Battle of wits : the complete story of codebreaking in World War II /

Saved in:
Bibliographic Details
Author / Creator:Budiansky, Stephen.
Imprint:New York : Free Press, c2000.
Description:436 p. : ill., maps ; 24 cm.
Language:English
Subject:
Format: Print Book
URL for this record:http://pi.lib.uchicago.edu/1001/cat/bib/4347080
Hidden Bibliographic Details
ISBN:0684859327
Notes:Includes bibliographical references (p. 363-417) and index.

Prologue Midway Green water breaking across her flat topside, the U.S. carrier Hornet plunged through a heavy sea and strong wind. A few hours earlier, as dawn broke on April 18, 1942, lookouts aboard the American ship had spotted a Japanese patrol boat. The Hornet' s planes were intending to carry out their raid under cover of darkness, but Admiral William Halsey, whose nickname "Bull" had not been bestowed lightly, gave the order to launch at once -- wind, wave, and daylight be damned. The ship swung into the wind, and at 7:25 A.M. the first of the twin-engine bombers groaned off the flight deck. The Army pilots at the controls of the B-25s had practiced short takeoffs from the comfortably dry land of a Florida airstrip but had never once tried it at sea. No one else ever had, either. Landing a B-25 on a carrier was impossible. Flying a B-25 off a carrier was, by comparison, merely insane. But the medium-weight bombers were the only aircraft in the American arsenal with a prayer of completing the daredevil mission. If all went according to plan, they would fly five hundred miles to Japan, drop their load, then continue another eleven hundred miles to a safe landing in unoccupied China. Leading the attack was an unflappable test pilot, Lieutenant Colonel James H. Doolittle; his was the first of the lumbering bombers to catapult down the heaving deck. Over the next hour fifteen others followed. One pilot hung on the verge of a stall for so long as he struggled to get airborne that, Halsey later recalled, "we nearly catalogued his effects." Thirteen of the planes headed for Tokyo, roared in over the rooftops from different directions, and dropped their four bombs apiece. The three others hit Nagoya and Osaka. Ever since the attack on Pearl Harbor four months earlier, President Franklin D. Roosevelt had been pressing for just such a morale-boosting coup to bolster some of America's wounded pride. The Doolittle raid had been his pet project, and he was exultant with the news. Asked by reporters where the planes had come from, FDR grinned and said, "Shangri-La." Doolittle's raiders did essentially no damage -- except to the Japanese psyche. On that they scored a direct hit. The Japanese Army claimed it shot down nine of the marauders; the true figure was zero. But the premature launch of the planes had added almost two hundred miles to their planned mission, strong head winds burned up still more fuel as they made their way toward China, and most of the crews ditched or bailed out. Eight who landed in Japanese-occupied territory were taken prisoner and three of them were executed, ostensibly for the crime of bombing civilian targets but in reality in an access of Japanese fury and mortification. In the months since his lightning strike against the American fleet on December 7, 1941, Admiral Isoroku Yamamoto, Commander in Chief of Japan's Combined Fleet, had grown accustomed to the adulation of a grateful public; each day brought sacks of adoring letters. After the bombs fell on Tokyo he was rattled to find he had become the target of hate mail. He was wracked, too, with anxiety over the Emperor's personal safety. Where had the bombers come from? Yamamoto pointed to Midway Island, America's westernmost outpost in the Pacific since the Philippines, Guam, and Wake Island had been overrun in Japan's seaborne blitzkrieg. It was a plausible conclusion, even if Shangri-La was actually closer to the mark. Midway was twenty-five hundred miles from Japan and thirteen hundred miles from Honolulu. Thus, Yamamoto argued, as long as it remained in American hands, bombers could fly from Hawaii to Midway, and from Midway to strike Dai Nippon. Japan's defensive perimeter would have to be pushed back farther still. Two days later, Yamamoto's fleet air officer, Captain Yoshitake Miwa, noted in his diary that if further raids on the mainland were to be prevented, "there would be no other way but to make a landing on Hawaii. This makes landing on Midway a prerequisite. This is the very reason why the Combined Fleet urges a Midway operation." But, truth be told, Yamamoto had had his eyes on Midway for months. Untouched by the "victory fever" that swept through Japan's high command, Yamamoto insisted that unless America could be forced swiftly to accept a negotiated settlement, Japan was ultimately doomed. The grand admiral was a gambler and something of a playboy; he had been to Harvard to learn English, served in Washington as a naval attach©, and his knowledge of America's industrial power made him view war with the United States as folly. But if war was inevitable, he had consistently argued, Japan's only hope was to risk all on a knockout blow. America's industrial might would take months or even years to fully mobilize: thus Yamamoto's bold stroke on December 7. Unfortunately, that had left the job only half done. America's battleships had been caught at anchor at Pearl Harbor, but her aircraft carriers, at sea on the morning of the Japanese attack, had escaped. Throughout March and early April a bitter fight roiled Japan's high command. Yamamoto pressed his case with mounting impatience: To draw the American carriers into the decisive battle, Japan must seize an objective that the United States would have to defend. If his plan to attack Midway was not approved, he would resign. The Naval General Staff sputtered. Midway might be of strategic value to an America on the defensive, the staff insisted, but it was worthless to Japan. Midway was a rocky atoll hardly larger than the small airstrip that stretched from one end of the island to the other; it could hold no more aircraft than a single carrier. The naval staff preferred a thrust to the south to cut off Australia, or, even more ambitiously, to seize Ceylon and India and link up with the German forces in the Near East. The Japanese Army, its eyes on China and on the threat Russia would pose if it entered the Pacific war, declared it would have nothing to do with Yamamoto's scheme, either. But as the dust from Doolittle's bombs settled, the Army staff came forward with a new demand: It now insisted that the Army must be included in Yamamoto's forthcoming assault on Midway. Four thousand miles to the east, Midway had become an obsession to another man during that winter and spring of 1942, a man as anonymous as Yamamoto was famous. Commander Joseph J. Rochefort came by his anonymity as much by the nature of his personality as by the necessity of his vocation. Above his desk hung a notice that read: "We can accomplish anything provided no one cares who gets the credit." He would later have reason to question the wisdom of that principle. But putting in twenty-hour shifts in a windowless basement was not a calling that appealed to the glory seekers in the United States Navy anyway. "The Dungeon," they called their cheerless command post in the basement of the administration building at Fourteenth Naval District Headquarters at Pearl Harbor; its more formal name was Station Hypo. Rochefort had enlisted in the Navy in 1918 with vague dreams of becoming a naval aviator. Nothing but the oddest of chances determined that 1942 would find him in charge of breaking JN-25, the Japanese Fleet General Purpose Code -- the code that carried the operational orders of the Combined Fleet, the code that would, in short, tell where Japan was going to strike next. Rochefort was driven but unflamboyant, a conventional career sailor who had pursued a conventional career path: sea duty, engineering school, ensign's commission, more sea duty. Rising from the enlisted ranks, he was an outsider to the elite fraternity of officers who had graduated from the Naval Academy. The coincidence that deflected him out of the ordinary course of duty occurred while he was serving aboard the battleship Arizona in 1925. The ship's executive officer, Commander Chester C. Jersey, liked crossword puzzles. Rochefort did too. Jersey remembered that fact when he was posted to Navy Department headquarters in Washington later that year. The Navy needed someone to work on codes, and Jersey recommended Rochefort. The informality of it all would seem fantastic by the standards of the huge and bureaucratic postwar Navy. But in 1925, the Navy's cryptanalytic staff consisted of a grand total of one person, and administrative matters throughout the service were frequently settled through personal contact and word of mouth. Lieutenant Laurance F. Safford, the Navy's one-man code breaking bureau, had not set out to be a cryptanalyst, either. In 1924 he was assigned the task of developing new codes for the Navy. No one in the Navy was paying much attention to foreign countries' codes at the time, and they certainly weren't trying to break them. But Safford figured that to make a good code he ought to first see what other navies were doing. And so the "research desk" was born in Room 1621 of the old Navy Department Building on the Mall in Washington. When Rochefort showed up for duty in October 1925, Safford put him through a six-month course in cryptanalysis that basically consisted of tossing him cryptograms to try to solve. When Safford was called to sea duty in February 1926, the "course" ended, and Rochefort, more or less by default, found himself officer in charge of the research desk. Under him was one cryptanalyst and one assistant with "no particular abilities." That was it. Rochefort's first dose of cryptanalysis left him decidedly disinclined for another. It was not that there was any particular pressure on him to produce results. No one in the Navy had much of an idea what he was up to anyway, and no one would have understood it if he had. But the work had a way of generating its own compulsive pressures. Rochefort would come home every evening at five or six o'clock with his stomach in knots from the tension of the problem he was tackling. It would be eight or nine at night before he could manage to force down his supper. He developed an ulcer and greeted his recall to sea duty in 1927 with unfeigned relief. But in those two years Rochefort scored America's first victory in a long shadow war with the Japanese Navy. Left over from 1918 was most of a $100,000 secret naval intelligence slush fund. To conceal it from Congress, the money was deposited in a Washington bank in a personal account belonging to the Director of Naval Intelligence. Whenever a new DNI took over, his predecessor just handed the money over to him along with the keys to the office. The money had begun to burn a hole in the pockets of successive DNIs, and in the early 1920s the incumbent decided to get rid of some of it by financing a series of break-ins at the Japanese consulate in New York City. The Japanese Navy's "Red" code book was secretly photographed and, over the course of several years, laboriously translated by linguists hired with more of the DNI's secret funds. (Just how hard it was to use up $100,000 was shown in 1931, when an acting DNI, in a fit of conscience for which his successors never forgave him, returned the money to the Treasury. The balance was $65,000.) A complete code book was a windfall, but there was still one crucial piece missing. Like almost all of the Japanese Navy codes that Rochefort and his colleagues would encounter over the course of their long battle of wits with their Japanese counterparts, Red was an enciphered code. Every word or syllable likely to be used in a message was assigned a numerical value -- that was the "code" part. But such a simple one-for-one substitution would not hold up a team of Boy Scouts, much less a determined military foe, for very long. So before the Japanese Navy sent any coded message over the airwaves, it was given a second disguise. The code clerk opened a second book, which contained page after page of random numbers; starting at the top of a page, he added the first of these random "additives" to the first code group of his message, the second to the second, and so on. An indicator buried in the message would tell what page in the additive book he had used for this "encipherment" of the basic code, so that the recipient could turn to that same page and strip off the additive before looking up the meaning of each code group. Thanks to the DNI's black-bag jobs, Rochefort had the code book. What he did not have was the additive book. To make matters worse, the Japanese changed the additive book frequently. With nothing to go on but the raw traffic that the Japanese Navy put out over the airwaves, Rochefort's job was to reproduce an additive book that he had never seen. Breaking a code when one has the underlying code book but no additive book is like finding a way across a strange country without a map or a compass. Breaking a code when one has neither code book nor additive book is like finding a way across a strange country with both eyes closed. Doing the former was what had given Rochefort his ulcer in 1926. His task in 1942 was to do the latter. America's very success, in September 1940, in breaking the Japanese diplomatic cipher, code named "Purple," had the ironic effect of distracting attention from where it could have been more profitably focused in the fateful months leading up to Pearl Harbor. The Purple cipher carried the highest-level diplomatic messages of the Japanese Empire; this was intelligence of such remarkable value that it was given the code name magic. The Purple cipher was generated by a complex machine. It used a cascade of rotating switches to encipher every letter of a message in a different key from the last or the next. In one position of the switches the letter A would become G; in the next it would become P. The U.S. Army's code breakers had, in eighteen months of intense effort, deduced the wiring and setup of the machine without ever seeing one, a feat of pure analysis the likes of which had scarcely before been seen. After hastily soldering together telephone switches and relays to produce a replica of the machine, they proceeded to decode the Japanese messages almost as quickly as they arrived. On the morning of December 3, 1941, a Purple message came through ordering Japan's embassy in Washington to destroy its code books, and even one of its two vital Purple machines. Frank Rowlett, a senior cryptanalyst of the Army's Signal Intelligence Service, arrived at his office at noon that day from a meeting, plucked this latest magic decrypt from his in-box, and proceeded to read its contents with mounting incredulity. With only a single machine it would obviously be impossible for the embassy to continue its normal flow of business. Colonel Otis Sadtler, who was in charge of distributing the magic decrypts, showed up in Rowlett's office at that moment and began to pepper him with questions. Had the Japanese ever sent anything like this before? Could they be getting ready to change their codes? Perhaps they suspected their current codes had been broken? Then the only possible meaning of this extraordinary message sank in. Sadtler pulled himself to attention. "Rowlett, do you know what this means? It means Japan is about to go to war with the United States!" And, decrypt in hand, Sadtler took off literally running down the corridor of the Munitions Building to alert the head of Army intelligence. On the night of December 6, an aide interrupted the President at a White House dinner to deliver him the latest magic decrypts. These erased all remaining doubt. Japan was preparing to break off diplomatic relations. War was inevitable. But diplomatic communications are not the place where military orders are delivered. America knew that Japan was going to strike; it did not know where she would strike. To know that would require breaking into the Japanese naval codes, and there was only one catch: Since mid-1939, America had not read a single message in the main Japanese naval code on the same day it had been sent. For most of the period from June 1, 1939, to December 7, 1941, the Navy was working on naval messages that were months, or even over a year, old. Partly this was a matter of manpower, partly it was a matter of human nature. Magic was such a dazzling prize that it blinded its possessors to the smaller but sometimes more valuable gems that lay buried among the dross and slag of supply orders and fleet maneuvers. JN-25 was the most recent descendant of the Japanese Navy's Red code; like its predecessors it was an enciphered code. At the time the new code first appeared on June 1, 1939, the U.S. Navy's Washington code breaking staff had grown to about thirty-six hands. By this time the "research desk" had acquired the official bureaucratic designation of OP-20-G, designating it as part of the Office of Naval Communications, OP-20, and Safford was back in charge after several tours of sea duty. The staff of thirty-six included translators, clerks, radio direction-finding experts, intelligence analysts, and officers responsible for the security of the Navy's own codes; only a handful were trained cryptanalysts, and of these only two or three could be spared to work on the new code, which was initially given the designation AN-1. Over the course of months they laboriously punched every message on IBM cards and searched for any clues that would give them a toehold on this completely uncharted terrain. But everything about it had the air of scholarly inquiry, far from the heat or urgency of battle. AN-1 was a "research" project, not a "current decryption" job; reconstructing the meaning of thirty thousand code groups and piecing together thirty thousand random additives was not going to be the work of a moment. The Japanese Navy's radio signals were intercepted by U.S. Navy operators in Hawaii, Guam, and the Philippines. The operators transcribed the Morse code signals by hand onto message blanks, bundled them up, and once a week handed them over to the captain of one of the Dollar Line's "President" passenger liners that plied the Pacific. The captains, all members of the Naval Reserve and therefore authorized to act as couriers of confidential documents, dropped the packages in the mail to Washington when their ships reached the West Coast. A very small amount of priority traffic could be entrusted to the Pacific "Clippers" of Pan American Airways; in the hull of each of these airplanes a small steel strongbox had been welded into place just for this purpose, the keys held by naval officers along the route. But delays of weeks from the time a message was transmitted by the Japanese to the time it arrived in Washington were the norm. Throughout 1939, 1940, and 1941 a slow-motion cat and mouse game ensued. Sorting the IBM cards by number and printing out huge catalogues of every code group in every message, the Navy cryptanalysts began to detect a few ghosts of the underlying code. The numbers in each message that appeared to serve as indicators, telling the recipient what page of the additive book had been used, were not quite as perfectly random as they should have been. The numbers bunched up, meaning that some lazy Japanese code clerks were reusing the same pages of additive over and over. That was a classic error; messages enciphered with the same additive pages could in principle be cracked. It was not until fall of 1940, however, that the first real break came -- and it came just in time to be rendered obsolete by a completely new, and much more complex, code book that the Japanese brought into service on December 1, 1940. By the summer of 1941, as tensions in the Pacific grew, every section of OP-20-G was desperately short of help. Messages in the Purple cipher, which could be read in their totality and almost always the same day they were transmitted, claimed first priority. Meanwhile seven thousand AN-1 messages were pouring in each month by mail to Washington, while only sixteen men could be spared to work on them. As the Navy began calling up Naval Reserve officers throughout the summer and fall, that figure crept up by about one per month. Station Cast, the Navy's intercept station at Cavite in the Philippines, was also trying its hand at compiling the reams of printouts and worksheets needed to tease apart the AN code and its additives; since spring 1941 a highly secret collaboration between Cavite and the British government's code breakers in Singapore had been under way on the project as well. But it simply wasn't enough. Slowly and laboriously, the new code book was being reconstructed; again, inexorably, on August 1, 1941, the Japanese introduced a new, 50,000-group additive book that sent the code breakers back to the beginning. By November 1941 only 3,800 code groups had been identified, along with only 2,500 additives reconstructed in the current system. It was far less than 10 percent of the total, nowhere near enough to read current traffic. Conspiracy theorists continue to weave elaborate scenarios "proving" that America had advance warning of the Japanese attack, with one branch of the "FDR knew" theorizers insisting that AN traffic was in fact being read in 1941. Yet month-by-month progress reports, internal histories, war diaries, logs -- some declassifed only in 1998 -- are all in agreement: Not a single AN message had ever been read currently by the time of Pearl Harbor, and not a single AN message transmitted at any time during 1941 was read by December 7. Five years later, with the war safely won, a few of OP-20-G's cryptanalysts were tidying up loose ends and decided to go back and try to crack the unread AN-1 traffic that had piled up in the months just before Pearl Harbor. What they found was enough to break an intelligence officer's heart. Over and over, the orders to the Japanese fleet during October and November 1941 repeated a single theme: Complete all preparations and be on a total war footing by November 20. Several messages referred to exercises in "ambushing" the "U.S. enemy." And one signal, dispatched November 4, ordered a destroyer to pick up torpedoes that Carrier Divisions 1 and 2 "are to fire against anchored capital ships on the morning in question." None specifically mentioned Pearl Harbor, and indeed many other intelligence indications in those critical months pointed to the Philippines, or even the Panama Canal, as possible targets of Japanese naval action if war broke out. Yet the pre­Pearl Harbor AN traffic, had it been broken at the time, would certainly have conveyed heavy hints of what was to come. In the chaos following the Japanese attack, mail service from the Pacific was thrown into disarray. On December 4, the Japanese had again changed the additive book for AN-1; it was back to square one yet again, and Washington fretted away a month waiting for enough current intercepts to arrive in the mail to renew the attack. But in the meanwhile the decision was made to allow "the field" to begin work without delay. On December 10, Rochefort's Station Hypo, which had been shunted off to work on a dead-end problem before Pearl Harbor, was given the go-ahead by Safford to tackle AN-1 on its own. The atmosphere throughout Hawaii in the days following the Japanese attack was one of stunned demoralization. During the attack a spent bullet actually ricocheted off the chest of Admiral Husband E. Kimmel, Commander in Chief of the Pacific Fleet, who would shortly become the scapegoat for the worst military disaster in American history. "Too bad it didn't kill me," Kimmel muttered. Two thousand four hundred and three Americans were dead. Two hundred aircraft were destroyed on the ground: Dutifully heeding warnings to be on the alert, the Army commanders had crowded their planes together wingtip to wingtip in midfield, well away from the perimeter fence and the Japanese saboteurs everyone imagined were lurking about the island. The Japanese torpedo planes and bombers caught all but one of the nine American battleships of the Pacific Fleet in port that morning, and all were left damaged or immobilized by the attack. Arizona, blown in two when her magazine went up, took eleven hundred of her crew with her to oblivion. Oklahoma lay capsized in the mud, never to see action again. The others sank at their moorings, had gaping holes ripped in their sides, or had run aground or were wedged between other crippled ships. Japan's force of ten battleships now had a seemingly insuperable command in the Pacific. If Pearl Harbor had been asleep, the American forces in the Philippines under General Douglas MacArthur had been comatose. The Philippines had been everyone's bet for where the Japanese blow would fall. When Secretary of the Navy Frank Knox was delivered the news of Pearl Harbor he exclaimed, "My God, this can't be true, this must mean the Philippines!" The Japanese would oblige soon enough. MacArthur received word of the Pearl Harbor attack just as it was ending, at 3:00 A.M. on December 8, Manila time. In the preceding weeks MacArthur had confidently assured Washington that with enough air power he could drive the Japanese back into the sea if they dared to come ashore. On that assurance he had been shipped dozens of top-of-the-line B-17 long-range bombers. When the decisive moment came, MacArthur, apparently frozen in indecision, barricaded himself in his penthouse suite in a downtown Manila hotel and did nothing. Nine hours later Japanese bombers and Zeros appeared over Clark Field; instead of meeting the swarm of enemy fighters they fully expected, the Japanese pilots looked down and rubbed their eyes in disbelief at the sixty neatly parked planes on the field below. That evening, Roosevelt kept a long-scheduled appointment with newsman Edward R. Murrow. FDR pounded his fist on the table in frustration: The American planes had been destroyed "on the ground, by God, on the ground!" he exclaimed. Three days later, the pride of Britain's Singapore-based Asiatic Fleet, the battleship Prince of Wales and the battle cruiser Repulse, were sunk by Japanese torpedo bombers flying from Saigon. Not a single Allied battleship or cruiser was left west of Hawaii. Japan, for the moment, was the unchallenged master of the Pacific and Indian oceans. At Pearl Harbor, the thoroughly shaken American commanders were certain that the Japanese were going to hit them again. Crews were hastily set to work tearing down fences, welding them together, and dangling them into the water around the docked ships as crude antitorpedo barriers. "Of course we had no knowledge whether that kind of net would be any good at all," admitted Rear Admiral Claude Bloch, commander of the Fourteenth Naval District, "but it was the best we had." A hypercautious mentality set in, bordering on paralysis. A carrier task force sent to relieve Wake Island was recalled at the last minute by Vice Admiral William S. Pye, who had been given temporary command of the fleet after Kimmel's ouster; the ships were actually in sight of the besieged atoll when the recall order came, setting off a near mutiny aboard the carrier Saratoga. When FDR received the news he said it was a worse blow than Pearl Harbor. Rochefort, blaming himself for not foreseeing the Pearl Harbor attack, reacted characteristically, driving himself and his men without mercy. Summoned to the Dungeon on December 7 by an 8:00 A.M. telephone call from his deputy