SPECIAL REPORT
ESPIONAGE
November 12th 2016
Shaken
and
stirred
SPECIAL REPOR T
ESPIONAGE
Shaken and stirred
Intelligence services on both sides of the Atlantic have struggled
to come to terms with new technology and a new mission. They are
not done yet, writes Edward Carr
IN THE SPRING thaw of 1992 a KGB archivist called Vasili Mitrokhin
walked into the British embassy in Riga. Stashed at the bottom of his bag,
beneath some sausages, were copies of Soviet intelligence files that he
had smuggled out of Russia. Before the year was out MI6, Britain’s foreign-intelligence service, had spirited away Mitrokhin, his family and six
large cases packed with KGB records which he had kept hidden in a milk
churn and some old trunks under the floor of his dacha.
The pages of “The Mitrokhin Archive”, eventually published in
1999, are steeped in vodka and betrayal. They tell the stories of notorious
spies like Kim Philby, a British intelligence officer who defected to Russia
in 1963. And they exposed agents like Melita Norwood, who had quietly
worked for the KGB for 40 years from her home in south-east London,
then shot to fame as a great-granny. Her unrelenting Marxist refusal to shop at Britain’s capitalist supermarkets earned her the
headline: “The Spy Who Came in
from the Co-op”.
Mitrokhin’s record for the
largest-ever haul of intelligence
was smashed in 2013 when an
American contractor, Edward
Snowden, fled from Hawaii to
Hong Kong with a secret archive
of his own that contained more
than 1.5m classified files from
America’s National Security
Agency (NSA). Mr Snowden uncovered
programmes
with
names like DISHFIRE and OPTIC
NERVE under which the NSA and
its British counterpart, GCHQ,
were alleged to be monitoring
phones and computers around
the world. Mr Snowden’s accusation was not that foreign agents
had infiltrated Western intelligence agencies but that Western agencies
were spying on ordinary people, including their own citizens.
To look at Mitrokhin’s meticulous typed-up transcriptions side by
side with Mr Snowden’s capacious pen-drives conveys a sense of how
deeply and rapidly the business of intelligence has changed. Western intelligence agencies used to inhabit a parallel world where spy battled spy.
Their trade was stealing or guarding secrets. Their masters were the men
and women in government. Today the intelligence services are part of
everyone’s world. Their main task has been to protect society from terrorists and criminals. They are increasingly held to account in the press, parliaments and courts. This special report is about their struggle in the past
15 years to come to terms with this transition. They are not done yet.
ACKNOWLEDGMENT S
Many of the people who helped with
this special report prefer to remain
anonymous, for understandable
reasons. As well as acknowledging
them, the author would like to single
out three people: Jeffrey Hiday, Oleg
Kalugin and Nigel Inkster.
CONTENT S
5 Technology
Tinker, tailor, hacker, spy
7 Governance
Standard operating
procedure
8 Edward Snowden
You’re US government
property
10 China and Russia
Happenstance and enemy
action
12 How to do better
The solace of the law
Who can spy on the spies?
The intelligence revolution is partly the result ofnew technology. As
recently as1999, on becoming director of the NSA, Michael Hayden asked
to send an e-mail to all staff. He was told: “We can’t actually do that.” The
organisation used computers to break codes rather than to surf the web
as everyone else did. The NSA’s new facility in Bluffdale, Utah, the first of 1
The Economist November 12th 2016
A list of sources is at
Economist.com/specialreports
1
SPECIAL REPOR T
ESPIONAGE
2 several, now stores exabytes of data drawn from everyday com-
munications. At Britain’s GCHQ, most code-breaking was done
on paper until well into the 1980s. Today, inside its doughnutshaped building in Cheltenham, south-west England, the hum
from banks of computers that stretch away into the half-light is
drowned out by the roar of air-conditioning.
The revolution has brought spying closer to ordinary people. After the attacks on America on September 11th 2001, counter-terrorism and counter-insurgency became the focus for the
American intelligence agencies, says John Parachini, who heads
intelligence policy for RAND, a think-tank. Almost two-thirds of
today’s intelligence personnel have been hired since 9/11. As the
world has moved online, so the spooks have become involved in
monitoring organised crime and paedophiles as well as terrorists. That might mean tracking a drugs syndicate from Latin
America to Europe, or working out how criminal gangs launder
their money, or following paedophiles on the web.
In Mitrokhin’s days spies sent coded messages using shortwave radios and dead letter boxes. Now the communications of
the spooks’ new targets are mixed in with everyone else’s, shuttling between computers and smartphones that are identical to
those on your deskand in your pocket. Counter-terrorism, in particular, is pre-emptive. Hence the security services have had to
act as hunters of conspiracies rather than gatherers of evidence.
I don’t believe you any more
And the revolution is taking place amid growing popular
suspicion of everyone in charge. The days are gone when the
word of Congress, the home secretary and the odd judge commanded enough public confidence to see off the accusations of a
private individual such as Mr Snowden. Belated official acknowledgment of secret programmes has often been met by
public dismay, even after assurances that they have been properly overseen. “It is not enough for the authorities just to say ‘trust
us’,” writes Paul Bernal, of Britain’s University of East Anglia.
“The public needs to know.”
Privacy advocates complain that the spooks have unprecedented scope to pry into people’s lives. They warn of a burgeoning surveillance state. The spooks retort that, on the contrary,
they cannot keep up with terrorists and criminals cloaked by encryption, the dark web and the fact that, as the world builds internet infrastructure, a smaller share of total traffic is routed
through accessible Western networks.
At the heart of the debate lies a conflict. The goal of a modern intelligence service, in the formulation of Sir David Omand,
a former British intelligence chief, is for citizens to trust the state
to manage the threats to their everyday lives. To maintain public
safety, the intelligence services must be able to employ secret
et to
sources and methods that inevitably involve intrus
command that public trust, they must also be transparent and
prepared to live by rules that protect individual privacy.
These contradictions cannot be wished away. Privacy is a
precondition for intimacy, trust and individuality, says David
Anderson, a senior lawyer asked by the British government to review intelligence legislation. It secures rights such as the freedom
ofassembly and fair trials. The knowledge that an all-seeing state
is watching has a chilling effect even if you have done nothing
wrong. Perhaps your words will be used against you later, under
laws passed by a different government. Perhaps the state will try
to crush the dissent that prefigures desirable social change—as
America’s FBI tried to destroy Martin Luther King by sending a
letter, supposedly from a disillusioned admirer, that accused him
of being a “colossal fraud and an evil, vicious one at that”.
But privacy is not an unalloyed good. A society that gives it
primacy over security invites paralysing disorder and injustice
that would inhibit the very intimacy and freedom of expression
which privacy is supposed to promote.
Likewise, although the public needs to know what is being
done in its name, some spying techniques lose their potency if
they are discovered. Early Enigma decrypts in 1940 from Bletchley Park, Britain’s code-breaking centre, were given the “CX” prefix of MI6 reports so that the Nazis would think they were based
on standard human intelligence (known in the jargon as HUMINT). A former CIA employee who is now at RAND tells how,
after a successful raid in 1998, journalists learned that the NSA
was intercepting calls from the satellite phone of Osama bin Laden, the founder of al-Qaeda. Immediately after the news got out,
the phone fell silent.
Since the Snowden revelations, Western security services,
and particularly those of America and Britain, have come in for
savage criticism. Much of this has focused on the intense years
immediately after 9/11. The CIA subjected prisoners to brutal interrogation techniques, including simulated drowning, or waterboarding. For some years the NSA operated a telephone-surveillance programme without judicial oversight. That programme
was later judged to be illegal.
This special report will look at those transgressions in greater det
et, even taking them into account, the criticism of
American and British intelligence is overblown. Rather than being James Bonds, real-life intelligence officers are bureaucrats.
Rather than acting as freewheeling individualists, most set out to
live by the rules. It is possible to argue about the merits of intercepting and warehousing data, about access to databases and
large-scale hacking, but the idea that controlling masterminds at
the NSA and GCHQ are plotting mass surveillance is a myth.
Such criticism is especially unfair when it comes from outside the English-speaking intelligence alliance embracing America, Australia, Britain, Canada and New Zealand, known as the
Five Eyes. Few countries say much about their intelligence ser- 1
2
The Economist November 12th 2016
SPECIAL REPOR T
ESPIONAGE
2 vices or have a clear framework for governing them. Britain’s op-
erated almost entirely in the shadows until the 1990s and acknowledged some of its activities for the first time only in 2015.
et by the end of this year it will have put its intelligence services
under a system of oversight that is a model. And America is more
open about its intelligence services than any other country.
The stark contrast is with countries like China and Russia,
where the security services answer to nobody except the men at
the very top. Russian and Chinese citizens are subject to untrammelled surveillance by their own leaders.
Before looking at Russia and China, and the growing awareness that they will become the Western agencies’ main antagonists again, start with the twin shocks of technology and terrorism. They have turned the world of intelligence on its head. 7
Technology
Tinker, tailor, hacker,
spy
Who is benefiting more from the cyberisation of
intelligence, the spooks or their foes?
“THE COMPUTER WAS born to spy,” says Gordon Corera,
who covers intelligence for the BBC, Britain’s national
broadcaster. The earliest computers, including Colossus and
SEAC, were used by signals intelligence (known as SIGINT) in
Britain and America to help break codes. But computers also
happen to have become supremely good at storing information.
Searching a database is a lot easier than searching shelves of files
like those compiled by the East German secret police, the Stasi—
which stretched for100km.
The job used to be to discover what a hostile country was
up to by attaching crocodile clips to telephone lines emerging
from its embassy, intercepting communications, collecting data
and decrypting them. It was an industrial process. Breaking code
was laborious, but once you had succeeded, the results endured.
“Twenty years ago we had a stable target,
a stately pace of new technology and
point-to-point communications,” says a
senior intelligence officer. Cryptography
evolved slowly, so “when you cracked a
code it could last from ten to 30 years.”
The internet changed everything.
Roughly $3.4trn a year is being invested in
networked computers, phones, infrastructure and software. The pace is set by
businesses, not spooks. Individual packets of data no longer travel on a dedicated
phone line but take the route that is most
convenient at that instant, blurring the
distinction between foreign and domestic communications. Signal intelligence
used to be hard to get hold of. Today it
gushes in torrents. The trick is to make
sense of it.
Civil-liberties groups rightly claim
that this new world presents untold opportunities for surveillance. This has
been especially true for the NSA and
GCHQ. Most of the traffic has passed
through America, which contains much
The Economist November 12th 2016
of the infrastructure of the internet, and much of the rest passed
through Britain, even if it originated and terminated elsewhere.
Everyone uses the same hardware and software, so if you can
break one device, you can break similar devices anywhere.
Knowing who communicates with whom is almost as revealing as what they say. In a technique called contact chaining,
agencies use “seed” information—the telephone number or email address of a known threat—as a “selector” to trace his contacts and his contacts’ contacts. A burst of activity may signal an
attack. In 2015 contact chaining let GCHQ identify a new terrorist
cell that the police broke up hours before it struck.
You are never alone with a phone
Mobile phones show where they are. According to Bruce
Schneier, a cyber-security expert, the NSA uses this information
to find out when people’s paths cross suspiciously often, which
could indicate that they are meeting, even if they never speak on
the line. The NSA traces American intelligence officers overseas
and looks for phones that remain near them, possibly because
they are being tailed. Location data can identify the owner of a
disposable phone, known as a “burner”, because it travels
around with a known phone.
The technical possibilities for obtaining information are
now endless. Because photographs embed location data, they
provide a log of where people have been. Touch ID is proof that
someone is in a particular place at a particular time. Software can
recognise faces, gaits and vehicles’ number plates. Commercially
available devices can mimic mobile-phone base stations and intercept calls; more advanced models can alter texts, block calls or
insert malware. In 2014 researchers reconstructed an audio signal from behind glass by measuring how sound waves were
bouncing off a crisp packet. The plethora of wired devices in offices and houses, from smart meters to voice-activated controllers to the yet-to-be-useful intelligent refrigerator, all provide an
“attack surface” for hacking—including by intelligence agencies.
Britain’s government has banned the Apple Watch from cabinet
meetings, fearing that it might be vulnerable to Russian hackers.
The agencies can also make use of the billows of “data exhaust” that people leave behind them as they go—including financial transactions, posts on social media and travel records. 1
3
SPECIAL REPOR T
ESPIONAGE
2 Some of this is open-source intelligence (known as OSINT),
which the former head of the Bin Laden unit of the CIA has said
provides “90% of what you need to know”. Private data can be
obtained by warrant. Data sets are especially powerful in combination. Facial-recognition software linked to criminal records,
say, could alert the authorities to a drug deal.
The agencies not only do more, they also spend less. According to Mr Schneier, to deploy agents on a tail costs $175,000 a
month because it takes a lot of manpower. To put a GPS receiver
in someone’s car takes $150 a month. But to tag a target’s mobile
phone, with the help of a phone company, costs only $30 a
month. And whereas paper records soon become unmanageable, electronic storage is so cheap that the agencies can afford to
hang on to a lot of data that may one day come in useful.
Vague, very vague
But not everything is going the agencies’ way. Indeed, many
SIGINTers believe that their golden age is already behind them.
As the network expands, more capacity is being added outside
America. By 2014, according to Mr Corera, the proportion of international data passing through American and British fibres
had nearly halved from its peak. And the agencies have the capacity to examine only a small fraction of what is available. The
NSA touches 1.6% of data travelling over the internet and selects
0.025% for review. Its analysts see just 0.00004%.
Data are also becoming harder to trace. Some protocols split
a message in such a way that it passes over different networks—a
phone connection and Wi-Fi, say. Others allocate IP addresses
dynamically, so that they may change many times in a single session, or they share one between many users, which complicates
identification. Still others take computing closer to the user,
which means that messages bypass the core network.
The internet has many channels and communications
apps, each with its own protocol. Work on new tools is 20-30% of
the spooks’ job. Even so, there are too many apps for the agencies
to reverse-engineer, so they have to choose. An easy protocol
might take a day to work around. A difficult one might take
months. A routine upgrade of an app can mean having to start
from scratch. And some means of communication are intrinsically hard to break. Messages worth collecting that are contained
in apps like FaceTime and Skype are hard to tell apart from entertainment in Netflix an ouTube when they pass through networks. Jihadists can contact each other through online gaming
chat rooms. Steganography hides messages inside images.
Encryption is becoming standard. Ifa message is sent via an
app provider like Telegram or WhatsApp, the identity of the re-
4
ceiver might be encrypted, too. In principle modern encryption
is uncrackable. Unless someone can build a quantum computer,
which could search for multiple solutions simultaneously, working through the permutations would take a chunk out of the rest
of history.
To get in, therefore, analysts often depend on human error.
But the targets are becoming more sophisticated. The New York
Times has reported that Abdelhamid Abaaoud, who directed a
wave of bloody attacks in Paris in November last year, ordered a
soldier to ring a mobile phone on Syria’s northern border so that
his call would pass through a lightly monitored Turkish network.
The result, case officers say, is that tracking jihadists takes increasing effort and skill. A few years ago one officer might watch
several jihadist targets; today you need to throw a lot more manpower at the task. Too many jihadists have travelled to Syria for
GCHQ to monitor them all. The intelligence services catch
glimpses of what is going on, but not the full picture. “With encryption,” says a British officer, “maybe you see a bit of content, a
bit of the puzzle.”
Some Western intelligence chiefs have tried to curb encryption, or argued that at least they should be given a set of secret
keys. That would be impractical and unwise. Impractical, because watertight encryption programmes will then be written
outside America and Europe, and there is little the authorities
can do to stop it. Unwise, because the intelligence services are
not the only ones prowling the web. Organised criminals and
fraudsters would like nothing better than weaker encryption.
A better way to cope with the difficulties of intercepting
traffic is to hack into machines sitting at the end of the communications chain. Once in, the agencies can look at a message before
it has been encrypted, split into packets and scattered across the
network. Again, though, that poses a dilemma, because governments are responsible for cyber defence as well as cyber offence.
To gain entrance to a machine, hackers use flaws in software. The
most prized of these are undisclosed and called zero-day vulnerabilities (because software engineers have zero days to write a
patch). Stuxnet, a computer worm written by the Americans and
the Israelis that attacked centrifuges in Iran’s uranium-enrichment programme, exploited five zero-day flaws.
There is a market in such tools. When Hacking Team, an Italian cyber-company, was itself hacked in 2015, the world learnt
that zero-day vulnerabilities were for sale. According to Wired, a
magazine, the price started at hundreds of thousands of dollars.
Among the buyers were governments and criminals. In their role
as defenders, the NSA and GCHQ should be revealing software
faults so that companies can write patches. In their role as attackers, they need some in reserve.
When machines are so powerful, where do people fit in?
Certainly, signal intelligence is relatively cheap, versatile and safer than running human agents et human spies still play a vital
complementary role. One taskis to furnish seed information that
can serve as selectors for tracing contacts. Another is to gain access to computers that are well-defended or “air-gapped” from
the internet. Most valuable of all is the human ability to bring
judgment and context.
People also provide oversight. There was a time when the
constraints on the agencies were technical and budgetary, because codes were hard to break and agents costly to deploy. In an
era of cheap technology, it is difficult to know precisely what the
technology will be able to accomplish. The constraints on the intelligence services’ conduct must therefore be legal—and robust.
Edward Snowden and others have suggested that the agencies are unwilling to live within the rules. But is that criticism deserved? In the anxious times after the attack on America on September11th 2001, how far did the CIA and the NSA really go? 7
The Economist November 12th 2016
SPECIAL REPOR T
ESPIONAGE
Governance
Standard operating
procedure
How the war on terror turned into a fight about
intelligence
AFTER THE COLLAPSE of the Soviet Union, intelligence
was becalmed. Apartheid came to an end, the Palestine Liberation Organisation said that it was abandoning terror, and
economies around the world embraced the Washington consensus. The NSA, isolated by its own secrecy, was out of touch with
the burgeoning internet; it lost 30% of both its budget and its
workforce. Budgets at the CIA and MI6 were cut by a quarter.
John Deutch, then the CIA’s director, thought the future lay in signals intelligence and began to retire old hands in what became
known as the “agent scrub”. At gatherings of senior mandarins
in Whitehall, Sir Colin McColl, then head of MI6, was asked by
colleagues: “Are you still here?”
Everything changed on September 11th 2001. When alQaeda struck America, the recriminations flew. The CIA had
been created after Pearl Harbour to guard against surprise attacks, yet in the 1990s the agency’s bin Laden hunters had been
marginalised as eccentric and obsessive. The intelligence agencies scrambled to make up for what the 9/11 Commission later
called their failure “to connect the dots”.
At the time, amid fears of the next assault, the intelligence
agencies were called on to make the homeland safe. But when
their conduct came to light later, in a less fearful world, they were
condemned for their methods. The story of this whipsaw is a
case study in how democratic, law-abiding societies struggle to
govern bureaucracies that act behind a veil of secrecy. America
has found the ensuing debate messy and bitter. The thing to remember, however, is that in other countries the debate barely
took place at all.
One set of accusations was levelled at the “President’s Surveillance Programme”. Under this, the NSA intercepted international communications that it suspected had a bearing on alQaeda, even if one of the callers was in the United States and
was thus protected by the Fourth Amendment, which guards
Americans against searches or seizures without a warrant. The
The Economist November 12th 2016
agency also collected “metadata” (the details but not the content)
of calls to, from and within America, acting outside the usual legal machinery. Administration lawyers advised that, as commander-in-chief, George W. Bush had war powers that overrode
other laws.
A second set of accusations dealt with harsh treatment of
prisoners by the CIA. In secret detention centres outside America
it employed 13 techniques, including slapping, nudity and, notoriously, waterboarding. The aim was not to extract information
directly but to break prisoners’ will, so that they tipped from a
“zone of defiance” to a “zone of co-operation” in which they
would talk freely. In “extraordinary renditions” some prisoners
were handed over to other governments. Although these were
supposed to give America assurances of fair treatment, critics
said that in practice nothing could stop them from using torture.
In all, the CIA dealt with fewer than 100 high-value prisoners, and half that number were rendered up. Bush administration lawyers advised that prisoners’ treatment at the hands of
the CIA stopped short of torture, which is illegal. Common Article Three of the Geneva Convention, which applies the stricter
standard of cruel, inhuman or degrading treatment, was irrelevant, they said, because it applies only to civil wars.
Would you waterboard your daughter?
Both the surveillance and the interrogation programmes
were to be mauled in the press, in Congress and in the courts. The
Detainee Treatment Act, passed in 2005, banned cruel, inhuman
or degrading treatment for any American prisoner. The same
year the Washington Post revealed the existence of secret prisons
in eastern Europe and others hinted at the harsh techniques.
General Hayden, by then head of the CIA, reports that sessions
between the agency and the House Security and Intelligence
Committee descended into shouting matches. During one, he
was asked if he would be prepared to waterboard his daughter.
In 2006 the Supreme Court found against Mr Bush’s legal team
and ruled that Common Article Three did in fact protect alQaeda prisoners. Early in his presidency, Barack Obama restricted interrogators to mild techniques, such as exploiting the subject’s fears and resentments or offering small rewards like cigarettes, laid out in the revised Army Field Manual. In effect, the
vestiges of the CIA interrogation programme were shut down.
A chunk of the surveillance programme followed a similar
trajectory. Reports about it surfaced in the New York Times in
2005 (though the paper had been sitting on the story for over a 1
5
SPECIAL REPOR T
ESPIONAGE
2 year), with an account of warrantless col-
lection of information. The extent of the
programmes became clear only in June
2013, when Edward Snowden released his
trove of NSA files (see box). Immediately
it became obvious that a few months earlier James Clapper, the Director of National Intelligence, had misled Congress.
When asked whether the NSA collected
“any type of data at all on millions or hundreds of millions of Americans” he had
replied under oath, “No sir”, and given a
chance to clarify his answer, he continued: “Not wittingly.”
At the end of 2013 a presidential review panel and in early 2014 a government agency, the Privacy and Civil Liberties Oversight Board, both issued
withering critiques of the metadata collection. The law says that the government
can seize metadata if they are “relevant”
to an FBI investigation. That language, the
oversight board concluded, is not broad
enough to allow the NSA to seize the
whole lot before an investigation has begun. In May 2015 a federal appeals court in
New York agreed. And a month later the
USA Freedom Act gave the NSA six
months to stop warehousing metadata—
though it allowed the agency to go to telecoms companies with specific queries.
Grey areas
You’re US government property
Is Edward Snowden a villain or a hero?
EDWARD SNOWDEN HAS plenty of fans. A
film about him by Oliver Stone describes
how, as a contractor with Booz Allen Hamilton, Mr Snowden turned against the system
and smuggled out files about its spying
activities. To coincide with the release of
the film in September, the fans have
launched a campaign for his pardon. No
one else has sparked such an intense debate on public policy, they say. He won a
change in the law and shifted global attitudes to privacy.
Having fled to Hong Kong, Mr Snowden later took refuge in Moscow, where he
now lives under the protection of the Russian government. If he returned to face
trial in America he would not be able to
mount a full defence. The Espionage act,
under which he would be tried, does not
allow him to appeal to the public interest.
Yet even if he could, he would probably be
convicted. And rightly so.
America’s House Permanent Select
Committee on Intelligence recently published its own verdict on Mr Snowden,
calling the leak “the largest and most
damaging public release of classified information in US intelligence history”. It endangered troops and agents overseas and
undermined defences against terrorism.
The vast majority of the documents Mr
Snowden stole did not touch on the privacy
of American citizens. Instead, they revealed details of how the NSA spies on
non-Americans, including foreign leaders,
who do not enjoy constitutional protection.
This saga raises two questions about
America’s system for running the intelligence agencies. The first involves the role
of the president. Both the surveillance
and the interrogation programmes, as
well as the legal opinions justifying them,
were secret. In itself, that was legitimate
and perfectly sensible, because otherwise the jihadists might have learnt about
them and altered their behaviour accordingly. But the Bush legal team rested on
maximalist interpretations of the president’s war powers, which the courts were later to strike down.
At the same time the secrecy the administration insisted on
was extreme. Even the chief counsel of the NSA was not allowed
to read the basis for his own agency’s surveillance programme,
and its inspector-general, in effect its regulator, was not told of
the programme’s existence for several months. If—or more likely
when—tight security fails, the combination of controversial legal
opinion and general shock risks a humiliating climbdown. That
does the agencies no good at all.
Second are doubts about governance. Congress and the
courts are supposed to check the executive, but questions hang
over both. At the start Congress was pliant. “There was some
oversight,” says Matthew Aid, a former intelligence officer who
writes about the NSA, “but I have seen kittens protest more loudly.” Later, amid popular anger at the programmes, members
queued up to chuck rotten tomatoes. Part of the problem is structural. The House and Senate Committees meet in camera and
much of their debate is classified. One former official at America’s Defence Intelligence Agency points out that, since the members get no chance to grandstand to their voters back home, sit6
The committee says that America may have
to spend hundreds of millions or even
billions of dollars to mitigate the damage.
Others point out the indirect costs.
Private companies were embarrassed by
being shown to co-operate with the American authorities. The very fact that the leak
took place may lead people and companies
to conclude that to work with America is not
safe. That feeling will have been reinforced
by the arrest last month of a second contractor, Hal Martin, on suspicion of having
stolen classified material, though as yet
there is no evidence that he passed it on.
Mr Snowden’s supporters claim that
he is a whistleblower. But the committee
found that he made little or no attempt to
raise his concerns with his superiors. If they
had proved unsympathetic, he could have
gone to the NSA’s inspector-general, or to
the committee itself.
Mr Snowden’s boss at the NSA in
Hawaii, Steven Bay, also worked for Booz
Allen Hamilton. He lost his job over the leak.
Speaking in September to Cipher Brief, a
newsletter, he attested to Mr Snowden’s
intelligence and ability but questioned his
qualifications for speaking out. “He never
actually had access to any of that data,” Mr
Bay said. “All of the ‘domestic-collection
stuff’ that he revealed, he never had access
to that. So he didn’t understand the oversight and compliance, he didn’t understand
the rules for handling it, and he didn’t
understand the processing of it…In my
mind Ed’s not a hero.”
ting on the committees offers little reward.
The worries extend to the special intelligence court, created
under the Foreign Intelligence Surveillance act. It was informed
early on about the collection of metadata, and in 2006 was formally brought into the process and asked to issue general warrants. The court asked for changes to strengthen protections for
Americans. However, compared with the New ork appeals
court and much other legal opinion, it leant heavily towards the
administration. The suspicion is that, like any regulator, it had
started to see the world through the eyes of its charge.
Before you conclude that the system is broken, however,
look for a moment at the other side of the coin. Intelligence law is
complex and often secret. This has meant that reasonable complaints against the agencies have become mixed up with unreasonable ones and with outright errors to form one great hairball
of moral outrage.
For instance, there were reports that the NSA broke its own
privacy rules thousands of times a year. That sounds alarming.
In fact, two-thirds of these breaches involved calls between legitimate non-American targets who just happened to be in 1
The Economist November 12th 2016
SPECIAL REPOR T
ESPIONAGE
2 America at the time—and were thus temporarily protected by
law. Most of the rest were selectors wrongly entered in the database because of poor typing or overly broad search criteria. Instances of genuine abuse tended to involve intelligence officers
checking up on their partners (known, inevitably, as LOVEINT).
Defending the programme, General Hayden points out that all
but a handful of the NSA’s 61m inquiries were legitimate. The
newspaper headlines, he says, should have said, “NSA damn
near perfect”.
From the press coverage you get the sense that the agencies
were out of control. In reality they are highly bureaucratic. In the
metadata programme each search of a seed had to be approved
by one of 22 supervisors. The foreign programme established
tests to ensure that targets are not American, likely to be outside
the United States and likely to provide useful intelligence. The
“audit trails are baked into the process”, says a former intelligence-oversight official at the Department of Defence. “There are
triggers and warnings to managers of improper searches within
the datasets.”
Despite this, there is a persistent notion that the intelligence
agencies undertake mass surveillance. That is partly because
some critics elide foreigners, who are not protected, with citizens, who are. Although the Privacy and Civil Liberties Oversight Board condemned the metadata programme, it made clear
that “compliance issues [did not] involve significant intentional
misuse of the system. Neither has the board seen any evidence
of bad faith or misconduct.” When a senior British judge was
asked whether GCHQ engaged in random mass intrusion into
citizens’ private affairs, he replied “emphatically no”. According
to Sir Iain Lobban, a former head of GCHQ, “if they were asked to
snoop, I wouldn’t have the workforce; they’d leave the building.”
The idea has also got about that intelligence is ineffective
against terrorists, and that its true purpose must therefore be to
spy on ordinary people. That conclusion has arisen partly because the oversight board found that the metadata programme
did not add anything to the NSA’s understanding of terrorism. Intelligence chiefs are to blame, too, for making claims about their
achievements that they could not substantiate.
However, the oversight board found that the other, foreign
programme made “a substantial contribution to the government’s efforts to learn about the membership, goals and activities of international terrorist organisations, and to prevent acts of
terrorism from coming to fruition.” For instance, it helped to
identify the courier who led to Osama bin Laden. Between 2002
and 2013 the NSA helped foil 17 terrorist plots against New ork. In
Britain MI5, MI6 and GCHQ convinced David Anderson, an in-
The subtle point critics of American intelligence often miss
is how the system, taken as a whole, has tended to right itself. Ben
Wittes, of the Brookings Institution and editor of the Lawfare
blog, says that after the initial reaction to 9/11 there was a broad
correction in the following years. The last waterboarding took
place in 2003. When General Hayden became director of the CIA
in 2006, he stopped the most extreme treatment. “Presidents—
any president—get to do one-offs based on raw executive authority,” he has said, “but long-term programmes, like this one had
become, needed broad political support.”
Likewise, thanks to growing discomfort within the Justice
Department, the FBI and the NSA—and a lot of courage from
some officials—the metadata programme was brought under the
control of the intelligence court. “When the terror threat receded
a bit,” says the former intelligence-oversight official, “people
stepped back and privacy and civil liberties came to the fore.”
Some intelligence folk think that the clamour for action immediately after 9/11 and the condemnation of the intelligence
services later, when the world no longer
seemed so dangerous, is an example of
double standards. There is something to
The subtle point critics of American intelligence often
that. But the whipsaw is also a consemiss is how the system, taken as a whole, has tended to
quence of secrecy. For the truth to
emerge, as it inevitably will, takes time.
right itself
And when it does, the intelligence services can seem sly and out of control. Mr
dependent reviewer appointed by the government, that commu- Wittes believes they would do better to be open about what they
nications data has played a “significant” role in every counter- do, and “to sacrifice some degree of effectiveness to win trust”.
Counter-terrorism has left its mark on the intelligence serterrorism operation in the decade to 2015.
The same is true for harsh interrogation. It would be conve- vices. The old guard had a variety of experience, say the experts
nient if inflicting pain on prisoners was pointless as well as at RAND, but the young tend to know only about Iraq or Afghaniwrong. However, many people in government and the intelli- stan. That will remain useful: even if Islamic State fades, jihadists
gence services attest to how the three people who suffered wa- will continue to attack the West. But the old adversaries never
terboarding gave up a lot of information; the CIA’s former coun- went away. Indeed, the spy agencies of Russia and China have
ter-terrorism chief, Jose Rodriguez, called them “walking taken advantage of the terrorist distraction to hack American netlibraries”. The decision to abstain from such techniques, just and works. That, says Seth Jones of RAND, is where the attention is
shifting right now. 7
wise though it was, came at a cost.
The Economist November 12th 2016
7
SPECIAL REPOR T
ESPIONAGE
China and Russia
Happenstance and
enemy action
Western intelligence agencies are turning to the old
rivalry with Russia and the new one with China
MOST COUNTRIES HAVE spy agencies of one sort or another, and their efforts may well be directed chiefly against
their own people. Many are a legacy ofcolonial rule. An agency’s
clout is often at odds with its country’s place in the world. Brazil’s
intelligence services are puny compared with those of Peru and
Colombia, which fought off Marxist narco-guerrillas. India’s Research and Analysis Wing is a minnow next to Pakistan’s tentacular Inter Services Intelligence. Israel’s Shin Bet and Mossad are
world-class.
In an era dominated by terrorism, many of these services
work with the big Western agencies such as the CIA or France’s
Direction Générale de la Sécurité Extérieure. The locals are more
successful at infiltrating their agents and have a better understanding of their own region. In return for collecting intelligence
on the ground, the liaison services get help, often in the form of
signal intelligence (SIGINT) or satellite imagery (IMINT).
Sometimes, however, the story is all about rivalry, most of
all between the West and Russia and China. Russia has the higher profile, probably intentionally. In 2015 James Clapper, America’s director of national intelligence, told Congress that Russia
was America’s main cyber threat. In the past few months alone it
is thought to have scored a number of points.
One was to humiliate the NSA by putting a stolen suite of its
hacking tools on sale under the cover name Shadow Brokers. Another was to hack the medical records of Simone Biles, an American gymnast who won four gold medals at the Rio Olympics.
Russia also undermined the presidential campaign of Hillary
Clinton by releasing e-mails from its hacks of the Democratic National Committee and Colin Powell, a former secretary of state.
“Active measures” like this draw on techniques of manipulation, misinformation and infiltration that go back to the tsars.
What is new, says Fiona Hill, a Russia expert at the Brookings Institution, is the lack of restraint. “Snowden blows everything
open,” she says. Now that Russia can say America is up to the
same tricks, there is no need for secrecy.
Influence by insinuendo
Russia’s foreign-language television station, RT, and news
agency, Sputnik International, work by what might be called “insinuendo”—a slur on the integrity of an opponent, the false reporting of an anti-Russian war crime in Ukraine, a relentless focus on racial tensions in American cities. The idea is to fan the
flames of fear, resentment and division. Russia is active across
the West. A recent report by the Centre for European Policy Analysis in Washington and the Legatum Institute in London, written
by Edward Lucas (a journalist on this paper) and Peter Pomerantsev, accuses it of “seeding fear of Western institutions and alliances (Lithuania); fomenting insurrection (eastern Ukraine); general denigration of a country’s international reputation (Latvia);
the development of native pro-Kremlin media (the Czech Republic and Estonia); and support for far-right and ultra-nationalist
movements and sentiments (Poland).”
Having seen how effective Russian misinformation was in
splitting off Crimea from Ukraine, some in Washington feared
that Russia might try to swing the presidential election in favour
8
of Donald Trump. By revealing that Bernie Sanders, a popular
candidate on the left, was locked out by the powers in the Democratic Party, it made American politics look rigged. And by undermining Hillary Clinton and casting doubt on the result, it could
weaken her. That would be a fine day’s work for Russia’s leader, a
former KGB officer called Vladimir Putin.
However, a recent paper from the Aleksanteri Institute in
Finland points out that Ukraine was vulnerable because of its
weak government and the presence of large numbers of Russians in Crimea, including soldiers, and goes on to question
whether Russian tactics would work more generally. Another
study, by the Finnish Institute of International Affairs, also concluded that Russian strategic deception has its limits. The authors
did not find a single case of Russian misinformation bringing
about meaningful change in the West.
That leaves policymakers in a quandary. If governments
complacently leave misinformation unanswered, they risk the
spread of potentially harmful ideas. If, on the other hand, they
build up Russia’s actions into a grave threat, they also build up
Russia’s stature. That would be to do Mr Putin’s work for him.
China has emerged only recently as a great power in intelligence. During the Cultural Revolution its security services persecuted the opponents of Mao Zedong. As part of the purge, the
Central Investigation Department—which was to become the
Ministry ofState Security (MSS)—eliminated officers with foreign
experience who, by definition, included those in its foreign-intelligence service. China had little expertise in SIGINT.
Its chance to catch up came in the late 1990s, with the shift
from breaking codes to hacking computers. Peter Mattis, a China
expert at the Jamestown Foundation, compares the innovation
to the launch of Britain’s Dreadnought battleship a century ago,
which revolutionised naval warfare. China has used the communications revolution to become a world SIGINT power.
Much of its effort is still focused inward. Nigel Inkster, a China expert who was a senior intelligence officer with MI6 and is
now at the International Institute for Strategic Studies, reports
how a defector defined the role of the security services as first of 1
The Economist November 12th 2016
SPECIAL REPOR T
ESPIONAGE
2 all to “control the Chinese people to maintain the power of the
Chinese Communist Party”. Their task was to counter the “evil
forces” of separatism, terrorism and religious extremism. They
accomplished this partly through sheer manpower. In one district of 400,000 people, fully 4% of the population was on the
payroll, outstripping the East German Stasi in its pomp. But they
also make good use of technology. Mr Mattis explains how their
Golden Shield project tags potential troublemakers. You never
know who is under scrutiny. In 2015 Qiu Jin, an MSS vice-minister, was briefly arrested, possibly after requesting the bugging of
senior leaders.
In the 18th century Jeremy Bentham, a British philosopher,
invented a prison in which a single watchman could observe all
the prisoners all the time, calling it the Panopticon. Mr Mattis believes that Bentham’s idea is coming to life. “China’s goal”, he
says, “is as close as you’re going to get to the real Panopticon.”
As China’s interests have become more international, so
have the intelligence services. For many years their specialism
was industrial espionage. As early as 1987, Deng Xiaoping
launched “Plan 863” to establish China’s independence in strategic industries. One of the first hacks to be detected was Titan Rain
in 2003, in which terabytes of data were taken from Sandia National Laboratories, NASA and American defence contractors.
Over the years, Chinese hackers are believed to have
sucked out details of the B1bomber, the B2 Stealth bomber, an advanced submarine-propulsion system and a miniaturised nuclear warhead, as well as countless industrial and scientific processes. China was also suspected of stealing the blueprint of
Australia’s new intelligence headquarters. Even today, according
to Matt Brazil, another fellow at the Jamestown Foundation, China’s five-year plans tell you what technologies the country will
seek to obtain through research, deals or, if necessary, theft.
By 2013 the Obama administration had had enough. With
official blessing, a computer-security company called Mandiant
released a report saying that one of the main hackers was Unit
61398 of the People’s Liberation Army, based in Shanghai. Mandiant claimed to have spotted the unit inside no fewer than 141
companies. Known as Comment Crew, it included hackers with
aliases such as UglyGorilla, and had broken into companies
linked to electric power, water and natural gas. Once in, the hackers typically stayed for almost a year.
In 2014 the Department of Justice charged five members of
Comment Crew with hacking into American steel, solar and nuclear firms, and published mugshots of the hackers, including
UglyGorilla. Two of the men were in military uniform. America
also threatened to bring suits against Chinese companies, including Chinalco, Boasteel and State Nuclear Power Technology Cor-
The Economist November 12th 2016
poration. The threats seemed to work. Since Barack Obama and
Xi Jinping agreed to curb cyber-espionage in September last year,
far fewer Chinese hackers have been detected (see chart).
China is less well known for its HUMINT, but it does pursue
businesspeople with a background in Western governments.
Since the mid-1980s it has often used sex as a lure. According to
Mr Inkster, a Japanese diplomat committed suicide in Shanghai
in 2005, supposedly after having got caught in a honey trap.
China’s spying is a fundamental expression of its rise as a
great power and its growing rivalry with America—just as the creation of modern espionage and counter-espionage dates back to
Germany’s challenge to Britain at the start ofthe 20th century. No
longer is China interested principally in looking after the Chinese diaspora. Today it cares about American policy in, say, Japan and South Korea, as well as Brazil, where it buys its food, and
Saudi Arabia, where it buys its oil.
This has a dark side. According to Mr Inkster, China is convinced that America is exploiting its hold over the internet to perpetuate its hegemony and to spread subversion. That was one
reason why China helped Iran suppress the liberal Green Movement when it rose up against the mullahs in 2009. Both China
and Russia suspect that America uses the internet to try to inject
Western values into their countries. Mr Putin has described the
internet as a “CIA project”. China sees American condemnation
of hacking as hypocrisy. Last year the Xinhua news agency published an article entitled “The USA Talks of Cyber Security and
the World Laughs”.
This could have consequences. China has put forward a
“new security concept” in which international law is subordinate to national interests. In June the Global Commission on Internet Governance warned that governments might further Balkanise the internet, at a cost to the global economy and to
freedom of expression.
Intelligence will partly define relations between China and
America. It need not always lead to hostility. By helping each side
understand the other better, intelligence can also lower tensions—much as in the late 1950s satellites and spy planes diffused
American fears of a “missile gap” with the Soviet Union. But the
stakes are high. General Hayden thinks that most intelligence domains can withstand some mistakes. With intelligence towards
China, he says, there is no room for error. “No one else is in the
same area code. It’s pass-fail.” 7
9
SPECIAL REPOR T
ESPIONAGE
How to do better
The solace of the law
A blueprint for the intelligence services
IN THE SP MUSEUM in Washington, DC, a floor is given
over to James Bond. Pay attention to the villains, say ince
Houghton, the museum’s historian: each tells you what the West
was scared of when that particular film was made. Bond is sandwiched between the paraphernalia of real-life spying, including
bugs, silk maps and cipher pads. But his wayward independence
looms over the whole business. The thing about spies, says Kelley Ragland, who publishes modern spy novelists, including
Olen Steinhauer, is that they are lone wolves who survive without help. “They are underdogs,” she says. “We root for them.”
The intelligence officers featured in this special report break
some rules, too. All nations make espionage against them a criminal offence. They consider foreign citizens fair game, on the
ground that their duty is to maximise the well-being of their own
people. But at home, too, they can intrude into lives, playing on
people’s fears or vanities, issuing threats or offering money. The
question is how an open, democratic society should govern
their behaviour. Too much power and secrecy, and they will go
astray. Too little, and they will fail.
Sir David Omand, a former head of Britain’s GCHQ, says
that lawful spying should be governed by ethics in the same way
that a just war is. And David Anderson, in his review of the Investigatory Powers bill, which by the end of this year will for the first
time put British intelligence on a unified statutory footing, offers
a blueprint for what this might look like.
Five principles
Because of the need for security, he argues for minimal
no-go areas. The state needs to be able in principle to bug bedrooms, read diaries and, if necessary, listen in to conversations
between lawyers and clients or journalists and sources. “The issue is when it should be lawful to exercise such powers,” he says,
“not whether they should exist at all”. Drawing on international
human-rights law, he sets out five principles for their use:
• The law must be accessible—easy to obtain and understand;
and it must operate in a foreseeable way.
• Spying must be necessary, which means more than useful. On
In principle,
the state
needs to be
able to bug
bedrooms,
read diaries
and listen to
privileged
conversations
10
September 12th 2001 necessity
was different from what it had
been on September10th.
• Measures must be proportionate. Squeezing privacy brings diminishing returns.
• There must be effective monitoring and oversight.
• There must be redress by an independent tribunal for those
who have been mistreated.
This legal footing serves as
a foundation. But the intelligence services also need to command public trust, says John Parachini of RAND. If you are seen
to deviate from expectations,
you run risks. Unless they explain why capabilities are needed, says Cortney Weinbaum,
also of RAND, agencies cannot
justify their budgets or programmes to voters and taxpayFossil fuels November 26th 2016
ers. As director of the NSA, MiLifelong education January 14th 2017
chael Hayden used to map out
Mass entertainment February 11th
what the agency did as
enn
2017
diagram with three circles, labelled technologically feasible,
operationally relevant and legal.
After he became director of the
CIA a few years later, he added a
fourth: politically sustainable.
The essential ingredient is transparency—or, rather, what
Michael Leiter, head of the National Counterterrorism Centre
under George W. Bush, has called “translucence”. The public
needs to know the broad outline of what the security service is
doing, but not the details.
Reporting to Barack Obama, the presidential advisory
group invoked what it termed the “front-page rule”: that the
agencies should forsake any programme which could not command the consent of ordinary people if leaked to a newspaper.
General Hayden thinks the intelligence services should be more
willing to let retired officers write books and speak to journalists.
“Too much is protected,” he says. “We need less secrecy. We need
to be the teller of our story, not the keeper of secrets.”
An effort to restrict classification is overdue, especially in
America, where nearly 1.5m people have top-secret clearance. In
2012 the presidential libraries contained 5bn pages waiting to be
reviewed for declassifying. Mr Parachini believes that a small
amount of secret intelligence must be guarded with extreme
care. Insights can come from publicly available sources at a small
fraction of the cost and be widely shared to prevent terrorist attacks or prepare for political and military surprises.
In terms of public relations, the West’s intelligence services
have endured a difficult decade and a half. In terms of their operations, however, the years since 9/11 have seen extraordinary
shifts in focus and capabilities. Increasingly, society is asking
them for protection from criminals and paedophiles as well as
terrorists and foreign powers. It is a vast agenda.
The rules governing their actions have not always kept pace
with the public mood. However, in fits and starts, the intelligence
services have adapted. It is right that they should be held to high
standards. But their critics should also remember that the world
is dangerous and hostile, and that the intelligence services are often the best protection ordinary people can hope for. 7
The Economist November 12th 2016