WASHINGTON — The surveillance machine grew too big for anyone to understand.
The National Security Agency set it in motion in 2006 and the vast network of supercomputers, switches and wiretaps began gathering Americans’ phone and Internet records by the millions, looking for signs of terrorism.
But every day, NSA analysts snooped on more American phone records than they were allowed to. Some officials searched databases of phone records without even realizing it. Others shared the results of their searches with people who weren’t authorized to see them.
It took nearly three years before the government figured out that so much had gone wrong. It took even longer to figure out why.
Newly declassified documents released Tuesday tell a story of a surveillance apparatus so unwieldy and complex that nobody fully comprehended it, even as the government pointed it at the American people in the name of protecting them.
“There was no single person who had a complete technical understanding,” government lawyers explained to a federal judge in 2009.
During a summer in which former NSA systems analyst Edward Snowden released America’s surveillance secrets to the world, the Obama administration has repeatedly tried to reassure people that the NSA’s powers were kept in check by Congress and the courts. The mistakes discovered in 2009 have been fixed, the president said, a reflection of that oversight.
But the documents from the Foreign Intelligence Surveillance Court show that, in developing the world’s most sophisticated surveillance network, even senior lawyers and officials weren’t sure how the system worked and didn’t understand what they were told.
“It appears there was never a complete understanding among the key personnel regarding what each individual meant by the terminology,” lawyers wrote in March 2009 as the scope of the problems came into focus.
As a result, the judges on the surveillance court, who rely on the NSA to explain the surveillance program, approved a program that was far more intrusive than they believed.
“Given the executive branch’s responsibility for and expertise in determining how best to protect our national security, and in light of the scale of this bulk collection program, the court must rely heavily on the government to monitor this program,” Judge Reggie B. Walton wrote in a 2009 order that found the NSA had repeatedly misrepresented its programs.
In Congress, meanwhile, only some lawmakers fully understand the programs they have repeatedly authorized and are supposed to be overseeing. For instance, Rep. Jim Sensenbrenner, R-Wis., one of the sponsors of the USA Patriot Act, has said he never intended it to be used to collect and store the phone records of every American.
And when Director of National Intelligence James Clapper was asked whether the government was doing that, he testified, “No.” Yet Snowden’s revelations, published in Britain’s Guardian newspaper, show that is what happened.
There is no evidence in the new documents suggesting the NSA used its surveillance powers to spy on Americans for political purposes, a fear of many critics who recall the FBI’s intrusive monitoring of civil rights leaders and anti-war protesters in the 1960s. Instead, the documents blame the years of government overreaching on technical mistakes, misunderstandings and lack of training.
From 2006 through early 2009, for instance, the NSA’s computers reached into the database of phone records and compared them with thousands of others without “reasonable, articulable suspicion,” the required legal standard.
By the time the problems were discovered, only about 10 percent of the 17,835 phone numbers on the government’s watch list in early 2009 met the legal standard.
By then, Walton said he’d “lost confidence” in the NSA’s ability to legally operate the program. He ordered a full review of the surveillance.
In its long report to the surveillance court in August 2009, the Obama administration blamed its mistakes on the complexity of the system and “a lack of shared understanding among the key stakeholders” about the scope of the surveillance.
“The documents released today are a testament to the government’s strong commitment to detecting, correcting and reporting mistakes that occur in implementing technologically complex intelligence collection activities, and to continually improving its oversight and compliance processes,” Clapper said in a statement Tuesday.
The surveillance court was satisfied by those improvements; it allowed the NSA to continue collecting phone records every day, a practice that continues today.
Now, the Obama administration is fending off lawsuits and a push in Congress to rein in the surveillance.
An unusual coalition of liberal Democrats and Republican civil libertarians has proposed several bills that would either scrap the phone surveillance entirely or require more oversight.
President Barack Obama has said he’s open to more oversight but says the surveillance is essential to keep the country safe.
Obama and Clapper have said the changes made in 2009 resulted in tightened controls. American data is still collected but only seldom looked at, officials said. And it is kept on secure computer servers equipped with special software to protect it from analysts looking to illegally snoop.
“There are checks at multiple levels,” NSA Deputy Director John Inglis told Congress in July. “There are checks in terms of what an individual might be doing at any moment in time.”
The same checks that protect Americans’ personal data were also supposed to protect the NSA’s information. Yet Snowden, a 29-year-old contractor, managed to walk out with flash drives full of the nation’s most highly classified documents.
The NSA is still trying to figure out, in such a complex system, exactly how Snowden defeated those checks.
“I think we can say that they failed,” Inglis said. “But we don’t yet know where.”
