In 2008, a team of software coders inside the National Security Agency started reverse-engineering the database that ran Google.
They closely followed the Google research paper describing BigTable — the sweeping database that underpinned many of the Google’s online services, running across tens of thousands of computer servers — but they also went a little further. In rebuilding this massive database, they beefed up the security. After all, this was the NSA.
Like Google, the agency needed a way of storing and retrieving massive amounts of data across an army of servers, but it also needed extra tools for protecting all that data from prying eyes. They added “cell level” software controls that could separate various classifications of data, ensuring that each user could only access the information they were authorized to access. It was a key part of the NSA’s effort to improve the security of its own networks.
But the NSA also saw the database as something that could improve security across the federal government — and beyond. Last September, the agency open sourced its Google mimic, releasing the code as the Accumulo project. It’s a common open source story — except that the Senate Armed Services Committee wants to put the brakes on the project.
In a bill recently introduced on Capitol Hill, the committee questions whether Accumulo runs afoul of a government policy that prevents federal agencies from building their own software when they have access to commercial alternatives. The bill could ban the Department of Defense from using the NSA’s database — and it could force the NSA to meld the project’s security tools with other open source projects that mimic Google’s BigTable.