by Steven R. Corman

Ars Technica reported yesterday on a letter sent by Congressman Brad Miller to the Inspector General of the Director of National Intelligence.  Miller, who is Chairman of a House subcommittee on technology oversight, sounded the alarm over a current initiative called “Railhead,” which is designed to upgrade the central database that contains information on suspected terrorists. Miller claims that the upgrade process is flawed, and if allowed to continue will actually decrease performance relative to the existing TIDE system.

TIDE is used to do things like generate the government’s “no-fly” list. Like the movie Brazil, in which a fly dropping into a printer morphs the name of an air-conditioning repairman into that of a wanted terrorist, the TIDE system has been excoriated for generating an unacceptable number of false positives–including some that flagged U.S. Air Marshalls as potential terrorists.

But that’s not the worst of it.  According to Miller’s letter, the TIDE system is composed of data spread across 463 different tables, 295  of them undocumented.  Users in the government rely on Structured Query Language (SQL) to extract information from the database.

I’m sure that to most COMOPS Journal readers that sounds like a lot of technical mumbo-jumbo.  So to illustrate just what it means, here is an example of a SQL query that would be used to combine information from (or “join”) three tables, in this case to find records with missing information:

select * from (bdg left join res on bdg.bid = res.bid) left join dom on res.rid = dom.rid where dom.rid is NULL and res.rid is not NULL;

Again, that is to combine data from just three tables.

Now imagine the poor CIA analyst trying to write a statement to get information out of six or ten tables, when s/he may not even know the variable names (in the examle res.id, bdg.bid) in some of them.  It’s no wonder that Miller quotes CIA Chief Information Officer Al Tarasiuk as saying:

The thing the worries me most is that we have buried, in some database, some piece of information that a person might need access to [but] doesn’t have the access or the data is not available to them somehow.

The upgrade process is described as “incremental” and will actually result in a reduction of current capabilities of the TIDE system.  Given the complexity of TIDE that may be unavoidable.  Yet even the revisions for partial functionality have been trouble-plagued.  In one test, Railhead “passed 148 tasks, but did not complete 26 others and failed 42 tasks,” according to Miller’s letter.

Ars Technica reports that

The NCTC fired back Friday in a statement, calling Miller’s description “inconsistent with the facts” and complaining that his subcommittee “has had no interaction with the NCTC or the Intelligence Community on the Railhead Program.”

Maybe, but the number of tables in a database is a pretty straightforward fact, and one would think that the CIO of the CIA qualifies as a member of the “intelligence community.”  You can collect all the information  you want, but unless it’s backed up by sound management practices it never becomes knowledge.