As Dean Vernon Wormer said in National Lampoon’s Animal House, “Well, as of this moment, they’re on double, secret probation.” It is one of my favorite movies and most will cheer on Delta as they get their revenge after being kicked out of school because of this. Unfortunately, having secret rules and regulations is the norm these days like the NSA’s (National Security Agency) Prism program (see figure). The FBI is involved and Prism is not the only program.
Massive snooping is not restricted to the U.S. The United Kingdom is tapping fiber optic cables to track voice and data trafffic (see GCHQ taps fibre-optic cables for secret access to world's communications). This information is being shared with the NSA giving it even a bigger data collection to play with.
Most know about the NSA’s use of drones but the FBI is starting to use them as well. Even local police and rescue groups are looking at them. There are significant benefits from using this technology but all of these programs are generating new kinds of data.
Obviously, keeping a program secret can be an advantage but the question is who watches the watchers? Trust us is not a great line given the track record of almost any organization going this kind of work. Small abuses are often overlooked and those in charge often have vested interests for keeping major ones hidden.
It remains to be seen if Edward Snowden is a hero, criminal or both (at this point there is a sealed criminal complaint). On the plus side his efforts have finally moved this discussion into the open. The challenge in the discussion is verifying any claims on the secret holders because few are allowed to examine the veracity of the claims.
There are tools for “wiretapping” and these are supposed to require a judge to issue the warrants. The Foreign Intelligence Surveillance Act (FISA) made secret warrants possible but Prism addresses meta-data like phone call source, destination and duration.
For Prism, a number of factors have come together to make it a high value proposition. One is digital communication and laws that force developers to include hooks for surveillance. The other is big data tools like Hadoop (see Essentials Of The Hadoop Open Source Project). The open source Apache Accumulo is one of the tools being used in the big data spacce. It builds on Apache Hadoop, Zookeeper, and Thrift. Tracking individual transactions is useful but tracking trends requires massaging large amounts of data. Having high speed, on-site hooks into service providers like Google and Microsoft means large amounts of data are readily available.
In the past, doing a wiretap required physical access to the switching systems. The on-site hooks now provide that in our digital environment. The problem is that most of these server sites have “lights out” operation. Unattended, remote management is the norm and controlling a drone is not the only thing that can be done from the other side of the planet. Who is doing what when will be logged but bypassing or modifying the logs is not just for movie plots.
One thing that is possible is to minimize the number of people involved in getting any particular piece of information. That number can be one even when obtaining large amounts of information. This highlights the efficiency and capabilities of a system but also increases the possibility for abuse.
The challenge is determining what is fact, fiction and fantasy. We in the industry have a good idea of what is possible and probable which is more than the average person, including our elected officials, knows about. Most will not understand how they work except at a very high level. For example, companies are already using browser information to track users.
Unfortunately many are willing to exchange an unknown level of privacy for an unknown level of security. Benjamin Franklin noted, those who would sacrifice freedom for security deserve neither.