You've probably heard about Echelon, the vast listening system run by the US, UK, Canada and Australia that scans the world's voice traffic looking for key words and phrases.
Aside from using the system for industrial espionage and bypassing international and national laws to listen in on people, it is also used to listen out for people like Osama bin Laden and assorted terrorists in the hope of preventing attacks.
All this is out in the relative open thanks to investigative journalists and a European Commission report into the system, concerned and annoyed that the Brits and Yanks has got there first.
It works like this: The calls are recorded by geo-stationary spy satellites and listening stations, such as the UK's Menwith Hill, which combine satellite-intercepted calls and trunk landline intercepts and forward them on to centres, such as the US' Fort Meade, where supercomputers work on the recordings in real time.
But what, you ask, can deal with that overwhelming mass of data that helps our government spy on the world? And how does it work?
Well, a Texas Memory Systems SAM product - a combined solid-state disk (SSD) and DSP (digital signal processor). Woody Hutsell, an executive VP at TMS, said: "Fifty percent of our revenue this year will come from DSP systems, more than last year. The systems are a combination of SSD with DSP ASICs." ASICs are application-specific integrated circuits - chips dedicated to a specific purpose.
TMS has a TM-44 DSP chip which has 8 GFLOPS of processing power - that's eight billion floating point operations per second. The processing uses floating point arithmatic operations to supply the accuracy needed for the analysis. A DSP chip turns analogue signals from a sensor or recorder into digital information usable by a computer. Digital cameras will use a DSP to turn the light signals coming through the lens into digital picture element, or pixel, information.
A SAM-650 product is called a 192 GFLOPS DSP supercomputer by TMS. It is just 3U high and has 24 DSP chips and is positioned as a back-end number cruncher controlled by any standard server - a similar architecture to that used by Cray supercomputers. There are vast streams of information coming from recorded telephone conversations. The ability to have the DSPs work in parallel speeds up analysis enormously. Spinning hard drives can't feed the DSPs fast enough, nor are they quick enough for subsequent software analysis of the data. Consequently TMS uses its solid state technology to provide a buffer up to 32GB that keeps the DSPs operating at full speed.
A cluster of five SAM-650's provides a terra flop of processing power; one trillion floating point operations per second.
Echelon is a global surveillance network set up in Cold War days to provide the US goverment with intelligence data about Russia. One of the main contractors is Raytheon. Lockheed Martin has been involved in writing software for it. Since then it has expanded into a general listening facility, an electronic vacuum cleaner, sucking up the world's telephone conversations. Information about it's existence has been reluctantly revealed, prompted by scandals such as the recordings of Princess Diana's telephone calls by the NSA.
Recorded signals are fed into the TMS SAM systems where the DSPs filter out the noise to produce much clearer signals that software can work on to detect individual voices, perform voice recognition, and listen out for keywords, such as, for example, "Semtex". Decryption of encrypted calls is also a likely activity.
Hutsell says the SAM systems, "are supplied to intelligence agencies and the military though system integrators like Raytheon, Lockheed Martin and Zeta. It's an intelligence community application involving data from various sources. This is loaded into RAM and then real-time analysis is carried out on it. Step one is to filter out the noise and our DSP chips are used for that. Then they look into patterns using other tools - images or voice. It's very high-speed."
TMS has supplied its RAMsan high-speed SSD technology to several US government agencies. Hutsell said, "We have recently sold another terabyte system to a federal agency. It's installed in the DC [District of Columbia] area via our partnerVion. There's another in a government data centre with Oracle indices that needed to be accelerated."
TMS has had 40 percent year on year growth for three years. It has no debt and is privately-owned. Hutsell said: "This year is the healthiest year ever." Half the company's revenue comes from the government sector.
Fast, very fast, database and recorded signal access is the name of this game. The US government wants to know what you and I are talking about. Spy in the sky satellites listen in to what we say and look at what we do. Then solid state disk keeps the real time analysis of these calls and images operating at full speed. The world's fastest storage system is used in the world's most sophisticated spying operation.
Impressive and scary at the same time.