return to ICG Spaces home    ICG Risk Blog    discussions    newsletters    login    

ICG Risk Blog - [ Dataveillance is reborn from data mining: parts of TIA return ]

Dataveillance is reborn from data mining: parts of TIA return


Senator Akaka asked the GAO to survey federal data mining systems and activities by identifying "planned and operational federal data mining efforts and describe their characteristics." GAO found that 52 of 128 federal agencies and departments (CIA and NSA did not respond to GAO's inquiry) are using, or are planning to use, data mining. Of 199 identified mining efforts, 131 are operational and 68 are planned. Of those 199 data mining efforts, 54 used private-sector data that could include a vast array of corporate data and other data sets in private hands that go far beyond credit card data.

The six most common data mining goals across all departments were to:

  • Improve service or performance
  • Detect fraud, waste, and abuse
  • Analyze scientific and research information
  • Manage human resources
  • Detect criminal activities or patterns
  • Analyze intelligence and detect terrorist activities

The focus of each department varied: DoD led on improving service or performance, managing human resources, and analyzing intelligence and detecting terrorist activities; DoE (Education) led in detecting fraud, waste, and abuse, NASA led in analyzing scientific and research information, while detecting criminal activities or patterns was widely distributed.

Of the 54 using private-sector data , four, and possibly more, appear to reconstitute significant parts of Poindexter's discredited Total Information Awareness (TIA) Program which had been renamed Terrorist Information Awareness (remaining TIA) Program in May 2003 before it was ostensibly quashed.

It should be remembered that DARPA's Information Awareness Office (IAO) had many projects beyond TIA, given its mission to "imagine, develop, apply, integrate, demonstrate and transition information technologies, components and prototype, closed-loop, information systems that will counter asymmetric threats by achieving total information awareness":

  • Effective Affordable Reusable Speech-to-text (EARS) - automated speech-to-text transcription
  • Futures Markets Applied to Prediction (FutureMAP)
  • Genisys - database to be implemented as the info center for all IAO activities
  • Genoa/Genoa II - structured decision-making argumentation for Genisys and other data
  • Human Identification at a Distance (HumanID) - automated biometric identification to detect, recognize and identify humans at a distance
  • Translingual Information Detection, Extraction and Summarization (TIDES) - was to be integrated into TIA
  • Wargaming the Asymmetric Environment (WAE) - automation to predict terrorist attacks and predictive indicators based on terrorist motivations

In whole or in part, data mining has suffered disappointed expectations and very bad press and so like old TIA became a new TIA, data mining is reemerging as predictive analytics, using next generation technology and less aggressive expectations.

Watchdog groups are especially concerned about four of the programs that constitute a "dragnet" or dataveillance, a surveillance of large groups of people, on citizen and terrorist alike. All are operational and all use personal information, private sector data, and other agency data:

  • Verity K2 Enterprise (DIA) - Identify foreign terrorists or US citizens connected to foreign terrorism activities
  • PATHFINDER (DIA) - Analyst tool to rapidly analyze government and private sector databases
  • Analyst Notebook I2 (DHS) - Correlate events and people to specific information
  • Case Management Data Mart (DHS) - Manage law enforcement cases, including Customs cases; reviews case loads, status, and case relationships

I find it interesting that a number of these apps rise from commercial CRM (Customer relationship management), "a comprehensive approach which provides seamless integration of every area of business that touches the customer - namely marketing, sales, customer service and field support-through the integration of people, process and technology, taking advantage of the revolutionary impact of the Internet."

DATA MINING: Federal Efforts Cover a Wide Range of Uses
May 2004

Data Mining is Dead - Long Live Predictive Analytics!
Lou Agosta
Forrester Research
October 30, 2003

Information Technology and Dataveillance
Roger Clarke
Principal, Xamax Consultancy Pty Ltd, Canberra
Visiting Fellow, Department of Computer Science, Australian National University
Version of November 1987

Gordon Housworth


InfoT Public  Infrastructure Defense Public  


  discuss this article

<<  |  September 2019  |  >>
view our rss feed