Data mining for national security

Some links to information on CSE and data mining (previously blogged about here and here):

• The Mathematics of Information Technology and Complex Systems (MITACS) project, a network of academia, industry and the public sector, including CSE, that was created to "harness the power of the mathematical sciences to address the inherent complexity of modern industrial and societal problems for the benefit of all Canadians". Projects include Semi-Supervised Learning in Large Graphs:

  As part of ongoing collaborations with the Communications Security Establishment (CSE), we are applying unsupervised and semi-supervised learning methods to understand transactions on large dynamic networks, such as telephone and email networks. When viewed as a graph, the nodes correspond to individuals that send or receive messages, and edges correspond to the messages themselves. The graphs we address can be observed in real-time, include from hundreds to hundreds of thousands of nodes, and feature thousands to millions of transactions. There are two goals associated with this project: firstly, there is the semi-supervised learning task, and rare-target problem, in which we wish to identify certain types of nodes; secondly, there is the unsupervised learning task of detecting anomalous messages. For reasons of efficiency, we have restricted our attention to meta-data of message transactions, such as the time, sender, and recipient, and ignored the contents of messages themselves. In collaboration with CSE, we are studying the problem of counter-terrorism, a semi-supervised problem in which some terrorists in a large network are labeled, but most are not.... Another common feature of counter-terrorism problems is the fact that large volumes of data are often "streamed" through various collection sites, in order to provide maximal information in a timely fashion. A consequence of efficient collection of transactions on very large graphs is that the data itself can only be stored for a short time. This leads to a nonstandard learning problem, since most learning algorithms assume that the full dataset can be accessed for training purposes. Working in conjunction with CSE, we will devise on-line learning algorithms that scale efficiently with increasing volume, and need only use each example once.

• The National Program on Complex Data Structures project on Data Mining with Complex Data Structures, a team of academic researchers partnered with CSE and Generation 5 Technologies.
  
  
• Examples of recent Canadian work in the field:
  
  • Mining Transactional Data on Networks (PDF file), Hugh Chipman with Erika Nahm and Alberto Nettel-Aguirre.
    
  • Classification Models for Transactional Graph Data (PDF file), Erika Nahm. (Can't imagine where she might be working these days...)
    
  • Automated Social Network Analysis for Collaborative Work (PDF file), Larry Korba et al. ("The authors acknowledge the Communications Security Establishment for its support in this work.")
  
  
• CSE career opportunities in data mining
  
  
• A contrary view of the intelligence value of data mining: Effective Counterterrorism and the Limited Role of Predictive Data Mining (PDF file), Jeff Jonas and Jim Harper.