Lux Ex Umbra: Mistaken metadata-sharing went on for years

The CSE Commissioner's classified report on CSE's bungled metadata-sharing program, parts of which have been made public during the BCCLA's lawsuit against CSE, indicates that the agency's failure to properly remove information that could identify Canadians went on for years and involved both DNR (phone-related) and DNI (Internet-related) metadata.

From the Globe and Mail's report (Colin Freeze, "Spy agency accidentally shared Canadians’ data with allies for years," Globe and Mail, 1 June 2016):

The confidential report was written by Jean-Pierre Plouffe, a retired Quebec judge who heads the Office of the CSE Commissioner, the spy agency’s watchdog agency. In it, he suggests the unlawful seepage of Canadians’ phone and Internet records to foreign intelligence agencies could date back to the mid-2000s, and that the overall amount of compromised material is unclear.

Given this, Mr. Plouffe is urging Parliament to pass laws spelling out how it wants the spy agency to function. “As CSE’s collection posture has strengthened, … the volume of metadata collected has increased considerably,” Mr. Plouffe writes in his 2015 report. He urged federal politicians to give clearer direction on surveillance.

“Metadata” are logs of communications without the content of the conversation. The watchdog’s report reveals that, during its international spying, CSE has been capturing phone logs and sharing them with allies since 2005. Internet logs have been shared since 2009.

In 2014, CSE suspended sharing both sorts of records when it realized its automated systems had failed to scrub out what it calls the “Canadian identifying information” that turned up in the wider mix. Mr. Plouffe, who has the last word on such matters, eventually ruled that although CSE’s system failures were inadvertent, they violated the Privacy Act and National Defence Act. ...

The report reveals that CSE refers to the phone logs it collects as “Dialled Number Recognition” (DNR) metadata. The agency started sharing such material with Five Eyes allies in 2005, thinking it had devised ways to automatically strike out telling portions of any Canadian phone numbers that turned up.

Then, starting two years ago, CSE discovered that “DNR metadata was not being minimized properly,” according to the watchdog report. Mr. Plouffe added: “CSE is unable to determine how many systems were impacted and for how long.”

CSE calls the Internet logs it collects “Digital Network Intelligence” (DNI) metadata, and this material can consist of e-mail addresses and Internet protocol addresses that indicate who is communicating to who.

A scrubbing system was developed for that material as well – but this, too, failed. “DNI metadata was being shared with [Five Eyes] Second Parties … with minimization applied to Canadian e-mail address fields, but no minimization applied to Canadian IP address fields,” Mr. Plouffe writes.

He adds that “CSE was under the impression that minimization was taking place, when in fact it was not.”

The spy agency suspended sharing when the problems were discovered in 2014, and apparently have not resumed it.