UPDATE 2015-07-02: the Dutch government released a draft intelligence bill, which implements (some of) the outcomes of the review by the Dessens Committee, into public consultation. Details here.

Between February 2013 and December 2013, the ‘Dessens Committee’ reviewed the Dutch Intelligence & Security Act of 2002 (aka `WIV2002′). On December 2nd their final report was published (.pdf, in Dutch). The conclusions and recommendations in that report will be debated in Dutch Parliament in January 2014. Eventually a bill will emerge for revising the WIV2002.

The report is an excellent 196-page read that covers the entire WIV2002, relevant parts of Parliamentary history, reflection on oversight in other countries, collaboration between the AIVD and MIVD, collaboration with foreign agencies, oversight on spy involvement in crime as part of investigations, and more. In this post I only address the parts concerning SIGINT (undirected interception) and wiretapping (directed interception).

As was generally expected, the committee recommends that SIGINT powers be extended from only non-cablebound (current situation) to cablebound communication as well. The report does not mention the annually recurring SIGINT oversight failure I described here, but does suggest changes to strengthen the oversight, including a proper specification of the motivation includes in the request for Ministerial approval for exercise of special powers. Lack of motivation was one of the main problems that led me to state that oversight on SIGINT is currently “still broken”. Other issues remain IMHO, including composition of the Review Committee on the Intelligence and Security Services (CTIVD) — which will be entirely composed of former crimefighters as of January 1st 2014 (a former District Attorney, a former police chief and a former chairman of the Attorneys General). More on that later. [UPDATE 2014-12-26: based on oversight reports that have been published by the new committee throughout 2014, I revoke these initial doubts.]

Below is my translation of the Dessens Committee’s conclusions and recommendations concerning the use of special powers in the digital world (wiretapping, SIGINT, hacking). Parts in [] are mine. Below that is my translation of the new legal system for interception that is proposed (note: the Dessens Committee did not propose a bill: a bill will probably emerge after the Parliamentary debate on surveillance etc. that is scheduled in January 2014).

I welcome your questions, corrections and requests for translation: koot at cyberwar dot nl (email) or @mrkoot (Twitter).

WARNING: this is an unofficial translation.

8.5 Conclusions and recommendations concerning the use of special powers in the digital world

The committee finds that the WIV2002, regarding the interception rules in Article 26 and Article 27, is ‘technology-dependent’ and has become outdated by technological advances and new communication possibilities. Technological development is here to stay, will only increase, and offers both advantages and disadvantages, including to the intelligence and security services.

Original Dutch: “De evaluatiecommissie constateert dat de Wiv wat betreft de interceptiebepalingen 26 en 27 Wiv ‘techniekafhankelijk’ is opgesteld en door de voortschrijdende technologie en nieuwe communicatiemogelijkheden gedateerd is. Technologisering is niet meer weg te denken, neemt alleen maar toe en biedt zowel voor- als nadelen, ook voor de I&V-diensten.”

The committee concludes that the current distinction between “ether” and “cable”, resulting in technology-dependent interception provisions of the WIV2002 no longer fitting the rapidly advancing technological developments in the field of data traffic and communication. The current system of interception provisions does little justice to the necessary powers in the context of national security in the year 2013, but also no (longer) provides the necessary safeguards that should apply in the exercise of special powers. The potential effects of medium-dependent or technology-dependent constraints are too great to ignore. The committee finds that undirected interception of cablebound communications should be allowed, under simultaneous review and strengthening of the legal safeguards of permission and oversight, linked to the interception provisions.

Original Dutch: “De evaluatiecommissie komt tot de conclusie dat het huidige onderscheid tussen ‘de ether’ en ‘de kabel’, resulterend in techniekafhankelijke interceptiebepalingen van de Wiv, niet meer rijmt met de snel voortschrijdende technologische ontwikkelingen op het gebied van dataverkeer en communicatie. Het systeem van interceptiebepalingen doet anno 2013 te weinig recht aan de noodzakelijke bevoegdheden in het kader van de nationale veiligheid maar voldoet ook niet (meer) aan de noodzakelijke waarborgen die moeten gelden bij de uitoefening van genoemde bijzondere bevoegdheden. De potentiële gevolgen van medium- of techniekafhankelijke beperkingen zijn te groot om te veronachtzamen. De evaluatiecommissie vindt dat ongerichte kabelgebonden interceptie van communicatie moet worden toegestaan, onder gelijktijdige herziening en versteviging van de wettelijke waarborgen van toestemming en toezicht, verbonden aan de interceptiebepalingen.”

The committee is of the opinion that as the invasion of privacy and the confidentiality of communications is more intrusive, the authorization procedure and supervision should be stronger embedded. It should be the intrusiveness of cognizance of communication, and not the transport medium or the current state of technology, that determines the requirements for obtaining permission and the supervision of legality.

Original Dutch: “De evaluatiecommissie is van mening dat naarmate de inbreuk op de privacy en het communicatiegeheim indringender is, de toestemmingsprocedure en het toezicht ook sterker ingebed moeten zijn. Hierbij moet de indringendheid van kennisname van communicatie, en niet meer het transportmedium of de stand der techniek, bepalend zijn voor de toestemmingsvereisten en het toezicht op rechtmatigheid.”

In the opinion of the committee, a new dichotomy could be created that replaces the current Articles 25, 26 and 27. This dichotomy is related to the type of (interception) activity, the intrusiveness of cognizance of communication and hence the degree of privacy infringement and the associated requirements for obtaining permission. Undirected interception of cablebound communications and several variations of ‘search’ (as established by the Review Committee on the Intelligence and Security Services (CTIVD)) can be part of this dichotomy. Section 5.3.2 outlines a conceptual framework to this end. The committee recommends requiring Ministerial permission for ‘search’, and additional requirements and immediate supervision by the CTIVD as the invasion of privacy is more targeted and more pervasive. At the stage where infringements are potentially most intrusive, oversight occurs immediately after the permission has been granted.

Original Dutch: “Er zou naar de opvatting van de evaluatiecommissie een nieuwe tweedeling gemaakt kunnen worden, die de huidige artikelen 25, 26 en 27 vervangt. Deze tweedeling houdt verband met het type (interceptie)activiteit, de indringendheid van kennisname van communicatie en daarmee de mate van privacyinbreuk, en de hieraan gekoppelde toestemmingsvereisten. Ongerichte kabelgebonden interceptie en de verschillende varianten van search (zoals geconstateerd door de CTIVD) kunnen van deze tweedeling dan deel uitmaken. In paragraaf 5.3.2 is hiervoor een conceptueel raamwerk geschetst. De evaluatiecommissie beveelt een ministeriële toestemming aan voor search, alsook aanvullende toestemmingsvereisten en onmiddellijk toezicht door de CTIVD naarmate de inbreuk op de privacy gerichter en indringender wordt. In de fase waarin de inbreuken potentieel het meest indringend zijn vindt het toezicht onmiddellijk na de toestemmingverlening plaats.”

Extending this line of reasoning of the conceptual framework, the committee recommends that the requirements for permission in Article 24 [=legal power to hack systems] and the oversight on these powers be aligned with the second phase of the legal framework for interception. This means that permission for the use of Article 24 powers should be given by the Minister himself and that the CTIVD immediately oversees the mandates. The committee also recommends that immediate oversight also be applied to the powers provided by Article 20, paragraph 3, and Article 22, paragraph 6, The committee also recommends to act with restrain concerning delegation of authority for exercising powers that can infringe on rights.

Original Dutch: “In het verlengde van de bovenstaande gedachtelijn van het conceptueel raamwerk beveelt de evaluatiecommissie aan om het toestemmingsvereiste in artikel 24 en het toezicht op deze bevoegdheden in lijn te brengen met de tweede fase van het interceptieraamwerk dat is geschetst. Dat betekent dat de toestemming voor de inzet van artikel 24 door de minister zelf moet worden gegeven en de CTIVD onmiddellijk toezicht houdt op de lastgevingen. De evaluatiecommissie beveelt aan om het onmiddellijk toezicht ook op de bevoegdheden ex artikel 20, lid 3, en 22, lid 6, te laten zien. De evaluatiecommissie beveelt verder aan om terughoudend te zijn met het (door)mandateren van bevoegdheden die inbreuk kunnen maken op grondrechten.”

The committee recommends that the legislature, in drafting a new bill to amend the WIV2002, explicitly accounts for the criteria defined by the motion Franken. Considering that it will be new legislation involving a restriction of privacy rights, the committee recommends that in the legislative process and in the final decision, the requirements of the motion Franken are included, as already promised by the Minister of Interior.

Original Dutch: “De evaluatiecommissie adviseert de wetgever, bij het formuleren van een nieuw wetsvoorstel tot wijziging van de Wiv 2002, om zich nadrukkelijk rekenschap te geven van (de criteria in) de motie-Franken. Aangezien het hier zal gaan om nieuwe wetgeving waarbij sprake is van een beperking van het recht op eerbiediging van de persoonlijke levenssfeer, beveelt de evaluatiecommissie aan dat in het wetgevingsproces en in de uiteindelijke besluitvorming hierover de vereisten van de motie-Franken worden meegenomen, zoals reeds toegezegd door de Minister van BZK.”

The committee has examined whether the oversight of the use of special powers is currently sufficiently strong. The committee has also included in this Article 13 of the Dutch Constitution [confidentiality of mail], the jurisprudence of the ECHR and developments in the neighboring countries involved in this issue. An analysis has been made of the different variants of oversight before and after the use of intelligence means. On this basis, the committee recommends to strengthen the oversight afterwards. In the view of the committee, reinforced oversight (afterwards) combined with better designed control (before) and other permission requirements result in better consistency of safeguards in the use of special powers.

Original Dutch: “De evaluatiecommissie heeft zich gebogen over de vraag of het huidige toezicht op de inzet van bijzondere bevoegdheden voldoende sterk is. De evaluatiecommissie heeft bij dit vraagstuk ook artikel 13 van de Grondwet, de jurisprudentie van het EHRM en ontwikkelingen in de landen om ons heen betrokken. Er is een analyse gemaakt van de verschillende varianten van toezicht vooraf en achteraf op de inzet van inlichtingenmiddelen. Op basis daarvan doet de evaluatiecommissie de aanbeveling om het toezicht achteraf te versterken. In de visie van de evaluatiecommissie zal verstevigd toezicht (achteraf), gecombineerd met beter vormgegeven sturing (vooraf) en andere toestemmingsvereisten resulteren in een betere samenhang van waarborgen bij de inzet van bijzondere bevoegdheden.”

The committee recommends making the assessments of legality by the CTIVD legally binding. The committee finds that the legally binding assessment by the CTIVD must cover all special powers. The committee believes it is not desirable that the WIV2002 leaves open the possibility that Ministers lay down opinions of CTIVD beside them. If the CTIVD concludes that a charge is issued illegally, the intelligence service should immediately cease the use of this special power. An exception should apply to the authority to open letters and other consignments (Article 23, first paragraph), for which a judicial approval should be asked, and for the use of special powers to trace the sources of journalists. In these situations a judicial approval will be prescribed. The legislature may also consider giving the CTIVD the power to suspend a mandate at first. There are situations where the (il)legality of a mandate is not immediately clear, and explanation or a more detailed justification is needed to use a specific power. In addition, the committee recommends that the CTIVD immediately oversees the mandates in the second phase of the legal framework for interception outlined by the committee. The committee recommends performing oversight as soon as possible after the permission is granted, because with these powers a major invasion of privacy potentially occurs. It is thereby necessary to expand the CTIVD, at least with respect to the departmental support.

Original Dutch: “De evaluatiecommissie beveelt daarbij aan om de rechtmatigheidsoordelen van de CTIVD een juridisch bindende kracht te geven. De evaluatiecommissie vindt dat het bindend rechtmatigheidsoordeel van de CTIVD moet zien op alle bijzondere bevoegdheden. De evaluatiecommissie acht het namelijk niet wenselijk dat de Wiv de mogelijkheid openlaat dat ministers een dergelijk rechtmatigheidsoordeel van de CTIVD naast zich neer leggen. Als de CTIVD tot de conclusie komt dat een afgegeven last onrechtmatig is, moet de dienst de uitvoering van deze bijzondere bevoegdheid, voor zover deze op dat moment al is aangevangen, direct staken. Een uitzondering moet gelden voor de bevoegdheid om brieven en andere geadresseerde zendingen te openen, waarvoor een rechterlijke last moet worden gevraagd (artikel 23, eerste lid, Wiv), en voor de inzet van bijzondere bevoegdheden om de bronnen van journalisten te achterhalen, als ook in die situaties een rechterlijke last zal worden voorgeschreven. De wetgever kan daarbij ook overwegen om de mogelijkheid te creëren dat de CTIVD de bevoegdheid krijgt om een lastgeving eerst te schorsen. Er zijn situaties denkbaar waarin de (on)rechtmatigheid van een lastgeving niet direct glashelder is en zij behoefte heeft aan een nadere uitleg of een uitgebreidere motivering van de noodzaak om een bepaalde bevoegdheid in te zetten. Daarnaast beveelt de evaluatiecommissie aan dat de CTIVD onmiddellijk toezicht uitoefent op de lastgevingen in de tweede fase van het interceptiestelsel dat de evaluatiecommissie heeft geschetst. De evaluatiecommissie beveelt aan om dit toezicht zo snel mogelijk na de toestemmingverlening te verrichten omdat bij deze bevoegdheden potentieel een grote inbreuk op de privacy plaatsvindt. Het is daarbij noodzakelijk om de CTIVD navenant uit te breiden, in ieder geval voor wat betreft de ambtelijke ondersteuning.”

The committee explains the following about the dichotomy they suggest as a replacement for Articles 25, 26 and 27:

5.3.2 Framework for a future legal system for interception
(…)

The first article, the first step in a new system, would give intelligence services the power to intercept telecommunications and test the data for usefulness/fitness. The committee finds that the power to ‘search’ must be subject to Ministerial approval. The committee is of the opinion that also in an exploration using general keywords, you should be able to justify what you’re looking for and why. Purpose and necessity must be demonstrated here too. The committee considers a mandate to be necessary for search. The mandate is possibly long-term but delineated in time and can be quite generic in nature but must be bound to a particular investigation. The committee herewith aligns with a judgement in CTIVD-report 28: “The fact that during search, contents of communication are [only] known for a brief period and do not involve the entire contents of the communications, does not change the fact that indeed infringement is made on the right to confidentiality of telephone and telegraph communications laid down in Article 13 of the Constitution.” From this, the CTIVD concludes that searching requires advance authorization in the sense of Article 13 of the Constitution. The committee emphasizes the recommendation to introduce a required mandate in a new legal system for interception that involves increased powers.

Original Dutch: “Een eerste artikel, de eerste trede in een nieuw stelsel, zou de diensten de bevoegdheid kunnen geven om telecommunicatie te mogen intercepteren en de gegevens te toetsen op bruikbaarheid. De evaluatiecommissie vindt dat de bevoegdheid tot ‘search’ onderworpen moet zijn aan ministeriële toestemming. De evaluatiecommissie is van mening dat je ook bij een verkenning met algemene zoektermen moet kunnen beargumenteren waarnaar je op zoek bent en waarom. Nut en noodzaak moeten ook hier kunnen worden aangetoond. Een lastgeving voor search, die eventueel langlopend maar wel afgebakend in tijd en vrij generiek van aard kan zijn maar in ieder geval gekoppeld moet zijn aan een bepaald onderzoek, acht de commissie dus noodzakelijk. Hiermee sluit de evaluatiecommissie aan bij een oordeel van de CTIVD in haar toezichtrapport nr. 28: ‘Ook het gegeven dat bij het searchen voor een korte tijd wordt kennisgenomen van de inhoud van de communicatie en dat het niet gaat om de volledige inhoud van de communicatie doet niets af aan het feit dat wel degelijk inbreuk wordt gemaakt op het in artikel 13 van de Grondwet neergelegde telefoon- en telegraafgeheim’. De CTIVD concludeert hieruit dat aan de uitoefening van de bevoegdheid te searchen een machtiging in de zin van artikel 13 van de Grondwet vooraf dient te gaan. De evaluatiecommissie benadrukt de aanbeveling om bij de introductie van een nieuw stelsel van interceptiebepalingen, waarbij sprake is van verruimde mogelijkheden, een lastgeving voor search in te voeren.”

By allowing search of cablebound communications, the intelligence services, even for the sole fact that the cable infrastructure is privately owned and the necessity to funnel the huge amount of data traffic (i.e., filtering for relevance to their tasks), end up in the situation that they need to make agreements with telecom providers about cable infrastructures and internet exchanges. The committee is of the opinion that the choice of a provider and the choice of an exchange or cable, to then ‘search’ it, must be subject to Ministerial permission, that then also involves an order for the provider to provide access to a particular cable.

 

Original Dutch: “Met het toestaan van kabelgebonden search komen de diensten in de situatie, alleen al vanwege het feit dat de kabelinfrastructuur in private handen is en vanwege de noodzaak om de enorme hoeveelheid dataverkeer te trechteren (dat wil zeggen te filteren op relevantie voor hun taakuitvoering), dat zij afspraken moeten maken met telecomaanbieders over kabelinfrastructuren en internetknooppunten. De evaluatiecommissie is van mening dat de keuze van een aanbieder en de keuze van een knooppunt of kabel, om daarop vervolgens te ‘searchen’ onderworpen moet zijn aan een ministeriële toestemming, die tevens een opdracht inhoudt voor de provider om toegang te verlenen tot een bepaalde kabel.”

The second step in a new legal system for interception could be said to be the phase in which the collected data are used. This [‘usage’] phase both the analysis (step 2a) and the subject-oriented in-depth investigation (step 2b). The analysis (2a) concerns the data that are collected via the aforementiond power of search. With a new Ministerial approval, the intelligence services could determine the relevance of certain traffic through metadata analysis and selection. In step 2a, further funneling or filtering of the (streams of) communication to be examined takes place through a further selection based on numbers, technical characteristics or keywords. This step includes elements of the current Article 27 but at the same time provides a legal basis for the two variations of search [already] carried out by the MIVD that are contended by the CTIVD and involve an additional, and sometimes new selection.

The committee is strongly in favor of a proper specification of the motivation that must precede the analysis step (2a). With the [introduction of] permission of undirected interception of cablebound communication, the intelligence services must provide  insight in the actual considerations[/deliberation] concerning necessity, proportionality and subsidiarity, and must include these in requests for permission sent to the Minister, specifying the person, organization, region or phenomenon that the investigation is targeting. Hence the committee is of the opinion that this power can only be exercised by the intelligence services after a new Ministerial approval. It involves a more invasive infringement on privacy than the aforementioned power of search, meaning that in the view of the committee, a new safeguard is necessary in the form of Ministerial approval.

Original Dutch: “De tweede trede in een nieuw stelsel van interceptiebepalingen zou aangeduid kunnen worden als de fase waarin de gegevens die zijn binnen gehaald, worden gebruikt. In deze gebruikfase vindt zowel de analyse plaats (trede 2a) als het subjectgerichte diepteonderzoek (trede 2b). De analyse (2a) heeft betrekking op de data die via de voornoemde searchbevoegdheid zijn verkregen. Met een nieuwe ministeriële toestemming zouden de diensten via metadata-analyse en selectie de relevantie van bepaald verkeer kunnen vaststellen. In fase 2a vindt aldus door een nadere selectie aan de hand van nummers, technische kenmerken of trefwoorden een verdere trechtering of filtering van de te onderzoeken communicatie(stromen) plaats. Deze fase herbergt elementen van het huidige art. 27 Wiv in zich maar geeft tegelijkertijd een wettelijke basis aan de twee door de CTIVD ‘betwiste’ search-varianten van de MIVD waarin sprake is van aanvullende, soms nieuwe selectie.
De evaluatiecommissie is sterk voorstander van een deugdelijke specificatie van de motivering die vooraf moet gaan aan de analysestap (trede 2a). Zo zullen de diensten, met het toestaan van ongerichte kabelgebonden interceptie, ruim inzicht moeten geven in de daadwerkelijk gemaakte afwegingen omtrent noodzakelijkheid, proportionaliteit en subsidiariteit en deze moeten opnemen in de verzoeken om toestemming aan de minister, toegespitst op de persoon, organisatie, regio of fenomeen waar het onderzoek op is gericht. De evaluatiecommissie is dan ook van oordeel dat deze bevoegdheid alleen kan worden ingezet door de diensten na een nieuwe ministeriële toestemming. Het gaat hier om een verdergaande inbreuk op de privacy dan de voornoemde searchbevoegdheid waardoor naar het oordeel van de evaluatiecommissie een nieuwe waarborg in de vorm van een ministeriële toestemming noodzakelijk is.”

Step 2b in a new legal system for interception should concern the actual use at the subject[/person] level of collected communication data and directed intercepts, as currently laid down in Article 25. The intelligence services would be permitted to examine the contents of the intercepted communication after Ministerial approval. In this last step, the communication that is considered to be relevant has been filtered with the aim of further, subject-oriented, investigation. In this step, the Ministerial approval must comply with the safeguards related to the largest infringement on citizen rights. The committee therefore considers it to be a safeguard if the intelligence services apply a functional separation [/separation of duty] between those who collect the data (in step 1) and those who process the data for the purpose of use (in step 2). The committee finds it reassuring that an organizational and/or functional separation will exist between an ‘interception unit’ (the Joint Sigint Cyber Unit) and its two clients, the AIVD and MIVD. Both services are ultimately the institutions that determine whether intercepts are relevant, processed into intelligence products and distributed to intelligence consumers.

Original Dutch: “Deel b van de tweede trede (2b) in een herzien stelsel van interceptiebepalingen zou toezien op het daadwerkelijk op subjectniveau gebruiken van verzamelde communicatiegegevens en gerichte interceptie, zoals vastgelegd in het huidige artikel 25. Hierbij zouden de diensten de inhoud van geïntercepteerde communicatie mogen onderzoeken na een nieuwe ministeriële toestemming. In deze laatste stap is het relevant geachte communicatieverkeer gefilterd ten behoeve van verder, op het subject gericht, onderzoek. In deze stap zou een afgegeven ministeriële toestemming moeten voldoen aan de waarborgen die gelden bij de meest vergaande inbreuk op de grondrechten van burgers. De evaluatiecommissie beschouwt het ook als een waarborg als er bij de diensten een functionele scheiding wordt aangebracht tussen diegenen die communicatiegegevens verwerven (in trede 1) en zij die deze gegevens verwerken ten behoeve van het gebruik (in trede 2). De commissie vindt het vertrouwenwekkend dat er straks een organisatorische en/of functionele scheiding is tussen een ‘interceptie-eenheid’ (straks de Joint Sigint Cyber Unit) en twee opdrachtgevers, zijnde de AIVD en de MIVD. Beide diensten zijn uiteindelijk ook de instanties die bepalen of de intercepts relevant zijn, verwerkt worden tot inlichtingenproducten en verspreid worden naar afnemers.”

Related:

• 2013-11-09: Oversight on Dutch SIGINT is still broken (blog)

EOF

UPDATE 2014-02-21: according to this (.pdf) declassified document “Recall from OVSC1100, the Overview of Signals Intelligence Authorities, that we learned that in addition to E.O. 12333, NSA may perform SIGINT functions under various FISA authorities to include NSA FISA, FBI FISA, FISA Amendments Act (FAA) Section 702, 704, and 705(b). While there are specific rules governing when and how these authorities may be applied, each of these authorities has the potential to provide a valuable and unique complement to our E.O. 12333 collection resources. Similarly, the BR and PR/TT Bulk Metadata Programs provide analysts with another opportunity to gain unique collection on a target. By leveraging as many of these various collection authorities available to them as permitted, analysts can fill existing knowledge gaps on their target. (…) One prime example of how an analyst leveraged several of these collection authorities to close crucial knowledge ga ps on a target occurred in Fall 2009, when a CT analyst pieced together information obtained from E.O. 12333, FAA 702, and BR FISA authorities to reveal a terrorist plot on the New York subway system, which was subsequently disrupted by the FBI.”
UPDATE 2014-01-{23,24}: in the U.S., the Privacy & Civil Liberties Oversight Board (PCLOB), an ‘independent federal privacy watchdog’ has found the telephony metadata collection under Section 215 to be illegal in their new report (.pdf, Jan 23). NY Times reported about it here, WaPo here, Ars Technica here, FAS here.
UPDATE 2014-01-17: DNI declassified 24 FISC orders approving NSA’s collection, use of telephony metadata under FISA Section 501, commonly referred to as ‘Section 215’, as does the EU-US Working Group report. Journalist Marcy Wheeler (@emptywheel) notes: “Of particular note, though, they seem to be withholding the BR 09-15 primary order, which was right in the middle of PATRIOT reauthorization, when NSA kept disseminating results in violation of Reggie Walton’s orders.”

UPDATE 2013-12-16: a partially declassified list of EU participants to the Working Group can be found here (.pdf).
UPDATE 2013-12-09: here is an excerpt from Viviane Reding’s speech (.doc) during the Civil Liberties Committee hearing of December 9th on Data Protection and U.S. Surveillance European Parliament/Strasbourg (original emphasis):

[…]

Let’s be honest. Some questions were not answered. The report is clear on this point. We know little about the use of some US legal bases on data collection (such as executive orders), the existence of other surveillance programmes, as well as limitations applicable to these programmes.

Many questions were answered. They are the raw material, the basis, of the recommendations that the Commission has made.

I would draw three main conclusions from the discussions.

First, the U.S. confirmed that these programmes exist and that their scope is broad. We had long discussions about the purpose of the surveillance programmes, and the conditions under which data can be collected and processed under U.S. law.

Second, the conditions and safeguards which apply are discriminatory. They protect EU citizens only to a limited extent. Whilst there are procedures regarding the targeting and minimisation of data collection for U.S. citizens, these procedures do not apply to EU citizens, even when they have no connection with terrorism, crime or any other unlawful or dangerous activity. In addition, while U.S. citizens benefit from constitutional protections, these do not apply to EU citizens not residing in the U.S.

Third, while some judicial oversight exists, it is of little added value from the perspective of a European. The orders of the Foreign Intelligence Surveillance Court, the FISA Court, are secret and companies providing assistance are required to maintain secrecy. There are no avenues (judicial or administrative), for either EU or U.S. data subjects to be informed whether their personal data is being collected or further processed. There are no opportunities for individuals to obtain access, rectification or erasure of data, or administrative or judicial redress.

While there are oversight mechanisms by the three branches of the U.S. Government, it is clear that they have loopholes. You are aware of the internal U.S. debates on this point.

In any case, there is no judicial oversight at all on the collection of foreign intelligence outside the U.S., which is conducted under the sole competence of the Executive Branch.

[…]

In addition, the following four steps for rebuilding trust in EU-US data flows “stand out” (for details, see the original document):

1. a swift adoption of the EU Data Protection Reform.
2. we must make Safe Harbour safer.
3. we have to agree strong data protection rules in the law enforcement context.
4. we must ensure that European concerns are addressed in the ongoing U.S. reform process.

 
============ ORIGINAL POST IS BELOW THIS LINE ============

The EU co-chairs of the ad hoc EU-US Working Group on Data Protection presented (.pdf, Nov 27) their findings in a report. The Working Group was established “to establish the facts about US surveillance programmes and their impact on fundamental rights in the EU and personal data of EU citizens” (p.2).

Summary of main findings (cited from p.26/27; original emphasis):

1. Under US law, a number of legal bases allow large-scale collection and processing, for foreign intelligence purposes, including counter-terrorism, of personal data that has been transferred to the US or is processed by US companies. The US has confirmed the existence and the main elements of certain aspects of these programmes, under which data collection and processing is done with a basis in US law that lays down specific conditions and safeguards. Other elements remain unclear, including the number of EU citizens affected by these surveillance programmes and the geographical scope of surveillance programmes under Section 702.
2. There are differences in the safeguards applicable to EU data subjects compared to US data subjects, namely:
   • i. Collection of data pertaining to US persons is, in principle, not authorised under Section 702. Where it is authorised, data of US persons is considered to be “foreign intelligence” only if necessary to the specified purpose. This necessity requirement does not apply to data of EU citizens which is considered to be “foreign intelligence” if it relates to the purposes pursued. This results in lower threshold being applied for the collection of personal data of EU citizens.
   • ii. The targeting and minimisation procedures approved by FISC under Section 702 are aimed at reducing the collection, retention and dissemination of personal data of or concerning US persons. These procedures do not impose specific requirements or restrictions with regard to the collection, processing or retention of personal data of individuals in the EU, even when they have no connection with terrorism, crime or any other unlawful or dangerous activity. Oversight of the surveillance programmes aims primarily at protecting US persons.
   • iii. Under both Section 215 and Section 702, US persons benefit from constitutional protections (respectively, First and Fourth Amendments) that do not apply to EU citizens not residing in the US.
3. Moreover, under US surveillance programmes, different levels of data protection safeguards apply to different types of data (meta-data vs. content data) and different stages of data processing (initial acquisition vs. further processing/analysis).
4. A lack of clarity remains as to the use of other available legal bases, the existence of other surveillance programmes as well as limitative conditions applicable to these programmes. This is especially relevant regarding Executive Order 12333.
5. Since the orders of the FISC are classified and companies are required to maintain secrecy with regard to the assistance they are required to provide, there are no avenues, judicial or administrative, for either EU or US data subjects to be informed of whether their personal data is being collected or further processed. There are no opportunities for individuals to obtain access, rectification or erasure of data, or administrative or judicial redress. 
6. Various layers of oversight by the three branches of Government apply to activities on the base of Section 215 and Section 702. There is judicial oversight for activities that imply a capacity to compel information, including FISC orders for the collection under Section 215 and annual certifications that provide the basis for collection under Section 702. There is no judicial approval of individual selectors to query the data collected under Section 215 or tasked for collection under Section 702. The FISC operates ex parte and in camera. Its orders and opinions are classified, unless they are declassified. There is no judicial oversight of the collection of foreign intelligence outside the US under Executive Order 12333, which are conducted under the sole competence of the Executive Branch.”

For my own purposes I keep my notes here (emphasis is mine):

1. AIM AND SETTING UP OF THE WORKING GROUP

p.2: “Given the central position of US information and communications technology companies in the EU market, the transatlantic routing of electronic data flows, and the volume of data flows across the Atlantic, significant numbers of individuals in the EU are potentially affected by the US programmes.”

p.3: “The report is based on information provided by the US during the meetings of the ad hoc EU-US working group, as well as on publicly available documents, including classified documents disclosed in the press but not confirmed by the US. (…) The US was provided with an opportunity to comment on possible inaccuracies in the draft. The final report has been prepared under the sole responsibility of the EU-co chairs.”

p.3: “The scope of the discussions was also limited by operational necessities and the need to protect classified information, particularly information related to sources and methods. The US authorities dedicated substantial time and efforts to responding to the questions asked by the EU side on the legal and oversight framework in which their Signal Intelligence capabilities operate.”

2. THE LEGAL FRAMEWORK

p.4: “Two legal authorities that serve as bases for the collection of personal data by US intelligence agencies are: Section 702 of the Foreign Intelligence Surveillance Act of 1978 (FISA) (as amended by the 2008 FISA Amendments Act, 50 U.S.C. § 1881a); and Section 215 of the USA PATRIOT Act 2001 (which also amended FISA, 50 U.S.C. 1861). The FISA Court has a role in authorising and overseeing intelligence collection under both legal authorities.”

p.5: “The US further clarified that not all intelligence collection relies on these provisions of FISA; there are other provisions that may be used for intelligence collection. The Group’s attention was also drawn to Executive Order 12333, issued by the US President in 1981 and amended most recently in 2008, which sets out certain powers and functions of the intelligence agencies, including the collection of foreign intelligence information. No judicial oversight is provided for intelligence collection under Executive Order 12333, but activities commenced pursuant to the Order must not violate the US constitution or applicable statutory law.“

2.1. Section 702 FISA (50 U.S.C. § 1881a)

p.5: “Section 702 FISA provides a legal basis for the collection of “foreign intelligence information” regarding persons who are “reasonably believed to be located outside the United States. (…) Under Section 702, information is obtained “from or with the assistance of an electronic communication service provider”.

p.5: “The US confirmed that it is under Section 702 that the National Security Agency (NSA) maintains a database known as PRISM. This allows collection of electronically stored data, including content data, by means of directives addressed to the main US internet service providers and technology companies providing online services, including, according to classified documents disclosed in the press but not confirmed by the US, Microsoft, Yahoo, Google, Facebook, PalTalk, AOL, Apple, Skype and YouTube.“

p.6: “The US also confirmed that Section 702 provides the legal basis for so-called “upstream collection“; this is understood to be the interception of Internet communications by the NSA as they transit through the US (e.g. through cables, at transmission points).”

p.6: “Section 702 does not require the government to identify particular targets or give the Foreign Intelligence Surveillance Court (hereafter ‘FISC’) Court a rationale for individual targeting. Section 702 states that a specific warrant for each target is not necessary.

The US stated that no blanket or bulk collection of data is carried out under Section 702, because collection of data takes place only for a specified foreign intelligence purpose.“

p.10: “Declassified FISC opinions confirm that US intelligence agencies have recourse to methods of collection under Section 702 that have a wide reach, such as the PRISM collection of data from internet service providers or through the “upstream collection” of data that transits through the US.”

The EU asked for specific clarifications on the issue of collection of or access to data not located or not exclusively located in the US; data stored or otherwise processed in the cloud; data processed by subsidiaries of US companies located in the EU; and data from Internet transmission cables outside the US. The US declined to reply on the grounds that the questions pertained to methods of intelligence collection.“

2.2. Section 215 US Patriot Act (50 U.S.C. § 1861)

p.10/11: “Section 215 of the USA-Patriot Act 2001 (…) permits the Federal Bureau of Investigation (FBI) to make an application for a court order requiring a business or another entity to produce “tangible things”, such as books, records or documents, where the information sought is relevant for an investigation to obtain foreign intelligence information not concerning a United States person or to protect against international terrorism or clandestine intelligence activities.

The US confirmed that this provision serves as the basis for a programme of intelligence collection via orders obtained by the FBI from the FISC directing certain telecommunications service providers to provide specified non-content telephony “meta-data”. For that programme, the information is stored by the NSA and queried only for counter-terrorism purposes.

That programme is limited to the collection of call detail records, or telephony “meta-data” maintained by specified telecommunications service providers. These records cover information such as telephone numbers dialled and the numbers from which calls are made, as well as the date, time and duration of calls, but do not include the content of the calls, the names, address or financial information of any subscriber or customer, or any cell site location information. According to the explanations provided by the US, this means that the intelligence agencies cannot, through this programme, listen to or record telephone conversations.

(…) The US also explained that, although the collection is broad in scope, the further processing of the meta-data acquired under this programme is limited to the purpose of investigation of international terrorism. It was stated that the bulk records may not be accessed or queried by intelligence agencies for any other purpose.”   

2.3 Executive Order 12333

p.12: “The US indicated that Executive Order 12333 serves as the basis for other surveillance programmes, the scope of which is at the discretion of the President. (…)”

p.12: “(…) The EU requested information in particular with regard to the application of Executive Order 12333 to bulk data collection, its impact on individuals in the EU and any applicable safeguards. The US explained that the part that covers signals intelligence annexed to the relevant regulation setting forth procedures under 12333 is classified, as are the supplementary procedures on data analysis, but that the focus of these procedures is on protecting information of US persons. (…)”

p.12: “The US confirmed that judicial approval is not required under Executive Order 12333 and that there is no judicial oversight of its use, except in limited circumstances such as when information is used in a legal proceeding. Executive oversight is exercised under Executive Order 12333 by the Inspector-Generals of each agency (…) The US was unable to provide any quantitative information with regard to the use or impact on EU citizens of Executive Order 12333. (…)”

p.13: “The US further confirmed that in the US there are other legal bases for intelligence collection where the data of non-US persons may be acquired but did not go into details as to the legal authorities and procedures applicable.“

3. COLLECTION AND FURTHER PROCESSING OF DATA

p.13: “(…) the US stated that the collection of personal information based on Section 702 FISA and Section 215 Patriot Act is subject to a number of procedural safeguards and limitative conditions. Under both legal authorities, according to the US, privacy is protected by a multi-layered system of controls on what is collected and on the use of what is collected, and these controls are based on the nature and intrusiveness of the collection.”

p.13: “It appeared from the discussions that there is a significant difference in interpretation between the EU and the US of a fundamental concept relating to the processing of personal data by security agencies. For the EU, data acquisition is synonymous with data collection and is a form of processing of personal data. Data protection rights and obligations are already applicable at that stage. Any subsequent operation carried out on the data collected, such as storage or consultation by human eyes, constitutes further processing. As the US explained, under US law, the initial acquisition of personal data does not always constitute processing of personal data; data is “processed” only when it is analysed by means of human intervention. This means that while certain safeguards arise at that moment of acquisition, additional data protection safeguards arise at the time of processing.“

3.1. Section 702 FISA

3.1.1. Certification and authorization procedure

p.14: “Section 702 does not require individual judicial orders or warrants authorizing collection against each target. Instead, the FISC approves annual certifications submitted in writing by the Attorney General and the Director of National Intelligence. Both the certifications and the FISC’s orders are secret, unless declassified under US law. The certifications, which are renewable, identify categories of foreign intelligence information sought to be acquired. They are therefore critical documents for a correct understanding of the scope and reach of collection pursuant to Section 702.

The EU requested, but did not receive, further information regarding how the certifications or categories of foreign intelligence purposes are defined and is therefore not in a position to assess their scope. The US explained that the specific purpose of acquisition is set out in the certification, but was not in a position to provide members of the Group with examples because the certifications are classified. (…) The FISC does not scrutinise the substance of the attestation or the need to acquire data against the purpose of the acquisition, e.g. whether it is consistent with the purpose or proportionate, and in this regard cannot substitute the determination made by the Attorney General and the Director of National Intelligence. Section 702 expressly specifies that certifications are not required to identify the specific facilities, places, premises, or property to which an acquisition of data will be directed or in which it will be conducted.

On the basis of FISC-approved certifications, data is collected by means of directives addressed to electronic communications services providers to provide any and all assistance necessary. On the question of whether data is “pushed” by the companies or “pulled” by the NSA directly from their infrastructure, the US explained that the technical modalities depend on the provider and the system they have in place; providers are supplied with a written directive, respond to it and are therefore informed of a request for data. (…)”

p.15: “According to the US, under Section 702, once communications from specific targets that are assessed to possess, or that are likely to communicate, foreign intelligence information have been acquired, the communications may be queried. This is achieved by tasking selectors that are used by the targeted individual, such as a telephone number or an email address. The US explained that there are no random searches of data collected under Section 702, but only targeted queries. Query terms include names, email addresses, telephone numbers, or keywords. When query terms are used to search databases, there is no requirement of reasonable suspicion neither of unlawful activity nor of a specific investigation. The applicable criterion is that the query terms should be reasonably believed to be used to return foreign intelligence information. The US confirmed that it is possible to perform full-text searches of communications collected, and access both content information and metadata with respect to communications collected.

(…) There is no judicial scrutiny of the selectors tasked, e.g. their reasonableness or their use. The EU requested further information on the criteria on the basis of which selectors are defined and chosen, as well as examples of selectors, but no further clarifications were provided.“

p.16: “Finally, the FISC review does not include review of potential measures to protect the personal information of non-US persons outside the US.”

3.1.2. Quantitative indicators

p.17: “(…) The US did not discuss the specific number of certification or selectors. Additionally, the US was unable to quantify the number of individuals in the EU affected by the programmes.

The US confirmed that 1.6% of all global internet traffic is “acquired” and 0.025% of it is selected for review; hence 0.0004% of all global internet traffic is looked at by NSA analysts. The vast majority of global internet traffic consists of high-volume streaming and downloads such as television series, films and sports1. Communications data makes up a very small part of global internet traffic. The US did not confirm whether these figures included “upstream” data collection.”

3.1.3. Retention Periods

p.17: “The US side explained that “unreviewed data” collected under Section 702 is generally retained for five years, although data collected via upstream collection is retained for two years.”

p.18: “The EU asked what happens to “non-responsive” information (i.e. data collected that does not respond to query on the basis of a query term). The US responded that it is not “collecting” non-responsive information. According to the US, information that is not reviewed pursuant to a query made to that database normally will “age off of the system”. It remains unclear whether and when such data is deleted.“

3.1.4. Onward transfers and sharing of information

(…)

3.1.5. Effectiveness and added value

(…)

3.1.6. Transparency and remedies ex-post

(…)

3.1.7. Overarching limits on strategic surveillance of data flows

p.19: “The EU asked whether surveillance of communications of people with no identified link to serious crime or matters of state security is limited, for example in terms of quantitative limits on the percentage of communications that can be subject to surveillance. The US stated that no such limits exist under US law.“

3.2. Section 215 US Patriot Act

(…)

3.2.1. Authorization procedure

(…)

3.2.2. Quantitative indicators

p.20: “The US explained that only a very small fraction of the telephony meta-data collected and retained under the Section 215-authorised programme is further reviewed, because the vast majority of the data will never be responsive to a terrorism-related query. It was further explained that in 2012 less than 300 unique identifiers were approved as meeting the “reasonable, articulable suspicion” standard and were queried. According to the US, the same identifier can be queried more than once, can generate multiple responsive records, and can be used to obtain second and third-tier contacts of the identifier (known as “hops”). The actual number of queries can be higher than 300 because multiple queries may be performed using the same identifier. The number of persons affected by searches on the basis of these identifiers, up to third-tier contacts, remains therefore unclear.”

3.2.3. Retention periods

p.21: “The US explained that, in principle, data collected under Section 215 is retained for five years, with the exception for data that are responsive to authorized queries. In regard to data that are responsive to authorized queries, the data may be retained pursuant to the procedures of the agency holding the information, e.g. the NSA or another agency such as the FBI with whom NSA shared the data.”

3.2.4. Onward transfers and sharing of information

p.22: “According to the US, the orders for the production of telephony meta-data, among other requirements, prohibit the sharing of the raw data and permit NSA to share with other agencies only data that are responsive to authorized queries for counterterrorism queries.”

4. OVERSIGHT AND REDRESS MECHANISMS

4.1. Executive oversight

p.23: “Once the data is collected, a number of executive oversight mechanisms and reporting procedures apply. There are internal audits and oversight controls (e.g. the NSA employs more than 300 personnel who support compliance efforts). Each of the 17 agencies that form the intelligence community, including the Office of the Director of National Intelligence has a General Counsel and an Inspector General. The independence of certain Inspectors General is protected by a statute and who can review the operation of the programmes, compel the production of documents, carry out on-site inspections and address Congress when needed. Regular reporting is done by the executive branch and submitted to the FISC and Congress.

As an example, the NSA Inspector-General in a letter of September 2013 to Congress referred to twelve compliance incidents related to surveillance under Executive Order 12333.In this context, the US drew the Group’s attention to the fact that since 1 January 2003 nine individuals have been investigated in relation to the acquisition of data related to non-US persons for personal interests. The US explained that these employees either retired, resigned or were disciplined.”

4.2. Congressional oversight

(…)

4.3. Judicial oversight: FISC role and limitations

(…)

EOF

Today the AIVD added (in English) the below content to its website. Don’t forget that oversight on Dutch SIGINT is still broken. The AIVD states that regarding mass interception of non-cablebound communications (for SIGINT purposes) it “must present a well-founded request to the minister”. From the CTIVD oversight reports it is clear that the AIVD has been failing to do so in the period 2008-2013.

Interception of telecommunications by the AIVD: rules and regulations

The interception of telecommunications by secret services is surrounded by myths and misunderstandings. What is interception according to the Dutch intelligence services and what kinds of interception do we define? What does the law say? By whom and how are the Dutch intelligence services supervised?

Telecommunications have always been a vital source of information in the world of intelligence and security services. And the importance has only increased further, because, just like everyone else, those subject to investigation by the AIVD increasingly use telecommunications and Internet services. In the old days telecommunications mainly consisted of landlines, fax machines and radio links. With the introduction of the Internet and mobile telephones the number of possibilities increased exponentially, including the use of wireless networks, text messaging, chat rooms, etc.

Telecommunications can take place in three ways:

• via a wireless connection (non cable-bound), also called ether communication (e.g. mobile telephony and satellite links);
• via cable (glass fibre and copper connections);
• via a combination of the two.

Content and metadata

Telecommunications consist of the message (content) and all the data added for the purpose of transport (metadata), such as a telephone number, an IP number, an email address or location data.

Intercepting telecommunications first and foremost means collecting metadata. Metadata is less substantial in size and can be analysed more quickly. In addition, gathering metadata is a less serious privacy infringement. Analysis of the metadata shows whether the matching content of traffic may be relevant for AIVD investigations.
Most data is irrelevant for AIVD investigations. If, based on a carefully designed assessment trajectory, the data does prove to be important, the Minister of the Interior and Kingdom Relations must be asked for permission to also look at the content.

What does the law say?

The Intelligence and Security Services Act 2002 (Wiv 2002) gives the AIVD special investigative powers to intercept, store and search telecommunications, and to gain access to the content, but there are restrictions.

Mass interception versus targeted interception of telecommunications

There are various differences between mass interception and targeted interception of telecommunications.

Mass interception of telecommunications (non cable-bound/ether traffic):

• The interception does not require prior permission.
• You do not yet know who or what you are looking for.
• Telecommunications of several users are intercepted, not just of potentially interesting people or organisations.
• You do not yet know all the characteristics, such as the mobile telephone number and/or email address of a person or organisation.
• You expect to find relevant communications in certain communication streams regarding specific topics, people or organisations.

Targeted interception of telecommunications (cable-bound and non cable-bound):

• This requires prior permission.
• You know who or what you are looking for: the communications of person or organisation ‘x’.
• You possess unique characteristics, such as a mobile telephone number and/or email address of person ‘x’.
• You are allowed to look at the content of the telecommunications.

Cable-bound versus non cable-bound

The difference between intercepting cable-bound and non cable-bound communications has been explained above. In practice, however, telecommunications run both via cable and in the ether. A telephone call to Australia does not run via a single ‘line’, for instance, but is usually transmitted via a ‘collection of wired and ether connections’. Whether telecommunications run via cable or not is irrelevant for content, range or type of communication means. The AIVD is permitted to mass intercept non cable-bound communications only.

Permission

The parameters for approved targeted and mass interception of telecommunications have been standardised in detail by law.

Mass interception of telecommunications (non cable-bound)

Mass interception of telecommunications, for example by means of satellites, does not require the approval of the minister, because its content is not being processed and thus -according to the law- does not infringe the secrecy of correspondence, which includes telephone and telegraph.

Within the bulk of data, which is acquired through mass interception, subsequently a selection can be made to isolate the telecommunications traffic pertaining to one specific person or organisation. Such a selection requires approval of the minister because it infringes on personal privacy. After the selection has been made the content of the communications may be disclosed. (Article 27, Wiv 2002).

The AIVD must present a well-founded request to the minister.

The request must state the reason why it is necessary to employ this special power, whether the invasion of privacy merits the intended result (proportionality) and whether the information cannot be acquired in a different, less invasive, manner (subsidiarity).

Targeted interception of telecommunications

The AIVD is also allowed to intercept, receive, store, research and take knowledge of the content of targeted telecommunications. Within specific parameters the AIVD is authorised to tap public telecommunications networks and services including Internet traffic in real time; the classic eavesdropping (Article 25 Wiv 2002).
A request for targeted interception also must show that the AIVD can substantiate the necessity, proportionality and subsidiarity of the means to be deployed.

Supervision

The AIVD is supervised by the independent Committee on the Intelligence and Security Services (CTIVD). The CTIVD oversees whether the AIVD’s (and the MIVD’s) activities are lawful. The CTIVD has also investigated the manner in which the AIVD and MIVD deploy mass interception (Signal Intelligence). The reports pertaining to this subject are accessible on the Committee’s site.
In addition, the Committee on the Intelligence and Security Services (CTIVD) of the Lower House is responsible for the parliamentary supervision of the intelligence and security services. This committee is comprised of the Lower House chairpersons of the parliamentary parties.

EOF

UPDATE 2017-03-22: related reading: “How good is your batting average?” Early IC Efforts To Assess the Accuracy of Estimates by Jim Marchio, in Studies in Intelligence Volume 60, Number 4 (December 2016).

These are my reading notes concerning `Evaluating the Quality of Intelligence Analysis: By What (Mis) Measure?‘ (Stephen Marrin, 2012).

Marrin provides an exposition of the problem of evaluating the quality of intelligence analysis. Marrin first discusses three measures that are employed for retrospective evaluation of the quality of intelligence analysis:

• Accuracy;
• Preventing surprise;
• Influence on policy.

According to Marrin, all measures are problematic, and considering the inevitability of intelligence failures, evaluation of intelligence quality should not rely on absolute measures (e.g. black/white: something is accurate or inaccurate) but be oriented toward relative measures and `improving on the margins’.

Below are snippets from various sections of Marrin’s paper that I keep here for my own purposes. Emphasis is mine.

Accuracy

“One way to evaluate intelligence analysis is according to an accuracy standard. (…) However, while using accuracy as an evaluative criterion is simple in theory, actually comparing the analysis to ground truth and determining whether the analysis was accurate or inaccurate can be very difficult to implement in practice.

First, there is the presence of qualifiers in the analysis. Uncertainty is part of the intelligence production process. (…) When precise information is desired, such as the condition of a country’s WMD program, a CIA analyst cobbles together bits and pieces of information to form a picture or story and frequently discovers many gaps in the data. As a result, an intelligence analyst’s judgment frequently rests on a rickety foundation of assumptions, inferences and educated guesses.

Caveats and qualifiers are necessary in finished intelligence as a way to communicate analytic uncertainty.  Intelligence agencies would be performing a disservice to policymakers if their judgments communicated greater certainty than the analysts possessed.

(…)

Words such as ‘probably’, ‘likely’ and ‘may’ are scattered throughout intelligence publications and prevent easy assessment of accuracy.

(…)

Removing caveats for sake of simplicity in assessing intelligence accuracy also unfairly removes the record of analytic uncertainty and, in the end, assesses something with which the analyst never would have agreed. For example, if an analyst says that a coup is likely to occur in a foreign country within six months, and the coup happened 12 months later, would that analysis be accurate or inaccurate?

(…)

In addition, the analytic judgment could not be considered completely accurate nor completely inaccurate; it is somewhere in between. It is for this reason that the then-Director of Central Intelligence George Tenet said: ‘In the intelligence business, you are almost never completely wrong or completely right’.

(…)

In addition, even if accurate analysis was produced, a ‘self-negating prophecy’ resulting from analysis produced within a decision cycle could occur. This means that intelligence analysis can help change what may happen in the future, making the analysis inaccurate. (…) This causal dynamic exists for all intelligence issues including political, economic, and scientific due to the nature of the intelligence mission.”

Preventing surprise

“Like accuracy, another absolute standard for evaluating analytic quality involves the prevention of decision-maker surprise. By describing, explaining, evaluating, and forecasting the external environment, intelligence analysts facilitate decision-maker understanding to the point that decision-makers are not surprised by the events that take place. When decision-makers are surprised, by definition there must have been an intelligence failure since it failed to achieve its objective; preventing surprise.

The problem with this expectation, of course, is that surprise is ever present in international relations. Many surprises are the intentional result of adversaries who employ secrecy to hide their intentions. Secrecy in policy creation and implementation magnifies the effectiveness of power application internationally because, when done successfully, the intended target has little or no time to effectively counter the respective policy. (…)

(…) According to Christopher Andrew: ‘Good intelligence diminishes surprise, but even the best cannot possibly prevent it altogether. Human behavior is not, and probably never will be, fully predictable’. Richard Betts, in his article on the inevitability of intelligence failure, suggests that policies should be implemented in such a way as to be able to withstand the inevitability of surprise, with ‘tolerance for disaster’. (…)”

Betts and Shifting the Standard from Absolute to Relative

“Richard Betts has made the greatest contributions to shifting the evaluative metric from the unattainable ideal of accuracy to something more realistic with his argument that intelligence failures, consisting of either inaccuracy or surprise, are inevitable. Betts’ argument is a sophisticated one which acknowledges that (1) the analytic task is really difficult; and (2) anything done to ‘fix’ or reform perceived problems will lead to other problems. (…)

Betts is primarily responding to earlier efforts to eliminate failure by identifying causes of inaccuracy or surprise and then trying to eliminate them one by one. Many causes of failure have been identified, including an individual analyst’s cognitive limitations, and as a result ‘analysis is subject to many pitfalls – biases, stereotypes, mirror-imaging, simplistic thinking, confusion between cause and effect, bureaucratic politics, group-think, and a host of other human failings’, according to Ron Garst and Max Gross. Many efforts to identify causes of failure then proceed to produce recommendations for ways to eliminate them.

Betts, on the other hand, does not believe failure can be eliminated. According to Betts, failure results from paradoxes of perception that include the impossibility of perfect warning, and the distortion of analysis due to motivated biases resulting from organizational or operational goals. Another source of analytic inaccuracy, according to Betts, is a byproduct of the inherent ambiguity of information, which is related to the limitations of intelligence analysis due to the inductive fallacy which Klaus Knorr highlighted in 1964.

(…) Betts’ conclusion that failure will be inevitable has become the consensus among intelligence scholars. (…)

(…) Yet at the same time Betts says that failure can become less frequent on the margins and also has recommendations for how policymakers can make the inevitable failures less costly or significant.”

The Batting Average Metaphor

“Metaphors from baseball are frequently employed by scholars to frame the evaluation of intelligence performance precisely because many useful inferences can be derived from them. For example, the difference between the fielding percentage, where most anything less than perfection is an error, and a batting average, which provides more room for error without condemnation, highlights the importance of the standard used to evaluate relative performance. In addition, the use of the batting average metric also makes it clear that it is relative success versus an opposing force in the context of a competition where the fates of the batters will, as Betts says, ‘depend heavily on the quality of the pitching they face’. The fact that relative success or failure is contingent on the skill of the opposition has clear parallels in the world of intelligence.”

Using Decision-makers’ Evaluative Framework

“(…) Intelligence analysis is regularly ignored by decision-makers, and frequently has limited to no impact on the decisions they make.

As a result, a new kind of theory or model more effectively explaining what happens at the intersection of intelligence analysis and decision-making has been developed. It conceptualizes the purpose of intelligence as to ensure that decision-makers’ power is used as effectively and efficiently as possible, with the purpose of intelligence analysis being to integrate and assess information as a delegated, subordinate, and duplicative step in a decision-making process. This conceptualization privileges the role of the decision-maker in the assessment process over that of the analyst, thus turning the standard model in its head.

(…) Unfortunately, this emphasis on the significance of the decision-maker in evaluating the analytic product has not been universally embraced by scholars, practitioners, or the general public. Instead, more simplistic measures such as accuracy or surprise tend to predominate the discussion of intelligence performance as a way of characterizing failures. Yet evaluating intelligence analysis using the decision-makers’ perspective could be important since, as Kuhns suggests, the decision-maker is ‘the only person whose opinion really matters’. If decision-makers find the analysis informative, insightful, relevant or useful, then the intelligence analysis has succeeded whereas if the decision-makers are left unsatisfied then the analysis has failed.

Intelligence analysis can be evaluated based on the decision-maker’s perception of its relevance, influence, utility or impact. First, there is intelligence analysis that is relevant to decision-makers. (…) Second, there is intelligence analysis that is influential in terms of shaping or influencing the decision-maker’s judgment on a particular issue. (…) Third, there is intelligence analysis that is useful – which also has to be relevant by definition – and could indicate analysis that is either useful in the sense of improving judgment (i.e. influential) or useful in the sense of achieving policy outcomes, or both.

Asking for feedback from decision-makers may be a way to evaluate the analysis’ relevance, influence on judgment or utility, but doing so can be fraught with peril. Policymaker satisfaction with intelligence analysis is a notoriously fickle and idiosyncratic metric. Decision-makers may not be satisfied with intelligence analysis if it conflicts with their own biases, assumptions, policy preferences, or conveys information that indicates a policy may be failing. (…)

(…) Unfortunately, as Ford goes on to say, reason is not the only factor that drives policymaking. Instead, ‘all kinds of forces go into their making of policy, not excluding timidity, ambition, hubris, misunderstandings, budgetary ploys, and regard for how this or that policy will play in Peoria’.“

Clarifying Purpose and Improving on the Margins

“In the end, there is no single metric or standard used to evaluate intelligence analysis, and different people use different standards. This highlights an even more significant issue: the reason that different standards are used is because there is no consensus in either the practitioner’s or scholar’s camp regarding the purpose of intelligence analysis.Some believe that the purpose of intelligence analysis is to be accurate; others believe the purpose is to prevent surprise; while yet others believe the purpose is to be influential or useful. If the intelligence analysis does not meet any of these criteria, then failure is the descriptor that is frequently used.

But the fact that different kinds of failures really represent different normative visions of what intelligence analysis is supposed to accomplish is not acknowledged by most participants. Despite the fact that intelligence analysis has existed as a function of government for decades, both practitioners and scholars have failed to develop a consensus on or even acknowledge differences of opinion regarding exactly what it is supposed to do.

If the failure is determined to be inaccuracy, is the implicit expectation perfection? If the failure is one of surprise, is the implicit expectation omniscience? If the failure is one of lack of influence, to what degree is that more of a policy failure than an intelligence failure? These are questions that both scholars and practitioners should make explicit when they discuss the quality of intelligence analysis and the causes of intelligence failure.

Perhaps the goal of policy should be trying to improve intelligence analysis across the board by improving accuracy, preventing surprise and increasing the value of the product for the decision-maker. But even this will not eliminate failures altogether. As Betts has said, echoing Knorr before him, we should focus on improving performance on the margins – raising the batting average by 50 points, or raising the level of liquid in the glass – not achieving perfection or omniscience.

(…)

Finally, understanding and improving intelligence analysis may also require clarifying what we believe the purpose of intelligence analysis is, or what the purposes of intelligence analysis are, and how to best achieve them. Rather than focusing on and studying failure, perhaps trying to achieve success will do more to improve the quality of intelligence analysis than trying to eliminate failure.”

EOF

For my own purposes I hereby post some reading notes from What’s the Harm? The Ethics of Intelligence Collection ($, 2012) by Ross Bellaby.

Bellaby states:

“If liberal democracies are to be seen abiding by the rules, norms and ideals to which they subscribe, then so must their intelligence communities.
(…)
At the centre of the topic of ‘intelligence ethics’, often ridiculed as ‘oxymoronic’, is the tension between the belief that ‘there are aspects of the intelligence business, as practised by all major countries, that seem notably disreputable’, and the argument that ‘without secret intelligence we will not understand sufficiently the nature of some important threats that face us’; that political communities have an ethical obligation to act so as to protect its people.”

Bellaby proposes an ethical framework for intelligence collection, building from primum non nocere — the Hippocratic injunction of “first, do no harm”. Next, Bellaby states (emphasis is mine):

“The first step in establishing an ethic against harm begins with the realization that individuals have certain requirements that are both ‘vital’ to their well- being and vulnerable to external interference. These ‘vital interests’ are the prerequisites or preconditions that must be maintained if individuals are to fulfil their ultimate goals and aspirations. Joel Feinberg calls these requirements ‘welfare interests’. John Rawls calls them ‘primary goods’. Essentially they amount to the same thing – that is, regardless of what one’s conception of the good life might be, these preconditions must be first satisfied in order to achieve them. These interests include individuals’ physical and mental integrity, their autonomy, liberty, sense of self-worth and privacy. Without these vital interests any individual is unable to pursue other ultimate interests, purposes, goals or plans. Of such fundamental importance are these interests to the individual that they have intrinsic value. Damaging them can therefore cause harm regardless of the repercussions. That is, even if on balance the individual does not experience the harm in a ‘tangible and material’ way, s/he can still be said to be harmed if the vital interests are violated or wronged. In this way, these interests are a person’s most important ones and thus demand protection. (…)”

Bellaby then identifies the following `vital interests’ of humans:

• Physical integrity: “for example, (…) what sort of physical treatment potentially dangerous suspects can expect.”
• Mental integrity : concerns “actions that can cause debilitating levels of stress or anxiety to the individual.”
• Autonomy: “the capacity for self-rule”; “one must be able to decide for oneself, without external manipulation or interference, what shape one’s own life will take.”
• Liberty: “closely connected to the concept of autonomy”. “Whereas autonomy is the freedom to decide one’s will, liberty is the freedom from constraints on acting out that will.”
• Human Dignity as Amour-propre: A Sense of One’s Own Self-Worth: how individuals view themselves and how others view them. “Confidence in one’s self-worth is so fundamental that without it one can become unable to continue or realize endeavours that are needed to fulfil one’s aspirations. Without self-respect individuals feel worthless; nothing ‘seems worth doing’, activities become ‘empty and vain’ and people ‘sink into apathy and cynicism’.”
• Privacy: in context of intelligence collection, two concepts are “of particular relevance”: privacy as boundaries and privacy as control. “Boundaries mark out areas where outside intrusion is unwelcome. (…) In comparison, privacy as control is the right of individuals to control those things pertaining to themselves, that is, ‘the control we have over information about ourselves’ or the ‘control over one’s personal affairs’.”

Regarding harm to privacy, Bellaby states the following:

“However, by intercepting another’s communications, their privacy is violated. This is because, first, the activity involves intercepting and utilizing without consent information that is essentially the individual’s property, and second, by violating a sphere with a strong expectation that the individual is ‘in’ private, represented by the clear distinction between the ‘inside’ of the communication where the message exists and the ‘outside’ where the rest of society exists.”

Regarding surveillance (dataveillance, CCTV, data mining, covert surveillance), he states:

“By collecting individual personal information in this way, an individual’s privacy and autonomy is violated. The individual’s privacy is violated in that the information collected is the individual’s personal property; it pertains to their actions and personal details. By collecting and collating this information, the individual’s right to control and keep it private is violated.”

In order to limit the harm done to vital human interests by intelligence collection, and outline exactly when harm is justified, Bellaby proposes the following `just intelligence principles’ as an ethical framework for intelligence collection:

• Just cause: there must be a sufficient threat to justify the harm that might be caused by the intelligence collection activity.
  

  “Thomas Aquinas argued that for a war to be just there must be some reason or injury to give cause, namely that ‘those who are attacked must be attacked because they deserve it on account of some fault’. Currently, international law frames ‘self-defence’ as the main justification for going to war.”

• Authority: there must be legitimate authority, representing the political community’s interests, sanctioning the activity.
  

  “For a war to be considered morally permissible according to the just war tradition it must be authorized by the right authority, that is, those who have the right to command by virtue of their position. As Aquinas stated, ‘the ruler for whom the war is to be fought must have the authority to do so’ and ‘a private person does not have the right to make war’. (…) Similarly, one can argue that in order for intelligence collection to be just, there must be a legitimate authority present to sanction the harms that can be caused.”

• Intention: the means should be used for the intended purpose and not for other (political, economic, social) objectives.
  

  “Leaders must be able to justify their decisions, noting that they had the right intentions; ‘for those that slip the dogs of war, it is not sufficient that things turn out for the best’.”

  “Another implication of this principle is reflected in the current debate on personal information databases and how crossover information collection should be restricted. If information is collected – DNA, fingerprints, personal data for example – under a just cause with the appropriate degree of evidence, but was incidentally connected to another crime, then the information can be used since the original just cause and correct intention was present. This would be analogous to finding illegal goods incidentally while performing a legal search. However, what is not permissible is to use a just cause such as tax fraud to justify the collection and retention of DNA, as this type of information is unrelated and is not reflecting the original just cause, clearly outside what should be the correct intention.”

• Proportion: the harm that is perceived to be caused should be outweighed by the perceived gains.
  

  “One can argue that, for the intelligence collection to be just, the level of harm that one perceives to be caused, or prevented, by the collection should be outweighed by the perceived gains.”

• Last resort [=subsidiarity]: less harmful acts should be attempted before more harmful ones are chosen.
  

  “In order for an intelligence collection means to be just, it must only be used once other less or none harmful means have been exhausted or are redundant.”

• Discrimination: There should be discrimination between legitimate and illegitimate targets.
  

  “The principle of discrimination for the just intelligence principles therefore distinguishes between those individuals without involvement in a threat (and thereby protected), and those who have made themselves a part of the threat (and by so doing have become legitimate targets). According to the degree to which an individual has assimilated himself, either through making himself a threat or acting in a manner that forfeits his rights, the level of harm which can be used against him will alter.”

The first five principles are borrowed from jus ad bellum, Just War Theory. That theory includes Likelihood of Success as a sixth criterium; for Just Intelligence, Bellaby replaced it with Discrimination.

For full context, always read the original document. (Which I’m sorry to say is paywalled in this case.)

Related:

• 2014-09-22: Collection of online articles on intelligence ethics (Intelligence Ethics blog / @vallandingham_c)
• 2013-11-09: Oversight on Dutch SIGINT is still broken (this blog)

EOF