Words that stand for numbers
Intelligence rarely knows anything for certain. An analyst receives fragments: a report from a source, an intercepted conversation, a satellite image. From these the analyst builds an assessment for a customer who has to make a decision. British doctrine JDP 2-00 puts it plainly: decision-makers should accept that intelligence must be bounded with degrees of probability and confidence, and that predictive judgements can only be expressed as probabilities.5
The trouble is that words such as “possible” or “probable” mean different things to different people. That is why the services of several countries have tied them to numbers. An assessment then stops being a string of adjectives and becomes a message the customer can compare with others.1,3
Sherman Kent’s lesson
The problem was described by Sherman Kent, one of the founders of American intelligence analysis, in “Words of Estimative Probability”, an article in the CIA journal Studies in Intelligence. His starting point was National Intelligence Estimate 29-51 of March 1951 on the prospect of an invasion of Yugoslavia. Its key judgement was that an attack in 1951 should be considered a “serious possibility”.1
When a State Department official asked what odds lay behind those words, Kent replied that he personally had in mind roughly 65 to 35 in favour of an attack. He then asked the colleagues who had approved the text. Their readings ranged from about 20 to 80 against an attack to 80 to 20 in favour. The same phrase meant quite different judgements to its own authors.1
Kent therefore proposed a vocabulary in which each expression has an approximate value: “almost certain” about 93%, “probable” about 75%, “chances about even” about 50%, “probably not” about 30%. He jokingly called the advocates of loose, descriptive language the “poets” and the advocates of a strict vocabulary of odds the “mathematicians”, and his own sympathies lay with the latter.1
United States: Directive ICD 203
Today the rules for writing assessments in the US intelligence community are set by Intelligence Community Directive 203 on analytic standards, issued by the Director of National Intelligence (ODNI) and signed by James R. Clapper on 2 January 2015. It requires analysis to indicate and explain the uncertainties of major judgements, to describe the quality and credibility of sources, and to distinguish clearly between underlying information and analysts’ assumptions and judgements.2
ICD 203 includes a likelihood scale with percentage ranges: from “almost no chance” (1–5%) through “roughly even chance” (45–55%) to “almost certain(ly)” (95–99%). The scale includes neither zero nor one hundred per cent.2
The directive also separates two measures that are easily confused. Likelihood says how probable something is. Analytic confidence says how sound the evidence and reasoning behind the assessment are. ICD 203 stipulates that the two are not combined in the same sentence.2
United Kingdom: the PHIA Probability Yardstick
In the UK a similar role is played by the Probability Yardstick, described in a publication of the Intelligence Analysis profession dated 24 March 2025. It splits the probability scale into seven ranges: “remote chance” (above 0% to roughly 5%), “highly unlikely” (roughly 10–20%), “unlikely” (roughly 25–35%), “realistic possibility” (roughly 40% to under 50%), “likely or probable” (roughly 55–75%), “highly likely” (roughly 80–90%) and “almost certain” (roughly 95% to under 100%).3
Gaps are left between the ranges, for example between roughly 5% and roughly 10%. According to the document, the choice of terms and ranges was informed by academic research. The yardstick is meant to prevent sentences such as “if X were to occur then Y might happen”, from which the reader cannot extract any definite judgement.3
The British rate analytical confidence on three levels: high, moderate and low. They take three categories into account: the information base, analytical rigour, and complexity and volatility. The lower the confidence, the greater the uncertainty of the assessment itself.3
NATO: grading the source and the information
NATO doctrine AJP-2 (Edition A, 2016) gives no percentage scale of likelihood. Instead it has an older and very practical system: the reliability of the source and the credibility of the information are graded separately. The source receives a letter from A (“completely reliable”) to F (“reliability cannot be judged”), the information a number from 1 (“confirmed by other sources”) to 6 (“truth cannot be judged”).4
The two grades are combined into a pair, such as B2: a usually reliable source, information that is probably true. AJP-2 stresses that reliability and credibility should be considered independently of each other. A trusted source can therefore pass on information graded low, and an unknown source information confirmed through other channels.4
The doctrine adds that even when fully exploited, intelligence will not produce complete certainty, and that staffs must state clearly where there are gaps in knowledge. British JDP 2-00 makes the same point.4,5
Example: the ODNI assessment of January 2017
How this works in practice can be seen in the declassified US intelligence community assessment ICA 2017-01D of 6 January 2017 on Russian activities concerning the US presidential election. It was drafted and coordinated by three agencies: the CIA, the FBI and the NSA. Here we are interested only in the language of the assessment, not in its content.6
The document explains that intelligence community judgements include judgements of how likely it is that something has happened or will happen, as well as confidence levels – low, moderate and high – that refer to the evidentiary basis, logic and reasoning, and precedents that underpin the judgements.6
For one of the key judgements the document notes that the CIA and FBI have high confidence and the NSA moderate confidence. The agencies thus agreed on the judgement itself but rated the strength of its basis differently, and the document says so openly. The Senate Select Committee on Intelligence recorded the same difference in its findings of 3 July 2018.6,7
The public version also states that it does not include the full supporting information, including specific intelligence. Readers get the conclusions and their level of confidence, but not the evidence. This is typical of published assessments: openness ends where the protection of sources and methods begins.6
A British example: “almost certainly”
The same language appears in government statements. In 2018 the UK government announced that the Russian military was “almost certainly” behind the NotPetya attack. In 2020 the UK National Cyber Security Centre (NCSC) assessed with high confidence that the attacks on the Olympic Games were “almost certainly (95%+)” the work of the GTsST unit of Russian military intelligence. One sentence shows both measures: likelihood and confidence.8,9
How such assessments turn into political attribution of an attack is covered in our article Attribution: how states name the perpetrators of cyber attacks.2,8
How to read an assessment: five questions
First: which likelihood term is used, and on which scale – the American or the British one? Second: is confidence stated, and if so, at what level? Third: do all the agencies hold the same confidence? Fourth: what does the document say about sources and gaps in knowledge? Fifth: is this the full assessment, or a public version without the supporting evidence?2,3,4,6
“Likely” in an intelligence assessment does not mean “certain”, and “almost certainly” does not mean “proven”. Such sentences are honest information about the limits of knowledge. An intelligence assessment is one stage of the intelligence cycle (see the intelligence cycle map): the customer receives a finished product, not raw data (see raw data and finished intelligence).2,5
Sources
- Sherman Kent, CIA – Center for the Study of Intelligence, Words of Estimative Probability, Studies in Intelligence vol. 8, no. 4 (CIA reprint)
- Office of the Director of National Intelligence (USA), Intelligence Community Directive 203: Analytic Standards, 2 January 2015 (Federation of American Scientists copy)
- UK Government (Intelligence Analysis profession), Explaining uncertainty in UK intelligence assessment, 24 March 2025
- NATO, AJP-2 Allied Joint Doctrine for Intelligence, Counter-Intelligence and Security, Edition A, Version 2, February 2016 (chapters 3 and 4)
- UK Ministry of Defence, JDP 2-00 Intelligence, Counter-intelligence and Security Support to Joint Operations, Edition 4, August 2023
- Office of the Director of National Intelligence (USA); copy published by the Senate Select Committee on Intelligence, Assessing Russian Activities and Intentions in Recent US Elections, ICA 2017-01D, 6 January 2017 (declassified version)
- US Senate Select Committee on Intelligence (SSCI), Committee Findings on the 2017 Intelligence Community Assessment (initial findings), 3 July 2018
- UK Government (Foreign Office), Foreign Office Minister condemns Russia for NotPetya attacks, 15.02.2018
- UK Government (FCDO), UK exposes series of Russian cyber attacks against Olympic and Paralympic Games (press release), 19.10.2020
Checked: 15 November 2026