Book
Words of Estimative Probability
by Sherman Kent · 1964 · 2 reading cards · public domain
2 cards
Words of Estimative Probability · 1964
A probability word with no number behind it is a projective test, not an estimate.
In March 1951 National Intelligence Estimate 29-51 called an attack on Yugoslavia that year a serious possibility. Kent, a member of the Board of National Estimates that had approved the text, had odds of about 65 to 35 in favour of an attack in mind; a State Department reader had understood a much lower chance, and Kent's colleagues each held a different figure, from 20 to 80 up to 80 to 20. His answer was a table: a few expressions, each tied to a range — probable around 75 per cent, almost certain around 93 per cent. Today ICD 203 requires a seven-step scale, from almost no chance (1–5 per cent) to almost certain (95–99 per cent). The British PHIA yardstick also has seven steps, from remote chance (up to about 5 per cent) to almost certain (from about 95 per cent), but deliberately leaves gaps between them. In practice: pick the number first, then the word from the scale; do not mix sets of terms; if you cannot defend the word, you do not yet have a judgement. The trap is what Kent called the poets' camp: phrases such as may well or distinctly possible, which suggest odds without committing to them.
“It was another jolt to find that each Board member had had somewhat different odds in mind and the low man was thinking of about 20 to 80, the high of 80 to 20.”
Words of Estimative Probability · 1964
The number is the judgement; the word is only its translation for the reader.
In March 1951 a US intelligence estimate said an attack on Yugoslavia ‘should be considered a serious possibility’. Sherman Kent, a member of the board that had approved it, had odds of 65 to 35 in favour of an attack in mind; readers at the State Department had understood something much lower. When he asked his colleagues on the board, the lowest had been thinking of about 20 to 80, the highest of 80 to 20. The same words, approved unanimously, hid a disagreement of 60 percentage points. Kent's remedy, published in 1964, was a table: ‘almost certain’ around 93%, ‘probable’ around 75%, ‘chances about even’ around 50%, ‘probably not’ around 30%, ‘almost certainly not’ around 7%. The US intelligence community's directive ICD-203, signed in 2015, does the same with seven terms and ranges: from ‘almost no chance’ (1–5%) to ‘almost certain’ (95–99%), with ‘likely’ between 55% and 80%. The practical rule has two halves. The analyst thinks in and writes the number; the word is only a translation for the reader, generated from the number. The reverse route — choosing the word first and turning it into a number later — loses information, because a word can cover a band up to 25 points wide. In the olivLaw engine the label is computed in Python by the function hedge(p); the AI model writing the analysis never chooses it.
“It was another jolt to find that each Board member had had somewhat different odds in mind and the low man was thinking of about 20 to 80, the high of 80 to 20. The rest ranged in between.”