Bayes' Theorem for Intelligence Analysis

1 card

1. 01

   Bayes does not ask you to guess the truth, only to make one judgement per item of evidence: how many times more likely it is if the hypothesis is true than if it is false.

   Bayes' Theorem for Intelligence Analysis · 1972

   An item of evidence matters only to the extent that it is more likely under one hypothesis than under the other.

   The CIA sponsored in-house research on applying Bayes' theorem to intelligence analysis, and Jack Zlotnick, an analyst who took part, described the method in Studies in Intelligence. It uses the odds form: the revised odds (R) equal the prior odds (P) multiplied by the likelihood ratio (L). Analysts never judged the conclusion directly; they judged only L for each new item — how many times more likely, say, a troop deployment to a border is if war is coming than if it is not. If twice as likely, L = 2 and the odds double. R then becomes the P for the next item. The steps: frame two mutually exclusive hypotheses; fix the starting odds explicitly; for each item of evidence, estimate separately how likely it is under each hypothesis; multiply and move on. It pays off when evidence arrives in a stream and the temptation is to react to the latest item. Zlotnick names a limit: close to the climax, much incoming evidence becomes undiagnostic — equally likely under both hypotheses. A second classic pitfall is treating two reports from the same source as independent; multiplied as if they were, they inflate the conclusion. The typical error remains mistaking evidence that is consistent with a hypothesis for evidence that is diagnostic.

   “The very best that intelligence can do is to make the most of the evidence without making more of the evidence than it deserves.”

   Open the card →