olivlaw

From books

Bayes does not ask you to guess the truth, only to make one judgement per item of evidence: how many times more likely it is if the hypothesis is true than if it is false.

Jack Zlotnick (CIA, Center for the Study of Intelligence) · Bayes' Theorem for Intelligence Analysis · 1972 · Jack Zlotnick, «Bayes' Theorem for Intelligence Analysis», Studies in Intelligence vol. 16, nr. 2 (1972), paragrafele introductive2 minutes readpublic domain
“The very best that intelligence can do is to make the most of the evidence without making more of the evidence than it deserves.”Jack Zlotnick (CIA, Center for the Study of Intelligence) · Bayes' Theorem for Intelligence Analysis · 1972 · Jack Zlotnick, «Bayes' Theorem for Intelligence Analysis», Studies in Intelligence vol. 16, nr. 2 (1972), paragrafele introductive

An item of evidence matters only to the extent that it is more likely under one hypothesis than under the other.

The CIA sponsored in-house research on applying Bayes' theorem to intelligence analysis, and Jack Zlotnick, an analyst who took part, described the method in Studies in Intelligence. It uses the odds form: the revised odds (R) equal the prior odds (P) multiplied by the likelihood ratio (L). Analysts never judged the conclusion directly; they judged only L for each new item — how many times more likely, say, a troop deployment to a border is if war is coming than if it is not. If twice as likely, L = 2 and the odds double. R then becomes the P for the next item. The steps: frame two mutually exclusive hypotheses; fix the starting odds explicitly; for each item of evidence, estimate separately how likely it is under each hypothesis; multiply and move on. It pays off when evidence arrives in a stream and the temptation is to react to the latest item. Zlotnick names a limit: close to the climax, much incoming evidence becomes undiagnostic — equally likely under both hypotheses. A second classic pitfall is treating two reports from the same source as independent; multiplied as if they were, they inflate the conclusion. The typical error remains mistaking evidence that is consistent with a hypothesis for evidence that is diagnostic.

The card shows how the likelihood ratio is computed. The next step is subtler: who sets the starting odds matters enormously, because every later multiplication builds on that starting point. If two analysts begin from different beliefs, the same evidence moves them in different directions, but not necessarily to the same place. A familiar example: two people read the same ambiguous restaurant review. One was already convinced it was worth trying, the other skeptical. Both update correctly, yet they end up with different conclusions. That is why the method demands that initial odds be stated openly, not hidden in one's head. Then you can see where each person started and how far the evidence moved them. A question remains: what do you do when you have no basis at all for the starting point?

When you have no basis for a starting point, the method gives a simple answer: even odds. It does not mean you know something; it means you officially declare that you don't. Then the evidence does all the work. A useful distinction appears here: a piece of evidence can be compatible with your hypothesis and still move you not at all. If its likelihood ratio is one, multiplying leaves the odds untouched. Example: you hear that a neighbor bought bread. That says almost nothing about whether they cook often or order takeout, because both hypotheses make bread equally likely. The disciplined analyst does not ignore the evidence; they measure it and accept the answer, even if it is zero movement. But what do you do when two pieces of evidence seem different and yet say the same thing?

Why it matters Without an explicit updating rule, the latest item of evidence gets the most weight merely because it is the latest, not because it says more.

Priorodds (P)Likelihoodratio ofRevisedodds R =
The human judges only L; the rest is arithmetic repeated for each item.

← Back to the feed