Richards J. Heuer Jr. (CIA, Center for the Study of Intelligence)

1 card

1. 01

   Analysis of competing hypotheses makes every reasonable explanation compete on the same evidence and keeps the one the evidence refutes least.

   Psychology of Intelligence Analysis · 1999

   The winner is the hypothesis with the least evidence against it, not the one with the most evidence for it.

   The method starts from an awkward observation about how we work: we pick the likely answer by intuition and then look for evidence that supports it. The problem is not that the evidence is false but that most of it fits other explanations just as well. Analysis of competing hypotheses (ACH) has eight steps in Heuer's version, and their core fits into three moves: write down every reasonable hypothesis, ideally with colleagues who think differently; build a matrix with hypotheses across the top and evidence down the side; mark, for each item, which hypotheses it is consistent with and which it is not. The key word is diagnosticity. A fever tells a doctor that the patient is ill, not which illness it is; evidence consistent with every hypothesis does not help you choose and drops out of the calculation. What remains are the few items that discriminate, and those are the ones you re-check first. Then you try to disprove rather than prove, report every hypothesis, and state in advance what would change your mind. The typical trap is confusing an unproven hypothesis with a disproved one. No indication that India will test soon does not mean India will not test — Heuer's example from 1998.

   “The most probable hypothesis is usually the one with the least evidence against it, not the one with the most evidence for it.”

   Open the card →