“The most probable hypothesis is usually the one with the least evidence against it, not the one with the most evidence for it.”Richards J. Heuer Jr. (CIA, Center for the Study of Intelligence) · Psychology of Intelligence Analysis · 1999 · Richards J. Heuer Jr., «Psychology of Intelligence Analysis» (CIA Center for the Study of Intelligence, 1999), capitolul 8 «Analysis of Competing Hypotheses», secțiunea «Summary and Conclusion», p. 108
The winner is the hypothesis with the least evidence against it, not the one with the most evidence for it.
The method starts from an awkward observation about how we work: we pick the likely answer by intuition and then look for evidence that supports it. The problem is not that the evidence is false but that most of it fits other explanations just as well. Analysis of competing hypotheses (ACH) has eight steps in Heuer's version, and their core fits into three moves: write down every reasonable hypothesis, ideally with colleagues who think differently; build a matrix with hypotheses across the top and evidence down the side; mark, for each item, which hypotheses it is consistent with and which it is not. The key word is diagnosticity. A fever tells a doctor that the patient is ill, not which illness it is; evidence consistent with every hypothesis does not help you choose and drops out of the calculation. What remains are the few items that discriminate, and those are the ones you re-check first. Then you try to disprove rather than prove, report every hypothesis, and state in advance what would change your mind. The typical trap is confusing an unproven hypothesis with a disproved one. No indication that India will test soon does not mean India will not test — Heuer's example from 1998.
Why it matters A report that counts the evidence for the favoured hypothesis looks solid right up to the moment the same evidence turns out to fit the alternative nobody tested.