Book
A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis
by US Government · 2009 · 4 reading cards · public domain
4 cards
A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis · 2009
You do not debate whether it will happen; you write how it would happen and what you would see first.
These are two related contrarian techniques for the moment when everyone is sure something will not happen. High-Impact/Low-Probability (HILP) analysis asks: if it does happen, what are the consequences, and by what path would we get there? What If? analysis accepts from the outset that the event has occurred and moves straight to how it came about. The primer recalls events once considered unlikely — the fall of the Shah, the collapse of the Soviet Union, the reunification of Germany — and a State Department assessment of 27 November 1941 that gave odds of five to one against the United States and Japan being at war by 15 December. The steps: state the dominant analytic line and the high-impact outcome clearly; assume it has happened; choose plausible triggers, such as the death of a leader, a natural disaster or an economic shock; think backwards, step by step, about what must have occurred; identify one or more pathways; write observable indicators for each and monitor them periodically. It matters most when a judgement rests on limited information or unproven assumptions. The trap: the exercise does not change the probability; it produces signposts to watch and hedging options. Taken out of context, it turns into alarmism.
“By shifting the focus from whether an event could occur to how it may happen, analysts allow themselves to suspend judgment about the likelihood of the event and focus more on what developments—even unlikely ones—might enable such an outcome.”
A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis · 2009
A single consensus calls for a devil's advocate; two strong camps call for Team A and Team B.
Both techniques are contrarian: they do not build an analysis from scratch but attack an existing one. The difference between them is diagnostic, and the primer warns that analysts often confuse them. Devil's advocacy is used when there is ONE dominant consensus: an analyst with doubts, or one designated by a manager, builds the best possible case for an alternative explanation. The steps: summarise the mainline judgement and key assumptions; pick the assumptions most open to challenge; check whether any information is questionable, whether deception is possible, or whether major gaps exist; present the findings and, if needed, draft a contrarian paper clearly labelled as an exercise. Team A/Team B is used when TWO equally strong positions coexist: each side writes out its full argument, with assumptions and data in plain view, and the decision-maker judges which is stronger. It is costly, so it is kept for long-running disputes or far-reaching decisions; analysts are sometimes made to argue the position they do not hold, to expose their own mind-set. The traps: a challenger with no real mandate who merely performs, and an unlabelled contrarian product the reader mistakes for the official view. Reaffirming the original line, with more confidence, is also a valid outcome.
“Its primary value is to serve as a check on a dominant mind-set that can develop over time among even the best analysts who have followed an issue and formed strong consensus that there is only one way of looking at their issue.”
A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis · 2009
The most dangerous premise is the one everybody shares and nobody has written down.
A key assumption is any hypothesis the analyst has accepted as true and on which the assessment rests — often without ever having stated it. The primer proposes four steps: write down the current analytic line where everyone can see it; list all the premises, stated or unstated, that must be true for the line to hold; challenge each one — why it must be true and whether it holds under all conditions; keep only the assumptions that really carry the conclusion, and note the conditions under which they would fail. The primer's example is the 2002 sniper case around Washington, where it was assumed the perpetrator was acting alone. At the CIA in the 1990s, Douglas MacEachin promoted a variant called linchpin analysis: assumptions about the key drivers become linchpins, the premises the argument stands on, and must be spelled out. The method pays at the start of a project, when it costs an hour or two, and again before judgements are finalised. The trap: the list turns into an inventory of platitudes, while the dangerous premise — the one nobody sees because everybody shares it — stays unwritten. The question that brings it out: if this proved wrong, would the conclusion change?
“The goal is not to undermine or abandon key assumptions; rather, it is to make them explicit and identify what information or developments would demand rethinking them.”
A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis · 2009
Five echoes of the same source are not five confirmations but one.
The quality of information check assesses how complete and sound the source base behind a judgement is. The 2009 primer describes it as a continuous process, not a one-off step: otherwise important judgements stay anchored to weak information, and caveats once attached to them are forgotten. The steps: keep an inventory of sources by type and date, with their strengths and weaknesses; identify the most critical or compelling sources; check that critical reporting has sufficient and strong corroboration; re-examine previously dismissed information in the light of new facts; flag recalled reporting and review any analysis built on it. The result is an honest picture of what we know and what we do not, and it feeds straight into the confidence level. The method pays before any major assessment and whenever an analytic line has run for months. It is also where an adversary's deception strategies can be caught. The main trap is counting: five articles repeating the same source are not five confirmations but one, copied. In the finished product, ICD 203 requires the quality of sources to be described, and source summary statements are strongly encouraged.
“Having multiple sources on an issue is not a substitute for having good information that has been thoroughly examined.”