Why does Heuer ask analysts to think about their own reasoning, not just their conclusions?
You cannot see the lens you look through until you take it off and put it on the table.
Richards J. Heuer Jr. · Psychology of Intelligence Analysis
What are the five analytic standards in ICD 203?
An assessment can be accurate in every sentence and still false if it was tailored to a policy.
Office of the Director of National Intelligence (ODNI) · Intelligence Community Directive 203: Analytic Standards
What did Kent discover after NIE 29-51 on Yugoslavia?
A probability word with no number behind it is a projective test, not an estimate.
Sherman Kent · Words of Estimative Probability
What does analytical confidence measure, as opposed to probability?
Missing evidence lowers confidence, not probability.
Professional Head of Intelligence Assessment (PHIA), Guvernul Regatului Unit · Explaining Uncertainty in UK Intelligence Assessment
What does the letter grade and what does the figure grade in the A–F / 1–6 grid?
A source's reputation does not confirm the information, and a lack of track record does not refute it.
Headquarters, Department of the Army (SUA) · FM 2-22.3 Human Intelligence Collector Operations
What are the steps of a quality of information check?
Five echoes of the same source are not five confirmations but one.
US Government · A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis
What are the four steps of the key assumptions check?
The most dangerous premise is the one everybody shares and nobody has written down.
US Government · A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis
Why does Betts say intelligence failures are inevitable?
A correct estimate that is heard and dismissed is still an intelligence failure.
Richard K. Betts · Analysis, War, and Decision: Why Intelligence Failures Are Inevitable
Why do you remove from the ACH matrix an item of evidence consistent with every hypothesis?
The winner is the hypothesis with the least evidence against it, not the one with the most evidence for it.
Richards J. Heuer Jr. (CIA, Center for the Study of Intelligence) · Psychology of Intelligence Analysis
According to Wohlstetter, what was the main reason the Pearl Harbor attack was not anticipated?
The signal is lost not through too little information but through too much irrelevant information.
Roberta Wohlstetter · Pearl Harbor: Warning and Decision
What is the difference between an indicator and an indication in Grabo's vocabulary?
The indicator is what you expect to see; the indication is what you actually see.
Cynthia M. Grabo (Joint Military Intelligence College) · Anticipating Surprise: Analysis for Strategic Warning
When do you choose devil's advocacy and when Team A/Team B?
A single consensus calls for a devil's advocate; two strong camps call for Team A and Team B.
US Government · A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis
What is the difference between a red team and a red cell in the British handbook?
The red team challenges your plan; the red cell plays the adversary.
UK Ministry of Defence — Development, Concepts and Doctrine Centre (DCDC) · Red Teaming Handbook (3rd Edition)
What does the premortem change in the question put to the team, compared with an ordinary critique session?
You do not ask what might go wrong but why it has already failed.
Gary Klein (Harvard Business Review) · Performing a Project Premortem
How does What If? analysis differ from High-Impact/Low-Probability analysis?
You do not debate whether it will happen; you write how it would happen and what you would see first.
US Government · A Tradecraft Primer: Structured Analytic Techniques for Improving Intelligence Analysis
What does Wack separate before building scenarios, and why?
A good scenario does not guess the future; it changes the map in the decision-maker's head.
Pierre Wack (Harvard Business Review) · Scenarios: Uncharted Waters Ahead
How is the Brier score computed for a 75% forecast on an event that happened, and what is the best value?
A forecast with no deadline and no resolution criterion cannot be lost, so it cannot teach you anything.
Barbara Mellers, Philip E. Tetlock et al. · Identifying and Cultivating Superforecasters as a Method of Improving Probabilistic Predictions
What separates the inside view from the outside view, according to Kahneman and Lovallo?
The base rate is the starting point, and the details of the case are only allowed to adjust it.
Daniel Kahneman & Dan Lovallo · Timid Choices and Bold Forecasts: A Cognitive Perspective on Risk Taking
What does the analyst judge in the Bayesian method tested at the CIA, and what does the arithmetic do?
An item of evidence matters only to the extent that it is more likely under one hypothesis than under the other.
Jack Zlotnick (CIA, Center for the Study of Intelligence) · Bayes' Theorem for Intelligence Analysis
Which condition most favours groupthink, according to Janis, and what three procedural remedies does he propose?
Disagreement survives in a close group only if a procedure turns it from betrayal into duty.
Irving L. Janis · Victims of Groupthink: A Psychological Study of Foreign-Policy Decisions and Fiascoes