Author
Philip E. Tetlock & Dan Gardner
1 reading card from 1 book · 2015.
1 card
Superforecasting: The Art and Science of Prediction · 2015
Almost every commandment is a balance between two opposing errors, and balance is learnt only with feedback.
The appendix of Superforecasting (Tetlock and Gardner, 2015) distils the Good Judgment Project's research into ten commandments, also reproduced on Good Judgment Inc.'s page: triage (pick the questions where effort pays off); break seemingly intractable problems into tractable sub-problems; balance the inside and outside views; balance under- and overreacting to evidence; look for the clashing causal forces; distinguish as many degrees of doubt as the problem permits but no more; balance under- and overconfidence; look for the errors behind your mistakes while avoiding hindsight bias; bring out the best in others; and master the error-balancing bicycle. The eleventh: don't treat commandments as commandments. Notice the recurring word: balance. Three commandments open with ‘strike the right balance’ between two opposing errors, and the tenth says that each one requires balancing opposing errors. That is why it cannot be learnt by reading, only by practising with feedback that says unambiguously whether you rolled along or crashed. Lessons 1–11 of this course already cover most of the commandments; here we connect them. For a forecasting engine, the commandments become fields: a deadline and a resolution source, components, a base rate, forces for and against, an unrounded probability, the history of one's own score. The olivLaw engine does not ask for them today: the Monte Carlo recalibration asks the AI model to report its Brier score ‘if history exists’, but never sends it the history from forecast_resolution.
“Implementing each commandment requires balancing opposing errors. Just as you can’t learn to ride a bicycle by reading a physics textbook, you can’t become a superforecaster by reading training manuals.”