Superforecasting: The Art and Science of Prediction

The Good Judgment Project (started in 2011) indicates that well-chosen selected amateur forecasters who demonstrate their ability to accurately forecast are often more accurate than subject matter experts.^[1]

Superforecasting: The Art and Science of Prediction is a 2016 book about forecasting written by Philip E. Tetlock and Dan Gardner.

The book reports on a wealth of data coming from the The Good Judgment Project (started in 2011) which indicates that well-chosen selected amateur forecasters who demonstrate their ability to accurately forecast are often more accurate than subject matter experts.^[2]

The Harvard Business Review likened the book Superforecasting: The Art and Science of Prediction to the book How Not to Be Wrong: The Power of Mathematical Thinking by Jordan Ellenberg.^[3]

Book summary:

“

Everyone would benefit from seeing further into the future, whether buying stocks, crafting policy, launching a new product, or simply planning the week’s meals. Unfortunately, people tend to be terrible forecasters. As Wharton professor Philip Tetlock showed in a landmark 2005 study, even experts’ predictions are only slightly better than chance. However, an important and underreported conclusion of that study was that some experts do have real foresight, and Tetlock has spent the past decade trying to figure out why. What makes some people so good? And can this talent be taught?

In Superforecasting, Tetlock and coauthor Dan Gardner offer a masterwork on prediction, drawing on decades of research and the results of a massive, government-funded forecasting tournament. The Good Judgment Project involves tens of thousands of ordinary people—including a Brooklyn filmmaker, a retired pipe installer, and a former ballroom dancer—who set out to forecast global events. Some of the volunteers have turned out to be astonishingly good. They’ve beaten other benchmarks, competitors, and prediction markets. They’ve even beaten the collective judgment of intelligence analysts with access to classified information. They are "superforecasters."

In this groundbreaking and accessible book, Tetlock and Gardner show us how we can learn from this elite group. Weaving together stories of forecasting successes (the raid on Osama bin Laden’s compound) and failures (the Bay of Pigs) and interviews with a range of high-level decision makers, from David Petraeus to Robert Rubin, they show that good forecasting doesn’t require powerful computers or arcane methods. It involves gathering evidence from a variety of sources, thinking probabilistically, working in teams, keeping score, and being willing to admit error and change course.

Superforecasting offers the first demonstrably effective way to improve our ability to predict the future—whether in business, finance, politics, international affairs, or daily life—and is destined to become a modern classic.^[4]

”

Personal characteristics of superforecasters according to the research from the Good Judgment Project

See also: Forecasting

Characteristics of "superforecasters":^[5]^[6]

- Temperament: Cautious (Many things are uncertain), Humble (Reality is infinitely complex), Non-Determinist.

- Cognition: Open-Minded (Beliefs are hypotheses to be tested, not treasures to be protected), Intelligent, Curious (Intellectually curious, enjoy puzzles and mental challenges), Reflective (Introspective and self-critical), Numerate (Comfortable with numbers).

- Analytical: Pragmatic (Not wedded to any idea or agenda), Dragonfly-Eyed (Value diverse views and synthesize them into your own), Probabilistic, Thoughtful Updaters (When facts change, they change their minds), Intuitive Psychologist (Aware of the value of checking thinking for cognitive biases and emotional biases).

- Work Ethic: Growth Mindset, Grit (Determined to keep at it however long it takes)