Political scientist Philip Tetlock and journalist Dan Gardner report on a research project that found some ordinary people can predict world events better than experts, and describe the habits that make them so good.
Overview
Superforecasting: The Art and Science of Prediction was published in 2015 by Philip E. Tetlock, a psychologist and professor at the University of Pennsylvania, and Dan Gardner, a journalist. Tetlock had earlier studied expert predictions for twenty years and found that the average expert was only a little better than chance, a result summarized as “the dart-throwing chimpanzee.” Later, he co-led the Good Judgment Project, a team in a forecasting tournament sponsored by IARPA, a US intelligence research agency. The best forecasters in it, whom he calls superforecasters, reportedly did better than intelligence analysts with access to classified information.
Full Summary
1. Why most forecasts are poor
The idea: The book begins with Tetlock’s earlier research. Over roughly twenty years, he collected predictions about political and economic events from hundreds of experts and scored them as events unfolded. On average, the experts did only slightly better than chance, and those with the biggest media profiles did especially badly. The best forecasters were those who thought like foxes, who know many small things and draw on many sources, not hedgehogs, who know one big thing and apply it everywhere.
In real life: A famous commentator with a bold theory may be less reliable than a careful person who weighs many views.
2. Making forecasts measurable
The idea: The authors say that most forecasts are vague, so that nobody can tell if they were right. Good forecasting requires clear questions, with a time limit and a probability, such as a seventy percent chance that a particular event will happen by a stated date. These can be scored with a method called the Brier score, which rewards being both accurate and well calibrated. The tournament, which began in 2011, gave several thousand volunteers such questions, and tracked their results.
In real life: “It might rain soon” can never be wrong, but “70 percent chance of rain by noon” can be tested.
3. The superforecasters
The idea: Tetlock describes the people who did best, who were not necessarily geniuses or experts, but ordinary volunteers, among them a Brooklyn filmmaker, a retired pipe installer and a former ballroom dancer. They tended to be open-minded, and above all, they were careful thinkers who liked to look at problems from many angles. Working with others helped. The top forecasters were selected into groups of superforecasters and performed consistently well over time and across different questions, which suggests that it is skill and not just luck.
In real life: Some people consistently pick the winners of matches, because they follow the evidence closely.
4. Breaking problems down
The idea: The authors explain a key technique: break a big question into smaller parts, in the style of the physicist Enrico Fermi’s estimation problems. First find a base rate, how often similar events happen, and then adjust it up or down based on specific evidence. Superforecasters start from the outside view, the general statistics, before the inside view, the details of the case. This protects against the common mistake of being drawn in by a story.
In real life: To judge whether a new restaurant will survive, start with how many new restaurants survive in general.
5. Updating and avoiding extremes
The idea: Superforecasters think in probabilities and revise their estimates as circumstances change, and are not too attached to their first view. They avoid both underreacting, sticking to an old belief, and overreacting, jumping at each news item. They are comfortable with shades of probability, and use fine distinctions, such as 63 percent, not just “likely.” They also keep a habit of admitting error, and learning from it.
In real life: A weather forecaster who adjusts the chance of rain as clouds arrive is updating well.
6. Humility, teamwork and the limits of prediction
The idea: The book gathers its lessons in a set of “Ten Commandments for Aspiring Superforecasters.” The overall recipe is to gather evidence from varied sources, think probabilistically, work in teams, keep score, and be willing to admit error and change course. Tetlock reminds readers that some things cannot be forecast, particularly in the long run. The book also argues that experts and public figures should be held accountable for their predictions, and that organizations should keep score to make better decisions.
In real life: A student who tracks their predictions about test scores can see where their judgment is off.
Why It Matters
Superforecasting has been influential in business, government and intelligence, in which readers have taken up ideas such as the scoring of forecasts, base rates and teamwork. It showed that forecasting is a skill that can be measured and improved, and is an antidote to both the cult of the confident expert and the idea that the future is entirely unknowable. Jason Zweig of the Wall Street Journal called it the most important book on decision making since Daniel Kahneman’s Thinking, Fast and Slow. The Good Judgment Project also gave rise to a commercial spin-off, Good Judgment Inc., which started in 2015.
Key Takeaways
- Be a fox, not a hedgehog. Use many sources and views.
- Make forecasts specific. Use clear questions and probabilities.
- Start with base rates. Look at the outside view first.
- Update often and gently. Adjust as evidence arrives.
- Keep score and learn. Track results and examine mistakes.
What We Can Learn
You can practice forecasting by making small, clear predictions, such as whether a team will win or whether you will finish a task by a date, with a percentage, and then checking the results. Before judging a case, ask how often things like it turn out a certain way. Look for opposing views, and ask what would change your mind. When you are wrong, treat it as information. And avoid vague claims in your own speech, since they cannot be tested. These habits can improve decisions in school and work.
Criticism and Counterpoints
Some critics say that the forecasting questions in the tournament were short-term and specific, so that the results may not apply to the big, rare events that matter most, such as financial crises, which Nassim Taleb argues cannot be forecast. One reviewer in the Journal of Intelligence Studies in Business was critical that the book did not give enough information on how the Good Judgment Project beat other forecasters. Others suggest that organizations may find the methods hard to apply, and that much expert work is not about prediction. Still, the research is widely respected.
Who Should Read It
This book is for readers interested in decision making, politics, business and statistics. It is clear and fairly short. Those who want a technical treatment of forecasting can read the original studies from the Good Judgment Project.
Want the whole book?
A summary is a taste, not the meal. Here is where to find the full book.
You reached the end 🎉
Nicely done. Mark this one as read, and we will suggest what to read next.
Read this next
Thinking in Bets
Annie Duke
Former professional poker player Annie Duke argues that life's decisions are bets made with incomplete information, and shows how to judge decisions by their quality and not their results, by thinking in probabilities and by learning with the help of honest groups.
🧩 Mind & PsychologyThe Signal and the Noise
Nate Silver
Statistician and election forecaster Nate Silver explains why most predictions fail, and shows that better forecasting comes from thinking in probabilities, updating with new evidence and being humble about uncertainty.
🔬 ScienceThe Scout Mindset
Julia Galef
Julia Galef contrasts the "soldier mindset," which defends our beliefs, with the "scout mindset," which tries to see things as they are, and shows how to build the habit of honest, curious thinking.
🧩 Mind & PsychologyThinking, Fast and Slow
Daniel Kahneman
Our minds run on two systems, a fast, intuitive one and a slow, careful one, and understanding how the fast system fools us can help us make better decisions about money, risk, work and happiness.
🧩 Mind & PsychologyThe Art of Thinking Clearly
Rolf Dobelli
Swiss author Rolf Dobelli collects about a hundred short chapters on common thinking errors, such as confirmation bias, survivorship bias and the sunk-cost fallacy, to help readers make fewer foolish decisions.
🧩 Mind & PsychologyPredictably Irrational
Dan Ariely
Behavioral economist Dan Ariely uses entertaining experiments to show that people's decisions are shaped by predictable mental quirks, such as relativity, the lure of "free" and social norms, rather than by cool, rational calculation.
🧩 Mind & PsychologyWant something different?
Pick another topic and we will show you three books to start with.





