Statistician and election forecaster Nate Silver explains why most predictions fail, and shows that better forecasting comes from thinking in probabilities, updating with new evidence and being humble about uncertainty.
Overview
The Signal and the Noise: Why So Many Predictions Fail, But Some Don’t was published in 2012 by Nate Silver, who had made his name by developing a system to forecast baseball performance, and then by his political blog, FiveThirtyEight, which correctly called 49 of 50 states in the 2008 US presidential election and, a few weeks after the book appeared, all 50 states in 2012. In the book, he looks at forecasting in many fields: the 2008 financial crisis, weather, earthquakes, economics, disease, chess, baseball, poker, politics, climate and terrorism. He asks why some forecasts succeed and others fail, and offers a philosophy based on Bayesian reasoning. The book is full of examples and is written in a conversational style.
Full Summary
1. Why we fail: a catastrophic failure of prediction
The idea: Silver opens with the 2007-2008 financial crisis. Rating agencies such as Standard & Poor’s gave certain mortgage-backed securities top safety ratings and badly underestimated their risk, and officials and economists did not see the housing bubble for what it was. He separates risk, which can be measured, from uncertainty, which is hard to measure, and says the agencies treated new, untested products as if their risks were known. He makes the same charge against confident economic forecasts made by government in early 2009. Using a target-practice picture, he shows that a forecast can be precise without being accurate: shots that always land in the same spot are consistent, but if they miss the bull’s-eye, they are not accurate. Forecasters, he says, often project certainty to protect their reputations, when better predictions begin with admitting uncertainty and bias.
In real life: A confident, exact-sounding number is not the same as a correct one.
2. Pundits, hedgehogs and foxes
The idea: Silver turns to the research of Philip Tetlock, who found that many expert predictions about politics were no better than chance. Tetlock divided thinkers into hedgehogs, who hold a few big ideas and apply them everywhere, and foxes, who draw on many sources and change their minds with new information. Hedgehogs did worse, and they tended to blame outside factors for their mistakes, while foxes owned their errors. Silver lists the habits of good forecasters as thinking in probabilities, adapting, and being open to several perspectives, and he says he tries to work this way himself. He contrasts this with television pundits, such as those on The McLaughlin Group, who, in his view, are not really trying to make accurate predictions.
In real life: Someone who changes their forecast when the facts change is usually more reliable than someone who never does.
3. Baseball and poker: models, scouts and process
The idea: Silver describes his own baseball system, PECOTA, in the era that Moneyball made famous. Models of this kind study players’ records, adjust for luck and aging, and compare players with similar ones from the past. He argues that they have not replaced scouts, who see things numbers miss, such as how a player behaves, and that the best forecasting uses both. He gives the example of Dustin Pedroia, a player whom PECOTA rated well but whom scouts doubted. Silver also played poker professionally for a time, and he describes the poker boom that began around 2003, when many inexperienced players made it easy for slightly better ones to profit. Good players think in conditional probabilities, including what opponents may be calculating. Because luck plays a large role in the short run, he says you should judge decisions by the quality of the process and not only by results.
In real life: A good player judges a decision by the odds, not just the result.
4. Weather: a success story
The idea: The author holds up weather forecasting as a field that has improved greatly over the past few decades, mainly through faster computers. Human forecasters still add value by correcting the computer’s output, especially for rain. Weather is a chaotic system, so small errors in the starting data can grow into large errors in the result, but meteorologists give probabilities, such as a thirty percent chance of rain, and check whether those probabilities were well calibrated. He also looks at Hurricane Katrina, where the forecasts were good but the response was not.
In real life: A rain forecast that says “70 percent” should be right about seven times out of ten.
5. Earthquakes and economics
The idea: In contrast, earthquakes cannot be predicted in time and place. Seismologists can estimate how likely quakes are in a region over a long period, but not say when one will strike. The data are noisy and the systems poorly understood, so it is easy to find patterns that are not real, a mistake called overfitting. Silver points to the 2011 tsunami at the Fukushima nuclear plant in Japan as a case where past evidence of large events was not given enough weight. Economic forecasting he finds poor too. The title of that chapter refers to the 1997 flood at Grand Forks, North Dakota, where the uncertainty in a forecast was not made clear. An economy is a complex system in which cause and effect are hard to separate, and forecasters can confuse correlation with causation and protect their reputations at the expense of accuracy. He sees promise in prediction markets, in which people put money behind their forecasts.
In real life: With many people guessing, someone will eventually be right by luck, which proves nothing about skill.
6. Disease and Bayes’s theorem
The idea: Epidemiologists work where mistakes cost lives, so Silver treats them as role models for caution. He recalls the 1976 swine flu scare in the United States, when officials expected an epidemic and President Ford backed a rushed vaccination program; the epidemic never came. Forecasts can be self-fulfilling or self-canceling, because people react to them. The central idea of the book is then Bayesian thinking, named after the eighteenth-century minister Thomas Bayes: start with a prior belief about how likely something is, then update it as new evidence arrives. Silver contrasts this with the frequentist approach of Ronald Fisher, which dominated twentieth-century statistics, and argues that it contributed to many false positives, citing John Ioannidis on how much published research is flawed.
In real life: If you hear a rumor, you weigh how likely it was before, and then adjust as you learn more.
7. Computers, groups and markets
The idea: Using accounts of chess matches between people and machines, Silver argues that computers do exactly what they are programmed to do. They excel where rules are clear and fast calculation matters, as in chess and weather, but struggle with messy systems such as the economy or earthquakes, and a forecaster’s blind spots are likely to be copied into the program. In the stock market, he finds that group forecasts made independently tend to beat individual ones, though not always the best individual. Traders often follow the herd, and institutions judged on short-term results help bubbles to form. We will never spot a bubble, he says, if we assume that the price is always right.
In real life: A crowd that thinks alike is not a crowd that checks its answers.
8. Climate, terrorism and the limits of knowledge
The idea: On climate change, Silver says the greenhouse effect almost certainly exists, that human emissions of carbon dioxide add to it, and that the planet is very likely to grow warmer, though the effects are less certain and scientists disagree about how reliable climate models are. Politics adds noise, since some aggressive forecasts come from modeling errors and others from efforts to influence policy, and the public often reads scientific uncertainty as weakness. On terrorism, he says it is very hard to tell signal from noise before an attack, and that people treat the strange as implausible. The pattern of attacks follows a power law, in which rare, very large events do occur, so analysts should imagine bigger attacks and take signs of them seriously. The worst failure, he adds, is to make no prediction at all. He closes by saying that people are biased toward thinking they are good at predicting, and that a Bayesian approach, with honesty about uncertainty and a willingness to revise, is the way to improve.
In real life: A forecaster who says what could change the forecast gives you a fuller picture.
Why It Matters
The Signal and the Noise reached the New York Times bestseller list, was named Amazon’s best nonfiction book of 2012, and won the 2013 Phi Beta Kappa Society book award in science. It is a leading popular introduction to forecasting, probability and Bayesian thinking, and it helped to bring statistical reasoning into public discussion, especially during election seasons. The book highlighted the value of humility, probabilities and testing, and it showed how overconfidence and incentives distort predictions. It draws on the research of Philip Tetlock on expert judgment, and it is a useful read for anyone who has to judge forecasts of any kind, in business, journalism, politics or sport.
Key Takeaways
- Separate signal from noise. Not every pattern in a set of data is real or meaningful.
- Think in probabilities. Avoid false certainty, and say how sure you are.
- Update with new evidence. Bayesian thinking helps you to adjust your view gradually.
- Test forecasts. Fields that check their results, like weather, improve over time.
- Be humble. Know the limits of knowledge, especially about complex systems.
What We Can Learn
You can start by expressing your own predictions as probabilities, and checking how often they come true. When you see a striking pattern in data, ask whether it might be chance. Be wary of confident forecasts from commentators who do not face consequences. Update your views when new evidence comes in, but avoid overreacting to each piece of news. And learn the basics of statistics, which help to read the news more critically. Combining numbers with judgment is a good habit.
Criticism and Counterpoints
The climate scientist Michael Mann criticized the book for treating the physical science of climate with the same approach used for voter preferences, resting on assumptions he called subjective and untestable. Because the book is written for general readers and leaves out most of the mathematics, as one admiring reviewer noted, it cannot treat every statistical method in depth, and Bayesian methods have difficulties of their own, such as how to choose prior beliefs. Silver’s later forecasts also fueled debate about probabilities: his 2016 model gave Donald Trump about a 29 percent chance, higher than other major forecasters, and Trump won. An outcome at those odds is entirely possible, and the episode showed how often probabilities are misread as predictions. Still, the book’s main message is widely valued.
Who Should Read It
This book is for readers interested in data, forecasting, sports and politics. It is engaging and requires little mathematics. Those who want a rigorous treatment of statistics or probability might consult a textbook.
Want the whole book?
A summary is a taste, not the meal. Here is where to find the full book.
You reached the end 🎉
Nicely done. Mark this one as read, and we will suggest what to read next.
Read this next
Superforecasting
Philip E. Tetlock and Dan Gardner
Political scientist Philip Tetlock and journalist Dan Gardner report on a research project that found some ordinary people can predict world events better than experts, and describe the habits that make them so good.
🧩 Mind & PsychologyThinking in Bets
Annie Duke
Former professional poker player Annie Duke argues that life's decisions are bets made with incomplete information, and shows how to judge decisions by their quality and not their results, by thinking in probabilities and by learning with the help of honest groups.
🧩 Mind & PsychologyHow to Lie with Statistics
Darrell Huff
Darrell Huff shows, in a light and witty tour of tricks used by advertisers, journalists and others, how numbers, charts and averages can mislead, so that readers can learn to spot the deceit.
🔬 ScienceMoneyball
Michael Lewis
Michael Lewis tells how the Oakland Athletics, a baseball team with a small budget, used statistics to find overlooked players and compete with richer teams, and how that challenged old beliefs about talent.
💼 BusinessThe Big Short
Michael Lewis
Michael Lewis follows a handful of unusual investors who saw that the American housing boom was a bubble before the 2008 financial crisis, bet against it, and discovered how deeply the financial system had misjudged its own risks.
💼 BusinessA Brief History of Time
Stephen Hawking
One of the world's greatest physicists explains, with almost no math, what we know about the origin, structure and fate of the universe, from the Big Bang to black holes and the nature of time itself.
🔬 ScienceWant something different?
Pick another topic and we will show you three books to start with.






