[Photograph by ImageParty under Creative Commons]
In 2005, Philip E Tetlock wrote a book called Expert Political Judgement which said that on an average experts do a pretty bad job in forecasting events. Even though there are some good forecasters, the rest pull the average down to make them no better than dart throwing monkeys.
In his new book, Superforecasting: The Art and Science of Prediction, Tetlock looks at those who get it right. He examines the superforecasters, how they are so good at it, and what others can learn from them. These forecasters were identified from the US Intelligence community’s Good Judgement Project, a contest that challenges participants to predict future events. The participants are volunteers, with varied backgrounds, and no access to inside information. Yet, some of them bettered intelligence professionals.
The book argues that these superforecasters didn’t achieve it because they had superior intelligence, or better math skills or because they were better updated about the world. All these help, but don’t guarantee accuracy. What matters, Tetlock found, was how they think about the issues. Among others, they used frameworks and ideas from some of the best minds in the world, past and present. Here are six of them.
Isaiah Berlin (1909–97)
[Isaiah Berlin for PIFAL by Arturo Espinosa under CC 2.0]
Who is he? He is a British philosopher, political theorist and a historian of ideas.
What did he say/do? In an essay The Hedgehog and the Fox (1953), Berlin compared the thinking styles of great writers, and used animal metaphors to describe them: Hedgehog for those who know one big thing, one big idea and pursue them relentlessly; and Fox for those who know many things.
How will it make you a better forecaster? Be a fox. Tetlock says foxes beat the hedgehogs hands down. Hedgehogs tend to gather more and more evidence in support of their big idea and use transitions like moreover, and furthermore in their arguments. Foxes on the other hand tend to approach an issue from several angles. Their narratives are complex. And they tend to use terms like however, on the other hand, etc. The result: hedgehogs can be impressive. Who doesn’t like a simple, confident narrative? But, foxes are more likely to see the truth. And as a result, they are better forecasters.
Enrico Fermi (1901-53)
[Enrico Fermi 1943-49 by Department of Energy. Office of Public Affairs via Commons]
Who is he? He is an Italian-American physicist who created the world’s first nuclear reactor and won a Nobel for his work on radioactivity.
What did he say/do? As a teacher, he challenged students with brain teasers such as “how many piano tuners are there in Chicago?” The students were expected to find the answers by breaking down the question into sub-questions that could be answered either with available information or educated guess.
How will it make you a better forecaster? Tetlock says the method—he calls it Fermi-izing—helps the forecasters figure out what’s known and what’s not known, what’s knowable and what’s not knowable. It also dares us to be wrong, and hence pushes us to shed the deep rooted fears of looking dumb. In fact, some of the top tech companies use Fermi questions in their recruitment, hoping to hire people who are comfortable working with little information and a lot of uncertainty.
John Maynard Keynes (1883-1946)
[John Maynard Keyes under CC 3.0]
Who is he? He is an influential British economist, upon whose ideas the Keynesian school of economic thought, which stresses on government spending during recessions, stands.
What did he say/do? “When the facts change, I change my mind, what about you, sir?” (He might not have actually said this, but that defined his approach to economics.)
How will it make you a better forecaster? The best forecasters constantly update their forecasts, says Tetlock. It’s not easy, because people tend to stand by the initial statement. It’s a cognitive bias that salesmen sometimes exploit. But, being prepared to change one’s position is the key to getting it right. Not being an expert might actually help. Tetlock says, “They aren’t deeply committed to their judgments, which makes it easier to admit when a forecast is offtrack and adjust.” Constant updating is also at the core of Bayesian method, a favourite tool among top forecasters.
Robert Rubin (1938 - )
[Robert Rubin, US Treasury Secretary (1995-99) by Chatam House under CC 2.0]
Who is he? He is a former US Treasury Secretary, and a Goldman Sachs veteran.
What did he say/do? When he was at Harvard University, he learnt the axiom that “there’s no provable certainty”, and that became his guiding principle through his career. (He called his memoir In an Uncertain World.)
He is a probabilistic thinker. A New York Times piece had this revealing anecdote from one of his colleagues: “One of the first times I met with him, he asked me if a bill would make it through Congress, and I said, ‘Absolutely,’” a young Treasury aide recalls. “He didn’t like that one bit. Now I say the probability is 60 percent—and we argue about whether it’s 59 or 60.”
How will it make you a better forecaster? Try to get very specific, as if there is a lot of difference between 59 percent probability of something happening and 60 percent. The Good Judgement Project data showed that “granularity predicts accuracy: the average forecaster who sticks with the tens—20 percent, 30 percent, 40 percent—is less accurate than the finer-grained forecaster who uses fives—20 percent, 25 percent, 30 percent—and still less accurate than the even finer-grained forecaster who uses ones—20 percent, 21 percent, 22 percent.” So crunch numbers a bit more, and get even granular.
Daniel Kahneman (1934 - )
[Daniel Kahneman by Eric Benson under CC 2.0]
Who is he? He is an Israeli-American psychologist who won a Nobel prize in Economics.
What did he say/do? In Thinking, Fast and Slow, he explained we use two types of thinking. System 1 is automatic, fast and based on simple heuristics. System 2 is slow, deliberate and calculating. System 1 has its uses, but prone to cognitive errors, and you jump to wrong conclusions. System 2, makes you pause, question, think over, and eventually helps you get a better grasp of the reality.
How will it make you a better forecaster? Superforecasters are good intuitive psychologists, says Tetlock. They know the value of checking thinking for cognitive and emotional biases. One of the biases that bad forecasters are prone to is to substitute the question we want answered with one that’s easy to answer.
Helmuth von Moltke (1800 - 1891)
["Helmuth Karl Bernhard von Moltke" by Kunstverlag der Photographischen Gesellschaft Berlin - Albumin-Foto via Commons]
Who is he? He is a German field marshal and military theorist.
What did he say/do? “War cannot be conducted from the green table. Frequent and rapid decisions can be shaped only on the spot according to estimates of local conditions,” he once wrote, a principle he believed in. Yet, he commanded one of the most successful armies of the 19th century—acting resolutely with no half measures. The two might sound contradictory, but in fact shows that it’s key to balance “nothing is certain” and “unwavering determination” to achieve results.
How will it make you a better forecaster? As a leader, you have to juggle both the balls. Leadership demands confidence, determination and resoluteness. Forecasting demands humility. Humility to accept that the world is uncertain; to change one’s own opinion when facts change; to acknowledge that our judgements are flawed and beliefs might be wrong. It’s intellectual humility, and it can sit comfortably with confidence. Pursue both.
