If expert predictions were always right, the world would be a different place: The Dow Jones would have hit 36,000 a decade ago, and Bill Clinton wouldn't have stood a chance in the 1992 presidential election (though Hillary would have trounced Barack Obama in the 2008 primaries). How do well-informed experts so often miss the mark? Dan Gardner, a bestselling author and award-winning journalist for the Ottawa Citizen, spoke to U.S. News about this phenomenon, which he explores in his new book, babbleFuture Babble: Why Expert Predictions Are Next to Worthless, and You Can Do Better. Excerpts:
Why do so-called experts have such a poor track record on making predictions?
The nature of reality is that the world is unpredictable, and the brains that we use to make predictions are systematically flawed. Combine a flawed brain with an impossible task, and you're going to get mistakes.
Are you saying that expert prediction-making is altogether bad?
Prediction is an inherent part of our lives. What I'm arguing is that we have to learn to distinguish between predictions that are reasonable to attempt and those that are completely unreasonable. The second aspect is learning to distinguish between reasonable [prediction] methods and unreasonable methods. Unfortunately, the worst combination is gross overcertainty combined with attempts to do grand-scale prediction, which is simply impossible. [See photos of the Obamas behind the scenes.]
What's a more reasonable method?
The expert who is comfortable with complexity and uncertainty, who gets info from diverse sources, who is prepared to say, "I can see many factors pointing in a certain direction, but there are countervailing factors which may cause things to go in a different direction"—they are the sorts of people who are most likely to be right.
What was your goal in writing this book?
Let me start by saying what it's not about: What I'm not saying is, "Ignore experts; experts know nothing." Experts do know a lot. But we have to learn to distinguish between those experts who have some reasonable likelihood of being correct and the other sort of experts who are, frankly, blowhards.
What have been some of political pundits' biggest recent blown calls?
If you were to round up all the pundits in the U.S. who said there was no way that Hillary Clinton would lose the  Democratic nomination for the presidency, you would round up, I think, pretty much every pundit in the U.S. One of my favorite examples is [Fox News commentator] Dick Morris's 2005 book about the 2008 presidential election, which was called Condi vs. Hillary. There's just an endless list of these things.
Is there some element of attention-seeking in making a grandiose prediction?
I think, in most cases, pundits sincerely believe what they're saying, if only because that's basic human psychology. If you commit to a position, you will convince yourself that it's true. In fact, I suspect that, frankly, they believe it more strongly after they say it. [Check out a roundup of this month's best political cartoons.]
Do you see the same types of predictions regularly being repeated?
Absolutely. One of the biggest and oldest, of course, is Malthusianism [which predicts that population growth will outpace food production]. That one has been recurring for a couple hundred years now. It had a big resurgence in the early 1970s, and it's having a big resurgence now. And no matter how many times it fails, experts don't learn to at least temper their predictions. Another big theme is the foreign threat that is going to take over our economy and control us all. Today it's China that's going to take the lead and dominate the United States. Twenty-five years ago, it was Japan that was going to take the lead and dominate the United States, and 25 years before that, it was the Soviet Union. No matter how many times these firmly held convictions fail to come to pass, the experts who then make predictions are just as confident and just as sure of themselves as they were in the past.