A short history of predicting the future

‘One might have reasonably predicted that a century or two after Malthus, people would have learned from his example to be much more cautious about using demography to predict the future. But that prediction would have been wrong. As they usually are.’

You can understand Dan Gardner’s sense of frustration. It’s been two centuries since the first ‘expert’ in demographics, the Reverend Thomas Malthus, made his predictions about impending mass starvation and food scarcity. His predictions were proven wrong. Yet equally spurious predictions, such as those by Paul Ehrlich in the 1970s, when he warned about an imminent ‘population explosion’, keep being made and keep getting taken seriously. Indeed, even after the prophecies are proven wrong, many, like Ehrlich, are lavished with awards and given influential platforms. What’s going on?

It’s not just population. In his book Future Babble: Why Expert Predictions Fail and Why We Believe Them Anyway., Gardner catalogues failed prediction after failed prediction. They range from Jimmy Carter’s claim in the 1970s that the era of cheap oil was over, which was followed by 20 years of oil being cheaper than on the day he made the pronouncement, to predictions about Japan’s emergence as the new superpower in the 1980s and journalist James Howard Kunstler’s prophesy about the ‘millennium bug’ bringing about widespread devastation as IT systems collapsed.

Even those who supposedly get it right are rarely the soothsayers they are made out to be. More often than not, they are simply lucky. Like the economist Peter Schiff, who is famously credited with having ‘called’ the financial crisis of 2008, they’ve managed to get it right in the way a stopped clock gets the time right twice a day. And, despite extremely complex models, even the weatherman can only get predictions broadly right a couple of days ahead with any degree of certainty.

Gardner’s book is extremely valuable in raising questions about forecasting and in systematically challenging the assumptions of those making predictions about the future. He even includes a qualified challenge to the models that predict the extent and effects of climate change. When even the most confidently made predictions are often proven false by subsequent events, why do we continue to rely on them? Unfortunately, his attempt at explaining this is as wrongheaded as the forecasts he criticises.

His thesis, in a nutshell, is: ‘Try to predict an unpredictable world using an error-prone brain and you get the gaffes that litter history.’ Unless we are omniscient, reasons Gardner, then ultimately any analysis of how future events are going to pan out will mean we miss things. The reason we cling to certainty, despite its impossibility, is because we have primitive brains, says Gardner: ‘Blame evolution. In the Stone Age environment in which our brains evolved, there were no casinos, no lotteries and no iPods. A caveman with a good intuitive grasp of randomness couldn’t have used it to get rich and marry the best-looking woman in the tribe.’

Because our brains haven’t evolved to deal with the complexities of an uncertain world, apparently, we are attracted to latter-day Nostradamuses such as Ehrlich and Kunstler, no matter how wrong they are. Gardner explains the reasons why using philosopher Isaiah Berlin’s concept of a fox and hedgehog – an analogy used to undertake a critique of Tolstoy that has since been bastardised by scores of two-bit management theory gurus and which Gardner would have been well-advised not to touch with a bargepole. Gardner suggests the problem is that people, with their cavemen brains, are attracted to the totalising approach of ‘hedgehogs’ to provide a universal framework for understanding matters, rather than considering the views of the fox, which is adept instead at analysing particular situations, remaining doubtful and making a virtue out of knowing ‘many little things’.

He calls for us to be more ‘fox-like’, arguing that in the face of such scientific uncertainty, and with our fallible brains, we need to accept more humility and exercise greater caution when dealing with the future. We should not ‘put all our eggs in one basket’. It’s only in the postscript, however, once he’s finished giving a grand (some might say ‘hedgehog-like’) theory about how we’ll never be able accurately to predict future events that Gardner becomes a bit more ‘fox-like’ himself, and acknowledges that his conclusions rest upon chaos theory and evolutionary theories of consciousness being correct. Given the spurious, unscientific nature of much of evolutionary theory when it is used to attempt to explain human behaviour and social phenomena, it comes as a surprise that such a proud sceptic swallows it wholesale.

In attempting to offer timeless, evolutionary arguments about why expert predictions appeal to us, Gardner misses the opportunity to get to the heart of the phenomenon of soothsaying and apocalypse-mongering in the twenty-first century. What does the propagation, and acceptance, of largely baseless predictions founded on shaky assumptions say about contemporary culture? Gardner is too quick to write off humans as having something akin to a mental disorder, rather than examining social trends critically. Ironically, as a result, he ends up simply regurgitating another of the contemporary prejudices of our age: that human beings are far more irrational and subject to delusions and unconscious pressures than we like to think we are.

Gardner’s talk of the future makes us appear alienated from it; the future becomes something that we merely watch unfold and try to second-guess at our peril. He, tellingly, fails to consider ways in which predictions themselves shape and influence the future. He claims, for example, that ‘it is a truism that novels set in the future say a great deal about the time they were written and little or nothing about the future’, neglecting to mention how influential were the works of Orwell and Huxley in raising awareness of possible futures and also warning against them. Arguably this is also being undertaken by neo-Malthusians such as Ehrlich, whose scaremongering declarations about overpopulation may well reflect a misanthropic desire to influence policy on the issue as much as predict the future. Gardner gives scant consideration to the political motivations behind predictions, and how such agendas may muddy the water when it comes to the accuracy of forecasts.

At one point he speaks of our desire to shape the future and be successful against the odds as an understandable, yet primitive evolutionary urge. We have the delusion, when starting a new business or getting married, that we will be successful, even though the probability is that we won’t be. Fundamentally, his call for us to be more realistic and cautious and to show humility in the face of an uncertain future reflects just how estranged we have become from the process of making our own history. We are, instead, reduced to the role of humble, passive observers of history – aware that, following Socrates, all we know is that we know nothing – rather than those struggling to overcome obstacles in order to bring about the future outcomes we desire.

There is, Gardner is right to point out, much babbling going on about the future, which should be met with robust, unrelenting criticism. But instead of accepting that the future is a dark, ultimately unknowable place, we should focus on how to gain greater understanding and greater control over it.

Patrick Hayes is a reporter for spiked.