Sunday, 8 January 2017

Something Fishy in the world of forecasting


The Bank of England’s chief economist, Andy Haldane, created something of a furore on Thursday evening when he admitted that “it’s a fair cop to say the profession is to some degree in crisis” following the EU referendum. Haldane suggested that this was a “Michael Fish moment[1]” for the economics community, whose models struggle to cope with irrationality. In a way this is heartening. Anyone who has read any of my posts over recent months will know that I am critical of much mainstream macroeconomic academic research, in which models are based on rational behaviour (here, for example). I am also highly wary of the output produced by economic models (including my own). What was less heartening was the way in which the press jumped all over the story and began to conflate some of the issues, which generated a number of below the line comments on many newspaper websites demonstrating widespread ignorance of the issues at hand.

The article which most raised my hackles was by Simon Jenkins in The Guardian (here), who wrote that “the profession owes the public an inquest and an apology” and accused economics of grovelling “before its paymasters in government and commerce.” Predictably, the Pavlovian responses of many reader comments were along the lines of “economics is not a science” and “experts don’t know what they are talking about”, thus confirming that Michael Gove was right all along. One small snag: Jenkins’s article was confused and wrong. He conflates the problems in economics and the issue of Brexit as though they were one and the same thing. They’re not.

One issue Jenkins failed to mention was the extent to which many mainstream macroeconomists believe that their discipline has taken a wrong turning. He only need read the blogs by Paul Krugman to get a flavour of that. You certainly won’t find me defending the use of rational behaviour in economic models (for the record, all the models I have ever built use adaptive, rather than rational expectations). His second mistake was to conflate economics as a discipline and the arcane practice of forecasting.  I don’t know how many more times I have to explain to people that economics is not a predictive discipline. We can build models which capture the aggregate performance of the economy over the past but their predictive ability depends on numerous exogenous factors which can throw the model-generated path off course.

It is true that the short-term forecasts in the wake of the referendum turned out to be wrong. But the forecasts were made against the backdrop of zero to partial information; a power vacuum at the heart of government; the expectation that Article 50 would be triggered sooner rather than later, as David Cameron had promised, and a hysterical media. They are the worst circumstances in which to make a forecast for the next two years. But as I have pointed out before, three of my four key pre-referendum predictions have been borne out: Sterling would collapse; the BoE would cut interest rates and the stock market would rally after an initial dip. I was wrong on gilt yields, which fell rather than rose. And time will tell whether my view that post-Brexit growth will slow relative to what we experienced previously will also be borne out.

Jenkins was also wrong to confuse the political and economic aspects of Brexit. The issue of Brexit is purely a political issue but it does have potentially significant economic consequences. Anyone trying to come up with a forecast for the UK over the next five years or so has to take into account the fact that trading arrangements will be different, and they will almost certainly lead to a loss of economic welfare. However, it is worth making the point (again) that most analysis produced before the referendum reckoned that a vote to leave the EU would knock an average of around 0.3% off annual growth for a period of ten to fifteen years. We will likely feel it, but it would put the UK’s growth rate more into line with that of Germany since the turn of the century, rather than the outperformance of recent years. In other words, not a headline-grabbing disaster but not a good outcome either.

Finally, it is worth repeating the point I have been making for some time (most recently last week, here) that point economic forecasts are more likely to be wrong than right and that the probability distribution of risks around that central case is a more useful metric of accuracy. In a way, it was surprising that Haldane did not mention this because the BoE has been a pioneer of probabilistic forecasting. And for all the criticism levelled at the BoE, its August forecast for Q3 GDP growth assigned a near-45% chance that it would exceed 2% - as indeed happened.

It can only be positive that economists hold their hands up and admit when they are wrong. But I am not sure that Haldane did the economics profession or the BoE many favours by giving the media a stick to beat us all with. Indeed, as Mark Harrison of Warwick University asked: “Is epidemiological science in crisis because public health officials did not predict the largest Ebola epidemic in history?”

[1] Michael Fish is a meteorologist who in 1987 famously predicted that rumours of an impending hurricane were unfounded (here). In the event, London was hit by its biggest storm since 1703 which caused major damage and disruption. Moreover, UK markets were effectively closed on the following day (Friday) at a time when Wall Street was tanking. When London markets reopened the following Monday, it prompted a far sharper London collapse, amplifying the negative global sentiment which contributed to the severity of Black Monday on 19 October 1987.

No comments:

Post a Comment