Of course, the press gleefully highlighted the prediction error – as they did with all those who failed to correctly predict the winner. The only thing was, I didn’t really get it wrong. Although I made Germany the most likely team to win, I only assigned an 18% probability to their chances of tournament success, implying an 82% chance of not winning. In bookmakers’ parlance, I put the odds against Germany winning the tournament at 4-1. Sure enough, Germany did not win the tournament – the most likely outcome predicted by the model.
The idea that we apply probabilistic assessments to outcomes strikes me as a sensible way to think about an inherently unknowable future and it is a point I have made on numerous occasions previously (here, for example). At a time when macroeconomics has come in for considerable criticism for its failure to accurately forecast future events, understanding the process of how forecasts are made is worthy of further investigation.
Critical to understanding the nature of an economic forecast is that they are heavily conditional. In fact everything in economics depends on everything else, so if some of the conditioning factors change the forecast is likely to be blown off course. Consider the case of forecasting how a central bank might set interest rates on the basis that it follows an inflation targeting regime. We assume that inflation is a function of the amount of spare capacity in the economy – the less slack there is, the more competition for resources which then bids up their price. The choice of model itself is a major conditioning factor. If central banks use different metrics in making their decision, this raises the chance that the forecast will be wrong.
But let us pursue our assumption a bit further: In order to determine how much slack there is in the economy, we have to understand trends on both the demand and supply side which introduces additional conditioning factors. On the demand side we need to know what is the likely path of driving forces such as incomes, taxes (which influence disposable incomes and labour supply decisions) and wealth (which can be used to finance consumption and which also impacts on desired saving levels). On the supply side, we need to know something about changes in the capital stock, which requires assumptions for investment and the rate of capital depreciation; the size of the labour force and the path of multifactor productivity. It should be pretty obvious by now that in a short space of time, we have identified a whole chain of events which could impact at any point to change our assessment of the amount of spare capacity and thus the potential inflationary threat.
It is pretty unlikely that we are going to predict all the inputs correctly, with the result that there is a considerable margin of uncertainty associated with our projections. When the economy is subject to an exogenous shock, such as in the wake of the Lehman’s bust or Brexit, the degree of uncertainty is significantly raised. Consider the UK in the wake of the Brexit vote: There was no effective government following David Cameron’s resignation and it was totally unclear whether the UK would invoke Article 50 in June 2016, as some had advocated. In this vacuum of uncertainty, large forecasting errors were made in the immediate post-referendum environment.
But contrary to the statements made by a number of pro-Brexit politicians about how the doomsayers were wrong before the referendum, much of what the forecasting profession said has stood up to scrutiny. Notably that the pound would collapse, inflation would rise and the economy would grow more slowly and thus suffer a loss of output relative to the case of no vote for Brexit. Obviously we don’t know what will happen from here because we do not yet know the nature of the UK’s future trading arrangements with the EU. One way to proceed is to outline a number of scenarios and assess what might happen to growth in each case. If we assign a probability to each scenario then our best guess for output growth is the probability-weighted average of the outcomes.
But how useful is the single point estimate for annual growth over the next five years in the case of Brexit? The answer, I suspect, is not much. We are more interested in the cost of our forecast being wrong (i.e. whether we are too optimistic or pessimistic relative to the outturn). We thus should focus on the loss function, which measures the cost of being wrong. This is not something that gets the attention it perhaps deserves because it can be a costly and time-consuming exercise. Instead we generally define a range of forecast extremes which encompass a median (or modal) forecast. The extent to which this forecast lies in the upper or lower half of the range determines the extent to which forecast risks are asymmetric, giving us some idea of the costs of being too optimistic versus being overly pessimistic.The Bank of England has long been an advocate of this approach (see chart, which assesses the range of outcomes for the August 2018 inflation forecast).
My efforts at forecasting future economic events are guided by Niels Bohr’s quip that “prediction is very difficult, especially if it's about the future.” Experience has taught me that we should treat our central case economic predictions as the most likely of a range of possibilities, and nothing more. After all, there is no certainty. When Germany can underperform so spectacularly on the international football stage, even the most confident of forecasters should take note.
No comments:
Post a Comment