Error correction (or blame deflection?)

For anyone interested in the practice and methodological issues associated with economic forecasting, you could do a lot worse than read the Bernanke Review of forecasting at the Bank of England. According to the FT, the former Fed chair who was commissioned to produce a report on the BoE’s forecasting practices after its failure to predict the rise in inflation in 2022, was “brutally honest about [its] failings.“ Brutal might be overstating it, but it was an honest assessment that one feels is shared by many BoE insiders. As one might expect, not all economists agreed with all of its conclusions but there was a lot to like about it.

The fact that Bernanke outlined many shortcomings in the BoE’s practices should come as no surprise. No system is ever perfect, and the fact that the current monetary framework has been in place for almost 30 years does suggest that it is time to have a close look. There are a number of questions around the whole process, however. Why was it necessary to have such a review in the first place? If the processes really are as poor as Bernanke highlighted, why did it require an external review to point it out? And if the purpose of the exercise was to address policy errors, should we not be spending time looking at the policy making process rather than putting a lot of effort into the forecast generation process? I will deal with these points below.

What were the conclusions?

It is perhaps instructive first to reflect on Bernanke’s main process recommendations. One of the most widely trailed in advance was the suggestion that the BoE publish scenarios alongside the main forecast. This would “help assess the costs of potential risks to the outlook” and “stress test the judgements made by the MPC.” There is a lot of merit in doing this: The experience of recent years which has produced the Covid-19 pandemic and the oil price shock, suggests that a single forecast with a univariate central case cannot adequately capture all future states of the world. Even allowing for risks in the form of a fan chart, no forecast could capture shocks of the magnitude of 2020 (see chart below). The Bernanke Review went as far as suggesting that “the fan charts as published in the MPR have weak conceptual foundations, convey little useful information over and above what could be communicated in other, more direct ways, and receive little attention from the public. They should be eliminated.” While there is some truth in this, it may be going too far to eliminate them, as fan charts are a very useful way of conveying risks around a central case in a stable environment, and there is a case for retaining them.

Another very important consideration was the nature of conditioning assumptions, particularly for the future path of interest rates. There are a number of reasons why using market rate expectations as the appropriate starting point is less than optimal. For one thing, “forward rates implied by the market curve are not pure forecasts of future rates, because forward rates may incorporate risk and liquidity premiums.” In addition they may not reflect the MPC’s best judgement of the path of rates, meaning that “a forecast conditioned on the market curve may be misleading.” One alternative is for the central bank to give a preferred path for rates, much as the Riksbank does, although as I argued in this post in 2019, this could simply create a hostage to fortune. Instead, the practice of offering alternative scenarios based on different rate paths will probably suffice.

A final big point, and one that is close to my heart, is that the software required to manage and manipulate data “is seriously out of date and difficult to use” and should be upgraded and constantly monitored. I don’t know which systems Bernanke is referring to but my own experience with languages such as R and Python, now en vogue in economic circles, is that they are far less user-friendly and flexible than some of the systems designed in the 1970s. The review was also critical of the BoE’s macro model, COMPASS, unveiled to great fanfare in 2013. Bernanke did not explicitly say that DSGE models may not be up to the job of forecasting but he offered the view that structural models (of the kind I have long advocated) still have a role to play in forecasting – after all, the Fed still uses them.

Policy considerations

The elephant in the room, however, is why it was felt that such a review was required in the first place. The answer, to put it bluntly, is that it was designed to keep politicians off the BoE’s back after it was accused of failing to predict the huge rise in inflation in 2022 (true) and the fact that its policy response was too slow (less true). In fact, the BoE's inflation forecast in February 2022 was above that of the consensus, predicting end-2022 inflation at 5.8% versus a consensus expectation of 4.6% (outturn: 10.8%) and end-2023 inflation at 2.5% versus the consensus prediction of 2.1% (outturn: 4.2%). Thus, while the BoE forecast was a significant under-estimate, it was less so than most forecasters.

As for the policy response, as I (and many others) have noted previously there was little anyone could have done to prevent an inflation spike in the face of an external oil price shock. Recall that the UK had just come off the back of a pandemic which had resulted in the steepest decline in output in 300 years and whose long-term effects were at that time still unknown. It did not feel like the right time for a sharp tightening of monetary policy. However a review of process is a standard response to issues that are more a matter of policy. Simply put, it is a way to deflect attention.

Another issue worth addressing is the question raised by the Sunday Times economics editor David Smith as to why it took an external review to highlight these shortcomings, which were well known internally. We are very much in speculative territory here, but since Bernanke took a lot of evidence from BoE insiders – past and present – it is hard to avoid the conclusion that this review offered an opportunity to tackle internal inertia. This may be the result of senior managers lack of knowledge of the issues involved; the fact that their attention has been diverted by other policy matters in recent years (Brexit, the pandemic) or simply a lack of budget resources. Either way the Review is a good way to get their attention.

Last word

It is always a good thing to review forecast models and processes, especially when they have been in place for so long and the Bernanke Review put the BoE’s process under a lot of scrutiny. In many ways it simply came across as a call to modernise a system, which in the grand scheme of things was already pretty decent but perhaps had been neglected a little over the past decade. However, the one thing it will not fix is that the future is inherently unknowable. No matter how state of the art, no forecasting system can cope with the kind of shocks to which we have been subject of late. Give it another decade and we will be having this debate all over again.

Telling it straight

Last week I was fortunate enough to attend a speech given by MPC member Gertjan Vlieghe in which he argued quite forcefully that the BoE’s method of communicating with the market is flawed and that there are better ways to deliver a clear message on interest rates. This issue is particularly topical given the concerns expressed about the BoE’s recent attempt to convince markets there is still scope to raise interest rates, despite the apparent change in global monetary conditions which argues against such a move.

We should, of course, recognise that central banks have come a long way since they told us nothing about their intentions (I am indeed old enough to remember when the Fed first started publishing its interest rate announcements in 1994). Even if the BoE’s communications are imperfect, they are a considerable improvement on where we were just over two decades ago. In Vlieghe’s view, however, the BoE should move away from a process of  communicating the path of interest rates by showing “a forecast of what will happen if we do something else” to explicitly telling us its preferred path for rates. It is recognised by a number of policymakers that there is a problem with the current communications policy because it is impossible to derive a unique path for interest rates “simply by observing how far the inflation forecast is above or below target at the end of the forecast period.” In effect, the MPC is “asking outside observers to solve a complex reverse-engineering problem that cannot be uniquely solved.” 

A number of other central banks have adopted a policy of publishing a preferred path for interest rates in a bid to improve policy transparency with most of the evidence suggesting that they are more satisfied with this approach than with the partial transparency adopted by the BoE. I have argued previously that one of the arguments against this idea is that it could be interpreted as a commitment rather than a forecast – a criticism that Vlieghe accepts. But in his view, “none of the central banks that have made the change, have reported this type of systematic misinterpretation between forecasts and promises … They have taken active steps to ensure public understanding of the uncertain nature of the interest rate forecasts, often by publishing uncertainty bands or fan charts rather than only a central path, and always by emphasising the uncertain nature of the path and its data-dependence.”

But the BoE has gone to great lengths to do much the same thing – indeed it was a pioneer of using fan charts – and that has still not prevented large parts of the commentariat from criticising the BoE for the inaccuracy of its forecasts. My own concern is that in an environment where central banking is becoming increasingly politicised, it would be extremely unwise for the BoE to create a hostage to fortune by publishing a preferred interest rate path. Recall the criticism levied at BoE Governor Carney by MP Pat McFadden that he was like an “unreliable boyfriend” due to the hints of interest rate hikes that never materialised (“one day hot, one day cold, and the people on the other side of the message are left not really knowing where they stand”). Imagine how much worse that criticism would be if the BoE produced a path for interest rates that was not subsequently adopted.

I do increasingly wonder whether giving ever more information to markets under the guise of improving transparency might in any case be counterproductive. It always helps to have a little something extra up one’s sleeve in case of need, and an unanticipated burst of monetary tightening would certainly be a good way to fire a warning shot across the bows of the market. Not that central banks appear to believe they have a duty to take the punchbowl away these days, but there is a strong argument that they should be doing more at a time when expectations of further easing have helped to drive equity markets to record highs, even though there are few good fundamental reasons for this. Moreover, it is slightly ironic that central bankers should even be talking about publishing interest rate paths when rates have barely moved in the past decade. If anyone had published a path for European rates in 2009 that showed them at similar levels – or lower – 10 years on, they would have been laughed out of court.

Whilst I do take Vlieghe’s point, and I have a lot of time for his analysis, I do wonder whether he may be somewhat missing the point about communications. It seems that the more central banks communicate, the more markets want. Sometimes it might be better to give them a little fright every now and then.

Don't be casual with words

They say that the pen is mightier than the sword. Consequently, it is incumbent upon us all to use our words judiciously. But it is also important that those consuming any given message are careful to interpret the information given to them without extrapolating beyond what is in front of them. This is particularly important in a world riddled with fake news in which messages can be subtly tweaked to say something that was not in the original communication, which is then passed on down the chain like the old game of Chinese whispers. It is also an issue for policymakers, particularly central bankers who are trying to communicate with markets and the wider public.

This issue was thrown into sharp relief by the recent TV interview by British Foreign Secretary Boris Johnson, who in response to the question of whether Russia was the source of the poison used in the Salisbury incident replied: “When I look at the evidence, the people from Porton Down, the laboratory… they were absolutely categorical, I mean, I asked the guy myself, I said, 'are you sure?' and he said 'there's no doubt.'” Only this week, Gary Aitkenhead, the chief executive of the government’s Defence Science and Technology Laboratory stated that whilst the government combined the laboratory’s scientific findings with information from other sources to conclude that Russia was responsible for the Salisbury attack, “we have not verified the precise source.”

It would appear that Johnson jumped to a conclusion that may not (yet) be supported by the evidence – statistically known as a Type I error. Meanwhile, the more cautious Aitkenhead refused to deal in speculation – as befitting someone leading a team of scientists. But whilst there is a discrepancy between these two versions of events, which has raised question marks against Johnson’s judgement, it is important to note that Aitkenhead did not say that the source was not Russian, as some of the more excitable media commentators have suggested.

I was similarly struck by a Twitter exchange involving the physicist Brian Cox who noted that “we have a generation of senior politicians who were not taught how to think properly - more science in their education would have helped. They use imprecise, woolly language, which is symptomatic of woolly thinking.” Cox was careful not to dismiss the arts and social sciences but was nonetheless inundated with comments accusing him of doing just that, thereby rather proving his point. People may disagree, but what I interpreted Cox as saying was that science demands very high levels of certainty and many people could benefit from understanding what constitutes a reasonable degree of proof before making pronouncements in public.

But perhaps the problem is as much to do with the medium through which many of our news stories are filtered. Take, for example, the way in which the actions of central banks are reported. In August 2013 the Bank of England unveiled a forward guidance strategy based on the unemployment rate. It announced that Bank Rate would not rise from its then-current level of 0.5% until the unemployment rate fell to 7%. This strategy was conditional upon ‘knockouts’ designed to allow for rate hikes if certain threats to inflation became evident.

Although in fact unemployment fell well below 7% over the next twelve months, the Bank did not raise rates for a variety of reasons – domestic inflation was falling whilst the international environment was plagued by euro zone uncertainty and concerns over Chinese events. Nonetheless, many people fell into the trap of arguing that the Bank’s intentions did not match with its actions which rather destroyed its credibility. But it is important to look at exactly what the BoE said: “the MPC intends not to raise Bank Rate from its current level of 0.5% at least until the … headline measure of the unemployment rate has fallen to a threshold of 7%.” This was not a commitment to raise rates once unemployment hit 7% – only a commitment not to do so as long as it remained above the threshold, which is a very different matter.

Despite the best efforts of the BoE to explain the conditional nature of economic forecasts and the risks surrounding the central case projection, the subtleties of this message are often lost in media translation. Thus the mechanistic nature of the initial forward guidance rule was always given more prominence than it deserved. Perhaps the BoE should have been more aware that the issue would be construed in this way, and framing a rule based on the unemployment rate laid it open to more criticism than was necessary. I am not convinced that the BoE did a great job of communicating its message at the time, but it certainly was not helped by some of the reporting surrounding its commentary. 

In his latest speech, MPC member Gertjan Vlieghe suggested that in his view it is “useful to provide a snapshot of how today’s central growth and inflation forecast map into my view of the likely central path of interest rates.” This is exactly the approach adopted by the Swedish Riksbank which sets out an illustrative path for the policy rate conditioned upon its economic forecast. Vlieghe pointed out that “if growth and inflation turn out differently from this central forecast, the path of interest rates will be different too. That should not be seen as a mistake, or a breaking of an earlier promise. It should be seen for what it is, namely an appropriate response to a changed economic outlook.“

Whilst this is totally correct, past UK experience suggests that if the BoE were to adopt such a strategy, a large number of people will misunderstand the nature of conditional versus unconditional forecasts and use this as a stick with which to beat the central bank when it is unable to deliver on its forecast. We all have to choose our words carefully, but it seems that central banks have to do so almost us much as foreign secretaries

How to spot a fake

They say that if you can fake sincerity you've got it made. These days it's fake news we worry about. But it is rare that anyone goes to such great lengths to highlight it as this fascinating Twitter thread I found the other day (also picked up by The Times). A data scientist with the Twitter handle Conspirador Norteño (CN) observed that bot and troll accounts on Twitter often have names that end with 8 random digits. He then took the time to trawl through a series of Twitter accounts, searching for those that referenced #unitetheright and #firemcmaster, both of which are trends followed by those on the Alt-right end of the political spectrum, and found 824 accounts with an 8 digit handle at the end of their user name. Searching their followers for similarly named accounts, and subsequently their followers' followers yielded 63099 accounts. It was (for CN) a simple task to trawl through the followers of these accounts in order to plot the node network. This research yielded the nugget that the largest node in the network belongs to a David Jones based in Southampton.

It then starts to get a little murky thereafter. CN observed that said account posted only between 8am to 8pm Moscow time "almost like it's his job or something". Breaking down the subject matter of the account reveals that this account posted a lot about Ukraine in 2014, then in 2016 moved on to the issues of Brexit and Trump (see chart). Some of the material on the Brexit topic was very inflammatory, particularly with regard to immigration. When it came to the US election, CN pointed out that the language was very similar to that used by adherents of the Alt-right, despite the fact the poster was supposed to be British. CN concludes that this account was "one of the more interesting troll accounts I've seen (and almost certainly human operated and not a bot)". Furthermore, the variety of topics was "aligned with the interests of the Kremlin at the time."

Now we may be maligning David Jones of Southampton unfairly and perhaps he really feels very strongly about the issues at stake. But as one commentator pointed out, if it were a UK based Tweeter their times would vary with the switch to daylight saving in the UK, which would change the time vis-a-vis Russia which has no DST. But they do not, hence accentuating CN's suspicions. This clearly highlights the ease with which it is possible to influence issues of the day by disseminating a particular view and creating a fake network of followers to provide "likes" and recommendations.

This is not to say that the likes of Russia are unduly influencing the democratic process in western economies – at least no more so than usual. Foreign powers have always used propaganda to influence beliefs in other countries. There is evidence to suggest its use as far back as the sixth century BC and it reached new heights during the Second World War, and the Cold War that followed. Even today, the TV channel RT and the BBC World Service provide a view of the world as seen from Moscow and London respectively.  Fake news is not new either: One of the more historically notable events was the publication by the Daily Mail of the infamous Zinoviev Letter in 1924 which purported to be a directive from the head of the international Communist movement, based in Moscow, to the British Communist Party encouraging it to engage in seditious activities. What appeared to be a direct attempt to influence British domestic policy turned out to be a forgery, but it cast a shadow over the Labour Party for decades thereafter, which (unfairly) blamed its heavy election defeat on the letter.

However, the rise of social media has changed the way in which propaganda can be disseminated. For one thing, it is easy to maintain online anonymity which means we can never be 100% sure of the source of the material. Moreover, social media operates on a decentralised basis so that it is straightforward to set up a series of apparently independent channels all feeding the same message. In this way, the message can be drip-fed rather than blasted out.

The impact of fake news on voting patterns is believed to be very small. In one study (Spenkuch and Toniatti 2016[1]) the authors suggest that exposing voters to one additional television campaign ad changes vote shares by approximately 0.02 percentage points. If exposure to one TV ad is as persuasive as one fake news article, each fake tweet influences voting patterns by mere hundredths of a percent. Preaching to the converted will not win more votes, so it does not matter how many times those convinced of a particular view are exposed to fake news because they only have one vote. But the cumulative effect of many thousands of such fake messages will start to mount up if they then influence other voters who otherwise might not be susceptible to such tactics. According to one source (Gottfried and Shearer 2016[2]), 62 percent of US adults get their news from social media and 18 percent do so often, with Facebook the most popular medium. Not everyone will believe the fake news of course, but the dissemination of fake news may have more of an effect than we often credit.

This appears to be a serious problem but I will leave it for others to debate the impact on voting patterns. As an economist, my concern is how such tactics could change the way politicians react to the groundswell of apparent public opinion. If social media is abuzz with reports of how health spending, for example, is scandalously low, do governments react by changing their priorities in order to win votes at the next election? And how would they do so: Do they change their defence budget? Given the low costs associated with fake news dissemination, it is easy to understand why foreign powers with a different world view might try to influence the policies of other governments. But the same applies to domestic interest groups which want to prioritize spending on one area over another. Companies might also apply the same tactics to make the case for a change in the law. If anything this highlights the extent to which all members with a vested interest in our society have a duty to do a little bit of due diligence rather than simply accepting the newsfeeds put before us.

Otherwise, as the often prescient and always quotable HL Mencken put it, "As democracy is perfected, the office of president represents, more and more closely, the inner soul of the people. On some great and glorious day the plain folks of the land will reach their heart's desire at last and the White House will be adorned by a downright moron."

---

[1] Spenkuch, Jörg L., and David Toniatti. 2016 “Political Advertising and Election Outcomes.” CESifo Working Paper Series 5780 
[2] Gottfried, Jeffrey, and Elisa Shearer. 2016. “News Use across Social Media Platforms 2016.” Pew Research Center, May 26. http://www.journalism.org/2016/05/26/news-use-across-social-media-platforms-2016