Covid-19: Forecasting and Modelling

I beg to move,

That this House has considered forecasting and modelling during covid-19.

It is a pleasure to speak under your chairmanship, Sir Edward. I speak not to bury science, but to praise it. During the covid pandemic, there has been some remarkable, wonderful science; I just question to what extent that includes the modelling and forecasts that have come from it. Thanks to some questionable modelling that was poorly presented and often misrepresented, never before has so much harm been done to so many by so few based on so little questionable and potentially flawed data.

I believe that the use of modelling is pretty much getting to be a national scandal. That is not just the fault of the modellers; it is how their work was interpreted by public health officials and the media—and yes, by politicians, including the Government, sadly. Modelling and forecasts were the ammunition that drove lockdown and created a climate of manipulated fear. I believe that that creation of fear was pretty despicable and unforgivable. I do not doubt that modelling is important or that there has been some good modelling, but too often it has been drowned out by hysterical forecasts. I am not, as Professor Ferguson implied, one of those with an “axe to grind”. I do, however, care about truth and believe that if someone influences policy, as the modellers and Imperial College London have done, they should be questioned. Frankly, they have not been questioned enough.

Above all, I want to understand why Government, parts of the media and the public health establishment became addicted to these doomsday scenarios, and then normalised them in our country with such depressing and upsetting consequences for many. I do not pretend to be an expert; I am not. I defended my own PhD at the end of last year, but it is not in epidemiology and I do not pretend to be particularly knowledgeable about that. But depending on time—I know others want to come in as well—I will quote from 13 academic papers and 22 articles authored by a total of approximately 100 academics.

This is a story of three scandals, and the first one took place 21 years ago. In 2001, we faced the foot and mouth emergency. We reacted drastically by slaughtering and burning millions of animals, and farmer suicides and bankruptcies followed. That policy was allegedly heavily influenced by Imperial College modelling and Professor Ferguson. Since foot and mouth, two peer-reviewed studies examined the method behind that particular madness. I quote from them now to show there are practical and ethical questions over modelling going back two decades.

In a 2006 paper, and I apologise for these wordy, long titles, titled “Use and abuse of mathematical models: an illustration from the 2001 foot and mouth disease epidemic in the United Kingdom”—they are not that catchy—the authors confirmed that Ferguson’s model

“probably had the most influence on early policy decisions”

and

“specifically, the introduction of the pre-emptive contiguous culling policy”.

That is the mass slaughter of animals near infected farms. The authors said that the consequences were “severe” and

“the models were not fit for the purpose of predicting the course of the epidemic”

—not a good start. They remain “unvalidated”. Their use was “imprudent” and amounted to

“the abuse of predictive models”.

Devastatingly, the authors wrote

“The UK experience provides a salutary warning of how models can be abused in the interests of scientific opportunism.”

It is difficult to find a more damning criticism of one group of scientists by another.

A 2011 paper, “Destructive tension: mathematics versus experience—the progress and control of the 2001 foot and mouth disease epidemic in Great Britain”—bit of a mouthful—by four academics said the models that supported the culling policy were “severely flawed” and based on flawed data with “highly improbable biological assumptions”. The models were

“at best, crude estimations that could not differentiate risk”.

That is not a very good “at best”. At worst, they were “inaccurate representations”.

Sadly, the paper said, impatience for results

“provided the opportunity for self-styled ‘experts’, including some veterinarians, biologists and mathematicians, to publicise unproven novel options.”

Some of the scientific work—some of it modelling, some of it not, with some modelling by Ferguson and some not—was cited as “unvalidated” and “severely flawed”, with inaccurate data on “highly improbable assumptions” leading to “scientific opportunism”. Is anybody reminded of anything more recent that would suggest the same?

I scroll forward 20 years. As with foot and mouth, with covid we had a nervous Government presented with doomsday scenarios by Imperial—the 500,000 dead prediction—that panicked them into a course of profound action with shocking outcomes. After the lockdown had gone ahead, Imperial publicised on 8 June a study by, I think, seven of them arguing the justification for lockdown. It claimed that non-pharmaceutical interventions saved more than 3 million lives in Europe. Effectively, Imperial marked its own homework and gave itself a big slap on the back.

That work is now being challenged. Because of time, I will quote only a small selection. In a paper entitled, “The effect of interventions on COVID-19”, 13 Swedish academics—Ferguson ain’t popular in Sweden, I can tell Members that much—said that the conclusions of the Imperial study were not justified and went beyond the data. Regensburg and Leibniz university academics directly refuted Imperial College in a paper entitled “The illusory effects of non-pharmaceutical interventions on COVID-19 in Europe”, which said that the authors of the Imperial study

“allege that non-pharmaceutical interventions imposed by 11 European countries saved millions of lives. We show that their methods involve circular reasoning. The purported effects are pure artefacts, which contradict the data. Moreover, we demonstrate that the United Kingdom’s lockdown was both superfluous and ineffective.”

I am not saying that this stuff is right; I am just saying that there is a growing body of work that is, frankly, taking apart Imperial’s. Remember, we spent £370 billion on lockdown that we will never get back. I could continue with other quotes, but I think Members get the flavour.

Moreover, a substantial number of other papers now question not Imperial per se but the worth generally of lockdowns. A pre-print article by four authors, “Effects of non-pharmaceutical interventions on COVID-19: A Tale of Three Models”, said:

“Claimed benefits of lockdown appear grossly exaggerated.”

In another paper, three authors found no clear, significant benefit of lockdowns on case growth in any country. Other papers continue that theme. I will quote one more, on adults living with kids. Remember: we shut schools because we were scared that kids would come home and infect older people, who would then die. This paper, in The BMJ, found

“no evidence of an increased risk of severe COVID-19 outcomes.”

We shut down society and schools just in case, doing extraordinary harm to people’s lives, especially young people. I am not a lockdown sceptic, as Ferguson casually describes some of his critics, but I am becoming so. Do you know why, Sir Edward? Because I read the evidence, and there is a growing body of it. In fact, there is one quote that I did not read out. There was a study of lots of countries that had lockdowns and lots that did not, and the data was inconclusive.

The third element of the scandal is the recent modelling. Swedish epidemiologists looked at Imperial’s work and compared it with their own experience. Chief epidemiologist Anders Tegnell said of Imperial’s work that

“the variables…were quite extreme…We were always quite doubtful”.

Former chief epidemiologist Johan Giesecke said Ferguson’s model was “almost hysterical”. In the House of Lords, Viscount Ridley talked of a huge discrepancy and flaws in the model and the modelling. John Ioannidis from Stanford University said that the “assumptions and estimates” seemed “substantially inflated”.

There was a second example last summer. In July 2021, the good Professor Ferguson predicted that hitting 100,000 cases was “almost inevitable”. He told the BBC that the real question was whether we got to double that or even higher. That is where the crystal ball starts to fail: we got nowhere near 200,000, and we got nowhere near 100,000. There was nothing inevitable about Professor Ferguson’s inevitability, and his crystal ball must have gone missing from the start. In The Times, he blamed the Euros for messing up his modelling because—shock horror—people went to pubs a lot to watch the games during the competition. When the tournament finished—shock horror—they did not. That seems to be the fundamental problem: where reality comes up against models, reality steamrollers them because models cannot cope with the complexity of real life. To pretend that they can and even that they are useful, when so many of them have proved not to be, is concerning.

Ferguson is only one of many people in Independent SAGE especially, but also SAGE, who did not cover themselves in glory. Raghib Ali—a friend of my hon. Friend the Member for Wycombe (Mr Baker), who I am delighted is present—is one of the heroes of covid. He noted that many left-wing SAGE members

“repeatedly made inaccurate forecasts overestimating infections”.

Very often, they were falsely described on the BBC.

I am grateful to my hon. Friend for mentioning my friend and constituent Raghib Ali, who has indeed been one of the absolute heroes of this pandemic—not only in his advice to us all, including online, but through his service in hospitals. I hope my hon. Friend will not mind my saying that I do not think any of us can speak for Raghib about his opinion of modelling, and I know my hon. Friend is not trying to.

I quite agree, and I thank my hon. Friend for that, but I am deeply grateful to Raghib and other people for speaking out. Just for the record, the communist Susan Michie, who is quoted quite often by the BBC, is not a medical doctor, a virologist or an epidemiologist. She is a health psychologist, so why on earth is she talking about epidemiology?

The third scandal took place this winter. Imperial, the London School of Hygiene and Tropical Medicine and others—I think they included Warwick—predicted 5,000 daily covid deaths, with 3,000 daily deaths as the best-case scenario. They were hopelessly inaccurate, and at this point the tide really begins to turn. Dr Clive Dix, a former vaccine taskforce head, said:

“It’s bad science, and I think they’re being irresponsible. They have a duty to reflect the true risks, but this just headline grabbing.”

As I say, the tide is turning. Oncology professor Angus Dalgleish describes Ferguson’s modelling as “lurid predictions” and “spectacularly wrong”. The great Carl Heneghan, another scientist who has emerged with great credit for his honesty and fairness of comment, says:

“it’s becoming clearer all that ministers see is the worst-case scenario”.

Professor Brendan Wren says:

“Dodgy data and flawed forecasts have become the hallmarks of much of the scientific establishment 2”—

what a damning quote!—

“which has traded almost exclusively in worst-case scenarios...this must stop now.”

I agree.

I will wind up in the next two to three minutes—I will speak for no longer than 15 minutes because other people wish to get in, and I am always mindful of that. What is the result of all this? The result, as UCL’s Professor Francois Balloux notes, is a

Order. To get everybody in, I will now have to ask for a five-minute limit for speeches.

Thank you very much, Sir Edward. I begin by referring to the declarations that I have made in connection to the Covid Recovery Group.

I am a professional aerospace and software engineer—at least I was in my former life. I have an MSc in computer science, and am very interested in models. However, there is an old joke among engineers, which derives from a “Dilbert” cartoon, that the career goal of every engineer is not to be blamed for a major catastrophe. I wonder whether that spirit infuses not only expert advice but modelling in particular. We are all indebted to The Spectator for its data hub, which shows how data has worked out against models. As anyone can see by going to data.spectator.co.uk, it is the same story again and again: enormous great molehills of death projections, and underneath them the reality of much lower lines. I will leave it to people to look for themselves at the data, rather than trying to characterise the curves for Hansard.

There is a great deal to be done in terms of institutional reform of the way in which modelling is done and informs public policy. That is a very old problem; I found a great article in Foreign Affairs that goes back a long time, to the post-war era, about this problem. It is time we did something about it, through institutional reform. The situation is now perfectly plain: under the Public Health (Control of Disease) Act 1984, even our most basic liberties can be taken away with a stroke of a pen if a Minister has been shown sufficiently persuasive modelling—not even data—that tells them that there is trouble ahead.

I have put this on the record before, and I hope that my right hon. Friend the Prime Minister will not mind. Before we went into the 2020 lockdown, he called me; I was amazed to be at home and to have the Prime Minister of the UK call me. “Steve, I have been shown death projections—4,500 people a day and the hospitals overwhelmed.” I gave him two pieces of advice: “First, if you really believe that we are going to have 4,500 people a day die, you’d better do whatever it takes to prevent that from happening,” which is not advice that anyone would have expected me to give, but that is what I said, and, “Secondly, for goodness’ sake, go and challenge the advice—the data.”

Where does my hon. Friend believe that body should sit: within BEIS or a separate scientific institution?

I think it should be in the Cabinet Office, because we see that scientific advice applies right across Government.

The code quality of the model that was released was really not fit for a hobbyist. The irony is that the universities that do modelling will overwhelmingly have computer science departments. For goodness’ sake, I say to modellers, go and talk to software engineers and produce good quality code. For goodness’ sake, stop using C++. People are using, as they so often do, the fastest computer programming language, but also the most sophisticated and dangerous. As a professional software engineer, the first thing I would say is, “Don’t use C++ if you don’t have to. Models don’t need to; they can run a bit slower. Use something where you can’t make the desperately poor quality coding errors that were in that released model code”. That is really inexcusable and fulfils all the prejudices of software engineers against scientists hacking out poor quality code not fit for hobbyists. As I think people can tell, I feel quite strongly about that, precisely because these poor modellers have had unacceptable burdens placed on them. All the incentives for them to be pessimistic can now be seen in the data. This all has to be changed with an office of research integrity.

I will try to be brief, Sir Edward. The hon. Member for Isle of Wight (Bob Seely) has raised a first-class, crucial issue. Clearly we cannot predict the future, but we can prepare for it. Traditional crisis management and risk management list the possible things that might happen and look at the severity and the likelihood, and based on that we produce a number of models. That is the old way of doing it.

The modern way of doing it, instead of creating models, is to create a playbook—a mechanism by which we can look at all the challenges that might face a country and at least put in place a mechanism for dealing with the crisis, whatever it is. Bizarrely, the US have taken that approach but, perhaps rather sadly and tragically, never used the playbook model that might have saved them.

We very much went down the model route and took out of our kit bag the one we had for flu. It was too rigid, did not fit and was too slow. The assumptions were not challenged; the real world evidence was not incorporated; and, worse, we limited how we looked at modelling absolutely to covid. We looked at the mortality of covid and the impact; we looked at the morbidity of covid and the impact, but we should have looked instead and as well at the impact of the crisis as a whole and the proposed solutions, including the lockdown and other restrictive measures, across the country, across society and across the economy.

We were told at the time—the Government were challenged on this—that it was too hard: we could not possibly do any modelling with regard to the mortality of lockdown and restrictions, the morbidity of lockdown and restrictions, or the economic impact. That was sadly relegated to second order, but we should never forget that there is a huge interaction between wealth and health.

The interconnectedness of the impacts of the steps that were taken was totally ignored. They were unexplored and unexplained for a good 12 months. So only today, as my hon. Friend the Member for Isle of Wight has articulated so clearly and eloquently, are we looking retrospectively at what happened. At last we are looking at the impact, not just on covid patients and those vulnerable to covid, but on those patients not subject to high risk for covid. We are looking at the impact on children’s life chances—not just on their education—as it is far more serious than just their education. We are beginning to look at the impact on society and communities and—at last—the economy. As my hon. Friend said, never again must we be faced with the question, “Did we unknowingly and unintentionally do more harm than good?”

I congratulate my hon. Friend the Member for Isle of Wight (Bob Seely) on securing this very important debate and making an excellent speech. I have no wish to repeat the brilliant research that he recited, but he did highlight the repeated failures of modelling throughout the pandemic, not just the modelling but how it is being used. The models have not been out by just a few per cent, as he said, but often by orders of magnitude. The way that the models have been used has had life-changing impacts on people across the country.

Before I was a politician, I was a science teacher. One of the joys of teaching science to teenagers is conducting practical experiments in the lab. Once the teacher has ensured that they are not going to burn down the lab, it is important to teach them how to conduct an experiment properly and write it up. The first thing is to create a hypothesis. They must write a statement of what they think will happen and why, using the scientific knowledge they have and some assumptions, then carry out the experiment, write up the research and, crucially, evaluate. They must look at the hypothesis and at what they have observed, and decide whether they match. If they do match, they go back to their assumptions and see why they were correct. If they do not match, if what has happened in the lab and been recorded does not match the hypothesis, they need to ask why—“What assumptions did I make that did not bear out in real life, that did not happen in the lab?”

It seems to me that those are the questions that have not been asked throughout this crisis. Perhaps we can understand why assumptions had to be made quickly the first time, for the first lockdown—assumptions that turned out not to be true. My hon. Friend said that perhaps we are repeating history of 20 years ago, and that there is not that excuse. However, during subsequent waves and restrictions, why were those assumptions not questioned? There were assumptions about how likely the different scenarios were, about people’s behaviour and fatality rates.

My hon. Friend spoke earlier about the repeatability of scientific experiments with hypotheses. One of the reasons I talked about C++ is that by using multithreading, it is possible to end up with code that does not produce repeatable outputs. Does she agree that it is very important that when models are run, they produce consistent and coherent outputs that can be repeated?

I absolutely agree with my hon. Friend. I would have said to my students, “It is not a valid experiment if you cannot follow the same method, repeat the experiment and produce the same results. It is completely invalid if you cannot do that.” I am not a software engineer, so I take my hon. Friend’s word for it when it comes to the use of programming languages, but he is absolutely correct that the whole experiment is not valid if the results cannot be repeated.

Over just the past few months, there has been an opening up of debate that has moved from The Spectator into mainstream media, where people such as my hon. Friends present have been able to speak more freely about the problems and costs of lockdown, and have not suffered so much criticism—I hesitate to say “abuse”—in the media and on social media. To avoid this happening again, we need institutional change, but we also need to understand that these are not black-and-white issues. It is good, right and wise to question the data and the science, and to put just as much weight on people’s quality of life—the things that make life living—as on the number of people in hospital at one time for a particular disease.

It is a pleasure to see you in the Chair, Sir Edward, and to follow all my hon. Friends, who I note have usually been in a different Lobby from me on most coronavirus measures. I am sure the Minister will be grateful to have somebody speaking from the Government Benches who has been supporting the Government on coronavirus throughout.

However, I too have issues with modelling, which is why I chose to speak in today’s debate. I have more sympathy with modelling, and I will be offering some sort of partial defence and explanation of it in my remarks, because before I was an MP, I was a modeller myself—a software engineer. I wrote in Visual Basic.NET, which is nice and simple: engineers can see what the code does. I worked for bet365, and I used to write models that worked out the chance of somebody winning a tennis match, a team winning a baseball game, or whatever. I had some advantages that Neil Ferguson and these models do not have, in that there are many tennis matches, and I could repeat the model again and again and calibrate it. If I got my model wrong, there were people out there who would tell me that it was wrong by beating me and winning money off me, so my models got better and better.

The problem we have with covid is that we cannot repeat that exercise—there is no counterfactual. We have heard the phrase “marking your own homework”.