Can we trust Covid modelling? More evidence from Sweden

At last we’re getting a debate about Covid-19 modelling. When people finally got to look under the hood of the famous Imperial College study, they found twisted and tangled code. And most of the model’s predictions bear little resemblance to what is actually happening. Some defend the models by saying that their predictions turned out to be wrong only because governments imposed harsher restrictions than the coders expected.

If so, we have a perfect experiment. Sweden did not close borders, shut down schools, businesses, restaurants, gyms or shopping centres and did not issue stay at home orders. So it should be the one country where the models fit. Let’s see.

Maria Gunther and Maria Westholm at Dagens Nyheter, Sweden’s biggest daily, just took a look at two of the most influential models in Sweden, both were inspired by the Imperial College study and published on the preprint server MedRxiv in April. Both were used by critics to argue that the Swedish model would quickly break our health care system - and that we had to make a U-turn into lockdown, as Britain did.

Here are the models’ prediction of the number of Covid-19 patients in Swedish intensive care units, ICU (the highest curve is a model without lockdown):

H. Sjödin et al: ‘Covid-19 health care demand and mortality in Sweden in response to non-pharmaceutical (NPIs) mitigation and suppression scenarios’, 7 April. The graph suggests critical care demand would peak above 16,000 patients per day by early May, and pre-pandemic intensive care unit capacity would be exceeded 30-fold.

Then came J. Gardner et al, ‘Intervention strategies against Covid-19 and their estimated impact on Swedish healthcare capacity’, 15 April.  It was an even more pessimistic assessment, showing a peak of over 20,000 patients by early May - with an ICU requirement around 40 times the actual capacity.

Sweden’s Public Health Agency rejected the models. It instead planned for a worst-case scenario that was much less pessimistic, suggesting a peak around 1,700 ICU patients in the middle of May. Still more than three times more than the pre-pandemic capacity. Sweden, almost alone in the world, refused to lock down. And here is how things eventually worked out.

The number of patients in ICU has been fairly stable around 500-550 since mid-April. This means that capacity was never exceeded. At this moment, when the models suggested that Sweden would have 30 to 40 patients fighting over every available ICU bed, there is spare capacity in beds, equipment and personnel of around 30 percent (partly as a result of a doubling of the pre-pandemic capacity).

Gardner et al predicted that Sweden would have 82,000 Covid-19 deaths by 1 July. That implies around 1,000 deaths every day since the paper was published in mid-April. However, the total number of Swedish Covid-19 deaths at the time of writing is 3,313.

One reason why the models failed is that they – just like most countries’ politicians – underestimated how millions of people spontaneously adapt to new circumstances. They only thought in terms of lockdowns vs business as usual, but failed to consider a third option: that people engage in social distancing voluntarily when they realise lives are at stake and when authorities recommend them to do so.

And obviously, there is an argument that these models scared us into changing our behaviour and ramping up capacity, and so helped us to avoid a disaster. But they were also clearly based on faulty assumptions that would always result in absurd predictions. We know this, because both models actually assumed that it was already too late, and estimated that ICU capacity would be exceeded by around 10 times even if Sweden switched to strong mitigation.

The need for ICU beds in Sweden will be ‘at least 10-fold greater [than capacity] if strategies approximating the most stringent in Europe are introduced by 10 April’, wrote Gardner et al.

Those strategies were never introduced in Sweden, and yet, additional ICU capacity is 30 percent and the number of patients in intensive care has been declining for two weeks. The newly constructed field hospital in Stockholm, with room for hundreds of patients, has still not received any patients. It will probably never have to open. Here's a zoomed-in graph of eventual ICU: numbering in the hundreds, not the predicted thousands.

In Britain, it’s argued that if the number of Covid deaths is far lower than the models predicted, well, this is the result of lockdown! Some even argue that the difference between the predicted and the actual is the lockdown effect. But you can't say that for Sweden. We banned gatherings of more than 50 people, but that was about it. All other behavioural change was voluntary: something the models did not properly understand.

As countries plan how to leave lockdown, they can look at Sweden and ask: what happens if you don't involve the police, if you don't issue edicts about how many of your relatives or neighbours you can visit, and just ask people to be careful? Might that work? The Swedish experiment casts huge doubts on the models, and makes the case for trusting the public.