On Sage’s Covid models

In the confusion that has arisen from the demonstrable inability of a certain type of mathematical model to predict the time course of the SARS-CoV-2 pandemic, many have taken to reciting – with a variable mixture of glee and sympathy – that ‘all models are wrong but some are useful’.

I have never been comfortable with this statement (it doesn’t really deserve to be called an aphorism) and even less so the smugness with which it is typically pronounced. One might as well say all metaphors are wrong but some are useful.

‘Wrong’ seems to be employed here to suggest that ‘statistical or scientific models always fall short of the complexities of reality (but can still be useful nonetheless)’. But a model, by definition, does not strive to recapitulate reality – instead, it replaces it with an abstraction which advances our comprehension of a system in a way that cannot be achieved simply by spectating upon it or by painstakingly cataloguing its complexities. In the words of Georgia O'Keeffe: ‘Nothing is less real than realism. Details are confusing. It is only by selection, by elimination, by emphasis, that we get at the real meaning of things.’

The level of detail a model can possess before becoming more confusing than reality itself is a matter of debate but the reason why the Sage and other models failed were not due to an excess of realism compared to other efforts, like the Danish modellers who were more successful in their predictions. Neither was it a matter of badly executed code as many have suggested. It is a fallacy to assume that their polishing code or updating their parameters would have allowed them to arrive at the correct answer. This is not so much a case of poor workmanship as an adherence to incorrect assumptions.

The observed dynamics of SARS-CoV-2 can be easily explained by assuming that the virus arrived at different times within the seasonal cycle of changes in viral transmissibility in different regions, and – as is the case with seasonal coronaviruses – that immunity from natural exposure does not durably block infection, even though protection from severe disease can be lifelong. Within such a framework, the acute phase of the pandemic would have occurred prior to March 2020 (leading to the wave of deaths in April 2020) in the UK and sufficient immunity would have accumulated in the population to keep infections at low levels over the summer. The general rule of thumb here is that infections only start to increase when the proportion immune falls below the herd immunity threshold. The latter is determined by the transmissibility of the virus at the time of year and therefore fluctuates with the seasons, typically being low in the summer. As anticipated by many of us who were then scrambling to put together a strategy to deal with this eventuality, a second wave occurred in the autumn when the proportion immune again fell below the herd immunity threshold. This happened as a consequence of both the waning of infection-blocking immunity and a seasonal increase in the herd immunity threshold itself. The current wave of the Omicron variant presents a similar increase in infections that might be expected of an endemic pathogen during the high transmissibility season.

The alternative explanation put forward by Sage and their satellites was that these patterns arose as a result of lockdowns and other non-pharmaceutical interventions rather than herd immunity. Under this scenario, the lifting of restrictions should have caused a very large increase in infections. That this did not happen allows us to dismiss the fundamental assumption that NPIs had a large effect. It does not mean that those who created these models are all crooked charlatans or that mathematical modelling is a fundamentally useless exercise. It just means their assumptions regarding the role of restrictions were incorrect.

But isn’t it just the case that the virus has evolved into a milder but more transmissible form? We must be very careful about making such an assumption, not least because it suggests that how we handle the situation now should be different from how we handled it in the first place. While some very elegant studies have shown that the Omicron variant has some clear functional differences from its antecedents, there is no reason to believe it is intrinsically less virulent or more transmissible. The idea that all viruses evolve in this direction is entirely incorrect. It would be most unreasonable to expect Sage modellers (most of whom are well acquainted with the fundaments of the evolution of virulence) to recalibrate their models on the basis that Omicron was intrinsically less capable of causing severe disease and death.

Instead, they assumed – correctly, in my opinion – that the reason why Omicron was so ‘mild’ in South Africa was because of high levels of previous exposure. Their predictions thus make complete sense under the assumption that the levels of natural exposure in the UK were low. It is this assumption that was likely flawed rather than any dereliction of duty on their part to ‘update’ their model to reflect that Omicron was intrinsically less severe.

The Omicron variant is capable of significant immune evasion – this gives it an edge over Delta in re-infecting those who have either been previously exposed or vaccinated. This is why it has replaced Delta. We see this happening all the time with influenza. It does not require any increase in transmissibility. There is no reason, therefore, to believe that Omicron is fundamentally more transmissible than any of the other variants. What we are witnessing now in terms of the increase in infection rates is very likely to reflect how rapidly the virus was spreading two years ago in many parts of the world before we had the means to properly document it.

However, because by now most of what we are seeing are re-infections or vaccine breakthroughs, the fatality rate is dramatically lower. The sad truth is that we did not need to wait for that to happen before lifting the restrictions that have caused so much damage to so many. The best course of action was always to protect the vulnerable during the periods of danger (for which anonymous surveillance is essential, as is the testing of those who are planning to come into contact with vulnerable individuals) and let the accumulation of immunity in the population steer us towards the endemic equilibrium we are now beginning to accept as the natural endpoint of the pandemic.