Introducing the Harding-Hancock Efficiency test

We are going to hear a lot about Test, Trace and Isolate (TTI) in coming weeks, as we approach autumn and fears of a second wave of Covid-19 grow. Now we have moved away from national lockdown but do not yet have a vaccine, the test-and-trace system is our main bulwark against a resurgence of the disease.

But how good a defence is it? A study published in the Lancet Child and Adolescent Health this week suggests there is a huge amount at stake. Academics attempted to model what would happen in various scenarios over the coming winter, assuming schools return either full-time or on a part-time rota basis — and came to the stark conclusion that, depending on how well the tracing scheme operates, we could end up with no second wave at all, or with a second wave that would be more than twice as large as the first outbreak in the UK.

How, then, do we assess the effectiveness of the programme? Today I start to answer this question by launching a first version of my rough-but-ready single measure of the overall effectiveness of TTI. I have named this measure the Harding-Hancock Efficiency (HHE), in honour of Secretary of State Matt Hancock and Baroness Dido Harding, chair of the test-and-trace programme in England. My first estimate of HHE suggests that, despite a great start in getting the test-and-trace system off the ground, there is much improvement needed if it is to contain the virus as we exit further from lockdown.

Test-and-trace, which has worked effectively in countries such as South Korea, seeks to contain the virus by tracing close contacts of those who have tested positive for Covid, and then isolating these contacts so as to prevent them spreading the virus further. From a standing start, the system in England has now tested 2.7 million people, with a team of around 27,000 tracers assembled, and almost 223,000 people identified as coming into close contact with someone who has tested positive. It’s pretty impressive to have built up this system in less than three months, and the programme is still developing further measures, such as working out the optimal time of day to call close contacts. Yet the worrying news is that, according to my analysis, the present system is still a long way from what we need.

I’ve created HHE using data from a variety of sources, including the Weekly NHS Test and Trace bulletin and the Office for National Statistics Coronavirus Infection Survey, and some extrapolation from survey work by UCL on compliance. It should be noted that the required data is not available in any one report, and indeed some of the data needed to assess performance of test-and-trace, such as estimates of isolation compliance or close contacts missed, is not currently available at all.

Harding-Hancock Efficiency is defined as the proportion of people who are successfully isolated compared with a perfect outcome. A perfect efficiency would involve a testing programme that finds everyone recently infected with Covid, identifies all their close contacts, reaches all of these contacts and gets them to isolate. And it does all of this in less than the four days before the close contacts could become contagious. If we managed this, we would have HHE of 100 per cent.

Now that is clearly impossible — but the HHE shows how close to perfection a test-and-trace system gets. Say that we manage to find 70 per cent of people infected across the country in our test programme, to identify 80 per cent of their close contacts, and to contact 90 per cent of these close contacts, all of whom we persuade to isolate. Put these together and we would be managing to isolate around 50 per cent of those we should isolate in a perfect scenario (the maths is 70 per cent times 80 per cent times 90 per cent). So, our HHE would be 50 per cent.

But my initial estimate is that at the moment, Harding-Hancock Efficiency is at a potentially catastrophic level of less than 5 per cent for England – see table below. That means that for every person successfully isolated, there are around 20 not isolated, potentially spreading the infection. Clearly that will not be enough to prevent a second peak.

So what are the major performance issues in our system? There are lots of little gaps, and one big one.

The big gap is that the testing is not finding nearly enough of the people who have been recently infected with Covid-19. The best estimate of how many are getting infected in England comes from the ONS Coronavirus Infection Survey, which tests individuals for Covid in a representative sample of households. During 13 to 19 July, the same week for which we have TTI data, the ONS estimates there were around 20,000 new cases in the community (excluding hospitals and care homes) in England — albeit the ONS estimates have a wide 95 per cent credible interval of 10,500 to 38,500. Yet the test-and-trace operation found only around 3,500 cases in the community in a week. In other words, around 16,000 infected people are not being tested — perhaps because they might not know how to access testing, or they might be asymptomatic and so not realise they are infected — or, of course, the ONS central estimate of infection might be wrong. Increasing testing to 500,000 per day as planned will help close this gap, but might still not be sufficient, given we are finding less than a fifth of infected individuals with the current 240,000 tests per day.

As for the little gaps which my analysis picked up, these include those who tested positive but were never reached (around 20 per cent), those contacts who were never traced (around 25 per cent), and those for whom the process took too long so that they were spreading the disease before they isolated (this could be as high as 30 per cent).

As Peter Drucker, the management guru, said: ‘If you can’t measure it, you can’t improve it.’ That’s why I will continue to refine and publish my metric. I will also try to estimate how good HHE needs to be to prevent a second spike, and try to compare performance on HHE across the devolved administrations in other countries.