Using OpenSAFELY to carry out a randomised trial

Written by Ben Ainsworth, Associate Professor, Digital Interventions Group, University of Southampton

During the COVID-19 pandemic, it became clear that supporting people to change their behaviours was going to be crucial in reducing the spread of the virus. Simple things like wearing masks, opening windows and washing hands would help reduce the impact of COVID, buying the NHS valuable time while a vaccine was developed and tested.

During the earlier H1N1 pandemic in 2009, we had developed a web app, Germ Defence, which encouraged people to improve hygiene practices at home. Between 2010 and 2013 this was tested in a large randomised controlled trial of over 20,000 people, and the test showed that people who used Germ Defence reduced their chances of developing a cold or respiratory infection by 14%.

So when the COVID-19 pandemic struck, we knew that some relatively minor updates to Germ Defence would be likely to make it an effective tool against the COVID virus. These were updates around the technology, but also to the behavioural advice.

After making those changes, we worked to disseminate Germ Defence via any mechanism we could. However, as with any medical intervention, we also recognised the need to evaluate it. We wanted to see whether GP practices that asked their patients to use Germ Defence reduced the amount of patients who were diagnosed with COVID-19 (and other infections), as well as other healthcare outcomes.

However, there are challenges to evaluating interventions that are already being used. The evaluation needs to happen quickly (so that it can inform ongoing intervention) and it needs to be unobtrusive – so that it doesn’t interfere with the healthcare that is already being delivered.

We were able to do this using OpenSAFELY. We designed the largest clinical randomised control trial to have ever been conducted (that we know of!). In Autumn 2020, we randomised every single GP practice in England to text their patients a link to Germ Defence (3,297 practices), or continue usual practice until Spring 2021 (3,282 practices). This gave a maximum sample size of every NHS patient in England!

We designed a trial that avoided many of the usual burdens that are placed upon already overstretched GP practices taking part in research. OpenSAFELY meant we could analyse data for 1,246 intervention practices and 1,252 control practices, representing 11.9 million and 12.3 million registered patients.

In the trial, only 460 practices sent Germ Defence on to their patients (16%), which was far below what we had hoped, and we didn’t see any reduction in infection rates. However, we were able to link patient website usage with infection rates at a practice level. OpenSAFELY allowed us to evaluate a large-scale public health intervention in real time, while it was actually being used.

We were able to deliver gold-standard randomised control trial results in a short enough time frame to inform current practice – in a fraction of the expected time, at a fraction of the expected cost, and with far fewer people working on it than expected. In the future, this method can be used for evaluating a huge range of new interventions – technological, social, behavioural, pharmaceutical and organisational.