Comparative Effectiveness of Pfizer and Moderna COVID-19 Vaccines for Boosting

Our OpenSAFELY study comparing the Pfizer and Moderna vaccines for protecting against covid-19 during the first booster programme in England is now published open-access in the BMJ. The study is part of an ongoing collaboration between the Bennett Institute, University of Bristol, London School of Hygiene and Tropical Medicine, and Harvard University, looking into COVID-19 vaccine effectiveness using the OpenSAFELY platform. This blog post describes the motivation for the study, what we did, and what we found.

Why did we do the study?

While there is a lot of evidence that getting a booster dose will substantially reduce the chances of being hospitalised or dying because of Covid-19 (at least in the short term for most people), there is little research that directly compares different types of vaccines to assess if one is better than another.

The Covboost trial compared 7 booster vaccines, and one placebo, and found differences in the body’s immunological response. However, as a phase II trial its sample size was too small to properly compare rates of rarer patient-centred outcomes, such as hospitalisation with severe COVID-19 disease. A randomised trial recruiting huge numbers of people would be needed to reliably identify a meaningful difference in such rare events. This is particularly true in the boosted population where the primary vaccine course provides a higher baseline level of protection than people who are completely unvaccinated.

Fortunately, England’s electronic health record infrastructure routinely captures information on who’s getting which vaccine when and where, and what happened to them afterwards. The OpenSAFELY research platform, run by the Bennett Institute, enables secure analysis of such data for millions of people without researchers ever needing to directly access and view it.

Using observational data like this to compare treatments is fraught with difficulties, such as confounding and selection bias. However, as the UK booster programme used both Pfizer and Moderna vaccines at the same time and in the same population, there was an element of chance about which type of booster vaccine a person would receive on any given day. This provided an opportunity to compare vaccines perhaps more reliably than most, as we could mitigate confounding in time and space.

All we needed was the right study design.

What did we do?

We compared the effectiveness of the Pfizer and Moderna COVID-19 vaccines during the 2021/22 booster vaccination programme in England, using the OpenSAFELY-TPP database, covering around 40% of English residents. The primary outcomes were rates of positive SARS-CoV-2 tests, hospital admissions related to COVID-19, and deaths relating to COVID-19.

Who did we include?

We started by considering all adults who received a third dose of Pfizer or Moderna from 29 October 2021 until 28 February 2022 (around 10.4 million people). This was a period where both Moderna and Pfizer were being administered concurrently in the booster roll-out.

We then applied a few exclusions to this long-list of potential participants based on data availability and other factors that helped us to make the two treatment groups as similar as possible.

For example, most people received two doses of Pfizer or two doses of Astra-Zeneca as their primary vaccine course, with only very few people receiving Moderna or a mixed course. Since the immunological response to the third dose, and therefore effectiveness, is likely to be affected by the type of vaccine received as the primary course, we wanted to make sure this characteristic was accounted for in our comparisons between treatment groups. We only had the numbers to do this properly for the people who got Pfizer-Pfizer and AZ-AZ, so we excluded the rest. This left us with 6.7 million Moderna recipients and 2.7 million Pfizer recipients.

How did we make the comparisons fair?

We matched each Moderna recipient to a Pfizer recipient, such that important characteristics potentially influencing COVID-related outcomes were identical or very similar for each matched pair.

We matched on date of vaccination, type of vaccine received for first and second dose, time since second dose, age, sex, region, and other clinical characteristics that were likely to influence the risk of being infected and subsequent risk of severe disease.

Anyone who could not be matched was excluded, leaving us with around 1.6 million recipients in each group, 3.2 million in total.

This may seem like a big drop in our original long-list of 10.4 million booster recipients, but this helped to ensure that we were comparing the two vaccines as fairly as possible.

Number of booster vaccine recipients available for matching on each day

What did we find?

Our main comparison point was 20 weeks after receiving the booster dose. The rate of positive tests after 20 weeks was 164 per 1,000 people for those who had Pfizer and 160 for Moderna. This means that there were around 4 fewer (95% confidence interval is 3.0 to 5.5) positive tests per 1,000 people for Moderna than for Pfizer after 20 weeks. For COVID-19 hospital admissions the rates were 0.75 for Pfizer and 0.65 for Moderna, so around 0.10 fewer (95% CI 0.05 to 0.16) events per 1000 people for Moderna than for Pfizer. The COVID-19 death rate after 20 weeks was 0.028 for Pfizer and 0.024 for Moderna, so just 0.004 fewer (95% CI 0.015 fewer to 0.007 more) deaths per 1000 people.

The figure below shows the cumulative rates of these outcomes over time for each vaccine. The differences are small, but potentially important at a population level.

Cumulative incidence per 1,000 people for each outcome, up to 28 weeks after boosting

How reliable are the results?

Observational studies will always carry some risk of residual confounding. In this study, we expect this to be small, for a few reasons.

First, whether a person received Pfizer or Moderna was based largely on local supply and availability, and clinical characteristics did not influence this at the point of vaccination.

Second, although we couldn’t match on all important characteristics (it gets harder to find matching pairs when you have more restrictive matching criteria), we could still check whether any unmatched characteristics were similar between the two groups. When we did this we didn’t find anything to suggest that there were large residual differences between vaccine groups but, of course, we couldn’t measure everything.

Third, since it takes a few days for vaccination to improve protection against infection and disease (the body’s immunological response is not immediate), we shouldn’t expect to see any differences in the first week after vaccination. If we did see early differences these would be unlikely to be caused by the vaccine itself (except maybe due to side-effects though these are very rare), suggesting that the groups were not adequately balanced by the matching procedure. As we didn’t see any early differences between vaccines in our study, this gives us further confidence that we were making reasonably fair comparisons.

COVID-19 hospital admission was the most reliable outcome we studied. It’s a well-defined event that accords fairly closely to clinical status, unlike positive tests which significantly under estimate actual infections because so many people don’t get tested; hospital admissions also happened frequently enough that we could detect important differences between the vaccines, unlike death which, fortunately, was too rare to provide strong evidence either way.

One limitation of our study is that it only included people whose first and second vaccine dose before boosting was either both Pfizer or both Astra-Zenena. People with other combinations were excluded due to low numbers. Given the potential role of mixed dosing for providing increased protection, it’s possible that our study population inherently favours Moderna boosting over Pfizer boosting, as all people getting a Moderna booster are dose-mixed, unlike those getting a Pfizer booster because some will have had Pfizer for all their doses.

Why is this important?

Comparing treatments using observational data is difficult. However, the nature of the UK booster programme, coupled with the rich clinical data available in the OpenSAFELY-TPP database, gave us a good opportunity to understand which vaccine performed better without relying on randomised controlled trials.

By closely matching recipients of each vaccine, we found a modest benefit of Moderna over Pfizer for protecting against COVID-19 related events during the 2021/22 booster vaccination programme in England. We think this study can help inform future booster vaccine procurement and delivery policies in the UK and elsewhere.