Specification of the target trials
We designed an observational analysis to emulate a hypothetical pragmatic trial of a third dose of BNT162b2 compared with mRNA-1273 and risks of COVID-19 outcomes in the VA healthcare system. Supplementary Table 3 summarizes the key protocol components.
Eligibility criteria included veteran status; age ≥65 yr or 18–64 yr with high risk of severe COVID-19 between 20 October and 18 November 2021 (based on the presence of at least one coexisting condition listed in Table 1)8 or ≥18 yr between 19 November 2021 and 8 February 2022 (based on national guidelines for third dose deployment)9,10; receipt of the second dose of an mRNA vaccine primary series at least 6 months earlier (based on the same guidelines); known residential address outside of a long-term care facility; and known smoking status and known body-mass index within the previous year. Individuals had to have used the VA healthcare system during the previous year (defined as receiving care at a station eligible to administer the vaccines under study and having at least one primary care visit) but not within the previous 3 d (which may indicate the start of symptomatic disease).
The interventions of interest were a third dose of either the BNT162b2 or the mRNA-1273 vaccine. To ensure balance of important characteristics across groups, eligible veterans in the target trial would be randomly assigned to one of these two vaccines within strata defined by calendar date of the third dose (5 d bins), calendar month of the second mRNA vaccine dose, age (5 yr bins), sex (male, female), race (White, Black, other, unknown), urbanicity of residence (urban, not urban), geographic location (coded as 19 categories of Veterans Integrated Services Network) and number of SARS-CoV-2 tests performed in the past 12 months (0, 1, ≥2).
The five outcomes of interest were documented SARS-CoV-2 infection, documented symptomatic COVID-19 and COVID-19-related hospital admission, ICU admission and death. For each eligible individual, follow-up started on the day the third dose of vaccine was received (baseline) and ended on the day of the outcome of interest, death, 112 d (16 weeks) after baseline, or the end of the study period (15 February 2022), whichever happened first.
Our target trial evaluates comparative effectiveness of a third dose of the vaccines during a period spanning times when SARS-CoV-2 Delta- and Omicron-variants were circulating. The Delta variant had decreased to a share of 26% of circulating variants in the United States as of 25 December 2021, as it was rapidly displaced by the Omicron variant, which rose to a 100% share as of 12 February 202211. To evaluate the comparative effectiveness during a period of Omicron-variant predominance, we considered a second target trial that was identical to the first trial except that the recruitment period was 1 January to 1 March 2022, and the only outcome of interest was documented SARS-CoV-2 infection because the period was too short to accrue a sufficient number of rarer, more severe outcomes.
Emulation of the target trials
We emulated both target trials using the VA healthcare databases3. Vaccination was identified using records from the Corporate Data Warehouse and the VA COVID-19 Shared Data Resource. SARS-CoV-2 infections were identified using the VA COVID-19 National Surveillance Tool3,12, which integrates data on laboratory tests with natural language processing of clinical notes to capture diagnoses documented inside and outside the VA healthcare system. Symptomatic COVID-19 was defined as ≥1 of the following symptoms documented within 4 d of SARS-CoV-2 infection: fever, chills, cough, shortness of breath or difficulty breathing, sore throat, loss of taste or smell, headache, myalgia/muscle pain, diarrhoea and vomiting. COVID-19 hospitalization was defined as a hospitalization within 21 d after documented SARS-CoV-2 infection, COVID-19 ICU admission was defined as an ICU admission during a COVID-19 hospitalization, and COVID-19 death was defined as a death within 30 d after documented SARS-CoV-2 infection. Supplementary Table 4 provides detailed information on all study variables and their ascertainment.
We attempted to mimic the stratified randomization of the target trial by matching persons who received a third dose of BNT162b2 and of mRNA-1273 on the basis of the calendar date of the third dose, calendar month of the second dose, age, sex, race, urbanicity of residence, geographic location, and number of SARS-CoV-2 tests performed in the past 12 months, using the same matching algorithm described in our previous study3.
To explore the possibility of residual confounding (for example, by underlying health status or healthcare-seeking behaviour), we incorporated two negative outcome controls7. First, we evaluated the risk of symptomatic COVID-19 in the first 7 d after the third vaccine dose. Second, we evaluated the risk of death from causes other than COVID-19 during the follow-up period.
Statistical analyses
Covariate balance after matching was evaluated by plotting the mean differences between variable values (standardized for continuous variables) for the vaccination groups, with a difference of 0.1 or less considered to be acceptable13.
Cumulative incidence (risk) curves for the vaccine groups were estimated using the Kaplan-Meier estimator14. We then calculated 16-week risks of each outcome and compared them between the vaccine groups via differences and ratios. We conducted subgroup analyses by age (<70 or ≥70 yr), race (Black or White), time since completion of the COVID-19 vaccine primary series (6–7, 8, or ≥9 months) and vaccine type of the primary series (BNT162b2 or mRNA-1273). We conducted sensitivity analyses that excluded eligible individuals who (1) had previously documented SARS-CoV-2 infection, (2) did not receive a homologous third dose compared with the COVID-19 vaccine primary series and (3) had a third dose that could not be identified as a booster dose on the basis of procedural codes available in the ‘Inpatient’ and ‘Outpatient’ domains, to compare recipients of vaccines at known doses, as the dose of mRNA-1273 differs for booster doses (50 μg) vs third primary doses (100 μg).
We used a nonparametric bootstrapping procedure (including both matching and subsequent analyses) with 500 iterations to calculate percentile-based 95% confidence intervals for all estimates.
Analyses were performed using R software version 3.6.0 (R Foundation for Statistical Computing) and SAS Enterprise Guide version 8.2 (SAS Institute).
Reporting summary
Further information on research design is available in the Nature Portfolio Reporting Summary linked to this article.