and huge economic and social upheaval internationally.
An efficacious vaccine is considered essential to prevent further morbidity and mortality.
To date, 44 candidate COVID-19 vaccines are in clinical development and 151 are in preclinical development, by use of a range of vaccine platforms.
In this unprecedented pandemic, vaccine development is time-dependent, and considerable collaborative efforts are being expended to expedite preclinical and clinical assessment of candidate vaccines.
The cost to manufacture and internationally deploy an efficacious COVID-19 vaccine will be vast, and the process will be at risk of politicisation.
Although some countries might deploy COVID-19 vaccines on the strength of safety and immunogenicity data alone, the goal of vaccine development is to gain direct evidence of vaccine efficacy in protecting humans against SARS-CoV-2 infection and COVID-19.
This definition is necessarily non-specific and reflects the complexities of assessing the clinical efficacy of candidate vaccines in the context of a novel pathogen. Indeed, a COVID-19 vaccine capable of reducing any of these elements might contribute to disease control where there are no efficacious prophylactic medications and few treatments.
The US Food and Drug Administration (FDA) suggested that laboratory-confirmed COVID-19 or SARS-CoV-2 infection are appropriate primary endpoints for vaccine efficacy studies, with an endpoint estimate of at least 50% for placebo-controlled efficacy trials.
However, protection against severe disease and death is difficult to assess in phase 3 clinical trials due to the unfeasibly large numbers of participants required. Instead, data to address this endpoint might be available only from large phase 4 trials or epidemiological studies done after widespread deployment of a vaccine. In this Review, we explore the challenges in assessing the efficacy of candidate SARS-CoV-2 vaccines and discuss the caveats needed to interpret reported efficacy endpoints.
Defining vaccine efficacy
Outcomes might include reduction in infection (ie, assessing sterilising immunity), severity of resultant clinical disease (ie, assessing disease-modifying immunity),
or duration of infectivity.
Such RCTs represent best-case scenarios of vaccine efficacy under idealised conditions in particular populations and provide key data necessary for vaccine licensure. However, vaccine efficacy does not always predict vaccine effectiveness—ie, the protection attributable to a vaccine administered non-randomly under field conditions.
For example, the effectiveness of rotavirus vaccines in children in low-income and middle-income settings was lower than the efficacy observed in children in high-income countries.
RCTs might not predict protection gained indirectly from herd protection (sometimes called herd immunity) following widespread vaccine deployment. Equally, RCTs done in a particular age group or geographical setting might not predict effectiveness if the vaccine is more widely deployed. It is possible that alternative vaccine platforms or the addition of adjuvants are required for adequate immunogenicity in older age groups, as for influenza vaccines.
For this reason, prospective studies of vaccine effectiveness in real-world scenarios post licensure are routinely needed.
On an individual level, the consequence of infection can range from paucisymptomatic states to hospital admission, requirement for respiratory support, and death.
Transmission dynamics of SARS-CoV-2 are not yet fully understood but the ability of infected individuals to transmit infection when asymptomatic or in a presymptomatic period means that infection control strategies that focus solely on preventing transmission from symptomatic individuals will be insufficient alone to interrupt the transmission of SARS-CoV-2.
The effect of an efficacious vaccine on the course of the SARS-CoV-2 pandemic is complex and there are many potential scenarios after deployment. The ability of a vaccine to protect against severe disease and mortality is the most important efficacy endpoint, as hospital and critical-care admissions place the greatest burden on health-care systems. However, the beneficial effects of such a vaccine on a population can be observed only if the vaccine is efficacious in older adults (eg, approximately >60 years) and widespread distribution of the vaccine exists, including to people who are most susceptible to COVID-19. High coverage among these groups who are at high risk of severe COVID-19 would have the greatest effect against disease endpoints. Alternatively, vaccines that do not affect the clinical course, but reduce the transmissibility of SARS-CoV-2, could still be valuable interventions on a population level.
Study design of SARS-CoV-2 vaccine efficacy trials
Although vaccine candidates can be assessed in isolation,
WHO and the US FDA suggest that an adaptive trial design, evaluating multiple vaccine candidates in parallel against a single placebo group, could be an acceptable method to increase efficiency, provided that the trials are sufficiently powered.
For example, if a phase 3 efficacy study enrolled participants aged only 20–29 years, the expected low mortality rate in this population would require an unfeasibly large sample size to adequately power the study to assess mortality as an endpoint, and the study would be reliant on a high rate of transmission to meet other efficacy endpoints (table 1). Selection of older participants with high rates of mortality, for example people older than 80 years, could reduce this number (table 1). However, given that older participants, especially people with comorbidities, are more likely to socially shield, they might be less likely to be exposed to SARS-CoV-2 and so a mortality efficacy endpoint might still not be met. Indeed, recruitment of older participants in vaccine trials has been historically challenging;
Cochrane reviews of influenza vaccine studies listed 52 RCTs in healthy adults with participants predominately aged 16–65 years, but only eight RCTs in adults older than 65 years,
despite the higher burden of disease in older adults. Given that mortality from SARS-CoV-2 disproportionately affects older adults, it is important that enrolment of older participants in COVID-19 vaccine trials is actively pursued via targeted engagement, minimising inconvenience to participants, and proactive sharing of study results.
Table 1Illustrative sample size calculations for a randomised controlled trial to assess efficacy of a SARS-CoV-2 vaccine candidate, calculated according to incidence of SARS-CoV-2 infection and age of participants
Hospital admission rates related to age and infection fatality ratio are taken from Verity and colleagues.
Each scenario presumes participants aged only either 20–29 years or >80 years are enrolled in the vaccine efficacy trial. SARS-CoV-2=severe acute respiratory syndrome coronavirus 2.
This unpredictability has a considerable effect on study sample sizes for efficacy endpoints (table 1), especially as infection rates are highly likely to change over the study follow-up period. For these reasons, assessment of vaccine efficacy against mortality is non-viable in current phase 3 clinical trials. However, as the pandemic spreads internationally, and other variables, such as poverty and decreased access to hospital care, contribute to high rates of severe disease in some populations, clinical trials in such settings could provide a measure of efficacy against severe disease with fewer participants than are suggested in table 1. Alternatively, pooling data from multiple trials that were not originally configured as a network of sites could mean that efficacy endpoints are met earlier, and conclusions about the efficacy of candidate vaccines are reached sooner.
The usefulness of pooling of data assumes that trial protocols can be sufficiently aligned and might come at the expense of a loss of statistical power if heterogeneity exists between trials.
Although this study and others did not provide direct data for vaccine efficacy against mortality,
this efficacy could be conjectured, and following a WHO recommendation for the widespread introduction of rotavirus vaccination,
substantial declines were noted in mortality from diarrhoeal illness.
This study design was used to show the efficacy of a recombinant vesicular stomatitis virus vectored Ebola vaccine (ie, rVSV-ZEBOV) during the Ebola virus disease outbreak in Guinea in 2015, when no cases were confirmed among contacts vaccinated immediately compared with 16 cases in the delayed-vaccination group.
However, for this study design to provide a measure of the efficacy of a SARS-CoV-2 vaccine, robust diagnostic and contact-tracing pathways and rapid inducement of protective immunity post vaccination would be needed. Given the respiratory route of transmission and the short incubation period of SARS-CoV-2, it is unlikely that this study design would be a feasible means of assessing the efficacy of SARS-CoV-2 vaccines.
This suggestion, combined with the selection of conserved antigens for most COVID-19 vaccine candidates,
means that vaccine efficacy detected against a particular circulating variant of SARS-CoV-2 in one region is likely to predict efficacy in other parts of the world.
However, to support this expectation, and because an efficacious vaccine can itself provide a selective pressure for SARS-CoV-2 mutation, serum samples from vaccinees in efficacy studies should be tested for neutralisation against a range of viral lineages.
Ongoing surveillance for viral escape from immunity induced by vaccines or mediated by antibodies will also be important.
This technique facilitates the rapid screening and deselection of candidate vaccines. A potential surrogate endpoint for a SARS-CoV-2 vaccine would most likely depend on the characteristics of the vaccine, including antigen structure, method of delivery, and antigen processing and presentation in vaccines.
and heterogeneity is seen in clinical outcomes.
However, the immunological mechanisms underlying protection or susceptibility to natural infection are unknown. Seroconversion of antibodies against SARS-CoV-2 is a marker of exposure, but whether the presence of neutralising antibodies is sufficient to provide protection against subsequent infection or disease is unclear.
Moreover, if these antibodies are sufficient, we do not know the titre that would be needed for protection or the diverse range of innate immune effector functions that can be relied on for antibody action, such as antibody-dependent complement deposition and antibody-dependent neutrophil phagocytosis.
Cellular immune responses have also been described in response to infection and are likely to be an important component of a protective adaptive immune response.
Indeed, individuals have been described who were seronegative and had T-cell responses to the SARS-CoV-2 spike protein.
However, the particular cellular signature that is required for protection is unknown, and whether protective T cells can be measured in peripheral blood samples is unclear. Additionally, an efficacious SARS-CoV-2 vaccine might provide protection by a mechanism that is distinct from the mechanism induced following natural infection. Distinguishing immunological markers of infection from mechanistic correlates of protection is difficult but important to inform rational design of a vaccine.
Other vaccine candidates could then be deemed efficacious and licensed if they induced similar levels of immune responses in non-inferiority studies, which would circumvent the need for large efficacy studies. Evidence of effectiveness against disease would be needed in post-licensure studies, however, this approach could markedly accelerate development of multiple SARS-CoV-2 vaccines. This approach relies on collaboration and standardisation of in-vitro assays to allow meaningful comparison of immunological outputs from different laboratories.
Protection against reinfection with SARS-CoV-2 has been observed in rhesus monkeys, who formed neutralising antibodies on initial exposure,
and a minimum neutralising antibody titre has been proposed.
However, since SARS-CoV-2 is a novel pathogen, any surrogate endpoints identified in animal studies would ideally need validation in clinical trials to ensure that they adequately predict efficacy in humans.
For example, the European Medicines Agency recommended marketing approval for an Ebola virus vaccine (ie, Ad26.ZEBOV and MVA-BN-Filo administered in a prime-boost regimen) on the basis of efficacy data that were extrapolated to humans from animal and immunobridging studies.
If it is impossible to collect human efficacy data, then SARS-COV-2 vaccines might be licensed on the basis of the animal rule, with effectiveness data collected after vaccine roll-out. However, the absence of accepted surrogate endpoints in humans or animals that are reasonably likely to predict the clinical benefit of a SARS-CoV-2 vaccine mean that investigators continue to pursue clinical evidence of vaccine efficacy in studies in humans.
Controlled human infection model
A COVID-19 CHIM model has several advantages over studies that are reliant on naturally occurring community transmission, which is difficult to predict and dependent on changes in behaviour and public health interventions (table 2).
Table 2A comparison of the key factors for clinical trials that are reliant on natural exposure to, or a direct challenge with, SARS-CoV-2
NA=not applicable. SARS-CoV-2=severe acute respiratory syndrome coronavirus 2.
If done, SARS-CoV-2 CHIM studies are likely to include carefully selected young volunteers at low risk of severe disease, who are exposed to low doses of SARS-CoV-2 with the aim of establishing only asymptomatic or mild infection. It is unclear whether efficacy shown in such a model will predict the key efficacy measure of protection against severe disease and death in the target older population who are at risk of severe disease.
CHIM studies could provide valuable immunological insights. For example, re-exposing individuals to SARS-CoV-2 who have recovered from naturally acquired infection could help to identify a surrogate marker of protection, which would inform vaccine design. Provided CHIM studies can be done safely, the information gained can be viewed as complementary to traditional RCTs, both to guide resources for large-scale phase 3 studies and in the efficacy evaluation of existing vaccines.
These studies can be logistically difficult and costly per participant, although the number of participants required is far lower than in large phase 3 studies. Although there are challenges to setting up a CHIM of SARS-CoV-2, there might also be substantial value in doing so, even in the context of a licensed product. Use of this ethically complex and controversial approach for vaccine assessment will require multidisciplinary, international oversight to ensure that outputs are rigorous and justify the potential risks to participants and their communities.
It is probable that there will not be a single vaccine winner; diverse platforms and technologies can offer different strengths and be relevant in distinct epidemiological contexts. Additionally, there will probably be insufficient supply, at least initially, of a single vaccine. However, collaboration and standardised approaches for assessing different efficacy endpoints will be important to allow meaningful comparison and ensure that the most effective candidates are deployed. Following deployment, well supported pharmacovigilance studies should be established to ensure the ongoing evaluation of vaccine safety.
Capacity to measure vaccine efficacy in field studies is reliant on ongoing SARS-CoV-2 transmission, which is rightly at odds with public health interventions. In the absence of a surrogate of protection, CHIM trials might provide the only means of rapidly assessing vaccine efficacy; however, the relationship between efficacy data from CHIM studies in young individuals and population-level protection is unclear. CHIM studies might help to identify a surrogate of protection. It is probable that any evidence for efficacy against severe disease and mortality in populations that are at risk will only be garnered post licensure via large epidemiological studies.
Finally, the development of SARS-CoV-2 vaccines is under great political and media scrutiny. In keeping with the development of any novel medical intervention, but particularly so in this context, it is imperative that efficacy outcomes for a SARS-CoV-2 vaccine are critically appraised with scientific rigour to understand their generalisability and clinical significance.
SHH, KM, GM, KRWE, and AJP conceived and wrote the manuscript. SHH, KM, GM, and KRWE created the figures. VH did the statistical calculations. All authors reviewed the publication.
SHH, KM, GM, and KRWE have worked, or are currently working on, the UK clinical trials of the SARS-COV-2 candidate vaccine (ChAdOx-1 nCoV-19). AJP is the chief investigator of these clinical trials. These clinical trials are funded by UK Research and Innovation (MC_PC_19055), Coalition for Epidemic Preparedness Innovations, the National Institute for Health Research, and the National Institute for Health Research Oxford Biomedical Research Centre. The University of Oxford has entered into a partnership with AstraZeneca on vaccine development. AJP is Chair of the UK Department of Health and Social Care Joint Committee on Vaccination and Immunisation but does not participate in discussions on COVID-19 vaccines and is a member of WHO’s Strategic Advisory Group of Experts. AJP is a National Institute for Health Research senior investigator. VH declares no competing interests. The views expressed in this Review do not necessarily represent the views of the UK Department of Health and Social Care, Joint Committee on Vaccination and Immunisation, National Institute for Health Research, or WHO.