Different countries have so far approved more than 6 Covid-19 vaccines mostly by Emergency Use Authorizations. With this development a few questions keep popping up. One such question is on how the vaccines stack up against each other. Last week, a friend that I had not spoken to in ages called. After talking to her I realized I had not followed my own advice to physically distance rather than socially distance. My old colleague needed advice for a friend on vaccination. A close friend of hers had decided she would only take the Moderna vaccine. Truthfully that lady is not the only one trying to find out how these vaccines compare to each other. Last weekend, I got another question on vaccine efficacy comparisons. So in this post I am going to address this question; “How good are we at comparing the efficacy of vaccines?”.
I know many want a simple chart comparing one vaccine to the next one. Unfortunately that would not be possible, the subject is not that simple. Vaccine efficacy is usually expressed as a percentage. This provides us with an idea on the number of infections prevented by the vaccine. This number is a form of currency that pharmaceutical companies brandish in advertisements and presentations on the quality of their product. In this post, I would use data from the Moderna vaccine trial to illustrate these calculations.
The Moderna mRNA-1273 Trial
Towards the end of December 2020 the New England Journal of Medicine published data from the Moderna vaccine trial. The study enrolled 30,420 volunteers who were assigned randomly into 2 groups. 15,210 were in the vaccine group and received mRNA-1273, with the same number of participants getting the dummy vaccine. 196 cases of SARS-CoV-2 infection occurred among the study participants during follow-up. These case numbers only included individuals diagnosed more than 14 days after receiving their second dose.
In the unblinded data there were 11 cases in the mRNA-1273 group and 185 in the dummy group. The calculated incidence rate in the dummy vaccine group was 56.5 per 1000 person years. For those who got mRNA-1273 the incidence was 3.3 per 1000 person years. Thus if a 1000 individuals were given the vaccine and followed for a year about 3 of them will get Covid-19. In those given the dummy about 56 are expected to get Covid-19. These numbers only apply if the same conditions as occurred during the study prevailed.
Efficacy Calculations From The Moderna Trial
Mathematically vaccine efficacy is defined as {1 – hazard ratio (mRNA-1273 vs. placebo)}. The hazard ratio based on these numbers is 3.3/56.5 which is 0.058. The efficacy is thus 1 – 0.058 or 0.94.2 about 94.2%. So what does this mean? This is where it gets a bit more complicated. This means in a setting similar to the study conditions an imaginary person given the vaccine avoids symptomatic SARS-CoV-2 infection on average 94.2% of the time. Similar conditions include, weather, human behavior, other disease conditions, housing, virus distributions including variants etc. The setting is really important as one company Novavax with a vaccine in the pipeline realized. In their early analysis of their vaccine study data it was 90% effective in the U.K. but 49% effective in South Africa. These factors influence the infectivity of the virus making comparing efficacy numbers from different studies impossible.
Can We Ever Compare The Efficacy of Two Vaccines?
Vaccine efficacy can only be compared adequately in head to head trials. In such studies the dummy group receives a different vaccine for the same condition. In 2020 one such study reported in the Lancet compared an oral Flu vaccine against an intramuscular one. For that study the oral vaccine did better than the injectable Flu vaccine. This was a small study with 179 individuals. Unfortunately when the current Covid-19 vaccines were going through trials there were no other vaccines available for SARS-CoV-2. Thus at this time there are no head to head studies of SARS-CoV-2 vaccines available. The current available vaccines were tested in different populations under different conditions. This makes any fair comparison of efficacy numbers impossible at this time. Below are some of the key information and numbers of the current available vaccines.
Table of Available Vaccines and Reported Efficacy From Clinical Trials
| Company | Type of Vaccine | Study Population | End point determination | Countries | Efficacy (%) | Protection against Severe disease (%) |
| Pfizer-Biontech | mRNA Vaccine | 43,448 individuals 16 years and older | Confirmed Covid-19 infection 7 days after 2nd dose | Argentina, Brazil, South Africa, Germany, Turkey, USA | 94.6 (89.9 – 97.3) | 100% |
| Moderna | mRNA Vaccine | 30,420 individuals 18 years and older | Covid-19 infection based on clinical criteria 14 days after 2nd dose | USA only | 94.1% (89.3 – 96.8) | 100% |
| Johnson and Johnson | Adenovirus Vector Vaccine | 39,321 individuals ages 18 years and older | PCR confirmed symptomatic Covid-19 diagnosis 14 days after vaccination | Brazil, Chile, Argentina, Columbia, Peru, Mexico, South Africa and USA | 66.1% (55.0 – 74.8) | 100% |
| AstraZeneca* | Adenovirus Vector Vaccine | 23,848 individuals (i) Ages 18 – 55 years in COV001 (UK). (ii) High risks professionals (UK) (iii) Healthcare workers (Brazil). (iv)Healthy non-HIV positive age 18 – 65 years (South Africa) | PCR confirmed, symptomatic COVID-19 14 days after the 2nd dose | Brazil, South Africa and UK | 70.4 (54.8 – 80.6) | 100% |
| Chinese Can Sino | Adenovirus Vector Vaccine | adults 18 years and older | PCR confirmed, symptomatic COVID-19 14 days after the 2nd dose | Brazil, Pakistan, Russia, Argentina Turkey | Varies from 50.4% Reported in Brazil to 91.2% in Turkey | |
| Russian Sputnik-V** | Adenovirus Vector Vaccine | 21,862 individuals 18 years and older | PCR Confirmed Covid-19, 21 days after the first dose | Russia | 91·6% (95% CI 85·6–95·2) | 100% |
**There were multiple methodological issues in the conduct of the Russian Sputnik-V vaccine which could be considered problematic by a regulatory authority
The Chinese Can Sino vaccine’s manufacturers have not presented their data to any regulatory authorities outside China nor published phase 3 results in peer reviewed journals
How Must One Evaluate These Efficacy Numbers?
At this point in time we must all look at these numbers as a guide on the likelihood of obtaining protection from Covid-19 disease. None of the current vaccines report protection from infection. There are other studies looking into that. Protection from infection has however never been the focus of vaccines anyway. For most of the vaccine that I could find information on protection from severe disease the protection is 100%. Effective vaccines work by interrupting infection or changing the course of the disease. We cannot compare the numbers to each other if they were obtained in different studies.
It is important to remember that the best vaccine is the one in your arm. It is ok to have a favorite but everyone needs to get vaccinated as soon as possible. Until we have head to head studies that compare the vaccines to each other we cannot compare these numbers.
By Dr. Leonard Sowah an Internal Medicine Physician in Baltimore, Maryland
Feature photo: Modified from different photos by Marco Verch
