Comparison of the effectiveness of four SARS-COV-2 v accines in Nuevo Leon, Mexico: A test-negative control study

Abstract

Objective

The objective of the present study was to provide statewide estimates of real-world effectiveness in reducing the odds of one primary (symptomatic COVID-19 infection) and two secondary outcomes (hospitalization and severe COVID-19 infection) by four vaccines BNT162b2 (Pfizer-BioNTech), ChAdOx1 (AstraZeneca), Ad5-nCoV (CanSinoBIO), and CoronaVac (Sinovac Life Sciences), used in Northeast Mexico.

Design

We conducted a test-negative case-control study and analyzed statewide surveillance data from December 2020 to August 2021.

Site

Primary attention and hospitalization.

Participants

Two inclusion criteria were applied, age ≥ 18 years and having a real-time reverse-transcriptase-polymerase-chain-reaction assay or a rapid test for antigen detection in postnasal samples (N = 164,052). The vaccination was considered complete if at least 14 days had passed since the application of the single or second dose and the beginning of symptomatology.

Interventions

Does not apply.

Main measurements

Point and 95% confidence intervals (CI) of vaccine effectiveness were calculated per type of vaccine using the formula 1 – odds ratio, adjusted by sex and age.

Results

Complete vaccination offered from none (CoronaVac – Sinovac) to 75% (95%CI 71, 77) (BNT162b2 – Pfizer) effectiveness in reducing symptomatic COVID-19 infection, regardless of sex and age. The fully ChAdOx1 (AstraZeneca) scheme reached the maximum effectiveness in hospitalization (80%, 95%CI 69, 87) and the fully BNT162b2 (Pfizer) scheme the maximum effectiveness in severity (81%, 95%CI 64, 90).

Conclusions

More studies are needed to compare benefits of different vaccines and guide policy makers select the best option for their population.

Resumen

Objetivo

Proporcionar estimaciones en el ámbito estatal de la efectividad en el mundo real de reducir las probabilidades de un resultado primario (infección sintomática por COVID-19) y 2 resultados secundarios (hospitalización e infección grave por COVID-19) para 4 vacunas: BNT162b2 (Pfizer-BioNTech), ChAdOx1 (AstraZeneca), Ad5-nCoV (CanSinoBIO) y CoronaVac (Sinovac Life Sciences) utilizadas en el noreste de México.

Diseño

Realizamos un estudio de casos y controles y analizamos los datos de vigilancia en todo el estado desde diciembre de 2020 hasta agosto de 2021.

Emplazamiento

Atención primaria y hospitalización.

Participantes

Se aplicaron 2 criterios de inclusión: edad ≥ 18 años y tener prueba de RT-PCR en tiempo real o una prueba rápida para la detección de antígeno en muestras posnasales (N = 164.052). La vacunación se consideró completa si habían transcurrido al menos 14 días desde la aplicación de la dosis única o desde la segunda dosis hasta el inicio de la sintomatología.

Intervenciones

No aplica.

Mediciones principales

Se calcularon los puntos e intervalos de confianza (IC) del 95% de la efectividad de la vacuna por tipo de vacuna utilizando la fórmula 1: razón de probabilidades, ajustada por sexo y edad.

Resultados

Vacunación completa que ofrece desde ninguna efectividad (CoronaVac-Sinovac) hasta el 75% de efectividad (IC95%: 71-77 de BNT162b2-Pfizer) en la reducción de la infección sintomática por COVID-19, independientemente del sexo y la edad. El esquema completo con ChAdOx1 (AstraZeneca) alcanzó la máxima efectividad en hospitalización (80%; IC95%: 69-87) y el esquema completo con BNT162b2 (Pfizer) la máxima efectividad en gravedad (81%; IC95%: 64-90).

Conclusiones

Se necesitan más estudios para comparar los beneficios de las diferentes vacunas y para guiar a los responsables en la formulación de políticas a seleccionar la mejor opción para su población.

Introduction

The COVID-19 pandemic continues affecting the whole world. As of September 2021, over 229 million people have been diagnosed with COVID-19 and over 4.7 million confirmed deaths have occurred.¹ An effective vaccine scheme is crucial to break the SARS-CoV-2 chain transmission^2,3 and as of October 2021, 6.6 billion doses of vaccines had been administered worldwide. Unfortunately, socioeconomic and health inequalities have been evident. From the beginning, developing countries have been the most affected due to weak health systems and limited access to vaccines.⁴ In Mexico, vaccination started in December 2020 and the administration was prioritized for the following groups: front-line health workers, adults 60 years and older, and teaching staff regardless of age. Then, according to the following age groups: 50–59, 40–49, 30–39, and 18–29. And the type of vaccine varied upon availability. Efficacy of COVID-19 vaccines varies from 66% to 95%,^5–6 but the maximum biological expected result might not be the same in day-to-day circumstances. Behaviors such as less use of safety measures and more social contact among vaccinated people may decrease the vaccine's potential.^7,8 Therefore, it is expected that the effectiveness be lower than efficacy.

Information on real-world effectiveness across diverse populations with different use of protection measures, prevalence of medical conditions, and compliance to cold-chain maintenance is needed for reassuring the benefits of vaccines. Effectiveness studies have included outcomes such as prevention or reduction of COVID-19 infection, hospitalization, intensive care admission, and/or death. And the results vary according to the type of vaccine.⁹ The objective of the present study was to provide statewide estimates of real-world effectiveness in reducing the odds of one primary (symptomatic COVID-19 infection) and two secondary outcomes (hospitalization and severe COVID-19 infection) by four vaccines BNT162b2 (Pfizer-BioNTech), ChAdOx1 (AstraZeneca), Ad5-nCoV (CanSinoBIO), and CoronaVac (Sinovac Life Sciences), used in Northeast Mexico.

Material and methods

We conducted a test-negative case–control study based on the state of Nuevo Leon surveillance data system (N = 178,822). In the region, various diagnostic centers offered COVID-19 testing free of charge to individuals with COVID-19 common symptoms such as high temperature, cough, or loss in sense of taste. Also, private laboratories provided diagnostic services, but at a cost. All of them must notify the State Ministry of Health. We analyzed data from December 2020 to August 2021, two inclusion criteria were applied, being at least 18 years old and having a real-time reverse-transcriptase-polymerase-chain-reaction [RT-PCR] assay or a rapid test for antigen detection in postnasal samples (N = 164,052); 771 (0.4%) registries were excluded for not knowing the vaccination status (Fig. 1). The sample size was large enough to provide statistical power greater than 95% with confidence level of 95%. The protocol was approved by Research Committee of Nuevo Leon Ministry of Health (DEISC-190122008). A waiver of informed consent was granted given the secondary use of a database and the confidentiality of personal data was always preserved.

Figure 1.

Cases and controls selection flowchart.

Exposure

History of vaccination (yes, no, if yes type of vaccine and number of doses). A post-vaccine window time was estimated between the date of the last dose and onset of symptoms. The scheme was considered complete if at least 14 days had passed since the application of the doses required for each vaccine: a single dose for Ad5-nCoV and two doses for BNT162b2, ChAdOx1, and CoronaVac, at the beginning of symptomatology.

Outcomes

One primary and two secondary outcomes were studied. The primary outcome was confirmed COVID-19 infection by RT-PCR assay or rapid antigen test in postnasal samples in symptomatic individuals. The date of appearance of the symptoms had to be later than the date of application of the vaccine. Cases and controls were categorized based on positivity: cases when the diagnosis was confirmed (n = 56,929) and controls, when the diagnosis was ruled out (n = 107,123). The secondary outcomes were analyzed only among cases: hospitalization (inpatient care upon positive diagnosis/the patient was hospitalized when the diagnosis was made vs ambulatory care) and COVID-19 severity (intensive care admission and/or death = severe infection vs no need for intensive care and/or no death = non-severe infection) (Fig. 1).

Statistical analysis

Frequencies were obtained for the categorical variables, as were means and standard deviations for the non-categorical variables with normal distribution. Three logistic regression models were run, one for estimating the odds of testing positive to SARS-CoV-2 in vaccinated compared with unvaccinated symptomatic individuals, adjusted by sex and age (primary outcome). The other two, for estimating the odds of hospitalization and COVID-19 severity (secondary outcomes). Point and 95% confidence intervals (CI) of vaccine effectiveness were calculated per type of vaccine using the formula 1 – odds ratio, adjusted by sex and age.¹⁰ We removed 165 records (1.7%) from the effectiveness analysis because of a small sample size: 60 of BBIBP-CorV (Sinofarm), 56 of ARNm-1273 (Moderna), 44 of Ad26.CoV2.S (Johnson & Johnson/Janssen), and 5 of other vaccine.

Results

Overall history of vaccination was 5.9%. The most frequently administered vaccine was BNT162b2 (Pfizer) (41.3%), followed by ChAdOx1 (AstraZeneca) (31.1%), CoronaVac (Sinovac) (16.6%), Ad5-nCoV (CanSinoBIO) (9.4%), and other (1.7%). It predominated the window time ≥ 14 days between the last or single dose and the onset of symptoms (78.7%); less than half had a complete vaccination scheme (47.1%). The cases were characterized by being male and older. Also, by having lower percentage of previous vaccination, ≥14 post-vaccination days, and complete scheme. But higher rate of secondary outcomes (Table 1).

Table 1.

Sociodemographic and vaccine history in symptomatic COVID-19 individuals. Study of COVID-19 vaccines effectiveness in Nuevo Leon, Mexico, December 2020–August 2021 (n = 164,052).

	Cases (n = 56,929)	Controls (n = 107,123)	Chi square p value
Sex, male	51.0%	45.1%	<0.001
Age (years), mean ± SD	42.1 ± 15.9	40.2 ± 16.1	<0.001
Age group (years)
18–24	13.5%	17.1%
25–44	45.6%	47.2%
45–60	27.8%	23.7%
61–64	3.3%	2.9%
65+	9.8%	9.0%	<0.001
History of vaccination (yes)	4.8%	6.5%	<0.001
Window time between the last dose and onset of symptoms
≥14 days	3.4%	5.2%
0–13 days	1.3%	1.2%	<0.001
Type of scheme
No vaccine	95.2%	93.6%
Incomplete (<14 days since single or 2nd dose, or 1st dose regardless of time)	3.2%	3.1%
Complete (≥14 days since single or 2nd dose)	1.6%	3.4%	<0.001
Hospitalization (yes)	15.1%	12.7%	<0.001
COVID-19 severity
Intensive care admission (yes)	3.9%	2.2%	<0.001
Death (yes)	5.5%	1.6%	<0.001

Effectiveness for reducing COVID-19 infection

The complete BNT162b2 (Pfizer) and Ad5-nCoV (CanSinoBIO) schemes made a difference for reducing COVID-19 infection. Fully BNT162b2 (Pfizer) effectiveness was much higher than fully Ad5-nCoV (CanSinoBIO) (74.5% 95%CI 71–77% vs 26.3% 95%CI 14–37%). The number of doses rather the number of days differentiated ChAdOx1 (AstraZeneca) effectiveness and having two doses was better than one regardless of the window time (33.2%, 95%CI 20–44%). CoronaVac (Sinovac) was not effective at all for reducing COVID-19 infection (Table 2).

Table 2.

Vaccines effectiveness for reducing the odds of COVID-19 infection, hospitalization, and severity in symptomatic SARS-CoV-2 individuals of four vaccines in Nuevo Leon, Mexico, December 2020–August 2021 (n = 164,052).

Vaccination status according to dose and window time since last application	Primary outcome			Secondary outcomes
	COVID-19 infection			Hospitalization			COVID-19 severity
	Yes	No	Effectiveness (95%CI)(Adjusted 1 – OR^b)	Yes	No	Effectiveness (95%CI)(Adjusted 1 – OR^b)	Yes	No	Effectiveness (95%CI)(Adjusted 1 – OR^b)
BNT162b2 (Pfizer)
No vaccine	98.92%	96.72%	Reference	99.23%	98.87%	Reference	99.45%	98.87%	Reference
1st dose 0–13 days	0.16%	0.24%	38.6% (22%, 52%)	0.17%	0.16%	38.7% (−13%, 67%)	0.15%	0.16%	44.9% (−19%, 74%)
1st dose ≥ 14 days	0.25%	0.58%	59.1% (51%, 66%)	0.19%	0.26%	42.0% (0%, 66%)	0.21%	0.25%	29.9% (−35%, 63%)
2nd dose 0–13 days	0.09%	0.24%	64.5% (52%, 74%)	0.05%	0.10%	77.9% (34%, 93%)	0.00%	0.10%	100%
2nd dose ≥ 14 days^a	0.58%	2.22%	74.5% (71%, 77%)	0.36%	0.61%	62.9% (44%, 75%)	0.19%	0.62%	81.1% (64%, 90%)
ChAdOx1 (AstraZeneca)
No vaccine	97.95%	98.20%	Reference	98.55%	97.84%	Reference	98.57%	97.88%	Reference
1st dose 0–13 days	0.63%	0.40%	−51.7% (−75%, −31%)	0.38%	0.67%	43.5% (17%, 61%)	0.36%	0.65%	40.1% (3%, 63%)
1st dose ≥ 14 days	1.03%	0.89%	−9.4% (−22%, 2%)	0.68%	1.09%	55.7% (41%, 67%)	0.58%	1.08%	58.6% (40%, 72%)
2nd dose 0–13 days	0.07%	0.10%	32.3% (2%, 53%)	0.08%	0.07%	53.6% (−9%, 80%)	0.11%	0.07%	30.0% (−74%, 72%)
2nd dose ≥ 14 days^a	0.32%	0.42%	33.2% (20%, 44%)	0.31%	0.33%	80.0% (69%, 87%)	0.38%	0.32%	74.6% (59%, 84%)
CoronaVac (Sinovac)
No vaccine	98.77%	99.10%		99.07%	98.72%	Reference	99.16%	98.73%	Reference
1st dose 0–13 days	0.15%	0.12%	−25.1% (−65%, 5%)	0.10%	0.17%	51.7% (−1%, 77%)	0.11%	0.16%	31.6% (−58%, 70%)
1st dose ≥ 14 days	0.62%	0.45%	−31.2% (−51%, −14%)	0.59%	0.63%	32.2% (7%, 50%)	0.49%	0.64%	38.5% (7%, 59%)
2nd dose 0–13 days	0.20%	0.11%	−74.2% (−128%, −33%)	0.08%	0.22%	73.2% (42%, 88%)	0.06%	0.21%	79.2% (34%, 94%)
2nd dose ≥ 14 days^a	0.25%	0.22%	−2.9% (−27%, 17%)	0.17%	0.27%	79.4% (63%, 88%)	0.17%	0.26%	77.9% (55%, 89%)
Ad5-nCoV (CanSinoBIO)
No vaccine	99.52%	99.36%	Reference	99.83%	99.47%	Reference	99.85%	99.24%	Reference
Only dose 0–13 days	0.05%	0.03%	−46.5% (−150%, 14%)	0.06%	0.04%	−31.9% (−287%, 55%)	0.04%	0.00%	28.9% (−245%, 85%)
Only dose ≥ 14 days^a	0.43%	0.61%	26.3% (14%, 37%)	0.11%	0.49%	72.8% (46%, 86%)	0.11%	0.00%	65.1% (20%, 85%)

a

Complete scheme.

b

Adjusted by sex and age.

Effectiveness for reducing hospitalization

Hospitalization rate in confirmed cases of COVID-19 was 15.1%. The complete ChAdOx1 (AstraZeneca) and CoronaVac (Sinovac) schemes registered the highest effectiveness in hospitalization (80% and 79.4%, respectively). It was followed by the fully Ad5-nCoV (CanSinoBIO) scheme (72.8%) and the fully BNT162b2 (Pfizer) scheme (62.9%). Having at least 14 days made a difference with ChAdOx1 (AstraZeneca) and Ad5-nCoV (CanSinoBIO) effectiveness, not so with BNT162b2 (Pfizer) and CoronaVac (Sinovac) (Table 2).

Effectiveness for reducing severity

The death rate in confirmed cases of COVID-19 was 5.5% and the intensive care admission rate was 3.9%. It was the number of doses and not the number of postvaccine days what made the difference with BNT162b2 (Pfizer) and CoronaVac (Sinovac) effectiveness. Having two doses reduced severity regardless of the window time. Not so with ChAdOx1 (AstraZeneca) and Ad5-nCoV (CanSinoBIO), whose effectiveness depended on a fully scheme (number of doses and window time since last application) (Table 2 and Table S1).

Discussion

Analysis of the real-world effectiveness across different regions is essential to fully understand vaccines impact. Therefore, we compared statewide real-world effectiveness of four vaccines in symptomatic SARS-CoV-2 adults with RT-PCR or a rapid test for antigen detection in postnasal samples.

COVID-19 infection (primary outcome)

All the vaccines were less effective than reports found in the literature. The fully BNT162b2 (Pfizer) vaccination scheme (two doses and ≥14 days since second dose) registered the highest effectiveness in reducing symptomatic COVID-19 infection at 75%, lower than 87% (−12% difference) and 97% (−22% difference) reported in USA (California residents) and Israel, respectively.^11,12 Two ChAdOx1 (AstraZeneca) doses produced an effectiveness of 32–33% regardless of the window time (0–13 days or ≥14 days), a figure much lower than the 78% (−46% to 47% difference) obtained with the complete scheme in Brazil.¹³ Results may depend on the SARS-CoV-2 variant strain as Sheikh et al.¹⁴ showed. They identified an effectiveness of 79% with a fully BNT162b2 (Pfizer) scheme in cases with the Delta variant compared to 92% in cases without such variant. A similar scenario occurred with the fully ChAdOx1 (AstraZeneca) vaccination, 60% and 73%, respectively. Another vaccine examined was CoronaVac (Sinovac). We found this was not effective for preventing a symptomatic COVID-19 infection. Other studies have estimated effectiveness between 37% and 66% for the fully,^13,15,16 while clinical trials have estimated efficacy between 51% and 84%.⁹ Regarding the Ad5-nCoV (CanSinoBIO) vaccine, the complete scheme was 26% effective compared to 69% (−43% difference) efficacy previously reported.⁹ Lower effectiveness results in all the vaccines reminds us of the importance of analyzing vaccines effectiveness across populations. It also points out the need to reinforce the use of protection measures. People could have reduced its use knowing they had been vaccinated, that is, because of a false security that any type of vaccine could had provided.

BNT162b2 was effective in preventing primary and secondary outcomes. On the contrary, CoronaVac was not effective in preventing infection with SARS-CoV-2, but it was effective in preventing severe COVID-19 outcomes. One study found that none of serum specimens showed neutralizing antibody seroconversion against Omicron after a two-dose regimen, implying the possibility that humoral immunity may represent not all protection against severe COVID-19 outcomes. At the present, it is not clear what level of antibody titer guarantees protection against SARS-CoV-2. Time is required for neutralizing antibodies to develop after vaccination. Furthermore, reduction of the antibody response over time is a natural process of humoral immunity.¹⁷

Hospitalization and severity (secondary outcomes)

Almost all or all the vaccines were less effective than other studies in reducing hospitalization or COVID-19 severity. The fully BNT162b2 (Pfizer) scheme was 63% effective in reducing hospitalization, lower than 97% (−34% difference) and 87% (−24% difference) found in Israel and USA,^12,18 respectively. While the fully ChAdOx1 (AstraZeneca) scheme was 80% effective, also lower than 91% (−11% difference) obtained in Brazil.¹³ The complete Ad5-nCoV (CanSinoBIO) was 73% effective whereas the complete CoronaVac (Sinovac) was 79% effective, compared to 71% (+2% difference) and 88% (−9% difference) identified in Brazil and Chile, respectively.^13,16 Regarding COVID-19 severity, the maximum effectiveness was 81% obtained with a complete BNT162b2 (Pfizer) scheme. Other authors have showed at least 90% effectiveness (−9% difference).^12,18 On the opposite side, the minimum effectiveness was 65%, obtained with a complete Ad5-nCoV (CanSinoBIO) scheme compared to reported efficacy > 95% (−30% difference).⁹ Fully ChAdOx1 (AstraZeneca) and CoronaVac (Sinovac) vaccinations registered in between effectiveness, 75% and 78%, respectively. In Brazil, ChAdOx1 (AstraZeneca) was at least 90% (−15% difference)¹³ and in Chile CoronaVac (Sinovac) was > 85% (−7% difference).¹⁶ Then, CoronaVac (Sinovac) had the least difference with respect literature effectiveness, even was better in reducing hospitalization than result found in one study. Moreover, CoronaVac (Sinovac) together with BNT162b2 (Pfizer), registered the least difference with respect other studies regarding severity. This has the implication of future decision to choose the vaccine with the greatest expected benefit in the target population.

Limitations

We did not have data on occupation. Some occupations may lead to higher exposure, but also to higher use of protection measures such as the health care sector. Information on comorbidities was not available either, a recognized risk factor for severe COVID-19. However, given that several comorbidities are intrinsically related to age, and that age was considered in the multivariate effectiveness analysis, indirectly, or at least partially, an adjustment for the effect of comorbidities was present. More importantly, we restricted the analysis to symptomatic individuals. The effect of the vaccines against asymptomatic disease may differ and future studies on asymptomatic individuals are needed. Finally, differences in effectiveness may be due to differences in the ability of the vaccine to deal with variants of the virus present during the data collection phase (December 2020 to August 2021). Unfortunately, this information was no available, future studies need to consider this factor.

Conclusions

Results evidenced that complete vaccination offered from none (CoronaVac – Sinovac) to 75% (BNT162b2 – Pfizer) protection in reducing symptomatic COVID-19 infection. Hospitalization protection ranged from 63% to 80% and severity protection, from 65% to 81%, regardless of sex and age. The fully ChAdOx1 (AstraZeneca) scheme reached the highest effectiveness to reduce hospitalization and the fully BNT162b2 (Pfizer) scheme the highest effectiveness to reduce severity. While the lowest effectiveness was reached by the fully BNT162b2 (Pfizer) and fully Ad5-nCoV (CanSinoBIO) schemes, respectively. More studies are needed to compare benefits of different vaccines and guide policy makers select the best option for their population.

Ethical considerations

The protocol was approved by the Research Ethics Committee of the Nuevo Leon Ministry of Health and was registered under the number DEISC-190122008.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Conflict of interest

The authors have no conflicts of interest to declare.