A systematic review on rapid antigen test devices for SARS-CoV-2 in nursing homes: Useful, but handle with care

Dear Editor,

The COVID-19 pandemic has disproportionately affected Nursing homes (NHs), with mortality rates largely exceeding those of the general population, particularly during the first wave.^1,2 A recent study from England and Wales has estimated an 18-fold difference in mortality rates when figures were compared to the pre-pandemic time period, but this is a likely an underestimate given the low levels of testing in NHs, particularly when nasal swabs with subsequent Real-Time quantitative polymerase chain reaction (RT-qPCR) represented the only validated diagnostic items.²

As recently pointed out by Domínguez Fernandez et al.,³ rapid antigen tests (RAT), with their reduced costs and turnaround times,⁴ could significantly speed and scale up diagnoses, benefiting residents’ and workers’ safety. However, available evidence appears far more controversial. We specifically performed a systematic review and meta-analysis on RAT in NHs according to PRISMA guidelines (see Annex 1A for the detailed search strategy),⁵ being able to retrieve 5 studies (Table 1), for a total of 1327 paired samples RAT vs. RT-qPCR from residents of NHs, three of them from Spain.^3,6–9

Table 1.

Summary of the studies included in the present meta-analysis.

Reference	Characteristics of the samples	Commercial test	No. of samples	TP	FP	FN	TN	Se.	Sp.	PPV	PNV	Accuracy	Cohen's Kappa
Paap et al., 2021^6	Three NHs organizations in the Netherlands; single swab (December 1st, 2020, to March 31st, 2021)	Aptima® SARS-CoV-2 Assay + Lightcycler 480 from Roche diagnostics	461	27	25	26	383	50.9%	93.9%	51.9%	93.6%	88.9%	0.452
Escrivá et al., 2021^7	Seven NHs belonging to the Health Department of Consorcio Hospital General Universitario de Valencia (November–December, 2020)	Panbio COVID-19 Rapid Test Device	448	99	0	18	331	84.6%	100%	100%	94.8%	96.0%	0.890
Diez Flecha et al., 2021^8	One NH from the Spanish State of Leon after the emergence of a single case (November 24–25th, 2020)	Panbio COVID-19 Rapid Test Device	55	36	0	13	6	73.5%	100%	100%	31.6%	76.4%	0.377
Dominguez Fernández et al., 2022^3	One NH from the Spanish State of Galicia (September, 2020)	Panbio COVID-19 Rapid Test Device	30	19	0	1	10	95.0%	100%	100%	90.9%	96.7%	0.927
McKay et al., 2021^9	Serial facility-wide testing in a NH from Georgia (USA) (October, 22–November 3rd, 2020)	BinaxNOW	333	68	8	30	227	69.4%	96.6%	89.5%	88.3%	88.6%	0.706
POOLED	–	–	1327	249	33	88	957	75.8%	99.0%	88.3%	91.6%	90.9%	0.670
Only asymptomatic	–	–	311	70	7	34	200	67.3%	96.6%	90.9%	85.5%	86.8%	0.683
Only cases with cycle threshold ≥ 25	–	–	31	8	-	31	-	25.8%	–	–	–	–	–

Notes: NH = nursing home; TP = true positive; FP = false positive; FN = false negative; TN = true negative; Se. = sensitivity; Sp. = specificity; PPV = predicted positive value; PNV = predicted negative value; Cohen's Kappa values should be interpreted as follows: 0.0–0.20 no agreement, 0.21–0.39 minimal agreement, 0.40–0.59 weak agreement, 0.60–0.79 moderate agreement, 0.80–0.90 strong agreement, >0.90 almost perfect agreement.

Overall, RT-qPCR detected 337 SARS-CoV-2 positive cases (25.4%), with a pooled sensitivity of 75.8% (95% Confidence Interval [95%CI] 61.0–86.2) that was affected by substantial heterogeneity (I² = 82%, p < 0.01), and a pooled specificity of 99.0% (95%CI 89.3–99.9) (see Annex 1B for details). Two studies included estimates of viral replication,^6,8 while other two studies reported RAT performances by symptom status.^8,9 Even though Escrivá et al.⁷ included both symptom and viral activity statuses, reporting strategy impaired their inclusion in subgroup estimates. When sensitivity was calculated for samples characterized by cycle threshold values ≥ 25, an overall estimate of 25.8% was calculated, that increased to 67.3% in asymptomatic individuals irrespective of their viral replication status.

Diagnostic agreement, reported by means of Cohen's Kappa, ranged between 0.377 (95%CI 0.352–0.401)⁸ and 0.927 (95%CI 0.909–0.944),³ with a pooled estimate of 0.670 (95%CI 0.452–0.889), suggesting a moderate agreement despite the substantial heterogeneity (I² = 100%, p < 0.01). Diagnostic Odds Ratio (DOR) was estimated in 95.552 (95%CI 16.125–565.859), i.e. the OR for the positive result among residents with SARS-CoV-2 was approximately 96 times higher than the OR for positive results among persons without SARS-CoV-2. Summary Receiver Operating Characteristic (SROC) Curve (Annex 1C) was estimated through a maximum likelihood estimation model (REML), and a fixed model. Not only both curves were quite asymmetrical, suggesting a substantial heterogeneity among retrieved studies, but the substantial difference between the curves suggested that a substantial threshold effect may present, i.e. higher content of viral antigen may lead to increased identification of positive cases by RAT.

In other words, real-world estimates suggest that actual reliability of RAT may be quite far from optimal, particularly for non-seriated testing strategy. As acknowledged by Dominguez-Fernández et al.,³ in cases characterized by high viral load, RAT may be quite reliable,^6,8 but they exhibited substantial lack of sensitivity when employed in individuals that exhibit low viral replication. Indeed, RAT may be quite unreliable when employed to screen earlier stages of SARS-CoV-2 infections, or in individuals who, because of their even transitory lack of symptoms, may actively spread the infection not only among other residents, but also in NH workers failing to cope with appropriate preventive measures.^1,2 As a consequence, as suggested by McKay et al.,⁹ early and frequent referral to RAT rather than a single and synchronous sampling campaign may be quite effective in identifying individuals with the greatest potential to transmit the virus.

In summary, as RAT are relatively easy to use, produce results in minutes, and do not require expensive laboratory instruments, they can provide actionable results, particularly during outbreaks, but require a rational and specifically tailored use. On the contrary, as previously stressed by Escrivá et al.,⁷ the improper referral to instruments that can be affected by substantial lack of sensitivity may lead to potentially dismal consequences.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conflict of interest

We declare no competing interest. The facts, conclusions, and opinions stated in the article represent the Authors’ research, conclusions, and opinions, and are believed to be substantiated, accurate, valid, and reliable. However, as this article includes the results of personal researches of the Authors, presenting correspondent, personal conclusions, and opinions, parent employers are not forced in any way to endorse or share its contents and its potential implications.