Concordance between two rapid diagnostic tests for the detection of antibodies against SARS-CoV-2

Abstract

Objective

To assess the agreement between two rapid detection tests (RDT) for antibodies against SARS-CoV-2 infection.

Materials and methods

This was a cross-sectional study that used a random sample of non-hospitalized patients from the primary care management division of the Healthcare Area of Leon (58 RT-PCR-positive cases and 52 RT-PCR-negative cases). Information regarding symptoms was collected and all patients were simultaneously tested using two RDTs (Combined - cRDT and Differentiated - dRDT). The results of both tests were evaluated using the chi-square test and, for degree of agreement, the kappa coefficient.

Results

About 52% of the participants were women (mean age: 48.2 ± 11.0 years). A total of 58.2% were positive for d-RDT and 41.2% were positive for c-RDT. In the subjects who were RT-PCR-positive, d-RDT was positive in 72.4% and c-RDT in 55.2%; in those who were RT-PCR-negative, the percentages were 42.3% and 26.9%, respectively. The kappa coefficient observed between the two RDTs was 0.644, and was higher in patients without a fever or anosmia (0.725) and lower in those with a fever or anosmia (0.524).

Conclusions

There is good agreement between the tests used in this study. Given the sensitivity observed, they can be very useful as a complement to RT-PCR.

Resumen

Objetivo

Evaluar la concordancia entre dos pruebas de detección rápida (PDR) de anticuerpos en la infección por SARS-CoV-2.

Materiales y métodos

Estudio transversal. Se utilizó una muestra aleatoria de pacientes no hospitalizados de la Gerencia de Atención Primaria del Área de Salud de León (58 con RT-PCR positiva y 52 con RT-PCR negativa). Se recogió información sobre síntomas y a todos se les realizó simultáneamente dos PDR (combinada: PRD-C y diferenciada: PRD-D). Los resultados de ambas pruebas fueron evaluados mediante X² y el grado de concordancia con el índice Kappa.

Resultados

Un 52% de los participantes fueron mujeres (edad media: 48,2 ± 11,0 años). El 58,2% fue positivo a la PDR-D y 41,2% a la PDR-C. En los sujetos RT-PCR + la PDR-D fue positiva en el 72,4% y la PDR-C en el 55,2%; en el caso de los RT-PCR – en el 42,3% y 26,9%, respectivamente. El índice Kappa observado entre las dos PDR fue del 0,644, siendo mayor en pacientes sin fiebre ni anosmia (0,725) y menor en aquellos con fiebre o anosmia (0,524).

Conclusiones

Existe una buena concordancia entre los test utilizados en este estudio. Dada la sensibilidad obtenida, pueden ser de gran utilidad como complemento a las RT-PCR.

Introduction

The detection and isolation of the sources of infection is well-recognized as one of the main strategies for the prevention and control of the COVID-19 pandemic.¹ The first-choice test for detecting sources of infection is the RT-PCR (Reverse transcription polymerase chain reaction), a technique that requires complex equipment and experienced personnel, but is not immune to false-negative results.² The qualitative tests to detect specific antibodies, known as rapid detection tests (RDT), are presented as an alternative and/or complement to the RT-PCR because they are simple, do not require equipment and can be performed and interpreted quickly.³ Although RDTs have the necessary qualities for use in clinical diagnostics, there is currently no scientific evidence to support their internal validity or consistency, and no experience with their use internationally.³ In this context, it may be of great interest to see the concordance between what are currently two of the most frequently used RDTs in Spain.

Therefore, the aim of this study was to evaluate the concordance between two RDTs (Combined and Differentiated) and the RT-PCR in the diagnosis of the SARS-CoV-2 infection, in non-hospitalized patients with Covid-19 or those suspected of having the virus in the Healthcare Area of León.

Material and methods

Study design

A cross-sectional study was carried out. A random selection was made of 58 non-hospitalized patients who were RT-PCR-positive and 52 who were RT-PCR-negative. Patients were selected from the register of confirmed or suspected COVID-19 cases from primary care management of the Healthcare Area of Leon. In all patients, more than 14 days had passed since the onset of their symptoms.

Procedure

Participants were invited to take part in the study through a telephone call, during which they were summoned for collection of a biological sample and personal information. Participation in the study was voluntary. During the collection of information and samples, all protection regulations were followed and the project was approved by the Ethics Committee of the health area of León and the Bierzo (reference: 2073). After signing an informed consent, each participant completed a brief ad hoc questionnaire that collected information on socio-demographic data, symptoms (date of onset and end), and the date of the RT-PCR.

All patients were tested simultaneously with two RDTs:³

Combined (c-RDT) (one band): Wondfo® SARS-COV-2 Antibody test (Lateral Flow Method) of GUANGZHOU WONDFO BIOTECH CO LTD,

Differentiated (d-RDT) (two bands): This test allows differentiation between IgG and IgM. All Test® 2019-nCoV IgG/IgM Rapid Test Cassette of HANGZHOU ALL TEST BIOTECH CO LTD.

Both tests were performed using a finger-stick whole-blood sample. The first test determined the presence of IgM and IgG, while the second test differentiates both antibodies’ subtypes. The results of the tests were read 10-15 minutes after they were completed.

Statistical analysis

Central and dispersion measurements were calculated in quantitative variables (mean and standard deviation (SD)) and frequencies with their 95% confidence intervals in qualitative variables. A bivariate statistical analysis between the results of the different tests and the variables collected was carried out by comparing frequencies and chi-square tests. Mean differences in quantitative variables were estimated using the Student's t-test. Agreement between the different measurements analyzed was estimated with the kappa coefficient. All analyses were performed with the STATA 15 statistical package.⁴

The patient's consent has been obtained and followed treatment workplace protocols of patient information.

Results

A total of 110 patients participated in the study (51.8% were women, with a mean age of 48.2 ± 11.0 years). Among the respondents, 35.5% reported a fever of 38.5 °C or higher, 41.8% hypo-anosmia, 60.0% a fever or hypo-anosmia and 17.3% a fever and hypo-anosmia.

Using the differentiated tests, 18.2% of the patients were IgM positive (95% CI = 11.5-26.7) and 58.2% were IgG positive (95% CI = 48.4-67.5). All IgM-positive patients were also IgG-positive, meaning the prevalence of IgM or IgG positives was 58.2% for the differentiated test and 41.2% for the combined test (95% CI = 32.5-51.6).

Table 1 shows the distribution of the RDTs results. It is important to note that out of 64 patients who tested positive for IgG-IgM, 45 (70.3%) were also positive for the combined RDTs. Only one patient tested negative for the differentiated IgM-IgG test but was positive for the combined RDTs.

Table 1.

Results of the different RDTs.

		Differentiated RDT(IgM or IgG)		Combined RDT(IgM and IgG)
	N	Positive	%	Positive	%
IgM
Positive	20	20	100.0	19	95.0
Negative	90	44	48.9	27	30.0
IgG
Positive	64	64	100.0	45	70.3
Negative	46	46	100.0	1	2.2
IgM or IgG
Positive	64			45	70.3
Negative	46			1	2.2
Combined RDT
Positive	46	45	97.8
Negative	64	1	2.2

A positive and statistically significant result was most frequently presented in both RDTs among patients with a previously positive RT-PCR, those with hypo-anosmia and fever or hypo-anosmia (table 2). The positive cases for both RDTs were older than the negatives (Combined RDT: Negative = 46.2 ± 10.9 years vs. 50.9 ± 10.7 years; p = 0.027 / Differentiated RDT: Negative = 44.7 ± 11.2 years vs. 50.7 ± 10.2 years; p = 0.005).

Table 2.

Distribution of the results of the RDTs, by variable.

		Differentiated RDT(IgM or IgG)			Combined RDT(IgM and IgG)
	N	Positive	%	p	Positive	%	p
Sex
Men	57	37	64.9	0.138	25	43.9	0.653
Women	53	27	50.9		21	39.6
RT-PCR
Positive	58	42	72.4	0.001	32	55.2	0.003
Negative	52	22	42.3		14	26.9
Fever
Yes	39	26	66.7	0.181	20	51.3	0,136
No	71	38	53.5		26	36.6
Hypo-anosmia
Yes	46	37	80.4	< 0.001	27	58.7	0.002
No	64	27	42.2		19	29.7
Fever or hypo-anosmia
Yes	66	49	74.2	< 0.001	36	54.6	< 0.001
No	44	15	34.1		10	22.7
Fever and hypo-anosmia
Yes	19	14	73.7	0.132	11	57.9	0.118
No	91	50	54.6		35	38.5

Table 3 shows the degree of agreement and the kappa coefficient between the two RDTs analyzed. The overall kappa coefficient was 0.646 and was higher in subjects without a fever or anosmia (0.725) and lower in those with a fever or anosmia (0.524). Based on the previous RT-PCR result, the kappa coefficient was higher in RT-PCR-negative cases (0.669) than in those that were RT-PCR-positive (0.566).

Table 3.

Distribution of the kappa coefficient, by variable.

Variables	N	Agreement (%)	Kappa	95% CI
Total	110	81.8	0.646	0.513-0.779
RT-PCR-positive	58	79.3	0.566	0.363-0.770
RT-PCR-negative	52	84.6	0.669	0.470-0.868
Fever or hypo-anosmia	66	77.3	0.524	0.332-0.717
No fever and no hypo-anosmia	44	88.6	0.725	0.507-0.943

Discussion

The information provided by the company regarding the internal validity of the RDTs used in this study stated that there was 100% sensitivity in both cases and 97% specificity for RDTs, differentiating between IgG and IgM and 90% for the combined RDT.³ However, the information available from various validation studies carried out in our country expressed a specificity close to 100% for both tests and a sensitivity of 56.5% in the differentiated RDT and 63% in the combined RDT. These results were found in samples from hospitalized patients in which the evolution time of the disease was not taken into account and in about 80% of the tested patients the evolution time was more than 7 days.³

Although it is not in the scope of this article to assess the internal validity of these RDTs, in the overall analysis we are able to see that the prevalence of positive tests is higher in the differentiated RDT than in the combined RDT (58.2% vs 41.2%). This is also true for RT-PCR-positive patients (72.4% vs 55.2%). In the most recent case, the sensitivity observed by the test differentiating between immunoglobulins was higher than what the authors cited previously and lower than the combined RDT. Among other reasons, these differences can be attributed to the use of capillary blood in both RDTs. Though in both cases the sensitivity improves with the use of venous blood over capillary blood, the difference is greater in the combined tests (61.5% vs 84.5%; 74% vs 86%).

In terms of the results obtained from RDTs in RT-PCR-negative patients, it is important to note that 42.3% of the differentiated RDTs and 26.9% of the combined RDTs were positive. Although the possibility of false negatives in RT-PCR tests is known, it is remarkable that the numbers are so high and may be related to the sampling techniques instead of problems related to the technique itself or the low viral load.² This finding highlights the necessity for the combined use of both techniques, RDTs and RT-PCRs, at least in places and professions with a high risk of SARS-CoV-2 transmission.⁵

According to the Fleiss classification,⁶ the level of agreement observed between the two RDTs is good, and in some cases, close to excellent, both globally and in the different subgroups analyzed.

The sample used in this study were patients with symptoms compatible with COVID-19 who had been tested by RT-PCR to confirm the diagnosis. In all cases, more than 14 days had passed since the onset of symptoms. Given the viral dynamics and the immune system's response, the results obtained were what was expected, both globally, in RT-PCR-positive cases^7,8 and in cases with symptoms, especially fever and hypo-anosmia.^9,10

The differences in positivity between IgM and IgG were also expected given the dynamics of the appearance of the various immunoglobulins, although it seems that they may appear simultaneously if there is agreement on a greater persistence of IgG in relation to IgM and therefore a greater probability of being selected.⁷ ELISA-based IgG and IgM seroconversion occurs in all patients between the third and fourth week of symptom onset and while IgM decreases to low levels by the fifth week and almost disappears by the seventh week, IgG continues beyond the seventh week.¹¹

This study is not without its limitations. The study design and the sample size in particular require that the results obtained be interpreted with caution. However, the COVID-19 pandemic is a global battle, so this study provides interesting results in the RDTs analyzed that may be of special interest in the clinical practice and diagnosis of the SARS-CoV-2 infection.

Conclusion

The agreement between the two RDTs analyzed in the study is good. Both tests have a sensitivity in the range that other authors have observed, meaning they may be useful as a complement to RT-PCR.

Funding

This research has not received specific support from public sector agencies, the commercial sector or non-profit organizations.

Conflict of interest

The authors declare no conflict of interest.

Acknowledgements

The authors would like to thank all patients for their participation in this study who have voluntarily agreed to collaborate in the evaluation of diagnostic techniques against COVID-19. Thank you to the primary care management division of the Healthcare Area of Leon and the University of Leon also, because their collaboration has been crucial in developing this study.