To assess the agreement between two rapid detection tests (RDT) for antibodies against SARS-CoV-2 infection.
Materials and methodsThis was a cross-sectional study that used a random sample of non-hospitalized patients from the primary care management division of the Healthcare Area of Leon (58 RT-PCR-positive cases and 52 RT-PCR-negative cases). Information regarding symptoms was collected and all patients were simultaneously tested using two RDTs (Combined - cRDT and Differentiated - dRDT). The results of both tests were evaluated using the chi-square test and, for degree of agreement, the kappa coefficient.
ResultsAbout 52% of the participants were women (mean age: 48.2±11.0 years). A total of 58.2% were positive for d-RDT and 41.2% were positive for c-RDT. In the subjects who were RT-PCR-positive, d-RDT was positive in 72.4% and c-RDT in 55.2%; in those who were RT-PCR-negative, the percentages were 42.3% and 26.9%, respectively. The kappa coefficient observed between the two RDTs was 0.644, and was higher in patients without a fever or anosmia (0.725) and lower in those with a fever or anosmia (0.524).
ConclusionsThere is good agreement between the tests used in this study. Given the sensitivity observed, they can be very useful as a complement to RT-PCR.
Evaluar la concordancia entre dos pruebas de detección rápida (PDR) de anticuerpos en la infección por SARS-CoV-2.
Materiales y métodosEstudio transversal. Se utilizó una muestra aleatoria de pacientes no hospitalizados de la Gerencia de Atención Primaria del Área de Salud de León (58 con RT-PCR positiva y 52 con RT-PCR negativa). Se recogió información sobre síntomas y a todos se les realizó simultáneamente dos PDR (combinada: PRD-C y diferenciada: PRD-D). Los resultados de ambas pruebas fueron evaluados mediante X2 y el grado de concordancia con el índice Kappa.
ResultadosUn 52% de los participantes fueron mujeres (edad media: 48,2 ± 11,0 años). El 58,2% fue positivo a la PDR-D y 41,2% a la PDR-C. En los sujetos RT-PCR + la PDR-D fue positiva en el 72,4% y la PDR-C en el 55,2%; en el caso de los RT-PCR – en el 42,3% y 26,9%, respectivamente. El índice Kappa observado entre las dos PDR fue del 0,644, siendo mayor en pacientes sin fiebre ni anosmia (0,725) y menor en aquellos con fiebre o anosmia (0,524).
ConclusionesExiste una buena concordancia entre los test utilizados en este estudio. Dada la sensibilidad obtenida, pueden ser de gran utilidad como complemento a las RT-PCR.
The detection and isolation of the sources of infection is well-recognized as one of the main strategies for the prevention and control of the COVID-19 pandemic.1 The first-choice test for detecting sources of infection is the RT-PCR (Reverse transcription polymerase chain reaction), a technique that requires complex equipment and experienced personnel, but is not immune to false-negative results.2 The qualitative tests to detect specific antibodies, known as rapid detection tests (RDT), are presented as an alternative and/or complement to the RT-PCR because they are simple, do not require equipment and can be performed and interpreted quickly.3 Although RDTs have the necessary qualities for use in clinical diagnostics, there is currently no scientific evidence to support their internal validity or consistency, and no experience with their use internationally.3 In this context, it may be of great interest to see the concordance between what are currently two of the most frequently used RDTs in Spain.
Therefore, the aim of this study was to evaluate the concordance between two RDTs (Combined and Differentiated) and the RT-PCR in the diagnosis of the SARS-CoV-2 infection, in non-hospitalized patients with Covid-19 or those suspected of having the virus in the Healthcare Area of León.
Material and methodsStudy designA cross-sectional study was carried out. A random selection was made of 58 non-hospitalized patients who were RT-PCR-positive and 52 who were RT-PCR-negative. Patients were selected from the register of confirmed or suspected COVID-19 cases from primary care management of the Healthcare Area of Leon. In all patients, more than 14 days had passed since the onset of their symptoms.
ProcedureParticipants were invited to take part in the study through a telephone call, during which they were summoned for collection of a biological sample and personal information. Participation in the study was voluntary. During the collection of information and samples, all protection regulations were followed and the project was approved by the Ethics Committee of the health area of León and the Bierzo (reference: 2073). After signing an informed consent, each participant completed a brief ad hoc questionnaire that collected information on socio-demographic data, symptoms (date of onset and end), and the date of the RT-PCR.
All patients were tested simultaneously with two RDTs:3
Combined (c-RDT) (one band): Wondfo® SARS-COV-2 Antibody test (Lateral Flow Method) of GUANGZHOU WONDFO BIOTECH CO LTD,
Differentiated (d-RDT) (two bands): This test allows differentiation between IgG and IgM. All Test® 2019-nCoV IgG/IgM Rapid Test Cassette of HANGZHOU ALL TEST BIOTECH CO LTD.
Both tests were performed using a finger-stick whole-blood sample. The first test determined the presence of IgM and IgG, while the second test differentiates both antibodies’ subtypes. The results of the tests were read 10-15minutes after they were completed.
Statistical analysisCentral and dispersion measurements were calculated in quantitative variables (mean and standard deviation (SD)) and frequencies with their 95% confidence intervals in qualitative variables. A bivariate statistical analysis between the results of the different tests and the variables collected was carried out by comparing frequencies and chi-square tests. Mean differences in quantitative variables were estimated using the Student's t-test. Agreement between the different measurements analyzed was estimated with the kappa coefficient. All analyses were performed with the STATA 15 statistical package.4
The patient's consent has been obtained and followed treatment workplace protocols of patient information.
ResultsA total of 110 patients participated in the study (51.8% were women, with a mean age of 48.2±11.0 years). Among the respondents, 35.5% reported a fever of 38.5°C or higher, 41.8% hypo-anosmia, 60.0% a fever or hypo-anosmia and 17.3% a fever and hypo-anosmia.
Using the differentiated tests, 18.2% of the patients were IgM positive (95% CI=11.5-26.7) and 58.2% were IgG positive (95% CI=48.4-67.5). All IgM-positive patients were also IgG-positive, meaning the prevalence of IgM or IgG positives was 58.2% for the differentiated test and 41.2% for the combined test (95% CI=32.5-51.6).
Table 1 shows the distribution of the RDTs results. It is important to note that out of 64 patients who tested positive for IgG-IgM, 45 (70.3%) were also positive for the combined RDTs. Only one patient tested negative for the differentiated IgM-IgG test but was positive for the combined RDTs.
Results of the different RDTs.
Differentiated RDT(IgM or IgG) | Combined RDT(IgM and IgG) | ||||
---|---|---|---|---|---|
N | Positive | % | Positive | % | |
IgM | |||||
Positive | 20 | 20 | 100.0 | 19 | 95.0 |
Negative | 90 | 44 | 48.9 | 27 | 30.0 |
IgG | |||||
Positive | 64 | 64 | 100.0 | 45 | 70.3 |
Negative | 46 | 46 | 100.0 | 1 | 2.2 |
IgM or IgG | |||||
Positive | 64 | 45 | 70.3 | ||
Negative | 46 | 1 | 2.2 | ||
Combined RDT | |||||
Positive | 46 | 45 | 97.8 | ||
Negative | 64 | 1 | 2.2 |
A positive and statistically significant result was most frequently presented in both RDTs among patients with a previously positive RT-PCR, those with hypo-anosmia and fever or hypo-anosmia (table 2). The positive cases for both RDTs were older than the negatives (Combined RDT: Negative=46.2±10.9 years vs. 50.9±10.7 years; p=0.027 / Differentiated RDT: Negative=44.7±11.2 years vs. 50.7±10.2 years; p=0.005).
Distribution of the results of the RDTs, by variable.
Differentiated RDT(IgM or IgG) | Combined RDT(IgM and IgG) | ||||||
---|---|---|---|---|---|---|---|
N | Positive | % | p | Positive | % | p | |
Sex | |||||||
Men | 57 | 37 | 64.9 | 0.138 | 25 | 43.9 | 0.653 |
Women | 53 | 27 | 50.9 | 21 | 39.6 | ||
RT-PCR | |||||||
Positive | 58 | 42 | 72.4 | 0.001 | 32 | 55.2 | 0.003 |
Negative | 52 | 22 | 42.3 | 14 | 26.9 | ||
Fever | |||||||
Yes | 39 | 26 | 66.7 | 0.181 | 20 | 51.3 | 0,136 |
No | 71 | 38 | 53.5 | 26 | 36.6 | ||
Hypo-anosmia | |||||||
Yes | 46 | 37 | 80.4 | <0.001 | 27 | 58.7 | 0.002 |
No | 64 | 27 | 42.2 | 19 | 29.7 | ||
Fever or hypo-anosmia | |||||||
Yes | 66 | 49 | 74.2 | <0.001 | 36 | 54.6 | <0.001 |
No | 44 | 15 | 34.1 | 10 | 22.7 | ||
Fever and hypo-anosmia | |||||||
Yes | 19 | 14 | 73.7 | 0.132 | 11 | 57.9 | 0.118 |
No | 91 | 50 | 54.6 | 35 | 38.5 |
Table 3 shows the degree of agreement and the kappa coefficient between the two RDTs analyzed. The overall kappa coefficient was 0.646 and was higher in subjects without a fever or anosmia (0.725) and lower in those with a fever or anosmia (0.524). Based on the previous RT-PCR result, the kappa coefficient was higher in RT-PCR-negative cases (0.669) than in those that were RT-PCR-positive (0.566).
Distribution of the kappa coefficient, by variable.
Variables | N | Agreement (%) | Kappa | 95% CI |
---|---|---|---|---|
Total | 110 | 81.8 | 0.646 | 0.513-0.779 |
RT-PCR-positive | 58 | 79.3 | 0.566 | 0.363-0.770 |
RT-PCR-negative | 52 | 84.6 | 0.669 | 0.470-0.868 |
Fever or hypo-anosmia | 66 | 77.3 | 0.524 | 0.332-0.717 |
No fever and no hypo-anosmia | 44 | 88.6 | 0.725 | 0.507-0.943 |
The information provided by the company regarding the internal validity of the RDTs used in this study stated that there was 100% sensitivity in both cases and 97% specificity for RDTs, differentiating between IgG and IgM and 90% for the combined RDT.3 However, the information available from various validation studies carried out in our country expressed a specificity close to 100% for both tests and a sensitivity of 56.5% in the differentiated RDT and 63% in the combined RDT. These results were found in samples from hospitalized patients in which the evolution time of the disease was not taken into account and in about 80% of the tested patients the evolution time was more than 7 days.3
Although it is not in the scope of this article to assess the internal validity of these RDTs, in the overall analysis we are able to see that the prevalence of positive tests is higher in the differentiated RDT than in the combined RDT (58.2% vs 41.2%). This is also true for RT-PCR-positive patients (72.4% vs 55.2%). In the most recent case, the sensitivity observed by the test differentiating between immunoglobulins was higher than what the authors cited previously and lower than the combined RDT. Among other reasons, these differences can be attributed to the use of capillary blood in both RDTs. Though in both cases the sensitivity improves with the use of venous blood over capillary blood, the difference is greater in the combined tests (61.5% vs 84.5%; 74% vs 86%).
In terms of the results obtained from RDTs in RT-PCR-negative patients, it is important to note that 42.3% of the differentiated RDTs and 26.9% of the combined RDTs were positive. Although the possibility of false negatives in RT-PCR tests is known, it is remarkable that the numbers are so high and may be related to the sampling techniques instead of problems related to the technique itself or the low viral load.2 This finding highlights the necessity for the combined use of both techniques, RDTs and RT-PCRs, at least in places and professions with a high risk of SARS-CoV-2 transmission.5
According to the Fleiss classification,6 the level of agreement observed between the two RDTs is good, and in some cases, close to excellent, both globally and in the different subgroups analyzed.
The sample used in this study were patients with symptoms compatible with COVID-19 who had been tested by RT-PCR to confirm the diagnosis. In all cases, more than 14 days had passed since the onset of symptoms. Given the viral dynamics and the immune system's response, the results obtained were what was expected, both globally, in RT-PCR-positive cases7,8 and in cases with symptoms, especially fever and hypo-anosmia.9,10
The differences in positivity between IgM and IgG were also expected given the dynamics of the appearance of the various immunoglobulins, although it seems that they may appear simultaneously if there is agreement on a greater persistence of IgG in relation to IgM and therefore a greater probability of being selected.7 ELISA-based IgG and IgM seroconversion occurs in all patients between the third and fourth week of symptom onset and while IgM decreases to low levels by the fifth week and almost disappears by the seventh week, IgG continues beyond the seventh week.11
This study is not without its limitations. The study design and the sample size in particular require that the results obtained be interpreted with caution. However, the COVID-19 pandemic is a global battle, so this study provides interesting results in the RDTs analyzed that may be of special interest in the clinical practice and diagnosis of the SARS-CoV-2 infection.
ConclusionThe agreement between the two RDTs analyzed in the study is good. Both tests have a sensitivity in the range that other authors have observed, meaning they may be useful as a complement to RT-PCR.
FundingThis research has not received specific support from public sector agencies, the commercial sector or non-profit organizations.
Conflict of interestThe authors declare no conflict of interest.
The authors would like to thank all patients for their participation in this study who have voluntarily agreed to collaborate in the evaluation of diagnostic techniques against COVID-19. Thank you to the primary care management division of the Healthcare Area of Leon and the University of Leon also, because their collaboration has been crucial in developing this study.