Accuracy of three hemoglobin A1c point-of-care systems for glucose monitoring in patients with diabetes mellitus

Abstract

Objectives

Use of hemoglobin A1c point-of-care devices in physician offices provides immediate results and reduces inconveniences for the patients. We compared the analytical performances of 3 point-of-care HbA_1c analyzers to high pressure liquid chromatography (HPLC).

Material and methods

We preselected a pool of 40 EDTA-preserved whole blood samples from our laboratory with HbA1c results obtained by HPLC (mean 6.6% [49 mmol/mol] and range: 4.6–9.9% [27–87 mmol/mol]). Aliquots of theses samples were tested by Afinion AS100, DCA Vantage and In2it point-of-care systems. According the Clinical Laboratory Standards Institute EP-09 protocol we determined linearity (linear regression and correlation coefficient between point-of-care and reference methods), bias (Bland–Altman analysis) and coefficient of variation (%). We used the acceptability criteria endorsed by the National Glycohemoglobin Standardization Program.

Results

The calculated correlation coefficients (r) were 0.98, 0.98 and 0.83 for Afinion AS100, DCA Vantage and In2it systems, respectively. The 95% confidence interval of the error between point-of-care systems and the reference method was −0.41% and +0.34% (p = 0.22) for Afinion AS100, −0.62% and +0.05% (p = 0.57) for DCA Vantage, and −1.15% and +1.26% (p < 0.001) for the In2it. The coefficients of variation for Afinion AS100, DCA Vantage and In2it systems were 1.80, 3.74 and 7.14%, respectively.

Conclusion

Only the Afinion AS100 point-of-care system met all NGSP performance criteria.

Resumen

Objetivos

El uso de sistemas de determinación rápida de HbA1c proporciona resultados inmediatos y reduce inconvenientes para los pacientes. Hemos comparado las características de tres sistemas de diagnóstico rápido de HbA1c respecto al método de referencia constituido por la cromatografía líquida de alta resolución (HPLC).

Material y métodos

Preseleccionamos un total de 40 muestras de sangre conservadas en EDTA cuyos valores de HbA1c (media: 6,6% [49 mmol/mol]; rango: 4,6 a 9,9% [27–87 mmol/mol]) habían sido medidos por el método de referencia HA 8160 (Menarini Diagnostics, Akray Factory Inc. Koji, Konan-Cho, Koka Shi, Shiga, Japón). Las alícuotas de estas muestras fueron testadas con los sistemas rápidos Afinion AS100 (Axis-Shield, Oslo, Noruega), DCA Vantage System (Siemens Healthcare Diagnostics Inc. Tarrytown, NY, EE. UU.) e In2it (Bio-Rad Hercules, CA. EE. UU.). Según las recomendaciones del Clinical Laboratory Standards Institute determinamos linealidad (regresión y coeficiente de correlación), sesgo (análisis de Bland–Altman) y coeficiente variación (%). Utilizamos los criterios de aceptabilidad publicados por el National Glycohemoglobin Standardization Program.

Resultados

Los coeficientes de correlación fueron 0,98, 0,98 y 0,83 para Afinion AS100, DCA Vantage System e In2it, respectivamente. El intervalo de confianza al 95% del error (sesgo) fue en Afinion AS100 -0,41% y +0,34% (p = 0,22), en DCA Vantage System −0,62% y +0,05% (p = 0,57) y en In2it −1,15% y +1,26% (p < 0,001). El coeficiente de variación intraensayo fue 1,80, 3,74 y 7,14% para Afinion AS100, DCA Vantage System e In2it, respectivamente.

Conclusiones

Solo el sistema Afinion AS100 cumplió los 3 requerimientos de aceptabilidad.

Introduction

Glycosylated hemoglobin results from nonenzymatic binding of glucose to amino acids forming hemoglobin (Hb). In adults, hemoglobin contains 97% of hemoglobin A, while HbA2 and HbF account for the remaining 3%. HbA analysis by chromatography allows for differentiating several Hb fractions called HbA1a, HbA1b, and HbA1c, which are collectively called glycohemoglobin HbA1. Glycosylated hemoglobin A_1c levels reflect the mean glucose concentration to which red blood cells have been exposed during their lives.

The American Diabetes Association recommends that hemoglobin A_1c, as a marker of diabetes control, is tested every six months in patients who have achieved the therapeutic goals and show stable blood glucose control. By contrast, in patients in whom treatment has been changed or treatment objectives have not been achieved, hemoglobin A_1c should be measured every three months.¹

Measurement of hemoglobin A_1c at a central laboratory has as disadvantages the delay in availability of results, the cost, and the need for patient travel. Use of hemoglobin A_1c point-of-care testing systems at physician offices could provide immediate results and decrease inconvenience for patients.^2–4 Some studies have shown that availability of a method for rapid measurement of hemoglobin A1c at the office improves blood glucose control by changing the attitude to treatment of both physician and patient.⁵ Several hemoglobin A_1c point-of-care testing systems are available, but there are no sufficient studies comparing their results to those of a reference method to conclude that rapid systems provide accurate data. We therefore decided to assess three point-of-care hemoglobin A_1c analyzers as compared to the reference method.

Methods

Scope

The study was conducted at the laboratory of biochemistry of Hospital Marina Baixa, which has a level I certification as regards methods, reagents, controls, and calibrators according to the requirements proposed by the NGSP (National Glycohemoglobin Standardization Program).

Blood samples

For inter-assay evaluation, 30 blood samples preserved in ethylenediaminetetraacetic acid (EDTA) were preselected at the laboratory. Blood samples tested with the reference method for HbA1c measurement had a mean value of 6.6% (49 mmol/mol), with a range from 4.6% and 9.9% (27–85 mmol/mol). For intra-assay evaluation, a total of 10 blood samples preserved in EDTA with a value of 7.0% (53 mmol/mol) were selected. Blood samples were stored at 4 °C for less than three days until measurement using the point-of-care devices. Hemoglobin A1c results are given in NGSP units as percentages (%) and are converted into IFCC (International Federation of Clinical Chemistry and Laboratory Medicine) units⁶ as mmol/mol using the master equation IFCC-HbA1c (mmol/mol) = [DCCT-HbA1c (%) − 2.15] × 10.929. The blood samples tested had a mean urea concentration (range) of 37.7 mg/dL (20–64 mg/dL).

Reference method

HbA1c is measured at the central laboratory of our hospital using a HA 8160 analyzer (Menarini Diagnostics, Arkray Factory Inc. Koji, Konan-Cho, Koka Shi, Shiga, Japan), which uses cation exchange high-performance liquid chromatography. All test procedures in this study were performed by a single technician.

HbA1c point-of-care testing systems

The Afinion AS100 System (Axis-Shield, Olso, Norway) uses the boronate affinity method. The DCA Vantage System (Siemens Healthcare Diagnostics Inc., Tarrytown, NY, USA) uses the method of latex immunoagglutination inhibition. In2it (Bio-Rad, Hercules, CA, USA) is a system using the boronate affinity method. All HbA1c point-of-care testing systems may use both capillary blood taken with a lancet or venous blood drawn by puncture and preserved in heparin, fluoride, or EDTA as anticoagulants. With all systems, manufacturer instructions were followed for handling or reagents, capillary tubes, sample collection and preparation, and instrument calibration. Table 1 shows the main characteristics of the HbA1c point-of-care testing systems we used.

Table 1.

Summary of characteristics of HbA1c point-of-care testing systems.

HbA1c point-of-care testing system	Chemical principle of the procedure	Sample volume (μL)	Simplicity	Time to result (min)	Safety in blood sample management
Afinion AS100	Boronate affinity	1.5	Yes	3	Yes
DCA Vantage	Latex immunoagglutination inhibition	1	Yes, requires an additional step	6	Yes
In2it	Boronate affinity	10	Yes	10	Yes

HbA1c: hemoglobin A1c.

Assessment of accuracy of HbA1c point-of-care testing systems

The following analytical properties were determined according to protocol EP-09 published by the Clinical Laboratory Standards Institute (CLSI): linearity (linear regression and correlation coefficient between the reference and point-of-care methods), bias (Bland–Altman analysis), and coefficient of variation (%).⁷ To determine the coefficient of variation, 10 HbA1c repeat samples with a value of 7.0% (53 mmol/mol) were measured with each of the methods assessed. Acceptability criteria were a value of the constant in linear regression whose confidence interval included zero. A value in the Student's t test for paired samples not exceeding 4.6 (p < .01) and a 95% confidence interval (95% CI) of differences between paired sample not exceeding ±0.75% HbA1c. The following formula was used for this: total error = bias ± 1.96 × standard deviation (SD). Finally, the acceptability criterion for checking intra-assay variability was a coefficient of variation less than 3%.

Statistical analysis

Normal data distribution was verified using a Kolmogorov–Smirnov test. Statistical analysis was performed using SPSS version 15.0 statistical software (SPSS Inc., Chicago, IL, USA). Bland–Altman linear regression plots were generated using Analyze-it statistical software (Analyze-it Software Ltd., United Kingdom).

Results

Study data were normally distributed according to a Kolmogorov–Smirnov test. Mean (SD) HbA1c in NGSP units was 6.6% (0.96%) with the reference method, as compared to 6.6% (0.94%), 6.3% (0.92%), and 6.7% (1.11%) with the Afinion AS100, DCA Vantage, and In2it systems respectively. After conversion into IFCC units, results were 49 mmol/mol for the reference method, and 49 mmol/mol, 45 mmol/mol, and 50 mmol/mol for Afinion AS100, DCA Vantage, and In2it respectively (Table 2). Table 3 summarizes the main accuracy results of the three HbA1c point-of-care testing systems as compared to the reference method. For the first validation test, the constant and the slope were estimated by linear regression (Fig. 1). The Afinion AS100 and DCA Vantage systems showed a constant with confidence intervals including zero, while the In2it system showed a significant bias (constant 1.78; 95% CI: 0.71–2.85; p = 0.002). Estimated correlation coefficients (r) were 0.98, 0.98, and 0.83 for Afinion AS100, DCA Vantage, and In2it respectively. The In2it system again showed less correlation to the reference method than the other two point-of-care systems.

Table 2.

Mean hemoglobin A1c value (standard deviation [SD]) in NGSP and IFCC units found with the HbA1c point-of-care testing systems and the reference method in 40 whole blood samples preserved in EDTA.

	Reference methodHA 8160	Afinion AS100	DCA Vantage	In2it
HbA1c, mean (SD)	6.6% (0.96%)49 mmol/mol	6.6% (0.94%)49 mmol/mol	6.3% (0.92%)45 mmol/mol	6.7% (1.11%)50 mmol/mol
HbA1c, range	4.6–9.9%27–85 mmol/mol	4.4–9.7%25–83 mmol/mol	4.5–9.5%26–80 mmol/mol	4.8–10.9%29–96 mmol/mol

Conversion from NGSP units to IFCC units using the master equation IFCC = [NGSP − 2.15] × 10.929. NGSP values should range from 3% and 20%.

EDTA: ethylenediaminetetraacetic acid; IFCC: International Federation of Clinical Chemistry and Laboratory Medicine; NGSP: National Glycohemoglobin Standardization Program.

Table 3.

Evaluation of HbA1c point-of-care testing systems based on NGSP criteria.

Criterion	Afinion AS100	DCA Vantage	In2it
Constant (95% CI)	0.01 (−0.44; 0.45)	0.08 (−0.30; 0.47)	1.78 (0.71; 2.85)
Acceptable limit according to NGSP (95% CI should include “0”)	Complies	Complies	Does not comply
Slope (95% CI)	1.01 (0.94; 1.07)	1.03 (0.97; 1.09)	0.72 (0.57; 0.88)
Correlation (r)	0.98	0.98	0.83
Student's t test for paired samples	1.24	10.46	−0.62
Limit accepted by NGSP (less than 4.6)	Complies	Does not comply	Complies
Mean difference between the HbA1c point-of-care testing system and the reference method, % (95% CI)	−0.04 (−0.10; 0.02)	−0.28 (−0.34; −0.23)	0.06 (−0.14; 0.26)
Upper limit of the difference, %	+0.34	+0.05	+1.26
Lower limit of the difference, %	−0.41	−0.62	−1.15
Limit accepted by NGSP (±0.75%)	Complies	Complies	Does not comply
Coefficient of variation, %	1.80	3.74	7.14
Limit accepted by NGSP (less than 3%)	Complies	Does not comply	Does not comply

NGSP: National Glycohemoglobin Standardization Program.

Figure 1.

Linear regression showing the systems (A) Afinion AS100; (B) DCA Vantage, and (C) In2it as compared to the reference method, HA 8160 HLPC.

For the second validation test, the reference method and point-of-care testing systems were compared using a Student's t test for paired samples. Values obtained were 1.24 (p = 0.22), 10.46 (p < 0.001), and −0.58 (p = 0.56) for Afinion AS100, DCA Vantage, and In2it respectively (Table 3). Mean difference (bias) (95% CI) between point-of-care testing systems and the reference method was: −0.04% (95% CI: −0.41% and +0.34%; p = 0.22) for Afinion AS100, −0.28% (95% CI: −0.62% and +0.05%; p = 0.57) for DCA Vantage, and 0.06% (95% CI: −1.15% and +1.26%; p < 0.001) for In2it (Fig. 2).

Figure 2.

Bland–Altman plots showing agreement between: (A) Afinion AS100; (B) DCA Vantage, and (C) In2it and the reference method, HA 8160 HLPC.

For the third validation test, the coefficient of variation of each method for an HbA1c value = 7.0% (53 mmol/mol) was estimated in 10 repeat samples; coefficients of variation were 0% for the reference method, 1.80% for Afinion AS100, 3.74% for DCA Vantage, and 7.14% for In2it (Table 3). The DCA Vantage and In2it systems did not meet the strict criteria established by the NGSP on September 1, 2012.

Discussion

The accuracy of three HbA1c point-of-care testing systems for taking rapid decisions for management of diabetes mellitus at the office was assessed. The reference method was measurement of HbA1c by cation-exchange high-performance liquid chromatography at our central laboratory. Of the HbA1c point-of-care testing systems tested (Afinion AS100, DCA Vantage, In2it), only Afinion AS100 met the accuracy requirements. Our results are similar to those of other studies assessing the Afinion AS100 system^8–12 where this was consistently shown to meet all the requirements after applying the evaluation protocols proposed by the CLSI. By contrast, the DCA Vantage and In2it systems have been evaluated by other investigators with conflicting results.^13,14 This study was conducted to select the most accurate point-of-care testing system for use in patient care at the office. All systems tested are marketed, and their instruction manuals state that they meet quality standards.

According to our study, the lack of accuracy of the Afinion AS100 results in a slight underestimation of HbA1c by −0.04%. By contrast, with the DCA Vantage system there is a mean underestimation of actual HbA1c values by −0.28%, with little variability, while the In2it overestimates actual HbA1c values by +0.06% with high variability. Underestimation of HbA1c with the DCA Vantage system may have consequences in the long term due to suboptimal control of the disease. The great inaccuracy of the In2it system advises against its use in clinical practice. As regards reproducibility, the DCA Vantage and In2it systems were shown to have a significant intra-assay variability, above the levels recommended in evaluation protocols.

The importance of having an HbA1c value in real time while the patient is at the office cannot be overemphasized. In 2012, the American Diabetes Association supported use of point-of-care systems for testing HbA1c with a level of evidence “E”, corresponding to recommendation by experts.¹⁵ Use of HbA1c point-of-care analyzers may be particularly helpful in areas with low resources or at primary or specialized care clinics where patients may be evaluated immediately for potential changes in treatment. A survey to patients and healthcare professionals showed greater acceptability of and satisfaction with use of point-of-care systems for testing HbA1c in capillary blood.¹⁶ A systematic review of studies assessing the value of HbA1c point-of-care analyzers through clinical trials showed that availability of such analyzers improved treatment intensification, resulting in HbA1c reduction by 0.1–1.5% in the intervention group as compared to the standard management group.¹⁷ A subsequent systematic review and meta-analysis showed that availability of point-of-care systems had little effect on HbA1c. Mean difference in HbA1c between the intervention group, where HbA1c was measured with a point-of-care device, and the control group was −0.09% (95% CI: −0.21% to 0.02%). No significant differences were found between the groups in the proportion of patients who achieved HbA1c values less than 7% at the end of the study.¹⁸ However, both meta-analyses showed a great heterogeneity between the HbA1c point-of-care testing systems evaluated and in the study population, and their conclusions are therefore not robust. On the other hand, there is unanimity in advising against use of HbA1c point-of-care analyzers for diagnostic purposes because of their inaccuracy and the clinical implications of diagnosis of diabetes mellitus. In such cases, use of a reference method such as high-performance liquid chromatography is recommended.¹⁹

Our study results were obtained using a rigorous methodology according to CLSI recommendations and to the NGSP acceptability criteria.

As regards limitations of our study, it should be noted that patients with advanced renal failure or subjects with reduced mean red blood cell lifespan were excluded. Urea levels found in the blood samples tested were close to normal limits, with a mean value of 37.8 mg/dL. It is well known that elevated urea levels in patients with renal failure lead to formation of carbamyl-Hb. Per each mmol/L of urea in blood (2.8 mg/dL), carbamyl-Hb increases by 0.063%. Carbamyl-Hb has an isoelectric point similar to HbA1c, and may therefore cause false HbA1c elevations. This is very important in cation exchange systems because of the interference caused. However, systems using the boronate affinity method, as the ones evaluated here, are not affected by the presence of carbamyl Hb, even in uremic patients.²⁰ Carbamylation of hemoglobin does not affect the values obtained with the reference method used at the central laboratory either (Menarini HA-8160).²¹ On the other hand, it should be taken into account that mean red blood cell lifespan is decreased in patients with advanced chronic kidney failure, hemodialysis, or hemoglobinopathies, falsely reducing HbA1c readings.²⁰ Patients with advanced renal failure or on dialysis which could influence the results were excluded from our study. Presence of genetic variants of abnormal hemoglobins, which do not affect more than 1% of the Caucasian population, was not verified.²² An additional potential study limitation is that analysis of intra-assay variability was done with a single HbA1c value. Although the CLSI recommends use of two different HbA1c values, one in the low range and the other in the mid-high range, we decided to select an HbA1c value of 7.0% (53 mmol/mol) because it is clinically relevant for making decisions in clinical practice. Finally, all HbA1c measurements in our study were done on blood taken by venipuncture and preserved in EDTA, while in real life the sample used at the office is capillary blood obtained by lancet. In all HbA1c point-of-care testing systems, both capillary blood obtained with a lancet and venous blood drawn by venipuncture and preserved in EDTA, heparin, or fluoride could be used as samples. Use of a venous blood sample taken by venipuncture, as in our study, instead of capillary blood obtained with a lancet, as would occur in real life, to measure HbA1c values does not represent a significant limitation considering the high correlation existing between the HbA1c values obtained with both procedures (r = 0.987).²³

In our view, the information provided is important because we have shown that not all point-of-care analyzers marketed meet all accuracy requirements. Variations between methods may be due to the different stability of reagents or the different need for periodic calibration of components. In any case, our study showed that the Afinion AS100 system is superior to the other two point-of-care analyzers for monitoring blood glucose control in patients with diabetes mellitus at the office.

Conflicts of interest

The authors state that they have no conflicts of interest.

Acknowledgement

To Sanofi, Siemens Diagnostics, and Ferrer for lending the Afinion AS100, DCA Vantage, and In2it point-of-care analyzers respectively, as well as the corresponding reagents.