Validation of a glucose meter at an intensive care unit

Abstract

Introduction

Hyperglycemic patients admitted to intensive care units (ICUs) have higher morbidity and mortality than normoglycemic patients. Blood glucose levels of ICU patients are usually measured with a glucose meter. The aim of this study was to evaluate a glucose meter (StatStrip, Nova Biomedical) to assess its agreement with the standard laboratory method for testing glucose.

Material and methods

Eighty-nine different samples were collected from patients (76.4% men and 23.6% women) admitted to an ICU from September to December 2010. Each blood sample was collected into two tubes, a lithium heparin tube and an EDTA tube. The total blood aliquot was used to measure glycemia using the glucose meter. The lithium heparin tube was processed at the same time for measuring plasma glucose (Cobas 6000 Analyzer, Roche Diagnostics, SA). Agreement between the two methods was assessed according to the EP-9-A2 Clinical Laboratory Standards Institute guideline.

Results

Mean whole blood glucose level measured by the glucose meter was 126.53 + 49.28 mg/dL (range, 33.5–431 mg/dL), while mean plasma glucose value measured by the laboratory reference method was 138.13 + 78.6 mg/dL (range, 43–451 mg/dL). Correlation coefficient was 0.99, with a 95% confidence interval of 0.98–0.99. Coefficient of determination (R²) was 0.97, and intraclass correlation coefficient was 0.99 with a 95% CI of 0.98–0.99.

Conclusions

The tested glucose meter (StatStrip) shows a good linear association, precision, and accuracy when compared to the laboratory reference method. This device is adequate for glucose monitoring.

Resumen

Introducción

Los pacientes ingresados en las unidades de cuidados intensivos (UCI) con hiperglucemia presentan mayor morbi-mortalidad que los pacientes normoglucémicos. Habitualmente, la monitorización de la glucemia de los pacientes en las unidades de cuidados intensivos es realizado por medio de glucómetros. El objetivo del estudio fue evaluar un glucómetro (StatStrip, Nova Biomedical) para determinar su grado de acuerdo con el método habitual de determinación de la glucemia en el laboratorio.

Material y métodos

Se recogieron 89 muestras de diferentes pacientes (76,4% hombres y 23,6% mujeres) ingresados en una UCI durante los meses de septiembre a diciembre del 2010. En cada extracción, se recogió un tubo de heparina litio y otro tubo de EDTA. La alícuota de sangre total era utilizada para la determinación de glucemia mediante el glucómetro. El tubo de heparina litio era procesado a la misma vez para la determinación de la glucemia plasmática (Analizador Cobas 6000, Roche Diagnostic, SA). Para evaluar el grado de acuerdo entre los dos métodos, seguimos el procedimiento indicado en la guía EP-9-A2 del Clinical and Laboratory Standards Institute (CLSI).

Resultado

La glucemia en sangre total medida por el glucómetro presentaba un valor medio de 126,53 ± 49,28 mg/dL con un rango de 33,5 a 431 mg/dL y la glucemia plasmática del método de laboratorio reflejaba un valor medio de 138,13 ± 78,6 mg/dL con un rango de 43–451 mg/dL. El coeficiente de correlación entre ambos métodos fue de 0,99 con un intervalo de confianza al 95% (IC) de 0,98 a 0,99; el coeficiente de determinación (R²) fue de 0,97 y el coeficiente de correlación intraclase fue 0,99 con un IC de 0,98 a 0,99.

Conclusiones

El glucosímetro evaluado (StatStrip) presenta una buena asociación lineal, precisión y exactitud, cuando es comparado con el método de referencia del laboratorio clínico. Es un dispositivo adecuado para la monitorización de la glucosa.

Blood glucose changes are among the most common metabolic changes in both diabetic and non-diabetic inpatients. Patients admitted to intensive care units (ICUs) have increased insulin requirements because of pain, trauma, surgery, sepsis, hypoxia, burns, cardiovascular changes, psychic stress, drugs, the administration of glucose solutions, and so on.¹ These patients experience a number of metabolic and circulatory changes defined as systemic inflammatory response syndrome (SIRS). SIRS induces the activation of a number of neuroendocrine and inflammatory mediators (cortisol, glucagon, growth hormone, catecholamines, glucocorticoids, and cytokines such as interleukin-1, inteleukin-6, and tumor necrosis factor alpha) which increase hepatic gluconeogenesis and peripheral insulin resistance, causing so-called stress hyperglycemia.

Until recently, stress hyperglycemia was thought to provide adequate glucose levels for the brain, skeletal muscle, myocardium, and other vital organs in conditions where glucose demand was increased, but it has been found to increase infections and morbidity and mortality as the result of increased oxidative damage and the enhancement of proinflammatory response, amongst other actions.²

Hyperglycemia can have severe consequences: it can increase cerebral ischemia, delay wound healing, increase the frequency of infections, and worsen the prognosis of primary disease.^3,4 In addition, transient hyperglycemia may cause water and electrolyte changes, dehydration (osmotic diuresis), and lactic acidosis, decrease brain flow, impair mental status, delay wound healing, delay gastric emptying, decrease drug elimination (particularly narcotics), impair WBC function, increase frequency of bacteremia and fungemia, and decrease implantation of skin grafts in patients with burns.^5–8

Patients with abnormal blood glucose values admitted to ICU have a higher mortality rate than normoglycemic patients.⁹ All studies recommend treatment of hyperglycemia (blood glucose > 140 mg/dL), but no agreement exists as to the criteria and the type of monitoring required in these patients. Today, most physicians consider hypoglycemia more serious than hyperglycemia.

Van den Berghe et al. showed that strict monitoring reduced morbidity and mortality rates in patients admitted to ICU.^10,11 Blood glucose monitoring is a routine practice at these units. However, recently reported studies state that monitoring causes a significant increase in the risk of hypoglycemia with increased morbidity and mortality.^12–14 Blood glucose levels are usually measured with glucose meters at ICUs and other hospital units. Glucose meters are widely used because they provide rapid results, avoid repeated blood sampling, and only very small sample volumes are required.

Glucose meters may give false readings due to various reported interferences (including drugs, maltose, galactose, hematocrit, ascorbic acid, xylose, uric acid, oxygen, and bilirubin, amongst others),^15–18 which often occur at ICUs. An assessment of the analytical accuracy and precision of glucose meters used at the different hospital units is, therefore, required. A comparison of their results to those of the routinely used laboratory method is also recommended.

The EP-9-A2 consensus guideline of the Clinical and Laboratory Standards Institute (CLSI) allows for the assessment and comparison of two methods that measure the same analyte using patient samples, provided that one of the methods is the reference or standard laboratory method.¹⁹ The purpose of this study was to assess the StatStrip glucose meter (Nova Biomedical, Boston, USA) in order to measure its agreement with the standard blood glucose measurement method used at the laboratory of our hospital using the CLSI EP-9-A2 consensus guideline.

Materials and methods

Eighty-nine samples were collected from different patients (77.6% males and 22.4% females) admitted to the ICU of Juan Ramón Jiménez Hospital from September to December 2010. A lithium heparin and an EDTA tube were collected at each sampling. Hematocrit was measured in the EDTA tube using a Sysmex blood cell counter (Roche Diagnostics, Mannheim, Germany).

The StatStrip glucose meter uses a procedure based on a strip consisting of four layers with different functions: a first isolating layer guarantees the preservation of the lower layers; a second layer distributes the sample (50 μL) to the third layer; a third layer contains four wells (12 μL in total) in which measurements are made of the total glucose (first well), the amount attributable to interferences (second well), and the hematocrit (third well). Adequate sample volume is controlled in the fourth well. The last layer is used for glucose meter control and calibration. This consists of a gold layer that confers stability in all environmental conditions.

Blood glucose was measured in the lithium heparin tube according to the protocol below. Upon sample reception, a 1 mL aliquot was separated and labeled with the same number as the sample. The tube was closed again and centrifuged at 3500 rpm for 10 min. The whole blood aliquot was used to measure blood glucose (in duplicate) using the glucose meter.

Blood glucose measurement in whole blood (aliquot) with the glucose meter and the measurement of plasma glucose in a Cobas 6000 analyzer (Roche Diagnostics) by the hexokinase method (in duplicate) were done at the same time. The reference method was the measurement of glucose using the hexokinase method routinely used at our center for measuring blood glucose levels.

The procedure indicated in the EP-9-A2 guideline was used for the statistical analysis of the results as summarized below:

• 1.

  Verification of the normality of the tested parameters, for which a Kolmogorov–Smirnov statistical test was used.

• 2.

  The detection of outliers. Absolute differences between duplicates of each method were compared. These differences were to be not greater than 4 times the mean of the absolute differences. No outliers were found in this study.

• 3.

  Charts for comparing the linear relationship between the values of both methods. Four scatter plots were used employing equal scales:

  • a.

    Between the mean values of both methods.

  • b.

    Between the individual Y values and the mean X values.

  • c.

    Between the differences between mean Y and mean X for each method versus the sum of mean Y and X values divided by 2.

  • d.

    Between the difference between each individual Y and X value versus the sum of mean Y and X values divided by 2.

• 4.

  These plots allowed for verifying whether a linear relationship existed, and whether systematic or random error existed between both the methods studied.

• 5.

  The correlation coefficient. If calculated r was 0.975 or greater or the coefficient of determination was 0.95 or greater, it was considered that the variability of X was acceptable, and that the potential measurement error of X was compensated for by the wide range of X values.

• 6.

  The performance of Bland–Altman plots. These allowed for determining whether significant differences existed between the two measurement techniques studied and if constant scatter existed. If constant scatter existed, a regression analysis comparing both techniques was to be made.

• 7.

  The estimate of predicted systematic error and its confidence intervals (CI).

• 8.

  Intraclass correlation coefficient. Agreement was considered to be very good when this coefficient was greater than 0.90.²⁰

MedCalc 11.4 software was used for statistical analysis.

Results

A total of 89 patients with a mean age of 57.4 ± 14.8 years (mean and standard deviation) were enrolled into the study. Table 1 shows other characteristics of the study population. Most admissions were for major surgery (28.4%). Of these patents, 38.8% had diabetes mellitus and 89.5% were receiving multiple drugs.

Table 1.

Characteristics of study patients.

		Sex
		Male No.	Female No.
Associated conditions
Diabetes mellitus	39.33%	23	12
Multiple drugs	89.9%	62	18
Reason for admission
Traffic accident	23.6%	15	3
Major surgery	28.1%	17	7
Stroke	9%	11	1
Severe respiratory insufficiency	21.4%	13	4
Other	17.9%	9	6

SD: standard deviation.

Most patients had low hematocrit values (mean ± standard deviation, 33.13 ± 6.56%; range, 18–55). Blood glucose (mean ± standard deviation) in whole blood measured by the glucose meter was 126.5 ± 49.28 mg/dL (range, 33.5–431 mg/dL), and laboratory blood glucose (Cobas 6000) 138.13 ± 49.28 mg/dL (range, 43–451). Coefficients of variation (CVs) were 3.57% for the glucose meter and 1.59% for the Cobas method. No outliers were found in the tested parameters of whole blood glucose measured with the glucose meter and plasma glucose measured with the reference method.

As regards linearity and constant scatter, both measurement methods followed a linear relationship, as they showed a Pearson's correlation coefficient >0.975. This coefficient of determination was greater than 0.95 with constant scatter, and linear regression could therefore be used to verify linearity of the measurement techniques tested (Fig. 1).

Figure 1.

Regression line between Cobas 6000 and StatStrip.

The variables tested, whole blood glucose using a glucose meter and plasma glucose using Cobas 6000, followed a normal distribution (p = 0.310 and p = 0.167 respectively).

The statistical parameters calculated included: (1) correlation coefficient: 0.99 (95% confidence interval (CI): 0.98–0.99); (2) regression line: Cobas 6000 blood glucose (y) = 1.67 + 1.03 × Cobas b221 blood glucose (X); (3) coefficient of determination (R²): 0.97; and (3) intraclass correlation coefficient: 0.99 (CI: 0.98–0.99).

An analysis was also made using Bland–Altman plots to verify that there were no significant differences between the pairs of points between the two types of equipment being compared (Fig. 2). It could be shown that 5.2% of data exceeded two standard deviations.

Figure 2.

Bland–Altman plot between Cobas 6000 and StatStrip.

Table 2 shows the quality specifications of the different international organizations. Table 3 gives the results achieved with the glucose meter based on the quality objectives established by these international organizations.

Table 2.

Quality objectives according to different international organizations.

CLIA	Values obtained ±10% or 5.4 mg/dL as compared to values with standard method
CLSI	100% values ± 14.7 mg/dL for values less than 75 mg/dL
	100% values ± 15% for values ≥75 mg/dL
ISO 15197	95% values ± 15% for values less than 75 mg/dL
	100% values ± 20% for values ≥75 mg/dL
ADA	100% values ± 5% from the reference method value

ADA: American Diabetes Association; CLIA: Clinical Laboratory Improvement Amendments; CLSI: Clinical and Laboratory Standards Institute; ISO: International Organization for Standardization.

Table 3.

Values obtained with the glucose meter assessed as a function of the quality objectives of the different international organizations.

	<75 mg/dL (21 samples)								≥75 mg/dL (68 samples)
	0–5.0%		5.1–8.0%		8.1–10.0%		>10.0%		0–5%		5–10%		10.1–15%		>15%
	No.	%	No.	%	No.	%	No.	%	No.	%	No.	%	No.	%	No.	%
ADA	15	71.43	4	19.04	2	9.52	0	0	51	73.9	9	13	7	10	1	1

	<5.40 mg/dL		5.40–10.85		10.86–14.7		>14.7 mg/dL		<5%		5.1–10%		10.1–15%		>15%
	No.	%	No.	%	No.	%	No.	%	No.	%	No.	%	No.	%	No.	%
CLSI	15	71.43	4	19.04	2	9.52	0	0	51	73.9	9	13	7	10	1	1

	0–5%		5.1–15%		15.1–20%		>20%		0–10%		10.1–20%		20–30%		>30%
	No.	%	No.	%	No.	%	No.	%	No.	%	No.	%	No.	%	No.	%
ISO	15	71.43	6	28.57	0	0	0	0	60	88.2	7	10	1	1.5	0	0

	0–5.4 mg/dL		>5.4 mg/dL		0–5%		5.1–10%		>10%
	No.	%	No.	%	No.	%	No.	%	No.	%
CLIA	63	70.79	26	29.21	66	74.2	15	16.9	8	8.99

ADA: American Diabetes Association; CLIA: Clinical Laboratory Improvement Amendments; CLSI: Clinical and Laboratory Standards Institute; ISO International Organization for Standardization; No.: number of samples.

The most interesting results given in Tables 2 and 3 are discussed below. Using the CLSI criterion, 100% of values obtained with the glucose meter were within ±14.7 mg/dL of the values obtained with the reference method for blood glucose values less than 75 mg/dL, and 96.01% were within 15% of values of the reference method for concentrations of 75 mg/dL or more. A single sample was 21.2% higher in the glucose meter as compared to the reference method. Using the criterion of standard 15197 of the International Organization for Standardization (ISO), 100% of the values obtained with the glucose meter were within ±15% for values less than 75 mg/dL, and 100% were within ±20% for values of 75 mg/dL or more. For the criteria of the American Diabetes Association (ADA), only 60.6% of values found with the glucose meter met the goal of ±5% as compared to the values obtained with the reference method. Finally, using the criterion of the Clinical Laboratory Improvement Amendments (CLIA), 91.01% of the values met the goal of ±5.4 mg/dL or 10% as compared to the values obtained with the reference method.

Discussion

The use of glucose meters at hospitals for measuring blood glucose in diabetics is very common. The main problems of such measurements include their accuracy and interference by hematocrit, very high protein levels, and the use of certain drugs. The use of glucose meters at the different units requires a verification of agreement with the routine laboratory method. Different authors have assessed the reliability of different glucose meters and have compared these to the standard laboratory methods.^16,21

The EP-9-A2 was used for method validation. This guideline may appear complex, but is easy to standardize when used. The procedures used by this guideline do not correspond to the classical statistical methods for method validation, but because of its effectiveness as a validation method, it has been proposed as a validation tool of both accredited laboratories with a flexible scope and for method validation of laboratories aiming at accreditation.²²

In this study, the Nova StatStrip glucose meter showed a good linearity within the studied range and a coefficient of variation of 3.57%. Skeie et al. recommended a maximum coefficient of variation of 5%.¹⁶ The comparison of the StatStrip glucose meter and the reference laboratory method (Cobas 6000) showed a very high correlation, with a slope close to 1 and an intercept close to 0.

The best parameter for assessing the agreement of a quantitative variable using two different instruments is intraclass correlation coefficient (ICC). Agreement is considered to be very good when ICC is >0.90.²⁰ In this study, the glucose meter showed a very good agreement with the reference method (ICC = 0.99). The glucose meter provided glucose values slightly lower than those found with the reference laboratory method, with a mean difference of 5.9 mg/dL, similar to that reported by other authors.^16,24,25 This difference may be explained by the samples used, because glucose concentration in whole blood was approximately 10–15% lower than plasma glucose levels.²⁴

In any case, the compliance of glucose meters with the established quality recommendations is required. With regard to CLSI and ISO 15197:2003 criteria,^22,23 the results of this study showed that 100% of glucose meter readings were within ±14.7 mg/dL of those obtained with the reference method for values less than 75 mg/dL. 96.01% of readings were within 15% of Cobas 6000 values for concentrations higher than 75 mg/dL. A 21.2% higher value was found with StatStrip as compared to Cobas 6000 in a single sample. The glucose meter assessed therefore met both the CLSI and the ISO criteria.

Another criterion for assessing the precision of evaluation was proposed by the ADA, which recommended that meter readings should be within ±5% of the reference method.¹⁶ However, no currently available glucose meter has achieved this goal.^16,25,26 The proportion of measurements meeting this requirement was 60.5%, and was lower in some reports and higher in others.^16,21,25,26 This suggests that StatStrip is an improved method but that it does not meet the ADA requirements.

According to the CLIA, the results should be within 10% of the results of the reference method or ±5.4 mg/dL. In our study, 91.0% of the results were within that limit.

Poirier et al.²⁷ classified glucose meters as good, acceptable, and unacceptable for clinical use. A glucose meter is defined as good when 60% of its results are within ±10% of the reference method values. The glucose meter assessed in this study may, therefore, be classified as good.

A variety of substances cause interference with glucose meters.^16,28–31 The main such substances are acetaminophen, ascorbic acid, and maltose. Other authors found that variations in hematocrit levels may affect glucose meter measurements.^30,32 This interference occurs with some meters, even when the manufacturer's recommendations are followed.³³ Different studies have reported that the above substances do not interfere with the StatStrip glucose meter, but most such studies were conducted in vitro. Although an interference study was not made in our study, the occurrence of interferences, particularly the most common ones,^16,28,30 may be shown, because the characteristics of the patients admitted to ICUs (multiple drug treatment, low hematocrit, severe diseases, various treatments, etc.) may have caused the appearance of such interferences. In this study, no significant interferences or accuracy losses were found.

The glucose meter assessed (Nova StatStrip) showed a good linearity, precision, and correlation as compared to the reference method of the clinical laboratory. The StatStrip meets the quality requirements of all the different international organizations, except for those of the ADA. However, there is currently no glucose meter that meets the requirements set by this international organization. It may, therefore, be concluded that this glucose meter is an adequate device for blood glucose monitoring in patients admitted to ICUs.

Conflicts of interest

The authors state that they have no conflicts of interest.