Accuracy and precision of online quantitative coronary angiography with automatic calibration: a pilot study

Abstract

Background

The objective of this study was to evaluate the degree of accuracy, precision, correlation, and agreement between the measurements performed by online Quantitative Coronary Angiography (QCA) software with automatic calibration.

Methods

Pilot study that analyzed angiographic images of ten patients through online QCA software using Auto ISO (automatic calibration isocenter) and Auto TOD (Table-to-Object Distance) automatic calibration. Catheter size was measured by both methods and the reference diameter was computed. These measurements were compared with the measurement of catheter diameter regarding accuracy, precision, and agreement.

Results

The actual average of the catheter diameter was 1.75 ± 0.32 mm (range 1.33 to 2.67 mm). The measurement of catheters by TOD QCA and ISO QCA resulted in mean diameters of 1.78 ± 0.37 mm and 1.88 ± 0.38 mm, respectively. The accuracy/precision of the TOD QCA and the ISO QCA was 0.03 mm/0.21 mm and 0.12 mm/0.20 mm, respectively. The TOD QCA and ISO QCA measures were among the limits of agreement in 96.3 and 94.7% of cases, respectively, and were significantly correlated (rs = 0.93, p < 0.01). However, despite the small difference between the methods (0.10 ± 0.10 mm), the ISO QCA measures were significantly higher than those obtained by the TOD QCA (p < 0.01).

Conclusions

Online QCA with automatic calibration has good accuracy, precision, and correlation, which may represent a promising tool in the catheterization laboratory.

Resumo

Introdução

O objetivo deste estudo foi avaliar o grau de acurácia, precisão, correlação e concordância entre as medidas realizadas por um software de angiografia coronária quantitativa (QCA) on-line de calibração automática.

Métodos

Estudo piloto que analisou imagens angiográficas de dez pacientes com um software de QCA on-line nas calibrações automáticas Auto ISO (calibração automática do isocentro) e Auto TOD (Table-to-Object Distance). Foi realizada a medida do calibre do cateter pelos dois métodos, e o diâmetro de referência foi computado. Essas medidas foram comparadas com a medida do diâmetro do cateter quanto à acurácia, precisão e concordância.

Resultados

O diâmetro médio real dos cateteres era de 1,75 ± 0,32 mm (variando entre 1,33 e 2,67 mm). A aferição dos cateteres pela QCA TOD e pela QCA ISO resultou em diâmetros médios de 1,78 ± 0,37 mm e 1,88 ± 0,38 mm, respectivamente. A acurácia/precisão da QCA TOD e da QCA ISO foi 0,03 mm/0,21 mm e 0,12 mm/0,20 mm, respectivamente. As medidas da QCA TOD e da QCA ISO estiveram entre os limites de concordância em 96,3 e 94,7% dos casos, respectivamente. As medidas da QCA TOD e da QCA ISO correlacionaram-se significativamente (r_s = 0,93; p < 0,01). No entanto, apesar da pequena diferença entre os métodos (0,10 ± 0,10 mm), as medidas da QCA ISO foram estatisticamente maiores que aquelas obtidas com a QCA TOD (p < 0,01).

Conclusões

A QCA on-line com calibração automática apresentou boa acurácia, precisão e correlação, podendo representar uma ferramenta promissora no laboratório de hemodinâmica.

Introduction

Clinical outcomes after percutaneous coronary intervention (PCI), both in the short and long-term, are directly associated with the interventional strategy used during the procedure. In particular, the appropriate choice of the dimensions of the devices used during the procedure is one of the most important factors that modulate post-intervention evolution. An oversized stent may cause coronary dissection or perforation, whereas an undersized stent may result in incomplete apposition of stent struts on the artery wall and inadequate expansion, increasing the risk of thrombosis and restenosis.^1,2

Moreover, with the arrival and incorporation of bioresorbable vascular scaffolds in daily clinical practice, the assessment of the appropriate size has become of major importance, as these devices have some peculiarities in their implantation. The choice of diameter should maintain a close association with the arterial caliber and its post-dilation should not exceed 0.5 mm in diameter, as there may be damage to its structure.

Online quantitative coronary angiography (QCA) is widely available in most modern angiographic services. It allows for the quantification of vessel size and lesion length, and can be used as a tool to guide the interventional technique during PCI.

Usually, QCA software utilizes measurement calibration by comparing it with an object of known dimension. The present study evaluated the diagnostic performance of online QCA software with automatic calibration, which eliminates the need for manual image calibration.

Methods

This pilot study evaluated the angiographic images of ten randomly selected patients who underwent diagnostic and therapeutic coronary procedures in the Hemodynamics Service of Hospital Sírio-Libanês, located in São Paulo, SP, Brazil.

The angiographic images, performed in different angles and field magnifications, were analyzed using the online QCA software Artis Zee syngo QCA (Siemens AG, Munich, Germany). This program has two automatic calibration options: (1) Auto ISO (automatic calibration of the isocenter), in which the system calculates the calibration factor based on the image geometry; the organ to be evaluated must be in the isocenter during image acquisition; and (2) Auto TOD (Table-to-Object distance), in which the system calculates the calibration factor according to the distance between the table and the center of the organ to be evaluated. For the calibration, the point of interest of the image must be established.

To evaluate the diagnostic performance of Auto ISO and Auto TOD QCA methods, both techniques were applied to measure the diameter of the catheter used during the procedure, which had known dimensions and against which the measurement results were compared. A measurement of at least 10 mm of the catheter segment in the same plane was standardized, without image shortening. The reference diameter automatically generated by the two automatic calibration methods was computed and compared to the catheter diameter. Accuracy was defined as the mean of the differences between the catheter measurements and those of the QCA. Precision was defined as the standard deviation (SD) of the differences between these measurements.

For the analysis of the differences between the measurements, the Wilcoxon signed rank test was used. Spearman's correlation coefficient (r_s) was used to evaluate the correlation. The agreement degree between measurements was observed according to the method suggested by Bland and Altman,³ in which the mean of the two individual measurements and the difference between them were graphically plotted and the mean(bias) was calculated, as well as the SD of the difference between measurements, for the calculation of the agreement limits (bias ± 1.96 SD).

The statistical significance limit was set at p < 0.05 and the confidence interval at 95%. Analyses were performed using SPSS version 21.0 (SPSS Inc., Chicago, USA).

Results

Analysis of TOD QCA and ISO QCA

A total of 107 angiographic images of 10 patients were analyzed in different projections. The measurements of QCA methods with automatic calibration were compared against the standard of coronary catheters 4F (17.8%), 5F (57%), 6F (9.3%), 7F (14%), and 8F (1.9%). In general, the catheters were diagnostic – for instance, non-guide catheter (74.8%) filled with contrast medium (88.8%). Regarding image acquisition, 94.4% were in field 22, with a mean angle of 27.5° ± 16.2° in the coronal axis, and 20.6° ± 14.4° in the sagittal axis.

The actual mean diameter of the catheters was 1.75 ± 0.32 mm (ranging from 1.33 to 2.67 mm). The measurement of catheters by TOD QCA and ISO QCA resulted in mean diameters of 1.78 ± 0.37 mm and 1.88 ± 0.38 mm, respectively. The accuracy (mean difference between the measured and actual values) and precision (SD of the differences between the measured and actual values) of TOD QCA and ISO QCA are shown in Table 1.

Table 1.

Accuracy and precision of ISO vs. TOD quantitative coronary angiography (QCA).

Measurements	Accuracy (mm)	Precision (mm)
QCA with TOD automatic calibration	0.03	0.21
QCA with ISO automatic calibration	0.12	0.20

After the determination of the respective upper and lower limits of agreement (Bland-Altman method), it was verified that the measurements of TOD QCA and ISO QCA were among the limits of agreement in 96.3% and 94.7% of cases, respectively.

When compared among them, the TOD and ISO reference diameter measures were significantly correlated (r_s = 0.925, 95%CI 0.839-0.956; p < 0.01) (Fig. 1). However, despite the small difference between the methods (0.10 ± 0.10 mm), ISO QCA measures were significantly higher than those obtained by TOD QCA (p < 0.01).

Figure 1.

Scatter plot showing the correlation between the measurements by automatic calibration isocenter (ISO) vs. automatic Table-to-Object Distance (TOD) quantitative coronary angiography (QCA).

Intraobserver analysis

For the intraobserver analysis, 30 angiographic images of the total sample were randomly selected and a new measurement was performed of TOD and ISO reference diameters in the same projection 2 days after the initial measure. The difference between the two measures of TOD reference diameter was 0.01 mm (accuracy) with SD of 0.11 mm (precision). The accuracy between the measurements of ISO reference diameter was –0.01 mm, and the precision was 0.14 mm. There was a significant correlation between the measurements of the first and second evaluation of TOD reference diameter (r_s = 0.899, p < 0.01) and ISO reference diameter (r_s = 0.861, p < 0.01) by the same observer. There was no statistically significant difference between measures of TOD reference diameter in the first (median 1.75 mm) and second evaluations (median 1.72 mm), with z = –0.540, p = 0.62, nor between the ISO reference diameter in the first (median: 1.81 mm) and second evaluations (median: 1.77 mm), with z = –0.148, p = 0.82.

Discussion

These results suggest that the assessed QCA methods using TOD and ISO automatic calibration have good accuracy and precision in absolute numbers, as well as a good correlation when used to assess measurements that ranged from 1.33 to 2.67 mm in diameter.

TOD calibration showed better accuracy, correlation, and agreement, although ISO calibration overestimated the catheter size by only approximately 0.1 mm. One potential reason for ISO to have shown this difference was the fact that, in practice, the catheter is not filmed in an isocentric position, thus increasing the margin of error. A deviation of 1 cm in the isocenter results in 1.5% error.

Both calibration methods showed good intraobserver correlation, with no statistically significant differences. The accuracy and precision of measurement reproducibility in this study were similar to those found in other studies, ranging from –0.02 to 0.03 mm and 0.18 to 0.29 mm, respectively.^4,5

Study limitations

The main limitations of this pilot study were the small number of patients and angiographic analysis, and use of the catheter as reference for comparison with the QCA methods (as the catheter has smaller diameters than the coronary arteries, commonly treated with stents and devices, which range from 2.25 to 4.5 mm in diameter). Moreover, the possible factors that can increase or decrease the correlation between the methods were not assessed, such as patient chest thickness and the angle used for image acquisition, among others. There was no interobserver evaluation.

Conclusions

On-line QCA with automatic calibration has good accuracy, precision, and correlation, which may represent a promising tool in the catheterization laboratory. However, larger and more robust studies are necessary to assess arterial caliber and length, featuring more angiographic images, in order to correct possible automatic calibration biases.

Funding sources

None declared.

Conflicts of interest

The authors declare no conflicts of interest.