Diagnostic performance of the American Thyroid Association ultrasound risk assessment of thyroid nodules in endocrinology (the ETIEN 3 study)

Abstract

Introduction

No study has been published to date in Spain about the value of the American Thyroid Association (ATA) ultrasound risk assessment of thyroid nodules applied by endocrinologists.

Objectives

To assess the diagnostic performance of ATA thyroid nodule risk classification applied by endocrinologists with respect to histological results.

Methods

A retrospective, observational study of 317 patients (mean age, 51.7 ± 13.7 years; 83.3% women) with thyroid nodules (maximum diameter: 3.2 ± 1.4 cm) who underwent US-guided FNA in endocrinology between October 2015 and December 2018, were classified based on the ATA ultrasound risk assessment. Surgery was performed in all of them. Quality criteria of the ultrasound classification were assessed as compared to histological results.

Results

Overall, 61.2% of nodules assessed were classified as benign (n = 3) and very low (n = 60) or low suspicion (n = 131), 11.7% as intermediate suspicion (n = 37), and 27.1% as high suspicion (n = 86). Benign nodular disease was found in 260 patients, and thyroid cancer in 57 patients. There were 14 incidental papillary microcarcinomas. Classification of thyroid nodules as high suspicion showed a high diagnostic performance to identify malignant nodules (87.7% sensitivity, 86.2% specificity, negative predictive value 97.0%), improving sensitivity (93.0%) and reducing specificity (73.1%) when considering high and intermediate suspicion nodules as a risk for thyroid cancer.

Conclusions

Application by endocrinologists of the ATA ultrasound risk assessment of thyroid nodules shows a high diagnostic performance to identify malignant thyroid nodules before surgery.

Resumen

Introducción

Hasta la actualidad no se ha publicado en nuestro país ningún estudio sobre la utilidad del sistema de evaluación del riesgo ecográfico del nódulo tiroideo de la American Thyroid Association (ATA) aplicada por endocrinólogos.

Objetivos

Evaluar el rendimiento diagnóstico de la clasificación ATA del nódulo tiroideo aplicada por endocrinólogos respecto a los resultados histológicos.

Metodología

Estudio observacional retrospectivo de 317 pacientes (51,7 ± 13,7 años de edad media; 83,3% mujeres) con nódulos tiroideos (diámetro máximo: 3,2 ± 1,4 cm) evaluados en consulta de eco-PAAF de endocrinología entre octubre de 2015 y diciembre de 2018, a los cuales se les aplicó la clasificación ATA y que fueron intervenidos. Se evalúan los criterios de calidad de la clasificación ecográfica prequirúrgica respecto a los resultados histológicos.

Resultados

El 61,2% de los nódulos evaluados fueron clasificados como benignos (n = 3), de muy baja (n = 60) o de baja sospecha (n = 131), el 11,7% como sospecha intermedia (n = 37) y el 27,1% como alta sospecha (n = 86). Se detectaron 260 casos de enfermedad nodular benigna y 57 cánceres de tiroides. Hubo 14 microcarcinomas papilares incidentales. La clasificación de nódulos de alta sospecha presentó un elevado rendimiento diagnóstico para identificar nódulos malignos (sensibilidad 87,7%, especificidad 86,2% y valor predictivo negativo 97,0%), mejorando la sensibilidad (93,0%) y reduciéndose la especificidad (73,1%) al considerar de riesgo los nódulos de sospecha alta e intermedia.

Conclusiones

La aplicación por endocrinólogos del sistema de evaluación de riesgo ecográfico del nódulo tiroideo de la ATA presenta un elevado rendimiento diagnóstico para identificar prequirúrgicamente nódulos tiroideos malignos.

Introduction

Ultrasound is the most important imaging technique for assessing nodular thyroid disease, since it is easily accessible, noninvasive, cost-effective, and predictive of the risk of thyroid nodule malignancy, making it possible to select those nodules that should be evaluated by fine needle aspiration biopsy (FNAB).^1,2 Ultrasound has evolved enormously in recent years thanks to the introduction of increasingly smaller and more manageable equipment with improved imaging quality and definition. This has made it increasingly possible to transfer ultrasound exploration from the Radiology Department to other units such as the Endocrinology Clinic.^3–9 In recent years, a number of studies in Spain have shown the integration of clinical evaluation, thyroid ultrasound and thyroid FNAB in single-intervention or high-resolution consultations conducted by specialists in endocrinology to be a cost-effective strategy,^5–7 with very good acceptance on the part of the patients.⁷ This strategy makes it possible to reduce the patient study time to thyroidectomy in cases of benign nodular disease or suspected malignancy.^5,8 Following a suitable learning period, a high diagnostic yield is achieved with punctures of both thyroid lesions and suspect adenopathies.^3,9

Different classifications have been developed for assessing the ultrasound malignancy risk of thyroid nodules.^10–15 These classifications have demonstrated their usefulness (and limitations) both in assessing thyroid nodule malignancy risk and in establishing indications of FNAB,^16–22 and have even contributed to reducing the number of unnecessary puncture procedures.¹⁷ The ultrasound risk assessment of thyroid nodules proposed by the American Thyroid Association (ATA) in late 2015 established 5 ultrasound risk categories: benign, very low suspicion, low suspicion, intermediate suspicion and high suspicion. Many studies have demonstrated the usefulness of this classification in identifying malignant nodules.^18–22 However, to date no studies have examined the usefulness of this ultrasound risk assessment tool in patients with thyroid nodules seen in single-intervention or high-resolution clinics in Spanish Endocrinology Units.

Since January 2013, our Unit has had both a high-resolution thyroid disease clinic in which single-intervention ultrasound is performed, and a thyroid ultrasound-FNAB clinic where thyroid nodule punctures are made. Since late 2015, both the ultrasound risk classification of thyroid nodules and the indications of FNAB have been adapted in our protocol to the criteria of the ATA. The primary objective of the present study (ETIEN 3 study) was to assess the diagnostic performance of the ATA ultrasound risk assessment of thyroid nodule malignancy applied by specialists in endocrinology, compared with the definitive histological findings.

Material and methods

A retrospective observational study was conducted at the Endocrinology Unit of Puerto Real University Hospital (Cádiz, Spain), involving the clinical, ultrasound and histological data of 317 patients subjected to thyroidectomy at our center, and who had been previously seen in the thyroid ultrasound-FNAB clinic between October 2015 and December 2018. The study was approved by the Research Ethics Committee of Cádiz in April 2018, and no informed consent was required to access the study information. However, all patients undergoing thyroid ultrasound-FNAB gave written consent before the procedure.

Evaluation of thyroid nodules

Since 2010, the diagnostic and therapeutic approach to thyroid nodules has been protocolized in coordination with the Units of Endocrinology, Radiodiagnosis, General Surgery and Pathology. In January 2013, thyroid ultrasound and ultrasound-FNAB were added to the endocrinology services portfolio. At present, all thyroid ultrasound explorations are made as a single intervention at any of our clinics (both at the monographic thyroid disease clinic and at all other clinics) by all the endocrinologists of our Unit. Patients eligible for puncture are subsequently seen at the ultrasound-FNAB clinic. This strategy has shown its efficiency in terms of cost savings in patients with thyroid disease (the ETIEN 1 study)⁶ and its effectiveness in improving the study or workup times in patients subjected to thyroidectomy (the ETIEN 2 study).⁸ In October 2015, our Unit adopted both the criteria proposed by the ATA for the ultrasound risk assessment of thyroid nodule malignancy and the thyroid FNAB indications recommended by the ATA, namely: (1) benign nodule: a purely cystic nodule with indication of FNAB in the event of discomfort or esthetic concerns; (2) very low suspicion nodule: a spongiform or partially cystic nodule without microcalcifications, irregular margins or spread beyond the thyroid, and measuring no taller than wide, with an indication of FNAB >2 cm in size; (3) low suspicion nodule: an isoechogenic or hyperechogenic solid nodule and partially cystic nodule with eccentric solid areas in the absence of microcalcifications, irregular margins or spread beyond the thyroid, and measuring no taller than wide, with an indication of FNAB >1.5 cm in size; (4) intermediate suspicion nodule: a hypoechogenic solid nodule with regular margins, no microcalcifications or spread beyond the thyroid, and measuring no taller than wide, with indication of FNAB >1 cm in size; (5) high suspicion nodule: a hypoechogenic solid nodule or partially cystic nodule with hypoechogenic solid areas, and with one or more of the following: irregular margins (infiltrative, microlobular), microcalcifications, measuring taller than wide, ring-form calcifications with extrusion of the nodule tissue or evidence of spread beyond the thyroid, with indication of FNAB >1 cm in size (<1 cm, to be individualized).

All patients seen in the ultrasound-FNAB clinic and meeting the ATA criteria underwent puncture by the same endocrinologist using a 20 ml syringe with a 23G needle, under imaging guidance from Sonosite MicroMaxx® (years 2015–2016) and Hitachi Aloka F37® ultrasound machines (2017 to date), using 10–18 MHz transducers. All the cases were entered in an MS Access® database named “Thyroid Process”, with the inclusion of the following variables, among others: (a) clinical and demographic data: case history, age, gender (F/M), family history of thyroid cancer (yes/no, type of relative), active smoking (yes/no); (b) endocrinological data: thyroid function (normal function/subclinical hypofunction/hypothyroidism/subclinical hyperfunction/hyperthyroidism) and thyroxin treatment (yes/no); (c) ultrasound parameters: date of puncture, maximum diameter of target nodule (mm), consistency (cystic/solid/mixed), echogenicity (anechoic/hypoechoic/isoechoic/hyperechoic), margins (regular lobular, infiltrative), calcifications (none/microcalcifications/macrocalcifications), morphology (wider than tall/taller than wide), the presence of suspect adenopathies (yes/no), the number of suspect findings (n = 0–6), ATA classification (benign/very low suspicion/low suspicion/intermediate suspicion/high suspicion or not classifiable); (d) histopathological findings: thyroid cytology results (description and Bethesda classification), pathology results (description), thyroid cancer (yes/no), type of thyroid cancer (description), thyroiditis (yes/no), incidental carcinoma (yes/no, description and size). For practical reasons, this study analyzed the ultrasound characteristics of a single thyroid nodule per patient (in the case of puncture of more than one nodule) and no evaluation was made of the cytological results before the intervention. The detected cases of incidental microcarcinoma <1 cm in size were not considered thyroid cancer cases in evaluating the diagnostic performance of ultrasound.

In all cases, the indication of thyroidectomy was established in a joint session with General Surgery, with the inclusion of the following as general criteria: uni- or multinodular goiters with nodules ≥4 cm in size or presenting compressive symptoms or thyroid hyperfunction, and nodules with cytological findings indicative of malignancy or suspected malignancy. In nodules with indeterminate cytological features (follicular lesion or atypia of uncertain significance) or suspected follicular neoplasm, the indication of thyroidectomy was established on an individualized basis in each case. The thyroidectomy specimens were analyzed at the Pathology Unit of our center.

Statistical analysis

The anonymized data of the MS Access® database named “Thyroid Process” were exported to the SPSS® version 12.0 statistical package for MS Windows® for analysis. Continuous variables were reported as the mean ± standard deviation (SD), and qualitative variables as absolute numbers and percentages. After verifying normal data distribution, the Student t-test or analysis of variance (ANOVA) was used to compare quantitative variables, while the chi-squared test or Fisher's exact test was used to compare qualitative or categorical variables. Statistical significance was considered for p < 0.05.

The diagnostic performance of the ATA ultrasound classification was evaluated after grouping the results as true positive (TP: high suspicion ± intermediate suspicion and pathology data indicating cancer), true negative (TN: benign or very low suspicion or low suspicion ± intermediate suspicion and benign pathology data), false positive (FP: high suspicion ± intermediate suspicion and benign pathology data) and false negative (FN: benign or very low suspicion or low suspicion ± intermediate suspicion and pathology data indicating cancer). The positive predictive value (PPV) was calculated using the formula TP/(TP + FP), while the negative predictive value (NPV) was calculated using the formula TN/(TN + FN). The 95% confidence interval (95%CI) was considered in all estimates.

Results

During the study period, a total of 317 patients (mean age 51.7 ± 13.7 years; 83.3% females) with thyroid nodules (maximum diameter: 3.2 ± 1.4 cm) and previously evaluated at our ultrasound-FNAB clinic were studied, with the detection of 260 cases of benign disease (82.8%) and 57 cases of malignant disease (44 papillary cancers, 5 follicular cancers, 3 Hürthle tumors, 2 anaplastic cancers, one poorly differentiated cancer and one lymphoma). There were also 14 cases (5.4% of the total series) of incidental papillary microcarcinoma measuring 0.26 ± 0.25 cm in maximum diameter (range 0.1–0.7 cm), all in the context of patients with multinodular goiter and/or thyroiditis.

There were no significant differences between patients with benign or malignant nodules in terms of age (51.8 ± 13.3 years vs. 51.3 ± 15.6; p = 0.812) or thyroxine prescription (17.7% vs. 17.2%; p = 0.949). However, in our series there was a higher malignancy rate among males (35.3% vs. 13.7%; p < 0.001), active smokers (26.3% vs. 14.4%; p = 0.017) and in patients with a family history of thyroid cancer (46.7% vs. 15.8%; p = 0.007).

Table 1 shows the main ultrasound characteristics of the operated benign and malignant nodules, with additional assessment in those cases diagnosed with papillary cancer. The malignant nodules were smaller (2.4 ± 1.6 vs. 3.4 ± 1.5 cm; p < 0.001) and were significantly more likely to be classified as solid (96.3 vs. 72.2%; p < 0.001), hypoechogenic (85.1 vs. 28.5%; p < 0.001), with irregular margins (57.4 vs. 10.8%; p < 0.001), taller than wide (33.3 vs. 11.9%; p < 0.001), with microcalcifications (64.0 vs. 12.4%; p < 0.001) or associated with suspect neck adenopathies (33.3 vs. 4.3%; p < 0.001), compared with the benign nodules. The presence of macrocalcifications was not associated with an increased risk of malignancy (10.3% vs. 9.3% in benign and malignant nodules, respectively; p = 0.876).

Table 1.

Comparison of the ultrasound characteristics of benign nodules, all malignant nodules, and nodules with papillary cancer.

Ultrasound characteristic	Benign (n = 260)	Malignant (n = 57)	p^a	Papillary cancer (n = 44)	p^b
Maximum diameter (cm), mean ± SD	3.4 ± 1.5	2.4 ± 1.6	<0.001	2.4 ± 1.2	<0.001
Solid nodule (%)	72.2	96.3	<0.001	97.6	<0.001
Hypoechogenicity (%)	28.5	85.1	<0.001	88.6	<0.001
Irregular margins (%)	10.8	57.4	<0.001	61.4	<0.001
Taller than wide (%)	11.9	33.3	<0.001	34.0	<0.001
Microcalcifications (%)	12.4	64.0	<0.001	63.6	<0.001
Suspect adenopathies (%)	4.2	33.3	<0.001	36.4	<0.001
Suspect findings (n)^c, mean ± SD	1.2 ± 1.1	3.6 ± 1.4	<0.001	3.8 ± 1.3	<0.001

SD: standard deviation.

a

p benign vs. malignant.

b

p benign vs. papillary cancer.

c

Solid, hypoechogenic, irregular margins, microcalcifications, taller than wide, or suspect adenopathies.

In our study, 61.2% of the nodules were classified as benign (n = 3) or very low (n = 60) or low suspicion nodules (n = 131); 11.7% were classified as intermediate suspicion nodules (n = 37); and 27.1% as high suspicion nodules (n = 86), with histopathological malignancy rates of 0% for the benign nodules (n = 3) and very low suspicion nodules (n = 60); 3.1% for the low suspicion nodules (n = 131); 8.8% for the intermediate suspicion nodules; and 56.7% for the high suspicion nodules (Table 2).

Table 2.

Risk of malignancy of thyroid nodules based on the ATA classification.

Ultrasound classification	Cases (n)	Maximum diameter (cm)	Benign (n = 260)n (%)	Malignant^a (n = 57)n (%)	Risk of malignancy (%)	ATA risk 2015 (%)^10
Benign	3	4.2 ± 0.7	3 (1.1)	0 (0)	0	<1
Very low suspicion	60	3.7 ± 1.3	60 (23.0)	0 (0)	0.0	<3
Low suspicion	131	3.6 ± 1.3	127 (48.9)	4 (7.0)	3.1	5–10
Intermediate suspicion	37	2.8 ± 1.6	34 (13.1)	3 (5.3)	8.8	10–20
High suspicion	86	2.6 ± 1.3	36 (13.9)	50 (87.7)	58.1	>70–90

a

Not including the 14 incidental microcarcinomas.

Table 3 shows the diagnostic performance on evaluating the ATA ultrasound classification for nodules that were classified as presenting high ultrasound suspicion (n = 83) or high and intermediate suspicion (n = 120). In the first case, high sensitivity (87.7%), specificity (86.2%) and NPV (97.0%) were seen; accordingly, only 3% of the operated patients had a malignant lesion that was not classified as a high suspicion nodule. On considering those patients classified as presenting intermediate or high suspicion nodules as being at high malignancy risk, a slight improvement in sensitivity as a diagnostic test for thyroid cancer was observed (93%), at the expense of a lower specificity (73.1%).

Table 3.

Diagnostic performance of the ATA classification using high suspicion or high and intermediate suspicion criteria.

	Sensitivity (95% CI)	Specificity (95% CI)	PPV (95% CI)	NPV (95% CI)	Diagnostic performance^a (95% CI)
High suspicion (n = 83)	87.7 (75.7–94.1)	86.2 (81.2–90.0)	58.1 (47.0–68.5)	97.0 (93.6–98.7)	86.4 (82.0–89.9)
High and intermediate suspicion (n = 120)	93.0 (82.1–97.7)	73.1 (67.2–78.3)	43.1 (34.3–52.3)	97.9 (94.5–99.3)	76.7 (71.5–81.1)

95% CI: 95% confidence interval; NPV: negative predictive value; PPV: positive predictive value.

a

Patients correctly diagnosed. The 14 incidental microcarcinomas are not included.

Discussion

Our results show that the application by endocrinologists of the ATA ultrasound risk assessment of thyroid nodules has a high diagnostic performance, comparable to that observed by international expert radiologists,^18,21–23 and even slightly greater than that reported by other authors,^19,16 thereby warranting its applicability at single-intervention or high-resolution endocrinology clinics. In our series, the identification of highly suspect nodules exhibited a diagnostic sensitivity in preoperatively identifying malignant nodules of 87.7%. This figure moreover increased to 93% when high and intermediate suspicion nodules were included in screening, the NPV in both cases being 97%. However, thyroid ultrasound could not identify 14 cases of incidental papillary microcarcinomas measuring 0.26 cm in mean maximum diameter, this finding being similar to those reported in studies similar to our own.¹⁶ In addition, our data agree with those of other authors in that the studied malignant nodules were usually smaller than the benign nodules, and that there is an inverse relationship between the maximum diameter of the studied nodule and the ATA ultrasound risk classification,^18,20 thus supporting the idea that size is not relevant in the study of thyroid nodules.²⁴

In the ultrasound study of the thyroid nodule, the presence of certain findings such as solidness, hypoechogenicity, irregular margins, microcalcifications, greater height than width, intranodular vascularization or suspected extrathyroid spread, may suggest the presence of cancer. However, no single individual finding has sufficient diagnostic performance,^10–12 and the detection of these features is moreover characterized by high inter-operator variability.^12,25 For this reason, when studying thyroid nodules, it is advisable for clinicians and radiologists to use some of the current ultrasound risk classification systems, which have revealed sensitivity and NPV ≥ 90% in expert hands.^17–23 In this regard, the TI-RADS and ATA classifications are possibly the most widely used options, and avoid a greater number of unnecessary punctures.¹⁷ In our opinion, the ATA classification offers the advantages of simplicity, the use of fewer risk categories, and easy comprehension by clinicians and patients of the ultrasound risk criteria, in contrast to systems that use numbers or acronyms for risk stratification, such as the TI-RADS system.¹⁴ However, strictly speaking, between 5 and 18% of all thyroid nodules might not be classified as belonging to any of the risk categories proposed by the ATA. These cases may either remain unclassified or be included in any of the malignancy risk categories more closely related to the characteristics of the nodule in question.^22,26 Lastly, we observed a lesser ultrasound risk of malignancy for all categories as compared to that established in the ATA document¹⁰ and also recorded by other authors,^20,22 particularly in relation to the “high suspicion” category (58.1% risk in our study vs. >70–90% risk according to the ATA system), though our results agree with those of other authors that attribute to this category a cancer risk ranging from 46.7 to 56%.^18,27,28

Our study has limitations. A first limitation is its retrospective design. Secondly, since ultrasound risk was classified by the same operator, inter-observer variability was avoided, though this in turn could limit generalization of the results, taking into account the strong correlation between observer experience and adequate ultrasound evaluation of the thyroid nodule.²⁴ Moreover, in our study ultrasound risk categorization was not related to the cytological results, because it was considered more useful to correlate the ATA classification to definitive histopathological results. Finally, our study only included patients with thyroid nodules who underwent surgery, patients with nodules presenting benign cytology but without surgical criteria being excluded from the analysis. In this regard, if the study had comprised both types of patients (surgical and non-surgical), it is probable that the observed differences in diameter between the benign and malignant nodules would have decreased (operated nodules are generally larger than those not subjected to surgery). Likewise, the NPV and PPV values would probably have changed, since the increase in non-operated benign nodules would have caused a decrease in the pre-test prevalence of malignancy, with an increase in NPV and a decrease in PPV.

In conclusion, our results support the proposal that the application by endocrinologists at single-intervention or high-resolution clinics of the ATA ultrasound risk assessment of thyroid nodules affords a high diagnostic performance in identifying suspect malignant nodules, with a low probability of not detecting thyroid cancer nodules. The exceptions are incidental microcarcinomas which are often found in patients with multinodular goiter.

Funding

This study was partially supported by unconditional research grants from Laboratorios Menarini, S.A. and by the Andalusian Society of Endocrinology, Diabetes, and Nutrition (SAEDYN).

Conflicts of interest

The authors of this article have no conflicts of interest in relation to the objective or results of the study.