Evaluation of pathologically confirmed benign inflammatory breast diseases using artificial intelligence on ultrasound images

Abstract

Objectives

It was aimed to use AI retrospectively to evaluate US images of pathologically confirmed benign inflammatory lesions, to compare the results of AI with our US reports, and to test the reliability of AI in itself.

Methods

US images of 71 histopathologically confirmed benign inflammatory breast lesions were analysed by the FDA-approved AI programme (Koios Decision Support) using 2 orthogonal projections. The lesions' probability of malignancy based on AI and BI-RADS categories of the lesion based on initial US interpretations were recorded. Categories obtained by both systems were divided into 2 groups as unsuspicious and suspicious in terms of malignancy and compared statistically. Reliability of AI was also evaluated.

Results

No statistically significant difference was found in the lesions' likelihood of malignancy based on the AI and initial US interpretations (P = .512). Additionally, a positive and substantial association (τ-b = 0.458, P < .001) between the levels of suspicion by AI and the initial US interpretation reports was discovered, as per Kendall-b correlation analysis. With a Cronbach alpha correlation coefficient of 0.727, the reliability was high for AI.

Conclusions

Benign inflammatory breast lesions may show suspicious appearances in terms of malignancy with US and AI. Artificial intelligence produces results comparable to radiologists' US reports for benign inflammatory diseases. AI has high reliability within itself.

Resumen

Objetivos

Nuestro objetivo fue utilizar la IA retrospectivamente, para evaluar imágenes ecográficas de lesiones inflamatorias benignas patológicamente confirmadas, y comparar los resultados de la IA con nuestros informes ecográficos, así como probar su fiabilidad.

Métodos

Se analizaron las imágenes ecográficas de 71 lesiones de mama inflamatorias benignas, histopatológicamente confirmadas, mediante el programa de IA aprobado por la FDA (Koios Decision Support) utilizando dos proyecciones ortogonales. Se registraron la probabilidad de malignidad de las lesiones basada en IA y las categorías BI-RADS de la lesión basadas en la interpretación ecográfica inicial. Las categorías obtenidas por ambos sistemas se dividieron en dos grupos: no sospechosas y sospechosas en términos de malignidad, comparándose estadísticamente. También se evaluó la fiabilidad de la IA.

Resultados

No se encontraron diferencias estadísticamente significativas en cuanto a probabilidad de malignidad de las lesiones, basada en IA e interpretaciones ecográficas iniciales (p = 0,512). Además, se encontró una asociación positiva y sustancial (τ-b = 0,458, p < 0,001) entre los niveles de sospecha mediante IA y la interpretación ecográfica inicial, de acuerdo con el análisis de correlación τ-b de Kendall. Con un coeficiente de correlación alfa de Cronbach de 0,727, la fiabilidad de la IA fue alta.

Conclusiones

Las lesiones de mama inflamatorias benignas pueden mostrar aspectos sospechosos en términos de malignidad, utilizando ecografía e IA. La inteligencia artificial produce resultados comparables a los informes ecográficos de los radiólogos para enfermedades inflamatorias benignas. La IA tiene una alta fiabilidad en sí misma.

Introduction

Mastitis is the term used to describe inflammation of the breast parenchyma.¹ In fact, it is a clinical diagnosis, and the primary objective of imaging in the case of any inflammatory breast disease is to look for a potential abscess or lump. Since both infectious and non-infectious mastitis can resemble breast cancer, a pathologic diagnosis is required for both conditions. Furthermore, if a treatment strategy is unsuccessful, a biopsy is required to rule out malignancy and establish a specific diagnosis.^2,3 Ultrasound (US) is the ideal initial imaging method for this frequently young population—40 years of age or younger—presenting with discomfort complaints.^4,5

One of the main advantages of the US is its accessibility, excellent diagnostic performance, non-invasiveness, and real-time imaging capability. Despite the fact that the US findings of benign inflammatory breast diseases and breast cancer are well characterised, this modality cannot be trusted to reliably make the differential diagnosis. The biggest problem is operator reliance.⁶ A histopathological diagnosis with a biopsy is also required before to beginning various treatment plans like chemo- or steroid therapy due to the varying therapeutic approaches.^4,7,8 When a benign lesion is subjected to a biopsy, the patient feels anxious and the healthcare system is burdened with additional expenses. To date, efforts have been done in this area to establish a non-invasive diagnostic method that will prevent unnecessary biopsies.⁹

There are limited studies on breast US in the literature compared to studies on artificial intelligence (AI) in breast radiology that predominantly focus on mammography.^10–18 Furthermore, no research has been done that specifically looks at how well AI addresses benign inflammatory breast diseases. On US images that were previously taken in 2 orthogonal projections and contained the lesion, Koios Decision Support employs machine learning and AI to create a likelihood of malignancy for a region of interest drawn manually or automatically.¹⁹ Following that, this probability is divided into 4 groups: benign (B) corresponds to BI-RADS 2, probably benign (P) corresponds to BI-RADS 3, suspicious (S) corresponds to BI-RADS 4A or 4B, and probably malignant (M) corresponds to BI-RADS 4C or 5.²⁰

In this investigation, we used AI to assess US images of benign inflammatory lesions with pathological confirmation, and we compared the outcomes to our interpretation reports. Additionally, the relationship between AI's levels of suspicion and US interpretation reports was evaluated, and the reliability of AI was assessed.

Materials and methods

The study was approved by the Istanbul Medipol University Ethics Committee (E-10840098-772.02-5234, 15 October 2021). Because the study was retrospective, no informed consent was sought. The data were anonymized, and the management of the patients was independent of the results of AI. Included were lesions that were examined between October 2019 and October 2021 and were histopathologically identified as benign inflammatory breast lesions. All biopsies were performed by a radiologist with a sub-specialty diploma and 12 years of experience in breast radiology (IDS).

Ultrasound images

Breast ultrasound using Logic E9 ultrasound was performed (GE Healthcare Ultrasound Systems, Chicago, USA). Digital Imaging and Communications in Medicine (DICOM) images of lesions were reviewed. Following were the requirements for inclusion:

(1) Biopsies for all of the lesions corroborated the diagnosis.

(2) The images were taken before the biopsy, antibiotic, or steroid therapies.

(3) For image acquisition, a minimum frequency of 12 MHz was used.

(4) The images were taken using 2 perpendicular planes (static, 2 orthogonal slices).

(5) Appropriate adjustments were made to the focus and gain.

The pursuing exclusion criteria were applied:

(1) Not having any of the aforementioned characteristics.

(2) Images that were taken as a follow-up after therapy.

(3) Low-resolution images with inadequately adjusted focus-gain.

(4) Printed images.

(5) The lesion contains any markers for measurement, a biopsy needle, or a colour code.

(6) Elastography maps or coloured Doppler images.

Artificial intelligence algorithm

The FDA-cleared diagnostic AI application Koios DS for Breast Study Tool was used to analyse the US images.¹⁹ Machine learning is used in this algorithm. The user adds a region interest to US images that completely encloses and contains the lesion. It examines previously taken, static US images to determine the possibility of malignancy. The 4 categories of this probability are: benign (B), probably benign (P), suspicious (S), and probably malignant (M).²⁰ The algorithms of Koios DS are produced using either pathologically proved cases or follow-up imaging from more than 1 000 000 cases collected with various ultrasound machine models and suppliers from more than 25 universities. After being educated on thousands of prior cases, it categorises suspicious lesions and automatically populates diagnostic reports. (Koios Medical, Inc. 444 North Michigan Avenue, Suite 3450 Chicago, Illinois 60611).

Data analysis and statistical methods

Using the first US interpretations, the lesion was classified using the BI-RADS system. Every lesion underwent AI analysis, and we kept track of how each one was categorised. Malignancy-related categories were divided into 2 groupings by both systems (the initial US interpretations and AI): unsuspicious and suspicious. The original US BI-RADS category and the AI category for benign inflammatory lesions were compared using the Chi-square test. A Kendall-b correlation analysis was conducted to look for a connection between the initial US report and the AI after the categories were divided into benign, low-, and high-suspicious categories. The intra-reader reliability of the AI system was also evaluated by re-evaluating the images 6 months later with the same AI system. The threshold for statistical significance was set at 0.05. The data was analysed using the SPSS version 22.0 software programme for Windows (IBM Corporation, Armonk, NY, USA).

Results

The study comprised a total of 71 benign inflammatory breast lesions (All female, patients' age varied between 25 and 44 (36 ± 7)). Based on core biopsy, benign inflammatory lesions included xanthogranulomatous inflammation (n = 4), granulomatous inflammation (n = 42), active chronic inflammation (n = 19), mastitis (n = 3), and periductal mastitis (n = 3) (Table 1). 58 of the patients (82%) had palpable mass and 27 of the patients (38%) had inflammatory findings.

Table 1.

Histopathological results and US imaging/AI classification of the all lesions and comparison of initial US and AI categories for benign inflammatory lesions (P = .512).

Histopathological results		Number	Percent
Granulomatous mastitis		42	59.2
Active chronic inflammation		19	26.8
Xanthogranulomatous inflammation		4	5.6
Mastitis		3	4.2
Periductal mastitis		3	4.2
US classification of benign inflammatory lesions
BI-RADS 3		36	50.7
BI-RADS 4A		16	22.5
BI-RADS 4B		14	19.7
BI-RADS 4C		5	7
BI-RADS 5		0	0
AI classification of benign inflammatory lesions
Benign (B)		4	5.6
Probably benign (P)		10	14.1
Suspicious (S)		48	67.6
Probably malignant (M)		9	12.7
		US categories
		Un-suspicious (BI-RADS 2,3) (n, %)	Suspicious (BI-RADS 4,5) (n, %)
AI categories	Unsuspicious (B,P)	6 (42.9%)	8 (57.1%)
	Suspicious (S, M)	30 (52.6%)	27 (47.4%)

Pearson Chi-Square, AI: artificial intelligence, BI-RADS: breast imaging reporting and data system, US: ultrasonography, n: number, B: benign, P: probably benign, S: suspicious, M: possibly malignant.

AI classified 48 lesions (67.6%) as suspicious (S), 10 lesions (14.1%) as probably benign (P), 9 lesions (12.7%) as probably malignant (M), and 4 lesions (5.6%) as benign (B) (Figs. 1–4, Table 1). The classification of benign inflammatory lesions between the initial US reports and the AI results was compared (n = 71). The difference was not statistically different (P = .512). Six (43%) of the 14 benign inflammatory lesions that AI deemed unsuspicious were similarly deemed unsuspicious in the initial US reports. Eight (57%) of these lesions were in the suspicious group based on the initial US reports. Thirty (53%) of the patients who were in the suspicious group according to the AI were assigned to the unsuspicious group based on the initial US reports, while 27 (47%) were categorised as suspicious according to the initial US reports.

Fig. 1.

A 49-year-old patient with granulomatous inflammation. A false-positive benign inflammatory lesion that was initially classified by the US as BI-RADS 3 but by AI as suspicious.

Fig. 2.

A 34-year-old patient with granulomatous inflammation. A true-positive benign inflammatory lesion that was initially assessed on US as BI-RADS 5 and by AI as probably malignant.

Fig. 3.

A 43-year-old patient with active chronic inflammation. A false-negative benign inflammatory lesion that was initially classified by the US as suspicious (BI-RADS 4B) but probably benign by AI.

Fig. 4.

A 43-year-old patient with granulomatous inflammation. A true-negative benign inflammatory lesion that was initially assessed on US as BI-RADS 3 and by AI as benign.

According to Kendall τ-b correlation analysis, a positive and significant relationship was found between the level of suspicion in the US reports and AI (τ-b = 0.458, P < .001).

The AI was quite reliable based on Cronbach’s alpha test (α = 0.727, 95% Confidence Interval of 0.560–0.829).

Discussion

In our investigation, there was no statistically significant difference between our initial US interpretation reports and AI in the categorising the benign inflammatory breast lesions. (P = .512). The amount of suspicion expressed in the US reports and the level of suspicion by AI were also found to be positively and significantly correlated (τ-b = 0.458, P < .001). For our study population with benign inflammatory breast lesions, the intra-reader reliability of AI was high (α = 0.727).

The AI application in our study has a challenging task: evaluating benign inflammatory breast lesions. Numerous factors, including patient characteristics, prior medical history, physical examination findings, and clinical trajectory, have an impact on doctors' daily decisions.^3,4 Despite having to discern between several possibilities without access to such information, the AI algorithm's classification performance in our study population was comparable to our initial US results.

Few AI studies have examined breast lesions on US images.^10–18 These studies typically focus on the distinction between breast cancer and benign breast lesions. In a study that employed the most images in the literature (7408 breast ultrasound images from 5151 patients), the GoogLeNet convolutional neural network (CNN) architecture was used to categorise benign and malignant tumours.¹¹ In a different study, 882 masses were analysed and classified as benign or malignant using the ImageNet data set using a pre-trained VGG19 deep learning model.¹² The predictions made by the model were compared with the BI-RADS classification offered by 4 expert radiologists. In a different study, 520 histopathologically validated breast lesions from 520 patients (275 benign, 245 malignant) were analysed using a deep learning model of stacked noise cancelling autoencoder (SDAE).¹³ These studies focused on the difference between benign and malignant tumours and compared radiologists to artificial intelligence.

Layer pooling is present in a sequential manner in GoogleNet, one of the first CNN architectures. This model makes extensive use of memory and energy. The likelihood that anything will be remembered increases when all the layers and filters are used, but there are additional computational and memory costs.^21,22 The VGG19 model is an advanced and complex CNN model. Each of the 19 layers that make up VGG19's convolutional layers has a kernel size of 3 and a stride of 1, which enables it to identify characteristics that are typical of the entire image. The rectified linear unit (ReLU) activation function is used to create the nonlinearity.²³ The SDAE architecture may be ideal to handle the intrinsically noisy nature of medical image data from diverse imaging modalities since it is well-equipped with the automatic feature exploration mechanism and noise tolerance advantage.¹³ In contrast to these deep learning architectures, the system we employed was a machine learning algorithm. Without explicit programming, machine learning enables a system to automatically learn from its experiences and get better over time.

Machine learning enables a system to automatically learn from given data without explicitly programming. In image classification problems, new algorithms are tested on a common dataset for benchmarking purposes. ImageNet is one of these benchmarking datasets with more than 10 million images in 1000 categories. After 2012, when Alex-Net surpassed the performance of the winner of the previous year with a large margin, Deep Learning methods set the new standard for image classification. In the following years, VGG16/19 and Inception (GoogleNet) further extended the performance of these models and surpassed the human level classification. VGG models stack a series of convolutional blocks which include convolution, max pooling, and batch normalisation operations to build a deeper network along with ReLu activation functions. On the other hand, GoogleNet employs inception blocks which employ a series of convolution operations with different kernel sizes, at the same layer level to capture features in different scales. Their building methodology are the basis of modern convolutional network architectures. These models can either be employed stand alone with leveraging their carefully designed architecture without pre-trained weights, if high number of sufficient data allows training from scratch or can be fine-tuned initializing with pre-trained weights. On the other hand, stacked denoising autoencoders exploit self-supervised learning to obtain latent representation of the input. Each layer learns to fix the manually corrupted input to obtain the original input. This approach leads to strong feature representation which can be further employed in any type of classifier. Since the whitepaper of the Koios doesn't include algorithm details, direct comparison is not possible. However, the whitepaper mentions that the algorithm starts with 2 orthogonal images and outputs BI-RADS categorisation along with confidence scores and shape and orientation features. This implies that the naturally selected algorithm is a CNN possibly with addition of residual, dense, or inception blocks and attention mechanisms. However, as this study is a reader study comparing the performance of radiologists with AI against ground-truth labels, exact training mechanism of the algorithm would have minimal impact on the results if has any.

In a study evaluating treatment response, a deep learning-based approach in US was preferred. A pre-trained CNN on the ImageNet dataset and double-transfer learning were implemented. The proposed approach demonstrated the feasibility of transfer learning with CNNs to predict treatment response with US.²⁴ In a multicentre retrospective study, 900 breast lesions were evaluated using the same artificial intelligence system to our study and assigned into 1 of 4 categories as follows: benign, probably benign, suspicious, and probably malignant. The positive likelihood ratio for AI was 1.98, higher than the positive likelihood ratio of all readers except one. Fourteen readers had better AUC with both US and AI than US with only. The mean Kendall τ-b inter-reader variability was 0.54 (95% CI, 0.53–0.55) for US-only group and 0.68 (95% CI, 0.67–0.69) for US plus Koios DS group. The intrareader variability was high with Koios DS; class switching (defined as upgrading from BI-RADS category 3 to BI-RADS category 4A or above) occurred in 13.6% of the cases with US only versus 10.8% of cases with US plus DS (P = .04). They concluded that AI improves the accuracy of sonographic breast lesion assessment while reducing inter- and intraobserver variability.²⁵ In our investigation, the same AI model was employed. As a result, the model produced results that were highly intra-observer reliable and at least somewhat consistent with our actual clinical practise. To further assess its performance in a larger population and clearly demonstrate its negative-predictive value, it would be desirable to include benign lesions, inflammatory malignant lesions, and non-mass lesions in the analysis.

Visionary planning for artificial intelligence applications in the healthcare system will have a positive impact on both today and tomorrow. Recent algorithms with high accuracy and repeatability will result in smart systems, ease of application, and an increase in service quality in the future. Therefore, the development of verified systems based on our own data could be possible with correct planning and action from today onwards. While one aspect of artificial intelligence is such fast and effective outputs, the other aspect is its clinical utility. To be more precise, commercial manufacturers might sometimes promise more than they could provide, or even without any malicious intent, the proposed algorithm might not be enough for a solution. In this context, healthcare professionals, especially physicians, have the responsibility to educate themselves about AI, evaluate its validity in the areas where it is used, and maintain an effective and safe balance to protect patients while applying new technologies.^15,26,27

There are several limitations of the study design that must be taken into consideration. Firstly, only one reader interpreted the sonographic images. Although she has considerable experience, a broader population of radiologists reading and analysing the breast ultrasound images would be more efficient. Secondly, there may be a potential selection bias due to selecting patients from one institution. In this regard, patients and images provided from various institutions would be more accurate. We compared artificial intelligence with our own practice specifically for the benign inflammatory group. It would be great if we had the malignant inflammatory group, but there were a very limited number of patients with malignant inflammatory lesions and their images did not meet our criteria. Other limitations include the retrospective design and the lack of intraoperator variability comparison for initial US reports. Lastly, images enrolled in this study were obtained by the radiologist in 2 perpendicular planes, which were thought to represent the pathology best. Since they could not fully represent a 3-dimensional structure, an automated US with a 3-D display would be more appropriate to test and apply in decision-making algorithms. The purpose of this study was to investigate how compatible artificial intelligence and our US application are. We did not intend to conduct a diagnostic performance study of AI. This would require a negative control group. Our next task may be a study in which the diagnostic performance is clearly demonstrated in a larger population with a negative control group.

In conclusion, artificial intelligence has similar results in terms of radiologic classification of benign inflammatory diseases. In light of this, the AI system has the capacity to offer insightful data as a second opinion in the assessment of benign inflammatory lesions. Future research should focus on assessing the effect of extended AI use on radiologists' performance as well as possible advantages to overcome experience and education disparities. In addition to measurements of factors like sensitivity, specificity, and accuracy that may have an impact on their clinical use, it is also important to assess how well they perform in specific clinical settings, how much they optimise workflow, how much they compensate for lack of training, how long the consultation process takes, and how much of a gap they can fill.

Funding

No funding received.

Ethical disclosure

Ethical approval: Istanbul Medipol University Ethics Committee (E-10840098-772.02-5234, 10/15/2021), informed consent waived.

Patients consent

The authors declare that they obtained the patient's consent for publication of the article.