ECOSHOT study: Prospective evaluation of a shoulder ultrasound training programme for general practitioners

Abstract

Background and objective

The effectiveness of shoulder ultrasound training programmes in primary care is unknown. This study aims to measure the diagnostic agreement between radiologists and general practitioners in shoulder ultrasound examination after a specific training programme.

Materials and methods

We designed a prospective study with patients who required a shoulder ultrasound examination. Ultrasound examination was performed by general practitioners after receiving specific training, consisting of 50 h of theoretical musculoskeletal ultrasound training online and 8 h of hands-on shoulder ultrasound practice. The findings were compared to those of radiologists using Cohen’s Kappa coefficient. Sociodemographic and quantitative variables were also measured.

Results

We included 153 patients. Finally, 121 radiologists’ reports were used for analyses. The average age was 61.8 years and 62.1% were women. Main occupations were domestic work/cleaning and administrative jobs. Average Visual Analogue Scale score was 5.25. Patients with previous treatment comprised 88.9% (mostly oral painkillers, rest, and physical measures). Complementary testing was performed in 28.8%, radiography was the most frequent. The most frequent motive for referral for ultrasound examination was unspecific shoulder pain. Our study reveals that the global agreement was moderate (Kappa = .4346) overall, but when measured over time improved from weak (Kappa = .3756) to good (Kappa = .6423). There was a tendency to overdiagnose most lesions and underdiagnose partial tears.

Conclusions

We conclude that shoulder ultrasound examination training is feasible in primary care. The interobserver agreement improved with the number of ultrasound examinations performed over time, suggesting that specific programmes may be effective in training general practitioners in musculoskeletal shoulder ultrasound examination.

Resumen

Antecedentes y objetivo

En la actualidad se desconoce la eficacia de la formación en ecografía de hombro para atención primaria, por lo cual el objetivo de este estudio es medir la concordancia diagnóstica entre radiólogos y médicos de atención primaria en ecografía de hombro tras una formación específica.

Materiales y métodos

Se diseñó un estudio prospectivo con pacientes que requerían una ecografía de hombro. Seis médicos de atención primaria realizaban este procedimiento tras una formación específica (50 horas teóricas en ecografía musculoesquelética online/8 horas prácticas en ecografía de hombro) y los hallazgos se comparaban con el radiólogo, empleando el coeficiente de correlación Kappa. También se recogieron variables cuantitativas y sociodemográficas.

Resultados

Se incluyó a 153 pacientes y al final se emplearon 121 informes del radiólogo para los análisis. La edad media fue de 61,8 años y el 62,1% fueron mujeres. Las principales ocupaciones eran trabajo doméstico/limpieza y trabajo administrativo. La puntuación media de la escala visual analógica fue de 5,25. El 88,9% de los pacientes habían seguido un tratamiento previo, las más de las veces analgésicos orales, reposo y medidas físicas. En el 28,8% de los casos se hicieron pruebas complementarias, en tanto que el motivo de derivación más frecuente fue la omalgia inespecífica. Se reveló que la concordancia global era moderada (Kappa = 0,4346) y que con el tiempo pasaba de débil (Kappa = 0,3756) a buena (Kappa = 0,6423). Se tendía a sobrediagnosticar la mayoría de las lesiones e infradiagnosticar las roturas parciales.

Conclusiones

La ecografía de hombro es factible en atención primaria. La concordancia interobservador mejoró con el número de ecografías y con el tiempo, lo cual sugiere que los programas de formación específicos pueden ser efectivos para entrenar a médicos de atención primaria en ecografía de hombro.

Introduction

Omalgia is one of the most common musculoskeletal disorders in primary care (PC). Its estimated prevalence is 7%–26% in the general population, almost 70% throughout life,¹ and an estimated incidence of 14.7/1000 patients per year.²

The most common causes of omalgia include rotator cuff disease, glenohumeral involvement, acromioclavicular joint disease, and referred pain from the cervical region.^3,4 Most cases are treated with rehabilitation, followed by non-steroidal anti-inflammatory drugs (NSAIDs), corticosteroid infiltrations, or combinations.⁵ Even so, the response to treatment is usually poor, with 51% of patients experiencing recurrences at 26 weeks and 41% at 12–18 months.⁶ This disease also has a considerable impact on daily living activities and can cause chronic pain and disability.^6,7

Most often, the diagnosis is based on a detailed clinical history and physical examination. Although the clinical course also offers important clues for the diagnosis considering the high incidence of morbidity and chronicity in this disease, it may be necessary to refer patients for additional examinations and in this way define the diagnosis and offer targeted treatment.^8–11 Furthermore, if there is pain refractory to treatment, an atypical clinical course, or warning signs, it is advisable to perform an ultrasound study to complete the diagnosis.^3,12

In most cases, ultrasound is recommended for non-traumatic injuries, especially in suspected rotator cuff tears or when the diagnosis is unclear.¹¹ Some guidelines limit its use to episodes of suspected gout or malignancy, along with magnetic resonance imaging (MRI),¹² while others recommend ultrasound as the first test for non-traumatic pain in patients under 40 years of age, to confirm X-ray findings, or if the initial X-ray is not revealing.¹³

The use of ultrasound is booming; it is used by many specialists due to its accessibility, cost-effectiveness, being minimally invasive, and good acceptance by the patient,^14,15 which has led to a growing demand for standardized training programs in ultrasound to increase its use and diagnostic accuracy.^15–17

In PC, ultrasound directed at specific anatomical areas has proven to be an effective diagnostic tool that can help reduce healthcare costs.¹⁸ However, there are still few training programs on this topic and there is no consensus on what they should include. There is also not enough information on the application of this test, nor studies on its results, and the evolution of patients in whom it is performed in PC.^19–21

Until it is known, a prospective study assessing a shoulder ultrasound training program has not been performed. The objective of this study is to measure the diagnostic concordance between expert radiologists and PC physicians in shoulder ultrasounds after specific training.

Methods

Training program

Before recruiting patients, all participants (6 PC physicians) underwent the following specific training in shoulder ultrasound offered by different medical societies:

• •

  An online theoretical course in musculoskeletal ultrasound (50 h), organized by the Society of Family and Community Medicine (SEMFYC)

• •

  A specific theoretical and practical course on shoulder ultrasound (4 h), organized by the Catalan Society of Family and Community Medicine (CAMFiC)

• •

  Practical training (4 h) with ultrasound, supervised by expert PC doctors at the Ca N’Oriac PC center (Sabadell, Barcelona)

Study design and sample selection

A prospective diagnostic testing study is presented that considers the radiological report as the gold standard, which includes subjects with omalgia referred for a shoulder ultrasound by their PC physician. Individuals were recruited in three PC centers (La Marina, Carles Ribes, and Bordeta-Magòria, in Barcelona, ​​Spain) between June 2019 and February 2020. The exclusion criteria were minor patients (<18 years), subjects with previous shoulder interventions or fractures, and individuals with little mobility or at home.

The PC physician of the candidate patients himself did the recruitment, for which he/she offered to participate in the study to those referred subjects for a regulated shoulder ultrasound. Those who accepted were included in a database and assigned a numerical code for confidentiality. Within 48 h they were contacted, and the project was explained to them in detail. In addition, they were given the date and time for the ultrasound in PC and they signed an informed consent before the examination was performed by the PC physician.

Interventions

The included subjects underwent two ultrasound scans, one by the trained PC physician and another regular ultrasound by the radiologist. The examination was not supervised by an expert professional; however, two of the PC physicians were present while it was being conducted. The ultrasound performed by the PC doctor took place before the radiologist’s assessment, and in those cases in which it was not possible, the PC professional did not have access to the radiological report. For the PC study, ultrasound devices were used: Mindray Diagnostic US System model Z6 and a DC-N3 model, with a 7.5 Hz linear probe.

After an interview with the patient, a data collection sheet was filled out with sociodemographic information and clinical variables, while the same was done with the descriptive variables of the ultrasound findings for each anatomical structure. It was concluded with a classification of normal or pathological, and then the main diagnosis was established (more than one if necessary).

Outcomes

The main outcome of the study was to assess the diagnostic concordance of shoulder ultrasound between the PC physician and the radiologist. Secondary objectives included an assessment of the learning curve over time and the number of examinations performed. In addition, the main diagnosis for which the referral for the examination was evaluated.

Statistical analysis

A sample of 141 ultrasound scans was estimated, based on an expected value of Cohen’s Kappa coefficient (k) of 0.6, with 85% of pathological classifications by the PC professional, 90% by the radiologist, and a confidence level of 95%.

The radiologist’s report was the gold standard for comparison. A bivariate correlation analysis and an estimation of the concordance coefficient were performed with the radiologist and PC physician variables. The Kappa coefficient was interpreted according to the following scale: 0–0.20: poor; 0.21–0.40: weak; 0.41–0.60: moderate; 0.61–0.80: good; 0.81–1: very good.²²

The sociodemographic and clinical data collected after interviewing the participating subjects were described with frequency measurements for the qualitative variables, while the mean and standard deviation were used for the quantitative variables. p-Values ​​of p < 0.05 were considered significant. The statistical package SPSS version 17.0 (Statistical Package for Social Sciences, IBM) was used for all statistical analyses.

Ethical considerations

This work was carried out following the Declaration of Helsinki. All patients signed an informed consent form before performing the ultrasound by the PC professional. Participants were assigned a numerical code for data protection. The study protocol was approved by the IDIAP Jordi Gol Primary Care Research Ethics Committee (CEI).

Results

From June 2019 to February 2020, a total of 207 requests for shoulder ultrasound were received. Fifty-four individuals were excluded from the study, most due to refusal to participate, failure to appear, or cancellation of the test due to the onset of the COVID-19 pandemic in Spain. After exclusions, a sample of 153 patients was obtained. Thirty-two patients did not have the radiologist’s report available (mostly due to the cancellation of the ultrasound due to the Covid-19 pandemic), so they were excluded from the analysis of the correlation coefficient. The flow chart is detailed in Fig. 1.

Fig. 1.

Flowchart showing cases and selection.

Characteristics of the patients

Of the 153 subjects, 62.1% were women and the mean age was 61.8 years. The main occupations were domestic work and cleaning (13.1%), administrative work (9.2%), retired (31.4%), and other occupations (27.5%).

The mean visual analog scale (VAS) score (used to assess baseline pain) was 5.25, with a standard deviation (SD) of 2.46. A score of more than 7 points on the VAS scale, considered intense pain, was obtained in 25% of the individuals. The median number of days from the onset of omalgia to the performance of the PC ultrasound was 120 (interquartile range 60–240); 25% of patients reported previous episodes of pain in the same shoulder.

A total of 88.9% of the patients had started previous treatments, including rest (35.9%), physical measures (37.3%), NSAIDs (75.2%), infiltrations (7.2%), and rehabilitation (18.3%). Likewise, in 28.8% of the cases, complementary tests were performed before the ultrasound examination, mainly X-ray assessments (24.8%). Data on population characteristics are depicted in Table 1.

Table 1.

Basic characteristics of the patients.

Characteristic	Total
Patients, N.	153
Mean age in years (SD)	61.8 (14.8)
Sex, N.° (%)
Female	95 (62.1)
Male	58 (37.9)
Main occupation, N.° (%)
Household and cleaning work	20 (13.1)
Administration	14 (9.2)
Retired	48 (31.4)
Manual work	11 (7.4)
Drivers	8 (5.3)
Dependents and hospitality	9 (6.1)
Other^a	43 (27.5)
Mean VAS (SD)	5.25 (2.46)
Time course in days, median (IQR)	120 (60–240)
>1 episode on the same shoulder (%)	25
Previous treatment, N.° (%)	136 (88.9)
Oral NSAIDs	115 (75.2)
Rest	55 (35.9)
Physical measurements	57 (37.3)
Rehabilitation	28 (18.3)
Infiltrations	11 (7.2)
Other^b	12 (8.1)
Previous complementary tests, N.° (%)	44 (28.8)
X-rays	38 (24.8)
Ultrasound	9 (5.9)
Magnetic resonance	2 (1,3)
Time in days from referral to ultrasound, N.° (SD)
Radiologist	75 (26.35)
Primary care physician	34 (31.79)

SD: standard deviation; VAS: Visual Analogue Scale; N.°: number; IQR: interquartile range; NSAIDS: non-steroidal anti-inflammatory drugs.

a

Other occupations: professional scientist, technician, nurse, farmer, security guard, hairdresser, designer, photographer, social worker, and salesperson.

b

Other treatments: acupuncture, anti-inflammatory creams, home exercises, use of weights, homeopathy and collagen, osteopathy, dry needling, and home remedies.

The reasons for referral by PC physicians to perform a shoulder ultrasound were omalgia (61.4%), unspecified rotator cuff tendonitis (14.4%), supraspinatus tendinitis (5.9%), subacromial syndrome (5.2%), supraspinatus complete tear (1.3%), bicipital tendinitis (1.3%), bursitis (0.7%), and others (–9.8%– included synovitis, fractures, calcifying tendonitis, infraspinatus partial tear, or no specified reason).

Statistical results

The delay between referral and performance of the ultrasound by the PC physician was a mean of 34 days (SD 31.795) and 75 days (SD 26.349) for the radiologist.

Concerning the results classified as normal or pathological, a level of concordance of 50% was achieved in those considered normal and 93.5% in those classified as pathological. The total Kappa level of concordance was k = 0.4346 (95% CI = 0.1876–0.6816; p < 0.0001).

The level of concordance evolved over time was analyzed and it was found that the first 6 months gave a weak level of concordance with the radiologists (k = 0.3756; 95% CI = 0.092–0.6592; p = 0.001); however, from the seventh to the ninth month, the Kappa value increased significantly to k = 0.6423 (95% CI = 0.1828–1.1018; p < 0.0001), considered good. Furthermore, when including only the last 2 months of the study, a k = 1 was obtained (p < 0.0001). However, it should be interpreted with caution, since in the last 2 months, 21 ultrasounds were performed, of which only 2 were classified by the radiologist as normal. Considering that PC professionals tended to overdiagnose, these results were very timely to improve the kappa value (Fig. 2).

Fig. 2.

Representation of the level of concordance according to the number of ultrasounds performed and time.

Subsequently, the level of concordance was analyzed according to the anatomical area. Consistent with previous observations, it was observed that the supraspinatus tendon was the most affected (78.5%), followed by the involvement of the acromioclavicular joint (41%), subscapular (24%), bicipital (13.2%), and infraspinatus tendons (7.4%). This sequence was consistent for both radiologists and PC physicians, except for findings of acromioclavicular joint involvement, yet the proportions were different. Indeed, PC professionals tended to overestimate the number of pathological tendons when compared to the radiologist’s examination (Fig. 3).

Fig. 3.

The proportion of lesions found according to affected anatomical structure comparing radiologist versus primary care physician.

A moderate level of concordance was found for the infraspinatus (k = 0.431; p < 0.001), weak for the bicipital tendon (k = 0.260; p = 0.003) and the supraspinatus (k = 0.314; p = 0.001), and poor for the subscapular tendon (k = 0.130; p = 0.148) and the acromioclavicular joint (k = 0.092; p = 0.243).

The type of injury found most frequently was tendinosis (43%), followed by calcifications (33.1%), partial tear (33.1%), tendinitis (26.4%), subacromial bursitis (25.8%), and complete tear (12.4%) (Fig. 4).

Fig. 4.

The proportion of injuries found according to the type of pathology comparing radiologist and primary care physician.

A weak concordance was noted for tendinosis (k = 0.235; p = 0.009), partial tears (k = 0.226; p = 0.008), subacromial bursitis (k = 0.350; p < 0.001), and complete tears (k = 0.315; p = 0.001). Otherwise, a poor level of concordance was found for calcifications (k = 0.186; p = 0.033) and tendonitis (k = 0.171; p = 0.059), while a trend among PC physicians to overdiagnose most injuries and underdiagnose partial tears was found.

Discussion and conclusions

The diagnosis of omalgia is a challenge since different shoulder disorders can have similar symptoms and the findings on physical examination are frequently non-specific.^23–25 In non-traumatic shoulder injuries, empirical treatment should be initiated before performing additional tests.^3,25 Shoulder ultrasound is useful in cases in which omalgia does not respond to treatment or presents warning signs. Relevantly, it has the same specificity as MRI for complete tears, but at a lower cost.^8,12,13

Complementary tests in PC should be interpreted carefully, considering that structural abnormalities may appear in asymptomatic subjects result in referrals to secondary care, and cause unnecessary concern to the patient.^3,19

However, a more specific diagnosis of persistent omalgia could help better guide the therapeutic approach and allow earlier treatment that could prevent chronicity, recurrences, and avoid referrals to secondary care.^8,10,18

The patient’s occupation could influence omalgia, although we see the same incidence in manual and non-manual jobs and in non-dominant arms,^3,26 which could be because external factors such as practicing sports or hobbies, apart from possible genetic factors.²⁷

It was found that a significant number of patients (25%) had a VAS score that indicated intense pain, highlighting the importance of omalgia as a major limitation.⁷

As expected, the delay in performing the ultrasound was shorter in PC compared to radiology (34 vs. 75 days), which makes evident the advantage of shoulder ultrasounds performed by trained PC doctors, since this is how could reduce the waiting list for radiology services, improve cost-effectiveness, as well as earlier diagnosis and targeted therapy.¹⁸ Likewise, recent studies show a high level of patient satisfaction with ultrasounds performed on PC, which makes it even more important to extend its use.¹⁸

In general, the diagnostic concordance of PC physicians’ ultrasounds was moderate (k = 0.4346), mostly due to overdiagnosis by PC professionals. However, the level of concordance between PC physicians and expert radiologists increased with the passage of time and the number of procedures. Therefore, specific training, with a longer period of practice, could significantly improve the accuracy of PC professionals in shoulder ultrasound.

To the best of our knowledge, this is the first assessment of a specific training program in shoulder ultrasound in PC. In a systematic review conducted by Andersen et al. in 2020, which brought together studies on ultrasound training programs in other anatomical areas performed in PC, it was found that diagnostic concordance increased with the number of ultrasounds performed and the passage of time.²¹ Also, Esquerrà et al. in 2011 evaluated the performance of low-complexity abdominal ultrasounds, and their results at 6 months describe good concordance (k = 0.85; 95% CI = 0.82–0.98; p < 0.05) in the main diagnosis among PC physicians and radiologists.²⁸

The limitations of our study were, firstly, that the patient recruitment period was shortened due to the beginning of the SARS-CoV-2 pandemic, the reason why there were fewer patients than had been planned for statistical analysis, with the consequent reduction in the power of the study.

Secondly, it is important to consider that the difference in time between the ultrasounds of the PC professional and the radiologist may have led to changes in the shoulder pathology that were initially visible or not. Furthermore, the lack of a standardized reporting structure could, in some cases, make comparability of the two reports difficult. Finally, the ultrasound devices used in PC and radiology were different, which could also influence the results.

In conclusion, specific training in shoulder ultrasound for primary care physicians is feasible and, after a period of training and practice, a progressive improvement in diagnostic skills can be achieved. Larger studies are needed to validate our results and assess whether shoulder ultrasound can improve prognosis and treatment response in the setting of primary care.

Financing

This research has not received specific support from public sector agencies, the commercial sector, or non-profit entities.

Conflict of interests

None.

Acknowledgments

We would like to thank all primary care physicians who participated by referring their patients to the primary care ultrasound consultation; Clara Alavedra Celada and Carmen Berbel Navarro, MDs from the Ca N’Oriac center for helping us with the practical training; and the patients for their time and disposition.