The usefulness and positive predictive value of ultrasonography and computed tomography in the diagnosis of acute appendicitis in adults: A retrospective study

Abstract

Objective

To analyze the evolution of the use of imaging tests (ultrasonography and computed tomography (CT)) in the diagnosis of acute appendicitis. To determine the positive predictive value of these tests and the percentage of negative and complicated appendectomies.

Material and methods

This retrospective study compared adults who underwent appendectomy for suspected acute appendicitis at our tertiary hospital during 2015 versus similar patients at our center during 2007.

Results

A total of 278 patients were included. The rate of negative appendectomies descended to 5%. The positive predictive value of ultrasonography increased to 97.4% in 2015, and the positive predictive value of CT and combined CT and ultrasonography was 100%. The rate of complicated appendicitis increased (23% in 2015).

Conclusions

The use of imaging tests increased, and the rate of “blind” laparotomies decreased. Nevertheless, the rate of complicated appendicitis increased.

Resumen

Objetivo

Analizar la evolución del uso de pruebas de imagen (ecografía y tomografía computarizada [TC]) en el diagnóstico de apendicitis aguda, determinar el valor predictivo positivo (VPP) y el porcentaje de apendicectomías negativas y complicadas.

Material y métodos

Estudio retrospectivo que incluye a los pacientes adultos con apendicectomía por sospecha de apendicitis aguda en 2015 en un hospital terciario. Se compara con los datos obtenidos en el 2007, publicados previamente.

Resultados

La muestra incluye 278 pacientes. La tasa de apendicectomías negativas descendió a un 5%. El VPP de la ecografía aumentó a 97,4% en 2015 y el VPP de la TC y del uso combinado de la ecografía y la TC fue del 100%. El porcentaje de apendicitis complicadas se incrementó (23% en 2015).

Conclusiones

Se observó un aumento en el empleo de pruebas de imagen y una disminución de laparotomías en blanco. No obstante, las apendicitis complicadas se han incrementado.

Introduction

The diagnostic approach to acute appendicitis is currently based on signs and symptoms, laboratory results and imaging tests. Traditional reports have indicated that early diagnosis and early surgery are essential for preventing complications such as abscesses, peritonitis and sepsis of abdominal origin.

In addition, negative appendectomies are not free from postoperative complications^1,2; hence the importance of preventing those complications and the importance of the role of imaging tests.

An inverse relationship between rates of negative appendectomies and rates of complicated appendicitis has been reported, meaning that the more negative appendectomies are performed, the lower the number of cases of complicated appendicitis. The ideal situation, however, would be to decrease negative appendectomies without increasing the rate for complicated appendicitis.³

Moreover, while the rate of negative appendectomies is considered a failure in diagnostic protocols, the rate of complicated appendicitis is a healthcare quality indicator.³

At present, the use of imaging tests in the diagnosis of acute appendicitis is on the rise, yet it is not clear whether this will lead to an improvement in the management of these patients, with a decrease in the number of negative and complicated appendectomies.^4,5

A retrospective study in our hospital published in 2009 looked at the usefulness of imaging tests, the use of these tests and the rate of negative appendectomies out of all appendectomies performed in 2007.

We analysed this study,³ which found a rate of 9.6% of negative appendectomies (38 patients out of a total of 394) and a rate of 11.8% of complicated appendicitis (46 patients out of 394). That study also collected data on imaging tests used: ultrasound alone in 49.7% of cases (196 patients), computed tomography (CT) alone in 21 patients (5.4%) and ultrasound plus CT in 19 patients (4.8%).

The objectives of this study were:

• 1

  To analyse changes over time in the use of imaging tests (ultrasound and CT) in the diagnosis of acute appendicitis at our hospital.

• 2

  To determine the positive predictive value (PPV) of ultrasound, CT and ultrasound plus CT for acute appendicitis.

• 3

  To observe changes over time in the proportion of requests for imaging tests, appendectomy outcomes and hospital stays in order to evaluate whether there might be some sort of relationship between them that should be considered in a separate and/or subsequent study.

• 4

  To indirectly assess which appendectomies were performed without imaging tests (under our hospital's protocol in 2007, imaging tests were not done in young men 20–40 years of age with suspected appendicitis).

To do this, the data collected were compared to the data from the study conducted in 2007, in the same population and an equivalent time period. With the results obtained, given the lack of established, internationally agreed-upon protocols, clinical guidelines for the use of imaging tests in acute and complicated appendicitis will be proposed.

Material and methods

This retrospective study enrolled all patients over 14 years of age who underwent an emergency appendectomy at a tertiary university hospital between 1 January and 31 December 2015.

Elective appendectomies and appendectomies associated with other surgical procedures, even emergency ones, were excluded.

The patient sample was drawn from the hospital's administrative database, and pathology reports, surgical protocols and radiology reports were assessed. Variables concerning age, sex, date of admission, date of discharge and imaging tests performed were collected. Acute appendicitis was confirmed by histology and defined as the presence of acute transmural inflammation of the appendix (classified as phlegmonous or gangrenous appendicitis depending on its extent and severity). A negative appendectomy, for its part, was defined by the absence of this finding. Findings of other appendiceal diseases were not considered negative appendectomies. Complicated appendicitis was defined as the gangrenous or microscopically perforated form determined by histological analysis or complications with an appendiceal abscess and/or perforation and/or mass (appendicular plastron).

At our hospital, the process of requesting imaging tests in cases of acute appendicitis was not restrictive; imaging tests were requested in cases of right iliac fossa (RIF) pain, fever, leukocytosis and/or elevated C-reactive protein (CRP) levels, especially in women, in whom the differential diagnosis of RIF pain is broader. In addition, in elderly people (over 65 years of age), an imaging test was done even in the absence of leukocytosis and/or elevated CRP levels.

The results were compared to those obtained from 1 January to 31 December 2007 in the same population and data source.

Ultrasounds were performed with a Toshiba Aplio ultrasound machine, model TUS-A500 (Toshiba Medical Systems Corporation, Otawara, Japan), with a targeted examination of the RIF using a gradual compression technique with a 14-MHz probe, following a complete examination of the abdomen and pelvis with a 6-MHz probe. The ultrasounds were done by attending and/or third- or fourth-year resident physicians from the radiology department. The results were classified as: suggestive of acute appendicitis, not suggestive of acute appendicitis or inconclusive. Findings suggestive of acute appendicitis included: appendix with a maximum transverse diameter exceeding 6 mm, a maximum wall thickness of 3 mm, non-compressibility, pain on palpation, changes in layers and periappendiceal hyperechogenic fat (Fig. 1). Ultrasounds lacking these findings were classified as not suggestive of acute appendicitis. Inconclusive ultrasounds included those in which the caecal appendix was not visualised, the caecal appendix was not visualised in its entirety, it was concluded that the imaging findings had to be correlated with the clinical context, or the ultrasound was deemed uncertain. We did not take inconclusive ultrasounds into account in calculating the PPV of ultrasound, since they were not taken into account in the prior study on which this study was based. As a result, the PPVs of ultrasound in the two periods can be compared.³

Figure 1.

Findings suggestive of typical acute appendicitis. Abdominal ultrasound of the right iliac fossa: appendix of enlarged calibre showing a thickened submucosa (white arrows) associated with increased echogenicity of periappendicular fat (asterisks).

Abdominal CT scans were done using a Philips Brilliance 16-slice helical CT scanner (Philips Medical Systems, Eindhoven, Netherlands) with image acquisition from the domes of the diaphragm to the pubic symphysis with 3-mm collimation and 1.5-mm reconstruction, using non-iodinated intravenous contrast calculated based on patient body weight (2 cc/kg of body weight with contrast of 300 mg iodine/mL) in the portal venous phase (70-s fixed delay). Intravenous contrast was not used in patients with kidney failure or contrast allergy. Oral contrast was not used in any of the cases.

Statistical analysis was performed with the SPSS software program (IBM, Inc.). To compare the 2007 data (contributed by the authors) and the 2015 data, Student's unpaired t test was used for continuous variables, with a normal distribution, and the χ² test or Fisher's exact test was used for categorical variables. A p value <0.05 was considered statistically significant.

Results

In 2015, surgery was performed in a total of 278 cases of acute appendicitis, versus 394 cases in 2007.

The characteristics of the two populations are described in Table 1. Mean hospital stay was also included; this showed a statistically significant decrease between the two periods, from 4.6 days in 2007 to 2.8 days in 2015 (p < 0.005).

Table 1.

Demographic characteristics and mean hospital stay of the two samples.

Characteristics	2007 population (n = 394)	2015 population (n = 278)	p
Sex (M/F)	225/169	143/135	0.154
Mean age (years)	34 ± 15	36 ± 18	0.190
Mean stay (days)	4.6 ± 3.9	2.8 ± 3.4	0.005

F: female; M: male. Data are expressed in terms of percentage or mean ± SD.

In 2007, the percentage of normal appendices or negative laparotomies was 9.6% (38 patients). In 2015, it was practically half that, at 5% (13 patients). This decrease was statistically significant (p < 0.01) (Table 2). Other pathology diagnoses were made in five patients (2%): one case of endometriosis of the appendicular wall (Fig. 2), one case of acute appendicular diverticulitis, one case of endometrioid adenocarcinoma, one case of appendicular deciduosis in a pregnant patient and, finally, one case of chronic inflammatory changes in a patient with an interval appendectomy. These cases were not considered negative appendectomies despite the absence of acute appendicitis, since appendiceal disease was present and indistinguishable from acute appendicitis on imaging tests. Regarding the imaging tests done in these cases, to calculate the PPV of these, only appendicular endometriosis, indistinguishable on imaging from acute appendicitis, was considered a false positive for ultrasound. In the case of appendicular deciduosis, an appendectomy was performed even though the ultrasound report was not suggestive of acute appendicitis. Therefore, we considered it neither a true positive nor a false positive ultrasound. In any case, it would be a true negative ultrasound result for acute appendicitis, but calculation of negative predictive value (NPV) was not the objective of our study. The case of acute appendicular diverticulitis and the case of adenocarcinoma of the appendix were diagnosed by CT, and although they were correctly diagnosed, they too were not included in the calculation of the PPV of ultrasound plus CT for acute appendicitis (for the same reasons, they would be true negatives). In the case of the interval appendectomy, the patient did not undergo any imaging tests.

Table 2.

Pathology of appendectomies.

Pathology	2007 population (n = 394)	2015 population (n = 278)
Appendix without transmural inflammation	38 (9.6%)	13 (5%)
Acute appendicitis	338 (85.8%)	260 (93%)
Other^a	18 (4.6%)	5 (2%)

χ² = 9.13, p = 0.010. Data are expressed in terms of percentage.

a

Others in 2015 period: endometriosis of the appendicular wall, acute appendiceal diverticulitis, appendicular deciduosis, endometrioid adenocarcinoma and chronic inflammatory changes in a patient with an interval appendectomy.

Figure 2.

Endometriosis of the appendicular wall. Abdominal ultrasound of the right iliac fossa showing imaging findings indistinguishable from acute appendicitis: hypoechoic appendix with loss of layer integrity (white arrows), increased echogenicity of periappendicular fat (asterisk) and an appendicolith (dotted line bounded by Xs). The definitive diagnosis was based on pathological findings.

Cases of complicated appendicitis increased significantly (p < 0.01), from 14% in 2007 (46 patients out of a total of 338 patients with appendicitis) to 23% in 2015 (60 patients out of 260 patients with appendicitis) (Table 3). Within the group of cases of complicated appendicitis in 2015, 45 were gangrenous and 15 were perforated. Of these, five were associated with a collection, two were associated with an inflammatory plastron and one was associated with generalised peritonitis with pneumoperitoneum.

Table 3.

Severity of cases of acute appendicitis.

Appendicitis	2007 population (n = 338)	2015 population (n = 260)
Phlegmonous	292 (86%)	200 (77%)
Gangrenous	46 (14%)	60 (23%)

χ² = 9.57, p = 0.002. Data are expressed in terms of percentage.

Fig. 3 shows the pathology results for the appendectomies performed in the two periods compared.

Figure 3.

Graph of the pathology results of the appendectomies performed in the two periods.

The use of imaging tests in appendectomy patients has greatly increased in recent years. In 2007, imaging tests were done in 59.9% of these patients, and in 2015, imaging tests were done in 87% (Table 4). Among the patients who did not undergo imaging tests in this study, 82% were male patients under 45 years of age, for whom imaging tests were not requested.

Table 4.

Variation in the use of imaging tests in the two periods compared.

Imaging test	2007 population (n = 394)	2015 population (n = 278)
None	158 (40.1%)	37 (13.2%)
Ultrasound	196 (49.7%)	200 (71.4%)
CT alone	21 (5.4%)	11 (4.3)
Ultrasound plus CT	19 (4.8%)	30 (11.1%)

χ² = 62.98, p < 0.001. Data are expressed in terms of percentage.

The increase in the use of imaging tests in appendectomy patients has come with greater use of ultrasound as a diagnostic test: 71.4% in 2015 versus 49.7% in 2007.

Of the 230 ultrasounds performed in 2015 in appendectomy patients, in 33 cases (14.7%) the patient's ultrasound was inconclusive, and in five cases (2.1%) the patient was misdiagnosed. By comparison, of the 215 ultrasounds performed in appendectomy patients 2007, in 15 cases (7%) the patient's ultrasound was inconclusive, and in 22 cases (10.2%) the patient was misdiagnosed. The PPV of ultrasound was 97.4% (95% confidence interval [CI]: 94.2–98.9) in 2015 — higher than in 2007, which was already high, at 92.2% (95% CI: 87.5–95.2).

Of the 33 patients undergoing appendectomy following an inconclusive ultrasound in 2015, in 11 cases the appendectomy was performed directly and in the 22 remaining cases the study was completed with a CT scan.

The use of ultrasound followed by CT has doubled (from 4.8% in 2007 to 11.1% in 2015) in appendectomy patients. This diagnostic approach was applied to 30 patients in 2015, in 22 patients due to an inconclusive ultrasound and in eight patients to assess the extent of complicated acute appendicitis detected on ultrasound. In 20 cases, the diagnosis of acute appendicitis was confirmed by CT, in line with the surgery and pathology findings. Hence, the PPV in 2015 for sequential use of ultrasound and CT was 100% (95% CI: 83.8–100), higher than in 2007, when it was 94.7% (95% CI: 75.3–99). In two cases, CT diagnosed other diseases also confirmed by pathology (appendicular diverticulitis and endometrioid adenocarcinoma); although the diagnoses were correct, we did not take them into account in the calculation of the PPV. Regarding the eight patients undergoing appendectomies in whom CT was used to assess the extent of appendicitis, we considered them true positives on ultrasound, since in these cases CT added no diagnostic value and was performed simply to assess the extent and/or possible surgical complications.

CT was used in very few cases as a sole diagnostic test, with a slight, non-statistically significant drop between the two periods (5.4% in 2007 and 4.3% in 2015). The use of imaging tests in both periods is shown in Fig. 4. Of the 11 CT scans performed directly without a prior ultrasound in 2015, all were true positives (Fig. 5). Hence, the PPV of CT did not vary, and was 100% (95% CI: 74.1–100).

Figure 4.

Use of imaging tests in appendectomies due to suspected acute appendicitis.

Figure 5.

Typical findings of acute appendicitis in computed tomography with intravenous contrast: distended appendicular lumen and increased appendicular wall uptake (white arrow), intra-appendicular appendicoliths (small yellow arrows) and free fluid in the pelvis (asterisk).

Fig. 6 shows data on the management — including requests for imaging tests (or the absence thereof), pathology tests and their results — of the patients who underwent surgery with suspected acute appendicitis in 2015. Tables 5 and 6 show the PPVs of ultrasound, CT and ultrasound followed by CT in the periods compared.

Figure 6.

Management of patients having undergone surgery with suspected acute appendicitis in relation to whether imaging tests were requested or not and pathology results at our hospital in 2015. AGA: acute gangrenous appendicitis; APA: acute phlegmonous appendicitis; NA: negative appendectomy.

Table 5.

Predictive positive values for 2015.

Imaging test	True positives	False positives	Positive predictive value (%)
			95% CI
Ultrasound*	191	5	97.4 (94.2−98.9)
CT	11	0	100 (74.1−100)
Ultrasound plus CT**	20	0	100 (83.8−100)

*

Neither inconclusive ultrasounds nor appendicular deciduosis were considered true negatives for ultrasound. The eight ultrasounds supplemented with CT to assess extent and/or complications were included as true positives, since in those cases a diagnosis of acute appendicitis had already been made by ultrasound. False positives: include appendicular endometriosis and negative appendectomies.

**

The eight ultrasounds supplemented with CT to assess extent were not included, nor were the cases of appendicular diverticulitis or endometrioid adenocarcinoma; the latter two were true negatives on CT.

Table 6.

Predictive positive values for 2007.

Imaging test	True positives	False positives	Positive predictive value (%)
			95% CI
Ultrasound*	178	15	92.2 (87.5−95.2)
CT	21	0	100 (84.5−100)
Ultrasound plus CT	18	1	94.7 (75.3−99)

*

Inconclusive ultrasounds were not included. There were 22 inconclusive ultrasounds altogether: 19 were supplemented with CT and are included in Ultrasound plus CT, and in the remaining three cases, surgery was performed directly.

Discussion

The data obtained in our study were consistent with the current trend in the management of acute appendicitis, which has translated to a very significant increase in the use of imaging tests in appendectomy patients. This increase was obvious in our comparative study, as we found that the use of imaging tests rose from 59.9% in 2007 to 87% in 2015. Other studies such as those by Sartelli et al.⁶ and Tseng et al.,⁷ with a significant sample of patients, confirmed this trend with preoperative imaging tests in 70% and 93.9% of cases, respectively.

Analysis of our results revealed that this increase essentially came with an increase in the use of ultrasound compared to CT. Despite our good experience with ultrasound, its systematic use remains surprisingly subject to doubts in the scientific literature, and some recent studies have continued to question its usefulness.^2,4,8

The use of CT is more widely accepted, and many authors have reported mainly using CT for the diagnosis of acute appendicitis.^2,9–11 However, it must not be forgotten that CT involves a concerning, and growing, use of radiation in a disease that primarily affects children and young adults.

Ultrasound was consolidated in our study as a first initial diagnostic test, improving PPV from 92% in 2007 to 97.4% in 2015, coinciding with other studies,^12–15 and is even higher it is grouped into presumptive diagnostic categories, as shown in a recent study by Benedetto et al.¹⁶ However, Giljaca et al.⁸ and Fields et al.¹⁷ in separate meta-analyses question its usefulness, due to differences in sensitivity attributable, among other causes, to the experience of the operator.

In our study, ultrasound had a high PPV (97.4%), although 14.7% of ultrasounds were inconclusive. CT does not have this limitation; whether it is done in isolation or combined with ultrasound, it has a PPV of 100%. However, CT involves the use of ionising radiation, which is of particular concern in young patients. In addition, the very high PPV obtained for CT according to the study design did not reflect its limitations well, since in one case not included in the study because acute appendicitis was not suspected, CT did not manage to demonstrate it as a cause of ileus (Fig. 7). Therefore, our proposed diagnostic algorithm is to perform ultrasound as an initial diagnostic test in all cases and to supplement it with CT in inconclusive ultrasounds.

Figure 7.

A 32-year-old man who visited the emergency department with abdominal pain, fever and vomiting, in whom bowel perforation was suspected. A and B) Axial images of computed tomography with intravenous contrast in venous phase. Distended caecal appendix with intraluminal gas (arrow) and small appendicolith of low attenuation (arrow tip), which went unnoticed. Note the distended small bowel loops, probably in relation to local paralytic ileus. Surgery and pathology tests showed acute gangrenous appendicitis.

Efforts to decrease radiation doses have yielded various diagnostic strategies including low-dose protocols, focused protocols and, more recently, the use of a low-voltage tube with dual-energy CT.^18–21

Despite the increase in imaging tests, there is still a group of patients who undergo neither ultrasound nor CT. These patients in our study were mostly young males (82%). This group might be the subject of a future study conducted with a view to determining whether this subgroup of patients really benefit from imaging tests or they can represent a diagnostic delay, since the differential diagnosis of RIF pain in these patients is very limited.

It has been reported that the morbidity of negative appendectomies is non-negligible and leads to lengthened hospital stays, increased postoperative infectious complications (2.6%) and increased mortality (1.5%), with the clinical and economic burden all these things entail.²² In this regard, our study found that the use of imaging tests in the diagnosis of acute appendicitis could bring about a significant drop in rates of negative appendectomies, since these decreased practically by half (from 9.6% in 2007 to 5% in 2015). These data were consistent with reports in the scientific literature in which rates of negative appendectomies have ranged from 4% to 7%.^7,10,23 However, they were not consistent with other studies such as those by Flum et al.²⁴ and Markar et al.,²⁵ either because those studies did not confirm the decrease in the rate of negative appendectomies or the reduction was not significant.

Mean hospital stay has become significantly shorter (p < 0.01), dropping from 4.6 days in 2007 to 2.8 days in 2015. This figure was consistent with the mean hospital stay of three days reported in the scientific literature.⁶ We believe that this decrease may have been due to the higher percentage of laparoscopic surgery performed in 2015.

Despite the improvement in these indicators, our study showed a statistically significant increase in the percentage of cases of complicated appendicitis, which went from 14% in 2007 to 23% in 2015. We do not have a clear explanation of the cause of the increase in cases of complicated appendicitis in our series. However, the literature offers various possibilities. One is that the delay resulting from the use of diagnostic tests is in itself the reason for that increase, as reported by Musunuro et al.²⁶ Another is that it is related to the time from the onset of symptoms to surgery, regardless of the use or non-use of imaging tests, as indicated by Bickell et al.,²⁷ who demonstrated in the pre-CT era that the risk of perforation in patients with less than 36 h since the onset of symptoms was less than 2%, and increased by 5% with every 12 h of delay once those first 36 h had elapsed.

However, other studies such as one by March et al.²⁸ found no significant differences in rates of complicated appendicitis when comparing two patient groups: one undergoing early appendectomy and the other undergoing appendectomy after 48 h. In the same vein, a study by Sauvain et al.²⁹ demonstrated that a delay in imaging tests (ultrasound and CT) did not represent an increase in cases of perforated/complicated appendicitis. This study took into account the time that elapsed between hospital admission and surgery. This was on average six hours, increased to seven hours with the use of CT and/or ultrasound, and was 4.3 h when surgery was indicated directly, with no diagnostic imaging tests.

We were unable to collect data on the time elapsed between the onset of symptoms and surgery, and this may be a limitation of our study. However, in line with Krajewski et al.,³⁰ we believe that the rate of cases of perforated appendicitis should not be influenced only by a delay in diagnosis due to imaging tests, since in our case they were performed in the emergency radiology unit on a high-priority basis.

Other factors, or this combined with other factors, must be at play. A study by Chen et al.³¹ proposed age (over 55 years) as a sole predictive factor of perforation. Other factors have recently been reported, such as differences in microbiology,³² differences in immune response³³ and even factors related to different ethnic groups.³⁴

Our study was limited by its retrospective nature and by its design, since we were unable to calculate sensitivity, the NPV of the diagnostic tests or the positive likelihood ratio, because the study was conducted based on data from appendectomies performed with a diagnosis of appendicitis. We were also unable to determine the time from the onset of symptoms to surgery — which could be a factor responsible for the increase in cases of complicated appendicitis — since this information is not always accurately registered in medical records. We were unable to obtain patients' body mass index, which would have enabled us to determine the reasons why CT was directly requested for certain patients, given that ultrasound has significant limitations in obese patients. This, added to differences in age and sex distribution between the two samples, means that the possibility of bias could not be ruled out and, as a result, the PPVs of the tests could not be compared to one another. In addition, although there were no significant differences between the 2007 population and the 2015 population, the 2007 population was younger and featured a higher proportion of male patients. This might have helped to account for why fewer imaging tests were requested in 2007. On the other hand, although the study data were from 2015, comparing them to the data previously obtained in 2007 gave us a perspective on changes over an eight-year period in the use of imaging tests in the diagnosis of acute appendicitis. We believe this use continues today, though this could be examined in a future study.

The use of imaging tests has increased in patients with suspected acute appendicitis who undergo appendectomy. At the same time, a decreased rate of negative laparotomies and a shortened mean hospital stay were observed. However, the percentage of complicated appendicitis was found to have not just not decreased, but increased. Ultrasound prevails as a first-line diagnostic imaging test, with CT reserved for cases in which ultrasound is uncertain or inconclusive. This diagnostic approach is very efficient, since it not only has a very high PPV but also makes it possible to reduce the use of ionising radiation in these patients.

Authorship

• 1

  Responsible for study integrity: MAD and SBN.

• 2

  Study concept: MAD, EMC and SBN.

• 3

  Study design: MAD, EMC, LIS and SBN.

• 4

  Data collection: MAD, ERC, EMC and SBN.

• 5

  Data analysis and interpretation: MAD, EMC, JCAM and SBN.

• 6

  Statistical processing: MAD, LIS and SBN.

• 7

  Literature search: MAD, EMC, LIS and SBN.

• 8

  Drafting of the article: MAD, EMC and SBN.

• 9

  Critical review of the manuscript with intellectually significant contributions: MAD, EMC, LIS, JCAM, ERC and SBN.

• 10

  Approval of the final version: MAD, EMC, LIS, JCAM, ERC and SBN.

Conflicts of interest

The authors declare that they have no conflicts of interest.