Impact of the endoscopist's experience on the negative predictive value of capsule endoscopy

Abstract

Introduction

The impact of the accumulated experience of the capsule endoscopy (CE) reader on the accuracy of this test is discussed.

Aim

To determine whether the negative predictive value of CE findings changes along the learning curve.

Methods

We reviewed the first 900 CE read by 3 gastroenterologists experienced in endoscopy over 8 years.

These 900 CE were divided into 3 groups (300 CE each): group 1 consisted of the sum of the first 100 CE read by each of the 3 endoscopists; group 2, the sum of the second 100 and groups 3, the sum of the third 100.

Patients with normal CE were monitored for at least 28 months to estimate the negative predictive value.

Results

A total of 54 (18%) CE in group 1, 58 (19.3%) in group 2 and 47 (15.6%) in group 3 were normal, although only 34 patients in group 1, 38 in group 2 and 36 in group 3 with normal CE completed follow up and were eventually studied.

The negative predictive value was 88.2% in group 1, 89.5% in group 2 and 97% in group 3 (p > .05).

Conclusion

The negative predictive value tended to increase, but remained high and did not change significantly after the first 100 when readers are experienced in conventional endoscopy and have preliminary specific training.

Resumen

Introducción

La influencia de la experiencia acumulada del médico que interpreta cápsulas endoscópicas sobre su capacidad diagnóstica es discutida.

Objetivo

Determinar si existen diferencias en el valor predictivo negativo de las cápsulas endoscópicas informadas por los mismos endoscopistas a lo largo del tiempo.

Métodos

Revisamos las 900 primeras cápsulas endoscópicas realizadas por tres gastroenterólogos expertos en endoscopia durante 8 años. Se dividieron en 3 grupos de 300 cápsulas cada uno. El grupo 1 fue la suma de las tres primeras centenas informadas por cada uno, el grupo 2 la suma de las tres segundas centenas y el grupo 3 la suma de las tres terceras centenas. Se hizo un seguimiento mínimo de 28 meses a los casos con exploración normal.

Resultados

Aunque se consideraron normales el 18% de las cápsulas del grupo 1, el 19,3% de las del grupo 2 y el 15,6% de las del grupo 3, solo fue posible seguir y finalmente analizar a 34 enfermos en el grupo 1, a 38 en el 2 y a 36 en el 3. Sobre estos casos, el valor predictivo negativo fue del 88,2% en el grupo 1, del 89,5% en el grupo 2 y del 97% en el grupo 3 (p > 0,05).

Conclusión

El valor predictivo negativo de la cápsula endoscópica, aunque con tendencia a aumentar, se mantiene alto y sin diferencias significativas desde las 100 primeras exploraciones si los médicos que la interpretan son expertos en endoscopia convencional y tienen formación específica previa.

Introduction

Capsule endoscopy (CE) has become an essential test for diagnosing small bowel (SB) disease since it was approved by the Food and Drug Administration in 2001.¹ CE is safer, easier to use, and more acceptable to patients than other techniques, and as such continues to be a key tool for studying and monitoring of lesions distal to the ligament of Treitz versus other techniques.^2–8

Although indications and contraindications for CE are well documented,^1,9,10 the length of training needed to acquire competence in reading and interpreting images¹¹ and the specific learning curve associated with the technique remain unclear.^12,13 Thus, the learning curve is currently assumed to be short, provided the clinician has sufficient experience with conventional endoscopy,¹⁴ although this has not been corroborated in specific studies.¹⁵

Our aim was to determine whether the accumulated experience gained from performing an increasing number of CE studies has any impact on the diagnostic yield of the technique, measured in terms of the negative predictive value (NPV) of test results.

Patients and methods

We performed a retrospective review of the first 900 complete, clean, valid CE studies performed between December 2003 and December 2011 and interpreted by 3 gastroenterologists with wide experience in conventional endoscopy. The endoscopists were self-taught in the interpretation of CE images, but had also attended conferences on the topic and had completed at least 1 specific training course during the 12 months prior and subsequent to the introduction of the technique in our hospital.

The 900 CE studies were divided into 3 groups: group 1 (300 CEs, consisting of the sum total of the first 100 CEs interpreted by each endoscopist); group 2 (300 CEs, consisting of the sum total of the second 100 CEs interpreted by each endoscopist), and group 3 (300 CEs, consisting of the sum total of the third 100 CEs interpreted by each endoscopist).

Only CEs reported as negative were included in the negative predictive value analysis. Based on previously published recommendations,^16,17 CE studies presenting erosions, aphthae, ulcers, stenosis, oedema or denudation, tumours, or any vascular lesion of any size that could cause haemorrhage were excluded. The endoscopists were aware of each patient's medical history. Time from the order to performance of the CE did not vary significantly over the study period, and was always less than 2 months.

The Given Imaging PillCam SB and SB2 with PillCam recorder and different versions of the Rapid software provided by the manufacturer since it was marketed were used in all patients. The protocol did not differ significantly during the study period: all patients were instructed to follow a liquid diet the day before the test and to fast the night before and until 2–4 h from the start of the test, when they were allowed to drink liquids. Prokinetics or intestinal lavage solutions were only used in a few cases. All patients had previously undergone gastroscopy and colonoscopy and had signed an informed consent form. Double video images were acquired at a rate of 8–14 frames per second.

Epidemiological data (age, sex) and date the study was performed were collected from the post-CE reports. The reasons for ordering the test were grouped as overt obscure gastrointestinal bleeding (OGIB), occult OGIB, or suspected inflammatory bowel disease (IBD). Cases involving follow-up of previously diagnosed pathology and screening for asymptomatic polyposis syndrome, together with incomplete or contaminated studies, were not included in the initial evaluation. Suspected neoplasm was included in the occult OGIB group.

The medical records of all patients with negative CE were reviewed retrospectively and all were contacted by telephone for a short interview. Patient data were accessed according to the protocols used in our hospital, and the telephone interviews were conducted in 2 sessions scheduled on different days and at different times in order to contact as many patients as possible. If the patient had died or could not be located, close family members with detailed knowledge of the patient's disease were interviewed. Priority was given to the information collected from medical records; information supplied by the patient was only used when the medical record was unavailable. The study was designed to ensure that medical records were examined or patients followed up by telephone at least 28 months after the CE, this being considered sufficient to determine the NPV. The following variables were collected: persistence or absence of symptoms motivating the CE order, performance of subsequent tests related to SB symptoms after CE, presence or absence of an SB diagnosis subsequent to negative CE, and if so, whether the patient received a final diagnosis relating to any part of the digestive tract connected with the symptoms that motivated the CE order.

NPV was defined as the ability of a negative CE result to predict the absence of a SB diagnosis after at least 28 months of follow-up, plus further tests and/or surgery.

Statistical analysis

Study data were entered into a specially designed Excel spreadsheet and subsequently analysed using SPSS version 15.0. Results are shown as absolute frequencies and percentages. The χ² test was used to compare proportions of qualitative variables between groups. Statistical significance was set at p < 0.05.

Results

Indication for CE in the 900 CEs initially included in the study were overt OGIB in 41.2% of the 300 cases in group 1, in 26.3% of the 300 cases in group 2, and in 19.4% of the 300 cases in group 3. CE was order due to occult OGIB in 47.1%, 44.7% and 41.7% of patients in each of the 3 groups, respectively, and on suspicion of IBD in 11.8%, 28.9% and 38.9% if cases, respectively (Fig. 1).

Figure 1.

Indications of first 900 capsule endoscopies performed. IBD: irritable bowel disease; OGIB: obscure gastrointestinal bleeding.

In group 1, 54 of the 300 CEs performed were considered negative, 58 in group 2 (19.3%) and 47 in group 3 (15.6%). Differences in the percentage of negative CEs between groups and in the percentage of negative CEs reported by the 3 endoscopists were not statistically significant. The mean age of patients with negative CE was well matched.

After a review of the medical records, 20, 20 and 11 patients were excluded from each group, respectively, due to missing data that could not be retrieved due to 2 failed attempts to contact the patient by telephone, or the patient's refusal to participate in the study. This gave a final total of 108 patients included in the NPV analysis (34 in group 1, 38 in group 2 and 36 in group 3), with a mean age of 54 years (range: 17–96 years), of whom 39.8% were men.

Symptoms persisted in 61 (56.5%) of the 108 study subjects that underwent at least 28 months of post-CE follow-up. Of these, 27% were tested for overt OGIB, 43% for occult OGIB, and 30% for suspected IBD. Table 1 shows the follow-up results by group.

Table 1.

Results by group after a minimum follow-up of 28 months in 108 patients with negative capsule endoscopy.

	Group 1 patients with negative CE No. = 34	Group 2 patients with negative CE No. = 38	Group 3 patients with negative CE No. = 36
Persistent symptoms (p = 0.201)	44.1%	71.1%	52.8%
Further SB tests (p = 0.055)	17.4%	5.2%	2.7%
Final SB diagnosis (p = 0.330)	11.8%	10.5%	2.7%
Final GI diagnosis (p = 0.663)	23.5%	30%	33.3%

CE: capsule endoscopy; SB: small bowel.

Of the 108 patients with negative CE, a definitive diagnosis was made of GI disease in 31 (28.7%). The pathology was located in the SB in only 8.3% of cases, and in other parts of the GI tract in the remaining 20.2% (22 patients). Thus, the CE results had an NPV of 88.2% in group 1, 89.5% in group 2, and 97% in group 3, with no significant differences between groups.

Discussion

Formal CE training has not yet been fully defined.¹¹ Although experts recommend using strict, reproducible criteria to assess competency, available guidelines have only recently been published and have not been re-evaluated, and no internationally standardised training programmes have been put in place.¹⁸ Some experts have recently called for measures to make CE more cost-effective by conducting studies to determine when an endoscopist is skilled enough to independently interpret CE studies, and whether the level of experience in CE increase diagnostic accuracy (to allocate the more difficult cases to more experienced endoscopists, distribute the work of interpretation, etc.).¹⁹

It stands to reason that training in CE should follow the same structure as that required in other radiological techniques, in which there are 2 basic processes: viewing images and subsequent interpretation. The basic requirement for capsule endoscopists is experience in conventional endoscopy, and they are expected to be able to identify, interpret and document their findings using appropriate medical terminology. However, the case volume needed to achieve basic competency varies depending mainly on the individual ability of each physician, although guidelines recommend that endoscopists should perform at least 20 supervised procedures before progressing to an independent interpretation.²⁰ This recommendation is largely based on expert consensus, which also recommends that preliminary understanding of the technique should be gained from a review of CE studies and attending specific training courses and conferences (as was our case), and should include tutored video viewing and subsequent periodic objective assessments using real cases.²⁰ These same guidelines consider that expert endoscopists could acquire adequate basic skills in a 1-day training programme, acquiring further competency in a continuing professional training course of at least 8 h duration. This should be followed by a review of skills supervised by a capsule endoscopist with experience in at least 10 cases.²⁰ As in other endoscopy training curricula where thresholds of competence have been established (200 procedures for colonoscopy, 30–35 for enteroscopy^21,22), CE training should include 3 levels of experience (low, intermediate and high). Endoscopists that have reported more than 300 CE studies are considered to have achieved expert status.^23,24

A recent multicentre study, albeit with a small sample size as the authors readily admit, set the threshold for CE competence at 20 or more reported studies, although the authors recommend that competency should be achieved and assessed by gradually increasing the number of reported CE studies. Instead of following participants over time as their skills developed, however, the findings of the study were based on a structured training programme and a non-validated competency test.¹⁸

NPV after long-term follow-up provides an objective indicator of the results of a diagnostic test, and for this reason it was chosen in this study. Although a positive diagnosis has been analysed in other studies to validate the diagnostic value of CE, the disadvantages of this approach include, among others, the difficulty in interpreting some injuries (presented by up to 14% of asymptomatic individuals²⁵), the lack of uniform criteria for a specific diagnosis,^26,27 the disappearance of lesions after empirical treatment, and the failure to perform further explorations after what is considered a definitive diagnosis has been reached. NPV, moreover, is more difficult insofar as it can be affected by possible inter-group differences.

For these reasons we, like other authors, evaluated the quality of CE findings on the basis of NPV after follow-up.²⁸ Our results are consistent with those reported in other studies: an NPV of 96–100% in IBD,^10,29–31 assuming figures under 80.1% in indications for suspected occult OGIB, with a mean follow-up of 24 months.²⁸

We evaluated whether NPV varied in accordance the endoscopist's learning curve (which we assumed would improve results over time), but observed little change between the first and second group of tests, although improvement peaked in group 3. This could be due as much to the high accuracy obtained in the first 100 tests as to the sample size. The few studies that are similar to ours in the literature have mostly been published by Asian groups. A recent study from a Korean group that also investigated whether diagnostic competency in CE studies was affected by the learning curve found that 10 cases of supervised CE were sufficient for trainees to attain competency. However, the authors compared the level of agreement between trainees over a learning curve of only 15 CEs.³² Another study reports a high overall level of inter-observer agreement, particularly between gastroenterologists, over the training period. The authors suggest, in line with our findings, that adequate diagnostic competency can be acquired with moderate experience.³³

There is no clear evidence to suggest that other elements, such as SB cleanliness, transit time, or the viewing mode and speed, influence CE results.^34,35 In their cohort study, Zheng et al. concluded that the ability to detect lesions on CE was not affected by increased experience, but by other variables, such as viewing speed. However, the number of CEs previously interpreted by each endoscopist varied greatly.³⁶ Reading technique, attention, and even visual acuity are important when viewing CE images, while experience acquired in both CE and other endoscopic procedures is particularly important in interpreting findings.^21,24,37

In our study, the NPV of CE findings did not differ significantly when measured against the experience of the reporting endoscopist, although a tendency towards better results was observed, despite high accuracy in the initial 100 cases. As CE technology advances, accumulated experience may become a more significant factor.²¹

Our study has some limitations: it is retrospective, although the sample size is large and cases date from the early years of CE in Spain; indications differed in each of the 3 groups. This is because indications for CE increased as the technique developed and the test became more readily available, although any potential bias in this regard was minimised by measuring our results on the basis of NPV; basing diagnostic competency on a “no pathology” probability measure such as NPV could overestimate the capacity of the endoscopist to detect, and to a lesser extent, interpret CE findings; and finally, patients and follow-up periods differed widely, although all cases were monitored for at least 28 months.

In conclusion, the negative predictive value of capsule endoscopy is high. No significant differences were found between study groups, although a trend towards improvement was observed when the reporting physicians are experts in conventional endoscopy with specific training in CE techniques.

Conflicts of interest

The authors have no conflict of interest to declare.