Outcome Quality Standards in Pancreatic Oncologic Surgery in Spain

Abstract

Introduction

To establish quality standards in oncologic surgery is a complex but necessary challenge to improve surgical outcomes. Unlike other tumors, there are no well-defined quality standards in pancreatic cancer. The aim of this study is to identify quality indicators in pancreatic oncologic surgery in Spain as well as their acceptable limits of variability.

Methods

Quality indicators were selected based on clinical practice guidelines, consensus conferences, reviews and national publications on oncologic pancreatic surgery between the years 2000 and 2016. Variability margins for each indicator have been determined by statistical process control techniques and graphically represented with the 99.8 and 95% confidence intervals above and below the weighted average according to sample size.

Results

The following indicators have been determined with their weighted average and acceptable quality limits: resectability rate 71% (>58%), morbidity 58% (<73%), mortality 4% (<10%), biliary leak 6% (<14%), pancreatic fistula rate 18% (<29%), hemorrhage 11% (<21%), reoperation rate 11% (<20%) and mean hospital stay (<21 days).

Conclusions

To date, few related series have been published, and they present important methodological limitations. Among the selected indicators, the morbidity and mortality quality limits have come out higher than those obtained in international standards. It is necessary for Spanish pancreatic surgeons to adopt homogeneous criteria regarding indicators and their definitions to allow for the comparison of their results.

Resumen

Introducción

Identificar los estándares de calidad en la cirugía oncológica es un reto complejo necesario para poder mejorar los resultados quirúrgicos. A diferencia de lo que ocurre en otros tumores, no existen unos estándares de calidad bien definidos en el cáncer de páncreas. El objetivo es identificar los indicadores de calidad en la cirugía pancreática oncológica en España, así como sus límites de variabilidad.

Métodos

Los indicadores de calidad se han seleccionado a partir de las guías de práctica clínica, conferencias de consenso, revisiones y publicaciones de ámbito nacional sobre cirugía pancreática oncológica entre los años 2000–2016. Los márgenes de variabilidad para cada indicador se obtienen mediante estadística de control de procesos y gráficas de representación, teniendo en cuenta el tamaño de las series. Los límites de variabilidad se establecen a partir de la media y los intervalos de confianza al 95 y al 99,8%.

Resultados

Se han determinado los siguientes indicadores con sus medias y límites de calidad aceptables: tasa de resecabilidad 71% (> 58%), morbilidad 58% (< 73%), mortalidad 4% (< 10%), tasa de fístula biliar 6% (< 14%), tasa de fístula pancreática 18% (< 29%), hemorragia 11% (< 21%), tasa de reintervención 11% (< 20%) y estancia media (< 21 días).

Conclusiones

El número de series publicadas es muy escaso y presentan limitaciones metodológicas importantes. Entre los indicadores seleccionados, los límites de calidad de morbimortalidad han resultado más elevados que los obtenidos en los estándares internacionales. Es necesario que los cirujanos pancreáticos españoles adopten unos criterios homogéneos consensuados de los indicadores y su definición que permitan comparar sus resultados.

Introduction

Pancreatic cancer is the fourth cause of cancer death in Spain, and surgical resection is the only treatment that is able to increase survival.^1,2 It is always a challenge for surgeons due to the technical difficulty of the procedure, and although mortality has been reduced in recent years, the rate of postoperative complications is still very high.³ Given the complexity of pancreatic surgery, it is necessary to establish standards that allow surgeons to assess the quality of the treatment provided and to consider what aspects should be improved, while comparing their results with those of other groups and specialized units. There is a great debate about which measurements should be used to determine surgical quality, as there are currently indicators for structure (case volume, availability of intensive care units or interventional radiology), procedure (perioperative antibiotic prophylaxis) and results (morbidity, mortality, pancreatic fistula rate).⁴

Standards would define the acceptable ranges of quality for a certain process and establish minimum permissible limits for specific indicators. This entails comparing the results from the management of a specific disease by means of measurable, valid and relevant indicators.⁵ Unlike what happens in other types of diseases where quality standards have been established (such as breast or colorectal cancer), in pancreatic cancer surgery there are no well-defined national standards, and only one study has published recommendations based on subjective criteria.⁶ Although approximations have been made based on objective international criteria,⁷ quality standards are influenced by multiple factors and those obtained in one country may not coincide with criteria established in another country or with those based on international reports. Therefore, it is necessary to define national quality standards and their variability ranges in order to establish what can be considered acceptable or unacceptable in the management of this disease.⁸ Consequently, the objective of our research has been to identify quality indicators for pancreatic cancer surgery in Spain, as well as its limits of variability according to a standardized methodology.

Material and Methods

Research Strategy, Selection of Quality Indicators and Studies

The research process was carried out in 2 parts. First, it was necessary to identify which quality indicators have clinical relevance in pancreatic cancer surgery. To this end, a systematic search was conducted for clinical practice guidelines,^9–15 consensus conferences^16–19 and review studies about the quality of pancreatic cancer surgery and the development of indicators^5,20–25 between 2000 and 2016 in MEDLINE/PubMed, Embase and Cochrane Library. Second, we proceeded with the systematic search of clinical research studies on oncological pancreatic surgery published in Spain between January 2000 and December 2016 that included a minimum of 20 surgical patients. To be selected, an indicator needed to have the following characteristics: be clinically relevant, so that it has an impact on prognosis or postoperative evolution; be clearly mentioned in the studies identified in the second part of the research, or values were able to be calculated easily from the data provided; and, appear with sufficient frequency to be able to be evaluated.

Statistical Analysis and Determining Standards

The standards for each quality indicator were determined by statistical techniques based on statistical process control.^26–28 Control charts for the quantitative variables (X-charts) and control charts for attributes for the qualitative variables (P-charts) were made, representing the studies according to the number of cases included in each series from lowest to highest volume of patients. In the graphs, the main horizontal line corresponds with the average result of all the units compared, weighted mathematically by the volume of cases that each one contributes to the analysis; the limits of variability are represented by lines corresponding with the 99.8% confidence intervals (±3 standard errors) and 95% (±2 standard errors), respectively. Any result outside these limits deviates significantly from the weighted average (P < .002 and P < 1.005, respectively) and is considered out of control according to the terminology of statistical process control. In our study, the acceptable quality limits for each indicator are defined within the 99.8% range. Therefore, if a result is within these limits, it is considered to be within quality standards. The 95 and 99.8% confidence intervals are limits that designate areas of alarm (99.8%) and precaution (95%); if a result is above or below them, it should be interpreted depending on the parameter we are measuring. Thus, for example, in the case of pancreatic fistula rate, if a result is below the limit of variability, it can be considered an excellent result; in contrast, in the case of the resectability rate, if the result is below the alarm zone, it is a negative result.

Results

Quality Indicators and Selected Studies

The quality indicators selected for oncological pancreatic surgery were resectability rate, morbidity, mortality, pancreatic fistula rate, biliary fistula rate, hemorrhage, reoperation rate and mean hospital stay. A total of 20 series were identified,^29–48 5 of which^29,33,34 were excluded because they did not include a minimum of 20 patients. The results of the 15 selected series are shown in Table 1.

Table 1.

Spanish Series of Pancreatic Cancer Surgery (2000–2016).

References	Year	Patients, n	Resectability	Morbidity	Mortality	Bile duct fistula	Pancreatic fistula (definition)	Pancreatic fistula rate	Hemorrhage	Reoperation	Hospital stay
García-Plata et al.^32	2005	58		49.80	13.70	8.60	Amylase in discharge	22.40	5.17	20.60	–
Balsells-Valls et al.^35	2006	97	41.99	74.55	14.35	–	Not defined	22.50	26.70	6.40	26
Jover et al.^36	2006	80		46.70	6.60	6.60	Amylase×3	20.00	7.50	7.50	21
Fernández-Cruz et al.^37	2008	108		33.00	0.00	1.00	ISGPF	12.00	2.00	1.85	14
Figueras et al.^38	2008	56		72.00	5.40	4.50	ISGPF	12.50	7.00	27.50	22
Sabater et al.^39	2009	124	77.50	38.70	4.00	5.50	Amylase×3	6.25	6.45	3.80	17
Montiel et al.^40	2010	82		55.65	5.75	9.50	Amylase×3	11.40	8.40	10.45	10
Busquets et al.^41	2010	204		45.00	7.00	5.00	ISGPF	10.00	11.76	13.00	20
Figueras et al.^42	2013	123		64.20	5.00	6.00	ISGPF	24.50	16.00		14
Dominguez-Comesaña et al.^43	2013	117		48.71	5.13	7.89	Amylase×3	9.90	11.10	14.53	–
Herrera-Cabezón et al.^44	2015	480		58.05	1.05	–	ISGPF	22.50	–	10.15	17
Rubio-Manzanares Dorado et al.^45	2015	151	83.40	100.00	0.66	–	Not defined	–	–	10.50	18
Landi et al.^46	2015	78		58.97	5.00	10.25	ISGPF	11.50	12.82	8.97	19
Sánchez Cabús et al.^47	2015	129		77.50	4.70	7.80	ISGPF	34.80	14.70	12.40	24
Morales Soriano et al.^48	2015	85		42.30	2.40	3.50	ISGPF	16.45	–	11.50	16

All the indicators present as percentages, except hospital stay, which is shown in days.

Resectability Rate

Only three series,^35,39,45 with a total of 372 patients, reviewed the resectability rate. The weighted average was 71% and the acceptable quality limit was >58% (Fig. 1A).

Fig. 1.

Rate of resectability (A), morbidity (B) and mortality (C); each point represents a study included in Table 1.

Green: within standard limits with variability due to chance; yellow: warning zone but still within the 95% confidence interval; red: over the 99.8% limit and the results cannot be randomly attributed to the process analyzed.

Morbidity

Fifteen series,^35–48 with a total of 1972 patients, included data on morbidity. The weighted average of this indicator was 58% and the acceptable quality limit was <73% (Fig. 1B).

Mortality

Mortality was evaluated in a total of 15 series,^35–48 with a total of 1972 patients. In these series, the weighted average mortality was 4%, with an acceptable quality limit of <10% (Fig. 1C).

Pancreatic Fistula Rate

Pancreatic fistula rate was studied in 14 series^{32,35–44,46–48} for a total of 1821 patients. The weighted average was 18%, with an acceptable quality limit of <29%. It is important to note that there was great variability depending on the fistula definition used. Thus, the 14 studies that referred to pancreatic fistula rate presented several different definitions, with a variation among the rates reported between 6.25 and 34.8%.

The quality indicators selected for pancreatic cancer surgery were resectability rate, morbidity, mortality, pancreatic fistula rate, biliary fistula rate, hemorrhage, reoperation rate and mean stay. A total of 20 series were identified,^29–48 5 of which^{29–31,33,34} were excluded because they did not include a minimum of 20 patients. The results of the 15 selected series are shown in Table 1.

When the pancreatic fistula rate was calculated in the series that followed the ISGPF criteria,^{37,38,41,42,44,46–48} the weighted average was 19%, while the acceptable quality limit was <31% (Fig. 2).

Fig. 2.

Rate of pancreatic fistula (A) and pancreatic fistula with ISGPF criteria (B); each point represents a study included in Table 1.

Green: within standard limits with variability due to chance; yellow: warning zone but still within the 95% confidence interval; red: over the 99.8% limit and the results cannot be randomly attributed to the process analyzed.

Biliary Fistula Rate

A total of 12 series^{32,36–43,46,48} including 1244 patients were evaluated to determine the rate of biliary fistula. The weighted average was 6%, with an acceptable quality limit of <14% (Fig. 3A).

Fig. 3.

Rate of biliary fistula (A) and hemorrhage (B); each point represents a study included in Table 1.

Green: within standard limits with variability due to chance; yellow: warning zone but still within the 95% confidence interval; red: over the 99.8% limit and the results cannot be randomly attributed to the process analyzed.

Hemorrhage

Hemorrhage of any origin (gastrointestinal, intra-abdominal) was evaluated from 12 series,^{32,35–43,46,47} totaling 1256 patients. The weighted average for hemorrhage was 11%, with an acceptable quality limit of <21% (Fig. 3B).

Reoperation Rate

The rate of reoperation was studied in 18 series,^{32,35–41,43–48} with a total of 1849 patients. The mean reoperation rate was 11%, with an acceptable quality limit of <20% (Fig. 4A).

Fig. 4.

Rate of reoperations (A) and average stay (B); each point represents a study included in Table 1.

Green: within standard limits with variability due to chance; yellow: warning zone but still within the 95% confidence interval; red: over the 99.8% limit and the results cannot be randomly attributed to the process analyzed.

Mean Hospital Stay

Mean hospital stay was calculated from 13 series^{35–42,44–48} with a total of 1797 patients. The mean stay was 18 days, with an acceptable quality limit of <21 days (Fig. 4B). The summarized description for each indicator with the weighted average and its limits of variability are shown in Table 2.

Table 2.

Summarized Description of the Results of the Quality Indicators.

	Weighted mean	99.8% CI	95% CI	Quality limit	Cases, n
Rate of resectability	71	83–58	79–62	>58	372
Morbidity	58	73–43	68–49	<73	1972
Mortality	4	10–0	8–0	<10	1972
Pancreatic fistula rate	18	29–7	25–10	<29	1821
Bile duct fistula rate	6	14–0	11–1	<14	1244
Hemorrhage	11	21–2	18–5	<21	1256
Rate of reoperation	11	20–2	17–5	<20	1849
Mean hospital stay	18	21–15	20–16	<21	1797

All the indicators are presented as percentages, except mean hospital stay, which is shown in days.

Discussion

Pancreatic oncological resection is a technically complex procedure associated with a high incidence of postoperative complications. Therefore, the identification of quality standards is a very important element to reduce variability in surgical practice, improve results and optimize available resources.^4,8 Research in quality standards is a difficult multidisciplinary challenge that includes not only structure and results indicators, but also process indicators.^4,5 These indicators allow the surgeon to assess the results of the treatment performed, as well as to enable the comparison of these with other groups or specialized units.

Until now, the limited antecedents of proposed standards in hepatobiliary surgery in our healthcare system focused on perioperative morbidity and mortality and case volume. This is similar to the study published by Figueras et al.⁶ in 2002, which established standards for mortality and morbidity (<10 and <50%, respectively) for periampullary tumors, below which they do not recommend performing these procedures, as well as a minimum number of annual procedures that should be performed at the hospital (24 annual pancreaticoduodenectomies). Recently, Herrera-Cabezón et al.⁴⁴ have reviewed a series of 480 pancreatic resections in comparison with international standards,⁷ with excellent results in terms of compliance with quality criteria. In the present study, we have tried to identify the quality standards and their limits of acceptable variability in pancreatic cancer surgery in Spain, based on publications in our country. Compared with data from the international literature,⁷ we found a similar resectability rate, but the acceptable quality limits for morbidity and mortality indicators are higher in the Spanish series: <73 vs <55% and <10 vs <5%, respectively.

The most important problem that arises when carrying out a study with this methodology is that it depends on the information that the different researchers have considered relevant to publish and is available in the series. Thus, there is information that is considered very important from the point of view of quality, such as the annual volume of cases, the R0 resection rate, overall survival or disease-free survival which, since it is not available in a minimum number of articles, cannot be analyzed. In this sense, this present study contrasts with the international standard study, where such information was available.⁷ Another difficulty that must be highlighted is the lack of definitions that different authors consider to assess and present their results. Although there is already consensus about the definitions of the majority of complications,^49–51 these have not been used in the published Spanish series, except for pancreatic fistula rate, which considered the ISGPF criteria in 8 of the 15 series. This implies that the incidence of complications shows great disparity, depending on the definition applied in each study. Despite these limitations, this study represents an initial attempt at obtaining indicators and their acceptable variability in oncological pancreatic surgery in our setting.

Furthermore, an additional element that our research highlights is the limited number of pancreatic surgery series published in Spain during the last 16 years, many of which with a small number of patients: Only since 2008 have series been published with more than 100 patients. In conclusion, we have defined a series of indicators in pancreatic cancer surgery in Spain, as well as their quality limits, according to a standardized methodology. Our research shows the limited number of series published, which present important methodological limitations, highlighting the heterogeneity and the lack of definition of the parameters used. It is necessary for researchers to adopt homogeneous criteria for the definition of relevant indicators, which would allow for comparison of results and an assessment of which aspects should be improved for the benefit of patients treated with pancreatic cancer resection. Despite such limitations, this study offers currently available information, providing a reference with which to compare different national groups.

Authorship

All the authors have participated in the research study and in the review of the manuscript: LS and JES designed the study; LS, IM and JMGC identified the articles for inclusion and the selected indicators; LS, IM and JES conducted the statistical analyses; LS, IM and JMGC composed the first version of the manuscript; and EMF, MGA, DD and JO gave their critical review and final approval of the manuscript.

Conflict of Interests

The authors have no conflict of interests to declare.