Detection of Adverse Events in General Surgery Using the “Trigger Tool” Methodology

Abstract

Introduction

Surgery is one of the high-risk areas for the occurrence of adverse events (AE). The purpose of this study is to know the percentage of hospitalization-related AE that are detected by the “Global Trigger Tool” methodology in surgical patients, their characteristics and the tool validity.

Materials and methods

Retrospective, observational study on patients admitted to a general surgery department, who underwent a surgical operation in a third level hospital during the year 2012. The identification of AE was carried out by patient record review using an adaptation of “Global Trigger Tool” methodology. Once an AE was identified, a harm category was assigned, including the grade in which the AE could have been avoided and its relation with the surgical procedure.

Results

The prevalence of AE was 36.8%. There were 0.5 AE per patient. 56.2% were deemed preventable. 69.3% were directly related to the surgical procedure. The tool had a sensitivity of 86% and a specificity of 93.6%. The positive predictive value was 89% and the negative predictive value 92%.

Conclusions

Prevalence of AE is greater than the estimate of other studies. In most cases the AE detected were related to the surgical procedure and more than half were also preventable.

The adapted “Global Trigger Tool” methodology has demonstrated to be highly effective and efficient for detecting AE in surgical patients, identifying all the serious AE with few false negative results.

Resumen

Introducción

La cirugía supone una de las áreas de alto riesgo para la aparición de efectos adversos (EA). El objetivo de este estudio es conocer el porcentaje de EA en hospitalización que se detectan mediante la metodología «Global Trigger Tool» en pacientes de cirugía general, las características de los mismos y la validez de la herramienta.

Material y métodos

Estudio retrospectivo, observacional y descriptivo sobre pacientes ingresados en cirugía general de un hospital de tercer nivel, sometidos a intervención quirúrgica durante el año 2012. La identificación de EA se lleva a cabo mediante una revisión de historias clínicas empleando una adaptación de la metodología «Global Trigger Tool» Una vez identificado el EA, se le asignó una categoría de daño y se determinó el grado en el que este podría haber sido evitado así como su relación con el procedimiento quirúrgico.

Resultados

La prevalencia de EA fue de 36,8%. Con un número de EA por paciente de 0,5. El 56,2% se consideraron evitables. Y un 69,3% se relacionaron directamente con el procedimiento quirúrgico. La herramienta demostró una sensibilidad del 86% y una especificidad del 93,6%. El valor predictivo positivo fue de 89%, el valor predictivo negativo de 92%.

Conclusiones

La prevalencia de EA es más alta de lo estimado en otros estudios. La mayoría de los EA detectados están relacionados con el procedimiento quirúrgico, y más de la mitad son evitables.

La metodología «Global Trigger Tool» adaptada ha demostrado ser altamente eficaz y eficiente para la detección de EA en cirugía, identificando todos los EA graves y con pocos falsos negativos.

Introduction

An adverse event is defined as an injury or harm to a patient that is caused by medical management, rather than an underlying disease.¹ Adverse events (AE) are a cause for concern due to the high rates that have been observed in hospitalized patients.^2–9 Furthermore, approximately half of these injuries are considered preventable.^4,5,10

The surgical specialties have higher concentrations of AE.^4–6,9 In general surgery, rates have been reported between 7% (the Harvard Medical Practice Study²) and 30.3% (Healey et al. study¹⁰). In the population study about adverse effects in hospitalized patients in Spain (ENEAS), the incidence of AE in general surgery was 10.3%.⁹

Historically, AE identification systems have focused on the voluntary notification of incidences, error tracking, and information obtained from clinical administrative databases (CAD) and complaints. The majority underestimate the actual incidence of AE.^11,12 Recently, different tools have been developed to examine medical records for screening and reviewing in order to detect AE.

In the 1990s, the Institute for Healthcare Improvement (IHI) developed the IHI Global Trigger Tool to quantify AE.¹³ Initially, this system was only used for adverse drug reactions, although later it was adapted for use in intensive care, perinatal, and surgical units. It is based on the selection of medical records that have a high probability of being associated with AE, which is determined by the identification of alarming “clues” or “triggers”. When a trigger is identified, the medical record is then reviewed in detail to confirm the AE. This tool has been shown to be highly effective and efficient for detecting up to 10 times more AE than other systems.¹⁴

The development of a tool that is able to identify AE in surgical patients in a reliable and effective manner is quite an interesting prospect, especially if it is a low-cost method.

The main objective of this study is to determine the percentage of AE in hospital records detected by applying the Global Trigger Tool methodology adapted to general surgery patients. We will also discuss the characteristics of this tool and its validity in our setting.

Materials and Methods

Study Design

This study has an observational, descriptive, and retrospective design.

The patients included for study had been admitted for general surgery at a tertiary hospital from 1 January 2012 to 31 December 2012.

The sample size was calculated for a population of 2900 cases, using data from the ENEAS study as a reference (incidence 10.3%), with a confidence interval of 95% and a precision of 0.03. The sample was made up of 350 patients who had been recruited by simple randomization.

The inclusion criteria were: patients ≥18 years of age who had undergone an urgent or scheduled surgical procedure; complete, closed clinical episodes; and admission for general surgery, either urgent or scheduled.

The exclusion criteria included: psychiatric patients or those in rehabilitation; patients with non-scheduled referral from other hospitals; or organ transplantation surgery.

When a patient did not meet the inclusion criteria, the following patient from the list of surgical patients was selected.

Preparation

After an exhaustive review of the literature, we adapted the tool and list of triggers to the characteristics of the information system at our hospital. The IHI proposes a list of triggers that are grouped into 6 modules: general care, medication, surgery, intensive care, perinatal and emergency department. From the first three modules, we selected the panel of triggers that were adequate for detection by the internal computer network of the hospital. Those that were not able to be detected by this system were ruled out, and others were substituted (Table 1).

Table 1.

Triggers Detected and Their Frequency.

Care module		Medication module		Surgical module		Added triggers
Blood transfusion	19	Stool culture+Clostridium difficile	2	Reintervention	21	Anatomic pathology not related with the diagnosis	1
Transfer to ICU	3	PTT>100 s	^a	Damage to or unplanned exeresis of an organ	10	Stay in PACU >24 h	12
Rehospitalization within 30 days of discharge	26	INR>6	^a	Mechanical ventilation >24 h	2	ER visit within 30 days of discharge	60
Positive blood culture	11	Glucose<50 mg/dl	^a	Intubation or reintubation in PACU	1	Invasive procedure during hospitalization (ERCP, CT-guided FNA, etc.)	21
Echo-Doppler LE/helical chest CT	1	↑ Creatinine ×2 baseline value	^a	Intra- or postoperative death	4	Unscheduled X-ray during hospitalization	69
Cardiac arrest	0	Hypotension or over-sedation	^a	Rehospitalization in recovery	2	Assessment by Infectious Disease Unit	43
Pressure ulcers	^a	Vitamin K	^a	Intra- or postoperative X-ray	^a	Assessment by Nutrition Unit	32
Acute dialysis	^a	Diphenhydramine	^a	Change of procedure	^a	Incidental finding	3
Use of restraints	^a	Flumazenil	^a	Postoperative troponin >1.5 ng/ml^a	^a
Nosocomial infection	^a	Naloxone	^a	Intra-operative epinephrine, norepinephrine, naloxone^a	^a
Patient fall	^a	Abrupt stop in medication	^a
Complication from a procedure	^a
Hemoglobin/hematocrit^a	^a

ERCP: endoscopic retrograde cholangiopancreatography; INR: international normalized ratio prothrombin time; LE: lower extremities; PTT: partial thromboplastin time; PACU: post-anesthesia care unit; CT: computed tomography; ICU: intensive care unit.

a

Trigger removed.

For the harm category assigned to each AE, we used the classification of the National Coordinating Council for Medication Error Reporting and Prevention,¹⁵ which is usually used with the Global Trigger Tool methodology.

Review Process (Fig. 1)

• 1.

  Review team

  The reviewers were two internal medicine residents and a senior surgeon consultant. Prior to the screening phase, the reviewers underwent a training phase in which they reviewed 20 medical files (Fig. 1).

  
  

  Fig. 1.

  Review process.
• 2.

  Screening phase

  The reviewers screened all the files in search of triggers to select which files would go through the review process. Specific screening guidelines were designed for this study, including the variables to be studied for each patient and the selected triggers. The medical file screening time was calculated.

• 3.

  Detection and characterization of AE

  AE were considered “unintentional events that cause harm to the patient as a consequence of medical management rather than an underlying disease”.¹

The information sources for the search of AE included hospital discharge reports, surgical intervention protocols and comments on evolution by physicians and nursing staff from the moment the patient was hospitalized until 30 days after discharge, including comments from outpatient visits. All this information was available in electronic format.

When an AE was detected, it was assigned a harm category and the extent to which the event could have been avoided was assessed. To determine the preventability of the AE, we adapted the classification used in the ENEAS study⁴; an adverse event was considered avoidable with a score of 4 or greater.

In order to minimize the subjectivity of the reviewers in the classification of avoidable AE and the degree of complexity assessment of the surgical procedures, a questionnaire was answered by 20 senior surgeons. These surgeons graded each AE according to the previous classification and each surgical procedure was scored from 1 to 4: a score of 1 was considered a low degree of complexity, while 4 was a highly complex procedure. A mean value was assigned to each AE and each surgical procedure. The reviewers were able to later adjust the score to each case.

The files in which no trigger was detected were reviewed in the same way as those that had been positively screened in the AE search, using the same information sources.

Data were collected for the following variables: age, sex, ASA category, main diagnosis, surgical procedure, days of hospital stay, urgent or scheduled surgery, complexity of the surgical procedure.

Data Analysis

The descriptive analysis included the means, median, standard deviation for continuous variables, and distribution of frequencies for categorical variables.

The comparison of the main variables according to AE was done with the Mann–Whitney U, chi-squared or Fisher's test, depending on whether the variable was continuous or categorical.

In order to measure the validity of the tool for detecting the presence of an AE, the sensitivity and specificity of the diagnostic test were used, as well as the positive predictive value (PPV) and negative predictive value.

Last of all, the prediction model was created by means of binary logistic regression with the forward conditional method. The dependent variable used was the appearance of AE, and independent variables were those that were statistically significant in the bivariate analysis or could have clinically plausible involvement. The model calibration analysis was carried out with the Hosmer–Lemeshow statistic. The discriminatory power of the model was evaluated using the area under the ROC (receiver–operator characteristics) curve obtained by analyzing the probability of the value predicted by the multivariate model. The results of the model are presented as odds ratio (95% confidence interval [CI]).

The statistical program used was the STATA/SE v10.0. P values <.05 were considered statistically significant throughout the analysis.

Results

In the training phase, the reviewers reached 100% agreement.

A total of 350 patient medical files were screened. Fourteen were excluded because they did not meet the inclusion criteria (8 liver transplantations, 3 hepatic chemoembolizations, and 3 incomplete medical files) and substituted as described in the Methods section. Mean screening time for the medical files was less than 3 min.

Mean age was 58, and median age was 60; 47.7% of the patients were women. Surgery had been scheduled for 255 patients (73%) and was urgent for 95 (27%). The surgical procedure types were: cholecystectomy 22%; appendectomy 13%; colectomy 11%; hernioplasty 9%; and thyroidectomy 8%. The remaining procedures represented less than 5% each.

Table 1 shows that 340 triggers were detected in 125 patients. In 111 of the 125 patients selected by the tool, we found at least one AE. There were 14 false positives; PPV of the tool was 89%. In 31 patients, a second AE was detected, while 15 patients had a third, 3 had a fourth, and 1 had a fifth. The total number of AE detected by the tool was 161.

After the review of the medical files in which no triggers were detected, 18 AE were detected in 18 patients: 5 peripheral intravenous catheter-related phlebitis, 4 postoperative hypocalcemia, 2 allergic reactions, 2 seromas and 1 hematoma of the surgical wound, 1 acute urinary retention, and 3 operating room cancellations.

The actual prevalence of AE was 36.8% (CI: 31.7%–41.9%). Excluding the cases of phlebitis as the only AE, the prevalence was 35.4%. There were 51.1 AE for every 100 patients and 5.57 AE for every 100 patient-days.

The harm categories for the detected AE are shown in Fig. 2.

Fig. 2.

Harm categories from the National Coordinating Council for Medication Error Reporting and Prevention.

Table 2 shows the types of AE that were most frequently detected. 56.2% of the AE were considered avoidable (Fig. 3), while 69.3% were directly related with the surgical procedure.

Table 2.

Adverse Events and Frequency.

	Frequency	Percentage
Surgical wound infection	24	13.5
Anastomotic fistula	17	9.5
Paralytic ileus	14	8
Intraabdominal abscess	12	6.5
Phlebitis (peripheral)	11	6
Intraabdominal bleeding	9	5
Wound seroma	9	5
Hypocalcemia	8	4.5
Hematoma of surgical wound	7	4
Respiratory infection	6	3.5
Operating room cancellation	6	3.5
Poor pain control	5	3
Urinary tract infection	5	3
Procedure complication	4	2.5
Allergic reaction to iv therapy	3	1.5
Decubitus ulcer	3	1.5
Central venous access infection	3	1.5
Death	2	1
Miscellaneous	28	15.5
Total	179	100%

Fig. 3.

Preventability of AE.

The appearance of AE in the sample was related with the complexity of the surgery and the degree of contamination (Table 3). The type of surgery (urgent/scheduled), patient age, and ASA category were not related with the appearance of AE. In urgent surgery, however, patients >65 years of age presented 3.16 times more risk for presenting an AE than those who were <65 years old (CI 1.04–9.58, P=.042).

Table 3.

Adverse Events and Risk Factors.

Variable	Odds ratio	P-value	Confidence interval
			Lower	Upper
Procedure complexity
Low	Ref	Ref	Ref	Ref
Medium-low	1.19	.63	0.57	2.50
Medium-high	3.26	<.001	1.75	6.09
High	8.27	<.001	2.83	24.21
Degree of surgical contamination
Clean	Ref	Ref	Ref	Ref
Clean-contaminated	0.73	.47	0.35	1.53
Contaminated	1.92	.03	1.04	3.53
Dirty	1.35	.47	0.59	3.10

The model presents only those variables that were significant. As for the procedure complexity, the reference category was low complexity, and the risk increased along with the complexity. As for the degree of surgical contamination, the reference category was “clean”, and “contaminated” was significant, with a risk for AE that was 1.92 times higher than “clean”.

The tool showed a sensitivity of 86% and a specificity of 93.6%. The negative predictive value of the tool was 92%, and the PPV was 89%.

Discussion

The screening of patient files for the detection of AE is a simple, effective and efficient method. As far as we know, this is the first specific study in Spain that assesses AE detection in surgery using the Global Trigger Tool methodology. This tool is used extensively throughout the world, mainly in English-speaking countries, and it produces consistent, reliable and relevant results at little cost.¹⁶

The prevalence of AE in our study (36.9%) is higher than what has been found in other studies using different methods. The ENEAS¹⁷ study reported a rate of 10.3% (12.5% in hospitals with more than 500 beds), and the Júdez Legaristi et al. study reported 17.8%.¹⁵ Nonetheless, the prevalence in our study is similar to other prospective studies: 30.3% in the Haley et al. study (which measured complications in exclusively surgical patients), and 36.9% in the Rebasa et al. study.¹⁸ The IHI carried out a study in surgery pateints¹⁹ and found an AE prevalence of 14.6%, although they used a trigger battery that was different than what we used for this article.

Very few AE were not detected by the tool,¹⁸ and in each of these instances the harm category was lower than F. Therefore, we consider the trigger tool to be reliable, useful and effective for reviewing AE in general surgery.

Differences with other studies may be caused by conceptual aspects, but we believe that the specific design of the tool was able to optimize medical file selection in the screening phase. The PPV of the tool in our study (89%) was higher than the reported one by other international studies,^20,21 which have reported PPV rates from 16.3% up to a maximum of 38% in the Australian article about quality health care.⁴ In the ENEAS study, the PPV of the screening guidelines for general surgery patients was 77.2% (including incidents and AE due to the disease),¹⁴ while in the Júdez Legaristi et al. study it was 53%.²²

The tool has demonstrated high sensitivity and specificity, which indicates its ability to reveal the reality of AE in hospitalized surgical patients.

The tool was adjusted to the electronic resources of our center, with the elimination or substitution of certain triggers. One of the advantages of the tool is its ability to be adapted according to the framework to which it is applied.²³

The subjectivity of the results is minimized with the evaluation of the surgical complexity and preventability of AE using surveys answered by an appropriate number of expert surgeons. These surveys are also a flexible guide for reviewers.

As 56.42% of the AE observed were considered preventable, there is a wide margin for improvement. These data are similar to other studies with preventable AE rates close to 50%.^4,5,9,10 In general, the causes of AE are multifactorial and it is necessary to analyze them with a systemic approach in order to reduce these percentages.

The harm category was similar to other published studies.¹³ The percentage of AE that were directly related with the surgical procedure (69.27%) would justify the controlled risk factors related with the appearance of AE (procedure complexity and degree of surgical contamination).

Only 3 medication-error AE were detected. This finding could be explained by the fact that most of the triggers proposed by the IHI methodology for errors in medication were eliminated because it was difficult to perform a rapid case screening. However, after having reviewed the medical files of the sample in great detail, we cannot conclude that the tool was not effective in this regard. In the ENEAS study, 24%¹⁷ of detected AE were medication-related, which leads us to believe that medication errors are not adequately recorded in patient medical files at our hospital. It should also be added that the IHI study for surgical patients reported a rate of medication-related AE of around 15%¹⁹ and a predominance of AE that are directly related with the surgical procedure.¹⁹

We conclude that the adapted Global Trigger Tool methodology has been shown to be highly effective and efficient for the detection of AE in surgery. It was able to identify all the serious AE and produced few false negatives.

The prevalence of AE was higher than estimated in other studies. Most of the detected adverse events were related with the surgical procedure, and more than half were preventable.

Funding

No funding was received for this study.

Conflict of Interests

The authors have no conflict of interests to declare.