Perceptions from nurses, surgeons, and anesthetists about the use and benefits of the surgical checklist in a teaching hospital

Abstract

Objective

To assess attitudes and perceptions from nursing staff, surgeons and anesthetists about compliance, utility, and impact on patient's safety of the surgical checklist in a teaching hospital. We also aimed to identify improvement opportunities for strengthening the usefulness of the checklist in the operating theater.

Methods

We carried out a questionnaire-based an observational cross-sectional study. A questionnaire was distributed to operating room staff, including nursing staff, surgeons, and anesthetists. In addition to the information about surgical checklist, We also collected information regarding years of experience in the operating theater. Fisher's exact was used to compare proportions in each statement. Group discussion meetings with key professionals were held to jointly assess the results, propose improvement actions, and evaluate their feasibility.

Results

The overall response rate was 36.2% (131/362). Nursing staff was perceived as the most supportive group in the use of surgical checklist. A 64.3% of surgeons considered that using the checklist prevented adverse events vs 84.2% and 85.7% among anesthetists and nurses, respectively; p = 0.028. Junior staff showed a supportive attitude toward the use of surgical checklist, considering it as a tool that gives them confidence. We ended up with a list of improvement actions aiming at strengthening the surgical checklist reliability and compliance.

Conclusions

The perception of the surgical checklist usefulness as a tool to prevent adverse events was moderate among surgeons, but well appreciated by junior staff. Nursing staff were especially critical regarding compliance and support by other professionals. To reinforce the usefulness perception of the surgical checklist it is needed to increase the involvement of all professionals, especially senior staff and surgical leaders.

Resumen

Objetivo

Evaluar las actitudes y percepciones del personal de enfermería, cirujanos y anestesistas con relación al cumplimiento, utilidad e impacto en seguridad del checklist quirúrgico en un hospital universitario. Asimismo nos propusimos identificar oportunidades de mejora para reforzar la utilidad del checklist quirúrgico en el quirófano.

Métodos

Estudio observacional de corte transversal basado en la información obtenida mediante un cuestionario distribuido entre el personal de quirófano: incluyendo personal de enfermería, cirujanos y anestesistas. Además de las cuestiones relativas al checklist quirúrgico, se incluyó información sobre años de experiencia en el quirófano. Se utilizó el test exacto de Fisher para comparar las proporciones de las respuestas para cada ítem. Los resultados fueron evaluados conjuntamente por parte de un grupo de discusión con profesionales clave, donde se propusieron acciones de mejora y se estudió su factibilidad.

Resultados

La tasa de respuesta fue del 36,2% (131/362). El personal de enfermería fue percibido como el grupo que daba mayor apoyo en el uso del checklist quirúrgico. Un 64,3% de cirujanos consideró que el uso del checklist quirúrgico previno eventos adversos vs. 84,2% y 85,7% entre anestesistas y personal de enfermería, respectivamente; p = 0,028. El personal junior mostró una actitud de apoyo hacia el uso del checklist quirúrgico, considerándolo como una herramienta que les daba confianza. Se elaboró una lista de acciones de mejora con el objetivo de fortalecer la confianza y el cumplimiento del checklist quirúrgico.

Conclusiones

La percepción de la utilidad del checklist quirúrgico como herramienta para evitar eventos adversos fue moderada entre cirujanos, pero bien considerada por el personal con menos experiencia en el quirófano. El personal de enfermería fue especialmente crítico respecto al cumplimiento y el apoyo por parte de otros profesionales. Para reforzar la percepción de la utilidad del checklist quirúrgico, se considera necesario realizar acciones de mejora para aumentar la participación de todos los profesionales, especialmente el personal sénior y los líderes quirúrgicos.

Introduction

Surgical care is an integral part of healthcare systems, with an estimated 313 million surgical procedures undertaken globally each year.¹ Surgery, although essential in managing several health conditions, it is not risk free. Around 4.2 million postoperative deaths are estimated to occur each year, 7.7% of total deaths worldwide.² Postoperative complications led the World Health Organization (WHO) to publish in 2008 a series of recommendations to improve the safety of surgical procedures. Among these recommendations stands out the implementation of the surgical checklist, which consists of 19 items for professionals to review at three different moments in the operating room,³ aiming at improving teamwork communication and preventing errors and omissions in the theater.

Several studies showed that the use of the surgical checklist is linked to fewer postoperative deaths and complications.^4–7 However, this finding has been discussed, because certain hospitals did not experienced an improvement in operative mortality or complications after the introduction of the surgical checklist.^8,9 Urbach et al. pointed out an inadequate adherence to the checklist or a non-standardized implementation as potential explanations for these results, as well as the possibility that the effect of the surgical checklist was over-estimated.⁸

The evidence of the surgical checklist's benefits along with the ease of implementing the checklist led to the quick and wide implementation of this tool worldwide.¹⁰ Nevertheless, there is great variability in the use of the surgical checklist among countries and across facilities.¹¹ Several papers have assessed barriers and facilitators for the surgical checklist implantation and have identified organizational and cultural factors, design elements, and professional engagement,¹² along with staff perception of the checklist utility, as key elements to achieve a successful uptake.^13–15

In our center, the circulating surgical nurse is the professional in charge of reading out the different statements of each phase, as well as of recording them into the computer system. An adapted version of the WHO surgical checklist promoted by the Department of Health in the context of World Alliance for Patient Safety¹⁶ was progressively introduced in 2011 and reached the full implementation in 2014, when it was also fully integrated into the hospital computer system. Its implementation was accompanied by an intense educational campaign aimed at all operating room professionals. Despite the consolidated integration in the theater routine, some signals of poor use have been detected, with the potential risk of the surgical checklist becoming a “tick box” exercise.

In the context of quality and patient safety improvement initiatives, we carried out a study with the objective to assess attitudes and perceptions of the operating room's staff about compliance, utility, impact on patient's safety and barriers for the surgical checklist use in a teaching hospital. We also aimed to identify improvement opportunities for strengthening the usefulness of the checklist in the operating room.

Methods

Study design

We carried out a questionnaire-based cross-sectional study to assess attitudes and perceptions about compliance, usefulness, and impact on patient safety of the surgical checklist among operating room staff, including nurses, surgeons, and anesthetists. Afterwards, group discussion meetings were held with key professionals to jointly assess questionnaire results and to discuss improvement actions aimed at enhancing the current use of the surgical checklist in our hospital.

Setting

The study was conducted at Parc de Salut MAR (PSMAR), a healthcare organization which includes two tertiary hospitals in Barcelona, Spain (Hospital del Mar and Hospital de l’Esperança) with surgical activity and serves an area of more than 300,000 inhabitants. The hospital covers a wide range of surgical specialties, including: Angiology and Vascular Surgery, General and Digestive Surgery, Maxillofacial Surgery, Orthopedic Surgery and Traumatology, Plastic Surgery, Thoracic Surgery, Dermatology, Neurosurgery, Obstetrics and Gynecology, Ophthalmology, Otorhinolaryngology, Urology.

Baseline situation

From 2014, the surgical checklist is mandatory at PSMAR to perform any surgical intervention, except for vital emergencies; in these cases, retrospective completion is allowed.

It is necessary to complete and finish each phase to start and/or end a surgery. Nevertheless, the system allows a retrospective completion for the abovementioned cases of vital emergencies. Since its integration into the hospital computer system, the checklist has been completed for an overall of 71,569 surgical interventions, which represent the 98.5% of the surgical interventions performed.

Phase I. Questionnaire administration

We adapted and translated the questionnaire used by O’Connor et al.¹⁷ to our setting. The current survey was a 23-item questionnaire, which assessed compliance, usefulness, and impact on patient safety of the surgical checklist along with barriers and facilitators of its use. The first 22 items were closed questions rated on a Likert scale of 1–5 to show the degree of agreement with the different statements: 1 = strongly disagree; 2 = slightly disagree; 3 = neutral; 4 = slightly agree and 5 = strongly agree. The last item was a dichotomous question that asks for a yes/no. Finally, an open-ended statement was included to identify potential unmentioned barriers.

We also collected information on: (a) task performed in the operating room (surgery, anesthesia and nursing), along with the surgical specialty in the case of surgeons; (b) years of experience in the operating room (<10 years, 10–20 years, and >20 years); and (c) Main surgical activity performed at Hospital del Mar/Hospital de l’Esperança.

The questionnaire was distributed to operating room nursing staff, surgeons, and anesthetists, during June 2018, including all professional staff and resident medical interns of the participating centers. First, the potential participants were informed through a cover letter that outlined the purpose and importance of the study, sent by the heads of each surgical department, head of the anesthesia department, and the operating room nurses’ managers in each center. Afterwards, paper copies were distributed through patient safety referents of each surgical/anesthesia service during clinical sessions. As for the nursing staff, the nursing coordinators distributed the questionnaires to the operating room nursing shifts. Email reminders were sent 1 week before the deadline (30 June 2018).

Phase II. Group discussion

Once results from the questionnaire were obtained and analyzed, they were presented and discussed with key professionals. Three meetings were held between November 2018 and April 2019, with the objective of jointly assessing the results, proposing improvement actions, and evaluating their feasibility. The components of the group discussion were surgical and nursing leaders, senior anesthetists, and quality and patient safety technicians. Meetings were moderated by the Director of the Quality and Patient Safety Program. Group discussion meetings were structured in 3 stages: (1) Results discussion: focused on the items with more disagreement that emerged from the questionnaire; (2) Prioritization: at this stage, we identified which areas were most important to address, and generated ideas for problem solving and action planning; (3) Action plan definition: A detailed action plan proposal was defined, identifying what should be done, by whom and by when. Afterwards, the actions proposed were transferred to the executive committee, surgical, anesthesia and nursing teams. Feedback from different stakeholders was put in common within the discussion group as well as the feasibility of implementing each action.

Data analysis

Likert's scale scores of 4 and 5 were grouped to calculate the percentage of participants who agreed with each questionnaire statement. For the last dichotomous question, the percentage of affirmative answers was calculated. We used Fisher's exact test to compare proportions in each statement according to task performed and years of experience in the operating room. Statistical significance was set at p < 0.05.

Data management and statistical analysis were performed using Microsoft Access 2010, and IBM SPSS Statistics program, version 25.

Results

The overall response rate of the questionnaire was 36.2% (131/362). The response rate for surgeons was 39.1% (70/179), 27.9% for anesthetists (19/68) and 35.9% (42/117) nursing staff.

Table 1 shows the percentage of participants who agree with the survey statements by task in the theater. Operating room staff agreed in most items regarding compliance and impact on patient safety of the surgical checklist. However, only 31.7% of the nursing staff answered that everyone stopped what they were doing when the checklist was being carried out, while this percentage was 71.4% and 68.4% for surgeons and anesthetists, respectively (p < 0.001). There was agreement between theater staff that the individual who signs the checklist personally ensures that the relevant steps have been completed (86.2%, 89.5% and 95.2% of surgeons, anesthetists, and nurses; p = 0.395). Regarding the use of the surgical checklist as a tool to improve teamwork in the theater, nursing staff reported a 64.3% of positive answers vs a 87.1% and 78.9% among surgeons and anesthetists, respectively (p = 0.019).

Table 1.

Proportion of affirmative answers according to the task in the operating room.

	Surgeon(N = 70)		Anesthetist(N = 19)		Nurse(N = 42)
	n* (%)	DK**	n** (%)	DK**	n** (%)	DK**	p-Value***
Compliance
The complete checklist is used for every procedure in every theater	57 (90.5)	7	18 (94.7)	0	39 (95.1)	1	0.722
When the checklist is being carried out, everyone in theater stops what they are doing	50 (71.4)^a	0	13 (68.4)^b	0	13 (31.7)^a,b	0	<0.001
Impact on patient safety
I believe that failing to use the checklist is poor professional practice	60 (85.7)	0	17 (89.5)	0	36 (90.0)	0	0.931
I believe that using the checklist gives me confidence	61 (87.1)	0	18 (94.7)	0	35 (83.3)	0	0.551
I believe that using the checklist improves teamwork in theater	61 (87.1)^a	0	15 (78.9)	0	27 (64.3)^a	0	0.019
Support
Surgeons support the use of the checklist	64 (97.0)^a,b	4	13 (68.4)^a	0	26 (63.4)^b	0	<0.001
Anesthetists support the use of the checklist	54 (91.5)^a	11	15 (78.9)	0	29 (70.7)^a	1	0.023
Nursing staff support the use of the checklist	64 (98.5)^a	5	16 (84.2)^a	0	40 (95.2)	0	0.035
Senior staff support the use of the checklist	55 (88.7)^a	8	12 (70.6)	2	27 (67.5)^a	2	0.019
Junior staff support the use of the checklist	61 (98.4)^a,b	8	13 (76.5)^a	2	34 (82.9)^b	1	0.003
Usefulness
Do you think the checklist has prevented any medical error?****	45 (64.3)^a	0	16 (84.2)	0	36 (85.7)^a	0	0.028

*

Absolute number and percentage of participants who agreed with the statements. We considered in the category ‘agree’ those who have a score of 4 and 5 of the Likert scale.

**

DK: Absolute number of answers ‘I don’t know’.

***

Fischer's exact test.

****

Absolute number and percentage of affirmative answers.

^a,b Values that share a letter in the same row are significantly different in p < 0.05 in the two-sided equality test for column proportions.

Nursing staff was perceived, especially by surgeons, as the most supportive professional group in the use of the surgical checklist (98.5%, 84.2% and 95.2% among surgeons, anesthetists, and nurses, respectively; p = 0.035). Surgeons reported high levels of support to the use of surgical checklist by all theater staff roles. However, anesthetists and nursing staff reported low levels of support, and both viewed surgeons as the least supportive group. Management support to the use of the checklist was perceived as high by all theater staff (95.7%, 85.7% and 97.3% among surgeons, anesthetists and nurses, respectively; p = 0.275). Regarding the usefulness of the checklist as a tool for decreasing surgical adverse events, 64.3% of surgeons considered that using the checklist prevented adverse events, whereas the percentage among anesthetists and nurses was 84.2% and 85.7%, respectively (p = 0.028) (Table 1).

The main barriers to the use of the checklist identified by theater staff were lack of assertiveness of staff (27.7%, 44.4% and 51.3% among surgeons, anesthetists, and nurses, respectively; p = 0.047), lack of staff commitment (26.9%; 57.9% and 45.2%, respectively; p = 0.022) and lack of training (25.4%, 50.0% and 12.5%, respectively; p = 0.011). Lack of time was perceived as a barrier by 22.1% of surgeons, 26.3% of anesthetists and 29.3% of nurses (p = 0.694) (data not shown).

In Table 2, we present the proportion of affirmative answers by years of experience in the theater. Over 90% of participants of all study groups agreed that the complete checklist was used for every procedure in every theater (90.7%, 92.5% and 93.8% among professionals of <10, between 10–20, and >20 years of experience in the theater, respectively; p = 1.000). There was agreement that the individual who signs the checklist personally ensures that the relevant steps have been completed (85.0%, 90.2% and 94.4% among staff with <10, 10–20, and >20 years of experience, respectively; p = 0.441).

Table 2.

Proportion of affirmative answers according to years of experience in the operating room.

	<10 years(N = 43)		10–20 years(N = 43)		>20 years(N = 36)
	n* (%)	DK**	n* (%)	DK**	n* (%)	DK**	p-Value***
Compliance
The complete checklist is used for every procedure in every theater	39 (90.7%)	0	37 (92.5)	3	30 (93.8)	4	1.00
When the checklist is being carried out, everyone in theater stops what they are doing	20 (46.5)	0	25 (59.5)	0	26 (72.2)	0	0.072
Impact on patient safety
I believe that failing to use the checklist is poor professional practice	42 (97.7)^a	0	33 (80.5)^a	0	31 (86.1)	0	0.028
I believe that using the checklist gives me confidence	40 (93.0)	0	36 (83.7)	0	30 (83.3)	0	0.347
I believe that using the checklist improves teamwork in theater	39 (90.7)^a	0	29 (67.4)^a	0	28 (77.8)	0	0.026
Support
Surgeons support the use of the checklist	36 (90.0)	3	33 (76.7)	0	26 (76.5)	1	0.228
Anesthetists support the use of the checklist	30 (78.9)	5	35 (85.4)	2	26 (81.3)	4	0.782
Nursing staff support the use of the checklist	38 (97.4)	4	40 (95.2)	1	33 (91.7)	0	0.515
Senior staff support the use of the checklist	28 (80.0)	8	32 (74.4)	0	25 (78.1)	4	0.846
Junior staff support the use of the checklist	35 (97.2)	7	37 (86.0)	0	27 (84.4)	4	0.133
Usefulness
In your experience, you think the checklist has prevented a medical error?****	34 (79.1)	0	30 (69.8)	0	25 (69.4)	0	0.543

*

Absolute number and Percentage of participants who agree with the statements. We considered in the category ‘agree’ those who have a score of 4 and 5 of the Likert scale.

**

Absolute number of answers ‘I don’t know’.

***

Fischer exact test.

****

Absolute number and percentage of affirmative answers.

^a,b Values that share a letter in the same row are significantly different in p < 0.05 in the two-sided equality test for column proportions.

In general, junior staff were more supportive of the use of checklist as a tool for improving patient safety: 97.7% of personnel with <10 years of experience considered that failing to use the checklist was poor professional practice, whereas this percentage was 80.5% and 86.1% (p = 0.028) among professionals with 10–20 and >20 years of experience in the theater, respectively. A 90.7% of junior staff believes that the use of checklist improves teamwork in the theater, in front of a 67.4% and a 77.8% among the other groups (p = 0.026, Table 2).

Although differences were not statistically significant, over 90% of junior staff affirmed that using the checklist gives them confidence, and also a 79.1% agreed that the checklist prevented medical errors, contrasting with the lower proportions among more experienced professionals (Table 2).

Barriers to the use of the checklist according to years of experience were lack of assertiveness of staff (30.2%, 41.5% and 43.3% among staff with <10, 10–20, and >20 years of experience, respectively; p = 0.432), lack of staff commitment (28.6%, 39.5% and 44.1%, respectively; p = 0.355), lack of time (30.2%, 28.6% and 14.7%, respectively; p = 0.249) and lack of training (24.4%, 17.5% and 31.3%, respectively; p = 0.417) (data not shown).

Results discussion with key professionals resulted in a list of improvement actions, summarized in Table 3. Specific actions included: (1) the elaboration of a new protocol to enhance surgeons and anesthetist's responsibility in initiating the surgical checklist instead of nursing staff; (2) closer monitoring of the checklist compliance with surgical leaders to strengthen the checklist reliability; and (3) formative actions in surgical services to reinforce the confidence and usefulness of the checklist for junior staff. In overall, actions proposed aimed at strengthening the checklist reliability and compliance by engaging senior staff and chiefs of surgical services in promoting a proper surgical checklist compliance and reinforcing the involvement of different professional roles in the theater, by reviewing the current approach where the weight of the process relies mainly on nursing staff. The ultimate responsibility of the check list procedure the most difficult agreement point.

Table 3.

List of improvement actions proposed by the discussion group.

Goals and improvement actions	Target professionals
Goal 1: reinforcing the involvement of different professional roles in the theater, by reviewing the current approach where the weight of the process relies mainly on nursing staff.
Elaboration of a new protocol for the surgical checklist administration, to enhance surgeons and anesthetist's responsibility in initiating the surgical checklist instead of nursing staff. Specifically, the professional responsible to initiate each phase would be: 1. Before the induction of anesthesia “sign in”: the anesthetist 2. Before the skin incision “time out”: the surgeon 3. Before the patient leaves the operating room “sign out”: the nurse	All theater staff
Goal 2: To strengthen the checklist reliability and compliance by engaging senior staff and chiefs of surgical services in promoting a proper surgical checklist compliance
Closer monitoring of the checklist compliance with surgical leaders by reviewing periodically compliance rates by service	Surgical leaders
To carry out formative actions in surgical services to reinforce the confidence and usefulness of the checklist for junior staff	Surgical teams

Discussion

Our results pointed out some differences in perceptions about the use of the surgical checklist according to the role in the theater and years of experience in the operating room. Whereas nursing staff was the most supportive group of the use of surgical checklist, surgeons were more skeptical regarding the usefulness of the surgical checklist as a tool for preventing surgical adverse events. Besides, junior staff showed a supportive attitude toward the use of the surgical checklist in the theater, considering it as a tool that gives them confidence and useful to avoid medical errors in the theater. These results place surgeons and senior staff as key targets for strengthening the usefulness of the checklist in the operating room.

Nursing staff was viewed by other professionals as the most supportive group in the use of surgical checklist and gave a positive assessment on its utility as a tool for preventing adverse events. At the same time, nurses are the most critical professional group regarding compliance and support by other professionals. Some previous studies observed great differences between surgeons and nursing staff perceptions regarding teamwork,^17,18 whereas nursing and anesthesia staff were more in agreement in terms of impact on patient safety, support barriers, and checklist utility than surgeons.¹⁹ The fact that the nursing staff was more critical than the rest of theater professionals, may be explained by the huge responsibility for guaranteeing the use of the checklist for each intervention, so they are probably the ones who are most aware of the limitations of putting it into practice.²⁰ Anesthetists are, in general, more familiar to the use of checklists, since it is common practice to check the anesthesia equipment before any intervention.

This study includes the analysis of perceptions by years of experience in the theater. Junior staff was critical in terms of checklist compliance but, at the same time, they are the ones who value the usefulness of the checklist more positively. These results are in line with prior works that pointed out that senior staff showed more reluctant attitudes for implementing the surgical checklist in the theater routine^3,15 and were less likely to perceive the surgical checklist as a tool to improve patient safety.²¹ Without a strong and constant commitment with the checklist strengthen from surgical leaders and theater managers, the utilization of the checklist in the theater may become a routine check-box activity, giving the surgeons a false sense of safety.²²

One of the main arguments in favor of using the checklist is that it improves communication in the operating room.^22,23 Previous evidence showed correlation between improving teamwork and decreasing the frequency of errors during surgery.²⁴ Our study shows that both surgeons and anesthetists perceived that checklist improves teamwork, as opposed to nursing staff. This difference in teamworking perception in the theater, has been explained by the hierarchy between professionals in the operating room, reflect educative culture differences, and maybe different perceptions of proper teamwork between professions.^18,22,25 Positive experiences aiming at improving teamwork climate at the theater have opted to encourage multidisciplinary teams and promoting multiprofessional team briefings as a complementary approach to checklists.²⁶

The most relevant proposed actions, after discussion with key professionals, aimed at strengthening the checklist reliability, by engaging senior staff in the operation room and chiefs of surgical services in promoting a proper surgical checklist compliance. There is agreement that engaging institutional leaders to explain why and show how to use the checklist is key for a successful checklist implementation.^10,27 So far, few studies^28–30 have proposed changing the current administration paradigm switching the standard practice in which the circulating nurse is responsible for initiating and administering the surgical checklist. Ong et al. have evaluated the migration leadership of the three checklist domains among the anesthesia, surgical and nursing subteams and have found positive results regarding compliance and professional's engagement.²⁸ Nevertheless, there is still scarce evidence on the impact at the long-term and whether these new practices end up leading to become routine over time.

There are several limitations in our study. First, we had a low response rate to the questionnaire, and we observed differences in the response rate between professional roles. Although O’Connor et al.¹⁷ obtained a similar response rate, we cannot rule out a participation bias, as we do not have information of non-participants. This bias may lead to an overrepresentation of positive responses about the use of the surgical checklist, since respondents may also be more likely to use the surgical checklist. Second, we used an adapted questionnaire, so maybe the results are not generalizable. Nonetheless, our results are consistent with those obtained in other studies.^15,17,19 Third, we have not assessed the impact of the improvement actions adopted. Implementation of any new tool is a process of constant change that can generate a reaction in one or more aspects, generating an entirely new environment. Although this information would enrich the current approach, this objective is out of the scope of the study.

Conclusions

We observed differences in perception by professional group and by years of experience regarding compliance, impact on patient safety and support to the use of surgical checklist. The perception of the surgical checklist usefulness as a tool to avoid adverse events in the theater was moderate among surgeons, but well appreciated by junior staff. Nurses were especially critic regarding compliance and support by other professionals. The results point to the need to increase the involvement of all professionals, especially senior staff and surgical leaders, to reinforce the usefulness of the surgical checklist as a real tool for improving patient safety. The implementation of the surgical checklist required structural changes and it needs periodical revision of the workflow of the operating room personnel in order to ensure quality requirements.

Source of funding

The authors received no specific funding for this work.

Conflicts of interest

None declared.

Acknowledgements

We would like to thank all professionals that participated in the group discussion sessions and all professionals that responded the questionnaire.