Lessons learnt from the development of the Patient Safety Incidents Reporting an Learning System for the Spanish National Health System: SiNASP

Abstract

Objective

To describe the development process and characteristics of a patient safety incidents reporting system to be implemented in the Spanish National Health System, based on the context and the needs of the different stakeholders.

Design

Literature review and analysis of most relevant reporting systems, identification of more than 100 stakeholder's (patients, professionals, regional governments representatives) expectations and requirements, analysis of the legal context, consensus of taxonomy, development of the software and pilot test.

Results

Patient Safety Events Reporting and Learning system (Sistema de Notificación y Aprendizajepara la Seguridad del Paciente, SiNASP) is a generic reporting system for all types of incidents related to patient safety, voluntary, confidential, non punitive, anonymous or nominative with anonimization, system oriented, with local analysis of cases and based on the WHO International Classification for Patient Safety. The electronic program has an on-line form for reporting, a software to manage the incidents and improvement plans, and a scoreboard with process indicators to monitor the system.

Conclusions

The reporting system has been designed to respond to the needs and expectations identified by the stakeholders, taking into account the lessons learned from the previous notification systems, the characteristics of the National Health System and the existing legal context. The development process presented and the characteristics of the system provide a comprehensive framework that can be used for future deployments of similar patient safety systems.

Resumen

Objetivo

Describir el proceso de desarrollo y las características del sistema de notificación de incidentes de seguridad del paciente para el Sistema Nacional de Salud, basado en el contexto y en las necesidades de los distintos implicados.

Diseño

Revisión bibliográfica y análisis de los sistemas de notificación más relevantes, identificación de las necesidades y expectativas de más de 100 implicados (pacientes, profesionales, representantes de las Comunidades Autónomas), análisis del contexto legal, conseso de la taxonomía del SiNASP, desarrollo del programa informático y prueba piloto.

Resultados

SiNASP es un sistema de notificación genérico para todo tipo de incidentes relacionados con la seguridad del paciente, voluntario, confidencial, no punitivo, anónimo o nominativo con deidentificación, con orientación sistémica, con análisis local de casos y basado en la Clasificación Internacional de Seguridad del Paciente de la OMS. La aplicación informática tiene un cuestionario para la notificación a través de internet, un software para gestionar los incidentes y los planes de mejora y un cuadro de mando de indicadores para monitorizar el sistema.

Conclusiones

El sistema de notificación se ha diseñado para dar respuesta a las necesidad y expectativas de los implicados, teniendo en cuenta las lecciones aprendidas de los sistemas de identificación previos, las características del Sistema Nacional de Salud y el contexto legal. El proceso de desarrollo descrito y las características del sistema proporcionan un marco que puede servir de base para el desarrollo de otros sistemas de seguridad del paciente.

Since the publication of the US Institute of Medicine report “To err is human”,¹ the UK Department of Health report “An organization with a memory”,² and other relevant studies, patient safety has been a cause for concern in healthcare systems all over the world. The rates of adverse events (AE) are estimated to range between 3% and 17% in acute care hospitals.³ In Spain, the incidence of patients with AE related to healthcare in hospitals was 9.3%⁴ while in primary care settings the AE occur in nearly 2% of all patient visits. This means, according to the use of primary care services, that 1 in every 7 citizens would eventually experience an adverse event.

Based on the impact of the AE in patients, professionals and organizations, the World Health Organization⁵ and the European Commission for Healthcare,⁶ have recommended developing reporting and learning systems (RLS) to facilitate the analysis of contributing factors that led to errors and to prevent them. The development of these systems was one of the objectives of the Patient Safety Strategy⁷ developed by the Spanish Ministry of Health since 2005.

RLS have been a key tool to improve safety in a range of high-risk organizations (commercial aviation, rail industry, etc.). Although incident reporting has been instituted in healthcare systems in many countries for some time now, similar positive experience is yet to be fully realized.⁸ The RLS is a complementary method to identify patient safety incidents, being medical records reviews and walk rounds some other relevant ones. Even when incident reporting has limitations, studies show that they capture more contextual information about incidents⁹ and, when actively promoted within the clinical setting, they can detect more preventable adverse events than medical records review¹⁰ at a fraction of the cost.

When the Spanish Ministry of Health, responsibile to keep health basic principles and general coordination for the NHS,¹¹ started the development of the RLS, two Health Regions had already implemented their own, and also some monographic ones existed (e.g.: anesthesia, medication, intensive care).

The objective of this study is to describe the design and development of a RLS for the Spanish National Health System, adapted to the legal context and the needs and expectations of different stakeholders.

Methods

Development of SiNASP

The development of the Patient Safety Events Reporting and Learning system (Sistema de Notificación y Aprendizaje para la Seguridad del Paciente, SiNASP) was based on analysis of exiting notification systems, expectations and demands from main stakeholders and the legal context related to these systems. Once the system had been defined and the electronic program had been developed, a pilot test was performed. The steps of the development process are shown on Fig. 1.

Fig. 1.

SiNASP development process.

Literature review and visits to some relevant notification systems

A scientific literature review was conducted to identify lessons learned from existing RLS, focused on articles that discussed the impact of the systems, the assessment of the most important characteristics and expert recommendations. Visits to countries with leading national notification systems (Denmark, England and Wales) were performed to learn about characteristics and workflow of the systems.

Identification of stakeholder's expectations and requirements

In the first step of the project, a meeting was held with the Regional authorities and national experts as well as two focus groups with 16 representatives members of patients associations to know their opinions expectations and position regarding a RLS.

Their suggestions were used as a basis to define the SiNASP main attributes. On a second stage, scientific societies were consulted by an online questionnaire, about the viability of implementing and pros and cons of a system with those attributes.

Analysis of the legal context

At the same time, a group of legal experts performed an analysis focused on the Spanish legal framework,¹² a comparison with the international legislation¹³ and recommendations for the development of the Spanish national RLS.¹⁴

Definition of SiNASP taxonomy and development of the application

The taxonomy of SiNASP was developed according to the information collected in previous phases, the WHO International Classification for Patient Safety (ICPS)¹⁵ and aiming to get a compatible taxonomy with other RLS running in the NHS (Andalucía, Bask Country, ISMP-Spain¹⁶ and SENSAR¹⁷) and from some leading international systems (National Patient Safety Agency,¹⁸ Veterans Administration¹⁹ and Joint Commission International).²⁰

The terms included were mapped using the WHO ICPS framework as the guiding structure, to identify the common contents in the different notification systems. The common terms in most systems were selected for the SiNASP taxonomy. A detailed assessment of the rest of the terms was performed in order to select only the ones providing relevant information. The usefulness of the information was balanced with the goal of developing a questionnaire that was simple and quick to fill out (requirement identified by healthcare professionals in prior phases of this work).

Pilot test

Once the electronic software and supporting documents were developed, a pilot test was conducted in two hospitals in 2009. The analysis of the incidents reported, the opinion of healthcare professionals that used the system (by questionnaire) and reporting system managers (by phone interview) was performed. The needed modifications to the system were performed.

Results

Results from the development process

Conclusions from literature review and visits to relevant notification systems were very similar and can be summarized by the characteristics of a successful reporting system: non punitive, confidential, independent, expert analysis, timely analysis of cases, systems oriented, responsive and capable of disseminating and implementing recommendations.²¹

Main requirements and recommendations identified by stakeholders are summarized on Table 1

The legal experts, taking into account the Spanish legislation, recommended the development of a RLS voluntary, confidential, preferably anonymous, non punitive, oriented to learning and focused on incidents that did not produce harm to patients.¹⁵

Characteristics and main results from the pilot test are summarized on Table 2. The general evaluation of the system was positive, and the agreement with the basic attributes of it was high.

Table 2.

Pilot test results participants.

Organizations	2 General Hospitals from 2 different regions
Characteristics	Hospital 1	Hospital 2	Total
N of participating services	9	6	15
N of beds in areas involved	83	160	243
Professionals involved	Over 500	Over 300	Over 800

Events notified during the pilot
N of events notified	263
Type of events notified	Medication (49.8%), Therapeutic procedures (12.5%), Diagnostic procedures (7.2%)
Severity assessment code (SAC)	SAC 4, low risk (85.2%), SAC 3, medium risk (10.3%), SAC 2 high risk (4.2%), SAC 1 extreme risk (0)
Areas where events were notified	Medical hospitalization unit (27.3%), Intensive care unit (18.6%), Surgical hospitalization unit (15.9%)
Professional notifying	Pharmacist (38.4%), Nurse (28.9%), Physician (24.7%)

Users satisfaction surveys
Users (staff notifying incidents)	Structured survey with 42 respondents from both hospitalsResponses to some selected items
Item/% of respondents by category	Disagreement	Unsure	Agreement
SiNASP is user friendly	20%	12%	68%
SiNASP is secure for staff	2%	27%	71%
SiNASP is secure for patients	0%	17%	83%
SiNASP works properly when needed	21%	23%	56%
I have computers to access SiNASP	33%	26%	41%
I know how to use the application	3%	3%	95%
I trust confidentiality of information	7%	17%	76%
I worry about disciplinary actions	68%	7%	24%
SiNASP can promote changes	2%	7%	90%
I receive feedback about improvements	54%	14%	32%
I have time to notify incidents	68%	7%	24%
SiNASP helps improving pt safety	8%	8%	85%

SiNASP hospital managers	Semi-structured phone interviews with 4 respondents Responses to some selected questions
SiNASP basic characteristics	Total agreement with all basic characteristics
SiNASP taxonomy, design and software	Agreement with taxonomy. SAC is difficult to score, more training needed. Good software but sometimes slow, the report generation area needs to be improved. A benchmarking system is needed
Internal management in hospitals	Training was sufficient to implement the system. Time and human resources needed to manage SiNASP is identified as main problem. Feedback has been limited
General assessment	SiNASP is useful and necessary. Main difficulties have been low safety culture and communication to staff. Improvements were implemented during the pilot based on notifications

Table 1.

Main requirements and recommendations identified by stakeholders involved in the development of SiNASP.

Stakeholder	Method and participants	Date	Main requirements and recommendations identified
Safety experts	Delphi (on-line questionnaire). 25 respondents (12 experts & 18 representatives) out of 30 invited (response rate 83%)	Spring2007	System with emphasis in learning, non punitive, that guarantees confidentiality of all people involved. Lack of agreement about if the notifications should be anonymous, tending to a mixed system.All different kinds of events should be reported to a unique questionnaire, preferably electronic. Notifications should be analyzed by the healthcare organization in a short timeframe, including quantitative and qualitative analysis and producing periodic reports, alerts and recommendationsConcepts and terminology should be unified, considering the WHO patient safety taxonomy as a good basis to do soThere is a need to analyze the legal framework and assess its possible modification
Regional authorities in patient safety
Patients	Two focus groups with representatives from 16 patient associations	Nov 2007	Positive opinion regarding the implementation of a reporting system, with the main goal of learning from errorsContextual obstacles: errors are not commonly recognized, lack of reporting culture, professionals’ fear of punitive consequencesPatient's role: patients are willing to participate, mainly in the phase to propose solutions, avoiding the attitude of inspectors or controllersConfidentially is generally accepted as the best possible solution for infra-reporting and for the culture of fear. However, there is no agreement about the convenience of modifying the legal framework in order to guarantee confidentiality of information
Healthcare professionals	Semi-structured on-line questionnaire to 59 professionals from 53 scientific societies	Oct 2008	High level of agreement with the proposed model: the healthcare organizations where incident occurred would be responsible for the analysis of the incident and for the improvement of the problems identified. The system should allow qualitative and quantities analysis of aggregated data by the regional government, and share standardized information with the Ministry's databaseIt is important to focus on the deep analysis of individual incidents that can facilitate learning opportunities, avoiding the “ranking effect” between institutionsMain barriers to notification: low participation from healthcare professionals due to fear, poor patient safety culture or lack of time; legal aspects related to the possible use of the information for punishment purposes; lack of patient safety culture (so staff could think this is not important); lack of response to the problems identified; and reporting systems previously implementedPossible facilitating factors: guarantee confidentiality, a simple and quick system which doesn’t require much time from the reporter, provide training to all staff involved, develop quick systems for feedback and provide evidence of adequate response to the incidents reported

Adverse event notification system developed: SiNASP

SiNASP main characteristics

SiNASP was developed by the Spanish Ministry of Health for the Health Regions that required a RLS with the goal to improve Patient Safety through the analysis of incidents and learning process at local level.

The basic principles of SiNASP were: voluntary reporting, no punitive consequences to professionals involved in the incidents, confidentiality of information, analysis of incidents and implementation of improvements performed at hospital level, systemic orientation for the analysis of incidents and anonymous reporting or nominative with anonymization (or de-identification, meaning that the software automatically eliminates this information after a 2 weeks period).

SiNASP has a pyramidal structure: 1) The healthcare organizations are responsible of the management, analysis of the incidents and improvement proposals; 2) The Health Regions are responsible for the periodic analysis of aggregated data in order to identify patterns and risks and to elaborate recommendations for all the organizations in the region; 3) The Spanish Ministry of Health has a similar role than the regions but with a broader scope, including all the hospitals participating in the system.

Reportable incidents to SiNASP are all kinds of patient safety incidents (events or circumstances which could have resulted, or did result, in unnecessary harm to a patient)¹⁶, excluding sabotages which are understood as serious offenses, deliberate deviations from norms or rules to harm either the patient or the system. These events have obvious legal implications and limited use from a learning point of view.

Reporting, analysis and learning process

The process to manage a safety incident within a healthcare organization follows the steps shown in Fig. 2. Incidents are detected either by direct observation (the professional is involved or directly observes the incident) or indirect information (the incident is identified by a third party or by any documentation that reflects it). The inclusion of non-healthcare professionals, patients and family members as reporters is still under discussion.

Fig. 2.

Process to manage a patient safety incident within a healthcare facility.

An adaptation of the “Severity Assessment Code” (SAC)²² is used to classify all incidents based on their associated risk. The SAC score guides the selection of the analysis method for the incident: Incidents with low and medium risk are commonly analyzed by monitoring of trended aggregated incident data, while for incidents with high and extreme risk, a deep analysis is recommended thought a root cause analysis or a similar method.

Finally the development of a risk reduction plan is needed, specifiying the actions to be accomplished, staff in charge, calendar and indicators to monitor the plan. Feedback to professionals includes the submition of a semi-automatic email to the reporter when the incident is received (if the reported included his/her contact details), phone calls in those cases where a deep analysis is required, periodic reports (with aggregated data, examples of incidents and the risk reduction plans developed), meetings to discuss about safety incidents, etc.

SiNASP taxonomy and software

The SiNASP software (https://www.sinasp.es) has a public area including the reporting questionnaire accessible to healthcare professionals through a code and a private area for the managers of the system, where individual incidents are managed and aggregated information is analyzed in order to developed feedback reports.

The reporting form has four sections: incident (characteristics and type of incident), patient (consequences for the patient and patient characteristics), contributing factors and risk reduction actions (Table 3). The form includes open and close questions with several response categories that facilitate the subsequent analysis of data thought groups, classifications and filters; some of the questions are mandatory. The form also includes specific questions for some incident types (for example for medication incidents) that show up in the form only after some alternatives have been selected, what reduces the perceived workload for the reporter.

Table 3.

Items included in SiNASP reporting form.

Incident
Where did it happen
	Care setting*
	Organization name*
	Unit/service
When did it happen
	Day
	Month*
	Year*
	Time when the incident happened
People involved in the incident
	N of people involved
	Type of people involved
Reporter
	Email address
	Name
	Profession
	Did the incident happen in the same organization that is reporting?*
	Region
	Reporting date*
Incident type
	Description of the incident*
	Incident type
	Frequency/likelihood of recurrence
Specific data for medication incidents
	No. of medications involved
	Commercial name
	Active substance
	Manufacturer
	Formulation or presentation
	Dose/Strength
	Type and size of packaging
	National code
	Administration route
	Stages involved
Patient
Patient outcomes
	Type of incident and degree of harm*
	Type of harm
Patient characteristics
	Age
	Gender
Incident's risk severity
	Severity Assessment Code (SAC)
Contributing factors
Factors that contributed to the incident*
	Category(s) of contributing factors
Actions to reduce risk
Actions to prevent similar incident*

*Fields with asterisk are mandatory, have to be filled out to be able to submit the report.

SiNASP Performance monitoring indicators

The SiNASP software includes an indicator scoreboard that provides information to facilitate the management of the system at different levels (Table 4).

Table 4.

SiNASP indicators.

Volume of incidents reported:

No. of incidents reported in the period

Ratio of incidents reported in the given period: no. incidents reported × 100/no. of beds

Characteristics of the incidents reported:

Severity of the incidents reported: % incidents with SAC 1 and SAC 2 out of the total number of incidents reported in the period

Identification of the reporter: % de incidents with reporter identified.

Management of incidents reported:

Average number of days since the incident is notified until it is assigned to a manager

Average number of days to manage an incident (since the incident is notified until it is closed)

Incidents reported >1 month ago that have not been managed jet (have not been closed)

Percentage of closed incidents that have at least one management action documented

Analysis of incidents:

% of incident with SAC 1 and SAC 2 that indicate a RCA is to be performed

% of total incidents that indicate a RCA is to be performed

N of RCA that have been finalized in the period*

N of meeting from the group in charge of the analysis of incidents that have been held in the period*

Feedback mechanisms:

Email to the reporters

N of reports saved in the system for the period

N of reports distributed for the period*

N of case presentation sessions held for the period*

SAC: Severity Assessment Code RCA: Root Cause Analysis.

*All the indicators are automatically calculated by the SiNASP software based on previously existing information, except for the ones that are marked with an asterisk, which require that SiNASP hospital managers introduce some specific information into the system.

Since incident RLS are relatively new in the healthcare arena, the development and use of indicators to monitor these systems imply some difficulties, such as the lack of background information to establish the standard for the indicators. For this reason, SiNASP's indicators provide objective data about the functioning of the system, but there is no value judgment associated to it. Indicators’ results can be analyzed mainly by comparative analysis (the organization receives information about its own results compared to the anonymous results from all the rest of the organizations using the reporting system) and trending analysis (observing the evolution of its own results).

Process to implement SiNASP in healthcare organizations

The implementation process includes an on-site training process for the reporting system managers, supporting documents and tools to facilitate the local management of the incidents, the internal training process and the implementation into the organization. An on-line training course is available to all healthcare workers from organizations using SiNASP, which provides basics topics on Patient Safety and information to the reporting process in SiNASP.

Discussion

SiNASP has been specifically designed to respond to the needs of Spanish healthcare organizations, taking into account the contextual characteristics and involving more than 100 representatives from patients, professionals, regional governments and safety experts.

SiNASP has been developed based on the current Spanish legal context. For this reason the design of this system allows the identification or anonymity of the reporter. After balancing the benefits of anonymous reporting (less risk of legal implications) and the benefits of identified reporting (information that allows the identification of the case in order to get additional information to perform an in deep analysis), the decision was to combine both options into a anonimization o de-identification system according to the reporter preferences. The de-identification system allows SiNASP managers to contact the reporter if it is needed during the first days after the report of the incident, but it would not be available after that time if it is required as part of a judicial process or for other reasons. Even when this is not an optimal solution, more than 60% of the reporters voluntarily included identification data during the pilot, what indicates that it was generally accepted. The most prevalent option at international level is the anonimization of the identification data from the people involved, which is the system implemented, for example, by Denmark and by the Veterans Administration in the United States. It is also important to highlight studies that show that there is very low coincidence between the incidents reported and legal sues, what considerably reduces the probability of facing this kind of problems.^23,24

Regarding the possible low participation of professionals, it was one of the main difficulties identified by stakeholders involved on this work and it is also one of the most commonly cited problems when reporting systems are analyzed,^25,26 so it was important to implement mechanism to approach the possible causes of under-reporting. The fear of a possible use of the information for punitive actions has been considered one of the main barriers to reporting by experts.^27,28 To mitigate this limitation, confidentiality of the information reported to SiNASP has been maximized thought a system that allows reporting only through an electronic questionnaire, the method that best guarantees confidentiality, even when its side effect is that it reduces accessibility to the reporting system in some organizations, as shown on the pilot. Other than that, formal management team commitment with the principles and characteristics of the reporting system is required before any healthcare organization get access to SiNASP.

The improvement of the safety culture within the healthcare organizations would also contribute to increase professionals’ confidence, as shown by studies that correlate the level of reporting in an organization with the existing safety culture.²⁹ For this reason, the on-site and on-line training courses for the implementation phase of SiNASP in any organization include patient safety culture concepts as an important element of the program. Another aspect that was identified as a main barrier on this study was the high workload and lack of time for reporting. Long reporting forms and insufficient time for reporting had also been previously identified as mayor obstacles,³⁰ so agile and simple software was a requisite. The system developed for SiNASP has an on-line questionnaire that includes a reduced number of closed multiple-choice questions and only four open free text questions, so reporting an incident is estimated to take less than five minutes, excepts for falls and medication incidents that have some additional questions and therefore require some extra time.

The development process presented and the characteristics of the system provide a comprehensive framework that can be used for future deployments of similar RLS.

Conflict of interest

The authors declare no conflict of interest.