Development and impact of computerised decision support systems for clinical management of depression: A systematic review

Abstract

One of the proposals for improving clinical practice is to introduce computerised decision support systems (CDSS) and integrate these with electronic medical records. Accordingly, this study sought to systematically review evidence on the effectiveness of CDSS in the management of depression. A search was performed in Medline, EMBASE and PsycInfo, in order to do this. The quality of quantitative studies was assessed using the SIGN method, and qualitative studies using the CASPe checklist. Seven studies were identified (3 randomised clinical trials, 3 non-randomised trials, and one qualitative study). The CDSS assessed incorporated content drawn from guidelines and other evidence-based products. In general, the CDSS had a positive impact on different aspects, such as the screening and diagnosis, treatment, improvement in depressive symptoms and quality of life, and referral of patients. The use of CDSS could thus serve to optimise care of depression in various scenarios by providing recommendations based on the best evidence available and facilitating decision-making in clinical practice.

Resumen

Una de las propuestas para conseguir mejorar la práctica clínica es la incorporación de sistemas informatizados de apoyo a las decisiones (SADC) y su integración con los registros clínicos electrónicos. El objetivo de este trabajo es revisar de forma sistemática la evidencia sobre la eficacia de los SADC en el manejo de la depresión. Para ello se realizó una búsqueda bibliográfica en Medline, EMBASE y PsycInfo. La calidad de los estudios cuantitativos se evaluó mediante el método SIGN y los estudios cualitativos mediante el checklist de CASPe. Se identificaron 7 estudios (3 ensayos clínicos aleatorizados, 3 ensayos no aleatorizados y un estudio cualitativo). Los SADC evaluados incorporaron contenidos derivados de guías u otros productos basados en la evidencia. En líneas generales, los SADC mostraron un impacto positivo sobre diferentes aspectos como el cribado y diagnóstico, tratamiento, mejora de síntomas depresivos y calidad de vida y derivación de pacientes a asistencia especializada. El empleo de SADC podría optimizar la atención de la depresión en diversos escenarios mediante la provisión de recomendaciones basadas en la mejor evidencia disponible y la facilitación de la toma de decisiones de los profesionales en la práctica clínica.

Introduction

Depression is one of the most frequent mental disorders and one of the main global causes of disability.^1,2 Despite advances on the approach, it is still very much associated to suicidal behaviour,³ a tendency towards recurrence and chronicity⁴ and, in some cases, the lack of response to different treatments.⁵ As regards care, there is evidence of an unsupported variability in the diagnosis and treatment, both by defect^6–10 and excess.^11,12

Clinical practice guidelines (CPG) may play an important role in the improvement of the those aspects, although its implementation entails many obstacles.^13,14 One of the proposals to overcome these barriers is to include it in the workflow^15,16 and to integrate it with electronic clinical records¹⁷ through the development of clinical decision support systems (CDSS), defined as “tools designed for the support of clinical decisions, where recommendations are made according to the characteristics of the patients”.¹⁸

Although some studies have shown that CDSS may contribute to improve the interaction of scientific evidence with the patients’ information, the results on its introduction into clinical practice are currently limited and at the time being, it is impossible to draw a final conclusion on aspects such as the cost-effectiveness, workload or efficiency of these systems.^18–23

The purpose of this article is to review the scientific evidence available on the efficacy of CDSS in the clinical management of depression. The intention was to answer 3 questions: (1) What is the efficacy/effectiveness of CDSS in the diagnosis and management of depression?; (2) What is the acceptability and satisfaction of professionals and patients with these systems? and (3) What are the characteristics of CDSS used which may be associated to an improvement in health processes or results?

Methodology

This systematic review has been carried out as part of a broader project to update the CPG on the management of major depression in adults²⁴ and follows the recommendations of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA).²⁵

Search strategy

A preliminary search of systematic reviews was carried out. The review prepared by the Agency for Healthcare Research and Quality (AHRQ)¹⁸ was chosen because it was the most recent and the one with the highest quality, and its bibliography search was updated from January 2011 to February 2014. For such purposes, its search strategy was replicated on general databases (Medline [PubMed] and EMBASE [OvidSP]) and specialised data bases (PsycInfo [OvidSP]). The search strategy combined terms related to the type of studies (case–control studies, cohort studies, clinical trial, multi-centre study, clinical practice guideline, validation studies, meta-analysis), intervention (clinical decision support systems, computer-assisted therapy, reminder systems, medical order entry systems, provider order entry, physician order entry) and disorder (depression, depressive disorder, major depressive disorder, mood disorders). The bibliography of all selected studies was also reviewed.

Inclusion and exclusion criteria

Studies where any electronic system specifically designed for the purposes of supporting the decision making for the clinical management of depression were included (at a preventive, diagnosis or therapeutic level); the specific characteristics of a patient were used to generate a certain indication for professionals; and both the efficacy/effectiveness and the satisfaction or acceptability of CDSS were evaluated, regardless of the design. No language restrictions were established.

Selection of studies

The results of the bibliographic search were subject to independent review by 2 authors through reading of the title and summary. Afterwards, the full text of potentially relevant articles was revised to consider inclusion. Discrepancies were solved through discussion with a third author.

Data gathering and extraction

Evidence tables with the most relevant characteristics, methodology and results of each study were prepared. Data extraction was carried out by one author and confirmed by another, so as to solve conflicts by consensus.

Assessment of the quality of evidence

The quality of quantitative studies was assessed using the method proposed by the Scottish Intercollegiate Guidelines Network (SIGN).²⁶ Qualitative studies were carried out using the checklist of the Critical Appraisal Skills Programme (CASP) and, following the proposal in Goldsmith et al.,²⁷ they were classified as ++ (meeting all or most methodological criteria), + (meeting some criteria) and − (meeting few or none of the methodological criteria).

Results

Studies included

The systematic review of the AHRQ¹⁸ included 4 studies on the inclusion of CDSS in the management of depression.^28–31 Since this work included basically randomised controlled trials (RCT), excluded studies were reviewed, although none of them met the inclusion criteria of this review. Further, the update of the bibliographic search provided 472 references. After the review of the titles and summaries, 5 studies were considered for full text reading, of which 3 finally met the inclusion criteria (Fig. 1).

Figure 1.

Diagram of selection of studies.

Characteristics and quality of studies included

Six quantitative studies were included: 3 RCT (level 1+ in the SIGN scale)^28,30 and 3 non randomised trials (level 2+).^31–33 Only one qualitative study was identified,³⁴ and was classified as Q+ (because it met most of the quality criteria required).

Five studies were carried out in the US,^{28,29,31–33} one in Europe³⁰ and another one in Australia.³⁴ Of those studies, 2 were carried out in the academic field with ambulatory patients,^28,29 3 were carried out in primary care,^30–32 one in speciality care³⁴ and some in primary care as well as speciality care.³³

All evaluated CDSS included contents derived from CPG^28,29,31,32 or other products based on the evidence^30,33,34 and only 2 of the systems were integrated with the electronic medical record (EMR).^29,32 Most were locally developed systems (designed in the context of a specific health care system), except the study of Rollman et al.²⁹ which used a commercial system.

The duration of the intervention varied from 6 to 20 months. In 4 of the studies, the intervention was compared to the common treatment^30–33 (although in one of them CPG access was strengthened³⁰) and in 2 others it was compared to another implementation method.^28,29

The most relevant methodological aspects and results of the quantitative studies reviewed are presented in Table 1.

Table 1.

Main methodological aspects, results and quality of the evidence of quantitative studies included.

StudyArea/country	Clinicians/patients (n)Duration	Intervention	Control	Result variables	Effect (intervention versus control)	DesignLOE
Cannon and Allen^28Academic/USA	1/789 months	The CDSS sends reminders to perform the screening and diagnosis according to DSM-IV	Written Checklist	Percentage of patients screened, percentage of documentation in the medical record according to DSM-IV	Screening: 86.5 versus 61%; p = 0.008Documentation according to DSM-IV: 100 versus 5.6%;p < 0.0001	RCT1+
Rollman et al.^29Academic/USA	16/21220 months	Interactive notification on the EMR when the depression module identified a patient with risk factors for depression, requiring mandatory response	Protocol of written diagnosis; common treatment	Percentage according to the diagnosis after notification	After 3 days: agreement: 120/186 (65%); disagreement: 24/186 (13%); uncertain: 42/186 (23%)After one month: agreement: 147/186 (71%); disagreement: 34/186 (16%); uncertain: 27/186 (13%)After 154 days: agreement: 166/186 (78%); disagreement: 36/186 (17%); uncertain: 10/186 (5%)“No significant differences were found on agreement rates or the treatments of the 3 interventions”	RCT1+
Thomas et al.^30PC/UK	NA/7626 months	Patients completed a computer psychosocial assessment based on which the system generated a report with treatment recommendations (for depression and/or anxiety)	Common treatment with access to CPG	Quality of life	Control:Baseline (n = 387): mean (95% CI): 4.7 (4.4–4.9)6 weeks (n = 319): mean (95% CI): 5.8 (5.4–6.1)6 months (n = 299): mean (95% CI): 6.2 (5.8–6.6)Intervention:Baseline (n = 358): mean (95% CI): 4.8 (4.5–5.1)6 weeks (n = 283): mean (95% CI): 5.9 (5.5–6.2), NS6 months (n = 243): mean (95% CI): 6.4 (6–6.9), NS	RCT1+
Kurian et al.^31PC/USA	4/559 months	Computerised algorithm of drug treatment for depression	Common treatment	Reduction of depressive symptoms, response and remission	“Reduction of depressive symptoms: decrease in the score of HDRS in the CDSS group (p < 0.001)”Response: 59 versus 61%Remission: 44 versus 39%	2+
Gill et al.^32PC/USA	256/17,1547 months	The system included one screening tool and other diagnosis and management tools for bipolar disorder	Common treatment	Percentage of patients with new diagnosis and treatment for the disorder	Diagnosis: 1.11 versus 0.36%; p < 0.01Treatment: 1.85 versus 1.19%; p < 0.01	2+
Williams et al.^33PC, MH/USA	NA/6526 months	Screening alerts system and treatment of depression in patients hospitalised after an CVA, treated after 6 months in primary care or neurology service + training	Common treatment	Percentage of patients screened 6 months after infarction and percentage of treatment	Screening: 85 versus 50% (OR 6.2; p < 0.001)Treatment: 83 versus 73% (OR 1.8; p = 0.13)	2+

CVA, cerebrovascular accident; PC, primary care; DSM-IV, Diagnostic and Statistical Manual of Mental Disorders, fourth edition; RCT, randomised clinical trial; CPG, clinical practice guidelines; EMR, electronic medical record; HDRS, Hamilton Depression Rating Scale; CI, confidence interval; NA, not available; LOE, level of evidence: NS, not significant; OR, odds ratio; CDSS, clinical decision support systems; MH, mental health.

Source: original content.

Effect of the intervention

Support for the decisions on diagnosis and treatment

Four studies specifically evaluated the impact of CDSS on the screening and diagnosis of depression. One RCT compared the effect of including 2 recommendations on screening and diagnosis of depression in computerised versus manual format (through a checklist). The rate of identification and diagnosis according to the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) increased significantly in the computerised version.²⁸ Another RCT compared the provision of an electronic feedback over diagnosis and treatment versus written reminders and the common treatment. No differences were found on the rate between the groups in the 3 evaluated scenarios. However, the professionals who agreed with the diagnosis (65% of 186 clinicians), documented it to a greater extent, prescribed significantly more drug treatments and referred to speciality care more frequently.²⁹ Other 2 quasi-experimental design studies evaluated the efficacy of CDSS on the screening of depression after a cerebral infarction³³ and on the screening of bipolar disorder in patients diagnosed with depression.³² In both studies, CDSS significantly improved the number of patients screened and those with positive screening had a higher likelihood of receiving the right treatment.^32,33

Of the 2 remaining studies, a RCT evaluated the effects on the quality of life of the use of CDSS in the management of anxiety and depression, as compared to the common treatment. The intervention consisted of a computerised psychosocial assessment which generated recommendations regarding clinical management. Although the experimental group presented higher levels of quality of life after 6 weeks, the difference was not sustained after 6 months.³⁰ Lastly, only one study assessed the effects of the implementation of CDSS over the reduction of depressive symptoms or the response/remission. This small, non-randomised trial, evaluated the effectiveness of the computerised algorithm for depression of the Texas Medication Algorithm Project as compared to the common treatment in primary care, and evidenced that patients seen by professionals using CDSS showed a significant reduction in depressive symptoms, although no differences were found on response/remission.³¹

Acceptability, satisfaction and the perspective of patients and professionals

Although not explicitly evaluated in any of the included quantitative studies, in the study of Rollman et al.,²⁹ agreement with the diagnosis of depression suggested by CDSS was moderate in the short term (<75% after 3 days and after one month) and high in the long term (>75%). Furthermore, in another study, professionals indicated that the tool was easy to use and stated that they preferred this option rather than the common treatment.³¹

The only qualitative design work identified investigated the perception of clinicians and patients on a CDSS for the electronic screening of depression after a cerebrovascular accident. Prior to consultation, patients filled out a computerised version of the PHQ-2 questionnaire and if the score was positive, the assessment was completed with PHQ-9 and items selected from the Distress Problem List. The clinician received a notification on the score, its interpretation and management options for patients where PHQ-9 ≥5. Most patients (n = 62) found the electronic screening tool easy to understand and answer and considered it a good screening method. In turn, professionals (n = 7) acknowledged that CDSS had in a way changed their professional involvement and that, in most cases, the assessment and follow-up of the depressive process had improved.³⁴

Technical characteristics of the interventions evaluated

The most relevant technical factors of CDSS are included in Table 2. Some studies do not describe them properly and, in addition, a high level of heterogeneity has been found in the systems used. None of the studies included all aspects associated with the effectiveness of CDSS according to the work of the AHRQ,¹⁸ although there seems to be a tendency towards the inclusion of a higher number of them in most recent studies. It is worth noting that all systems were centred only on favouring the decision making of professionals and in no case did the tool have the purpose of favouring the patients’ decision making.

Table 2.

Technical characteristics of systems to support the decisions of evaluated interventions.

	Rollman et al.^29	Cannon and Allen^28	Thomas et al.^30	Kurian et al.^31	Gill et al.^32	Williams et al.^33	White et al.^34
General
Integration with the entry into the system, to guarantee introduction into the workflow	Yes	No	Yes	Yes	No	Yes	NA
Clinician-system interaction
Automatic introduction of the information as part of the workflow	NA	No	No	Yes	No	Yes	Yes
No introduction of additional data by the clinician	No	No	Yes	NA	No	Yes	Yes
Provision of support where and when decisions are made	No	Yes	NA	Yes	Yes	Yes	Yes
Communication of the content
Providing recommendations	Yes	No	No	Yes	No	Yes	Yes
Advocacy of action rather than inaction	No	Yes	No	NA	No	NA	Yes
Justification of the decision on the basis of scientific evidence	No	Yes	No	Yes	Yes	NA	NA
Others
Involvement of professionals in the development of the tool	No	Yes	NA	Yes	NA	Yes	NA
Facilitation of decision making for professionals and patients	No	No	No	No	No	No	No

NA, not available.

Source: original content and adaptation from Lobach et al.¹⁸

Discussion

The management of depression is an area with many possibilities for improvement, mainly due to its under- and over-diagnosis and the great variability evidenced in clinical practice. On the other hand, the decision making involves a number of interactions among a great variety of people involved and sources of information. At the clinical management level, the information needs of professionals and the sources they use may be several, including personal experience, published scientific research, reliance on clinical experts, CPG and technology assessment reports.³⁵ The successful use of scientific evidence and the implementation of the recommendations obtained through it may imply an important step in the improvement of care.¹⁵ The EMR is of particular importance here, since it may be a tool to provide recommendations on the management of a certain entity or of interaction with relevant information for the decision making.^29,36 Thus, different studies have proved the effectiveness of integrating CDSS into the EMR, evidencing an increase of adherence to the recommendations and an improvement in the results on health.^15,37,38 Moreover, these systems may be oriented to primary and speciality care, guaranteeing continuity and the process of assistance. It all seems to indicate that, in the future, EMRs will no longer be mere information stores, and an will have a greater interaction with professionals through the provision of information related to preventive activities, specific risks of the patient, more suitable therapeutic options and facilitating follow-up or providing reference information, such as the information for patients, among others.

In this systematic review, the impact of CDSS applied to the clinical management of depression has been evaluated, and a positive impact was observed over screening^32,33 and diagnosis,^28,29 treatment^29,32,33 and referral of the patient to mental health.²⁹ The support for the decision making has also been associated with a significant reduction of the symptoms of depression³¹ and an increase in the level of quality of life.³⁰ Although most of the studies did not include variables related to the satisfaction and acceptability of users, in one of those studies, professionals stated that they preferred this option to the common treatment³¹ and in another study, there were high rates of agreement with the information generated by the system.²⁹ In turn, the qualitative study evidenced high satisfaction, both from patients and clinicians, with the tool used.³⁴

Although they cannot be strictly considered as CDSS, there is also evidence of other interventions based on the EMR directed to improving the screening of depression in primary care.^39,40 As regards the use of algorithms in depression, the experience of the Texas Medication Algorithm Project, designed for the purpose of promoting the use of pharmacological algorithms combined with clinical support and information to patients, stands out globally.¹⁵ The computerised version of the Texas Medication Algorithm Project⁴¹ includes aspects related to the prevention, diagnosis, support of decisions regarding treatment and advice on secondary effects and monitoring and, although insufficient, the data published on its effectiveness show improvements in the reduction of depressive symptoms as compared to the common treatment.³¹ Further, the STAR*D study also evidenced adherence to algorithms in the management of depression and its positive impact on the clinical practice.^42,43 At a national level, the Departament de Salut de la Generalitat de Catalunya has promoted the project to computerise the CPG of major depression in adults of the Spanish National Health Service (Servicio Nacional de Salud, SNS)⁴⁴ to use in all primary care centres of the Institut Català de la Salut, by integrating it into the EMR. As the guideline has been simultaneously activated at all centres of the Institut Català de la Salut, it was not possible to carry out a randomised study on the effectiveness and we have opted to evaluate a strategy for multifaceted implementation at primary care centres of the same region in a controlled way. The contrasted hypothesis is that an active implementation process significantly improves the use of the guideline and, consequently, the results on health. An automatic feedback will be regularly added to the initial actions for the implementation of a computerised guideline, which may have a momentary impact, for physicians with information on process results (use of instruments of the guideline) and on the health of patients on which it is used.

As regards the characteristics of CDSS, the review of the AHRQ¹⁸ identified the factors associated with the efficacy or success in the implementation of the integrated systems and confirmed other previously proposed factors (integration with entry into the system, clinician-system interaction factors, factors related to the communication of content, among others).¹⁶ None of the studies reviewed included all these aspects. The introduction with the entry into the system and the provision of information at the time of the decision making were the characteristics that the studies presented more consistently.

The results of this review must be interpreted taking into consideration its limitations. The most important limitations are the number and quality of the studies included, which may be considered insufficient to establish a final conclusion. Regarding these limitations, it is worth noting that the systems reviewed are part of a heterogeneous group with different types of tools, samples, objectives, methodology and variables assessed, which did not allow a meta-analysis. In addition, there is a clear absence of information regarding key characteristics of the implementation, which are neither explicitly considered nor explicitly described in the reviewed studies.

The importance of CDSS in the management of depression becomes more relevant currently with the strong investment of health authorities in the development of new information and communication technologies in the field of clinical practice. Future research shall be oriented towards a deeper assessment of potential adverse effects and the relevant variables to measure its real impact on patients and health systems, as well as the related cost-effectiveness. Effective implementation of CDSS is a complex challenge demanding the interaction of specific characteristics of organisations, technology and the scientific evidence available. In this regard, it has been stated that next-generation integrated systems should be based on evidence, with a continued update of scientific advances, and must be flexible and adaptable to the different scopes of application.¹⁸ Further, the broad introduction of CDSS requires great advances in understanding what information to include, and when and how it should be available for clinicians and patients, as well as a critical assessment of its implementation.¹⁹

In summary, the use of CDSS and its integration with the EMR may improve depression care in several scenarios through the provision of recommendations based on the best evidence available and facilitating the decision making process for professionals. However, current evidence is limited, and it is therefore necessary to deepen the real impact of these systems in the clinical practice and issues that may affect its implementation.

Funding

This study has been carried out within the development of activities of the Spanish Network of Agencies for Health Technology Assessment and SNS Services (“Red Española de Agencias de Evaluación de Tecnologías Sanitarias and Prestaciones del SNS”), funded by the Spanish Ministry of Health, Social Services and Equality.

Conflict of interest

The authors declare that there are no conflicts of interest.

Acknowledgements

The authors would like to thank the Workgroup of the Clinical Practice Guideline for Depression on Adults, issued by the Health Technologies Assessment Agency of Galicia (avalia-t) (“Agencia de Evaluación de Tecnologías Sanitarias de Galicia, avalia-t”) and Servizo Galego de Saúde, for their support and collaboration in the conceptualisation and development of this work.