People with psychotic disorders can experience a lack of active involvement in their decisional process. Clinical decision aids are shared decision-making tools which are currently rarely used in mental healthcare. We examined the effects of Treatment E-Assist (TREAT), a computerized clinical decision aid in psychosis care, on shared decision-making and satisfaction with consultations as assessed by patients.
MethodsA total of 187 patients with a psychotic disorder participated. They received either treatment as usual in the first phase (TAU1), TREAT in the second phase or treatment as usual in the third phase of the trial (TAU2). The Decisional Conflict Scale was used as primary outcome measure for shared decision-making and patient satisfaction as secondary outcome.
ResultsA linear mixed model analysis found no significant effects between TAU 1 (β = −0.54, SE = 2.01, p = 0.80) and TAU 2 (β = −1.66, SE = 2.63, p = 0.53) compared to TREAT on shared decision-making. High patient rated satisfaction with the consultations was found with no significant differences between TAU 1 (β = 1.48, SE = 1.14, p = 0.20) and TAU 2 (β = 2.26, SE = 1.33, p = 0.09) compared to TREAT.
ConclusionWe expected TREAT to enhance shared decision-making without decreasing satisfaction with consultations. However, no significant differences on shared decision-making or satisfaction with consultations were found. Our findings suggest that TREAT is safe to implement in psychosis care, but more research is needed to fully understand its effects on the decisional process.
Psychotic disorders occur in approximately three to four percent of the Western population.1,2 They are often characterized by severe and persistent positive symptoms (such as hallucinations, delusions and incoherent thoughts) and negative symptoms (such as flat affect, loss of initiative and social withdrawal).3 Apart from psychiatric symptoms, people with psychotic disorders are usually faced with other challenges, such as somatic issues, loneliness, stigmatizing and difficulty to participate in work or study. This all contributes to a lower quality of life4-6 and a severely reduced life expectancy.7 Some symptoms, somatic problems and psychosocial issues remain unresolved and are considered care needs that persist over time.8-11 Identifying these needs and subsequently indicating adequate treatment is important, but challenging, requiring a long-term iterative process between patient and clinician. Routine outcome monitoring (ROM), described as the use of standard instruments to systematically assess the health and wellbeing of patients,12 can be helpful in identifying care needs of patients. The systematic feedback of ROM results improves the treatment process in mental healthcare.13 Ideally, ROM provides input for shared decision-making (SDM) about patients’ existing issues and the course of treatment but the integration into daily clinical practice of psychosis care can be challenging.14
Shared decision-makingIn recent years there has been a gradual transition in mental healthcare from a paternalistic model, in which clinicians hold most of the knowledge and decisional power, towards a more patient-centered informed choice model of decision-making.15 SDM is a process intended to facilitate this transition by strengthening the exchange of information and the decisional position of patients with a severe mental illness.16 Many definitions of SDM exist but a systematic review identified some of the most consistent aspects: present treatment options to the patient, clarify decisions to be made, discuss benefits and risks of treatment options, incorporate patient preferences and values, make recommendations and decide on treatment course.17 Mental health professionals acknowledge the importance of SDM yet it is not regular practice in the treatment of severe mental illness.16 In a UK survey (n = 5028) for example nearly two-third of the people with psychotic illnesses did not feel actively involved in the decisional process regarding their own treatment.18 This highlights the importance for continued efforts to increase SDM in mental healthcare,19 for which different types of interventions are available.20 A concrete example are clinical decision aids (CDAs), which gain popularity and facilitate aspects of SDM in various medical disciplines.20,21
Decision aids and treatment E-assistCDAs are tools such as evidence-based booklets, websites or computer applications, supporting patients and clinicians with healthcare decisions.22 These tools improve the decisional position of patients by increasing their knowledge about decisions that need to be made and available treatment options.22 A growing body of evidence shows the efficacy of CDAs,21 yet research on these tools in the treatment of severe mental illness is limited.23 In psychosis care, lower re-hospitalizations rates and reduced clinical symptoms were demonstrated when a CDA was used.24,25 Concerns exist about potential negative effects of CDAs, for example by depersonalizing interactions during clinical encounters.26 Furthermore, computerized CDAs might require high technological proficiency to use, potentially disrupting existing workflows during consultations.27 Drawing on these findings, we developed Treatment E-Assist (TREAT). TREAT is a pragmatic and user-friendly, computer based CDA for the treatment of people with psychotic disorders. To the best of our knowledge, TREAT is the first CDA in mental healthcare combining ROM results with guidelines and standards of care to generate personalized treatment advice for individual patients. The developmental process and pilot results are published elsewhere.28 TREAT highlights patients’ care needs during treatment plan consultations based on their latest ROM screening and provides relevant treatment suggestions for these needs based on the Dutch multidisciplinary guideline for schizophrenia and the care standard for psychosis.29,30 We expect TREAT to improve all aspects of SDM between patient and clinician by presenting treatment decisions, increasing the exchange of information and knowledge about guideline-recommended treatment options as well as opening the discussion about these options and patients’ personal preferences without decreasing their overall satisfaction with the consultations.
Research aimThis first aim of this study is to investigate whether patients with psychotic disorders working with a clinical decision aid (TREAT) experience more shared decision-making than patients receiving treatment as usual. Secondly, we examine the effects of this CDA (TREAT) on the overall satisfaction of these patients with their consultations.
Material and methodsSetting and participantsFrom the 33 clinicians originally contracted, a total of 27 clinicians actually participated in the trial and worked with TREAT. They were recruited from four mental healthcare institutions in the Northern-Netherlands. All clinicians worked in flexible assertive community treatment (FACT) teams,31 as psychiatrists (n = 13), psychologists (n = 3) or nurse-practitioners (n = 11). Clinicians received a brief training on how to use TREAT. All patients of the participating clinicians which were successively scheduled for treatment plan consultations were asked to participate in this study by their secretariat. Patients had to be adults with a DSM-5 diagnosis of a psychotic disorder (295.90, 295.40, 295.70, 297.1, 298.8 or 298.9).32 They needed to complete their most recent Pharmacotherapy Monitoring and Outcome Survey (PHAMOUS) screening,33 be fluent in Dutch and able to give an informed consent. Clinicians were reminded by the research team about upcoming measurements after which the questionnaires were provided by mail. Participation was voluntary and all patients signed an informed consent. The Medical Ethical Committee of the University Medical Center Groningen (UMCG) approved this study (Research registration number 201700763, date January 9, 2018). Our procedures were in accordance with local legislation and the Declaration of Helsinki. Data was collected from April 2018 until March 2020 (aborted 7 weeks sooner than planned due to the COVID-19 pandemic).
Trial design and sample sizeAn ABC study design with three phases lasting 8 months (the last phase was seven weeks shorter due to the COVID-19 pandemic) was used. In the A phase clinicians provided treatment as usual (TAU) in their treatment plan consultations. During the TAU consultations the treatment plans for the upcoming year were drafted. The outcomes of the PHAMOUS screening were available for clinicians and patients for discussion during these consultations in the form of a standardized letter. In the B phase the clinicians used TREAT in their treatment plan consultations. TREAT is CDA available in the electronic patient file of the patient which summarized the outcomes of the PHAMOUS screening and combined them with treatment recommendations. TREAT was only accessible in this phase and intervention fidelity was checked by examining whether TREAT reports had been generated for all included patients. In the C phase, the clinicians provided only TAU again. Clinicians participated with a minimum of 1 and a maximum of 4 consultations during each phase (phase 1: M = 3.22, phase 2: M = 2.66, phase 3: M = 2.25). Patients were not allowed to be included more than once in this study. We calculated the approximate needed sample size based on a review by Stacey et al.21, in which CDAs were tested against TAU, with SDM as main study outcome. The CDA condition had a mean average of 20,61 and a pooled standard deviation of 16,68. The TAU condition had a mean average of 28,48. Based on a power of 0.8, an alpha of 0.05 using an independent t-test: an N = 71 was needed for each group. At least 71 participants should be included in each condition, totaling N = 213 patients. Attrition rate was estimated at 14%, therefore we strived to include a minimum total of 243 patients.
Outcome measuresDecisional conflict (DC) provides insight into the quality of the decisional process and decisional outcomes from a patient perspective.34,35 DC is a useful construct for evaluating the application of SDM in daily clinical practice.21,36,37 Therefore, we used DC measured with the Decisional Conflict Scale (DCS),38 as the primary outcome to measure SDM. The DCS consists of 16 items on a 5-point Likert scale, ranging from strongly agree (0 points) to strongly disagree (4 points). The sum of all items was divided by 16 and multiplied by 25. Resulting scores range from 0 to 100 with higher scores indicating more decisional conflict. Scores below 25 indicate an absence of decisional conflict and thus complete SDM, whereas scores above 37,5 indicate an uncomfortable decisional process and thus suboptimal SDM.35 The DCS consist of five subscales, reflecting important aspects of SDM: 1) feeling informed about the treatment options, 2) being clear about ones values regarding the treatment options, 3) feeling supported in decision-making, 4) feeling uncertainty about the decision and 5) feelings about the quality of the decision.38 Scores for all subscales were recalculated to range from 0 to 100. The DCS has test-retest correlations and Cronbach alpha coefficients ≥0.78.38 A validated Dutch translation of the DCS was used.37
Patient satisfaction ratings of the consultations were used as a secondary outcome, measured by the Session Rating Scale (SRS).39 The SRS is a Visual Analog Scale (VAS) which measures satisfaction with clinical encounters on four dimensions: (1) relationship, (2) goals and topics, (3) approach and method and (4) overall satisfaction. The total score of these 4 items ranges from 0 to 40 points. Higher scores indicate a more positive rating of the session. A validated Dutch translation of the SRS was used with a Cronbach alpha of 0.89 and test-retest correlation of 0.57.40 Patients filled in paper versions of the DCS and SRS directly after their consultations and returned the questionnaires in a closed envelope, ensuring anonymity.
AnalysisDemographic characteristics between conditions were tested with independent-samples t-tests (age and duration of illness) or Pearson's chi-square (χ2) test (gender and diagnosis). The effects of TREAT on SDM were analyzed with a multilevel analysis comparing both TAU conditions to the TREAT condition. Prior to the analysis, the intraclass correlations (ICC) were calculated for the DCS and SRS to assess the ratio of within-groups variance to between group-variance components as an indication for clustering of the data within clinicians.41 The ICCs were <0.05 for both the DCS and SRS, indicating low levels of clustering within clinicians. A two-level linear mixed model was built for all outcome measures. Clinicians were modeled as level 2 and patients were modeled as level 1. To account for the ABC design, both TAU 1 (1, 0, 0) and TAU 2 (0, 0, 1) were dummy coded and added to the model as a fixed effects. In total 3% of the data on the DCS and SRS was missing at random. Random effects for the intercepts were used with variance components as covariance structure.42 Multiple imputations with predictive mean matching were used to impute missing data, adding to a total of five imputed datasets which were combined using Rubin's rule.43 The pooled effects were compared with the original dataset to evaluate the impact of the imputation. All statistical analyses were tested against a 0.05 significance level and performed using the Statistical Package of the Social Sciences (SPSS), version 27.44
ResultsDemographicsWe included 187 patients in this study, of which n = 87 in the first TAU condition, n = 64 in the TREAT condition and n = 36 in the second TAU condition (see Table 1). The unequal distribution can be attributed to the drop-out of participating clinicians (3 dropouts for the TREAT condition and 7 for the second TAU condition). In total 17 clinicians completed the trial. Drop-outs were due to job changes or a lack of eligible patients in their caseloads. TREAT reports were generated for all 64 patients in the TREAT condition, indicating appropriate intervention fidelity. Our sample contained slightly more men (69.9% vs 65.8%), were slightly older (49.2 vs 45.1) and had a longer average duration of illness (23.3 vs 17.6 years) compared to the PHAMOUS population.31,33 No significant differences were found in age, gender, duration of illness or diagnosis between conditions except for the percentage of people with a schizoaffective disorders (see Table 1).
Demographics and clinical characteristics of patients in TREAT trial (n = 187).
| Demographics | TAU 1 Mean | TREAT Mean | TAU 2 Mean | p-value/χ2 |
|---|---|---|---|---|
| Age, years (SD) | 48.1 (9.2) | 49.9 (10.2) | 52.3 (10.2) | 0.13 |
| Gender male,% (n) | 65.0 (52) | 74.0 (37) | 75.0 (21) | 0.44 |
| Illness duration years (SD) | 22.4 (10.3) | 22.2 (11.8) | 27.5 (11.6) | 0.11 |
| Diagnosis% (n) | TAU 1 Mean | TREAT Mean | TAU 2 Mean | χ2 |
| Schizophrenia | 46.0 (40) | 42.2 (27) | 55.6 (20) | 0.08 |
| Schizoaffective disorder* | 19.5 (17)* | 12.5 (8) | 0 (0)* | 0.03 |
| Substance induced | 10.3 (9) | 14.1 (9) | 8.3 (3) | 0.51 |
| Definitive diagnosis missing | 9.1 (6) | 15.3 (10) | 14.3 (5) | 0.58 |
| Schizophreniform disorder | 2.3 (2) | 0 (0) | 2.8 (1) | 0.45 |
| Delusional disorder | 1.2 (1) | 1.6 (1) | 2.8 (1) | 0.73 |
| Definitive diagnosis missing | 12.6 (11) | 12.5 (8) | 13.9 (5) | 0.91 |
The total mean scores on the DCS indicated high levels of SDM (see Table 2) but did not significantly differ between both TAU 1 (β = −0.54, SE = 2.01, p = 0.80) and TAU 2 (β = −1.66, SE = 2.63, p = 0.53) compared to the TREAT condition. Furthermore, mean scores on the different subscales of the DCS were also not significantly different between both TAU conditions and the TREAT condition (see Table 2).
Mean scores, betas, confidence intervals and p-values decisional conflict scale and session rating scale.
DCS = Decisional Conflict Scale; SRS = Session Rating Scale.
* Range DCS Total and subscales = 0 – 100.
* Range SRS Total = 0 – 40.
* Range SRS subscales = 0 – 10.
There were no statistically significant differences in total mean scores on the SRS between both TAU 1 (β = 1.48, SE = 1.14, p = 0.20) and TAU 2 (β = 2.26, SE = 1.33, p = 0.09) compared to the TREAT condition. Nor were there significant differences between both TAU conditions compared to the TREAT condition on the individual items of the SRS (see Table 2).
The original data and pooled data were compared to test the impact of multiple imputation on the outcomes (Supportive Information S2). Deltas between the pooled effects and the effects of the original dataset across both models varied between β = 0.35 and β = 0.40 on the DCS and β = 0.08 and β = 0.03 on the SRS, indicating adequate imputation.
DiscussionThe first aim of this study was to investigate the effects of the clinical decision aid (CDA) Treatment E-Assist (TREAT) on the level of shared decision-making (SDM) in the treatment experienced by people with psychotic disorders. No significant differences in SDM between both TAU conditions and the TREAT condition were found. This lack of measurable difference in SDM could be explained by the so-called ceiling effect. When score distributions of a certain variable tend to be skewed, as happens with ceiling, a regression could lead to inaccurate predictions.45 More than half of all patients scored 25 or lower on the DCS indicating absence of decisional conflict during consultations. Average scores on the DCS and its subscales were well below the threshold of 37.5 for indicating an uncomfortable decisional process. Therefore, potentially facilitating effects of TREAT on aspects of SDM are harder to detect. These high levels of SDM were unexpected and different from another recent study on Dutch patients in specialized mental healthcare (17% psychosis), in which an uncomfortable decisional process was reported with mean DCS scores of 38.8.37 These differences might stem from the relatively long duration of illness in our sample (mean: 23 years), likely meaning that most participants are very knowledgeable about their illness. They mostly receive long-term care, focused on long-term disease management. These treatment decisions tend to be less acute compared to first episodes of psychosis or other medical disciplines, such as oncology or cardiology. CDAs in those medical fields have shown to increase aspects of SDM, but those treatment decisions were surrounded by considerably more decisional conflict at baseline compared to our sample.46-48 Notably, our results did not confirm earlier findings from a qualitative assessment in which the same clinicians perceived more SDM while using TREAT because of a perceived increase in the exchange of information and discussion about treatment options.14 This exchange of information aspect of SDM correlates to the informed subscale of the DCS, yet no significant differences were observed on this subscale. Another aspect of SDM is the incorporation of patients’ personal preferences and values in the decisional process, but the non-significant differences on the value clarity subscale do not indicate improvements on this aspect either. In sum, it would be valuable to repeat this experiment in a sample of patients and clinicians who experience more decisional conflict.
The second aim of this study was to examine the effects of TREAT on the overall satisfaction of patients with their consultations. Although the satisfaction scores in the TREAT condition were lower than both TAU conditions, these differences were not statistically significant. High satisfaction was found in all conditions, indicating that TREAT did not cause large changes in patients’ perceived quality of the clinical encounters. Existing concerns about potential negative effects of CDAs on the therapeutic relationship in mental healthcare settings, potentially depersonalizing consultations, were not confirmed by our findings. Overall, these findings suggest that computer based CDAs such as TREAT are safe to implement in psychosis care from a patients’ point of view but so far no indication was found for its use in increasing SDM.
Strengths & limitationsTo the best of our knowledge, this is the first study in psychosis care evaluating the effects of a CDA during clinical encounters across multiple mental healthcare institutions. Considering the broad inclusion criteria for participants, this study has high ecological validity.
Randomization was impossible as TREAT can only be switched on or off in the electronic patient record for all participating clinicians. We opted for an ABC design to ensure that clinicians and had no prior exposure to TREAT. A strength of this design is that participating clinicians provided care in both conditions of the trial. This limits the variance on a clinician level, as was shown by their low intra class correlations. The second TAU mitigated potential time effects and served as a control-comparison for the first TAU phase, since conditions may change over time, for example due to changes in institutional policies or implementation of new guidelines and treatments. The COVID-19 pandemic caused a potential time effect hence the last TAU phase was stopped earlier than intended. This resulted in a somewhat underpowered study but with almost no missing data. At the same time we were able to study a potential carryover effect from the TREAT intervention.
A TREAT report was created for every patient in the TREAT condition, indicating that all clinicians attempted to use the application in their consultations. However, alternative indicators of intervention fidelity were absent. In other words, we do not fully understand how TREAT was utilized in this trial to facilitate aspects of SDM. Much of the content of clinical encounters in mental health remains a black box, making intervention fidelity and implementation of CDAs in daily clinical practice a tenuous endeavor.49
A final limitation of this study is that some participants had difficulty in filling in the DCS, because the formulation of the questions was sometimes perceived as complicated.
Future researchDifferent SDM interventions exist in mental healthcare, such as training clinicians in problem definition and agreement, or interventions aimed at patients for enhancing involvement and autonomy in the decisional process.20 SDM interventions in the form of CDAs are still scarce in mental healthcare23 and their implementation in regular clinical practice can be challenging.50 Currently available tools are generally used outside of clinical encounters,21 for example in the form of evidence-based booklets or online educational tools which patients use to prepare themselves for upcoming consultations.23 TREAT is unique in mental healthcare because it is used collaboratively during clinical encounters, aiming to shift a potential knowledge and decisional asymmetry between patient and clinician. Not all patients responded well to TREAT as some found it confrontational or complicated.14 Moreover, some patients in psychosis care experience reduced decisional ability due to their illness,51 limiting the potential benefits of a CDA. Future research should examine which patients benefit most from CDAs such as TREAT. Alternative patient outcomes such as empowerment or autonomy are also worth investigating in relation to CDAs.52 The systemic feedback of ROM results has the potential to improve clinician performance.13 TREATs feedback and treatment recommendations are used collaboratively by patient and clinician, therefore a second part of this trial will investigate its effects on clinical decision-making.
ConclusionsThis study investigated the effects of a clinical decision aid named TREAT on shared decision-making (SDM) during treatment plan consultations in psychosis care. We expected TREAT would improve SDM by increasing the exchange of information and by aligning decisions with patient's personal preferences, but no differences in SDM were detected between TREAT and TAU. In contrast to other patient populations in mental healthcare, participants experienced minimal decisional conflict. SDM and satisfaction rates were high in both conditions suggesting that TREAT did not depersonalize clinical encounters, disrupt existing workflows or decrease the quality of consultations. These findings suggests that decision aids such as TREAT can be safely implemented in psychosis care, but so far have not demonstrated to increase SDM. A second part of this trial will investigate if TREAT can benefit patients by improving decisional process in other ways during consultations.
Consent for publicationNot applicable.
Ethics approval and consent to participateParticipants signed an informed consent before participation in this study. All participants were ruled mentally competent to ethically consent to their participation by the participating clinicians. The procedures of this study were in accordance with the declaration of Helsinki. This study was approved by the Medical Ethics Committee of the University Medical Center Groningen.
Submission declarationThis work is not under consideration for publication elsewhere. All authors and participating institutions approve of this publication.
Formatting of funding sourcesThis study was funded by the (Dutch) ROOS foundation (Stichting ROOS). The ROOS foundation had no part in the study design, data collection, analysis or publication of the data.
Availability of data and materialsData will be made available upon reasonable request.
Author contributionsLR: conception and design of the study, organizing and conducting the trial and the data collection, wrote the draft article, data analysis and interpretation. JB: conception and design of the study, data analysis and interpretation, critical review of the manuscript. AB: conception and design of the study, data analysis and interpretation, critical review of the manuscript. PD: conception and design of the study, data analysis and interpretation, critical review of the manuscript. SC: conception and design of the study, data analysis and interpretation, critical review of the manuscript.
We would like to thank all the patients and clinicians and participating mental healthcare organizations Lentis, GGZ Drenthe, GGZ Friesland and the University Center for Psychiatry of the University Medical Center Groningen for their cooperation. We would like to thank Klaas Wardenaar for his statistical advice.