This project aims at developing an integral platform of digital support for mental health problems in PC: PRimary carE digital Support ToOl in mental health (PRESTO), which will allow to improve the quality of care and reduce waiting times through the prioritization of the most severe cases to be assessed timely in specialized care, while it assigns mild to moderate cases to one of these interventions: (1) Watchful waiting and periodic follow-up by the GP, (2) Symptoms monitoring and a brief evidence-based psychological intervention through an app (PRESTOapp), or (3) A personalized psychopharmacological treatment. To achieve this objective, first, ML-based predictive severity models will be developed taking into account clinical variables from electronic health records (EHR) of all referrals from GPs to the Primary Care mental health support programme (PCMHSP) during the last 5 years from all PCCs in Catalonia, including a total population of 7.5 million users. Simultaneously, the effectiveness of an app for monitoring and delivering evidence-based psychological interventions for mild to moderate anxiety-depressive symptoms (PRESTOapp) will be developed and evaluated in a feasibility study and a randomized clinical trial. Finally, the ML models and the PRESTOapp will be integrated into a decision support platform (PRESTO) that will triage and personalize the interventions for each case in a supervised manner. Yet, the platform will be dynamically updated, and the triage algorithm will be automatically re-trained with the results of the evolution and response to the treatment of each case included to improve its accuracy.

The PRESTO project will be structured in three phases, described in detail below, in order to finally integrate the different modules validated in each phase in the final objective of evaluating the PRESTO platform (Fig. 1). Phase 1 and 2 will be conducted simultaneously.

Fig. 1.

PRESTO platform development process phases. The figure shows the final PRESTO platform assembling process divided by phases. Phase 1 and Phase 2 will be conducted in parallel. EHR: Electronic Health records.

(0.22MB).

The regional shared EHR of all referrals from GPs to the PCMHSP during the last 5 years from all PCCs in Catalonia and a control sample of non-referred PC users will be independently retrieved and anonymized by the Data analytics programme for health research and innovation (PADRIS)38 from the Catalan Agency for Health Quality and Evaluation (AQuAS).39 Afterwards, the data will be analyzed in an aggregated manner.

Based on previous research of predictors of depression severity in patients from PC,40 the baseline data that will be prioritized in the initial model are the following variables: sex, age, education level, ethnicity, employment status, marital status, housing and living conditions, present or past medical and psychiatric co-morbidities, past-present drug use and pharmacological treatments, and recent life stressors. Additionally, any previous mental health care service provided such as emergency room or hospitalizations admissions registered in any public health care facility in Catalonia will be included to the model. Simultaneously the same variables from PC users who were not referred or required to the PCMHSP will be retrieved from public EHR to train the ML models.

Once all the data is retrieved and merged from the diverse sources aforementioned, the team (AS, JR) will use ML techniques to building, debugging, modelling, training and evaluating predictive variables to differentiate PC users that required specialized mental health care from those who didn’t, as well as predictive variables of severity. The severity (target feature) will be determined by the diagnosis by the PCMHSP of a severe mental disorder or not, time off work, the use of second- or third-line antidepressant treatments, prolonged follow-up, referral to the mental health centre, hospitalization in a psychiatric units or consultations in the psychiatric emergency departments.

Potential problems that can be assumed in predictable data cleansing and modelling include cases that left the monitoring by the PCMHSP, people who changed the basic health area sector, or cases that preferred to be monitored and treated in the private network. It will also be decided on the basis of the quality and granularity of the data available whether to proceed with a binary, categorical or cluster classification.

Based on the team's experience in the development and evaluation of mental health apps,31,32 the next steps will be followed:

a. Development of PRESTOapp 1.0: A User-centred development (UCD) approach will be adopted which has evidence of improved user retention and satisfaction with the end product.41,42

This phase will begin by conducting two online surveys: one directed to PC users who have been referred to PCMHSP about their interests and preferences on the use of a mental health app, and another survey directed to GPs and mental health professionals (psychiatrists, psychologists, specialized nurses, etc.) about the needs they think ought to be covered by a mental health app.

Subsequently, two focus groups, one with PC users/people who have been referred to PCMHSP and another with GPs and mental health professionals, with a minimum of 8 people each and a duration of 1.5hours approximately, will be performed in an online format. During the focus group, ideas, models, designs, characteristics and functionalities of the proposed mental health app, considering the results from the surveys, will be presented to find out the participants’ interest and opinions.

Based on the surveys and the data collected during the focus groups, the research team will outline the priority aspects of the development of the application as well as the contents of the evidence-based psychological interventions to be included.

The technical development of PRESTOapp will be carried out with the software development collaborating company through an iterative, collaborative, multidisciplinary process that preserves patient confidentiality at all times and will be monitored by the researchers as it will be displayed in real time at the collaborative working platform. This approach has been successfully implemented by the team in previous e-mental health projects.43

b. Evaluation of the feasibility of PRESTOapp 1.0: A minimum of 50 PC users will be recruited to participate in the feasibility study of PRESTOapp 1.0 app for 1 month. The inclusion criteria will take into account persons between 18 and 65 years of age who are referred to or are being followed up by the PCMHSP who are found to have mild-to-moderate depressive (PHQ-9: 5–14) and/or anxious (GAD-7: 5–14) symptoms,44,45 and have a compatible smartphone (i.e. Android or iPhone) and sufficient digital skills to download and use an app. The criteria for exclusion will be the previous or present diagnosis of a severe mental disorder, or a mild, moderate or severe risk of suicide. A baseline assessment will be conducted to collect sociodemographic (including occupation to identify health workers), medical and psychiatric history and evaluate depressive (PHQ-9) and anxious (GAD-7) symptoms. At a single follow-up visit after 1 month, depressive and anxious symptoms will be reassessed and acceptability will be explored with the Technology Acceptance Model (TAM) questionnaire,46 usability with the System usability scale (SUS),47 and satisfaction with the Health App Usability Questionnaire (MAUQ).48 Data on usage, retention and adherence with the app will be retrieved and analyzed directly from the servers.

c. Randomized controlled clinical trial with PRESTOapp 2.0: Based on participants’ feedback and suggestions from the feasibility study, PRESTOapp will be further improved to a 2.0 version. We will conduct a single-blind randomized controlled clinical trial offering their participation to all patients referred to our PCMHSP from our catchment area that includes 5 PCCs. The sample size calculation will contemplate as a primary outcome the reduction of depressive symptoms assessed by PHQ-9 taking into account two groups (PRESTOapp 2.0 vs. treatment as usual (TAU) in addition to previous results of effect sizes between 0.30 and 0.35 from similar studies.49,50 We have set for this purpose a power of 0.80 and a α of 0.05. Experience in similar studies indicates an expected 25–30% drop-out. Therefore, it was decided to add 15 more subjects per branch for preventive purposes and to ensure that at the end of the study there would be a sufficient sample to guarantee the strength of the data. Hence, 76 participants per branch will be recruited. Considering similar inclusion and exclusion criteria as the previous feasibility study, the intervention group will be asked to use the app for a period of 2 months. The control group will receive the usual follow-up and treatment during the same time by the PCMHSP team. Participants from both groups will be asked to participate in an initial evaluation and a follow-up in 2 months, unless the app indicates or symptoms that require urgent care arise. Both interviews will include the collection of sociodemographic and clinical variables alongside standardized scales PHQ-9, GAD-7, Holmes and Rahe Stress Scale.51

Statistical analyses will be conducted using specific R packages. The analyses of all the subjects included will be considered until the dropout or end of the study. The main variable is the change in symptoms measured by PHQ-9, GAD-7 during the 2 months, controlled by Holmes and Rahe Stress Scale. We will use a mixed effects linear model with random interception for each participant. The differences in the primary measures will be analyzed first in an unadjusted manner and then adjusted for sociodemographic and clinical factors. An analysis by intention to treat (ITT) with last observation carried forward (LOCF) will also be employed. A threshold of statistical significance p≤0.05 (two-tailed) will be set.

Among the limitations of this study, it is worth mentioning that researcher and patients blinding will not be possible for logistical and ethical reasons, nor is there a placebo in the case of the control group. The main reason for this design is that the methodological effects of providing a placebo app to the control group are unclear. Assumable risks are the placebo and Hawthorne effect in the intervention group, natural progression and regression to the mean. Digital literacy will likely be widely variable among the included sample. This may hinder the use of the app in some of the included subjects. To minimize bias in measuring engagement, inclusion criteria in phase 2 takes into account having sufficient digital skills to download and use an app. Moreover, the design of the app's user-interface will be clean, user-friendly, and include preliminary instructions of use.

Health workers are one of the most affected populations regarding mental health due to the COVID-19 pandemic. Beside from anxiety and depression symptoms, they may suffer from posttraumatic stress disorder (PTSD) symptoms.9 Moreover, their medical literacy is usually higher than the general population. To avoid selection bias, health workers included in the study will be identified and a sensitivity analysis performed. It is worth noting that a version of the PRESTOapp adapted to specific situations and particularities of health workers – in particular on the detection and management of PTSD symptoms and the medical language used – is under current development in a parallel project.

a. Calculation of predictive models of response to each assigned intervention based on ML: development and integration in the PRESTO platform: From the ML predictive models built in phase 1 and the data collected on the evolution of patients with continuous monitoring through the app in phase 2, the PRESTO platform will be integrated and developed in a web interface.

The main aim of this phase is to test the PRESTO platform, a dynamic triage system for all patients who are referred to PCMHSP by GPs that will be assigned, in a supervised manner, the most appropriate and personalized approach to each case. However, beyond the friendly web environment that will host the platform, the background algorithms will progressively be fed to increase with outcomes and follow-up data derived from face-to-face visits as well as those collected by the PRESTOapp. This will allow a two-way loop flow of information that with a machine learning approach will “learn in a supervised manner” and will allow to predict with increasing precision the evolutionary course of the patients and the best type of the approach that will be required (Fig. 2).

Fig. 2.

Final PRESTO platform framework and dataflow prototype. The picture shows a diagram of the final PRESTO platform prototype 1.56 alongside their dataflow channels allowing to created loop of self-learning data from data collecting during the follow-up and treatment assigned to each specific case.

(0.1MB).

b. Stepped-wedge cluster randomized trial to assess effectiveness in reducing waiting lists: The investigation of PRESTO effectiveness in reducing waiting times will be carried out by means of a pragmatic design study without distorting or interrupting the health care of service users as well as minimizing its risks. The design will therefore be of a Stepped-wedge cluster randomized trial, in which, for predetermined periods of 2 months, the platform will be implemented randomly to clusters of referrals to PCMHSP of each of the 5 PCCs from our catchment area. The rest of the time the current care method will be used as a control. All users will be informed of the system being used through the PRESTO platform and they must accept and sign the informed consent for study participation. Otherwise, they be provided with the current healthcare approach. The main variable for assessing efficiency will be the reduction in waiting time of a first contact with the PCMHSP of GP referrals, which will be calculated for each PCC.

Regarding statistical analyses, the waiting times will be compared with other centres and also with the average of each period of the historical records of the last 5 years. In order to minimize selection and unbalance bias, individual and cluster characteristics will be compared among groups beforehand and appropriate corrections will be conducted. A generalized mixed-effects model will be adopted. This model will make it possible to determine the efficiency of the platform by being able to adjust it by age, sex, seasonal patterns, as well as the characteristics and care burden of each PCC (population covered, socioeconomic situation, number of individuals per GP). A randomized slope model for treatment will also be considered given the variability of intervention by each centre following the recommendations of Davey et al.52 As a sensitivity analysis, the use of a mixed-effects linear Gaussian model for the different variances within each cluster will be adopted. Calculations will be made using specific R packages. In order to provide an extensive use of the PRESTO platform among GPs, and avoid a restricted use due to limited digital literacy, the user-interface will be clean, user-friendly and intuitive.

The PRESTO project has been approved by the Clinical Research Ethics Committees (CEIC) of the Hospital Clinic of Barcelona (HCB/2020/0735). All the data will be collected by the project researchers, and stored in encrypted and secure servers following the guidelines and standards of the 2018 European General Data Protection Regulation (GDPR). The project protocols as described in this manuscript were registered at Clinicaltrials.gov (NCT04559360) in September 2020.

At the time of writing this protocol, the process of anonymized data extraction from EHR are already underway with the collaboration of the PADRIS programme from the AQuAS agency, as planned in Phase 1. In parallel, we have finished the exploratory surveys about the attitudes, uses and preferences of a digital mental health platform from both PC users and health professionals.

The participation in PC users survey was offered at the centres waiting rooms with cards and posters including a QR code, while the invitation to complete the health professionals survey was distributed by internal mailing list containing a link. The link to the surveys were accessible from June to November 2020. Both surveys were composed of 47 sequential, multi-choice and non-mandatory questions.

One hundred and sixty-two PC users and 72 health professionals took part in the survey. The sociodemographic and clinical characteristics of the respondents are described in Table 1. About 1 in 5 (22%) of PC users had previously been treated for mental health problems, most of them due to “anxiety” (51.6%), “depression” and “sleep problems” (35.5% both).

The ownership and use of digital devices of PC users are reported in Table 1, most of the them reported owning a smartphone (91.2%) with an Android operating system (66.1%), and 54.1% reported having been using a smartphone for more than 10 years. Fig. 3 details the most common uses of their smartphone informed by the participants.

Fig. 3.

Most Common Smartphone Daily Use Purposes reported by service users. The figure shows the percentage of the most common smartphone daily use purposes reported by service users.

(0.07MB).

The use of smartphone applications (app) for mental health reasons and the potential use of a clinically validated mental health app are detailed in Table 2. Almost one-quarter of PC users (23.4%) reported having used a mental health app and an important percentage of health professionals (32.8%) found out that their patients used or were already using mental health apps. The most common type of mental health apps used were for meditation/mindfulness, to reduce stress and improve sleep quality. As a result, more than half of PC users (56.4%) and 82% of health professionals were interested on having available a clinically-validated mental health app for use or to prescribe to their patients with the purposes and features detailed in Fig. 4.

Fig. 4.

PRESTOapp preferred purposes and features. Potential preferred purposes and features of a smartphone app for mental health problems. Interests from primary care users (left) and health professionals (right).

(0.06MB).

Overall, PC users were predominantly interested in symptoms monitoring (46%) and receiving psychological contents (54%) through a smartphone app preferably only when they needed (on-demand) (57.3%) during 1 to 10min per day (58.2%), 1 to 3 times per week (81%), over a period from 1 to 6 months (58.2%). Most of the participants preferred to share with the health care team the information collected by the app only when they decided it (51.8%), by e-mail (39.1%) and alert the mental health specialist if worsening (55.5%). Most of the respondents (59.5%) were extremely or very worried about the confidentiality and safety of the potential data collected by the app, stating that they would feel safer about it if the app clarifies beforehand who might have access to their data (68%), where and how it is stored, and for which aims it might be used (60%). The main drawback identified to use the app was an excessive amount of information (51.5%).

Health professionals that responded the survey, mostly psychiatrists and GPs, regularly cared for people suffering from depression and anxiety problems (more than 80% of patient's attended). The preferences of a clinically-validated mental health app were very similar to those from PC users (Table 2). The greatest discrepancies between them were that health professionals were more interested in apps including psychoeducation modules (52 vs 29%) whereas PC users showed more interest in meditation/mindfulness modules (43 vs 24%). Also, compared to PC users, only few health professionals considered that an app would be useful for diagnostic (6 vs 37%) or treatment orientation purposes (10 vs 28%). Finally, health professionals considered that self-evaluations should be performed more times per week as compared to patients’ preferences.

Based on all the feedback received with both surveys, we have drafted the first mock-ups and prototypes of the potential mental health app to present and discuss at the focus groups which are scheduled to be held in February 2021 with a modified online approach due to the covid-19 crisis.