metricas
covid
Buscar en
Revista de Psiquiatría y Salud Mental
Toda la web
Inicio Revista de Psiquiatría y Salud Mental Digital mental health interventions for anxiety and depressive symptoms in unive...
Información de la revista
Vol. 16. Núm. S1.
Psycho-COVID: Long-term effects of COVID19 pandemic on brain and mental health
Páginas 47-58 (enero - marzo 2023)
Compartir
Compartir
Descargar PDF
Más opciones de artículo
Visitas
1284
Vol. 16. Núm. S1.
Psycho-COVID: Long-term effects of COVID19 pandemic on brain and mental health
Páginas 47-58 (enero - marzo 2023)
Acceso a texto completo
Digital mental health interventions for anxiety and depressive symptoms in university students during the COVID-19 pandemic: A systematic review of randomized controlled trials
Intervenciones digitales de salud mental para los síntomas ansiosos y depresivos en estudiantes universitarios durante la pandemia de COVID-19: una revisión sistemática de pruebas controladas aleatorizadas
Visitas
1284
Ilaria Riboldia,
Autor para correspondencia
i.riboldi1@campus.unimib.it

Corresponding author.
, Daniele Cavaleria, Angela Calabresea, Chiara Alessandra Capogrossoa, Susanna Piacentia, Francesco Bartolia, Cristina Crocamoa, Giuseppe Carràa,b
a Department of Medicine and Surgery, University of Milano-Bicocca, via Cadore 48, 20900 Monza, Italy
b Division of Psychiatry, University College London, Maple House 149, London W1T 7BN, United Kingdom
Este artículo ha recibido
Información del artículo
Resumen
Texto completo
Bibliografía
Descargar PDF
Estadísticas
Figuras (1)
Tablas (3)
Table 1. Characteristics of the included studies.
Table 2. Summary of findings of digital interventions supporting higher-education students’ mental health.
Table 3. Quality assessment of the included studies by the Randomized Controlled Trial of Psychotherapy Quality Rating Scale.
Mostrar másMostrar menos
Material adicional (1)
Suplemento especial
Este artículo forma parte de:
Vol. 16. Núm S1

Psycho-COVID: Long-term effects of COVID19 pandemic on brain and mental health

Más datos
Abstract

University students are particularly vulnerable to mental health issues, with anxiety and depression identified as the most common conditions. During the COVID-19 pandemic, social distancing, self-isolation, and difficulties linked to online teaching and learning have increased their burden of anxiety and depressive symptoms. Thus, the urgent need to intervene in favour of these vulnerable subjects, together with the difficulties in delivering in-person interventions because of lockdowns and restrictions, has led to prioritize digital mental health strategies. This study aimed at systematically reviewing the existing literature on digital mental health interventions targeting anxiety and depressive symptoms in university students during the COVID-19 emergency. Systematic searches of Medline, Embase, and PsycInfo databases identified eight randomized controlled trials. Regarding anxiety symptoms, digitally delivered cognitive behavioural therapy, dialectical behaviour therapy, and mind-body practice techniques emerged as valid strategies, while digital positive psychology and mindfulness-based interventions showed mixed results. On the other hand, digitally delivered dialectical behaviour therapy and positive psychology interventions have shown some efficacy in reducing depressive symptoms. Overall, the available literature, albeit of low quality, seems to support the role of digital interventions in promoting the mental health of university students during the COVID-19 pandemic.

Keywords:
Anxiety
COVID-19
Depression
Digital mental health interventions
Mental health
University students
Resumen

Los estudiantes universitarios son particularmente vulnerables a los problemas de salud mental, siendo la ansiedad y la depresión las condiciones más comunes. Durante la pandemia de COVID-19, el distanciamiento social, el autoaislamiento y las dificultades relacionadas con la enseñanza y el aprendizaje en línea han aumentado su carga de ansiedad y síntomas depresivos. Así, la urgente necesidad de intervenir a favor de estos sujetos vulnerables, junto con las dificultades para realizar intervenciones presenciales a causa de los confinamientos y restricciones, ha llevado a priorizar estrategias de salud mental digital. Este estudio tuvo como objetivo revisar sistemáticamente la literatura existente sobre intervenciones digitales de salud mental dirigidas a la ansiedad y los síntomas depresivos en estudiantes universitarios durante la emergencia de COVID-19. Las búsquedas sistemáticas en las bases de datos Medline, Embase y PsycInfo identificaron ocho ensayos controlados aleatorios. Con respecto a los síntomas de ansiedad, la terapia cognitiva conductual, la terapia conductual dialéctica y las técnicas de práctica mente-cuerpo entregadas digitalmente surgieron como estrategias válidas, mientras que la psicología positiva digital y las intervenciones basadas en el mindfulness mostraron resultados mixtos. Por otro lado, la terapia conductual dialéctica y las intervenciones de psicología positiva proporcionadas digitalmente han demostrado cierta eficacia en la reducción de los síntomas depresivos. En general, la literatura disponible, aunque de baja calidad, parece respaldar el papel de las intervenciones digitales en la promoción de la salud mental de los estudiantes universitarios durante la pandemia de COVID-19.

Palabras clave:
Ansiedad
COVID-19
Depresión
Estudiantes universitarios
Intervenciones digitales de salud mental
Salud mental
Texto completo
Introduction

Several challenges expose higher education students to increased vulnerability to psychological distress and mental illness, especially during the first years of academic courses.1,2 The transition to university often leads to increased social, performance, and financial demands, as well as to major changes in lifestyle habits.3 A rising trend for psychological distress and mental health issues among university students, in terms of both incidence and severity, had been already highlighted in the past decades.4 Amongst these, anxiety and depressive features were identified as the most common, often turning into mental health burden.4,5 Notwithstanding the efforts made by educational institutions to deploy more structured interventions and programmes for common mental disorders among students,6,7 the relevant delivery is often complicated by their reluctance to access university counselling and supporting services, mainly because of barriers such as stigma or lack of adequate information.8,9

However, during the last two years, social distancing and self-isolation attributable to the COVID-19 pandemic, together with the difficulties linked to online teaching and learning (at best blended with face-to-face support), have further amplified the psychological burden among higher-education students.10–12 Unfortunately, the prevalence of anxiety and depression in this population during the COVID-19 emergency has increased in comparison to pre-pandemic levels, with about one in three university students reporting anxiety and depressive symptoms.10 Thus, the urgency to provide them with adequate, personalized, evidence-based psychological support amid these difficult times has been advocated.13

However, in the previous few years, many digital mental health interventions had been developed and implemented for special populations such as youths and students with the aim of improving mental health care access.14 These are based not only on the online delivery of conventional psychotherapies – e.g., cognitive behavioural (CBT) and dialectical behaviour (DBT) therapies, or mindfulness-based interventions6,15,16 – but also on specific digital tools, including video-based and mobile health (mHealth) approaches,17,18 and have been increasingly acknowledged as convenient, accessible, easy to use, and helpful options.19 A systematic review and meta-analysis run before the COVID-19 pandemic suggested that web-based and computer-delivered interventions might be effective, compared to inactive controls, in improving depression and anxiety of university students.20 Considering its efficacy and feasibility, the digitalization of interventions has been rapidly identified by educational institutions as suitable to overcome the many barriers encountered in the delivery of traditional, in-person interventions for higher-education students during periods of lockdowns and restrictions.21 Consistently, an increasing number of studies on digital interventions targeting the mental health of university students during the COVID-19 pandemic has been carried out.22 However, the available evidence on the efficacy of such interventions appears fragmented and has not been synthesized to date. To fill this gap, we performed a systematic review of randomized controlled trials (RCTs) testing digital mental health interventions for anxiety and depression in university students during the COVID-19 global emergency, both synthesizing the available evidence on their efficacy and assessing its quality.

Material and methods

This systematic review was carried out following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) 2020 Statement.23 A protocol was registered in Open Science Framework Registries on 12 January 2022 (https://doi.org/10.17605/OSF.IO/7UNV5).

Search strategy and inclusion criteria

Systematic searches of Embase, Ovid MEDLINE, and APA PsycInfo databases (via Ovid) were performed on 5 January 2022. The search phrase used was: “(student* or university or varsity or college or tertiary or higher or campus or undergrad* or postgrad* or freshm*) AND (psychotherap* or mindfulness or cognitive or CBT or dialectic* or DBT or psychoeducation or intervention*) AND (random* or controlled or RCT) AND (mental or psychiatr* or psychol* or depress* or anxi* or stress or resilien* or coping or wellbeing or well-being) AND (coronavirus or SARS-COV-2 or COVID-19 or nCoV or pandemic).mp” as a multiple purpose search of title, abstract, heading words, and keywords. Since the COVID-19 outbreak was declared a Public Health Emergency of International Concern by the World Health Organization on 30 January 2020,24 the search was restricted to articles published from 2020 onwards. No language restrictions were applied.

We included RCTs comparing any type of digital mental health interventions, i.e., interventions based on digital technology for mental health support, prevention, and treatment,21 and an appropriate control condition (e.g., wait-lists or psychoeducational/behavioural interventions) for anxiety and depression in individuals in higher education during the COVID-19 outbreak. Studies were considered eligible if they involved students enrolled in university or college courses. Both undergraduate and postgraduate students (e.g., doctoral or master students) were included. To be eligible, RCTs had to report measures of efficacy on anxiety and/or depressive symptoms, as estimated by the mean change difference between index and control interventions. To improve the consistency and comparability of data, we excluded: (i) studies with a non-randomized and/or non-controlled design; (ii) studies testing interventions which addressed other outcomes, such as stress, quality of life, or wellbeing; (iii) studies examining in-person interventions; (iv) “grey” literature, conference abstracts, dissertations, and all publications not having undergone a peer-review process.

After a preliminary screening based on titles and abstracts, full texts were retrieved to evaluate eligibility. Articles were independently screened and read in full text by four authors (IR, AC, CAC, SP), and reasons for exclusion were recorded (Table A.1). Any disagreement was resolved by discussion with the other authors.

Data extraction

Four authors (IR, AC, CAC, SP) independently extracted data and blindly cross-checked them for accuracy. A data extraction template was used to collect key information from the eligible studies, including author(s) and year of publication, country, setting, study design, main characteristics of experimental and control groups, type of interventions and comparators, tools used, outcomes assessed, and related measures. We extracted information on effect size measures, evaluating the magnitude of the effects according to standard cut-offs.25,26

Quality assessment of the included studies

A formal quality assessment of the studies included in the systematic review was performed using the Randomized Controlled Trial of Psychotherapy Quality Rating Scale (RCT-PQRS), a 25-item validated tool, specifically designed to assess the quality of psychotherapy RCTs.27,28 Items #1 through #24 refer to several domains including the description of subjects, treatment/intervention definition and assignment, and outcome measures, as well as data analysis and related reporting procedures with a score of 0, 1, or 2 for each. In addition, Item #25 represents an omnibus rating of the quality of the entire study ranging from 1 (“exceptionally poor”) to 7 (“exceptionally good”). A total score of 24 or more for the first 24 items (corresponding to an average value of 1 on each scale item) is assumed to represent the minimum adequate total quality-of-study score.27

The included studies were independently evaluated by two authors (IR, CAC), and disagreements were resolved by discussion involving all authors.

ResultsStudy selection

As a whole, our systematic search generated 1411 records (1138 from Embase, 218 from Ovid MEDLINE, and 55 from APA PsycInfo), reduced to 1204 unique articles after deduplication. After the screening of titles and abstracts, 25 articles were retrieved and examined in full text. Finally, eight articles29–36 were found to be eligible. The study selection process is fully described in Fig. 1.

Figure 1.

Flowchart of the systematic review according to PRISMA criteria.20

(0.43MB).
Study characteristics

The eight studies included a total of 1403 participants. Three main clusters of digital interventions administered to study participants were identified: CBT-/DBT-based interventions33,34; mindfulness-based interventions32,35,36; other types of interventions, i.e., positive psychology interventions (PPIs)30,31 and mind-body interventions.29 Control conditions mostly consisted of wait-lists and behavioural/psychoeducational interventions. All studies assessed the efficacy of digital interventions on anxiety symptoms, while five31–33,35,36 also on depressive symptoms. The duration of interventions ranged from two to eight weeks. Main study characteristics are reported in Table 1. Detailed descriptions of the various interventions are available in Table A.2.

Table 1.

Characteristics of the included studies.

Study  Country  SubjectsEducational setting  Pandemic restrictions  EG intervention  CG intervention  Outcomes  Assessment tools 
    N tot  N EG  N CG  Female gender (%)             
Cozzolino et al., 2021  Italy  266  144  122  78%a  University students from different educational levels  National lockdown (March–April 2020)  Mind-body practice named “Brain Wave Modulation Technique”  Online psychoeducational intervention  Anxiety  STAI-Y 
Datu et al., 2021  Hong Kong  107  69  38  72%  Undergraduate psychology students  N/R  Positive psychology intervention  Online behavioural intervention  Anxiety  CAS 
Krifa et al., 2021  Tunisia  161  88  73  94%  1st–3rd year healthcare university students  Restrictions and quarantine (Spring 2020)  Positive psychology intervention  Wait-list  AnxietyDepression  DASS-21 
Lahtinen et al., 2021  Finland  406  195  211  N/R  University students and staff  N/R  Mindfulness-based intervention  Online psychoeducational intervention  AnxietyDepression  GAD-7BDI-21 
Liang et al., 2021  China  52  26  26  62%  Medical students  June-August 2020, when medical students were still at home  Dialectical behaviour therapy  Online psychoeducational intervention  AnxietyDepression  GAD-7PHQ-9 
Shabahang et al., 2021  Iran  150  75  75  51%  Undergraduate- and graduate-level students  March–April 2020, during the second wave of COVID-19  Cognitive behavioural therapy  Wait-list  Anxiety  CVAQSHAIASI-3 
Simonsson et al., 2021  United Kingdom  162  77  85  70%b  Undergraduate- and graduate-level students  Third national lockdown (January–March 2021)  Mindfulness-based intervention  Wait-list  AnxietyDepression  PROMIS 
Sun et al., 2021  China  99  52  47  74%c  Undergraduate- and graduate-level students  March–April 2020 (different restrictions across 27 provinces in China)  Mindfulness-based intervention  Online social support  AnxietyDepression  GAD-7PHQ-9 

CG=control group; EG=experimental group; N=number of subjects; N/R=not reported; ASI-3=Anxiety Sensitivity Index-3; BDI-21=Beck Depression Inventory-21; CAS=COVID-19 Anxiety Scale; CVAQ=COVID-19 Anxiety Questionnaire; DASS-21=Depression, Anxiety and Stress Scales; GAD-7=Generalized Anxiety Disorder-7; PHQ-9=Patient Health Questionnaire-9; PROMIS=Patient-Reported Outcome Measurement Information System; SHAI=Short Health Anxiety Inventory; STAI-Y=State-Trait Anxiety Inventory, form Y.

a

Data from the original eligible sample (N=310).

b

Data from the original eligible sample (N=177; 4/177 subjects preferred not to say; 2/177 non-binary; 1/177 genderfluid).

c

Data from the original eligible sample (N=114).

Study findings

Table 2 shows a summary of findings of the effects of digital mental health interventions designed to support higher-education students’ mental health on relevant outcomes.

Table 2.

Summary of findings of digital interventions supporting higher-education students’ mental health.

Study  AnxietyDepression
  Results  Effect measures  Results  Effect measures 
Cozzolino et al., 2021  Greater reduction in STAI-Y scores in the EG compared to the CG.  Trait anxiety: η2=0.03; p<0.01State anxiety: η2=0.08; p<0.001  N/A  N/A 
Datu et al., 2021  No differences in CAS scores between the EG and the CG.  ηp2=0.003; p=0.88  N/A  N/A 
Krifa et al., 2021  Greater reduction in DASS-21 anxiety scores in the EG compared to the CG.  d=0.21; p<0.001η2=0.02  Greater reduction in DASS-21 depression scores in the EG compared to the CG.  d=0.29; p<0.001η2=0.05 
Lahtinen et al., 2021  No differences in GAD-7 scores between the EG and the CG.  N/R  Greater reduction in BDI-21 scores in the EG compared to the CG.  d=0.45 in EG vs. d=0.29 in CGPost-test: p=0.11 
Liang et al., 2021  Greater reduction in GAD-7 scores in the EG compared to the CG.  N/R; p<0.001  Greater reduction in PHQ-9 scores in the EG compared to the CG.  N/R; p=0.014 
Shabahang et al., 2021  Greater reductions in CVAQ, SHAI, and ASI-3 scores in the EG compared to the CG.  COVID-19 anxiety: ηp2=0.49; p=0.001Health anxiety: ηp2=0.23; p=0.001Anxiety sensitivity: ηp2=0.22; p=0.001  N/A  N/A 
Simonsson et al., 2021  Greater reduction in PROMIS anxiety scores at the end of the intervention and at the 1-month FU in the EG compared to the CG.  Post-test: d=−0.39; p=0.025(FU: d=−0.20No differences in PROMIS depression scores between the EG and the CG at the end of the intervention period, nor at the 1-month FU.  Post-test: d=−0.26; p=0.35(FU: d=–0.06
Sun et al., 2021  Greater reduction in GAD-7 scores in the EG compared to the CG.  d=0.72; p=0.024  No differences in PHQ-9 scores between the EG and the CG.  d=0.36; p=0.43 

CG=control group; EG=experimental group; d=Cohen's d; FU=follow-up; p=p-value; N/A=not assessed; N/R=not reported; η2=eta-squared; ηp2=partial eta-squared; ASI-3=Anxiety Sensitivity Index-3; BDI-21=Beck Depression Inventory-21; CAS=COVID-19 Anxiety Scale; CVAQ=COVID-19 Anxiety Questionnaire; DASS-21=Depression, Anxiety and Stress Scales; GAD-7=Generalized Anxiety Disorder-7; PHQ-9=Patient Health Questionnaire-9; PROMIS=Patient-Reported Outcome Measurement Information System; SHAI=Short Health Anxiety Inventory; STAI-Y=State-Trait Anxiety Inventory, form Y.

p-Value not reported.

CBT-/DBT-based digital interventions

Two studies tested CBT-/DBT-based digital interventions. Shabahang and colleagues investigated the efficacy of video-based CBT, including video clips and online booklets, in a sample of 150 students, divided in two equal-size groups.34 Different specific domains of anxiety were evaluated, including COVID-19 anxiety (assessed through the COVID-19 Anxiety Questionnaire – CVAS),37 health anxiety (i.e., anxiety related to the likelihood and severity of having an illness; Short Health Anxiety Inventory – SHAI38), and anxiety sensitivity (i.e., concern associated with negative consequences of anxiety-related symptoms; Anxiety Sensitivity Index-3 – ASI-339). The authors estimated a small-to-moderate effect of video-based CBT on all anxiety-related outcomes, i.e., COVID-19 anxiety (ηp2=0.49; p<0.01), health anxiety (ηp2=0.23; p<0.01), and anxiety sensitivity (ηp2=0.22; p<0.01), as compared with the wait-list control condition.

Liang and colleagues33 administered an online DBT intervention, which included exercises aimed at improving emotion regulations skills and techniques of pain tolerance, to an experimental group of 26 medical students. This intervention was compared with an online psychoeducational programme administered to an equal-size sample. The authors uncovered that the Generalized Anxiety Disorder-7 (GAD-7)40 score reductions were greater in the DBT experimental group (p<0.001). This intervention was also associated with a greater decrease in depressive symptoms assessed by the Patient Health Questionnaire-9 (PHQ-9),41 as compared with the control group, with a significant Time X Group interaction (p=0.01).

Mindfulness-based digital interventions

Three studies32,35,36 explored the use of mindfulness-based digital interventions in higher education students. Lahtinen and colleagues32 tested a mindfulness programme carried out using a mobile app among university students and staff in Finland. The 4-week intervention trained 195 participants to recognize bodily signs of psychological stress and to manage it. The control condition group was involved in an online psychoeducational intervention based on mini-lectures on mindfulness practice. The authors showed that, despite a progressive decrease in anxiety symptoms during the intervention period, GAD-7 scale scores did not differ between the experimental and the control group at trial endpoint. Moreover, they estimated a greater reduction in depressive symptoms42 in the intervention group, even though the post-test between-group difference was not statistically significant (p=0.11).

A different kind of online mindfulness programme,35 developed through an eight-week course, was carried out in the United Kingdom within a group of undergraduate- and graduate-level students, in comparison to a wait-list control group. The authors showed larger reductions in anxiety, as measured by the Patient-Reported Outcome Measurement Information System Scale (PROMIS)43 (d=−0.39), with a significant Time X Group interaction (p=0.025). This was also true at 1-month follow-up (d=−0.20). Otherwise, no significant differences in the reduction of depressive symptoms on the same scale were found between the two groups (d=−0.26; p=0.35), both immediately after the end of the experiment and at 1-month follow-up.

Finally, Sun and colleagues36 measured the efficacy of a mindfulness-based mHealth intervention in improving anxiety and depressive symptoms in 52 university students as compared with 47 students who were offered online social support. The authors reported a greater improvement in anxiety symptoms, as measured by GAD-7, in the experimental group, with a medium-to-large effect size (d=0.72; p=0.024), as well as a significant Time X Group interaction when considering the evaluations done at subsequent follow-ups (p=0.024). Conversely, no differences between index and control interventions in depressive symptom improvement, as measured by PHQ-9 Scale, were estimated (p=0.43).

Other interventions

Three studies29–31 tested other digital interventions for anxiety and depressive symptoms. Two distinct PPIs were examined.30,31 Datu and colleagues30 introduced an intervention based on weekly tasks of gratitude and kindness to 69 psychology students. The comparison condition, an online behavioural intervention, was tested in a sample of 38 students from the same degree programme. No differences between the experimental and control groups were found in anxiety symptoms at study endpoint, as measured by the Coronavirus Anxiety Scale (CAS)44 (ηp2=0.003; p=0.88).

Krifa et al.31 proposed an online PPI named “CARE program”, which was based on the reorientation of attention towards the positive and satisfying aspects of life and on the engagement in meaningful action. Each online session consisted of lectures, videos of experts, psychoeducation, positive psychology practices, and one or two weekly activities, which 88 healthcare students were asked to complete. This study revealed a small improvement of both anxiety (d=0.21; p<0.001) and depressive symptoms (d=0.29, p<0.001) as measured by the Depression Anxiety Stress Scale-21 (DASS-21)45 in subjects receiving the index intervention as compared with the wait-list control group.

Finally, Cozzolino et al.29 explored a new mind-body practice named brain wave modulation technique (BWM-T) in a pool of 144 students from an Italian university. This method consisted of an easy-to-implement 4-step finger movement procedure, while the control condition involved a video psychoeducational intervention that suggested 122 students how to reduce stress. The authors found a higher reduction of both trait-anxiety (η2=0.03; p<0.01) and state-anxiety (η2=0.08; p<0.001), measured by the State-Trait Anxiety Inventory (STAI-Y),46 in the experimental group as compared with the control group.

Quality assessment

The quality assessment of the included studies—performed using the RCT-PQRS—resulted in scores ranging from 13 to 29 (mean=19.0; SD=5.55), indicating that, on average, according to the cut-off of 24 points, trials failed to achieve a minimum adequate quality.27 Definition and delivery of treatment (i.e., the method to demonstrate that the treatment which is being studied is the treatment which is actually being delivered) and outcome measures (i.e., outcome assessment by raters blind to treatment group and with established reliability) were among the domains rated with the lowest score. The complete quality assessment is reported in Table 3.

Table 3.

Quality assessment of the included studies by the Randomized Controlled Trial of Psychotherapy Quality Rating Scale.

Author(s), year  RCT-PQRS items
  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  Totala  25b 
Cozzolino et al., 2021  18 
Datu et al., 2021  13 
Krifa et al., 2021  17 
Lahtinen et al., 2021  22 
Liang et al., 2021  15 
Shabahang et al., 2021  14 
Simonsson et al., 2021  24 
Sun et al., 2021  29 

RCT-PQRS=Randomized Controlled Trial of Psychotherapy Quality Rating Scale.24

a

The total quality-of-study score is obtained by summing items from #1 to #24 and ranges from 0 to 48. A total quality-of-study score equal or greater than 24 corresponds to the minimum adequate study quality.23

b

The omnibus score, which coincides with item #25, ranges from 1 (exceptionally poor study) to 7 (exceptionally good study).

Discussion

To our knowledge, this is the first systematic review aimed at synthesizing the existing evidence from RCTs on digital interventions designed to support higher-education students’ mental health during the COVID-19 pandemic. We comprehensively evaluated the efficacy of these interventions on two major outcomes, i.e., anxiety and depressive symptoms. With regard to anxiety, CBT, DBT, and mind-body practice techniques emerged as potentially effective strategies, while PPIs and mindfulness-based programmes showed mixed results, faring better than wait-list control conditions or unstructured online social support but not being superior to online behavioural/psychoeducational interventions in reducing anxiety symptoms. In addition, as far as depressive symptoms are concerned, DBT proved to be effective in reducing them in comparison to psychoeducation, while PPIs emerged as superior to wait-list condition to manage depression. Mindfulness-based interventions did not show any effects.

Overall, notwithstanding the small-to-medium magnitude of the effects and the average low quality of the included RCTs according to the RCT-PQRS, the available literature seems to support the promising role of different types of digital interventions and their possible benefits on crucial mental health outcomes for university students during the COVID-19 public health emergency.

CBT, DBT, and mindfulness-based techniques had previously proven to be effective options to reduce depressive and anxiety symptoms in university students, even independently of the pandemic.47–49 Similarly, PPIs50 and mind-body interventions51,52 had been formerly tested in university settings, even if the existing evidence remains scarce. Clarifying their potential role as beneficial options for higher education students’ mental health is even more necessary during the COVID-19 pandemic and related waves. Telemental health services, which have been increasingly proposed in the past two years, hold several advantages over traditional, in-person, counselling, being easily provided, instantly accessible to several subjects at the same time, affordable, and highly customisable53: all these advantages seem particularly beneficial for university students.19,54 Digital approaches for mental health may play a key role in supporting them by delivering remote mental health interventions when social contacts are limited due to COVID-19-related restrictions.55

The COVID-19 pandemic, determining lifestyle changes, poorer social relationships, and impoverishment of healthy behaviours,56 is likely to have a major impact on university students also over the coming months. One of the first measures universally adopted in all countries amid COVID-19 pandemic was a transition to remote care via phone, chat, or video calls,57 and the need for online services for mental well-being is anticipated to persist for several years. Findings of this systematic review could thus be useful to outline future directions on multiple levels, considering the peculiar needs of this special population. University students, whose traditional displacement from familiar environments has been made even more confused during the pandemic, through cyclic lockdowns and social distancing measures enforcements, have faced disrupted schooling and inequalities exacerbated by online education,58 as well as severe psychosocial distress due to isolation from peers, lack of interactivity with instructors, and difficulty adjusting to family needs.59 The implementation of effective telemental health interventions could thus empower university students and enhance their engagement. Key elements of digital interventions in terms of mental health promotion include increasing awareness, overcoming stigma related to help-seeking, and favouring adherence to psychotherapy, ultimately improving outcomes in university students.8,60–62 On the other hand, universities should be encouraged to include within their policies for mental health and wellbeing support, also digital interventions as promising and feasible opportunities in emergency times such as the ongoing pandemic.63,64 They should consider routinely providing students with adequate psychological support, based on interactive therapeutic interventions.65–67 In addition, social media platforms, albeit amenable of an excessive and uncontrolled use,68 are often deemed appropriate by higher-education students for self-disclosure or help-seeking. Therefore, relevant strategies should include social media-based mental health support tools as essential surveillance approaches, taking into account privacy and boundary regulation mechanisms to sustain healthy behaviours.69–71

Also, the current favourable conditions for the design of digital strategies72 and the related promising role of recently developed digital mental health interventions, both suggest the importance of making digital approaches more available in routine care and not only in response to the COVID-19 crisis.73

The findings of this systematic review should be interpreted with caution considering some limitations. First, heterogeneity issues across studies in terms of design, primary sampling units and related variations as of local COVID-19 lockdowns and restrictions measures, characteristics and length of interventions, assessment tools, outcome definition, and effect size measures, all must be considered when approaching this piece of work. This also precluded the possibility of a quantitative, meta-analytical, synthesis of the available data. Second, assessment via the RCT-PQRS showed on average moderately poor quality for the included studies: the lack of blinding, which is an intrinsic characteristic of these interventions, as well as the high attrition rates (a common phenomenon in online interventions),74 could have possibly influenced RCTs’ findings. Third, not all tools used in the studies included in our systematic review are validated for remote use, although there is mounting evidence supporting the adequacy of common instruments (e.g., the GAD-7) in terms of psychometric properties when used for online assessment.75 Fourth, available data did not allow us to verify if outcomes of digital interventions could be influenced by specific characteristics in terms of, for example, gender, year of course, or type of degree programme. Clarifying this issue would favour the development of more tailored digital mental-health interventions for university students.76,77 Finally, only very few studies reported data on follow-up and long-term assessments, not allowing us to explore beneficial effects over time, rather than the need of periodical boosts, which is typical of psychoeducational interventions.78

Conclusions

Taken together, the findings of our review support the hypothesis that different types of digital mental health interventions could be effective in promoting the mental health and well-being of university students during the COVID-19 pandemic.79 Nonetheless, since the available body of evidence from RCTs is neither extensive nor of high quality, additional studies are needed before firmly advocating the application of such psychological strategies for higher-education students on a large scale. In particular, future rigorous research should provide additional insight into this important field, also in terms of special needs of more vulnerable groups, minorities, and frail populations.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Conflict of interest

None.

Appendix A
Supplementary data

The following are the supplementary data to this article:

References
[1]
P. Pedrelli, M. Nyer, A. Yeung, C. Zulauf, T. Wilens.
College students: mental health problems and treatment considerations.
Acad Psychiatry, 39 (2015), pp. 503-511
[2]
A.R.J. Briggs, J. Clark, I. Hall.
Building bridges: understanding student transition to university.
Qual High Educ, 18 (2012), pp. 3-21
[3]
R.P. Auerbach, J. Alonso, W.G. Axinn, P. Cuijpers, D.D. Ebert, J.G. Green, et al.
Mental disorders among college students in the World Health Organization World Mental Health Surveys.
Psychol Med, 46 (2016), pp. 1-16
[4]
R.P. Auerbach, P. Mortier, R. Bruffaerts, J. Alonso, C. Benjet, P. Cuijpers, et al.
WHO World Mental Health Surveys International College Student Project: prevalence and distribution of mental disorders.
J Abnorm Psychol, 127 (2018), pp. 623-638
[5]
P.E. Jenkins, I. Ducker, R. Gooding, M. James, E. Rutter-Eley.
Anxiety and depression in a sample of UK college students: a study of prevalence, comorbidity, and quality of life.
J Am Coll Health, 69 (2021), pp. 813-819
[6]
P. Barnett, L.L. Arundell, R. Saunders, H. Matthews, S. Pilling.
The efficacy of psychological interventions for the prevention and treatment of mental health disorders in university students: a systematic review and meta-analysis.
J Affect Disord, 280 (2021), pp. 381-406
[7]
C.T. Mowbray, D. Megivern, J.M. Mandiberg, S. Strauss, C.H. Stein, K. Collins, et al.
Campus mental health services: recommendations for change.
Am J Orthopsychiatry, 76 (2006), pp. 226-237
[8]
E. Ennis, M. McLafferty, E. Murray, C. Lapsley, T. Bjourson, C. Armour, et al.
Readiness to change and barriers to treatment seeking in college students with a mental disorder.
J Affect Disord, 252 (2019), pp. 428-434
[9]
J. Hunt, D. Eisenberg.
Mental health problems and help-seeking behavior among college students.
J Adolesc Health, 46 (2010), pp. 3-10
[10]
J. Deng, F. Zhou, W. Hou, Z. Silver, C.Y. Wong, O. Chang, et al.
The prevalence of depressive symptoms, anxiety symptoms and sleep disturbance in higher education students during the COVID-19 pandemic: a systematic review and meta-analysis.
Psychiatry Res, 301 (2021), pp. 113863
[11]
G. Sampogna, V. Giallonardo, V. Del Vecchio, M. Luciano, U. Albert, C. Carmassi, et al.
Loneliness in young adults during the first wave of COVID-19 lockdown: results from the multicentric COMET study.
Front Psychiatry, 12 (2021), pp. 788139
[12]
C. Wang, W. Wen, H. Zhang, J. Ni, J. Jiang, Y. Cheng, et al.
Anxiety, depression, and stress prevalence among college students during the COVID-19 pandemic: a systematic review and meta-analysis.
J Am Coll Health, (2021), pp. 1-8
[13]
A. Rudnik, P. Anikiej-Wiczenbach, A. Szulman-Wardal, P. Conway, M. Bidzan.
Offering psychological support to university students in poland during the COVID-19 pandemic: lessons learned from an initial evaluation.
Front Psychol, 12 (2021), pp. 635378
[14]
L. Orsolini, S. Pompili, V. Salvi, U. Volpe.
A systematic review on TeleMental health in youth mental health: focus on anxiety, depression and obsessive–compulsive disorder.
Medicina (Kaunas), 57 (2021), pp. 793
[15]
L.R. Rith-Najarian, M.M. Boustani, B.F. Chorpita.
A systematic review of prevention programs targeting depression, anxiety, and stress in university students.
J Affect Disord, 257 (2019), pp. 568-584
[16]
C.S. Conley, J.B. Shapiro, A.C. Kirsch, J.A. Durlak.
A meta-analysis of indicated mental health prevention programs for at-risk higher education students.
J Couns Psychol, 64 (2017), pp. 121-140
[17]
C.A. Figueroa, A. Aguilera.
The need for a mental health technology revolution in the COVID-19 pandemic.
Front Psychiatry, 11 (2020), pp. 523
[18]
M.L. Ryan, I.M. Shochet, H.M. Stallman.
Universal online interventions might engage psychologically distressed university students who are unlikely to seek formal help.
Adv Ment Health, 9 (2010), pp. 73-83
[19]
N.L. Hadler, P. Bu, A. Winkler, A.W. Alexander.
College student perspectives of telemental health: a review of the recent literature.
Curr Psychiatry Rep, 23 (2021), pp. 6
[20]
E.B. Davies, R. Morriss, C. Glazebrook.
Computer-delivered and web-based interventions to improve depression, anxiety, and psychological well-being of university students: a systematic review and meta-analysis.
J Med Internet Res, 16 (2014), pp. e130
[21]
B. Wies, C. Landers, M. Ienca.
Digital mental health for young people: a scoping review of ethical promises and challenges.
Front Digit Health, 3 (2021), pp. 697072
[22]
C. Rauschenberg, A. Schick, D. Hirjak, A. Seidler, I. Paetzold, C. Apfelbacher, et al.
Evidence synthesis of digital interventions to mitigate the negative impact of the COVID-19 pandemic on public mental health: rapid meta-review.
J Med Internet Res, 23 (2021), pp. e23365
[23]
M.J. Page, J.E. McKenzie, P.M. Bossuyt, I. Boutron, T.C. Hoffmann, C.D. Mulrow, et al.
The PRISMA 2020 statement: an updated guideline for reporting systematic reviews.
BMJ, 372 (2021), pp. n71
[24]
World Health Organization (WHO) Statement on the second meeting of the International Health Regulations (2005) Emergency Committee regarding the outbreak of novel coronavirus (2019-nCoV); 2020. https://www.who.int/news/item/30-01-2020-statement-on-the-second-meeting-of-the-international-health-regulations-(2005)-emergency-committee-regarding-the-outbreak-of-novel-coronavirus-(2019-ncov).
[25]
J. Miles, M. Shevlin.
Applying regression and correlation. A guide for students and researchers.
1st ed., SAGE Publications Ltd., (2001),
[26]
J.A. Rosenthal.
Qualitative descriptors of strength of association and effect size.
J Soc Serv Res, 21 (1996), pp. 37-59
[27]
A.J. Gerber, J.H. Kocsis, B.L. Milrod, S.P. Roose, J.P. Barber, M.E. Thase, et al.
A quality-based review of randomized controlled trials of psychodynamic psychotherapy.
Am J Psychiatry, 168 (2011), pp. 19-28
[28]
J.H. Kocsis, A.J. Gerber, B. Milrod, S.P. Roose, J. Barber, M.E. Thase, et al.
A new scale for assessing the quality of randomized clinical trials of psychotherapy.
Compr Psychiatry, 51 (2010), pp. 319-324
[29]
M. Cozzolino, G. Celia, L. Girelli, P. Limone.
Effects of the brain wave modulation technique administered online on stress, anxiety global distress, and affect during the first wave of the COVID-19 pandemic: a randomized clinical trial.
Front Psychol, 12 (2021), pp. 635877
[30]
J. Datu, J. Valdez, D.M. McInerney, R.F. Cayubit.
The effects of gratitude and kindness on life satisfaction, positive emotions, negative emotions, and COVID-19 anxiety: an online pilot experimental study.
Appl Psychol Health Well Being, (2021),
[31]
I. Krifa, Q. Hallez, L.E. van Zyl, A. Braham, J. Sahli, S. Ben Nasr, et al.
Effectiveness of an online positive psychology intervention among Tunisian healthcare students on mental health and study engagement during the Covid-19 pandemic.
Appl Psychol Health Well Being, (2021),
[32]
O. Lahtinen, J. Aaltonen, J. Kaakinen, L. Franklin, J. Hyönä.
The effects of app-based mindfulness practice on the well-being of university students and staff.
[33]
L. Liang, L. Feng, X. Zheng, Y. Wu, C. Zhang, J. Li.
Effect of dialectical behavior group therapy on the anxiety and depression of medical students under the normalization of epidemic prevention and control for the COVID-19 epidemic: a randomized study.
J Palliat Med, 10 (2021), pp. 10591-10599
[34]
R. Shabahang, M.S. Aruguete, L. McCutcheon.
Video-based cognitive-behavioral intervention for COVID-19 anxiety: a randomized controlled trial.
Trends Psychiatry Psychother, 43 (2021), pp. 141-150
[35]
O. Simonsson, O. Bazin, S.D. Fisher, S.B. Goldberg.
Effects of an eight-week, online mindfulness program on anxiety and depression in university students during COVID-19: a randomized controlled trial.
Psychiatry Res, 305 (2021), pp. 114222
[36]
S. Sun, D. Lin, S. Goldberg, Z. Shen, P. Chen, S. Qiao, et al.
A mindfulness-based mobile health (mHealth) intervention among psychologically distressed university students in quarantine during the COVID-19 pandemic: a randomized controlled trial.
J Couns Psychol, (2021),
[37]
R. Shabahang, M.S. Aruguete, L.E. McCutcheon.
Online health information utilization and online news exposure as predictors of COVID-19 anxiety.
N Am J Psychol, 22 (2020), pp. 469-482
[38]
P.M. Salkovskis, K.A. Rimes, H.M. Warwick, D.M. Clark.
The Health Anxiety Inventory: development and validation of scales for the measurement of health anxiety and hypochondriasis.
Psychol Med, 32 (2002), pp. 843-853
[39]
S. Taylor, M.J. Zvolensky, B.J. Cox, B. Deacon, R.G. Heimberg, D.R. Ledley, et al.
Robust dimensions of anxiety sensitivity: development and initial validation of the Anxiety Sensitivity Index-3.
Psychol Assess, 19 (2007), pp. 176-188
[40]
R.L. Spitzer, K. Kroenke, J.B. Williams, B. Löwe.
A brief measure for assessing generalized anxiety disorder: the GAD-7.
Arch Intern Med, 166 (2006), pp. 1092-1097
[41]
K. Kroenke, R.L. Spitzer, J.B. Williams.
The PHQ-9: validity of a brief depression severity measure.
J Gen Intern Med, 16 (2001), pp. 606-613
[42]
A.T. Beck, R.A. Steer, M.G. Carbin.
Psychometric properties of the Beck depression inventory: twenty-five years of evaluation.
Clin Psychol Rev, 8 (1988), pp. 77-100
[43]
P.A. Pilkonis, S.W. Choi, S.P. Reise, A.M. Stover, W.T. Riley, D. Cella, et al.
Item banks for measuring emotional distress from the Patient-Reported Outcomes Measurement Information System (PROMIS®): depression, anxiety, and anger.
Assessment, 18 (2011), pp. 263-283
[44]
S.A. Lee.
Coronavirus anxiety scale: a brief mental health screener for COVID-19 related anxiety.
Death Stud, 44 (2020), pp. 393-401
[45]
P.F. Lovibond, S.H. Lovibond.
The structure of negative emotional states: comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories.
Behav Res Ther, 33 (1995), pp. 335-343
[46]
C.D. Spielberger, R.L. Gorsuch, R. Lushene.
The state-trait personality inventory STAI-Y, Form Y.
Consulting Psychologists press, (1983), pp. 1983
[47]
A.F. Dawson, W.W. Brown, J. Anderson, B. Datta, J.N. Donald, K. Hong, et al.
Mindfulness-based interventions for university students: a systematic review and meta-analysis of randomised controlled trials.
Appl Psychol Health Well Being, 12 (2021), pp. 384-410
[48]
J. Huang, Y.T. Nigatu, R. Smail-Crevier, X. Zhang, J. Wang.
Interventions for common mental health problems among university and college students: a systematic review and meta-analysis of randomized controlled trials.
J Psychiatr Res, 107 (2018), pp. 1-10
[49]
A.A. Uliaszek, T. Rashid, G.E. Williams, T. Gulamani.
Group therapy for university students: a randomized control trial of dialectical behavior therapy and positive psychotherapy.
Behav Res Ther, 77 (2016), pp. 78-85
[50]
Y.F. Guo, X. Zhang, V. Plummer, L. Lam, W. Cross, J.P. Zhang.
Positive psychotherapy for depression and self-efficacy in undergraduate nursing students: a randomized, controlled trial.
Int J Ment Health Nurs, 26 (2017), pp. 375-383
[51]
I. Strehli, R.D. Burns, Y. Bai, D.H. Ziegenfuss, M.E. Block, T.A. Brusseau.
Mind–body physical activity interventions and stress-related physiological markers in educational settings: a systematic review and meta-analysis.
Int J Environ Res Public Health, 18 (2020), pp. 224
[52]
S.B. Stillwell, A.L. Vermeesch, J.G. Scott.
Interventions to reduce perceived stress among graduate students: a systematic review with implications for evidence-based practice.
Worldviews Evid Based Nurs, 14 (2017), pp. 507-513
[53]
E.R. Dorsey, E.J. Topol.
Telemedicine 2020 and the next decade.
[54]
T.D. Becker, J.B. Torous.
Recent developments in digital mental health interventions for college and university students.
Curr Treat Opt Psychiatry, 6 (2019), pp. 210-220
[55]
C. Oliveira, A. Pereira, P. Vagos, C. Nóbrega, J. Gonçalves, B. Afonso.
Effectiveness of mobile App-based psychological interventions for college students: a systematic review of the literature.
Front Psychol, 12 (2021), pp. 647606
[56]
K. Fila-Witecka, A. Senczyszyn, A. Kołodziejczyk, M. Ciułkowicz, J. Maciaszek, B. Misiak, et al.
Lifestyle changes among polish university students during the COVID-19 pandemic.
Int J Environ Res Public Health, 18 (2021), pp. 9571
[57]
E. Vieta, V. Pérez, C. Arango.
Psychiatry in the aftermath of COVID-19.
Rev Psiquiatr Salud Ment (Engl Ed), 13 (2020), pp. 105-110
[58]
M. Pwnall, R. Harris, P. Blundell-Birtill.
Supporting students during the transition to university in COVID-19: 5 key considerations and recommendations for educators.
Psychol Learn Teach, (2021),
[59]
M.E. Morris, K.S. Kuehn, J. Brown, P.S. Nurius, H. Zhang, Y.S. Sefidgar, et al.
College from home during COVID-19: a mixed-methods study of heterogeneous experiences.
PLOS ONE, 16 (2021), pp. e0251580
[60]
K.F. Johnson, M.T. Kalkbrenner.
The utilization of technological innovations to support college student mental health: mobile health communication.
J Technol Hum Serv, 35 (2017), pp. 314-339
[61]
G. Carrà, C. Crocamo, F. Bartoli, D. Carretta, A. Schivalocchi, P.E. Bebbington, et al.
Impact of a mobile e-health intervention on binge drinking in young people: the digital-alcohol risk alertness notifying network for adolescents and young adults project.
J Adolesc Health, 58 (2016), pp. 520-526
[62]
P. Wilansky, J.M. Eklund, T. Milner, D. Kreindler, A. Cheung, T. Kovacs, et al.
Cognitive behavior therapy for anxious and depressed youth: improving homework adherence through mobile technology.
JMIR Res Protoc, 5 (2016), pp. e209
[63]
A.L. Schuh Teixeira, A.V. Spadini, V. Pereira-Sanchez, et al.
The urge to implement and expand telepsychiatry during the COVID-19 crisis: Early career psychiatrists’ perspective.
Rev Psiquiatr Salud Ment (Engl Ed), 13 (2020), pp. 174-175
[64]
S. Liu, L. Yang, C. Zhang, Y.T. Xiang, Z. Liu, S. Hu, et al.
Online mental health services in China during the COVID-19 outbreak.
Lancet Psychiatry, 7 (2020), pp. e17-e18
[65]
M.H.E.M. Browning, L.R. Larson, I. Sharaievska, A. Rigolon, O. McAnirlin, L. Mullenbach, et al.
Psychological impacts from COVID-19 among university students: risk factors across seven states in the United States.
PLOS ONE, 16 (2021), pp. e0245327
[66]
T. Elmer, K. Mepham, C. Stadtfeld.
Students under lockdown: comparisons of students’ social networks and mental health before and during the COVID-19 crisis in Switzerland.
PLOS ONE, 15 (2020), pp. e0236337
[67]
E.A. Holmes, R.C. O’Connor, V.H. Perry, I. Tracey, S. Wessely, L. Arseneault, et al.
Multidisciplinary research priorities for the COVID-19 pandemic: a call for action for mental health science.
Lancet Psychiatry, 7 (2020), pp. 547-560
[68]
U. Volpe, L. Orsolini, V. Salvi, U. Albert, C. Carmassi, G. Carrà, et al.
COVID-19-related social isolation predispose to problematic Internet and online video gaming use in Italy.
Int J Environ Res Public Health, 19 (2022), pp. 1539
[69]
C. Crocamo, M. Viviani, L. Famiglini, F. Bartoli, G. Pasi, G. Carrà.
Surveilling COVID-19 emotional contagion on twitter by sentiment analysis.
Eur Psychiatry, 64 (2021), pp. e17
[70]
M. Viviani, C. Crocamo, M. Mazzola, F. Bartoli, G. Carrà, G. Pasi.
Assessing vulnerability to psychological distress during the COVID-19 pandemic through the analysis of microblogging content.
Future Gener Comput Syst, 125 (2021), pp. 446-459
[71]
P. Vornholt, M. De Choudhury.
Understanding the role of social media-based mental health support among college students: survey and semistructured interviews.
JMIR Ment Health, 8 (2021), pp. e24512
[72]
C. Crocamo, B. Bachi, R.M. Cioni, et al.
Professionals’ digital training for child maltreatment prevention in the COVID-19 era: a pan-European model.
Int J Environ Res Public Health, 19 (2022), pp. 885
[73]
C.B. Taylor, E.E. Fitzsimmons-Craft, A.K. Graham.
Digital technology can revolutionize mental health services delivery: the COVID-19 crisis as a catalyst for change.
Int J Eat Disord, 53 (2020), pp. 1155-1157
[74]
G. Eysenbach.
The law of attrition.
J Med Internet Res, 7 (2005), pp. e11
[75]
W. van Ballegooijen, H. Riper, P. Cuijpers, P. van Oppen, J.H. Smit.
Validation of online psychometric instruments for common mental health disorders: a systematic review.
BMC Psychiatry, 16 (2016), pp. 45
[76]
M. Wathelet, S. Duhem, G. Vaiva, et al.
Factors associated with mental health disorders among university students in france confined during the COVID-19 pandemic.
JAMA Netw Open, 3 (2020), pp. e2025591
[77]
C. Crocamo, B. Bachi, A. Calabrese, T. Callovini, D. Cavaleri, R.M. Cioni, et al.
Some of us are most at risk: systematic review and meta-analysis of correlates of depressive symptoms among healthcare workers during the SARS-CoV-2 outbreak.
Neurosci Biobehav Rev, 131 (2021), pp. 912-922
[78]
G. Carrà, C. Montomoli, M. Clerici, C.L. Cazzullo.
Family interventions for schizophrenia in Italy: randomized controlled trial.
Eur Arch Psychiatry Clin Neurosci, 257 (2007), pp. 23-30
[79]
W. Cao, Z. Fang, G. Hou, M. Han, X. Xu, J. Dong, et al.
The psychological impact of the COVID-19 epidemic on college students in China.
Psychiatry Res, 287 (2020), pp. 112934
Copyright © 2022. SEP y SEPB
Opciones de artículo
es en pt

¿Es usted profesional sanitario apto para prescribir o dispensar medicamentos?

Are you a health professional able to prescribe or dispense drugs?

Você é um profissional de saúde habilitado a prescrever ou dispensar medicamentos

Quizás le interese:
10.1016/j.rpsm.2020.05.004
No mostrar más