Digital empowerment in mental health: A meta-analysis of internet-based interventions for enhancing mental health literacy

Abstract

Background

Poor mental health literacy (MHL) in the global population significantly contributes to the treatment gap associated with mental disorders. In the digital age, leveraging Internet-based MHL interventions offers scalability and broader accessibility. This meta-analysis aimed to evaluate the effects of Internet-based interventions in improving MHL and mental health.

Method

Up to Feb 2024, seven databases were searched for Internet-based interventions on MHL (knowledge, stigma, help-seeking attitudes and intentions) and mental disorders (general distress, anxiety, and depressive symptoms). The random-effects meta-analyses at post-intervention and long-term follow-up assessments were performed.

Results

Twenty-nine eligible studies involving 11,582 participants were included. Significant positive effects were observed across various domains: knowledge increase (immediate: g = 0.459, 95 %CI: 0.285 to 0.634; follow-up: g = 0.487, 95 %CI: 0.348 to 0.626), immediate stigma reduction (g = -0.332, 95 %CI: -0.479 to -0.186), immediate enhancement of help-seeking attitudes (g = 0.168, 95 %CI: 0.046 to 0.3291) and help-seeking intentions (g = 0.135, 95 %CI: 0.072 to 0.198), as well as immediate mental health improvements (g = -0.074, 95 %CI: -0.115 to -0.033).

Conclusion

Overall, these findings underscore the promising effects of internet-based interventions in improving MHL and mental health, while maintaining these effects over time remains challenging, particularly in reducing stigma and promoting long-term help-seeking behaviors. Addressing methodological limitations, adopting a more interactive approach, and implementing targeted interventions are crucial to maximizing the effectiveness and advancing mental health care worldwide.

Introduction

Mental disorders are a leading cause of global disability (Vigo et al., 2016). Although it affects one in five adults annually (Steel et al., 2014), the typical public's poor understanding of mental disorders delays recovery and imposes significant health and economic burdens worldwide (Arias et al., 2022). Elevated levels of mental health literacy (MHL) within society offer a promising avenue to address and potentially reverse these challenges.

MHL is the knowledge and beliefs about mental disorders that facilitate their recognition, management, and prevention (Jorm et al., 1997). Addressing poor MHL has the potential to significantly narrow the treatment gap associated with mental disorders (Renwick et al., 2022; Tay et al., 2018; WHO, 2022). Enhanced MHL has been associated with various beneficial outcomes, including improved recognition of mental disorders and understanding of effective treatments (Morgan et al., 2018). Moreover, it plays a vital role in reducing discrimination against individuals with mental disorders (Ma et al., 2023), promoting personal mental well-being, encouraging help-seeking behaviors, and is negatively associated with psychological distress (Kutcher et al., 2016; Pehlivan et al., 2021).

In recent years, there has been a growing interest in utilizing the Internet to deliver MHL interventions due to their cost-effectiveness and accessibility (Donker et al., 2015; Massoudi et al., 2019). Online platforms offer advantages in overcoming geographical barriers, reaching underserved communities, and providing tailored and interactive educational materials within a secure environment (Andersson et al., 2019). Supporting this trend, a meta-analysis highlighted the effectiveness of online stigma reduction programs, demonstrating that these programs are not inferior to traditional face-to-face interventions in reducing personal stigma (Griffiths et al., 2014).

Despite some research indicating the effectiveness of online MHL interventions, there remains a significant gap in comprehensive analysis. For instance, one systematic review (Brijnath et al., 2016), including 14 studies published until August 2015, examined the impact of web-based interventions on MHL. It suggests that web-based interventions, particularly those with structured content that include various forms like audio, videos, animations, and illustrations, are effective. However, the authors could not conduct a meta-analysis to comprehensively assess the overall effectiveness of MHL interventions due to the limited number of randomized trials. Another recent meta-analysis focused on web-based educational interventions for young people aged 10 to 25 years, limiting insights into intervention impacts across diverse age groups (Nazari et al., 2023). While both studies demonstrated improvements in mental health knowledge, they did not address long-term intervention effects, underscoring the need for updated and inclusive analyses to provide a more comprehensive understanding of the effectiveness of MHL interventions for diverse populations.

Another persistent question regarding the effectiveness of MHL interventions concerns their impact on promoting psychological well-being. While earlier studies explored the relationship between MHL interventions and improved psychological well-being, research has yielded mixed results. For instance, a recently developed psychoeducation website called UTSMed significantly reduced depression among participants in the high-risk subgroup (Imamura et al., 2016). Conversely, other studies indicated no difference in depression symptoms before and after the intervention (Costin et al., 2009). While the systematic reviews suggested a positive association between increased MHL and improved mental health outcomes, especially for mild to moderate depression (Brijnath et al., 2016), no meta-analysis has thoroughly assessed the effects on psychological well-being.

Therefore, it is timely to synthesize the literature comprehensively to evaluate a broader range of MHL outcomes for various populations. The aim of this meta-analysis is to assess and analyze the immediate and sustained effectiveness of internet-based interventions in improving MHL and mental health. By summarizing empirical evidence, this study aims to guide the expansion and refinement of MHL interventions, thereby making valuable contributions to advancing mental health support strategies.

Methods

This systematic review and meta-analysis were conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines (Moher et al., 2009). The PROSPERO register number for this study is CRD42021264219.

Search strategy and selection of studies

We searched (last updated Feb 2024) the PsycINFO, PsycArticles, PubMed, Web of Science, Medline, EMBASE, and CINAHL, using index terms and synonyms related to Internet-based, mental health literacy, intervention, mental health, and RCTs. The complete search strategy can be found in the Appendix 1. We also reviewed the reference lists of included studies and relevant reviews identified in our search to ensure that all eligible research was included.

Eligible studies were randomized controlled trials (RCTs) examining the effects of web- or app-based self-help interventions on improving MHL against a control condition. The primary outcomes were MHL and its related components, including knowledge, stigma, help-seeking attitudes, and help-seeking intentions. The secondary outcomes included measures of mental health, such as anxiety, depression, and distress. Trials were excluded if the intervention: 1) required help from professionals, therapists, or peer support; 2) included in-person components; 3) was not predetermined in terms of content (for example, allowing participants to self-study through Chrome). Trials were also excluded if the control group received mental health information, or the study was not published in a peer-reviewed journal or was not written in English. Trials in which participants were over 13 years old were eligible for inclusion. Studies lacking data for effect size calculation were excluded unless the authors provided the data upon request.

After removing duplicates, titles, abstracts, and full texts were assessed by two reviewers independently in Covidence. Any disagreements between reviewers were resolved through discussion with a third reviewer until a consensus was reached.

Quality assessment and data extraction

We used the Cochrane Risk of Bias (RoB) 2.0 tool to assess the risk of bias (Sterne et al., 2019). Criteria include risks associated with the randomization process, deviations from intended interventions, bias due to missing outcome data, bias in outcome measurement, and bias in the selection of reported results for each study. Each domain was rated as high risk, low risk, or with some concerns. The randomization process was rated as low risk if a clear method for randomization in the allocation sequence was reported or if no baseline differences between groups were reported. Deviations from intended interventions were rated as low risk if the majority of participants received the intervention as designed. Bias due to missing outcome data was rated as low risk if appropriate methods (e.g., intent-to-treat analysis) were employed in the analysis of missing data. Given that the majority of studies utilize knowledge-based interventions with the aim of influencing measured outcomes, such as knowledge, bias in outcome measurement was rated as low risk if the outcome measures were self-reported and did not involve direct contact with the researcher. Bias in the selection of reported results was rated as low risk if all pre-specified primary outcomes were reported and there was no apparent reporting bias.

We extracted several characteristics pertaining to the study (author, year, country, sample size), participants (age, gender, target group, selection criteria), intervention (delivery method, primary target), comparison (type of control group), and outcome assessment (assessment tool, length assessed). Two researchers performed data extraction, and any discrepancies were resolved through consensus with a third reviewer.

During data extraction, we prioritized certain aspects of MHL due to its complexity. For knowledge, we focused on mental health-related knowledge, which is fundamental to MHL, while excluding help-seeking knowledge. Stigma-related outcomes were prioritized, with a focus on public stigma followed by self-stigma, social distance scale, reported and intended behaviour scale future domain (RIBS-F), and attribution questionnaires when necessary (Howard et al., 2018; Tippin & Maranzan, 2019). Help-seeking behaviours were refined to help-seeking intentions, such as extracting data from the General Help-seeking Questionnaire (GHSQ) instead of the Actual Help-seeking Questionnaire (AHSQ) due to a lack of available data (Nickerson et al., 2020). Formal strategies in help seeking of GHSQ were prioritized over informal sources, and data extraction focused on emotional problems rather than suicidal ideation. Furthermore, anxiety was prioritized over depression in articles assessing multiple mental disorders due to its higher global prevalence rate. This prioritization is supported by findings of Our World in Data (Dattani et al., 2023) and GBD 2019 Mental Disorders Collaborators (2022), which show that anxiety disorders are the most prevalent mental illness worldwide, followed by depressive disorders. Additionally, posttraumatic stress disorder (PTSD) was categorized under anxiety, consistent with its classification in existing literature.

Data synthesis

The primary outcomes of this study consisted of data obtained from post-intervention and/or follow-up evaluations of MHL and its components (i.e., mental health knowledge, stigma, help-seeking attitudes, and help-seeking intentions). Secondary outcomes included mental health outcomes, such as depression, anxiety, and stress.

Meta-analysis was performed in R (version 4.3.1) with the Meta package (Balduzzi et al., 2019). Random-effect models were employed for all analyses, with MHL and mental health outcomes assessed separately. Effect estimates were determined using standardized mean differences (SMD) at post-intervention and follow-up assessments. In cases where standard deviations were unavailable, we obtained standard errors or other statistical indicators and converted them per the Cochrane Handbook of Systematic Reviews. Groups stratified by gender and different distress levels were merged into one group by combining means and standard deviations. If studies reported multiple qualified arms in their sample, each arm was treated as a separate 'trial'.

We conducted separate meta-analyses for knowledge, stigma, help-seeking attitudes, help-seeking intentions, and mental health conditions, including all studies with estimable effect sizes. Intervention effects were analyzed separately for immediate and follow-up effects. Heterogeneity was assessed using the I² statistic. Subgroup analysis was performed if I² indicated significant heterogeneity and the confidence interval included high levels of heterogeneity (Morgan et al., 2018). Publication bias was evaluated using funnel plots and Egger's test. Statistical significance was set at p < 0.05 for all tests.

Results

The database search yielded 13,431 records, supplemented by 17 articles from relevant reviews, culminating in 13,448 records, with 7479 remaining after deduplication. Subsequent screening of titles and abstracts excluded 7325 records, resulting in 154 for full-text review. Among these, 29 articles met the inclusion criteria for the meta-analysis. Since two articles originated from the same trial (Reavley et al., 2018, 2021), the total number of distinct RCTs included was 28. Among the trials evaluating the effectiveness of various interventions against a control group, four trials included multiple intervention groups that fulfilled our inclusion criteria, and all eligible intervention arms were incorporated into the analysis (Amsalem et al., 2022; Fong & Mak, 2022; Griffiths et al., 2004; Gulliver et al., 2012). The literature selection process is shown in Fig. 1.

Fig. 1.

Flow chat of literature searching and screening.

Study characteristics

The characteristics of the total sample from each included study are presented in Table 1. The included 28 RCTs encompassed a combined sample size of 11,582, ranging from 44 to 1802 participants. Geographically, a substantive proportion of these RCTs were conducted in Australia (k = 13, 46.4 %), followed by America (k = 5, 17.9 %), Japan (k = 2, 7.1 %), the UK (k = 2, 7.1 %), Canda (k = 1, 3.6 %), China (k = 1, 3.6 %), Germany (k = 1, 3.6 %), Norway (k = 1, 3.6 %), Singapore (k = 1, 3.6 %), and Spain (k = 1, 3.6 %).

Table 1.

Characteristics of included studies.

Study	Country	Study population	% female	Age mean (S.D.)/range	N	Intervention	Delivery method	Comparison	ITT	Outcomes	Measurements
Afsharnejad et al., 2023	Australia	University students	80	25.22 (7.43)	129	Talk-to-Me: "Talk-to-Me" is an online psychoeducational suicide prevention program for young adults. It aims to increase awareness of mental health-promoting activities, improve resilience, develop distress management skills, and identify early signs of suicidal ideation.	Structured programs	Waitlist control	Yes	Help-seeking attitudes	ATSPPH-SF
Amsalem & Martin, 2022	USA	Adolescents or young adults	48	16.8 (1.2)	1183	Social contact-based videos: Two intervention videos (∼102 to 113 s each) featured young actors discussing their struggles with depressive symptoms and their journey to seek help. These videos aimed to humanize mental health issues through relatable social contact.	Video-based materials	Active control	No	Stigma	DSS - Personal stigma subscale
										Help-seeking intentions	GHSQ - Mental health professional
Amsalem et al., 2022	USA	Hospital or healthcare participants	74	34.8 (11.5)	350	Social contact-based videos: The 3-minute video featured a nurse discussing her struggles with stress, anxiety, and depression and emphasizing the challenges of overcoming misconceptions about treatment. She emphasizes the importance of social support and psychotherapy in managing COVID-19 stressors. The video concludes with her positive message on the benefits of seeking mental health treatment.	Video-based materials	No-treatment control	Yes	Help-seeking attitudes	ATSPPH-SF - Subscale of openness to help-seeking
Billings et al., 2008	USA	General population	70.6	20∼69	309	Stress and Mood Management: It is a web-based, audio-narrated health promotion program aimed at helping working adults manage stress, prevent mood issues, and seek early treatment for depression and anxiety. The program uses cognitive-behavioral techniques like goal setting, problem solving, and relaxation.	Structured programs	Waitlist control	No	Knowledge	Self-designed_The early identification, prevention, and treatment of stress, anxiety, and mood disorders
										Help-seeking attitudes	ATSPPH-SF
										Mental health	BAI, CES-D, Symptoms of Distress Scale
Costin et al., 2009	Australia	Adolescents or young adults	77.6	21.4 (1.5)	348	Mental Health E-cards: The intervention included three personalized emails over three weeks, each containing a URL link to brief health information on a webpage, described as health e-cards. These e-cards provided information on depression symptoms, a personal vignette, and guidance on finding evidence-based treatment.	Educational materials	Active control	No	Help-seeking intentions	GHSQ
										Mental health	CES-D
Davies et al., 2018	UK	Hospital or healthcare participants	65.5	19.9 (3.2)	55	Mental Health First Aid (MHFA) eLearning course: The eLearning course consisted of six modules delivered through various multimedia formats. Participants were educated about specific mental disorders and taught the disorder-specific MHFA action plan.	Structured programs	No-treatment control	Yes	Stigma	DSS - Personal stigma subscale
Deitz et al., 2008	USA	General population	45.45	42 (N/R)	99	Youth Mental Health - A Parent's Guide: It is a web-based program with four interactive, multimedia-rich modules based on social cognitive theory. It covers anxiety, depression, treatment options, and parental strategies, aiming to increase self-efficacy and promote behavioral improvement. The program provides detailed information, interactive tools, practical advice, and links to advanced resources on specific mental health issues.	Structured programs	Waitlist control	Yes	Knowledge	Self-designed Knowledge of Childhood Depression and Anxiety
										Help-seeking attitudes	Adapted ATTPHS
Farrer et al., 2012	Australia	Participants with mental health problems	83.56	40.5 (12.5)	73	Web-based Psychoeducation and CBT: The intervention included web-based psychoeducation (via BluePages in week 1) and web-based CBT (via MoodGYM in weeks 2–6). BluePages is a web-based psychoeducation platform designed to provide reliable and comprehensive information about depression. MoodGYM is an interactive, web-based program designed to help users learn and apply CBT techniques to manage and reduce symptoms of depression and anxiety.	Structured programs	Waitlist control	Yes	Knowledge	D-lit
										Stigma	DSS - Personal stigma subscale
										Mental health	CES-D
Fong & Mak, 2022	China	Adolescents or young adults	69.2	22.56 (6.16)	199	Amazing Adventure Against Stigma: The website offers a 20-minute interactive story about a person experiencing mental illness stigma. Participants engage with the story through animated backgrounds and diverse avatars, choosing actions and responses to interact with the protagonist. The story guides participants through encounters with microaggressions and public misunderstandings, highlighting the connection between people with and without mental illness.	Stigma group: websites	Active control	Yes	Stigma	The 21-item Public Stigma Scale-Mental Illness-Short Version
Griffiths et al., 2004	Australia	Participants with mental health problems	74	36.5 (9.4)	525	BluePages: It is a web-based resource providing comprehensive information about depression. It covers medical and psychological treatments and prevention strategies, emphasizing that depression is treatable and the importance of seeking help. MoodGYM: It is an interactive online program with five modules teaching cognitive-behavioral techniques for managing depression and anxiety.	Structured programs	Attentional control	Yes	Stigma	DSS-Personal stigma subscale
Griffiths et al., 2016	Australia	General population	73.6	44.5 (11.3)	507	Mental Health Guru: It is an online psychoeducation workplace induction program. It consists of two modules: one focused on depression and the other on generalized anxiety disorder. Each module includes information on prevalence, symptoms, identification, treatments, risk factors, myth-busting, advice for supervisors and colleagues, and sources of help. The program features a multimedia, interactive format with graphics, exercises, and video vignettes of individuals with lived experience.	Structured programs	Waitlist control	Yes	Knowledge	A-lit
										Stigma	GASS - Personal stigma subscale
										Help-seeking attitudes	ATSPPH-SF
										Help-seeking intentions	GHSQ
Griffiths et al., 2017	Australia	Participants with mental health problems	91.5	43.8 (15.3)	82	Shyness Information Online: It is a text-based, online, multi-component program designed to increase help-seeking behavior based on the Social Anxiety Disorder Help-seeking Behaviour Framework. The program addresses themes such as literacy, stigma reduction, normative feedback, help-seeking information, and motivational interviewing, presented across 21 linear pages with three interactive tasks.	Structured programs	Active control	Yes	Knowledge	Self-designed: Social Anxiety Literacy Questionnaire
										Stigma	SASS-I
										Help-seeking intentions	GHSQ
										Help-seeking attitudes	ATSPPH-SF
Gulliver et al., 2012	Australia	General population	75	25.30 (6.01)	44	Psychoeducation Webpages: It involved written material delivered through 34 brief linear webpages designed to increase MHL and decrease stigma, particularly for depression and anxiety	Educational materials	No-treatment control	Yes	Knowledge	A-lit
										Stigma	GASS
										Help-seeking attitudes	ATSPPH-SF
										Help-seeking intentions	GHSQ
Imamura et al., 2016	Japan	General population	29.6	39.5 (8.9)	1236	The University of Tokyo website for Stress Management and Education on Depression (UTSMed): It is an information website developed to provide psychoeducational content and CBT for managing depression and stress. UTSMed consists of approximately 90 pages of text and illustrations, including information on symptoms, diagnosis, treatment, and mechanisms of depression, as well as stress management tips.	Educational materials	No-treatment control	No	Knowledge	Self-designed: Improvement of knowledge in knowing and coping with stress and depression
										Mental health	BDI-II, K6
Kauer et al., 2017	Australia	Adolescents or young adults	76.5	20.95 (1.98)	51	The Link Intervention: It involved a self-directed triage process designed to help participants seek appropriate services for their issues. Participants followed three steps: (1) selecting their issues (e.g., depression/anxiety, body image, etc.), (2) indicating severity on a five-point interactive pictorial sliding scale, and (3) selecting their service preference (e.g., face-to-face, phone helpline, etc.). Based on these inputs, three suitable services were recommended, including a description, expectations, costs, and direct access links.	Educational materials	Active control	No	Help-seeking attitudes	Mental health help-seeking perceptions
										Help-seeking intentions	GHSQ
Kerber et al., 2023	Germany	Participants with mental health problems	73.6	38.3 (11.19)	1045	MindDoc: The MindDoc app is a self-guided transdiagnostic intervention designed for mental health care across various stages, including prevention, early recognition, treatment, and aftercare. The core features of the app encompass regular self-monitoring and automated feedback and psychological courses and exercises based on cognitive behavioral therapy principles.	Structured programs	Waitlist control	Yes	Knowledge	MHLQ
										Help-seeking attitudes	IATSMHS
										Mental health	GAD-7, PHQ-9
Lintvedt et al., 2013	Norway	Participants with mental health problems	76.7	28.2 (7.4)	163	BluePages: It is a web-based resource providing information about depression, including its symptoms, sources of help, and evidence-based treatments. It covers medical, psychological, and treatments, as well as prevention strategies. The site emphasizes that depression is a treatable illness and the importance of seeking help. MoodGYM: It is an interactive online program consisting of five modules that teach CBT for managing depression and anxiety.	Structured programs	Waitlist control	Yes	Knowledge	Treatment depression literacy
										Mental health	CES-D
Martin et al., 2022	USA	Adolescents or young adults	45.29	16.8 (1.1)	1291	Social contact-based videos: A brief stimuli video (lasting 117 s) edited down from filmed interviews with a young professional actor, a 16-year-old Black girl named “Jasmin”. In a direct manner, she describes difficulties coping with depressive symptoms, thoughts that life is not worth living, false assumptions about treatment, and how and when she decided to seek help.	Video-based materials	Active control	No	Stigma	DSS-Personal stigma subscale
										Help-seeking intentions	GHSQ
Nickerson et al., 2020	Australia	Participants with mental health problems	0	39.37 (9.88)	103	Tell Your Story: This intervention comprised 11 short, interactive web-based modules designed to reduce stigma and increase help-seeking behaviors. The modules included information, short videos, and activities focused on both formal and informal sources of help, particularly for individuals from collectivist cultures. Strategies such as psychoeducation, social contact, and cognitive reappraisal of negative beliefs about mental health and help-seeking were implemented.	Structured programs	Waitlist control	Yes	Stigma	Self-Stigma for PTSD
										Help-seeking intentions	GHSQ
O'Connell et al., 2021	UK	Hospital or healthcare participants	87	36.16 (N/R)	139	MindED: This study used MindED which is a free educational resource for adults to support children and young people's mental health. The training content consisted of two modules, “What Goes Wrong” and “Mind and Body: The Interface,” with additional information on depression and oppositional defiant disorder . It also provided guidance for staff on how to recognize and support children needing help.	Educational materials	Waitlist control	No	Stigma	RIBS
O'Dea et al., 2021	Australia	Adolescents or young adults	51.6	14.30 (0.87)	1802	Smooth Sailing service: This intervention began with screening participants using the GAD-7 and the PHQ-9. Based on their scores, students were assigned a care level and given a personalized dashboard with symptom feedback and recommended modules. The psychoeducation component included five 10-minute self-directed modules on mental health topics, supplemented by animations and links to credible youth mental health services. Additionally, a module referred students to two Internet CBT programs: MoodGym and The BRAVE Program.	Structured programs	No-treatment control	Yes	Knowledge	MHLS
										Help-seeking intentions	GHSQ
										Mental health	GAD-7, CES-DC, DQ5
Reavley et al., 2018, 2021	Australia	General population	75.49	41.23 (10.93)	409	MHFA eLearning course: This intervention was a 6-hour course accessed via the MHFA Australia online portal. It taught a 5-component action plan for addressing developing mental health issues and crises. This course consisted of five sequential modules: Introduction to mental health; Depression; Anxiety Problems; Psychosis and Substance Use Problems. Each module included interactive case studies, quizzes, and audio and video content showcasing lived experiences and mental health first aid techniques, followed by related activities.	Structured programs	Active control	Yes	Knowledge	Beliefs about treatment_PTSD
										Stigma	SDS-PTSD
Rodriguez-Ferrer et al., 2022	Spain	University students	81.4	23.34 (8.37)	306	Without Memories escape room: This intervention was based on the experiences and daily challenges faced by people with serious mental illness. It featured a web-based, synchronous escape room game with a linear structure, where solving one clue provided the necessary object to solve the next. This sequential task completion continued until participants escaped. The average duration of the escape room was 60 min.	Escape room game	Active control	No	Stigma	Attribution Questionnaire-14
Sanci et al., 2019	Australia	Adolescents or young adults	83.29	20.70 (2.42)	413	The Link Intervention: Link is a self-directed mental health help-seeking service navigation website designed to guide young people to appropriate Web-based and computer-based sources of mental health information and care.	Educational materials	Active control	Yes	Help-seeking intentions	GHSQ
										Mental health	K10
Tay et al., 2022	Singapore	University students	71.26	N/R	175	HOPE: The HOPE intervention was designed to improve MHL and included four sessions over two weeks, with two sessions per week. Each session lasted about ten minutes and featured pre-post quizzes, videos, and illustrations about mental health. Session one addressed myths, symptoms, causes, self-help strategies, and treatments for depression. Session two focused on positive psychology. Session three covered myths, causes, symptoms, self-help strategies, and treatments for anxiety disorders. Session four included relaxation exercises and CBT techniques for managing unhealthy thoughts.	Structured programs	Attentional control	Yes	Knowledge	A-Lit
										Stigma	Personal Stigma Scale
										Mental health	PSS
Taylor-Rodgers & Batterham, 2014	Australia	Adolescents or young adults	74.63	21.9 (1.94)	67	Online information program: A three-week online program with content on depression, anxiety, and suicide. Each topic included a vignette of a typical young person experiencing the issue, a description of symptoms, strategies for challenging stigmatizing views, treatment options, and help resources. Information was based on DSM criteria and synthesized from mental health websites such as BluePages, Youth beyondblue, and the Black Dog Institute.	Structured programs	Active control	Yes	Knowledge	A-Lit
										Stigma	GASS
										Help-seeking attitudes	ATSPPH-SF
										Help-seeking intentions	GHSQ
Tippin & Maranzan, 2019	Canada	University students	73.9	21.44 (5.04)	303	Photovoice‐based video: This intervention featured photographs taken by eight individuals living with mental illnesses over nine months. These photographs, selected and organized into themes (experience of mental illness, barriers to recovery, supports to recovery, and recovery experience), were accompanied by voiceovers explaining their significance, and text highlighting the main themes. The video addressed various mental health issues, emphasizing recovery and avoiding biological explanations to prevent increasing stigma.	Video-based materials	Attentional control	No	Stigma	SDS
Yamaguchi et al., 2019	Japan	University students	30.68	20.22 (1.13)	176	Filmed social contact intervention: This intervention featured a 30-minute film showcasing interviews with two men living with schizophrenia, a portrayal of a woman with obsessive-compulsive disorder, and educational lecture slides explaining mental health concepts.	Video-based materials	No-treatment control	No	Stigma	RIBS-Japan version_Future domain
										Knowledge	MIDUS

Note. ITT = Intent-to-Treat analysis. A-lit = Anxiety Literacy Questionnaire. ATSPPH-SF = Attitudes Toward Seeking Professional Psychological Help-Short Form scale. BAI = Beck Anxiety Inventory. BDI-II = Beck Depression Inventory. CES-D = Center for Epidemiological Studies Depression Scale. DQ5 = Distress Questionnaire-5. DSS = Depression Stigma Scale. d-lit = Depression Literacy Scale. GAD = Generalized Anxiety Disorder. GALS = General Anxiety Literacy Scale. GASS = Generalised Anxiety Stigma Scale. GHSQ = General Help-Seeking Questionnaire. IATSMHS = Inventory of Attitudes Toward Seeking Mental Health Services. K6, K10 = Kessler Psychological Distress Scale. MHLS = Mental Health Literacy Scale. MHLQ = Mental Health Literacy Questionnaire. MIDUS = The Mental Illness and Disorder Understanding Scale. PHQ = Patent Health Questionnaire. PSS = The Perceived Stress Scale. RIBS = Reported and Intended Behaviour Scale. SASS-I = Social Anxiety Stigma Scale. SDS = Social Distance Scale. SASS-I = Social Anxiety Stigma Scale.

The included studies featured a diverse range of demographic characteristics among participants. The average age of participants ranged from 14.3 to 44.5 years. Female participants constituted between 0 % and 91.5 % of the samples, with a median percentage of 66.39 %. Of the 28 RCTs, eight (29.6 %) focused on adolescents and young adults, six (22.2 %) were recruited from the general population, six (22.2 %) focused on participants with mental health concerns, five (17.9 %) focused on university students or postgraduates, and three (11.1 %) focused on hospital or healthcare workers. Among the six trials focusing on people with mental health concerns, two recruited people with mild mental distress, two recruited people with elevated depressive symptoms and other internalizing symptoms, one recruited refugee men with PTSD symptoms, and the other recruited people with social anxiety.

Regarding the intervention approaches, more than half of the studies (k = 15, 53.6 %) implemented structured programs incorporating a variety of media such as audio clips, video segments, animations, illustrations, interactive exercises, and real-life case studies. Another 21.4 % (k = 6) of the trials utilized educational materials, where participants were tasked with reading text-based content aimed at improving MHL, reducing stigma, and promoting help-seeking behavior. Notably, one study introduced 'Link', a dedicated online mental health help-seeking navigation tool that matches users' mental health issues, severity, and service-type preferences (Imamura et al., 2016). Additionally, five studies (17.8 %) used video-based materials to disseminate MHL concepts.

Most interventions were conducted via websites (k = 24, 85.7 %), while three (10.7 %) utilized Qualtrics as the research platform, and one (3.6 %) was delivered through mobile apps. The duration of these interventions varied widely from brief videos lasting between 102 and 113 s (Amsalem & Martin, 2022) to more extensive programs spanning up to six sessions over the course of one year (Yamaguchi et al., 2019).

Regarding the outcome measurements, 21 studies (75.0 %) reported multiple dimensions of MHL (knowledge, k = 14, 66.7 %; stigma, k = 12, 57.1 %; help-seeking intentions, k = 11, 52.4 %; help-seeking attitude, k = 7, 33.3 %; mental health symptoms, k = 9, 42.9 %). Seven studies (25.0 %) reported only a single dimension of MHL (stigma, k = 5, 71.4 %; help-seeking attitude, k = 2, 16.7 %). Among these trials, ten (35.7 %) included a follow-up test conducted over a timeframe ranging from 1 week to 24 months.

Concerning knowledge (k = 14, 50.0 %), ten trials utilized validated scales (anxiety, k = 4, 40 %; generic mental health, k = 3, 30 %; depression, k = 2, 20 %; PTSD, k = 1, 10 %). The remaining four trials used self-designed questionnaires (Billings et al., 2008; Deitz et al., 2008; Griffiths et al., 2017; Imamura et al., 2016). These questionnaires aimed to measure participants' understanding of early identification, prevention, and treatment of mental health problems. One of these studies measured knowledge improvement by assessing participants' confidence in coping with stress and depression (Imamura et al., 2016).

Of the 17 trials assessing stigma, all used or adapted validated scales. Specifically, 11 studies (64.7 %) targeted stigma towards a specific mental disorder (depression, k = 5, 45.4 %; anxiety, k = 3, 27.3 %; PTSD, k = 2, 18.2 %; social anxiety, k = 1, 9.1 %), and six (35.3 %) focused on generic mental health stigma.

The majority of trials utilized the Attitudes Toward Seeking Professional Psychological Help scale (ATSPPH) (Fischer & Turner, 1970) to measure help-seeking attitudes (k = 8, 80 %). For the other two studies that reported help-seeking attitudes as their outcome, one used a self-designed questionnaire to assess the mental health help-seeking perceptions, and the other used the Inventory of Attitudes Toward Seeking Mental Health Services (IATSMHS). Furthermore, help-seeking intentions were measured by 11 trials (39.3 %) through the General Help-seeking Questionnaire (GHSQ).

In addition to MHL-related outcomes, nine studies assessed participants' mental health, including depression, k = 7, 77.8 %, stress, k = 4, 44.4 %, anxiety, k = 3, 33.3 %. Notably, all measures were validated scales, including the Beck Anxiety Inventory (BAI) (Beck et al., 1988), Center for Epidemiologic Studies Depression Scale (CES) (Eaton et al., 2004), Beck Depression Inventory (BDI) (BECK et al., 1961), and Kessler Psychological Distress Scale (Kessler et al., 2002).

Control groups primarily consisted of active control (k = 10, 35.7 %), waitlist control (k = 9, 32.1 %), no treatment control (k = 6, 21.4 %), and attentional control (k = 3, 10.7 %). Most included trials applied intent-to-treat analysis (k = 18, 64.3 %), while ten trials did not (35.7 %). However, in two of the ITT analysis studies, data for completers were used for analysis because the ITT data were unavailable for extraction (Nickerson et al., 2020; Taylor-Rodgers & Batterham, 2014).

Effects of mental health literacy intervention

The results of the intervention effect are presented in Table 2 and Fig. 2 separately for immediate and follow-up intervention effects.

Table 2.

Effect sizes, heterogeneity test, and publication bias test.

Outcome variables	k	N	g [95 % CI]	Z	p	Heterogeneity test			Publication bias test
						Q	df	I^2 %	Egger's intercept	SE	p
Mental health literacy
Knowledge
Post	15	5320	0.459[0.285; 0.634]	5.16	<0.0001	107.28	14	87.00	0.03	0.129	0.034
Follow-up	7	1322	0.487[0.348; 0.626]	6.89	<0.0001	9.19	6	11.08
Stigma
Post	20	5168	−0.332[−0.479; −0.186]	−4.44	<0.0001	89.55	19	78.80	−0.199	0.162	0.510
Follow-up	11	1786	−0.679[−1.418; 0.061]	−1.80	0.072	557.78	10	97.48
Attitudes
Post	11	2690	0.168[0.046; 0.291]	2.69	0.007	18.08	10	44.70	0.217	0.110	0.708
Follow-up	5	941	0.120[−0.008; 0.248]	1.84	0.066	3.03	4	0
Help-seeking intentions
Post	12	4890	0.135[0.072; 0.198]	4.18	<0.0001	8.98	11	0	0.138	0.059	0.952
Follow-up	4	645	0.118[−0.037; 0.273]	1.49	0.1369	0.95	3	0
Mental health
Post	16	9841	−0.074[−0.115; −0.033]	−3.56	0.0004	10.82	15	0	−0.014	0.041	0.124
Follow-up	2	218	−0.086[−0.353; 0.181]	−0.63	0.528	0.77	1	0

Note: k represents the number of independent effect sizes, N is the sample size, and 95 % CI is the 95 % confidence interval for the effect size g corresponding to the outcome variable; ** p < 0.05, *** p < 0.001. Heterogeneity test: Q represents the within-group heterogeneity test statistic (two-tailed test).

Fig. 2.

Forest plots of the meta-analysis at post-intervention assessment.

Knowledge

The interventions significantly increased the level of knowledge immediately after the intervention and in the subsequent follow-up assessments. The pooled effect size for immediate intervention effects was statistically significant and of medium magnitude (g = 0.459, 95 % CI: 0.285 to 0.634, p < 0.0001). However, it is important to note that there was significant heterogeneity between studies (I² = 87.0 %). Moreover, little evidence suggests that publication bias influenced the conclusions (Egger's intercept = 0.03, p = 0.034). Regarding follow-up effects, the pooled mean effect size across all interventions indicated a medium effect (g = 0.487, 95 % CI: 0.348 to 0.626, p < 0.0001).

Stigma

The interventions significantly reduced stigma immediately after the intervention but not in follow-up assessments. The pooled mean effect size for immediate effects, including all conditions and interventions, was significant (g = −0.332 95 % CI: −0.479 to −0.186, p < 0.0001). However, a substantial heterogeneity was noted between studies (I²= 78.80 %). There was little evidence that the conclusions were affected by publication bias (Egger's intercept = −0.199, p = 0.510). The follow-up intervention effect was insignificant (g = −0.679, 95 % CI: −1.418 to 0.061, p = 0.072).

Help-seeking attitudes

The interventions significantly increased professional help-seeking attitudes immediately after the intervention. The pooled mean effect size for immediate effects, including all conditions and interventions, was very small yet significant (g = 0.168, 95 % CI: 0.046 to 0.291, p = 0.007). A low level of heterogeneity between studies was detected (I²=44.70 %). There was little evidence that the conclusions were influenced by publication bias (Egger's intercept = 0.217, p = 0.708). However, the intervention did not show significant follow-up effects (g = 0.120, 95 % CI: −0.008 to 0.248, p = 0.066).

Help-seeking intentions

The interventions significantly enhanced professional help-seeking intentions immediately after the intervention. The pooled mean effect size for immediate effects was very small yet statistically significant (g = 0.135, 95 % CI: 0.072 to 0.198, p < 0.0001). There was no heterogeneity among the studies (I²=0 %). However, the intervention did not yield significant follow-up effects (g = 0.118, 95 % CI: −0.037 to 0.273, p = 0.1369). There was little evidence that the conclusions were influenced by publication bias (Egger's intercept = 0.138, p = 0.952).

Mental health

The MHL interventions significantly improved mental health immediately following the intervention. The mean effect size for immediate effects was very small yet statistically significant (g = −0.074, 95 % CI: −0.115 to −0.033, p = 0.0004). There was no heterogeneity among the studies (I²=0 %). However, when evaluating follow-up effects, no statistically significant effects were observed (g = −0.086, 95 % CI: −0.353 to 0.181, p = 0.528). There is limited evidence to suggest that publication bias might influence the conclusions (Egger's intercept = −0.014, p = 0.124).

Subgroup analysis

Subgroup analyses were conducted exclusively on the immediate intervention effects because only a limited number of studies reported follow-up results. The table for subgroup analysis is available in the Appendix 2. Due to the high heterogeneity, subgroup analyses were solely conducted for knowledge (I²=87 %) and stigma (I²=78.8 %).

For knowledge, significant differences were observed across study regions (Q = 19.93, p = 0.0013), with interventions conducted in Singapore yielding the largest effects. In addition, we found comparison with attention control had a larger effect than other types of comparisons (Q = 72.88, p < 0.0001). However, these findings were mainly driven by one study(Tay et al., 2022), reporting large effects (g = 0.952, 95 % CI: 0.637 to 1.268). No significant differences were found for other selected variables in subgroup analysis.

For stigma, the effects significantly differed in intervention formats (Q = 50.46, p < 0.0001), with the online game showing the largest effect. Additionally, the intervention effects differed significantly based on region (Q = 67.74, p < 0.0001), with Spain tending to have the largest effect. However, these findings were mainly driven by one study (Rodriguez-Ferrer et al., 2022), which reported large effects (g = −1.225, 95 %CI: −1.479 to −0.972). Interventions aimed at increasing MHL were observed to be more effective in reducing stigma compared to those aimed at addressing mental disorders (Q = 3.9, p = 0.0482). No significant differences were found for other selected variables in subgroup analysis.

Risk of bias

Only seven studies had an overall low risk of bias rating (25 %). Half of the trials were rated as having some concerns regarding the risk of bias (k = 14, 50 %), and the remaining seven were classified as high risk (25 %). The primary issues identified, i.e., some concerns or high risk, included the selection of the reported result (k = 12, 42.9 %), the randomization process (k = 11, 39.3 %), deviations from the intended intervention (k = 9, 32.1 %), and missing outcome data (k = 6, 21.4 %). The summary of the RoB results can be seen in Fig. 3 and Fig. 4.

Fig. 3.

Results from risk of bias assessment.

Fig. 4.

Summary of risk of bias assessments for each study.

Discussion

To our knowledge, this study is the first comprehensive evaluation of the immediate and sustained effectiveness of internet-based interventions in improving MHL and mental health outcomes across diverse populations. Our meta-analysis examined data from 28 RCTs conducted in 10 countries involving 11,582 participants. It found that internet-based interventions significantly improve participants' knowledge immediately after treatment (g = 0.459), and this effect persists during follow-up (g = 0.487). Moreover, these interventions also demonstrate a very small yet significant positive post-treatment effect on promoting help-seeking attitudes (g = 0.168) and intentions (g = 0.135), reducing stigma (g = −0.332), and alleviating symptoms of mental disorders (g = −0.074). While these effects were not sustained over time or reassessed during follow-up, they highlight the initial benefits of internet-based interventions in addressing these mental health challenges.

Regarding mental health knowledge, internet-based programs can increase people's awareness and understanding of mental health issues, such as the symptoms and treatments of common mental disorders and the available sources of support and have a long-term sustained effect. Consistent with our findings, a meta-analysis also found a significant increase in mental health knowledge, persisting up to six months following school-based MHL programs (Amado-Rodríguez et al., 2022). Compared to stigma and behavior, knowledge enhancement benefits from a relatively straightforward objective of delivering accurate information about mental health. Additionally, internet-based interventions may use various formats to actively engage participants in learning, facilitating better retention and application of knowledge. The accessibility and flexibility of Internet-based interventions may further allow individuals to learn at their own pace and access resources continually, enhancing their understanding and knowledge retention.

However, we also noted several discrepancies among studies that warrant attention, particularly in intervention content and assessment methods. While some RCTs concentrated on improving the understanding of individual mental disorders (Gulliver et al., 2012), others tackle multiple disorders (O'Connell et al., 2021), leading to variations in effectiveness. The use of self-designed questionnaires further complicates assessments, reflecting a lack of consensus among researchers regarding conceptualizations in this field (Wei et al., 2015). Despite this variability, expecting a universal assessment of MHL is impractical, and interventions tailored to specific contexts or cultures should be responsive to their unique needs and circumstances. For example, educating teenagers about a variety of mental health conditions may be more beneficial for prevention than concentrating on just one. Conversely, targeted MHL interventions may efficiently benefit individuals at high risk of specific conditions, such as depression in breast cancer patients. It is imperative to develop assessment tools that strike a balance between standardization and cultural or contextual sensitivity. Given the global attention to MHL, we advocate for assessment tools that incorporate both universal components and adaptive elements to enhance the accuracy of MHL assessment and facilitate meaningful comparisons across studies. For instance, when designing a measure for depression knowledge in Asian cultures, researchers could include questions about common depressive symptoms alongside culture-specific misconceptions. By adopting this tailored approach, researchers can improve the assessment of MHL and contribute to developing more effective and culturally sensitive interventions.

Our study highlights the transient nature of the very small effects of internet-based interventions in reducing stigmatized beliefs. Previous reviews and meta-analyses yielded mixed results regarding the effectiveness of such interventions in reducing stigma, with outcomes varying across populations and types of interventions. For instance, a meta-analysis showed that educational interventions and social contacts can effectively decrease public stigma among adults and adolescents with mental illness (Corrigan et al., 2012). Conversely, no effects of web-based programs on reducing stigma were found among young people aged 18 to 25 (Nazari et al., 2023). Importantly, few reviews or meta-analyses have differentiated the short- and long-term effects of stigma reduction (Ran et al., 2021). The present study enriches our understanding by elucidating the immediate, albeit unsustainable, effects of Internet-based interventions across diverse groups. The transient nature of the effects may be attributed to various factors, such as diminishing novelty, lack of ongoing reinforcement, or the persistence of societal stigma despite individual interventions. Thus, intervention participants could experience a resurgence of stigma when exposed to negative environments without ongoing support or reminders. This finding emphasizes the importance of providing continuous assistance, such as follow-up sessions, periodic reminders, or access to support networks to reinforce positive attitudes towards mental health. The subgroup analysis highlights the potential of interactive components to enhance intervention effectiveness in reducing stigma. Although this finding is primarily based on one study reporting a substantial effect size (Rodriguez-Ferrer et al., 2022), the use of an online game is noteworthy. By immersing participants in the complexities of mental health, the game could foster empathy towards individuals facing similar struggles, significantly reducing stigma. Previous research has suggested that individuals who have interacted with people with mental illness tend to hold more positive attitudes towards them (Addison & Thorpe, 2004). The finding underscores the value of interactive Internet-based interventions, mimicking real-world contexts to engage participants and facilitate meaningful learning experiences for stigma reduction.

Furthermore, our analysis revealed subtle but significant post-treatment effects of internet-based interventions in improving help-seeking attitudes and intentions. This aligns with the Knowledge-Attitudes-Behaviour continuum (Bettinghaus, 1986), indicating that increased knowledge of mental health issues through Internet-based interventions may foster positive changes in attitudes and behaviors related to help-seeking. However, follow-up effects were not significant in a subset of studies (attitudes: 5/11, p = 0.168; intentions: 4/12 p = 0.135), indicating that while these interventions could initiate positive changes in help-seeking attitudes and behavior towards mental health issues, such changes may not be immediately noticeable or substantial. Therefore, ongoing reinforcement or follow-up sessions are necessary for sustaining these changes. Notably, all 12 studies used the General Help-Seeking Questionnaire (GHSQ) to assess help-seeking intentions, with few reported actual help-seeking behaviours. This discrepancy may be attributed to the primarily low-risk population-based studies that lack opportunities for real-world help-seeking experiences, making intentions a more reflective measure. However, reliance on self-reported scales may introduce bias. Integrating situational or contextual scenario questions into online programs could compensate for the limitations of current assessment methods.

Our study also examined the effects of Internet-based interventions on mental health and found a very small but significant post-treatment effect (g = −0.074) for symptoms of mental disorders. This finding is consistent with the prior review, which also noted a slight alleviating effect on mental health symptoms (Brijnath et al., 2016). One possible explanation for this trend is the normalization process, wherein learning about mental disorders may help individuals perceive their reactions as normal and understandable, thereby contributing to symptom reduction (Tursi et al., 2013). However, it's important to acknowledge that while this effect is statistically significant, its practical implications may be limited. It's crucial to note that MHL interventions alone were insufficient to produce clinically significant improvements in psychological symptoms. Moreover, the predominance of low-risk populations in most studies may be prone to ceiling effects as they were already in relatively good psychological condition, reducing the likelihood of significant improvements from MHL interventions. Furthermore, while many studies mentioned assessments of psychological distress in articles, they often omitted relevant data, and only 2 studies reported follow-up results, making it difficult for us to comprehensively evaluate the effects.

Our study also identified methodological shortcomings in RCTs, which echo previous research. Specifically, there is a noticeable lack of follow-up assessments of interventions, as highlighted in prior studies (Ma et al., 2023; O'Connell et al., 2021). Additionally, the field lacks high-quality studies, necessitating RCTs conducted and reported following the CONSORT statement guidelines (Renwick et al., 2022; Tay et al., 2018; Yamaguchi et al., 2019). Moreover, the considerable heterogeneity of studies due to different conceptualizations and measurements of MHL complicates the interpretation of overall effects in the meta-analysis (Seedaket et al., 2020).

Several advantages underscore the robustness of our analysis. The large participant pool in our meta-analysis (N = 11,582) enhances the reliability of our aggregated findings. By examining diverse populations and categorizing intervention effects into immediate and follow-up outcomes, we promote understanding intervention dynamics over time, facilitating thorough, long-term assessments of intervention effects. Additionally, our subgroup analysis offers nuanced insights into effectiveness across diverse demographics and factors, supporting the adaptation of interventions to various populations and contexts.

Moving forward, our findings have important implications for developing, researching, and implementing Internet-based interventions targeting enhancing MHL. First, we note a predominance of RCTs in developed areas, highlighting an oversight in addressing MHL problems in developing regions. This imbalance undermines the generalizability of findings, underestimates the global mental health burden, and represents missed opportunities for tailored interventions (Dang et al., 2020; Renwick et al., 2022). To address this, more equitable research efforts are needed to reduce mental health disparities and improve intervention implementation globally. Adapting interventions to address specific cultural beliefs and societal norms enhances their relevance and effectiveness for diverse populations. Second, we suggest that intervention strategies be intricately tailored to the varying needs of both low-risk and high-risk populations. For those at low risk, leveraging the Internet's advantages, such as developing more interactive programs, can enhance participant engagement and retention of intervention content. Conversely, for high-risk individuals, even though simple and general interventions could be beneficial due to higher demand, targeted approaches are crucial to address their specific needs. Furthermore, integrating MHL into clinical practice can be effectively achieved by developing and implementing tailored interventions for specific clinical settings, such as primary care. These customized interventions can address the distinct needs and characteristics of patients within these settings, promoting early identification and proactive management of mental health issues. For instance, incorporating brief MHL modules into routine primary care visits can educate patients about mental health, reduce stigma, and encourage help-seeking behaviors. Additionally, these interventions can be designed to include culturally sensitive content and utilize digital tools. By aligning MHL interventions with the specific demands of different clinical environments, healthcare providers can improve the overall quality of mental health care and outcomes. Lastly, incorporating ongoing support, such as peer support components or discussion forums, can provide valuable encouragement and validation, enhancing the long-term effects of interventions, which is especially beneficial for high-risk individuals who may need additional support and guidance.

Several limitations warrant acknowledgment. First, caution is warranted in interpreting the effects of follow-ups due to the limited number of studies addressing this aspect. Second, the inclusion of studies with a high risk of bias may result in an uneven quality of literature in this analysis. Third, we did not investigate the possibility of different findings depending on the type of mental disorder investigated. Last, it is important to note that our study exclusively considered English-language articles.

In conclusion, our meta-analysis of Internet-based MHL interventions reveals significant post-treatment effects on knowledge improvement, help-seeking attitudes, stigma, and mental health promotion, albeit with varying effect sizes and sustainability over time. While these interventions hold promise in enhancing mental health awareness and positive attitudes, challenges remain in maintaining long-term effects, particularly in reducing stigma and promoting help-seeking behaviors. To maximize the effectiveness of Internet-based MHL interventions and advance mental health care on a broader scale, addressing methodological shortcomings and tailoring interventions to the needs of diverse populations are critical.