The role of behaviour problems in screening for mental ill-health in adults with intellectual disability

Abstract

Depression and anxiety are common conditions in adults with intellectual disabilities (ID) and often coexist with behaviour problems. We examined whether behaviour problems can be used to screen for depression and anxiety in ID. Clinical prediction models (CPM) generated from independent databases supported the utility of the depression screen, especially in severe/profound ID. CPM did not support the utility of the anxiety screen at any ID level. Given the paucity of screening tools to improve ascertainment of mental ill-health in ID, the short depression screen would be clinically useful in identifying those who need to undergo a full diagnostic evaluation.

The term behaviour problems is used to describe observable behaviours including irritability, lethargy, stereotypic behaviour, hyperactivity and inappropriate speech.¹ Intellectual disability (ID) is present when an individual has significant limitations in both intellectual functioning and adaptive behaviour (including social, daily living skills) during the developmental period.² In the UK, the National Institute for Health and Care Excellence (NICE; 2016) highlighted the need for suitable tools for the reliable screening and early identification of common mental health problems within adults with ID.³ In this population, mental health problems are common⁴ but often overlooked and untreated as professionals can attribute these symptoms to an individual’s ID.⁵ Individuals with ID can have reduced ability to reflect on internal states, communicate their thoughts and feelings, and understand the content of questions.⁶ Thus, assessing for mental health problems in this population can be challenging, particularly as severity of ID increases.

Behaviour problems are associated with mental health problems in ID.^7,8 The nature of their association remains unclear, i.e., whether behaviour problems are behavioural equivalent symptoms or secondary features of mental health problems.⁹ Due to cognitive limitations, symptoms of mental health problems may present differently in those with ID, compared to those without.¹⁰ For example, irritation may present as assaults, self-injurious behaviours, spitting, yelling and destructive behaviours.¹¹ Clinicians typically rely on self-reporting of internal states to assess mental health. Individuals with ID experience varying difficulties in expressive and receptive language as well as limitations in cognitive and verbal skills, worsened with severity of ID, which make it challenging to articulate their internal states.¹⁰ For those who are non-verbal some symptoms of mental health problems may be undetectable.¹⁰ Behaviour is observable and, as such, it could provide an objective indication of internal states or mental health problems in those with ID, particularly for adults who are non-verbal or have limited communication skills due to severe/profound ID. We aimed to explore whether brief screening tools for depression and anxiety in adults with ID could be developed from measures of behaviour problems.

Method

CPM combines statistical modelling with clinical judgment.¹² Clinical judgement was based on the consensus of three intellectual disability and mental health experts (Professor of Psychiatry of Intellectual Disability, Associate Professor in Intellectual Developmental Disability and MSc student in Clinical Mental Health Sciences). In this study, we followed the recommended steps for building a CPM, namely developing a prediction model in a model generation database, and then checking its validity in an external, independent, database.¹² In both databases, behaviour problems were measured using the Aberrant Behaviour Checklist¹ (ABC), the scale includes 58 items on a 4-point scale ranging from 0 to 3 (0: no behaviour displayed and 3: behaviour displayed often). Mental health was measured using the Mini PAS-ADD.¹³ The model generation database included 245 participants with ID (mean age = 39; male = 64.1%; severe/profound ID = 53.1%; depression = 8.5%; anxiety = 33.7%)¹⁴; the short version of the Adaptive Behaviour Scale measured severity of ID. The model validation database included 100 participants with ID (mean age = 37.1; male = 63%; severe/profound ID = 45%; depression = 6%; anxiety = 30%)¹⁵; a functional/adaptive behaviour scale based on the ICD-10 measured severity of ID.

Behaviour problems from the ABC were identified as potentially related to anxiety and depression using clinical judgment. Receiver operating characteristic (ROC) curves examined the association between each ABC item, depression and anxiety, and an Area Under the Curve (AUC) effect size >.57 was used to select the final set of ABC items to be included in the anxiety and depression screens (see Table 1).¹⁶ A final ROC curve was fitted to examine the association between the screen’s overall score and each outcome. At this stage, an AUC greater than .70 was considered a sufficient threshold to determine the screen’s predictive accuracy.¹⁶ We examined likely moderation of ID severity by re-fitting the final ROC curves separately for those with mild/moderate and severe/profound ID. Moderation was established if the relationship between the screen and the outcome was significantly different between the groups. Analyses were repeated in the validation database, and concordance between the two CPM phases was examined by statistically comparing the magnitude of AUCs between phases.

Table 1.

ABC items included in the depression and anxiety screens and each screen’s association with the outcome.

		AUC value	Standard error	95.0% CI lower	95.0% CI upper
Depression screen
ABC items included in screen^a	Listless and sluggish	.720	.001	.585	.856
	Seeks isolation	.594	.154	.486	.702
	Preoccupied	.584	.202	.456	.713
	Irritable and whiny	.640	.035	.508	.771
	Fixed facial expression	.623	.064	.488	.757
	Uncooperative	.666	.012	.544	.789
	Depressed mood	.749	.000	.640	.857
	Isolates self	.575	.258	.441	.709
	Sits of stands for a long time	.575	.254	.429	.722
	Cries over minor annoyances	.577	.245	.439	.715
	Mood changes quickly	.612	.092	.486	.737
	Unresponsive to structured activities	.648	.025	.519	.776
	Difficult to reach	.660	.015	.522	.798
	Pays no attention when spoken to	.692	.004	.576	.807
Model generation database^b	Overall score in full sample	.749	.064	.623	.874
	Overall score in mild/moderate ID	.648	.087	.478	.818
	Overall score in severe/profound ID	.950	.023	.904	.995
Model validation database^c	Overall score in full sample	.755	.121	.517	.992
	Overall score in mild/moderate ID	.510	.277	.000	1.000
	Overall score in severe/profound ID	.893	.051	.794	.993
Combined sample^d	Overall score in full sample^b	.746	.055	.696	.792
	Overall score in mild/moderate ID^b	.655	.082	.577	.728
	Overall score in severe/profound ID^b	.870	.051	.810	.916
Anxiety screen
ABC items included in screen^a	Irritable and whiney	.601	.010	.525	.677
	Odd/bizarre	.608	.006	.532	.683
	Disobedient	.581	.040	.506	.655
	Yells inappropriate times	.609	.006	.534	.684
	Repetitive speech	.585	.030	.508	.663
	Depressed mood	.582	.038	.504	.660
	Resists any form of physical contact	.578	.048	.500	.656
	Talks to self loudly	.596	.015	.518	.674
	Mood changes quickly	.613	.004	.538	.687
	Unresponsive to structured activities	.572	.066	.495	.650
	Cries and screams inappropriately	.588	.025	.511	.665
	Responds negatively to affection	.592	.019	.514	.670
	Temper tantrums when doesn’t get own way	.570	.074	.495	.646
Model generation database^b	Overall screen score in full sample^b	.674	.037	.602	.746
	Overall screen score in mild/moderate ID^b	.681	.051	.582	.781
	Overall screen score in severe/profound ID^b	.677	.055	.569	.785
Model validation database^c	Overall screen score in full sample^b	.689	.058	.575	.803
	Overall screen score in mild/moderate ID^b	.750	.087	.579	.921
	Overall screen score in severe/profound ID^b	.635	.082	.474	.796
Combined sample^d	Overall screen score in full sample^b	.672	.031	.619	.722
	Overall screen score in mild/moderate ID^b	.713	.042	.637	.781
	Overall screen score in severe/profound ID^b	.638	.047	.561	.709

a

ROC curves were fit between each ABC item and both depression and anxiety. An AUC effect size >.57 was used to select the final set of ABC items to be included in the anxiety and depression screens. Analyses completed in Hassiotis et al (2018) model generation sample.

ROC curves were fit to examine the association between each screen’s overall score and each outcome:

b

Analyses completed in Hassiotis et al (2018) model generation sample (N = 245).

c

Analyses completed in Unwin (2014) model validation sample (N = 100).

d

Analyses completed in the combined sample of the model generation and validation databases (N = 345; Hassiotis et al, 2018; Unwin, 2014.

Primary studies were ethically reviewed and approved by NHS Research Ethics Committee, University of Birmingham Ethical Review Committee and UCL Research Ethics Committee, and written informed consent to participate was obtained from participants. The current study was reviewed and approved by UCL Research Ethics Committee.

Results

The 14-item depression screen was moderately associated to a depression diagnosis AUC = .75, 95.0% CI [.62–.87]). This association was moderated by severity of ID as the depression screen was weakly associated to depression in the mild/moderate ID group (AUC = .65, 95.0% CI [.48–.82]), but strongly in the severe/profound ID group (AUC = .95, 95.0% CI [.90–.99]). Findings were replicated in the validation phase with AUCs of similar magnitude obtained between the generation and validation phase analyses (all p > 0.05).

A weak association was found between the 13-item anxiety screen and anxiety diagnosis (AUC = .67, 95.0% CI [.60–.75]). Subgroup analyses indicated a weak association in the mild/moderate ID group (AUC = .68, 95.0% CI [.58–.78]) and in the severe/profound ID group (AUC = .67, 95.0% CI [.57–.79]). Therefore, the weak association seen between the screen and anxiety diagnosis was not moderated by ID severity. Findings were replicated in the validation phase with AUCs of similar magnitude obtained (all p > 0.05).

Finally, to determine the specificity, sensitivity and optimum cut-off value for the depression screen, data from the two databases were combined to increase power. In the combined sample (N = 345), the depression screen achieved an AUC = .75 (p < 0.001, 95% CI [.70–.79]) (see Table 1), with an optimal cut-off value at 16 (screen score range 0–42). At this cut off, the screen had a sensitivity value of 70.37% and a specificity value of 66.01%. In the severe/profound ID subgroup, the depression screen has an AUC = .87 (p < .001, 95% CI [.81–.92]) (see Table 1) and an optimal cut-off value of 23 (range 0–42). At this cut off, the screen had a sensitivity value of 75.00% and a specificity value of 84.37%.

Discussion

In those with ID, mental health problems develop at similar rates or higher to that of the general population.⁴ Sheehan and colleagues note that different presentations, communication deficits and difficulty accessing services may mean mental illness is under-reported in people with ID.¹⁷ Our findings suggest that the 14-item depression screen could be a useful tool in clinical practice and research. The moderate association with a depression diagnosis across the full sample, and the strong association in those with severe/profound ID, indicates that behaviour problems are a good proxy for depression in adults with ID. Our findings support evidence from previous studies⁷, but also extend it to offer researchers and practitioners a brief screen that does not require specialist training to be completed, and can identify with relative accuracy those who require a more thorough clinical assessment.

By contrast, the weak associations seen for anxiety suggested that behaviour problems cannot offer a reliable tool to screen for anxiety among those with any level of ID. This finding may support previous evidence of a weak/moderate association between behaviour problems and anxiety,⁸ but future research should examine whether a different cluster of behaviour problems may have resulted in stronger associations.

The use of two independent databases is a methodological strength as it allowed for an external independent validation of the initial findings. The measures of severity of ID differed between databases, but the fact that subgroup analyses resulted in similar findings across study phases supports the validity of findings. Participants in the included studies had been selected to present with behaviour problems and, as such, were at high risk for mental health problems. Therefore, CPM results may not be generalisable to the wider adult ID population. Future replication across different samples is warranted. Replication with an appropriate measure of behaviour problems for children and young people with ID would be helpful to explore whether these findings are also relevant to this age group. Current promising findings on the utility of the depression screen to directly support clinical practice respond to NICE recommendations to develop brief and user-friendly screens for adults with ID at risk of psychiatric morbidity.

Ethical considerations

Primary studies were ethically reviewed and approved by NHS and UCL Research Ethics Committees and written informed consent to participate was obtained from participants. The use of data for secondary analysis was reviewed and approved by University of Birmingham Ethical Review Committee. The current study was reviewed and approved by UCL Research Ethics Committee, approval number 12953/002.

Funding

This research received no specific grant from any funding agency, commercial or not-for-profit sectors.

Conflict of interests

The authors have no conflict of interest to declare.