metricas
covid
Buscar en
International Journal of Clinical and Health Psychology
Toda la web
Inicio International Journal of Clinical and Health Psychology Clinical utility of new cut-off scores for the world health organization ADHD se...
Información de la revista
Vol. 23. Núm. 4.
(octubre - diciembre 2023)
Compartir
Compartir
Descargar PDF
Más opciones de artículo
Visitas
3540
Vol. 23. Núm. 4.
(octubre - diciembre 2023)
Original article
Acceso a texto completo
Clinical utility of new cut-off scores for the world health organization ADHD self-report scale among adolescents in psychiatric outpatient care
Visitas
3540
Susanne Olofsdottera,b,
Autor para correspondencia
susanne.olofsdotter@regionvastmanland.se
susanne.olofsdotter@psyk.uu.se

Corresponding author at: Center for Clinical Research, Uppsala University, Västmanland County Hospital, S-721 89 Västerås, Sweden.
, Águeda Fernández-Quintanaa, Karin Sonnbya,c, Sofia Vadlina
a Center for Clinical Research, Uppsala University, Västmanland County Hospital Västerås, Västerås, Sweden
b Department of Psychology, Uppsala University, Uppsala, Sweden
c Center for Clinical Research and Education, Central Hospital, County of Värmland, Karlstad, Sweden
Este artículo ha recibido
Información del artículo
Resumen
Texto completo
Bibliografía
Descargar PDF
Estadísticas
Figuras (2)
Tablas (3)
Table 1. Socio-demographic and Clinical Characteristics Among Adolescent Psychiatric Outpatients by ADHD Status and Sex.
Table 2. Diagnostic Accuracy of Four New Cut-Off Scores of the ASRS-18 Among Psychiatrically Referred Adolescents.
Table 3. Diagnostic Accuracy of Eight New Sex-Specific Cut-Off Scores of the ASRS-18 Among Psychiatrically Referred Adolescents.
Mostrar másMostrar menos
Abstract

Previous analyses of the proposed cut-off score for the 18-item World Health Organization ADHD Self-Report Scale (ASRS-18) among psychiatrically referred adolescents have shown limited clinical utility. This prospective study examined the diagnostic accuracy and clinical utility of new cut-off scores of the ASRS-18 in a consecutive sample of 111 Swedish adolescent psychiatric outpatients. Using the Kiddie Schedule of Affective Disorders and Schizophrenia (K-SADS) as the reference standard and based on predefined sensitivity and specificity criteria, multiple new general and sex-specific cut-off scores were evaluated. Results showed that clinical utility was greater for sex-specific cut-off scores than for general cut-off scores. The greatest change in pre-test to post-test probability of ADHD diagnosis was observed with a balanced, high-specificity diagnostic cut-off score for girls, where the probability of ADHD increased from 40% pre-test to 82% post-test. The proposed new cut-off levels for the ASRS-18 are useful for the detection and identification of ADHD among adolescents in general psychiatric outpatient settings.

Keywords:
Attention deficit disorder with hyperactivity
Adolescents
ASRS
Measurement
Clinical utility
Child and adolescent psychiatry
Texto completo
Introduction

Attention-deficit hyperactivity disorder (ADHD) has a lifetime prevalence of 4–11% among young people with high persistence rates into adulthood (Danielson, Visser, Chronis-Tuscano & DuPaul, 2018; Polanczyk, Willcutt, Salum, Kieling & Rohde, 2014; Sayal, Prasad, Daley, Ford & Coghill, 2018; Shaw & Sudre, 2021; van Lieshout et al., 2016). When left untreated, ADHD is associated with multiple negative outcomes, including educational underachievement, impaired social interactions, risky behaviours, poor physical health, and increased risk for suicide, addiction, antisocial behaviours and injuries (Becker, Luebbe & Langberg, 2012; Flory, Molina, Pelham, Gnagy & Smith, 2006; Howard et al., 2019; Impey & Heun, 2012; Mohr-Jensen, Muller Bisgaard, Boldsen & Steinhausen, 2019; Pham, 2016; Prasad, West, Sayal & Kendrick, 2018; Pulgaron, 2013; Shaw et al., 2012; Thompson, Molina, Pelham & Gnagy, 2007). Accordingly, early identification and treatment are important in the prognosis of affected youth. However, among adolescents, recognizing ADHD can be challenging. One reason for this is that symptoms of ADHD may be more subtle in adolescents compared to younger children, making it more difficult to detect the disorder. Another reason is that adolescents with ADHD are more likely to have co-occurring mental health disorders which can further complicate identification (Sultan, Liu, Hacker & Olfson, 2021). Identification of ADHD in adolescent girls can be even more difficult, due to a typical symptom presentation of inattention and co-occurring internalizing symptoms rather than hyperactivity and impulsivity and use of compensatory strategies that may disguise ADHD symptoms (Hinshaw, Nguyen, O'Grady & Rosenthal, 2022; Young et al., 2020). Hence, girls with ADHD are less likely to be disruptive and their symptoms can be overlooked by teachers and other informants (Coles, Slavec, Bernstein & Baroni, 2012).

Self-rating scales are a time- and cost-efficient method to collect information on symptoms of ADHD and can be helpful in the detection and identification of impairing levels of inattention and hyperactivity in adolescents. However, to be useful in clinical settings self-report scales must have sound psychometric properties established in settings that allow for generalization to equivalent clinical populations.

One self-report scale commonly used to assess ADHD is the 18-item World Health Organization Adult ADHD Self-Report Scale (ASRS-18) (Kessler et al., 2005, 2007). The ASRS-18 is a Likert-type self-report questionnaire designed to identify symptoms of ADHD. Respondents indicate the frequency of occurrence of 9 attention-deficit symptoms and 9 hyperactivity symptoms during the last 6 months on a 5-point scale, ranging from “never” to “very often”. Kessler and colleagues suggested a summation of scores based on dichotomized item-ratings, yielding a maximum score of 18, and then use a cut-off score of 9 to identify elevated levels of ADHD symptoms. However, alternative scoring methods yielding different total scores for the ASRS-18 have been used in a range of studies based on either adult psychiatric samples or adolescent community samples (Corbisiero, Hartmann-Schorro, Riecher-Rössler & Stieglitz, 2017; Hagen et al., 2017; Kim, Lee & Joung, 2013; Somma, Borroni & Fossati, 2019; van de Glind et al., 2013).

Originally developed for the assessment of ADHD symptoms in adults, the ASRS-18 has been increasingly used among adolescents. Reports from a limited number of evaluations of the ASRS-18 among adolescents have shown adequate psychometric properties in community samples (Adler et al., 2012; Green et al., 2019; Somma et al., 2019). Only one study has examined the performance of the ASRS-18 among adolescent psychiatric outpatients (Sonnby et al., 2015). In their study, Sonnby and colleagues analysed diagnostic accuracy and post-test probability of ADHD based on the dichotomous summation system and the proposed cut-off of 9 (Kessler et al., 2005). The results showed 79% sensitivity (girls 83%; boys 75%) and 60% specificity (girls 60%; boys 59%). However, the cut-off score of 9 showed limited clinical utility because results at or above this score yielded a post-test probability of an ADHD diagnosis of about 50%, leaving clinicians with a high level of uncertainty about the presence of a diagnosis after ASRS-18 administration among adolescent psychiatric outpatients (Sonnby et al., 2015). Furthermore, the ASRS-18 showed better psychometric properties among girls than boys.

The present study provides follow-up analyses on the clinical sample in Sonnby et al.´s study. The overall aim was to determine new, tailored cut-off scores for the ASRS-18 for ADHD assessment purposes among psychiatrically referred adolescent boys and girls. The specific aims were to (a) examine the reliability and known-groups validity of the ASRS-18 and (b) evaluate the diagnostic accuracy and clinical utility of new general and sex-specific cut-off scores.

MethodParticipants and setting

This prospective study was carried out in two child and adolescent psychiatry (CAP) outpatient units in the county of Västmanland, Sweden. Data collection was conducted during 63 predefined recruitment weeks between August 2011 and June 2013. During the study period, 37,494 children and adolescents were living in urban, sub-urban and rural regions within the CAP units’ catchment area. Patients were eligible for study inclusion if they met the age criteria (13–17 years at the time of giving consent to participate), irrespective of reasons for help-seeking. Exclusion criteria were intellectual disability and inadequate Swedish-speaking skills.

During the study period, 202 consecutive patients were eligible for inclusion. Of these, 77 were excluded: 28 declined to participate, 45 were missed as eligible and four did not return after their first appointment. Of the remaining 125 patients, 14 were excluded from analysis because information was missing from their ASRS-18 (Fig. 1). There were no significant differences in sex, age, or proportions of ADHD between those excluded because of missing data and the remaining participants.

Fig. 1.

Flow Chart of Participant Recruitment and Exclusion.

(0.3MB).
Note. ASRS-18 = The World Health Organization ADHD Self-Report Scale, 18 items; K-SADS = Kiddie Schedule for Affective Disorders and Schizophrenia for School-Age Children

The study was approved by the Ethical Review Board in Uppsala, Sweden, in concordance with the Declaration of Helsinki. Written informed consent to participate was obtained at each participant's first CAP appointment.

Measures

The World Health Organization ADHD Self-Report Scale for Adolescents (ASRS-18) (Adler et al., 2012). The ASRS-18 is an 18-item, Likert-type (0 = ‘never’, 4 = ‘always’) questionnaire designed to assess symptoms of ADHD in adolescents. The questionnaire is designed in accordance with DSM-IV diagnostic criteria for ADHD and explores how often symptoms occurred in the last six months. In the current study, a total score of the ASRS-18 was computed by summing up individual participant scores on the 18 items, yielding a total maximum score of 72. The study used the Swedish version of the ASRS-18, with minor adaption of the wording to suit adolescents (Sonnby et al., 2015).

Schedule for Affective Disorders and Schizophrenia for School-Age Children – Present and Lifetime Version 2009 (K-SADS)( Ambrosini, 2000; Kaufman et al., 1997). The K-SADS is a semi-structured diagnostic interview designed to guide clinicians in the assessment of psychiatric disorders in children and adolescents aged 6–18 years. The K-SADS consists of an introductory interview, a screening interview and eight diagnostic supplements covering affective disorders, psychotic disorders, anxiety disorders, behavioural disorders, substance use disorders, eating disorders, tic disorders and autism spectrum disorders. Clinicians score the symptom severity and determine diagnoses based on all available information. Widely used as a diagnostic tool in child and adolescent mental health research, it has been shown that psychiatric diagnoses based on the K-SADS interview conducted at a single visit have excellent validity (Ambrosini, 2000; Fitzpatrick et al., 2012; Jarbin, Andersson, Rastam & Ivarsson, 2017).

Procedure

The ASRS-18 was administered to adolescents as part of a regular assessment procedure at the CAP units. Diagnostic assessment was conducted by clinicians who used the K-SADS interview to gather information on history of psychiatric symptoms and age of onset, current symptoms, and functional impairment in school, at home, with friends and during leisure-time. Clinicians were unaware of the results of the ASRS-18. Adolescents and parents were interviewed together. Diagnoses were based on DSM-IV criteria using information gathered during the diagnostic interview. The K-SADS interview was conducted within 14 days of ASRS-18 completion (M = 3.65, SD = 3.62), with interviewers unaware of ASRS-18 results. The ASRS-18 was re-administered to 14 adolescents (12.6%) directly after the interview because the time frame of 14 days was violated. These reassessments replaced initial assessments in all analyses.

Interviewer training and inter-rater reliability

All interviewers were experienced CAP clinicians with extensive K-SADS interview training before data collection. A free-marginal multi-rater kappa (multi-rater κfree) (Brennan & Prediger, 1981; Randolph, 2005) was chosen to calculate inter-rater reliability (IRR). IRR was calculated before data collection, based on interviewers’ ratings on videotaped model interviews, and showed adequate agreement: multi-rater κfree = 0.88, percent overall agreement (Po) = 0.94. Average multi-rater κfree and Po reliability during data collection was multi-rater κfree = 0.94 (Po = 0.97) for all diagnoses, and multi-rater κfree =0.73 (Po = 0.87) for the composite diagnostic category of any ADHD.

Statistical analysis

Statistical analyses were conducted using IBM Statistical Package for the Social Sciences (SPSS, version 26). A p-value < 0.05 was considered significant. To explore differences between groups for continuous and ordinal data, independent t tests were performed and validated using the non-parametric Mann–Whitney U test. A chi-squared test was used for dichotomous and categorical variables. The internal consistency of the ASRS-18 total score was calculated with Cronbach's alpha.

The diagnoses of ADHD, ADD, and ADHD not otherwise specified were combined into one category: any ADHD. Receiver operating characteristic (ROC) analyses were conducted to examine the ability of the ASRS-18 total score to predict the presence or absence of a diagnosis of any ADHD in the sample. The strength of the prediction is indicated by the area under the curve (AUC), which ranges from 1.0 (perfect association) to 0.5 (no association) (Hanley & McNeil, 1982). Tailored cut-off scores both for the total sample and specifically for boys and girls were determined using the coordinates of the ROC curve plot (sensitivity vs. 1-specificity) with the K-SADS as the reference standard. Four cut-off scores were identified for each of three groups: the total sample, and for boys and girls separately. The following criteria were used to identify cut-off scores: the screening cut off represents the highest score with a minimum sensitivity of 0.90; the diagnostic cut off represents the lowest score with a minimum specificity of 0.90; the balanced screening cut off represents the score that maximizes sensitivity when specificity is set at minimum 0.50; the balanced diagnostic cut off represents the score that maximizes specificity when sensitivity is set at minimum 0.50.

Dichotomized variables were used for calculating sensitivity, specificity, and accuracy (i.e., proportion of participants correctly classified as either having or not having a diagnosis of ADHD). Post-test probabilities for different cut-off scores were calculated using the methods of positive (PPV) and negative (NPV) predictive values and likelihood ratios (LRs). LRs range from 0 to infinity and indicate how much a test result will alter the pre-test probability of diagnosis. LRs above 1 indicate the presence of a diagnosis, whereas LRs below 1 indicate the absence of a diagnosis. LRs equal to 1 lack diagnostic information (McGee, 2002). Results are reported in accordance with the recommendations of the Standards for Reporting of Diagnostic Accuracy (Bossuyt et al., 2015)

ResultsDescription of sample

The study group consisted of 111 psychiatrically referred adolescents (67 girls, 60.4%) aged 12–18 years (M = 15.7, SD = 1.5), predominantly ethnic Swedes (98%) and their parents. The most common causes for help-seeking were symptoms of depression (28.8%), ADHD (27.9%) and anxiety (20.7%). According to the K-SADS, the number of diagnoses per adolescent ranged from 0 to 8 (median = 2, interquartile range = 1–3). The majority of adolescents, 70.3%, had at least one comorbid disorder. Nine (8.1%) had no diagnoses. Fifty-seven (51.4%) adolescents (40.3% of all girls, 68.2% of all boys) were diagnosed with ADHD. Several group differences were observed. First, there was a higher proportion of boys among adolescents with ADHD than among adolescents without ADHD. Second, when girls and boys were analysed separately, mean age differences were observed among girls only; girls with ADHD had a lower mean age than girls without ADHD. Third, among adolescents diagnosed with at least one psychiatric disorder, differences in the mean number of disorders were observed among girls only, with more concurrent disorders among girls with ADHD than girls without ADHD.

In the total sample, the mean ASRS-18 score was 34.3 points (median = 36.0, SD = 13.5) with no statistically significant sex differences (girls: median = 34.6, SD = 14.4; boys: median = 33.8, SD = 12.3) (not shown in tables). There were no differences in total mean scores between those who completed the ASRS-18 before the K-SADS and those who were re-administered the questionnaire after the K-SADS. Table 1 shows socio-demographic and diagnostic characteristics of girls and boys with and without ADHD.

Table 1.

Socio-demographic and Clinical Characteristics Among Adolescent Psychiatric Outpatients by ADHD Status and Sex.

Allp-valueGirlsp-valueBoysp-value
ADHD  Non-ADHD  ADHDNon-ADHD
ADHD  Non-ADHD  n = 27n = 40n = 30n = 14
n = 57  n = 54 
Sex           
Boys, n (%)  30 (52.6)  14 (25.9)  .006       
Age, mean (SD)  15.5 (1.5)  15.9 (1.4)  .122  15.4 (1.4)  16.2 (1.3)  .01715.6 (1.6)15.1 (1.5)  .335
Prevalence of ADHD by type, n (%)             
Any ADHD disorder  57 (100)      27 (100)    30 (100) 
ADHD, combined type  23 (40.4)      13 (48.1)    10 (33.3) 
ADHD, inattentive  25 (43.9)      10 (37.0)    15 (50) 
ADHD, hyperactive-impulsive  1 (1.8)      1 (3.7)    0 
ADHD, not otherwise specified  8 (14.0)      3 (11.1)    5 (16.7) 
Number of disorders, n (%)a      .275b      .211b  .441b
11 (19.3)  13 (28.9)    4 (14.8)  9 (24.3)  7 (23.3)4 (50.0) 
22 (38.6)  14 (31.1)    8 (29.6)  12 (32.4)  14 (46.7)2 (25.0) 
6 (10.5)  11 (24.4)    2 (7.4)  9 (24.3)  4 (13.3)2 (25.0) 
6 (10.5)  3 (6.7)    4 (14.8)  3 (8.1)  2 (6.7)
≥5  12 (21.1)  4 (8.8)    9 (33.3)  4 (10.8)  3 (10.0)
Number of disorders, mean (SD)a  2.9 (1.7)  2.4 (1.3)  .125  3.5 (1.9)  2.5 (1.4)  .0382.3 (1.2)1.8 (0.9)  .213
ASRS-18, mean (SD)  40.2 (12.5)  28.1(11.7)  <0.001  43.6(13.8)  28.5(11.3)  <0.00137.0(10.5)26.9(13.4)  .009
a

Estimates among adolescents diagnosed with at least one psychiatric disorder, n = 102.

b

Fischer's exact test, two-tailed.

Note: K-SADS = Kiddie Schedule for Affective Disorders and Schizophrenia for School-Age Children; ASRS-18: The World Health Organization ADHD Self-Report Scale, 18 items.

Reliability and known-group validity

High internal consistency was observed for the ASRS-18, both in the total sample (Cronbach's α = 0.91) and separately among girls (α = 0.92) and boys (α = 0.89). Known-group validity was examined by analyses of between-group differences in mean ASRS-18 scores. In the total sample, ASRS-18 scores of adolescents with a diagnosis of ADHD were significantly higher, with a large effect size as determined by Cohen's d (Cohen, 1988), compared with the ASRS-18 scores of adolescents without ADHD (d = 0.99). Similarly, analyses of known-group validity by sex showed that the ASRS-18 scores of girls and boys with a diagnosis of ADHD were significantly higher, with large effect sizes, than the ASRS-18 scores of girls and boys without ADHD (d = 1.2 and d = 0.84 for girls and boys, respectively). Means and standard deviations for the ASRS-18 by ADHD status and sex are presented in Table 1.

Diagnostic accuracy and clinical utility of new general and sex-specific cut-off scores

ROC analyses were used to examine the overall ability of the ASRS-18 to predict a diagnosis of any type of ADHD as determined using the K-SADS. The predictive ability, measured as AUC, fell in the fair to good range both in the total sample (AUC = 0.74, 95% CI = 0.65–0.83, p < .001) and in separate analyses for girls (AUC = 0.80, 95% CI = 0.69–0.91, p < .001) and boys (AUC = 0.71, 95% CI = 0.53–0.89, p = .027) (Fig. 2). There were no significant differences between the AUCs of girls and boys (AUC difference = 0.10, Z = 0.88, p = .378).

Fig. 2.

Receiver Operating Characteristic (ROC) Curves for the ASRS-18 Total Score (Range 0–72 Points) in a Clinical Sample of Adolescent Psychiatric Outpatients (Total Sample: n = 111; Girls: n = 67, Boys: n = 44).

(0.15MB).
Note: ASRS-18: The World Health Organization ADHD Self-Report Scale, 18 items

The clinical utility of multiple new general and sex-specific cut offs, derived from ROC analyses using predefined criteria of sensitivity and specificity, were examined. Overall, the diagnostic cut off for girls (ASRS-18-score = 43; sensitivity = 56%; specificity = 90%) and balanced diagnostic cut off for girls (ASRS-18-score = 44; sensitivity = 52%; specificity = 93%) showed the best performance, yielding a moderate increase in the likelihood of a diagnosis of ADHD. An ASRS-18 score above these cut-off scores raised the probability of diagnosis by 39 and 42 percentage points, respectively, i.e., from 40% to 79% and 82%. In contrast, the corresponding diagnostic cut-off scores for boys lacked clinical value because the 95% confidence interval for the LRs for these cut-offs crossed the value of 1 (diagnostic cut off for boys: ASRS-18-score = 45; sensitivity = 17%; specificity = 93%; balanced diagnostic cut off for boys: ASRS-18-score = 38; sensitivity = 56%; specificity = 90%).

In the total sample, all four new general cut-off scores showed some clinical utility and yielded post-test probabilities above 50%, ranging from 60% to 80%, when ASRS-18 scores fell above these cut offs.

The general and sex-specific high-sensitivity screening cut-off scores representing the highest score with a minimum sensitivity of 90% (total sample: ASRS-18-score = 24, sensitivity = 91%, specificity = 35%; girls: ASRS-18-score = 25, sensitivity = 93%, specificity = 35%; boys: ASRS-18-score = 24, sensitivity = 90%, specificity = 43%) all lowered the probability of diagnosis to a similar degree (∼30 percentage points) when ASRS-18 results fell below these cut offs.

The accuracy of the general cut-off scores was fairly consistent at just above 60%, whereas accuracies for the sex-specific cut offs were more varied. The highest accuracy was observed for the balanced diagnostic cut-off score for girls (76%) and the two screening cut-off scores for boys (75%). Estimates of diagnostic accuracy and clinical utility for the new cut-off scores are presented in Tables 2 and 3.

Table 2.

Diagnostic Accuracy of Four New Cut-Off Scores of the ASRS-18 Among Psychiatrically Referred Adolescents.

ASRS-18  Base rate%  AUC (95% CI)  Sens.%  Spec.%  Accuracy%  PPV%  NPV%  LR+ (95% CI)  LR– (95% CI) 
Total sample (n = 111)  51.35  0.75⁎⁎⁎(0.66–0.84)               
Screening cut off (≥24 points)a      91.23  35.19  63.96  59.77  79.17  1.41 (1.14–1.74)  0.25 (0.10–0.62) 
Diagnostic cut off (≥44 points)b      35.09  90.74  62.16  80.00  56.98  3.79 (1.53–9.38)  0.72 (0.58–0.88) 
Balanced screening cut off (≥30 points)c      77.19  51.85  64.86  62.86  68.29  1.60 (1.18–2.19)  0.44 (0.26–0.76) 
Balanced diagnostic cut off (≥40 points)d      50.88  81.48  65.77  74.36  61.11  2.75 (1.49–5.08)  0.60 (0.45–0.81) 
a

cut off with minimum 90% sensitivity.

b

cut off with minimum 90% specificity.

c

cut off with maximized sensitivity at minimum 50% specificity.

d

cut off with maximized specificity at minimum 50% sensitivity.

⁎⁎⁎

p < .001.

Note: ASRS-18: The World Health Organization ADHD Self-Report Scale, 18 items; Base rate: rate of adolescents with a diagnosis of ADHD/ADD; AUC: area under the curve; Sens: Sensitivity; Spec: Specificity; Accuracy: correctly classified; PPV: Positive Predictive Value; NPV: Negative Predictive Value; LR+: Positive Likelihood Ratio; LR–: Negative Likelihood Ratio.

Table 3.

Diagnostic Accuracy of Eight New Sex-Specific Cut-Off Scores of the ASRS-18 Among Psychiatrically Referred Adolescents.

ASRS-18  Base rate%  AUC (95% CI)  Sens.%  Spec.%  Accuracy%  PPV  NPV  LR+ (95% CI)  LR– (95% CI) 
Girls (n = 67)  40.30  .80⁎⁎⁎ (0.69–0.92)               
Screening cut off (≥25 points)a      92.59  35.00  58.20  49.01  87.50  1.42(1.11–1.83)  0.21(0.05–0.86) 
Diagnostic cut off (≥43 points)b      55.56  90.00  72.12  78.94  75.01  5.56(2.07–14.94)  0.49(0.32–0.76) 
Balanced screening cut off (≥30 points)c      81.50  50.00  62.68  52.37  80.01  1.63 (1.14–2.33)  0.37(0.16–0.87) 
Balanced diagnostic cut off (≥44 points)d      51.90  92.50  76.12  82.35  74.01  6.91(2.19–21.78)  0.52(0.35–0.78) 
Boys (n = 44)  68.18  .71⁎⁎ (0.53–0.89)               
Screening cut off (≥24 points)a      90.00  42.86  75.00  77.14  66.67  1.58(0.99–2.52)  0.23(0.07–0.80) 
Diagnostic cut off (≥45 points)b      16.67  92.86  40.91  83.33  34.21  2.33(0.30–18.15)  0.90(0.72–1.11) 
Balanced screening cut off (≥26 points)c      83.30  57.10  75.00  80.64  61.54  1.94 (1.04–3.64)  0.29(0.12–0.73) 
Balanced diagnostic cut off (≥38 points)d      56.70  71.40  61.63  80.95  43.48  1.98(0.82–4.80)  0.61(0.36–1.03) 
a

cut off with minimum 90% sensitivity.

b

cut off with minimum 90% specificity.

c

cut off with maximized sensitivity at minimum 50% specificity.

d

cut off with maximized specificity at minimum 50% sensitivity.

⁎⁎

p < .01.

⁎⁎⁎

p < .001.

Note: ASRS-18: The World Health Organization ADHD Self-Report Scale, 18 items; Base rate: rate of adolescents with a diagnosis of ADHD/ADD; AUC: area under the curve; Sens: Sensitivity; Spec: Specificity; Accuracy: correctly classified; PPV: Positive Predictive Value; NPV: Negative Predictive Value; LR+: Positive Likelihood Ratio; LR–: Negative Likelihood Ratio.

Discussion

This study examined psychometric properties and the clinical utility of new, empirically derived screening and diagnostic cut-off scores for the ASRS-18. We present follow-up analyses based on our previous work on the ASRS-18 using data from an adolescent psychiatric sample aged 12 to 18 years old (n = 111) to examine the ability of tailored ASRS-18 cut-off scores to predict a K-SADS-based diagnosis of any ADHD in a psychiatric outpatient setting.

Consistent with previous reports, the ASRS-18 showed good psychometric properties in our sample, with high reliability and validity (Adler et al., 2012; Somma et al., 2019; Sonnby et al., 2015). The examination of known-groups validity in the present study demonstrated that the ASRS-18 can distinguish between adolescents with and without an ADHD diagnosis in a general CAP outpatient setting.

In our prior evaluation of the ASRS-18 among psychiatrically referred adolescents, we found that the proposed general cut-off score of 9 (score range 0–18 based on the dichotomized scoring method) was of limited clinical value in a CAP setting because it yielded a post-test probability of ADHD of 50%, leaving clinicians with a high degree of uncertainty after ASRS-18 administration (Kessler et al., 2005; Sonnby et al., 2015). In this follow-up study, we conducted ROC analyses in the total sample and separately among girls and boys to obtain four new ASRS-18 cut-off scores for each of these groups, using predefined criteria: a screening cut off with 90% sensitivity with no restraints on specificity, a balanced screening cut off with maximum sensitivity and minimum 50% specificity, a diagnostic cut off with 90% specificity with no restraints on sensitivity, and a balanced diagnostic cut off with maximum specificity and minimum 50% sensitivity.

Overall, most of the new cut-off scores presented in this study were of greater clinical value than the cut-off of 9 proposed by Kessler et al. The majority of the new cut-off scores yielded post-test probabilities above or below chance level with the strongest performances observed for general and sex-specific diagnostic and balanced diagnostic cut-off levels.

To our knowledge, only one other study has explored alternative cut-off scores for the ASRS among adolescents (Somma et al., 2019). For their study, Somma et al. recruited adolescents attending professional education in Italy, but only those reported by their teachers to have externalizing problem behaviours were included in the analyses of diagnostic accuracy. This procedure resulted in a high ADHD prevalence (33%) in the sample, which makes it more akin to CAP outpatient settings, which are characterized by high ADHD prevalence. Using the same ASRS-18 scoring and summation method as the present study, Somma et al. suggested a general ASRS-18 cut-off score of 31 points for the detection of ADHD, yielding a sensitivity of 78%, a specificity of 72%, and post-test probability of ADHD of 58% (25 percentage point increase from pre-test probability) (Somma et al., 2019). We identified two ASRS-18 cut-off scores close to that of the Italian study, namely the general and girl-specific balanced screening cut-offs, both representing a score of 30. However, whereas the sensitivity and specificity of the Italian cut-off were fairly balanced, we observed substantially different estimates with higher sensitivities (77% and 82%, respectively) than specificities (50% and 52%, respectively) for both cut-offs and smaller changes in pre-test to post-test probability of ADHD (∼12 percentage points). The Italian study did not explore sex-specific cut-off scores; however, results of the current study indicate that the clinical usefulness of sex-specific cut-off scores is greater than that of general cut off levels.

It is important to note that the prevalence of ADHD varies greatly between population-based and clinical samples. The high prevalence of ADHD in our sample leads to a higher PPV and a lower NPV for the ASRS-18, compared with community-dwelling adolescent samples. However, the PPV and NPV values are not intrinsic to the ASRS-18 and are influenced by the high ADHD prevalence in outpatient settings. Hence, we included likelihood ratios, a prevalence-independent and clinically useful measure of test performance, in our analyses (Kent & Hancock, 2016).

Strengths and limitations

The present study has several methodological strengths. The sample consisted of consecutively referred unselected adolescent outpatients who were included in the study regardless of the cause of referral to CAP, which decreases spectrum bias and increases the generalizability of our results to general CAP settings. Our study can be considered representative of adolescent populations with high ADHD prevalence rates who are usually referred and treated in CAP across Europe and worldwide (Danielson et al., 2018). Another methodological strength is the assessment procedure where experienced clinicians gathered information on ADHD diagnostic criteria from both adolescents and their parents with the aid of a semi-structured diagnostic interview, the K-SADS interview, which strengthened the accuracy of diagnoses. Finally, our study sample had a predominance of adolescent girls, which supported the identification of cut-offs for girls. Although ADHD is more prevalent among boys than girls (Nussbaum, 2012), and sex is associated with different symptom profiles (Solberg et al., 2018), both groups experience the same negative consequences (Halmoy, Fasmer, Gillberg & Haavik, 2009) and benefit from early diagnosis (Hamed, Kauer & Stevens, 2015). However, ADHD is often underrecognized in girls because of differences in symptom profile; it tends to be characterized by more inattentive and fewer hyperactive/impulsive symptoms compared to ADHD in boys (Gershon, 2002). Therefore, there is an urgent need for suitable screening and diagnostic tools to identify affected girls. We believe our study can contribute to more successful recognition and diagnostic accuracy for ADHD in girls, raising awareness of the need for specific cut-off scores by sex.

Some limitations of the current study must be acknowledged. The most important limitation is the small sample size which may have reduced power to identify sex-specific cut off scores, especially among boys. Moreover, data-driven selection of cut-off levels may lead to overestimation of sensitivity and specificity, with smaller sample sizes increasing the amount of bias (Whiting, Rutjes, Westwood & Mallett, 2013). This study used DSM-IV (American Psychiatric Association, 2000) diagnostic criteria for ADHD diagnosis. Although core diagnostic criteria are unchanged in DSM-5 (American Psychiatric Association, 2013), a change of the minimum number of symptoms required for diagnosis among older adolescents (age 17 or older) may have implications for assessment with the ASRS-18. Furthermore, high rates of comorbidity were noted among participants, especially among girls with ADHD. Comorbidity may imply a symptom overlap, influencing the ratings of the ASRS-18. However, the internal consistency of the ASRS-18 was high in both sexes, indicating adequate reliability for both girls and boys.

Directions for future research

Despite the ASRS-18 being one of the most widely used self-report ADHD measures in psychiatry (Gray, Woltering, Mawjee & Tannock, 2014), there is a lack of studies exploring the clinical utility of alternative cut-off scores for the detection and identification of ADHD in outpatient settings. To our knowledge, our study is the first to analyze alternative cut-off scores for the ASRS-18 among psychiatrically referred adolescents.

The results of the current study suggest that the two new diagnostic cut-off scores for girls showed the best performance among all new cut-off scores, yielding a moderate increase in the probability of an ADHD diagnosis. Further studies in larger samples are warranted to validate these findings.

Conclusion

The ASRS-18 is a valid tool for assessing ADHD among adolescents in CAP settings. Results of this study showed that the ASRS-18 can distinguish between psychiatrically referred adolescents with and without an ADHD diagnosis. The clinical utility of the ASRS-18 in CAP settings can be enhanced by using the total score summation method with the new general and sex-specific cut offs proposed in this study. Multiple sources of information need to be carefully considered when assessing adolescents with ADHD symptoms. No single screening or diagnostic tool can replace a full clinical assessment. The ASRS-18 can be a useful tool for the detection and identification of ADHD among adolescents but must always be used together with other adequate indicators and information sources in the context of ADHD diagnostic assessment (Austerman, 2015).

References
[Adler et al., 2012]
L.A. Adler, D.M. Shaw, T.J. Spencer, J.H. Newcorn, P. Hammerness, D.J. Sitt, et al.
Preliminary examination of the reliability and concurrent validity of the attention-deficit/hyperactivity disorder self-report scale v1.1 symptom checklist to rate symptoms of attention-deficit/hyperactivity disorder in adolescents.
Journal of Child and Adolescent Psychopharmacology, 22 (2012), pp. 238-244
[Ambrosini, 2000]
P.J. Ambrosini.
Historical development and present status of the schedule for affective disorders and schizophrenia for school-age children (K-SADS).
Journal of the American Academy of Child and Adolescent Psychiatry, 39 (2000), pp. 49-58
[American Psychiatric Association 2000]
American Psychiatric Association.
Diagnostic and statistical manual of mental disorders: Dsm-iv-tr.
The Author, (2000),
[American Psychiatric Association 2013]
American Psychiatric Association.
Diagnostic and statistical manual of mental disorders.
5th ed., American Psychiatric Publishing, (2013),
[Austerman, 2015]
J. Austerman.
ADHD and behavioral disorders: Assessment, management, and an update from DSM-5.
Cleveland Clinic journal of medicine, 82 (2015), pp. S2-S7
[Becker et al., 2012]
S.P. Becker, A.M. Luebbe, J.M. Langberg.
Co-occurring mental health problems and peer functioning among youth with attention-deficit/hyperactivity disorder: A review and recommendations for future research.
Clinical Child and Family Psychology Review, 15 (2012), pp. 279-302
[Bossuyt et al., 2015]
P.M. Bossuyt, J.B. Reitsma, D.E. Bruns, C.A. Gatsonis, P.P. Glasziou, L. Irwig, et al.
STARD 2015: An updated list of essential items for reporting diagnostic accuracy studies.
BMJ (Clinical research ed.), 351 (2015), pp. h5527
[Brennan and Prediger, 1981]
R.L. Brennan, D.J. Prediger.
Coefficient kappa: Some uses, misuses, and alternatives.
Educational and Psychological Measurement, 41 (1981), pp. 687-699
[Cohen, 1988]
J. Cohen.
Statistical power analysis for the behavioral sciences.
2nd ed., Lawrence, S. Erlbaum Publishers, (1988),
[Coles et al., 2012]
E.K. Coles, J. Slavec, M. Bernstein, E. Baroni.
Exploring the gender gap in referrals for children with ADHD and other disruptive behavior disorders.
Journal of attention disorders, 16 (2012), pp. 101-108
[Corbisiero et al., 2017]
S. Corbisiero, R.M. Hartmann-Schorro, A. Riecher-Rössler, R.-D. Stieglitz.
Screening for Adult Attention-Deficit/Hyperactivity Disorder in a Psychiatric Outpatient Population with Specific Focus on Sex Differences [Original Research].
Frontiers in Psychiatry, 8 (2017),
[Danielson et al., 2018]
M.L. Danielson, S.N. Visser, A. Chronis-Tuscano, G.J. DuPaul.
A National Description of Treatment among United States Children and Adolescents with Attention-Deficit/Hyperactivity Disorder.
J Pediatr, 192 (2018), pp. 240-246
[Fitzpatrick et al., 2012]
C. Fitzpatrick, N.N. Abayomi, A. Kehoe, N. Devlin, S. Glackin, L. Power, et al.
Do we miss depressive disorders and suicidal behaviours in clinical practice?.
Clinical child psychology and psychiatry, 17 (2012), pp. 449-458
[Flory et al., 2006]
K. Flory, B.S. Molina, W.E. Pelham Jr., E. Gnagy, B. Smith.
Childhood ADHD predicts risky sexual behavior in young adulthood.
Journal of clinical child and adolescent psychology : the official journal for the Society of Clinical Child and Adolescent Psychology, American Psychological Association, Division 53, 35 (2006), pp. 571-577
[Gershon, 2002]
J. Gershon.
A meta-analytic review of gender differences in ADHD.
Journal of attention disorders, 5 (2002), pp. 143-154
[Gray et al., 2014]
S. Gray, S. Woltering, K. Mawjee, R. Tannock.
The Adult ADHD Self-Report Scale (ASRS): Utility in college students with attention-deficit/hyperactivity disorder.
PeerJ, 2 (2014), pp. e324
[Green et al., 2019]
J.G. Green, G. DeYoung, M.E. Wogan, E.J. Wolf, K.L. Lane, L.A. Adler.
Evidence for the reliability and preliminary validity of the Adult ADHD Self-Report Scale v1.1 (ASRS v1.1) Screener in an adolescent community sample.
International journal of methods in psychiatric research, 28 (2019), pp. e1751
[Hagen et al., 2017]
E. Hagen, A.H. Erga, S.M. Nesvag, J.R. McKay, A.J. Lundervold, E. Walderhaug.
One-year abstinence improves ADHD symptoms among patients with polysubstance use disorder.
Addictive behaviors reports, 6 (2017), pp. 96-101
[Halmoy et al., 2009]
A. Halmoy, O.B. Fasmer, C. Gillberg, J. Haavik.
Occupational outcome in adult ADHD: Impact of symptom profile, comorbid psychiatric problems, and treatment: A cross-sectional study of 414 clinically diagnosed adult ADHD patients.
Journal of attention disorders, 13 (2009), pp. 175-187
[Hamed et al., 2015]
A.M. Hamed, A.J. Kauer, H.E. Stevens.
Why the Diagnosis of Attention Deficit Hyperactivity Disorder Matters.
Frontiers in psychiatry, 6 (2015), pp. 168
[Hanley and McNeil, 1982]
J.A. Hanley, B.J. McNeil.
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Radiology, (1982), pp. 29-36
[Hinshaw et al., 2022]
S.P. Hinshaw, P.T. Nguyen, S.M. O'Grady, E.A. Rosenthal.
Annual Research Review: Attention-deficit/hyperactivity disorder in girls and women: Underrepresentation, longitudinal processes, and key directions.
Journal of Child Psychology and Psychiatry, 63 (2022), pp. 484-496
[Howard et al., 2019]
A.L. Howard, T.M. Kennedy, E.P. Macdonald, J.T. Mitchell, M.H. Sibley, A. Roy, et al.
Depression and ADHD-Related Risk for Substance Use in Adolescence and Early Adulthood: Concurrent and Prospective Associations in the MTA.
Journal of abnormal child psychology, 47 (2019), pp. 1903-1916
[Impey and Heun, 2012]
M. Impey, R. Heun.
Completed suicide, ideation and attempt in attention deficit hyperactivity disorder.
Acta psychiatrica Scandinavica, 125 (2012), pp. 93-102
[Jarbin et al., 2017]
H. Jarbin, M. Andersson, M. Rastam, T. Ivarsson.
Predictive validity of the K-SADS-PL 2009 version in school-aged and adolescent outpatients.
Nordic journal of psychiatry, 71 (2017), pp. 270-276
[Kaufman et al., 1997]
J. Kaufman, B. Birmaher, D. Brent, U. Rao, C. Flynn, P. Moreci, et al.
Schedule for Affective Disorders and Schizophrenia for School-Age Children-Present and Lifetime Version (K-SADS-PL): Initial reliability and validity data.
Journal of the American Academy of Child and Adolescent Psychiatry, 36 (1997), pp. 980-988
[Kent and Hancock, 2016]
P. Kent, M.J. Hancock.
Interpretation of dichotomous outcomes: Sensitivity, specificity, likelihood ratios, and pre-test and post-test probability.
Journal of physiotherapy, 62 (2016), pp. 231-233
[Kessler et al., 2007]
R.C. Kessler, L.A. Adler, M.J. Gruber, C.A. Sarawate, T. Spencer, D.L. Van Brunt.
Validity of the World Health Organization Adult ADHD Self-Report Scale (ASRS) Screener in a representative sample of health plan members.
International Journal of Methods in Psychiatric Research, 16 (2007), pp. 52-65
[Kessler et al., 2005]
R.C. Kessler, L. Adler, M. Ames, O. Demler, S. Faraone, E. Hiripi, et al.
The World Health Organization Adult ADHD Self-Report Scale (ASRS): A short screening scale for use in the general population.
[Kim et al., 2013]
J.H. Kim, E.H. Lee, Y.S. Joung.
The WHO Adult ADHD Self-Report Scale: Reliability and Validity of the Korean Version.
Psychiatry Investigation, 10 (2013), pp. 41-46
[McGee, 2002]
S. McGee.
Simplifying likelihood ratios.
[Mohr-Jensen et al., 2019]
C. Mohr-Jensen, C. Muller Bisgaard, S.K. Boldsen, H.C. Steinhausen.
Attention-Deficit/Hyperactivity Disorder in Childhood and Adolescence and the Risk of Crime in Young Adulthood in a Danish Nationwide Study.
Journal of the American Academy of Child and Adolescent Psychiatry, 58 (2019), pp. 443-452
[Nussbaum, 2012]
N.L. Nussbaum.
ADHD and female specific concerns: A review of the literature and clinical implications.
Journal of attention disorders, 16 (2012), pp. 87-100
[Pham, 2016]
A.V. Pham.
Differentiating Behavioral Ratings of Inattention, Impulsivity, and Hyperactivity in Children: Effects on Reading Achievement.
Journal of attention disorders, 20 (2016), pp. 674-683
[Polanczyk et al., 2014]
G.V. Polanczyk, E.G. Willcutt, G.A. Salum, C. Kieling, L.A. Rohde.
ADHD prevalence estimates across three decades: An updated systematic review and meta-regression analysis.
International Journal of Epidemiology, 43 (2014), pp. 434-442
[Prasad et al., 2018]
V. Prasad, J. West, K. Sayal, D. Kendrick.
Injury among children and young people with and without attention-deficit hyperactivity disorder in the community: The risk of fractures, thermal injuries, and poisonings.
Child: care, health and development, 44 (2018), pp. 871-878
[Pulgaron, 2013]
E.R. Pulgaron.
Childhood obesity: A review of increased risk for physical and psychological comorbidities.
Clinical Therapeutics, 35 (2013), pp. A18-A32
[Randolph, 2005]
J.J. Randolph.
Free-marginal multi-rater kappa. An alternative to Fleiss' fixed-marginal multirater kappa.
Joensuu University Learning and Instruction Symposium Joensuu, (2005),
[Sayal et al., 2018]
K. Sayal, V. Prasad, D. Daley, T. Ford, D. Coghill.
ADHD in children and young people: Prevalence, care pathways, and service provision.
The lancet. Psychiatry, 5 (2018), pp. 175-186
[Shaw et al., 2012]
M. Shaw, P. Hodgkins, H. Caci, S. Young, J. Kahle, A.G. Woods, et al.
A systematic review and analysis of long-term outcomes in attention deficit hyperactivity disorder: Effects of treatment and non-treatment.
BMC Medicine, 10 (2012), pp. 99
[Shaw and Sudre, 2021]
P. Shaw, G. Sudre.
Adolescent Attention-Deficit/Hyperactivity Disorder: Understanding teenage symptom trajectories.
Biological Psychiatry, 89 (2021), pp. 152-161
[Solberg et al., 2018]
B.S. Solberg, A. Halmoy, A. Engeland, J. Igland, J. Haavik, K. Klungsoyr.
Gender differences in psychiatric comorbidity: A population-based study of 40 000 adults with attention deficit hyperactivity disorder.
Acta psychiatrica Scandinavica, 137 (2018), pp. 176-186
[Somma et al., 2019]
A. Somma, S. Borroni, A. Fossati.
Construct validity and diagnostic accuracy of the Italian translation of the 18-Item World Health Organization Adult ADHD Self-Report Scale (ASRS-18) Italian translation in a sample of community-dwelling adolescents.
Psychiatry Research, 273 (2019), pp. 753-758
[Sonnby et al., 2015]
K. Sonnby, K. Skordas, S. Olofsdotter, S. Vadlin, K.W. Nilsson, M. Ramklint.
Validation of the World Health Organization Adult ADHD Self-Report Scale for adolescents.
Nordic Journal of Psychiatry, 69 (2015), pp. 216-223
[Sultan et al., 2021]
R.S. Sultan, S.M. Liu, K.A. Hacker, M. Olfson.
Adolescents With Attention-Deficit/Hyperactivity Disorder: Adverse Behaviors and Comorbidity.
Journal of Adolescent Health, 68 (2021), pp. 284-291
[Thompson et al., 2007]
A.L. Thompson, B.S. Molina, W. Pelham Jr., E.M. Gnagy.
Risky driving in adolescents and young adults with childhood ADHD.
Journal of pediatric psychology, 32 (2007), pp. 745-759
[van de Glind et al., 2013]
G. van de Glind, W. van den Brink, M.W. Koeter, P.J. Carpentier, K. van Emmerik-van Oortmerssen, S. Kaye, et al.
Validity of the Adult ADHD Self-Report Scale (ASRS) as a screener for adult ADHD in treatment seeking substance use disorder patients.
Drug and Alcohol Dependence, 132 (2013), pp. 587-596
[van Lieshout et al., 2016]
M. van Lieshout, M. Luman, J.W. Twisk, H. van Ewijk, A.P. Groenman, A.J. Thissen, et al.
A 6-year follow-up of a large European cohort of children with attention-deficit/hyperactivity disorder-combined subtype: Outcomes in late adolescence and young adulthood.
European Child and Adolescent Psychiatry, 25 (2016), pp. 1007-1017
[Whiting et al., 2013]
P.F. Whiting, A.W. Rutjes, M.E. Westwood, S. Mallett, Quadas-2 Steering Group-1.
A systematic review classifies sources of bias and variation in diagnostic test accuracy studies.
Journal of Clinical Epidemiology, 66 (2013), pp. 1093-1104
[Young et al., 2020]
S. Young, N. Adamo, B.B. Asgeirsdottir, P. Branney, M. Beckett, W. Colley, et al.
Females with ADHD: An expert consensus statement taking a lifespan approach providing guidance for the identification and treatment of attention-deficit/hyperactivity disorder in girls and women.
BMC psychiatry, 20 (2020), pp. 404
Copyright © 2023. The Author(s)
Descargar PDF
Opciones de artículo