Assessment of adult psychopathology tends to rely on data obtained mainly from the person being assessed. However, reports by other informants may not agree with self-reports. For example, Meyer et al. (2001) found a mean kappa of only .18 between diagnoses based solely on adults’ self-reports versus diagnoses based on collateral reports. Meyer et al. also found that 70% of personality disorder diagnoses based on clinical interviews with adult patients were wrong when compared with diagnoses made from multiple sources of data. Furthermore, meta-analyses have yielded mean correlations between adult self- and informant-ratings of only .43 for internalizing problems (e.g., anxiety, depression) and .44 for externalizing problems (e.g., aggression, lying) (Achenbach, Krukowski, Dumenci, & Ivanova, 2005).

One method intended to overcome the limitations of self-report instruments is to use validity scales to correct for misleading self-reports. However, several reviews have concluded that validity scales do not improve the validity of results (e.g., Archer, Fontaine, & McCrae, 1998; Barthlow, Graham, Ben-Porath, Tellegen, & McNulty, 2002; McGrath, Mitchell, Kim, & Hough, 2010). In a cross-cultural study that found validity scales to reduce validity, the authors concluded that multimethod approaches are superior to validity scales, because “The best evidence on protocol validity, and the best alternative to the use of validity scales, comes from the comparison of self-report scores with independent assessments,” i.e., reports by informants who know the people being assessed (Piedmont, McCrae, Riemann, & Angleitner, 2000, p. 590). Although standardized multi-informant methods are often used for assessing child psychopathology (De Los Reyes, Thomas, Goodman, & Kundey, 2013), such methods may also improve assessment of adult psychopathology (De Los Reyes, Bunnell, & Beidel, 2013).

One way to advance standardized multi-informant assessment of adult psychopathology is to develop instruments for obtaining collateral reports that parallel self-report instruments. Accordingly, the Adult Behavior Checklist (ABCL) was developed for completion by collaterals to assess many of the same problems that are assessed by the ASR. The ABCL has counterparts of 115 of the 120 ASR problem items but omits items not likely to be ratable by informants (e.g., “heart pounding”). The ABCL also has three items that are not likely to be self-reported (e.g., “stares blankly”). Like the ASR items, ABCL items are rated 0-1-2, based on the preceding 6 months. EFAs and CFAs of ABCL ratings by collaterals of 1,660 U.S. referred and nonreferred 18-59-year-olds yielded counterparts of the eight syndromes found in ASR self-ratings (Achenbach & Rescorla, 2003). The collaterals included spouses/partners, family members, and friends. Correlations between ASR and ABCL scores were .43 for internalizing problems and .44 for externalizing problems, the same as the mean cross-informant correlations found in the meta-analyses of many instruments in many studies (Achenbach et al., 2005).

There are multiple reasons why we cannot assume that assessment instruments developed in one society would be applicable in other societies. As an example, genetic factors affecting behavioral covariation may differ across societies (Way & Lieberman, 2010). Additionally, different behaviors may be encouraged or discouraged in different societies, leading to different groupings of behaviors across societies (Weisz, Weiss, Suwanlert & Chaiyasit, 2006).

To evaluate the applicability of assessment instruments to societies other than where they were developed, they must be used to assess large samples from those societies. To test the degree to which ratings of problems in other societies fit particular syndrome models, CFAs should be applied to the ratings obtained in each society. CFAs of ASR self-ratings by 17,152 18-59-year-olds from 29 diverse societies have supported the eight-syndrome model in each society (Ivanova et al., in press). However, to our knowledge, no published studies report multi-society tests of syndromes of adult psychopathology derived statistically from collateral ratings.

Although focused on personality dimensions rather than psychopathology, EFAs of ratings of the NEO-PI-R (Costa & McCrae, 1992) by collaterals in multiple societies may be instructive. The NEO-PI-R is a 240-item questionnaire for assessing personality dimensions designated as Neuroticism, Extraversion, Openness-to-Experience, Agreeableness, and Conscientiousness. McCrae Terracciano and 78 Members of the Personality Profiles of Cultures Project (2005) performed EFAs on NEO-PI-R collateral ratings for 11,985 18- to 59-year-olds. Data were collected in 50 societies representing a wide range of world regions. Results supported the five NEO-PI-R personality factors in all tested societies, although support was weaker for the five African societies.

Various studies indicate that reports by collaterals provide important information about adult psychopathology beyond what self-reports provide. The ABCL was developed to obtain informants’ ratings of most of the problems that are self-rated on the ASR. Factor analyses of the ABCL and ASR for U.S. samples have yielded eight similar syndromes that enable researchers and clinicians to rigorously compare collateral versus self-reports. CFAs of ASR ratings from 29 societies have supported the eight-syndrome model in self-ratings but the multi-society generalizability of the syndromes scored from the ABCL needs to be tested. The purpose of this study was to test whether the ABCL eight-syndrome model would be supported in collateral ratings of adults in societies that differed in many ways from the U.S. The multi-society CFA findings for ASR self-ratings suggest that ABCL collateral ratings would also support the eight-syndrome model.

The correlated eight-factor model converged for all samples. RMSEAs ranged from .016 (China and Iceland) to .029 (Albania and Lithuania), indicating good fit for all societies (Table 2). The RMSEAs equaled .021, .025, and .027 at the 25th, 50th, and 75th percentiles, respectively. CFIs ranged from .775 for Portugal to .955 for Iceland. TLI values were similar to CFI values within societies, ranging from .768 for Portugal to .954 for Iceland. CFIs and TLIs indicated acceptable to good model fit for all societies, except Portugal and Algeria.

For 10 societies, all 93 tested items had statistically significant loadings on their respective factors. As shown in Table 2, 17 item loadings were not statistically significant (one for Albania and Iceland, two for China, France, Lithuania, and Taiwan, three for Portugal, and four for Algeria). The 17 nonsignificant loadings comprised 1.01% of the 1,669 loadings we tested (92 for Taiwan and 89 for Japan, plus 93 for the remaining 16 societies).

Table 2 presents medians and ranges of factor loadings for each society. The median factor loading ranged from .53 (Algeria) to .79 (Japan), with an overall median of .70. Table 2 also presents medians and ranges for correlations between latent factors across the 18 societies. Median latent factor correlations ranged from .55 in the Argentine sample to .78 in the Albanian sample, with an overall median of .66.

Table 3 presents the means, medians, standard deviations, and ranges of the loadings for each item and for the items comprising each syndrome across the 18 societies. For each item, the median factor loading across societies ranged from .49 (121. Late for appointments) to .87 (54. Feels tired without good reason), with an overall median of .71. For the items comprising each syndrome, median loadings ranged from .65 (Attention Problems) to .74 (Thought Problems).

For four societies, one item had a negative residual, indicating that it was misspecified (item 56c. Nausea, feels sick for Iceland; item 104. Is unusually loud for Japan; item 91. Talks about killing self for Serbia; and item 40. Hears sounds or voices that aren’t there (describe) for the UK). Because the 95% confidence intervals for all four parameters included the admissible parameter space, sampling fluctuations may explain their misspecification (Chen, Bollen, Paxton, Curran, & Kirby, 2001). These four out-of-range parameters comprised only 0.08% of the 5,007 parameters that we estimated (93 thresholds, item loadings, and residuals for 16 societies, 92 thresholds, item loadings, and residuals for Taiwan, and 89 thresholds, item loadings, and residuals for Japan).

The ABCL was developed to meet needs for multi-informant assessment of adult psychopathology by obtaining informant ratings of the same clinical constructs as rated by the people being assessed. Because findings in the society where an instrument originated cannot be assumed to generalize to other societies without empirical evidence, instruments and constructs should be empirically tested in multiple societies. The more diverse the societies in which an instrument's constructs are supported, the more confidence their generalizability warrants. We used CFAs to test the ABCL's eight-syndrome model in 18 societies from such very different regions as Africa, Asia, South America, and Eastern, Northern, and Western Europe.

We found that the eight-syndrome model scored from ABCL ratings by collaterals was supported by RMSEAs ranging from .016 for China and Iceland to .029 for Albania and Lithuania. The small size of the RMSEAs indicated very good fit for the eight-syndrome model in all 18 societies in which collaterals completed ABCLs. The secondary fit indices indicated acceptable to good fit in all societies except Algeria and Portugal. Of the 1,669 loadings that were tested, the median loading was .71, and only 17 (1.01%) were not significant. Of the 5,007 parameters that were estimated, 4 (0.08%) were out of range, but 95% confidence intervals included the admissible range for all 4. Our findings thus supported the same eight-syndrome model for collateral ratings as was previously supported for self-report ratings (Ivanova et al., in press).

Our findings indicate that the eight-syndrome model was supported for operationalizing phenotypes of mutually associated problems rated by collaterals in 18 societies that differed in ethnicity, language, religion, geographical region, and economic, mental health, and political systems. Combined with similar findings for the eight-syndrome model in ASR self-ratings from 17 of these societies (Ivanova et al., in press), our findings support the applicability of the syndromes scored from the parallel collateral- and self-rating instruments for 18-59-year-olds in many parts of the world. Additional items, instruments, and models are likely to be useful. However, the ASR and ABCL items and the eight syndromes scored from them provide a core data language for describing and quantifying psychopathology that can be widely used for research and clinical purposes in conjunction with whatever additional items, instruments, and/or models may be warranted in particular contexts and societies.

The findings support use of the ABCL to assess 18-59-year-olds in terms of the eight-syndrome model scored from collateral ratings in many societies. When the people being assessed rate themselves on the ASR, the parallel items and scales of the two instruments enable clinicians and researchers to identify specific similarities and differences between collateral- and self-reports. To evaluate the degree of deviance indicated by particular scores on each syndrome, research is needed to test differences between scores for different societies, males versus females in different age groups, and collateral- versus self-ratings, as is now being done (Rescorla, 2014;Rescorla et al., 2014). The findings provide a basis for constructing norms that take account of differences in distributions of scores for different societies, each gender/age group, and collateral- versus self-ratings. Computer software will then enable users to display syndrome scale scores in relation to norms appropriate for each society and gender/age group for collateral- and self-ratings.

The present study tested the configural invariance of the ABCL syndrome structure in data from 18 societies. Configural invariance, which is the most fundamental component of measurement invariance, means that an assessment instrument measures the same construct(s) across different populations (e.g., Vandenberg & Lance, 2000). Because our tested model was complex and the WLSMV estimator (which we used to account for the ordered categorical nature of our data) is very computationally intensive, we were unable to successfully execute tests of other components of measurement invariance (i.e., metric, scalar, item residual, and factor), which all require multi-sample modelling.

Although the study reported here extends evidence-based collateral assessment of psychopathology across diverse societies, it is limited to what Pike (1967) called etic research, which applies the same standardized assessments to people in different societies. We did not hypothesize that the ABCL syndrome structure is universal nor that every ABCL item is equally applicable in all societies. Instead, the study's purpose was to test the degree to which the ABCL syndrome structure fit ratings of the ABCL items in 18 specific societies.

Detailed comparisons of scores in different societies (Rescorla, 2014; Rescorla et al., 2014) may reveal societal differences that warrant what Pike (1967) called emic research, which is customized for particular societies to explore society-specific phenomena. Emic research may also be used to test the validity of the ABCL items and syndromes according to society-specific criteria. The findings may argue for use of other items and syndrome models in particular societies.