Validity and reliability of a Spanish version of the Standardised Assessment of Personality-Abbreviated Scale (SAPAS) for personality disorder screening in community mental health settings

Abstract

Backgrounds and objectives

Personality disorders (PDs) are an essential public health problem being frequently underdiagnosed in the mental health (MH) services.

Methods

This is a cross-sectional study performed in patients attending a community mental health care out-patient clinic. The Standardised Assessment of Personality-Abbreviated Scale (SAPAS) was administered to 59 consecutively agreeing to take part. SAPAS factorial structure was analyzed utilizing principal component analysis with calculation of Cronbach's α to check for internal consistency. Subsequently, SAPAS criterion validity was established comparing it with the Spanish version of the International Personality Disorder Examination (IPDE) by performing a ROC curve to determine sensitivity, specificity, positive and negative predictive values for different cut-off points. Finally, correlations were estimated between the SAPAS scores and dimensional scores for the different IPDE PD types.

Results

Three factors were achieved, each one closely related to the three DSM-5 PD groups (A–C) explaining 64.1% of total variance in the model. SAPAS internal consistency measured with Cronbach's α was 0.66. The area under the curve (AUC) was found to be 0.89. A cut-off point of 5 showed 84% sensitivity and 79% specificity, correctly classifying 81.5% of the individuals.

Conclusions

SAPAS is a valid and reliable PD screening instrument in patients attending community mental health settings.

Introduction

Personality disorders (PDs) are a serious public health problem given they are difficult to treat and associated with a significant increase in morbidity and mortality.¹ Persons with PD have social, occupational, recreational, and global functioning difficulties. They also have an increased risk of depression or anxiety² and an earlier-than-expected onset of disability.³ They also show higher cardiovascular risk,^4,5 and more substance abuse, adversity during childhood, especially abuse and neglect, are also risk factors for its appearance.⁶ Adversity during childhood, especially abuse and neglect, are also risk factors for PD⁶ which is also correlated with other psychosocial difficulties such as higher rates of divorce or separation and unemployment.⁷ The prevalence of PD in the general population ranges between 5.9% and 22.5%.⁸ Thus it was estimated at 6% by the World Mental Health Survey⁹ and at 10.5% and 13.5%, respectively, by other population studies (i.e., Samuels et al.⁶ and Torgensen et al.¹⁰). The prevalence of PD in the province of Granada (Spain) was estimated to be 3.6%.¹¹ In clinical populations, the prevalence is much higher, reaching figures up to 45.5%.¹² However, PD is generally underdiagnosed in clinical populations even when it as a risk factor for other medical and mental diseases, especially if they are not detected adequately. There is evidence that PD increases the use of health services, especially those of Primary Health Care (PHC).⁵ The diagnosis of PD is time-consuming, and the costs are high.

For these reasons, a simple, valid, reliable, and self-administered screening scale of PD could be clinically useful. Until now, there is a limited number of instruments useful for the diagnostic screening of any type of PD in the Spanish population. One of them is the Spanish version¹³ of IPDE (International Personality Disorder Examination),¹⁴ which is a somewhat lengthy self-administered screening instrument with 59 items, yet it has proven to be reliable and acceptable for clinicians in different countries.¹⁴ The Standardised Assessment of Personality-Abbreviated Scale (SAPAS)¹⁵ could also be an useful instrument since it has shown its validity and reliability as a PD screening questionnaire for English-speaking clinical populations. The SAPAS is a short self-administered PD screening scale composed of just 8 dichotomous yes/no questions that derive from a much larger scale, the Standardised Assessment of Personality (SAP).¹⁶ The SAPAS was validated in a sample of English psychiatric patients, where it has shown excellent psychometric properties, correctly classifying 90% of cases, exhibiting a sensitivity of 0.94 and a specificity of 0.85.¹⁵ It has also been validated in adolescent patients,¹⁷ patients with substance abuse¹⁸ and as a screening tool for the general population¹⁹ even at a lower predictive value of 58%. The SAPAS has not been validated so far in any Spanish-speaking clinical population. Our goal is to improve the detection of PD in mental health community settings by using a short tool able to easily flag potential PD cases as PD is very frequently underdiagnosed if its diagnosis is exclusively based on unstructured clinical evaluations.¹²

We hypothesize that the Spanish version of SAPAS can be a valid and reliable instrument for PD screening in the clinical population attending to community mental health services in Spain.

Material and methods

Design

This is a cross-sectional study to test validity and reliability of a translated and back-translated Spanish version of the SAPAS.

Sample size

A sample size of 59 individuals was calculated as providing sufficient power under the assumptions of a PD prevalence of 4% in the general population of Granada, considering a precision of 5% and using 95% as 1 − α value.

Inclusion criteria

Adult individuals who provided informed consent to take part whilst they were under follow-up at the community mental health facilities within San Cecilio University Hospital in Granada.

Exclusion criteria

An inability of either provide an informed consent or to fully grasp purpose of the study and/or the questions asked in our study below-described study protocol.

Variables and measure instruments

• 1.

  Sociodemographic variables such as age, sex, educational level and urbanicity.

• 2.

  Full medical history using clinical standards.

• 3.

  The SAPAS which a self-administered screening scale for PD consisting of 8 dichotomous-answer (yes/no) questions.¹⁵

• 4.

  Diagnosis of PD using the ICD10-based IPDE questionnaire.¹⁴ The IPDE questionnaire is a semi-structured interview for the ascertainment of PD diagnoses. It was developed by the WHO and internationally validated. It consists of an initial self-administered screening questionnaire conformed by 59 dichotomous-answer (yes/no) items followed by a semi-structured hetero-administered interview consisting of 67 questions that can be scored between a range of 0 through to 2 points. The IPDE provides a negative, probable, or positive diagnosis of PD using a 3/4 points cut-off for each probed PD providing additional dimensional scores for each PD type. We used the validated Spanish version.¹³

Procedure

The SAPAS scale was firstly translated into Spanish by a bilingual psychiatrist, taking into account both cultural and linguistic aspects, and such version was hence back-translated from Spanish to English by another bilingual psychiatrist so that a fully acceptable degree of concordance with the original English version could be verified. Subsequently, two psychiatrists reviewed the resulting Spanish version of the scale to confirm appropriate content validity, and the final questionnaire was reached. Consecutively attending patients were recruited by one psychiatrist and one clinical psychologist who collected sociodemographic information, the medical history and the IPDE screening.¹³ Then the patients answered the SAPAS scale and subsequently a psychiatrist blinded to previous results administered the rest of the diagnostic IPDE interview.

Statistical analyses

Initially, we analyzed the SAPAS internal consistency using Cronbach's α and checked the requirements for factor analysis using the Kaiser–Meyer–Olkin test (KMO test) and the Bartlett sphericity test. Subsequently, the factorial structure was determined using principal components analysis using Varimax rotation. The concurrent validity of the SAPAS was then determined by comparing it with that of the semi-structured IPDE interview. The concurrent validity between the SAPAS and the IPDE was obtained utilizing a ROC curve considering as gold standard “positive” IPDE PD diagnosis. Sensitivity, specificity, positive predictive value, negative predictive value, and cut-off points for PD diagnosis were then calculated. Finally, Pearson's correlation was used to explore the associations between SAPAS scores and dimensional scores of the different PD subtypes identified by the IPDE. All statistical calculations were performed with the SPSS 24 program.

Results

The sample

Sixty individuals were finally included of which 60% were women. The mean age was 37.5 (SD: 12.4). The most frequent marital status was being single (55%), followed by being married or living with a partner (35%), the rest situations amounting to 10%. 41.7% of the sample had achieved primary or lower educational levels, whilst 36.7% had completed university studies and the remaining 21.6% had other educational levels. Regarding employment status, 36.7% of the sample was unemployed, 30% employed, 16.7% were retired or on long-term sick leave and 16.6% had other employment stata. As for recorded psychiatric diagnosis, 20% had a diagnosis of affective disorder (20%), 16.7% had an anxiety disorder, 15% were diagnosed as PD and 5% had psychosis, whilst the the most prevalent recorded diagnosis “unspecified/delayed” (43.3%). Administering the IPDE interview identified some 36.7% potential cases of PD. Thus, the most frequently identified PD type was emotionally unstable borderline type (11.7%), followed by “unspecified” PD (10%), anankastic PD (8.3%), paranoid PD (6.7%), dependent PD (6.7%), anxious PD (5%), dissocial PD (5%), emotionally unstable impusive type PD (3.3%), histrionic PD (1.7%) and schizoid PD (1.7%). 11.7% of the sample had more than one PD diagnosis (i.e., four patients presented two concurrent PD diagnoses, two patients had three different PD diagnoses and one patient was identified as having four different types of PD. 100% of the patients in our sample were on drug treatment, with both antidepressants (74%) and benzodiazepines (74%) as the most frequently used drugs.

Construct validity

The sample fulfilled conditions for adequate factorial analysis exploration (KMO test = 0.611 and Barlett sphericity test = 87.913; p ≤ 0.0001). Hence, after performing principal component analysis (PCA) a three-factor solution explaining 64.1% of the total variance was obtained. The first factor (eigenvalue = 2.06) grouped the first three items of SAPAS: “difficulty in making friends”, “usually loner”, and “generally trusting others.” The second factor (eigenvalue = 1.56) included items number 6 (normally worried) and 7 (generally dependent). Finally, the third factor (eigenvalue = 1.50) was composed by items 4 (normally losing your temper) and 5 (normally impulsive) (see Table 1).

Table 1.

Construct validity of the Spanish SAPAS (factorial analysis).

SAPAS item	Factor 1	Factor 2	Factor 3
Difficulty making friends	0.722	0.241	−0.051
Usually a loner	0.809	0.244	0.160
Generally trusting others	0.842	0.165	−0.124
Normally loses temper easily	0.313	0.132	0.780
Normally impulsive	0.053	0.067	0.809
Normally a worrier	0.047	0.809	0.044
Generally depending on others a lot	0.124	0.793	0.041
Generally a perfectionist	0.242	0.332	−0.442
Eigenvalues	2.062	1.561	1.507
% Variance	25.774	19.511	18.835
% Total model variance	64.119

Concurrent validity and cut-off points for SAPAS score

The SAPAS cut-off points to identify a probable PD diagnosis were determined using the IPDE as the gold standard. In the event, an area under the curve of 0.89 was obtained (IC = 0.79–0.98; p ≤ 0.0001) (see Figure 1). A score of 4 points in the SAPAS showed a sensitivity and specificity of 89% and 58%, respectively. Additionally, using a score of 5 points and above provided sensitivity and specificity figures of 84% and 79%, respectively (see Table 2). Overall, total the SAPAS scores showed a positive and statistically significant correlation with dimensional scores for each individual PD type diagnosed using the IPDE with correlation estimates as follows: paranoid PD (r = 0.48; p ≤ 0.0001), schizoid PD (r = 0.41; p ≤ 0.002), dissocial PD (r = 0.43; p ≤ 0.001), impulsive type emotionally unstable PD (r = 0.55; p ≤ 0.0001), borderline type emotionally unstable PD (r = 0.67; p ≤ 0.0001), histrionic PD (r = 0.51; p ≤ 0.0001), anankastic PD (r = 0.43; p ≤ 0.001), anxious PD (r = 0.56; p ≤ 0.0001) and dependent PD (r = 0.54; p ≤ 0.0001).

Figure 1.

The area under the curve, sensitivity, and false positives for the different cut-off points for the SAPAS score.

Table 2.

Concurrent validity: sensitivity, specificity and positive (PPV) and negative predictive (NPV) and Spanish SAPAS cut-off points.

Cut-off point	Sensitivity	Specificity	PPV	NPV	Correctly classified (%)
2 or more	0.98	0.20	0.55	0.91	59%
3 or more	0.93	0.38	0.60	0.84	65.5%
4 or more	0.89	0.58	0.68	0.84	73.5%
5 or more	0.84	0.79	0.80	0.83	81.5%
6 or more	0.70	0.93	0.91	0.76	81.5%
7 or more	0.36	0.99	0.97	0.61	67.5%

Reliability

A Cronbach's α was calculated to analyze the internal consistency of the scale, and its value was 0.66 (see Table 3 for further details).

Table 3.

Reliability analysis of the Spanish SAPAS (Cronbach's alpha).

SAPAS item	Cronbach's alpha if item omitted
Difficulty in making friends	0.591
Normally a loner	0.546
Generally trusting others	0.613
Normally loses the temper easily	0.611
Normally impulsive	0.659
Normally a worrier	0.639
Generally depending on others a lot	0.630
Generally a perfectionist	0.680
Total Cronbach's alpha	0.665

Discussion

We demonstrate that our Spanish version of the SAPAS is a valid and reliable instrument for PD screening in patients with a diagnosis of mental disorder attending to community mental health services. To the best of our knowledge, we do not know of any PD screening scale shorter than this making it a potentially excellent tool for PD screening in clinical settings. It is also a self-administered instrument and easy to interprete. PCA revealed the existence of 3 factors that adequately explained a large part of the variability in SAPAS scores and showed an adequate degree of agreement with all PD types and clusters (A–C).²⁰ The first factor that could be named schizo-paranoid groups items 1, 2, and 3 in the scale, and it evaluates aspects that have fundamentally to do with the psychopathology of schizoid (items 1 and 2) and paranoid (item 3) PD types, respectively. The second factor, anxious-dependent, encompasses items 6 and 7 and measures variables such as the degree of worry and dependence, closely related to cluster C PDs. The third factor incorporates SAPAS items 4 and 5 exploring impulsivity and anger and is more related related to cluster B PDs. Finally, item 8 that evaluates perfectionism, did not aggregate with any previous factors but it is meant to explore obsessional traits. The internal consistency of the scale was fairly acceptable, showing very similar levels if compared with the original English version,¹⁵ taking into account that the SAPAS evaluates with just eight questions a complex construct with multiple subtypes, such as PD.²⁰ The concurrent validity of the SAPAS was excellent when compared with that of a validated questionnaire used as gold standard (IPDE).^13,14

The area under the curve was 0.886, comparable to that of 0.94 obtained in the original SAPAS validation.¹⁵ The sensitivity of the scale was excellent, showing values similar to the ones reported for the English version, although the specificity was lower for cut-off points lower than 5.¹⁵ A cut-off point of 4 gives us almost 90% sensitivity, which makes it an excellent screening tool. Although a cut-off point of 5 still offers 84% sensitivity decreasing false positives from 42% to 21% and correctly classifying 81.5% of the patients. We believe that the latter cut-off point is a more balanced option as it maintains high sensitivity and significantly decreases false positives, whils the 4 points cut-off could be ideal for screening purposes. In the English version,¹⁵ cut-off point 3 offered 94% sensitivity and the 4 points cut-off point demonstrated 82% sensitivity.

Besides demonstrating excellent concurrent validity, the SAPAS scores were moderately and statistically correlated with the IPDE dimensional scores, showing a positive agreement between both scales. This critical finding is in concordance with findings that the English version of SAPAS also showed a dimensional correlation between the SAPAS scores and the PD clusters A and C.²¹ It is especially remarkable the finding that the highest correlation (r = 0.67) was with borderline personality disorder since this kind of PD is particularly severe, causing a significant impact on personal and social functioning.²² Thus, we pose that the SAPAS could also be a useful tool to screen this particularly troublesome PD type, a hypothesis that should be proved in further studies. The finding that the SAPAS could even capture the dimensional structure of PD might be of particular interest in population studies, or indeed clinical settings, where it is necessary to estimate individuals under subclinical thresholds. We reckon that this is a potential and practical advantage in favor of its utility, especially in a moment when the evaluation and classification of PD seems to be moving from a categorical to a more dimensional approach.²³ There are several scales in the Spanish language that evaluate personality traits^24,25 personality dimensions^26–28 or PD,²⁸ but we do not know of any previous validated PD screening questionnaires in the Spanish language other than the more protracted IPDE screening section.¹³

Limitations

We think it is challenging to extrapolate these findings to populations where the prevalence of PD might be lower, such as in Primary Care or community studies. It is very likely that in these contexts, the scale had less discriminative power, and the optimal cut-off points might differ. Hence the Spanish SAPAS must also be validated for such populations as the previous validation of the English version in the general population has indicated.¹⁹ Another limitation, consubstantial within these types of studies, has to do with the difficulty in finding the ideal gold standard to explore concurrent validity. Moreover, regarding construct validity, PDs are not all that a stable construct as is the case for other better-established mental disorder categories.²⁰

Conclusions

The Spanish version of SAPAS proved to be a valid and reliable PD screening instrument at mental health services settings.

Ethical considerations

All participants gave their informed consent to participate in the study, and it was approved by the Andalusian Ethics Committee of Biomedical Research (PEIBA).

Funding

The present investigation has not received any specific grant from granting agencies of either the public sector, commercial sector or non-profit entities.

Conflicts of interest

The authors declare none conflict of interest.