In patients with severe mental disorders outcome measurement should include symptoms, cognition, functioning and quality of life at least. Shorter and efficient instruments have greater potential for pragmatic and valid clinical utility. Our aim was to develop the Spanish UPSA Brief scale (Sp-UPSA-Brief).
Materials and methodsNaturalistic, 6-month follow-up, multicentre study. 139 patients with schizophrenia, 57 with bipolar disorder and 31 controls were evaluated using the Sp-UPSA, CGI-S, GAF, and PSP. We conducted a multivariate linear regression model to identify candidate subscales for the Sp-UPSA-Brief.
ResultsThe stepwise regression model for patients with schizophrenia showed that communication and transportation Sp-UPSA subscales entered first and second at p<0.0001 (R2=0.88, model df=2, F=395.05). In patients with bipolar disorder transportation and communication Sp-UPSA subscales entered first and second at p<0.0001 (R2=0.87, model df=2, F=132.32). Cronbach's alpha was 0.78 in schizophrenia and 0.64 in bipolar patients. Test–retest was 0.66 and 0.64 (p<0.0001) respectively. Pearson correlation coefficients between Sp-UPSA and Sp-UPSA-Brief were 0.93 for schizophrenia and 0.92 for bipolar patients (p<0.0001). The Sp-UPSA-Brief discriminated between patients and controls. In schizophrenia patients it also discriminated among different levels of illness severity according to CGI-S scores.
ConclusionThe Sp-UPSA-Brief is an alternate instrument to evaluate functional capacity that is valid and reliable. Having a shorter instrument makes it more feasible to assess functional capacity in patients with severe mental disorders, especially in everyday clinical practice.
En los pacientes con trastornos mentales graves, la medición de los desenlaces clínicos debería incluir como mínimo la evaluación de los síntomas, la cognición, la funcionalidad y la calidad de vida. Desde un punto de vista pragmático, los instrumentos abreviados y eficientes presentan mayor potencial en cuanto a utilidad y validez clínica. Nuestro objetivo fue desarrollar la versión española de la escala UPSA abreviada (SP-UPSA-Brief).
Material y métodosEstudio multicéntrico de diseño naturalista y con 6 meses de seguimiento. Se evaluaron 139 pacientes con esquizofrenia, 57 con trastorno bipolar y 31 controles mediante el empleo de la Sp-UPSA, la CGI-S, la GAF, y la PSP. Se realizó un modelo de regresión lineal multivariante para identificar las subescalas candidatas para la Sp-UPSA-Brief.
ResultadosSegún el modelo de regresión por pasos para los pacientes con esquizofrenia las subescalas de comunicación y de transporte de la Sp-UPSA fueron la primera y la segunda introducidas, con p<0,0001 (R2=0,88, modelo df=2, F=395,05). En los pacientes con trastorno bipolar las subescalas de transporte y de comunicación de la Sp-UPSA se introdujeron en primer y segundo lugar, respectivamente, con p<0,0001 (R2=0,87, modelo df=2, F=132,32). El alfa de Cronbach fue de 0,78 para la esquizofrenia y de 0,64 en pacientes bipolares. El test–retest fue de 0,66 y de 0,64 (p<0,0001), respectivamente. Los coeficientes de correlación de Pearson entre la Sp-UPSA y la Sp-UPSA-Brief fueron de 0,93 para la esquizofrenia y de 0,92 para los pacientes bipolares (p<0,0001). La Sp-UPSA-Brief discriminó entre los pacientes y los controles. En los pacientes con esquizofrenia también discriminó entre los grados de gravedad de la enfermedad de acuerdo con las puntuaciones CGI-S.
ConclusionesLa escala Sp-UPSA-Brief es un instrumento alternativo, válido y fiable, para la evaluación de la capacidad funcional. Su formato abreviado lo convierte en un instrumento más útil a la hora de evaluar la capacidad funcional de los pacientes con trastornos mentales graves, sobre todo en la práctica clínica diaria.
Schizophrenia and bipolar disorder are severe mental disorders leading to functioning and quality of life impairments in several life domains.1–4 Consequently, the main long-term therapeutic goals of schizophrenia should go beyond the symptoms and include the improvement of patients’ psychosocial functioning and quality of life5–9 in agreement with the Food and Drug Administration definition of “treatment benefit”10 that refer to the effect of treatment on how a patient survives, feels or functions. This means that outcome measurement should not be limited to the assessment of symptoms but also includes at least measures of other domains such as cognition, functioning, and quality of life. Since multiple constructs should be systematically assessed, there is a growing need for new assessment tools capable of being both shorter and “diagnostically sensitive” that potentially minimise the burden placed on patients and clinicians, making them easier to incorporate into both research and daily clinical practice settings.
The University of California San Diego Performance-Based Skills Assessment (UPSA) is a performance-based instrument of functional capacity that plays a leading role as a co-primary measure for trials designed to improve cognition in schizophrenia11 as it was included in the Consensus Battery for Clinical Trials in Schizophrenia (NIMH-MATRICS initiative).12 Functional capacity instruments, unlike those evaluating real world functioning (i.e., PSP, FAST, etc.), evaluate the capacity of persons with schizophrenia to adequately perform the skills necessary for daily functioning such as paying bills, scheduling medical appointments or complying with a work schedule under optimal conditions.13,14 Thus it provides an important new perspective on functional outcome measures in schizophrenia and severe mental illnesses14 since it minimises the potential influence of environmental and social factors as well as informant biases and lacunae in knowledge in measuring real-world functioning.15
Recently two short-forms of the UPSA have been developed by using techniques of factor analysis16 and multiple linear regression analysis17 for patients with severe mental illness and for patients with mild cognitive impairment (MCI) and Alzheimer disease (AD) respectively. These two forms demonstrated to be valid, reliable and time-efficient measures of functional capacity for these patients.16,17
Since we recently culturally adapted and validated the Spanish version of the UPSA scale -Sp-UPSA-,18 we further designed a study to develop and validate a short version, the Sp-UPSA-Brief scale.
Materials and methodsStudy designThis is a naturalistic, 6-month follow-up validation study conducted at 7 centres in Spain. It was approved by the Ethics Committee for Clinical Research of one of the centres, Hospital Universitario Central de Asturias, Oviedo, Spain and it is in accordance with 1975 Declaration of Helsinki, as revised in 1983. Written informed consent was obtained from all subjects prior to enrolment.
SubjectsParticipants included 139 patients with stable schizophrenia, 57 patients with stable bipolar disorder and 31 healthy controls. Stable was defined as those patients who were clinically stable and did not require any change in their current pharmacological treatment during the 3 past months.
Patients’ inclusion criteria were (1) age≥18 years; (2) ICD-10 diagnosis of schizophrenic disorder or bipolar disorder; (3) currently on treatment for his/her illness; and (4) written informed consent to participate in the study. Controls’ inclusion criteria were 1) age≥18 years; (2) without mental or relevant physical disorder; (3) CGI-S score=1; and (4) written informed consent to participate in the study. In controls, the presence of no relevant physical or mental illness was determined by clinicians valuing the candidates’ answer to the question if he/she has a physical or mental illness. Clinicians determined whether illnesses reported were relevant or not. Moreover, in the case of mental illness, after the interview with the subject clinicians should complete the CGI-S and the subject was included only if the score was 1. Exclusion criteria were designed to be minimal, due to the naturalistic design of the study, and only included patients with intellectual developmental disorder, acquired brain injury, or refusal to participate in the study.
Clinical measuresDemographic and clinical data collection as well as assessments was administered at baseline and 6-month follow-up. For each patient, both assessments were made by the same clinician. Severity of the illness status was assessed by means of the Clinical Global Impression, Severity scale -CGI-S-.19 Functional capacity was assessed using the Spanish version of the University of California San Diego Performance-Based Skills Assessment (Sp-UPSA),18 and real-world functioning using the Global Assessment of Functioning (GAF)20 and the Spanish version of the Personal and Social Performance scale (PSP).21 Information was collected from the patients themselves and, when possible, from the main caregiver.
Statistical analysisThe statistical analyses were done using SPSS 17.0. The two-tailed level of significance used was 0.05. Chi-square and ANOVA (Duncan post hoc) were used to determine statistically significant differences according to demographic and clinical status.
We conducted a multivariate linear regression model to identify candidate subscales for the Sp-UPSA-Brief. Specifically, a stepwise regression analysis was carried-out with the following independent variables: age, years of education, and the four Sp-UPSA domains, i.e., finances, communication, planning recreational activities, and transportation, and with the Sp-UPSA total score as the dependent variable. Age and years of education were forced to enter in the regression model as they had been found to be related with functional outcome measures.17 We performed the analysis for the healthy controls and for patients with schizophrenia and bipolar disorder separately.
The following distribution characteristics of the Sp-UPSA-Brief scores were evaluated: (1) Mean and standard deviation, (2) skewness and kurtosis to measure the shape of the distributions (values of skewness and kurtosis +/−1 were considered as good), (3) coefficient of variation (standard deviation/mean), and (4) ceiling and floor effects in the control group and in the patients’ groups respectively (number of controls with scores greater than 95% and number of patients with scores smaller than 5%).
In addition, internal consistency for the total Sp-UPSA-Brief using the Cronbach's alpha coefficient, and test–retest reliability by means of the Pearson correlation coefficient between Sp-UPSA-Brief total scores at baseline and at 6-month follow-up (49 schizophrenia patients and 25 bipolar patients respectively) were determined. As a measure of the level of practice effects,15 effect sizes for changes in performance at month 6 were calculated using the Cohen's d coefficient. Construct validity was calculated using the Pearson correlation coefficient between the Sp-UPSA and Sp-UPSA-Brief scales. Pearson correlation coefficients between the total Sp-UPSA-Brief score and total scores on the PSP and GAF were also determined under the hypothesis that a moderate r coefficients will be found as they are related but different constructs.
In order to determine the discriminant validity, patients with schizophrenia were classified in four groups based on their CGI-S scores: minimally ill (CGI-S=1–2), mildly ill (CGI-S=3), moderately ill (CGI-S=4) and severely ill (CGI-S=5–7). Patients with bipolar disorder were classified in three groups based on their CGI-S scores: minimally ill (CGI-S=1–2), mildly ill (CGI-S=3) and moderately-markedly ill (CGI-S=4–5). An ANOVA test (with Duncan's post hoc comparisons) was used to identify statistically significant differences in the Sp-UPSA-Brief scores according to clinical status (schizophrenia, bipolar or healthy control) and severity groups. The diagnostic accuracy of the Sp-UPSA-Brief to discriminate between both group of patients and controls was analysed using the receiver operating characteristic (ROC) curve analysis.
ResultsDemographic characteristics of the sampleA more detailed description of the sample characteristics can be found elsewhere.18 Briefly, patients with bipolar disorder were older than schizophrenic patients and healthy controls (46.8 versus 39.9 and 39.2 respectively, F=9.485, p<0.0001), and a smaller proportion of them were males (56.1% versus 73.4% and 65.6% respectively, Chi-square=10.065, p=0.007). Healthy controls had more years of education than both patients with schizophrenia and bipolar disorder (15.9 versus 11.2 and 11.5 respectively, F=11.590, p<0.0001).
Sp-UPSA-BriefThe stepwise linear regression model for patients with schizophrenia showed that communication and transportation Sp-UPSA domains entered first and second at p<0.0001 and accounted for almost 90% of the variance (R2=0.88, model df=2, F=395.05, p<0.0001) (see Table 1). When age and years of education were not modelled the results were very similar (R2=0.87, model df=2, F=451.78, p<0.0001).
Linear regression model (stepwise method) for the entire sample and for patients with schizophrenia and bipolar disorder separately (with age and years of education forced to enter in the model).
| R2 | df | F | p | B | 95% CI | t | p | |
| Healthy controls (n=31)a | ||||||||
| Model | 0.81 | 2 | 60.25 | <0.0001 | – | – | – | – |
| Constant UPSA transportation | – | – | – | – | 32.63 | 21.02–44.25 | 5.75 | <0.0001 |
| UPSA finances | – | – | – | – | 1.36 | 1.01–1.72 | 7.89 | <0.0001 |
| – | – | – | – | 1.20 | 0.69–1.70 | 4.86 | 0.001 | |
| Patients with schizophrenia (n=139) | ||||||||
| Model | 0.88 | 2 | 395.05 | <0.0001 | – | – | – | – |
| Constant UPSA communication | – | – | – | – | 19.30 | 15.56–23.05 | 10.21 | <0.0001 |
| UPSA transportation | – | – | – | – | 1.78 | 1.49–2.06 | 12.34 | <0.0001 |
| – | – | – | – | 1.41 | 1.16–1.66 | 11.24 | <0.0001 | |
| Patients with bipolar disorder (n=57) | ||||||||
| Model | 0.87 | 2 | 132.32 | <0.0001 | – | – | – | |
| Constant UPSA transportation | – | – | – | – | 28.77 | 22.48–35.06 | 9.25 | <0.0001 |
| UPSA communication | – | – | – | 1.38 | 1.02–1.74 | 7.85 | <0.0001 | |
| – | – | – | 1.28 | 0.94–1.62 | 7.52 | <0.0001 | ||
df: degrees of freedom; CI: confidence interval.
In patients with bipolar disorder, transportation and communication Sp-UPSA domains entered first and second at p<0.0001, and accounted for almost 90% of the variance (see Table 1). Again, similar results were obtained when age and years of education were not modelled.
Finally, in healthy controls, we decided not to model age and years of education as there were nine missing values for years of education. The linear regression model showed that transportation (B=1.36) and communication (B=1.20) entered first and second at p<0.0001, and accounted for 81% of the variance (model df=2, F=60.25, p<0.0001) (see Table 1).
Consequently, the Sp-UPSA-Brief, derived from the regression model detailed above (see also Table 1), included communication and transportation domains.
Psychometric properties of the Sp-UPSA-BriefDistribution characteristics of Sp-UPSA-Brief scoresThe distribution characteristics of the total and domain Sp-UPSA-Brief scores in terms of mean and standard deviation, coefficient of variation, skewness, kurtosis, and ceiling and floor effects are shown in Table 2.
Distribution characteristics of Sp-UPSA-Brief scores for healthy controls and patients with schizophrenia or bipolar disorder separately.
| Mean (SD) | Coefficient of variation (%) | Skewness | Kurtosis | Ceiling effect* | Floor effect** | |
| Healthy controls (n=31) | ||||||
| UPSA-Brief Communication1,a | 44.22 (4.12) | 9.50 | −0.67 | 0.37 | – | – |
| UPSA-Brief Transportation2,b | 45.16 (5.87) | 13.00 | −1.00 | −0.20 | – | – |
| UPSA-Brief Total3,b | 89.38 (8.13) | 9.10 | −0.49 | −0.69 | 11 (36.5) | – |
| Patients with schizophrenia (n=139) | ||||||
| UPSA-Brief Communication1,a | 30.63 (11.27) | 36.79 | −0.72 | 0.23 | – | – |
| UPSA-Brief Transportation2,b | 33.73 (12.86) | 38.13 | −1.07 | 0.80 | – | – |
| UPSA-Brief Total3,b | 64.37 (21.89) | 34.01 | −0.93 | 0.62 | – | 3 (2.2) |
| Patients with bipolar disorder (n=57) | ||||||
| UPSA-Brief Communication1,a | 37.28 (10.61) | 28.46 | −0.69 | 0.20 | – | – |
| UPSA-Brief Transportation2,b | 33.92 (10.89) | 32.10 | −0.71 | 0.62 | – | – |
| UPSA-Brief Total3,b | 71.20 (18.41) | 25.86 | −0.65 | 0.41 | – | 0 (0.0) |
1. ANOVA F value=24.938, p<0.0001; 2. ANOVA F value=12.723, p<0.0001; 3. ANOVA F value=20.670, p<0.0001.
Duncan test showed that in the Communication domain the three groups were different among them (the mean difference was significant at the 0.05 level).
The mean total Sp-UPSA-Brief score for all the subjects was 69.5 (SD=21.4). Patients with schizophrenia or bipolar disorder scored significantly lower compared to healthy controls (64.4 and 71.2 versus 89.4, F=20.670, p<0.0001). However, although bipolar disorder group showed better performance than the schizophrenia group, this difference did not reach statistical significance.
Total Sp-UPSA-Brief score exhibited symmetrical and mesokurtic distributions in both healthy controls and patients with schizophrenia or bipolar disorder. While a moderate ceiling effect was found in controls (11 subjects scored >95%), minimal or no floor effects were found in patients with schizophrenia and bipolar disorders (3 and 0 patients scored <5% respectively).
ReliabilityThe Sp-UPSA-Brief scale had good internal consistency for patients with schizophrenia, with a Cronbach's alpha of 0.78. In the case of patients with bipolar disorder and controls the internal consistency was moderate (Cronbach's alpha=0.64 and 0.44 respectively).
Test–retest reliability was adequate with Pearson correlation coefficients of 0.66 (p<0.0001) and 0.64 (p=0.001) for patients with schizophrenia and bipolar disorder respectively. In both schizophrenic and bipolar patients groups the highest correlation coefficient was found in the communication domain (0.66 and 0.72 respectively).
Practice effects on the Sp-UPSA-Brief were small in patients with schizophrenia while in bipolar patients they were medium (Cohen's d=0.15 and 0.26 respectively).
Construct validityPearson correlation coefficients between the total Sp-UPSA and Sp-UPSA-Brief scores were 0.93 (p<0.0001) for patients with schizophrenia, 0.92 for bipolar (p<0.0001), and 0.90 (p<0.0001) for healthy controls.
Correlation coefficients between the Sp-UPSA-Brief total score and the total score of the PSP was 0.37 (p<0.0001) for patients with schizophrenia, 0.29 (p=0.02) for patients with bipolar disorder and 0.33 (p=0.072) for healthy controls. In the case of the GAF, the Pearson coefficients were 0.41 (p<0.0001), 0.38 (p=0.003), and 0.33 (p=0.072) respectively.
Discriminant validityAs stated above (see Distribution characteristics of Sp-UPSA-Brief scores and Table 2) the total Sp-UPSA-Brief was able to discriminate between patients (schizophrenia or bipolar) and healthy controls, but not between patients with schizophrenia and bipolar disorder. The same result was found for the transportation domain. However, the Sp-UPSA-Brief communication domain was able to differentiate among the three groups (F=24.938, p<0.0001).
The Sp-UPSA-Brief total score was also able to discriminate between the different levels of illness severity according to CGI-S scores in patients with schizophrenia (F=7.663, p<0.0001) but not in bipolar patients (F=0.213, p=0.809). In patients with schizophrenia, those minimally and mildly ill (mean total scores 76.2 and 70.1 respectively) scored significantly higher than moderately and severely ill patients (56.2 and 57.1).
In subjects with schizophrenia, the area under the ROC curve was 0.88 (95% CI=0.83–0.93), indicating good accuracy of the test. A cut-off point of 85 provided good sensitivity (83.5%) and moderate specificity (74.2%) (see Fig. 1). Predictive values were: positive 93.5% and negative 50.0%.
In subjects with bipolar disorder, we found similar results. The area under the curve was 0.82 (95% CI=0.73–0.90). A cut-off point of 90 provided good sensitivity (80.7%) and moderate specificity (61.3%) (see Fig. 1). Predictive values were: positive 73.1% and negative 61.9%.
DiscussionThe current study developed and validated the brief Spanish version of the UPSA scale (Sp-UPSA-Brief) in a sample of Spanish patients with schizophrenia and bipolar disorder under standard maintenance treatment. Findings indicated that the brief version, like the parent scale, has adequate psychometric properties and that it is highly correlated with the Sp-UPSA full scale.
The development of a brief version of the Sp-UPSA will allow clinicians and researchers to assess other areas of interest by reducing the time employed in the administration of the Sp-UPSA, while minimising the patient's burden associated with the assessment.16 Furthermore, it is important to note that the time in completing the Sp-UPSA-Brief is reduced by 15–20min compared to the full version of the scale. This fact represents an evident advantage in the assessment of psychiatric populations that often suffer from fatigue, lack of motivation, and difficulty on the capacity for sustaining attention.
Likewise previous published abbreviated UPSA versions16,17 in English language, the Sp-UPSA-Brief also included the communication domain, while we included instead the transportation domain as compared to the finances domain16 in patients with schizophrenia, or the planning recreational activities domain17 in patients with MCI and AD. This may be due to various causes; firstly, cultural differences between Spain and North-America in the use of transportation may have contributed to make the tasks included on the transportation domain more difficult to be applied in Spain. While in the USA and Canada the use of public transportation is more widely spread, in Spain, with the exception of a few big cities, the majority of trips can be done by walking, thus psychiatric patients may not be very familiar with the use of transportation maps to plan a trip. Consequently, the opportunity of everyday real-world practice of transportation abilities is not as common in Spain as it could be compared to USA. Secondly, in the case of Gomar et al. UPSA Short Form,17 this difference could be also due to the potentially distinct patterns in functional capacity across different disorders, MCI and AD versus schizophrenia and bipolar disorder patients, although this issue deserves further research. The functional impact of the onset of amnestic and dementia disorders varies as compared to psychiatric disorders such as schizophrenia and bipolar disorder; whereas function impairments are very subtle in MCI and early stages of AD, these deficits may be profound in schizophrenia even at first manifestations of psychotic symptoms (although it could be argued not to be so evident in bipolar disorder). Furthermore, the processes engaged in performing the transportation domain of the UPSA could have been consistently practiced and systematically performed in real-world settings by people suffering from MCI or AD during their entire adulthood, while in the case of people with schizophrenia both the typical onset of the disorder at first stage of adulthood and the devastating effect on functioning independently of the disorder itself dramatically affects practice of these sort of activities in the real world during adulthood and therefore undermine their proper acquisition.
The correlation between the Sp-UPSA-Brief and the full Sp-UPSA was high in both samples the original and the newly created random sample, thus allowing us to verify the equivalence between the two versions of the scale, the brief and the full Spanish forms. In patients with schizophrenia, Pearson correlation coefficients were 0.93 and 0.95 respectively; while in patients with bipolar disorder were 0.92 in both samples. These results confirm those reported by previous studies, which found high correlation coefficients in patients schizophrenia and schizoaffective disorders16 and in patients with MCI and AD.17
We also found that the Sp-UPSA-Brief version has psychometric properties as adequate as the full Spanish UPSA. The Sp-USA-Brief showed good levels of internal consistency with similar Cronbach's alphas compared to those reported for the Sp-UPSA.18 Our findings support the construct validity of the Sp-UPSA-Brief through high correlations with the total Sp-UPSA and low correlations with the PSP and GAF, thus confirming the difference between functional capacity and functional performance in real world as in the case of the full Sp-UPSA. Results on the discriminant validity by subgroup comparisons according to levels of severity showed that the Sp-UPSA-Brief could distinguish between severe mentally ill patients and healthy controls. Furthermore, in the case of patients with schizophrenia, the brief version was able to distinguish between minimally and mildly ill patients versus moderately and severely ill patients as the full version of the scale.
Limitations of the study are presented and discussed in depth in García-Portilla et al.18 and briefly included the lack of specific training for clinicians on the application of the Sp-UPSA scale, and the small sample size of patients with bipolar disorder. In addition, another limitation is the inability to know, given the design of the study, the inter-rater reliability.
In conclusion the Sp-UPSA-Brief is an alternate instrument to evaluate functional capacity that has been found in the present study to be valid and reliable. Having a shorter instrument makes it more feasible to assess functional capacity in patients with severe mental disorders, especially in everyday clinical practice.
Ethical responsibilitiesProtection of human and animal subjectsThe authors declare that no experiments were performed on humans or animals for this investigation.
Confidentiality of dataThe authors declare that they have followed the protocols of their work centre on the publication of patient data and that all the patients included in the study have received sufficient information and have given their informed consent in writing to participate in that study.
Right to privacy and informed consentThe authors must have obtained the informed consent of the patients and /or subjects mentioned in the article. The author for correspondence must be in possession of this document.
Conflict of interestsThe authors have no conflicts of interests to declare.
This research was support, in part, by BICIBERSAM 677 by the Centro de Investigación Biomédica en Red de Salud Mental CIBERSAM to Dr. Bobes.
Please cite this article as: Garcia-Portilla MP, Gomar J, Bobes-Bascaran MT, Menendez-Miranda I, Saiz PA, Muñiz J, et al. Desarrollo de la versión española de la Escala Breve para la Evaluación de la Capacidad Funcional [Sp-UPSA-Brief] para pacientes con esquizofrenia y trastorno bipolar. Rev Psiquiatr Salud Ment (Barc.). 2014;7:113–120.