Validation of the Spanish-language version of the Relevant Outcome Scale for Alzheimer's Disease

Abstract

Introduction and objectives

The Relevant Outcome Scale for Alzheimer's Disease (ROSA) is a useful tool for evaluating and monitoring dementia patients. This study aims to evaluate the validity and reliability of the Spanish version of ROSA.

Patients and methods

Spanish multicentre study involving 39 researchers and including 237 patients with Alzheimer disease (78 mild, 79 moderate, and 80 severe). The patients were tested with the following: Mini–Mental State Examination (MMSE), Fototest, Neuropsychiatric Inventory (NPI), Blessed dementia scale, and a Spanish-language version of ROSA. A subsample of 40 subjects was retested in the 14 days following the initial evaluation. The construct validity was evaluated with the Spearman correlation coefficient (r), internal consistency with Cronbach's alpha (alpha), and test–retest reliability with the intraclass correlation coefficient (ICC).

Results

ROSA requires 13.8 ± 7.4 minutes to administer and its results show a significant association with the clinical stage of AD (mild, 116.7 ± 23.1; moderate, 92.9 ± 19.8; and severe, 64.3 ± 22.6), and with results on the MMSE (r = 0.68), Fototest (r = 0.63), NPI (r = 0.53), and Blessed dementia scale (r = −0.80). ROSA shows high internal consistency (alpha = 0.90) and excellent test–retest reliability (ICC 0.97).

Conclusion

The Spanish version of ROSA is a brief, valid, and reliable tool permitting overall evaluation of patients with dementia.

Resumen

Introducción y objetivos

La escala Relevant Outcome Scale for Alzheimer's Disease (ROSA) es una herramienta útil para la evaluación y seguimiento de pacientes con demencia. Nuestro objetivo es evaluar la validez y fiabilidad de una versión española de la escala ROSA.

Pacientes y métodos

Estudio multicéntrico nacional en el que 39 investigadores han incluido 237 sujetos con enfermedad de Alzheimer (78 en estadio leve, 79 moderado y 80 grave) a los que se les ha aplicado Mini-Mental, Fototest, Neuropsychiatric Inventory (NPI), escala de Blessed y una versión adaptada al español de la escala ROSA. En una submuestra de 40 sujetos se realizó un retest en los 14 días siguientes a la evaluación inicial. La validez de constructo se ha evaluado mediante el coeficiente correlación de Spearman (r), la consistencia interna con el coeficiente alfa de Cronbach (alfa) y la fiabilidad test-retest con el coeficiente correlación intraclase (CCI).

Resultados

La escala ROSA se aplica en 13,8 ± 7,4 min y sus resultados están asociados de forma significativa al estadio clínico (leve 116,7 ± 23,1, moderado 92,9 ± 19,8 y grave 64,3 ± 22,6), Mini-Mental (r = 0,68), Fototest (r = 0,63), NPI (r = 0,53) y escala de Blessed (r = –0,80). La escala ROSA muestra una alta consistencia interna (alfa = 0,90) y una excelente fiabilidad test-retest (CCI = 0,97).

Conclusión

La versión española de la escala ROSA es un instrumento breve, válido y fiable para la evaluación global de pacientes con demencia.

Introduction

Alzheimer disease (AD) is a neurodegenerative disease characterised by extracellular deposition of amyloid beta in the form of senile plaques and intraneuronal deposition of hyperphosphorylated tau protein forming neurofibrillary tangles. These alterations lead to mitochondrial and synaptic dysfunction resulting in neuronal loss and atrophy; this translates clinically into progressive cognitive dysfunction (mainly memory loss) and behavioural disorders which result in increasing functional impairment.^1,2 Mounting evidence suggests that the pathological processes of AD start decades before onset of the first symptoms; involvement occurs progressively and in an organised fashion, explaining the progressive and sequential character of symptoms and the deficits caused by this condition.³

Scales are both necessary and frequently used for diagnosis, follow-up, and assessment of response to AD treatment. Ideally, using one single scale for assessing AD globally and throughout its course would make intra- and inter-patient comparisons easier. Nonetheless, the progressive and multifaceted character of AD explains the use of multiple scales to measure such specific domains as cognition (e.g. ADAS-Cog),⁴ behaviour (e.g. Neuropsychiatric Inventory⁵ [NPI]), functional status (e.g. Pfeffer),⁶ or patients’ and carers’ quality of life (e.g. Zarit).⁷ However, we still lack a global scale which is easy and quick to administer, applicable at any stage of the disease, and sensitive to progression and the effects of treatment.⁸

Some scales measure multiple AD domains (cognition, behaviour, functional status, and quality of life), but none of them meets each and every one of the requirements mentioned above; some of them are time-consuming and complex to administer (e.g. the Sandoz Clinical Assessment Geriatric⁹ and the Gottfries-Brane-Steen¹⁰ scales require specific training and take 30 minutes to administer), while others have been specifically designed for certain stages of the disease (e.g. the Vienna List¹¹ for advanced stages), require direct observation (these are more suitable for closed institutions than for clinical practice; e.g. the Nurses’ Observation Scale for Geriatric Patients),¹² or focus on a single domain of the disease (e.g. Behavioral Rating Scale for Geriatric Patients¹³ for behaviour, Clinical Dementia Rating scale¹⁴ for cognition and functional status).

The Relevant Outcome Scale for Alzheimer's Disease (ROSA)¹⁵ is a new instrument that can be applied at any stage of the disease, requires little time to administer, and enables the assessment of cognition, behaviour, functional status, quality of life, and caregiver burden. This scale, which has been validated in a German population,¹⁵ has been shown to be valid, reliable, and very sensitive to changes in progression. The psychometric properties of this scale and the fact that it assesses different aspects of the disease make the ROSA scale a very useful instrument for long-term patient follow-up in clinical practice, observational studies, and clinical trials.

In this study, we aim to assess the validity, reliability, and factorial structure of a Spanish-language version of the ROSA scale which may be administered to more than 500 million Spanish speakers.

Patients and methods

Design

We conducted a prospective multicentre study including 39 Spanish neurology, psychiatry, and geriatrics specialists with experience managing patients with AD.

Patients

Each specialist recruited 6 prospective patients diagnosed with AD according to the DSM-IV¹⁶ and NINCDS-ADRDA¹⁷ criteria (2 for each stage: mild, moderate, and severe); stages were assigned based on each specialist's experience and clinical practice. Patients were recruited at the convenience of specialists, patients, and caregivers. One of the specialists gathered 2 independent groups of patients from different clinics. Each patient included in the study had a primary caregiver who acted as an informant. The person living with the patient and directly involved in their care, or the person bearing the burden of the disease, was considered the primary caregiver. Primary caregivers were not professionals, nor did they belong to a social support network.

ROSA scale

The ROSA scale includes 16 items evaluating 6 dimensions: cognition (3 items), communication (3), behaviour (5), functional status (3), quality of life (1), and caregiver burden (1). The scale's content validity is high. Patients are assessed using everyday scenarios (events or situations) (supplementary material). It is completed with the help of an informant, who rates the patient's functional status in every scenario on a scale from 0 to 10, with higher scores reflecting better quality of life or better functional status or performance in a given scenario.

The scale was culturally adapted to maintain semantic content and preserve conceptual equivalence between the original culture and the Spanish culture,¹⁸ and it was subsequently translated and back-translated by 2 professional translators, a native German speaker and a native Spanish speaker. The main researcher also collaborated in this process.

Procedure

Each specialist collected data in a single visit recording demographic (sex, age, education level) and clinical data (time from symptom onset, time from diagnosis, stage according to the Global Deterioration Scale),¹⁹ and administered the Mini–Mental State Examination (MMSE),²⁰ the Fototest,²¹ the NPI,⁵ parts A and B of Blessed's scale,²² and the ROSA scale. Time of administration of the ROSA scale was also recorded. The scale was administered in normal clinical practice. Therefore, the conditions of administration varied: in some cases, the scale was administered by the researcher, whereas in others it was administered by another professional (neuropsychologists, etc.).

For the test–retest reliability study, every researcher selected one patient at the convenience of both parties; this subject was further assessed after 1 to 14 days using the ROSA scale.

Sample size

Based on the data from the original ROSA scale validation study¹⁵ and following the recommendations by Fleiss,²³ a minimum of 200 patients were necessary for validation and 38 for the test–retest reliability study.

Statistical methods

Quantitative variables are expressed as means ± standard deviation and qualitative variables as number of patients and percentages. Quantitative variables were compared using ANOVA (normally distributed variables) and the Kruskal–Wallis test (non-normally distributed variables). Qualitative variables were compared using the chi-square test.

Construct validity was assessed using the Spearman rank correlation coefficient between ROSA scores and scores on the administered cognitive (MMSE, Fototest), behavioural (NPI), and functional scales (Blessed).

The dimensional structure of the ROSA scale was analysed using principal component analysis and orthogonal Varimax rotation according to the Kaiser criterion (eigenvalue ≥ 1) and excluding the items with scores < 0.4 in each factor.

Internal consistency was calculated using Cronbach's alpha, and test–retest reliability was assessed with the intraclass correlation coefficient (ICC).

Ethical and formal considerations

Our study complied with the requirements of the Declaration of Helsinki (revised version, Seoul, October 2008) and Spanish legislation on observational studies (Ministerial Order SAS/3470/2009). The protocol was approved by the Clinical Research Ethics Committee at Hospital Universitario Virgen de las Nieves in Granada and notified to the Spanish Agency for Medicines and Medical Devices. All participants (or their legal representatives) signed informed consent forms.

Results

The final sample included 237 subjects, as data were lacking in 3 cases. Table 1 lists the sociodemographic characteristics and scale scores of our sample. As expected, patients at more severe stages of the disease displayed longer disease progression times and poorer scores on all scales. However, no significant differences were observed between severity groups in terms of age (77.9 ± 7.7 years), sex distribution (predominantly women [62%]), education level (59.9% with primary studies), and time taken to administer the ROSA scale (13.8 ± 7.4 min). All patients received some type of pharmacological treatment, with the most frequently used drugs being donepezil (38% of the patients), memantine (34.6%), rivastigmine (27%), galantamine (7.6%), quetiapine (5.9%), and trazodone (3.4%).

Table 1.

Sociodemographic and clinical characteristics of our sample.

	Mild	Moderate	Severe	Total	P
n	78	79	80	237
Age (years)	76.6 ± 6.2	78.4 ± 7.6	78.7 ± 8.8	77.9 ± 7.7	ns
Sex (women)	43 (55.1)	52 (65.8)	52 (65.0)	147 (62.0)	ns
Education level
No studies	3 (16.7)	11 (13.9)	27 (33.8)	51 (21.5)	ns
Primary studies	50 (64.1)	51 (64.4)	41 (51.3)	142 (59.9)
Secondary studies or higher	15 (19.2)	17 (21.5)	12 (15.0)	44 (18.6)
Time from symptom onset^a	37.9 ± 24.9	57.7 ± 29.5	77.5 ± 35.0	57.6 ± 34.0	^*
Time from diagnosis^b	14.7 ± 19.7	31.5 ± 24.1	49.9 ± 34.6	32.1 ± 30.4	^*
GDS	3.7 ± 0.7	4.6 ± 0.7	5.8 ± 0.7	4.7 ± 1.1	^*
MMSE	21.6 ± 3.5	17.3 ± 3.9	8.5 ± 4.8	15.8 ± 6.8	^*
Fototest	25.4 ± 5.6	19.0 ± 5.4	9.5 ± 6.3	18.1 ± 8.7	^*
NPI	10.6 ± 13.3	16.3 ± 15.0	24.3 ± 19.1	17.2 ± 16.9	^*
Blessed^c	5.6 ± 3.2	10.3 ± 3.8	17.2 ± 4.6	11.1 ± 6.2	^*
ROSA	116.7 ± 23.1	92.9 ± 19.8	64.3 ± 22.6	91.1 ± 30.6	^*
ROSA administration time^d	13.9 ± 7.5	14.7 ± 8.1	12.7 ± 6.6	13.8 ± 7.4	ns

Data are expressed as means ± standard deviation or number of cases (%).

GDS: Global Deterioration Scale; MMSE: Mini–Mental State Examination; NPI: Neuropsychiatric Inventory; ns: not significant.

a

Time from symptom onset, in months.

b

Time from diagnosis, in months.

c

Blessed scale, parts A and B.

d

Time required for the administration of the ROSA scale, in minutes.

*

P < .05.

Fig. 1 shows the distribution of ROSA scores broken down by stage of severity. We found significant differences in ROSA scores between stages, both globally (Fig. 2) and by domain (Table 2). This reflects the decline inherent to the disease's progression as well as the progressive deterioration of patients’ quality of life and the increasing caregiver burden.

Figure 1.

Distribution of ROSA scores broken down by severity stage.

Figure 2.

ROSA scores by stage.

Table 2.

Distribution of partial and total ROSA scores by domain.

Domain	Mild (n = 78)	Moderate (n = 79)	Severe (n = 80)	Total (n = 237)	P
Cognition (3 items)	19.1 ± 5.8	13.1 ± 5.3	6.3 ± 4.8	12.8 ± 7.4	^*
Communication (3 items)	22.3 ± 5.3	18.4 ± 5.6	11.9 ± 7.2	17.5 ± 7.4	^*
Behaviour (5 items)	39.7 ± 9.1	34.4 ± 8.9	30.1 ± 10.9	34.7 ± 10.4	^*
Functional status (3 items)	21.4 ± 6.9	15.4 ± 5.8	7.1 ± 5.9	14.6 ± 8.6	^*
Quality of life (1 item)	7.6 ± 2.0	6.8 ± 1.9	5.8 ± 3.0	6.7 ± 2.5	^*
Caregiver burden (1 item)	6.5 ± 2.8	4.8 ± 2.6	3.1 ± 2.9	4.8 ± 3.1	^*
Total score (16 items)	116.7 ± 23.1	92.9 ± 19.8	64.3 ± 22.6	91.1 ± 30.6	^*

Data are expressed as means ± standard deviation.

*

P < .05.

The factorial analysis of the ROSA scale (Table 3) resulted in the extraction of 2 factors which explain 52.3% of total variability; all items showed an Eigenvalue > 0.4 after Varimax rotation. The first factor groups items assessing cognition, communication, functional status, and caregiver burden; the second factor includes items evaluating behaviour and quality of life.

Table 3.

Factorial analysis of the ROSA scale.

Item number and description	Domain	Factor 1^a	Factor 2^a
1. The patient is able to remember events from a long time ago.	Cognition	0.71941
2. The patient is able to remember events of the recent past.	Cognition	0.74855
3. The patient is able to plan and carry out complex procedures.	Cognition	0.80784
4. The patient is able to make himself understood.	Communication	0.68579
5. The patient is able to communicate.	Communication	0.72205
6. The patient shows social competence.	Communication	0.73873
7. The patient behaves aggressively.	Behaviour		0.72587
8. The patient is restless.	Behaviour		0.70439
9. The patient shows behavioural changes due to delusions.	Behaviour		0.70794
10. The patient is insecure.	Behaviour		0.63603
11. The patient's behaviour is cooperative.	Behaviour		0.41758
12. The patient is competent at everyday tasks.	Functional status	0.77966
13. The patient is attentive, shows interest in his surroundings.	Functional status	0.72524
14. The patient is independent.	Functional	0.75014
15. The patient's quality of life is (very good/very poor).	Quality of life		0.42139
16. The burden for the caregiver is (very small/very large).	Caregiver burden	0.42256

a

Eigenvalue.

ROSA scores showed a significant correlation with MMSE (r = 0.68, P < .01), Fototest (r = 0.63, P < .01), and Blessed scale scores (r = −0.80, P < .01), and with global NPI scores (r = −0.53, P < .05).

The ROSA scale showed a high internal consistency, with a Cronbach's alpha of 0.90 in the baseline visit. Forty patients (19 with mild AD, 15 with moderate AD, and 6 with severe AD) were evaluated with the ROSA scale in a follow-up visit taking place 1 to 14 days after the baseline visit (median of 13 days). In these cases, no significant differences were seen between baseline and follow-up scores (106.8 ± 27.3 vs 106.5 ± 28.1, ns), which showed high test–retest reliability (ICC = 0.97, P < .01) (Fig. 3).

Figure 3.

Test–retest reliability of the ROSA scale.

Discussion

According to our results, the Spanish-language version of the ROSA scale, like the original, is a quick-to-administer, valid, and reliable instrument for assessing patients with AD throughout the entire course of the disease, from mild to severe stages.

Content validity is ensured by the structure of the scale and the procedure followed to create it.¹⁵ Construct and convergent validity are shown by the significant association between ROSA scores and AD stages (Fig. 2) and scores on the rest of the scales specifically assessing each domain of AD. Unlike the original study, which used the ADAS-Cog⁴ scale for cognitive assessment, we used the MMSE,²⁰ which is the most widely used cognitive assessment tool in spite of its many limitations,²⁴ and the Fototest,²¹ a novel quick-to-administer cognitive test suitable for illiterate subjects, which is very widely used in our setting as it is more appropriate for populations with a low level of education²⁵ and more cost-effective than the MMSE.²⁶ Correlations with both scales were highly significant (MMSE, r = 0.68; Fototest, r = 0.63), even more so than the correlation with the ADAS-Cog found in the original study (r = −0.47). The fact that the correlation coefficients reported in our study were positive, whereas those of the original study were negative, is explained by the different scoring systems of these scales: higher scores on the ROSA scale, the MMSE, and the Fototest indicate less severe involvement, whereas the ADAS-Cog scale assigns higher scores to more severe stages. The original study used the Disability Assessment for Dementia²⁷ (DAD) to assess functional status. Our study, in contrast, used the Blessed scale (parts A and B), an instrument with demonstrated validity which is more frequently used in our setting. The association found in our study is also highly significant (r = −0.80), also to a greater extent than the correlation with the DAD scale found in the original study (r = 0.70). Again, the fact that the correlation coefficient reported in our study was negative whereas that of the original study was positive is explained by the different scoring systems of the Blessed and the ROSA scales. Both studies used the NPI to evaluate behavioural symptoms, reporting a similar association (r = 0.53 in both studies).

As in the case of the original ROSA scale, our results have showed high internal consistency and test–retest reliability (Fig. 3). Furthermore, the results of our factorial analysis are virtually identical to those of the original study: we extracted 2 independent factors. The first factor included the items evaluating cognition, functional status, communication, and caregiver burden, and the second factor included the items assessing behaviour and quality of life.

The mean time taken to administer the ROSA scale was less than 15 minutes (13.8 ± 7.4), with no significant differences between stages of the disease. The time of administration in our study is similar to that reported in the original study (14.3 ± 6.8), and clearly shorter than that required to administer other scales assessing the different domains of AD covered by the ROSA scale.

The Dementia Severity Scale²⁸ and the HABC-Monitor²⁹ are also new multidimensional scales which are completed with the help of an informant; both scales have shown adequate internal consistency, reliability, and construct validity. However, both include more items than the ROSA scale (47 and 31 items, respectively). The usefulness and applicability of these 3 instruments should be compared using the same sample.

The main limitation of our study is the method of sample collection: patients were selected at the convenience of researchers and patients, which limits the representativeness of the sample. Furthermore, our sample did not include patients without dementia, whether or not they had cognitive impairment, which prevents us from evaluating the extent to which the scale displays the same properties in these subjects, especially in those with cognitive impairment and no dementia. On the other hand, our study also has a number of strengths, including the large sample size and its multicentre and naturalistic design, which supports the external validity of our results and its potential applicability to normal clinical practice.

In summary, the Spanish-language version of the ROSA scale is similar to the original version in that it is a quick-to-administer, valid, and reliable instrument for global assessment of patients with dementia.

Funding

This study was financed by Grünenthal Pharma S.A.

Conflict of interest

C. Carnero-Pardo designed the Fototest and the Eurotest. Isabel Sánchez Magro works for Grünenthal Pharma S.A.