Validation of the Spanish version of Addenbrooke's Cognitive Examination III for diagnosing dementia

Abstract

Introduction

Addenbrooke's Cognitive Examination is a screening test used to diagnose dementia. The third edition of this test (ACE-III) was recently developed. The aim of this study was to translate and validate the ACE-III in Spanish.

Methods

The ACE-III was translated and adapted to Spanish. It was then administered to a group of healthy subjects as well as a group of patients with different types of mild dementia treated in 2 hospitals in Spain.

Results

Internal reliability (Cronbach's alpha = 0.927), inter-rater reliability (intraclass correlation coefficient = 0.976) and test–retest reliability (kappa 0.995) were excellent. Age (r = −0.512) and education (r = 0.659) showed a significant correlation with total test scores. The diagnostic accuracy of ACE-III was higher than that of the Mini-Mental State Examination, particularly for the group with the highest educational level. Researchers obtained normative data and cut-off points for the diagnosis of dementia.

Conclusions

The Spanish version of the ACE-III is a reliable and valid test for diagnosing dementia. Its diagnostic accuracy is high, especially in patients with a higher level of education.

Resumen

Introducción

El Addenbrooke's Cognitive Examination (ACE) es un test de cribado para el diagnóstico de demencia. Recientemente, se ha desarrollado la tercera versión del test (ACE-III). El objetivo del estudio fue la traducción y adaptación del ACE-III al español y su validación.

Material y métodos

El ACE-III fue traducido y adaptado al español. Se administró a un grupo de sujetos cognitivamente sanos y a pacientes con demencia leve de diferentes tipos en 2 centros españoles.

Resultados

La consistencia interna del test (alfa de Cronbach = 0,927), la fiabilidad interevaluador (coeficiente de correlación intraclase = 0,976) y la fiabilidad test-retest (kappa = 0,995) fueron elevadas. Edad (r = −0,512) y escolaridad (r = 0,659) se correlacionaron significativamente con la puntuación total del test. La capacidad diagnóstica del ACE-III fue superior al Mini-Mental State Examination, especialmente en el grupo con mayor escolaridad. Se obtuvieron datos normativos por edad, y puntos de corte para la detección de demencia.

Conclusiones

La versión española del test ACE-III es un instrumento válido para el diagnóstico de demencia, con una alta capacidad discriminatoria especialmente en pacientes con un mayor nivel educativo.

Introduction

Both the incidence and the prevalence of dementia have risen in recent years, and it is believed that the frequency of dementia will continue rising due to the ageing population. As a result, screening for and diagnosing dementia are immensely important tasks because they provide the gateway to proper treatment.¹

The high numbers of patients seen in general and neurology clinics for cognitive symptoms point to a need for sensitive and specific instruments enabling doctors to distinguish between physiological and pathological states.^2,3 Screening tests constitute the first step in the neuropsychological evaluation; they allow us to identify subjects that may have a disease, but not deliver a diagnosis.⁴ The most frequently employed screening test is the Mini Mental State Examination (MMSE), although it does present a number of limitations.⁵ Several other neuropsychological tests have been developed in order to reduce those limitations. One of the best-known is Addenbrooke's Cognitive Examination (ACE), which was initially developed as a modified form of the MMSE and which has proved its utility for diagnosing and monitoring different types of cognitive decline.^6–12 In addition to serving as a screening tool, the ACE may also permit differential diagnosis of different types of dementia, and as such, it can be used for both screening and diagnostic ends. The ACE test and its revised version (ACE-R) have both been adapted and validated in Spanish.^13–15

The third English-language version of this test (ACE-III) has recently been developed and validated with a view to increasing the diagnostic utility of earlier versions.¹⁶ The goal of this study was to produce a translated, adapted version of the ACE-III in Spanish and validate it for diagnosing dementia.

Material and methods

Study design

Prospective, observational, comparative, cross-sectional study intended to validate the ACE-III test in a population of healthy controls and subjects with various types of dementia.

Study population

The test was validated in 217 subjects recruited from the neurology departments at Hospital Clínico San Carlos in Madrid and Hospital de la Santa Creu i Sant Pau in Barcelona. Subjects were recruited between January and April 2014.

Inclusion criteria were being at least 50 years old and having sufficient hearing, sight, and physical ability to complete the evaluation.

Healthy controls were selected among patients’ family members who accompanied them to appointments, or among healthy volunteers. Exclusion criteria were neurological or systemic diseases potentially able to affect cognitive function, current psychiatric disease, and history of abusing alcohol or other substances.

Pathological cases were included consecutively from among patients attending appointments in the listed neurology departments if they met inclusion criteria. We included patients diagnosed with dementia according to NINCDS-ADRDA criteria¹⁷ and rated as ‘mild’ according to the Clinical Dementia Rating scale.¹⁸ Patients with different types of dementia were recruited in proportion to the prevalence of each dementia type^19,20 according to the latest diagnostic criteria. The sample therefore included patients with Alzheimer disease,¹⁷ vascular dementia,²¹ degenerative dementia with a vascular component,²² dementia associated with Parkinson's disease,²³ dementia with Lewy bodies,²⁴ frontotemporal dementia (including the behavioural variant and primary progressive aphasia),²⁵ and alcoholic dementia.²²

Translation and adaptation of Addenbrooke's Cognitive Examination III

The English-language version of the ACE-III was translated to Castilian Spanish and certain items on the test were adapted culturally and linguistically. The 3 words provided for the patient to repeat and memorise were adapted to present the appropriate length, imaginability, and frequency in the target language: limón, tren and pelota. The name and address each subject was asked to remember underwent a similar adaptation so that the length, familiarity, and imaginability of the item would resemble those in the English version. For the address recognition item, the patient was offered alternative words that were phonetically or semantically similar to the originals. Other adaptations were that the subject was asked to name the Prime Minister of Spain rather than of the UK, and Spain's first democratically elected president rather than the woman who was the UK's Prime Minister. In the repeated words section, ‘caterpillar’ was replaced by cucaracha and Spanish popular sayings were chosen to represent the length, familiarity, and difficulty of the original sayings. In the semantic memory subtest in which the patient was asked to point to a drawing after hearing a description, we replaced “point to the marsupial” with “point to the reptile” after concluding that the Spanish population might not be familiar with the term ‘marsupial’. Irregular spellings do not exist in Spanish, so the subtest in which subjects read oddly-spelled words lists foreign words instead: ‘Hollywood’, ‘vedette’, ‘blues’, ‘tour’ and ‘a capella’. The population older than 50 is more likely to recognise this selection than a list consisting entirely of English words. Some of these words had already appeared in the previous Spanish-language versions of the ACE and ACE-R, but others were replaced because a large part of our target population would have found them unfamiliar. The rest of this subtest followed the original version. The test was translated by 2 of the authors working independently, and we reached a consensus version based on experience gained with the previous versions of the ACE. Lastly, the test was administered to a group of 20 healthy subjects to gauge its comprehensibility and applicability. Completing the test delivers a maximum score of 100 points, with 5 scores for each of the 5 sections or domains represented in the test: attention (18 points max), memory (26 points max), fluency (14 points max), language (26 points max) and visuospatial abilities (16 points max). The test is corrected according to the guidelines given for the English-language version (http://www.neura.edu.au/).

Statistical analysis

Statistical analysis was performed with the IBM program SPSS Statistics 20.0. We analysed the psychometric properties of the test by calculating internal validity (Cronbach's alpha coefficient), concurrent validity with the MMSE (Pearson correlation coefficient, r), inter-rater reliability, and test–retest reliability. The effects of age, years of education, and sex were studied using the correlation coefficient (r) and the coefficient of determination (r²). Inter-rater reliability was studied using 2 raters and a subgroup of 15 controls and patients. The test–retest reliability study was performed in a subgroup of 15 patients and controls, with an interval of 20 to 40 days between testing sessions. The ROC curves (receiver operating characteristic) and sensitivity and specificity values for different cut-off points were estimated so as to distinguish between controls and patients with dementia.

Results

Of the 217 cases included, 130 were cognitively healthy and 87 displayed cognitive impairment. Patients with cognitive impairment had the following diagnoses: Alzheimer disease (47 cases, 54%), vascular dementia (4 cases, 4.6%), mixed dementia (9 cases, 10.3%), dementia associated with Parkinson's disease (11 cases, 12.6%), Lewy body dementia (6 cases, 6.9%), frontotemporal dementia (6 cases, 8.8%), alcoholic dementia (3 cases, 3.4%), and parkinsonism with dementia (1 case, 1.1%). Table 1 lists the main demographic characteristics of control subjects and patients with dementia.

Table 1.

Demographic characteristics for the sample, with scores on ACE-III and MMSE.

	Total sample (n = 217)		Group aged 65 and older (n = 175)		P-value
	Healthy controls (n = 130)	Dementia patients (n = 87)	Healthy controls (n = 92)	Dementia patients (n = 83)
Age (years)	71.0 ± 11.0	77.3 ± 8.4	77.0 ± 6.4	78.4 ± 6.8	.142
Sex (women)	84 (64.6%)	53 (60.9%)	57 (62.0%)	52 (62.7%)	.925
Years of schooling	9.8 ± 5.9	7.5 ± 4.6	8.4 ± 5.8	7.4 ± 4.7	.202
MMSE	27.2 ± 2.5	19.6 ± 4.7	26.5 ± 2.6	19.5 ± 4.6	<.0001
ACE-III	81.8 ± 12.7	50.4 ± 16.0	77.9 ± 12.6	49.9 ± 15.0	<.0001
Attention	15.9 ± 1.8	11.0 ± 3.1	15.5 ± 5.9	11.0 ± 3.1	<.0001
Memory	19.6 ± 5.0	9.3 ± 4.2	18.5 ± 5.0	9.2 ± 4.2	<.0001
Fluency	10.2 ± 2.9	5.0 ± 2.9	9.4 ± 2.9	4.9 ± 2.7	<.0001
Language	22.4 ± 3.7	15.2 ± 5.8	21.5 ± 3.8	15.0 ± 5.5	<.0001
Visuospatial	13.5 ± 2.1	9.5 ± 3.4	12.9 ± 2.1	9.4 ± 3.2	<.0001

Values are given as mean ± standard deviation or as frequencies (percentages).

Consistency and reliability

Cronbach's alpha was 0.927, which indicates excellent internal consistency. The test–retest reliability analysis yielded an intraclass correlation coefficient of 0.976 (0.917-0.993, 95% confidence interval). The kappa coefficient for inter-rater agreement was 0.995. Convergent validity, assessed using the kappa coefficient with the MMSE score, was 0.891 (P < .0001).

Normative data

Table 2 shows the ACE-III scores (totals and domains) for healthy controls, broken down by age group.

Table 2.

Normative data for the total score and the domain scores by age group.

Age (years)	Number of subjects	Years of education	ACE-III total	Attention	Memory	Fluency	Language	Visuospatial
50-59	26	14.1 ± 4.6	91.8 ± 6.9	17.0 ± 1.0	23.1 ± 3.1	12.0 ± 1.9	24.6 ± 2.2	15.0 ± 1.4
60-69	25	11.2 ± 5.3	87.7 ± 8.9	16.4 ± 1.6	20.8 ± 3.6	11.7 ± 2.0	24.0 ± 2.9	14.6 ± 1.6
70-79	45	7.4 ± 6.1	78.0 ± 13.2	15.3 ± 2.8	18.7 ± 5.2	9.5 ± 2.6	21.9 ± 3.8	12.8 ± 2.2
≥80	34	9.8 ± 5.9	74.7 ± 11.4	15.7 ± 1.5	17.4 ± 5.1	8.6 ± 3.1	20.3 ± 3.7	12.5 ± 1.8

Values are given as mean ± standard deviation.

Age, education, and sex effects

The correlations between total/domain scores and the variables age, education, and sex are shown in Table 3. Age and education had a significant impact on total score and domain scores on the test. In contrast, sex was only linked to the attention domain.

Table 3.

Correlation coefficients and coefficients of determination of scores by age, education, and sex.

	Age (years)		Education (years)		Sex
	r	r^2	r	r^2	r	r^2
ACE-III total	−0.512^*	0.262	0.659^*	0.434	−0.130	0.016
Attention	−0.285^*	0.081	0.428^*	0.183	−0.199^**	0.039
Memory	−0.391^*	0.152	0.562^*	0.315	−0.034	0.001
Fluency	−0.461^*	0.212	0.568^*	0.322	−0.141	0.019
Language	−0.451^*	0.203	0.552^*	0.304	−0.077	0.005
Visuospatial	−0.482^*	0.232	0.531^*	0.281	−0.182^**	0.033

*

P-value < .01 (two-tailed).

**

P-value < .05 (two-tailed).

Diagnostic properties

Since the sample contained only a few dementia patients younger than 65, the test's diagnostic properties were assessed in subjects aged 65 and older. We did not observe any statistically significant differences in age, education, or sex between the group of healthy controls and the group with mild dementia (Table 1). ROC curves were estimated for the total ACE-III and MMSE scores. Areas under the curve, indicating the distinction between healthy controls and patients with mild dementia, were 0.923 for ACE-III and 0.910 for MMSE (Fig. 1). The optimum cut-off point was 65.5, with a sensitivity of 83.1% and a specificity of 80.4%. Other cut-off points are shown in Table 4.

Figure 1.

ROC curve for discriminating between mild dementia and healthy controls. The solid line shows the ACE-III; the dashed line shows the MMSE.

Table 4.

Cut-off points for a diagnosis of mild dementia.

ACE-III total	Sensitivity	Specificity
58/59	0.723	0.924
60/61	0.735	0.913
61/62	0.759	0.891
62/63	0.783	0.870
63/64	0.807	0.848
64/65	0.819	0.837
65/66	0.831	0.804
66/67	0.855	0.772
67/68	0.867	0.750
68/69	0.904	0.728
69/70	0.916	0.696

The cut-off point found to be optimal is shown in bold.

We observed variations in the diagnostic ability of ACE-III according to the patient's educational level. As such, the group with 10 or more years of schooling showed an area under the curve of 0.982 for the ACE-III and of 0.934 for the MMSE. For this group, the optimal cut-off point was 73.5, with a sensitivity of 90.5% and a specificity of 96.7%. In the group with less than 10 years of schooling, the area under the curve was 0.922 for the ACE-III and 0.908 for the MMSE (Fig. 2). In this case, the cut-off point was 61.5, with a sensitivity of 82.3% and a specificity of 83.9%. The diagnostic ability of the ACE-III exceeded that of the MMSE for subjects with at least an elementary education, but the MMSE was more useful for the group with little to no education (Fig. 3). In contrast, the diagnostic accuracy of the ACE-III was similar to that of the MMSE for all age groups (Fig. 4).

Figure 2.

ROC curves for discriminating between mild dementia and healthy controls. (A) Group with 10 or more years of schooling. (B) Group with less than 10 years of schooling. The solid line shows the ACE-III; the dashed line shows the MMSE.

Figure 3.

Area under the ROC curve as a function of education. The solid line shows the ACE-III; the dashed line shows the MMSE. The ACE-III showed a higher discriminatory ability for subjects with more years of schooling.

Figure 4.

Area under the ROC curve as a function of age. The solid line shows the ACE-III; the dashed line shows the MMSE. Discriminatory ability is lower for older subjects, with no differences between the MMSE and the ACE-III.

Discussion

This study shows that the Spanish-language version of the ACE-III test is very reliable and has a high diagnostic capability. It is therefore a valid instrument for diagnosing dementia in our setting.

The test's reliability was evaluated using test–retest, inter-rater agreement, and internal consistency analyses. Scores were high for all 3 variables and similar to those obtained by earlier versions of the test.

The test demonstrated a high diagnostic ability, with values above 90% for discrimination between healthy controls and subjects with mild dementia. The optimal cut-off point was near 65.5, although a slightly higher point would probably be recommendable to increase sensitivity, which is the main purpose of a screening test. This cut-off point is lower than those proposed for the English-language versions of the ACE-R (82) and the ACE-III (88).^7,16 Furthermore, earlier versions of the Spanish-language test found optimal cut-off points of 86 and 85. Nevertheless, these studies were carried out in highly educated populations with a mean of 12 years of education in both the dementia and control groups. This being the case, the validation study of the Italian-language version of ACE-R,²⁶ carried out in a population with a mean of 7 to 8 years of education, proposed a cut-off point of 60 for subjects aged 75 or older. Another validation study of the first Spanish-language version of the ACE in a rural Spanish population proposed a cut-off point of 86 for highly-educated subjects and of 68 for those with a lower educational level.¹⁴ Furthermore, the patients included in our study were also older than those in other validation studies,^14,16 another factor that could have yielded a lower cut-off point because of the age effect on scores on this test. On the other hand, including multiple types of dementias rather than just Alzheimer disease and frontotemporal dementia, as other validation studies have done, could also have affected the test's cut-off point and diagnostic ability. Each type of cognitive decline will be associated with a different profile of cognitive impairments.²⁷

In our study, we observed that age and years of education exerted a significant effect on the test score. Each of these variables accounted for 15% to 43% of the total score and of the domain scores. These age and education effects underline the importance of establishing reference values according to the age and educational level of our subjects.^3,4 This study has obtained preliminary normative data for different ages, but studies with larger sample sizes must be carried out in order to obtain reference values adjusted by age, years of schooling, and general knowledge.

In the full sample, the ACE-III showed a higher diagnostic ability than the MMSE, as measured by the area under the ROC curve (AUC). This was also true of subjects with at least 6 to 9 years of schooling (Fig. 3). Nevertheless, in patients with 0 to 5 years of schooling (illiterate or largely uneducated), the ACE-III's diagnostic ability was lower than that of the MMSE (AUC_ACE-III 0.899; AUC_MMSE 0.944). A similar result was also found in the Spanish version of the first version of the test. These findings, added to the fact that most commonly-used screening tests display lower diagnostic ability in subjects with less education, probably justify the need to develop tests specifically aimed at this segment of the population.^5,28

Our study has some limitations. It includes only a few patients with certain rarer forms of dementia, and we therefore cannot draw conclusions about the diagnostic utility of the test for each dementia type, or speculate about its utility for differential diagnosis between dementia types. Nevertheless, including different types of dementia, in accordance with the dementia type proportions and ages cited in epidemiological studies, supports the utility of the ACE-III as a means of diagnosing dementia in normal clinical practice.

We conclude that the Spanish version of the ACE-III is a valid instrument for diagnosing dementia. The ACE-III demonstrates a greater discriminatory ability than the MMSE, especially among more educated patients. Given that age and education have a substantial effect on scores on this test, further studies in larger populations of healthy controls are needed in order to provide normative data for different ages and educational levels.

Conflicts of interest

The authors have no conflicts of interest to declare.

Acknowledgements

We would like to thank Professor John R. Hodges and his team for their support for this study.