Validation of the Spanish-language version of the Montreal Cognitive Assessment test in adults older than 60 years

Abstract

Introduction

Few studies have validated the Spanish-language version of the Montreal Cognitive Assessment (MoCA-S) test in Latin American populations.

Objetive

To evaluate the psychometric properties and discriminant validity of the MoCA-S in elderly patients in Santiago de Chile.

Methods

172 individuals were grouped according to their clinical diagnosis based on the Clinical Dementia Rating (CDR) scale as follows: amnestic mild cognitive impairment (aMCI; n = 24), non-amnestic MCI (naMCI; n = 24), mild dementia (n = 20), and cognitively normal (n = 104). Participants were evaluated with both the MoCA-S and the Mini-Mental State Examination (MMSE) to determine the discriminant validity of the MoCA-S.

Results

Mean age and years of schooling were 73 ± 6 and 11 ± 4 years, respectively, with no significant intergroup differences. The MoCA-S displayed good internal consistency (Cronbach’s α: 0.772), high inter-rater reliability (Spearman correlation coefficient: 0.846; P < .01), and high intra-rater reliability (test-retest reliability coefficient: 0.922; P < .001). The MoCA-S was found to be an effective and valid test for detecting aMCI (AUC = 0.903) and mild dementia (AUC = 0.957); its effectiveness for detecting naMCI was lower (AUC = 0.629). The optimal cut-off points for aMCI and mild dementia were < 21 and < 20, respectively, with sensitivity and specificity rates of 75% and 82% for aMCI and 90% and 86% for mild dementia. The level of education had a great impact on scores: as a result, 2 points were added for patients with less than 8 years of schooling and one point for patients with 8-12 years of schooling (MoCA-S1-2). The MoCA-S1-2 showed significantly greater discriminant validity than the MMSE for differentiating aMCI from dementia.

Conclusions

The MoCA-S1-2 is a short, easy-to-use, and useful test for diagnosing aMCI and mild dementia.

Resumen

Introducción

Existen pocas validaciones de la versión en español de la prueba Montreal Cognitive Assessment (MoCA-S) en Latinoamérica.

Objetivo

Evaluar las propiedades psicométricas y la validez discriminativa del MoCA-S en adultos mayores de Santiago de Chile.

Métodos

Ciento setenta y dos individuos agrupados según diagnóstico clínico basado en Clinical Dementia Rating (CDR) en: deterioro cognitivo leve tipo amnésico (DCL-a, n = 24), DCL no-amnésico (DCL-na, n = 24), demencia leve (n = 20) y 104 cognitivamente sanos fueron evaluados con el MoCA-S y Mini-Mental test de Folstein (MMSE) como prueba de contraste, para determinar la validez discriminativa del MoCA-S.

Resultados

Los promedios ± desviación estándar de edad y escolaridad fueron 73 ± 6 y 11 ± 4, respectivamente, sin diferencias significativas entre los grupos. La consistencia interna fue buena (α de Cronbach 0,772), la fiabilidad interevaluador muy buena (coeficiente de correlación de Spearman 0,846 [p < 0,01]) y la fiabilidad intraevaluador (test-retest) fue 0,922 bilateral (p < 0,001). La prueba fue eficaz y válida para la detección del DCL-a (ABC = 0,903) y demencia leve (ABC = 0,957); menos eficaz en DCL no-a (ABC = 0,629). El punto de corte de mejor rendimiento para DCL-a fue < 21 y para demencia leve < 20; sensibilidad/especificidad de 75/82% y 90/86%, respectivamente. La escolaridad mostró una importante influencia en el puntaje, por ello se adicionaron 2 puntos para escolaridad < 8 años y un punto para escolaridad entre 8 y 12 años (MoCA-S1-2). El MoCA-S1-2 fue significativamente más discriminativo que el MMSE para diferenciar DCL-a y demencia.

Conclusiones

El MoCA-S1-2 es una prueba breve, de fácil administración y útil para el diagnóstico de DCL-a y demencia leve.

Introduction

Population ageing in Chile has accelerated in recent decades^1–3; Chile currently has the third-oldest population in Latin America (after Cuba and Uruguay) and one of the highest rates of dementia among individuals older than 60 years (8%) worldwide.^4,5

Mild cognitive impairment (MCI) is defined as cognitive decline which is greater than that expected for the patient's age, but which does not meet criteria for dementia. The annual conversion rate of MCI to dementia is 10%-30%, compared to a rate of 1%-2% in cognitively normal elderly individuals.^6–9 Amnestic MCI (aMCI; both single- and multiple-domain subtypes) is the type most frequently associated with progression to Alzheimer disease (AD). The new diagnostic guidelines published in 2011 refer to aMCI as “MCI due to AD.”⁹

To date, MCI and dementia have been diagnosed clinically, through medical history, neuropsychological assessment, and laboratory analysis to rule out confounding factors.^10,11 Biomarkers of AD, such as CSF amyloid beta peptide and total and phosphorylated tau protein levels or presence of amyloid in the brain, detected with positron emission tomography,^9,12 are increasingly useful; however, these techniques continue to be costly or invasive.^9,10,13 Therefore, neuropsychological assessment is still the basic tool used in daily practice for the diagnosis and screening of cognitive impairment.¹⁰

The Mini–Mental State Examination (MMSE) is the most widely used cognitive screening test in patients with memory problems.¹⁴ However, it is not free from limitations^15,16: it does not assess abstraction or executive functions; it is strongly biased by social/educational factors, with differences of around 7 points between healthy populations of different educational levels¹⁷; its sensitivity to early stages of cognitive impairment is poor; and it is subject to copyright. New cognitive screening tests address these issues.^15,16,18,19 In the Chilean validation of the MMSE, Quiroga et al.²⁰ found the test to have a specificity of 44%; in other words, the false positive rate is 56%. However, the MMSE combined with the Pfeffer Functional Activities Questionnaire (administered to a valid informant) showed a sensitivity for detecting dementia of 94.4% and a specificity of 83.3%. This combination of questionnaires is commonly used in Chile.²⁰ However, these tests lack sufficient validity for screening for early stages of cognitive impairment.

The Montreal Cognitive Assessment (MoCA) is an MCI screening tool developed in 2005 by Nasreddine et al.²¹ Numerous studies comparing the MoCA and the MMSE have found the former to be superior for detecting MCI, with high levels of reliability in international validation studies.^21–26 The MoCA assesses executive functions, attention, abstraction, memory, language, visuospatial skills, calculation, and orientation. It is easy to administer and takes approximately 10 minutes to complete.²¹ The maximum score is 30 points; the cut-off point for MCI and dementia in developed countries is < 26.²¹ The main limitation is its considerable educational bias; therefore, the original version recommends adding a point to the score of individuals with less than 12 years’ schooling.²¹ However, one additional point may be insufficient in populations with very low levels of education.^23,27,28

There are very few validation studies of the use of the Spanish-language version of the MoCA (MoCA-S). Lozano Gallego et al.²³ validated the test in Spain; the only validations in Latin America were conducted in Colombia.^26,28 The test has also been validated in Spanish speakers in the United States, although these results cannot be compared as some study participants were bilingual.²⁷ Therefore, it is necessary to study the cut-off point for the test in the Chilean population. The objectives of this study were: 1) to study the psychometric properties of the MoCA-S; 2) to study the discriminant validity of the MoCA-S for diagnosing MCI and mild dementia in the Chilean population and to compare the test to the MMSE, which is the validated screening test in Chile; and 3) to determine which MoCA items show the greatest sensitivity for detecting MCI and dementia.

Patients and methods

Design

We performed a cross-sectional, mixed (descriptive and correlational) validation study of a diagnostic test.

Participants

We recruited individuals aged ≥ 60, either with cognitive impairment or without cognitive impairment or any other disease affecting the assessment. Participants were recruited from community centres for elderly people (Recoleta, Santiago Centro, and Las Condes) and at hospitals and health centres (Hospital Clínico Universidad de Chile; Consultorio Rosita Renard, Ñuñoa; and Hospital San José, Melipilla). We excluded individuals who were illiterate or had significant sensory alterations, decompensated medical conditions, psychiatric disorders, or moderate-to-severe dementia. Participants were clinically assessed and classified as cognitively normal controls or patients with mild dementia or MCI (which was in turn subdivided into aMCI and non-amnestic MCI [naMCI]). We estimated the minimum required sample size for controls, MCI, and mild dementia using 3 likelihood ratio contingency tables, with a unilateral hypothesis, prevalence and false positive rate, using the Fleiss correction, aiming to attain a minimum of 75% for sensitivity and specificity.

Procedure

Our study was approved by the Ethics Committee of the Faculty of Medicine and Hospital Clínico de la Universidad de Chile. All participants signed informed consent forms. The gold standard technique was clinical diagnosis based on the Clinical Dementia Rating (CDR)²⁹ semi-structured interview, the Diagnostic and Statistical Manual of Mental Disorders (4th edition) criteria for dementia, and the Petersen et al.^6,8 and Albert et al.⁹ criteria for MCI. CDR scores of 0, 0.5, and 1 correspond to cognitively healthy individuals, MCI, and mild dementia, respectively.²⁹ Diagnosis of aMCI or naMCI was defined according to the patient’s/informant’s main complaint (memory or other cognitive function), according to the 6 subitems of the CDR (memory, orientation, judgement and problem-solving, community affairs, home and hobbies, and personal care).²⁹ Patients were diagnosed with aMCI if they displayed progressive deterioration of episodic memory but no significant impairment of functional status (this corresponds to very mild AD, or MCI due to AD according to the Albert et al.⁹ criteria); individuals with cognitive alterations in domains other than memory were diagnosed with naMCI. The latter group therefore included all individuals with cognitive impairment but not aMCI, constituting a very heterogeneous group. The MoCA-S and MMSE questionnaires were administered by 2 trained professionals at the participating community and health centres. MMSE results for the naMCI group were not included in the analysis due to missing data. We adapted the official version of the MoCA-S (http://www.mocatest.org/pdf_files/test/MoCA-Test-Spanish.pdf) to Chilean Spanish via minor lexical substitutions; as per the recommendations of the authors, scores were given for 7 subitems: visuospatial/executive, naming, attention, language, abstraction, delayed recall, and orientation (www.mocatest.org). Within 3 months of MoCA-S and MMSE administration, participants were evaluated by neurologists blinded to the test results in the presence of a valid informant. The same test was applied to subsamples of participants 10 days after the first assessment to calculate inter- and intra-rater reliability.

Statistical analysis

We determined the discriminant validity of the MoCA-S using 3 types of analysis: 1) we compared different groups’ MoCA-S performance (corrected for age, sex, and level of education of the different groups) using the ANCOVA test and Bonferroni post hoc test; 2) calculated standardised mean differences, 95% confidence intervals, and Cohen d effect sizes to determine the magnitude of the differences³⁰; and 3) constructed receiver operating characteristic (ROC) curves and compared the area under the ROC curve (AUC) according to the Hanley and McNeil³¹ method. A z statistic ≥ 1.5 was considered statistically significant; AUC ≥ 0.9 was classified as excellent, 0.8-0.89 as good, and < 0.8 as poor.³² Sensitivity, specificity, positive and negative predictive values, and 95% confidence intervals were calculated. The Youden index was used to determine the cut-off point for optimal performance, with specificity prioritised over sensitivity to reduce false positives. Convergent validity between the MoCA-S and the MMSE was determined using the Pearson correlation analysis. We assessed internal consistency using the Cronbach α and reliability using the Spearman correlation coefficient. Statistical analysis was performed using version 17 of the SPSS statistics software.

Results

Sample characteristics

Of 404 individuals recruited, 172 were included in the study sample after the inclusion and exclusion criteria were applied. Participants were classified as cognitively normal controls (104 individuals, CDR = 0); patients with MCI (48, CDR = 0.5), of which 24 had aMCI and 24 had naMCI; and patients with mild dementia (20, CDR = 1). All patients with cognitive impairment (MCI or dementia) were included in the “all cognitive impairment” group (Table 1). No statistically significant intergroup differences were observed in age or years of schooling. Thirty individuals (17%) had ≤ 6years of schooling. The dementia group had a higher proportion of men.

Table 1.

Demographic data and mean MoCA-S, MoCA-S1, MoCA-S1-2, and MMSE scores.

	Controls (n = 104)	aMCI (n = 24)	naMCI (n = 24)	Mild dementia (n = 20)	All cognitive impairment (n = 68)
Age, years	72.3 ± 5.4	75.3 ± 7.8	74.3 ± 7.9	75.1 ± 8.2	74.9 ± 7.8
Education, years	11.4 ± 4.2	9.2 ± 3.8	12.0 ± 4.3	11.3 ± 4.37	10.8 ± 4.4
Women, n (%)	68 (65)	17 (71)	14 (58)	4 (20)^*	35 (51.5)
MoCA-S1-2 score	24.2 ± 3.7	17.0 ± 3.9^a,c	21.8 ± 4.37	14.6 ± 4.3^a,c	18.2 ± 5.2^a
Visuospatial/executive (max. 5 points)	3.8 ± 1.1	2.3 ± 1.2^a	3.0 ± 1.5^a	2.2 ± 1.3^a,c	2.5 ± 1.4^a
Naming (max. 3 points)	2.7 ± 0.6	2.1 ± 0.8^a	2.5 ± 0.8	2.5 ± 0.7	2.4 ± 0.8^a
Attention (max. 6 points)	4.8 ± 1.4	2.8 ± 1.3^a,c	4.3 ± 1.4	3.1 ± 1.9^a,c	3.4 ± 1.7^a
Language (max. 3 points)	2.0 ± 1.0	1.0 ± 0.9^a,c	1.8 ± 0.9	1.1 ± 0.9^a	1.3 ± 1.0^a
Abstraction (max. 2 points)	1.3 ± 0.7	0.9 ± 0.7	1.3 ± 0.7	0.9 ± 0.8	1.0 ± 0.7^a
Delayed recall (max. 5 points)	3.0 ± 1.5	1.5 ± 1.4^a,c	2.8 ± 1.6	0.4 ± 0.7^a,b,c,d	1.6 ± 1.6^a,c
Orientation (max. 6 points)	5.8 ± 0.5	5.5 ± 0.8	5.8 ± 0.5	3.8 ± 1.5^a,b,c,d	5.1 ± 1.3
Total MMSE score	27.4 ± 2.2	26.0 ± 3.0	NT	21.9 ± 4.5^a,b	24.1 ± 4.2^a

Demographic data, total MMSE and MoCA-S1-2 scores, and MoCA-S1-2 subitem scores for the control, aMCI, ncMCI, mild dementia, and all cognitive impairment groups. Data are expressed as means ± SD.

aMCI: amnestic mild cognitive impairment; MMSE: Mini–Mental State Examination; MoCA-S: original Spanish-language version of the Montreal Cognitive Assessment; MoCA-S1: Spanish-language version of the Montreal Cognitive Assessment, corrected for years of schooling (+1 point for individuals with ≤ 12 years of schooling); MoCA-S1-2: Spanish-language version of the Montreal Cognitive Assessment, corrected for years of education (+2 points for individuals with ≤ 8 years of schooling and +1 point for individuals with 8-12 years); naMCI: non-amnestic mild cognitive impairment; NT: not tested.

a

Lower score than controls.

b

Lower score than aMCI.

c

Lower score than naMCI.

d

Lower score than all cognitive impairment.

*

P < .05 vs control and aMCI groups.

Psychometric properties of the MoCA-S

The 7 subitems of the MoCA-S (visuospatial/executive, naming, attention, language, abstraction, delayed recall, and orientation) had a Cronbach α of 0.772, considered to indicate good internal consistency. Inter-rater reliability was very good (two-tailed Spearman correlation coefficient, 0.846; P < .01). Intra-rater reliability (n = 49) was good and statistically significant for all MoCA-S subitems with the exception of orientation (P = .308). The most stable items were attention, language, delayed recall, and naming. Inter-rater (test-retest) reliability was 0.922 (P < .001). We identified an inverse correlation with age (r = 0.237; P = .004) and a direct correlation with years of schooling (r = 0.408; P < .001). Correlation with MMSE score showed significant convergent validity (r = 0.661; P < .001).

Given the strong correlation between MoCA-S score and years of schooling in our and other studies^23,27 and the fact that 17% of participants had ≤ 6years of schooling (20%-30% among Chileans aged > 65, according to the 2007 Chile National Socioeconomic Characterisation Survey³³), we tested discriminant validity by adding one point to the scores of individuals with ≤ 12years of schooling (MoCA-S1), as recommended by Nasreddine et al.,²¹ or 2 points to individuals with ≤ 8years of schooling and 1 point to participants with 8-12 years (MoCA-S1-2). Our results, figures, and tables are based on the latter modification, which showed the best discriminant validity.

Discriminant validity of the MoCA-S

Comparison of mean MoCA-S scores between groups

Table 1 shows means ± standard deviations (SD) for total MoCA-S1-2 score, total MMSE score, and MoCA subitem scores. Fig. 1 shows means ± SD for MoCA-S1-2 and MMSE scores in each cognitive classification group. ANCOVA analysis (covariates: age, years of schooling, and sex) with post hoc Bonferroni correction showed significant differences between mean MoCA-S1-2 scores for all groups (F = 45.2; P < .001). Controls scored significantly higher than patients with aMCI and mild dementia and the naMCI group scored significantly higher than the aMCI and mild dementia groups; no significant differences were observed between the mild dementia and aMCI groups or between the naMCI and control groups (Table 1; Fig. 1). MMSE scores only showed significant differences between the mild dementia and the control and aMCI groups (F = 19.3; P < .001); no differences were observed between controls and patients with aMCI.

Figure 1.

Comparison of total MoCA-S1-2 and MMSE scores for the aMCI, naMCI, mild dementia, and control groups. Significant differences between groups are indicated with parentheses. aMCI: amnestic mild cognitive impairment; MMSE: Mini–Mental State Examination; MoCA-S1-2: Spanish-language version of the Montreal Cognitive Assessment, corrected for years of schooling; naMCI: non-amnestic mild cognitive impairment.

Effect size

The MoCA-S1-2 showed a moderate effect size for diagnosing aMCI (Pearson r = 0.692; Cohen d = 1.90) and a good effect size for diagnosing mild dementia (r = 0.79; d = 2.55). A small effect size (r = 0.277; d = 0.58) was observed for diagnosing naMCI. For the MMSE, effect size was moderate for diagnosing dementia (r = 0.681; d = 1.86) and small for aMCI (r = 0.22; d = 0.45).

ROC curve analysis

Table 2 shows AUC values for the MoCA-S1-2 and MMSE tests for discriminating controls from patients with aMCI, naMCI, mild dementia, and all cognitive impairment. The MoCA-S1-2 showed an excellent capacity for differentiating between controls and patients with aMCI (AUC > 0.9); this was significantly better than the AUC for the MMSE (> 0.6 ,z = 3.69) (Fig. 2). For diagnosing mild dementia, the discriminant capacity of the MoCA-S1-2 was excellent and the MMSE’s was good (with the MoCA-S1-2 showing a non-significant trend towards better discrimination; z = 1.39). For naMCI, no data are available for the MMSE, whereas the MoCA-S1-2 showed poor discriminant capacity (AUC > 0.6). The MoCA-S1-2 showed good discriminant validity for all cognitive impairment (AUC > 0.8) (Table 2; Fig. 2).

Table 2.

Diagnostic usefulness of the MoCA-S1-2 for differentiating controls from patients with aMCI, naMCI, mild dementia, and all cognitive impairment.

Diagnostic group	Test	Area under the ROC curve	Standard deviation	Significance level	95% confidence interval
					Lower limit	Upper limit
aMCI	MoCA-S1-2	0.903	0.030	.000	0.844	0.961
	MMSE	0.652	0.061	.021	0.532	0.772
naMCI	MoCA-S1-2	0.629	0.066	.075	0.501	0.757
Mild dementia	MoCA-S1-2	0.957	0.018	.000	0.921	0.993
	MMSE	0.890	0.043	.000	0.899	0.987
Cognitive impairment	MoCA-S1-2	0.820	0.034	.000	0.753	0.886

aMCI: amnestic mild cognitive impairment; MMSE: Mini–Mental State Examination; MoCA-S1-2: Spanish-language version of the Montreal Cognitive Assessment, corrected for years of education; naMCI: non-amnestic mild cognitive impairment; ROC: receiver operating characteristic.

Figure 2.

Comparison between ROC curves for the MoCA-S1-2 and MMSE tests for discriminating between controls and patients with aMCI, naMCI, mild dementia, and all cognitive impairment (aMCI + naMCI + mild dementia). aMCI: amnestic mild cognitive impairment; MMSE: Mini–Mental State Examination; MoCA-S1-2: Spanish-language version of the Montreal Cognitive Assessment, corrected for years of schooling; naMCI: non-amnestic mild cognitive impairment.

Sensitivity, specificity, and predictive values of the MoCA-S

The MoCA-S1-2 score showing the best performance for distinguishing controls from patients with aMCI, according to the Youden index, was < 21 (sensitivity, 75%; specificity, 82%) (Table 3). The sensitivity and specificity of this cut-off point for diagnosing naMCI were 42% and 82%, respectively. A cut-off point of < 20 was identified for discriminating between patients with mild dementia and controls (90% sensitivity, 86% specificity). The same cut-off point was identified for all cognitive impairment. Table 3 shows positive and negative predictive values for the MoCA-S1-2, assuming a prevalence of 5%-30% for cognitive impairment among individuals aged >65.

Table 3.

Proposed cut-off points for MoCA-S1-2 scores for different population prevalences of cognitive impairment and dementia.

Score	S	Sp	YI	Predictive value	Population prevalence
					0.05	0.1	0.15	0.2	0.3
aMCI
< 20	0.71	0.86	0.56	Positive	0.64	0.63	0.62	0.62	0.60
				Negative	0.75	0.77	0.78	0.80	0.83
< 21	0.75	0.82	0.57	Positive	0.73	0.72	0.72	0.71	0.69
				Negative	0.68	0.70	0.71	0.72	0.75
< 22	0.79	0.76	0.55	Positive	0.85	0.84	0.83	0.82	0.79
				Negative	0.59	0.60	0.61	0.62	0.64
naMCI
< 21	0.42	0.82	0.23	Positive	0.35	0.34	0.33	0.32	0.30
				Negative	0.70	0.73	0.75	0.78	0.84
< 22	0.42	0.76	0.18	Positive	0.40	0.39	0.38	0.37	0.34
				Negative	0.61	0.64	0.67	0.69	0.75
Mild dementia
< 20	0.90	0.86	0.76	Positive	0.94	0.94	0.93	0.92	0.91
				Negative	0.74	0.75	0.75	0.76	0.77
Cognitive impairment
< 20	0.62	0.89	0.5	Positive	0.22	0.37	0.49	0.57	0.70
				Negative	0.98	0.95	0.93	0.90	0.84

aMCI: amnestic mild cognitive impairment; naMCI: non-amnestic mild cognitive impairment; ROC: receiver operating characteristic; S: sensitivity; Sp: specificity; YI: Youden index.

Values with the highest Youden index score are shown in bold.

Psychometric performance by subitem

In the ROC curve analysis by subitem, the control and aMCI groups showed statistically significant differences between means in all domains except orientation (Table 4). The subitems with the greatest discriminant capacity were attention, visuospatial/executive, language, and delayed recall. The only significant difference observed between the naMCI and control groups was in the visuospatial/executive subitem. Significant differences were observed between the control and mild dementia groups for all subitems except abstraction and naming; delayed recall, orientation, visuospatial/executive, and language showed the greatest discriminant capacity. The aMCI group performed significantly worse than the naMCI group in the attention, language, and delayed recall domains (P < .05) and significantly better than the mild dementia group in delayed recall and orientation (P < .005). Finally, the naMCI group scored higher than the mild dementia group for attention, language, delayed recall, and orientation (P < .05).

Table 4.

Comparison of areas under the ROC curve for MoCA-S1-2 subitems for differentiating controls from patients with aMCI, naMCI, and mild dementia.

Cognitive classification MoCA item	Area under the ROC curve	Standard deviation	Significance level	95% confidence interval
				Lower limit	Upper limit
aMCI
Visuospatial/executive	0.799	0.046	.000	0.709	0.888
Naming	0.724	0.063	.001	0.601	0.846
Attention	0.823	0.046	.000	0.733	0.913
Language	0.773	0.047	.000	0.680	0.866
Abstraction	0.645	0.062	.030	0.522	0.767
Delayed recall	0.754	0.053	.000	0.649	0.858
Orientation	0.619	0.070	.073	0.483	0.756
naMCI
Visuospatial/executive	0.666	0.065	.013	0.539	0.792
Naming	0.549	0.068	.463	0.415	0.683
Attention	0.566	0.064	.319	0.440	0.692
Language	0.583	0.063	.212	0.459	0.707
Abstraction	0.517	0.064	.794	0.392	0.643
Delayed recall	0.528	0.067	.672	0.397	0.659
Orientation	0.534	0.068	.612	0.400	0.668
Mild dementia
Visuospatial/executive	0.817	0.052	.000	0.716	0.918
Naming	0.582	0.074	.251	0.437	0.727
Attention	0.737	0.059	.001	0.621	0.853
Language	0.758	0.056	.000	0.649	0.867
Abstraction	0.632	0.070	.066	0.495	0.769
Delayed recall	0.916	0.028	.000	0.862	0.969
Orientation	0.897	0.051	.000	0.798	0.996

Italics: statistical significance < .05; bold italics: statistical significance < .05 and area under the ROC curve ≥ 0.8.

aMCI: amnestic mild cognitive impairment; MoCA: Montreal Cognitive Assessment; naMCI: non-amnestic mild cognitive impairment; ROC: receiver operating characteristic.

Discussion

We determined the psychometric properties and discriminant validity of total and subitem MoCA-S scores for diagnosing aMCI, naMCI, and mild dementia, with clinical diagnosis as the gold standard, in a sample of 172 individuals recruited from several community and health centres in Santiago de Chile. The MoCA-S has good psychometric properties, although the number of years of schooling has a strong effect; therefore, we added 1 point to the scores of individuals with 8-12 years of schooling and 2 points to individuals with < 8 years (MoCA-S1-2). This modification of the test was valid and useful for diagnosing aMCI and mild dementia, but performed worse for naMCI. The MoCA-S1-2 performed better than the MMSE for discriminating aMCI and mild dementia. The visuospatial/executive domain was the only subitem showing discrimination in all comparisons, whereas attention and delayed recall showed strong discriminant validity for diagnosing aMCI and mild dementia, respectively.

The mean scores for the different diagnostic groups in our study are within the ranges reported in the literature for MoCA-S, although scores vary greatly depending on participants’ level of schooling. In the validation study performed in Spain in a slightly younger population (mean age, 68.8 vs 73.3 years in the present study) with a lower mean level of schooling (7.15 vs 11.0 years) Lozano Gallega et al.²³ report a similar mean score (after correcting for years of education) to our own for the MCI group, although their mean score for the control group is more than 3 points lower (19.83 vs 23.4). The only Latin American validation studies of the MoCA-S were performed in Colombia. Gómez et al.²⁸ report that a low level of education had a very strong effect on MoCA-S score in a population of elderly individuals without dementia and with a very low level of education (mean, 4.8 years). Illiterate individuals and those with less than 5 years of schooling had a mean score of 17.1, whereas those who had completed primary school (5 years) or had more than 5 years of schooling scored 18.9 and 21.6, respectively. Another Colombian study, in which participants’ mean age was 73 years, reports considerable variation in MoCA performance according to years of schooling and socioeconomic status, with mean scores of 15 in individuals with no education and 25 in individuals who had completed university study.³⁴ In another study, performed in a sample from Bogotá with a high level of education (mean, 12 years), Gil et al.²⁶ found significantly higher scores, with a mean of 25.2 in the control group.

The cut-off point of < 21 for MCI is lower than that indicated in the original version,²¹ but is consistent with the majority of international validations of the MoCA. Community-level validation studies performed in the United States also report lower cut-off points: Luis et al.²⁴ indicate a cut-off point of < 23, as a cut-off point of < 26 had very poor specificity (35%); Rossetti et al.³⁵ report a mean score of 23 in a sample of 2653 individuals without cognitive impairment, with 66% of participants falling below the cut-off point of 26. MoCA-S validation studies also recommend lower cut-off points. The Spanish study indicates cut-off points of < 21 for MCI and < 14 for dementia,²³ whereas the Colombian study of individuals with a high level of education suggests cut-off points of < 23 for MCI and dementia.²⁶ Validation studies performed in Asian countries also report lower cut-off points than that proposed in the original version, recommending similar scores to our own: < 22 in the Cantonese version³⁶ and < 23 in the Beijing study.³⁷ A recent validation study of the Cantonese version of the MoCA, performed in southern China,³⁸ recommends cut-off points of < 23 for aMCI (78% sensitivity, 73% specificity) and < 20 for dementia (94% sensitivity, 92% specificity). The advantages of prioritising sensitivity over specificity in the detection of cognitive impairment are subject to debate.³⁹ Given the costs associated with referring individuals classified as false positives to specialist care, we prioritised specificity, which results in lower cut-off points.

The literature includes few studies into the use of the MoCA to discriminate between patients with aMCI and those with naMCI. A study of patients with cardiovascular disease found a sensitivity of 100% for aMCI and 83% for naMCI and specificity of 50% for aMCI and 53% for naMCI for a cut-off point of < 24.⁴⁰ Another study of individuals assessed a year after suffering stroke or transient ischaemic attack, who were classified as having aMCI or naMCI, found the MoCA to be most useful for diagnosing aMCI. A cut-off point of < 25 gave a sensitivity of 100% and a specificity of 83% for diagnosing aMCI, and a sensitivity of 75% for naMCI. The authors argue that the MoCA’s low sensitivity for detecting naMCI may be related with the lack of subitems testing information processing speed.^41,42 Our results with the MoCA-S1-2 with a cut-off point of < 21 show very good discrimination for aMCI (AUC = 0.903; 82% specificity, 75% sensitivity). In the naMCI group, on the other hand, the MoCA-S1-2 showed poorer discrimination (AUC = 0.629; 42% sensitivity, 82% specificity); this is probably due to the heterogeneity of this subgroup of our sample. However, the naMCI group also had a lower level of cognitive impairment, showing no significant difference in mean MoCA score with respect to the control group. A MoCA-S1-2 cut-off point of < 20 showed very good discrimination for detecting dementia (AUC = 0.957; 90% sensitivity, 86% specificity) (Table 3; Fig. 2).

Comparison of MoCA-S1-2 and MMSE ROC curves for distinguishing controls from patients with aMCI showed significantly better AUCs for MoCA-S1-2 (0.903 vs 0.652; z = 3.7). In contrast, both tests performed well for discriminating between the control and dementia groups; the MoCA-S had a slightly better AUC (0.957 vs 0.890 for the MMSE; z = 1.4), although this difference was not significant. The MMSE is therefore an appropriate test for detecting dementia but not MCI.

Surprisingly, delayed recall was not the most discriminant subitem for detecting aMCI. This may be due to the fact that 5 words are insufficient for detecting mild amnesia; a similar phenomenon occurs with such “brief” memory tests as the Memory Impairment Screen,⁴³ which includes 4 words and is valid for detecting dementia but not MCI. Similarly, the memory task of the Fototest includes 6 objects and is sensitive for detecting dementia but not MCI.⁴⁴ More extensive memory tests have been shown to have predictive value even for cases of MCI that are likely to convert to dementia.⁴⁵

The visuospatial/executive subitem, on the other hand, discriminated between controls and all other groups; this was the only domain in which the MoCA detected naMCI. It also had a good level of discrimination for aMCI and mild dementia, with an AUC ≥ 0.8. Previous studies have demonstrated similar high sensitivity^46,47; this may be explained by the fact that executive function is affected early in AD and other dementias.⁴⁵ Patients with aMCI scored slightly lower for attention than those with mild dementia (non-significant difference). This may be because members of the aMCI group had a very low level of education and this item has the strongest correlation with years of schooling. Attention, delayed recall, and language also showed good discrimination between controls and patients with aMCI and mild dementia. It should be noted that the 3 subitems showing the best discriminant capacity (visuospatial/executive, attention, and language) each include multiple tasks. Attention, delayed recall, and language showed the best test–retest reliability. We may therefore consider whether the visuospatial/executive, language, attention, and delayed recall domains are valid and reliable measures in populations with medium-to-high levels of education, as in our sample, and whether they may be used in isolation. In fact, several abbreviated versions of the MoCA test have been developed: the mini-MoCA,⁴⁶ short-MoCA,⁴⁷ MoCA 5 min protocol,⁴⁸ T-MoCA short,⁴⁹ and the MEFO⁵⁰; these tests either use parts of these 4 items or use the orientation domain instead of the visuospatial in order to administer the test by telephone. These versions show very good validity compared to the original test: the T-MoCA short,⁴⁹ MoCA 5 min protocol,⁴⁸ and MEFO⁵⁰ include memory (delayed recall), fluency (phonology with the letter P), and orientation, but score them differently; the MEFO is a 13-point scale validated in Chile by our study group.⁵⁰

One limitation of our study is that the level of education in our sample was higher than the national average; we also excluded illiterate individuals. Therefore, our data cannot be extrapolated to populations with a lower level of education, such as that included in the Colombian study by Gómez et al.²⁸ According to our results, the most discriminant items are strongly correlated with years of schooling (r > 0.3); Gómez et al.²⁸ found that individuals with low levels of education had the greatest difficulty with the visuospatial/executive (inability to complete the trailmaking, clock, or cube tasks), attention (inability to perform the serial subtractions), and phonological fluency subitems (poor vocabulary). Screening of these populations therefore requires adapting the test by substituting these items for others, or using a different test.²⁸ In conclusion, we consider that the MoCA-S is useful and reliable for diagnosing aMCI and dementia in populations with medium-to-high levels of education, but do not recommend its use in populations with very low levels of education.

Funding

This study was financed through the following grants: Fondecyt1110189 and 1151297 (MIB) and ACT1403 (CD).

Conflicts of interest

The authors have no conflicts of interest to declare.