Validity and reliability of the Spanish Version of the Montreal Cognitive Assessment (MoCA) for the detection of cognitive impairment in Mexico

Abstract

Objective

To establish the validity and reliability of the Montreal Cognitive Assessment in Spanish (MoCA-S) to identify mild cognitive impairment (MCI) and dementia in the Mexican elderly population.

Material and methods

168 participants from a memory clinic in Mexico City were enrolled and divided into 3 groups: 59 cognitively healthy (CHG), 52 with mild cognitive impairment (MCI) (DSM-5 criteria) and 57 with dementia (NINCDS-ADRDA criteria). The MoCA-S and Mini-Mental State Evaluation (MMSE) were applied at baseline and during the last months to establish intra-observer reliability. ROC curves and a multinomial regression model were constructed to evaluate the effect of age and education on MoCA-S performance.

Results

The mean age of the participants was 76 ± 8.1 years and the education rate was 10.7 ± 5.2. The MoCA-S scores by group were: CHG, 27.3 ± 1.9; MCI, 22.9 ± 2.9; and dementia, 13.7 ± 4.9 (p < 0.001). The reliability of the MoCA-S was 0.89 and the intraclass correlation coefficient was 0.955. Sensitivity was 80% and specificity was 75%, with a cut-off point of 26 points for MCI (area under the curve, 0.886; p < 0.001). For the dementia group, the sensitivity was 98% and specificity was 93%, with a cut-off point of 24 points (area under the curve, 0.998; p < 0.001). The multinomial regression showed no association with education and age for both the MCI and dementia groups.

Conclusions

The MoCA-S is a valid and reliable instrument for MCI and dementia screening in the Mexican population, even after adjusting for age and education.

Resumen

Objetivo

Establecer la validez y confiabilidad del Montreal Evaluación Cognitiva en Español (MoCA-E) para identificar deterioro cognitivo leve (DCL) y demencia en adultos mayores mexicanos.

Material y métodos

Se incluyó a 168 participantes en una clínica de memoria de la ciudad de México, en 3 grupos: 59 cognitivamente sanos (GCS), 52 con DCL (criterios del DSM-V) y 57 con demencia (criterios NINCDS-ADRDA). Se aplicó el MoCA-E y el Mini-Mental State Evaluation al inicio y en los últimos meses, para establecer la confiabilidad intraobservador. Se construyeron curvas ROC y un modelo de regresión multinomial para evaluar el efecto de la edad y la escolaridad en el desempeño del MOCA-E.

Resultados

El promedio de edad de los participantes era 76 ± 8,1 años; la tasa de escolaridad, 10,7 ± 5,2. Las puntuaciones de MoCA-E por grupo fueron: GCS, 27,3 ± 1,9; DCL, 22,9 ± 2,9, y demencia, 13,7 ± 4,9 (p < 0,001). La confiabilidad del MoCA-E fue 0,89 con un coeficiente de correlación intraclase de 0,955. La sensibilidad fue del 80% y la especificidad, del 75% con el punto de corte de 26 puntos para DCL (área bajo la curva = 0,886; p < 0,001). Para demencia, la sensibilidad fue del 98% y la especificidad, del 93% con el punto de corte de 24puntos (área bajo la curva = 0,998; p < 0,001). La regresión multinomial no mostró asociación con la escolaridad y la edad tanto para DCL como para demencia.

Conclusiones

El MoCA-E es un instrumento con validez y confiabilidad para el cribado de DCL y demencia en la población mexicana, aun después de ajustar por edad y escolaridad.

Introduction

Cognitive impairment is a significant cause of morbidity and mortality associated with population ageing worldwide, including in developing countries like Mexico,¹ where the annual incidence of dementia in the over 65 s is 30.4 per 1000 people. It is also estimated that in Mexico 8% of people over 65 years of age might have some form of cognitive impairment.^2,3 On the other hand, the concept of mild cognitive impairment (MCI) stems from the need to identify people presenting memory or other cognitive deficits which are not severe enough to support a dementia diagnosis.^4,5 The presence of MCI has been associated with a greater risk of dementia, primarily the amnestic type, which most often progresses to Alzheimer's disease.^6,7 A neuropsychological assessment is crucial for the diagnosis of MCI. However, many of the screening tests available do not differentiate this form of cognitive impairment, so it is important and necessary to have tests that facilitate the identification of people with memory problems in subclinical stages.⁸

The Montreal Cognitive Assessment (MoCA) is a short screening test to evaluate cognitive function and explores six domains: memory (5 points), visuospatial capacity (4 points), executive function (4 points), attention/concentration/working memory (5 points), language (5 points) and orientation (6 points). The score ranges from 0 to 30 points, and the higher the score, the better the cognitive function. The administration time is approximately 10 min and 1 point is added to subjects who have spent ≤12 years in education. Its sensitivity and specificity for detecting patients with Alzheimer's disease are 100% and 87%, respectively, and 90% and 87% for the diagnosis of MCI.^9,10 It has also been shown that, for cognitive function testing, it generally performs better than other more common tests, such as the Mini-Mental State Examination (MMSE).¹¹ It has been translated into multiple languages, and the Spanish version (MoCA-E) has now been validated in Colombia and Spain.^12–14 To our knowledge, the MoCA-E is still yet to be validated in the Mexican population. Thus, the objective of our study is to establish the validity and reliability of the MoCA-E for MCI and dementia screening in elderly Mexican adults.

Material and methods

A cross-sectional validation study that included 168 subjects aged over 60. All of the participants signed an informed consent form and underwent a clinical and cognitive assessment at the memory clinic of a tertiary hospital in Mexico City, in the period between March and December 2015. The calculated sample was estimated with the aim of critically studying diagnostic and validation performance by comparing the MoCA and MMSE with a moderate correlation, an error (α = 5%) and a power of 80%. At least 51 patients were needed per group to test diagnostic performance and at least 23 per group for the validation hypothesis (cognitively healthy [CHG], MCI and dementia groups).

Geriatric and/or neurology specialists assessed the patients, based on the criteria proposed by the National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer's Disease and Related Disorders Association (NINCDS/ADRDA).¹⁵ The Clinical Dementia Rating (CDR)¹⁶ classified subjects with dementia, comprising categories 0 (cognitively healthy), 0.5 (MCI) and ≥1 (dementia). Moreover, the Diagnostic and Statistical Manual of Mental Disorders (DSM–5)¹⁷criteria for mild neurocognitive disorder or MCI were used, which include:

• 1.

  Evidence of cognitive decline from a previous level of performance in one or more cognitive domains: (a) concern of the individual, a knowledgeable informant or the physician that there has been a significant decline in cognitive function; (b) a modest decline in cognitive performance, preferably documented by standardised neuropsychological testing or, in its absence, another quantified clinical assessment.

• 2.

  The cognitive deficits do not interfere with capacity for independence in everyday activities (i.e. instrumental activities are preserved but require greater effort or need compensatory strategies or adaptation).

• 3.

  The cognitive deficits do not occur exclusively in the context of a delirium.

d. The cognitive deficits are not better explained by another mental disorder.

In this study, a score of <2.58 on the Bayer – Activities of Daily Living (B-ADL) scale¹⁸ was taken into account to determine functional independence. To complete the neuropsychological assessment, the NEUROPSI¹⁹ (Mexican neuropsychological test standardised by age and education) was employed; subjects rated with a score of >1.5 standard deviations were considered to meet the MCI criterion.

Sociodemographic variables such as gender, age, education and the Katz²⁰ and Lawton²¹ indices were obtained from the comprehensive geriatric assessment.

Patients with severe or uncontrolled toxic, metabolic, infectious or vascular neurological diseases; uncontrolled psychiatric disorders, such as depression and/or schizophrenia; heart, liver or kidney disease; cancer; or any other uncontrolled systemic disease were excluded.

Statistical analysis

The validity of the instrument's content was already established by the original authors in 2005.⁹ To that effect, Spearman's correlation coefficient was used to determine the validity of the construct (convergence) by comparing the MoCA-E to the MMSE (standardised by education and age.²² Analysis of variance (ANOVA) was used to evaluate the sociodemographic differences, as well as to establish the criterion validity on testing the MoCA-E's performance against the reference standard (clinical assessment) among the three groups.

Reliability was analysed using test-retest from estimating the intraclass correlation coefficient in a three-month interval; 23 CHG subjects, 23 with MCI and 23 with dementia were reassessed by two different observers who did not know their clinical diagnosis (APG, SGAN).

To determine the internal consistency index, Cronbach's coefficient was applied,²³ which is considered very good if >0.80; good from 0.70 to 0.80; moderate from 0.45 to 0.60 and poor if <0.45. To determine concordance, we compared the result obtained on each of the instruments with the clinical diagnosis using the kappa coefficient, with a degree of agreement ≥0.45 deemed acceptable.^24,25

Receiver operating characteristic (ROC) curves were constructed and area under the curve was calculated to estimate sensitivity, specificity, the positive predictive value (PPV) and the negative predictive value (NPV) (95% confidence intervals [95% CI]).²⁶

Finally, a multinomial regression model was constructed to determine the association of age and education with MoCA-E performance both in the MCI and dementia groups.

The SPSS software package, version 20.0 for Windows (SPSS Inc.; Chicago, IL, USA), was used for the statistical analysis. The protocol was approved by the Institutional Ethics Committee (REF. 1158).

Results

The mean age was 70.0 ± 1.1 years in the CHG, 75.0 ± 6.2 in the MCI group and 82.0 ± 5.8 in the dementia group (p < 0.001). 80.4% were female and the mean education level in the CHG was 12.4 ± 3.7 years; in the MCI and dementia groups it was 10.0 ± 5.6 and 9.5 ± 5.8 years, respectively (p = 0.008).

Table 1 shows the comparative analysis of the sociodemographic characteristics and the psychometric performance among the participants. The total mean score of the MoCA-E was 27.2 ± 1.8 in the CHG, 22.9 ± 2.9 in the MCI group and 13.6 ± 4.9 in the dementia group (p < 0.001).

Table 1.

Clinical and cognitive characteristics of the groups.

	Control	MCI	Dementia	p
Subjects (n)	59	52	57
Age (years)	69.91 ± 1.11^a,b	75.15 ± 6.23^a,c	81.70 ± 5.88^b,c	≤0.001
Education (years)	12.41 ± 3.77^b,d	10.08 ± 5.60^b	9.58 ± 5.82^d	0.008
MMSE	28.59 ± 1.24	27.10 ± 2.12	20.44 ± 4.75	0.632
MoCA	27.26 ± 2.00^a,b	22.94 ± 2.93^a,b	13.68 ± 4.99^b,c	≤0.001^e
Katz	5.78 ± 0.42	5.40 ± 0.75	5.05 ± 1.18	0.649
Lawton	7.81 ± 0.54^a,b	6.65 ± 1.7^a,c	2.67 ± 2.1^b,c	≤0.001^e
ADL-B	1.38 ± 0.32^a,b	1.77 ± 1.77	3.30 ± 2.3	≤0.001

MCI, mild cognitive impairment; MMSE, Mini-Mental State Examination; MoCA, Montreal Cognitive Assessment.

The data is presented as the mean ± standard deviation. The analysis shows the differences between the groups using the ANOVA test, post hoc DSM.

a

p ≤ 0.001 between control and MCI.

b

p ≤ 0.001 between control and dementia.

c

p ≤ 0.05 between control and dementia.

d

p ≤ 0.05 between control and MCI.

e

Age and gender covariables.

Table 2 shows the performance characteristics per cognitive domain among the MoCA-E and MMSE, with statistically significant differences observed between both groups.

Table 2.

Cognitive test performance of the control, MCI and dementia groups.

Cognitive domains and tasks assessing them	Control	MCI	Dementia	p
Orientation
Temporal orientation (MMSE)	4.86 ± 0.40^a	4.67 ± 0.64^b	2.91 ± 1.54^a,b	≤0.001
Spatial orientation (MMSE)	4.95 ± 0.30^a	4.83 ± 0.39^b	3.88 ± 1.12^a,b	≤0.001
Orientation (MoCA)	5.97 ± 0.18^a,c	5.42 ± 0.89^b,c	3.63 ± 1.83^a,b	≤0.001
Attention
100-7 (MMSE)	4.45 ± 0.84^a	4.12 ± 1.13^b	2.47 ± 1.73^a,b	≤0.001
Attention section (MoCA)	5.22 ± 1.06^a	4.80 ± 0.99^b	2.50 ± 1.77^a,b	≤0.001
Memory
Registration (MMSE)	3.00 ± 0	2.98 ± 0.14	2.98 ± 0.13	0.586
Evocation (MMSE)	2.50 ± 0.66^a	2.23 ± 0.78^b	0.91 ± 1.05^a,b	≤0.001
Delayed recall (MoCA)	3.48 ± 1.50^a,d	2.48 ± 1.56^b,d	1.00 ± 0.82^a,b	≤0.001
Language
Naming (MMSE)	2.00 ± 0^a	1.98 ± 0.14^b	1.86 ± 0.39^a,b	0.005
Repetition (MMSE)	0.97 ± 0.18^a	0.81 ± 0.39^b	0.74 ± 0.44^a,b	0.003
Following instructions (MMSE)	2.88 ± 0.46^a	2.75 ± 0.62^e	2.42 ± 0.94^a,e	0.002
Reading comprehension (MMSE)	1.03 ± 0.46	1.06 ± 0.64	0.95 ± 0.23	0.423
Writing (MMSE)	1.00 ± 0^a	0.96 ± 0.19^b	0.75 ± 0.34^a,b	≤0.001
Identification (MoCA)	2.95 ± 0.22^a	2.79 ± 0.41^b	2.30 ± 0.82^a,b	≤0.001
Language section (MoCA)	2.54 ± 0.70^a,d	1.84 ± 0.87^b,d	0.84 ± 0.88^a,b	≤0.001
Semantic verbal fluency (animals)	20.44 ± 4.50^a,d	17.37 ± 4.56^b,d	9.49 ± 3.80^a,b	≤0.001
Phonological verbal fluency (letter F)	13.34 ± 3.27^a,d	9.88 ± 3.88^b,d	6.26 ± 3.33^b,d	≤0.001
Visuospatial
Copying the pentagons (MMSE)	0.91 ± 0.28^a,b	0.71 ± 0.45^c,e	0.56 ± 0.50^a,e	≤0.001
Executive functions
Executive functions section (MoCA)	4.53 ± 0.65^a,d	3.40 ± 1.19^b,d	1.93 ± 1.48^a,b	≤0.001
Abstraction (MoCA)	1.88 ± 0.33^a,d	1.63 ± 0.63^b,d	0.30 ± 0.46^a,b	≤0.001

MCI, mild cognitive impairment; MMSE, Mini-Mental State Examination; MoCA, Montreal Cognitive Assessment.

The data is presented as the mean ± standard deviation. The analysis shows the differences between the groups using the ANOVA test, post hoc DSM.

a

p ≤ 0.001 between control and dementia.

b

p ≤ 0.001 between MCI and dementia.

c

p ≤ 0.05 between control and MCI.

d

p ≤ 0.001 between control and MCI.

e

p ≤ 0.05 between MCI and dementia.

^fp ≤ 0.05 between control and dementia.

On comparing the MoCA-E and MMSE (using Spearman's correlation test), the validity of the construct was ρ = 0.830 (p < 0.001).

Reliability

The internal consistency of the MoCA-E, estimated with Cronbach's alpha index, was α = 0.891. The intraclass correlation coefficient was 0.955 (95% CI, 0.918–0.975; p < 0.001).

[1] Sensitivity, specificity and area under the ROC curve

The area under the curve for the MoCA-E was 0.886 (95% CI, 0.826–0.947), with a cut-off value ≤26 points, a sensitivity of 80%, specificity of 75%, PPV of 90% and NPV of 82%. Meanwhile, the MMSE had an area under the curve of 0.721 (95% CI, 0.627–0.818), a sensitivity of 75% and a specificity of 60% for the diagnosis of MCI (Fig. 1).

Fig. 1.

MoCA-E versus MMSE ROC curve in MCI. MCI, mild cognitive impairment; AUC, area under the ROC curve; 95% CI, 95% confidence interval.

Fig. 2 shows the area under the curve for the MoCA-E versus the reference standard in the dementia group, which was 0.997 (95% CI, 0.990–1) with 98% sensitivity and 93% specificity, and a cut-off value of 24 points. As regards the MMSE, the area under the curve was 0.998 (95% CI, 0.993–1), with 88% sensitivity and 100% specificity.

Fig. 2.

MoCA-E versus MMSE ROC curve in dementia. AUC, area under the ROC curve; 95% CI, 95% confidence interval.

Table 3 shows the inverse association between the MoCA-E score in the MCI (odds ratio [OR] = 0.481; 95% CI, 0.360–0.644; p < 0.001) and dementia groups (OR = 0.265; 95% CI, 0.188–0.392; p < 0.001) after the age and education adjustment.

Table 3.

Multinomial regression model between the 3 groups, showing the inverse probability of scoring between CHG, MCI and dementia after making adjustments according to age and education level.

	MCI, OR (95% CI)	p	Dementia, OR (95% CI)	p
Non-adjusted, MoCA	0.493 (0.381–0.637)	<0.001	0.271 (0.193–0.386)	<0.001
Adjusted, MoCA	0.481 (0.360–0.644)	<0.001	0.188 (0.188–0.392)	<0.001

MCI, mild cognitive impairment; CHG, cognitively healthy group (reference category).

The average application time for the test was 10 min.

Discussion

This study once again demonstrates that the MoCA, version 7.0 (translated into Spanish), is a valid and reliable test for the detection of dementia in a population of elderly Mexican adults. The instrument also showed adequate intraobserver reliability (0.95) and adequate internal consistency (0.89). With a cut-off point ≤24, it showed 98% sensitivity and 93% specificity for the diagnosis of dementia. This is compatible with previous publications.^10,27 However, we consider the most salient aspect of this study to be the demonstration of the MoCA-E's capacity to detect MCI in Mexican subjects; sensitivity was 80% and specificity 75%, with a cut-off point ≤26 points, which is higher than the MMSE, as reported previously by Costas et al.²⁸ and Julayanont et al.²⁹

Gil et al. have studied the performance of the Spanish version of the test, and report 89% sensitivity and 80% specificity for the detection of MCI, with a 85% PPV and 85% NPV, establishing a cut-off value ≥23 (a rating ≤22 was considered an abnormal result.¹³

Moreover, something that caused concern regarding the instrument's design was the education adjustment. Assuming that this could directly impact the test's performance, Gómez et al. studied the influence of education on the MoCA-E in a sample of Colombian subjects with dementia and a low level of education (average, 4.8 years). The average scores of the MoCA-E were 16.1/30 points among the illiterate subjects, 18.2/30 among those who did not complete their primary education, and 20.3/30 among those who had a complete primary education (p < 0.001). The most common errors were: the cube and clock drawing, subtraction, serial attention, verbal fluency and abstraction. Test-retest reliability was high (ICC = 0.86; 95% CI, 0.76–0.93), and it was concluded that the cut-off point should be modified according to education.¹² This generated some contradictions about the need to make adjustments due to education and the relevant cultural adaption. However, recently Parunyou et al. demonstrated in Asian subjects with a low education level that, on maintaining the cut-off of 26 points for MCI and 24 for dementia, sensitivity and specificity were >80%, and it should also be emphasised that the various translations of the instrument did not require any form of transformation or cultural adaptation.³⁰

Since the MoCA is considered to be a screening instrument, age-based standardisation is proposed. Particularly with regard to MoCA, various publications have considered the influence of age on the performance of the test and have shown a linear association regarding a worse performance at a higher age.^31–33 In this study, we were able to directly assess the effect of age and its correlation with the total MoCA-E score. A correlation between a higher age and worse score was indeed discovered, but it was possible to differentiate between the MCI and dementia groups.

On the other hand, the MMSE has traditionally been used as a screening test to detect any form of cognitive impairment. The MMSE has demonstrated high sensitivity for the identification of moderate-to-severe stage dementia,²⁰ yet this popular test has proved fairly useless in the identification of early stages, such as MCI. This is a result of the fact that the assessment is geared towards the memory and language functions and does not consider executive functioning, which is significantly implicated in cases of vascular disease, Parkinson's disease³⁴ and other non-Alzheimer's forms of dementia.³⁵ Therefore, the fact that the MoCA is a screening test that incorporates more complex tasks should be considered one of its main virtues since, as mentioned previously, it features domains that are not assessed by the MMSE, including executive functioning, attention and delayed recall, enabling better identification of preclinical stages such as MCI, for example.³⁶ The sensitivity and specificity of the MoCA has also been demonstrated in the detection of MCI in patients with Parkinson's disease,³⁷ vascular cognitive impairment,³⁸ amyotrophic lateral sclerosis, frontotemporal dementia, AIDS dementia complex, Huntington's disease, Lewy body dementia, multiple system atrophy and multiple sclerosis.^39,40 Likewise, in its Spanish and other language versions (French, Arabic, Cantonese, German and Portuguese)^41–43 it has also been shown to discriminate between people with normal cognition and cognitive impairment.⁴⁴ This reflects the universality of the instrument and how easily it is adapted to different cultures, which generally occurs with minimal or no modifications.⁴⁵

The main limitation of our study is that the population comprises patients from a tertiary hospital whose average education level was 11 years, which could generate selection bias. Having used a representative sample to evaluate diagnostic performance and validation ensures neither representativity nor normative data on elderly Mexican adults. Likewise, only having included subjects with Alzheimer's dementia and using the CDR as a test of dementia severity has the same effect. However, to our knowledge, this is the first study to validate the MoCA-E in a Mexican population. Nevertheless, more studies with a greater number of subjects are needed that include this test and prove that the MoCA-E can be reliable and valid in a Spanish-speaking population and that it can be used by various levels of care and/or in the community on people with different education levels and age ranges.

Conclusions

The Spanish version of the MoCA is a reliable and valid instrument for screening MCI. It could prove a good resource for screening MCI and correctly differentiating between people with and without the condition. Moreover, it effectively distinguishes dementia patients from cognitively healthy individuals compared to the MMSE. Nevertheless, more studies are needed that prove that the MoCA-E can be reliable and valid in a Spanish-speaking population and that it can be used in different scenarios, on groups with both high and low education levels.

Ethical disclosures

Protection of human and animal subjects

The authors declare that no experiments were performed on humans or animals for this research.

Confidentiality of data

The authors declare that they have followed the protocols implemented in their place of work regarding the publication of patient data.

Right to privacy and informed consent

The authors have obtained the informed consent of the patients and/or subjects referred to in the article. This document is in the possession of the corresponding author.

Conflicts of interest

The authors have no conflicts of interest to declare.