Memory, fluency, and orientation (MEFO): A five-minute screening test for cognitive decline

Abstract

Background

The prevalence of cognitive impairment (CI) will double in the next 20 years, making early detection a key priority.

Objectives

Validation of a 5-minute CI screening test.

Methods

Adults aged 60 years and older were recruited from memory clinics and the community at large in the Santiago, Chile metropolitan area. Based on clinical examination they were categorised as No CI (NCI), Mild CI (MCI) and dementia sufferers (DS). We measured the validity of a new test, MEFO, evaluating memory (5 points), phonetic verbal fluency (2 points) and orientation (6 points) by comparing its results with those from the MMSE.

Results

We evaluated 214 subjects, comprising 49 with dementia, 47 with MCI, and 118 with no CI. The MEFO differentiated between all 3 groups whereas the MMSE did not discriminate between the MCI and NCI groups. The area under the ROC curve (AUC) for the MEFO distinguishing NCI subjects from dementia sufferers was 0.97; for NCI vs CI (dementia + MCI), 0.89; and for NCI vs MCI, 0.80. On the MMSE these values were 0.95, 0.84, and 0.73, respectively. A cut-off score of 6/7 on the MEFO identified dementia sufferers with a sensitivity of 86% and a specificity of 96%. A cut-off score of 8/9 distinguished CI from NCI subjects with a sensitivity of 83% and a specificity of 75%.

Conclusions

The MEFO is a valid and reliable test for discriminating between dementia and CI sufferers and subjects with no CI. Its validity is similar to that of the MMSE under these conditions, but it is more effective for identifying subjects with MCI and its administration time is shorter.

Resumen

Antecedentes

En los próximos 20 años se duplicará la cantidad de personas con deterioro cognitivo (DC) apremiando su pronta detección.

Objetivos

Validar una prueba de cribado de DC aplicable en < de 5 minutos.

Métodos

Adultos ≥60 años reclutados desde la comunidad y clínicas de memoria de la zona urbana de Santiago de Chile fueron clasificados clínicamente en: Dementes, DC leve (DCL) y Sin DC (SDC). Se evaluó la validez de un nuevo test: MEFO, consistente en la evaluación de: Memoria (5 puntos), Fluidez Fonética (2 puntos) y Orientación (6 puntos) contrastado con los resultados del MMSE.

Resultados

Se evaluaron 214 personas: 49 con demencia, 47 con DCL y 118 SDC. El MEFO diferenció a los 3 grupos clínicos entre sí, mientras que el MMSE no permitió discriminar DCL de SDC. El área bajo la curva ROC en el MEFO para diferenciar los grupos SDC versus: i: Demencia fue = 0,97; ii: DC (dementes + DCL)fue = 0,89; y iii: DCL fue = 0,80; siendo estos valores para el MMSE de 0,95, 0,84, y 0,73, respectivamente. Un punto de corte (PC) del MEFO = 6/7 distinguió a las personas con demencia con una sensibilidad (S) = 86% y especificidad (E) = 96%; y un PC = 8/9 distinguió a aquellos con DC con una: S = 83% y E = 75%.

Conclusiones

El MEFO resultó una prueba confiable y válida para diferenciar a personas con demencia y DC, similar al MMSE en éstos casos, pero con mayor utilidad para distinguir a los pacientes con DCL y de menor duración de aplicación.

Introduction

Because of progressive population ageing, a phenomenon which is especially marked in Asian and Latin American countries,¹ researchers believe that the number of individuals affected by dementia and cognitive impairment will double every 20 years.² While a precise diagnosis will require a detailed clinical evaluation,^3,4 the public health perspective is that there is increasing need for brief, easy-to-use screening tools with good levels of sensitivity and specificity and a minimal educational bias.⁵ Screening tests should be sensitive to the initial stages of cognitive impairment (CI). As a minimum, they must include an evaluation of episodic memory and executive functions, since these are the domains affected the most by Alzheimer disease (AD)⁶ and vascular CI,⁷ which are the most frequent causes of CI.⁸ Several screening tests that can be completed in 10 minutes or less and that have been validated in Spanish are available.^3–5 Those with high levels of sensitivity and specificity include MIS,^9–11 the Clock Drawing Test,¹² the Mini-Cog Test,¹³ the 7-minute screen,¹⁴ Fototest,^15,16 Eurotest,¹⁷ and the Memory Alteration Test (M@T).¹⁸ Within the framework of an international agreement for studying dementia,¹⁹ researchers validated the Mini-Mental State Examination (MMSE),²⁰ in addition to the Pfeffer functional activities questionnaire (PFAQ),²¹ as dementia screening methods. The Chilean study showed that while the MMSE was 93% sensitive with a cut-off point of 21/22, its specificity was low at 46%. When it was associated with PFAQ, however, specificity increased to 83%.²² One of the disadvantages of the MMSE is its sizeable educational bias which results in variations of up to 7 points in the healthy population.²³ Other significant drawbacks are its ceiling effect, inability to measure executive functions, and low sensitivity in amnestic mild CI²⁴ and predominantly subcortical CI.^25,26 In this study, we wish to test the accuracy of a new test named ‘Memory, Fluency and Orientation’ (MEFO), which measures the following: (a) deferred free recall; (b) phonetic fluency for words beginning with ‘P’; and (c) spatial and temporal orientation. These tests were chosen based on the high levels of sensitivity determined by studies on deferred free recall in screening for early AD.^6,9–18 Phonetic verbal fluency testing has proven itself useful for evaluating executive functions,²⁷ and spatial/temporal orientation is a very sensitive parameter for screening for dementia in general.^18,20 Furthermore, evaluating these 3 cognitive domains is recommended as a method of screening for CI in 5 minutes or less.²⁸ The purpose of this study is to validate a Spanish version of MEFO as a screening test for dementia and CI and compare its results with those from the MMSE.

Subjects and methods

The sample was made up of Chilean subjects (≥60 years) of both sexes, residing in Santiago. They were divided into 3 groups as follows: (a) patients with dementia; (b) subjects with mild CI (MCI), and (c) controls with no dementia (CND). Patients with dementia or MCI were prospectively recruited from memory consults at the University of Chile's Hospital Clínico (HCUCH in Spanish) and Hospital Salvador between 2007 and 2011. The control group (CND) was recruited from among family members of patients receiving care at those centres, by word-of-mouth campaigns, and using Santiago's municipal centres. All subjects were assessed by neurologists with experience in dementia. They conducted structured interviews with patients and family members/informants to diagnose each individual's cognitive state and establish the severity of the condition using the Global Deterioration Scale or GDS.²⁹ Two-thirds of patients with dementia had already undergone a neuroimaging study and serological tests that included complete blood count, biochemical profile, and thyroid scan. Diagnoses of dementia were established according to DSM-IV diagnostic criteria.³⁰ PFAQ scores ≥ 5 were considered to indicate an important and significant change in function. Researchers used NINCDS-ADRDA criteria³¹ for probable cases of Alzheimer disease and NINDS-AIREN criteria³² for vascular dementia. Patients who could not be classified because studies were insufficient were placed in the non-specific dementia category. Diagnoses of MCI were established based on International Working Group criteria: the person is neither normal nor demented; there is evidence of cognitive deterioration shown by either objectively measured decline over time and/or subjective report of decline reported by self and/or informant; and complex instrumental functions are either intact or minimally impaired.³³ Control group subjects had experienced no declines in cognitive function according to both self-provided and informant-provided information. Upon recruitment, all subjects signed the informed consent form approved by the HCUCH ethics committee. Exclusion criteria for all 3 groups were as follows: age < 60 years; sensory changes that would prevent use of testing instruments; major depressive disorder; changes in level of consciousness; and lack of reliable informants.

Neuropsychological assessment

Evaluators blinded to the clinical diagnosis assessed all subjects included in the study. The evaluation began with the MEFO and the MMSE. MEFO includes sections testing deferred free recall, verbal fluency, and spatial/temporal orientation. We chose these tests because they are recommended as 5-minute cognitive screening devices by the Group on Standards for Determining Vascular Dementia.²⁷ The session begins with a memory test in which the examiner pronounces 5 frequently used words and the subject is given 2 attempts to repeat them. After that, the subject is asked to utter as many words beginning with ‘P’ as possible in one minute. The orientation-related questions follow that section, and last of all, the examiner asks the subject to list the 5 words from the memory test (Appendix 1). Each subject then completed the MMSE (version validated for Chile).²² The global cognitive state was also evaluated using Addenbrooke's Cognitive Examination Revised Version (ACE-R).³⁴ A depression screening was performed using Yesavage's geriatric depression scale (GDS-15).³⁵ Informants were asked to complete the PFAQ^21,22 and the Alzheimer Disease 8 (AD8), on which a score >2 has been shown to be sensitive for detecting CI.³⁶ This test has been validated in Chile.³⁷

Statistics

SPSS-17^® software was used for the statistical analysis. We completed a descriptive study of the entire sample, including both demographic and neuropsychological data. Results were compared between the 3 cognitive categories using one-way ANOVA corrected for age and educational level. The Bonferroni test was used for post hoc analysis of quantitative variables, and the chi-square test for categorical variables.

To study the influence of demographic variables on the diagnostic category, we performed multivariate logistic regression, using cognitive groups as the dependent variables. The NCI group was compared to the following groups: (a) dementia; (b) CI (MCI + dementia); and (c) MCI only. The independent variables were scores on the MEFO or MMSE cognitive tests (continuous variables), age (continuous variable), education (continuous variable), and sex (dichotomous variable).¹⁸ All of these comparisons were interpreted as significant, with a 5% bilateral error considering a P-value of <.05. Internal consistency was calculated using Cronbach's alpha statistic. To determine test-retest reliability, researchers administered the test on 2 different occasions a month apart and examined how well results correlated with each other. Pearson correlations were found between MEFO and other overall cognitive assessment scales and informant scales to determine convergent validity. This is used as a gold standard method for diagnostic classification of clinical + neuropsychological assessments (without including MMSE or MEFO results). Discriminant validity for MEFO was measured by measuring the area under the ROC curve (AUC). This was done to differentiate between subjects in the dementia, CI (dementia or MCI), MCI, and NCI groups. These values were compared with AUCs for the MMSE using the Hanley and McNeil method of calculating the difference between ROC curves.³⁸ We also determined the diagnostic precision for each test (percentage of individuals categorised correctly). Positive and negative correlation coefficients were calculated to determine the probability of presence or absence of disease.

Results

Sample

We recruited a total of 287 subjects older than 60 years according to inclusion and exclusion criteria. Of these subjects, 73 (45 controls, 11 with MCI and 17 with dementia) were excluded due to having incomplete neuropsychological assessments. These excluded subjects were significantly older than those in the study group (74 ± 8 vs 72 ± 7, P = .035); there were no differences in educational level, sex, or distribution into cognitive groups. The reasons why evaluations were not completed were as follows: drop-outs (n = 19), inability to attend evaluation sessions (n = 23), patient or informant illness (n = 8), loss of contact (n = 11), and unspecified (n = 12). As a result, 214 subjects completed the study; 118 were controls (NCI = 55%), 47 had MCI (MCI = 22%), and 49 had dementia (DEM = 23%). The control group consisted of 118 subjects, of whom 66 (56%) were recruited from municipal adult care centres; 29 (25%) were related to patients or had heard about the study by word of mouth; and 13 were patients receiving care from geriatric departments in the hospitals participating in the study. The 47 members of the MCI group included 10 (21%) with amnestic CI and 37 (79%) with non-amnestic CI according to the results of the clinical/neuropsychological assessment. Of the 49 patients with dementia, 25 (51%) met criteria for probable AD, 7 (14%) for vascular dementia, and 5 (10%) for other types of dementia (3 cases of Lewy body dementia, 1 case associated with Parkinson's disease, and 1 case of frontotemporal dementia). The remaining 12 (25%) were classified as non-specific dementia. Dementia severity in patients with that diagnosis ranged from mild (GDS = 3, n = 3) to moderate (GDS = 4, n = 46).

Demographic data

Mean age was 72 ± 7 years; the NCI group was younger than the others (F = 9.5; P < .001). The mean number of years at school was 11 ± 4, with 45 patients (21%) attending school <6 years, 99 (46%) attending 6 to 12 years, and 70 (33%) attending more than 12 years. The MCI group had a lower educational level than the other 2 groups (F = 6.5; P = .002). It contained a higher percentage of individuals with <6 years at school (44%) than did either the control group (12%) or the dementia group (24%) (chi-square = 18, P = .001). The study contained a total of 122 women (57%), with a smaller percentage of women in the dementia group (39%) than in the NCI group (66%) (chi-square = 11; P = .004) (Table 1).

Table 1.

Demographic and performance characteristics for neuropsychological tests, broken down by diagnostic group.

	(a) NCI (n = 118)	(b) MCI (n = 47)	(c) DEM (n = 49)
Age (years)	70 ± 7	75 ± 7^a	73 ± 7^a
Women, n (%)	78 (66)^c	25 (53)	19 (38)
Education (years)	12 ± 4^b	9 ± 5	12 ± 4^b
MMSE^*	28 ± 3^c	27 ± 3^c	21 ± 3
MEFO^*	10 ± 2^b,c	8 ± 2^c	4 ± 2
Free recall	2.8 (1.5)^bc	1.3 (1.5)^c	0.5 (0.6)
Phonetic fluency for words beginning with P^*	14 (5)^b,c	11 (5)^c	8 (4)
Orientation^*	6 (0.5)^c	6 (0.9)^c	4 (1.9)

Means are shown ±SD.

CI: cognitive impairment: MCI; mild CI; MEFO: memory, fluency, and orientation test; MMSE: Mini-mental state examination; NCI: no CI.

*

Values corrected for age and educational level. a,b,c: values considered significant for P < .05. ^aGreater than NCI group. ^bGreater than MCI group. ^bcGreater than MCI and dementia groups. ^cGreater than dementia group.

Influence of demographic variables on classifying cognitive groups

Comparing cognitive test scores broken down by sex to whole-group scores revealed no significant differences, but in the dementia group, women performed more poorly than men on MEFO (t = 2.1; P = .04) and the MMSE (t = 2.4; P = .03). Diagnostic test scores showed small but significant correlations (P < .001) to age and educational level. Scores were inversely related to age (MEFO r = −0.24, MMSE r = −0.23) and directly related to years of schooling (MEFO r = 0.20, MMSE r = 0.22). Multivariate logistic regression analyses were used to study the influence of demographic variables on subjects’ diagnostic categories. There was a sex effect on categorisation in the dementia group according to results from both MEFO (P = .009) and MMSE (P = .002). While MEFO showed no demographic variables with a significant link to subjects being placed in the CI group, sex was a factor for the MMSE (P = .002). None of the demographic variables affected how the MCI group was identified by MEFO; the MMSE showed both a sex effect (P = .049) and an age effect (P = .04).

Table 1 and Fig. 1 show a comparison between values on neuropsychological tests among the 3 cognitive groups adjusted for age and education. MEFO revealed significant differences between all groups (F = 110; P < .001) while the MMSE did not find differences between the MCI and NCI groups (P = .6).

Figure 1.

Comparison of performances on neuropsychological tests by diagnostic group.

Internal reliability

Standard MEFO items had a Cronbach alpha of 0.69; the correlation between orientation and deferred recall was r = 0.44, that between orientation and phonetic fluency was r = 0.39, and between phonetic fluency and free recall, r = 0.45.

The validity of each separate item was determined. Those that best discriminated for dementia were orientation, AUC = 0.89 (CI 95%, 0.83–0.96) and deferred recall, AUC = 0.917 (CI 95%, 0.88–0.96). Items that best discriminated for MCI were deferred recall, AUC = 0.77 (CI 95%, 0.69–0.85) and verbal fluency, AUC = 0.69 (CI 95%, 0.69–0.85).

Duration

The average duration of the test was 270 ± 30 seconds.

Convergent validity

MEFO showed significant direct correlations (P < .001) with the MMSE (Pearson r = 0.77) and the ACE-R (r = 0.85). Correlations were inversed for questionnaires about functionality (PFAQ, r = −0.58) and memory loss (AD8, r = −0.74). The depression scale was not used to establish correlations.

Test-retest reliability

Seventy-nine subjects from all 3 groups were tested on 2 occasions, and results showed a high level of correlation between the 2 measurements (r = 0.82, P < .001). Performance improved on the second assessment by 1 ± 1.8 points; the increase in performance on the item requiring recall of 5 words was significant.

Discriminant validity

We used ROC curves to test the discriminant validity of the following: (a) dementia (n = 49); (b) CI (MCI or dementia) (n = 96); and (c) MCI (n = 47). NCI subjects made up the control group (n = 118). We used clinical diagnoses in addition to results from the ACE-R, PFAQ, and AD8 tests as the gold standard, and then compared results to those from MEFO and the MMSE. A comparison between ROC curves and the AUCs for each test are shown in Fig. 2. The Z-values determined after comparing AUCs from the MEFO with those from the MMSE were as follows: Z = 0.85, P = .19 for dementia; Z = 1.34, P = .08 for CI; and Z = 1.3, P = .11 for MCI. In calculating sensitivity, specificity, and diagnostic accuracy for the screening tests, we used the cut-off point (COP) with the best compromise between sensitivity and specificity according to the ROC curve coordinates (Youden index). These data and the MEFO normative data are shown in Table 2.

Figure 2.

Comparison of areas under the curve for different neuropsychological tests, broken down by diagnostic group.

Table 2.

Normative data from MEFO.

COP	YI	DA	SE	SP	PPV	NPV
A. Dementia
<7	0.81	0.93	0.86	0.96	21.5	0.15
B. Cognitive impairment
<8	0.59	0.8	0.72	0.87	5.54	0.32
<9	0.58	0.79	0.82	0.75	3.3	0.24
C. Mild cognitive impairment
<9	0.44	0.73	0.68	0.76	2.83	0.42
<10	0.44	0.68	0.81	0.63	2.19	0.30

NPV: negative predictive value; PPV: positive predictive value; SP: specificity; YI: Youden index; COP: cut-off point; DA: diagnostic accuracy (percentage of correctly classified subjects); SE: sensitivity.

Discussion

Our objective is to validate MEFO, a new screening test for CI and dementia that can be applied in less than 5 minutes, and compare its validity to the MMSE's. The 2 tests showed a high discriminant validity for dementia, with acceptable and similar values for sensitivity (SE) and specificity (SP). The resulting level of diagnostic accuracy approaches 95%. MMSE's discriminant value for dementia as found by our study resembled values published in other studies of highly educated populations.^23,39 It was greater than that in studies of less-educated populations, such as the Chilean validation of MMSE, in which half the population had completed less than 6 years at school.²² Due to the educational status in our study population (11 ± 4 years; only 21% had completed <6 years at school), the specificity of all the tests employed was probably higher than would have been the case if the study had included rural populations. Despite this observation, we believe that our results are valid; MEFO was shown to have a low education effect. Furthermore, unlike the MMSE, it does not demonstrate an age effect on the diagnostic category.

Since MEFO does not require that patients be able to read or use a pencil and paper, it could be administered to illiterate subjects.⁴⁰ However, we have not tested its discriminant validity in this study owing to the low number of illiterate subjects in the sample.

Regarding the ability to identify CI of any type (MCI or dementia) (Fig. 2 and Table 2), MEFO's validity was shown to be similar to the MMSE's, with a diagnostic accuracy of approximately 80%. Evaluating the discriminant validity for MCI showed that MEFO was more accurate than the MMSE, given that it could differentiate between MCI and NCI groups. Amnestic MCI is the most homogeneous type of MCI. This initial stage of AD can be detected using episodic memory tests.^6,33,41 In our study, only 10 of the 47 subjects with MCI were amnestic-type. This may have contributed to MEFO's lower diagnostic accuracy compared to studies that included only those patients with amnestic MCI.^18,23

Estimates indicate that some 20% to 30% of individuals older than 60 years have CI (3%–19% with MCI^41,42; 5%–10% with dementia^2,42). We believe that the most important task, from a public health perspective, is to differentiate between groups of patients with varying degrees of CI, since this category includes both individuals with marked cognitive disorders and others with initial deficits that may be treatable. For general screening studies, we recommend using a MEFO cut-off point of 7 to 8. This will provide a sensitivity of 0.72, specificity of 0.87, positive predictive value of 60%, and negative predictive value of 93%. In turn, we recommend using a cut-off point of 8 to 9 in memory clinics where a greater prevalence of CI is anticipated. This will increase sensitivity to 82% (Table 2).

Our study's limitations include the fact that it was performed exclusively in metropolitan Santiago and the subjects have a high educational level compared to those in other parts of the country. Furthermore, most of the sample was not gathered from primary care centres, although primary care is our target population. Another bias in the sample is that subjects in the control group were younger and more educated than those in other groups. This is a common bias in cognitive impairment studies,^18,40 and it may be due to the fact that advanced age and low educational level are risk factors for CI. Results were corrected for age and education to decrease the effects of those factors on scores. Another distinguishing feature of our study is that the dementia group contained more men than women. We believe this trend resulted from female companions being more thorough than male companions about bringing patients to all the evaluations needed in order to complete the study.

Conclusions

MEFO was shown to have good internal reliability and test-retest reliability with a high discriminant ability for dementia and CI. These characteristics resemble those of the MMSE and other brief tests. MEFO is more useful than the MMSE for identifying subjects with MCI. This trait, plus its short application time (less than 5 minutes) and the fact that subjects do not have to be able to read or use a pencil and paper, makes it appropriate for population-wide CI screening.

Funding

This study received funding from Fondecyt projects 1100975 and 1110189.

Conflict of interest

The authors have no conflicts of interest to declare.