Cognitive impairment (CI) in general, and Alzheimer disease in particular, pose considerable challenges for healthcare systems and for society due to their high prevalence and associated costs.1 In Spain, CI affects 14.5% to 19% of individuals older than 65 years,2–4 although many cases go undetected (over half, according to the recent Gómez de Caso study4); despite this, there is broad consensus that the current circumstances do not meet the minimum required standard to recommend screening for CI in asymptomatic individuals.5–9 However, most clinical practice guidelines agree on the value of maintaining an appropriate level of vigilance in primary care and performing screening with brief cognitive tests (BCT) if CI is suspected,7,8 whether due to subjective complaints, complaints from the patient’s caregiver or family members, or clinical suspicion among healthcare professionals.

In Spain, these recommendations are reflected in the National Health System’s (SNS, for its Spanish abbreviation) clinical practice guidelines on the integral care of patients with Alzheimer disease and other dementias,10 promoted by the Ministry of Health, Social Services, and Equality (hereinafter, SNS guidelines), which recommend the use of BCTs in patients with suspected CI. Specifically, the document recommends the normalised Spanish-language version of the Mini–Mental State Examination (MMSE)11 or the Mini–Examen Cognoscitivo (MEC)12 (grade of recommendation A); secondly (grade of recommendation B), it recommends such other instruments as the Short Portable Mental State Questionnaire (SPMSQ),13 the Memory Impairment Screen (MIS),14 the Seven-Minute Screen (7MS),15 the clock-drawing test (CDT),16 the Eurotest,17 the Fototest,18 and the Memory Alteration Test (M@T).19 The SNS guidelines do not establish specific recommendations on cut-off points or corrections, and recommend that instruments be selected according to the available time, clinical experience, and the availability of normative data and validation studies for the language and setting in which they will be applied (good clinical practice criterion). However, the SNS guidelines do not mention certain other characteristics that may also be relevant when selecting a test; these include whether an instrument can be applied to illiterate individuals, whether it evaluates one or several cognitive domains, and the potential costs associated with its use (Table 1).20

Few studies have evaluated and compared these instruments in Spain,21–24 and no study to date has used a single sample to compare the diagnostic performance and effectiveness in everyday practice of the different BCTs recommended in the SNS guidelines. This study aims to evaluate and compare the diagnostic performance and effectiveness of the different screening strategies recommended by the SNS guidelines in a prospective cohort of individuals attended at primary care centres, with full, independent verification of the diagnosis.

We selected a total of 156 individuals (70 [44.9%] with subjective cognitive complaints, 76 [48.7%] with complaints raised by caregivers, and 10 [6.4%] with suspicion reported by the primary care physician). Three did not consent to be included in the study; one did not meet the selection criteria (previous diagnosis of CI); 4 did not attend the neurology consultation; 2 withdrew consent during the study period; 3 could not be evaluated due to sensory deficits and sequelae of previous strokes; and 2 were uncooperative in the evaluation due to behavioural alterations (Fig. 1).

Figure 1.

Participant flow chart.

CB: cognitive/behavioural; DEM: dementia; Dx: diagnosis; MCI: mild cognitive impairment; nCI: no cognitive impairment; PC: primary care.

(0.21MB).

Table 2 shows the sociodemographic characteristics of the final sample of 141 participants (55 [39.0%] without CI, 39 [27.7%] with MCI, and 47 [33.3%] with dementia); mean age (standard deviation; range) was 72.0 years (11.5; 31-90), with a clear predominance of women (73.0%) and individuals with low levels of schooling (14.2% of participants were illiterate and only half [49.6%] had completed primary education or higher). The table also shows the results and application times for the different BCTs administered, stratified by cognitive diagnosis. As expected, the results show a considerable difference between groups in all cases, with poorer scores and longer application times in individuals with CI.

All participants (n=141) completed the MMSE and MEC; 140 completed the Eurotest and SPMSQ; and 139 completed the Fototest and M@T. Twenty-two participants (20 of whom were illiterate) were unable to complete the MIS; the CDT was only administered to 126 participants as 15 illiterate individuals were unwilling to complete it.

Table 3 summarises the results for the different instruments, indicating the number of participants who completed each one, application time, diagnostic performance (AUC), sensitivity and specificity, and the proportion of correct diagnoses for the selected cut-off points.

Diagnostic performance was compared for pairs of BCTs in the subsamples of participants who completed both (Table 4). The results show that there was no significant difference between diagnostic performance for the Eurotest and the M@T, but that both performed significantly better than the MMSE, MEC-30, MEC-35, and SPMSQ; the M@T also performed significantly better than the CDT, with the result of the comparison between the Eurotest and the CDT being on the verge of significance (P=.08). However, the Eurotest and M@T did not perform significantly better than the Fototest and the MIS.

Fig. 2 shows the administration times for each of the BCTs studied (except the SPMSQ and the MIS, for which administration was not timed). No significant difference was observed in application times for the Eurotest and M@T (7.1 [1.8] vs 6.8 [2.2] minutes); the Fototest (2.8 [0.8] minutes) and CDT (1.3 [0.7] minutes) were clearly quicker to administer. The administration time for the MMSE (5.5 [1.6] minutes) is underestimated, as we did not time the items related to orientation in time and space; despite this, it took significantly longer to administer than the Fototest and CDT, needing longer than 5 minutes. We would expect the MEC, which includes more items than the MMSE, to take even longer.

Figure 2.

Time taken in minutes to complete each brief cognitive test.

MMSE: Mini–Mental State Examination; M@T: Memory Alteration Test; CDT: clock-drawing test.

(0.06MB).

The results from this phase III study evaluating diagnostic tests, following a prospective, paired methodology with full verification, show that the Eurotest and M@T took the longest to apply (7.1 [1.8] and 6.8 [2.2] minutes, respectively) and presented the best diagnostic performance (0.91 [0.02] and 0.90 [0.02], respectively), with no significant difference between the 2; both tests correctly classified 84% of participants. Both instruments were developed and validated in Spain, can be administered to illiterate individuals, and are freely available.17,19 However, the M@T presents certain limitations: no formal normative study has been conducted, and it only evaluates memory, and therefore would not be sensitive to CI affecting other cognitive domains. The M@T presented very similar diagnostic performance to that reported in a study of primary care patients with memory complaints (AUC of 0.91), although the overall scores in our sample are lower, probably due to the low level of education.39

The MMSE, MEC-30, and MEC-35, which are the most widely used instruments and the most strongly recommended in the SNS guidelines, and which also presented application times longer than 5 minutes (at least 5.5 [1.6] minutes), displayed significantly poorer diagnostic performance than the M@T and the Eurotest. The low diagnostic yield for CI of these tests, whose use is so universal and generalised, is more a rule than an exception, and has been reported in numerous Spanish and international studies.40–42 Our results are consistent with those of previous studies by our research group, showing a lower optimal cut-off point (22/23) than that which is typically used.42 The poor diagnostic performance, long application times, and other disadvantages (influence of level of education, etc), above all the costs associated with the MMSE, have led many authors to cease to consider it the instrument of choice43; other authors even omit the test from their recommendations.44

The SPMSQ also presents significantly poorer diagnostic performance; however, the application time for this instrument (not measured in this study) is considerably shorter. While the SPMSQ is widely used to screen for dementia in Spain,13 its diagnostic performance in screening for CI has not previously been evaluated. Our results show a clear floor effect, with even scores of 0 having a sensitivity of 0.88; this would imply that 12% of individuals with CI present perfect execution (no errors), and therefore will never be detected by this instrument. Another limitation of the SPMSQ is that for 3 of the 10 items (telephone number, date of birth, and mother’s maiden name), scoring requires reference to information that is often difficult to access or verify, particularly for older individuals attending the consultation alone.

The CDT, which presented the shortest application time (1.3 [0.7] minutes), had the poorest diagnostic performance (0.83 [1.03]), a significantly lower value; it also presents the disadvantage that it requires a certain level of graphomotor skill that individuals who are illiterate or have a low level of education may not possess; this would explain why 15 participants (10.6%), most of whom were illiterate, were unwilling to complete the test. These results confirm the previous recommendation that the CDT is not an appropriate BCT for individuals without a minimum level of education.45

However, the M@T and the Eurotest presented no significant differences in diagnostic performance when compared to the Fototest and the MIS (0.87 [0.02] for both tests), which are far quicker to administer (2.8 [0.8] minutes for the Fototest). This confirms the results of previous studies22,23 suggesting that these instruments are more efficient on account of their similar diagnostic performance and shorter administration time. Despite this, the MIS presents several limitations: no normative study has been performed, it only evaluates memory, and it is not applicable to illiterate individuals. Therefore, the test cannot be administered to a considerable percentage of Spanish adults (14.2% in our sample).

The AUC was selected as a measure of diagnostic performance because it enables evaluation of an instrument’s global discriminative capacity for all possible scores, and enables comparison of different instruments in a single sample,46,47 independently of the prevalence of the diagnosis and the sociodemographic characteristics of the sample (age, schooling, etc) and, therefore, independently of the representativeness of the sample. We also opted to study diagnostic performance for CI rather than for dementia only, as we deem this approach to be more pragmatic from a clinical perspective. For healthcare professionals in general, and primary care physicians in particular, it is beneficial to focus more on diagnosing CI, as targeting dementia exclusively would prevent us from identifying patients with CI with no functional impact, who may be eligible for preventive, corrective, or palliative measures to delay or even prevent progression to dementia.20

The main limitation of this study is the fact that the BCTs were applied consecutively, which may have introduced bias related to tiredness or fatigue, despite the short application times of these instruments; to partially mitigate this effect, the scales were administered in 3 different orders and we confirmed that results for the different BCTs showed no association with the serial position in which they were administered (data not shown). Another limitation is that the 7MS, also recommended in the SNS guidelines, was not included in the evaluation. However, this test48 is a complex instrument, both in terms of administration and scoring; despite its name, it takes longer than 10 minutes to administer; it cannot be administered to illiterate individuals; and it is not free of cost. Therefore, we do not consider this instrument to be appropriate for use at primary care centres.20

The main strength of our study is its design. On the one hand, the prospective, consecutive, systematic recruitment of participants; the use of a one-year recruitment period; the near absence of exclusion criteria; and the low drop-out rate mean that our sample accurately reflects the issue of diagnosing CI in clinical practice in this healthcare setting, conferring naturalistic and pragmatic value to the study. On the other, the fact that all participants were assessed with all BCTs, the fact that all participants were administered the gold-standard diagnostic protocol independently of their results in the BCTs, and the independent, blinded evaluation of the instruments studied and the gold-standard assessment ensure that we controlled for the main biases affecting studies evaluating diagnostic tests.25,49

In conclusion, there is no perfect, ideal BCT for use in primary care, and these professionals must be familiar with the use of various instruments; the selection of a specific test to use in a particular case should be based on the clinical circumstances (time available), patient characteristics (level of schooling, sensory deficits, etc), available data (normative data, specific validation for diagnosis of CI), and the healthcare professional’s preferences and experience.20 Based on our findings and the desirable characteristics summarised in Table 1, we recommend the Eurotest, T@M, and Fototest for use in primary care, with the latter being the most efficient as it takes less than half as long to administer.

Agencia de Evaluación de Tecnologías Sanitarias, Instituto de Salud Carlos III, Spain (Project no. PI06/90034).

C. Carnero Pardo created the Fototest and Eurotest and has received professional fees for academic and consulting activities for Nutricia, Schwabe Farma Ibérica, Biogen, Piramal, Janssen Cilag, Pfizer, Eisai, Esteve, Novartis, Lundbeck, and Grunenthal.

Under the terms of the BY-NC-ND Creative Commons licence, the Fototest and Eurotest may be used and distributed for non-commercial purposes, provided that they are not modified and their authorship is explicitly acknowledged.