Fototest is a short simple test, applicable to illiterate subjects and not influenced by educational level. The purpose of this study is to assess its validity for cognitive impairment (CI) and dementia (DEM) under routine clinical practice conditions.
Material and methodsA cross-sectional, multicentre and naturalistic study conducted in General Neurology clinics. The subjects were over 60 years old, with a previously established diagnosis of no cognitive impairment (NoCI), cognitive impairment without dementia (CInoDEM), or dementia (DEM). The discriminant validity was assessed using the area under the ROC curve (AUC), sensitivity (Sn), specificity (Sp), and probability coefficients. Partial correlations were calculated, adjusted for age, sex and education level with Eurotest, verbal fluency test (VFT) and Global Deterioration Scale (GDS) score to evaluate the concurrent and construct validity.
ResultsNineteen neurologists included a total of 589 subjects: 361 NoCI, 106 CINoDEM and 122 DEM. The discriminant value was better for DEM (AUC 0.94±0.02) than for CI (0.86±0.02). For DEM the best cut-off point was 26/27 (Sn=0.88 [95% CI, 0.93-0.97], Sp=0.87 [95% CI, 0.84–0.90]); and for CI, 28/29 (Sn=0.71 [95% CI, 0.65–0.77], Sp=0.84 [95% CI, 0.80–0.88]). Fototest showed a high and significant correlation with Eurotest (r=0.70±0.02), VFT (r=0.68±0.02) and the GDS score (r=−0.77±0.02).
ConclusionThe Fototest showed adequate validity for CI and DEM in general neurology clinics in an extensive and wide geographical area.
El Fototest es un test breve, fácil, aplicable a analfabetos y no influido por el nivel educativo. El objetivo de este estudio es evaluar su validez para deterioro cognitivo (DC) y demencia (DEM) en condiciones de práctica clínica habitual.
Material y métodosEstudio transversal multicéntrico y naturalístico realizado en consultas de neurología general. Se incluyó a sujetos mayores de 60 años con diagnóstico previamente establecido de no deterioro cognitivo (NoDC), deterioro cognitivo sin demencia (DCsD) o demencia (DEM). La validez discriminativa se evaluó mediante el área bajo la curva ROC (aROC), sensibilidad (S), especificidad (E) y cocientes de probabilidad. Se calcularon correlaciones parciales ajustadas por edad, sexo y nivel educativo con Eurotest, fluidez verbal semántica (FVS) y el estadio GDS para evaluar la validez concurrente y de constructo.
ResultadosDiecinueve neurólogos incluyeron a un total de 589 sujetos: 361 NoDC, 106 DCsD y 122 DEM. La validez discriminativa es mejor para DEM (aROC 0,94±0,02) que para DC (0,86±0,02). Para DEM el mejor punto de corte es 26/27 (S=0,88 [IC del 95%, 0,93-0,97], E=0,87 [IC del 95%, 0,84-0,90]), y para DC, 28/29 (S=0,71 [IC del 95%, 0,65-0,77], E=0,84 [IC del 95%, 0,80-0,88]). El Fototest muestra una alta y significativa correlación con el Eurotest (r=0,70±0,02), la FVS (r=0,68±0,02) y el estadio GDS (r=–0,77±0,02).
ConclusiónEl Fototest muestra una adecuada validez para DC y DEM, en consultas generales de neurología de un extenso y variado ámbito geográfico.
Fototest (www.fototest.es) is a cognitive test developed recently in our country, whose use is becoming increasingly widespread. Its distinguishing features are its brevity (it can be applied in less than 3min), the possibility of use by illiterate patients and absence of paper and pencil tasks. These features make it especially suitable for people with a low educational level and for short consultation visits such as those in primary care.1
Fototest consists of 3 tasks2; the initial task uses colour photographs and consists of naming 6 common objects shown in the foreground and corresponding to the categories “games”, “vehicles”, “fruits”, “musical instruments”, “items of clothing” and “kitchen utensils”; the frequency of use of each object varies between moderately frequent (“trumpet”) and frequent (“car”). Other variable factors are semantic field amplitude, from narrow (“kitchen utensils”) to extensive (“fruits”), and representativeness of each object with respect to the semantic field, including some items that are very typical (“spoon”) and others that are not as representative (“shoe”). A verbal fluency task is carried out upon completion of the task described above, consisting of providing names of people. Subjects are asked to say the names of persons of the opposite sex for half a minute. Next, they are asked to repeat the task with names of persons of the same sex for another half minute. It has been proven that this verbal fluency task is not influenced by educational variables.3 This verbal fluency task also acts as a distracting task. Once it has been completed, subjects are asked to remember the pictures named initially, for which they are given a time of 20s. After this period, they are offered names of item categories as an aid, for only those items not remembered spontaneously. Therefore, this instrument includes several cognitive domains, language (naming), executive tasks (verbal fluency) and episodic memory, both free and aided by clues.
The results of Fototest, which are not influenced by the educational level of subjects,4 fall between a minimum of zero and an unlimited maximum, meaning that there is no ceiling effect. The distribution follows a normal function.4 Fototest has high internal consistency (Cronbach's alpha 0.94) and high test-retest (intraclass correlation coefficient [ICC] 0.89±0.04) and interobserver reliability (ICC=0.98±0.01).4
Concurrent and discriminant construct validity has been evaluated in 2 studies: 1 case–control study with a total of 60 subjects5 and 1 cross-sectional study with 378 subjects.2 Finally, a blind, prospective study conducted in primary care has shown that Fototest has greater effectiveness and lower cost than the Mini-Mental test,6 and similar efficacy to other instruments applicable to illiterate subjects, such as T@M and Eurotest, but more efficiency due to its lower cost and requiring half the time.7,8 We recommend the cut-off points 26/27 for dementia (DEM) and 28/29 for cognitive impairment without dementia (CIwD).9
So far, all studies have been carried out by the group that developed Fototest, with no external verification of these results. Our goal now is to evaluate the diagnostic usefulness of Fototest in routine clinical practice conditions in a multicentre sample.
Material and methodsDesignFOTOTRANS was a phase II study for the evaluation of diagnostic tests.10 It was a cross-sectional multicentre naturalistic study, carried out in general neurology departments throughout Spain during the period between May and October 2006. The participation of researchers was voluntary and unpaid, and recruitment was conducted through the RedIris Neurology list11 and the Neurology Study Group for Behaviour and Dementia of the Spanish Society of Neurology (GENCD-SEN).
ParticipantsThe study included patients over 60 years, monitored continuously at the general neurology departments of the researchers involved, with previously established clinical and cognitive diagnoses. The only exclusion criteria were lack of consent and prior participation in the study.
Patients were classified as DEM (DSM-IV criteria12), CIwD (mild cognitive impairment criteria according to GENCD-SEN13) and no cognitive impairment (NoCI).
During consultation visits, each investigator selected the first subject on each day who met the inclusion criteria in a prospective, systematic and predefined manner. Only 1 subject was selected per day and researcher. The study required the inclusion of at least 25 subjects per investigator to ensure a minimum degree of consistency and validity of data from each of the researchers.
ProcedureFototest was administered at the end of the consultation visit and its results did not modify the previous clinical or cognitive diagnosis. In addition, Eurotest14,15 and a test of semantic verbal fluency (SVF) (naming animals during 1min) were also administered.16 The cognitive diagnosis, which had been established previously, was based on the medical history, cognitive assessment and complementary studies conducted by each of the researchers.
We recorded the following variables: gender (male/female), age in years, literacy (literate/illiterate), educational level (none or incomplete primary, primary and secondary or higher education), cognitive diagnosis (DEM, CIwD, NoCI), possibility that the clinical diagnosis might affect cognitive performance (yes/no), use of drugs that might affect cognitive performance (yes/no), subjective complaints of memory loss (yes/no), results of short tests administered (Fototest, Eurotest, SVF) and global deterioration scale (GDS) stage.
Statistical analysisA comparison of continuous variables between groups was performed using the ANOVA univariate test, and that of categorical variables using the chi-square test. The diagnostic usefulness was evaluated independently for NoCI vs. CI (CIwD+DEM) and DEM vs. NoDEM (CIwD+NoCI), through the area under the ROC curve (AUC), using the established cognitive diagnosis as a gold standard. We considered as the best cut-off point that which maximised the sum of sensitivity (Se) and specificity (Sp). We calculated specific likelihood ratios (LR) for various result ranges that maximised their interpretation, as well as their post-test probabilities for various prevalences (pre-test probabilities). The comparison of usefulness between tests was carried out through the comparison of AUC values derived from the same cases.17 Convergent validity was assessed by calculating the partial correlations between Fototest and other instruments employed, and that of the construct with the GDS stage. All these correlations were adjusted for age, gender and educational level. All parameters were estimated with their 95% confidence intervals (95% CI). Contrasts were bidirectional with an error α=0.05. Calculations were performed using SPSS 11.5 (SPSS Inc., Chicago) and MedCalc 7.0.
Ethical and formal aspectsThe FOTOTRANS study was approved by the research ethics committee of Hospital Universitario Virgen de las Nieves in Granada, and had the scientific backing of the Spanish and Andalusian societies of Neurology. All subjects or their caregivers were informed in detail and gave their consent to participate in the study. The study design and preparation of this manuscript conformed to the STARD recommendations for diagnostic test studies.18
ResultsA total of 23 researchers began the study, of which 3 did not reach the minimum established set of 25 subjects and 1 applied Fototest incorrectly (fluidity of names for 1min instead of half a minute). The 19 researchers finally considered included a total of 597 subjects, 8 of whom did not complete Fototest, thus resulting in a final sample of 589 subjects: 361 NoCI, 106 CIwD and 122 DEM (Fig. 1). Among the 361 NoCI subjects, 120 (33.2%) suffered a neurological process that could potentially induce cognitive impairment (cerebral vascular disease, epilepsy, Parkinson's disease, etc.), 76 (21.0%) were taking drugs that could negatively affect cognitive performance (tranquilisers, anticholinergics, etc.) and 93 (25.8%) reported subjective complaints of memory loss; 190 (52.6%) presented at least 1 of the above conditions and 62 (17.2%) reported more than 1.
The socio-demographic characteristics of the sample and the results in the tests employed are summarised in Table 1. The overall sample had a mean age±standard deviation of 72.69±7.4 years, with a predominance of females (56.5%) and low educational level (7% of subjects were illiterate and 49.9% had no education or had not completed primary education). There were no significant differences in gender distribution between the diagnostic groups. Additionally, subjects with DEM were significantly older than those with CIwD and these in turn were older than those with NoCI (76.75±6.0, 73.50±6.8 and 71.09±7.4 years, respectively; P<.001). Illiterate patients were more frequent in the DEM group (11.5%) than in the NoCI and CIwD groups (5.8% and 5.7%, respectively), although this difference did not reach statistical significance (P=.08). There was a significant difference in the test results for different groups with NoCI>CIwD>DEM (P<.001 for all comparisons). Distribution by GDS stage is shown in Fig. 2.
Socio-demographic characteristics and results by diagnostic groups.
Total | NoCI | CIwD | DEM | P | |
Number of subjects | 589 | 361 | 106 | 122 | |
Age (years) | 72.69±7.4 | 71.09±7.4 | 73.50±6.8 | 76.75±6.0 | <.001 |
Gender (females) | 333 (56.5) | 205 (56.8) | 58 (54.7) | 70 (57.4) | .91 |
Illiterate | 41 (7.0) | 21 (5.8) | 6 (5.7) | 14 (11.5) | .08 |
Educational level | |||||
None/incomplete | 294 (49.9) | 180 (49.9) | 44 (41.9) | 70 (57.4) | |
Primary | 192 (32.6) | 115 (31.9) | 41 (38.7) | 36 (29.5) | |
Secondary or higher | 103 (17.5) | 66 (18.3) | 21 (19.8) | 16 (13.1) | .18 |
Fototest | 29.63±7.4 [589] | 33.14±4.8 [361] | 28.89±4.8 [106] | 19.89±6.8 [122] | <.001 |
Eurotest | 22.98±8.1 [548] | 26.59±4.9 [349] | 22.41±5.4 [101] | 10.72±7.5 [98] | <.001 |
SVF | 12.17±5.3 [578] | 13.99±4.7 [361] | 12.08±4.7 [103] | 6.46±3.5 [114] | <.001 |
CIwD: cognitive impairment without dementia; DEM: dementia; NoCI: no cognitive impairment; SVF: semantic verbal fluidity.
Values are presented as number of subjects (percentage) or mean±standard deviation [number of subjects].
The results of Fototest for DEM are shown in Fig. 3A; the AUC for DEM was 0.94 (95% CI, 0.92 to 0.96), similar to Eurotest (0.95 [95% CI, 0.93 to 0.97], P=.54); both were higher than the SVF (0.90 [95% CI, 0.88 to 0.92], P<.05). For DEM, the best cut-off point was 26/27, with Se=0.88 (95% CI, 0.81–0.94) and Sp=0.87 (95% CI, 0.84–0.90). Table 2 shows the distribution by ranges of results and their specific LR. Fig. 3B shows the distribution of results for CI. The AUC of Fototest for CI was 0.86 (95% CI, 0.83 to 0.89), similar to Eurotest (0.84 [95% CI, 0.80–0.88], P=.71); both were higher than SVF (0.78 [95% CI, 0.74–0.82], P<.01). The cut-off point which maximised the sum of Sp and Se for CI was 28/29, for which Se=0.71 (95% CI, 0.65–0.77) and Sp=0.84 (95% CI, 0.80–0.88). Table 3 shows the distribution by ranges of results and their specific LR. The results of Fototest showed a high and significant correlation with Eurotest (r=0.70±0.02, P<.001) and SVF (r=0.68±0.02, P<.001) (Fig. 4), as well as a significant negative association with GDS stage (r=−0.77±0.02, P<.001) (Fig. 5), which supported the convergent and construct validity of the instrument.
Distribution of results, specific likelihood ratios and post-test probability of dementia according to Fototest scores (grouped by ranges).
Fototest | DEMa | No DEMa | LR | Prevalence (pre-test probability) | |||
0.05 | 0.1 | 0.26b | 0.5 | ||||
≤21 | 67 (0.55) | 8 (0.02) | 33.3 | 0.64c | 0.78 | 0.92 | 0.97 |
22–24 | 22 (0.18) | 19 (0.04) | 4.4 | 0.19 | 0.32 | 0.61 | 0.81 |
25–26 | 19 (0.16) | 33 (0.07) | 2.2 | 0.10 | 0.19 | 0.43 | 0.69 |
27–29 | 10 (0.08) | 76 (0.16) | 0.5 | 0.03 | 0.05 | 0.15 | 0.33 |
≥30 | 4 (0.03) | 331 (0.71) | 0.05 | 0.003 | 0.005 | 0.017 | 0.05 |
Total | 122 (100) | 467 (100) |
DEM: dementia; No DEM: no dementia; LR: likelihood ratio.
Distribution of results, specific likelihood ratios and post-test probability of dementia according to Fototest scores (grouped by ranges).
Fototest | CIa | No CIa | LR | Prevalence (pre-test probability) | ||||
0.05 | 0.1 | 0.25 | 0.39b | 0.5 | ||||
≤24 | 106 (0.465) | 10 (0.028) | 16.60 | 0.47c | 0.65 | 0.85 | 0.91 | 0.94 |
25–26 | 37 (0.162) | 15 (0.042) | 3.86 | 0.17 | 0.30 | 0.56 | 0.71 | 0.79 |
27–30 | 45 (0.197) | 73 (0.202) | 0.98 | 0.05 | 0.1 | 0.25 | 0.38 | 0.49 |
31–33 | 23 (0.101) | 96 (0.266) | 0.38 | 0.02 | 0.04 | 0.11 | 0.20 | 0.27 |
≥34 | 17 (0.074) | 167 (0.463) | 0.16 | 0.008 | 0.02 | 0.05 | 0.09 | 0.14 |
Total | 228 (100) | 361 (100) |
CI: cognitive impairment; No CI: no cognitive impairment; LR: likelihood ratio.
The results of the FOTOTRANS study show that Fototest is a valid, useful instrument in the identification of DEM and CI in general neurology consultations under everyday clinical practice conditions. The discriminant validity was slightly lower for CI, a logical finding given the more difficult diagnosis of CI. This tends to occur in all instruments, whether abbreviated or full.
The overall discriminant value measured as the AUC of Fototest for DEM (0.95±0.02) and CI (0.86±0.02) was lower than that of T@M.19 The latter is a recent instrument evaluating various types of memory and specifically designed for the detection of Alzheimer's disease in the prodromal phase. In the only study conducted, T@M showed a very high unspecified dementia (UD) level for both Alzheimer's disease (AUC 0.99) and for amnestic CI (AUC 0.93). The reasons for this could be manifold. Firstly, the controls were healthy volunteers in the T@M study; those with subjective memory complaints were explicitly ruled out. These control subjects were probably a normal population. This would make them much easier to differentiate with respect to subjects with AD and DEM than our controls, who were from a clinical sample in which all were neurological patients and reported memory loss (>25%) or some circumstance (disease, drugs) that could compromise their cognitive performance (>50%) in a high proportion of cases. Secondly, T@M is a specific instrument for memory assessment. Therefore, the study paid attention to the diagnosis of Alzheimer's disease and amnestic cognitive impairment. In contrast, in our study the attention was on dementia and cognitive impairment in general, rather than on amnestic profile impairment. A recent study using the 2 instruments reported a similar diagnostic usefulness and associated cost. The only significant difference was that Fototest required less than 3min to be administered whereas T@M needed almost 7min.7,8
There was no difference between the discriminant capacity of Fototest and that of Eurotest, an instrument of proven effectiveness and validity in subjects with a low level of education.20 The Eurotest results for DEM in this study (AUC 0.95) were very similar to those found in another multicentre study21 conducted with this instrument (AUC 0.93), thus ensuring its strength and validity. While easy and short, Eurotest is more complex to implement and evaluate and takes longer to administer (about 7min). Therefore, Fototest should be the first choice, since it has the same usefulness and is much easier and faster to apply.
Other brief instruments for CI that have been evaluated in our country are the adaptations of the MIS22 and MoCA23 instruments. The results of the Spanish adaptation of MoCA,24 a new abbreviated instrument that is becoming widespread and was specifically developed for the identification of CI, are very modest, especially for CI (DEM AUC 0.89 and CI AUC 0.69). In a prospective study in a clinical sample, the Spanish adaptation of MIS25 with the cut-off point 4/5 showed an acceptable level of validity for the identification of DEM and CI (AUC 0.92 for both). The negative side is that MIS requires that subjects be able to read and write; consequently, it cannot be applied to illiterate subjects, and an additional tool must be available for such cases.
The results were also strong and consistent, and remained stable in various subanalyses. There were no significant differences between the results obtained in the southern regions, where the overall level of education was lower, and centres in the northern part of the country. Neither were the results altered when the sample excluded cases provided by researchers from hospitals in Granada or in all of Andalusia (data not shown).
The great strength of this study was its external validity, due not only to its nature as a multicentre study with a large sample size, but above all due to the clinical setting in which it was developed and the virtual absence of exclusion criteria, which allowed the incorporation of subjects systematically excluded from other studies. These features not only gave the study a naturalistic character, since the sample was a true reflection of the population in which the instrument should be applied, but also underestimated the usefulness of the instrument. Another strength of this study was the independence of the cognitive diagnosis with respect to the results of the instrument and the main researcher.
This study had the limitations inherent to phase II studies for the evaluation of diagnostic tests, primarily its cross-sectional character and non-random recruitment and allocation, which made it impossible to assess the true predictive validity. The application of the test in a non-blind manner by the researchers, who knew the diagnoses, was also a known source of bias that may have overestimated validity. Finally, the multiplicity of researchers could also lessen the validity of the cognitive diagnoses due to heterogeneity in the interpretation of criteria. However, they were all experienced professionals in the field of cognitive-behavioural neurology and, in addition, the diagnoses considered as gold standard were those the subjects already had in real life, with their corresponding social and healthcare implications, thus offering a high level of validity.26
In conclusion, Fototest has confirmed, in an ample and varied clinical setting, its usefulness and discriminant validity for CI and DEM, as well as its concurrent and construct validity. This validity, along with its reliability, brevity, simplicity and applicability, makes it a perfect alternative for use in everyday clinical practice.
FinancingThis study was partly funded by the Ministry of Health of the Junta de Andalucía (Exp. 441/06) and a research support grant from Janssen-Cilag Laboratories. None of these organisations participated in the design, data collection, analysis and interpretation, or in the elaboration of this study.
Conflict of interestsC. Carnero-Pardo is the creator of Fototest and Eurotest.
Hospital Universitario Virgen de las Nieves, Granada: Ángel Ortega Moreno (35), Francisco Escamilla Sevilla (35), Rosa Vilchez Carrillo (31), Samuel Lopez Alcalde (–). Hospital Divino Vallés, Burgos: Miguel Goñi Imízcoz (35). Hospital Universitario Virgen del Rocío, Sevilla: Emilio Macías Franco (33). Hospital Virgen del Puerto, Plasencia (Cáceres): Fernando Castellanos Pinedo (32). Complejo Hospitalario de Cáceres: Ignacio Casado Naranjo (32). Centro de Prevención del Deterioro Cognitivo, Madrid: Marcos Llanero Luque (31). Hospital de Zumárraga, Guipúzcoa: Fermín Moreno Izco (31). Hospital Punta Europa, Algeciras (Cádiz): Antonio Zambrano Toribio (30). Hospital Universitario Nuestra Señora de la Candelaria, Tenerife: Norberto Rodriguez Espinosa (30). Hospital Universitario La Paz, Madrid: Ana Frank Garcia (30). Hospital Alvarez Buylla, Mieres (Asturias): Manuel Menéndez González (30). Hospital Sant Camil, Sant Pere de Ribes (Barcelona): Elisabeth Franquet Gomez (30). Hospital Clínico San Cecilio, Granada: Ángel Gomez Camello (29). Hospital de Navarra, Pamplona: Francisco Lacruz Bescos (29). Hospital Central de Asturias, Oviedo: Teresa Calatayud Noguera (28). Hospital Puerta del Mar, Cadiz: Santiago Cousido Martínez-Conde (28). Hospital Universitario La Fe, Valencia: Miguel Baquero Toledo (27). FIDYAN Neurocenter, Granada: M. Teresa Montoro Ríos (–). Hospital Clínico San Carlos, Madrid: M. Sagrario Barquero Jiménez (32)†.
The number in brackets indicates how many subjects the researcher contributed to the study.
This article is dedicated to the memory of Maria Sagrario Barquero Jimenez, who took part in it.
Please cite this article as: Carnero-Pardo C, et al. Estudio FOTOTRANS: estudio multicéntrico sobre la validez del Fototest en condiciones de práctica clínica. Neurología 2011; 27: 68–75.