A new assessment for episodic memory. Episodic memory test and caregiver's episodic memory test

Abstract

Introduction and objectives

The purpose of the episodic memory test and the caregiver's episodic memory test is to evaluate episodic memory according to its definition in a way that is feasible for families and achieves high degrees of sensitivity and specificity.

Methods and results

We administered a test consisting of 10 questions about episodic events to 332 subjects, of whom 65 had Alzheimer's disease (AD), 115 had amnestic MCI (aMCI) and 152 showed no cognitive impairment according to Reisberg's global deterioration scale (GDS). We calculated the test's sensitivity and specificity to distinguish AD from episodic aMCI and from normal ageing. The area under the ROC curve for the diagnosis of aMCI was 0.94 and the best cut-off value was 20; for that value, sensitivity was 89% and specificity was 82%. For a diagnosis of AD, the area under the ROC curve was 0.99 and the best cut-off point was 17, with a sensitivity of 98% and a specificity of 91%. A subsequent study using similar methodology yielded similar results when the test was administered directly by the caregiver.

Conclusions

The episodic memory test and the caregiver's episodic memory test are useful as brief screening tools for identifying patients with early-stage AD. It is suitable for use by primary care medical staff and in the home, since it can be administered by a caregiver. The test's limitations are that it must be administered by a reliable caregiver and the fact that it measures episodic memory only.

Resumen

Introducción y objetivos

Valorar la memoria episódica, adaptándola a la definición del concepto, haciéndola asequible a las familias, y aplicándola con una alta sensibilidad y especificidad, es el concepto del test episódico y el test episódico del cuidador.

Métodos y resultados

Se aplicó un test formado por 10 preguntas relacionadas con hechos episódicos a 332 sujetos, de los cuales 65 tenían enfermedad de Alzheimer (EA), 115 deterioro cognitivo leve amnésico (DCLa) y 152 no mostraron alteración cognitiva, según la escala de deterioro global (GDS) de Reisberg. Se calculó la sensibilidad y especificidad del test episódico para detectar EA frente a DCLa y normalidad. El área bajo la curva ROC para el diagnóstico de DCLa fue de 0,94 y el mejor punto de corte fue 20, valour con el que se obtuvo una sensibilidad del 89% y una especificidad del 82%. Con respecto al diagnóstico de EA, el área bajo la curva ROC fue de 0,99 y el mejor punto de corte fue 17, con el que se obtuvo una sensibilidad del 98% y una especificidad del 91%. Un estudio posterior con similar metodología demostró resultados similares cuando el test era pasado directamente por el cuidador.

Conclusiones

El test episódico y el test episódico del cuidador son herramientas útiles como test breves de cribado para la captación de enfermos con EA en estadios iniciales, adecuadas para la utilización en atención primaria y en él domicilio, al poder ser pasado directamente por el cuidador. Las limitaciones del test vienen dadas por la necesidad de un cuidador fiable y por la medición exclusiva de la memoria episódica.

Introduction

Most patients with dementia have memory problems at the onset of the disease.¹ Memory refers to the mental process of recovering stored information after time²; rather than being a unitary construct, it includes interrelated subsystems.³ Episodic memory refers to the system responsible for specific memories of episodes or experiences in our lives. These memories depend on context and each is associated with a specific time and place.⁴ Episodic memory depends on neural networks essentially limited to the temporal lobes, hippocampus, and the frontal lobes.³ The hippocampus is the crucial structure that consolidates information in long-term memory storage circuits. Lesions to the hippocampus result in failure to learn new material, although immediate and long-term memory will be spared.² Some scholars argue that recognition memory may be supported by 2 separate processes: recall and familiarity. ‘Recall’ is recovery of the context in which a prior event took place, including space–time placement, and it requires an intact hippocampus. This process is similar to the mental journey through time concept proposed by Endel Tulving.⁴ In contrast, ‘familiarity’ is a non-contextual sensation of having had an earlier encounter, and it depends on the perirhinal cortex.^5–7 To cite an example, when watching films, we often think that we have seen an actor before, even if we are not able to recall any details about that person (familiarity). Furthermore, we all know the feeling of suddenly remembering where we had seen the actor before, or other items of information such as his name (recall). Our memory allows us to simulate and predict future events, two activities with important repercussions for planning and decision-making.^8–13

In general terms, there are 2 types of episodic memory disorders: amnestic syndromes and memory recall deficits. Amnestic syndromes occur in patients (such as Alzheimer's disease (AD) patients) with medial temporal lobe, limbic system, and hippocampal dysfunction. Memory recall deficits are typical in patients with subcortical or frontal circuit dysfunction (frontotemporal dementia, Parkinson disease, progressive supranuclear palsy, corticobasal degeneration; most cases of vascular dementia, Lewy body dementia, and normal ageing). The classic test of verbal memory involves asking the patient to repeat a list of 3 to 10 words up to 5 times, or until the patient is able to repeat the full list without pausing. In the second phase of the examination, the patient is allowed to focus on the learned words and practice them silently for a minute, with no distractions, before repeating them. After a 5 to 10 minutes period during which he is occupied with other tasks, he is asked to repeat the words once again. Next, the patient identifies any omitted words from a list of target words (those learned previously) mixed with other words not included in the first task. Patients with amnesia will be unable to learn or store new information, and they have difficulty both repeating and identifying target words. Patients with memory recall deficits have trouble repeating words, but they are better at identifying the target words they missed. Another method of assessing recall involves providing semantic clues.^14–17

Active, immediate, or working memory is an attention function based on actively maintaining information in the short-term. It is closely related to prefrontal network integrity and its subcortical connections, and to the ascending activating reticular system. The terms ‘working memory’ or ‘short-term memory’ refer to the type of memory used to retain numbers, words, names, or other items of information for short periods of time. The working memory model proposed by Baddeley and Hitch¹⁸ consists of a control system or central executive which is responsible for maintaining information in the memory and providing the selective attention the mind needs to focus on a certain operation. The central executive is linked to the frontal lobe; damage to this lobe produces dysexecutive syndrome, which causes patients to have difficulty maintaining and manipulating the information needed to plan or coordinate activities. It has 3 slave systems: (a) the phonological loop, involved in immediate retention of auditory and written information; (b) the visuospatial scratchpad, related to immediate retention of visuospatial information; and (c) the episodic buffer, which Baddeley added to the model in 2000.¹⁹ The episodic buffer provides temporary, limited-capacity multi-modal storage, and it integrates information from the phonological loop and visuospatial scratchpad into long-term memory.

Managing information over several minutes is a task for working memory. Assessing episodic memory in clinical practice requires tests that exceed the reach of working memory. We know that many normal patients score 0 or 1 on the 3-word deferred recall test on the Mini-mental state examination (MMSE), and we must therefore exercise caution when interpreting simple recall as a measure of memory.²⁰ A more appropriate way of evaluating memory is to use list-learning followed by recall and recognition tasks (California Verbal Learning Test-II, Rey Auditory Verbal Learning Test).^21–23 Two very brief tests, the MIS²⁴ and Fototest,²⁵ have yielded excellent results in Spain. Their administration time is less than 5 minutes, and they appear to be feasible brief tests for detecting AD in a primary care setting. For specialist consultations lasting between 10 and 25 minutes, brief neuropsychological test batteries examining multiple cognitive areas are more recommendable. Such tests include the 7-Minute Screen²⁶ and the Memory Alteration Test (M@T).^27,28

Mild cognitive impairment is defined as the transitional state between the cognitive changes of normal ageing and very early dementia.²⁹

The last few years have seen increased interest in the construct of mild cognitive impairment. This interest lies in the need to identify early characteristics of such diseases as AD and other forms of dementia. Mild cognitive impairment (MCI) is the earliest stage in these diseases. As such, studying this entity must include research into its clinical, epidemiological, neuroimaging, biomarker, and neuropathological aspects.³⁰

Some of these biomarkers have been added to new research criteria^31,32 as a way of increasing the probability that cognitive impairment will be due to the pathophysiological changes in Alzheimer disease. This tendency gives rise to the concept of prodromal AD or dementia caused by probable AD with evidence of an AD-related pathophysiological process.

Several biomarkers have been listed as prognostic factors for MCI: hippocampal atrophy in MRI studies,^33,34 amyloid beta 1–42 and tau biomarkers in cerebrospinal fluid,³⁵ changes on PET scans with 2-deoxy-2-(18F) fluoro-d-glucose (FDG-PET),³⁶ and atypical PET scans with Pittsburgh compound B (PiB-PET).³⁷

Loss of episodic memory is a defining characteristic in AD, but it is also found in the elderly and in patients with amnestic MCI (aMCI). Memory loss in the elderly differs quantitatively and qualitatively from AD-related memory loss. Quantitatively, the former is less intense; qualitatively, there is more marked impairment of tasks requiring recollection of associative information, as also occurs in patients with frontal-subcortical deficits.³⁸ Nevertheless, the distinction between memory deficits in elderly subjects and in patients with aMCI is less clear and has not been proved. The exact role aMCI plays in episodic memory deficits in AD remains undetermined. The cause of episodic memory loss in AD probably includes multiple factors. In addition to neurofibrillary disease being present in the medial temporal lobe in early stages, especially in neuron layer II, other factors should be considered.³⁹ These include pathological changes in the inferior brain regions which reduce cholinergic input to the neocortex significantly.⁴⁰ Some scholars propose that the presence of amyloid and neurofibrillary disease in frontal lobes in early stages,⁴¹ and soluble amyloid itself, have a specific effect on long-term memory ability in the hippocampus, which would concomitantly affect learning.⁴²

Objective

We propose a new test intended to measure episodic memory exclusively for purposes of detecting initial stages of AD with high levels of sensitivity and specificity. The distinguishing features of the test are as follows: brevity, lack of interference with working memory, no education effect, being user-friendly and caregiver-applied, and being an ecological test adapted to the concept of episodic memory.

Subjects and methods

Description of the interview

The interview was designed to fit Tulving's concept of episodic memory according to which the key components of memory are time, place, and context.⁴ Our interview is contrasted with a caregiver's answers in an approach similar to that used in the Hughes Clinical Dementia Rating. However, our interview is given as specific questions and observations about episodic events, and it evaluates episodic memory exclusively.

First, we chose a universal, common task, such as having lunch. This social activity involves a place (where?), companions (who?), and the food itself (what?).

All these events are time-related, meaning that we classify them as having occurred the same day, the day before, or more than a day before (the weekend). We may refer to Saturdays and Sundays indiscriminately as ‘the weekend’, as long as both days were more than 24 hours before the reference time. However, if the test is given on Monday, it is important to say ‘last Saturday’.

Scores are assigned according to the degree of memory loss, which is more significant if it affects memories from the same day than if it affects memory of the day before or preceding days (Fig. 1).

Figure 1.

Episodic memory test.

Questions must always be read in the same order. This way, when we ask where the subject ate, or with whom, we leave clues allowing the subject to recollect what he ate on that occasion.

The score corresponds to 3 possible responses: correct, incorrect, and approximate. Examples of an ‘approximate’ response may be the time the subject left the house if it is off by less than an hour; companions present, if they were numerous and one is missing; or the food description, if the subject was partially mistaken about the content. The caregiver may assign a partial score as he or she sees fit, without there being strict rules.

It is absolutely necessary for the patient to be accompanied by a reliable caregiver so that we can compare two sets of responses; if the caregiver's is missing, we cannot continue.

The episodic test was designed as a means of assessing patients during a consultation. The Caregiver's Episodic Memory test is one in which the caregiver completes the test prior to the consult. It was validated by a later study.

The first 3 items were modified so that caregivers would be able to complete the test at home. Similarly, the test can also be used with institutionalised patients to whom the questions “who took you to the doctor?” and “how did you get there?” do not apply (Fig. 2).

Figure 2.

Modifications to the first 3 items for patients taking the test in institutions or at home.

Participants

Patients were selected in consecutive order from a tertiary-level memory clinic. It was their first visit to that centre after being referred by primary care and general neurology, psychiatry, and other clinics. All patients and caregivers were requested to give informed consent. GDS was used as the basis for selection. Patients were classified according to such data as the MMSE and MIS scores and the Blessed scale for categorising the GDS stage.

Patients with aMCI or AD were selected from among those who came to the clinic with memory concerns and who were accompanied by caregivers. Caregivers were required to be in daily contact with patients. Controls were selected from among healthy family members who accompanied patients to the memory clinic. Following the Reisberg Global Deterioration Scale, subjects were categorised as normal (GDS stage 1 or 2); aMCI (Petersen criteria and GDS stage 3); and mild-moderate AD (NINCDS ASDRA criteria for AD and GDS stage 4 or 5).

Of the 180 selected patients, 115 presented mild cognitive impairment, while 65 had mild to moderate AD. The control group without cognitive impairment contained 152 subjects.

A second study was performed following the same methods, except that this time, the caregiver filled in the interview. Caregiver data are also provided.

Thirty-five patients were excluded from the first study. Of this number, 6 did not have daily contact with reliable caregivers; 10 patients had a significant vascular component; 4 patients had severe and long histories of psychiatric disease; 8 met criteria for Lewy body dementia; and 6 met criteria for frontotemporal dementia. The last patient had a continuing history of chronic alcoholism.

Nineteen patients were excluded from the second study: 4 did not have reliable caregivers, 1 was in an advanced stage of AD, and the rest met criteria for other types of dementia (vascular, Lewy body, or frontotemporal dementia).

Procedure

During the consult, the doctor had patients complete the following tests: the Spanish-language version of the Memory Impairment Screen (MIS),²⁴ the MMSE (Folstein test),⁴³ the Blessed Dementia Scale,⁴⁴ the GDS scale,⁴⁵ and the episodic test, following instructions for every test (Table 1).

Table 1.

Sociodemographic traits and neuropsychological scales used in the validation study of the caregiver's episodic memory test that yielded 3 subject groups.

	AD n = 65	Amnestic MCI n = 115	Control n = 152	P-value
Age (mean ± SD)	77.9 ± 6.6	75.4 ± 8	71.4 ± 9.1	<0.001^*
Educational level n (%)				<0.001^*
Basic studies	16 (24.6%)	13 (11.3%)	37 (11.1%)
Primary studies	46 (70.8%)	91 (79.1%)	243 (73.2%)
Secondary studies	3 (4.6%)	7 (6.1%)	42 (12.7%)
University studies	0	4 (3.5%)	10 (3%)
Sex, n (%)				0.03^*
Male	22 (34%)	34 (30%)	68 (45%)
Female	43 (66%)	81 (70%)	84 (55%)
MIS (mean ± SD)	2.23 ± 0.2	5.69 ± 0.8	7.6 ± 1.2	<0.001^*
MMSE (mean ± SD)	19.2 ± 2.6	23.9 ± 1.9	28.2 ± 1.4	<0.001^*
Blessed Dementia Scale (mean ± SD)	9.9 ± 3.1	2.97 ± 2.4	1.5 ± 2.4	<0.001^*
GDS (mean ± SD)	4.3 (0.4)	3 (0.1)	1.4 (0.4)	<0.001^*
Episodic memory test (mean ± SD)	15.4 (3.5)	18.3 (3)	21.3 (1.8)	<0.001^*

*

P < 0.05.

While the same tests were completed in the second study, the caregiver administered the episodic test this time, without the intervention of any healthcare professionals, while caregiver and patient were in the doctor's waiting room (Table 2). Tests were not administered to patients in their institutions or homes.

Table 2.

Sociodemographic traits and neuropsychological scales used in the validation study of the caregiver's episodic memory test that yielded 3 subject groups.

	AD n = 55	Amnestic MCI n = 50	Control n = 63	P-value
Age (mean ± SD)	73.7 ± 6.4	74.2 ± 7.8	69 ± 8.1	0.002^*
Educational level n (%)				0.72
Basic studies	22 (40%)	6 (12%)	6 (9.5%)
Primary studies	29 (52.7%)	28 (56%)	33 (52.3)
Secondary studies	3 (5.4%)	8 (16%)	15 (23.8)
University studies	1 (1.8%)	8 (16%)	9 (14.2)
Sex, n (%)				0.27
Male	21 (38%)	19 (39%)	32 (51%)
Female	34 (62%)	31 (61%)	31 (49%)
MIS (mean ± SD)	2.13 ± 0.3	5.2 ± 0.7	7.8 ± 1.3	<0.001^*
MMSE (mean ± SD)	20.6 ± 2.2	25.7 ± 2.1	28.3 ± 1.6	<0.001^*
Blessed Dementia Scale (mean ± SD)	11.5 ± 3.3	2.9 ± 2.6	1.7 ± 2.7	<0.001^*
GDS (mean ± SD)	4.2 (0.4)	3 (0.1)	1.8 (0.4)	<0.001^*
Episodic memory test (mean ± SD)	15.3 (2.6)	18.6 (0.9)	21.5 (1.2)	<0.001^*

*

P < 0.05.

All patients underwent a 1.5 Tesla MR scan, general analytical studies, a syphilis test, and tests for TSH, vitamin B₁₂, and folic acid levels. We did not evaluate the degree of temporal atrophy, run routine functional neuroimaging scans, or measure other biomarkers.

Statistical analysis

We completed a descriptive analysis of the variables; continuous variables were expressed as mean ± standard deviation (SD) and qualitative variables as absolute frequencies and percentages. The Kruskall–Wallis test was used to compare patient characteristics and values from the total test score across the 3 groups for continuous variables, and the chi-square test was used for categorical variables. Patients with AD were contrasted with patients with MCI and with healthy controls to calculate the test's sensitivity and specificity for detecting AD. Likewise, healthy controls were contrasted with patients with AD or MCI to determine the test's ability to detect MCI.

We calculated positive and negative predictive values to determine with what degree of accuracy the test was able to determine the presence or absence of disease. Efficacy was calculated as an indicator to evaluate performance on each of the diagnostic tests we used. Confidence intervals of 95% were calculated for all validity indicators.

The same method was used in the second study in which the caregiver administered the test in the doctor's waiting room.

Statistical analysis was performed using R software.

Results

Sociodemographic characteristics: mean ages, percentage of male subjects, educational level, and the different scales used to classify patients according to GDS stage are shown in Tables 1 and 2.

Applicability: all patients who took the test were able to complete it without problems. No limitations due to educational level or sensory loss were detected. Test administration time ranged from 1 to 5 minutes.

The area under the receiver operating characteristic (ROC) curve for a diagnosis of MCI was 0.94 and the best cut-off point was 20. At this value, sensitivity was 89% and specificity was 82%. For a diagnosis of AD, the area under the curve (AUC) was 0.99 and the best cut-off point was 17, yielding a sensitivity of 98% and specificity of 91% (Table 3).

Table 3.

Sensitivity (SE), specificity (SP), positive and negative predictive values (PPV/NPV), area under the ROC curve (AUC), and cut-off points for the episodic memory test.

	SE (CI 95%)	SP (CI 95%)	AUC (CI 95%)	PPV (CI 95%)	NPV (CI 95%)	COP
AD vs MCI + controls	0.98 (0.95–1)	0.91 (0.87–0.95)	0.99 (0.97–1)	0.72 (0.63–0.83)	0.99 (0.98–1)	17
Control vs AD + MCI	0.89 (0.81–0.97)	0.82 (0.77–0.87)	0.94 (0.91–0.96)	0.55 (0.45–0.65)	0.96 (0.94–0.99)	20
AD vs. MCI	0.90 (0.83–0.98)	0.91 (0.86–0.97)	0.98 (0.96–1)	0.85 (0.76–0.94)	0.94 (0.89–0.99)	18

Results were very similar for tests administered by caregivers (Table 4).

Table 4.

Sensitivity (SE), specificity (SP), positive and negative predictive values (PPV/NPV), area under the ROC curve (AUC), and cut-off points for the episodic memory test administered by the caregiver.

	SE (CI 95%)	SP (CI 95%)	AUC (CI 95%)	PPV (CI 95%)	NPV (CI 95%)	COP
AD vs. MCI + control	0.93 (0.84–1)	0.94 (0.89–0.99)	0.96 (0.93–0.99)	0.87 (0.77–0.97)	0.96 (0.92–1)	17
Control vs AD + MCI	0.97 (0.96–1)	0.87 (0.79–0.96)	0.98 (0.96–1)	0.91 (0.85–0.97)	0.98 (0.94–1)	20
AD vs. MCI	0.92 (0.84–1)	0.87 (0.76–0.47)	0.94 (0.89–0.98)	0.87 (0.77–0.96)	0.92 (0.83–1)	18

Discussion

As we learn more about the neuropsychological and pathological profiles of normal ageing,^46–49 mild cognitive impairment,^50–56 and AD,^57–63 we draw closer to achieving early diagnosis of AD. However, we face serious difficulties when it comes to finding a single instrument able to distinguish between those 3 conditions in a practical and universal way.

In clinical practice, we can differentiate ageing from Alzheimer disease with high levels of sensitivity and specificity by using neuropsychological studies and confirming results with pathology studies. And yet, this tendency is less true for subjects older than 80 than for those younger than 70,⁶² and it is easier to differentiate between entities if deficits are asymmetrical (for example, if verbal memory deficit is unequal to visual memory deficit).⁶³

Drawing the line between normal ageing and MCI is more difficult. In light of this added complication, biological markers are being used more frequently to aid with neuropsychological diagnosis and to define early stages of the disease more precisely.

Today, the conversion rate of MCI to dementia ranges from 1% to 72% per year⁵⁶ and between 10% and 40% tip back into the normal range if they are re-evaluated.^54,55 The variability in these figures occurs for 3 reasons. First of all, doctors lack a feasible and universal instrument providing an accurate assessment of memory or other cognitive capacities.⁴⁸

Second, there is a lack of consensus regarding which cognitive domains should be evaluated to determine the presence or absence of MCI. Most experts support verbal memory,⁵⁰ but a large minority of cases would go undetected without an assessment of visual memory,⁵¹ or of cognitive domains other than memory.⁵² Third, we do not know what level of impairment should be defined as a deficit, for example, 1.5 SD below the normal level for the age group.^53,54

As a result of these uncertainties, doctors tend to make decisions according to the consistency of the results and the impairment patterns, and not based on isolated values suggesting a deficit.

In consequence, results for MCI frequency can be altered substantially by changing the number of neuropsychological indexes used to determine objective impairment or the cut-off points defining mild cognitive deficit in the elderly.^64,65 This has been pointed out by Jak.⁶⁶

Evaluating subtle episodic memory deficits is difficult, and classic evaluation methods require administration times of more than 15 minutes so as to avoid interferences with active or working memory. This disadvantage becomes increasingly obvious with shorter tests, which may evaluate only working memory and not episodic memory.

In contrast, our test does not show the disadvantages listed above. For purposes of this test, it is not necessary to know what level of impairment is defined as a deficit according to SDs in the study population. Nor is it necessary to distinguish between amnesia and memory retrieval deficits. And while the test is short, interference with working memory is avoided by drawing upon information that has already been recorded in time.

In the aMCI phase in which encoding is partially affected, anatomical pathology studies show early impairment of frontal circuits⁶⁷; this is also corroborated by PiB. The above makes it difficult to distinguish MCI patients from healthy elderly subjects, especially as both may show signs of memory retrieval deficit.

Conclusions

We believe that this test is adapted to the ecological concept of episodic memory in which location, context, and time play a fundamental role in how that type of memory is evaluated. This test, which has high levels of sensitivity and specificity, is able to distinguish between healthy subjects and those in early stages of AD by means of well-defined cut-off points. Other strong points for the test are its simplicity, short administration time, and lack of education effect. These traits mean that the test resembles what Parker and Philp describe as the optimal test for Alzheimer disease screening.⁶⁸

The fact that the test can be administered by a caregiver makes it suitable for use as a population-wide screening test or in primary care centres.

Nevertheless, the test does have obvious limitations, such as requiring a reliable caregiver and the fact that it measures only episodic memory. Another limitation is the simple correlation between the GDS stage and the episodic memory test result in subjects whose sampling bias consisted of their being referred to a tertiary-level memory clinic based on the authors’ expert opinion, and without a biomarker analysis. However, the favourable results led to our decision to publish the test and to continue with future studies that will employ new biomarkers.

Conflicts of interest

The authors have no conflicts of interest to declare.