Validity of a computerised population registry of dementia based on clinical databases

Abstract

Introduction

The handling of information through digital media allows innovative approaches for identifying cases of dementia through computerised searches within the clinical databases that include systems for coding diagnoses. The aim of this study was to analyse the validity of a dementia registry in Gipuzkoa based on the administrative and clinical databases existing in the Basque Health Service.

Methods

This is a descriptive study based on the evaluation of available data sources. First, through review of medical records, the diagnostic validity was evaluated in two samples of cases identified and not identified as dementia. The sensitivity, specificity and positive and negative predictive value of the diagnosis of dementia were measured. Subsequently, the cases of living dementia in December 31, 2016 were searched in the entire Gipuzkoa population to collect sociodemographic and clinical variables.

Results

The validation samples included 986 cases and 327 no cases. The calculated sensitivity was 80.2% and the specificity was 99.9%. The negative predictive value was 99.4% and positive value was 95.1%. The cases in Gipuzkoa were 10 551, representing 65% of the cases predicted according to the literature. Antipsychotic medication were taken by a 40% and a 25% of the cases were institutionalised.

Conclusions

A registry of dementias based on clinical and administrative databases is valid and feasible. Its main contribution is to show the dimension of dementia in the health system.

Resumen

Introducción

El manejo de la información mediante soportes digitales permite abordajes innovadores de la identificación de los casos de demencia mediante búsquedas automatizadas en las bases de datos clínicas con sistemas de codificación de los diagnósticos. El objetivo de este trabajo fue analizar la validez de un registro de demencia en Gipuzkoa basado en los sistemas de registro administrativos y clínicos existentes en el Servicio Vasco de Salud.

Métodos

Es un estudio descriptivo basado en la evaluación de las fuentes de datos disponibles. Primero, mediante revisión de historias clínicas se evaluó la validez diagnóstica en dos muestras de casos identificados y no identificados como demencia. Se midió la sensibilidad, especificidad y valor predictivo positivo y negativo del diagnóstico de demencia. Posteriormente se buscaron los casos de demencia vivos a fecha 31 de diciembre de 2016 en toda la población guipuzcoana y se recogieron variables sociodemográficas y clínicas.

Resultados

Las dos muestras de validación incluyeron 986 casos y 327 no casos. La sensibilidad calculada fue de 80,2% y la especificidad del 99,9%. El valor predictivo negativo fue del 99,4% y el positivo del 95,1%. Los casos registrados en toda la población guipuzcoana fueron 10.551 que supone un 65% de los casos previstos según la literatura. Un 40% tomaban medicación antisicótica. La población institucionalizada fue del 25%.

Conclusiones

Un registro de demencias basado en las bases de datos clínicas y administrativas es válido y factible. Su principal aportación es mostrar la dimensión que tiene la demencia en el ámbito del sistema sanitario.

Introduction

Dementia constitutes a major public health challenge due to rapid population ageing. The condition is one of the main causes of disability, loss of quality of life, and institutionalisation.^1,2 The information available is fundamentally based on cross-sectional and cohort studies, which provide data on the progression of dementia in terms of cognitive function, risk factors, and mortality, as well as on the prevalence and incidence of the disease.^3–5 However, experts report that information is dispersed and difficult to access, which may partially explain the insufficient attention paid to dementia.⁶ Hospitalisation databases (such as the Spanish minimum basic dataset) provide information about the incidence and use of healthcare resources associated with such other age-related diseases as stroke or hip fracture.^7,8 However, since dementia is managed on an outpatient basis, the information in the minimum basic dataset is limited, and does not allow us to determine the impact of the disease on the healthcare system in terms of resource utilisation.^9,10 Furthermore, prevalence data are especially relevant in the case of disabling neurological diseases; prevalence represents the frequency of a disease in a given population at a given time, and helps estimate the healthcare and social burden of the disease.¹¹

Managing the resources necessary to respond to these patients’ needs in the short, medium, and long term requires dynamic information systems for monitoring the impact of ageing in terms of incidence and prevalence.¹⁰ Population-based registries constitute a good example of dynamic information systems, gathering real data about the incidence of a particular disease; this information is extremely useful for epidemiological studies and healthcare resource planning.¹² Several types of dementia registry have been developed worldwide, but not all of them are population-based.¹² Some registries include several cohorts to increase sample size and for research purposes; the CERAD registry is an example of such a system.¹³ Other registries are based on active reporting by neurologists and gather clinical data from medical records; an example of these is the Registry of Dementia of Girona.^14,15 Lastly, population-based registries automatically and comprehensively collect data from the databases available and incorporate them into a single registry. The best example of a population-based registry is the South Carolina Alzheimer’s disease patient registry, which contains data collected from 225 938 patients since 1998.¹⁶ The registry of reference in Spain is the Registry of Dementia of Girona^14,15; however, this registry is limited by the difficulty of accessing sufficient resources to collect data from medical records on an individual basis.

Digital files enable new approaches to data management: cases of dementia can be identified through automated searches in clinical databases using diagnostic coding systems.^17,18 One of the issues with this approach is validating diagnoses.^19–21 Results vary greatly depending on the source of the cases included and the validation system. Confirming the validity of automated systems will enable the creation of dementia registries that are not limited by the number of patients. The lack of diagnostic coding systems in outpatient clinics has prevented the application of an automated registry for dementia. In 2013, the Basque Health Service launched a new version of the electronic medical record, which required physicians to assign a diagnostic code for each outpatient consultation. This new version of the clinical history was incorporated in all hospitals in the Basque Country, leading to the development of an automated population-based registry of dementia including all healthcare databases (primary care, hospitals, and pharmacies).

The main purpose of this study was to analyse the validity of a dementia registry in the Basque province of Gipuzkoa, based on the administrative and clinical registry systems of the Basque Health Service. As a secondary objective, we sought to determine the registry’s capacity to describe the clinical and sociodemographic profile of the patients included.

Patients and methods

We conducted a descriptive study aimed at evaluating the data sources available and their diagnostic validity. The study protocol was approved by the research ethics committee of the Basque Country, with registry number EPA2016069.

The study was conducted in several phases. In 2014, we reviewed studies on registries of dementia and Alzheimer disease in different countries. In Spain, we identified the Registry of Dementia of Girona^14,15 as a reference and explored the possibility of implementing this model in Gipuzkoa. We subsequently described the protocol followed in Gipuzkoa for the diagnosis and management of patients with cognitive impairment and dementia. Outpatient neurology clinics were identified as the key element in the information system, as this is where dementia is diagnosed. We confirmed that records from these clinics contained the information necessary to identify patients (they record diagnosis of dementia or cognitive impairment). However, electronic medical records (Osabide Global) were not available in all centres, with some hospitals still using paper records. With the Osabide Global software, physicians are required to assign a diagnosis for each outpatient consultation; diagnosis is automatically recorded as an ICD-9 code, which helps identify patients with dementia among all patients in the database. At that time, however, outpatient consultation data were not available in the Oracle Business Intelligence (OBI) database of the Basque Health System, hindering the creation of an automated registry. The complete computerisation of medical records in 2014 to 2015 and the inclusion of outpatient consultation data into the OBI database in 2015 made it possible to create a registry of dementia and other cognitive disorders. Lastly, we evaluated the validity of a dementia registry through the analysis of administrative and clinical databases of the Basque Health Service.

Validation of diagnoses

The study to validate diagnoses of dementia and cognitive impairment was performed in the population attended by the hospital and health centres of Bajo Deba, which serve 72 751 people. The objective of the study was to determine the diagnostic reliability of each data source. We initially worked with a set of inhabitants of Bajo Deba aged 50 years or older (38 952 individuals on 6 October 2016). Within this group, we searched the OBI database using diagnostic codes for dementia from the ICD-9 (290xx; 331.0; 331.1; 331.2; 331.82) and ICD-10 (F03.5x; F03.8x; F03.9x; G30.0x; G30.1x; G30.8x; G30.9x; G31.0x; G31.1x; G31.2x; G31.8x; G31.9x). ICD-10 codes were used during the database search process as this has been the official version of the ICD since 1 January 2016. The OBI database is anonymised, with each individual being assigned an identification number to prevent direct connections between personal and clinical data. We conducted searches to gather data on primary care, outpatient neurology consultations, hospital care, and prescription of drugs for Alzheimer disease or drugs with ATC code N06D (donepezil, rivastigmine, galantamine). These drugs must be prescribed by a specialist (neurologist, geriatrician, or psychiatrist) and the prescription must be authorised by the medical inspection services for the drug to be covered by the Spanish Health System.

We also recorded whether patients had attended an outpatient neurology clinic. The validation criterion was diagnosis recorded in the medical history by a neurologist; therefore, only patients who had visited outpatient neurology consultations could be regarded as validated cases of dementia. Some of the patients who had visited these consultations were diagnosed with dementia. Other cases, however, had been recorded before Osabide Global electronic medical records were introduced; therefore, diagnosis was not automatically recorded using ICD codes in these cases. These were validated based on drug prescription and primary care and hospitalisation data. The neurologists participating in this study agreed that a diagnosis of dementia, established by a neurologist, geriatrician, or psychiatrist, should be regarded as valid. If diagnosis of dementia was established in primary care or in the hospital setting, it was labelled as “probable dementia.” The validation study included all cases with a diagnostic code of dementia (primary care, hospital, outpatient clinic) plus an outpatient neurology consultation. We also reviewed a random sample of 327 individuals older than 60 years without a diagnosis of dementia in any database.

Diagnosis of dementia was considered to be confirmed when diagnosis or follow-up data from the outpatient neurology consultation included any of the following terms: dementia, dementia under study, AD, Alzheimer, Lewy bodies, Lewy body dementia, frontotemporal dementia, FTD, vascular dementia, or senile dementia. A professional with prior experience reviewing clinical histories was trained to search for these terms in outpatient medical records.

The results from both samples were used to calculate positive and negative predictive values. Cases of dementia were validated for the total population. However, we only validated a random subgroup of non-cases. In this subgroup, confidence intervals were estimated by bootstrapping.²² We generated 1000 samples with replacement and calculated percentiles 2.5 and 97.5. To estimate the sensitivity and specificity of the registry for the population over 50 years of age from the region of Bajo Deba, we applied the negative predictive value to the total population of non-cases in order to estimate the number of undetected cases in the total population

Search for cases in the population from Gipuzkoa

We subsequently searched the databases including the whole population of Gipuzkoa to identify all cases meeting diagnostic criteria for dementia. Data from these searches were anonymised: we did not record personal details but rather the OBI identification number, which is specific to the OBI database.

We collected the following data: demographic variables (age and sex), clinical variables (date of diagnosis of dementia, diagnosis of dementia, and concomitant diseases: arterial hypertension, diabetes, Parkinson’s disease, stroke, cardiovascular disease, head trauma, depression, Down syndrome), medications prescribed (cholinesterase inhibitors, memantine, antidepressants, antipsychotics, anxiolytics, hypnotics), and whether the individual was institutionalised. The list of ICD-9 and ICD-10 codes used to search for comorbidities is shown in Appendix B.

To evaluate the level of diagnostic coverage of the registry we compared the rates of total and confirmed cases in our study against those reported by Ferri et al.²³

Results

The cases included in the validation study of databases from primary care, hospitals, outpatient clinics, and medication prescriptions are described in Table 1. The validation study included 986 cases and 327 non-cases; results are shown in Table 2. The percentage of valid diagnoses of dementia (positive predictive value) was 95.1% in the total sample and 96.9% in the group of cases identified in outpatient clinics and those who had been prescribed a drug specific for Alzheimer disease. The positive predictive value fell to 91.7% for patients diagnosed by primary care physicians and/or hospitalised patients. Two of the 327 non-cases had dementia, giving a negative predictive value of 99.4% (Table 2). Bootstrapping gave an error rate of 0.6% and a confidence interval of 0% to 1.52% for the sample of non-cases. Projecting the error rate to the total population of non-cases of the region of Bajo Deba gave an estimate of 232 unconfirmed cases of dementia (95% CI, 69–396). Our registry had 80.2% sensitivity and 99.9% specificity (Table 3).

Table 1.

Cases identified in the region of Bajo Deba, by source of information.

Total population aged over 50 years	38 952	100%
No diagnosis	37 771	97%
With diagnosis or medication	1181	3%
Diagnosed at OC	317	0.8%
Diagnosed at OC, hospital, or PC	970	2.5%
Receiving drugs with ATC code N06D	498	1.3%
Patients with diagnosis (OC, hospital, or PC) or receiving medication and consulting OC	986	2.5%

ATC: Anatomical Therapeutic Chemical Classification System; OC: outpatient clinic; PC: primary care.

Table 2.

Results of the validation study of a sample of cases and non-cases and extrapolation of data from the group of non-cases of dementia.

Validated cases	Cases	Unconfirmed	Error (%)
Total	986	48	4.9
Diagnosed at OC	457	14	3.1
Not diagnosed at OC	529	34	6.4
Diagnosed at PC or hospital	348	29	8.3
Not diagnosed at PC or hospital (only pharmacy)	181	5	2.8

Validated non-cases	Non-cases	Unconfirmed	Error
Sample	327	2	0.6
Medium population projection	37 966	232	0.6^a

OC: outpatient clinic; PC: primary care.

a

We assumed the same level of error detected in our sample during projection.

Table 3.

Sensitivity and specificity of the automated search for cases of dementia in clinical databases.

Classification of the population	Confirmed cases	Confirmed non-cases	Total
Non-cases in registry	232	37 734	37 966
Cases in registry	938	48	986
Total	1170	37 782	38 952

	Mean	Lower bound 95% CI	Upper bound 95% CI
Sensitivity	80.16%	61.77%	100%
Specificity	99.87%	99.87%	99.87%
Positive predictive value	95.13%	95.13%	95.13%
Negative predictive value	99.39%	98.47%	100%

CI: confidence interval.

After validation using data from the region of Bajo Deba, we extracted data from the province of Gipuzkoa as a whole. Table 4 shows the distribution of all living patients with probable and confirmed dementia in Gipuzkoa on 31 December 2016 (n = 10 551), according to the database in which cases were recorded. Given that many patients are included in more than one database, the total number of cases does not coincide with the sum of the number of cases from each database. Table 5 shows the distribution of total, confirmed, and probable cases of dementia by age and sex. The sample included more women than men. The distribution of total cases by concomitant diseases, medications used, and institutionalisation status is shown in Table 6. The most frequent concomitant disease was dyslipidaemia. Patients were most frequently prescribed anxiolytics and antidepressants. Twenty-five percent of patients with dementia were institutionalised.

Table 4.

Distribution of all living patients with probable or confirmed dementia in Gipuzkoa by database.

	Dementia (n = 10 551)
	Probable dementia (n = 3184)	Confirmed dementia (n = 7367)
Primary care	3053 (96%)	4448 (60%)
Outpatient clinic	0	3464 (47%)
Hospitalisation	245 (8%)	414 (6%)
Specific medication	0	5867 (80%)

Table 5.

Distribution of all cases of dementia (probable and confirmed dementia) by age and sex.

	Probable dementia	Confirmed dementia	Total no. cases with dementia
	3184	7367	10 551	100%
Sex
Men	930	2508	3438	33%
Women	2254	4859	7113	67%
Age group
<60	84	390	474	4%
60-64	52	136	188	2%
65-69	97	216	313	3%
70-74	186	521	707	7%
75-79	280	916	1196	11%
80-84	581	1947	2528	24%
85-89	784	2120	2904	28%
≥90	1120	1121	2241	21%

Table 6.

Distribution of cases by concomitant diseases, medications used, and institutionalisation status.

Concomitant diseases	Dementia (n = 10 551)
Arterial hypertension	1680 (16%)
Diabetes	2603 (25%)
Dyslipidaemia	2074 (20%)
Parkinson’s disease	424 (4%)
Stroke	1899 (18%)
Cardiovascular disease	997 (9%)
Head trauma	1220 (12%)
Depression	1199 (11%)
Down syndrome	16 (0%)

Medication use	Dementia (n = 10 551)
Donepezil	1804 (17%)
Rivastigmine	3602 (34%)
Galantamine	494 (5%)
Memantine	2111 (20%)
Antiepileptics	2284 (22%)
Antipsychotics	4177 (40%)
Anxiolytics	5884 (56%)
Hypnotics and sedatives	2756 (26%)
Antidepressants	6629 (63%)

Institutionalisation status	Dementia (n = 10 551)
Yes	2589 (25%)
No	7940 (75%)
No data	22 (0%)

Fig. 1 compares the prevalence of dementia by age group, in the total sample and in the sample of confirmed cases in Gipuzkoa, and in the study by Ferri et al.²³ The prevalence of dementia in our population represents 65% of the prevalence of dementia in Europe according to Ferri et al.²³

Figure 1.

Prevalence of dementia (total and confirmed) by age group in our registry and in the article by Ferri et al.²³

Discussion

Our results suggest that a registry of dementia based on clinical and administrative databases is valid and viable. This study aims to demonstrate the impact of dementia on the healthcare system. We established whether dementia was confirmed or probable according to the opinion of the neurologist at the outpatient consultation. This automated coding system, which is based on the diagnoses assigned in each consultation, seems to be sufficiently valid for the purposes of epidemiological analysis. Furthermore, this design allows the process to be repeated each year; thus, the progression of this population can be analysed dynamically, as data would be available for the dates of first contact with the healthcare system (probable initial diagnosis) and death. This would enable analysis of disease trends and such epidemiological characteristics as incidence, prevalence, and mortality.²⁴

The literature includes very few validation studies of diagnostic algorithms in clinical databases.²⁴ A model similar to our own has been proposed in Canada.²⁵ However, the Canadian model lacks coded diagnoses from outpatient consultations; rather, it uses hospitalisation and primary care data. It shows similar sensitivity (80.7%) and specificity (98.7%) to our registry. Both models have limitations for identifying cases (80% sensitivity); however, they show very good diagnostic reliability (99% specificity). In our view, sensitivity will improve as neurologists become more accustomed to the automated coding system. Our study included data from paper medical records. The collection of data from scanned documents may have interfered with validation and increased the number of cases of unconfirmed dementia. In any case, we feel that our registry is of sufficient quality to measure the burden of dementia on the Basque Health System.

The definitions and classifications of registries are usually included in larger public health surveillance models and healthcare quality reports.²⁶ A recent review addressing dementia registries¹² distinguishes between the terms “register” and “registry.”²⁷ In epidemiology, “register” refers to “the file of data concerning all cases of a particular disease or other health-relevant condition in a defined population,” whereas “registry” is a “system of ongoing registration.”²⁷ The first category would include population-based dementia registers, such as the South Carolina Alzheimer’s disease patient registry.¹⁶ The CERAD registry, on the other hand, would fall within the second category.¹³ Although cases were gathered from clinical databases, our dementia registry follows a population-based approach and may therefore be considered a “register,” as it locates dementia diagnoses among all medical encounters. Its greatest strength is its diagnostic coverage: 65% of the expected prevalence.²³ This is a high rate considering that some patients do not come into contact with the healthcare system or do not receive a diagnosis. Furthermore, the number of confirmed cases may drop dramatically among patients older than 90 years, as at that age, cognitive symptoms are assessed differently and patients do not visit specialist consultations. Our registry shows similar rates of Parkinson’s disease, stroke, and diabetes to those appearing in the Registry of Dementia of Girona.²⁸ In contrast, the rates of arterial hypertension and thyroid disorders are different between registries. Our registry gathers fewer clinical data than other registries that actively search medical records, as is the case of the Registry of Dementia of Girona.¹⁴ A consequence of this limitation is the difficulty of differentiating between types of dementia. This may improve with the standardisation of dementia coding in outpatient clinics.

According to a communication made by the Department of Social Services of the local government of Gipuzkoa on the World Alzheimer Day 2011, 4746 individuals receiving economic benefits according to the Spanish law promoting personal autonomy and care had a diagnosis of dementia; 34% (1631 patients) were institutionalised. In our study, 2589 individuals were institutionalised; assuming that the 2011 figure remains accurate, the Spanish law promoting personal autonomy and care for dependent people provided social support to 63% of patients with dementia in 2016.

Our registry includes the patients who were alive by the time the search was conducted (31 December 2016); we intend to add to the registry by performing a search each year in order to provide data from patients included in previous years (prevalent cohorts) as well as new cases from the last year (incident cohort). In this way, patient data will be updated on a yearly basis. This would also enable population analyses, since each field includes the date when data were recorded. Maintenance of a dynamic registry involves managing an anonymised database and does not require patients to be identified.

In conclusion, our project to develop a dynamic, population-based registry of dementia using Basque Health System databases seems valid and viable from an economic viewpoint, as it involves an efficient allocation of resources. Although our study only included the population of Gipuzkoa, the fact that it is based on automated searches means that minimal human resources would be needed to expand it to include the whole population of the Basque Country.

Conflicts of interest

The authors have no conflicts of interest to declare.

Funding

The study was funded by the Basque Foundation for Healthcare Research and Innovation (Bioef; grant no. BIO12/AL/005).