Comparison between Screening Cognitive Impairment in Psychiatry (SCIP-S) and Minimental State Examination (MMSE) in Forensic Psychiatry. Pilot study

Abstract

Objectives

The objective is to compare 2 brief cognitive screening tests, MMSE and SCIP-S, and determine their usefulness in assessing the cognitive status of subjects diagnosed with psychiatric pathology.

Material and methods

Pilot study of a series of 33 cases from the Institutes of Legal Medicine of Catalonia and Andorra, whose inclusion criterion was the existence of a psychiatric pathology categorised in ICD-10. In all cases, in addition to the forensic medical interview, MMSE was administered followed by SCIP-S.

Results

In MMSE, 31 of the 33 interviewees obtained a total direct score ≥ 24 (normal cognitive performance). In SCIP-S, only 5 of the 33 participants were in a percentile ≥ 15 (normal cognitive performance). The difference between the results of both tests was statistically significant (P < 0.05).

Conclusions

This study suggests a greater usefulness of SCIP-S compared to MMSE in detecting cognitive deficits in psychiatric patients. Therefore, the use of the former is recommended in forensic medical practice.

Resumen

Objetivos

El objetivo es comparar MMSE y SCIP-S y determinar su utilidad en la valoración del estado cognitivo en sujetos con patología psiquiátrica en el ámbito forense.

Material y métodos

Estudio piloto de una serie de 33 casos procedentes de los Institutos de Medicina Legal de Catalunya y Andorra, cuyo criterio de inclusión fue la existencia de una patología psiquiátrica categorizada en CIE-10. En todos los casos, además de la entrevista médico forense se administró MMSE seguido de SCIP-S.

Resultados

En MMSE 31 de los 33 entrevistados obtuvieron una puntuación directa total ≥ 24 (rendimiento cognitivo normal). En SCIP-S solo 5 de los 33 participantes se situaron en un percentil ≥ 15 (rendimiento cognitivo normal). La diferencia entre los resultados de ambas pruebas resultó estadísticamente significativa (p < 0.05).

Conclusiones

Este estudio sugiere una mayor utilidad del SCIP-S frente al MMSE en la detección de déficits cognitivos en pacientes psiquiátricos. Por tanto se recomienda el uso del primero en la práctica médico forense.

Introduction

We define neurocognitive disorder as any alteration capable of affecting the way in which the brain processes and stores information, resulting in impaired memory, attention, perception, and/or thinking.¹ Low cognitive performance can have many different causes: neurodegenerative, vascular, infectious, traumatic, congenital, and even mental disorder.

This last cause has been the subject of study in recent years.¹ There are numerous literature reviews showing neurocognitive deficits in subjects with psychiatric illnesses.^2–4

Starting with the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) initiative of the National Institute of Mental Health in the USA,⁵ research focused especially on severe mental illnesses such as bipolar disorder and schizophrenia. Subsequently, these studies have been extended to other psychiatric illnesses such as attention deficit hyperactivity disorder,^6–8 major depressive disorder,⁹ and borderline personality disorder.^10,11

The cognitive domains most frequently affected in subjects with psychiatric illness are: verbal and visual memory, processing speed, sustained attention, executive functions, working memory, and verbal fluency.^4,12

All this translates into a loss of functionality, productivity, and quality of life that makes these subjects vulnerable, and they are often involved in legal proceedings.^1,13,14 The intervention of the forensic doctor in legal proceedings for support measures for people with disabilities or in criminal proceedings is one of the most frequent interventions in the field of forensic psychiatry. An impairment in one or more cognitive domains could entail a modification of the psychobiological bases of imputability, a limitation in the development of basic, instrumental, and legal–economic–administrative activities or a difficulty or impossibility to carry out the usual profession.^15,16

Forensic medical examination sometimes requires, in addition to the mandatory clinical interview, the use of complementary brief cognitive screening tests. One of the most commonly used is the Folstein Mini-Mental State Examination (MMSE). It is an easy to apply, quick and widely used instrument for screening cognitive impairment in dementia patients. This test assesses: temporal and spatial orientation, fixation and recall memory, attention and calculation, language and perceptual-motor skills.¹⁷ The most widely accepted and frequently used cut-off point for the MMSE of 30 points is 23; scores equal to or lower than this figure would indicate the presence of a cognitive deficit.^17,18 Despite the above, the MMSE has no proven utility in psychiatric patients, as it has limitations in detecting small cognitive deficits.^1,19

In 2005, in the absence of short and specific cognitive screening tests, Purdon designed the Screen for Cognitive Impairment in Psychiatry (SCIP) test, the Spanish adaptation of which was carried out by Pino et al. in 2014 and is known as the Screening for Cognitive Impairment in Psychiatry (SCIP-S).²⁰ The SCIP-S is composed of 5 subtests that explore: memory, attention, executive function, and processing speed.²⁰ Although specifically developed for subjects with some form of mental illness, it can also be used to assess the cognitive status of adults without psychiatric illness.²⁰ It has been shown that although the SCIP-S cannot replace the diagnostic value of a full neuropsychological examination, it offers a rapid and valid mechanism to detect cases without significant cognitive impairment.²¹

The aim of the present pilot study is to compare MMSE and SCIP-S and to determine their usefulness in detecting cognitive impairment in psychiatric patients.

Material and methods

Participants

Data were collected from 33 individuals examined between January 2019 and August 2020 at the Tarragona Division of the Institute of Legal Medicine and Forensic Sciences of Catalonia and at the Forensic Medicine Area of the Principality of Andorra. Their mean age was 43.3 ± 14.8 years. A total of 39.4% were female and 60.6% were male. Inclusion criteria: presentation of a psychiatric illness categorised in ICD-10. Exclusion criteria: coexistence of an intellectual or sensory disability that prevented the performance of the brief cognitive screening tests, previous diagnosis of cognitive impairment, and/or non-cooperation of the interviewee.

Material

a. MMSE: version translated by Lobo et al. known as the Mini Cognitive Examination (MEC) on 30 points. It is a hetero-administered test in paper format and lasts 5–15 min. It consists of 6 sections that explore: temporal orientation, spatial orientation, immediate recall, attention and calculation, delayed recall, and language.

b. SCIP-S by Tea Ediciones. Version translated and validated in Spanish by Pino et al. in 2014 of the Purdon SCIP, under the auspices of the NHS MATRICS initiative. It is a hetero-administered test in paper format and lasts about 15 min. It consists of 5 short tests exploring memory, attention, executive function, and processing speed. It has 3 parallel forms, in all cases the first one was used.

Method

In a single visit per participant, a semi-structured interview lasting approximately half an hour was conducted, from which data on the variables under study (psychobiographical data, pathological history, treatment, and substance use) and verbal consent for the performance of the cognitive screening tests were obtained. Next, the MMSE was administered, as it was a priori simpler, lasting approximately 15 min, and immediately afterwards the SCIP-S was administered, which lasted approximately 15 min more. There was no waiting time between the two tests. Each scan was performed entirely by a single examiner. In total, 3 different examiners participated in the study.

The variables were coded as follows:

• a.

  Age. The study subjects were divided into 3 groups: 18–39 years, 40–55 years and ≥ 56 years.

• b.

  Sex. They were classified into 2 gorups: male and female. No interviewee manifested their wish to not define their gender.

• c.

  Education. They were divided into 2 groups: primary level studies (< 8 years of education) and secondary/higher level studies (> 8 years of education).

• d.

  Primary diagnosis. Taken from active (unresolved) psychiatric history of the medical reports provided to the procedure and classified according to the International Classification of Diseases, version 10 (ICS-10).

• e.

  Toxic habits. This was stratified into 2 subgroups; without consumption and with consumption, without specifying the type of substance consumed. The following were considered as toxic substances: alcohol, cannabis, hallucinogens, stimulants, inhalants, opioids, sedatives, hypnotics, and anxiolytics (the last 3 only if they were not subject to medical prescription). Tobacco and caffeine were not considered as toxic substances, since the disorders they most frequently induce are sleep-wake disorders and anxiety disorders according to DSM-V.

• f.

  Prescribed treatment. Prescribed treatment was considered to be those drugs with action on the central nervous system according to the ATC classification: antipsychotics (N05A), anxiolytics (N05B), and antiepileptics (N03A), used as euthymotics. The prescribed treatment was also broken down into 2 categories: without prescribed treatment and with treatment.

Results of the cognitive screening tests. It was divided into 2 subgroups: corrected MMSE and SCIP-S (general scale). The maximum direct score in the version of the MMSE used is 30 points. Puntos.¹⁸ Since the MMSE scores are influenced by age and cultural level, the corresponding correction factor was applied. ¹⁸ A direct score (DS) ≥ 24 points was considered as a cut-off point given the balance between specificity and sensitivity.¹⁷ In SCIP-S a total direct score (total DS) ≥ 70 equated to preserved cognitive performance. In cases with total DS < 70, transformed scores (T) and percentiles (Pc) were used, taking into account the corrective factor of age and educational level. Preserved cognitive performance was considered with Pc ≥ 15.²⁰

After the forensic medical interviews, each participant was given a number from 1 to 33 in order to anonymise their identity and then the data was recorded in a Microsoft Excel spreadsheet shared between the 3 examiners. After the recording was completed, the descriptive and statistical treatment of the data was carried out with SPSS version 18 for Windows. To determine the existence of significant differences between the results obtained in both tests, McNemar's statistical test was used, a non-parametric test for paired samples, since in this case the same dichotomous variable (cognitive impairment Yes/No) was measured on 2 different occasions (MMSE and SCIPS), for each of the individuals in the sample. On the other hand, to determine the existence of some kind of dependence between the variable "cognitive impairment" and any of the other variables included in the study (age, sex, level of studies, etc.), we used Fisher's exact test, since the small sample size made it impossible to meet the conditions for applying the Chi-square test. A significance level of P < .05 was established for both statistical tests.

Results

In Table 1 and Table 2, the data obtained were collected using the descriptive analysis of the sample. Table 3 contains the results of the MMSE and SCIP-S in each one of the study variables.

Table 1.

Demographic and clinical distribution of the sample.

Gender	N	%
Man	20	60.6%
Woman	13	39.4%
Age group
18–39 years	16	48.5%
40–55 years	11	33.3%
56 years or above	6	18.2%
Total	33	100%
Education
Primary level	19	57.6%
Secondary level	10	30.3%
Higher studies	4	12.1%
Total	33	100%
Prescribed treatment
Yes	30	90.9%
No	03	9.1%
Total	33	100%
Toxic habits, number of toxic substances
Yes, 1	11	33.3%
Yes, 2	10	30.3%
Yes, 3	02	3.0%
No	03	33.3%
Total	33	100%
Diagnosis
Squizophrenia	10	30.3%
Schizoaffective disorder	2	6.1%
Psychotic-nonspecific disorder	5	15.1%
Bipolar disorder	6	18.2%
Borderline-personality disorder	5	15.1%
Non-specific personality disorder	4	12.1%
Others	1	3.0%
total	33	100%

Source: own creation.

Table 2.

Distribution of prescribed drugs in the sample.

Treatment	N/N. total	%
Antipsychotics	27/33	82.2%
Ansiolytics	19/33	57.5%
Eutimisers	10/33	30.3%

Source: own creation.

Table 3.

Average direct scores in MMSE and percentiles in SCIP-S according to the different variables under study.

Variables	Subtype	MMSE	SCIP-S
Sex	Male	27.65	9.25
	Female	27.92	8.00
Age	18–39^y	28.56	9.75
	40–55^y	26.36	6.91
	56^y or +	28.17	9.50
Education level	< 8 years	27.58	11.42
	≥ 8 years	28.00	5.14
Primary diagnosis	SCZ	27.20	11.10
	SCAD	28.50	8.50
	NPD	27.60	2.80
	BD	27.83	5.50
	BPD	27.00	7.25
	NPSD	29.25	20.75
	Others	29.00	1.00
Toxic substances	No	28.27	9.73
	Yes	27.50	8.27
Prescribed drugs	No	28.67	3.33
	Yes	27.67	9.30

Source: own creation. BD: bipolar disorder; BPD: border-line personality disorder; MMSE: Mini-Mental State Examination; NPD: non-specific psychotic disorder; NPSD: non-specific personality disorder; Others: non-specific cognitive disorder; SCAD: schizoaffective disorder; SCIP: Screening of cognitive impairment in psychiatry; SCZ: schizophrenia.

a. SCIP-S (general scale)

Overall, 31 of the 33 cases (94%) obtained a total direct score ≥ 24 points on MMSE (normal cognitive performance). In SCIP-S, only 5 of the 33 participants (15.2%) were in a percentile ≥ 15 (normal cognitive performance) (Fig. 1). If we compare the results obtained in MMSE and SCIP-S, the differences observed are statistically significant using McNemar's test for paired samples (P < .05).

Fig. 1.

Normal cognitive performance based on the cut-off point established for MMSE and SCIP-S.

When comparing the results of MMSE and SCIP-S between the subgroups of the different variables (toxic habits, age groups, educational level, primary diagnosis and prescribed treatment) no statistically significant differences were found in any case (Table 4).

Table 4.

P-values obtained in the Fisher's exact test for the different variables vs. MMSE-SCIP-S

Variables	MMSE (P-value)	SCIP-S (P-value)
Toxic substances	.542	1.000
Medication	1.000	1.000
Primary diagnosis (SCZ vs BD)	.500	.500
Nivel estudios	1.000	.366
Edad	.133	.833

Source: own creation. BD: bipolar disorder; SCZ: schizophrenia. Table: P-values obtained when contrasting the result in each of the tests for the other variables taken into account in the study using Fisher's exact test. It can be seen that in no case has P < .05 been obtained, so we can affirm that there is no relationship of dependence.

When comparing the results of MMSE and SCIP-S between the subgroups of the different variables (toxic habits, age groups, educational level, primary diagnosis and prescribed treatment) no statistically significant differences were found in any case (Table 4).

Discussion

The aim of this exploratory research was to compare the ability to detect cognitive deficits in people with mental illness by comparing two brief cognitive screening tests: the MMSE and the SCIP-S. According to our results, after administering the MMSE to 33 participants, less than 10% showed poor cognitive performance. However, for the same 33 respondents, the SCIP-S suggested that 85% of them exhibited low cognitive performance (Fig. 1), and these differences were statistically significant. This disparity is probably explained by the fact that the SCIP-S was specifically developed to detect cognitive deficits in people with some type of mental illness.^20–22 In contrast, the Spanish adaptation of the MMSE was assessed on a geriatric population aged 65 years or older with suspected cognitive impairment,²³ so that the MMSE is highly sensitive to moderate and severe cognitive impairment, although its sensitivity decreases for mild impairment.²⁴

Previous research suggests cognitive impairment in 30–80% of subjects with substance use disorder.²⁵ In particular, patients with opioid misuse had lower scores on memory and visuospatial skills when compared to cannabis and stimulant users. Alcohol users also had poorer cognitive performance.²⁵ In a comparative study between patients with a diagnosis of schizophrenia and patients with a diagnosis of schizophrenia and alcohol abuse, the latter showed more marked impairment.²⁶ However, establishing differences between the different substances was complicated by the high prevalence of polydrug users.²⁶ In our study, 2/3 of the sample consumed at least one substance. In MMSE, no statistically significant differences were found between the group that reported intoxicating habits and the group that did not (total DS 27.50 and 28.27, respectively). Ridley et al. in 2017 considered that the MMSE was not a valid and efficient screening tool for detecting cognitive impairment in the context of substance use due to its low sensitivity.²⁷ In SCIP-S the differences were also not statistically significant between the two groups (Pc 8.27 and Pc 9.73, respectively).

These data do not allow valid conclusions to be drawn, given that the sample analysed is not representative (the subgroup without toxic habits consisted of only 3 participants), although it is of interest to expand the sample in the future to analyse the extent to which toxic substances affect cognitive performance.

The most frequently prescribed drugs were: antipsychotics (82.2%), benzodiazepines (53.3%), and euthynotics (30.3%). More than 2/3 of the participants were prescribed more than one drug. In MMSE no statistically significant differences were found between the group with and without a prescribed regular treatment (total DS 27.67 and 28.67, respectively). In SCIP-S the differences between the two groups were not statistically significant either; however, the subjects with regular medication were at a higher percentile (Pc 9.30 and 3.33, respectively). These data do not allow conclusions to be drawn, since the sample analysed is not representative (the subgroup without medical prescription only consisted of 3 participants) and the previous cognitive performance of the interviewees is unknown. However, the observed difference in the SCIP-S percentile and the existence of previous studies suggesting that drugs such as lithium, benzodiazepines, antipsychotics and antidepressants could have an effect on cognitive performance,^28–30 call for further research.

Regarding the variables age and educational level, no statistically significant differences were observed between the overall MMSE and SCIP-S results, although no definitive conclusions can be drawn from our results for two reasons: because some subgroups are underrepresented (only 6 participants were in the group aged 56 years and older) and because age and educational level correction factors were used for the final results. However, in the SCIP-S20 User's Manual, specifically in the basic descriptive statistics of the scale groups of the general population sample, it is observed that the average total cognitive performance of the youngest group (18–39 years) is higher than that of the group aged 40-55 years, and this in turn is higher than that of the group aged ≥ 56 years. In the case of educational level, mean total cognitive performance is higher in those respondents with secondary or higher education.²⁰ The same occurs in the MMSE MMSE.¹⁸ Therefore, to obtain conclusive results it would be necessary to increase the sample and use direct scores without correction factors, which is complex considering the statistical basis on which the SCIP-S is based.

With regard to the main diagnosis, only the most prevalent diagnoses, schizophrenia and bipolar disorder, were compared. In MMSE no statistically significant differences were found between the different diagnoses (total DS 27.20 for schizophrenia and 27.83 for bipolar disorder). In SCIP-S the differences between the 3 illnesses were not statistically significant either (Pc 11.10 for schizophrenia and Pc 5.50 for bipolar disorder). These latter data contrast with the literature, where in general patients with bipolar disorder exhibit qualitatively similar but quantitatively less pronounced neurocognitive impairment than patients with schizophrenia.^20,30 Therefore, our preliminary research invites a study with a larger sample size where a comparison of cognitive performance between the different illnesses is carried out. From a forensic point of view, this could be interesting to suggest more effective disability support or assistance measures.

In terms of the limitations encountered, the main one was the sample size, since by separating each of the variables into subgroups, some of them were minimally represented. This was addressed by using a statistical method designed for small samples, the so-called Fisher's test, as described in the methodology section. Another limitation was the difference in the process of interpreting the results of MMSE and SCIP-S, since the former is based on direct scores and the latter on transformed scores or percentiles. To overcome these differences, the corrective factor for age and educational level was applied to the total direct score of the MMSE.¹⁸

Given that this is a pilot study, from which no conclusive results can be drawn, but hypotheses can be drawn to encourage the development of a more in-depth study in the future, it would be advisable to continue expanding the sample in order to increase the reliability of the data. It would also be interesting to analyse how each of the psychiatric illnesses specifically affects each of the cognitive domains and/or how toxic substances influence cognitive performance at the individual level. We also consider it useful to analyse the role of certain drugs on neuropsychological performance.

From the present preliminary investigation, we can conclude that the SCIP-S is considered a more useful alternative to the MMSE in the cognitive assessment of psychiatric patients. The SCIP-S is a test specifically developed for the cognitive screening of subjects with mental illness, but not the MMSE, whose Spanish adaptation was assessed on the geriatric population aged 65 years or older with suspected cognitive impairment. In view of the above, the use of the SCIP-S for cognitive screening of psychiatric patients in forensic medical practice is recommended.

Funding

This research did not receive specific support from public sector agencies, the commercial sector or non-profit organisations.

Conflict of interests

The authors declare that they have no conflict of interest in relation to this article.