Validity of the clinical and content scales of the Multiphasic Personality Inventory Minnesota 2 for the diagnosis of psychogenic non-epileptic seizures

Abstract

Introduction

The use of the Multiphasic Personality Inventory Minnesota 2 (MMPI-2) for the diagnosis of psychogenic non-epileptic seizures (PNES) is controversial. This study examines the validity of the clinical scales and, unlike previous works, the content scales.

Methods

Cross-sectional study of 209 patients treated in the epilepsy unit. We performed a logistic regression analysis, taking video-electroencephalography as the reference test, and as predictor variables age, sex, IQ and clinical (model A) or content scales (model B) of the MMPI-2. The models were selected according to the Aikake index and compared using the DeLong test.

Results

We analysed 37 patients with PNES alone, or combined with seizures, and 172 patients with seizures only. The model consisting of sex, Hs (hypochondriasis) and Pa (paranoia) showed a sensitivity of 77.1%, a specificity of 76.8%, a percentage of correct classification of 76.8%, and an area under the curve (AUC) of 0.836 for diagnosing CNEP. Model B, consisting of sex, HEA (health concerns) and FRS (fears), showed a sensitivity of 65.7%, a specificity of 78.0%, a percentage of correct classification of 75.9% and an AUC of 0.840. DeLong's test did not detect significant differences.

Conclusions

The MMPI-2 has a moderate validity for the diagnosis of PNES in patients referred to an epilepsy unit. Using content scales does not significantly improve results from the clinical scales.

Resumen

Introducción

La utilidad del Inventario Multifásico de Personalidad de Minnesota 2 (MMPI-2) para el diagnóstico de crisis no epilépticas psicógenas (CNEP) es controvertida. Este estudio analiza la validez de las escalas clínicas y, a diferencia de trabajos previos, las escalas de contenido.

Métodos

Estudio transversal de 209 pacientes atendidos en la unidad de epilepsia. Se realizó un análisis de regresión logística tomando como prueba de referencia la vídeo-electroencefalografía y como variables predictoras edad, sexo, cociente intelectual y las escalas clínicas (modelo A) o de contenido (modelo B) del MMPI-2. Los modelos se seleccionaron según el índice de Aikake y se compararon con el test de DeLong.

Resultados

Se analizó a 37 pacientes con CNEP solas o combinadas con crisis epilépticas y 172 pacientes solo con crisis epilépticas. El modelo A, compuesto por sexo, hipocondría (Hs) y paranoia (Pa), mostró una sensibilidad del 77,1%, una especificidad del 76,8%, un porcentaje de clasificación correcta del 76,8% y un área bajo la curva (AUC) de 0,836 para el diagnóstico de CNEP. El modelo B, compuesto por sexo, preocupación por la salud (HEA) y miedos (FRS), mostró una sensibilidad del 65,7%, una especificidad del 78,0%, un porcentaje de clasificación correcta del 75,9% y un AUC de 0,840. El test de DeLong no detectó diferencias significativas.

Conclusiones

El MMPI-2 presenta una validez moderada para el diagnóstico de CNEP en los pacientes remitidos a una unidad de epilepsia. El uso de las escalas de contenido no mejora de forma significativa los resultados obtenidos con las escalas clínicas.

Introduction

Among patients who visit a general neurology clinic for the first time, 30% present medically unexplained symptoms.¹ In addition to ruling out neurological disorders, assessment of psychiatric and personality characteristics is a necessary step in diagnosing functional disorders. Psychogenic non-epileptic seizures (PNES) are of particular interest to many specialists due to the problems they pose from a diagnostic and therapeutic point of view and their major financial and social/familial consequences in addition to their impact on health. Patients with this type of seizures frequently visit healthcare centres and are at risk for unnecessary examinations and treatments. Meanwhile, delays in diagnosis prevent them from receiving the multidisciplinary treatment which could be beneficial.² The DSM-5 classification system includes PNES in the somatic symptom disorders category.³ It is believed that 10%-20% of patients diagnosed with epilepsy and up to 50% of the patients monitored in video-EEG (vEEG) units actually present PNES.⁴ Furthermore, 7%-32% of the patients with PNES experience or have experienced epileptic seizures (ES).

Psychological factors in patients with PNES have been studied using both projective and non-projective tests. These studies suggest the presence of a personality with a prior tendency to somatisation⁵ in association with avoidant, dependent, histrionic, and borderline personality disorders.⁶ While many different techniques are used to diagnose these disorders, we would like to highlight the Minnesota Multiphasic Personality Inventory (MMPI).⁷ However, validity studies on the MMPI and its second edition (MMPI-2)⁸ for the diagnosis of PNES have yielded contradictory results. Some authors have reported sensitivity and specificity values higher than 80%,⁹ while others calculate them below 50%.¹⁰ Most of these studies have analysed basic clinical scales exclusively without considering content scales. Basic clinical scales screen for psychopathological symptoms as well as assessing the personality traits, interests, and preferences of the evaluated patient. Content scales provide additional information on singular symptomatic behaviours, less adaptive beliefs, and distorted thinking.

The main objective of this study is to create a logistic regression model able to identify patients with PNES among patients seen in an epilepsy unit. As our test of reference, we recorded seizure episodes using vEEG. The predictive variables were age, sex, IQ, scores on the MMPI-2 clinical scales, and, in contrast with other studies, scores on the MMPI-2 content scales.

Patients and methods

We performed a cross-sectional analysis of a series of 209 consecutive patients assessed within the framework of the epilepsy programme of the neurology department at Hospital Ruber Internacional in Madrid between 2000 and 2008.

Video-electroencephalography

All patients were studied with vEEG. Records were made with a digital acquisition system and stored on hard disk drives and video tapes. Durations ranged from 1 to 5 days. We used silver electrodes attached with collodion at the locations described by the international 10–20 system, with additional electrodes at the T1, T2, FT9, FT8, F9, F8, T9 and T8 locations. Every 15 min, the system records background activity to the hard disk, while the complete study is recorded on video tapes. A nurse controls the procedure, including monitoring patients, electrode impedance, and the quality of the recordings. Studies were analysed using bipolar and referential montages (longitudinal and transverse).

Neuropsychological studies

Patients were assessed with a battery of psychological and neuropsychological tests including the Wechsler Adult Intelligence Scale (WAIS-III), Wechsler Memory Scale (WMS-III), and the MMPI-2. The battery was administered by a clinical neuropsychologist (ADB). Administration time ranged between 4 and 6 hours. In our study, we used the Spanish-language version of the MMPI-2, which is a modified and standardised version of the MMPI for adults.¹¹ MMPI-2 includes 7 validity scales, the original 10 basic clinical scales on the MMPI, 31 specific subscales, 15 content scales, and 15 supplementary scales. Results for the WAIS-III included scores corresponding to the full-scale IQ (FSIQ), verbal IQ (VIQ), and the performance IQ (PIQ).

Demographic variables

In order to assess the diagnostic ability of MMPI-2 in cases of PNES, the statistical analysis only evaluated easily gathered demographic variables, such as sex, age, and IQ. Since other clinical and demographic variables such as history of psychiatric disorder, history of child abuse, and frequency of use of emergency services were not recorded systematically throughout the study, they were removed from the final statistical analysis. These variables are very susceptible to information biases.^12,13

Patient classification

We recorded seizures using vEEG as the test of reference. Three epileptologists (the authors) exhaustively evaluated the different signs indicative of PNES.^4,14 After we reviewed the vEEG recordings, patients were classified into 2 groups according to whether or not they presented PNES. Episodes with the following 2 characteristics were considered to be PNES: (1) unequivocal paroxysmal movements with no anatomical correlates and no congruent electrical features in EEG recordings on at least 2 occasions, and (2) manifestations which epileptologists and patients or their relatives identify as corresponding to the patient's habitual episodes. Patients only presenting the following were excluded from our analysis: (1) subjective paroxysmal events, or (2) paroxysmal events to which several epileptologists were unable to assign a consensus diagnosis after a thorough analysis of ictal signs. All patients with psychogenic seizures were placed in the PNES group, including patients who also presented ES. Although some review articles¹² recommend analysing patients who only present PNES independently from those also affected by ES, studies which compare the psychological profiles of these subgroups have not detected statistically significant differences.^15,16 Patients presenting ES only were assigned to the ES group.

Statistical analysis

Statistical analyses were conducted using SPSS Statistics v.19 (IBM SPSS Statistics, Armonk, USA), the !DT macro for SPSS,¹⁷ the script AllSetsReg developed by J.M. Doménech et al. (Universitat Autònoma de Barcelona), R software version 2.10.1,¹⁸ and the pROC package for R.¹⁹ Bivariate analyses were conducted using the t-test for quantitative or ordinal variables and Pearson's chi-square test for nominal variables. Bilateral tests and a significance level of P < .05 were used in all cases. We subsequently performed a multivariate logistic regression analysis in order to obtain a model able to differentiate between patients in the ES and PNES groups. We estimated 2 regression models from demographic variables (age and sex) combined with clinical scales (model A) or content scales (model B) from the MMPI-2 test. We selected the best model out of all possible models. This selection was based on the Akaike information criterion. Considering the size of the least frequent group (PNES = 37 cases), we removed variables with P-values > .2 on bivariate tests and limited variables to a maximum of 4 in the final model. Diagnostic validity of models was assessed based on parameters for sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and area under the ROC curve (AUC). Goodness-of-fit of the models was assessed using Nagelkerke's R² value and the Hosmer and Lemeshow test. Sensitivity and specificity of final models were compared using the McNemar test, and AUCs were compared using the DeLong test.

Results

Our study analysed 209 patients, including 37 patients with psychogenic seizures with or without associated ES (PNES group) and 172 patients with ES exclusively (ES group). The PNES group comprises 26 patients (70%) without associated ES and 11 patients (30%) with associated ES. In this group, women were more numerous than men (54.1% vs 45.9%). Mean age was 34.4 years with a 95% confidence interval (95% CI) of 32.6 to 36.1.

Bivariate analyses

Demographic variables

The percentage of women in the PNES group was 83.8% vs 47.7% in the ES group. This difference was statistically significant (P < .0001). Mean age in the PNES group was 36 years (95% CI, 31.1-40.8) compared to 34 years in the ES group (95% CI, 32.2-36). This difference was not statistically significant (P = .412).

Neuropsychological variables

Analysis of scores obtained in the MMPI-2 tests showed statistically significant differences (P < .5) between PNES and ES groups on the following scales: F, Hs, D, Hy, Pd, Pa, Pt, Ma, ANX, FRS, DEP, HEA, BIZ, CYN, FAM, WRK, and TRT. Table 1 presents itemised data and an explanation of these variables.

Table 1.

Comparison of MMPI-2 scores between patients with epileptic seizures only (ES group) and patients with psychogenic non-epileptic seizures, with or without associated epileptic seizures (PNES group).

MMPI-2 scales	PNES group (n = 37)	ES group (n = 172)	Significance level (P)
Clinical scales
F (incoherence)	58.2 ± 12.6	52.8 ± 9.2	.016
Hs (hypochondriasis)	68.5 ± 11.1	57.7 ± 10.4	<.001
D (depression)	64.5 ± 9.6	59.2 ± 9.8	.004
Hy (hysteria)	68.6 ± 13.3	58.2 ± 10.8	<.001
Pd (psychopathic deviate)	57.7 ± 11.7	52.6 ± 10.4	.009
Pa (paranoia)	61.1 ± 13.1	52.4 ± 9.9	<.001
Pt (psychasthenia)	62.1 ± 13.7	55.4 ± 9.8	.007
Ma (hypomania)	57.1 ± 11.5	53.0 ± 10.3	.003
Content scales
ANX (anxiety)	60.3 ± 11.1	53.2 ± 9.0	.001
FRS (fears)	55.4 ± 11.0	49.0 ± 8.8	.002
DEP (depression)	59.5 ± 11.9	53.9 ± 10.6	.005
HEA (health concerns)	69.5 ± 10.5	58.1 ± 9.9	<.001
BIZ (bizarre mentation)	56.5 ± 11.3	50.1 ± 8.9	.002
CYN (cynicism)	53.7 ± 9.4	49.7 ± 9.7	.024
FAM (family problems)	54.8 ± 10.9	50.3 ± 9.9	.016
WRK (work interference)	58.2 ± 10.7	53.1 ± 9.7	.005
TRT (negative treatment indicators)	57.0 ± 11.3	52.2 ± 9.8	.009

Only variables with a significance level of P < .05 on the t-test are shown. Results are expressed as mean ± standard deviation.

Multivariate logistic regression analysis

The variables excluded in the preselection stage (P-values < .2 in the bivariate analyses) were age (P = .412), FSIQ (P = .490), VIQ (P = .526), PIQ (P = .533), ? (P = .453), L (P = .233), Mf (P = .533), Si (P = .530), ANG (P = .238), ASP (P = .334), TPA (P = .411), and SOD (P = .837). After exclusion of 6 cases with missing values, 203 patients were included in the regression analysis.

Model A, including sex (odds ratio [OR] = 6.232 [95% CI, 2.120-18.316], P < .001), and clinical scales Hs (OR = 1.073 [95% CI, 1.027-1.121], P = .001), and Pa (OR = 1.068 [95% CI, 1.022-1.116], P = .002), showed a sensitivity of 34.3% and a specificity of 97% for diagnosing PNES with a cut-off point of .5 (default value). By lowering the cut-off point to .2, we achieved a better balance between groups: sensitivity of 77.1% (95% CI, 61-88) and a specificity of 76.8% (95% CI, 69.8-82.5). The correct classification rate of the model was 76.8%. Table 2 presents values for positive and negative likelihood ratios, AUC, Nagelkerke's R², and the Hosmer–Lemeshow test. Model A was selected from among 2047 possible models.

Table 2.

Diagnostic validity parameters and goodness-of-fit of the 2 selected logistic regression models.

	Model A^a	Model B^b
Sensitivity, % (CI)	77.1 (61.0-88.0)	65.7 (49.2-79.2)
Specificity, % (CI)	76.8 (69.8-82.5)	78.0 (71.1-83.6)
Positive LR	3.232	2.984
Negative LR	3.359	2.274
Area under ROC curve (CI)	0.836 (0.758-0.915)	0.840 (0.772-0.907)
Nagelkerke's R^2	0.355	0.367
Hosmer–Lemeshow, P	.730	.349

CI: 95% confidence interval; LR: likelihood ratio.

a

Model A: logit CPC = −11.004 + 1.830 × Sex + .070 × Hs + .066 × Pa.

b

Model B: logit CPC = –11.076 + 1.937 × Sex + .087 × HEA + .051 × FRS.

Model A was obtained from MMPI-2 clinical scales. Model B was obtained from MMPI-2 content scales. Sex was also included in both models (man = 0, woman = 1). The dependent variable CPC corresponds to the group of patients (epileptic seizures = 0, psychogenic non-epileptic seizures = 1).

Model B, including sex (odds ratio [OR] = 6.940 [95% CI, 2.362-20.389], P < .001), and content scales HEA (OR = 1.091 [95% CI, 1.043-1.141], P = .001), and FRS (OR = 1.052 [95% CI, 1.005-1.100], P = .002), showed a sensitivity of 45.7% and a specificity of 95.8% for diagnosing non-epileptic seizures with a cut-off point of .5 (default value). By lowering the cut-off point to .2, we achieved a better balance between groups: sensitivity of 65.7% (95% CI, 49.2-79.2) and a specificity of 78.0% (95% CI, 71.1-83.6). The correct classification rate for this model was 75.9%. Table 2 presents values of positive and negative likelihood ratios, AUC, Nagelkerke's R², and the Hosmer–Lemeshow test. Model B was selected from among 8191 possible models.

Although sensitivity and specificity values suggest a slight difference between these 2 models, with model A showing a higher sensitivity, comparative analyses did not find this difference to be statistically significant. The significance level obtained with the McNemar test for comparing paired data was .362; that obtained with the DeLong test for comparing ROC curves was .911. Fig. 1 shows the ROC curves for each of the 2 models.

Figure 1.

ROC curves corresponding to the multivariate logistic regression models A (upper curve) and B (lower curve). Model A includes the variables sex, Hs, and Pa. Model B includes the variables sex, HEA, and FRS.

Discussion

Our study shows that the MMPI-2 test has a diagnostic validity that is statistically significant, although moderate, for PNES in patients referred to an epilepsy unit. Differential diagnosis between ES and PNES was achieved using vEEG recordings of the events as the test of reference. We subsequently performed a multivariate logistic regression analysis, including scores on the clinical and content scales of the MMPI-2, using 2 separate models. The resulting models A and B permitted the correct classification of approximately 76% of the patients and showed sensitivity values of 65.7% and 77.1% and specificity values of 76.8% and 78%, respectively. The combination of female sex and high scores on the hypochondriasis (Hs) clinical scale or the health concerns (HEA) content scales was an especially good predictor of PNES.

Clinical and demographic variables displaying statistically significant differences between patients with ES and PNES include female sex and physical and sexual abuse,^13,20 history of psychiatric treatment,²¹ age of seizure onset, and prolonged duration of seizures.²² While we did not find statistically significant age differences in our series, we did observe differences in sex distribution with a clear predominance of women in the PNES group (83.8% vs 54.1%). This difference is in line with that observed by other authors²³ and it is reflected by the inclusion of sex as a variable in both final logistic regression models with a high OR (6.2 and 6.9).

Psychological tests have long been considered useful tools in diagnosing epilepsy.^10,24 The first studies using the MMPI for differential diagnosis of ES and PNES used the MMPI PseudoNeurologic (Pn) scale. This scale was developed to distinguish patients with neurological damage from patients with symptoms suggesting central nervous system dysfunction, but no objective anomalies in the neurological examination. The Pn scale includes 17 MMPI items taken from the Hs, Hy, and Pd scales.^25,26 It did not demonstrate statistically significant utility. Wilkus et al.²⁷ later designed classification rules that achieved correct classification rates of 80% to 90%. The Derry and McLachlan studies⁹ conducted with the MMPI-2 showed even higher values, with correct classification rates of 92% for patients with PNES and 94% for patients with ES. Using a logistic regression model based on MMPI-2 scores, EEG recordings and years since the first episode, Storzbach et al.²⁸ obtained an overall classification accuracy of 86%. However, other authors have obtained lower values. Cragar et al.¹² used the method by Derry and McLachlan and observed a sensitivity of 48% and a specificity of 58% for diagnosis of PNES. A restructured form of the MMPI-2, known as the MMPI-2-RF, was recently published. Studies performed with this version of the test proved that it is useful for identifying patients with PNES, but only preliminary data are available.²⁹ The results obtained with our 2 models, whose sensitivity and specificity values fall between 65% and 78%, are in line with the mean values described in the literature. In our study, we also observed that the models with content scales do not have a significantly higher diagnostic capability compared to traditional models based on clinical scales. The differences found between our results and those described in other articles may stem from several factors, including the different characteristics of the study populations, different sample sizes, techniques for selecting variables, and inclusion of other variables in the models (e.g. EEG).

Our series found no statistically significant differences in WAIS-III test scores, including both FSIQ and VIQ and PIQ, between patients with ES and PNES. These results agree with those described by other authors,^{24,27,30–37} while differing from those reported by Dodrill and Holmes²² who observed a higher FSIQ and VIQ in patients with PNES.

Our study has several limitations which should be considered before applying these results to clinical practice. Our sample consists of patients referred to an epilepsy unit dealing with especially complex cases, including patients with refractory epilepsy and many others seeking second opinions. Given the limited number of patients with pseudoseizures, multivariate analyses were estimated using the entire sample, and restricting the number of variables to a maximum of one per 10 cases. Furthermore, we did not perform reliability analyses on these models. Results were assumed to be reproducible based on the data obtained from the MMPI-2 validation study.¹¹ Although the sensitivity and specificity of the 2 final models are acceptable, the classification error rate is high, which points to a need for models with greater diagnostic accuracy.

Future projects might feature standardised questionnaires designed to record clinical variables with potential diagnostic value (e.g., history of psychiatric diseases or abuse) and add them to the models along with the MMPI-2 scales. Recently published results for diagnosing PNES using a new personality questionnaire, the Personality Assessment Inventory (PAI), are promising. Combined use of scales derived from the PAI and the MMPI-2 surpassed results obtained by each of these tests separately.³⁸ However, our results are preliminary and will require validation by subsequent studies.

Funding

This research has received no funding of any kind.

Conflicts of interest

The authors have no financial or personal relationships with other persons or organisations which could create conflicts of interest with regard to this article.