Adaptation of the Hinting Task theory of the mind test to Spanish

Abstract

Introduction

Many studies have found that patients with schizophrenia have a deficit in theory of mind. Some authors associate this deficit with the presence of symptoms, while others maintain that it can also be observed in patients in the remission phase. There is no reference test to assess theory of mind in schizophrenia, although one of the most used is the Hinting Task. The aim of the present study consists of adapting and validating, in Spanish, the 10 histories that make up this test.

Material and methods

The study was conducted on a sample of 39 control subjects and 40 patients with schizophrenia. The internal consistency and the between-observer reliability and test–retest were assessed in both sample groups. The performance of the patients and control subjects were also compared.

Results

Good reliability data was obtained in the inter-observer and test–retest in the two samples. On the other hand, the internal consistency was somewhat low for all of the 10 histories. For this reason, and starting from a previous study, a reduced version of 5 histories was prepared, which showed good internal consistency. The patients with schizophrenia obtained a significantly lower score than the control subjects in 8 out of the 10 histories.

Conclusions

The reduced Spanish version of the Hinting Task demonstrated good psychometric properties. When compared to the control group, the patients with schizophrenia had a deficit in theory of mind.

Resumen

Introducción

Numerosos estudios han encontrado que los pacientes con esquizofrenia realizan un manejo deficitario de la teoría de la mente. Algunos autores relacionan este déficit con la presencia de sintomatología, mientras que otros mantienen que también es observable en pacientes en fase de remisión. No existe una prueba de referencia para evaluar la teoría de la mente en esquizofrenia, si bien una de las más utilizadas es el Hinting Task. El objetivo del presente estudio consiste en adaptar y validar en español las 10 historias que componen esta prueba.

Material y métodos

El estudio se ha desarrollado con una muestra de 39 sujetos control y 40 pacientes con esquizofrenia. En ambas muestras se ha valorado la consistencia interna y la fiabilidad entre observadores y test-retest. Asimismo, se ha comparado el rendimiento de pacientes y sujetos control.

Resultados

Se obtuvieron buenos datos de fiabilidad entre observadores y test-retest en las dos muestras. Por su parte, la consistencia interna resultó algo baja para el total de las 10 historias. Por este motivo, y partiendo de un estudio anterior, se elaboró una versión reducida de 5 historias, que mostró una buena consistencia interna. Los pacientes con esquizofrenia obtuvieron una puntuación significativamente menor que los sujetos control en 8 de las 10 historias.

Conclusiones

La versión reducida en español del Hinting Task mostró buenas propiedades psicométricas. En comparación con el grupo control, los pacientes con esquizofrenia presentaron un déficit en teoría de la mente.

Introduction

Social cognition is defined as the set of abilities necessary to function effectively in social interactions.¹ It has an adaptive function that allows us more flexibility in the way we act in social situations, considering the people involved and the context in which said situations might occur. Most authors agree that there are 5 specific cognitive processes that integrate social cognition: emotional processing, theory of mind, attributional style, social perception and social knowledge.²

Emotional processing is defined as the capacity to perceive, understand and appropriately manage emotions. Theory of mind is understood as the ability to characterise the mental states of other people–like their thoughts, beliefs and intentions–and to consider them in explaining or predicting their behaviour.³ Attributional style refers to the causes through which a person explains the occurrence of positive or negative events. Finally, the processes of social perception and social knowledge are closely interrelated; the former refers to the capacity to value rules and social roles and to integrate them into context, while social knowledge is defined as the capacity to apply those elements in a specific social situation–a certain context of interaction (for more detailed definitions of these processes and their relevance to schizophrenia, consult the Ruiz et al.’s² article).

Studies performed concerning social cognition in schizophrenia have been centred, above all, on evaluating emotional processing and theory of mind. The data obtained indicate that, compared to control subjects, patients with schizophrenia display impairment of these abilities, which leads to poor psychosocial functioning.^4,5 Likewise, it seems that there is a specific deficit in identifying negative emotions, as differences in recognising positive emotions–and more specifically, perceiving happiness–were not observed between the 2 groups.⁶ On the other hand, the majority of studies carried out regarding theory of mind conclude that it is a characteristic impairment involved in schizophrenia, as it has been observed in subjects with high risk of developing psychosis,^7,8 subjects in their first episodes,⁹ and immediate family members who have yet to receive a psychiatric diagnosis. Thus, some authors conclude that it could be an endophenotype of the disease.^7,10

Nonetheless, despite the results previously mentioned, there is a current controversy over whether theory of mind deficit can be considered a stable trait of schizophrenia, or if it involves instead a variable state that improves when symptomatology has subsided, mainly positive symptoms.¹¹ In his most recent research, Frith^12,13 tended to agree with the latter, and concluded that theory of mind task performance improves when positive symptoms subside. Safarti and Hardy-Baylé¹⁴ also support the hypothesis that theory of mind impairment is a transient state, and they relate it to symptoms of disorganisation, like incoherent speech or disorganised thought. Drury et al.¹⁵ arrived at the same conclusion in a study where they found that patients with schizophrenia showed worse theory of mind task performance than patients without schizophrenia, but only in the acute phase and not when the crisis had stabilised. Based on this conclusion, they question whether there is a real theory of mind impairment, as it might be an epiphenomenon in the acute phase of the illness. In the same vein, Pousa et al.¹⁶ found impairment only in patients with active symptomatology, while those in the remission phase demonstrated the same performance in second order tasks as control subjects.

Contrary to this idea, various authors uphold that theory of mind deficit constitutes a stable trait in patients with schizophrenia, that it is also observed in remission phases and in the absence of positive syntomology.^17,18 In a review performed by Brüne,¹⁹ this author includes various studies^20–22 that provide sufficient data to consider that theory of mind deficit is independent of the chronicity or of the acute phase of the illness. In a later study, Bora et al.²³ found that patients with positive symptoms displayed greater difficulties than those without positive symptomatology in making inferences about mental states of other people. In addition, these patients performed significantly worse in theory of mind tests than control subjects. In light of these data, these authors also concluded that theory of mind impairment is present in patients in the remission phase and is therefore a stable trait of schizophrenia. Herold et al.²¹ expanded this idea, and maintained that patients without positive symptoms only show difficulties with second order tasks (understanding irony), but they achieve good results in first order tasks (understanding metaphors). Thus, comprehension of second order theory of mind tasks becomes the stable deficit.

On the other hand, various studies have found a relation between theory of mind deficit and impairment in other neurocognitive functions (understood as they are presented in the Salvador-Carulla and Aguilar's article²⁴) like memory, executive functioning or the presence of general cognitive impairment.^12,23,25,26 In light of these results, there are authors who conclude that it is unclear whether patients with schizophrenia display a specific theory of mind impairment or, instead, if the observed deficit in their capacity to understand and interpret the intentions of other people is the consequence of impairment of basic cognitive functions, which in turn would affect their comprehension of the materials involved in the tests used to assess theory of mind in schizophrenia. In the opinion of Bora et al.²³ maintaining and later manipulating information are abilities necessary to successfully complete theory of mind tasks. This conclusion is in agreement with that argued by McCabe,²⁷ who affirmed that patients with schizophrenia do not present problems in comprehending others’ intentions in real interactions in daily life. For this author, it would be recommendable for theory of mind assessments to be carried out through observations in a natural context.

Finally, it should be pointed out that adequate theory of mind management has been related to patients’ social competence and functional performance.^18,19 Using a regression model, Roncone et al.²⁸ determined that the capacity to represent the mental states of other people was among the best predictors of good social functioning, together with recent onset of illness, good verbal fluency and low levels of positive or negative symptomatology. Based on these results, these authors concluded that it would be necessary to include theory of mind measurements in neuropsychological evaluations of patients, moving toward designing programmes that focus on this aspect and so increase the probability of good social adjustment.

Despite the growing importance that social cognition has acquired in the last few years in general–and that theory of mind has acquired in particular–there is no broad consensus concerning the most appropriate evaluative tests for this area. In 2002, the initiative Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) was started. One of its objectives was to design a neuropsychological battery that would serve as a reference for evaluating deficits of cognitive characteristics in schizophrenia. To evaluate social cognition, the Mayer–Salovery–Caruso Emotional Intelligence Test²⁹ was selected. However, of the 5 cognitive processes that make up social cognition, this test is centred on evaluating emotional processing, which does not specifically evaluate the ability to make inferences about the intentions of others, which would be more akin to theory of mind tests. With the purpose of more precisely evaluating this process, many authors have designed tests that include understanding metaphors, jokes, hints, irony, tricks, blunders or false beliefs (some of these tests are detailed in Table 1), all the abilities that are integrated into theory of mind. However, as Rodríguez Sosa and Touriño González³⁸ noted, the problem is that most tests lack psychometric validation and were developed by authors for their specific use. The use of these tests in our context presents an added difficulty in their validation and adaptation to Spanish.

Table 1.

Tests that evaluate theory of mind.

Authors	Test
Baron-Cohen^30	“Sally and Anne”, first order false-belief task“The ice cream man”, second order false-belief task
Happé^31	Stories that include metaphors, sarcasm and irony
Happé and Frith^32	“The cigarettes”, first order false-belief task“The thief”, second order false-belief task
Corcoran et al.^33	Hinting Task
Baron-Cohen^34	Joke comprehension
Stone et al.^35	Faux pas comprehension (blunders)
Adolphs^36	Humorous vignettes
Adachi et al.^37	Metaphor and Sarcasm Scenario Test

One of the most-used tests in the references for evaluating theory of mind is the Hinting Task, a test created by Corcoran et al.³³ for their specific use in schizophrenia and with good psychometric properties.^39–41 The Hinting Task is a test that includes 10 brief stories–with the objective of reducing the interference of possible memory impairment or verbal comprehension–that the evaluator can read to the subjects as many times as needed to assure a correct understanding of them. All of the stories have 2 characters and, at the end of each story, one of the characters drops a fairly clear hint. The subject is asked what the character in the story really wanted to say with the comment he or she made. If the subject responds correctly, they receive 2 points; if not, information is added to make the hint clearer. If the subject responds correctly on this occasion, 1 point is given. An incorrect response amounts to a 0. The total test score ranges from 0 to 20.

The objective of the present study consisted of evaluating the reliability of the Spanish adaptation of the Hinting Task, as well as comparing the performance in said task of control subjects and stabilised patients of long-term evolution.

Methods

The adaptation and validation of the Hinting Task was developed in 3 phases. In the first phase, 3 professionals, specialising in mental health, independently translated the original test from English to Spanish. Each translation was compared and 1 version was selected for the test (see Appendix A). Then, a person isolated from the first group of translators performed a back-translation of the selected version. Both the Spanish version and the back-translated English version were supervised and approved by Dr. Corcoran, one of the authors of the original test.

Once this process was finished, in the second phase of the study the test was administered to 39 control subjects, workers from the Padre Menni Hospital Centre (the sample's characteristics are provided in Table 2). Internal stability was evaluated by using Cronbach's alpha coefficient, and inter-rater and test–retest reliability were assessed by using the kappa index. The 2 administrations of the test, both in this case and that of the patient sample, were carried out 15 days apart.

Table 2.

Sample characteristics.

Sociodemographic characteristics	Control subjects (n = 39)	Patients (n = 40)	Difference
Sex
Male	17 (53.6%)	23 (57.5%)
Female	22 (56.4%)	17 (42.5%)
Age
18–29	8 (20.5%)	1 (2.5%)
30–39	21 (53.8%)	17 (42.5%)
40–49	9 (23.1%)	11 (27.5%)
≥50	1 (2.6%)	11 (27.5%)
Educational level
Primary	1 (2.6%)	16 (40%)	χ^2 = 18.720
Secondary	25 (64.1%)	12 (30%)	P < .001
University	13 (33.3%)	12 (30%)
Mean age	35.86 ± 7.28	42.38 ± 9.81	t = −3.286P = .002

Clinical characteristics
Age at onset of illness	25.74 ± 7.72
Years of evolution	14.26 ± 9.42
Years of evolution by group
<5	8 (20%)
5–10	10 (25%)
<5	22 (55%)
PANSS score^a
Positive scale (range 7–49)	11.40 ± 5.58
Negative scale (7–49)	16.45 ± 8.94
General psychopathology (16–112)	26.70 ± 8.94
Total score (30–210)	54.55 ± 14.82

a

Positive and negative syndrome scale for schizophrenia.

Once the psychometric properties of the control subjects were analysed, the Hinting Task was given to the group of patients with schizophrenia (see Table 2) and internal stability, inter-rater and test–retest reliability were assessed. Finally, scores between the control subjects and the patients with schizophrenia were compared in each of the 10 stories in the test, as well as the total score.

The patient sample was composed of 40 subjects diagnosed with schizophrenia that were referred to the Padre Menni Psychosocial Rehabilitation Centre of Santander by a psychiatrist in the Health System of Cantabria. At the moment of the study, all the patients were treated with antipsychotic medication. The clinical characteristics of the sample are specified in Table 2. As observed in the table, the entire sample is characterised by long-term evolution of the illness and a low level of symptomatology.

Both the control subjects and the patients signed an informed consent document giving their authorisation to be part of the study.

Results

In the control group, Cronbach's alpha coefficient was 0.64. Inter-rater reliability was above 0.80 for all stories (see Table 3). In story 4, inter-rater reliability could not be assessed since both evaluators gave all subjects a 2 for this story. This also affected the calculation of test–retest reliability, as once again, all subjects scored a 2 in both the first and second administration. The test–retest reliability values for the rest of the stories are shown in Table 4. As observed in the table, the test–retest reliability measure was equal to or greater than 0.70 for all the stories.

Table 3.

Inter-rater reliability.

	Control group			Patient group
	Kappa value	Confidence interval	Standard error	Kappa value	Confidence interval	Standard error
Story 1	1		0	0.82	0.65–0.98	0.08
Story 2	1		0	0.89	0.76–1.04	0.07
Story 3	1		0	1		0
Story 4	a*			1		0
Story 5	1		0	1		0
Story 6	0.89	0.69–1.09	0.10	0.96	0.88–1.04	0.04
Story 7	1		0	0.96	0.88–1.04	0.04
Story 8	0.85	0.69–1.01	0.081	0.92	0.81–1.02	0.05
Story 9	1		0	0.96	0.88–1.04	0.04
Story 10	1		0	1		0
Total score – 10 stories	0.83	0.69–0.97	0.07	0.85	0.72–0.97	0.06
Total score – 5 stories (stories 2, 3, 6, 7 and 9)	0.91	0.78–1.03	0.06	0.94	0.85–1.02	0.04

a*: variance equal to 0.

Table 4.

Test–retest reliability.

	Control group			Patient group
	Kappa value	Confidence interval	Standard error	Kappa value	Confidence interval	Standard error
Story 1	0.72	0.52–0.93	0.10	0.75	0.49–1.01	0.13
Story 2	0.72	0.42–1.02	0.15	0.73	0.45–1.01	0.14
Story 3	1		0	0.85	0.65–1.05	0.10
Story 4	a*			0.82	0.58–1.06	0.12
Story 5	1		0	0.75	0.48–1.01	0.13
Story 6	0.77	0.46–1.08	0.16	0.80	0.59–1.01	0.11
Story 7	0.95	0.85–1.05	0.05	0.92	0.76–1.08	0.08
Story 8	0.70	0.49–0.92	0.11	0.70	0.44–0.97	0.13
Story 9	0.77	0.46–1.08	0.16	0.70	0.42–0.97	0.14
Story 10	1		0	0.82	0.57–1.06	0.12
Total score – 10 stories	0.79	0.64–0.95	0.08	0.77	0.57–0.97	0.10
Total score – 5 stories (stories 2, 3, 6, 7 and 9)	0.81	0.63–0.99	0.09	0.83	0.67–1.00	0.09

Internal stability
	Control group	Patient group
10-Story version	α = 0.64	α = 0.69
5-Story version	α = 0.73	α = 0.78

a*: variance equal to 0.

In the group of patients with schizophrenia, Cronbach's alpha coefficient was 0.69. In Tables 3 and 4, the inter-rater and test–retest reliability results are shown, respectively, as well as the internal stability values. As can be seen in the tables, inter-rater reliability was also more than 0.80 for all the stories, and test–retest reliability was equal to or greater than 0.70.

Finally, the comparison between control subjects and patients with schizophrenia (see Table 5) showed that –except for the first and the last stories–there were significant differences between both samples in all the stories that compose the Hinting Task, as well as in the total test scores. In the case of the first story, it is emphasised that both control subjects and patients had low scores. In the last story, the opposite occurred. Likewise, in story 8, despite there being a significant difference between control subjects and patients, the subjects from the control group also had worse results than in the rest of the stories.

Table 5.

Comparison between control subjects and patients with schizophrenia and total score in the 10-story Hinting Task.

	Control subjects (mean ± SD)	Patients (mean ± SD)	t value
Story 1	1.36 ± 0.58	1.18 ± 0.59	1387
Story 2	1.79 ± 0.41	1.50 ± 0.75	2159^a
Story 3	1.95 ± 0.32	1.65 ± 0.74	2330^a
Story 4	2	1.85 ± 0.36	2590^a
Story 5	1.97 ± 0.16	1.78 ± 0.58	2081^a
Story 6	1.85 ± 0.37	1.18 ± 0.71	5250^b
Story 7	1.97 ± 0.16	1.80 ± 0.41	2504^a
Story 8	1.31 ± 0.57	1 ± 0.72	2111^a
Story 9	1.85 ± 0.37	1.38 ± 0.70	3715^b
Story 10	1.97 ± 0.16	1.85 ± 0.43	1707
Total score – 10 stories	18.03 ± 1.39	15.15 ± 2.83	5707^b
Total score – 5 stories (stories 2, 3, 6, 7 and 9)	9.41 ± 0.85	7.50 ± 2.02	5441^b

a

P < .05.

b

P < .001.

In light of the results above, and based on a previous study,²² a reduced version of the Hinting Task was selected, composed of 5 stories. Story 6 was added to the 4 stories used in the study referenced (stories 2, 3, 7 and 9), since it had the greatest difference in the present study between control subjects and patients. The reduced version had an internal stability of 0.73 for the control subjects and 0.78 for the patients. Similarly, the 5-story version showed a high correlation with the complete test in both control subjects (r = .751, P < .001) and the patients (r = .911, P < .001).

The control subjects and the patients with schizophrenia presented significant differences regarding mean age (t = −3.286, df = 75, P = .002) and educational level (χ² = 18.720, P < .001). However, neither group showed significant differences related to age, educational level or sex in the results of the Hinting Task. In particular, in the control group, these variables were as follows: age (F = 0.792, df = 3, P = .507), educational level (F = 1.376, df = 2, P = .265) and sex (t = 0.360, df = 37, P = .721). In the patient group, the following values were obtained: age (F = 0.115, df = 3, P = .951), educational level (F = 2.887, df = 2, P = .069) and sex (t = −1.186, df = 38, P = .823).

Conclusions

Given the importance that social cognition has acquired in schizophrenia over recent years, it is useful to rely on an instrument in Spanish to be able to evaluate cognitive function as a whole, or the specific processes involved within it, like theory of mind. In this study, psychometric properties of 2 Spanish-adapted versions of the Hinting Task were analysed: 1 that included the 10 original stories and a reduced version composed of 5 stories. The 10 stories that form the test have good inter-rater and test–retest reliability data in both the control group and the patient group. However, Cronbach's alpha data suggest that the reduced version should be used instead of the complete test, because the latter had a lower internal stability index. In light of these data, the 10-story version does not seem to be an adequate measure for theory of mind evaluation, while the 5-story version does. On the other hand, it should also be considered that reducing from 10 to 5 stories might decrease the sensitivity of the test.

As indicated previously, the reduced version is composed of the stories used in the Janssen et al. study²² (stories 2, 3 7 and 9, which have also shown a significant difference in the present study between control subjects and patients with schizophrenia), plus story 6, as it demonstrated the greatest difference between both groups. The remaining 5 stories are excluded from the reduced version. Stories 1 and 10 were not included, as they did not show a significant difference between control subjects and patients with schizophrenia, consequently making them less discriminatory for either group. Stories 4 and 8 were excluded because, despite there being significant differences, in the former the patients obtained a high mean score (1.85), while in the latter the control subjects achieved a very low mean score (1.31). Finally, including story 5 in the reduced version did not maintain internal stability (Cronbach's alpha = 0.71).

The results in respect to the sociodemographic variables in both samples indicated that neither educational level nor age significantly influenced the capacity to understand the intentions of others, confirming previous research results.¹⁷

The mean scores obtained by the control subjects and the patients with schizophrenia were similar to those found in other studies.^26,33 Likewise, the comparison between control subjects and patients showed that the latter achieved a significantly lower total score than the control subjects in both the 10-story and 5-story versions. In agreement with previous studies,^{19,21–23,33} these results indicate that, on the whole, patients with schizophrenia display difficulties in making inferences about the mental states of other people. As noted in Table 2, patients included in the sample used were generally characterised by a low symptomatology profile, which could support the idea that theory of mind deficit is a stable trait in schizophrenia, as it is also observed in stabilised patients of long-term evolution.^20–22 However, to confirm this conclusion, it would be necessary to perform a more detailed analysis of the results obtained, evaluating the possible difference between patients, considering their scores in the items that compose the positive scale of the PANSS. Similarly, as this is a cross-sectional study, whether the conclusion includes patients in the remission phase cannot be adequately assessed. This is also a question to be considered in future studies in order to provide more conclusive data.

It is important to emphasise that 25% of the sample (10 patients) performed similarly to the control subjects. It would be interesting to analyse these data in more detail in future studies with the objective of evaluating the differences between patients who demonstrate good performance in the Hinting Task and those who do not, and seeing which aspects may be influencing the preservation of this ability. To this end, it would be necessary to evaluate a relation between theory of mind and another series of important variables in schizophrenia, like symptomatology, years of evolution of the illness, neuropsychological functioning and psychosocial impairment.

Finally, it would also be recommended to repeat the psychometric analyses performed with the Hinting Task with a broader sample of control subjects and patients.

Ethical disclosures

Protection of human and animal subjects. The authors declare that the procedures followed were in accordance with the regulations of the responsible Clinical Research Ethics Committee and in accordance with those of the World Medical Association and the Helsinki Declaration.

Confidentiality of Data. The authors declare that they have followed the protocols of their work centre on the publication of patient data and that all the patients included in the study have received sufficient information and have given their informed consent in writing to participate in that study.

Right to privacy and informed consent. The authors have obtained the informed consent of the patients and/or subjects mentioned in the article. The author for correspondence is in possession of this document.

Conflict of interest

The authors have no conflict of interest to declare.

Acknowledgement

The authors wish to thank philologist Sandra Peredo for her work in the back-translation of the Hinting Task.