Different measures for auditory hallucinations in populations with psychosis. The Validation of the Spanish versions of the Auditory Vocal Hallucination Rating Scale (AVHRS) and the Positive and Useful Voices Inquiry (PUVI)

Abstract

Introduction

An updated summary of the most used instruments assessing auditory hallucinations in population with psychosis, allows us to underline the scarceness and need of Spanish versions of important instruments. The aim of the study is to examine the psychometric characteristics of two different and complementary instruments for assessing auditory hallucinations, the Spanish version of the Auditory Vocal Hallucination Scale (AVHRS) and the Spanish version of the Positive and Useful Voices Inquiry (PUVI).

Materials and methods

A sample of 68 patients from four different centres, with a DSM-IV diagnosis of schizophrenia or schizoaffective disorder presenting with auditory hallucinations were included. Apart from the AVHRS and the PUVI, the Psychotic Symptom Rating Scales-Auditory Hallucinations subscale (PSYRATS-AH) and the Positive and Negative Syndrome Scale (PANSS) were also administered to all patients, plus an acceptability questionnaire.

Results

The Spanish version of the AVHRS showed a good internal consistency, a moderate to high inter-rater reliability, a medium to moderate test–retest reliability, and a good convergent and discriminant validity. The Spanish version of the PUVI showed a good internal consistency and a heterogeneous, but in general moderate, test–retest reliability.

Conclusions

The Spanish versions of the AVHRS and the PUVI have good psychometric properties and are well accepted among patients.

Resumen

Introducción

Un resumen actualizado de los instrumentos más utilizados en la evaluación de las alucinaciones auditivas en poblaciones con psicosis, nos permite subrayar la escasez y necesidad de versiones españolas de importantes instrumentos. El objetivo del estudio es examinar las características psicométricas de dos instrumentos para la evaluación de las alucinaciones auditivas diferentes y complementarios, la versión española de la Escala de Valoración de Alucinaciones Auditivas Vocales (AVHRS) y la versión española de la Encuesta sobre Voces Positivas y Útiles (PUVI).

Material y métodos

Se incluyó una muestra de 68 pacientes de cuatro centros diferentes, con diagnóstico de esquizofrenia o trastorno esquizoafectivo según el DSM-IV, que presentaban alucinaciones auditivas. Además de la AVHRS y de la PUVI, se administraron también a todos los pacientes la subescala de Alucinaciones Auditivas de la Escalas de Evaluación de Síntomas Psicóticos (PSYRATS-AH) y la Escala de Evaluación de Síndrome Positivo y Negativo (PANSS), además de un cuestionario de aceptabilidad.

Resultados

La versión española de la AVHRS mostró una buena consistencia interna, una fiabilidad inter-jueces de moderada a alta, una fiabilidad re-test de media a moderada, y una buena validez convergente y discriminante. La versión española de la PUVI mostró una buena consistencia interna y una fiabilidad test-retest heterogénea pero, en general, moderada.

Conclusiones

Las versiones españolas de la AVHRS y la PUVI tienes buenas propiedades psicométricas y son bien aceptadas entre los pacientes.

Introduction

Auditory hallucinations (AHs) are one of the most frequent and characteristic symptoms in patients with schizophrenia,¹ with epidemiological studies presenting a prevalence around 60–80%.²

The complexity of hallucinatory phenomena is well recognized, including different perspectives in the approach for its understanding: phenomenological studies, psychological models, and neuropsychological and functional neuroimaging studies.³

Psychological models have pointed out AHs’ multidimensional nature.⁴ Thus, assessing the different dimensions and characteristics of AHs is an essential issue for research purposes. It facilitates a better phenomenological description to obtain, for instance, psychometrically satisfactory subtypes of auditory verbal hallucinations as stated in the work of McCarthy-Jones et al.,⁵ or the identification of associations with specific environmental⁶ and biological factors.⁷

Multidimensional assessment is also important for clinical practice, providing a complete description of the different aspects of the symptom at baseline, to subsequently more accurately assess the impact of therapy.⁸ Actually, the most popular instrument for assessing AHs in the dimensional approach, the Psychotic Symptom Rating Scales (PSYRATS⁹), was developed in order to cover the need of standardized assessment and monitoring of symptom dimensions. This is essential to yield precise information reflecting dimensional change coinciding with treatment outcome.⁹ Our group previously performed the validation of the Spanish version of this instrument.¹⁰

In clinical practice, apart from assessing the dimensions of AHs, assessing specific other components is also necessary for providing tailored psychological treatments. Cognitive Behavioural Therapy (CBT) is the psychological approach that has accumulated more evidence in demonstrating improvements in treating persistent AHs.^11,12 According to cognitive models of hallucinations, the level of distress caused by hearing voices is associated to the appraisals the person makes on his/her voices. Negative appraisals, such as perceiving the voices as omnipotent and malevolent, are distressing and resisted, whereas voices believed to be benevolent are associated with positive emotions and engagement.^13–15 Therefore, CBT requires not only an exhaustive multidimensional assessment but also the evaluation of the three different components of the model: the cognitive component with both negative and positive voices, and emotional and behavioural components.

Recent promising developments in psychotherapy, such as application of acceptance and mindfulness-based approaches, and developments that emphasize the connection between voices and views of others, self and relationships,¹⁶ require specific assessment.

Therefore, assessment of AHs can be approached from different perspectives, and for different objectives. In a previous review, Gonzalez et al.¹⁰ presented a summary with the principal scales for the evaluation of AHs. More recently, Ratcliff et al.,¹⁷ based on Frederick and Killeen's work,¹⁸ published an updated review of AHs assessment tools that were developed and/or validated in English, focusing on patients with psychosis. They classified the instruments in four groups: (i) multidimensional rating scales, (ii) coping with AHs, (iii) beliefs about AHs rating scales, and (iv) acceptance and mindfulness of AHs. Based on their classification, in Table 1 we provide an updated summary of the most important instruments for assessing AHs in populations with psychosis presented in the different reviews.^10,17,18 Instruments developed or validated in other languages than English and, those included in book chapters on hallucinations assessment, are also presented.^1,19–21 Where a Spanish version of the instrument exists, this is specified. As can be observed in Table 1, the scarce number of AHs tools currently available and validated in the Spanish language underlines the need of well-adapted instruments.

Table 1.

Most important instruments for the evaluation of auditory verbal hallucinations (AVH) in psychotic populations: an updated summary based on Frederick and Killeen,¹⁸ Gonzalez et al.¹⁰ and Ratcliff et al.¹⁷

Instrument (author, year)	Brief description and psychometric characteristics
Multidimensional rating scales
Semi or structured interviews
Lowe^40	Semi-structured interview constructed to investigate the phenomenology of hallucinations and to discover which aspects of hallucinations differentiated diagnostic subgroups.Lack of evidence of reliability and validity limits its current use.
Rating Scale for the Phenomenology of Hallucinations^37)	Semi-structured interview that is an adaptation of Lowe's attempt of assessing hallucinations, adding also questions on patients’ attitudes towards their hallucinations.There is no psychometric studies of the instrument, limiting its use.
MUPS (Mental Health Research Institute Unusual Perception Schedule)^41	Semi-structured interview that assess comprehensively subjects’ experiences of auditory hallucinations.In a sample of 100 in-patients was found to be an acceptable instrument and in a subsample of 30 patients showed a good inter-rater reliability.
PSYRATS-AH subscale (Psychiatric Symptoms Rating Scale)^9 (Spanish validation Gonzalez et al.^10)	Structured interview that examines 11 dimensions of auditory hallucinations: frequency, duration, location (inside head or external), loudness, conviction in beliefs about origin, amount and degree of negative content, amount and intensity of distress, disruption, and controllability.Psychometric properties of the Spanish version were excellent: high reliability with coefficients higher than 0.85, good concurrent validity studied with Krawiecka scale, and good factorial validity with four factors explaining the 62% of variance.
Rating Scale for Psychotic Symptoms (RSPS) subscale 1 perception symptoms (illusions and hallucinations)^42	It is a 44-item rating instrument with a seven-point severity scale for each item. The subscale 1 includes a total of seven illusions and 11 hallucinations.
AHRS (Auditory Hallucinations Rating Scale)^43	It is a brief structured clinical interview that measures frequency, reality, loudness, number of voices, length, attentional salience and distress.Useful in measuring the effects of repetitive transcranial magnetic stimulation (rTMS)
AVHRS (Auditory Vocal Hallucination Rating Scale)^22	It is a semi-structured interview that was developed combining the AHRS with items from the PSYRATS-AH,^9 to which three items were added: (1) voices talking separately or simultaneously, (2) hypnagogic and/or hypnopompic hallucinations, and (3) form of address of voices (i.e. first, second or third person voices).The psychometric properties of the AVHRS were studied in a sample of Dutch patients and in Dutch children from the general population, and showed an excellent inter-rater agreement and a good internal consistency.^28
Clinical Characteristics of Auditory Hallucinations rating scale^44	Clinical interview developed in Greek, which assess broadly the characteristics of auditory hallucinations.Little psychometric evidence.
Self-report questionnaires
Junginger and Frame^45	It requires patients to estimate the frequency of their hallucinations and to rate loudness, clarity, location and reality of their most recent hallucinations using visual analogue scales.Psychometric evidence limited to one study.
Topography of Voices Rating Scale^46	It consists of 13 items measuring frequency, loudness, clarity, distress and intrusiveness on a five-point rating scale.Psychometric adequacy limited to one study.
HPSVQ (Hamilton Programme for Schizophrenia Voices Questionnaire)^47	It covers similar domains that the PSYRATS-AH but in a self-report format.Test–retest reliability over one week was acceptable. Internal consistency and convergent validity with PSYRATS-AH were adequate.
CAHQ (Characteristics of Auditory Hallucinations Questionnaire)^48	It examines the broad characteristics of AHs in the past 24 h, using a 6-point rating scale. It covers frequency, loudness, controllability, clarity, tone, distraction and distress.The psychometrics and validity of the scale have not been investigated.
Delusion and Voices Self-Assessment^49	Brief self-report instrument, developed in Italian, that assess delusions and hallucinations from the client's perspective.It demonstrate to be a reliable, valid and sensitive instrument for assessing subjective experiences.
Matsuzawa Assessment Schedule for Auditory Hallucinations^50	Self-report that offers a broad assessment of multiple dimensions of AHs in Japanese. It consisted of three sections: (i) perceptual characteristics, (ii) patient responses and judgments(related or accompanying symptoms); and (iii) beliefs and general judgments. It includes also some rater judgement items.
Computerized binary scale of auditory speech hallucinations (cbSASH)^51	This a scale that was primarily designed for the identification of phenomenologically defined subtypes of AVH. It also provides measures of the severity of hallucinations with variables such as frequency and loudness.
Coping with AHs
RAHQ (Responses to Auditory Hallucinations Questionnaire)^52	It is a self-report questionnaire, based on stress-appraisal coping model, that comprises three subscales: active coping, withdrawal coping and suppression coping. Three subscales have acceptable internal consistency; other psychometric characteristics studied were weak.
MACS (Maastricht Assessment of Coping Strategies)^53	It is a semi-structured brief interview developed to assess the amounts of distress, control and coping in relation to psychotic symptoms. The instrument showed to be reliable and useful to assess coping in relation to subjective experience of distress by and control over psychotic symptoms.
VCS (Voice Compliance Scale)^54	This is an observer rated scale to measure specifically the frequency of command hallucinations and level of compliance/resistance with each identified command.
Beliefs about AHs rating scales
BAVQ-R (Beliefs about Voices Questionnaire Revised)^14(Spanish validation of the BAVQ by Robles-Garcia et al.^55)	Self-applied questionnaire on beliefs, emotions and behaviours on auditory hallucinations. It consisted of 5 subscales: malevolence, benevolence, omnipotence, resistance and engagement.Psychometric properties of the Spanish version were good, appearing as a valid and reliable instrument to evaluate beliefs about identity, power, purpose, and consequences of obeying auditory: good internal consistency, good criteria validity and the same factorial structure than the original version.
CAS (Cognitive Assessment Schedule)^4	It is used in conjunction with the BAVQ-R to assess further the individual's feelings and behaviour in relation to the voice, and his/her beliefs about the voice's identity, power, purpose or meaning and the likely consequences of obedience or resistance.
VPD (Voice Power Differential)^56	It is a self-report scale based on the social rank theory that examines subordination and dominance in social relationship. It provides a measure of the respondent's perception of the disparity of power between themselves and the voices in six constructs: strength, confidence, respect, ability to harm, superiority and knowledge. VPD is both internally and temporally consistent.
PUVI (Positive and Useful Voices Inquiry)^23	The questionnaire includes a number of items from the Interpretation of Voices Inventory^57, the Beliefs About Voices Questionnaire^13, and the Auditory Vocal Hallucination Rating Scale^22. It explores the prevalence, course and characteristics of both positive and useful voices. The inventory also contains two subscales exploring reasons for attributions (the positive voices and the useful voices subscales). The PUVI was administered to a sample of hallucinating patients from the Voices Outpatient Psychiatric Department (VOPD) and the Dutch Resonance Foundation (DRF), i.e., two samples of psychiatric patients and voice-hearers not receiving psychiatric treatment, respectively. Reliability analysis demonstrated good internal consistency of the two subscales (Cronbach's alpha: positive voices scale = 0.92, useful voices scale = 0.89).
VAY (Voice and You Scale)^58(Spanish validation Perona-Garcelán et al.^59)	A scale measures the interrelation between the person who hears voices and his or her predominant voices based on two axes representing the power and proximity of this relationship. It consisted of four scales: voice dominance, voice intrusiveness, hearer distance and hearer dependence.Spanish version of VAY is a reliable and valid instrument: good internal consistency and acceptable test–retest reliability and concurrent validity.
Acceptance and mindfulness of AHs
SMVQ (Southamptom Mindfulness of Voices Questionnaire)^60	It is a self-report scale that measures four constructs: (1) clarity of awareness of the present moment vs being lost in responding to the voices; (2) allowing attention to remain on an unpleasant voices vs avoiding it, (3) acceptance of the difficult situation and of oneself vs judgement of the situation and self, (4) letting go vs struggle and rumination. It was found to have good internal consistency and adequate convergent validity.
VAAS (Voices Acceptance and Action Scale)^61VAAS brief versions (VAAS-9 and VASS-12)^62	It is a self-report scale, developed to measure two constructs: (a) acceptance of having the experience of AHs and (b) the ability to act autonomously. It has a general section for all patients with any AH, and a section specific for command hallucinations. Internal consistency was high and temporal stability over a 4-month period was adequate, as also convergent and discriminant validity.Two brief versions were found to be robust measures of acceptance of psychotic experiences.

Thus, given the importance of assessing the multidimensionality and the cognitive components of AHs and the limited availability of tools in Spanish, two different and complementary instruments for assessing the experience of auditory hallucinations were selected to be translated: the Auditory Vocal Hallucination Rating Scale (AVHRS) and the Positive and Useful Voices Inquiry (PUVI), both developed and validated at the Voices Outpatient Department (VOPD) of the University Medical Centre Groningen, the Netherlands. The aim of the current study is to present the Spanish versions of the AVHRS and the PUVI and to examine their psychometric properties.

Material and methods

Sample

The sample consisted of 68 patients (both inpatients and outpatients). Inclusion criteria were (i) a DSM-IV diagnosis of schizophrenia or schizoaffective disorder, presenting with AHs, (ii) age between 18 and 65 years, and (iii) a good command of the Spanish language. Exclusion criteria were having an organic mental disorder and/or comorbidity with mental retardation.

Recruitment was performed at four different centres in Spain: Valencia Clinic Hospital (n = 20), Barcelona Sant Pau Hospital (n = 17), Barcelona Parc Sanitari Sant Joan de Déu (n = 16) and Jerez de la Frontera Hospital (n = 15).

This study was integrated as part of a broader project on CBT for AHs. Patients were first informed about the study and then signed an informed consent form. The study was approved by the clinical research ethics committee of the participating institutions.

Materials

The following instruments were translated and validated:

• -

  The Auditory Vocal Hallucination Rating Scale (AVHRS),²² a 16-item structured interview which provides detailed information on auditory vocal hallucinations during the last month (or other time periods). It assesses: number of voices (separately or simultaneously), hypnagogic and/or hypnopompic voices, frequency, duration, localization, loudness, origin of the voices, negative content, severity of negative content, frequency of distress or suffering, intensity of distress or suffering, interference with daily functioning, control, anxiety, interference with thinking, and first, second and third person voices. Each item consists of a compulsory question, followed by optional support questions. Items are scored on a 4- or 5-point scale, ordered in increasing severity. The items are dichotomized in not severe (0) and severe (1), and a severity score was calculated by adding up the (severe) scores: Number of voices (more than 1 voice), Separately or simultaneously (score 3, 4), Frequency (3, 4), Duration (3, 4), Volume (3, 4), Origin (3, 4), Negative content (3, 4), Severity of negative content (3, 4), Frequency of distress (3, 4), Intensity of distress (3, 4), Interference with daily functioning (3, 4), Control (3, 4), Anxiety (3, 4), Interference with thinking (3, 4), and Third person voices (present). The Spanish version included in the Annex and also can be downloaded from http://bi.cibersam.es/).

• -

  Positive and Useful Voices Inquiry (PUVI),²³ a 39-item self-report AH inventory designed to assess sociodemographic data and psychopathology (five items), prevalence, course and characteristics of both positive (those experienced as pleasant or pleasurable) and useful (either positive or negative, but with a clear useful function) auditory vocal hallucinations (6 items each). Also, attribution of positiveness and usefulness is explored through 9 items each on a 5 point-Likert scale. The Spanish version is included in the Annex and also can be downloaded from http://bi.cibersam.es/).

• -

  A 4-item acceptability questionnaire asking about length (1-very brief, 4-very long), level of difficulty (1-very easy, 4-very difficult), understandability (1-very difficult to understand, 4-very easy to understand), and usefulness (1-useless, 4-very useful) of both questionnaires.

Demographic and other clinical data were obtained from all patients. The following instruments were administered:

• -

  Positive and Negative Syndrome Scale (PANSS)²⁴ (Spanish version Peralta and Cuesta²⁵). It consists of 30 items, scoring the symptom severity on a 7-point scale. The PANSS has three subscales: positive symptoms (PANSS-P), negative symptoms (PANSS-N) and general psychopathology (PANSS-G).

• -

  Psychotic Symptoms Rating Scales-Auditory Hallucinations subscale (PSYRATS-AH)⁹ (Spanish version Gonzalez et al.¹⁰). It consists of 11 items assessing different dimensions of auditory hallucinations in the past month: frequency, duration, localization, loudness, origin of voice, amount and degree of negative content, amount and intensity of distress, disruption, and controllability.

Procedure

The English version of the AVHRS and PUVI were initially translated into Spanish, and subsequently back translated by another native Spanish person with a very good command of the English language.

All instruments were administered in the same session. First, the PANSS interview was administered. Subsequently, the PSYRATS-AH, AVHRS, and PUVI were administered. A second rater scored the AVHRS at the same time. The acceptability questionnaire for both the AVHRS and the PUVI was administered after the respective instrument.

Finally, with an interval of two weeks, in which patients did not receive any CBT, the AVHRS and PUVI were again administered in order to examine the temporal stability of the instruments.

Statistical analysis

To establish the reliability of the AVHRS interview and the PUVI, the internal consistency was calculated using Cronbach's alpha. To obtain inter-rater reliability and temporal stability, Spearman correlations for ordinal items (1 to 15) and Cramer's V for the nominal item (16) were calculated between AVHRS scores obtained by the two different raters or at the two different time points, respectively. Spearman correlations between basal PUVI items and those obtained two weeks later were calculated to establish the test–retest reliability of the PUVI.

To establish the convergent validity of the AVHRS, Spearman correlations with the PSYRATS-AH and with PANSS-P – more specifically with the Hallucination behaviour item – were calculated. To establish the discriminant validity of the AVHRS, Spearman correlations with PANSS-N were calculated.

The acceptability results of both instruments are presented in terms of frequency and proportions of different responses.

Analyses were performed with the statistical package IBM SPSS version 22. To calculate Cronbach's alphas we used the “psych” package (Procedures for Psychological, Psychometric and Personality Research)^26,27 from the statistical application R (R: A Language and Environment for Statistical Computing), version 3.2.2. This package makes calculations without losing power when missing data are found.

Results

Sample description

Demographic, clinical and psychopathological characteristics of the sample are presented in Table 2. As it can be seen, the majority of the participants were male and ages ranged from 18 to 64 years (mean = 39.60).

Table 2.

Sample description: demographic, clinic and psychopathological variables (N = 68).

	Mean (SD)/N (%)
Gender
Male	44 (64.7%)
Female	24 (35.29%)
Age	39.6 (10.59)
Diagnosis
Schizophrenia	62 (91.2%)
Schizoaffective disorder	6 (8.8%)
Illness duration (years)	17.3 (9.3)
Illness course
Episodic course with inter-episodic persistent symptoms	17 (25%)
Continuous course	42 (61.8%)
Episodic course with partial remission	6 (8.8%)
Not specified	3 (4.4%)
PANSS-P	21.06 (6.14)
PANSS-N	20.30 (6.88)
PANNS-G	38.58 (9.82)
PSYRATS-AH total score	27.43 (5.36)

Most patients had a diagnosis of schizophrenia. Mean illness duration was 17.3 years (range 1–50). In terms of course of the illness, most patients presented a continuous course. The average duration of AHs was 12.5 years (range 0.25–40).

The sample was moderately symptomatic as is evident from all three PANSS subescales. Regarding to PSYRATS-AH, items scores range from 0 to 4. Sum score was calculated obtaining a moderately symptomatic score.

AVHRS acceptability

Fifty-nine patients (80.88%) completed the acceptability questionnaire. Forty-two patients (71.2%) considered the instrument brief or very brief. Thirty-seven patients (62.7%) scored the inquiry as easy or very easy. With respect to understandability, twenty-one patients (35.59%) scored it as easy or very easy to understand. Finally, forty-six patients (78%) considered the instrument as useful or very useful.

AVHRS reliability: internal consistency, inter-rater reliability, temporal stability

Cronbach's alpha coefficient for all items of the AVHRS (with the item Number of voices dichotomized in 1 or >1) amounted to 0.71.

Data for inter-rater reliability were available for 52 out of the 68 patients (76.47%). Patients not included in the second rating did not differ in gender (X² = 2.13, p = 0.14), age (t = 0.07, p = 0.94), duration of illness (t = 0.23, p = 0.82), and in the AVHRS severity score (t = 1.13, p = 0.26). All correlations were statistically significant, ranging from 0.41 to 0.96, most of them >0.70. The lowest correlation (0.41) was in item 16 (Third person voices). Regarding the severity score, the correlation was 0.79 (p < 0.01), indicative of a strong positive correlation. Data are shown in Table 3 in the Annex.

Table 3.

Reliability: Spearman and V-Cramer correlation coefficients between rater#1 and rater#2 scores of all AVHRS items (N = 52), and between test re-test scores in all AVHRS items (N = 57).

	1a	1b	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16 _1	16_2	16_3	Severity score
Inter-rater	.96^**	.80^**	.71^**	.87^**	.86^**	.78^**	.77^**	.82^**	.80^**	.71^**	.63^**	.56^**	.86^**	.63^**	.72^**	.77^**	.61^**	.69^**	.41*	.79^**
Test–retest	.67^**	.66^**	.76^**	.67^**	.62^**	.70^**	.56^**	.73^**	.55^**	.73^**	.58^**	.51^**	.63*	.53^**	.45^**	.26	.63^**	.64^**	.53^**	.48^**

*

p < 0.05.

**

p < 0.01.

To study the temporal stability reliability, a subsample of 57 out of the original 68 patients (83.82%) participated in the second assessment. Patients not included in retest analysis did not differ in gender (X² = 0.001, p = 0.97), age (t = −0.55, p = 0.58), duration of illness (t = 1.32, p = 0.19), and AVHRS severity index score (t = −1.31, p = 0.20).

Spearman's and Cramer's V correlations obtained in the test–retest data, were statistically significant for all items (ranging from 0.45 to 0.76), except for item 15 (Interference with thinking) (r = 0.26). The highest correlation coefficients (≥0.70) were obtained in item 2 (Hypnagogic and/or hypnopompic voices), item 5 (Location), item 7 (Origin of the voices), and item 9 (Severity of negative content). The severity score had a correlation of 0.48 (p < 0.01) between the two assessment points. The results are presented in Table 3.

AVHRS validity: convergent and discriminant validity

To establish the convergent validity, the correlations were calculated between a subset of AVHRS items and the PSYRATS-AH, both assessing similar dimensions of AHs (see Table 4). All correlations, ranging from 0.61 to 0.90, were statistically significant. In addition, the correlation coefficient between PSYRATS-AH overall score and the AVHRS severity score was significant (r = 0.62, p < 0.01).

Table 4.

Spearman correlation coefficients between AVHRS items, equivalent PSYRATS items, PANSS hallucination item and PANSS subscales (convergent and discriminant validity) (N = 68).

	PSYRATS's equivalent items^a	PANSS
		Hallucinatory behaviour item	Positive	Negative
1a Number of voices		0.19	0.07	-0.03
1b Separately or simultaneously		0.18	0.09	-0.02
2 Hypnagogic hypnopompic		0.26*	0.13	0.21
3 Frequency	0.78^**	0.43^**	0.34^**	0.15
4 Duration	0.76^**	0.38^**	0.10	0.23
5 Localization	0.85^**	−0.02	0.16	0.10
6 Loudness	0.66^**	0.17	−0.04	0.02
7 Origin	0.59*	0.11	0.48^**	0.14
8 Negative content	0.69^**	0.12	0.29*	−0.05
9 Severity of negative content	0.75^**	0.32*	0.44^**	0.19
10 Frequency of distress	0.59^**	0.15	0.11	0.19
11 Intensity of distress	0.65^**	0.15	0.17	0.25*
12 Interference functioning	0.56^**	0	0.36^**	0.37^**
13 Control	0.47^**	−0.03	0	0.16
14 Anxiety		0.17	0.24	0.09
15 Interference with thinking		0.1	0.10	0.18
16 1st person voice		0.42^**	0.03	−0.35
16 2nd person voice		0.16	−0.08	−0.09
16 3rd person voice		0.18	−0.05	0
Severity score	0.62^**	0.40^**	0.34^**	0.16

a

PSYRATS items: Frequency, Localization, Loudness, Origin of voice, Amount of Negative Content, Degree of negative content, Amount of distress, Intensity of distress, Disruption, Controllability, Overall score.

*

p < 0.05.

**

p < 0.01.

Still regarding convergent validity, the Hallucinatory behaviour item from the PANSS presented the highest correlation with Frequency (0.43), First person voices (0.42), Duration (0.38) and with the AVHRS severity score (0.40). The PANSS-P Total score was significantly correlated with five AVHRS items: Origin (0.48), Severity of negative content (0.44), Interference with functioning (0.36), Frequency (0.34), and Negative content (0.29). Also the correlation with the AVHRS severity score was significant (0.34).

As to the discriminant validity, the correlations between AVHRS items and PANSS-N were mostly very low and statistically non-significant (see Table 4).

PUVI acceptability

Responses from 48 patients (70.59%) to the PUVI acceptability questionnaire were available. Thirty-two patients (66.7%) considered it brief or very brief. Twenty-three patients (47.9%) scored the inquiry as easy or very easy. Twenty-one patients (43.7%) scored the PUVI as easy or very easy to understand. Finally, thirty-three patients (68.7%) considered the instrument as useful or very useful.

PUVI reliability: internal consistency and temporal stability

Regarding the internal consistency, Cronbach's alpha for the Positive Voices subscale using was 0.71, and for the Useful Voices subscale it was 0.79.

Scores at retest were available for 49 patients (72.06%). Patients not included in the second rating did not differ in gender (X² = 0.16, p = 0.69), age (t = 1.70, p = 0.95), and duration of illness (t = 0.58, p = 0.12), and in the AVHRS severity score (t = 1.04, p = 0.30).

In general, most Spearman correlations of the PUVI scores at first and second assessment were moderate but statistically significant, although heterogeneous, with correlations ranging between 0.1 and 0.92 (see Table 5).

Table 5.

Test–retest Spearman correlation coefficients of PUVI items (N = 49).

1a	1b	2a	2b	3	4	5	6	7	8	9	10	11
0.61**	0.65**	0.53**	0.64**	0.43**	0.63**	0.48**	0.55**	0.1	0.45**	0.47**	0.49**	0.38*

12	13	14	15	16	17	18	19	20	21	22
0.53**	0.72**	0.60**	0.91**	0.54**	0.56**	0.66**	0.64**	0.57**	0.3	0.62**

23	24	25	26	27	28	29	30	31	32	33	34	35	36	37	38	39
0.74**	0.70**	0.44*	0.38	0.76**	0.50*	0.13	0.19	0.57**	0.70**	0.49**	0.72**	0.52*	0.48*	0.56**	0.75**	0.92**

* p < 0.05; ** p < 0.01.

(N = 31)

* p < 0.05; ** p < 0.01

(N = 25)

* p < 0.05; ** p < 0.01.

Discussion

This study provides an updated summary of the most used instruments assessing AHs in populations with psychosis, which has outlined the scarceness of and a need for Spanish versions of important instruments. The main objective of the current study was to examine the psychometric properties of the Spanish versions of the AVHRS, and the PUVI. The results show that both instruments present good psychometric properties and are well-accepted by patients. The Spanish version of the AVHRS has good reliability and validity indexes: a good internal consistency, a moderate to high inter-rater reliability, a medium to moderate test–retest reliability, and a good convergent and discriminant validity. The Spanish version of the PUVI presents acceptable psychometric properties: a good internal consistency, and a heterogeneous but in general moderate, test–retest reliability.

In general, patients considered the AVHRS interview brief, easy, understandable and useful, pointing at good acceptability. The AVHRS acceptability had not been studied before, but Bartels-Velthuis et al.²⁸ examined face validity, questioning the participants about the comprehensibility and comprehensiveness of the scale using a short questionnaire. They observed that all patients understood the meaning of the questions, and recognized the content as a part of their voice hearing. Therefore, we may conclude that the Spanish version of the AVHRS is well accepted among patients, just like the Dutch version.

The AVHRS reliability was good. The internal consistency for the overall interview was high. In this sense, a score of 0.70 is usually considered the cut-off for an acceptable measure of internal consistency.^29,30 Our results are in the line with the Bartels-Velthuis et al.²⁸ study, where they found an internal consistency of 0.84 in a sample of 62 adult patients, and of 0.88 in a non-clinical children's group (n = 347), both considered good.³¹ Other studies validating hallucination rating scales also calculated their internal consistency. With PSYRATS-AH's, Drake et al.³² obtained Kendall's tau-b correlations between 0.63 and 0.76 except for the item ‘control over hallucinations’ that was lower (0.16). Steel et al.³³ also calculated item inter-relationships as a measure of internal consistency, obtaining 11 of 55 correlations significant at the 0.1% level, ranging between 0.20 and 0.71. Therefore, although lower than in the Dutch version, the AVHRS’ internal consistency could be considered acceptable, and within the usual values.

In the current study, in a subsample of 52 patients rated by two different raters, the inter-rater reliability of the AVHRS was moderate to high. The lowest correlation was for the ‘Third person voices’ item, although still statistically significant. Previous studies have analyzed the inter-rater reliability with small samples. To assess the inter-rater reliability, Bartels-Velthuis and colleagues²⁸ analyzed 23 successive interviews from a clinical adult sample, which were observed by four raters. Cohen's kappa as a measure of agreement was 0.84, which is considered excellent.³⁴ On the other hand, Haddock et al.⁹ studying the psychometric properties of the PSYRATS-AH, found a good inter-rater reliability, established for six patients. All AH items except two (Disruption and Control) had an unbiased estimate of reliability above 0.9. Drake et al.³² also used the PSYRATS, and selected thirteen patients to calculate interclass correlation between raters for subscales and total, obtaining excellent results (AH subscale 0.99–1). Interclass correlations (ICCs) for AH items ranged from 0.74 to 1, except ‘location of voices’ which had an ICC of 0.42. Our study showed a good inter-rater reliability, although less consistent for some items, in line with the general literature.^9,32

The test–retest reliability of the Spanish version of AVHRS, within a period of two weeks, yielded medium to moderate correlations. Only the item assessing ‘interference with thinking’ was statistically not significant. The most stable dimensions of auditory hallucinations were hypnagogic and/or hypnopompic voices, location, origin of the voices, and severity of negative content. This stability was expected as patients did not receive any psychological intervention between these two time points. In the Dutch AVHRS study, Bartels-Velthuis et al.²⁸ did not assess the test–retest reliability as the AVHRS administration was always incorporated as part of the assessment of the psychological treatment, providing patients with coping strategies, thus hampering an objective comparison of the two time points. Regarding the PSYRATS, Drake et al.³² assessed the temporal stability in a subgroup of patients with a first psychotic episode, interviewing them weekly for the first six weeks. They compared the scores obtained between the fifth and the sixth week, in which the symptoms were stable. The ICC for AH was 0.70. Individual items had an ICC between 0.55 and 0.74. It should be noted that the greater variability observed in the Spanish version of the AVHRS could be explained by the study sample consisting of patients being diagnosed mainly with schizophrenia but also with schizoaffective disorder (8.8%), and although most of them presented persistent and chronic symptoms, some of them (8.8%, not only being those with schizoaffective disorder) presented a more acute state, and therefore their symptoms were more unstable.

A high correlation was found between the AVHRS and the PSYRATS indicating a good convergent validity. As expected, all item correlations were statistically significant, obtaining medium to high correlations. With regard to the correlations with the PANSS, the specific Hallucination item had the highest correlation with probable the most classical hallucinatory dimension, the Frequency, and, interestingly, with First person voices. The PANSS-P subscale was also significantly correlated with other hallucinatory dimensions than the Hallucination item itself, especially ‘origin’ and ‘interference with functioning’. Convergent validity was not examined in the Dutch AVHRS validation study. Several studies using the PSYRATS-AH explored this validity aspect. Haddock et al.⁹ compared it with the Kravieka scale (KGV), and Drake et al.³² and Steel et al.³³ compared it with the PANNS, obtaining also moderate to high correlations, in line with our findings. Based on our results, we may conclude that the AVHRS has a good convergent validity with regard to PSYRATS-AH, the PANSS Hallucination item, and the PANSS-P.

As expected about the discriminant validity, most AVHRS items presented low and non-significant correlations with the PANSS-N, showing that the interview is able to discriminate between different constructs.

Regarding the PUVI psychometric study, the majority of patients offered positive responses about length and utility, and a substantial amount of the patients was positive about its understandability, thus showing a good acceptability. To our knowledge, the PUVI acceptability has not been studied before. In general, the Spanish version of the PUVI was well accepted by patients.

The PUVI's reliability was also good. Cronbach's alpha was high for both the Positive and the Useful Voices subscales, indicating a good internal consistency. Jenner et al.³⁵ studied the internal consistency of the Dutch version also for the two subscales obtaining better results than for the Spanish version (Positive Voices subscale: Cronbach's alpha = 0.92; Useful Voices subscale: Cronbach's alpha = 0.89). The Benevolence subscale from the revised Beliefs About Voices Questionnaire (BAVQ-R),¹⁴ that assesses positive intentions attributed to hallucinations, also showed a high internal consistency (Cronbach's alpha = 0.88).

Correlations between the PUVI scores obtained in the two different temporal assessments were, in most cases, moderate and statistically significant, showing the inquiry an acceptable test–retest reliability. However, some items presented unexpectedly low correlations. This heterogeneity found in the two temporal moment correlations needs an explanation. In the literature, it has been described an association between impaired insight and less self-report accuracy in populations with diagnosis of schizophrenia and schizoaffective disorder.³⁶ As the PUVI is a self-report measure, it could be possible that differences in insight in our sample would explain such heterogeneity, with some items being more sensitive to the lack of insight than others. Jenner et al.³⁵ nor other authors studied the temporal stability of the instrument. The BAVQ-R psychometric study¹⁴ did not include either this reliability aspect. Anyway, our extensive examination of the PUVI indicates that this scale is adequate and useful for the assessment of positive and useful AHs.

Although some other studies have approached positive³⁷ or pleasurable³⁸ hallucinations, they did not examine the psychometric properties of the instrument³⁷ nor did they use a comprehensive specific instrument for its assessment,³⁸ so it is not possible to compare our results with other studies. Regarding useful hallucinations, to our knowledge, the PUVI is the only instrument that assesses them. Future studies may elaborate on useful hallucinations.

Conclusions

Although nowadays the idea that AHs are not necessarily a hallmark of psychosis³⁹ is becoming popular, its high prevalence and relevance in populations with psychosis is undeniable. The complexity of hallucinatory phenomena requires different approaches to its study and to attend multiple characteristics and components. The importance of an appropriate assessment has been outlined in this paper, and the variety of approaches and objectives for performing such an assessment have been described. We have shown the scarceness of Spanish validation studies of instruments assessing AHs. The current paper has focused on presenting and studying the psychometric properties of two different and complementary instruments.

The AVHRS is conceptualized and makes sense mainly within the phenomenological approach and the psychological models that consider a dimensional conceptualization of AHs. The PUVI shows its utility in the context of a cognitive model of voices and the CBT interventions, where the beliefs about voices are crucial in order to understand the patients’ feelings and behaviours. Not much is known about the significance and implications of the positive and useful voices, therefore deserving further study.

A limitation of the current study is the rather small sample size. Besides, we had to deal with some missing data. However, a strength of the study is the participation of four different centres in the research project which provides quite heterogeneous raters and patients with different backgrounds. In spite of these differences, both instruments showed good psychometric properties. Moreover, we presented novelty regarding the validation of both the AVHRS and the PUVI.

In conclusion, the Spanish versions of the AVHRS and the PUVI were shown to have good psychometric properties and were well accepted among patients.

Ethical standards

The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with Helsinki Declaration of 1975, as revised in 2008.

Financial support

This project has received financial support from CIBERSAM, Instituto de Salud Carlos III, ISCIII (BICIBERSAM grant) and Ministerio de Sanidad y Consumo (ETES 09/91030 grant), and partly funded by grant FEDER funds of the EU and CERCA Programme (Generalitat de Catalunya).

Conflict of interests

None.

Acknowledgements

We would like to acknowledge the patients for their participation, and Blanca Llácer, Sabrina Cruz, Concha Esteban, Elena Huerta-Ramos, Maria Serra-Blasco, Anna Alonso and Mireia Rabella for their help in recruiting and assessing patients.