Spanish and Catalan translation, cultural adaptation and validation of the Picker Patient Experience Questionnaire-15

Abstract

Objective

To develop and test a culturally adapted core set of questions to measure patients’ experience after in-patient care.

Material and methods

Following the methodology recommended by international guides, a basic set of patient experience questions, selected from Picker Institute Europe questionnaires (originally in English), was translated to Spanish and Catalan. Acceptability, construct validity and reliability of the adapted questionnaire were assessed via a cross-sectional validation study. The inclusion criteria were patients aged >18 years, discharged within one week to one month prior to questionnaire sending and whose email was available. Day cases, emergency department patients and deaths were excluded. Invitations were sent by email (N = 876) and questionnaire was fulfilled through an online platform. An automatic reminder was sent 5 days later to non-respondents.

Results

A questionnaire, in Spanish and Catalan, with adequate conceptual and linguistic equivalence was obtained. Response rate was 44.4% (389 responses). The correlation matrix was factorable. Four factors were extracted with Parallel Analysis, which explained 43% of the total variance. First factor: information and communication received during discharge. Second factor: low sensitivity attitudes of professionals. Third factor: assessment of communication of medical and nursing staff. Fourth factor: global items. The value of the Cronbach alpha was 0.84, showing a high internal consistency.

Conclusions

The obtained experience patient questionnaire, in Spanish and Catalan, shows good results in the psychometric properties evaluated and could be a useful tool to identify opportunities for health care improvement in our context. Email could become a feasible tool for greater patient participation in everything that concerns his health.

Resumen

Objetivo

Diseñar y validar un conjunto básico de preguntas culturalmente adaptadas para medir la experiencia de los pacientes después de un ingreso hospitalario.

Material y métodos

Siguiendo la metodología recomendada por guías internacionales, se realizó la traducción y adaptación al español y al catalán de un conjunto básico de preguntas sobre experiencia del paciente, seleccionadas de cuestionarios en inglés del Picker Institute Europe. Se realizó un estudio transversal de validación para evaluar la aceptación, la validez de constructo y la fiabilidad del cuestionario adaptado. Los criterios de inclusión seleccionaron adultos dados de alta del hospital entre una semana y un mes antes, y de los que se disponía de su correo electrónico. Se excluyeron los ingresos de menos de 24 h, las altas del servicio de urgencias y las altas por defunción. Se les invitó a participar por correo electrónico (N = 876) y el cuestionario fue completado a través de una plataforma online. Un recordatorio automático fue enviado a los no respondedores 5 días después del primer correo electrónico.

Resultados

Se obtuvo un cuestionario, en español y en catalán, con una equivalencia conceptual y lingüística adecuada. La tasa de respuesta fue del 44,4% (389 respuestas). La matriz de correlación fue factorizable. Cuatro factores fueron extraídos en un análisis paralelo, que explicaron el 43% de la varianza total. Primer factor: información y comunicación recibida durante el alta; Segundo factor: actitudes poco sensibles de los profesionales; Tercer factor: evaluación de la comunicación con médicos y enfermeras; Cuarto factor: ítems globales. El valor del coeficiente alfa de Cronbach fue de 0,84, indicando una alta consistencia interna.

Conclusiones

El cuestionario de experiencia del paciente obtenido, en español y en catalán, muestra buenos resultados en las propiedades psicométricas evaluadas y puede constituir un instrumento útil para identificar oportunidades de mejora de la asistencia sanitaria en nuestro entorno. El correo electrónico podría convertirse en una herramienta viable para una mayor participación del paciente en todo lo que concierne a su salud.

Introduction

Patient-centered care concerns the relationship between health providers and patients (and their relatives and caregivers) along the healthcare process.¹ It implies patient involvement in decision-making about their healthcare and treatment, as well as patient and public participation in healthcare planning.² Both of them require the establishment of mechanisms to identify preferences, problems or unmet needs of patients. Satisfaction analysis has shown limitations in understanding these problems, as satisfaction is a subjective construct related to expectations rather than with problem solving.^3,4

The Beryl Institute defines patient experience⁵ as “the sum of all interactions, shaped by an organization's culture, that influence patient perceptions across the continuum of care”. The assessment and measurement of patients’ experience facilitates both strategic and shared decision-making affecting every patient.⁶ The increasing interest from policy makers to obtain this information and build a “health service designed around the patient”⁷ shows a trend toward studying patients’ experience.^8,9

However, there are numerous ways to measure and understand patient experience, and no one ‘best’ method for all circumstances. A combination of methods, both qualitative and quantitative, must be often used for its assessment.¹⁰ Patient-reported experience measures capture a person's perception of their experience with health care or service, providing insight into what truly matters most to patients and information about the patient-centeredness of the health care they receive.¹¹

One example of patient's experience assessment is the Hospital Consumer Assessment of Healthcare Providers and Systems,¹² which results influence decisions about health services financing in MEDICARE. Another example are the surveys conducted by the Picker Institute Europe in the National Health Service for assessing patient experience of in-patient care.^13,14 In Spain, there are some surveys to measure satisfaction during hospitalization,^15–17 yet not directly experience.

Building on extensive qualitative research to determine which aspects of care are important to patients, the Picker Institute developed standardized instruments to measure the quality of care.^14,18 The Picker Adult in-Patient Questionnaire consists of a number of sections asking patients about their condition, demographic details, and aspects of their health care experience. A core set of 15 questions selected from that questionnaire, known collectively as the Picker Patient Experience questionnaire (PPE-15), has been developed and validated for use in comparative and longitudinal benchmarking. In 2014, another short form scale, the Oxford Patient Involvement and Experience scale,¹⁹ was developed from the adult inpatient survey to compare the hospital experiences of patients regarding the presence of long-term conditions.

Before being used in a new cultural or linguistic context, any questionnaire should be translated and adapted in an appropriate manner so that the results are a good reflection of what is being measured and to allow comparability between different contexts.²⁰ The PPE-15 questionnaire was translated to Spanish and extended in 2009 by Barrio-Cantalejo et al.²¹ in order to obtain a tool of 33 questions that explored the perception of patients about the process of information and participation in decision-making, specifically directed to the Informed Consent procedure. However, the objective of that translation was different from ours and the questionnaire was translated only to Spanish.

The purpose of this study was to develop and test an adapted core set of questions to measure patients’ experience after in-patient care in Spanish and Catalan.

Material and methods

Study design

Cross-cultural adaptation of the Picker Patient Experience questionnaire and cross-sectional validation study

Questionnaire development

Picker Institute was asked for permission. The original PPE15 was modified, with their collaboration, in order to adapt it to the on-going objectives. First, the question referring to whether medical staff talked as if the patient was not there was divided in two; doctors and nurses, according to how it is done in the most recent version of the Adult In-Patient Survey. Secondly, four questions from the Oxford Patient Involvement and Experience scale¹⁹ were incorporated referring to feeling enough emotional support from staff; being treated with respect and dignity; being involved in decisions about discharge; and being told who to contact after discharge. Demographic questions from the National Health System survey were used, besides the ethnicity question, which was adapted following the Barcelona Health Survey.²² Finally, the questionnaire had 16 questions about experience (Q2–Q9, Q11–Q18), an introductory question about the hospitalization route (Q1), a filter question about whether or not they had had pain (Q10) and seven socio-demographic questions (Table 1).

Table 1.

Problems identified by the 16 questions about patient experience included in the questionnaire.

Item	Item content
Q2	Doctors’ answers to questions not clear
Q3	Doctors sometimes talked as if patient wasn’t there
Q4	Nurses’ answers to questions not clear
Q5	Nurses sometimes talked as if patient wasn’t there
Q6	Staff gave conflicting information
Q7	Patient not sufficiently involved in decisions about treatment and care
Q8	Staff didn’t discuss worries and fears
Q9	Staff didn’t give enough emotional support
Q11	Staff did not do enough to control pain
Q12	Patient not sufficiently involved in decisions about hospital discharge
Q13	Purpose of medicines not explained
Q14	Patient not told about medication side effects
Q15	Patient not told about danger signals to look for at home
Q16	Patient's family not given information needed to help patient recovery
Q17	Patient not told about the contact person after discharge
Q18	Patient not always treated with respect and dignity

The PPE15 was designed to be easily and quickly completed by patients, and to enable straightforward scoring. To this end the instrument is brief, and a simple additive scoring algorithm has been adopted.¹⁵ Following Jenkinson et al.,¹⁸ each item was coded for statistical analysis as a dichotomous “problem score”, indicating the presence or absence of a problem. A problem was defined as an aspect of health care that could, in the eyes of the patient be improved upon.

The obtained questionnaire underwent a cross-cultural adaptation process following the guidelines established by the International Society for Pharmacoeconomics and Outcome Research²³ in order to obtain Catalan and Spanish versions. Including investigation of conceptual and item equivalence, two original instrument translation and two back translations for each target language, synthesis of the documents and revision by an expert committee. For the Spanish translation, the previous Spanish version of the PPE-15 was taken into account.²¹

The versions obtained were then tested via cognitive debriefing. In December 2015, 10 patients discharged from different hospitalization units of Hospital Clínic of Barcelona were asked to complete the questionnaire. Afterwards, they were asked about how they had understood it. We tried to get as much diversity as possible in the group.

Study participants

Patients aged 18 years or more, discharged from Hospital Clínic within one week to one month prior to questionnaire sending (10^th March to 10^th April 2016) and whose e-mail address was available in the hospital database were invited to participate. The following patients were excluded: (i) patients whose length of stay lower than 24 h (ii) patients discharged from emergency department and (iii) deaths. To estimate a confidence interval of 95% for the Cronbach's alpha of a scale of 16 items with a margin of error of +/−0.1 and hypothetical value of alpha = 0.8, a sample size of 147 patients was required. This sample size was considered fair for factor analysis.²⁴

LimeSurvey^® virtual platform was used to send an e-mail invitation to participate with a unique link to an online version of the questionnaire, allowing patients to choose between Spanish and Catalan. An automatic reminder was sent five days later to non-respondents.

Statistical analysis

Descriptive statistics included frequencies and percentages of the categorical variables and means and standard deviation, or median and interquartile range, of the continuous variables. In order to identify possible differences regarding questionnaire language, bivariate analysis was carried out using the chi-squared test for categorical variables and Student's t test for continuous variables.

Questionnaire acceptability was judged according to the total percentage of respondents, the percentage of respondents to answer individual items, and the total completion time.

Exploratory factor analysis was carried out to test construct validity of the questionnaire. A correlation matrix was calculated, using pairwise deletion in the case of item nonresponse. The Kaiser–Mayer–Olkin index and Barlett's test of sphericity first measured the strength of inter-correlation between items and tested whether the population correlation matrix was an identity matrix. Factor extraction was performed using the minimum residual (minres) procedure, and Parallel Analysis was used to determine the number of factors. After oblimin rotation, the extracted factors were interpreted considering loading of 0.32 or above as indicative of the underlying dimension.²⁴

Questionnaire internal consistency was tested using Cronbach's alpha coefficient for the whole questionnaire and after the suppression of each item. The level generally regarded as adequate is 0.7.²⁵ Finally, item-total correlations were evaluated expecting moderate correlations, since they assess a construct but are not considered to be paraphrases.

Ethical considerations

The virtual platform emailed a link allowing access to the questionnaire, storing data anonymously. Patients were informed about the study objectives and the voluntary and anonymity of the survey. Receipt of a completed questionnaire was taken as consent to participate in the survey. Ethical approval for this study was sought and obtained from the Hospital Clinic Research Ethics Committee with reference number HCB/2015/1041.

Results

Cross-cultural adaptation process

As a result of the translation and adaptation process, the questionnaire was modified in order to guarantee conceptual and linguistic equivalence. As there is no difference between the concept “emergencia/emergència” (emergency) and “urgencia/urgència” (urgency) among general population, the term “Servicio de Urgencias/Servei d’Urgències” was considered easier to understand. The literal translation of the term ‘involved’ would be “involucrado/involucrat” but it was replaced by the verb “participar” (to participate in), because in Spanish and Catalan language it matches better to the real aim of the question. The comprehension problems identified during the cognitive debriefing were discussed and changes were made to the questionnaire as follows. Q7 and Q12 were rephrased in a way that helps understanding while keeping the question's original aim. A clarification sentence was added to Q15 so that patients could contextualize the term “danger signals”. It was also observed that many patients needed support of a relative or friend to answer the questionnaire. Therefore, the possibility to do so was noted in the invitation letter.

Questionnaires are available on request to the authors.

Study participants

After applying the selection and exclusion criteria, 876 patients were invited to participate. Table 2 summarizes the socio-demographic data, from the hospital database.

Table 2.

Socio-demographic characteristics according to hospital database.

	Patients who meet inclusion criteria (n = 2010)	Patients who meet inclusion criteria with email registered (n = 876)
Sex^a
Male	1120 (55.7)	480 (54.8)
Female	890 (44.3)	396 (45.2)
Age^b	66 (52–76)	60 (46–70)
Department^a
Medicine	1070 (53.2)	481 (54.9)
Surgery	801 (39.9)	336 (38.4)
Other	139 (6.9)	59 (6.7)
Hospitalization days^b	5 (2–9)	4 (2–8)

a

Frequency (percentage).

b

Median (interquartile range).

In total, 389 patients completed the questionnaire, 178 in Spanish and 211 in Catalan. The respondents socio-demographic characteristics, according to the questionnaire, are given in Table 3. The median age was 56 and 61 years respectively. The number of participants with at least one limiting long-term condition was 95 (45%) and 62 (34.8%) respectively. About half of the participants had a planned admission, 94 (52.8%) among Spanish respondents and 111 (52.6%) among Catalan respondents. In each group, more than 70% of patients completed the questionnaire themselves.

Table 3.

Socio-demographic characteristics and experience results from the questionnaire.

	Catalan (n = 211)	Spanish (n = 178)
Admission type^a			P = 0.93^c
Emergency	84 (39.8)	70 (39.3)
Planned	111 (52.6)	94 (52.8)
Other	10 (4.7)	12 (6.7)
NR	6 (2.8)	2 (1.1)
Sex^a			P = 0.98^c
Female	99 (46.9)	85 (47.8)
Male	102 (48.3)	88 (49.4)
NR	10 (4.7)	5 (2.8)
Age^b	61 (51–72)	56 (43–67)	P = 0.56^d
NR	39	36
Long-standing limiting conditions^a			P = 0.03^c
None	111 (52.6)	109 (61.2)
One	62 (29.4)	49 (27.5)
Two or more	33 (15.6)	13 (7.3)
NR	5 (2.4)	7 (3.9)
Nationality^a			P = 0.11^c
Spanish	190 (90.1)	160 (89.9)
Foreigner	2 (1.0)	5 (2.8)
Both	4 (1.9)	9 (5.1)
NR	15 (7.1)	4 (2.3)
Respondent^a			P = 0.01^c
Patient	148 (70.1)	130 (73.0)
Relative/Friend	32 (15.2)	21 (11.8)
Both	24 (11.4)	20 (11.2)
With health professional	0	0
NR	7 (3.3)	7 (3.9)

a

Frequency (percentage).

b

Median (interquartile range).

c

Chi-squared test.

d

T-test. NR: no response.

Acceptability

On average, participants spent 9 min 11 s completing the questionnaire. The initial response rate was 29.3%, which increased to a final response rate of 44.4% after the reminder. The number of missing values per item ranged from 4 (1.9%) to 19 (9%) in Catalan and from 1 (0.6%) to 15 (8.4%) in Spanish (Table 4).

Table 4.

Item study.

Item	Catalan		Spanish
	Missing values (%)	Item-total correlations	Missing values (%)	Item-total correlations
Clear doctors answers	8 (3.8)	0.60	5 (2.2)	0.59
Doctors talking in front of patient	13 (6.2)	0.33	4 (2.2)	0.26
Clear nurses answers	14 (6.6)	0.58	15 (8.4)	0.63
Nurses talking in front of patient	19 (9.0)	0.41	13 (7.3)	0.35
Discrepant information	4 (1.9)	0.52	6 (3.4)	0.56
Involved in decisions	13 (6.2)	0.47	11 (8.2)	0.56
Worries and fears	9 (4.3)	0.69	8 (4.5)	0.59
Emotional support	11 (5.2)	0.67	5 (2.8)	0.62
Pain control	12 (5.7)	0.57	5 (2.8)	0.44
Involved in discharge	14 (6.6)	0.50	7 (3.9)	0.56
Purpose of medicines explained	8 (4.0)	0.59	6 (3.4)	0.51
Side effects explained	9 (4.3)	0.57	7 (3.9)	0.57
Danger signal explained	11 (5.2)	0.64	11 (8.2)	0.64
Information delivered to caregivers	12 (5.7)	0.59	6 (3.4)	0.63
Contact information	12 (5.7)	0.50	9 (5.1)	0.46
Respect and dignity	6 (2.6)	0.57	1 (0.6)	0.49

Validity

Bartlett's test of sphericity gave a value of 1522.7 (P < 0.001) and KMO index of sample adequacy was 0.86, thus indicating that the correlation matrix is factorable. As a result of the Parallel Analysis, four factors were extracted, which explained 43% of the total variance. Factor loadings are presented in Table 5.

Table 5.

Factor loadings.

Factor number:	1	2	3	4
Q2. When you had important questions to ask a doctor, did you get answers that you could understand?	01	−01	74	−04
Q3. Did doctors talk in front of you as if you weren’t there?	10	50	00	−11
Q4. When you had important questions to ask a nurse, did you get answers that you could understand?	00	24	36	27
Q5. Did nurses talk in front of you as if you weren’t there?	−01	100	−.01	01
Q6. Sometimes in a hospital, a member of staff will say one thing and another will say something quite different. Did this happen to you?	04	−04	34	27
Q7. Were you involved as much as you wanted to be in decisions about your care and treatment?	39	02	07	05
Q8. Did you find someone on the hospital staff to talk to about your worries and fears?	22	01	36	17
Q9. Do you feel you got enough emotional support from hospital staff during your stay?	12	00	35	31
Q11. Do you think the hospital staff did everything they could to help control your pain?	−01	01	−03	73
Q12. Did you feel you were involved in decisions about your discharge from hospital?	46	−10	04	10
Q13. Did a member of staff explain the purpose of the medicines you were to take at home in a way you could understand?	50	07	00	11
Q14. Did a member of staff tell you about medication side effects to watch for when you went home?	71	01	−04	−09
Q15. Did a member of staff tell you about any danger signals you should watch for after you went home?	58	−01	17	00
Q16. Did the doctors or nurses give your family or someone close to you all the information they needed to help care for you?	31	06	32	4
Q17. Did hospital staff tell you who to contact if you were worried about your condition or treatment after you left hospital?	48	00	−14	15
Q18. Overall, did you feel you were treated with respect and dignity while you were in the hospital?	08	04	04	54
Percentage of explained variance	14	9	10	9
Accumulated percentage of explained variance	14	22	32	41

In bold: loadings >0.32. Loadings have been multiplied by 100.

The first factor is mainly loaded by items Q12 to Q15, which refer to information and communication received during discharge. The second factor, loaded by Q3 and Q5, reflects low sensitivity attitudes of professionals, not taking into account the patient presence. The third factor encompasses items Q2, Q4, Q6, Q8 and Q9 which relate to the assessment of communication of medical and nursing staff. Finally, the fourth factor is loaded by global items: Q18 (respect and dignity) and Q11 (pain control). The Q16 (Did the doctor or nurses give your family or someone close to you all the information they needed to help you recover?) appeared in factors 1 and 3 with a similar magnitude, but met the 0.32 threshold for inclusion only in factor 3.

These four factors significantly correlated between them, indicating that the assessment of the communication, information and emotional support received are related.

Reliability

The Cronbach's alpha coefficient of the overall scale was 0.84 (CI 0.81–0.87) indicating high internal consistency. Specifically for Catalan version was 0.84 (CI 0.81–0.87); and for Spanish 0.83 (CI 0.79–0.87).

These figures do not improve substantially after the removal of each item individually (between 0.82 and 0.85 in Catalan; between 0.81 and 0.84 in Spanish). Correlations between each item and the total scale indicate adequate discrimination for each item (Table 4).

Discussion

The cross-cultural adaptation process included a literature review, team discussions, opinions of experts and professionals and a pilot test in a group of patients, thus ensuring the conceptual and linguistic equivalence of the translated questionnaires. From the perspective of patient experience evaluation, increasing the number of assessment tools which are available allows greater flexibility and adaptation to specific needs.

The response rate is at the bottom end of the 46%–74% range that has been reported in other countries using the PPE-15¹⁴; however, in other cases the questionnaire was paper-based and sent by post. The response rate obtained with the previous adaptation of the PPE15 performed by Barrio-Cantalejo²¹ was 71.4% but the survey was phone-based. If we look at the response rate of satisfaction questionnaires used in Spain,^15–17,26 a considerable variability can also be observed. According to a meta-analysis of 39 studies the median response rate on internet surveys was 27% (range 14.5%–51%).²⁷

However, none of the mentioned studies has used an online-based questionnaire. Although some comparative studies showed a lower response rate for digital questionnaires compared to paper when they started to be used, this trend is reversing, probably related to the general increase in internet use and information and communication technology literacy, and later studies have shown similar response rates between them and also a decrease in costs when digital questionnaires are used.^28,29 Couturier et al. showed that the information reported online did not differ significantly from those collected by phone.³⁰ Consequently, we believe that an online-based approach could become a feasible alternative for patients’ experience surveys in our context, subject to availability of e-mail addresses for contacting patients.

The results of the exploratory factor analysis showed four interpretable factors explaining 41% of the variance. This indicates that the 16 survey questions cannot be reduced to four factors. In any case these four dimensions can be considered the most important. The number of respondents indicating a problem in each item ranged from a minimum of 17 (9.6%) for Spanish and 28 (13.3%) for Catalan, to a maximum of 97 (54.5%) and 147 (69.7%), respectively. These results are similar to those obtained with PPE-15 in other countries^14,18 although the Catalan maximum limits exceeds the figure by 10 percentage points.

Regarding the reliability of the questionnaire, both Catalan and Spanish questionnaire versions had high Cronbach's alpha, indicating good internal consistency.

With regard to limitations some points must be taken into account. Firstly, e-mail addresses were unavailable for a majority (56.4%) of eligible patients. Baseline demographic characteristics show a possible selection bias, since patients whose e-mail was registered in the database are younger and had shorter hospitalization than the total sample. This may cause bias in results, particularly if the trend for younger people to give more critical reports of their care is present as it is in other countries. However, we would obtain the most critical opinion and therefore this would identify opportunities for improvement.

Moreover, there may be unmeasured characteristics associated with whether or not patients have e-mail that are also related to their experiences of care. For example, one might hypothesize that more affluent and better educated patients may be more likely to have e-mail addresses recorded; if these factors are, as might be expected, related to people's experiences of care then this may create further bias.

Another possible limitation is the data collection during a particular year period, which can influence the pathology prevalence and hospital disease burden. We therefore recommend that the survey is carried out in different periods of the year. These limitations should be considered when generalizing the results obtained in relation to the patient's experience in this study and when using this questionnaire in other contexts. The translated questionnaire is orientated toward patient experience and patient-centered care and designed to be easily and quickly completed by patients, as well as to allow straightforward interpretation by professionals. These features, together with the fact that it is currently being used in other countries, make it a useful tool for comparison between centers.

In conclusion, the process of adaptation of a questionnaire is a dynamic process that must add up evidence to obtain the best tool for a given context. The questionnaire obtained showed conceptual and linguistic equivalence, as well as acceptable validity and reliability. Using e-mail as the way of invitation and a web-based survey tool was pioneer in our hospital and the response rates were higher than expected. Despite the possible selection bias, which should always be considered when interpreting the data, e-mail and web-based approach seems to be feasible and economically sustainable. It is therefore believed that this approach may allow the systematic study of patient experience in order to assess temporal evolution and to implement improvements.

Funding

This study has been funded in part by the Master Plan for Respiratory Diseases, Ministry of Health (Catalonia). The Catalan Society of Healthcare Quality (SCQA) contributed to the dissemination of the study results at the XXXV Congress of the Spanish Society of Healthcare Quality (SECA) and XIX Conference of the SCQA.

Conflict of interest

The authors declare no conflict of interest.

Acknowledgements

The authors are grateful to Margarida Jansà, Jordi Vilardell, Roser Cadena, Montserrat Núñez, Imma Grau and Mercè Vidal, as members of the Hospital Clinic's working group for information and patient education, for providing constructive suggestions based on an initial draft of the study protocol.

Special thanks to Carolina Casañas, for her advisory contributions.