Validation of the Spanish-language version of the simplified modified Rankin Scale telephone questionnaire

Abstract

Introduction

The modified Rankin Scale (mRS) is the main functional assessment tool used after stroke. The simplified mRS questionnaire (smRSq) has recently been validated in English, and represents a simpler, reliable scale with an excellent reproducibility via telephone. The present study aims to validate a Spanish-language version of the telephone smRSq.

Methods

We conducted a prospective cohort study, assessing 50 patients 3 months after they presented an ischaemic stroke. We assessed the Spanish-language smRSq by telephone and in person, the mRS with a structured interview, the Spanish-language smRSq, and the Stroke Impact Scale-16. Inter-rater reliability, test-retest reliability, construct validity, and criterion validity were assessed.

Results

Inter-rater reliability showed strong agreement (k = 0.810); test-retest reliability and criterion validity showed moderate agreement (k = 0.639 and k = 0.759, respectively), and construct validity showed moderate agreement (Spearman correlation coefficient = –0.728).

Conclusions

The Spanish-language telephone smRSq is reliable and simple, and saves time in the functional assessment after a stroke.

Resumen

Introducción

La escala modificada Rankin (mRS) es la principal herramienta para la valoración funcional tras un evento cerebrovascular. Recientemente, fue validado en inglés, el cuestionario simplificado de la mRS (smRSq) telefónico, consiguiendo una escala basada en la mRS, más sencilla, fiable y con buena reproductibilidad telefónica. Nuestro objetivo es validar en castellano el cuestionario smRSq telefónico.

método

Se realizó un estudio de cohortes prospectivo, evaluando a 50 pacientes, a los 3 meses de haber sufrido un ictus isquémico. Se evaluó el cuestionario smRSq en castellano por teléfono y posteriormente de forma presencial la mRS mediante entrevista estructurada, el cuestionario smRSq y la SIS-16 (Stroke Impact Scale-16). Se estudió la fiabilidad interobservador, test–retest y la validez de constructo y de criterio.

Resultados

La fiabilidad interobservador obtuvo una correlación muy alta (k = 0,810); la fiabilidad test-retest y la validez de criterio respecto a la entrevista estructurada de la mRS de forma presencial, obtuvieron una correlación alta (k = 0,639 y k = 0,759) y la validez de constructo respecto a la SIS-16, obtuvo una correlación lineal alta (coeficiente de Spearman = −0,728).

Conclusiones

El cuestionario smRSq telefónico en castellano es una herramienta fiable, rápida y sencilla para la obtención del estado funcional de los pacientes tras un evento cerebrovascular, basado en la mRS.

Introduction

The Rankin Scale was developed in 1957 to assess functional status after a cerebrovascular event.¹ A modified version (the mRS) was later designed with a view to improving inter-rater agreement; this version uses a structured interview.²

In order to further facilitate the use of the mRS and making it an easier and more reliable scale, the simplified mRS questionnaire (smRSq) has recently been developed and validated. It consists of a series of simple yes/no questions that distinguish between the different categories of the mRS and may be administered to patients or their caregivers; these questions are reliable for obtaining the mRS score.^3–5

Patients’ functional status after a cerebrovascular event, assessed with the mRS, is one of the main primary outcome variables analysed in studies of cerebrovascular disease.⁶ An mRS telephone questionnaire would facilitate the collection of this information and would represent an advantage for those studies; it would also be a more time-efficient way to assess patients in clinical practice. Therefore, after the validation of the smRSq, its authors also compared its administration in person and over the telephone, concluding that the smRSq presents excellent telephone reproducibility.⁷

In view of the increasing need to use brief, simple scales with good inter-rater agreement to assess functional status after a cerebrovascular event, both in clinical practice and in research, we aimed to validate a Spanish-language version of the telephone smRSq in order to contribute such a tool.

Material and methods

We conducted a prospective cohort study at Hospital Universitario Miguel Servet in Zaragoza between January 2017 and May 2018.

The authors of the original English-language smRSq^3,7 authorised us to translate the questionnaire and validate the Spanish-language version. Using the translation and back-translation method proposed by Guillemin,⁸ we translated the smRSq into Spanish (Fig. 1).⁷

Figure 1.

Spanish-language version of the simplified modified Rankin Scale questionnaire (smRSq).

We specifically trained the researchers performing the interviews in the use of the mRS.

Based on previous studies performed for the validation of the original scale and its validation in other languages,^3–5,7,9 as well as on the “10-times rule” for sample size calculation (the sample must be at least 10 times the number of items¹⁰), our sample included 50 patients.

Our inclusion criterion was diagnosis of ischaemic stroke with admission to our hospital’s stroke unit; ischaemic stroke was defined as a sudden-onset focal neurological deficit progressing for more than 24 hours with a brain imaging study ruling out other causes. Exclusion criteria were age <18 years, not being a native Spanish speaker, not being able to attend a follow-up consultation at 3 months, and being unwilling to participate in the study.

Patients were selected during hospitalisation in the stroke unit and called for follow-up appointments 3 months after the cerebrovascular event for assessment using the scales.

In an initial phase, 15 patients were assessed by telephone using the definitive Spanish-language version of the smRSq, the week before the in-person assessment. The questionnaire was administered by 2 different raters (lead researcher and secondary researcher) to assess comprehension, format, usefulness, and inter-rater reliability. At the in-person consultation, the same patients were assessed using the structured mRS interview,² the smRSq, and the Stroke Impact Scale-16 (SIS-16).¹¹ The following week, patients were once more assessed by telephone by the lead researcher, who used the smRSq to assess the test-retest or inter-rater reliability.

In a second phase, the remaining patients were assessed by telephone with the smRSq one week before the in-person consultation. At the in-person consultation, they were assessed using the structured mRS interview, the smRSq, and the SIS-16.

Scales were administered directly to those patients who were able to answer the questions themselves, and with the help of their primary caregivers in the event of difficulties.

We performed a descriptive analysis of the data; qualitative variables were expressed as percentages and quantitative variables as measures of central tendency and dispersion.

We analysed the psychometric properties of the scale using reliability and validity tests: inter-rater and test-retest reliability were evaluated with the kappa index, construct validity with respect to the SIS-16 with the Spearman correlation coefficient, and criterion validity with respect to the mRS structured interview with the kappa index.

Results

Of the 50 patients included in our study, 60% were women; mean age (standard deviation) was 71.36 (15.84). Baseline characteristics are described in Table 1; Table 2 includes the characteristics of the cerebrovascular event and admission.

Table 1.

Baseline characteristics of our sample.

Baseline characteristics	N = 50
Hypertension	76%
Dyslipidaemia	54%
Diabetes mellitus	26%
COPD	10%
Atrial fibrillation	16%
Previous stroke	22%
Previous TIA	12%
Previous cerebral haemorrhage	2%
Ischaemic heart disease	14%
Peripheral artery disease	6%
Cognitive impairment	10%
Previous mRS score
0	70%
1	10%
2	6%
3	12%
4	1%
5	0%

COPD: chronic obstructive pulmonary disease; mRS: modified Rankin Scale; TIA: transient ischaemic attack.

Table 2.

Characteristics of the cerebrovascular event in our sample.

Characteristics	N = 50
NIHSS at admission, median (percentile)	4 (9)
Reperfusion therapy	28%
OCSP
TACI	26%
PACI	30%
LACI	30%
POCI	14%
TOAST
Atherosclerotic	4%
Cardioembolic	26%
Lacunar	16%
Other determined	2%
Undetermined	52%
Haemorrhagic transformation	6%
Respiratory infection	4%
Urinary tract infection	8%
Confusional syndrome	12%
mRS score at discharge
0	16%
1	28%
2	30%
3	8%
4	12%
5	6%

LACI: lacunar circulation infarct; mRS: modified Rankin Scale; NIHSS: National Institutes of Health Stroke Scale; PACI: partial anterior circulation infarct; POCI: posterior circulation infarct; TACI: total anterior circulation infarct; OCSP: Oxfordshire Community Stroke Project classification system.

In 82% of cases, the questionnaire was administered to the patient. Mean time between stroke and telephone assessment was 89.16 (6.62) days. Mean duration of the telephone smRSq was 74.34 (33.64) seconds, whereas mean duration of the in-person administration of the mRS structured interview was 153.42 seconds (87.81) (P < .001).

Tables 3 and 4 show the results obtained with the scales assessed 3 months after the cerebrovascular event.

Table 3.

Results of scales analysed.

Scales	N = 50
Telephone smRSq
0	6%
1	28%
2	8%
3	46%
4	2%
5	10%
In-person smRSq
0	12%
1	24%
2	14%
3	38%
4	2%
5	10%
In-person structured mRS interview
0	12%
1	24%
2	16%
3	34%
4	4%
5	10%
SIS-16, mean (SD)	72 (28.08)

mRS: modified Rankin Scale; SIS-16: Stroke Impact Scale-16; smRSq: simplified mRS questionnaire.

Table 4.

Results of scales analysed in the first phase.

Scale	N = 15	Scale	N = 15	Scale	N = 15
Telephone smRSq (LR)		Telephone smRSq (SR)		Telephone smRSq (LR*)
0	6.7%	0	6.7%	0	13.3%
1	33.3%	1	26.7%	1	33.3%
2	0.0%	2	13.3%	2	13.3%
3	46.7%	3	40.0%	3	26.7%
4	0.0%	4	0.0%	4	0.0%
5	13.3%	5	13.3%	5	13.3%

LR: lead researcher; SR: secondary researcher; smRSq: simplified modified Rankin Scale questionnaire.

*

Assessment by the lead researcher after the in-person consultation.

In terms of the reliability and validity of the scale, inter-rater (lead researcher vs secondary researcher) reliability presented a kappa index of 0.810 (very strong correlation); test–retest reliability (lead researcher at 2 different time points) presented a kappa index of 0.639 (strong correlation); construct validity with respect to the SIS-16 presented a Spearman correlation coefficient of –0.728 (strong correlation) (P < .001), suggesting a linear correlation; and criterion validity with respect to the mRS structured interview presented a kappa index of 0.759 (strong correlation).

If we classify the results of the smRSq and mRS as independent (0–2) or dependent (3–5),¹² we observe the same reliability and validity: inter-rater reliability presented a kappa index of 0.865; test–retest reliability presented a kappa index of 0.615, and criterion validity with respect to the mRS structured interview presented a kappa index of 0.722.

Discussion

The researchers who assessed the Spanish-language version of the telephone smRSq concluded in the first stage that the questionnaire was easily understood both by patients and by caregivers, easy to administer, and very useful for assessing functional status after a cerebrovascular event, based on the mRS. Furthermore, the questionnaire takes less than 2 minutes to complete in the majority of cases, which makes it a faster and more time-efficient tool that the mRS structured interview.

Our results show that the Spanish-language version of the telephone smRSq presents strong correlations for inter-rater reliability, test-retest reliability, construct validity, and criterion validity. Of these 4 measurements, we consider the most important to be inter-rater reliability, which indicates very high concordance between the results obtained by 2 different raters administering the Spanish-language version of the telephone smRSq, and criterion validity, which indicates a strong correlation between the results of the Spanish-language version of the telephone smRSq and those obtained with the mRS structured interview when assessed in-person, which we used as the reference test.

Furthermore, the Spanish-language version of the telephone smRSq presents good test-retest reliability and a strong correlation with the scale chosen to assess construct validity (SIS-16), which measures the degree with which the questionnaire reflects the concept it measures through theoretical concepts related to the item. In this case, a negative linear correlation was observed between the Spanish-language telephone smRSq and the SIS-16, indicating that higher results in the Spanish-language telephone smRSq are associated with lower scores in the SIS-16.

All these results are consistent with those obtained in the validation of the original version of the questionnaire.^3,7

One limitation of our study may be the repetition of the telephone smRSq during the same telephone call to study inter-rater reliability, which we attempted to minimise by instructing the interviewees to consider each interview independently. Furthermore, the similarity between the questions in the telephone smRSq and those in the structured mRS interview may also represent a bias; therefore, telephone interviews were held at least 3 days before the in-person consultation. In order to minimise the possible changes in the functional status of the patient between the telephone and in-person assessment, the telephone interview was performed no more than one week before the in-person consultation.

We believe that the Spanish-language version of the telephone smRSq is a reliable, fast, and simple tool for assessing the functional status of patients after a cerebrovascular event, based on the mRS. At present, with reperfusion therapies mainly being performed in large hospitals, the validation of the Spanish-language version of this questionnaire and its administration by telephone makes it easier for us to obtain information on the functional status of patients who may have difficulties attending a follow-up visit 3 months after stroke. Furthermore, a review revealed that the final result of clinical trials of cerebrovascular disease, based on the mRS in the majority of cases, was assessed by telephone in 13% of the 126 trials assessed¹³; therefore, the telephone smRSq represents a valid and reliable tool for obtaining these results.

Conflicts of interest

The authors have no conflicts of interest to declare.