When trying to create new PRO and PRE from ground it is very important to work with a qualitative approach, since they allow researchers to identify key areas for patients, that can be later used for a scale or a questionnaire.7,8 The important question at this stage of the research is to identify what matters most to patients and caregivers of HRT.9 In the initial stages of research, an exploratory approach is needed that produces new knowledge10,11 (Fig. 2).

Fig. 2.

Article selection algorithm.

(0.2MB).

The methods used for inquiry and item identification were interviews and workshops. As recommended by the American Society for Anesthesiology it is important to “gather the opinions of patient focus groups and relevant health care professionals”.12

Their nature is inductive, and the accuracy depends on the right selection of participants.13 The sample can be small and does not aim to be fully representative, but to produce information that is both rich and coherent with research questions.14 It is only in a second phase, when a quantitative approach is needed in order to assess validity and reliability with large samples.15,16

The driver for the research was a tender for home respiratory care issued by the Catalan Health Department. The tender demanded from applicants the use of PROM and PREM in HRT in order to meet following criteria: patient monitoring, patient empowerment, adherence to therapy, patient satisfaction, lifestyle changes, intelligent storage of data and changes in care processes and care relationships.

Bibliographical research was conducted in two phases. The first one was conceived to determine whether there were already PRO and PRE scales for home respiratory therapies (HRT) that could be used. In case there would not be already validated scales, a second phase should be started in order to identify the most relevant scales for those pathologies treated with HRT. Relevant items that were related to the dimensions of the tender in Catalonia were chosen by patients and health care professionals as an input for specific HRT. For the first phase a broad research was conducted on National Center for Biotechnology Information (NCBI, https://www.ncbi.nlm.nih.gov). 85 articles were found on follow up of patients suffering diseases treated with HRT with PRO scales. Yet, no articles were found specifically for PRO and PRE use for HRT.

A second phase in our study had to be started in order to identify specific PRO and PRE scales for pathologies treated with HRT: sleep apnea, COPD, cystic fibrosis, lung fibrosis, neuromuscular diseases and asthma. Inclusion criteria: specific and generic validated scales for the above-mentioned pathologies in Spanish language. Exclusion criteria: scales not culturally adapted to Spanish context and questionnaires for prediction of exacerbations or disease detection, as well as age margins narrower than 6 years and questionnaires filled in by healthcare professionals.

130 articles were found in databases and finally 20 articles met the inclusion and exclusion criteria.

To create the scales, patient involvement was sought, in such a way that the items were adapted to their needs. To this aim, patients who were HRT users were interviewed. The patient sample was selected through so called “Punto Inspira” (facilities of VitalAire – company responsible for HRT in certain areas of the Community of Madrid – in public hospitals in order to support respiratory patients) and patient associations.

As a starting point, the criteria indicated in the tender specifications of the Catalonia HRT service tender were taken into account, as they were the first specifications in Spain to consider PROMs and PREMs for HRT.

Characteristics of the patient sample: a total of 12 interviews with patients took place (4 from oxygen therapy, 4 from CPAP, 2 from mechanical ventilation and 2 from aerosol therapy) and there were 4 interviews with caregivers (2 from aerosol therapy and 2 from mechanical ventilation). The sample was completed when saturation of obtained data appeared.14

All the participating patients were residents in Madrid and their ages ranged from 20 to 85. The information gathered from interviews was organized into 4 different patient journey maps, one for each kind of home respiratory therapy in order to organize and represent patient needs and preferences according to perceptions, emotions and improvement opportunities, as growing evidence finds the tool adequate.15

After listening to patients’ needs and priorities, the research team selected 40 PRO items for each therapy from a total of 51 PRO questionnaires found for respiratory diseases. The items were selected taking as a reference the information provided by the patients in semi-structured interviews. The patients identified the most significant and important particularities of each of the therapies in relation to their quality of life. With this information, questions were developed to include the most relevant PRE for each of the HRTs.

With the aim of reducing and concretizing the selection of questionnaire items, 3 validation workshops were arranged: two workshops for selecting and classifying the PROs and one to validate the PREs. The workshops had two kinds of goals: (a) screening the items coming from PRO scales for respiratory diseases in order to select those that were most adequate to build 4 PRO scales, one for each HRT; (b) validating the items prepared by the research team to create PRE questionnaires. Participants were patients and healthcare professionals with following inclusion criteria: pneumologists with more than 5 years of experience in HRT, physiotherapists with more than 3 years of experience with HRT, nurses also an experience of over 3 years in HRT, HRT technicians with an experience of over 3 years in the area and also expert patients having used HRT over 3 years. Professionals not related to HRT or with less than 3 years of experience in the area were excluded. In total 17 healthcare professionals from the different hospitals in Madrid (La Princesa, Ramón y Cajal, Gregorio Marañón) and in Bilbao (Basuto, Cruces and Galdácano) and 21 patients participated in the workshops.

The patient sample was selected from patient associations and “Punto Inspira”. The characteristics of the participants were: expert patients of both genders in an age range from 20 to 85 years with 3 or more years using HRT. For the first workshop 8 patients of HRT participated, namely users of oxygen therapy, CPAP and aerosol therapy. Mechanical ventilation patients were excluded due to the severity of their conditions and the impossibility of their caregivers to leave patients for attending the workshop. For the second patient validation workshop 8 patients (different from those in the first workshop) of the mentioned therapies attended. For the PRE workshop 8 patients (different from those in previous workshops) from were enrolled of both genders, users of the mentioned therapies and in an age range from 20 to 65 years. None of the patients had participated in prior interviews. All of the patients received and signed and informed consent.

The workshops were divided into 4 work groups, one for each HRT. Each group was made up by users of the corresponding HRT and by specialists. When selecting PROs two different workshops were needed. The first one to pre-select possible items from a wide range of scales. The second one aimed to make a final selection according to the criteria of the Catalonia tender: patient empowerment and self-care; support in changing patient habits; tracing of adherence, support in better patient monitoring and – finally – satisfaction with the HRT service. The selected item list had to be reduced to a final list of 15–35 items that could be used in a manageable questionnaire.

Two activities were carried out in each group: (a) discussions were held about each of the items in the HRT questionnaire, about their understanding and relevance; (b) It was evaluated whether the item actually belonged to the dimension to be measured (referring to the starting dimensions of the HRT tender in Catalonia). Cognitive validation of questionnaires.

With the items selected after the workshops, 8 preliminary questionnaires were made (4 questionnaires for gathering PROs and 4 questionnaires for PREs). Online questionnaires were created to validate the items making up each of the 8 preliminary questionnaires. In these validation questionnaires, those surveyed could state whether each of the items was relevant and make suggestions on the language, the terms, the approach and the scales. Questionnaires were furnished both to 21 patients and 17 healthcare professionals in Madrid and Bilbao. The patient sample was selected through “Punto Inspira” and patient associations. The research team checked the results of the questionnaires and removed the items that were validated by less than 60% of the sample. Statements were changed taking into consideration some of the suggestions made by respondents. After the second validation, the questionnaires were modified with the list of PREs and PROs resulting from the whole process.

The objectives of the psychometric analysis with exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) performed for the PROMs and PREMs questionnaires were:

• -

  Validity analysis through constructs of the scales.

• -

  Analysis of the internal consistency of the scales through Cronbach's Alpha, at a global level and factors.

• -

  Reliability analysis of the questionnaires through the CRI (Composite Reliability Index).

• -

  Know the global adjustment of the questionnaire following the criteria of Kline.16

• -

  Descriptive analysis of patient patients, in relation to their sociodemographic characteristics.

The number of questionnaires answered was 1299 responded by 650 patients (n=650). Table 1 shows the number of patients who responded to each of the questionnaires. Furthermore, Table 1 shows the sociodemographic characteristics of the sample. In order to find representativeness, the questionnaires were passed in different regions of Spain. Table 2 shows the origin of the sample.

Statistical analysis was performed as follows. Descriptive statistics (mean, standard deviation and frequencies) were calculated for the sociodemographic variables. An exploratory factor analysis was performed to assess whether any of the items was redundant. Each item was correlated with the main factor. Items were eliminated if they scored less than 0.3 (correlation with the main factor), were completed by<95% of respondents, or if they had more than 80% support in a single response category.16 The remaining items were ranked according to item-total correlations and then, starting with the lowest rank, items were removed if they were highly correlated (>0.75) with another question.16 Cronbach's alpha was applied later. Internal consistency was assessed using Cronbach's alpha statistic and item-total correlations.16,17

Content validity was assessed by comparing the items in the reduced instrument of the final item with the conceptual model. This was completed through factor analysis, which tested the hypothesis that all variables would be loaded into one factor.16 Construct validity was assessed to see if the questionnaires correlated with measures such as quality of life or subjective perception of compliance using Spearman's rank correlation coefficient.

Reliability was calculated for the CFA with the CRI (it provides a theoretical estimate of the correlation between the actual scores from a psychometric test and the assumed true scores). In addition, following the criteria of Kline,16 the global fit was calculated through Chi-square to assess the general fit and the discrepancy between the sample and the adjusted covariance matrices, the comparative fit index (CFI), the root Mean Square Error of Approximation (RMSEA) and the Standardized Root Mean Residual (SRMR). SPSS V22 and MPLUS 8.2 were used.

During the research no papers were found using specific HRT instruments; what was found was a list of PROMs and PREMs used in studies on respiratory diseases. A total of 51 specific PROM scales and 6 PREM scales used in studies on respiratory diseases treated with HRT were found. None of the scales that were found covered all the domains required by the project (Table 2), especially the domains of empowerment (D1) and satisfaction with services (D5), being domains that are better suited to PREM scales.

Characteristics of the patient sample: the workshops featured a sample of 21 patients from Madrid and Bilbao--not the same sample used for the interviews. The age of the patients ranged from 20 to 85. Characteristics of the sample of specialists: at the workshops there were a total 7 pulmonologists and nurses from different hospitals in Madrid (La Princesa, Ramón y Cajal and Gregorio Marañón) and 10 lung specialists from hospitals in Bilbao (Basurto, Cruces and Galdakao). After the Participatory Healthcare Workshops for the selection and the first validation for the 4 PRO questionnaires and the 4 PRE questionnaires, the following was obtained: 17 PROs and 17 PREs for oxygen therapy; 31 PROs and 17 PREs for aerosol therapy; 18 PROs and 17 PREs for CPAP, and 14 PROs and 17 PREs for mechanical ventilation.

Characteristics of the patient sample: the workshops featured a sample of 8 patients (4 men and 4 women) from Madrid and Bilbao (not the same sample used for the interviews). Ages ranged from 20 to 65. Characteristics of the sample of specialists: the workshops featured 20 pulmonologists from the Hospital of Basurto, the University Hospital of Cruces and the University Hospital of Galdakao in the Basque Country, and from Hospital La Princesa, Hospital Ramón y Cajal and Hospital Gregorio Marañón in Madrid. This validation led to the second and current version of the questionnaires on PROs, structured as follows: the questionnaire on mechanical ventilation comprises 14 items, the questionnaire on oxygen therapy has 15 items, the questionnaire on CPAP has 15 items, and the questionnaire on aerosol therapy comprises 28 items. The validation questionnaires on PREs were furnished to 10 pulmonologists and 8 patients. This validation led to the second and current list of PREs, structured as follows: 16 items for oxygen therapy, 17 items for aerosol therapy, 16 items for CPAP, and 16 items for mechanical ventilation.

Nevertheless, the research team considered keeping some of the items in some of the questionnaires because literature shows the importance of the contents they refer to. In other words, the items were kept in order to ensure coherence with the theoretical model. Once the psychometric validation has been done it will be possible to find out if these items can be empirically maintained. The items that are to be kept concern the questionnaires PRE of noninvasive ventilation (17), oxygen therapy (17), CPAP (17), and the PROs of oxygen therapy (16) and aerosol therapy (28).

In Tables 3 and 4 the results of the statistical analysis are shown. In general, all questionnaires presented a good fit, both in the EFA and in the CFA, with the exception of the PREM questionnaire for mechanical ventilation, where Cronbach's alpha was low, although the HR was very good. Taking into account that the CRI is a more robust statistic related to reliability, it was taken as a reliability criterion. The fit, following the Kline criteria, was good in all the questionnaires.