Construct validity and reliability of the BARRIERS scale in the Spanish context

Abstract

Objective

To establish the construct validity of the Spanish version of the BARRIERS scale.

Method

Methodological study of validation of a measurement instrument based on data from previously published studies. The study population consisted of nurses from the Basque Health Service and the Canary Health Service. The following variables were extracted and unified: Years of professional experience, possession of a specialist nursing degree, possession of a doctorate, type of activity performed by the professional and field of work. For construct validation, a confirmatory factor analysis (CFA) was performed based on the initial model proposed for the scale and RASCH analysis. A polychoric correlation matrix, factor extraction by unweighted least squares and PROMIN oblique rotation were used. For the RASCH analysis, the Joint Maximun Likelihood estimation (JMLE) method was used; the fit of the items and persons were estimated by means of outfit - Unweighted Mean Square fit statistic (UMS) and infit -Weighted Mean Square Fit Statistic (WMS), as well as the reliability and separation of items and persons.

Results

A total of 1200 nurses and midwives made up the final validation sample (n = 1200), with a mean professional experience of 21.22 ± 9.26 years. The CFA presented a good fit to the data (KMO = 0.935 [95% CI: 0.921-0.945]), changing the factorial assignment in 6 items, while 5 items received factorial scores in more than one factor. The fit values for the 4-factor solution were RMSEA = 0.026 [95% CI: 0.026-0.027] and GFI = 0.991 [95% CI: 0.986-0.991]. In the RASCH analysis most items presented infit-WMS and outfit-UMS values with a good fit.

Conclusions

The Spanish version of the BARRIERS scale has adequate construct validity although there are changes in the assignment of items to the dimensions compared to the original model. The RASCH analysis indicates adequate fit for both persons and items.

Resumen

Objetivo

Establecer la validez de constructo de la versión española de la escala BARRIERS.

Método

Estudio metodológico de validación de un instrumento de medida a partir de los datos de estudios previamente publicados. La población a estudio estuvo conformada por enfermeras del Servicio Vasco de Salud y del Servicio Canario de la Salud. Se extrajeron y unificaron las variables: Años de experiencia profesional, tenencia de la titulación de enfermera especialista, posesión del doctorado, tipo de actividad que desempeñaba el profesional y ámbito de trabajo. Para la validación de constructo se realizó un análisis factorial confirmatorio (AFC) en base al modelo inicial propuesto para la escala y análisis RASCH. Se usó una matriz de correlaciones de tipo policórico, extracción de factores por mínimos cuadrados no ponderados y rotación oblicua PROMIN. Para el análisis RASCH se usó el método de Joint Maximun Likelihood estimation (JMLE); el ajuste de los Ítems y las personas se estimaron mediante outfit - Unweighted Mean Square fit statistic (UMS) e infit -Weighted Mean Square Fit Statistic (WMS), así como la fiabilidad y separación de ítems y personas.

Resultados

Un total de 1.200 enfermeras y matronas conformaron la muestra final de validación (n = 1200), con una media de experiencia profesional de 21,22 ± 9,26 años. El AFC presentó buena adecuación de los datos (KMO = 0,935 [IC 95%: 0,921-0,945]), modificando la asignación factorial en 6 ítems, mientras que 5 ítems recibieron cargas factoriales en más de un factor. Los valores de ajuste para la solución de 4 factores fueron RMSEA = 0,026 [IC 95%: 0,026-0,027] y GFI = 0,991 [IC 95%: 0,986-0,991]. En el análisis RASCH la mayoría de ítems presentaron valores infit-WMS y outfit-UMS con un buen ajuste.

Conclusiones

La versión española de la escala BARRIERS posee una adecuada validez de constructo, aunque existen cambios en la asignación de ítems a las dimensiones en comparación al modelo original. El análisis RASCH indica adecuado ajuste tanto para las personas como para los ítems.

What is known

• •

  The BARRIERS scale is the one most widely used to assess the barriers nurses perceive regarding research.

• •

  Validity of the Spanish version of BARRIERS has not yet been produced.

What it contributes

• •

  BARRIERS may be applied as a valid and reliable instrument with an internal structure with subscales adjusted to the original model.

• •

  BARRIERS measures a well-defined latent variable leading to it equally being considered as an essentially unidimensional tool.

Introduction

Research is an essential tool for training nurses in making informed and contextualised decisions, leading to the development of an evidence-based practice that becomes the “gold standard” for people’s healthcare through scientific methods, resulting in the quality of care and the general health of society.^1–3 However, a gap exists between knowledge achieved through research and the clinical application of results, which are perceived by nurses as barriers to research.⁴ These barriers have been mainly linked to organisational culture⁵ and the organisational and leadership characteristics of the institutions. To a lesser extent, they have been linked to the individual characteristics of the nurses and their ability to present and disseminate the results of their research.^3,4

Although there are several instruments to evaluate the perceptions that nurses have about these barriers, the most used and widespread instrument is the BARRIERS scale, developed by Funk et al,⁶ the original version of which comprises twenty-nine items. Each item reflects the degree to which they said element is perceived as an obstacle through a scale ranging from 1 to 4 points (1: Never; 2: To a lesser degree; 3: To a moderate degree; 4: To a large extent); In addition, a non-scoring response option is included to describe the absence of opinion. The determination of the psychometric properties of the BARRIERS scale construct has been carried out using the classic principal component analysis procedure on a sample (n = 1948) divided into two identical groups.⁶ Thus, the items of the BARRIERS scale have been structured into four dimensions with satisfactory internal consistency: Characteristics of the nurse (α = .80), Characteristics of the organisation (α = .80), Characteristics of the quality of the research (α = .72) and Characteristics of the accessibility and communication of research results (α = .65). In the original study by Funk et al., one of the items (n^o. 27 “The amount of research information is overwhelming”) did not obtain satisfactory factor loadings in any of the previous dimensions, so the authors decided to show it in the original instrument. Independently.⁶ To obtain the global score of the scale, the scores of all the items are added. To obtain the dimension scores, the scores of the items that make up each dimension are added. The maximum score of the BARRIERS scale is 116 points, the sum of the items in each dimension corresponding to 32 for the first 2 and 24 for the last 2.

The BARRIERS scale has been adapted and applied in many different countries^4,7 including Sweden^8,9 Australia,^10,11 Finland,¹² Greece,¹³ Ireland,¹⁴ USA,¹⁵ Korea,¹⁶ Turkey,¹⁷ China,¹⁸ Poland,¹⁹ Saudi Arabia²⁰ and Pakistan.²¹ It is currently considered the most accepted scale worldwide to measure the barriers associated with research and EBP in nursing.^4,7 In Spain, Moreno-Casbas et al.^22,23 carried out the cross-cultural adaptation of the BARRIERS scale with adequate reliability (α = .842), so that, to date, it has been used on several occasions.^24–28 However, the analysis of the construct validity of the Spanish version of the scale has not yet been undertaken. Given that there are differences between the various validation studies regarding the dimensionality and structure of the instrument, the objective of this study was to establish the construct validity of the Spanish version of the BARRIERS scale.

Method

Study design

Methodological validation study of a measurement instrument, carried out using data from the studies published by Cidoncha-Moreno et al. (study 1)²⁷ and González-de la Torre et al. (study 2).²⁸

Study population

The study population was made up of the nurses participating in study 1 (nurses from the Osakidetza-Basque Health Service) and in study 2 (nurses and midwives from the Canary Islands Health Service). Both studies had considered serving in these organisations as the only inclusion criterion and did not establish any exclusion criteria.

Sample size calculation and sampling method

Study 1 carried out simple random probabilistic sampling by strata²⁷ while study 2 used non-probabilistic convenience sampling.²⁸

For the present validation study, a minimum necessary sample size of 400 nurses was considered to carry out a factor analysis (FA). This calculation was based on the classic recommendation that establishes using at least 10 subjects for each item that the instrument to be validated has, a minimum of 200 subjects if a polychoric type matrix is used and the possible need to carry out a cross-validation analysis (where the sample is divided into two subsamples to explore the stability of the results obtained).²⁹

Instruments and data collection system

In both studies online data collection systems were used, through the company email of the participants (study 1) or the intranet of the participating centres (study 2). The data collection periods in both cases may be consulted in the reference studies.^27,28

In the Cidoncha-Moreno et al.²⁷ and González-de la Torre et al.²⁸ studies, the Spanish version of the BARRIERS scale had been used, which consists of 29 items that score on a Likert-type scale from 1 to 4 points. There is a fifth response (Does not know/No comment) that scores 0 points. According to this model, the maximum score for the scale is 116 points. In these two studies, the item “The amount of research information is overwhelming was assigned to the dimension “Characteristics of research communication (presentation and access)” for analysis.

Study variables

Both studies had collected several socio-demographic, occupational and specific variables related to the area of research, although some had not been measured in the same way or had not been collected in the two studies. After meetings and discussions with the research team it was decided that variables could be considered to have a similar operational definition and could be combined and analysed together. These variables were: years of professional experience; specialist nurse qualification in any of its modes (Yes/No); possession of a doctorate (Yes/No); type of activity practised by the professional (carer, manager, teacher, researcher, and area of work (Primary care, specialised-hospital care setting, and other areas). The other areas may be consulted with the rest of the variables originally collected in the studies of reference.^27,28

Statistical analysis

For the descriptive and inferential analysis of the variables studies the IBM© SPSS Statistics v.24.0. software was used. The qualitative variables were expressed in percentages and frequencies and the quantitative variables in means, standard deviation (SD) and minimum-maximum values. An analysis for assessing whether there were differences between the two groups comprising the sample was performed (nurses from the Osakidetza-Basque Health Service and nurses and midwives from the Canary Islands Health Service). To do this they used the Chi squared test or the Fisher statistic, depending on the cases, for the qualitative variables and the student’s t-test for the quantitative variables. A p ≤ .05 value was established as being statistically significant.

FA and model reliability were performed using the freely available software FACTOR© Release Version 12.02.01 × 64 bits³⁰ and JASP Version 0.17.2.1. Computer software.

Confirmatory factor analysis (CFA) was performed based on the initial model proposed by the BARRIERS scale. Preliminary detection of the inappropriate items was carried out, according to Gulliksen’s pool, from the values obtained from the Overall Item Threshold (OIT), Overall Item Slope (OIS) and Measure of Sampling Adequacy (MSA).³¹ The MSA values under .500 suggested that the item did not measure the same construct as the other items of the group, and its removal was therefore recommended.^31,32

In the semi-specified CFA matrix, the item “The amount of research information is overwhelming” was assigned to factor 4, according to the model used for studies Nº1 and Nº2. FA data adequacy was assessed with the Kaiser Meyer Olkin (KMO) index and the Barlett statistic, with values above .75 being considered adequate for the first and values of p ≤ .05 being statistically significant for the second.^29,30

To assess the adequacy of the factor solution the Root Mean Square of Residuals (RMSR), Root Mean Square Error of Approximation (RMSEA), Non-Normed Fit Index (NNFI), Comparative Fit Index (CFI), Goodness of Fit Index (GFI) and Adjusted Goodness of Fit Index (AGFI) were used.²⁹ RMSR reference values under .05 were considered a good fit whilst values between .05–.08 were considered a reasonable fit. NNFI and CFI values of .95 or above and values of GFI and AGFI above .90 were considered indicators of a good model fit.

The Root Mean Square of Residuals (RMSR) was also calculated to assess the fit of the factor solution based on the magnitude of the residual correlations.²⁹ Kelly’s criterion was used for this, where the RMSR value is compared with the standard error which would have a 0 correlation in the population.³³

According to the analysis of symmetry and kurtosis of the items, a polychoric, correlation matrix was used for the FA.^29,30 Factor extractions was performed by unweighted least square rules (RULS) and PROMIN oblique rotation. Using bootstrapping, the 95% confidence intervals of the scores of the items and the model measures were calculated. Values higher than .300 were considered factor sufficient in the factor loading.³⁴

Factor constancy was assessed with the ORION (Overall Reliability of fully-Informative prior Oblique N-EAP scores) coefficient and the Factor Determinacy Index (FDI).³⁵ If factor scores were to be used for individual assessment, FDI values above .90 and ORION scores above .80 are recommended.³⁵

To assess unidimensionality the Unidimensional Congruence (UniCo), Explained Common Variance (ECV) and the Mean of Item REsidual Absolute Loadings (MIREAL)were used.³ UniCO values above 95, ECV values above .85 and below .30 for the MIREAL were considered to indicate that the data could be considered as essentially unidimensional.³⁶ Based on this analysis, the convenience to perform an analysis from the RASCH focus was established.

RASCH analysis using J Metrik© Software was performed. The Joint Maximun Likelihood estimation (JMLE) method was used to assess the parameters. Item fit and people was calculated using the outfit - Unweighted Mean Square fit statistic (UMS) and infit -Weighted Mean Square Fit Statistic (WMS). For fit indexes values between .8 and 1.2 were considered a good fit and between .5 and 1.5 an acceptable fit.³⁷ To establish quality statistics the separation indexes were estimated (considering adequate values above 2) and reliability (desirable reliability values above .8), both for the items and for people.³⁷ Suposition of local independence between the items was tested using the Yen Q3 test.³⁸

In addition, a reliability analysis was performed with the Cronbach's alpha coefficient (α), the Omega coefficient (ω) and the Greatest Lower Bound (GLB).³⁹ Values above .80 were considered adequate for the three coefficients.

Ethical aspects

Both reference studies had authorisation from their respective ethics committees (Hospital Universitario de Gran Canaria Dr. Negrín [HUGCDN] Code CEIm: 2022-358-1, Hospital Universitario de Canarias Code CEIm: EQS Project and Code CEIC-E: 27-09-2011), the participates having been informed of voluntary participation and guaranteeing anonymity. The unified data base that was generated from the data of the two studies was blinded and will be kept in accordance with the current legislation by the research team.

Results

A total of 1,200 nurses and midwives made up the final validation sample (n = 1200), 688 from study Nº1 (57.3%) and 512 from study Nº2 (42.7%), with a professional experience mean of 21.22 years (SD = 9.26) [range 0–43]. A total of 323 participants (26.9%) had at least one nursing speciality and 13 had doctorates (1.1%).

Regarding area of work, 479 (39.9%) were occupied in primary care, 711 (59.3%) in specialised hospital care and 10 (.8%) in other services. Regarding activity, 1006 (83.8%) were involved in care, 168 (14.0%) in management activities and 19 (1.6%) in teaching. Only 7 (.6%) were exclusively dedicated to research. The frequencies and percentages may be consulted for the originally collected in the reference studies.

Analysis between the two groups revealed statistically significant differences in the variables: Having a speciality (Fisher/statistics p ≤ .001), main activity (X²/ p = .001), having a doctorate (Fisher/statistic p ≤ .001) and work environment (X²/ p ≤ .001). The mean of professional experience in Basque nurses was 23.77 (DE = 8.57) years, compared with the mean of the nurses from the Canary Islands which was 17.85 (DE = 9.06), with this difference of means being statistically significant (p ≤ .001).

Mean scores, SD; symmetry-kurtosis and minimum-maximum scores of the items may be consulted in Table 1.

Table 1.

Mean scores, standard deviations, symmetry-kurtosis values, and ceiling and floor scores.

	M(SD)	Asymmetry	Kurtosis	n (%)Ceiling response*	n(%)Floor response**
1-Research reports/articles are not readily available	2.25(1.09)	−.239	−.330	169(14.1%)	98(8.2%)
2-Implications for practice are not made clear	2.41(1.17)	−.368	−.478	251(20.9%)	104(8.7%)
3-The statistical analyses are not understandable	2.56(1.13)	−.359	−.602	307(25.6%)	60(5.0%)
4-The research is not relevant to the nurse’s practice	1.77(1.09)	.907	−.411	138(11.5%)	28(2.3%)
5-The nurse is unaware of the research	2.85(1.12)	−.526	−.723	471(39.3%)	29(2.4%)
6. The facilities are inadequate for implementation	2.49(1.24)	−.410	−.741	326(27.2%)	106(8.8%)
7-The nurse does not have the time to read the research	2.81(1.11)	−.463	−.893	435(36.3%)	19 (1.6%)
8-The research has not been contrasted	1.54(1.13)	.345	−.655	65(5.4%)	249(20.8%)
9-The nurse feels the benefit of changing practice will be minimal	2.05(1.13)	.149	−.729	157(13.1%)	88(7.3%)
10-The nurse is uncertain whether to believe the result of the research	2.20(1.11)	−.035	−.664	173(14.4%)	76(6.3%)
11-The research has methodological inadequacies	1.71(1.23)	.108	−.919	100(8.3%)	270(22.5%)
12-The relevant literature is not compiled in one place	2.45(1.44)	−.493	−1.048	397(33.1%)	205(17.1%)
13-The nurse does not feel they have enough authority to change patient care procedures	2.56(1.19)	−.240	−1.132	354(29.5%)	34(2.8%)
14-The nurse feels results are not generalisable to their own setting	2.34(1.13)	−.193	−.741	212(17.7%)	68(5.7%)
15-The nurse is isolated from colleagues with whom to discuss the research results	2.48(1.19)	−.240	−.913	319(26.6%)	63(5.3%)
16-The nurse does not see the value of research for practice	2.25(1.12)	.141	−1.019	215(17.9%)	34(2.8%)
17-The research reports/articles are not published fast enough	1.88(1.38)	−.085	−1.193	164(13.7%)	322(26.8%)
18-Physicians will not cooperate with implementation	2.45(1.38)	−.500	−.917	366(30.5%)	183(15.3%)
19-Administration Will not allow implementation	2.21(1.38)	−.292	−1.078	269(22.4%)	219(18.3%)
20-The nurse does not see the results of research relevant for practice	2.20(1.13)	.008	−.852	184(15.3%)	68(5.7%)
21-There is not a documented need to change practice	1.97(1.19)	.133	−.883	151(12.6%)	124(10.3%)
22-The proposed conclusions are not duly justified	1.68(1.14)	.093	−.908	57(4.8%)	224(18.7%)
23-The literature reports conflicting results	1.78(1.20)	.039	−.830	101(8.4%)	237(19.8%)
24-The research is not reported clearly and readably	2.29(1.20)	−.167	−.828	244(20.3%)	101(8.4%)
25-Other staff are not supportive of implementation	2.26(1.21)	−.286	−.665	216(18.0%)	140(11.7%)
26-The nurse is not willing to change/try the new ideas implemented	2.39(1.14)	−.077	−.904	257(21.4%)	47(3.9%)
27-The amount of research is overwhelming	2.09(1.27)	−.070	−.944	210(17.5%)	166(13.8%)
28-The nurse does not feel capable of evaluating the quality of research	2.46(1.27)	−.358	−.890	65(5.4%)	249(20.8%)
29-There is insufficient time on the job to implement new ideas	2.97(1.11)	−.762	−.356	157(13.1%)	88(7.3%)

M(SD) = Mean (Standard deviation).

n(%) = Frequency (% percentage).

*

Only frequencies and percentage that correspond to the response To a great extent, (score 4 point) are expressed.

**

Only the frequencies and percentage that correspond to the response No comment (score 0 points) are expressed.

Validation of the construct using Confirmatory Factor Analysis

Preliminary detection of inappropriate items according to Gulliksen’s pool did not find any items susceptible to removal. According to the OIT and OIS values obtained. MSA values were above .850 in all items. (supplementary material 1).

Initial CFA presented a good fit with data, with a KMO = .935 (95% CI: .921–.945) and a Bartlett statistical value that was significant (p < .001). The 4 factor dimension solution came from explained variance of 59.12%, in keeping with the parallel analysis. Adjustment values for this model were RMSEA = .026 [95% CI: .026–.027], NNFI = .995 [95% CI: .993–.995], CFI = .996 [95% CI: .995–.996], GFI = .991 [95% CI: .986–.991] and AGFI = .988 [95% CI: .981–.988], indicating a good fit of the model. RMSR was .0375 [95% CI: .038–.038] (the expected value of RMSR according to the Kelley criterion for the acceptable model in this case was .0289).

Table 2 shows the factor loadings (after rotation) of the model with its respective confidence intervals. Depending on the factor loadings obtained, the CFA made changes in the factor assignment of items 7, 10, 12, 13, 14 and 27. Furthermore, five items received factor loadings in more than one factor (items 5, 6, 15, 25 and 29).

Table 2.

Factor loadings obtained after rotation with their respective confidence intervals.

	Factor 1	Factor 2	Factor 3	Factor 4
Item 1	−.377(−0.572 −.266)	.093(−.056 .226)	.108(.012 .215)	.707(.604 .903)
Item 2	−.313 (−.502 −.194)	.051(−.122 .186)	.100(.002 .195)	.840(.730 .974)
Item 3	−.149 (−.317 −.050)	−.104(−.240 .029)	.099(.013 .191)	.831 (.715 .943)
Item 4	.113(−.063 .267)	−.198(.378 .007)	.086(−.012 .205)	.380(.228 .538)
Item 5	.351(.231−.453)	−.078(−.207 .032)	−.213(−.307 −.139)	.476(.365 .589)
Item 6	−.169 (−.314 −.026)	.340(.160 .508)	−.054(−.174 .052)	.404(.257 .575)
Item 7	−.132(−.270 −.013)	.259 (.127 .396)	−.208(−.383 −.203)	.637(.514 .746)
Item 8	−.053 (−.165 .052)	−.025 (−.151 .092)	.676(.596 .768)	.149(.028 .304)
Item 9	.703 (.585 .834)	−.123(−.303 .003)	.144(.061 .248)	.011(−.124 .136)
Item 10	.710 (.601 .821)	−.177(−.31 −.006)	.127(.056 .217)	.068(−.043 .197)
Item 11	.058 (−.051−.185)	−.032(−.184 .109)	.676(.580 .762)	.021(−.104 .136)
Item 12	−.140(−.308 −.025)	.144(−.024 .276)	.422(.317 .507)	.183(.067 .338)
Item 13	.450(.300 .571)	.076(−.086 .219)	−.142(−.241 −.043)	.281(.149 .429)
Item 14	.627(.507 .765)	.015(−.139 .145)	−.036(−.143 .045)	.147(.013 .279)
Item 15	.394(.270 .522)	.352(.213−.486)	−.204(−.302 −.123)	.126(−.011 .257)
Item16	.862(.771 .967)	−.070(−.171 .046)	−.067(−.143 .005)	−.037(−.146 .062)
Item17	−.032(−.147 .098)	.189(.036 .354)	.569(.486 .666)	−.024(−.146 .108)
Item18	.058(−.040 .151)	.882(.778 1,067)	.065(−.031 .135)	−.239(−.360 −.156)
Item19	−.132(−.247 −.054)	.984(.894 1,124)	.136(.075 .190)	−.245(−.337 −.179)
Item20	.811(.706 .930)	−.069(−.204 .066)	.079(−.012 .161)	−.052(−.171 .046)
Item21	.371(.238 .508)	.075(−.070 .226)	.253(.157 .341)	.050(−.061 .184)
Item22	.122(−.001 .228)	.043(−.113 .155)	.578(.500 .656)	.104(−.019 .217)
Item23	−.041(−.155 .077)	−.019(−.179 .135)	.610(.502 .709)	.222(.081 .342)
Item24	.050(−.094 .162)	−.037(−.190 .120)	.174(.071 .270)	.550(.413 .684)
Item25	.543(.428 .667)	.302(.168 .443)	.152(.063 .231)	−.183(−.316 −.068)
Item26	.883(.756 1,014)	.063(−.121 .191)	−.078(−.163 .007)	−.239(−.395 −.109)
Item27	−.079(−.213 .073)	.013(−.143 .181)	.388(.285 .484)	.272(.132 .417)
Item28	.394(.252 .557)	−.017(−.242 .173)	.114(−.002 .228)	.241(.053 .377)
Item29	.007(−.124 .150)	.375(.233−.529)	−.218(−.320 −.130)	.440(.302 .568)

Item 5-The nurse is unaware of the research received loadings in factor 1 (that correspond to the dimension Researcher characteristics) of .351 and in factor 4 (Characteristics of the research communication) of .476. Due to this it was considered more appropriate to assign it to factor 4.

Item 6-The facilities are inadequate for implementation, received loadings for factor 2 (.340) and factor 4 (.404), with fit being considered adequate maintaining its assignment to factor 2 (Characteristics of the organisation), according to the original model. It was also considered that item 29-There is insufficient time on the job to implement new ideas should be kept in factor 2 (.375), despite the fact it received greater factor 4 loading (.440).

Item 15-The nurse is isolated from knowledgeable colleagues with whom to discuss research results received loadings for factor 1 and 2, being higher for factor 1, and the assignment was therefore maintained in accordance with the original model.

Lastly tem 25-Other staff are not supportive of implementation, was initially assigned to factor 2 in the original model, but with an insufficient factor loading in this factor (.302), receiving higher factor 1 loadings (.543), which led to its factor change.

The final proposed model was therefore composed of four factors-dimensions: factor 1, “Characteristics of the researcher (the nurse)”, comprising items 9, 10, 13, 14, 15, 16, 20, 21, 25, 26 and 28; factor 2, “Characteristics of the organisation”, comprising items 6, 18, 19 y 29; factor 3 “Characteristics of research (Quality)”, comprising items 8, 11, 12, 17, 22, 23 and 27 and finally factor 4 “Characteristics of research communication” to which items 1, 2, 3, 4, 5, 7 and 24 were assigned. In this model all the items received loadings above .300. The ORION and FDI for the factors may be consulted in Table 3.

Table 3.

Reliability of the factors according to the Factor Determinacy Index and the Overall Reliability of fully-Informative prior Oblique N-EAP scores.

Factors	FDI*(95% CI)	ORION** (95% CI)
Factor 1	.960(.955–.963)	.921(.911–.928)
Factor 2	.942(.927–.957)	.887(.860–.917)
Factor 3	.924(.914–.930)	.854(.836–.866)
Factor 4	.924(.914–.930)	.889(.875–.901)

If the factor scores are to be used for individual assessment, FDI values above .90 and ORION values above .80 are recommended.

*

Factor Determinacy Index.

**

Overall Reliability of fully-Informative prior Oblique N-EAP scores.

Furthermore, unidimensionality analysis obtained the following results: UniCo = .957 [95% CI: .947–.968], ECV = .832 [95% CI: .816–.855], MIREAL = .232 [95% CI: .140–.232]). Since two of these indexes (UniCo and MIREAL) indicate possible assumption of a unidimensional model, a RASCH analysis was performed.

RASCH analysis

A RASCH analysis based on the theory of item response was made. In Table 4 model fit obtained data may be consulted.

Table 4.

RASCH analysis of the BARRIERS scale.

Item	Difficulty index*	Infit-WMS**	Outfit-UMS**
Item1	.09	1.14	1.25
Item2	−.04	.99	.99
Item3	−.30	.97	.96
Item4	−.10	1.34	1.79
Item5	−.74	1.10	1.14
Item6	−.12	1.20	1.21
Item7	−.89	1.16	1.19
Item8	.80	.96	.93
Item9	.14	.86	.87
Item10	.03	.87	.86
Item11	.66	1.03	1.05
Item12	.03	1.25	1.34
Item13	−.51	.95	.98
Item14	−.10	.83	.82
Item15	−.26	.96	.94
Item16	−.30	.90	.94
Item17	.53	1.13	1.14
Item18	.02	1.00	1.03
Item19	.21	1.06	1.09
Item20	−.02	.82	.82
Item21	.27	.90	.88
Item22	.76	.84	.83
Item23	.60	.95	.95
Item24	−.00	.92	.91
Item25	.13	.83	.82
Item26	−.28	.98	.99
Item27	.23	1.17	1.21
Item28	−.11	.91	.92
Item29	−.76	1.05	1.06

*

The difficulty index indicates the highest values with respect of the research barriers in this case.

**

Unweighted Mean Square fit statistic (UMS)- and infit - Weighted Mean Square FitStatistic (WMS): Values of the adjustment indexes between .8 and 1.2 signify a Good fit and values between .5 and 1.5 signify an acceptable fit.

All items presented infit-(WMS) values with good fit, except items 4 and 12 which presented an acceptable fit. Regarding the outfit-UMS values, items 1, 6, 12 and 27 presented an acceptable fit, whilst item 4 had a poor fit. The other items obtained good fit. Fig. 1 shows the items and people mapping. RASCH analysis enables the value of the latent (theta) variable to be calculated, i.e. Research barriers in nursing, in relation to the total score. Fig. 2 shows the adjustment curve of the latent variable.

Figure 1.

Map showing items of the BARRIERS scale.

Figure 2.

Fit curve of latent variable.

Regarding the quality statistics of the scale, the reliability values of the items and the people were .995 and .922 respectively, whilst the separation indexes were 19.207 and 4.929 respectively (above 2 points in both cases) which indicates adequate reliability for both the items and the people. In Yen’s Q3 test most of the values in the correlation matrix were below .2 so the assumption of local item independence was met.

Reliability

The total values for the scale of Cronbach’s alpha and Omega coefficient were α = .92 [95% CI: .91–.93] and ω = .92 [95% CI: .91–.93] respectively. Only the elimination of item 4 increased the value of reliability by one hundredth in the Cronbach’s alpha coefficient. Given the high sample size, the GLB = .96 [95% CI: .96–.96] was additionally calculated. Regarding the internal reliability of the subscales, Cronbach’s alpha coefficient values ranged between .68 [95% CI: .64–.72] for the subscale “Characteristics of the organisation” and .89 [95% CI: .88–.90] for the subscales “Characteristics of the researcher (the nurse).”. For these same subscales the Omega coefficients were .71 [95% CI: .68–.74] and .89 [95% CI: .88–.90] respectively. The values of all the coefficients for the original model may be consulted in the Spanish version in Table 5.

Table 5.

Reliability values for the original model and the Spanish version of BARRIERS scale.

	Original model			Spanish version
	Cronbach’s Alpha coefficient (95% CI)	Omega coefficient (95% CI)	Greatest LowerBound (95% CI)	Cronbach’s Alpha coefficient (95% CI)	Omega coefficient (95% CI)	Greatest LowerBound (95% CI)
Factor1	.84(95% CI: .82–.85)	.84(95% CI: .82–.85)	.86(95% CI: .85–.88)	.89(95% CI: .88–.90)	.89(95% CI: .88–.90)	.92(95% CI: .92–.93)
Factor2	.79(95% CI: .77–.81)	.80(95% CI: .77–.82)	.87(95% CI: .85–.88)	.68(95% CI: .64–.72)	.71(95% CI: .68–.74)	.78(95% CI: .76–.81)
Factor3	.80(95% CI: .77–.82)	.80(95% CI: .78–.82)	.82(95% CI: .81–.84)	.80(95% CI: .78–.82)	.81(95% CI: .79–.82)	.84(95% CI: .82–.86)
Factor4	.71(95% CI: .67–.73)	.71(95% CI: .68–.74)	.77(95% CI: .74–.79)	.74(95% CI: .72–.77)	.75(95% CI: .72–.77)	.79(95% CI: .77–.82)

Discussion

The BARRIERS scale has been widely used in many studies and settings worldwide.^4,7 However, the practical use of the scale is highly uneven. For example, in several studies the response format was modified,^7,40 variations were made regarding how the items were formulated or the results were communicated incompletely, ignoring the response on the absence of opinión.⁷ Moreover, assessment of psychometric properties of the scale between the different studies was highly irregular,⁷ being applied in some countries without having passed a previous validation process.^13,16,41

In Spain, despite having been used in several research studies,^22–28 assessment of its psychometric properties has also been scarce. Although the values contributed up to date indicate an adequate reliability,^22,23,27,28 the internal structure of the Spanish version of the scale is pending study, the initially proposed model by Funck et al.⁶ being used in the different studies. With this study we have tried as far as possible to bridge this knowledge gap, attempting to improve the comprehension of the internal structure and behaviour of the scale, contributing data to enable its practical use in our environment.

There is also a disparity regarding the result from the different factor analyses with the BARRIERS, reporting factor solutions of 3 to 8 factors and differing to a greater or lesser extent from the initial model with regards to the item distribution.⁷ This is explained mainly based on the disparity of FA focal points and the differences between the study populations (both in their characteristics and their sample sizes). In the FA performed, a realistic non-lineal model was followed based on the sample size, the number of subscales-dimension considered, the symmetry and kurtosis analysis of the items, in keeping with the current recommendations for advanced FA.^29,30

The FA undertaken by Funck⁶ was based on the standard extraction method through principal components and varimax rotation,³⁴ outmoded in the present day. The majority of FAs carried out with the BARRIERS scale suffer from the same problem,^7,42 with the additional disadvantage that often the factor extraction system is not reported, or the adjustment indexes, being solely based on factor loadings for determining the composition of the factors-dimensions. This practically prevents any comparisons regarding FA to be made. One of the most recent studies, conducted by Ozga et al. with 349 Polish nurses reported RMSEA and CFI values, although little information on the type of FA undertaken was given.¹⁹ In its case, RMSEA values reported were of .066 [90% CI: .056–.076] on following the original model and of .075 [90% CI: .069–.080] in the model proposed for Poland, very far from those obtained in our study. It should be noted that the sample size used in our study is high (n = 1200). The FA is highly sensitive to the sample size, this parameter being an aspect that requires special attention when planning analysis strategies.^29,30,34 This has doubtless been able to penalise some of the FA made up until now with BARRIERS.

Sample size is something which at times has not been sufficiently considered in validation studies. In these studies with respect to the sample estimation, not only the standard recommendation based on the number of items of the tool to be validated should be considered, but also other parameters such as the variability existing in the sample, the presence of ambiguous items or the range of item responses.^29,30,34 For this reason no single unequivocal recommendation exists in this respect. Comrey and Lee, for example, propose several quality criteria regarding sample size based on the number of participants⁴³: 100 = poor, 200 = sufficient, 300 = good, 500 = very good and 1000 = excellent.

Based on this recommendation and that of Ferrando et al.,²⁹ who also used 200 as the minimum admissible sample size to obtain stable estimations in polychoric correlations in sample calculation we estimated the sample size in this study to be 400 subjects, which was widely surpassed. In this estimation the prevision of performing a cross-validation analysis was also taken into account (where the sample is divided into two subsamples to explore stability of results obtained). Although finally it was not necessary.²⁹ For subsample division our recommendation would be to use the Solomon method proposed by Lorenzo.^29,44

A key, but problematic, aspect when evaluating the psychometric properties of a measurement instrument is the evaluation of dimensionality.^45–47 Frequently, instruments initially designed as unidimensional are revealed to be multidimensional when their structure is explored and vice versa.⁴⁸ Although several systems have been proposed (based, for example, on the evaluation of Eigen values).⁴⁷ it is not clear which of the procedures is the most suitable for evaluating the dimensionality of an instrument.^45–47 It must be taken into account that, beyond the strictly statistical-mathematical criteria, the decision made by the researcher may be based on purely theoretical aspects of the construct.⁴⁵ In this specific case, the unidimensionality analysis carried out with the FACTOR software was inconclusive. This encouraged us, within the framework of a methodological study, to carry out a RASCH analysis, where several premises must be assumed, one of them being the unidimensionality of the instrument.^37,43 The RASCH model is a probabilistic model that states that a response is the result of an interaction between the person’s ability and the difficulty of the item.⁴⁹ If the data fit the model, the scale is defined as unidimensional.⁴⁹ The results obtained indicate a fit between good and acceptable for almost all items (except for item 4), which allows us to assume that there is a single latent variable that can be measured with the BARRIERS scale. This is consistent with what has been called in the literature as “Essential unidimensionality”^49–51 by pointing out that, in reality, there is only one predominant latent variable (in this case Barriers to research in nursing). To our knowledge, this is the first RASCH approach analysis performed with the BARRIERS scale in the world.

Another premise of the RASCH model is the local independence of the items and, generally, this is an aspect rarely reported in studies.⁴⁹ In this study, values were met, for the most part, below .2.³⁸

Regarding the specific analysis of each item, apart from item 4, we found no other item problematic. Item 27, a problematic item that in many studies has not been assigned to any dimension,^6,7 had discreet but sufficient behaviour, with factor loadings greater than .350 and an acceptable infit and outfit adjustment.

It must be understood that the BARRIERS scale was developed more than 30 years ago. This may have especially affected the functioning of the items that refer to the dimension “Characteristics of research communication (presentation and accessibility)”. While it is true that the publication of scientific literature has experienced exponential growth in recent years (and the trend continues to rise), it is also true that the development of the Internet and other new technologies has provided improvement in access to scientific documentation. To ensure the proper management of the enormous amount of information, it is necessary to promote the training of professionals⁵²; Perhaps this aspect is insufficiently represented in the BARRIERS scale, since the lack of training is undoubtedly a major barrier for nursing research.^4,5,53

One aspect related to the antiquity of this scale is linguistic. The Spanish version mentions “nursing” and “the nursing staff”, when in reality today it is accepted that the correct term is “nurses”. Although this terminology is still used, it would be advisable to adapt the formulation of the BARRIERS items to a more current format, replacing these generic terms with “nurses”.

The results obtained in the different reliability indices calculated support a high total reliability (internal consistency) of the Spanish version of the scale. Most previous studies with the BARRIERS have based the evaluation of internal consistency simply on the calculation of the Cronbach coefficient.⁷

Despite its wide use in health sciences, it has been reported that the use of Cronbach’s alpha coefficient is linked to several myths or misconceptions.^54,55 As Cho and Kim explained, one of them is that it is the best choice with respect to the published reliability coefficients.⁵⁶ Regarding the Omega coefficient, many authors defend that its use is clearly preferable to Cronbach’s.^39,55,57,58 Trizano-Hermosilla and Alvarado propose assessing the distribution and symmetry of the test scores to decide which reliability coefficient to use.³⁹ Thus, according to these authors, when the total test scores are normally distributed (all items), the Omega coefficient should be the first option, followed by Cronbach, since they avoid the overestimation problems presented by, for example, the Greatest Lower Bound (GLB) especially when the sample size is small-moderate⁵⁹). Given the sample size achieved in this study, we believe that there is no problem of overestimation. GLB is recommended when the proportion of asymmetric items is high, since in these conditions the use of Cronbach and Omega as reliability estimators is not advisable, regardless of the sample size.³⁹ Taking into account that we are dealing with a methodological study, and based on all these considerations, the three coefficients (Cronbach, Omega and GLB) have been calculated and reported. We trust that in future studies researchers will be encouraged to evaluate the reliability of the scale not only with Cronbach’s alpha coefficient.

Other aspects remain to be studied, such as exploring the temporary reliability-stability, logical validity or divergent-convergent validity of the BARRIERS scale.

The limitations of this study arise directly from its design. It is a study of a marked methodological nature that is based on data from two previous studies carried out in different time periods and environments and whose objective was not to validate a scale. There were statistically significant differences between the two groups, although we believe that these differences do not affect the interpretation of the construct, although this possibility cannot be ruled out.

Finally, in conclusion, we would highlight that the results obtained support the use of the BARRIERS scale for the evaluation of the perceptions that Spanish nurses have about the barriers in research. The Spanish version constitutes an instrument that has good construct validity and adequate internal consistency. Although it can be used as an instrument with four domains, it measures a well-defined latent variable and for practical purposes may therefore also be considered an essentially unidimensional instrument. The decision on use approach remains that of the researchers who use it, having to evaluate whether the use of the domains provides any additional information with practical value to their research and appreciating that the internal consistency values ​​of some of the domains are limited.

Funding

This research did not receive any specific funding from public sector agencies, the business sector or non-profit making entities.

Conflict of interests

The authors have no conflict of interests to declare with this manuscript.

Acknowledgements

Our gratitude to the nurses and midwives who participated in the studies which have enabled this research. Also to the Servicio Canario de la Salud and Osakidetza-Servicio Vasco de Salud managements. Our thanks to doctor María Teresa Moreno-Casbas for her involvement in the BARRIERS scale study in Spain.