Measuring work engagement among community health workers in Sierra Leone: Validating the Utrecht Work Engagement Scale

Abstract

This study examines the concept of volunteer work engagement in a sample of 334 community health workers in Bonthe District, Sierra Leone. Structural equation modelling was used to validate both the 9-item and the 17-item Utrecht Work Engagement Scale (UWES-9 and UWES-17, respectively). Results assessing the UWES-17 invalidated the three-factor structure within this cohort of community health workers, as high correlations were found between latent factors. Findings for the validity of the UWES-9 were largely consistent with those of the UWES-17. Model fit for the UWES-9 were generally equivalent for the one-factor, three-factor, and bifactor solutions, however the three-factor model was once again rejected due to high factor correlations. Based on these results, the current sample provides evidence that work engagement is best represented as a unidimensional construct in this context. Findings are considered alongside previous research to offer support for the utilization of the shortened UWES-9 in this context, as it appears to provide a good representation of work engagement and possess a parsimonious unidimensional scoring scheme.

Resumen

Este estudio analiza el concepto de compromiso en el trabajo voluntario de una muestra de 224 trabajadores sanitarios comunitarios del distrito de Bonthe, Sierra Leona. Se utilizó el modelado de ecuaciones estructurales para validar la Escala Utrecht de Engagement, tanto la de 9 (UWES-9) como la de 17 (UWES-17) elementos. Los resultados de la evaluación de la UWES-17 invalidaron la estructura de tres factores en esta muestra de trabajadores sanitarios comunitarios, ya que se hallaron correlaciones elevadas entre los factores latentes. Los resultados de la validez de la UWES-9 mostraban gran congruencia con los de la UWES-17. El ajuste de modelo para la UWES-9 era en general equivalente para las soluciones de uno, tres y dos factores, aunque el modelo de tres factores fue una vez más rechazado por las elevadas correlaciones entre factores. A la vista de los resultados, esta muestra prueba que el compromiso con el trabajo se representa mejor como constructo unidimensional en este contexto. En la línea de la investigación precedente, se considera que estos resultados respaldan la utilización de la forma abreviada, UWES-9, en este contexto, dado que parece representar mejor el compromiso con el trabajo y dispone de un método unidimensional de puntuación parsimonioso.

Sierra Leone has consistently faced a shortage and misdistribution of its health workers. Health worker attrition was particularly acute during Sierra Leone's decade-long civil war, lasting from 1991 to 2002. The introduction of the Free Healthcare Initiative (FHCI) in 2010 saw rapid recruitment of new health staff and salary increases for existing health workers. More recently, the Ebola outbreak in West Africa resulted in the loss of thousands of lives and an estimated 6.85% loss in the country's health workforce (Evans, Goldstein, & Popova, 2015). As a result, the Ministry of Health and Sanitation is increasingly relying on a volunteer cadre of community health workers (CHWs) for the delivery of primary care.

Though CHWs are effective in delivering health services to communities and improving health outcomes across specific health indicators, they remain vulnerable to high rates of attrition (Kironde & Klaasen, 2002; Nkonki, Cliff, & Sanders, 2011). Attrition rates are particularly high in rural areas where, despite greater need, health workers face an increased workload, poor or non-existing financial remuneration, difficult terrain, and high levels of isolation (Fritzen, 2007). Rates of attrition for CHW programmes are known to vary widely across different contexts and programmes (Bhattacharyya, Winch, LeBan, & Tien, 2001; Haines et al., 2007), suggesting that there are a number of interacting factors influencing a CHW's decision to remain in a programme.

Organizational psychology is increasingly recognised as an important field to address current health challenges in human resources for health (Carr et al., 2012; Vallières & McAuliffe, 2015). Specifically, organisational psychology has the potential to widen our current understanding of the factors that contribute to CHW retention over time. A better understanding of the psychological factors that lead to long-term engagement of CHWs in their work is considered of critical importance and recent studies have called for greater evidence to mitigate the high attrition rates of health workers in rural Sierra Leone (Wurie, Samai, & Witter, 2016).

Literature Review

The Utrecht Work Engagement Scale (UWES; Schaufeli, Salanova, González-Roma, & Bakker, 2002) is commonly used to study engagement among workers and volunteers alike (Vecina, Chacon, Sueiro, & Barron, 2012). The UWES was developed to measure three related factors of work engagement: vigour, dedication, and absorption. Categorised by levels of high energy and mental resilience, vigour describes a willingness to persevere in the face of obstacles in the workplace or the willingness to continue to invest effort in one's work when confronted with a challenge (Schaufeli, Bakker, & Salanova, 2006). Dedication implies heavy involvement in one's work and, through this work, one experiences a sense of challenge, pride, and inspiration (Schaufeli & Bakker, 2003). Individuals with high levels of dedication see their work as meaningful and inspiring. Lastly, absorption is defined as being happily engrossed in and concentrated on one's work, where time seems to pass rather quickly, and where one has difficulty detaching themselves from their work (Schaufeli & Bakker, 2003).

To date, the UWES has been translated into 23 different languages and applied in a variety of cultural contexts, including China (Fong, 2012; Zhang & Gan, 2005), Finland (Hakanen, 2002), the United States of America (Mills, Culbertson, & Fullagar, 2012), and a number of countries in continental Europe (Balducci, Fraccaroli, & Schaufeli, 2010; Pisanti, Paplomatas, & Bertini, 2008; Schaufeli, Martinez, Marques-Pinto, Salanova, & Bakker, 2002). With the exception of two studies among South African samples (De Bruin, Hill, Henn, & Muller, 2013; Storm & Rothmann, 2003), the UWES has primarily been utilised among workers in high and middle-income countries. To date, no research has been conducted to assess the reliability and validity of the scale with individuals from low-income countries.

Despite the widespread use of the UWES (Schaufeli et al., 2006) there are inconsistencies in the literature regarding the appropriate factor structure of the scale. Establishing the appropriate factor structure of the scale is necessary in order to develop an accurate scoring scheme for the UWES. A number of studies with culturally distinct samples have provided empirical support for the original three factor solution of vigour, dedication, and absorption (Schaufeli, Salanova et al., 2002; Storm & Rothmann, 2003). Conversely, other studies have failed to support this factorial structure (Sonnentag, 2003; Wefald & Downey, 2009), arguing that the UWES can be most accurately and parsimoniously represented as a unidimensional construct (Christian & Slaughter, 2007). In a number of other studies, the UWES has been reduced in length to 15 items (Salanova, Agut, & Peiro, 2005; Storm & Rothmann, 2003) and to 9 items (Fong, 2012). Even with these shortened versions there is conflicting evidence as to whether the scale is best explained as a unidimensional or multidimensional measure of work engagement (Fong, 2012; Mills et al., 2012; Vecina et al., 2012).

The objectives of the present research are twofold. First, this study seeks to examine the factorial validity of the UWES within the low-income context of Sierra Leone and, where necessary, suggest changes that would render the scale more appropriate for use in this context. Second, this study thoroughly examines the 9-item version as a possible, more parsimonious alternative to the 17-item UWES for community health workers in Sierra Leone.

Method

Participants and Procedure

Data collection took place over three weeks in May 2012 as part of an ongoing longitudinal cohort study of CHWs participating in World Vision Ireland's Access to Infant and Maternal Health (AIM-Health Programme). As part of the same programme, all CHWs were expected to carry out similar tasks which included timely visits to households with pregnant women and children under the age of five to deliver key health messages for the prevention of maternal and child morbidity and mortality. Participants were contacted from a list of all CHWs active in the AIM-Health programme areas, as was provided by the Bonthe District Health Management Team (DHMT) and World Vision Sierra Leone. A total of 327 out of 334 CHWs from the chiefdoms of Jong, Imperi, Sogbeni, and Kpanda Kemoh on the mainland chose to participate in this study. This sample size exceeded the requirements of N ≥ 200, or ratio of N to the number of variables in the model (p), N/p ≥ 10 considered necessary for confirmatory factor analysis using structural equation modelling (Boomsma & Hoogland, 2001).

Of these 327 participants, 54.7% (n = 179) identified as female and 43.5% (n = 142) identified as male. Just under a third of participants identified as Christian (30.9%, n = 103), with the remaining identifying as Muslim (67.0%, n = 223). Over a third (36%, n = 117) of CHWs had never attended school, less than a fifth (19.1%, n = 62) had attended some level of secondary school, 11.7% (n = 38) attended some level of primary school, and 20.9% (n = 68) had attended some level of junior secondary school. Only 9.8% (n = 32) of all CHWs interviewed had some level of tertiary education. Just less than half of the participants (46.2%, n = 154) of CHWs were aged between 26 and 40 years, 15.9% (n = 53) were < 26 years of age, and 33.3% (n = 111) were aged between 41 and 60 years old.

Scale Translation and Adaptation

Given that CHWs in Sierra Leone are volunteers, the term ‘work’ in the original UWES was substituted for ‘working as a CHW’. Respondents indicated their level of engagement on a five-point Likert scale: 1 (strongly disagree) to 5 (strongly agree). Total scores for vigour can range from 6 to 30, where higher scores indicate a greater willingness to persevere in the face of obstacles in the workplace, or when confronted with a challenge. An example of an item statement used to measure vigour includes, “At my work as a CHW, I feel strong and vigorous”. Total scores for dedication can range from 5 to 25, where higher scores indicate that individuals see their work as meaningful and inspiring. Dedication is characterised by items such as “I am enthusiastic about my work as a CHW”. Lastly, absorption total scores can range from 6 to 30, where high scores indicate that individuals report being happily engrossed and concentrated while working as a CHW. Absorption is captured in the UWES through items such as “I get carried away when I’m working as a CHW”. Cronbach's alpha for the three sub-scales of the UWES-17, vary between .80 and .90 across a number of studies (Demerouti, Bakker, Janssen, & Schaufeli, 2001; Schaufeli, Salanova et al., 2002).

To ensure that the content of the questionnaire was well understood, the questionnaires were first translated into Krio, Sierra Leone's lingua franca (see Supplementary Material). The UWES was then back-translated into English to ensure consistency and that no content was lost in translation. A total of eight enumerators were carefully trained to administer the questionnaire in Krio, Mende, and English, prior to the administration of the time-one assessment. Mende being an almost exclusively oral language¹, the questionnaires were ultimately left in English but administered by an enumerator in the language most familiar to the participant. In the case of illiterate CHWs, questionnaires were administered in a mixture of Krio and Mende with the help of a trained enumerator. In the case of literate CHWs, questionnaires were typically completed in English without the help of an enumerator. Of the CHWs who completed the questionnaire, 47.8% were illiterate, and therefore required the assistance of an enumerator, and 52.2% were able to read and write.

Data Analysis

The factor structures of the 17- and 9-item versions of the UWES were investigated through the use of traditional CFA techniques and confirmatory bifactor modelling. Three alternative models representing the latent structure of both versions of the UWES were assessed. Model 1 represented the UWES as a unidimensional structure in which the 17 items loaded on to a single engagement factor. Model 2 reflected the original three-factor model of vigour (6 items), absorption (6 items), and dedication (5 items). Model 3 was the proposed bifactor model, which combines Models 1 and 2 in a hierarchical manner (see Figure 1). Within models 1 and 2, items were restricted to load onto a single factor, while in the bifactor model each item was allowed to load onto the general factor (engagement) and one group factor (vigour, absorption, and dedication), as per recommendations (Reise, Moore, & Haviland, 2010). The UWES-9 contains three items for each of the three factors of vigour, absorption, and dedication. We tested for group differences in work engagement scores across sex, literate vs. illiterate respondents, and a one-way between groups ANOVA was conducted to examine differences in work engagement mean scores based on four levels of educational status (1 = never been to school, 2 = some primary/junior school, 3 = some secondary school, and 4 = some college/third level education). All measurement modelling analyses were carried out using Mplus version 7.0 (Muthén & Muthén, 1998-2010) with robust maximum likelihood (MLR) estimation (Yuan & Bentler, 2000). The MLR estimator allows parameters to be estimated using all available information and has been found to be superior to alternative methods requiring list-wise deletion of cases with missing values (Schafer & Graham, 2002). Model fit was determined using standard criteria: A chi-square to degree of freedom ratio (χ²:df) of less than 3:1 reflecting good model fit; Comparative Fit Index (CFI) and Tucker Lewis Index (TLI) values greater than .90 reflect acceptable model fit, and values greater than .95 reflect excellent model fit; Root-Mean-Square Error of Approximation with 90% confidence intervals (RMSEA 90% CI) and Standardized Root-Mean-Square Residual (SRMR) values of .05 or less reflect excellent model fit, while values less than .08 reflect acceptable model fit. Furthermore, the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC) were used to evaluate alternative models with the smaller value in each case indicating the best fitting model. The CFI, RMSEA, BIC, and AIC all have explicit penalties for model complexity.

Figure 1.

Bi-factor Model of the UWES-17.

Note. The unidimensional model is reflected in the lower half of the table (17 items loading on to the Engagement factor), and the correlated three-factor model is reflected in the top half of the table (three factors are correlated).

Results

CFA of the 17-UWES

The CFA results suggested that the unidimensional model provided an unsatisfactory representation of the data with the CFI and TLI indices falling well below acceptable levels (see Table 1). The correlated three-factor model (Model 2) was not identifiable due to the occurrence of a Heywood case (correlation greater than 1 between the absorption and vigour factors). A Heywood case suggests model mis-specification due to the extraction of too many factors (Byrne, 2012). Overall, the bifactor model (Model 3) produced the most superior model fit indices of the three tested models; however, this model also failed to satisfy the minimum criteria for acceptable model fit according to the CFI and TLI indices. On the basis of these findings, the UWES-17 does not appear to be a valid measure of work engagement for CHWs within the Sierra Leonean context.

Table 1.

Model Fit Indices for the Alternative Models of the UWES-17.

Models	χ^2	df	CFI	TLI	RMSEA 90% CI	SRMR	AIC	BIC
1 Factor Model	265.712^*	119	.728	.689	.062 (.052 - .072)	.069	13733	13926
3 Factor Model	-	-	-	-	-	-	-	-
Bi-factor Model	196.313^*	102	.825	.767	.054 (.042 - .065)	.057	13663	13920

Note. N = 323; χ² = chi square goodness of fit statistic; df = degrees of freedom; RMSEA = Root-Mean-Square Error of Approximation with 90% Confidence Intervals; AIC = Akaike Information Criterion; BIC = Bayesian Information Criterion; CFI = Comparative Fit Index; TLI = Tucker Lewis Index; SRMR = Standardized Square Root Mean Residual.

*

p < .05.

CFA of the 9-UWES

We subsequently assessed the latent structure of the 9-item UWES by comparing Models 1-3. As can be seen in Table 2, the three models provided an acceptable representation of the data and each model exhibited similar fit statistics. For each model, the χ²:df ratios were approximately 2:1, the CFI and RMSEA results indicated acceptable fit, and the SRMR results indicated excellent fit. Only the TLI results failed to satisfy minimal acceptable levels for acceptable fit. The AIC suggested the statistical superiority of the bifactor solution, while the BIC suggested the three-factor model to be the best fit. Given the difficulty of assessing the correct factorial structure on the basis of the model fit and model comparison indices alone, we thus assessed the model parameters to determine the most appropriate factorial solution.

Table 2.

Model Fit Indices for the Alternative Models of the UWES-9.

Models	χ^2	df	CFI	TLI	RMSEA 90% CI	SRMR	AIC	BIC
1 Factor Model	49.121	27	.911	.882	.050 (.027 - .072)	.050	6908	7010
3 Factor Model	48.328	24	.902	.854	.056 (.033 - .079)	.049	6910	6929
Bi-factor Model	36.333	17	.921	.844	.059 (.035 - .081)	.038	6897	7036

Note. N = 323; χ² = chi square goodness of fit statistic; df = degrees of freedom; RMSEA 90% CI = Root-Mean-Square Error of Approximation with 90% Confidence Intervals; AIC = Akaike Information Criterion; BIC = Bayesian Information Criterion; CFI = Comparative Fit Index; TLI = Tucker Lewis Index; SRMR = Standardized Square Root Mean Residual.

* p < .05.

Inspection of the factor correlations of the three-factor model indicated poor discriminant validity, as factors were very highly correlated, ranging from .87 to .95, suggesting that this model was not appropriate. Additionally, inspection of the factor loadings for the bifactor model indicated the primacy of a single ‘engagement’ latent factor. Factor loadings on the general engagement factor were all positive, statistically significant (p < .001), and had a mean factor loading of .47. Contrastingly, factor loadings on the three general factors of vigor, dedication, and absorption were predominately non-significant, weak, and varied in direction. These results further indicated the unsuitability of a multidimensional representation of the structure of the UWES-9.

Overall, the BIC was lower for the unidimensional model than the bifactor model, suggesting that the increase in model complexity associated with inclusion of the three method factors (vigor, dedication, and absorption) in addition to the general engagement factor did not lead to a superior representation of the UWES-9. Accordingly, taking into account model fit results, model parameter results, and favouring parsimony and theoretical interpretability, we suggest that the unidimensional model of the UWES-9 reflects the preferred scoring scheme for CHWs within the Sierra Leonean context. The factor loadings for each item of the UWES-9 are reported in Table 3 along with the item level and total score descriptive statistics.

Table 3.

Standardised Factor Loadings, Standard Errors, and Descriptive Statistics for the UWES-9.

Item	β	SE	Mean	SD
Work Engagement
When I am working as a CHW, I feel bursting with energy (Vigour-1)	.56^*	0.07	4.14	1.06
At my work as a CHW, I feel strong and vigorous (Vigour-2)	.58^*	0.06	4.31	0.89
I am enthusiastic about my work as a CHW (Dedication-1)	.48^*	0.07	4.45	0.67
My work as a CHW inspires me (Dedication-2)	.44^*	0.06	4.40	0.82
When I get up in the morning, I feel like going to work as a CHW (Vigour-3)	.39^*	0.06	4.21	0.85
I feel happy when I am working intensely as a CHW (Absorption-1)	.42^*	0.06	3.95	1.07
I am proud of the CHW work that I do (Dedication-3)	.32^*	0.07	4.36	0.81
I am immersed in my work as a CHW (Absorption-2)	.50^*	0.07	4.43	0.72
I get carried away when I am working as a CHW (Absorption-3)	.63^*	0.06	4.52	0.82
Total Score of the UWES-9 (possible score range from 9-45)	--	--	38.88	4.26

*

p < .001.

Group Differences in Work Engagement Scores

Mean work engagement scores were compared across sex and no differences were found, t(305) = 0.48, p = .63, d = 0.06. Similarly, no significant differences in mean work engagement scores were identified between literate and illiterate respondents, t(295) = 0.15, p = .88, d = 0.02. The ANOVA results revealed no statistically significant differences in mean levels of work engagement based on educational level, F(3, 293) = 0.80, p = .50, ¿² = .01. These results indicate that the UWES-9 is likely stable across sex, literacy, and educational status with this Sierra Leonean cadre of CHWs.

Discussion

The primary objective of the current study was to assess the factorial structure of the 17-item and 9-item versions of the UWES within a novel sample of volunteer CHWs in Sierra Leone. This work was carried out in order to determine the most accurate and parsimonious method of scoring this widely-used scale within a unique cultural context where the ability to quickly and reliably measure and track levels of work engagement is critical to the management of a volunteer workforce.

Considerable debate exists in the literature as to whether the UWES, and by extension the construct of work engagement, should be considered a unidimensional psychological construct or a multidimensional construct. The CFA results failed to support the factorial validity of the UWES-17, though the shortened UWES-9 performed well within this cadre of CHWs. Our findings are largely consistent with previous studies that failed to find evidence for a multi-dimensional construct of work engagement (Sonnentag, 2003; Wefald & Downey, 2009). Contrastingly, and while our findings support the use of the 9-item UWES in this context, our results are inconsistent with the three-factor model of the UWES-9 advocated for use among Chinese workers by Fong (2012).

The three models tested produced similar model fit estimates; however, the extremely high correlations between the three factors of vigour, dedication, and absorption undermined the viability of this scoring scheme. Poor discrimination between the three factors of the UWES is consistent with findings of a meta-analytic review by Christian and Slaughter (2007), who reported high correlations between vigour and absorption (.95) between dedication and absorption (.90) and between vigour and dedication (.88). While the bifactor model produced slightly superior statistical indices of model fit as compared to the unidimensional model, inspection of the factor loadings for the bifactor model produced strong evidence of unidimensionality. Given the high correlations between the three factors of vigour, dedication, and absorption within the three-factor model, the strong evidence of unidimensionality within the bifactor model, and the satisfactory model fit results observed for the unidimensional model, we reason that the latter provides the best statistical representation of the UWES-9 within this sample of Sierra Leonean CHWs. The unidimensional model of the UWES-9 has the added advantage of reflecting the most parsimonious, and rapid, scoring scheme for use by programme managers.

Implications for Theory and Practice

Henrich, Heine, and Norenzaya (2010) discuss how psychology is often overwhelmingly concerned with, generated from, and utilised in Western, Educated, Industrialised, Rich and Democratic, or WEIRD, settings. It follows, and as Berry, Poortinga, Breugelmans, Chasiotis, and Sam (2011) argue, that the transfer of scales from predominantly high- and middle-income contexts to a low-income context could result in irrelevant or even harmful findings. The results of this study are consistent with the need to pay greater attention to indigenous psychologies and refining theories to take account of the setting (Gelfand, Leslie, & Fehr, 2008). Therefore, while drawing on organisational psychology theories and hypothesising the existence of work engagement in the Sierra Leonean context, this study remains mindful of the potential need to refine these theories for a low-income context.

Conclusion

The current study suggests that among a novel sample of CHWs working in Sierra Leone with diverse levels of education and literacy standards the construct of work engagement can be validly and reliably captured by the 9-item version of the UWES. In addition, the current study contributes a Krio version of the UWES-9 for use in future studies, and represents the 24^th language in which this scale has been translated worldwide. To the best of our knowledge, this is the first study to test the validity of the UWES in a low-income context.

Limitations of the Study

The current study is not without limitations. The use of a very specific group of CHWs in Sierra Leone limits the extent to which these findings can be generalised to other volunteer health workers in different social contexts. Translation of the English version of the UWES to Krio may have resulted in a distinct response pattern emerging and thus suggesting that findings from this study may not generalise to English speaking samples of community health workers.

Conflict of Interest

The authors of this article declare no conflict of interest.

Financial Support

This research was supported in part by grants from Irish Aid and the Department for International Development (DFID).