Little is known about how the knowledge, attitudes, and behaviors of the public child welfare workforce influence implementation of evidence-based practice (EBP) as most research has focused on the private workforce. This paper reports on public child welfare staff knowledge, attitudes, and practices in a state implementing the EBP, SafeCare®. A survey of public child welfare staff (N = 222) was conducted to assess knowledge, familiarity, and referral barriers and practices. Knowledge of and familiarity with SafeCare were low, especially among front line staff (case managers). Attitudes toward SafeCare were fairly positive, but somewhat less so than attitudes toward a standard, non-evidenced based parenting program. Case managers were significantly less likely to have made a referral (15%) than other staff (46%). Job tenure had few effects on familiarity, knowledge, attitudes, or referrals. The strongest predictors of having made referrals were familiarity with SafeCare and job position.
Se sabe poco sobre cómo influyen el conocimiento, las actitudes y las conductas de los profesionales del sistema público de protección infantil en la implantación de programas basados en la evidencia (PBE), ya que gran parte de la investigación sobre este tema se ha centrado en el ámbito privado. Este artículo informa acerca de los conocimientos, las actitudes y las prácticas de un equipo público de protección infantil que lleva la implantación en un Estado de EE.UU. de un PBE (SafeCare®). Se aplicó una encuesta a 222 profesionales que trabajaban en protección infantil para evaluar el conocimiento, la familiaridad y las dificultades y prácticas de derivación de casos. El conocimiento y la familiaridad con SafeCare® era bajo, especialmente entre profesionales de primera línea (responsables de casos). Las actitudes hacia SafeCare® eran bastante positivas, pero un poco menos que las actitudes hacia un programa utilizado habitualmente y no basado en la evidencia. Los responsables de casos tenían significativamente menos posibilidades de derivar (15%) que otros profesionales (46%). La antigüedad en el trabajo influye poco en el conocimiento, las actitudes o las derivaciones al programa. Los predictores que más influyeron en el número de derivaciones a SafeCare® fueron la familiaridad con el programa y el tipo de contrato laboral.
The impact of child maltreatment on children, families, and socie-ty is well documented. An estimated 675,000 children are victims of officially substantiated maltreatment each year (U.S. Department of Health and Human Services, 2012), and these numbers represent only the cases that come to attention of child welfare officials. True estimates from survey data demonstrate that maltreatment is a much bigger problem than official reports indicate (Finkelhor, Ormrod, Turner, & Hamby, 2005; Sedlak et al., 2010). The impacts of maltreatment include negative psychological, social, and health impacts for individual victims (Felitti et al., 1998; Horwitz, Widom, McLaughlin, & White, 2001; Widom, Schuck, & White, 2006; Wilson & Widom, 2011), and a large economic impact on society of about $100 billion per year (Fang, Brown, Florence, & Mercy, 2012).
For many years, families with substantiated cases of maltreatment have been offered in-home family preservation services aimed at preventing the removal of the child from the home and future maltreatment reports. Typically, services are delivered by community-based providers using what some have called a “supportive case management” approach, which includes some concrete assistance, social support, referrals for specific problems, and parenting advice, but little structured intervention. Evaluations of these services showed disappointing results with few or no differences between families who received services and those who did not (Chaffin, Bonner, & Hill, 2001; Littell, 1997; MacMillan et al., 2005; Schuerman, Rzepnicki, & Littell, 1994; Westat, 2002).
Currently, there are mandates from federal, state, and local child welfare agencies to offer evidence-based programs (EBP). The research community strongly recommends that behaviorally-based parenting programs should serve as a foundation for child welfare services (Barth et al., 2005; Chaffin & Friedrich, 2004; Whitaker, Lutz-ker, & Shelley, 2005). Several behavioral parenting programs have been found to reduce maltreatment reports in randomized trials, including SafeCare (Chaffin, Hecht, Bard, Silovsky, & Beasley, 2012), Triple P (Prinz, Sanders, Shapiro, Whitaker, & Lutzker, 2009), and Parent-Child Interaction Therapy (PCIT; Chaffin et al., 2004). These programs have also been shown to offer good economic return on investment (Lee et al., 2012). With these results, much attention has focused on how best to implement such programs on a widespread basis.
Implementing an evidence-based practice is not a simple matter. EBP must be implemented with some degree of fidelity to ensure the desired outcomes are achieved (Durlak & DuPre, 2008). Theories of implementation suggest that processes at multiple levels can influence the uptake, delivery quality, and sustainment of EBP (Damschroder et al., 2009; Fixsen, Naoom, Blasé, Friedman, & Wallace, 2005; Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004). Organizational and systems factors such as leadership and organization culture and climate play an important role in determining the utilization and uptake of an EBP (Glisson & Hemmelgarn, 1998), especially in large, complex service systems such as child welfare systems. Most U.S. child welfare systems are structured in public/private partnerships. Public agency staff conduct investigations of alleged abuse, make a determination of the allegations (substantiate the case as maltreatment or not), and provide recommendations for service. Private staff at community-based agencies provide interventions based on those recommendations that can include parenting, substance use, and mental health.
With this infrastructure, implementation of EBP in child welfare has typically focused on training private providers within community-based organizations to deliver EBP, while less attention has been placed on training the public system staff who are often responsible for referring and even reimbursing services. This is problematic because public child welfare agency staff serve as referral agents or service brokers for parents and children (Dorsey, Kerns, Trupin, Conover, & Berliner, 2012). As new EBP are adopted and become avai-lable, it is critical that public child welfare staff be aware of the avai-lability of the EBP, understand the content of the EBP, understand which families are appropriate for referral, and understand any new procedures for completing referrals or closing out cases. One example in which public child welfare caseworkers were trained on an EBP resulted in increased awareness of an EBP, and a trend toward better identification of referrals suitable for an EBP (Dorsey et al., 2012).
Most of the research on implementation of EBP has focused on the early stages, such as decision to adopt EBP (Panzano & Roth, 2006; Wang, Saldana, Brown, & Chamberlain, 2010), initial adoption, and early indicators of utilization and fidelity following training (Kolko et al., 2012; Whitaker et al., 2012). This is particularly true for research focused on how public staff interact with EBP. Often, the role of public agents is seen as one of deciding to adopt an EBP, provide funds for training, and create policies to support the EBP. Once this is accomplished, the focus turns to private providers who will implement the EBP practice as they are trained and supported to implement that practice. Yet, public child welfare staff can play a critical role in the ongoing utilization and sustainment of an EBP for the reasons described above. Data show that increased collaboration between public child welfare staff and private service providers increase service utilization (Hurlburt et al., 2004), and increasingly, public child welfare workers are seen as service brokers or gateway providers that link youth and families to appropriate services (-D-orsey et al., 2012; Stiffman, Pescosolido, & Cabassa, 2004). Yet, little information is available on what impacts public service workers’ attitudes and behaviors regarding EBP.
Aims of the Current PaperThe goal of the current paper is to report on results from a survey of public child welfare staff in one state regarding their knowledge, attitudes, and behavior regarding SafeCare, an evidence-based mo-del that had been implemented for a few years in this particular state. SafeCare is a home-based, behavioral parenting program designed to address child neglect and physical abuse. SafeCare addres-ses three skill deficits that are proximal antecedents to child neglect and physical abuse: positive parenting skills, home safety, and child health care skills. The evidence base for SafeCare indicates that it can reduce child welfare recidivism (Chaffin, Hecht et al., 2012), improve parenting skills (Carta, Lefever, Bigelow, Borkowski, & Warren, 2013), decrease parental depression (Chaffin, Bard, Bigfoot, & Maher, 2012), increase program completion (Damashek, Doughty, Ware, & Silovsky, 2011), and improve service satisfaction (Damashek, Bard, & Hecht, 2012; Silovsky et al., 2011).
In this particular state, SafeCare had been implemented since 2009 by trained workers from private community-based agencies to deliver the model. Over 200 private providers have been trained throughout the state, but service utilization for SafeCare was gene-rally low (Whitaker et al., 2012). Several attempts have been made to increase service utilization, including intensive training workshops focusing on the program, referral procedures, appropriate families, and reimbursement infrastructure across the majority of service regions in the state. In order to further examine the low service utilization, a survey was conducted in 2012 to assess public staff know-ledge and attitudes about SafeCare, barriers to SafeCare utilization, and referral practices for SafeCare.
In understanding this particular implementation, it is important to note that SafeCare was implemented alongside a standard paren-ting program which had no specific curriculum or protocol for deli-very (referred to in this paper as “standard parenting”). That is, staff could refer to either SafeCare or the standard parenting. Thus, in the rollout of SafeCare, the existing non-evidence-based parenting program was not removed, something has been referred to as de-adoption or exnovation (Stirman et al., 2012). Second, because of differential levels of effort required for SafeCare versus standard parenting, reimbursement structures for each program were different, and SafeCare was the longer and more costly of the two programs. There were also different procedures and expectations regarding referring clients to SafeCare versus standard parenting. These issues – costs and administrative procedures – were suspected barriers to implementation and were assessed directly in the survey.
The survey was designed to assess factors thought to be related to service utilization. We assessed familiarity with and knowledge about SafeCare, attitudes toward and perceived effectiveness of SafeCare and standard parenting, perceived barriers to referrals, and actual SafeCare referrals. We also measured some aspects of the res-pondent, including the job position with an interest in examining differences between case managers and other supervisory and administrative staff. Case managers are the front line staff and as such make the bulk of the service referrals in child welfare. However, as the front line staff, they may be the last to be informed of new initiatives. Because of frequent turnover in child welfare positions, we also pay attention to job tenure. Newer staff may have had less of an opportunity to learn about a new EBP such as SafeCare, but may also may be more open to new service models (Whitaker et al., 2012).
Thus, the specific goals of this article are:
- 1.
To describe the knowledge, familiarity, and attitudes about SafeCare among public child welfare staff three years into a statewide implementation.
- 2.
To examine referral practices, barriers to referrals, and predictors of referrals.
- 3.
To examine differences by position and length of tenure at the child welfare system.
The anonymous survey was conducted via email. State child welfare leadership constructed an email to all public child welfare staff with a link to a web-based survey, created using www.Psychdata.com. The survey link was sent via email to all public child welfare staff once per week for four weeks (repeated emails were sent to encourage participation). The email encouraged staff to complete the 20-minute survey. Of the 294 respondents, 113 responded after the first email prompt, 80 after the second prompt, 58 after the third prompt, and 41 after the fourth prompt. No personally identifying information about respondents was collected, as the survey was anonymous. Because of this, there is no way to determine if any individual responded more than once to the survey, though this seems unlikely given that there were no incentives to complete the survey. The study was approved by the University IRB.
MeasuresRespondent work characteristics. Job position was measured with a single item assessing the type of position (case manager, field program specialist, administrator, county or regional director, or other). We also assessed the employment unit of the respondent (investigations, family preservation, permanency, or other), the region in which they worked, and how long they had worked for the public child welfare system (1 year or less, 2-3 years, 3-5 years, 5-10 years, 10+ years).
Familiarity with SafeCare was measured with the single question, reading, “Overall, how familiar are you with the SafeCare program?” Respondents indicated their familiarity using a four-point Likert scale, 1 = not at all, 2 = slightly, 3 = somewhat, 4 = very.
Knowledge of SafeCare was measured in a few different ways. First, respondents were asked to list the three skills SafeCare targets (parent-child interactions, health, and safety), which were reviewed in the marketing and training workshops given in service regions, and open-ended responses were coded. A score of 0-3 representing the number of modules successfully identified was created for each participant. A dichotomous variable was also created indicating whether participants correctly identified any of the three modules. Finally, respondents were asked to identify the target age group for SafeCare via a multiple-choice question (0-5, 6-12, 12-18, 0-18), and responses were coded as correct or not.
Attitudes. A series of attitude items were presented to measure attitudes toward SafeCare and standard parenting. Based on a factor analysis, three scales were developed for analyses in this paper. The first two scales assessed satisfaction with SafeCare (5items) and satisfaction with standard parenting services (5 items), and included parallel items for each scale. Sample items are: “SafeCare (standard pa-renting) addresses family's needs very well” and “How much does SafeCare (standard parenting) seem to help families you refer?” Scale scores were created by averaging across items. A third scale, titled concerns about SafeCare, was formed by averaging three items (“SafeCare is too expensive”, “The referral process and paperwork for SafeCare is too cumbersome”, “SafeCare is no better than our usual fami-ly preservation programs”).
Perceived barriers to making SafeCare referrals were measured with 15 items. Respondents rated the extent to which each item posed a barrier to SafeCare utilization using a 4-point scale (1 = not a barrier, 2 = a small barrier, 3 = a moderate barrier, 4 = a large barrier). Factor analyses indicated that barriers formed two factors: (1) lack of knowledge about SafeCare (7 items, e.g., “I have never heard of SafeCare”), (2) lack of fit between SafeCare and clients/context (8 items, e.g., “I prefer families to get parenting services other than SafeCare”, “The family needs other services like substance use counseling, domestic violence counseling, instead of SafeCare”). Scores for each barrier factor were created by averaging the items within the scale.
SafeCare referrals were measured by asking respondents if they had made any SafeCare referrals (“Have you made any referrals to agencies for FVS/SafeCare services?” yes/no). Because many higher level administrators may have taken the survey, respondents had the option to indicate that it was not their job to make referrals to providers. Respondents were also asked about how many referrals they had made. Because relatively few staff made referrals (n = 43), ana-lyses focus only on whether or not each made a referral and not the number of referrals.
SampleNo information was available on the sample frame or the total number of potential respondents. A total of 294 respondents completed the survey. Respondents were case managers (n = 167, 59%), supervisors (n = 44, 15.5%), administrators or field program specia-lists (n = 24, 8.5%), regional or county directors (n = 17, 6%), and other (n = 32, 11%). Almost 44% of participants worked for the child welfare system for 10 or more years, another 28% worked for the child welfare system for between 5 and 10 years, 17% percent between two and five years’ experience, and 12 percent had one year or less experience. Thus, overall, respondents represented an experienced workforce. Staff from all regions of the state responded to the survey. No additional demographic data were collected about respondents.
Analytic PlanTo describe the knowledge, attitudes, and referral practices of child welfare staff, simple descriptive statistics were used (frequencies, means). To examine differences in knowledge, attitudes, and practices by position type and job tenure, chi-square, t-tests, and ANOVAs were used, depending on the distribution of the dependent variable. Logistic regression was used to model predictors of SafeCare referrals.
ResultsOf the 294 respondents, 72 completed only the initial workforce questions (position, job tenure, region of work) and no additional parts of the survey, and thus, those 72 respondents do not appear in the analyses. Frequency analyses indicated that the 72 respondents who did not complete any survey items did not differ from the 222 respondents in terms of length of service (p = .69), but were more likely to indicate a position of “other” than respondents who completed the survey (27.1% vs. 8.5%), x2(5, N = 222) = 20.07, p < .01. Because of missing data on other variables, sample for analyses ranges from 173 to 222.
Of the 222 remaining respondents, 130 reported their position as case managers, 13 were field program specialists, 38 were supervisors, 16 were directors, 6 were administrators, and 19 were “other.” Because we were particularly interested in the actions of case mana-gers and because of the distribution of the variable, we re-classified position variable as case managers (n = 130) vs. all others staff (n = 92). Regarding job tenure, based on the distribution of responses, the tenure variable was grouped into three cate-gories: 0-5 years (n = 64), 5-10 years (n = 65), and 10+ years (n= 92).
Familiarity with and Knowledge of SafeCareTable 1 displays measures of familiarity and knowledge of SafeCare, overall and by position type and job tenure. On average, staff reported being slightly familiar with SafeCare (M = 2.08 on a 4-point scale). Only about a third could correctly name any of the three focal areas of SafeCare, but most (83%) could identify the age group targe-ted by SafeCare (0-5). The mean number of modules correctly named was less than one out of three (M = 0.74). When compared by position type, differences emerged between case managers and other staff: case managers reported lower familiarity with SafeCare, t(220) = 5.03, p < .01, named fewer modules, t(220) = 2.16, p = .03, and were less likely to correctly identify the age group SafeCare targets, x2(1) = 6.45, p = .01.
Familiarity with SafeCare and SafeCare knowledge, overall and by position and tenure.
| Position | |||||
|---|---|---|---|---|---|
| Overall M (SD) or N (%) | Case managers M (SD) or N (%) | Others M (SD) or N (%) | p | ||
| Familiarity and knowledge | |||||
| Familiarity | 2.08 (.97) | 1.81 (90) | 2.45 (.94) | < .01 | |
| # Modules correctly named | .74 (1.08) | .61 (.98) | .92 (1.19) | .03 | |
| Could Name Any Module | 84/222 (37.8%) | 41/92 (44.6%) | 43/130 (33.1%) | .08 | |
| Correctly named age group | 146/176 (83%) | 75/98 (76.5%) | 71/78 (91.0%) | .01 | |
| Job tenure | |||||
| Overall M (SD) or N (%) | 0-5 yrs M (SD) or N (%) | 5-10 yrs M (SD) or N (%) | 10+ yrs M (SD) or N (%) | p | |
| Familiarity | 2.08 (.97) | 1.95 (.93) | 1.97 (1.00) | 2.24 (.96) | .11 |
| # Modules Names | 74 (1.08) | .81 (1.07) | .75 (1.10) | .68 (1.09) | .77 |
| Could Name Any Module | 84/222 (37.8%) | 28/64 (43.8%) | 24/65 (36.9%) | 32/92 (34.8%) | .51 |
| Correctly named age group | 146/176 (83%) | 41/54 (75.9%) | 38/44 (86.4%) | 66/77 (85.7%) | .27 |
Table 1 also depicts means and percentages for familiarity and knowledge by job tenure (overall means are repeated for sake of comparison across rows). There were no differences in these va-riables by job tenure.
AttitudesTable 2 displays the three attitude measures – satisfaction with SafeCare, satisfaction with standard parenting, and concerns about SafeCare – overall, and by position and job tenure. Overall, attitudes were mildly favorable toward both SafeCare and standard parenting. There was no difference in SafeCare attitudes or standard parenting attitudes by position. Regarding concerns about SafeCare, non-case managers expressed greater concerns than case managers (Ms = 2.42 vs. 2.16, p = .01). No differences emerged by job tenure.
Attitudes toward SafeCare, extant parenting, and concerns about SafeCare by position and tenure.
| Position | |||||
|---|---|---|---|---|---|
| Overall M (SD) | Case managers M (SD) | Others M (SD) | p | ||
| Attitudinal variables | |||||
| SafeCare satisfaction | .462.51 (.81) | 2.40 (.85) | 2.68 (.73) | .05 | |
| Standard parenting satisfaction | 2.72 (.66) | 2.65 (.68) | 2.81 (.61) | .15. | |
| SafeCare concerns | 2.28 (.57) | 2.17 (.52) | 2.42 (.60) | .01 | |
| Job tenure | |||||
| OverallM (SD) | 0-5 yrsM (SD) | 5-10 yrsM (SD) | 10+ yrsM (SD) | p | |
| Attitudinal variables | |||||
| SafeCare satisfaction | 2.51 (.81) | 2.40 (.85) | 2.51 (.86) | 2.64 (.73) | .46 |
| Extant parenting satisfaction | 2.72 (.66) | 2.57 (.68) | 2.69 (.70) | 2.82 (.59) | .15 |
| SafeCare concerns | 2.28 (.57) | 2.30 (.43) | 2.28 (.58) | 2.24 (.62) | .84 |
Because parallel measures were used to create the satisfaction measures for SafeCare and standard parenting, we conducted ana-lyses to compare satisfaction with the two measures. A 2 3 2 repeated measured analysis of variance was conducted with Position (case manager vs. others) and Attitude Type (SafeCare vs. standard parenting) as the within subject factor. There was a significant effect of Attitude Type F(1, 137) = 3.90, p = .05, indicating that satisfaction with standard parenting was higher than satisfaction with SafeCare (Ms = 2.72 vs. 2.56). Because familiarity with SafeCare was overall low, we repeated this analysis using only the respondents who indicated a high degree of familiarity with SafeCare (those who responded were ‘somewhat’ or ‘very’ familiar with SafeCare, n = 69). Results of this analysis indicated no effects for Attitude Type or Position. Interestingly, the direction of the repor-ted satisfaction levels was reversed, such that satisfaction with SafeCare was nominally higher than satisfaction with standard parenting (2.96 vs. 2.88), but the Attitude Type effect was not significant (p = .18).
ReferralsOf the 222 respondents, 47 (21%) indicated it was not their job to make referrals (12 case managers, and 35 non-case managers) and 2 responses were missing. Of the remaining 173 respondents, 43 (24.8%) reported having made a referral for SafeCare. Case managers were significantly less likely to have made a referral (15.1%) than administrators/supervisors (46.3%), x2(1) = 19.3, p < .01. No differences in referral rates were found by job tenure (p = .66).
Barriers to SafeCare referrals are displayed in Table 3, overall and by position and tenure. Based on factor analysis, two categories of barriers were created, (1) Lack of knowledge regarding SafeCare (7 items; e.g. “I have never heard of SafeCare,” “I don’t know enough about SafeCare,”) and (2) Lack of fit of SafeCare (8 items; e.g., “I prefer families to get parenting services other than SafeCare,” “The family needs other services like substance abuse counseling, domestic violence counseling, instead of SafeCare”). Regarding lack of know-ledge as a barrier, overall means indicated that lack of knowledge was a small to moderate barrier (M = 2.31 on 4-point scale). However, there was a difference by position with case managers rating lack of knowledge as a greater barrier than those in other positions, t(185) = 4.27, p < .001. The second barrier – the fit of SafeCare – was rated as a small barrier overall (M = 1.80), with no difference by position (p=.18). No differences in barriers to referrals were found by job tenure (bottom on Table 3).
Barriers to SafeCare referrals, overall and by position and tenure.
| Position | |||||
|---|---|---|---|---|---|
| OverallM (SD) | Case managersM (SD) | OthersM (SD) | p | ||
| Lack of knowledge | 2.31(1.16) | 2.59 (1.13) | 1.89 (1.07) | < .01 | |
| Poor fit with client needs | 1.80 (.93) | 1.87 (1.02) | 1.68 (.75) | .60 | |
| Job tenure | |||||
| OverallM (SD) | 0-5 yrsM (SD) | 5-10 yrsM (SD) | 10+ yrsM (SD) | p | |
| Lack of knowledge | 2.31(1.16) | 2.33 (1.13) | 2.36 (1.19) | 2.28 (1.17) | .93 |
| Poor fit with client needs | 1.80 (.93) | 1.89 (1.00) | 1.84 (1.05) | 1.69 (0.75) | .48 |
Logistic regression analyses were conducted to predict having made a referral. Predictors included position, job tenure, knowledge of SafeCare (# of modules correctly named), satisfaction with SafeCare, and the knowledge and fit barrier measures. In bivariate models, se-veral of the predictor variables were statistically related to having made a SafeCare referral (Table 4). Being a Case Manager was related to fewer referrals, while familiarity with and knowledge of SafeCare and having more positive attitudes toward SafeCare were related to increased likelihood of referral. Endorsing lack of knowledge as a barrier to SafeCare referrals was related to fewer referrals. In the multivariate model, only two variables remained statistically significant: position and familiarity with SafeCare (OR = 4.3). Case managers were 80% less likely to report making a referral than others (OR = 0.20), and increased familiarity was associated with greater referrals (OR = 4.3)
Results from bivariate and multivariate logistic regression predicting SafeCare referrals.
| Predictor | Binary OR (95% CI) | Multivariate OR (95% CI) |
|---|---|---|
| Case Manager (Ref = all others) | 0.2 (0.1 – 0.4)* | 0.2 (0.1 – 0.9)* |
| Job Tenure | 0.8 (0.5 – 1.3) | 0.6 (0.3 – 1.3) |
| Familiarity with SafeCare | 7.0 (3.8 – 13.0)* | 4.3 (1.8 – 10.6)* |
| Knowledge | 2.9 (2.1 – 4.2)* | 1.1 (0.6 – 2.0) |
| Attitude: Satisfaction with SafeCare | 4.0 (2.6 – 7.8)* | 2.6 (0.8 – 8.3) |
| Attitude: Concerns about SafeCare utilization | 1.4 (0.6 – 2.8) | 2.5 (0.7 – 8.9) |
| Knowledge issues as a barrier | 0.2 (0.1 – 0.4)* | 1.0 (0.4 – 2.4) |
| Fit with client needs as barrier | 0.7 (0.4 – 1.0) | 1.1 (0.4 – 3.0) |
* p = .05
This paper examined the knowledge, awareness, attitudes, and referral practices and barriers of public child welfare workers in a state that had implemented the evidence-based parenting program, SafeCare. A large body of literature has demonstrated that implementation success is dependent upon a multiplicity of factors involving multiple individuals and systems (Aarons, Hurlburt, & Horwitz, 2011). In systems such as child welfare that utilize public/private partnerships for assessing and serving clients, it is critical that both public and private systems be engaged in implementation. Public service workers typically assess and refer clients to private community-based organizations who provide the EBP services. If public service workers are unaware or lack knowledge of evidence-based programs, lack of referrals may result in poor uptake of the EBP. Very few studies have examined public child welfare workers knowledge, attitudes, and behaviors toward an EBP during an implementation.
There were several notable findings. First, overall there was a ge-neral lack of familiarity with and knowledge of the evidence-based program, SafeCare, and this was especially pronounced among the case managers, the front line staff, who reported the lowest levels of familiarity and knowledge about SafeCare. Second, attitudes about SafeCare were moderately favorable, but slightly less favorable than the standard parenting program. This latter finding appeared to be driven by those staff who were not familiar with SafeCare. Finally, a relatively low percentage of staff had made referrals for SafeCare, and this was especially true for the front line staff, case managers, who would be most likely to make specific referrals. The strongest predictors of referrals in multivariate analyses were position and familiarity with SafeCare.
It is unclear as to why case managers had less awareness and knowledge of SafeCare, but there are several possibilities. Case ma-nagers are entry level positions with high degrees of turnover (Barak, Nissly, & Levin, 2001), and newer staff may have yet to learn about new program models such as SafeCare. In the current data, case ma-nagers had shorter tenure than other positions (e.g., 42% of case ma-nagers had fewer than five years’ experience vs. 10% of other staff). However, job tenure was not related to familiarity with and know-ledge of SafeCare. Another possibility pertains to how systems typically roll out an EBP. When large systems implement EBP, the impetus to do so typically begins with leadership (Aarons et al., 2011), who consider what EBP to adopt, and how and when to roll those EBPs out. Front line staff may not be informed until the roll out is underway. If there are not concerted efforts to inform front line staff of new programs, they may remain unaware. In the current implementation, there was a concerted ongoing effort to inform the public child welfare staff of SafeCare, with regular email blasts, webinars, and even in-person visits from the implementation team and the state level program coordinator. However, the efforts for this implementation began at the top of the state hierarchy and it is unclear how much it trickled down to regional/county leadership and ultimately front line staff.
Related, it is noteworthy that the staff who were primarily in higher level positions (i.e., the non-case managers) including supervisors and directors, reported greater concerns about SafeCare than case managers. Leadership plays a central role in implementation models (Aarons et al., 2011), and leadership variables have been demonstrated to affect staff attitudes and practices (Aarons & Palinkas, 2007). Here, those in leadership positions expressed concerns about SafeCare (cost, process/paperwork), and this may ultimately affect referral patterns. It is noteworthy that non-case-managers, des-pite greater concerns about SafeCare, were still more likely to have made a referral. The data do not speak to why this might be, but managerial/administrative staff may experience more pressure to use the newer, evidence-based program than lower level staff.
The study has important implications for implementation. In systems with public/private partnerships, it is critical to educate the public workforce who may be assessing and referring clients, as well as training the private workforce who may be implementing the EBP. Public workers tend to have less positive attitudes toward EBP than private providers (Aarons, Sommerfeld, & Walrath-Greene, 2009). Informational or educational efforts must be sustained over time. Staff turnover is high in child welfare systems (Aarons, Sommerfeld, Hecht, Silovsky, & Chaffin, 2009; Barak et al., 2001; Glisson, Dukes, & Green, 2006), and new staff must be educated on the various EBP available within their system. Implementation models that take a systems approach, such as those that use community development teams, or CDTs (Saldana & Chamberlain, 2012) are one method of engaging both public and private systems when implementations of EBP are conducted. CDTs involve peer networks of system leaders, agency directors, and practitioners to implement a model and pro-blem-solve when barriers are encountered. As applied to the current data, CDTs could certainly assist with the dissemination of information on the EBP, and hopefully promote referrals. When CDTs are not an option, careful planning regarding training of referral sources is needed. Such training may need to be ongoing because of staff turnover. As systems add more EBP to their repertoire, staff training should include information on how to determine which services are appropriate for specific families.
In this particular implementation, the EBP was introduced without de-adoption/exnovation of the extant parenting program. Although this provides flexibility to referral agents, it may not be an ideal arrangement for promoting the use of the new EBP; when new practices are adopted without explicit de-adoption of the old practices, it may be especially difficult for new practices to succeed. This may be because of lack of familiarity with the newer model, fear of change, inertia, or commitment to the status quo (Fixsen et al., 2005). Successful implementation likely involves replacing a non-evidence based program with an EBP rather than utilizing programs side-by-side and allowing them to compete for referrals. The de-adoption of non-EBP can send a clear message from an organization's leadership of the importance of adopting the new EBP, a variable which affects staff attitudes about EBP (Aarons, Sommerfeld, & Walrath-Greene, 2009). The role of de-adoption or exnovation efforts in the success of an implementation has not been well-studied.
The study has several weaknesses that should be noted. First, the survey method involved a brief, one-time, cross sectional survey. There was no specific information on the sample frame (e.g., numbers of staff for certain positions), and thus the response rate and representativeness of the sample is unknown. The nature of the pro-ject dictated that a limited number of questions would be included, and as a result, limited questions on demographics, work history, and other important constructs were included, and no standardized questionnaires were included. Because data were collected in a cross sectional manner, the temporal and causal relationship between variables cannot be known. For example, it is not known if more favorable attitudes toward SafeCare would lead to greater SafeCare referrals, or that referrals (and thus experience with SafeCare) would lead to more favorable attitudes.
Implementation of EBP in large service systems is a challenging but critically important endeavor. It has become clear that implementation involves a complex set of processes that involve many individuals at various points within a system to ensure utilization and uptake of an EBP. Simply training practitioners to conduct the EBP is insufficient for uptake, as has been demonstrated in a number of studies (Kolko et al., 2012; Whitaker et al., 2012). Efforts must be made to engage system leaders, referral agents, and other members of practice team to ensure effective utilization of an EBP.
Conflict of InterestThe authors of this article declare no conflict of interest.
Financial SupportParts of this research were supported by a contract with the state in which the research was carried out.
