This paper analyses the implementation characteristics of the Family Education and Support program, a theory-driven, needs-based, and evidence-based positive parenting program originally developed for the Andalusian family preservation services. The implementation process of 34 trials of the FAF program with 155 participants was analyzed. Cluster analyses were also performed to explore variability in implementation conditions from a comprehensive perspective. Results showed different implementation profiles that moderated the FAF effectiveness (namely lengthier interventions, higher program fidelity, and practitioners’ positive perceptions and satisfaction with the program). The relevance of examining implementation process across several trials is discussed in order to distinguish core and non-core FAF components, as well as the need for combining faithful and adaptable implementations that guarantee the ecologic validity of evidence-based positive parenting programs.
Este artículo analiza las características de implementación del programa de «Formación y apoyo familiar» (FAF). A partir de un análisis de necesidades, con una sólida fundamentación teórica y cumpliendo con los criterios de los programas basados en evidencias, el FAF se aplica en los servicios de preservación familiar de Andalucía para la promoción de la parentalidad positiva. En este artículo se analiza el proceso de implementación de 34 aplicaciones del programa FAF en las que participaron 155 madres. Desde una aproximación integral, se realizó un análisis de clústeres para explorar la variabilidad en las condiciones de implementación. Los resultados mostraron diferentes perfiles de implementación que moderaban la efectividad del programa FAF. Los grupos que mostraron mejores resultados fueron los de más duración, mayor tamaño y los que fueron aplicados con más fidelidad y por profesionales que valoraban muy positivamente el programa. Se discute la utilidad de examinar el proceso de implementación de diferentes aplicaciones del programa para identificar los elementos centrales del mismo y que necesitan ser respetados. Asimismo, se concluye con la necesidad de combinar fidelidad y flexibilidad para asegurar la validez ecológica de los programas de parentalidad positiva basados en la evidencia.
There are both ethic and economic reasons to justify that scarce financial and human resources are directed to those programs that have proven to significantly impact the target population (Small, Cooney, & O’Connor, 2009). In positive parenting programs, this means that the question of how much a program improves families’ quality of life needs to be answered (Flay et al., 2005).
Despite the relevance of the efficacy question, it does not give us much information about the best way to deliver the intervention in diverse real-world settings (Dodge, 2011). This point is a legitimate concern, because evidence indicates that efficacious programs can be poorly implemented with negative consequences for the participants (Durlak & DuPre, 2008; Fixsen, Naoom, Blase, Friedman, & Wallace, 2005). Consequently, the implementation process—together with a solid theoretical basis and an adequate evaluation design—is nowadays considered one of the main quality criteria for evidence-based parent education programs (Flay et al., 2005; Máiquez, Rodrigo, & Byrne, 2015). Accordingly, attention must be paid to the factors that influence the implementation process in order to explore variability in program outcomes. In line with these assumptions, in this paper we analyze the implementation characteristics of the Family Education and Support program (Programa de Formación y Apoyo Familiar), an evidence-based positive parenting program originally developed for the municipal Andalusian family preservation services in Spain (Hidalgo et al., 2011).
Despite the broad consensus about the implementation process as a quality standard for parenting programs and the existing orientations from international agencies to this matter (Walsh, Rolls-Reutz, & Williams, 2015), there is still not agreement about which factors should be specifically examined. Some authors focus on the implementation strategies followed to develop a program, while others analyze implementation outcomes. In this paper is stated that both questions (strategies and outcomes) constitute key elements for the implementation process and should be considered in program evaluation research as effectiveness moderators (Baumann, Kohl, Proctor, & Powell, 2016).
Implementation strategiesThe implementation strategies include the requirements and the recommendations that have to be considered in order to implement a program (Fixsen et al., 2005; Herrera, León, & Medina, 2007). These strategies need to be considered when designing the program and should be manualized, in order to guarantee both program dissemination and ecologic validity. The implementation strategies that have proven to be relevant in positive parenting programs are listed: organizational support; professional training; adaptability; participant selection and group composition (Rodrigo, Máiquez, Martín, & Rodríguez, 2015).
Organizational support is one of the most important components in determining whether a program is successful or not (De Melo & Alarcao, 2012). First, technical support and adequate setting and material conditions are needed to guarantee implementation success (Paúl, Arruabarrena, & Indias, 2015; Paúl, Indias, & Arruabarrena, 2015). However, the adequacy of the logistic conditions is not enough for a suitable implementation. In addition, programs should be officially supported, properly publicized through media and presented as an opportunity for all the families to improve their parenting (Rodrigo, Byrne, & Álvarez, 2012). Moreover, organizations responsible for the intervention have to ensure resources not only for developing the program, but also for guaranteeing its sustainability. According to Herrera et al. (2007), the organizational support will increase as long as following items are achieved: program and institution goals converge; professional selection and training is planned by the institution; and long-term funding is planned.
Professional training is a critical component of the implementation process (Fixsen et al., 2005). An adequate professional selection that matches program's characteristics as well as an optimum providers’ training constitute key elements to guarantee program fidelity. The most salient components in practitioners’ training are: what needs to be trained, and when should it be done (Herrera et al., 2007). Concerning the content there is variability according to the specific nature of the program; although theoretical bases of the program, as well as communication and group-work skills should be included (Máiquez et al., 2015). Regarding training timing, a structured proposal and suitable to program characteristics should be outlined before program implementation. Afterward, ongoing support and counselling from program developers should be provided (Small et al., 2009).
Adaptability refers to the strategies that enable the program to adapt to diverse contexts. Thus, flexibility constitutes a key factor in implementation that guarantees both program dissemination and ecologic validity. In order to ensure this balance while keeping program fidelity, core and non-core program components need to be distinguished. The latter can be modified in order to adjust the program to specific characteristics of the context for implementation (Dalton, Elias, & Wandersman, 2001; Kumpfer, Magalhães, & Xie, 2012). program developers should manualize this distinction. For this purpose, not only the former application but also several trials need to be implemented in order to identify core and non-core components (Fixsen et al., 2005; Kumpfer et al., 2012).
Finally, strategies related to participant selection and group composition constitute core components related to both standards and effectiveness of positive parenting programs. Individual interviews are a valuable resource for selecting program participants. Information provided from these interviews should be employed to compose groups with a moderate level of heterogeneity (Rodrigo et al., 2012). In line with this idea, available evidence indicates that preservation interventions achieve higher effect sizes with participants of mixed socioeconomic status (MacLeod & Nelson, 2000).
Implementation outcomesManualization of the aforementioned implementation strategies is needed, although is not enough for going to scale with parenting program standards. Evaluation of implementation outcomes can help investigators to disentangle implementation effectiveness from treatment effectiveness and to know, for example, if an intervention failed because it was poorly designed or because it was incorrectly implemented (Baumann et al., 2016). The most evidenced variables in positive parenting programs regarding implementation evaluation are the following: Fidelity; intensity and dosage; and participant responsiveness.
Fidelity is the extent to which a specific application corresponds to the originally intended program (Fixsen et al., 2005). This component is related to practitioner performance, and it has been measured through several indicators in the parenting program literature. Some authors refer to provider's adherence or compliance, this is the extent to which the practitioner uses the core intervention components prescribed and is committed with the program. Some of them refer to it as provider competence or quality of delivery (i.e., the way in which the intervention is delivered) (Baumann et al., 2016; Fixsen et al., 2005). Whatever the indicator, it has been consistently proved that higher levels of fidelity lead to better program outcomes (Durlak & DuPre, 2008; Oats et al., 2014). Assuring fidelity by stating the core elements to be followed across different settings is important; however, it should not be forgotten that balancing fidelity with adaptability throughout the implementation process is also crucial (Cabassa & Baumann, 2013).
Along with fidelity, those components related to participants that benefit from the intervention tend to be included in implementation evaluations. On one hand, we can refer to program dosage and intensity, that are usually operationalized in terms of duration of the program, specifically, the amount of hours in contact with the participants. The research indicates that dosage and intensity need to fit the characteristics of the targeted population in order to guarantee program effectiveness (Small et al., 2009). On the other hand, participants’ responsiveness refers to the degree to which the program stimulates the interest or holds the attention of participants (Durlak & DuPre, 2008). Empirical data shows that programs are more efficacious when they use active and varied teaching methods (e.g., role-playing, brainstorm, debates, etc.) that engage participants and enable them to learn and practice new skills (Small et al., 2009).
Family Education and Support programThe Family Education and Support program (FAF stands in Spanish), is a secondary preventive intervention implemented from the municipal family preservation services in Andalusia (Spain). It is a parenting program that integrates psycho-educational and community models, and it is originally designed for families at psychosocial risk (Hidalgo et al., 2014).
The FAF it is a theory-driven, needs- and evidence-based program (López, Menéndez, Lorence, Jiménez, Hidalgo, & Sánchez, 2007; Menéndez, Hidalgo, Jiménez, Lorence, & Sánchez, 2010). It is aimed at: (1) increasing parenting knowledge and skills, (2) ameliorating efficacy and satisfaction as a parent as well as personal adjustment, and (3) promoting informal social support networks (Hidalgo, Menéndez, Sánchez, Lorence, & Jiménez, 2009). These aims are translated into 14 units about 5 topics: (1) process and contexts of human development; (2) parent–child interaction; (3) other family interactions; (4) the family and the community; and (5) adjustment problems in childhood and adolescence. In addition, it should be stated that this program is based on a group-based, active, participative and experiential methodology. The aforementioned units can be trained through one or several sessions, performing activities aimed at enhancing both behavioural and emotional parental competences. The proposed activities are based on parents’ implicit theories and go through inductive strategies, transforming their experiential knowledge from exchanging mental schemes about parenting with other adults in similar situations (Hidalgo et al., 2015).
Concerning implementation strategies, it should be noticed that strong organizational support has been offered. The FAF program is included in the Municipal plan for at-risk children in Seville (Plan Municipal de Prevención y Atención a la Infancia y Adolescencia en situación de riesgo). As a consequence, the FAF program is being regularly implemented since 2009 in every district of Seville (Spain). In addition, a training plan for providers has been manualized. The FAF developers provide two courses about the theoretical and methodological principles, the structure, and the requirements for implementation. In addition, an ongoing support and counselling resource is offered.
Probably the most salient implementation strategy of the FAF is its adaptability. This program is fully manualized and structured, but adaptable to meet specific family needs and context characteristics. This adaptability refers only to non-core components in order to guarantee the fidelity to the program but at the same time ensuring its dissemination and guaranteeing its ecologic validity. For example, a consequence of FAF flexibility is variability in trials length, although core components are guaranteed.
Finally, participant selection and group composition should be commented. The FAF program was originally designed for families with a moderated risk level, although due its flexibility it can be adapted. Providers are recommended to compose mixed groups with diverse levels of risk, in order to enhance the diversity of life experiences and perspective-adoption from alternative models. Specific advice about group composition and management are manualized because these are considered key components for FAF effectiveness.
In a more comprehensive framework of the FAF evaluation, the purpose of this paper was to describe implementation characteristics of the FAF program. Specifically, two aims were dealt in this article: (1) the characteristics of FAF program implementation in family preservation services in Andalusia (Spain) were described; (2) the variability in implementation conditions was analyzed in order to explore variability in FAF effectiveness. According to previous research, it was expected that some components of implementation process (as fidelity, intensity and participants’ responsiveness) moderate FAF effectiveness. As a contribution from existing evidence in this area, different implementation profiles that moderate the FAF effectiveness are identified.
MethodParticipantsPractitioners responsible for the intervention provided information from 34 annual trials of the FAF program delivered by family preservation services in Andalusia (Spain), this is the whole population of families participating at FAF groups in a four-year period. Variability in number of applications occurred, with practitioners applying from one to four trials. Information was provided about 155 mothers with an average age of 40 years (M=40.57, SD=8.30), and a quite low educational level: 41.78% had completed elementary school and 23.29% were illiterate. The majority of them were employed (59.21%) but with precarious labour circumstances: 53.41% of the working mothers had unstable jobs, mainly without employment contract (57.83%). More than 50% of the families were two-parent (53.95%), with two or three children as average (M=2.46, SD=1.05). The practitioners conducting the FAF program were the families’ case managers at preservation services; they were all psychologists, mostly women (71.43%), trained in group dynamics and systemic techniques (75%).
MeasuresThe parents fulfilled a broad battery of questionnaires about family dynamics. In this paper, information refers to:
Self-esteem: A 4-item scale of the AUT-17 questionnaire was used (Gracia, Herrero, & Musitu, 2002) in order to measure the emotional self-esteem of the participants (e.g., “I have problems to control myself”). The scale is rated from 1 (completely disagree) to 5 (completely agree). The Cronbach's alpha in this study was α=.61.
Parental sense of control: An adapted version validated for Spanish at-risk families of the Parental Sense of Competence scale was used (Johnston & Mash, 1989; Menéndez, Jiménez, & Hidalgo, 2011). Concretely, a subscale to evaluate perception of control as a parent (e.g., “I do not know why it is, but sometimes when I’m supposed to be in control, I feel more like the one being manipulated”) was considered in this research. This dimension is measured through four items in a six-point scale ranging from 1 (strongly disagree) to 6 (strongly agree). The Cronbach's alpha in this study was α=.53.
Inductive discipline: The DP-R questionnaire about parental discipline for at-risk families was used (Hidalgo, Sánchez, Jiménez, & Sánchez, 2014); specifically the 8-item inductive discipline subscale was included (e.g., “When child's behaviour breaks family rules, first step is asking about it”). The items were rated on a scale from 1 (totally disagree) to 4 (totally agree). The Cronbach's alpha was α=.82.
The practitioners provided additional information for the FAF implementation through several tools developed ad hoc for the program:
Family socio-demographic profile: A semi-structured interview to collect socio-demographic information about the mothers (age, gender, educational level and labour conditions) and the family as a whole (structure and children at homes).
Records of the FAF sessions: A grid for each session was fulfilled by practitioners, including information about the program implementation (objectives, contents, and activities) and participants’ responsiveness: Involvement, engagement, and attendance (the formers in a 3-point Likert scale from 1=low to 3=high).
Professionals’ perception and satisfaction with the FAF program: A Delphi exercise with a 5-point Likert scale was used, with practitioners rating core components of the program design and implementation requirements (objectives, contents, techniques, dosage, etc.), as well as the perceived usefulness for practitioners, participants and preservation services (from 1=a highly weak aspect of the program to 5=a highly salient aspect of the program). To obtain consensus, the mean was employed as the statistic for integrating the scores in a second round.
The developers of the FAF who were also responsible for professionals’ training and supervision participated in an experts’ panel in order to reach an agreement about practitioners’ profile concerning: Training in family intervention and FAF program, and a multi-component evaluation of fidelity through adherence and compliance (e.g., “The practitioner implemented faithfully the FAF techniques”). Consensus from the expert panel was reached through a 5-point Likert scale (from 1=not at all to 5=completely yes).
ProcedureThe composition of the FAF groups was made by practitioners, from families being attended by the family preservation services who meet the following requirements: (1) the family needed an intervention for parent education purposes; (2) the nuclear family had school-aged children or adolescents; and (3) the family's level of psychosocial risk had been evaluated by practitioners and was deemed to be average or moderate.
The parents fulfilled a broad battery of questionnaires about family dynamics during semi-structured interviews at pre-test and post-test conducted by two researchers. The practitioners provided additional information for the FAF implementation in each session. The FAF developers participated in an experts’ panel after each trial. Every informant participated in this study voluntarily and was informed about its objectives and the anonymous use of the data.
ResultsImplementation characteristics of the FAF program are described below. The average and the standard deviation were used as descriptive statistical for continuous variables, and percentages for qualitative variables. According to Tabachnick and Fidell's (2007) recommendations, parametric (Student t test) and nonparametric (Mann–Whitney) tests were performed to exam possible differences between groups. According to Hair, Anderson, Tathan, and Black (2008), two-step cluster analysis was performed for groups’ classification. Finally, the moderating role of clusters for program effectiveness was explored with repeated measure ANOVA's with family dimensions included as intra-subject factor (self-esteem, parental sense of control, and inductive discipline) and cluster (C1, C2, and C3) as inter-subject factor. The statistical analyses were performed with SPSS v-18.
Characteristics of the FAF implementationThirty-four groups were implemented, composed by 8.74 participants as average (SD=2.28). Two-hour weekly sessions were developed with every group, with total dosage ranging from 9 to 22 sessions (M=16.39, SD=3.08). Concerning FAF contents, in average 4.19 units were covered (SD=1.26). None of the groups covered all the existing units. The groups with larger dosage implemented 7 units. The most frequently implemented units were Process and contexts of human development (unit 1) and Parent–child interaction (unit 2). The less frequently used units where those not directly focusing on parenting issues. There was also group variability in the number of sessions per unit, ranging from 1 to 8 sessions (M=3.16). The unit Adjustment problems in childhood and adolescence were the lengthiest with an average of 5 sessions (M=4.67; DT=1.53).
Participant responsiveness was rated through several indicators. Stability was observed regarding group composition. Moreover, in most trials no dropout was observed (67.74%). As average, participants attended 70.13% of the sessions. Only 8.15% of the participants quit the program, because of incompatible work schedule in 30% of the cases. High rates of both involvement (M=2.40, SD=0.54) and engagement were observed (M=2.50, SD=0.51). Involvement and engagement rates were negatively related to group size (r=−.24, p<.005 for engagement; r=−.27, p<.005 for motivation) and higher in stable groups (Mann–Whitney's U=1150.00, p<.05 for engagement; Mann–Whitney's U=893.50, p<.005 for motivation).
All practitioners who delivered the intervention had previous experience in family intervention (100%). According to the results from the experts’ panel, medium-high rates were obtained for FAF training (M=3.43; SD=.97), adherence (M=3.76; SD=.93), and compliance (M=3.80; SD=.96).
Information provided by practitioners revealed that the FAF program was highly valued and supported from preservation services. Specifically, 80% of providers rated the FAF design as a highly relevant aspect, and 100% did considered FAF objectives and methodology as relevant aspects. Concerning the contents, 11 units were positively rated by 100% of practitioners, and the 3 remaining units by 92.86%. A broad majority (70.83%) highlighted implementation characteristics (e.g., weekly sessions, participant selection) as highly relevant aspects. Usefulness for participants, practitioners and preservation services were considered as highly relevant aspects by 95.71% of respondents. It was also stated that the FAF program covered a specific intervention area not included in other resources.
Integrative clustering of implementation variabilityCluster analyses were performed in order to explore variability in implementation conditions across trials from a comprehensive perspective. Thirty-four annual trials described above referred to 19 different groups, because a group could include two annual trials. These 19 groups were employed as unit of analysis to meet the clustering objective. The following information was examined: group size; intensity and dosage; number of units covered; professional experience; fidelity to the program; and professionals’ perception of FAF program concerning its design and implementation requirements.
According to Hair et al. (2008) recommendations, a two-step cluster analysis (hierarchical and non-hierarchical) with z-scores was performed. Initial grouping was explored through hierarchical cluster analysis employing the squared Euclidean distance and the Ward method. The decision about the number of clusters was based on the dendogram, the statistical validity, the cluster-group size, as well as theoretical coherence and the interpretability of the clusters. This exam revealed the existence of three types of implementation (C1, C2 and C3). After determining the number of clusters through the hierarchical method, the centroids of these initial clusters were submitted to an iterative non-hierarchical clustering procedure (K-means) to confirm and refine the 3-cluster solution. The mean cluster values for each variable of this solution are presented in Table 1.
Implementation characteristics per cluster.
| C1 (n=4) | C2 (n=9) | C3 (n=6) | F | LSD | ||||
|---|---|---|---|---|---|---|---|---|
| Z | M | Z | M | Z | M | |||
| Group size | −1.18 | 6.5 | −0.36 | 8.5 | 1.11 | 10.6 | 10.14** | 1-2*; 1-3***; 2-3* |
| Intensity and dosage | −0.41 | 21.7 | −0.42 | 21.6 | 0.64 | 35.6 | 3.09† | 1-2; 1-3†; 2-3* |
| Number of units | −0.58 | 4.5 | −0.36 | 5.2 | 0.69 | 8.6 | 3.43† | 1-2; 1-3*; 2-3* |
| Prof. experience | −1.11 | 2.1 | 0.89 | 4.2 | −0.81 | 2.5 | 19.93*** | 1-2***; 1-3; 2-3*** |
| Fidelity | −1.69 | 2.6 | 0.35 | 4.2 | 0.56 | 4.4 | 15.08*** | 1-2**; 1-3**; 2-3 |
| Prof. design | −1.26 | .6 | −0.22 | 1.1 | 1.42 | 1.9 | 31.14*** | 1-2**; 1-3***; 2-3*** |
| Prof. implementation | −1.25 | .5 | 0.42 | .9 | 0.41 | .9 | 9.91** | 1-2**; 1-3**; 2-3 |
Note: Z scores were included for ANOVAs analyses.
Results revealed three clusters differing by size and implementation characteristics (C1=4 groups, C2=9 groups, and C3=6 groups). Post hoc comparisons between clusters were conducted merely for purposes of interpretation of the cluster solution through ANOVA analyses (see Table 1). The group size and scores of professionals’ perception of FAF design differed significantly among the three clusters, and the remaining variables differed by pairs. C1 presented the lowest scores in all the variables. C2 was different from C1 in several aspects: Medium group size, medium professional experience, and higher fidelity to the program. Furthermore, the perception of these professionals about the FAF program was much more positive. Finally, C3 comprised trials with larger groups, lengthier interventions (both in number of sessions and in contents), and a more positive perception of the professionals about the FAF design. The professionals of this cluster presented professional experience similar to C1 and a level of fidelity similar to C2.
Effectiveness of the FAF: the moderating role of implementation profilesIn order to explore the moderating role of implementation characteristics for the FAF effectiveness, repeated measures ANOVAs with inter- and intra-subjects effects were performed. Pre-test and post-test scores of self-esteem, parental sense of control and inductive discipline were included as dependent variables and the interaction time*cluster was examined as moderator effect. The 155 participants were distributed across the clusters as follows: nC1=40, nC2=68, and nC3=47. The results indicated significant interaction effects; therefore, significant changes in specific cluster slopes were examined. The interaction effects are plotted, and both the statistics for the ANOVAs and significant slope changes are presented for self-esteem (Fig. 1), parental sense of control (Fig. 2), and inductive discipline (Fig. 3).
The repeated measures ANOVAs indicated that implementation characteristics significantly moderated FAF effectiveness concerning self-esteem, parental sense of control and inductive discipline. Examination of slopes showed significant improvements for C3 in comparison to other clusters for all the variables considered. C2 also demonstrated a significant improvement for parental sense of control. In contrast, C1 did not show any significant change after the intervention.
DiscussionIn this study, an evaluation of the FAF program implementation is presented as a key component for assessing internal and external validity of interventions (Durlak & DuPre, 2008). For this purpose, implementation characteristics were described first. Second, a comprehensive analysis of variability in implementation conditions was explored across several trials, in order to examine its moderating role for FAF effectiveness.
Descriptive results revealed similarities across the FAF trials, which points to its reliable implementations. The providers followed the manualized implementation strategies concerning participant selection, group composition, and core units, as revealed by the adherence and compliance scores. Nonetheless, variability was observed concerning group size, intensity and dosage. This variability in implementation has been previously observed elsewhere (De Melo & Alarcao, 2012; Durlak & DuPre, 2008), and can be explained by the adaptability of the FAF. This enables taking into account the specific characteristics of each group and accomplishing ecologic validity while ensuring core components (Herrera et al., 2007; Máiquez et al., 2015), which remains a challenge for parenting programs (Orte, Amer, Pascual, & Vaqué, 2014). The results from this study confirm the need for considering implementation characteristics of several trials to distinguish non-core and adaptable characteristics from core components that identify evidence-based programs (De Paúl et al., 2015a; Fixsen et al., 2005; Máiquez et al., 2015).
Moreover, in this paper it has been stated that the FAF counts on organizational support and that it is delivered by trained professionals with stable positions in preservation services. These aspects have been highlighted in the literature as key components for quality implementation standards (Rodrigo et al., 2012). These results can help us to understand the positive perception of practitioners about the FAF program, as well as the high scores on compliance: organizational support likely promotes professionals’ commitment. Moreover, the organizational support constitutes a key element for giving prestige to the program, which facilitates logistic conditions and enhances providers’ training (Herrera et al., 2007).
The high-observed levels of participants’ involvement and engagement in this study, as well as the low dropout rate underscore participants’ responsiveness in FAF implementation. This responsiveness constitutes a key element for implementation process, although it has not been exhaustively examined and there is still not a golden standard way to assess it (Oats et al., 2014; Small et al., 2009; Staudt, 2001). Thisresearch and practice gap is probably explained by the inability to give families a voice regarding their experiences with parenting programs, which is an essential element to empower at-risk parents (Ayala-Nunes, Jiménez, Hidalgo, & Jesus, 2014; Baker, 2007; Daly, 2007; Tilbury, Osmond, & Crawford, 2010).
In order to explore variability between implementation conditions cluster analyses were performed. This approximation led us to assume a comprehensive perspective, considering various aspects involved in the implementation process through the examination of a large amount of trials. It constitutes a useful approach for identifying and distinguishing effectiveness core components from non-core elements adaptable to specificities, without jeopardizing program fidelity (Fixsen et al., 2005).
The key elements for the FAF implementation were group size, intensity and dosage, fidelity and professionals’ perception and satisfaction with the program. As it has been previously established, more intense interventions are linked to better results for the FAF program, which can be easily explained by the pressing needs of at-risk families (Menéndez et al., 2010; Small et al., 2009). Larger groups also achieved better results for self-esteem, parental sense of control and inductive discipline. This is not a surprising result, considering that larger groups include more heterogeneity, which facilitates the adoption of perspectives from alternative models (MacLeod & Nelson, 2000; Martín et al., 2009). Finally, the providers played an important role for FAF implementation and its effectiveness. On one hand, fidelity appeared as an effectiveness moderator, which has been previously established in literature (Fixsen et al., 2005). On the other hand, it seems that not only sticking to the manual of the program influences its effectiveness, but also feeling committed with the intervention. These results provide support to the idea that the attitudinal component should be examined in more detail, going beyond the professional profile, program training or program fidelity (Rodrigo et al., 2012). Other authors have stated as well that professional competence as a whole constitutes a core component of delivery quality, consequently, of program effectiveness (Baumann et al., 2016; Fixsen et al., 2005).
In conclusion, results discussed in this paper lead us to confirm two hypotheses. First, implementation standards play a crucial role in program effectiveness. Second, adequate implementations require manualized strategies. These strategies should ensure fidelity through the application of the core components combined with adaptability of non-core elements, what have proved to have a positive effect on program outcomes (Durlak & DuPre, 2008).
Some limitations of this study are the absence of observation records for the groups as well as lacking information about parents’ perceptions of the program. Although these data have been collected, there were space limitations to include them in this paper. Despite these gaps, in our opinion this study offers an innovative approximation to the implementation process in parenting interventions. Variability in implementation conditions through several FAF trials has been explored, identifying core components that ensure fidelity to the program in future replications and, consequently, led the FAF program to meet the evidence-based standards for parents at risk.
FundingThis study has been funded by the Spanish Government (MINECO, Ministerio de Economía y Competitividad). Project reference: EDU2013-41441-P, “Evaluación de intervenciones psicoeducativas con familias en situación de riesgo psicosocial. Un análisis de buenas prácticas basadas en la evidencia”.
Conflict of interestThe authors have no conflict of interest to declare.
