“Learning together, growing with family”: The implementation and evaluation of a family support programme

Abstract

The “Learning together, growing with family” programme is targeted to at-risk parents and children from 6 to 11 years old, with a preventive focus on promoting positive parent-child relationships. In this study, we examined the quality of the programme implementation and its influence on the programme results in a sample of 425 parents and 138 facilitators drawn from the first trial. Mixed methods were used, consisting of: parental self-reports on parenting dimensions, professionals’ records on parental attendance and appraisals on six topics of the implementation process, and focus group discussions in which facilitators reported on the initial steps of the implementation. Results showed a high quality of implementation with respect to the group facilitator and the programme organization factors, followed by the coordination with services and the support facilities offered to participants and, finally, by the factors of fidelity and prior organization steps. Results of the focus groups confirmed that the prior steps were challenging and offered the more effective strategies. Better quality in the implementation factors predicted better parenting styles and parental competencies after the programme, as well as a higher attendance rate. In sum, this study demonstrates the importance of good implementation in at-risk contexts and provides some clues as to the key elements that moderate programme effectiveness.

Resumen

El programa «Aprender juntos, crecer en familia» está dirigido a familias en situación de riesgo psicosocial con hijos de 6 a 11 años, mediante un enfoque preventivo y de promoción de relaciones positivas entre padres e hijos. En este estudio, se analizó la calidad de la implementación del programa y su influencia en los resultados del programa en una muestra de 425 padres y 138 facilitadores. La metodología fue mixta y consistió en el uso de autoinformes sobre dimensiones parentales, registros de asistencia y evaluaciones sobre 6 temas de la implementación que fueron posteriormente factorizados. También se realizaron grupos de discusión en los que los facilitadores informaron sobre el proceso de implementación. Los resultados indicaron una alta calidad de implementación en relación con los factores de facilitador del grupo y organización del programa, seguido de los factores de coordinación con los servicios y los recursos de apoyo a los participantes y, por último, de los factores de la fidelidad y organización previa. Los resultados de los grupos de discusión confirmaron que los preparativos de la implementación resultan fundamentales y ofrecen las estrategias más eficaces para el desarrollo del programa. Una mayor calidad en todos los factores de implementación predijo mejoras en los estilos y competencias parentales después del programa, así como una tasa de asistencia más alta. En resumen, este estudio demuestra la importancia de una buena implementación en contextos de riesgo y proporciona algunas pistas sobre los elementos clave que modulan la efectividad del programa.

Introduction

The CaixaProinfancia Programme is an initiative of “La Caixa” Foundation, a non-profit body that belongs to the financial entity CaixaBank, aimed at facilitating the promotion and integral development of childhood and families in situation of poverty and vulnerability. Studies of childhood poverty indicate that children who live in this situation have a greater probability of having learning difficulties, health problems, low school performance, premature pregnancies, and fewer job prospects (Brewer, Browne, Joyce, & Payne, 2011; UNICEF, 2007). In this context, the “Learning together, growing up in family” parenting programme (Aprender juntos, crecer en familia, in Spanish) was created (Amorós, Balsells, Fuentes-Peláez, Mateos, & Pastor, 2011; Amorós, Rodrigo, et al., 2011; Amorós, Balsells, Fuentes-Peláez, Molina, et al., 2011; Amorós, Kñallinsky, Martin, & Fuentes-Peláez, 2011), co-authored by researchers from four Spanish universities (Barcelona, La Laguna, Lleida and Las Palmas de Gran Canaria) and with the promotion and funding of the Area of Social Integration of the “la Caixa” Foundation. The programme is aimed at promoting wellbeing and family coexistence in those families with children aged 6–12, by means of increasing the positive relationships between parents and children, according to the core principles for exercising positive parenting (Rodrigo, 2010).

Today, parent education programmes (Amorós, Kñallinsky, et al., 2011) are oriented towards fostering family communication between parents and children and learning coping skills against adversities through positive and resilient parenting (Gómez & Kotliarenco, 2010; Walsh, 2004). Decreasing risk factors, promoting protection factors and strengthening family capacities are the best points of reference for a parenting intervention (Amorós, Balsells, Fuentes-Peláez, Molina, et al., 2011; Amorós, Palacios, Fuentes, León, & Mesas, 2003; Balsells, Amorós, Fuentes-Peláez, & Mateos, 2011). A growing body of evidence suggests that the improvements in the quality of parenting children receive can contribute to reducing child problems and can enhance positive development (e.g., Kumpfer & Alvardo, 2003; O’Connor & Scott, 2006; Rodrigo, Almeida, Spiel, & Koops, 2012).

The “Learning together, growing up in family” programme responds to a holistic view of a development process for family life in 3 dimensions: the emotional dimension, which aims to help manage emotions, the behavioural dimension that helps to face situations competently, and the cognitive dimension, which facilitates a better comprehension of the family life (Amorós, Balsells, Fuentes-Peláez, Mateos, & Pastor, 2015). The programme contains 14 weekly structured sessions aimed at parents and children separately and 14 joint sessions. In six modules and 1 follow-up module (each module has 2 sessions for parents, 2 sessions for children, and 2 sessions for the family) the programme is aimed at promoting the affective links in the family, the negotiated compromise about rules and values, the improvement in family organization and chore sharing, communication and resolution of conflicts, strengthening healthy habits and positive lifestyles, use of Internet at home, and the joint participation in family leisure activities. As for the methodology, participants had the opportunity to engage in activities following group-oriented didactic strategies and techniques, such as role-playing, group work, guided fantasy, brainstorming, case studies, and the use of audio-visual resources (videos and animated stories). A set of materials and resources for parents and children and for the facilitators was also provided for the adequate application and evaluation of the programme.

Since the launching of the programme, the increase in the groups and the families who have been able to participate has been notable. In the year 2011–2012, there were 1270 participants, which grew to 1764 in 2012–2013, to 2234 in 2013–2014, and to 3678 in 2014–2015. In total, 8946 family members (4262 adults and 4684 children) have participated over the course of these 4 years, delivered though a wide network of NGOs under the coordination of the CaixaProinfancia Programme. Each trial of the programme has been rigorously evaluated according to the premise of evidence-based programmes (Rodrigo, 2010). In particular, in the first trial an RCT design was applied for the evaluation of the programme involving 1834 participants, of whom 1270 (609 parents and 661 children) completed the programme, and 564 (296 parents and 268 children) were randomly assigned to the control group. The results showed that after the programme, and in comparison with the control group, the strict (authoritarian) style, the permissive style, and the use of criticism and rejection had significantly decreased and parental affection and communication had increased. There was also an increase in family leisure activities, satisfaction with family life improved and there was greater community integration for the families, as self-reported by parents and children and by focus groups performed with parents, children, and professionals (Amorós, Balsells, Buisan, Byrne, & Fuentes-Peláez, 2013).

In this study, 138 facilitators and 53 collaborating bodies of CaixaProinfancia in 12 Spanish cities reported positively on the impact of the programme on their professional development as a means to assure the programme sustainability over time (Rodrigo, Martin, Mateos, Pastor, & Guerra, 2013). Following a quantitative and qualitative methodology, the results indicate that after the application of the programme the facilitators considered themselves as most competent in the interaction with the families, in the organization of the work, in the coordination of the programme with other services and in the relation with other professionals as well as they have positively valued the impact of the programme on the families.

In the present evaluation study, also performed with the data from the first trial, we addressed the study of the implementation process. The implementation of the programme is crucial to understanding which factors make a programme work when applied in real-life conditions (Durlak & Dupre, 2008; Fixsen, Naoom, Blase, Friedman, & Wallace, 2005). It is important to define the conditions and human and material resources that allow the programme to function properly with the greatest possible guarantee, as well as its impact on programme outcomes (Rodrigo, Máiquez, & Martín, 2010). In particular we examined in this study: (a) the quality of the programme implementation, and (b) to what extent the implementation influenced the programme results. The programme results were changes in childrearing styles and parental sense of competence as well as attendance rate. It is expected that higher quality of implementation predicted better programme results. Increased levels of quality in implementation tend to be an indication of positive results from parent education programmes (Durlak & Dupre, 2008). We also expected that higher quality of implementation predicted higher attendance rate. Attendance rate was selected because drop-out rates are usually high in at-risk families, in spite of the economic support and the facilities provided during the sessions (Baker, Arnold, & Meagher, 2011).

To these aims, we used a mixed-method approach, which consists of the combination of the quantitative and qualitative methodology (Amorós, Balsells, Fuentes-Peláez, Mateos, et al., 2011; Amorós, Rodrigo, et al., 2011; Amorós, Balsells, Fuentes-Peláez, Molina, et al., 2011; Amorós, Kñallinsky, et al., 2011). The mixed method is a process of investigation that compiles, analyses and links quantitative and qualitative data, as well as its integration and joint discussion, in order to achieve a broader and deeper perspective of the phenomena under study (Creswell, 2005; Hernández, Fernández, & Baptista, 2008; Mertens, 2005). For the quantitative approach we have used scales to measure child-rearing styles, parental sense of competence reported by parents, implementation process and assistance rates reported by the facilitators. For the qualitative approach we have used focal groups (Denzin & Lincoln, 2005; Gutiérrez, 2008) in which participants (facilitators of the groups) were asked questions regarding the selection, attraction and motivation of the participants to the programme, as this is a crucial implementation aspect. Other implementation aspects (the evaluation process, the development of the sessions of the programme and the valuation of the programme) has been analyzed in a previous research (Rodrigo, Byrne, & Rodríguez, 2013; Rodrigo, Martin, et al., 2013).

Method

Participants

We examined the quality of the programme implementation and their influence on the programme results in a sample of 425 parents and 138 facilitators and 53 collaborating bodies of CaixaProinfancia in 12 Spanish cities.

Table 1 shows that participants were mainly composed of at risk families mothers (87%) and were mothers (86%) with a mean age of 37 years, and with a mean of 2.4 children, the majority with a two-parent family situation in 54.7%. The majority of the mothers have a basic education (58%) and are unemployed (62%). It is important to note the large proportion of participants coming from immigrant families and their geographic distribution (46.2%). The greatest percentage comes from the Northern African region, followed by South America, and in smaller proportion from the European Union itself and Asian regions.

Table 1.

Sociodemographic distribution of the participants.

	M (SD) or %
Level of risk
Risk	87.0
Gender
Mothers	86.1
Parent age	37.02 (6.77)
Number of children	2.41 (1.15)
Child age	8.89 (2.26)
Child gender
Boys	61.6
Origin
Spanish	53.8
Immigrants	46.2
Family status
Two-parent	54.7
One-parent	45.3
Level of studies
With no studies	15.2
Basic studies	58.9
Bachelor	22.9
University	3.0
Work situation
Active	35.9
Unemployed	61.7
Retired	2.4

There were 138 facilitators, over 61 groups, with an average age of 36.4 (4.2). The majority had more than 6 years of experience. All had graduate degrees; just over one third (39%) were social educators, 17.6% were social workers, 16.4% were psychologists, and 26.8% had degrees in pedagogy (teaching). All facilitators had attended the initial training workshop.

Instruments

Quality of Implementation Scale. The quality of implementation process was examined using a questionnaire elaborated ad hoc based on Byrne (2010). The objective was to assess the conditions in which the programme was implemented to contribute to its development and improvement in future trials. Consisted of 23 items with a response scale ranging from 0 to 4 (0. Not at all; 1. A little; 2. Some; 3. Plenty; 4. A lot). To obtain the factor model we used the structural equation exploratory model (Asparouhov & Muthén, 2009) using MPLUS6 that combines the exploratory factor analysis and the confirmatory factor analysis, the RMSEA index for the sample was 0.026. The resulting model includes 6 factors: (1) Prior steps to the beginning of the programme (4 items; vg., “The programme has been advertised”, “the programme has been officially inaugurated”), (2) Fidelity (4 items; vg., “Fidelity with respect to content and methodology is adequate”), (3) Programme organization (3 items; vg., “the best schedule for the meetings has been sought according to the groups”), (4) Facilities to the participants (5 items; vg., “Volunteers have been taken care of young children during the sessions”, “Some refreshments and coffee breaks have been arranged for the families during the sessions”), (5) Quality of the group facilitator (4 items; vg., “Facilitators have been stable during the program”), and (6) Coordination to other services (3 items; vg., “Facilitators have collaborated with the professional team and the coordinator”). Reliability was adequate (α = 0.650 for the Prior steps scale, α = 0.610 for the Fidelity scale, α = 0.876 for the Programme organization scale, α = 0.867 for the Facilities to the participants scale, α = 0.674 for the Quality of the group facilitator scale and α = 0.791 for the Coordination to other services scale.

Identification of Family Educational Practices scale (PEF) (short version) (Alonso & Román, 2003). The scale explores in a scale of 54 items, (0–5 frequency scale) the reported use of educational practices most frequently used by families across three hypothetical family situations. The styles examined are Authoritarian (low affect, low sensitivity, imposing), Inductive or Balanced (democratic, affective, communicative, clear rules, flexible control), and Permissive (indulgence, lack of demands and control, affective expressions). We used the parent version for this study. Reliability was adequate (α = 0.85 for the authoritarian scale, α = 0.85 for the inductive scale and α = 0.83 for the permissive scale).

Parental Sense of Competence (PSOC) (Johnston & Mash, 1989; Spanish version by Menéndez, Jiménez, & Hidalgo, 2011). This is a self-report scale of perceived self-efficacy and satisfaction in the parental role. It is a 16-item self-report questionnaire rated on a 6-point Likert scale (1 = strongly disagree; 6 = strongly agree). The PSOC provides two subscales: parents’ self-efficacy (α = .77), and satisfaction in the parenting role (α = .78). Higher mean scores for the subscales indicate more self-efficacy and satisfaction with the parental role.

Parental attendance rate. This was recorded on an individual basis by the facilitator. At the end of the programme each participant's rate of attendance was computed as a percentage of the total sessions assisted.

Focus group

There are two essential elements in the focus group technique: the list of questions to trigger off the discourse and selection of the participating people to pick up their perception, attitude or feeling about a specific subject responding to the objective of the investigation. Literal transcriptions were made of the contents of the focus groups (one per collaborating entity, 20 in total) with the professionals that had animated the groups. For the analysis of the content of the focus groups we used the Atlas Ti 6.2 computer program. Prior to the analysis of the textual information, some initial categories of reference were selected based on the literature. In the first stage of textual analysis paragraphs, fragments and significant quotes in the transcription documents of the discussion groups were selected. The second stage, consisted of the construction of the empirical categories extracted and their relevance which respect to the initial categories of reference.

Results

First, we included the descriptive analyses performed on the quality of the implementation factors. Then, we described the pre-post changes observed in the child-rearing practices and Parental competence. Finally, we included the regression models performed to analyze the influence of implementation factors on the Child-rearing styles, Parental sense of competence and attendance rate. Finally, we included the results obtained in the focus groups.

Descriptive analyses

Table 2 shows the average values of the implementation factors. In general, the values indicate a medium-high level of the quality of implementation, but they are especially high with respect to the quality of the group facilitator and the organization of the programme, followed by coordination with other services and the individual support facilities offered to participants and, finally, by adherence to programme contents and methodology and prior organization steps.

Table 2.

Means and standard deviations of the implementation factors.

Implementation factors (1–5 scale)	M (SD)
Prior steps	3.34 (0.61)
Fidelity	3.57 (0.52)
Programme Organization	4.20 (0.52)
Facilities to the participants	3.68 (0.55)
Quality of the group facilitator	4.88 (0.21)
Coordination to other services	3.84 (0.48)

Table 3 shows the pre-test and post-test comparisons on Child-rearing practices and Parental competence. Repeated measured ANOVA analyses were performed on pre– post measures for each of the factors involved in the outcome dimensions (Child-rearing practices and Parental competences). Significant positive results were obtained after the programme in Inductive and Permissive parenting, with increases scores in the former and decreases scores in the latter. Satisfaction and Efficacy also significantly increased after the programme.

Table 3.

Pre-test and post-test mean contrasts on parental outcomes.

Dimension	Factor	Pre-test M (SD)	Pos-test M (SD)	F (1,424)	Effect size (R^2)
Child-rearing practices	Authoritarian	2.02 (1.01)	1.80 (1.18)	1.654	.03
	Inductive	2.58 (1.03)	3.07 (0.93)	13.422^***	.21
	Permissive	2.30 (0.88)	1.71 (0.98)	10.302^***	.19
Parental competence	Satisfaction	3.27 (0.71)	3.54 (0.82)	3.55^*	.06
	Efficacy	3.91 (0.81)	4.20 (0.85)	5.61^**	.08

*

p ≤ .05.

**

p ≤ .01.

***

p ≤ .001.

With regard to the parental attendance in percentage the average rate of attendance was 80% of the sessions (minimum 70% and maximum 85%) what means a drop-out rate quite low given the poor psychosocial conditions of the families and motivated by personal and family circumstances.

Regression models

In order to determine which factors from the implementation process are those that most determine an improvement in the exercise of positive parenting after the programme, linear regression analyses has been performed. To that end, changes in parental education practices, parent efficacy and level of attendance of participants were analyzed as a function of the six factors of the implementation scale (Prior organization, Fidelity, Programme organization, Facilities to the participants, Quality of the group facilitator and Coordination with other services). Linear regression analyses were applied for the changes in Child-rearing practices and Parental sense of competence, as predicted by the six implementation measures Change scores for each factor of the outcome measures were calculated by subtracting the pre-test score from the post-test score, so that a higher score indicated an increase in the factor and a low score a reduction of the factor. All the variables included in the regression models were standardized (Tabachnick & Fidell, 2007). To interpret the global significance of the model, we examined the statistic F, the values for the Adjusted R² (AdjR²), as well as the specific contribution of each variable to the total variance explained by the model through the significance and the value of the squared semi-partial correlation (rs²). All analyses were conducted using the SPSS 18.0 statistical software assuming a confidence level of 95% for Type I error.

The models were only significant for Permissive and Inductive practices as well as for parental Efficacy and Assistance to the programme. The model of Permissive practices accounted for 47% of variance (Table 4). The results showed that Fidelity, Programme Organization, Facilities provided to the participants and Positive qualities of the facilitator predicted a reduction in permissive practices. The variables that contributed most to the model was the Programme Organization (rs² = .29) and Facilities provided to the participants (rs² = .15). The model of Inductive practices accounted for 36% of variance (Table 4). The results showed that Fidelity, Facilities provided to the participants and Positive qualities of the facilitator predicted an increase in inductive practices. The variable that contributed most to the model was the Facilities provided (rs² = .15), closely followed by the rest.

Table 4.

Regression models predicting pre-post changes in permissive and inductive practices from the implementation factors.

	Permissive, F(6,425) = 6.854				Inductive, F(6, 425) = 3.104
	β	rs^2	AdjR^2	p	β	rs^2	AdjR^2	p
			.47	.000			.36	.016
Prior steps	−.58			.111	.07			.775
Fidelity	−1.76	.08		.019	.71	.14		.016
Organization	−2.23	.29		.000	.25			.545
Facilities	−1.69	.15		.002	.54			.257
Facilitator	−2.14	.07		.027	.74	.15		.018
Coordination	−.41			.177	.61	.13		.014

The model of parental Efficacy accounted for 23% of variance (Table 5). The results showed that Fidelity, and Positive qualities of the facilitator predicted an increase in the perception of self-efficacy. The variable that contributed most to the model was the Positive qualities of the facilitator (rs² = .11). Finally, the model of programme assistance accounted for 28% of variance (Table 5). The results showed that Prior steps taken, Fidelity, Programme Organization, and Facilities provided to the participants predicted an increase in parental assistance to the programme. The variable that contributed most to the model was the performing of preparatory actions prior to the starting of the programme (rs² = .14).

Table 5.

Regression models predicting pre-post changes in parental Efficacy and Assistance to the programme from the implementation factors.

	Efficacy, F(6,425) = 2.346				Assistance, F(6, 425) = 20.166
	β	rs^2	AdjR^2	p	β	rs^2	AdjR^2	p
			.23	.040			.28	.000
Prior steps	−.26			.361	.46	.14		.000
Fidelity	.90	.08		.030	.14	.01		.020
Organization	.48			.207	.28	.04		.000
Facilities	−.46			.201	.12	.01		.028
Facilitator	1.00	.11		.012	.02			.633
Coordination	.13			.460	.08			.170

Results of the focus groups

Here we draw on facilitators’ comments on the elements that influence in the selection and attraction of the families as well as the strategies of motivation and forming groups, as they are the preparatory steps in the implementation process. We include some examples of transcriptions from the professionals’ discourse that illustrate the elements analyzed. Regarding the elements that have influenced in attracting the families the facilitators mentioned some of the strategies that had good results:

• •

  Pre-selection of families: In accordance with the inclusion and exclusion criteria of the families, the social workers of the centre and/or centres in the same network pre-selected families that could benefit from the programme. This “first” list of families served to begin the process of group formation.

• •

  Individual interview: After making the pre-selection, the attention in an in-person, individual interview helps families to better understand the proposal of participating in the groups. It is especially relevant when the programme is offered by a co-ordinating organization to families from other organizations. From the discussion groups, we acknowledged the importance of having the first presentation of the programme be given by a professional known to the family and then having a presentation (individual or collective) by the group facilitators. Typically, the referrer of the family or the professional who knows the family should make this first contact.

• •

  Group presentation meetings: These can be held after pre-selecting and contacting the families individually, but there are also cases in which an open meeting is held for all interested families. This strategy seems to show that the programme is on offer as another resource; in the centres that have many editions, the programme has “caught on,” and individualized recruitment is no longer necessary.

  • -

    For the first session, we called an open meeting at the school, and many families came. We explained the programme to them, gave a PowerPoint presentation, played a video of the families from last year with music, trying to motivate them, too.

  • -

    We usually present the programme, hold a joint meeting, and explain the schedule, what we are going to do, and to whom it is for. We give them the example of a family that has already come to the programme, or we even invite a family that has already done the programme to come.

• •

  Families who inform families: As a result of the gradual consolidation of the programme, a new recruitment strategy has appeared that consists of inviting families who have already completed the programme to come and explain their experience. The “experienced” families benefit because their sense of usefulness in helping other families strengthens their own family bonds, and the “new” families benefit because the visit helps them see the resource being offered more clearly.

  • -

    We also showed them a bit of the work from last year; a mother from last year came and talked about her experience so the people in the room could see people from their social environment, who had attended the programme.

  • -

    The truth is, it works, and it helps them. Besides, they also get asked about it on the street, what did they do last year, etc. They all know each other, these are neighbourhoods that are in touch with each other on the street, so they tell each other about it. I think it is also very important that they are in touch.

  • -

    The first year is always the most difficult for recruitment and acceptance, but after that, the same families [who participate] advertise the programme.

• •

  With regard to motivation and predisposition, one of the most notable aspects in selecting families is the level of motivation that they have to participate. The professionals consider the level of motivation and involvement of the families to be key factors to carry on the programme. Motivation is understood as not being static or a constituent part of the family but rather as a key element to take into account and stimulate in the recruitment phase.

  • -

    In our case, since we have already been doing workshops for 3 years, we were very clear that the families we were going to contact were going to be families whom we had contacted before, who are part of the association, and who had a reference and predisposition to be able to carry out a process of change and, above all, reinforce the positive parenting.

  • -

    To phrase it positively: We think that you are capable of changing things, that you want to change things, that this is definitely going to help you, that it will offer you tools that maybe you do not know about or that you use in different ways, etc. To give positive strength to everything we think they are capable of doing.

• •

  The child age plays an important role for group forming. The range age initially considered to form groups was children from 6 to 12 years old. However, since the 2012/13 edition of the programme (taking into account the recommendations from the previous evaluative report from 2011/12), the child groups have been formed to be more homogeneous in age. In the focal group facilitators recommended to form more homogeneous age groups (6–8 or 8–12 years old). Despite this recommendation, in some instances, it has been impossible to form these groupings, which has caused difficulties in adapting the activities for age groups with the broad range of 6–12 years of age.

• •

  Finally, the level at which the families function in the language were also used as an important factor to form groups. Faced with the linguistic impossibility of holding programme sessions in a normalized manner, in some groups, external resources have been used to overcome the language barrier, allowing families who could benefit from the content (but who do not know the language) to participate in a satisfactory manner.

  • -

    We have made the selection taking certain criteria into account but, above all, the language issue, so that they are able to understand each other since that limits us a lot.

  • -

    In our case, since all of the families were foreigners with little knowledge of Spanish, an educator native to that country participated.

Discussion

The present study examined the quality of the implementation, and its influence on the programme results, based on data drawn from the RCT design aimed at evaluating the first trial of the Learning together, growing up in family programme. Results showed that the quality of implementation was medium to high on average according to the facilitators’ appraisals. This is expected, since we have designed a training workshop directed towards all group facilitators before the programme begins. This training workshop is indispensable for group facilitators so that they can gain a deep understanding of the program's characteristics, the strategies and techniques to perform the activities and the best ways to be implemented (Lyon, Stirman, Kerns, & Bruns, 2011). The quality of the group facilitator (i.e., being stable during the programme; ensures confidentiality) and the organization of the programme (i.e., the inclusion of only high-risk families in a group has been avoided; the schedule and place most convenient to the groups have been sought) received the highest scores of the implementation factors. Medium scores were given to the coordination with other services (i.e., facilitators have collaborated with the professional team) and the individual support facilities offered to participants (i.e., there have been refreshments and snacks during the break, volunteers have been sought to support the programme). The relatively lower scores of implementation referred to the fidelity to the programme (i.e., fidelity with respect to content and methodology is adequate) and prior organization steps (i.e., the programme has been officially inaugurated, previous interviews have been conducted with the families to organize the groups). Results of the focus groups also confirmed that the prior steps were challenging and benefited from the professional experience accumulated from other editions of the programme. The analysis of the group discussions revealed the use of serial strategies directed at selecting participants, motivating them and forming the groups, three crucial steps that if not done properly they may produce high drop-out rates and interferences with the programme effectiveness.

The results on the changes in parenting dimensions showed that after the programme the permissive style have decreased and parental inductiveness have reliably increased as reported by the parents, resulting in a more adequate pattern of socialization styles (Maccoby, 2007; Rodrigo, Byrne, et al., 2013). Inductive parents are able to set clear, reasonable standards for responsible behaviour that are consistent with children's developing abilities, are firm in their enforcement, and provide explanations for their positions. They are also kind, warm, and responsive to children's needs and negotiate their expectations, whereas permissive parents respond affectively but they are not demanding. The authoritarian style did not show any significantly change what means that these parents still place high values on obedience to rules, and prevent the child from taking any initiative. However, authoritarian parenting is not so negative in at-risk environments where children are exposed to dangerous situations (Mason, Walker-Barnes, Tu, Simons, & Martinez-Arrue, 2004). Parents after the programme also increased their level of satisfaction and their perceived competence, both changes are crucial for positive parenting in at-risk psychosocial contexts (Barth, 2009; Jones & Prinz, 2005).

To what extent the quality of the implementation predict better programme results? This is a crucial question that needs to be answered to gain knowledge about the conditions affecting the effectiveness of the programme. Overall, higher quality in implementation predicted better programme results, as expected (Durlak & Dupre, 2008; Fixsen et al., 2005). Results showed that reductions in permissiveness reported after attending the programme were predicted by higher standards in fidelity, programme organization, facilities provided to the participants and positive qualities of the facilitator, explaining a 47% of the variance of change in permissiveness. Increases in inductiveness after the programme were predicted by higher levels of fidelity, facilities provided to the participants and positive qualities of the facilitator, accounting for 36% of variance of change in inductiveness. In the case of parental efficacy the key elements were programme organization and the quality of the facilitator, accounting for a 23% of the variance. Finally, a regular programme attendance, which is crucial for at-risk families (Baker et al., 2011) was predicted by practically all aspects of implementation, accounting for 28% of the variance. Converging evidence from the quantitative and qualitative results demonstrated that the participants’ reception of the programme at initial and continued attendance were not only the responsibility of the parents or parental figures but also depend on certain prior steps being taken for the service and the group facilitators to achieve a good level of attendance. As a limitation of this study we had no data on the participants’ motivations to abandon the programme to compare with the facilitators’ point of view in the focus groups. Also, the effects of implementation factors on the rest of programme outcomes and on long-term outcomes are unknown.

In conclusion, this study provides some important clues on the key elements of the implementation process that influence the programme outcomes examined. This is relevant since little is known about the most effective ways to implement the programmes in the real-world settings, especially in at-risk contexts, despite the increasing availability of evidence-based parenting programme. The results indicate that the effectiveness of the programme, measured by the changes in parental dimensions and the attendance rate, depended on the quality of the implementation: how to advertise the programme, to select and motivate the families and to form the groups, how to negotiate the objectives and deliver the contents of the programme rigorously and in a manner that is attractive and interesting to keep the attention and facilitate the reflection of parents and their children, how to train the facilitators, how to arrange the facilities for the participants, how to evaluate results in order to make decisions on solid and sure bases, and how to coordinate with other agencies. All these implementation factors are key elements for programme improvement that should be taken into account in future trials. The programme “Learning together, growing up with family” is widely used in Spain and with a high level of sustainability. Therefore, from these results we have possibilities to improve implementation and all this will be possible with the coordination of CaixaProinfancia who promoted the idea and encouraged the participation of organizations; with organizations that have taken on the challenge of carrying out an innovative group experience with families and with the group of university experts who have collaborated and knowledge on the part of design, implementation and evaluation.

Conflict of interest

The authors have no conflict of interest to declare.