Participation was initiated by families who contacted the Yale Parenting Center, an outpatient treatment service for children and families. After self-referral, caregivers of the referred child completed an initial evaluation to assess child, parent, and family functioning and then began treatment. Informed consent was solicited and obtained from all families who completed treatment. The study met all university approved and government required ethical standards for the conduct of research and protection of patient privacy. To be included in this study symptoms of oppositional defiant or conduct disorder needed to be the primary basis for referral. These symptoms were formally assessed at diagnostic screening. In addition, the children were required to be between the ages of 6-13 years old. The study included 138 children (39 girls and 99 boys) and their caregivers who completed treatment.

Not all participants who met screening criteria and began treatment, completed the program (see Figure 1). Of the sample that began treatment (N=164), 26 (15.9%) of the families dropped out before completing treatment. The drop-out rates were no different as a function of treatment condition or therapist (χ2 for each, < 1.0, ns) and did not differ from completed subjects on demographic variables or measures at pretest designed to evaluate child and parent functioning (see Kazdin, 2017a for our research on dropouts vs completers). In the present study, dropouts were replaced. That is, new cases were randomly assigned to conditions to achieve a minimal number of participants who completed treatment.

Figure 1.

Flow of participants through screening, randomization, and treatment.

(0.41MB).

The 138 children included in the study ranged in age from 6 to 13 years (M = 8.59, SD = 1.79); 73.2% of the children were European American, 13.0% were African American, 2.9% were Asian American, 7.2% multiracial, and 2.9% identified as other race, based on parent identification of race and ethnicity. 13% of the children identified as Hispanic. Diagnoses of the children were obtained from the Research Diagnostic Interview (Kazdin, Siegel, & Bass, 1992, a structured parent interview that assesses the presence, absence, and duration of child symptoms using criteria of the Diagnostic and Statistical Manual of Mental Disorders (4th ed. TR, American Psychiatric Association, 2000). Reliability of diagnoses, assessed by independent observers, for approximately 10% of randomly selected children was high (kappa =.95 across all diagnoses).

Principal diagnoses of the children were oppositional defiant disorder (55.1%) and conduct disorder (44.4%). Most children (92.8%) met criteria for more than one disorder (M = 2.96 disorders, SD = 1.18). The primary caretakers of the children included biological (73.2%), adoptive or foster mothers (9.4%), biological (8.0%) or adoptive (3.6) fathers, grandmothers (4.3%) or other relatives (1.4%). iPrimary caregivers ranged in age from 29 to 71 years (M =43.07, SD = 7.31); 23.9% of the children came from single-parent families. Median monthly household income was US $6,001 to 8,000 (range: < $2,000 to > $16,000); 3.0% of the families received social assistance.

The overall goals of assessment were to evaluate five domains: treatment outcome, emergent processes during treatment (therapeutic alliance, perceived barriers), evaluation of the treatment manipulation in generating expectancies, treatment integrity and adherence, and treatment acceptability. Assessments included multiple assessment formats (e.g., clinical interviews, parent-report questionnaires) and informants (i.e., parents and therapists). Measures were completed at three points, before treatment began, halfway through treatment, and after treatment ended (pre-, mid-, and posttreatment, respectively). Most parent- and therapist-report measures were collected online using a secure web-based survey program protected for privacy with unique links for each client.

Treatment outcome. To evaluate child outcomes, multiple measures of symptom and adaptive functioning were included. First, to assess a broad range of both internalizing and externalizing symptoms, parents completed the Child Behavior Checklist (CBCL; Achenbach, 1991). The total problem score was evaluated to assess severity of dysfunction across a broad range of symptom domains. The total social competence scale was evaluated to assess participation in activities (e.g., athletics, clubs), social interactions (e.g., number of friends, amount of contact with friends), and academic performance. Multiple forms of reliability and validity of the CBCL have been studied extensively in clinic and non-clinic samples (e.g., Achenbach, 1991). Second, parents completed the Interview for Antisocial Behavior (IAB; Kazdin & Esveldt-Dawson, 1986), a semi-structured parent-report form that measures multiple overt (e.g., fighting) and covert (e.g., lying) antisocial behavior of the child. This measure reflects primary symptoms for which children are referred to the clinic (i.e., conduct problems). Each of the 30 items is rated on a 5-point scale for severity of dysfunction (1 = not a problem at all, 5 = very much a problem) and a 3-point scale for duration (1 = recent or new problem [6 months], 3 = always). Total antisocial behavior is obtained by summing severity and duration scores. Internal consistency, convergent and discriminant validity, and overall construct validity have been supported in multiple studies (e.g., Kazdin & Esveldt-Dawson, 1986; Kazdin et al., 1992). Third, the therapist responsible for the case completed the Child Global Assessment Scale (CGAS; Shaffer et al., 1983) to provide an overall measure of impairment and adaptive functioning. This measure summarizes the child's psychiatric, adaptive, and social functioning in everyday life and consists of a single score between 1 and 100 (with a higher number indicating better functioning). High interrater reliability and concurrent and discriminant validity have been demonstrated for the CGAS (e.g., Bird, Canino, Rubio-Stipec, & Ribera, 1987). Finally, we counted the total number of symptoms present across all diagnoses at pre- and posttreatment. This number represents DSM-IV-TR symptoms for which there was significant impairment in everyday life and in the present sample correlates with (but is distinguishable from) number of diagnoses (r(138) = .76, p < .001).

Therapeutic changes of the parent and the family were assessed by measures that focused on parent depression, perceived stress, parenting practices, and family relationships. First, parents completed the Beck Depression Inventory (BDI; Beck, Steer, & Garbin, 1996) before and at the end of treatment. For each of 21 items, the parent selected one of several statements that best described their experience of a depression symptom; a higher score on the measure indicates more symptoms and greater severity of depression. The psychometric properties of the BDI have been studied extensively (e.g., Dozois, Dobson, & Ahnberg, 1998). Second, parent perceptions of stress were assessed through the Parenting Stress Index (PSI; Abidin, 1995; Lloyd & Abidin, 1985), which was completed before and at the end of treatment. The PSI consists of 120 items, most rated on a 5-point scale, that reflect multiple areas of stress related to the parents’ views of their own functioning. The measure assesses perceived sources of stress, delineates perceived stress from life events, and distinguishes sources of stress from the child (e.g., subscales such as adaptability, demandingness, and child mood) and sources of stress related to the parent functioning (e.g., subscales such as restrictions of role, social isolation, and relations with others). Diverse types of reliability and validity for the PSI have been reported (Abidin, 1995). Finally, a shortened version of the Family Environment Scale (FES; Moos, 1990) was completed by parents before and at the end of treatment. The three subscales of the FES relationship domain (cohesion, expressiveness, and conflict) were used in this study. Parents responded to 27 true-false items that assess quality of interpersonal relationships, support, and family functioning. The FES has been studied extensively, and multiple types of reliability and validity have been demonstrated, as reviewed in the prior citation.

Emergent processes during treatment. To measure the alliance, we administered the Working Alliance Inventory to both parents and therapists. The scale includes 36 items rated on a 7-point scale (1 = never; 7 = always) and yields a total score to reflect a higher quality of the therapeutic alliance (Horvath & Greenberg, 1989). Items focus on the therapist–parent agreement on the tasks and their relevance in therapy; the mutual endorsement of the goals or outcomes of treatment; and the extent to which there is a positive personal attachment, acceptance, and confidence in the relationship. To evaluate perceived barriers to participation in treatment, parents and therapists independently completed the Barriers to Treatment Participation Scale (Kazdin, Holland, Crowley, & Breton, 1997). The scale consists of 44 items rated on a 5-point scale (1 = never a problem, 5 = very often a problem) that relate to stressors and obstacles that interfere with participating in and coming to treatment, treatment demands and issues, perceived relevance of treatment, and parent relationship with the therapist. Prior validation work has shown that parents given similar treatments can perceive the barriers associated with those treatments quite differently (Kazdin, 2000; Kazdin et al., 1997).

Reactions to and evaluation of treatment. We assessed the extent the manipulations designed to distinguish the two treatment conditions and included expectancies and credibility of the treatment. First, we assessed Parent Expectations for Treatment before treatment began and midway through treatment. This scale consists of 17 items, rated on a five-point Likert scale, most from strongly agree to strongly disagree. Questions assessed parents’ belief that the presented treatment was valid, worthwhile and would improve a variety of behavioral difficulties in their child. We used a total score (Cronbach's alpha = .94) to assess total expectations for treatment, a higher score indicating higher expectations for improvements in therapy. Second, parents completed the Credibility/Expectancy Scale. This 6-item measure was adapted from a frequently used measure to evaluate the extent to which a given intervention generates positive expectancies for change (Devilly & Borkovec, 2000). Parents rate items on an anchored scale from one to nine. A summary score is derived where a higher score indicates a higher believed credibility about the likely impact of treatment (Cronbach's alpha =.91). Credibility was assessed at pretreatment assessment and midway through the treatment intervention.

Treatment acceptability. Treatment acceptability refers to the extent to which consumers of treatment (e.g., parents, mental health professionals) view the treatment procedures as reasonable, justified, fair, and palatable (Kazdin, 2000). We evaluated acceptability to ensure that overall the treatments were viewed positively and to explore whether the two versions of treatment differed in acceptability. Parents and therapists evaluated the acceptability of treatment and the progress made in treatment through the Parent Evaluation Inventory (PEI) and the Therapist Evaluation Inventory (TEI), respectively (Kazdin et al., 1992). These measures each include 19 items and assess the extent to which the treatment was viewed positively by the parents. Parents rated their own perception of the treatment, and therapists rated the parents’ perception of treatment acceptability.

Two therapists (women, European American) participated. Each had a master's degree and training in family treatment or counseling. The therapists had 18 and 23 years of experience administering PMT to families and in training and supervision of other mental health professionals in the treatment. PMT was the intervention provided to all families included in the study. For treatment, the parents are the clients and attend individual sessions with an assigned therapist; they practice and develop several skills and apply these in the home to change child behavior. The focus is on developing concrete behavior changes that emphasize antecedents, behaviors, and consequences. Practice, modeling, role playing, feedback, shaping, and specific behavior-change programs for use outside of the session were used to develop concrete parenting skills (e.g., use of multiple antecedents, ways of crafting behavior, and consequences) (see Kazdin, 2009). We and others have shown that improvement in parenting skills are reflected directly in therapeutic changes of the children (e.g., Eddy & Chamberlain, 2000; Henderson, Rowe, Dakof, Hawes, & Liddle, 2009).

The treatment included a core set of sessions to convey central content areas, themes, and skill areas but were individualized to address child dysfunction at home, at school, and in special family circumstances (e.g., living conditions, custody issues, involvement of extended family members). Treatment lasted for approximately six weekly sessions (weekly basis, administered individually) plus the option of up to two additional session as needed. The treatment manual for the intervention and a review of several controlled trials of the treatment are presented elsewhere (Kazdin, 2009, 2017a).

Parents were randomly assigned to receive the standard treatment, as highlighted previously, or standard treatment with several enhancements. The enhancements, highlighted in Table 1 were designed to increase credibility and expectations for therapeutic change and to improve the effectiveness of treatment. The manipulation included efforts to increase the professional statue of the setting (special certificates and credentials of the therapists in the rooms for the enhanced treatment), more pleasant accoutrements in the session (e.g., fresh flowers in the treatment room), an orientation video by the clinic director touting the favorable outcomes in the past, testimonials, a special binder with materials about the treatment and its effects, a sham ‘delivery’ interruption to the therapist from an important client during a session, a mailed book on PMT written by the clinic director along with a welcome letter, and more attractive and color materials throughout the treatment program (e.g., appointment reminders, handouts, and charts). During treatment, a variety of special expectancy statements, special emails, and or handwritten notes were interspersed that were designed to increase expectations for improvement and bonding between therapist and client.

To maintain the integrity of treatment, (a) therapists followed a treatment manual; (b) materials (checklists, notes, and outlines) were used within each session to foster correct execution of the treatment; (c) postsession documentation summarized what transpired; (d) videotaping of all treatment sessions, some of which were reviewed weekly to provide feedback to the therapists; (e) review of randomly selected sessions by independent observers who rated the integrity of standard and enhanced PMT and (f) weekly case progress and challenge reviews to evaluate the clinical status of parent and child functioning while also maintaining high treatment integrity and adherence to the treatment manual.

In addition, we evaluated the extent to which parents implemented the intervention techniques in the home. To do this, we administered the Treatment Adherence Inventory (TAI) separately to both parents and therapists at the end of treatment. The TAI included 16 items, with each question rated on 5-point scale (1 = almost never, 5 = almost always), and that evaluated how well or how often parents used certain skills. A higher total score indicates a greater level of adherence to the treatment method. Second, we evaluated treatment integrity from the perspective of therapists’ implementation of the treatment. Sessions were videotaped and randomly selected to assess whether in-session enhancements and treatment components were completed as indicated. Across both therapists and all rated sessions, therapist integrity to the treatment method was 99% across 206 (23.5%) of the sessionsiii.

Preliminary analyses using t-and chi-square tests for continuous and categorical variables, respectively, revealed no differences between the two intervention groups in demographic variables including child or parent age, child or parent race or ethnicity, parental marital or employment status, and educational level or family income or in pretreatment clinical characteristics. Also, there were no pretreatment differences among child outcome measures or parent outcome measures, as described previously. There were no therapist differences in relation to the child or parent and family treatment outcome measures, either at pretreatment or at posttreatment. Overall, at the beginning of treatment, no differences on characteristics of the groups that we measured were evident between treatment conditions or therapists.

There were no differences between standard and enhanced PMT conditions in parents’ report of adherence (t(136) =1.64, p = .10) or therapist (t(127) = < 1.00) evaluations of parent adherence. Combining both groups, the means on the measures for both parent and therapist versions (M = 67.34 and 64.53, respectively, were above 64, which was the cutoff for good adherence on the measure (4s on all 16 items of the 5-point scale, where a 4 indicates a skill was used ‘most of the time’) indicating good adherence to the PMT procedures characterized both groups. For the enhanced treatment, 97.2% of the cases met criteria for correct execution of enhancements. For the standard treatment, 100% of the cases met criteria for standard treatment, i.e., they did not receive the described enhancements.

Measures were included to check on the extent to which the manipulations designed to distinguish the two treatment conditions were reflected in critical facets of the manipulation. Two other measures (expectancy, credibility) were administered to parents before intake and later again at midtreatment (after most manipulations were completed but there was still the opportunity for further change). Separate analyses of covariance for each measure indicated no significant differences between the two treatment groups before intake when those measures were first administered or at midtreatment when they were administered on a second occasion. Over the course of treatment, parents in both conditions (and not differentially) increased significantly in their expectations for change and viewed the treatment as much more credible than they did before the treatment began. At the second assessment, both standard and enhanced PMT expectations and treatment credibility were very high in absolute values on the scale. The data were negatively skewed (bunched toward the high end of the scale) indicating high levels of expectancies and credibility for both treatments.

Child treatment outcomes. A series of analyses were conducted to determine if children improved over the course of treatment. Within-group t-tests were computed for the entire sample (N =138) to evaluate these changes over time (i.e., from pre- to posttreatment). The means for pretreatment and posttreatment scores on child, parent, and family outcome measures are presented in Table 2 for the sample. The t-tests, also in Table 2, show that children improved over the course of treatment, as reflected in significantly lower total problems (CBCL), improved social competence (CBCL), reduced aggressive and antisocial behavior (IAB), improved overall adaptive functioning (CGAS), and fewer psychiatric symptoms (RDI) (all ps < .001). The magnitude of the improvements (effect size, calculated by d), as noted in Table 2, was large for all but one measure, using Cohen's (1988) criteria for such designations, as noted in the table.

Additional analyses evaluated if there were differences in these outcome measures for the two treatment groups (i.e., standard versus enhanced PMT). A series of analyses of covariance (ANCOVAs) were run for each of the child outcome measures, with treatment group as the main factor and pretreatment scores for measures as the covariate. The means for pre- and posttreatment scores on child outcome measures are presented in Table 3 for each treatment group along with F-ratios (for the group effect). As evident in the table, no differences were found at posttreatment between the two treatment conditions. Overall, the consistent finding is improvements for both groups but no support for one version of treatment surpassing the effects of the other.

A critical question is the extent to which the treatment produced clinically important changes. Although there is no standardized way to assess clinical significance in outcome research, one means is to evaluate the extent to which treatments brought child behavior within the nonclinical (normative) range of functioning (Kazdin, 2017b). We drew on data from the well-studied CBCL, mentioned previously, which provides cutoff scores that fall above and below a normative range, as devised from comparisons of clinic and non-referred samples separately for boys and girls (e.g., Achenbach, 1991). We examined whether children placed within the normative range for both the total symptom score and the total social competence score. For these analyses, we excluded children who began treatment in the nonclinical range or borderline range. Of the remaining cases, we examined the proportion in each treatment condition that fell within the normative range at posttreatment. For the total behavior problem scale, there was no difference in the proportion of children who fell within the nonclinical range for standard and enhanced PMT (χ2 < 1.00). Clearly, there was the equivalent of a ceiling effect: 97.6% of children fell within the normative range in the total sample at posttreatment. Similarly, the total social competence scale showed no significant group difference in the proportion of children in who fell within the normative range at posttreatment (χ2 = 1.62, p >.20). Here again, across both conditions 89.4% of the children fell within the normative range at posttreatment. Overall, there were no group differences and, in many ways, differences would be difficult to detect if present. The treatment, provided to each group, whether enhanced or not, appeared to produce effects that placed most of the cases within the normative range.

Parent and family outcomes. We also evaluated outcomes related to parent and family functioning. As with the child outcome data, within-group t-tests were computed for the entire sample (N = 138) to evaluate these measures changes over time (i.e., from pre- to posttreatment) (see Table 2). By the end of treatment, parents reported significantly fewer symptoms of depression, reduced stress levels, and improved relationships among family members (all ps < .001). Effect sizes (d), also in the table, reflect large effects for two of the three parent and family measures.

To evaluate differences in parent and family outcomes based on treatment group, ANCOVAs were conducted, parallel to those reported in the evaluation of child outcomes. In each case, treatment group (standard versus enhanced PMT) was evaluated with pretreatment on a given measure as the covariate. The results (summarized in Table 3) show no differences between the two treatment groups on depression, stress, or family relationships at the end of treatment.

Using 2 x 2 analyses of variance (treatment group x therapist), we evaluated the therapeutic alliance and perceived barriers to treatment. There was no main effects of treatment nor interactions for either the therapeutic alliance or perceived barriers. Overall, the two emergent processes in treatment did not vary as a function of treatment group.

Parents and therapists completed their respective versions of the acceptability measure. Of primary interest was whether standard and enhanced PMT were viewed differently in relation to treatment acceptability. Yet, the two treatment were no different on either parent (t(135) = 1.38. p < .20) or therapist ratings of acceptability (t(< 1.0). The distribution of the scores for both parent and therapist ratings of acceptability were negatively skewed (-.89 and -1.06, respectively), indicating that ratings were “bunched up” toward the high end of the scale. Thus, parents in both groups tended to evaluate the treatment as highly acceptable and as such it would be difficult to discriminate among the treatments (ceiling effect) if, in fact, there were a different reaction in treatment acceptability.