Effectiveness of Acceptance and Commitment Therapy compared to CBT+: Preliminary results

Abstract

Background and objectives

Acceptance and Commitment Therapy (ACT) is a young psychotherapeutic approach. It expands traditional Cognitive Behavioral Therapy (CBT) especially by mindfulness and valued living. Available research findings indicate ACT to be generally efficacious compared to control conditions. Even though ACT is increasingly applied in clinical practice, only little research has been conducted in this field. Hence, to examine the effectiveness of ACT, the present study aims to compare treatment outcome of group ACT and group CBT+, which consists of CBT and Interpersonal Psychotherapy (IPT) sessions, within a naturalistic setting.

Methods

Sixty-seven inpatients of a German psychiatric department were assigned to either ACT or CBT+ condition and assessed with respect to symptom measure as well as ACT-specific outcomes.

Results

Both groups improved on measures of symptom severity as well as ACT-specific components. There were no significant between-group differences.

Conclusion

Results indicate that ACT appears to show comparable results as CBT+. Findings are discussed in terms of to what extend outcomes of ACT and CBT+ are distinct and which possibly influencing factors could be considered in future research and treatment.

Introduction

Acceptance and Commitment Therapy (ACT) is a relatively new psychotherapeutic approach and is considered as a part of the so-called “third wave” in behavior therapy.¹ ACT seeks to strengthen six psychological skills: acceptance, diffusion, contact with the present moment, self-as-context, values and committed action.² The first three components can be seen as acceptance and mindfulness skills and the latter three aspects can be described as commitment and behavior change skills.³ The core processes of ACT pursue the greater goal to increase psychological flexibility.² Therefore ACT, as a transdiagnostic treatment approach, is considered to be used for the treatment of a broad variety of mental disorders.

In a meta-analytic review Öst¹ has found ACT to be superior to wait-list and TAU. However, he considered the methodological quality of ACT studies as insufficient and therefore not comparable to Cognitive Behavioral Therapy (CBT) studies.

The results of a meta-analytic review by Powers, Zum Vörde Sive Vörding and Emmelkamp⁴ indicate a similar pattern as presented by Öst¹: ACT could also be found as superior compared to control condition, but not significantly more effective than established treatments (e.g., CBT). Öst⁵ published an update on the efficacy of ACT himself and states ACT to be a not yet well-established treatment.

So far only a few studies have examined the effectiveness of ACT. Based on an outpatient sample, ACT and Cognitive Therapy (CT) have been compared with regard to their effectiveness on depression and anxiety.⁶ The results led to the conclusion, that there are no differences between the outcomes of the two treatments.

Only very few studies are available on the outcome of ACT as treatment for psychiatric inpatients. The results of an examination of inpatients with psychotic symptoms indicated greater improvements on primary outcomes in favor of ACT compared to extended TAU (ETAU).⁷

In sum, there seems to be a general agreement that previous results indicate ACT to be superior to control conditions and TAU. Nevertheless, a few aspects relating to ACT remain unclear. Most notably are divergent results with regard to the comparison of ACT and established treatments, especially CBT. Moreover, most studies have used RCTs to investigate the efficacy of ACT. Only a few studies have focused on psychiatric inpatients.^7,8 Following the theory of ACT, symptom reduction is not the only factor, which should be addressed.⁹ Referring to Hayes¹⁰ most research lacks to investigate more ACT specific variables, such as mindfulness or life satisfaction.

Present study

We present data drawn from a quasi-randomized effectiveness trial of patients from a psychiatric inpatient department. The aim of this study was to contribute to the clarification of the effectiveness of ACT as a transdiagnostic approach compared to established disorder specific treatments for depression within a naturalistic setting. The psychiatric ward focused on the treatment of patients with a primary mood disorder. Patients received either ACT or CBT+, where CBT treatment was supplemented by Interpersonal Psychotherapy (IPT) within a group setting. Like CBT, IPT is a validated effective treatment for depression¹¹ and both treatment approaches are well established in practice and endorsed in clinical guidelines.^11,12 Thus the use of both can be assumed as a strict control condition.

As dependent variables, we used symptom-related assessment tools as well as a measurement of patients’ life functioning, which is by concept a main goal of ACT.¹⁰ Furthermore, we added ACT-specific questionnaires to examine to what extend ACT and CBT+ patients improve in ACT specific skills. We focus on two important factors: mindfulness and valued living.

Following the idea that ACT is a third wave enhancement of CBT, we expected ACT to be more effective than CBT+. Therefore we tested group differences and within-differences (pre- and post comparisons) for all symptom-related measures. We expected a clinical significant admission-to-discharge symptom reduction for both groups. Furthermore, due to the paradigm of psychological flexibility, we hypothesized a significantly lower perceived impairment of life functioning at post treatment for the ACT group. Additionally, we expected the ACT group to develop significantly higher levels of mindfulness and valued living than patients in the CBT+ group.

Method

The study was conducted within a naturalistic setting as a pre- and post treatment design and confirmed by the ethics committee of the Institute of Psychology of Humboldt-University Berlin. All clients were recruited, assessed and treated by a German Clinic Evangelisches Krankenhaus Königin Elisabeth Herzberge. Group allocation was realized in terms of quasi-randomization as delineated below.

Sample

For this study a total of 67 patients were recruited from the psychiatric unit for mood disorders as primary diagnosis and allocated to either ACT or CBT+ group. Patients treated in this hospital ward were either referred by the emergency ward or were sent electively. The allocation to the therapy groups was carried out as follows: Included participants were assigned quasi-randomly to the ACT and CBT+ group by a nurse. Quasi-randomly means the allocation of each patient was dependent on the capacity of either group. We chose this quasi-random allocation procedure to ensure immediate treatment service for all patients. Patients’ diagnoses were based on the ICD-10 symptom rating (ISR)¹³ and expert rating of the assigned therapist.

Cross-tabulations with χ²-tests and unpaired t-test showed no significant differences between the two groups on sociodemographic characteristics except for work status, χ² (1, n = 67) = 4.68, p < .05 (Table 1). Likewise cross-tabulations with χ²-tests and unpaired t-test showed no significant differences between the two groups on clinical characteristics except for neurotic, stress-related and somatoform disorders (ICD-10: F40-F48)¹⁴χ² (1, n = 67) = 13.53, p < .001 (Table 2).

Table 1.

Sociodemographic characteristics of patients of CBT+ and ACT group.

Characteristic	CBT+ (n = 28)		ACT (n = 39)
Gender^a^,^d
Male	8	(28.60)	14	(35.90)
Female	19	(67.90)	25	(64.10)
Other^b	1	(3.60)	0	(0.00)
Age^c^,^d	41.68	(12.97)	38.76	(14.89)
IQ^c^,^d	100.64	(6.70)	98.51	(6.71)
Immigration background^a^,^d	3	(10.70)	3	(7.70)
Work status^a^,^*
Student/apprentice/employed	21	(75.00)	19	(48.70)
Retiree/unemployed	7	(25.00)	20	(51.30)
Education^a^,^d
College degree/university degree	10	(35.70)	11	(28.20)
Other	18	(64.30)	28	(71.80)

a

Values shown as absolute frequency (%).

b

e.g. transgender or other cases not meeting either male or female category.

c

Values shown as absolute mean (SD).

d

Non-significant.

*

p < .05.

Table 2.

Clinical characteristics of CBT+ and ACT group.

Characteristic	CBT+ (n = 28)		ACT (n = 39)
Days of hospitalization^a^,^e	33.64	(11.33)	35.77	(10.32)
Persistence of symptoms^b^,^e
≤12 months	14	(50.00)	19	(48.70)
>12 months	14	(50.00)	20	(51.30)
Pretreatments^b^,^e
None	10	(35.70)	10	(25.60)
One or more	18	(64.30)	29	(74.30)
Diagnoses^b^,^c^,^d
F10–F19^e	4	(14.30)	12	(30.80)
F20–F29^e	1	(3.60)	–
F30–F39^e	25	(89.30)	29	(74.40)
F40–F48^***	4	(14.30)	23	(59.00)
F50–F59^e	–		2	(5.10)
F60–F69^e	5	(17.90)	11	(28.20)

a

Values shown as mean (SD).

b

Values shown as absolute frequency (%).

c

International Statistical Classification of Diseases and Related Health Problems (10th ed., rev.; ICD-10; World Health Organization, 2010).

d

More than one diagnose per patient possible.

e

Non-significant.

***

p < .001.

Participants were included if they (1) signed the written inform consent, (2) were older than 18 years old and (3) met ICD criteria (axis-I).

Participants were excluded, if they were (1) acutely suicidal, (2) mentally disabled, (3) suffering from dementia, (4) acutely psychotic, (5) overextended by the requirements of the study (e.g., lack of language competence), (6) addicted to any kind of substance, (7) under treatment for a crisis intervention or (8) if they had spent more than seven days in the clinic before pre-assessment due to clinical processes.

Treatment

ACT occurred as a group therapy twice a week for 50 min each. One of the weekly sessions was mindfulness training. For a standardized delivery of mindfulness skills, the German manuals for mindfulness training from Wolf-Arehult and Beckmann¹⁵ as well as from Huppertz¹⁶ were used. The second therapy session provided psychoeducation and focused on one of the other ACT components modularly to deliver all ACT core processes other than mindfulness. As there is no overall manual for different diagnoses so far, this treatment was conducted according to literature of Wengenroth¹⁷ and Harris¹⁸ (for a detailed description, please contact the corresponding author). Average group size was ten patients; group composition was changing because of the open, circular form. Average number of participated group sessions was 10.2 (SD = 2.95).

Similarly, CBT+ group therapy took place in a group setting twice a week for 50 min each. One session per week was interpersonal psychotherapy (IPT, manual according to Schramm¹⁹), the other session was CBT (manual according to Hautzinger and Kischkel²⁰). Average group size was ten patients; group composition was changing because of the open, circular form. Average number of participated group sessions was 9.6 (SD = 3.24).

A package of auxiliary therapeutic activities was offered to patients, primarily occupational therapy and psychoeducation.

Each group (ACT and CBT+) was offered by a female therapist. The ACT therapist was a psychological psychotherapist in training and had six months of specific training and clinical experience in providing ACT treatment groups. The CBT+ therapist was a medical resident trained in both, CBT and IPT. Inter- and supervision (peer consulting) was offered on a weekly base.

Measures

We composed a battery of self-report measures to assess change in specific domains. Ranging from 0 to 6 days (X¯=2.7, SD = 1.6) after inclusion and 0 to 7 days (X¯=1.9, SD = 1.3) before discharge, all clients received electronic versions of questionnaires in German language presented on a tablet computer.

Diagnoses

ICD-10 symptom rating (ISR)¹³ is a screening for psychological syndromes. It consists mainly of six clinical subscales: depression (Cronbach's α = .79), anxiety (Cronbach's α = .83), obsessive-compulsive symptoms (Cronbach's α = .75), somatoform symptoms (Cronbach's α = .86), eating disorder (Cronbach's α = .49) and other disorders (one item for each disorder, among others posttraumatic stress disorder).

Control variables

To control for a potential distortion of our results, we used the Wortschatztest (WST)²¹ as an indicator for IQ and to make sure that all questionnaires were understood. Internal consistency was satisfying, Cronbach's α = .90.

To control for the effect of medication between groups, we documented the medication for each person on the 10th day of stay and on the day of discharge. We have chosen the 10th day of stay because usually medicinal treatment should be adjusted until that day. Chi-square test showed no significant differences between the two groups in any category at any time of measurement.

Primary outcome measure

Beck Depression Inventory II (BDI II)²² was used to assess the severity of depression. Internal consistency was satisfying, Cronbach's α = .90.

Secondary outcome measures

In addition to our primary outcome, we used the following measurements as secondary outcome variables to determine ACT-specific effects. In order to measure mindfulness, we used the Freiburger Fragebogen zur Achtsamkeit Kurzform (FFA-KF),²³ which consists of 14 items. Internal consistency was satisfactory with Cronbach's α = .83. A German translation of the Valued Living Questionnaire (VLQ)²⁴ was used to measure the importance of valued areas of life and the extent to which one is living consistently with one's values by a single item format on a 10 point Likert scale for each area.

Statistical analyses

Kolmogorov–Smirnov test was used to test for normal distribution of data. To analyze changes in symptom severity, multiple repeated measure analyses of variance (MANOVA) and ANOVA was used for parametric variables. Post hoc t-tests were used for calculation of between-group differences at pre and post treatment and for within group differences from pre to post treatment. The calculation of clinical significant change is based on Jacobson and Truax.²⁵ According to their requirements, patients have to shift from a clinical to a nonclinical status, where improvement is quantified by the reliable change index (RCI). To determine a clinical to nonclinical shift, retest reliability and norms of the relevant measures were used. Differences in recovery rates were examined using cross-tabulations with χ²-tests. To define effect sizes either partial eta square (ηp2) or Cohen's d_(z), were used and rule of thumb was applied for interpretation: following Cohen²⁶ the terms small, moderate and large were used for effect sizes of 0.01, 0.06 and 0.14 in case of partial eta squared, 0.20, 0.50 and 0.80 in case of Cohen's d_(z), respectively.

Results

Participant flow

From 68 included patients, 67 completed the intervention (Fig. 1). The dropout (n = 1, 1.47%) appeared within the ACT group and since it was the only one, no systematic effects can be tested and it must be considered as random. Table 3 shows means and SDs for all measures from the two assessment occasions for both treatment groups.

Figure 1.

Participant flow for the total sample, divided by group.

Table 3.

Descriptive statistics of measurements for CBT+ and ACT group from pre to post.

Measure/time of measurement	CBT+ (n = 28)		ACT (n = 39)
BDI-II
Pre	33.79	(10.77)	31.67	(10.78)
Post	12.32	(8.58)	14.85	(11.50)
Life functioning
Pre	5.87	(1.79)	5.89	(2.27)
Post	4.06	(1.92)	4.63	(2.03)
FFA-KF
Pre	28.39	(6.93)	29.67	(6.60)
Post	34.32	(8.43)	34.79	(5.92)
VLQ
Pre	307.04	(199.46)	349.74	(186.98)
Post	374.36	(188.20)	456.28	(179.37)

Note. Values shown as mean (SD).

BDI-II: Beck Depression Inventory; FFA-KF: Freiburger Fragebogen zur Achtsamkeit; VLQ: Valued Living Questionnaire.

Primary outcome

A repeated measures ANOVA indicates significant main effects for time with large effect sizes for BDI-II, F(1, 65) = 307.56, p < .001, ηp2=.83. In case of the BDI-II a significant interaction effect for time × group with a small to moderate effect size was found, F(1, 65) = 4.53, p < .05, ηp2=.07. Between-group effects are not significant (Fig. 2).

Figure 2.

Means of measurement scores for each group and each time of measurement. Error bars represent standard errors.

Referring to the norms given in the BDI-II manual, results suggest that a total number of 41 patients reliably recovered with treatment, i.e. shift from BDI-II-score ≥14 to BDI-II-score <14 and RCI ≥1.96. Cross-tabulations with χ²-tests reveal no significant difference between treatment groups.

Reduction of impairment of life functioning is comparable in both groups. A repeated measure ANOVA indicates large main effects for time on perceived impairment of life functioning, F(1, 65) = 47.73, p < .001, ηp2=.42. Effect sizes for admission to discharge improvement are moderate to large for ACT condition and large for CBT+ condition (ACT: t(38) = 4.06, p < .001, d_z = .65; CBT+: t(27) = 6.12, p < .001, d_z = 1.16). Significant between-group differences do not reveal at any times of measurement (Fig. 2).

Secondary outcome

A repeated measure MANOVA including the independent variables time and group and the dependent variables mindfulness (FFA) and valued living (VLQ) was applied and indicated large significant main effect for time, F(1, 65) = 18.81, p < .001, ηp2=.22, but no main effect for group.

Contrary to our expectations an increase in mindfulness from pre to post treatment was not only detected for ACT but also for the CBT+ group (ACT: t(38) = −5.56, p < .001, d_z = −.89; CBT+: t(27) = −4.15, p < .001, d_z = −.79).

With respect to ACT group post hoc test shows a moderate to large effect regarding the increase of valued living, ACT: t(38) = −3.96, p < .001, d_z = −.63. Whereas the post hoc test results for CBT+ are only close to significance with a small to moderate effect, CBT+: t(27) = −2.04, p = .051, d_z = −.39 (Fig. 2).

Discussion

Primary outcome

We assumed a general improvement in patients’ wellbeing, which is reflected in a reduction of symptoms as well as an improvement of several domains in life functioning. We expected a greater general improvement for ACT group in comparison to CBT+ group.

Results show a significant decrease with a large effect for BDI-II scores as well as a significant clinical change for more than half of the patients. Contrary to our expectation, the mean improvement of participants in the ACT group is not significantly higher than for CBT+ group patients. Results indicate a comparable and substantial improvement of depressive symptoms for both groups as well as the number of individuals, who shifted from a pathological into a non-pathological range. Results are non-coincidental at both, group and individual level. In our study patients are considered as healthy, if the BDI-II score is lower than 15. According to the manual, a BDI-II score below 15 depicts a minimal depression and represents the lowest cut-off.²² From our point of view, it is advisable that patients require subsequent outpatient treatment after inpatient stay in the hospital. Hence we expect our set cut-off as conservative and representative.

In line with our hypothesis, a significant decrease in perceived impairment of life functioning has appeared. The results show similar effects for both treatments. We presume that, along with the symptom reduction, the offered treatments provoked a better well-being in different life areas for all patients. However, average effect was slightly greater for patients participating in CBT+ treatment.

Secondary outcome

With regard to mindfulness and valued living we supposed a significant improvement for ACT patients from admission to discharge.

As anticipated, mindfulness scores increased significantly for patients of the ACT group and a large effect size appeared. However, we could show similar effects for patients of the CBT+ group. These results indicate that not only patients of the ACT group, who explicitly perceived mindfulness training, improved their mindfulness skills, but also CBT+ patients’ mindfulness, which had not been trained specifically. There are some reasons one could consider this effect is due to the clinical setting: first, even though ACT and CBT+ group program was strictly separated, an overlap of both approaches within other therapeutic services (e.g. occupational therapy) cannot be ruled out completely. Secondly, ACT and CBT+ patients were accommodated at the same inpatient unit and therefore it is very likely that they exchanged their therapeutic experiences. Otherwise this might also be an indication for an indirect delivery of mindfulness skills by CBT+. Even if CBT+ does not comprise or name specific mindfulness intervention explicitly, mindfulness may be increased within the therapy process – but labeled differently. Another very important aspect, which may have led to the increasing mindfulness skills in CBT+ group are desirable response sets. This effect cannot be ruled out by any of our measures, but at least should be distributed equally in both groups due to quasi-randomization.

In case of the variable “valued living”, results show a comparable change-pattern. Again, patients of both treatment groups attained improved scores at post-test in this ACT specific measurement, showing a large effect size. However, only results for the ACT group show a significant improvement from pre to post. The changes in the CBT+ group, slightly missed the critical level of significance. Results illustrated a change in the same direction and almost to the same extend. We presume the same explanation as responsible for the observed increase in valued living for the CBT+ group as mentioned above.

For further investigations of processes of change for ACT and CBT+, more ACT and CBT+ specific measures should be added.

Limitations

One of the main goals of our study was to investigate differential effects of ACT and CBT+ primarily on depressive symptoms in an inpatient setting resulting in relatively high external validity. However, limitations are primarily related to the lower level of internal validity of the study.

Since we compared the effectiveness of ACT to a combination of CBT plus IPT, the effects of both (IPT, CBT) cannot be untangled. The participation in different therapy groups is common in inpatient treatments and therefore reflects the naturalistic study setting.

A randomized controlled trial was incompatible with the inpatient setting. Thus we decided to use a quasi-randomization to establish as equal conditions between the two treatment-groups as possible. However, the groups were parallel regarding clinically and methodologically relevant variables, especially diagnoses and symptom severity. This objective seems to be accomplished for most aspects. As described above, in two cases, distribution of data turned out to be uneven between ACT and CBT+ group.

What might also be criticized is the lack of a control group in the form of a treatment as usual (TAU) condition. Given that the results of the above mentioned research revealed neither TAU, nor waitlist control group effects and due to ethical concerns, we renounced the use of these kinds of control group.

Another critical aspect is the lack of a classical session-by-session therapy manual as a basis for the general ACT group sessions. Even though the ACT therapist delivered the therapeutic content based on ACT literature and in standardized manner, the comparability of the ACT sessions and CBT+ sessions could be distorted. Additionally, with the given resources, we were not able to systematically assess therapists’ competence and treatment adherence.

Also for further research larger sample sizes are recommended. In addition this would allow a more detailed statistical analysis. Interpreting dropout patterns a larger sample size could reveal more information.

In addition, two methodological limitations regarding the German version of VLQ are noteworthy. First, no back-translation of the German version was done. Hence it cannot be guaranteed that the German VLQ measures exactly the same as the original version of the VLQ. Secondly, the German assessment tool has not been validated in Germany yet and so far the exact reliability for the translated version is unknown. Therefore, the showed improvement in the variable of valued living for our sample has to be taken with caution.

Conclusion

In summary ACT and CBT+ appear to be effective interventions for treating inpatients with depression. Contrary to our expectations no superiority in favor of ACT could be detected. However, these findings are consistent with most results in the field of research, which has been done to compare ACT and established treatments to date.⁶

Regarding ACT-specific variables, as expected, greater mean increase for variables in mindfulness and valued living occurred for participants of the ACT treatment. Since “mindfulness” and “valued living” also increased in CBT+ group it is questionable, if this appears due to the discussed occasional blending of both approaches or due to the content of CBT, IPT or both. Further research could reveal inherent components of the therapeutic approaches and by this give insight in possibly similar active factors of ACT, CBT and IPT.

We conclude that – based on our results – the implementation of ACT in clinical practice can be justified. The results raise new issues that should be addressed in further research: as a transdiagnostic treatment approach, one could assume that ACT may be a more effective treatment for patients with comorbid disorders compared to diagnosis specific treatments. Instead of comparing both treatments, further research should focus on what ACT can add to established treatments.

Financial disclosure

There was no funding for this work.

Conflict of interest

The authors have no conflict of interest to declare.