This study was approved by the Tasmanian Human Research Ethics Committee (Approval No. H0011792). Recruitment was via a newspaper advertisement for a “Happiness Training Program”. Inclusion criteria were being over 18 years of age and not being depressed, indicated by a score of above seven on the Hamilton Rating Scale for Depression (HRSD; Hamilton, 1960). At the outset of the study each participant was asked to provide basic demographic information and to complete a 60-minute semi-structured interview. Participants’ mean age was 45.8 years (SD=11.77) and ranged from 23 to 62 years. The majority of participants (73.3%) were women. After providing consent, participants were allocated to one of the six counterbalanced patterns of interventions (Table 1). Participants were sent one diary per week according to the intervention in their schedule. The diaries contained instructions on the particular intervention. At the end of each day, participants were required to complete the PANAS.

A power analysis based on Raudenbush and Liu's (2000) recommendations for cluster-randomised trials using Optimal Design (Spybrook et al., 2011) suggested that 12 sites (persons) with more than 64 units (days) are sufficient to detect small to medium-sized effects (δ=.29; (Sin & Lyubomirsky, 2009)) of the interventions (clusters) on individual (daily) measures with sufficient power (.8) at an alpha level of .05.

The PANAS (Watson et al., 1988) was used to measure the primary outcome, the affective facet of subjective well-being. Neither the AHI nor the CES-D used in the original study were designed for repeated daily use, whereas the PANAS is sensitive to short-term fluctuations in the affective component of subjective well-being (e.g., Brose, Voelkle, Lövdén, Lindenberger, & Schmiedek, 2015) and therefore more suitable for use in the current N-of-1 design. In the PANAS, respondents rate the extent to which they have experienced feelings and emotions such as “scared”, “inspired” and “hostile” every day using a 5-point scale from “very slightly or not at all” to “extremely”. Three scores were computed: A positive affect (PA) score, which is the mean score of the 10 positive emotion adjectives; a negative affect (NA) score, which is the mean score of the 10 negative emotion adjectives; and an affect balance score, calculated by subtracting the negative from the positive affect scores. As there is evidence that PA and NA are relatively independent dimensions, we considered PA and NA separately as well as examining the affect balance score (Crawford & Henry, 2004; Tellegen, Watson, & Clark, 1999).

This study used a counterbalanced N-of-1 design. Counterbalancing was achieved by having participants complete a predetermined sequence of interventions in the first half of the study and then complete the reverse sequence of the same interventions in the second half of the study (Tervo et al., 2003). The particular ordering of interventions is shown in Table 1. Participants were randomly allocated to one of the six counterbalanced patterns of interventions.

Three of the six PPIs used by Seligman et al. (2005) were used in this study.

Gratitude visit: Participants were asked to write and deliver personally a letter of appreciation to someone who had been kind to them, but who they had never properly thanked.

Three good things in life: Participants were instructed to write down three good things that happened each day, together with a causal explanation for each of these things.

Using signature strengths in a new way: After completing the Inventory of Character Strengths (Peterson, Park, & Seligman, 2005) at the beginning of the program and receiving their top five signature strengths, participants were asked to use one of these five over the week, in a new way for each day of the week.

Based on previous recommendations (Seligman et al., 2005), we also tested a combination of the Three Good Things in Life and Using Signature Strengths in a New Way exercises, in which participants were asked to complete both exercises.

The control condition consisted of a week with daily affect assessments only.

In order to account for the hierarchical structure of the data (daily measurements nested within participants), multilevel analyses were performed using the lme4 package for R (Bates, Maechler, Bolker, & Walker, 2014). Multilevel analyses allow decomposing the variance of the dependent variables (repeated daily assessments of affect balance scores, PA score and NA score) into within- and between-person variance. The proportion of the total variance in PANAS scores accounted for by between-person (level-2) variance is represented by the intraclass correlation coefficient (ICC). Substantial ICCs (> .05) indicate that the data is structured in multiple levels (Snijders & Boskers, 2012).

Both the affect balance score and the PA and NA subscores (level-1 dependent variable) can be decomposed as follows:

Here, Yti represents a PANAS score of a measurement occasion t (level-1) within the level-2-unit (participant) i. Yti is regressed on the level-1-variables TIMEti (time indicated by study day), PPIti (dummy-coded intervention group), and the interaction of time and intervention group TIME*PPIti with the regression coefficients π1i, π2i, and π3i and a level-1 residual eti.

On level 2 (participant), both the intercept and the regression coefficients can be decomposed into mean levels and individual differences from this mean:

This implies that the mean intercept π0i (across all participants) can be decomposed into a mean intercept β00 on level-2 and individual differences from this mean r0i. Similarly, for all regression coefficients on level-1, πni, a mean coefficient βn0, and individual differences from this mean coefficient rni, can be estimated to account for individual differences between participants.

The level-1-predictors (time, intervention, time x intervention) were group-mean-centered. First, a null model containing the intercept only was analyzed to test for substantial ICCs of the primary outcome variables (affect balance score, PA score, and NA score). Second, a model using all level-1 predictors but ignoring the multilevel structure was fitted to obtain a baseline model. Thirdly, a model with random intercepts was used to examine whether there were differences in the intercepts of the level-1 outcome variables according to the level-2 units (in our case, basically whether there were between-individuals differences in the within-individual means of the outcome variables). Fourth, we examined whether the effects of the interventions differed between persons in a random slopes model in which we allowed the slopes of the intervention in predicting the outcomes to vary between persons. The difference in fit between model 2 (ignoring the multilevel structure), random intercepts and random slopes models was tested using the–2 Log-likelihood (–2LL) test (Snijders & Boskers, 2012). The study was not powered to detect cross-level interactions. This process of analysis was analogously repeated for all primary outcome variables (affect balance score, PA score and NA score).

Model 2 (Table 2) found no significant effects of time, intervention, and time x intervention. Although the -2LL test suggested a significantly better fit for the random intercepts model (Model 3), (Δ-2LL=287.67, df=1, p<.01), indicating that there are substantial individual differences in the mean level of affect balance, patterns of results did not change. The -2LL test suggested a significantly better fit of the random slopes model where the slopes of the intervention were allowed to vary between participants (Model 4; Δ-2LL=6.48, df=1, p<.05). The residual variance of the slopes however only approached significance.

Model 2 (Table 3) found no significant effects of time, or intervention on PA. However, a significant effect for the interaction term of time and Using Signature Strengths in a New Way was found. The -2LL test suggested a significantly better fit for the random intercepts model (Model 3), (Δ-2LL=459.74, df=1, p<.01). Allowing for random intercepts, the effect of time within the Using Signature Strengths in a New Way intervention was significant at B=.01 p<.05. Figure 1 illustrates the differential changes between interventions over time.

Figure 1.

Means and 95% confidence intervals of positive affect over the 7 days in the intervention blocks.

(0.16MB).

This effect was further probed using simple slopes analyses at three time points (for illustrative purposes): 10, 40, and 70 days into the study. The slopes of signature strength in predicting PA increased from B=-.1 at 10 days over B=.2 at 40 days to 0.5 at 70 days, suggesting that the effects of Using Signature Strengths in A New Way increased with time (Figure 2). Comparing Model 3 to Model 4, the -2LL test suggested no significantly better fit of the random slopes model where the slopes of the intervention were allowed to vary between participants (Δ-2LL=1.18, df=1, n.s.).

Figure 2.

Interaction of Using Signature Strength in a New Way and time in predicting positive affect.

(0.06MB).

Model 2 (Table 4) found no significant effects of time, intervention or time*intervention on NA. Although the -2LL test suggested a significantly better fit for the random intercepts model (Model 3), (Δ-2LL=407.09, df=1, p<.01), the pattern of results did not change. This suggests that participants differ with regard to their mean NA scores, but that these scores are not different between interventions. The -2LL test suggested a significantly better fit of the random slopes model (Model 4; Δ-2LL=22.72, df=1, p<.01). The residual variance of the slopes was significant as well, suggesting that there might be differences between participants in the effects of the interventions on NA.

Apart from the small interaction effect between the Using Signature Strengths intervention and Time, our study found none of the effects of the PPIs suggested by Seligman et al. (2005). Specifically, the two interventions Gratitude Visit and Three Good things had no significant effects on positive affect, and all four interventions had no significant effects on negative affect or overall well-being as indicated by the affect balance score.

Apart from the obvious fact that our study used the PANAS, as opposed to the AHI, a further possible explanation why we found no effects of the PPIs on well-being may be due to differences between our sample and that of Seligman et al. (2005). Whereas they used a relatively well-educated, financially comfortable, mildly depressed, motivated to become happier sample recruited through the university website, our sample was a non-depressed Australian community sample who responded to a newspaper advertisement. It is possible that a subject-expectancy effect might have contributed to the considerable effect sizes of the interventions in Seligman et al.’s research and to the lack of effect in our study.

A direct replication (Mongrain & Anselmo-Matthews, 2012) of Seligman et al.’s (2005) research found that the PPIs increased happiness levels, albeit with much smaller effect sizes. Furthermore, in this study, the PPIs did not exceed the control condition in reducing depression levels. The results from our study, using a strict test of within-individual changes in affect, when considered together with previous replications (Mongrain & Anselmo-Matthews, 2012), suggest that the PPI effects need further replication before conclusions can be drawn either about their general effectiveness, their support of basic tenets of positive psychology (Fernández-Ríos & Novo, 2012), or about their use in evidence-based practice. The significant residual variance in the slopes of the interventions in our study suggest that the PPIs might affect people differentially, which suggests more research into moderators.

Seligman et al. (2005) suggest that the Using Signature Strengths in a New Way exercise is effective because participants should, with practice, improve in their ability to effectively implement the exercise and will become more inclined to keep using the exercise. However, this describes nothing more than a positive feedback loop (i.e., the intervention keeps working because it already worked). Moreover, Seligman et al. provide this same ‘explanation’ for why the Three Good Things in Life exercise is effective in their original study, yet our study found no significant effects for the Three Good Things in Life exercise. As long as no clear theoretical framework identifying potential mediators of the effects (or lack thereof) of these interventions is provided, it is very difficult to speculate about the effective ingredients in the interventions (Michie & Abraham, 2004).

Another possible explanation for the small but significant increase in positive affect of the Using Signature Strengths in a New Way intervention is that there exists great variety in how to implement it. According to the instructions, participants have the opportunity to choose from five signature strengths, and the specific implementation of the signature strength is left to the participants. This extent of freedom in implementing the exercise might itself have led to increases in well-being, as previous theory and research has suggested that being able to choose and implement paths of action is associated with increases in well-being (Fredrickson, 2008).

Our study assigned participants to various tasks for 9-10 weeks, whereas Seligman et al. (Seligman et al., 2005) relied on self-selected adherence to interventions over time. A meta-analysis of positive psychology interventions (Sin & Lyubomirsky, 2009) showed that self-selected individuals benefited more from the interventions than individuals who were assigned a task. This suggests that interventions with a good person-activity fit might have more substantial effects on happiness (Giannopoulos & Vella-Brodrick, 2011; Schueller & Parks, 2014). However, if PPIs are to be implemented as evidence-based practice (Rashid & Seligman, 2013; Seligman et al., 2006) and recommended as self-help tools, their effectiveness should not rely on the self-selection of participants.