A method for comparing correlations based on confidence intervals: Contributions to Tejedor et al.

Dear Editor,

The scientific community in the United States of America and Europe has paid a lot of attention recently to the creation of psychometric instruments to measure different aspects of mindfulness.¹ Within this context, the validation study of the Spanish version of the Philadelphia Mindfulness Scale (PHLMS) by Tejedor et al.² is relevant as it is one of the first instruments for evaluating the concept of mindfulness validated in Spanish. The process of gaining evidence for reported converging validity based on the significance of the P values of the correlations of PHLMS scores with those of other instruments that evaluate the same construct was justified conceptually. Nevertheless, the conclusions deriving from the said comparisons are solely based on descriptive consideration which restrict the exactitude and magnitude of the estimated effects.

This manuscript therefore has the purpose of filling this methodological gap by performing a re-analysis with a clearer procedure for comparing independent correlations, based on the interval of confidence (IC) of the difference.³ Given the considerations of this method, if the IC of a difference includes the zero, evidence will be obtained in favour of the null hypothesis of the statistical equality of the correlations (H₀: r_i − r_j = 0). The mathematical expression is:

where I and S are the lower and upper limits of the IC of the difference between the correlations. r₁ is the first correlation coefficient. r₂ is the second correlation coefficient. i₁ and s₁ are the lower and upper limits of the IC of r₁. i₂ and s₂ are the lower and upper limits of the IC of r₂.

This type of analysis is important as it avoids the conclusions deriving from the apparent overlap between the IC of the correlations studied, and it provides more information than the comparison of the classic hypothesis (P < .05) as it provides the range of possible parameter values (r_i − r_j) for a certain level of confidence.

The magnitude of the effect is also calculated for the difference in correlations (Cohen's q)⁴ according to the following criteria: 0.10, a small difference; 0.30, a moderate difference; 0.50 a large difference.⁴ The mathematical expression is:

where Zr1 and Zr2: Fisher's transformation of the first and second correlation coefficient, respectively.

Taking into consideration the data reported in Table 3 of the study by Tejedor et al. the IC for each correlation were calculated, as well as for the difference between correlations of the magnitude of the effect. The conclusions derived from the findings with the proposed method differ from those supported by the said authors. The manuscript underlines the correlation between PHLMS-awareness and FFMQ-non-reactivity in the psychiatric sample; nevertheless, the results indicate that there are only statistically significant differences, and that with a large magnitude of the effect between groups respecting the correlation between PHLMS-awareness and FFMQ-not judging (Table 1), and this result was not analysed in greater detail.

Table 1.

Comparison of correlations for the Philadelphia Mindfulness Scale-awareness.

	Student sample (n = 83)		Psychiatric sample (n = 208)		Differences
	r1	IC 95%	r2	IC 95%	r1 − r2 IC 95%	q
FFMQ-observation	0.446	0.255; 0.604	0.499	0.389; 0.595	−0.267; 0.139	0.043
FFMQ-description	0.562	0.394; 0.694	0.496	0.386; 0.592	−0.127; 0.238	0.050
FFMQ- conscious action	0.362	0.159; 0.536	0.304	0.175; 0.423	−0.177; 0.274	0.052
FFMQ-not judging	0.281	0.070; 0.468	0.026	−0.110; 0.161	0.003; 0.487	0.383
FFMQ-no reactivity	0.219	0.003; 0.415	0.458	0.343; 0.559	−0.477; 0.012	0.213
MAAS	0.230	0.015; 0.424	0.381	0.258; 0.492	−0.392; 0.079	0.138

FFMQ: Five Facet Mindfulness Questionnaire; IC: interval of confidence; MAAS: Mindful Attention Awareness Scales; q: Cohen's q; r₁ and r₂: observed correlations; in bold type: intervals of confidence that do not include zero.

On the other hand, there are only differences in the correlation of PHLMS-acceptation with FFMQ-observation and FFMQ-description between both samples (Table 2). This partially coincides with the report by Tejedor et al., which states that in the psychiatric sample PHLMS-acceptation correlates significantly with FFMQ-conscious action and FFMQ-observation, and with a moderate magnitude of the effect; and that in the sample of students the only significant correlation was between PHLMS-acceptation and FFMQ-description.

Table 2.

Comparison of correlations for Philadelphia Mindfulness Scale-acceptation.

	Student sample (n = 83)		Psychiatric sample (n = 208)		Differences
	r1	IC 95%	r2	IC 95%	r1 − r2 IC 95%	q
FFMQ-observation	0.154	−0.063; 0.358	−0.250	−0.373; 0.118	0.149; 0.642	0.398
FFMQ-description	0.543	0.371; 0.679	0.136	−0.004; 0.267	0.191; 0.599	0.360
FFMQ-conscious action	−0.051	−0.264; 0.167	0.361	0.237; 0.474	−0.653; 0.161	0.397
FFMQ-not judging	0.592	0.431; 0.716	0.462	0.348; 0.563	−0.060; 0.299	0.100
FFMQ-no reactivity	0.095	−0.123; 0.304	0.003	−0.133; 0.139	−0.165; 0.342	0.092
MAAS	0.236	0.021; 0.430	0.271	0.140; 0.393	−0.282; 0.199	0.033

FFMQ: Five Facet Mindfulness Questionnaire; IC: interval of confidence; MAAS: Mindful Attention Awareness Scales; q: Cohen's q; r₁ and r₂: observed correlations; in bold type: intervals of confidence that do not include zero.

Finally, Tejedor et al.² conclude that the differences between the correlations PHLMS-acceptation and PHLMS-awareness in the student sample (r_u = −0.050) and the psychiatric sample (r_p = −0.218) seem to be different, but when the said differences are analysed statistically they are not found to be statistically significant and present a trivial effect magnitude (IC_dif = −0.083; 0.419; q = 0.165).

We conclude our remarks by restating the importance of using types of statistical analysis that offer proper empirical support for conclusions deriving from the observed data. The lack of scientific rigour in the process of validating psychometric inventories means that any argument that is not properly supported may lead to poor use of the instrument, to the detriment of the user.