How to limit bias in quasiexperimental studies

Dear Editor,

In the abridged original entitled “Dual prophylaxis with teicoplanin added to cefazolin in the prevention of prosthetic joint infection”,¹ the authors try to demonstrate in a retrospective work the effectiveness of this regimen in the reduction of prosthetic joint infections caused by gram-positive cocci in primary and elective surgery of hip and knee arthroplasty. We would like to thank the authors for their contribution, and, while it is true that prophylaxis with beta-lactams plus teicoplanin could be considered in some situations, there is no quality evidence in the literature that supports the use of dual prophylaxis.^2–8 The authors recognise the limitations of their work as it is not a randomised study. However, in order to evaluate the effectiveness of an intervention in a quasi-experimental study, it is desirable, in addition to the usual bivariate analyses, to take into account other factors that have not been considered and which are frequent with before–after studies, such as mean regression, maturation effect and confounding variables. Regression to the mean is a phenomenon whereby when results are at their extreme points, they are most likely about to start the way back to a midpoint and, therefore, the change could have occurred without the intervention. The maturation effect is another phenomenon whereby the results obtained are due to changes that patients experience over time or seasonal cycles. To reduce the probability of these two phenomena occurring, it is necessary to observe the trend of the outcome variable before the intervention, and to make a longer observation after the intervention or the change of regimen to verify that there is neither a tendency to the mean nor a maturation effect. Confounding factors are frequent in all before-after studies, especially if the work is retrospective. In the work in question, a logistic regression would have made it possible to control various confounding factors, such as the Charlson index and transfusions, which were significantly more frequent in the control group, or colonisation by methicillin-resistant Staphylococcus aureus which, although without significant differences, occurred in twice as many patients in the control group. In this type of work, it is also desirable to reflect the adherence to preventive measures that have been proven to be effective in preventing surgical infection such as the decolonisation of S. aureus carriers, the adequate preparation of patients (hygiene, shaving, disinfection of the skin), the control of perioperative blood glucose, adherence to antibiotic prophylaxis used or changes in surgical practice (drainage, etc.). The “standard” logistic regression in this type of study does not estimate the trend or the slope of the changes after an intervention, so the results obtained may be biased and the changes in time (trend) may not be detected. Segmented regression techniques make it possible to estimate the association between an intervention and the outcome variable controlling for confounding factors, and estimate the changes in the mean at different levels (interception) and trends (slopes). The limitations of these statistics are that they require data from multiple time intervals before and after the intervention (≥10 observations/model and parameter to be studied) to avoid over-adjustments, but it is possible to have ≥24 observations (e.g. 12 months before and 12 months after the intervention). Potential stationary changes can even be detected.

Quasi-experimental studies are widely used to observe the impact of certain interventions in the prevention and treatment of osteoarticular infection due to the difficulty of conducting randomised clinical trials, so it is important that they are carried out with an appropriate methodology. Otherwise, biased results may lead to inappropriate clinical practice. We encourage all researchers to take these concepts into account when designing and analysing the results of a quasi-experimental work. We believe that the results of this study may encourage conducting a randomised trial, but should not lead to a change in clinical practice.