Towards enhanced transparency and accuracy of scientific reporting in biomedical journals

I am very grateful to have this opportunity to share personal observations and to reflect on a subject which is a core element of the medical scientific enterprise, namely, accurate and transparent reporting of research. This article does not refer to intentional breaches of good publication practices, in the form of either scientific or ethical misconduct which, regrettably, has been observed with increasing frequency in recent years. The issues herein relate to incomplete or inaccurate reporting. The issues are not new, but nevertheless remain very topical and highly important, as, far too often, reporting limitations compromise many articles submitted to biomedical journals. Reporting oversights frequently become reasons for article rejection. We recognize that if a trial is never published, the results will not be disseminated and the study will in effect, not "exist". Mandatory clinical trials registration may be helpful to avoid publication bias. Nevertheless, without publication, a fundamental ethical principle relating to recruitment and randomization of patients into a clinical trial will have been breached. A related guiding principle in scientific publication is that, to achieve external validity, the results of an experiment (or randomized controlled trial) must be reproducible. Accordingly, the methods and results of a clinical trial should be described in sufficient detail such that a knowledge reader with access to the original data could replicate the results.

Medical editors usually base editorial decisions on three key elements: 1) the overall importance of an article (is it a good question that will ultimately impact clinical practice?); 2) the overall novelty (what is the incremental new knowledge of the study?); and 3) scientific merit of the study being reported (is the study design appropriate, and was the trial conducted properly?). Incomplete and/or inaccurate reporting makes it difficult for the editor, the reviewer, and ultimately the reader, to assess these fundamental aspects of the underlying clinical trial. Fortunately, valuable resources and guidelines to foster accurate and transparent reporting of clinical trials now exist and are freely available, although their user uptake across journals in the specialty of anesthesiology has been somewhat variable. The "Uniform Requirements for Manuscripts Submitted to Biomedical Journals", first published by the "Vancouver Group" in 1979, is regularly updated by the International Committee of Medical Journal Editors (ICMJE) (http://www.icmje.org). The most recent iteration (2010) provides an invaluable resource for authors and reviewers alike.¹ The ICMJE "Uniform Requirements" article details many important issues beyond basic reporting elements, including essential rules of authorship, and ethical principles that are shared and endorsed by many journals. These issues should be incorporated in editorial policy, and detailed and communicated in online journal-specific "Instructions for Authors". Journals that agree to use the Uniform Requirements are encouraged to state in their "Instructions to Authors" that their requirements are in accordance with the Uniform Requirements, and to cite the 2010 version.¹

Other invaluable resources include standardized reporting guidelines. Over 80 reporting guidelines have been developed in recent years. The most commonly cited reporting guideline is the CONSORT (CONsolidated Standards of Reporting Trials) Statement (http://www.consort-statement.org).² First published in 2000, and updated most recently in 2010, a key element of the CONSORT Statement is a standardized checklist of 25 reporting items. "The checklist includes the 25 items selected because empirical evidence indicates that not reporting the information is associated with biased estimates of treatment effect, or because the information is essential to judge the reliability or relevance of the findings."² A second key element of the CONSORT Statement is a flow diagram, which depicts the flow of patients through a study, from assessment screening for eligibility, to recruitment and intervention and follow-up. For smaller trials, a statement at the beginning of the methods section reporting this information may be adequate, whereas for larger trials involving larger patient cohorts, a flow diagram should ideally accompany the article. Equally important to the standardized checklist and flow diagram is the related elaboration document which justifies the rationale for each of the 25 reporting elements.³

For observational studies, the STROBE (STrengthening the Reporting of OBservational studies in Epidemiology) Statement (http://www.strobe-statement.org) should be considered for reporting cohort and case control studies.⁴ For reporting systematic reviews, our preference is the PRISMA (PReferred reporting Items for Systematic Reviews and Meta-Analyses) statement (http://www.prisma-statement.org).⁵ Many journals now endorse these and other reporting guidelines and in so doing, require the relevant reporting item checklist to be completed and uploaded at the time of article submission. Editors and reviewers can cross check the various reporting items according to the pages in the manuscript where each specific reporting item is addressed. In practice, these reporting guidelines are only as effective as the users who apply them, including the authors, reviewers, and the editors.

Appreciating that keeping abreast of the latest iterations of the commonly used reporting guidelines may be overwhelming, a new international initiative called the EQUATOR (Enhancing the QUality and Transparency Of Scientific Reporting) Network (http://www.equator-network.org) was recently established. The EQUATOR network is emerging as an important resource for authors, reviewers and editors. The EQUATOR website provides links to a host of valuable resources, including a library for health research reporting, and is highly recommended.

From personal observations related to manuscripts submitted to the Canadian Journal of Anesthesia, there are a number of common issues that are either incompletely addressed, or commonly overlooked. A recurring problem is that the rationale for many studies is not adequately justified. Frequently, authors fail to critically appraise the relevant literature on a given subject. This process must involve a detailed and comprehensive literature review, including a critical assessment of interval estimates (confidence intervals, not just P values) and any related systematic reviews. This process should, of course, have taken place at the protocol writing stage. The introduction of each article should end with a clear and unambiguous statement of purpose, framed around a validated primary outcome of interest, and based upon the underlying hypothesis. It is often at this stage of the article that blurring occurs, due to a failure to properly distinguish primary from secondary outcomes, and use of terminology that is vague. Without a clearly defined research question, any conclusions will, at best, be difficult to formulate, at worst - the conclusions may be potentially erroneous or misleading.

In the methods section, the exact method of patient randomization (eg. computer-based, or random numbers table) is frequently not specified. If this relatively simple issue has to be clarified in a revision, the initial author oversight will tend to undermine the credibility of the work. A key element of the randomization process is to conceal the sequence generation from the investigators, to reduce potential bias. This important element of study design is usually referred to as the method of allocation concealment. Examples include masking the group assignment in sequentially numbered sealed opaque envelopes. Whatever method used, it should be described transparently. If not undertaken and not reported, concerns for potential observer bias arise. The description of blinding procedures is frequently incomplete.

The statistical reporting in many manuscripts is frequently problematic. All too often, it becomes evident during editorial peer review that authors have not consulted with an experienced biostatistician at the study design phase, and later, during data analysis and interpretation. Authors should be aware that the editorial boards of many journals now include statistical editors, which we believe is gradually enhancing the rigor of peer review. The statistical reporting section of the Canadian Journal of Anesthesia was recently updated with the following recommendations. Whenever possible, quantify findings and present them with appropriate indicators of measurement error or uncertainty (such as confidence intervals). Avoid relying solely on statistical hypothesis testing, such as the use of P values, which fails to convey important information about effect size.⁶ References for the design of the study and statistical methods should be to standard works when possible (with pages stated). In reporting randomized controlled trials (RCTs), state exactly how the sample size was determined. If a formal sample size calculation was used in the design of the trial, provide all elements of the calculation. Do not calculate and report "post-hoc power". Power is a pre-study concept, useful in the design of a study, but it has no role after the data has been collected.⁷ A commonly overlooked item is the measure of variability of the primary outcome, an estimate of which should be identified in the sample size calculation.⁸ The results should be based on interpretation of the confidence interval, again, not just P values.

For observational studies, we prefer use of the term historical cohort study in lieu of retrospective cohort study when the cohort is identified and assembled in the past, on the basis of existing records or health care registries.⁶ It is important to include a clear and complete description of how and when data collection took place, and to describe any existing data sources that were used in the study such as administrative data or patient registries. We often observe confusion over whether a study is a double-cohort design or a case-control design. In a double-cohort study the two groups of subjects are sampled, based on the exposure of interest; whereas in a case-control study the two groups are sampled based on the outcome of interest.⁸

If a study has multiple outcomes a coherent strategy for dealing with these should be developed before starting the study⁹ and should be reported in the Methods section. Adjustments for multiplicity may be necessary.¹⁰ It is important to avoid using the word "trend" or "marginally significant" when referring to P-values that are near, but not below 0.05 (or whatever is the pre-specified Type 1 error). Null hypothesis significance tests (NHSTs) to compare baseline characteristics in randomized trials are not appropriate - we frequently request authors to remove the related P values during manuscript revision. Finally, many trials in anesthesiology report repeated measures of a number of variables (eg. heart rate and blood pressure) as a function of time. We caution that repeated use of significance tests at every time point should be avoided unless each time point is of interest in its own right.⁶ Otherwise, the treatment effect may become hyperinflated. Correction should be made for multiple testing. In most situations analysis of response profiles or linear mixed-effects models are the preferred methods of analysis for longitudinal data.¹¹ Once again, the importance of consulting with an experienced biostatistician at the study design and data analysis phases is crucial.

Another important aspect of transparent clinical trial reporting is to provide full disclosure of adverse events (AEs). Without sound AE reporting, it is impossible to provide an appropriate assessment of benefits versus harms of a given intervention or treatment. Even if a new drug or treatment has been shown to be clinically effective, if it is associated with common minor side effects (eg postoperative nausea), or rare but serious AEs (eg. perioperative stroke) that may become apparent only in larger Phase 4 surveillance studies, these side effects or AEs may be more important than any therapeutic advantage. The practicing clinician and ultimately, the patient, need to be duly informed. Many manuscripts submitted to biomedical journals fail to report on these issues completely, and adverse event reporting is regrettably suboptimal in many published studies.^12,13 It is the authors' responsibility to ensure accurate reporting of AEs.

In conclusion, due to inexperience, incomplete training or lack of good mentoring, many reports of clinical trials submitted to biomedical journals contain reporting errors or inconsistencies that make interpretation of the underlying research difficult to assess. There are considerable opportunities for improvement. Through better training in scientific writing, and increased adherence to validated standardized reporting guidelines, there are means and ways to enhance the accuracy and clarity of scientific reporting. Authors, reviewers, editors, as well as our university departments and faculties of medicine alike, all have responsibilities in this high-stakes process that influences clinical decision-making, and ultimately, the welfare of our patients.

Competing Interests

None declared.

Funding sources: None.