How to improve the integrity of clinical trial articles

Scandalous news is sometimes published about the integrity of medical research. Thus in 2017 The New England Journal of Medicine published a letter stating that in a clinical trial promoted by the United States National Institutes of Health (NIH), in 6 countries and published in the same journal in 2014, it was proven that 30% of the patients recruited in Russia had not received the drug being studied.¹ This fact led some people to ask themselves whether the samples of the drug in question had been sold in the country's black market, as something must have happened to them.² Although news like this is very striking, the best-known problem regarding the (lack of) integrity in clinical trials arises in how their results are communicated. This is more important than particular details of how trials are conducted, as usually only the promoters or main researchers are aware of these aspects. The problem in question here is the “distorted publication” or “selective reporting” of the results of clinical trials³: reporting the most interesting results (usually the “positive” ones that have attained statistical significance), while not publishing some of the results that, although they were said to be relevant in the protocol, had not attained such significance. This is especially important in mental health clinical trials, given that they have certain well-known particularities. These include the use of psychometric scales (which are therefore subject to a certain degree of subjectivity) as the main variables for measuring efficacy.⁴

To publish results in the more than 400 journals⁵ that belong to or have accepted the rules of the International Committee of Medical Journal Editors, clinical trials have to be registered in an open-access database (or registry) before the first participant is recruited. The registry with the largest number of clinical trials is the NIH clinicaltrials.gov, which contained more than 210,000 trials in January 2018. Registering trials prospectively is the best way of ensuring that results are published in an unbiased way.⁶ Comparison of the variables and analysis published in papers with the ones included by researchers in the registry makes it possible to know if there are discrepancies (omissions, additions or changes) between what they said they were going to evaluate and what they eventually said they had evaluated. This therefore makes it possible to know whether a paper contains a distortion in its publication of results. Some recently published studies show a discouraging state of affairs. Thus only 14% of the trials published by the five most important psychiatry journals were registered prospectively and did not selectively communicate their results.⁷ 85% of the clinical trials which evaluated antipsychotic drugs showed discrepancies between what they published and what had been registered.⁸ The situation is even worse in psychotherapy, as 95% of the trials published in the five highest-impact clinical psychology journals had publication distortion.⁹ Lastly, 63% of papers made no mention at all of severe side effects, and when they did so,¹⁰ 49% had discrepancies between the papers and registrations.¹¹ Regarding the mortality rate during clinical trials of antidepressant and antipsychotic drugs, cases of death and suicide were only reported in a minority of the papers included in registries (38% and 47%, respectively).¹¹

However, do these discrepancies have any clinical relevance in terms of care? This is what Becker et al.¹² found in 6% of 96 papers published in journals with an impact factor of at least 10. 55% of these papers were in fact published in The New England Journal of Medicine, The Lancet or the JAMA), and discrepancies in the main variable of efficacy were so great that they altered the interpretation of results. Although this percentage is striking, it could be expected to be even higher in papers published in journals with a less strict editorial policy.

Readers of medical journals have the right to expect that the information published will be true, exact and complete. To this end journals have to apply quality controls to prevent (or minimise) selective communication of results. Measures which are based on the responsibility of authors are not enough.¹³ Quality control should consist of individuals independent of the authors, such as reviewers or the editorial team, checking the variables and analyses in manuscripts against those shown in the registry for each trial, or in the trial protocol, which is usually not available.¹⁴ There are two problems here, as the majority of reviewers understand that this is not something they have to do,¹⁵ and moreover the majority of journals do not even ask them to do it.¹⁶ It may therefore be concluded that medical journals do not consider the selective reporting of the results of clinical trials to be a serious problem that they should correct.¹⁷

Resolving the selective reporting distortion will have to be based on the commitment of everyone involved: promoters and financers, the hospitals where trials take place, researchers/authors and journals. There can be no doubt that over the long term the best solution is to train all of the healthcare professionals involved in clinical trials in the values of scientific integrity.¹⁸ However, that would take a long time to produce results. Over the short to medium term the best way to prevent the proliferation of papers containing publication distortion would be to implement a preventive measure: to check, during the editorial process, that what the authors say in their paper corresponds to the information in the registry. It has to be taken into account that discrepancies which may arise may be scientifically necessary; in this case the authors have to state this and give reasons for it in their paper.¹⁹ Given that editorial teams are unable to do this work, it has been suggested that mixed committees of clinical research experts and students do it, similar to the team that undertook the COMPare²⁰ project in the United Kingdom on the same subject. It has been calculated that from 150 to 200 teams would be necessary to do this checking work. They would be specialised in different therapeutic areas and belong to universities, research centres and scientific associations throughout the world.¹⁹ These teams would work in the same way as external reviewers do now, interacting directly with the editorial teams of journals. The results of checking papers against their registries would be published as a supplement to the paper, so that readers would know if there were any discrepancies and, if there were any, the reasons for this¹⁹ which have sometimes been required during the editorial process.²¹

It is possible that many readers will consider that this proposal would complicate editorial reviewing by involving another agent to the process. Nevertheless, it is hard to imagine another solution over the short- to medium term when the credibility of clinical trials is questioned together with the correct interpretation of their results by clinicians and patients. On the other hand, similar measures have been applied in other areas of knowledge due to similar situations. For example, it is known that 4% of scientific papers contain duplicated images.²² Because of this The Journal of Biological Chemistry, which belongs to the American Society for Biochemistry and Molecular Biology, which is more than one hundred years old—and following the steps taken by The Journal of Cell Biology—, decided in 2017 to implement a preventive measure as a quality control: it contracted three people for its editorial team who, among other functions, examine the images contained in papers to detect any alteration or manipulation.²³ A tool has in fact already been proposed to aid this work.²⁴ If some journals have implemented measures like this to prevent the publication of duplicated images—a problem that affects 4% of images and which is of questionable and even minor relevance for public health—, what should medical journals be expected to do to prevent the publication of clinical trials that, in at least 6% of cases, may alter how their results are interpreted and therefore influence clinical decisions?