The PRISMA statement extension for systematic reviews incorporating network meta-analysis: PRISMA-NMA

Introduction

Systematic reviews incorporating meta-analysis of randomized clinical trials, when properly designed and implemented, can provide the best scientific evidence on the effect of health interventions. Systematic reviews with meta-analyses allow us to study the efficacy and safety of one treatment versus another with a high level of quality and scientific rigor in order to assist decision-making in healthcare. However, sometimes the presentation and description of some systematic reviews and meta-analyses are still not entirely clear or important information is still lacking, mainly in the categories of methods and results.¹

During the last decades, major initiatives have been undertaken in order to improve transparency, quality and consistency of the methodological information and results presented in systematic reviews and meta-analyses. Among them, the publication in 1999 of the statement Quality of Reporting of Meta-analyzes-QUOROM-² and its subsequent revision and expansion in the statement Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA).^3–5 The PRISMA statement is a guide on research publishing designed to improve the integrity of reporting systematic reviews and meta-analyses. Since its publication in 2009, authors and researchers around the world have used the PRISMA statement to plan, prepare and publish systematic reviews and meta-analyses. Dissemination and implementation of the PRISMA statement seems to indicate improvements in the quality of the publication of methods and results obtained from systematic reviews and meta-analyzes.⁶

Traditionally, systematic reviews and meta-analyses have focused primarily on assessing the efficacy or safety of a treatment's compared to a single comparator. In recent years, new methods have been developed such as network meta-analysis or NMA,^7,8 which, from a previous systematic review, it allows to compare multiple treatments simultaneously in situations where there are several alternatives that have been compared against a common comparator, providing estimates of the effects of each treatment relative to the others. Recently, deficiencies have been observed in the publication of systematic reviews assessing multiple treatments using NMA techniques.^9–11 Given the constant evolution of this kind of systematic reviews, more complex, if possible, when comparing multiple interventions (some of them only indirectly) the authors of these reviews are facing new challenges, including the full presentation of the methods applied and the results found. Also clinicians, researchers, health technology assessors, as well as peer reviewers and scientific magazine editors need guidelines to facilitate their understanding and correct interpretation of the information presented in systematic reviews incorporating NMA. To improve clarity, transparency and quality of this type of systematic reviews, the PRISMA extension statement for NMA was developed¹² (PRISMA-NMA), whose Spanish translation is presented in this Medicina Clínica article.

PRISMA-NMA checklist

Following the approach set out for the development of research publishing guidelines,^13,14 a group of experts participated in a systematic review,⁹ Delphi survey, discussions and a consensus meeting to establish new items to extend the PRISMA checklist and to clarify some existing items. As a result, PRISMA-NMA¹² consists of 32 items (Table 1), of which 5 are brand new (items S1–S5) and 11 are modifications of already existing items. The PRISMA-NMA extension incorporates new concepts and terminologies to areas of scientific evidence synthesis, systematic reviews and meta-analyses, such as the geometry of treatment networks (e.g., see items S1, S3 and S4 in Table 1), or the consideration and evaluation of transitivity and inconsistency assumptions (for example, see items S2, S5 and modification of item 25 in Table 1). Generally, transitivity is considered when different studies are comparable because they do not differ in the distribution of effect modifying factors (e.g. study design, assessed interventions, concomitant therapy, severity of patients, etc.); and inconsistency when there are disagreements between the effects of treatments derived from direct and indirect comparisons.^12,15 The lack of transitivity would cause inconsistencies between direct and indirect evidence, so the statistical combination in a NMA could provide effect estimates of doubtful usefulness to guide decision-making. As an important contribution, the description of the methods used to explore the treatment network structure and shape (geometry) can help establish the suitability of comparisons present in the studies incorporated and determine whether there is any evidence for any comparison. In this regard, authors can provide a treatment network graph of the studies and briefly discuss the characteristics of the network (e.g., the number of studies and the number of patients for each of the comparisons). For more information on the development of each of the 32 items, the rationale of its importance and practical examples of appropriate information extracted from NMA published in the biomedical literature, the explanation article on the PRISMA-NMA extension statement¹² should be consulted.

Table 1.

PRISMA-NMA checklist of items to include when reporting a systematic review involving a network meta-analysis.

Section/topic	Item #	Checklist item	Reported on page #
Title
Title	1	Identify the report as a systematic review incorporating a network meta-analysis (or related form of meta-analysis).
Abstract
Structured summary	2	Provide a structured summary including, as applicable: Background: main objectives Methods: data sources; study eligibility criteria, participants, and interventions; study appraisal; and synthesis methods, such as network meta-analysis. Results: number of studies and participants identified; summary estimates with corresponding confidence/credible intervals; treatment rankings may also be discussed. Authors may choose to summarize pairwise comparisons against a chosen treatment included in their analyses for brevity. Discussion/conclusions: limitations; conclusions and implications of findings. Other: primary source of funding; systematic review registration number with registry name.
Introduction
Rationale	3	Describe the rationale for the review in the context of what is already known, including mention of why a network meta-analysis has been conducted.
Objectives	4	Provide an explicit statement of questions being addressed, with reference to participants, interventions, comparisons, outcomes, and study design (PICOS).
Methods
Protocol and registration	5	Indicate whether a review protocol exists and if and where it can be accessed (e.g., Web address); and, if available, provide registration information, including registration number.
Eligibility criteria	6	Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics (e.g., years considered, language, publication status) used as criteria for eligibility, giving rationale. Clearly describe eligible treatments included in the treatment network, and note whether any have been clustered or merged into the same node (with justification).
Information sources	7	Describe all information sources (e.g., databases with dates of coverage, contact with study authors to identify additional studies) in the search and date last searched.
Search	8	Present full electronic search strategy for at least one database, including any limits used, such that it could be repeated.
Study selection	9	State the process for selecting studies (i.e., screening, eligibility, included in systematic review, and, if applicable, included in the meta-analysis).
Data collection process	10	Describe method of data extraction from reports (e.g., piloted forms, independently, in duplicate) and any processes for obtaining and confirming data from investigators.
Data items	11	List and define all variables for which data were sought (e.g., PICOS, funding sources) and any assumptions and simplifications made.
Geometry of the network	S1	Describe methods used to explore the geometry of the treatment network under study and potential biases related to it. This should include how the evidence base has been graphically summarized for presentation, and what characteristics were compiled and used to describe the evidence base to readers.
Risk of bias within individual studies	12	Describe methods used for assessing risk of bias of individual studies (including specification of whether this was done at the study or outcome level), and how this information is to be used in any data synthesis.
Summary measures	13	State the principal summary measures (e.g., risk ratio, difference in means). Also describe the use of additional summary measures assessed, such as treatment rankings and surface under the cumulative ranking curve (SUCRA) values, as well as modified approaches used to present summary findings from meta-analyses.
Planned methods of analysis	14	Describe the methods of handling data and combining results of studies for each network meta-analysis. This should include, but not be limited to: • Handling of multi-arm trials; • Selection of variance structure; • Selection of prior distributions in Bayesian analyses; and • Assessment of model fit.
Assessment of inconsistency	S2	Describe the statistical methods used to evaluate the agreement of direct and indirect evidence in the treatment network(s) studied. Describe efforts taken to address its presence when found.
Risk of bias across studies	15	Specify any assessment of risk of bias that may affect the cumulative evidence (e.g., publication bias, selective reporting within studies).
Additional analyses	16	Describe methods of additional analyses if done, indicating which were pre-specified. This may include, but not be limited to, the following: • Sensitivity or subgroup analyses; • Meta-regression analyses; • Alternative formulations of the treatment network; and • Use of alternative prior distributions for Bayesian analyses (if applicable).
Results^a
Study selection	17	Give numbers of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions at each stage, ideally with a flow diagram.
Presentation of network structure	S3	Provide a network graph of the included studies to enable visualization of the geometry of the treatment network.
Summary of network geometry	S4	Provide a brief overview of characteristics of the treatment network. This may include commentary on the abundance of trials and randomized patients for the different interventions and pairwise comparisons in the network, gaps of evidence in the treatment network, and potential biases reflected by the network structure.
Study characteristics	18	For each study, present characteristics for which data were extracted (e.g., study size, PICOS, follow-up period) and provide the citations.
Risk of bias within studies	19	Present data on risk of bias of each study and, if available, any outcome level assessment.
Results of individual studies	20	For all outcomes considered (benefits or harms), present, for each study: (1) simple summary data for each intervention group, and (2) effect estimates and confidence intervals. Modified approaches may be needed to deal with information from larger networks.
Synthesis of results	21	Present results of each meta-analysis done, including confidence/credible intervals. In larger networks, authors may focus on comparisons versus a particular comparator (e.g. placebo or standard care), with full findings presented in an appendix. League tables and forest plots may be considered to summarize pairwise comparisons. If additional summary measures were explored (such as treatment rankings), these should also be presented.
Exploration for inconsistency	S5	Describe results from investigations of inconsistency. This may include such information as measures of model fit to compare consistency and inconsistency models, P values from statistical tests, or summary of inconsistency estimates from different parts of the treatment network.
Risk of bias across studies	22	Present results of any assessment of risk of bias across studies for the evidence base being studied.
Results of additional analyses	23	Give results of additional analyses, if done (e.g., sensitivity or subgroup analyses, meta-regression analyses, alternative network geometries studied, alternative choice of prior distributions for Bayesian analyses, and so forth).
Discussion
Summary of evidence	24	Summarize the main findings, including the strength of evidence for each main outcome; consider their relevance to key groups (e.g., healthcare providers, users, and policy-makers).
Limitations	25	Discuss limitations at study and outcome level (e.g., risk of bias), and at review level (e.g., incomplete retrieval of identified research, reporting bias). Comment on the validity of the assumptions, such as transitivity and consistency. Comment on any concerns regarding network geometry (e.g., avoidance of certain comparisons).
Conclusions	26	Provide a general interpretation of the results in the context of other evidence, and implications for future research.
Funding
Funding	27	Describe sources of funding for the systematic review and other support (e.g., supply of data); role of funders for the systematic review. This should also include information regarding whether funding has been received from manufacturers of treatments in the network and/or whether some of the authors are content experts with professional conflicts of interest that could affect use of treatments in the network.

PICOS, population, intervention, comparators, outcomes, study design.

Text in italics indicates wording specific to reporting of network meta-analyses that has been added to guidance from the PRISMA statement.

a

Authors may wish to plan for use of appendices to present all relevant information in full detail for items in this section.

Translation process

The Spanish translation of PRISMA-NMA has been carried out by a process of translation and back-translation, following the policy established by the PRISMA group¹⁶ for official translations regarding research publishing guidelines. One of the authors of this article (FCL) prepared a first version in Spanish based on the original checklist (published in English).¹² Then, a translator specialized in health sciences and biomedicine (familiar with the terminology used) made the reverse translation (back translation) of the checklist into English, which was reviewed and approved by the other 2 authors of this article (BH and DM). Finally, a final version was agreed and approved, which aimed to use a neutral, concise and clear Spanish, understandable both in Spain and other Spanish-speaking countries in Central and South America. The 3 authors of this article (BH, FCL and DM) are, in turn, authors of the original in English.¹²

Final considerations

This article provides a Spanish translation of the PRISMA-NMA checklist. Other extensions have been developed^17–21 in recent years, including PRISMA for systematic review protocols (PRISMA-P)^17,18 and individual patient data meta-analysis (PRISMA-IPD).²¹ PRISMA-NMA extension is primarily aimed at researchers, authors and readers of systematic reviews that incorporate NMA as well as peer reviewers and editors of biomedical journals interested in this type of research. The reading of The PRISMA Extension Statement for Reporting of Systematic Reviews Incorporating Network Meta-analyses of Health Care Interventions: Checklist and Explanations¹² is recommended, as it can help answer questions about newly incorporated items or modifications of existing items. The explanatory document¹² can also help achieve a better understanding of the scientific rationale of the NMA, through teaching examples from the literature and presenting the description of specific terminology used in this type of systematic reviews. Where appropriate, we also recommend reading the explanation and elaboration document of the PRISMA statement.^3,4 For any revision or update of these documents it is also advised to consult the PRISMA statement websites (http://www.prisma-statement.org/) and the EQUATOR network – Enhancing the Quality and Transparency of Health Research (http://www.equator-network.org/), website also available in Spanish thanks to the collaboration of the Pan American Health Organization (http://www.equator-network.org/library/spanish-resources-recursos-en-espanol/).

Financing

BH has received funding from the Canadian Institutes of Health Research (New Investigator Award, Canadian Institutes of Health Research/Drug Safety and Effectiveness Network). FCL has received funding from the Alicia Koplowitz Foundation and the Generalitat Valenciana (PROMETEOII/2015/021). DM has received funding from the Research Chair at the University of Ottawa, Canada.

Conflict of interests

BH has received help from Amgen Canada and Cornerstone Research Group for methodological advice on network meta-analysis. The other authors declare no conflicts of interest.

Acknowledgements

The authors wish to express thanks to Sylvia Cambray for her support during the reverse translation process.