Reproducibility and replicability of systematic reviews

Abstract

Irreproducibility of research causes a major concern in academia. This concern affects all study designs regardless of scientific fields. Without testing the reproducibility and replicability it is almost impossible to repeat the research and to gain the same or similar results. In addition, irreproducibility limits the translation of research findings into practice where the same results are expected. To find the solutions, the Interacademy Partnership for Health gathered academics from established networks of science, medicine and engineering around a table to introduce seven strategies that can enhance the reproducibility: pre-registration, open methods, open data, collaboration, automation, reporting guidelines, and post-publication reviews. The current editorial discusses the generalisability and practicality of these strategies to systematic reviews and claims that systematic reviews have even a greater potential than other research designs to lead the movement toward the reproducibility of research. Moreover, I discuss the potential of reproducibility, on the other hand, to upgrade the systematic review from review to research. Furthermore, there are references to the successful and ongoing practices from collaborative efforts around the world to encourage the systematic reviewers, the journal editors and publishers, the organizations linked to evidence synthesis, and the funders and policy makers to facilitate this movement and to gain the public trust in research.

Keywords: Systematic review; Meta-analysis; Reproducibility of results; Automation; Data science; Data anonymization; Datasets; Guideline adherence; Guideline; Peer-review

Core tip: Reproducibility increases the practicality of the research findings and gains the public trust in research. The ongoing developments in automation of systematic reviews, availability of pre-registration platform, dealing more with secondary data or anonymized primary data, the collaboration culture among the organizations who produce systematic reviews, and finally having an update step that mandates replicability are all reasons that systematic reviews have the potential to lead the movement toward the reproducibility among the other research designs. Meanwhile, reproducibility can help the systematic reviews to be considered as research design rather than literature review.