You are here

  • Home
  • Archive
  • Volume 27, Issue 2
  • Framework for the synthesis of non-randomised studies and randomised controlled trials: a guidance on conducting a systematic review and meta-analysis for healthcare decision making

Article Text

Article menu
Evidence synthesis
General medicine
Framework for the synthesis of non-randomised studies and randomised controlled trials: a guidance on conducting a systematic review and meta-analysis for healthcare decision making
  1. Grammati Sarri1,
  2. Elisabetta Patorno2,
  3. Hongbo Yuan3,
  4. Jianfei (Jeff) Guo4,
  5. Dimitri Bennett5,
  6. Xuerong Wen6,
  7. Andrew R Zullo7,
  8. Joan Largent8,
  9. Mary Panaccio9,
  10. Mugdha Gokhale10,
  11. Daniela Claudia Moga11,
  12. M Sanni Ali12,13,14,
  13. Thomas P A Debray15,16
  1. 1 Real World Evidence Sciences, Visible Analytics Ltd, Oxford, UK
  2. 2 Division of Pharmacoepidemiology and Pharmacoeconomics, Dept. of Medicine, Brigham and Women’s Hospital and Harvard Medical School, Boston, Massachusetts, USA
  3. 3 Canadian Agency for Drugs and Technologies in Health (CADTH), Ottawa, Ontario, Canada
  4. 4 Department of Pharmacy Practice & Administrative Sciences, University of Cincinnati College of Pharmacy, Cincinnati, Ohio, USA
  5. 5 Takeda, Cambridge, Massachusetts, USA
  6. 6 Pharmacy Practice, College of Pharmacy, University of Rhode Island, Kingston, Rhode Island, USA
  7. 7 Health Services, Policy, and Practice, Brown University, Providence, Rhode Island, USA
  8. 8 Real-World Solutions, IQVIA, California, Colorado, USA
  9. 9 Epidemiology and Outcomes Research, Research Outcomes Innovations LLC, New York City, New York, USA
  10. 10 GlaxoSmithKline USA, Philadelphia, Pennsylvania, USA
  11. 11 University of Kentucky, Department of Pharmacy Practice and Science, Lexington, Kentucky, USA
  12. 12 NDORMS, Center for Statistics in Medicine, University of Oxford, Oxford, UK
  13. 13 Department of Non-communicable Disease Epidemiology, Faculty of Epidemiology and Population Health, London School of Hygiene and Tropical Medicine (LSHTM), London, UK
  14. 14 Department of Public Heath, Environments and Society, Faculty of Public Health and Policy, London School of Hygiene and Tropical Medicine (LSHTM), London, UK
  15. 15 Julius Center for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht, The Netherlands
  16. 16 Smart Data Analysis and Statistics, Utrecht, The Netherlands
  1. Correspondence to Dr Grammati Sarri, Visible Analytics, Oxford OX2 0DP, UK; grammati.sarri{at}visibleanalytics.co.uk

Abstract

Introduction: High-quality randomised controlled trials (RCTs) provide the most reliable evidence on the comparative efficacy of new medicines. However, non-randomised studies (NRS) are increasingly recognised as a source of insights into the real-world performance of novel therapeutic products, particularly when traditional RCTs are impractical or lack generalisability. This means there is a growing need for synthesising evidence from RCTs and NRS in healthcare decision making, particularly given recent developments such as innovative study designs, digital technologies and linked databases across countries. Crucially, however, no formal framework exists to guide the integration of these data types. Objectives and Methods: To address this gap, we used a mixed methods approach (review of existing guidance, methodological papers, Delphi survey) to develop guidance for researchers and healthcare decision-makers on when and how to best combine evidence from NRS and RCTs to improve transparency and build confidence in the resulting summary effect estimates. Results: Our framework comprises seven steps on guiding the integration and interpretation of evidence from NRS and RCTs and we offer recommendations on the most appropriate statistical approaches based on three main analytical scenarios in healthcare decision making (specifically, ‘high-bar evidence’ when RCTs are the preferred source of evidence, ‘medium,’ and ‘low’ when NRS is the main source of inference). Conclusion: Our framework augments existing guidance on assessing the quality of NRS and their compatibility with RCTs for evidence synthesis, while also highlighting potential challenges in implementing it. This manuscript received endorsement from the International Society for Pharmacoepidemiology.

  • evidence-based practice
  • health care economics and organisations
  • health services research

Data availability statement

N/A.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

https://doi.org/10.1136/bmjebm-2020-111493

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Data availability statement

    N/A.

    View Full Text

    Supplementary materials

    • Supplementary Data

      This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

    Footnotes

    • Collaborators Recommendations from the Working Group of the International Society for Pharmacoepidemiology (ISPE) Comparative Effectiveness Research Special Interest Group for the cross-design synthesis of evidence and endorsed by the ISPE Board of Directors.

    • Contributors All authors conceived and developed the framework described in this manuscript. GS and TPAD drafted the manuscript and responded to other authors’ comments. All authors reviewed, contributed to revisions and approved the final version of the manuscript.

    • Funding Funding to support this manuscript development was provided by ISPE (https://www.pharmacoepi.org/ISPE/assets/File/General/FINAL20Call20for20Manuscripts204-24-19.pdf).

    • Disclaimer This article reflects the views and opinions of the authors and does not necessarily represent the views of the organisations where they are employed.

    • Competing interests We have read and understood BMJ Evidence-Based Medicine policy on declaration of interests and declare the following interests: GS is employed by Visible Analytics, Ltd; DB is employed by Takeda; ARZ holds a grant from Sanofi Pasteur (direct to institution); MP owns stocks from Merck, Sanofi, and Johnson & Johnson; MG is employed by GSK; and TD is an advisor to pharma industry.

    • Provenance and peer review Not commissioned; externally peer reviewed.

    • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.