Article Text
PDF
Abstract
This paper, part of the Cochrane Rapid Review Methods Group series, offers guidance on determining when to conduct a rapid review (RR) instead of a full systematic review (SR). While both review types aim to comprehensively synthesise evidence, RRs, conducted within a shorter time frame of typically 6 months or less, involve streamlined methods to expedite the process. The decision to opt for an RR depends on the urgency of the research question, resource availability and the impact on decision outcomes. The paper categorises scenarios where RRs are appropriate, including urgent decision-making, informing guidelines, assessing new technologies and identifying evidence gaps. It also outlines instances when RRs may be inappropriate, cautioning against conducting them solely for ease, quick publication or only cost-saving motives.
When deciding on an RR, it is crucial to consider both conceptual and practical factors. These factors encompass the urgency of needing timely evidence, the consequences of waiting for a full SR, the potential risks associated with incomplete evidence, and the risk of not using synthesised evidence in decision-making, among other considerations. Key factors to weigh also include having a clearly defined need, a manageable scope and access to the necessary expertise. Overall, this paper aims to guide informed judgements about whether to choose an RR over an SR based on the specific research question and context. Researchers and decision-makers are encouraged to carefully weigh potential trade-offs when opting for RRs.
- Methods
- Systematic Reviews as Topic
- Clinical Decision-Making
Data availability statement
Data sharing not applicable as no datasets generated and/or analysed for this study.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
WHAT IS ALREADY KNOWN ON THIS TOPIC
Rapid reviews (RRs) are increasingly prevalent in the published literature due to their speed and efficiency in providing evidence synthesis compared with full systematic reviews (SRs). While methods guidance for conducting RRs exists, there is currently a lack of specific guidance on determining when it is appropriate to do an RR over an SR.
WHAT THIS STUDY ADDS
This paper outlines considerations for determining the appropriateness of conducting an RR. It emphasises the importance of context and the research question balanced against the backdrop of time-sensitive decision-making needs. It discusses suitable scenarios and potential limitations, and provides guiding questions for making a balanced assessment of appropriateness for RRs that is broadly applicable to RR producers and users.
HOW THIS STUDY MIGHT AFFECT RESEARCH, PRACTICE OR POLICY
This paper offers a framework for making informed judgements about whether to opt for RRs over an SR. Policy-makers, healthcare providers and researchers can use the provided considerations to determine when an RR is most viable and valuable. Organisations involved in producing or commissioning RRs can also apply the guiding questions to ensure the appropriateness of evidence synthesis for timely decision-making and relevant policy development.
Introduction
This paper is part of a Cochrane Rapid Reviews Methods Group series providing methodological guidance for rapid reviews (RRs).1–5 The main goal of this paper is to explain how to assess whether it is appropriate to conduct an RR instead of a full systematic review (SR).
Various literature review types exist, each serving distinct purposes based on the research question, available resources and timeline.6 SRs and RRs aim to answer a clearly defined research question and comprehensively review, synthesise and analyse existing evidence to make conclusions. However, an RR is typically conducted in a shorter time frame and is often a less resource-intensive way to synthesise evidence than a full SR.7 This is achieved by streamlining or accelerating certain steps of the review process.8 Generally, RRs are conducted over 6 months or less, a much shorter timeline than the 1–2 years needed for most SRs.9 But RRs should not be defined solely by the time taken to conduct the review, as other reviews may be conducted in short periods (eg, by assigning additional resources or if the review has zero or few included studies). Instead, RRs should be defined by the abbreviated methods used to reduce the time to completion.
The similarities in the methodologies between RRs and SRs raise the question of when an RR is appropriate to conduct for evidence synthesis. RRs have proven useful in both emergent (eg, COVID-19 pandemic, disaster relief)10–12 and non-emergent yet urgent situations where there is still a need for timely evidence (eg, to inform the development of a new health policy or programme)13–15 or in resource-limited environments (eg, low-income countries).16–19 However, there are cases where an RR may not be appropriate. For example, a full SR is likely preferable if the evidence synthesis will be used to make decisions or develop guidelines on a large scale (eg, international, regional), which could have wide-sweeping resource or implementation implications and if time allows to wait for evidence to inform a decision.
RRs present a challenge in maintaining rigorous methodology and ensuring the validity of findings within tight deadlines. RRs may be more biased and less reliable than well-conducted SRs due to time constraints, limited scope and potential biases introduced by the accelerated review process.20 21 The exact degree of bias remains to be determined. However, many published SRs are of low quality (ie, quality of conduct and/or reporting) and are susceptible to biases.22 Hence, the evidence synthesis type alone is insufficient to judge its reliability and quality. Regardless of which evidence synthesis approach is taken, it is essential to follow sound methodological guidance9 23 24 and ensure transparent reporting of methods.25
Several organisations produce or commission RRs within academia, government, research institutions and non-profit organisations to provide evidence for decision-making related to clinical care, healthcare funding, services, policy, technologies and programme development.26–29 Although methods guidance is available to support the conduct of RRs,9 24 30 there are no specific guiding principles on when it is appropriate to use an RR approach instead of an SR. This decision, which is often situational and involves several factors, is left to the discretion of the author teams that produce evidence syntheses and the organisations that commission them.
Cochrane, a global leader in producing high-quality SRs and methodological guidance, conducts RRs driven primarily by requests for timely evidence for decision-making and only for urgent and high-priority topics.9 It, too, needs more concrete criteria when making this determination. Therefore, this paper aims to outline considerations to support whether it is appropriate to undertake an RR. To replace arbitrary decision-making, this paper discusses the importance of considering the specific context surrounding an RR topic and the research question(s) to be addressed. Potential limitations are also discussed along with scenarios where RRs may be appropriate. Last, guiding considerations to help to make a balanced assessment are presented. Although examined within the context of Cochrane, an organisation that has been instrumental in leading the development of RR methods guidance,9 it is widely applicable to those who produce, request or use RRs.
When is it appropriate to do an RR?
In evidence synthesis, various approaches can be considered for quick action when time is limited. Decision-makers can rely on existing SRs that are up to date and relevant to the question at hand. If this is not available, another option is to update an existing SR by incorporating the latest evidence to ensure its relevance. It is important to note that not all updates will simply involve repeating the original steps of the review. The process could be time-consuming, especially if there are changes in the broader context of clinical decision-making or if updated methods need to be considered. These factors can contribute to a lengthy updating process, sometimes taking several months or longer to complete. If there are no existing reviews to work from, accelerated SRs, often confused with RRs, actually refer to traditional SRs conducted more quickly, often facilitated by additional resources such as expertise and an expanded review team. Accelerated SRs aim to maintain the methodological rigour and comprehensiveness of SRs, while also expediting their completion, distinguishing them from RRs. If these methods cannot meet the required timelines, or when initiating any new synthesis is impractical, decisions may be based on the best available evidence without a formal synthesis. The choice among these synthesis types depends on the urgency with which decision-makers need evidence, the availability of resources and expertise, and the impact on decision outcomes. RRs emerge as a distinct and valuable option when there is an immediate need for evidence, and other methods cannot adequately balance timeliness with the breadth and depth of a traditional SR. General distinctions between SRs and RRs, and other types of evidence syntheses have been previously published.6 31 32
Conducting an RR is appropriate in various scenarios for reasons that can often overlap. The categories outlined below address a variety of practical scenarios where timely access to synthesised evidence is crucial. They provide a clear framework for considering when to use RRs in decision-making and research, helping healthcare professionals, policy-makers, decision-makers and researchers better understand when and how to employ RRs effectively. Regardless of the scenario, the appropriateness of conducting an RR should depend on the specific context and the urgency of the decision or inquiry.
Scenarios
Urgent decision-making: RRs are valuable when policy-makers, healthcare providers or public health authorities face urgent decisions, such as responding to disease outbreaks, natural disasters or emerging health threats and need evidence to inform immediate actions.33 34 RRs can also guide clinical care decisions by synthesising available evidence for healthcare professionals requiring evidence for time-sensitive direct patient care decisions.35 36
Informing guidelines: RRs are valuable for informing the rapid development or updating of clinical practice guidelines recommendations, ensuring that healthcare practices are based on the latest evidence.28 37 38
New or emerging technologies and interventions: RRs may be suitable when assessing the evidence on newly introduced medical technologies, interventions or diagnostic tools that have potential clinical implications.39
Rapidly evolving research areas: RRs can help provide an up-to-date synthesis of evidence in rapidly evolving fields, such as infectious diseases, biotechnology or digital health interventions.40
Identify evidence gaps: RRs can efficiently identify evidence gaps and areas where evidence is scarce or lacking, guiding future research priorities.41
Justify or inform new primary research: RRs can justify or inform the design of new primary studies in situations with limited resources.42
Resource constraints: RRs provide a valuable alternative to full SRs by offering a concise yet evidence-based summary within project constraints in situations with limited resources, such as low-resource settings or tight timelines with limited funding.19 43
Time-sensitive opportunities: RRs expedite the research process and provide timely evidence to support proposals or initiatives when time is critical, such as time-limited funding opportunities or to meet decision-makers’ urgent evidence needs.14
Other possible scenarios: RRs may be conducted as a precursor to SR and can offer initial insights and may help identify whether there is a need for a more comprehensive SR to validate findings further. This approach is context-dependent and should be considered based on the specific research question. RRs can also assist researchers and decision-makers in gauging whether additional evidence is through SRs or primary research. This is particularly relevant when existing evidence is scarce, outdated or not directly applicable to the target population or context. By extension, RRs can also support grant submissions for SRs or primary studies.
When is it inappropriate to conduct an RR?
There are instances when conducting an RR may not be justified or inappropriate. One such situation arises when the researchers need more experience in conducting SRs, leading them to opt for an RR merely because it is perceived as easier. RRs may be more difficult, and the researcher should be aware of the potential biases introduced by the accelerated methods.
Similarly, if the primary motivation behind conducting an RR is to achieve a quick publication, and this is perceived to be less work, it may compromise the rigour and comprehensiveness of the review process. Another concern arises when the decision to conduct an RR is primarily driven by the desire to save money, even though the subject under investigation has far-reaching consequences and requires evidence-based decision-making.
Further, if there are already up-to-date full SRs available on the specific topic of interest, conducting an RR might duplicate efforts and fail to add significant value to the existing evidence base. Lastly, conducting an RR only for academic purposes should be discouraged unless, in the context of evidence-based research, the findings have immediate practical implications, such as contributing to the broader knowledge base to potentially inform future research or decision-making processes. In such cases, it is essential to carefully assess the need for an RR and consider alternative approaches for conducting more comprehensive and reliable research.
Considerations when deciding to do an RR
Undertaking RRs is predicated on their utility in scenarios where traditional SR processes are unable to meet the necessary time and resource constraints. It is also important to consider the pros and cons of this approach, as it offers the advantage of resource-efficient evidence synthesis but also comes with potential drawbacks.21 44 Limitations of RRs include potential methodological weaknesses, biases due to the expedited methods used and a narrower scope, which could impact the trustworthiness of their findings. Some of these limitations can also be found in SRs that are not well conducted or well reported.21 Moreover, limitations can arise from inadequately reported or conducted primary studies included in a review, irrespective of the type of evidence synthesis (see online supplemental file 1 for potential limitations of RRs and approaches to mitigate drawbacks). Regardless of the type of synthesis being conducted, if the process is poorly executed or inadequately reported, it can lead to unreliable results. To address these limitations, it is crucial to adhere to best practices9 23 24 and maintain transparent reporting.25
Supplemental material
At the outset, the initial planning stage of any RR may be guided by a set of factors categorised as either conceptual or practical. Conceptual factors represent higher-level considerations or broader aspects influencing the need for an RR. Considerations involve assessing the reasons for needing an RR, the potential risks of incomplete evidence, the novelty of the situation and the level of uncertainty among decision-makers. It also includes evaluating the impact on decision-making if waiting for an SR or going without any evidence synthesis and considering whether the findings from the RR will be acted on promptly. These factors help determine the urgency and importance of obtaining timely evidence. Practical factors represent the more operational and logistical aspects determining the feasibility of implementing an RR approach.
Guiding questions to help assess the appropriateness of conducting an RR include
Conceptual factors
Why is an RR needed, and what are the potential risks to the populations being studied if the evidence is incomplete? When considering the need for an RR, examining the event or situation driving the request is crucial. RRs have demonstrated utility in informing urgent health issues, such as rapidly spreading infectious diseases, where immediate access to evidence is crucial for decision-making.37 However, conducting an RR comes with risks if the evidence base is incomplete, potentially leading to suboptimal decision-making. Balancing the need for timely evidence synthesis and ensuring completeness is vital to minimise potential negative impacts on target populations. While decision-makers might be willing to prioritise speed over certainty,45 it is essential to approach this with caution. The potential risks and trade-offs linked to incomplete evidence should be carefully considered in the context of each RR.
Is there a need for an RR based on a ‘novel’ event? RRs may be beneficial for reviewing evidence in cases of new events, such as introducing treatment interventions, detecting new virus strains, considering new distinct outcomes or introducing new technologies. Depending on the degree of novelty, one should assess whether to rapidly review what could be a limited or premature literature base or to conduct a full SR.38
Is there significant uncertainty for decision-makers as to the best position to take? RRs can be valuable when facing uncertainty and time constraints in decision-making. Decision-makers may need clarification about the best course of action, or there may be conflicting viewpoints and opinions in the field. Therefore, assess the level of uncertainty and divergence among views to determine if an RR is the most appropriate approach.
What would be the impact of waiting for an SR? The impact of waiting for an SR vs an RR depends on the specific situation and context. While waiting for an SR can help ensure a more thorough and reliable evidence synthesis, it also takes longer and delays decision-making. On the other hand, choosing an RR provides evidence faster but may come with limitations in terms of comprehensiveness, reliability and potential bias. The decision between the two approaches hinges on the urgency of the decision and the level of evidence needed. If the event or situation is expected to last beyond the typical time frame of 1–6 months for RRs, then an RR may not be the appropriate choice. However, if decision-making would otherwise proceed without evidence, even a moderately robust RR could be better than having no evidence to inform healthcare decisions.
What would be the risk of not using evidence synthesis to inform decision-making? The risk of not using any evidence synthesis to inform decision-making should be carefully considered. Proceeding without any evidence synthesis, whether an RR or SR, could lead to suboptimal choices that negatively impact the well-being and outcomes of the populations involved. In contrast, conducting even a moderately robust RR can be a more suitable option to quickly provide relevant evidence than having no evidence. Assessing the potential impact of forgoing evidence synthesis or delaying decision-making while waiting for synthesis is essential.
Will the findings from the RR be rapidly acted on? If there is no existing mechanism for disseminating and implementing the findings of the RR, there may be no point in developing it. It will be necessary to carefully consider the decision-making context (eg, the existing health system, infrastructure, acceptability and resource implications) before embarking on an RR process.38
Practical factors
How quickly does the uncertainty need to be addressed? Evaluate the urgency/time sensitivity of the research question. If there is a pressing need for evidence to inform immediate decision-making or address an emerging issue, an RR may be more appropriate due to its shorter turnaround time. On the other hand, if time is less critical and a comprehensive synthesis is required, an SR may be preferred.
What are the stakeholders needs and expectations? Engage with decision-makers, policy-makers and relevant stakeholders to determine their priorities. Some stakeholders may prioritise rigour and comprehensiveness and therefore, prefer an SR while others may require timely evidence and lean towards an RR. Aligning with stakeholder preferences can help determine the most appropriate approach.
What is the scope of the research question? Consider the breadth and complexity of the research question. If the question is narrowly focused and specific, an RR may be sufficient to address it. However, an SR is more likely to be appropriate if the research question requires a comprehensive analysis of a wide range of studies and outcomes.
What are the available resources? Assess the availability of resources, including personnel, funding and expertise that are available for the review. Conducting an SR can be resource-intensive and time-consuming. However, an RR may also be all consuming and intense but over a more condensed time frame. An RR may be more feasible if resources are limited, as it typically requires fewer resources and can be conducted by a smaller team. Both require a team with expertise in SR methods.
Within Cochrane, the decision to conduct an RR is influenced by two main factors: the urgency of the question and whether it addresses a high-priority issue. High-priority situations may include, for example, urgent requests from funding agencies or the need to inform a quickly convened guideline panel. A full SR is deemed impractical due to time constraints in these cases. By considering the above-mentioned factors, we can better understand both the conceptual underpinnings and the practical considerations to help make an informed judgement about whether an RR or an SR is more appropriate for a specific research question and context.
Equity considerations for RRs
Low-resource settings face specific challenges in synthesising and delivering evidence to knowledge users. In these settings, limited resources may hinder the comprehensive conduct of SRs, leading to delays or knowledge gaps. RRs become particularly relevant in such contexts, offering a more feasible and timelier alternative to full SRs. Their appropriateness in low-resource settings stems from the ability to provide a concise, evidence-based summary within project constraints, aligning with the need for efficient resource use. Moreover, RRs expedite access to available evidence for decision-making in time-sensitive scenarios, addressing equity concerns through the provision of timely and relevant information to knowledge users.
Conversely, RRs may pose several concerns from an equity perspective. While RRs offer a quick and efficient approach to evidence synthesis, conducting them for subgroups can pose several limitations. One major concern is the potential for missing relevant studies on specific subpopulations. The expedited nature of RRs may lead to a more cursory search, increasing the risk of overlooking studies that focus on particular demographic, geographical or clinical subgroups. This limitation can result in a skewed representation of the evidence, potentially leading to inaccurate conclusions or recommendations for certain populations. Furthermore, exclusions based on language or publication type are common shortcuts in RRs to expedite the review process. However, this practice could introduce a language bias and exclude valuable studies, particularly those published in languages other than the primary language of the review. This exclusion may disproportionately impact research from certain regions or communities, contributing to disparities in evidence representation. Therefore, while RRs offer several advantages, researchers and decision-makers should be cautious and consider the trade-offs carefully when opting for RRs, particularly when focusing on specific subpopulations.
Data availability statement
Data sharing not applicable as no datasets generated and/or analysed for this study.
Ethics statements
Patient consent for publication
Ethics approval
Not applicable.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @cgarritty
Collaborators On behalf of the Cochrane Rapid Reviews Methods Group: Chantelle Garritty, Barbara Nussbaumer-Streit, Candyce Hamel, Declan Devane.
Contributors All authors (CG, BN-S, CH and DD) contributed to the conceptualisation of this paper on behalf of the Cochrane Rapid Reviews Methods Group. CG wrote the first draft of the manuscript. All authors (CG, BN-S, CH and DD) read and approved the final version. CG is the guarantor and attests that all authors meet authorship criteria and that no others meeting the criteria have been omitted.
Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.