• epidemiology

One of the key elements in evidence-based medicine (EBM) is reliable information from research on the benefits and harms of specific interventions, actions or strategies. This is true for resolving uncertainties about interventions that might be used to treat illnesses or improve well-being and also for choosing screening or diagnostic tests, understanding risk factors and estimating the current and future burden of disease. As the principles and practice of EBM have become more accepted and widespread over the last few decades, there has been an accompanying tremendous growth in the number of systematic reviews and wider recognition of their value. From sporadic examples before the 1980s, through the estimated 3000 that were indexed in MEDLINE during the two decades to 2000,1 200 000 or more might now be available.2 More than 10 000 systematic reviews are published every year, and over 30 000 are registered in the prospective registry, PROSPERO.3 They are a vital part of EBM, and many of the reasons that we value them today have echoes in history.

We have written elsewhere about this history of systematic reviews4–6 but reflect here on two aspects of the history and their relevance to EBM today and in the future: quality and quantity.

People making decisions and choices about health and social care need access to high-quality evidence from research. Systematic reviews provide this by both highlighting the quality of existing studies and by themselves providing a high-quality summary. In a 1753 example of what we might now call a systematic review, James Lind, in his treatise on scurvy, presented a ‘Critical and Chronological View of what has been published on the subject’. He wrote ‘It became requisite to exhibit a full and impartial view of what had hitherto been published on the scurvy’ and ‘before the subject could be set in a clear and proper light, it was necessary to remove a great deal of rubbish’.7

This uncovering of rubbish research continues to be an important role for systematic reviews, not least in demonstrating the enormous amount of research waste caused by poor quality studies.8 Furthermore, reviews are now widely accepted as the most reliable source of knowledge from research. They are emphasised as the top of the hierarchy of evidence across the whole range of clinical questions in the 2011 revision of the Oxford Centre for Evidence Based Medicine levels of evidence (www.cebm.net/2011/06/explanation-2011-ocebm-levels-evidence) and are the core building blocks for clinical and policy guidelines published by organisations such as the WHO.9 This dominance of systematic reviews looks set to continue, but must be tempered by the need for users to appraise the reviews they might use and not simply assume that the label ‘systematic review’ implies quality.10

Turning to quantity, or more appropriately perhaps, quantitative analysis, when Gene Glass introduced the term ‘meta-analysis’, it was to capture in part the combining of results from separate studies.11 This statistical synthesis of the findings of separate but similar studies is now a prominent feature in systematic reviews, with perhaps millions of meta-analyses across the hundreds of thousands of reviews in the literature. However, history shows us that this is not new. In the 1720s, James Jurin, the secretary of the Royal Society, Thomas Nettleton and John Gasper Scheuchzer combined the results of multiple studies to estimate the effects of inoculation for smallpox in England.12–15

In the early 20th century, Karl Pearson, director of the Biometric Laboratory at University College London, England, combined five studies of immunity and six studies of mortality to investigate the effects of a vaccine against typhoid,16 and Park and his colleagues synthesised the results of three studies of serum treatment for lobar pneumonia.17

The mathematical techniques have been refined over time,2 18 and systematic reviewers today are able to draw on numerous statistical packages to help them and to draw the ubiquitous forest plots to display their results.19 However, the future may see important developments in both the data used in these analyses and the way that studies are brought together. One of the first individual participant data meta-analyses used data from nearly 2500 patients who had taken part in nine trials to assess the effects of anticoagulant therapies after myocardial infarction.20 Today, increased access to this type of data looks set to deliver more such reviews,21 22 and the newer techniques of mixed treatment comparisons or network meta-analyses look set to change how studies are combined to identify the most effective, acceptably safe interventions.23

In summary, an understanding of history can reveal that things we might think of as ‘new’ or novel often have a past that goes back decades or centuries. This is the case with the need for systematic reviews and the value of science accumulating evidence.24 What seems different now is how wider awareness of the need for robust evidence and technological and other advances make it much easier to find and review individual studies, leading to such dramatic increases in the number of systematic reviews. This raises the importance for reviews themselves not to become wasteful, to be kept up-to-date, and to be done to the high standards needed to justify their status in evidence-informed decision making.