Skip to main content
A university for the real world

What is a Systematic Review

Definition

A systematic review is an appraisal and synthesis of primary research papers using a rigorous and clearly documented methodology in both the search strategy and the selection of studies. This minimises bias in the results. The clear documentation of the process and the decisions made allow the review to be reproduced and updated.

Characteristics of a Systematic review

  • a clearly stated set of objectives with pre-defined eligibility criteria for studies
  • an explicit, reproducible methodology
  • a systematic search that attempts to identify all studies that would meet the eligibility criteria
  • an assessment of the validity of the findings of the included studies, for example through the assessment of risk of bias
  • a systematic presentation, and synthesis, of the characteristics and findings of the included studies.

(Cochrane Handbook for Systematic Reviews of Interventions, 2011)

Team Work

Systematic Reviews are often a team effort. Important areas of expertise to cover are:

  • Content experts - It is important to have team members or an active consultant to provide expertise in the area covered by the review. Input is usually needed from practitioners and researchers representing a variety of perspectives.
  • Systematic Reviews methods experts - One or more persons with expertise in the methods of conducting Systematic Reviews is needed. This person may be responsible for developing the procedures and documentation standards for the review. A Systematic Review methods expert may also be a content expert, but more than one investigator-level reviewer is necessary, since some steps in the process require dual review or data checking that requires expertise in research and statistical methodology.
  • Statistician - If meta-analysis is to be considered, access to a statistician with experience in meta-analysis is needed.
  • Medical librarian - Database searching requires specialized knowledge that general research training does not provide. Preferably, the librarian searcher has experience with the extensive searching and documentation procedures that are a part of a systematic review.
  • Reference management - Someone must be responsible for maintaining the database of references. Most SRs involve thousands of abstracts, and the use of software to manage the references is necessary. This person must be able to track which abstracts have been reviewed and their disposition (e.g., included or excluded, reason for exclusion).

O'Connor, E., Whitlock, E., & Spring, B. (2007).
Introduction to Systematic Reviews, Retrieved from http://ebbp.org/course_outlines/systematic_review/#1

Why systematic reviews are important: an example

From 1956 until the late 1970s, the best selling book Baby and Child Care by influential American pediatrician Dr Spock, advised that infants be placed to sleep on their stomachsDr Spock argued  ‘There are two disadvantages to a baby’s sleeping on his back. If he vomits he’s more likely to choke on the vomitus. .. I think it is preferable to accustom a baby to sleeping on his stomach from the start.’  This became standard practice in hospitals and millions of households. Dr Spock's advice was based on logic and common sense, but had not been tested. Systematic reviewing of the evidence later proved that this practice had led to tens of thousands of avoidable cot deaths.

 

Evans I, Thornton H, Chalmers I, et al. Testing Treatments: Better Research for Better Healthcare. 2nd edition. London: Pinter & Martin; 2011. Chapter 2, Hoped-for effects that don’t materialize. Available from: http://www.ncbi.nlm.nih.gov/books/NBK66199/

What is the difference between a Literature and a Systematic Review?

  Systematic Review Literature Review
Question Focused on a single question Not necessarily focused on a single question, but may describe an overview
Protocol A peer review protocol or plan is included No protocol is included
Background Both provide summaries of the available literature on a topic
Objectives Clear objectives are identified Objectives may or may not be identified
Inclusion and Exclusion Criteria Criteria stated before the review is conducted Criteria not specified
Search Strategy Comprehensive search conducted in a systematic way Strategy not explicitly stated
Process of Selecting Articles Usually clear and explicit Not described in a literature review
Process of Evaluating Comprehensive evaluation of study quality Evaluation of study quality may or may not be included
Process of Extracting Relevant Information Usually clear and specific Not clear or explicit
Results and Data Synthesis Clear summaries of studies based on high quality evidence Summary based on studies where the quality of the articles may not be specified. May also be influenced by the reviewer's theories, needs and beliefs
Discussion Written by an expert or group of experts with a detailed and well grounded knowledge of the issues

Reproduced from: Bettany-Saltikov, J. (2010). Learning how to undertake a systematic review: Part 1. Nursing Standard, 24(40): 47-55.

How are they used

To help groups and individuals make decisions to improve people’s health. Examples include:

  • Recommendations and guidelines
    [e.g., United States Preventive Services Task Force (USPSTF), National Institute for Health and Clinical Excellence (NICE).]
    Should smokers routinely be advised to use quit smoking medications? Should primary care providers routinely screen patients for depression? What should be the components of an intervention to help overweight children manage their weight?
  • Benefit design, coverage and policy decisions
    [e.g., Centers for Medicare and Medicaid Services (CMS), Drug Effectiveness Review Project (DERP), UK National Health Service (NHS).]
    Should we cover the use of medication to quit smoking? Should we reimburse visits with breast feeding specialists in mothers with babies?
  • Public Policy
    Would it improve the health of our community if we increase funding for mass transit and bike facilities?
  • Performance measures
    [e.g., Assessing Care of Vulnerable Elderly (ACOVE).]
    If a patient receives a new prescription for an antidepressant, what frequency and duration of followup constitute good quality care?
  • Research agendas
    [e.g., National Institutes for Health (NIH), Agency of Healthcare Research and Quality (AHRQ).]
    What are the gaps in the evidence related to treatment of anxiety in children?
  • Individual patient care
    Should I advise this client to use behavioral treatment in addition to medication to help her quit smoking?
  • Patient decisions
    Should I try hypnosis to help me get over my fear of flying?

Elizabeth, O. C., Whitlock, E., & Spring, B. Introduction to Systematic Reviews, Retrieved from http://ebbp.org/course_outlines/systematic_review/