Systematic reviews , Ngā arotakenga nahanaha

A systematic review goes beyond a literature review.

A systematic review:

• uses strict methods for finding, appraising, and synthesising evidence to answer a specific research question
• requires an advanced literature search strategy and has rules for filtering results
• uses methods that are explained and justified to control bias
• includes a peer review process for article selection
• covers a large body of published literature that points to contradictory or uncertain results or outcomes.

What type of review is right for you?

Systematic reviews – includes timelines for each stage (University of Sydney Library)

Systematic searches – the first tutorial in a series about searching health literature systematically (Yale Library)

Systematic quantitative literature review – aimed at postgraduate environmental science students, includes video tutorials (Griffith University's School of Environment and Science)

Systematic reviews and meta-analysis in business/management (University of Calgary Library)

Systematic review teams

Teams of two or more people undertake systematic reviews. A team may include:

• a lead reviewer, who generally has systematic reviewing experience and is like a project lead
• reviewers
• a librarian
• a statistician.

Read about who should do a systematic review from Cochrane Training

Your research question

When formulating your research question, you need to look at:

• what’s been done already
• whether there’s enough research in the area you’re looking at to support a systematic review.

If you want to explore the literature further, learn about using your search results to find out more about your subject area.

How to formulate your question

Methods for formulating your question include:

• PEO – Population or Problem, Exposure or Experience, Outcome
• PICO – Population, Intervention, Comparison, Outcomes
• PICo – Population, Interest, Context
• PICOT – Population, Intervention/Exposure, Comparison, Outcome, Time
• SPIDER – Sample, Phenomenon of Interest, Design, Evaluation, Research type
• SPICE – Setting, Perspective, Intervention, Comparison, Evaluation.

For more about these methods, see:

• Using PICO or PICo – includes a section on SPICE and SPIDER, and examples and worksheets for qualitative and quantitative health studies (Murdoch University)
• Rationale for well-formulated questions – outlines criteria and discusses scope, focuses on writing for health and Cochrane reviews (Cochrane Training)
• Systematic and systematic-like review toolkit – advice on selecting, using and searching, and guidelines for frameworks focusing on topics relating to exercise, sport, health and medicine (Deakin University)
• Ask – tool for veterinary medicine (RCVS Knowledge)
• BestBETS for vets (University of Nottingham)
• Evidence-based veterinary medicine toolkit (RCVS Knowledge)
• PICO.vet.

What to include or exclude

Some criteria for what to include or exclude in your systematic review are:

• funding sources
• language
• methodologies
• population studied
• size of samples
• types of data.

Inclusion and exclusion criteria from the University of Melbourne

Defining the criteria for including studies and how they will be grouped for the synthesis from Cochrane Training

Sources you could include in a systematic review are:

• blogs
• book chapters
• clinical trials
• grey literature (literature that’s published informally or non-commercially)
• journal articles and conference papers
• nzresearch.org.nz
• open access repositories
• patents
• policy documents
• standards
• theses
• unpublished research papers
• working papers.

Register your review protocol

If you are publishing your systematic review you need to register a protocol. Your protocol should be developed and registered before you start your data extraction.

Learn about what a review protocol is from PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses)

Some example protocol registries include:

• Campbell Collaboration – social sciences and business focus
• Open Science Framework – sciences focus
• Proposing and registering new Cochrane reviews – health focus
• PROSPERO – international prospective register of systematic reviews
• Research Registry – human research studies, not just systematic reviews
• Systematic Review Register (Joanna Briggs Institute) – health focus

Advanced literature searching

Learn about advanced search strategies

Find out how to select databases to search

Learn about receiving new research alerts

Screen your studies and extract data

Examples of data screening and management tools include:

• Abstrackr – free screening programme for systematic reviewers (registration required)
• Covidence – platform for collaborative title and abstract screening, full-text review, risk of bias assessment and data extraction (paid subscription required for full access – check if your college has one)
• Critical appraisal tools – assess the trustworthiness, relevance, and results of published papers (Joanna Briggs Institute)
• EndNote referencing software – can manage duplicates, enable smart groups, rank and flag
• Nvivo – staff and students can install Nvivo on Massey-owned computers and home computers
• Rayyan – a free system for collaborative citation screening and full-text selection.

You can also customise workbooks and spreadsheets as the most basic tools to screen and extract your data using Microsoft Excel and Google Sheets.

Analyse and synthesise your evidence

Evaluate the quality of the research

Check for bias. This could include:

• attrition or incomplete data
• blinding
• participant selection
• selective reporting.

Categorise your findings

Identify commonalities and areas of difference, then combine all the evidence together and summarise the research that addresses your question.

Levels of Evidence from EBVM – an overview of levels of evidence and how they can be applied to analyse literature within veterinary science

Tools for analysing and synthesising your evidence include:

• MyHub provides resources for Massey students and staff, including some software
• Nvivo – software for qualitative and mixed-methods research, available on MyHub
• SPSS – software for statistical analysis, available on MyHub

Report your findings

When you report your findings, you need to describe:

• the steps you took and what you discovered
• themes that emerged
• gaps in the research
• contradictions in your findings
• recommendations
• best practices that you can highlight.

Some resources to help your report your findings include:

• PRISMA checklist (Preferred Reporting Items for Systematic Reviews and Meta-Analyses)
• Writing your report from Dalhousie University Libraries
• Reporting results from the University of Michigan Library
• Academic support – Massey writing consultants can give you feedback on academic writing including structure, focus, paragraph structure, flow, presentation, use of sources and referencing.