Systematic reviews , Ngā arotakenga nahanaha

A systematic review goes beyond a literature review.

A systematic review:

• uses strict methods for finding, appraising, and synthesising evidence to answer a specific research question
• requires an advanced literature search strategy and has rules for filtering results
• uses methods that are explained and justified to control bias
• includes a peer review process for article selection
• covers a large body of published literature that points to contradictory or uncertain results or outcomes.

What type of review is right for you?

Systematic reviews – includes timelines for each stage (University of Sydney Library)

Systematic searches – the first tutorial in a series about searching health literature systematically (Yale Library)

Systematic quantitative literature review – aimed at postgraduate environmental science students, includes video tutorials (Griffith University's School of Environment and Science)

Systematic reviews and meta-analysis in business/management (University of Calgary Library)

Teams of two or more people undertake systematic reviews. A team may include:

• a lead reviewer, who generally has systematic reviewing experience and is like a project lead
• reviewers
• a librarian
• a statistician.

Read about who should do a systematic review from Cochrane Training

The library may be able to provide support for a systematic review team. Please contact Alison Wallbutton, Associate University Librarian Client Services a.l.wallbutton@massey.ac.nz.

When formulating your research question, you need to look at:

• what’s been done already
• whether there’s enough research in the area you’re looking at to support a systematic review.

If you want to explore the literature further, learn about using your search results to find out more about your subject area.

How to formulate your question

Methods for formulating your question include:

• PEO – Population or Problem, Exposure or Experience, Outcome
• PICO – Population, Intervention, Comparison, Outcomes
• PICo – Population, Interest, Context
• PICOT – Population, Intervention/Exposure, Comparison, Outcome, Time
• SPIDER – Sample, Phenomenon of Interest, Design, Evaluation, Research type
• SPICE – Setting, Perspective, Intervention, Comparison, Evaluation.

For more about these methods, see:

• Using PICO or PICo – includes a section on SPICE and SPIDER, and examples and worksheets for qualitative and quantitative health studies (Murdoch University)
• Rationale for well-formulated questions – outlines criteria and discusses scope, focuses on writing for health and Cochrane reviews (Cochrane Training)
• Systematic and systematic-like review toolkit – advice on selecting, using and searching, and guidelines for frameworks focusing on topics relating to exercise, sport, health and medicine (Deakin University)
• Ask – tool for veterinary medicine (RCVS Knowledge)
• BestBETS for vets (University of Nottingham)
• Evidence-based veterinary medicine toolkit (RCVS Knowledge)
• PICO.vet.

Some criteria for what to include or exclude in your systematic review are:

• funding sources
• language
• methodologies
• population studied
• size of samples
• types of data.

Inclusion and exclusion criteria from the University of Melbourne

Defining the criteria for including studies and how they will be grouped for the synthesis from Cochrane Training

Sources you could include in a systematic review are:

• blogs
• book chapters
• clinical trials
• grey literature (literature that’s published informally or non-commercially)
• journal articles and conference papers
• nzresearch.org.nz
• open access repositories
• patents
• policy documents
• standards
• theses
• unpublished research papers
• working papers.

If you are publishing your systematic review you need to register a protocol. Your protocol should be developed and registered before you start your data extraction.

Some example protocol registries include:

• Campbell Collaboration – social sciences and business focus
• Open Science Framework – sciences focus
• Proposing and registering new Cochrane reviews – health focus
• PROSPERO – international prospective register of systematic reviews
• Research Registry – human research studies, not just systematic reviews
• Systematic Review Register (Joanna Briggs Institute) – health focus

This is a key area where the library can support your systematic review.

Learn about advanced search strategies

Find out how to select databases to search

Learn about receiving new research alerts

Examples of data screening and management tools include (please note that Massey University does not necessarily provide access or support for these):

• Abstrackr – free screening programme for systematic reviewers (registration required)
• Covidence – platform for collaborative title and abstract screening, full-text review, risk of bias assessment and data extraction (paid subscription required for full access – check if your college has one)
• Critical appraisal tools – assess the trustworthiness, relevance, and results of published papers (Joanna Briggs Institute)
• EndNote and Zotero referencing software – can organise groups, manage duplicates, create tags and notes. Contact us for help with EndNote or Zotero
• Nvivo – staff and students can install Nvivo on Massey-owned computers and home computers
• Rayyan – a free system for collaborative citation screening and full-text selection.

You can also customise workbooks and spreadsheets as the most basic tools to screen and extract your data using Microsoft Excel and Google Sheets.

Evaluate the quality of the research

Check for bias. This could include:

• attrition or incomplete data
• blinding
• participant selection
• selective reporting.

Categorise your findings

Identify commonalities and areas of difference, then combine all the evidence together and summarise the research that addresses your question.

Levels of Evidence from the Joanna Briggs Institute – a grading approach to evaluating evidence

Levels of Evidence from EBVM – an overview of levels of evidence and how they can be applied to analyse literature within veterinary science

Tools for analysing and synthesising your evidence include:

• MyHub provides resources for Massey students and staff, including some software
• Nvivo – software for qualitative and mixed-methods research, available on MyHub
• SPSS – software for statistical analysis, available on MyHub

When you report your findings, you need to describe:

• the steps you took and what you discovered
• themes that emerged
• gaps in the research
• contradictions in your findings
• recommendations
• best practices that you can highlight.

Some resources to help your report your findings include:

• PRISMA Checklist (Preferred Reporting Items for Systematic Reviews and Meta-Analyses)
• Writing your report from Dalhousie University Libraries
• Reporting results from the University of Michigan Library
• Academic support – Massey writing consultants can give you feedback on academic writing including structure, focus, paragraph structure, flow, presentation, use of sources and referencing.