Systematic review

If you are a staff member or PhD/Doctoral researcher conducting a systematic review this guide is for you. If you are an MSc or undergraduate dissertation student conducting a structured literature review, you will also find some parts of this guide helpful. This guide is from a health and life science research perspective but may contain useful information for researchers in other disciplines. If you are conducting a review in another area you can contact your librarian for support. There are many different types of literature review, for further information check our guide.

For subject specific help and resources including key databases access your subject guide

What is a systematic review?

A high quality systematic review identifies, evaluates, analyses and interprets research evidence to answer a clearly defined and answerable research question. One of the first steps in conducting a systematic review is to develop a protocol that explicitly states the review question, objectives, methods, search strategy, eligibility criteria and dissemination plans for the review. This should be registered with PROSPERO (International prospective register of systematic reviews) as a permanent record of the plan for the review.

Some, but not all, systematic reviews include statistical methods and techniques to analyse and integrate the results of independent studies to produce a meta-analysis. Results are synthesised and presented in an impartial manner for publication and to support evidence based practice.

A systematic review cannot be carried out by one person alone. In order to reduce bias there must be at least two researchers independently involved in the selection of studies. Often a systematic review includes a team of people and can include researchers, statisticians, librarians, project leader and may take months or even years to complete. 

A structured literature review adopts some of the same principles and approaches to gathering evidence as a systematic review but it can be conducted by one person and is more limited in scope. This is typically the type of review that would be expected at undergraduate, postgraduate and Masters level dissertation. You may be required to develop a formative plan for this type of review but only a full systematic review requires a protocol to be registered with PROSPERO.

For more information about the difference between a full systematic review and a structured literature review Penn State University have produced a useful chart. For information on other kinds of review see our accompanying guide.

Where should I start?

You should check whether a review has already been done. There are a number of places you can look, including the Cochrane Library, the Campbell Collaboration, the Joanna Briggs Database of Systematic Reviews, PEDro, and Google Scholar to name a few. You can also check whether a protocol has been registered by searching PROSPERO. For help using the Cochrane Library you can access video guides or attend a library workshop.

It will be necessary to carry out some simple scoping searches on the topic of interest in the early stages to get an idea of how much research evidence is available and the terminology that is in use. This will help you to refine the research question and to identify appropriate search terms and inclusion criteria. Google Scholar provides a simple broad search across scholarly content from journal publishers, university repositories, and other websites including peer-reviewed journal articles, books, abstracts, theses, dissertations, technical reports and more. This makes it useful for simple scoping searches. For help using Google Scholar more effectively visit our guide.

Consult other members of the review team regularly during this process as they may already know of influential research in the area and have a specialist insight to the terminology that is in use. This can be particularly valuable if different professional or geographical expertise is available within the team.

Use a framework such as PICO (population, intervention, comparison, outcome) or SPICE (setting, perspective, intervention, comparison, evaluation) to help structure your question and your search terms. This will inform the research protocol and the final search strategy for the review.

Once the protocol has been agreed by the review team it should be registered with PROSPERO.

The Cochrane Handbook and Systematic review: CRD’s guidance for undertaking reviews in health care both include sections on developing a protocol.

What sources should I search?

In order to search comprehensively it will be necessary to use multiple sources. Bibliographic databases provide the best functionality for performing and saving a structured search. There is some overlap between the databases in terms of the journals that they index but each one also has unique content and therefore it is essential to search more than one. For information on which databases to search you should consult a relevant library subject guide. In the health field it would be common to search using Cinahl, Medline, PsycInfo and others depending on the topic. 

It may be necessary to search supplementary sources where it is unlikely that all of the required information will be indexed in a bibliographic database. This may involve citation searching, journal hand searching and searching for information that has not been commercially published which is also known as grey literature. This type of searching should be documented in your protocol and in the final search strategy and methods section of the review.

How do I develop a search strategy?

Initial scoping searches, background reading, personal experience and input from the other members of the review team will all help to inform the search terms that you use in the final search strategy. You should consider synonyms, closely related terms, abbreviations and acronyms, alternative spellings and words with the same stem as you develop your search terms. Also check the search strategies of related systematic reviews for the terms they have used.

You should use a combination of free text (or keyword) and subject heading (or thesaurus) terms. Each database uses its own thesaurus to help with more precise searching. They are named after the database, apart from Medline which uses MeSH, Web of Science which uses Key Words Plus and EMBASE which uses Emtree. EMBASE is not currently available via GCU Library but you can get access by registering for an OpenAthen password with NHS Knowledge Network. For help using subject headings in CINAHL watch our video.

Once you have identified your search terms you will need to make sure they are combined effectively using the Boolean operators  AND, OR and NOT. If you need help planning your search strategy watch our short video: Planning a Structured Literature Search. You may also need to use advanced search techniques such as proximity or adjacency searching. The help sections of individual databases provide guidance on how to do this.

Some systematic searches also use advanced filters to enhance the focus a search, for example, to retrieve a particular study design or some other aspect of the research question. Advanced filters consist of a pre-tested combination of search terms and field codes that are translated for different databases. The Cochrane Collaboration publishes a number of advanced search filters including for RCTThe Scottish Intercollegiate Guidelines Network (SIGN) have developed and adapted a number of filters for the development of clinical guidelines. The InterTASC Information Specialists’ Sub-group (ISSG) identifies, evaluates and tests search filters designed to retrieve research by study design or focus and includes filters for qualitative review.

You should focus on one database while developing your final electronic search strategy but will then need to translate it for different databases to accommodate the thesaurus terms that are available. Keep your thesaurus terms as consistent as possible and always use the same free text terms across all of the databases.

You can also get support with developing your search strategy from your librarian.

How can I evaluate the quality of my search strategy?

Search strategy errors will affect the validity of a review. Check your searches have retrieved papers that you would expect to find. For example, if a relevant and high quality paper has been retrieved during the initial scoping stage using Google Scholar but not by your database searches you should investigate the reasons why and adapt your search strategy accordingly.

It is best practice to have someone evaluate your search. This could be another member of the review team or, where possible, a librarian or information specialist. The Press peer review of electronic search strategies: 2015 guideline statement contains a checklist to support this process. GCU Library is currently unable to offer this service.

How can I record my search strategy?

The methodology of a systematic review should be accurately documented and reported. Therefore you should record your searches as you do them. The easiest way to do this is to ensure that you save your searches.

Databases provide the option to create a personal account: requiring you to sign in, create or register an account. These details may differ from your domain username and password and will be unique to you. Personal accounts provide access to more functionality: ability to create alerts, save searches and save individual or batches of results. When saving searches, make sure you have a naming system that allows you to distinguish between test and final searches.

You may wish to use the PRISMA flowchart when writing up your search strategy.

How can I manage the research?

Search results can be exported from a database to reference management software such as RefWorks. Reference management software allows you to store and organise all of your search results in one place and to conduct the screening of abstracts including your inclusion and exclusion decisions. You will also be able to easily remove duplicates, share your results and decisions with other team members and to create and modify your reference list. 

Essential and recommended reading for systematic review
Essential reading

CENTRE FOR REVIEWS AND DISSEMINATION, 2009. Systematic review: CRD’s guidance for undertaking reviews in health care [online].York: CRD. Available from: https://www.york.ac.uk/media/crd/Systematic_Reviews.pdf

HIGGINS, J. & Green, S., 2008. Cochrane handbook for systematic reviews of interventions [online]. Chichester: John Wiley & Sons Ltd. Available from: http://www.dawsonera.com

SCOTTISH INTERCOLLEGIATE GUIDELINES NETWORK, 2015. SIGN 50: a guideline developer’s handbook [online]. Edinburgh: Health Improvement Scotland. Available from: https://www.sign.ac.uk/sign-50.html

Recommended reading

BETTANY-SALTIKOV, J. & MCSHERRY, R., 2016. How to do a systematic literature review in nursing : A step-by-step guide. Maidenhead: Open University Press.

DE BRUN, C., & PEARCE-SMITH, N., 2014. Searching skills toolkit: Finding the evidence [online]. 2nd ed. Hoboken: BMJ. Available from: https://www.dawsonera.com

GOUGH, D., OLIVER, S., & THOMAS, J., 2017. An introduction to systematic reviews. 2nd ed. London ; Thousand Oaks ; New Delhi ; Singapore: SAGE Publications Ltd.

POLLOCK, A., & BERGE, E., 2017. How to do a systematic review. International Journal of Stroke : Official Journal of the International Stroke Society [online]. 13(2), pp.138-156. Available from: https://doi-org.gcu.idm.oclc.org/10.1177/1747493017743796

Are there any other helpful tools that I should know about?

The Systematic Review Toolbox is a community-driven catalogue of tools to support all aspects of systematic review. The toolbox is developed and edited by Dr Chris Marshall.