definition of systematic review in nursing

• Subscribe to journal Subscribe
• Get new issue alerts Get alerts

Secondary Logo

Journal logo.

Colleague's E-mail is Invalid

Your message has been successfully sent to your colleague.

Save my selection

The Systematic Review

An overview.

Aromataris, Edoardo PhD; Pearson, Alan PhD, RN

Edoardo Aromataris is the director of synthesis science at the Joanna Briggs Institute in the School of Translational Health Science, University of Adelaide, South Australia. Alan Pearson is the former executive director and founder of the Joanna Briggs Institute. Contact author: Edoardo Aromataris, [email protected] . The authors have disclosed no potential conflicts of interest, financial or otherwise.

The Joanna Briggs Institute aims to inform health care decision making globally through the use of research evidence. It has developed innovative methods for appraising and synthesizing evidence; facilitating the transfer of evidence to health systems, health care professionals, and consumers; and creating tools to evaluate the impact of research on outcomes. For more on the institute's approach to weighing the evidence for practice, go to http://joannabriggs.org/jbi-approach.html .

This article is the first in a new series on systematic reviews from the Joanna Briggs Institute, an international collaborative supporting evidence-based practice in nursing, medicine, and allied health fields. The purpose of the series is to show nurses how to conduct a systematic review—one step at a time. This first installment provides a synopsis of the systematic review as a scientific exercise, one that influences health care decisions.

This first article in a new series from the Joanna Briggs Institute provides a synopsis of the systematic review as a scientific exercise, and introduces nurses to the steps involved in conducting one.

Research in the health sciences has provided all health care professions, including nursing, with much new knowledge to inform the prevention of illness and the care of people with ill health or trauma. As the body of research has grown, so too has the need for rigorous syntheses of the best available evidence.

Literature reviews have long been a means of summarizing and presenting overviews of knowledge, current and historical, derived from a body of literature. They often make use of the published literature; generally, published papers cited in a literature review have been subjected to the blind peer-review process (a hallmark of most scientific periodicals). The literature included in a literature review may encompass research reports that present data, as well as conceptual or theoretical literature that focuses on a concept. 1

An author may conduct a literature review for a variety of reasons, including to 1

• present general knowledge about a topic.
• show the history of the development of knowledge about a topic.
• identify where evidence may be lacking, contradictory, or inconclusive.
• establish whether there is consensus or debate on a topic.
• identify characteristics or relationships between key concepts from existing studies relevant to the topic.
• justify why a problem is worthy of further study.

All of these purposes have been well served by a “traditional” or “narrative” review of the literature. Traditional literature reviews, though useful, have major drawbacks in informing decision making in nursing practice. Predominantly subjective, they rely heavily on the author's knowledge and experience and provide a limited, rather than exhaustive, presentation of a topic. 2 Such reviews are often based on references chosen selectively from the evidence available, resulting in a review inherently at risk for bias or systematic error. Traditional literature reviews are useful for describing an issue and its underlying concepts and theories, but if conducted according to no stated methodology, they are difficult to reproduce—leaving the findings and conclusions resting heavily on the insight of the authors. 1, 2 In many cases, the author of the traditional review discusses only major ideas or results from the studies cited rather than analyzing the findings of any single study.

With the advent of evidence-based health care some 25 years ago, nurses and other clinicians have been expected to refer to and rely on research evidence to inform their decision making. The need for evidence to support clinical practice is constantly on the rise because of advances that continually expand the technologies, drugs, and other treatments available to patients. 3 Nurses must often decide which strategies should be implemented, yet comparisons between available options may be difficult to find because of limited information and time, particularly among clinical staff. Furthermore, interpreting research findings as presented in scientific publications is no easy task. Without clear recommendations for practice, it can be difficult to use evidence appropriately; it requires some knowledge of statistics and in some cases extensive knowledge or experience in how to apply the evidence to the clinical setting. 4 Also, many health care devices and drugs come with difficult-to-understand claims of effectiveness. 5

As a result of research, the knowledge on which nursing care is based has changed at a rapid pace. This inexorable progress means that nurses can access biomedical databases containing millions of citations pertinent to health care; these databases are growing at a phenomenal rate, with tens of thousands of publications added every year. The volume of literature is now too vast for nurses and other health care professionals to stay on top of. 3 Furthermore, not all published research is of high quality and reliable; on the contrary, many published studies have used inappropriate statistical methods or have otherwise been poorly conducted.

Such issues affecting research quality can make for research findings that are contradictory or inconclusive. Similarly, using the results of an individual study to inform clinical decision making is not advisable. When compared with other research on the topic, a study may be at risk for bias or systematic error. 5 Therefore, it can be difficult for nurses to know which studies from among the multitude available should be used to inform the decisions they make every day. As a result, reviews of the literature have evolved to become an increasingly important means by which data are collected, assessed, and summarized. 5-7

THE SYSTEMATIC REVIEW

Since the traditional literature review lacks a formal or reproducible means of estimating the effect of a treatment, including the size and precision of the estimate, 2, 7 a considerably more structured approach is needed. The “systematic review,” also known as the “research synthesis,” aims to provide a comprehensive, unbiased synthesis of many relevant studies in a single document. 2, 7, 8 While it has many of the characteristics of a literature review, adhering to the general principle of summarizing the knowledge from a body of literature, a systematic review differs in that it attempts to uncover “all” of the evidence relevant to a question and to focus on research that reports data rather than concepts or theory. 3, 9

As a scientific enterprise, a systematic review will influence health care decisions and should be conducted with the same rigor expected of all research. Explicit and exhaustive reporting of the methods used in the synthesis is also a hallmark of any well-conducted systematic review. Reporting standards similar to those produced for primary research designs have been created for systematic reviews. The PRISMA statement, or Preferred Reporting Items for Systematic Reviews and Meta-Analyses, provides a checklist for review authors on how to report a systematic review. 10 Ultimately, the quality of a systematic review, and the recommendations drawn from it, depends on the extent to which methods are followed to minimize the risk of error and bias. For example, having multiple steps in the systematic review process, including study selection, critical appraisal, and data extraction conducted in duplicate and by independent reviewers, reduces the risk of subjective interpretation and also of inaccuracies due to chance error affecting the results of the review. Such rigorous methods distinguish systematic reviews from traditional reviews of the literature.

The characteristics of a systematic review are well defined and internationally accepted. The following are the defining features of a systematic review and its conduct:

• clearly articulated objectives and questions to be addressed
• inclusion and exclusion criteria, stipulated a priori (in the protocol), that determine the eligibility of studies
• a comprehensive search to identify all relevant studies, both published and unpublished
• appraisal of the quality of included studies, assessment of the validity of their results, and reporting of any exclusions based on quality
• analysis of data extracted from the included research
• presentation and synthesis of the findings extracted
• transparent reporting of the methodology and methods used to conduct the review

Different groups worldwide conduct systematic reviews. The Cochrane Collaboration primarily addresses questions on the effectiveness of interventions or therapies and has a strong focus on synthesizing evidence from randomized controlled trials (RCTs) (see http://handbook.cochrane.org ). 9 Other groups such as the Centre for Reviews and Dissemination at the University of York ( http://bit.ly/1g9WoCq ) and the Joanna Briggs Institute ( http://joannabriggs.org ) include other study designs and evidence derived from different sources in their systematic reviews. The Institute of Medicine issued a report in 2011, Finding What Works in Health Care: Standards for Systematic Reviews , which makes recommendations for ensuring “objective, transparent, and scientifically valid reviews” (see http://bit.ly/1cRIAg7 ).

How systematic reviews are conducted may vary; the methods used will ultimately depend on the question being asked. The approach of the Cochrane Collaboration is almost universally adopted for a clear-cut review of treatment effectiveness. However, specific methods used to synthesize qualitative evidence in a review, for example, may depend on the preference of the researchers, among other factors. 7 The steps for conducting a systematic review will be addressed below and in greater detail throughout this series.

Review question and inclusion criteria. Systematic reviews ideally aim to answer specific questions, rather than present general summaries of the literature on a topic of interest. 5, 8 A systematic review does not seek to create new knowledge but rather to synthesize and summarize existing knowledge, and therefore relevant research must already exist on the topic. 3, 5 Deliberation on the question occurs as a first step in developing the review protocol. 5, 7 Nurses accustomed to evidence-based practice and database searching will be familiar with the PICO mnemonic (Population, Intervention, Comparison intervention, and Outcome measures), which helps in forming an answerable question that encompasses these concepts to aid in the search. 3, 8, 11 (The art of formulating the review question will be covered in the second article of this series.)

Ideally, the review protocol is developed and published before the systematic review is begun. It details the eligibility of studies to be included in the review (based on the PICO elements of the review question) and the methods to be used to conduct the review. Adhering to the eligibility criteria stipulated in the review protocol ensures that studies selected for inclusion are selected based on their research method, as well as on the PICO elements of the study, and not solely on the study's findings. 3 Conducting the review in such a fashion limits the potential for bias and reduces the possibility of altering the focus or boundaries of the review after results are seen. In addition to the PICO elements, the inclusion criteria should specify the research design or types of studies the review aims to find and summarize, such as RCTs when answering a question on the effectiveness of an intervention or therapy. 9 Stipulating “study design” as an extra element to be included as part of the inclusion criteria changes the standard PICO mnemonic to PICOS.

Searching for studies can be a complex task. The aim is to identify as many studies on the topic of interest as is feasible, and a comprehensive search strategy must be developed and presented to readers. 3, 10 A strategy that increases in complexity is common, starting with an initial search of major databases, such as MEDLINE (accessed through PubMed) and the Cumulative Index to Nursing and Allied Health (CINAHL), using keywords derived from the review question. This preliminary search helps to identify optimal search terms, including further keywords and subject headings or indexing terms, which are then used when searching all relevant databases. Finally, a manual search is conducted of the reference lists of all retrieved papers to identify any studies missed during the database searches. The search should also target unpublished studies to help minimize the risk of publication bias 3, 5 —a reality that review authors have to acknowledge. It arises because researchers are more likely to submit for publication positive rather than negative findings of their research, and scientific journals are inclined to publish studies that show a treatment's benefits. Therefore, relying on findings only from published studies may result in an overestimation of the benefits of an intervention. To date, locating unpublished studies has been difficult, but resources for locating this “gray” literature are available and increasing in sophistication. For example, Web search engines can search across many governmental and organizational sites simultaneously. Similarly, there are databases that index graduate theses and doctoral dissertations, abstracts of conference proceedings, and reports that aren't commercially published. Contacting experts in the field may also yield otherwise difficult-to-obtain information. Finally, studies published in languages other than English should be included, if possible, despite the added cost and complexity of doing so. (The art of searching will be addressed in the third paper in this series.)

Study selection and critical appraisal. The PICO elements can aid in defining the inclusion criteria used to select studies for the systematic review. The inclusion criteria place the review question in a practical context and act as a clear guide for the review team as they determine which studies should be included. 3 This step is referred to as study selection . 8 Once it's determined which studies should be included, their quality must be assessed during the step of critical appraisal . (Both of these steps will be further addressed in the fourth paper in this series.)

During study selection , reviewers look to match the studies found in the search to the review's inclusion criteria—that is, they identify those studies that were conducted in the correct population, use interventions of interest, and record the predetermined and relevant outcomes. 3 The optimal research design for answering the review question is also determined. For example, for a systematic review evaluating the effectiveness of an intervention, the most reliable evidence is thought to come from RCTs, which allow the inference of causal associations between the intervention and outcome, rather than from other study designs such as the cohort study, which lacks randomization and experimental “control.” Any exclusion criteria should also be documented—for example, specific populations or modes of delivery of an intervention.

During critical appraisal , all studies to be included are first assessed for methodologic rigor. 3 Although there are some subtle differences, this appraisal is akin to assessing the risk of bias in reviews that ask questions related to the effectiveness of an intervention. A systematic review aims to synthesize the best evidence for clinical decision making. Assessing the validity of a study requires careful consideration of the methods used during the research and establishing whether the study can be trusted to provide a reliable and accurate account of the intervention and its outcomes. 5-8 Studies that are of low or questionable quality are generally excluded from the remainder of the review process. Exclusion of lesser-quality studies reduces the risk of error and bias in the findings of the review. 3 For the most part, critical appraisal focuses squarely on research design and the validity and hence the believability of the study's findings rather than on the quality of reporting, which depends on both writing style and presentation. 10 For example, when assessing the validity of an RCT, critical appraisal generally focuses on four types of systematic error that can occur at different stages of a study: selection bias (in considering how study participants were assigned to the treatment groups), performance bias (in considering how the intervention was provided), attrition bias (in considering participant follow-up and drop-out), and detection bias (in considering how outcomes were measured). 3

To aid the transparency and reproducibility of this process in the systematic review, standardized instruments (checklists, scales) are commonly used when asking the reviewers about the research they are reading.

Data extraction and synthesis. Once the quality of the research has been established, relevant data aligned to the predetermined outcomes of the review must be extracted for the all-important synthesis of the findings. (These steps will be addressed in the fifth paper in this series.) Data synthesized by systematic reviews are the results extracted from the individual research studies; as with critical appraisal, data extraction is often facilitated by the use of a tool or instrument that ensures that the most relevant and accurate data are collected and recorded. 3 A tool may prompt the reviewer to extract relevant citation details, details of the study participants including their number and eligibility, descriptive details of the intervention and comparator used in the study, and the all-important outcome data. Generic extraction tools for both quantitative and qualitative data are readily available. 12 The data collected from individual studies vary with each review, but they should always answer the question posed by the review. While undertaking a review, reviewers will find that data extraction can be quite difficult—often complicated by factors of the included studies such as incomplete reporting of study findings and differing ways of reporting and presenting data. When these issues arise, reviewers should attempt to contact the authors to obtain missing data, particularly for recently published research. 5

Data synthesis is a principal feature of the systematic review. 3, 6, 7, 9 There are various methods available, depending on the type of data extracted that's most appropriate to the review question. 7 An example of a systematic review addressing a question of the effectiveness of a nursing intervention is one examining nurse-led cardiac rehabilitation programs following coronary artery bypass graft surgery; the review aims to give an overall estimate of the intervention's effectiveness on patients’ health-related quality of life and hospital readmission rates. 13 Depending on the question asked, such a synthesis of the results of relevant studies also allows for exploration of similarities or inconsistencies of the treatment effect in different studies and among various settings and populations. 5 Where inconsistencies are apparent they can be analyzed further. The synthesis either provides a narrative summary of the included studies or, where possible, statistically combines data extracted from the studies. This pooling of data is termed “meta-analysis.” 14

A meta-analysis may be included in a systematic review as a practical way of evaluating many studies. Meta-analysis should ideally be undertaken only when studies are similar enough; studies should sample from similar populations, have similar objectives and aims, administer the intervention of interest in a similar fashion, and (most important) measure the same outcomes. 3 Meta-analysis is rarely appropriate when such similarities do not appear across studies. Meta-analysis requires transforming the findings of treatment effect from individual studies into a common metric and then using statistical procedures across all of the findings to determine whether there is an overall effect of the treatment or association. 8, 9, 14 The typical output from a statistical synthesis of studies is the measure or estimate of effect; the confidence interval, which indicates the precision of the estimate; and the quantification of the differences (heterogeneity) between the included studies and the statistical impact of these differences, if any, on the analysis. There are many different statistical methods by which results from individual studies can be combined during the meta-analysis. The results of the meta-analysis are commonly displayed as a forest plot, which gives the reader a visual comparison of the findings.

Owing to the limited availability of relevant trials, reviews that aim to examine the effectiveness of an intervention may resort to evidence from experimental studies other than RCTs and even from observational studies; such reviews have the potential to play a greater role in evidence-based nursing, where trials, historically, have been rare. 15 But when conducting a systematic review of studies using designs other than the RCT, a reviewer must take into account the biases inherent in those designs and make definitive recommendations about the effectiveness of a practice with caution.

Other types of evidence, including qualitative evidence and economic evidence addressing questions related to health care costs, can also be synthesized using methods established by organizations such as the Joanna Briggs Institute. 12 While the methods of synthesizing quantitative data are relatively straightforward and accepted, there are numerous methods for synthesizing qualitative research. Such reviews may appear as a meta-synthesis, a meta-aggregation, a meta-study, or a meta-ethnography 7, 16 ; the differences between these approaches will be discussed in the fifth article in this series.

A systematic review that addresses both quantitative and qualitative studies, as well as theoretical literature, is referred to as an “integrative” or “comprehensive” systematic review. 6, 15 The motivation for conducting a comprehensive review is often to provide further insight into why an intervention appears to have a benefit (or not). “Realist” reviews, another emerging form of evidence synthesis, often look to answer questions surrounding complex interventions, including how and for whom an intervention works. 7, 16 Formalized methods for these types of reviews are still being validated.

Interpretation of findings and recommendations to guide nursing practice. The conclusions of the systematic review, along with recommendations for clinical practice and implications for future research, should be based on its findings. Questions to ask when considering the recommendations of a systematic review include the following: Has a clear and accurate summary of findings been provided? Have specific directives for further research been proposed? Are the recommendations, both for practice and future research, supported by the data presented? (Such issues will be explored in the sixth and last paper in this series.)

Reviewers must consider the quality of the studies when arriving at recommendations based on the results of those studies. For example, if the best available evidence was of low quality or only observational studies were available to answer a question of effectiveness, results based on this evidence must be interpreted with caution.

Nurses are increasingly expected to make evidence-based decisions in their practice, and nursing researchers are increasingly driven to develop advanced methods of evidence synthesis. Systematic reviews aim to summarize the best available evidence using rigorous and transparent methods. We've provided a brief introduction to the steps taken in conducting a systematic review; the remaining papers in this series will explore each step in greater detail, addressing the synthesis of both quantitative and qualitative evidence.

• + Favorites
• View in Gallery

Readers Of this Article Also Read

Evidence-based practice: step by step: the seven steps of evidence-based..., skin assessment in patients with dark skin tone, evidence-based practice, step by step: asking the clinical question: a key step ..., jbi's systematic reviews: data extraction and synthesis, systematic reviews: constructing a search strategy and searching for evidence.

This website is intended for healthcare professionals

• { $refs.search.focus(); })" aria-controls="searchpanel" :aria-expanded="open" class="hidden lg:inline-flex justify-end text-gray-800 hover:text-primary py-2 px-4 lg:px-0 items-center text-base font-medium"> Search

Search menu

Bashir Y, Conlon KC. Step by step guide to do a systematic review and meta-analysis for medical professionals. Ir J Med Sci. 2018; 187:(2)447-452 https://doi.org/10.1007/s11845-017-1663-3

Bettany-Saltikov J. How to do a systematic literature review in nursing: a step-by-step guide.Maidenhead: Open University Press; 2012

Bowers D, House A, Owens D. Getting started in health research.Oxford: Wiley-Blackwell; 2011

Hierarchies of evidence. 2016. http://cjblunt.com/hierarchies-evidence (accessed 23 July 2019)

Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Research in Psychology. 2008; 3:(2)37-41 https://doi.org/10.1191/1478088706qp063oa

Developing a framework for critiquing health research. 2005. https://tinyurl.com/y3nulqms (accessed 22 July 2019)

Cognetti G, Grossi L, Lucon A, Solimini R. Information retrieval for the Cochrane systematic reviews: the case of breast cancer surgery. Ann Ist Super Sanita. 2015; 51:(1)34-39 https://doi.org/10.4415/ANN_15_01_07

Dixon-Woods M, Cavers D, Agarwal S Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups. BMC Med Res Methodol. 2006; 6:(1) https://doi.org/10.1186/1471-2288-6-35

Guyatt GH, Sackett DL, Sinclair JC Users' guides to the medical literature IX. A method for grading health care recommendations. JAMA. 1995; 274:(22)1800-1804 https://doi.org/10.1001/jama.1995.03530220066035

Hanley T, Cutts LA. What is a systematic review? Counselling Psychology Review. 2013; 28:(4)3-6

Cochrane handbook for systematic reviews of interventions. Version 5.1.0. 2011. https://handbook-5-1.cochrane.org (accessed 23 July 2019)

Jahan N, Naveed S, Zeshan M, Tahir MA. How to conduct a systematic review: a narrative literature review. Cureus. 2016; 8:(11) https://doi.org/10.7759/cureus.864

Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1997; 33:(1)159-174

Methley AM, Campbell S, Chew-Graham C, McNally R, Cheraghi-Sohi S. PICO, PICOS and SPIDER: a comparison study of specificity and sensitivity in three search tools for qualitative systematic reviews. BMC Health Serv Res. 2014; 14:(1) https://doi.org/10.1186/s12913-014-0579-0

Moher D, Liberati A, Tetzlaff J, Altman DG Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009; 6:(7) https://doi.org/10.1371/journal.pmed.1000097

Mueller J, Jay C, Harper S, Davies A, Vega J, Todd C. Web use for symptom appraisal of physical health conditions: a systematic review. J Med Internet Res. 2017; 19:(6) https://doi.org/10.2196/jmir.6755

Murad MH, Asi N, Alsawas M, Alahdab F. New evidence pyramid. Evid Based Med. 2016; 21:(4)125-127 https://doi.org/10.1136/ebmed-2016-110401

National Institute for Health and Care Excellence. Methods for the development of NICE public health guidance. 2012. http://nice.org.uk/process/pmg4 (accessed 22 July 2019)

Sambunjak D, Franic M. Steps in the undertaking of a systematic review in orthopaedic surgery. Int Orthop. 2012; 36:(3)477-484 https://doi.org/10.1007/s00264-011-1460-y

Siddaway AP, Wood AM, Hedges LV. How to do a systematic review: a best practice guide for conducting and reporting narrative reviews, meta-analyses, and meta-syntheses. Annu Rev Psychol. 2019; 70:747-770 https://doi.org/0.1146/annurev-psych-010418-102803

Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008; 8:(1) https://doi.org/10.1186/1471-2288-8-45

Wallace J, Nwosu B, Clarke M. Barriers to the uptake of evidence from systematic reviews and meta-analyses: a systematic review of decision makers' perceptions. BMJ Open. 2012; 2:(5) https://doi.org/10.1136/bmjopen-2012-001220

Carrying out systematic literature reviews: an introduction

Alan Davies

Lecturer in Health Data Science, School of Health Sciences, University of Manchester, Manchester

View articles · Email Alan

Systematic reviews provide a synthesis of evidence for a specific topic of interest, summarising the results of multiple studies to aid in clinical decisions and resource allocation. They remain among the best forms of evidence, and reduce the bias inherent in other methods. A solid understanding of the systematic review process can be of benefit to nurses that carry out such reviews, and for those who make decisions based on them. An overview of the main steps involved in carrying out a systematic review is presented, including some of the common tools and frameworks utilised in this area. This should provide a good starting point for those that are considering embarking on such work, and to aid readers of such reviews in their understanding of the main review components, in order to appraise the quality of a review that may be used to inform subsequent clinical decision making.

Since their inception in the late 1970s, systematic reviews have gained influence in the health professions ( Hanley and Cutts, 2013 ). Systematic reviews and meta-analyses are considered to be the most credible and authoritative sources of evidence available ( Cognetti et al, 2015 ) and are regarded as the pinnacle of evidence in the various ‘hierarchies of evidence’. Reviews published in the Cochrane Library ( https://www.cochranelibrary.com) are widely considered to be the ‘gold’ standard. Since Guyatt et al (1995) presented a users' guide to medical literature for the Evidence-Based Medicine Working Group, various hierarchies of evidence have been proposed. Figure 1 illustrates an example.

Systematic reviews can be qualitative or quantitative. One of the criticisms levelled at hierarchies such as these is that qualitative research is often positioned towards or even is at the bottom of the pyramid, thus implying that it is of little evidential value. This may be because of traditional issues concerning the quality of some qualitative work, although it is now widely recognised that both quantitative and qualitative research methodologies have a valuable part to play in answering research questions, which is reflected by the National Institute for Health and Care Excellence (NICE) information concerning methods for developing public health guidance. The NICE (2012) guidance highlights how both qualitative and quantitative study designs can be used to answer different research questions. In a revised version of the hierarchy-of-evidence pyramid, the systematic review is considered as the lens through which the evidence is viewed, rather than being at the top of the pyramid ( Murad et al, 2016 ).

Register now to continue reading

Thank you for visiting British Journal of Nursing and reading some of our peer-reviewed resources for nurses. To read more, please register today. You’ll enjoy the following great benefits:

What's included

Limited access to clinical or professional articles

Unlimited access to the latest news, blogs and video content

Signing in with your registered email address

• Become Involved |
• Give to the Library |
• Staff Directory |
• UNF Library
• Brooks College of Health
• Systematic Reviews
• Nursing Databases
• PICO(T) from Nutrition
• Search Strategy

What is a Systematic Review?

Types of systematic reviews, the prisma checklist and diagram, the systematic review process, structuring a systematic review article, finding the right journal for publication.

• Tools to Manage the Systematic Review Process
• Conducting a Literature Review This link opens in a new window

Call Us (904) 620-2615

Chat With Us Text Us (904) 507-4122   Email Us Schedule a Research Consultation

Visit us on social media!

PRISMA Documents

• PRISMA Checklist
• PRISMA Flow Diagram

Formulating a Research Question using PICO(T)

P=Population or problem or patient

• What are the characteristics of the patient or population? What is the condition?

I=Intervention or issue of interest

• What do you want to do with/for the patient or population?

C=Comparison

• What is the alternative to the intervention?
• What are the relevant outcomes?

A systematic review is a scientific study of all available evidence on a certain topic. It requires the most exhaustive literature search possible, not only in published literature, but also in gray literature. It may also require searches in disciplines outside the researchers primary area of study.

• Qualitative:  In this type of systematic review, the results of relevant studies are summarized but not statistically combined.
• Quantitative:  This type of systematic review uses statistical methods to combine the results of two or more studies.
• Meta-analysis:  A meta-analysis uses statistical methods to integrate estimates of effect from relevant studies that are independent but similar and summarize them.

Anybody writing a systematic literature review should be familiar with the  PRISMA statement . The PRISMA Statement is a document that consists of a 27-item  checklist  and a  flow diagram . It is designed to guide authors on how to develop a systematic review and what to include when writing the review.

A protocol will include:

• Databases to be searched and additional sources (particularly for grey literature)
• Keywords to be used in the search strategy
• Limits applied to the search.
• Screening process
• Data to be extracted
• Summary of data to be reported

The essence of a systematic review lies in being systematic. A systematic review involves detailed scrutiny and analysis of a huge mass of literature. To ensure that your work is efficient and effective, you should follow a  clear process :

1.  Develop a research question

2.  Define inclusion and exclusion criteria

3.  Locate studies 

4.  Select studies

5.  Assess study quality

6.  Extract data

7.  Analyze and present results

8.  Interpret results

9.  Update the review as needed

It is helpful to make notes at each stage. This will make it easier for you to write the review article.

A systematic review article follows the same structure as that of an original research article. It typically includes a title, abstract, introduction, methods, results, discussion, and references.  

Title:  The title should accurately reflect the topic under review. Typically, the words “a systematic review” are a part of the title to make the nature of the study clear.

Abstract:  A systematic review usually has a structured Abstract, with a short paragraph devoted to each of the following: background, methods, results, and conclusion. 

Introduction : The Introduction summarizes the topic and explains why the systematic review was conducted. There might have been gaps in the existing knowledge or a disagreement in the literature that necessitated a review. The introduction should also state the purpose and aims of the review.

Methods:  The Methods section is the most crucial part of a systematic review article. The methodology followed should be explained clearly and logically. The following components should be discussed in detail:

• Inclusion and exclusion criteria
• Identification of studies
• Study selection
• Data extraction
• Quality assessment
• Data analysis

Results:  The Results section should also be explained logically. You can begin by describing the search results, and then move on to the study range and characteristics, study quality, and finally discuss the effect of the intervention on the outcome.

Discussion:  The Discussion should summarize the main findings from the review and then move on to discuss the limitations of the study and the reliability of the results. Finally, the strengths and weaknesses of the review should be discussed, and implications for current practice suggested.

References:  The References section of a systematic review article usually contains an extensive number of references. You have to be very careful and ensure that you do not miss out on a single one. You can consider using reference management software to help you tackle the references effectively. 

Enter your manuscript's title and abstract and other requested information and these systems will identify journals that are best suited for publishing.  Each resource provides journal information and additional information such as impact factors, publishing model, time to publication, etc.

These tools search the journals of the individual publisher.  

• Elsevier Journal Finder
• IEEE Publication Recommender
• Journal/Author Name Estimator (JANE) - PubMed
• Sage Journal Selector
• SpringerNature Journal Suggester
• Taylor & Francis Journal Suggestor
• Web of Science Manuscript Matcher You will need to create a free account to access Match.
• Wiley Journal Finder (Beta)
• A young researcher's guide to a systematic review
• How to conduct systematic literature reviews in management research: a guide in 6 steps and 14 decisions P. C. Sauer and S. Seuring, “How to conduct systematic literature reviews in management research: a guide in 6 steps and 14 decisions,” Review of managerial science, vol. 17, no. 5, pp. 1899–1933, 2023, doi: 10.1007/s11846-023-00668-3

• << Previous: Search Strategy
• Next: PRISMA >>
• Last Updated: Sep 2, 2024 12:14 PM
• URL: https://libguides.unf.edu/nursing

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• Systematic Review | Definition, Example, & Guide

Systematic Review | Definition, Example & Guide

Published on June 15, 2022 by Shaun Turney . Revised on November 20, 2023.

A systematic review is a type of review that uses repeatable methods to find, select, and synthesize all available evidence. It answers a clearly formulated research question and explicitly states the methods used to arrive at the answer.

They answered the question “What is the effectiveness of probiotics in reducing eczema symptoms and improving quality of life in patients with eczema?”

In this context, a probiotic is a health product that contains live microorganisms and is taken by mouth. Eczema is a common skin condition that causes red, itchy skin.

Table of contents

What is a systematic review, systematic review vs. meta-analysis, systematic review vs. literature review, systematic review vs. scoping review, when to conduct a systematic review, pros and cons of systematic reviews, step-by-step example of a systematic review, other interesting articles, frequently asked questions about systematic reviews.

A review is an overview of the research that’s already been completed on a topic.

What makes a systematic review different from other types of reviews is that the research methods are designed to reduce bias . The methods are repeatable, and the approach is formal and systematic:

• Formulate a research question
• Develop a protocol
• Search for all relevant studies
• Apply the selection criteria
• Extract the data
• Synthesize the data
• Write and publish a report

Although multiple sets of guidelines exist, the Cochrane Handbook for Systematic Reviews is among the most widely used. It provides detailed guidelines on how to complete each step of the systematic review process.

Systematic reviews are most commonly used in medical and public health research, but they can also be found in other disciplines.

Systematic reviews typically answer their research question by synthesizing all available evidence and evaluating the quality of the evidence. Synthesizing means bringing together different information to tell a single, cohesive story. The synthesis can be narrative ( qualitative ), quantitative , or both.

Prevent plagiarism. Run a free check.

Systematic reviews often quantitatively synthesize the evidence using a meta-analysis . A meta-analysis is a statistical analysis, not a type of review.

A meta-analysis is a technique to synthesize results from multiple studies. It’s a statistical analysis that combines the results of two or more studies, usually to estimate an effect size .

A literature review is a type of review that uses a less systematic and formal approach than a systematic review. Typically, an expert in a topic will qualitatively summarize and evaluate previous work, without using a formal, explicit method.

Although literature reviews are often less time-consuming and can be insightful or helpful, they have a higher risk of bias and are less transparent than systematic reviews.

Similar to a systematic review, a scoping review is a type of review that tries to minimize bias by using transparent and repeatable methods.

However, a scoping review isn’t a type of systematic review. The most important difference is the goal: rather than answering a specific question, a scoping review explores a topic. The researcher tries to identify the main concepts, theories, and evidence, as well as gaps in the current research.

Sometimes scoping reviews are an exploratory preparation step for a systematic review, and sometimes they are a standalone project.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

A systematic review is a good choice of review if you want to answer a question about the effectiveness of an intervention , such as a medical treatment.

To conduct a systematic review, you’ll need the following:

• A precise question , usually about the effectiveness of an intervention. The question needs to be about a topic that’s previously been studied by multiple researchers. If there’s no previous research, there’s nothing to review.
• If you’re doing a systematic review on your own (e.g., for a research paper or thesis ), you should take appropriate measures to ensure the validity and reliability of your research.
• Access to databases and journal archives. Often, your educational institution provides you with access.
• Time. A professional systematic review is a time-consuming process: it will take the lead author about six months of full-time work. If you’re a student, you should narrow the scope of your systematic review and stick to a tight schedule.
• Bibliographic, word-processing, spreadsheet, and statistical software . For example, you could use EndNote, Microsoft Word, Excel, and SPSS.

A systematic review has many pros .

• They minimize research bias by considering all available evidence and evaluating each study for bias.
• Their methods are transparent , so they can be scrutinized by others.
• They’re thorough : they summarize all available evidence.
• They can be replicated and updated by others.

Systematic reviews also have a few cons .

• They’re time-consuming .
• They’re narrow in scope : they only answer the precise research question.

The 7 steps for conducting a systematic review are explained with an example.

Step 1: Formulate a research question

Formulating the research question is probably the most important step of a systematic review. A clear research question will:

• Allow you to more effectively communicate your research to other researchers and practitioners
• Guide your decisions as you plan and conduct your systematic review

A good research question for a systematic review has four components, which you can remember with the acronym PICO :

• Population(s) or problem(s)
• Intervention(s)
• Comparison(s)

You can rearrange these four components to write your research question:

• What is the effectiveness of I versus C for O in P ?

Sometimes, you may want to include a fifth component, the type of study design . In this case, the acronym is PICOT .

• Type of study design(s)
• The population of patients with eczema
• The intervention of probiotics
• In comparison to no treatment, placebo , or non-probiotic treatment
• The outcome of changes in participant-, parent-, and doctor-rated symptoms of eczema and quality of life
• Randomized control trials, a type of study design

Their research question was:

• What is the effectiveness of probiotics versus no treatment, a placebo, or a non-probiotic treatment for reducing eczema symptoms and improving quality of life in patients with eczema?

Step 2: Develop a protocol

A protocol is a document that contains your research plan for the systematic review. This is an important step because having a plan allows you to work more efficiently and reduces bias.

Your protocol should include the following components:

• Background information : Provide the context of the research question, including why it’s important.
• Research objective (s) : Rephrase your research question as an objective.
• Selection criteria: State how you’ll decide which studies to include or exclude from your review.
• Search strategy: Discuss your plan for finding studies.
• Analysis: Explain what information you’ll collect from the studies and how you’ll synthesize the data.

If you’re a professional seeking to publish your review, it’s a good idea to bring together an advisory committee . This is a group of about six people who have experience in the topic you’re researching. They can help you make decisions about your protocol.

It’s highly recommended to register your protocol. Registering your protocol means submitting it to a database such as PROSPERO or ClinicalTrials.gov .

Step 3: Search for all relevant studies

Searching for relevant studies is the most time-consuming step of a systematic review.

To reduce bias, it’s important to search for relevant studies very thoroughly. Your strategy will depend on your field and your research question, but sources generally fall into these four categories:

• Databases: Search multiple databases of peer-reviewed literature, such as PubMed or Scopus . Think carefully about how to phrase your search terms and include multiple synonyms of each word. Use Boolean operators if relevant.
• Handsearching: In addition to searching the primary sources using databases, you’ll also need to search manually. One strategy is to scan relevant journals or conference proceedings. Another strategy is to scan the reference lists of relevant studies.
• Gray literature: Gray literature includes documents produced by governments, universities, and other institutions that aren’t published by traditional publishers. Graduate student theses are an important type of gray literature, which you can search using the Networked Digital Library of Theses and Dissertations (NDLTD) . In medicine, clinical trial registries are another important type of gray literature.
• Experts: Contact experts in the field to ask if they have unpublished studies that should be included in your review.

At this stage of your review, you won’t read the articles yet. Simply save any potentially relevant citations using bibliographic software, such as Scribbr’s APA or MLA Generator .

• Databases: EMBASE, PsycINFO, AMED, LILACS, and ISI Web of Science
• Handsearch: Conference proceedings and reference lists of articles
• Gray literature: The Cochrane Library, the metaRegister of Controlled Trials, and the Ongoing Skin Trials Register
• Experts: Authors of unpublished registered trials, pharmaceutical companies, and manufacturers of probiotics

Step 4: Apply the selection criteria

Applying the selection criteria is a three-person job. Two of you will independently read the studies and decide which to include in your review based on the selection criteria you established in your protocol . The third person’s job is to break any ties.

To increase inter-rater reliability , ensure that everyone thoroughly understands the selection criteria before you begin.

If you’re writing a systematic review as a student for an assignment, you might not have a team. In this case, you’ll have to apply the selection criteria on your own; you can mention this as a limitation in your paper’s discussion.

You should apply the selection criteria in two phases:

• Based on the titles and abstracts : Decide whether each article potentially meets the selection criteria based on the information provided in the abstracts.
• Based on the full texts: Download the articles that weren’t excluded during the first phase. If an article isn’t available online or through your library, you may need to contact the authors to ask for a copy. Read the articles and decide which articles meet the selection criteria.

It’s very important to keep a meticulous record of why you included or excluded each article. When the selection process is complete, you can summarize what you did using a PRISMA flow diagram .

Next, Boyle and colleagues found the full texts for each of the remaining studies. Boyle and Tang read through the articles to decide if any more studies needed to be excluded based on the selection criteria.

When Boyle and Tang disagreed about whether a study should be excluded, they discussed it with Varigos until the three researchers came to an agreement.

Step 5: Extract the data

Extracting the data means collecting information from the selected studies in a systematic way. There are two types of information you need to collect from each study:

• Information about the study’s methods and results . The exact information will depend on your research question, but it might include the year, study design , sample size, context, research findings , and conclusions. If any data are missing, you’ll need to contact the study’s authors.
• Your judgment of the quality of the evidence, including risk of bias .

You should collect this information using forms. You can find sample forms in The Registry of Methods and Tools for Evidence-Informed Decision Making and the Grading of Recommendations, Assessment, Development and Evaluations Working Group .

Extracting the data is also a three-person job. Two people should do this step independently, and the third person will resolve any disagreements.

They also collected data about possible sources of bias, such as how the study participants were randomized into the control and treatment groups.

Step 6: Synthesize the data

Synthesizing the data means bringing together the information you collected into a single, cohesive story. There are two main approaches to synthesizing the data:

• Narrative ( qualitative ): Summarize the information in words. You’ll need to discuss the studies and assess their overall quality.
• Quantitative : Use statistical methods to summarize and compare data from different studies. The most common quantitative approach is a meta-analysis , which allows you to combine results from multiple studies into a summary result.

Generally, you should use both approaches together whenever possible. If you don’t have enough data, or the data from different studies aren’t comparable, then you can take just a narrative approach. However, you should justify why a quantitative approach wasn’t possible.

Boyle and colleagues also divided the studies into subgroups, such as studies about babies, children, and adults, and analyzed the effect sizes within each group.

Step 7: Write and publish a report

The purpose of writing a systematic review article is to share the answer to your research question and explain how you arrived at this answer.

Your article should include the following sections:

• Abstract : A summary of the review
• Introduction : Including the rationale and objectives
• Methods : Including the selection criteria, search method, data extraction method, and synthesis method
• Results : Including results of the search and selection process, study characteristics, risk of bias in the studies, and synthesis results
• Discussion : Including interpretation of the results and limitations of the review
• Conclusion : The answer to your research question and implications for practice, policy, or research

To verify that your report includes everything it needs, you can use the PRISMA checklist .

Once your report is written, you can publish it in a systematic review database, such as the Cochrane Database of Systematic Reviews , and/or in a peer-reviewed journal.

In their report, Boyle and colleagues concluded that probiotics cannot be recommended for reducing eczema symptoms or improving quality of life in patients with eczema. Note Generative AI tools like ChatGPT can be useful at various stages of the writing and research process and can help you to write your systematic review. However, we strongly advise against trying to pass AI-generated text off as your own work.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Student’s  t -distribution
• Normal distribution
• Null and Alternative Hypotheses
• Chi square tests
• Confidence interval
• Quartiles & Quantiles
• Cluster sampling
• Stratified sampling
• Data cleansing
• Reproducibility vs Replicability
• Peer review
• Prospective cohort study

Research bias

• Implicit bias
• Cognitive bias
• Placebo effect
• Hawthorne effect
• Hindsight bias
• Affect heuristic
• Social desirability bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

A systematic review is secondary research because it uses existing research. You don’t collect new data yourself.

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Turney, S. (2023, November 20). Systematic Review | Definition, Example & Guide. Scribbr. Retrieved September 16, 2024, from https://www.scribbr.com/methodology/systematic-review/

Is this article helpful?

Shaun Turney

Other students also liked, how to write a literature review | guide, examples, & templates, how to write a research proposal | examples & templates, what is critical thinking | definition & examples, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

A .gov website belongs to an official government organization in the United States.

A lock ( ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

• Table of Contents
• Chapter 1: Introduction
• Chapter 2: Creating Trustworthy Guidelines
• Chapter 3: Overview of the Guideline Development Process
• Chapter 4: Formulating PICO Questions
• Chapter 5: Choosing and Ranking Outcomes
• Chapter 6: Systematic Review Overview
• Chapter 7: GRADE Criteria Determining Certainty of Evidence
• Chapter 8: Domains Decreasing Certainty in the Evidence
• Chapter 9: Domains Increasing One's Certainty in the Evidence
• Chapter 10: Overall Certainty of Evidence
• Chapter 11: Communicating findings from the GRADE certainty assessment
• Chapter 12: Integrating Randomized and Non-randomized Studies in Evidence Synthesis

Related Topics:

• Advisory Committee on Immunization Practices (ACIP)
• Vaccine-Specific Recommendations
• Evidence-Based Recommendations—GRADE

Chapter 6: Systematic Review Overview

• This ACIP GRADE handbook provides guidance to the ACIP workgroups on how to use the GRADE approach for assessing the certainty of evidence.

The evidence base must be identified and retrieved systematically before the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach is used to assess the certainty of the evidence and provide support for guideline judgements. A systematic review should be used to retrieve the best available evidence related to the Population, Intervention, Comparison, and Outcomes (PICO) question. All guidelines should be preceded by a systematic review to ensure that recommendations and judgements are supported by an extensive body of evidence that addresses the research question. This section provides an overview of the systematic review process, external to the GRADE assessment of the certainty of evidence.

Systematic methods should be used to identify and synthesize the evidence 1 . In contrast to narrative reviews, systematic methods address a specific question and apply a rigorous scientific approach to the selection, appraisal, and synthesis of relevant studies. A systematic approach requires documentation of the search strategy used to identify all relevant published and unpublished studies and the eligibility criteria for the selection of studies. Systematic methods reduce the risk of selective citation and improve the reliability and accuracy of decisions. The Cochrane handbook provides guidance on searching for studies, including gray literature and unpublished studies ( Chapter 4: Searching for and selecting studies ) 1 .

6.1 Identifying the evidence

Guidelines should be based on a systematic review of the evidence 2 3 . A published systematic review can be used to inform the guideline, or a new one can be conducted. The benefits of identifying a previously conducted systematic review include reduced time and resources of conducting a review from scratch 3 . Additionally, if a Cochrane or other well-done systematic review exists on the topic of interest, the evidence is likely presented in a well-structured format and meets certain quality standards, thus providing a good evidence foundation for guidelines. As a result, systematic reviews do not need to be developed de novo if a high-quality review of the topic exists. Updating a relevant and recent high-quality review is usually less expensive and requires less time than conducting a review de novo. Databases, such as the Cochrane library, Medline (through PubMed or OVID), and EMBASE can be searched to identify existing systematic reviews which address the PICO question of interest. Additionally, the International Prospective Register of Systematic Reviews (PROSPERO) database can be searched to check for completed or on-going systematic reviews addressing the research question of interest 3 . It's important to base an evidence assessment and recommendations on a well-done systematic review to avoid any potential for bias to be introduced into the review, such as the inability to replicate methods or exclusion of relevant studies. Assessing the quality of a published systematic review can be done using the A Measurement Tool to Assess systematic Reviews (AMSTAR 2) instrument 3 . This instrument assesses the presence of the following characteristics in the review: relevancy to the PICO question; deviations from the protocol; study selection criteria; search strategy; data extraction process; risk of bias assessments for included studies; and appropriateness of both quantitative and qualitative synthesis 4 . A Risk of Bias of Systematic Reviews (ROBIS) assessment may also be performed 5 .

If a well-done systematic review is identified but the date of the last search is more than 6-12 months old, consider updating the search from the last date to ensure that all available evidence is captured to inform the guideline. In a well-done published systematic review, the search strategy will be provided, possibly as an online appendix or supplementary materials. Refer to the Evidence Retrieval section (6.3) for more information.

If a well-done published systematic review is not identified, then a de novo systematic review must be conducted. Once the PICO question(s) have been identified, conducting a systematic review includes the following steps:

• Protocol development
• Evidence retrieval and identification
• Risk of bias assessment
• A meta-analysis or narrative synthesis
• Assessment of the certainty of evidence using GRADE

6.2 Protocol development

There are several in-depth resources available to support authors when developing a systematic review; therefore, this and following sections will refer to higher-level points and provide information on those resources. The Cochrane Handbook serves as a fundamental reference for the development of systematic reviews and the PRISMA guidance provides detailed information on reporting requirements. To improve transparency and reduce the potential for bias to be introduced into the systematic review process, a protocol should be developed a priori to outline the methods of the planned systematic review. If the methods in the final systematic review deviate from the protocol (as is not uncommon), this must be noted in the final review with a rationale. Protocol development aims to reduce potential bias and ensure transparency in the decisions and judgements made by the review team. Protocols should document the predetermined PICO and study inclusion/exclusion criteria without the influence of the outcomes available in published primary studies 6 . The Preferred Reporting Items for Systematic review and Meta-Analysis Protocols (PRISMA-P) framework can be used to guide the development of a systematic review 7 . Details on the PRISMA-P statement and checklist are available at https://www.prisma-statement.org/protocols . 7 If the intention is to publish the systematic review in a peer-reviewed journal separately from the guideline, consider registering the systematic review using PROSPERO before beginning the systematic review process 8 .

To ensure the review is done well and meets the needs of the guideline authors, it is important to consider what type of evidence will be searched and included at the protocol stage before the evidence is retrieved 9 . While randomized controlled trials (RCTs) are often considered gold standards for evidence, there are many reasons why authors will choose to include nonrandomized studies (NRS) in their searches:

• To address baseline risks
• When RCTs aren't feasible, ethical or readily available
• When it is predicted that RCTs will have very serious concerns with indirectness (Refer to Table 12 for more information about Indirectness)

NRS can serve as complementary, sequential, or replacement evidence to RCTs depending on the situation 10 . Section 9 of this handbook provides detailed information about how to integrate NRS evidence. At the protocol stage it is important to consider whether or not NRS should be included.

The systematic review team will scope the available literature to develop a sense of whether or not the systematic review should be limited to RCTs alone or if a reliance on NRS may also be necessary. Once this inclusion and exclusion criteria has been established, the literature can be searched and retrieved systematically.

6.3 Evidence retrieval and identification

6.3a. searching databases.

An expert librarian or information specialist should be consulted to create a search strategy that is applied to all relevant databases to gather primary literature 1 . The following databases are widely used when conducting a systematic review: MEDLINE (via PubMed or OVID); EMBASE; Cochrane Central Register of Controlled Trials (CENTRAL). The details of each strategy as actually performed, with search terms (keywords and/or Medical Subject Headings/MESH terms) the date(s) on which the search was conducted and/or updated; and the publication dates of the literature covered, should be recorded.

In addition to searching for evidence, references from studies included for the review should also be examined to add anything relevant missed by the searches. It is also useful to examine clinical trials registries maintained by the federal government ( www.clinicaltrials.gov ) and vaccine manufacturers, and to consult subject matter experts. Ongoing studies should be recorded as well so that if the review or guideline were to be updated, these studies can be assessed for inclusion.

6.3b. Screening to identify eligible studies

The criteria for including/excluding evidence identified by the search, and the reasons for including and excluding evidence should be described (e.g., population characteristics, intervention, comparison, outcomes, study design, setting, language). Screening is typically conducted independently and in duplicate by at least two reviewers. Title and abstract screening is done first based on broader eligibility criteria and once relevant abstracts are selected, the full texts of those papers are pulled. The full-text screening is also usually conducted by two reviewers, independently and in duplicate with a more specific eligibility criteria to decide if the paper answers the PICO question or not. At both the title and abstract, and at the full-text stages, disagreements between reviewers can be resolved through discussion or involvement of a third reviewer. The goal of the screening process is to sort through the literature and select the most relevant studies for the review. To organize and conduct the systematic review, Covidence can be used to better manage each of the steps of the screening process. Other programs, such as DistillerSR or Rayyan can also be used to manage the screening process 11 12 . The PRISMA Statement ( www.prisma-statement.org ) includes guidance on reporting the methods for evidence retrieval. A PRISMA flow diagram (Figure 3) presents the systematic review search process and results.

Figure 3. PRISMA flow diagram depicting the flow of information through the different phases of the systematic review evidence retrieval process, including the number of records identified, records included and excluded at each stage, and the reasons for exclusions.

References in this figure: 13

*Consider, if feasible to do so, reporting the number of records identified from each database or register searched (rather than the total number across all databases/registers).

**If automation tools were used, indicate how many records were excluded by a human and how many were excluded by automation tools.

From: Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. doi: 10.1136/bmj.n71. For more information, visit: http://www.prisma-statement.org/

6.3c. Data extraction

Once included articles have been screened and selected, relevant information from the articles should be extracted systematically using a standardized and pilot-tested data extraction form. Table 3 provides an example of an ACIP data extraction form (data fields may differ by topic and scope); Microsoft Excel can be used to keep track of and extract relevant details about each study. Data extraction forms typically capture information about: 1) study details (author, publication year, title, funding source, etc.); 2) study characteristics (study design, geographical location, population, etc.); 3) study population (demographics, disease severity, etc.); 4) intervention and comparisons (e.g., type of vaccine/placebo/control, dose, number in series, etc.); 5) outcome measures. For example, for dichotomously reported outcomes, the number of people with the outcome per study arm and the total number of people in each study arm are noted. In contrast, for continuous outcomes, the total number of people in each study arm, the mean or median, as well as standard deviation or standard error are extracted. This is the information needed to conduct a quantitative synthesis. If this information is not provided in the study, reviewers may want to reach out to the authors for more information or contact a statistician about alternative approaches to quantifying data. After extracting the studies, risk of bias should be assessed using an appropriate tool described in Section 8.1 of this handbook.

Table 3. Example of a data extraction form for included studies

*total and per group

6.4 Conducting the meta-analysis

After the data has been retrieved, if appropriate, it can be statistically combined to produce a pooled estimate of the relative (e.g., risk ratio, odds ratio, hazard ratio) or absolute (e.g., mean difference, standard mean difference) effect for the body of evidence of each outcome. A meta-analysis can be performed when there are at least two studies that report on the same outcome. Several software programs are available that can be used to perform a meta-analysis, including R, STATA, and Review Manager (RevMan).

The results from a meta-analysis are presented in a forest plot as presented in figure 4. A forest plot presents the effect estimates and confidence intervals for each individual study and a pooled estimate of all the studies included in the meta-analysis 14 . The square represents the effect estimate and the horizontal line crossing the square is indicative of the confidence interval (CI; typically 95% CI). The area the square covers reflects the weight given to the study in the analysis. The summary result is presented as a diamond at the bottom.

Figure 4. Estimates of effect for RCTs included in analysis for outcome of incidence of arthralgia (0-42 days)

References in this figure: 15

The two most popular statistical methods for conducting meta-analyses are the fixed-effects model and the random-effects model 14 . These two models typically generate similar effect estimates when used in meta-analyses. However, these models are not interchangeable, and each model makes a different assumption about the data being analyzed.

A fixed-effects model assumes that there is one true effect size that can be identified across all included studies; therefore, all observed differences between studies are attributed to sampling error. The fixed effect model is used when all the studies are assumed to share a common effect size 16 . Before using the fixed-effect model in a meta-analysis, consideration should be made as to whether the results will be applied to only the included studies. Since the fixed-effect model provides the pooled effect estimate for the population in the studies included in the analysis, it should not be used if the goal is to generalize the estimate to other populations.

In contrast, a random-effects model, some variability between the true effect sizes studies is accepted. These effect sizes are assumed to follow a normal distribution. The confidence intervals generated by the random-effects model are typically wider than those generated by the fixed-effect model, as they recognize that some variability in the findings can be due to differences between the primary studies. The weights of the studies are also more similar under the random-effects model. When variations in, for example, the participants or methods across different included studies is suspected, it is suggested to use a random-effects model. This is because the studies are weighed more evenly than the fixed effect model. The majority of analyses will meet the criteria to use a random effects mode. One caveat about the selection of models: when the number of studies included in the analysis is few (<3), the random-effects model will produce an estimate of variance with poor precision. In this situation, a fixed effect model will be a more appropriate way to conduct the meta-analysis 17 .

• Lefebvre C, Glanville J, Briscoe S, et al. Chapter 4: Searching for and selecting studies. In: Higgins J, Thomas J, Chandler J, et al, eds. Cochrane Handbook for Systematic Reviews of Interventions version 63 (updated February 2022). Cochrane; 2022. www.training.cochrane.org/handbook.
• Committee on Standards for Developing Trustworthy Clinical Practice Guidelines BoHCS, Institute of Medicine. Clinical Practice Guidelines We Can Trust. National Academies Press; 2011.
• World Health Organization. WHO handbook for guideline development, 2nd ed. 2014: World Health Organization. 167.
• Shea BJ, Reeves BC, Wells G, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017/09/21/ 2017:j4008. doi:10.1136/bmj.j4008
• Bristol Uo. ROBIS tool.
• Lasserson T, Thomas J, Higgins J. Chapter 1: Starting a review. In: Higgins J, Thomas J, Chandler J, et al, eds. Cochrane Handbook for Systematic Reviews of Interventions version 63. 2022. www.training.cochrane.org/handbook
• Moher D, Shamseer L, Clarke M, et al. Preferred reporting items for systematic review and meta analysis protocols (PRISMA-P) 2015 statement. Syst Rev. Jan 1 2015;4:1. doi:10.1186/2046- 4053-4-1
• PROSPERO. York.ac.uk. https://www.crd.york.ac.uk/PROSPERO/
• Cuello-Garcia CA, Santesso N, Morgan RL, et al. GRADE guidance 24 optimizing the integration of randomized and non-randomized studies of interventions in evidence syntheses and health guidelines. J Clin Epidemiol. 2022/02// 2022;142:200-208. doi:10.1016/j.jclinepi.2021.11.026
• Schünemann HJ, Tugwell P, Reeves BC, et al. Non-randomized studies as a source of complementary, sequential or replacement evidence for randomized controlled trials in systematic reviews on the effects of interventions. Research Synthesis Methods. 2013 2013;4(1):49-62. doi:10.1002/jrsm.1078
• DistillerSR | Systematic Review and Literature Review Software. DistillerSR.
• Rayyan – Intelligent Systematic Review. https://www.rayyan.ai/
• Page MJ, McKenzie JE, Bossuyt PM, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. doi:10.1136/bmj.n71
• Deeks J, Higgins J, Altman D. Chapter 10: Analysing data and undertaking meta-analyses. In: Higgins J, Thomas J, Chandler J, et al, eds. Cochrane Handbook for Systematic Reviews of v.04_2024 20 Interventions version 63 (updated February 2022). Cochrane; 2022. www.training.cochrane.org/handbook .
• Choi MJ, Cossaboom CM, Whitesell AN, et al. Use of ebola vaccine: recommendations of the Advisory Committee on Immunization Practices, United States, 2020. MMWR Recommendations and Reports. 2021;70(1):1.
• Borenstein M, Hedges LV, Higgins JP, Rothstein HR. Introduction to meta-analysis. John Wiley & Sons; 2021.
• Borenstein M, Hedges LV, Higgins JP, Rothstein HR. A basic introduction to fixed-effect and random-effects models for meta-analysis. Research Synthesis Methods. 2010;1:97-111. doi:DOI: 10.1002/jrsm.12

ACIP GRADE Handbook

This handbook provides guidance to the ACIP workgroups on how to use the GRADE approach for assessing the certainty of evidence.

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals

You are here

• Volume 19, Issue 1
• Reviewing the literature
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Joanna Smith 1 ,
• Helen Noble 2
• 1 School of Healthcare, University of Leeds , Leeds , UK
• 2 School of Nursing and Midwifery, Queens's University Belfast , Belfast , UK
• Correspondence to Dr Joanna Smith , School of Healthcare, University of Leeds, Leeds LS2 9JT, UK; j.e.smith1{at}leeds.ac.uk

https://doi.org/10.1136/eb-2015-102252

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Implementing evidence into practice requires nurses to identify, critically appraise and synthesise research. This may require a comprehensive literature review: this article aims to outline the approaches and stages required and provides a working example of a published review.

Are there different approaches to undertaking a literature review?

What stages are required to undertake a literature review.

The rationale for the review should be established; consider why the review is important and relevant to patient care/safety or service delivery. For example, Noble et al 's 4 review sought to understand and make recommendations for practice and research in relation to dialysis refusal and withdrawal in patients with end-stage renal disease, an area of care previously poorly described. If appropriate, highlight relevant policies and theoretical perspectives that might guide the review. Once the key issues related to the topic, including the challenges encountered in clinical practice, have been identified formulate a clear question, and/or develop an aim and specific objectives. The type of review undertaken is influenced by the purpose of the review and resources available. However, the stages or methods used to undertake a review are similar across approaches and include:

Formulating clear inclusion and exclusion criteria, for example, patient groups, ages, conditions/treatments, sources of evidence/research designs;

Justifying data bases and years searched, and whether strategies including hand searching of journals, conference proceedings and research not indexed in data bases (grey literature) will be undertaken;

Developing search terms, the PICU (P: patient, problem or population; I: intervention; C: comparison; O: outcome) framework is a useful guide when developing search terms;

Developing search skills (eg, understanding Boolean Operators, in particular the use of AND/OR) and knowledge of how data bases index topics (eg, MeSH headings). Working with a librarian experienced in undertaking health searches is invaluable when developing a search.

Once studies are selected, the quality of the research/evidence requires evaluation. Using a quality appraisal tool, such as the Critical Appraisal Skills Programme (CASP) tools, 5 results in a structured approach to assessing the rigour of studies being reviewed. 3 Approaches to data synthesis for quantitative studies may include a meta-analysis (statistical analysis of data from multiple studies of similar designs that have addressed the same question), or findings can be reported descriptively. 6 Methods applicable for synthesising qualitative studies include meta-ethnography (themes and concepts from different studies are explored and brought together using approaches similar to qualitative data analysis methods), narrative summary, thematic analysis and content analysis. 7 Table 1 outlines the stages undertaken for a published review that summarised research about parents’ experiences of living with a child with a long-term condition. 8

• View inline

An example of rapid evidence assessment review

In summary, the type of literature review depends on the review purpose. For the novice reviewer undertaking a review can be a daunting and complex process; by following the stages outlined and being systematic a robust review is achievable. The importance of literature reviews should not be underestimated—they help summarise and make sense of an increasingly vast body of research promoting best evidence-based practice.

• ↵ Centre for Reviews and Dissemination . Guidance for undertaking reviews in health care . 3rd edn . York : CRD, York University , 2009 .
• ↵ Canadian Best Practices Portal. http://cbpp-pcpe.phac-aspc.gc.ca/interventions/selected-systematic-review-sites / ( accessed 7.8.2015 ).
• Bridges J , et al
• ↵ Critical Appraisal Skills Programme (CASP). http://www.casp-uk.net / ( accessed 7.8.2015 ).
• Dixon-Woods M ,
• Shaw R , et al
• Agarwal S ,
• Jones D , et al
• Cheater F ,

Twitter Follow Joanna Smith at @josmith175

Competing interests None declared.

Read the full text or download the PDF:

Nursing - Systematic Reviews: Levels of Evidence

• Levels of Evidence
• Meta-Analyses
• Definitions
• Citation Search
• Write & Cite
• Give Feedback

"How would I use the 6S Model while taking care of a patient?" .cls-1{fill:#fff;stroke:#79a13f;stroke-miterlimit:10;stroke-width:5px;}.cls-2{fill:#79a13f;} The 6S Model is designed to work from the top down, starting with Systems - also referred to as computerized decision support systems (CDSSs). DiCenso et al. describes that, “an evidence-based clinical information system integrates and concisely summarizes all relevant and important research evidence about a clinical problem, is updated as new research evidence becomes available, and automatically links (through an electronic medical record) a specific patient’s circumstances to the relevant information” (2009). Systematic reviews lead up to this type of bio-available level of evidence.

What are systematic reviews, polit–beck evidence hierarchy/levels of evidence scale for therapy questions.

"Figure 2.2 [in context of book] shows our eight-level evidence hierarchy for Therapy/intervention questions. This hierarchy ranks sources of evidence with respect the readiness of an intervention to be put to use in practice" (Polit & Beck, 2021, p. 28). Levels are ranked on risk of bias - level one being the least bias, level eight being the most biased. There are several types of levels of evidence scales designed for answering different questions. "An evidence hierarchy for Prognosis questions, for example, is different from the hierarchy for Therapy questions" (p. 29).

Advantages of Levels of Evidence Scales

"Through controls imposed by manipulation, comparison, and randomization, alternative explanations can be discredited. It is because of this strength that meta-analyses of RCTs, which integrate evidence from multiple experiments, are at the pinnacle of the evidence hierarchies for Therapy questions" (p. 188).

"Tip: Traditional evidence hierarchies or level of evidence scales (e.g., Figure 2.2), rank evidence sources almost exclusively based on the risk of internal validity threats" (p. 217).

Systematic reviews can provide researchers with knowledge that prior evidence shows. This can help clarify established efficacy of a treatment without unnecessary and thus unethical research. Greenhalgh (2019) illustrates this citing Dean Fergusson and colleagues (2005) systematic review on a clinical surgical topic (p. 128).

Limits of Levels of Evidence Scales

Regarding the importance of real-world clinical practice settings, and the conflicting tradeoffs between internal and external validity, Polit and Beck (2021) write, "the first (and most prevalent) approach is to emphasize one and sacrifice another. Most often, it is external validity that is sacrificed. For example, external validity is not even considered in ranking evidence in level of evidence scales" (p. 221). ... From an EBP perspective, it is important to remember that drawing inferences about causal relationships relies not only on how high up on the evidence hierarchy a study is (Figure 2.2), but also, for any given level of the hierarchy, how successful the researcher was in managing study validity and balancing competing validity demands" (p. 222).

Polit and Beck note Levin (2014) that an evidence hierarchy "is not meant to provide a quality rating for evidence retrieved in the search for an answer" (p. 6), and as the Oxford Center for Evidence-Based Medicine concurs that evidence scales are, 'NOT intended to provide you with a definitive judgment about the quality of the evidence. There will inevitably be cases where "lower-level" evidence...will provide stronger than a "higher level" study (Howick et al., 2011, p.2)'" (p. 30).

Level of evidence (e.g., Figure 2.2) + Quality of evidence = Strength of evidence .

The 6S Model of Levels of Evidence

"The 6S hierarchy does not imply a gradient of evidence in terms of quality , but rather in terms of ease in retrieving relevant evidence to address a clinical question. At all levels, the evidence should be assessed for quality and relevance" (Polit & Beck, 2021, p. 24, Tip box).

The 6S Pyramid proposes a structure of quantitative evidence where articles that include pre-appraised and pre-synthesized studies are located at the top of the hierarchy (McMaster U., n.d.).

It can help to consider the level of evidence that a document represents, for example, a scientific article that summarizes and analyses many similar articles may provide more insight than the conclusion of a single research article. This is not to say that summaries can not be flawed, nor does it suggest that rare case studies should be ignored. The aim of health research is the well-being of all people, therefore it is important to use current evidence in light of patient preferences negotiated with clinical expertise.

Other Gradings in Levels of Evidence

While it is accepted that the strongest evidence is derived from meta-analyses, various evidence grading systems exist. for example: The Johns Hopkins Nursing Evidence-Based Practice model ranks evidence from level I to level V, as follows (Seben et al., 2010): Level I: Meta-analysis of randomized clinical trials (RCTs); experimental studies; RCTs Level II: Quasi-experimental studies Level III: Non-experimental or qualitative studies Level IV: Opinions of nationally recognized experts based on research evidence or an expert consensus panel Level V: Opinions of individual experts based on non-research evidence (e.g., case studies, literature reviews, organizational experience, and personal experience) The American Association of Critical-Care Nurses (AACN) evidence level system , updated in 2009, ranks evidence as follows (Armola et al., 2009): Level A: Meta-analysis of multiple controlled studies or meta-synthesis of qualitative studies with results that consistently support a specific action, intervention, or treatment Level B: Well-designed, controlled randomized or non-randomized studies with results that consistently support a specific action, intervention, or treatment Level C: Qualitative, descriptive, or correlational studies, integrative or systematic reviews, or RCTs with inconsistent results Level D: Peer-reviewed professional organizational standards, with clinical studies to support recommendations Level E: Theory-based evidence from expert opinion or multiple case reports Level M: Manufacturers’ recommendations (2017)

EBM Pyramid and EBM Page Generator

Unfiltered are resources that are primary sources describing original research. Randomized controlled trials, cohort studies, case-controlled studies, and case series/reports are considered unfiltered information.

Filtered are resources that are secondary sources which summarize and analyze the available evidence. They evaluate the quality of individual studies and often provide recommendations for practice. Systematic reviews, critically-appraised topics, and critically-appraised individual articles are considered filtered information.

Armola, R. R., Bourgault, A. M., Halm, M. A., Board, R. M., Bucher, L., Harrington, L., ... Medina, J. (2009). AACN levels of evidence. What's new? Critical Care Nurse , 29 (4), 70-73. doi:10.4037/ccn2009969

DiCenso, A., Bayley, L., & Haynes, R. B. (2009). Accessing pre-appraised evidence: Fine-tuning the 5S model into a 6S model. BMJ Evidence-Based Nursing , 12 (4) https://ebn.bmj.com/content/12/4/99.2.short

Fergusson, D., Glass, K. C., Hutton, B., & Shapiro, S. (2005). Randomized controlled trials of Aprotinin in cardiac surgery: Could clinical equipoise have stopped the bleeding?. Clinical Trials , 2 (3), 218-232.

Glover, J., Izzo, D., Odato, K. & Wang, L. (2008). Evidence-based mental health resources . EBM Pyramid and EBM Page Generator. Copyright 2008. All Rights Reserved. Retrieved April 28, 2020 from https://web.archive.org/web/20200219181415/http://www.dartmouth.edu/~biomed/resources.htmld/guides/ebm_psych_resources.html Note. Document removed from host. Old link used with the WayBack Machine of the Internet Archive to retrieve the original webpage on 2/10/21 http://www.dartmouth.edu/~biomed/resources.htmld/guides/ebm_psych_resources.html

Greenhalgh, T. (2019). How to read a paper: The basics of evidence-based medicine and healthcare . (Sixth ed.). Wiley Blackwell.

Haynes, R. B. (2001). Of studies, syntheses, synopses, and systems: The “4S” evolution of services for finding current best evidence. BMJ Evidence-Based Medicine , 6 (2), 36-38.

Haynes, R. B. (2006). Of studies, syntheses, synopses, summaries, and systems: the “5S” evolution of information services for evidence-based healthcare decisions. BMJ Evidence-Based Medicine , 11 (6), 162-164.

McMaster University (n.d.). 6S Search Pyramid Tool https://www.nccmt.ca/capacity-development/6s-search-pyramid

Polit, D., & Beck, C. (2019). Nursing research: Generating and assessing evidence for nursing practice . Wolters Kluwer Health.

Schub, E., Walsh, K. & Pravikoff D. (Ed.) (2017). Evidence-based nursing practice: Implementing [Skill Set]. Nursing Reference Center Plus

Seben, S., March, K. S., & Pugh, L. C. (2010). Evidence-based practice: The forum approach. American Nurse Today , 5 (11), 32-34.

• Systematic Review from the Encyclopedia of Nursing Research by Cheryl Holly Systematic reviews provide reliable evidential summaries of past research for the busy practitioner. By pooling results from multiple studies, findings are based on multiple populations, conditions, and circumstances. The pooled results of many small and large studies have more precise, powerful, and convincing conclusions (Holly, Salmond, & Saimbert, 2016) [ references in article ]. This scholarly synthesis of research findings and other evidence forms the foundation for evidence-based practice allowing the practitioner to make up-to-date decisions.

Standards & Guides

• Cochrane Handbook for Systematic Reviews of Interventions The Cochrane Handbook for Systematic Reviews of Interventions is the official guide that describes in detail the process of preparing and maintaining Cochrane systematic reviews on the effects of healthcare interventions.
• Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) PRISMA is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. PRISMA focuses on the reporting of reviews evaluating randomized trials, but can also be used as a basis for reporting systematic reviews of other types of research, particularly evaluations of interventions.
• Systematic Reviews by The Centre for Reviews and Dissemination "The guidance has been written for those with an understanding of health research but who are new to systematic reviews; those with some experience but who want to learn more; and for commissioners. We hope that experienced systematic reviewers will also find this guidance of value; for example when planning a review in an area that is unfamiliar or with an expanded scope. This guidance might also be useful to those who need to evaluate the quality of systematic reviews, including, for example, anyone with responsibility for implementing systematic review findings" (CRD, 2009, p. vi, "Who should use this guide")

• Carrying out systematic literature reviews: An introduction by Alan Davies Systematic reviews provide a synthesis of evidence for a specific topic of interest, summarising the results of multiple studies to aid in clinical decisions and resource allocation. They remain among the best forms of evidence, and reduce the bias inherent in other methods. A solid understanding of the systematic review process can be of benefit to nurses that carry out such reviews, and for those who make decisions based on them. An overview of the main steps involved in carrying out a systematic review is presented, including some of the common tools and frameworks utilised in this area. This should provide a good starting point for those that are considering embarking on such work, and to aid readers of such reviews in their understanding of the main review components, in order to appraise the quality of a review that may be used to inform subsequent clinical decision making (Davies, 2019, Abstract)
• Papers that summarize other papers (systematic reviews and meta-analyses) by Trisha Greenhalgh ... a systematic review is an overview of primary studies that: contains a statement of objectives, sources and methods; has been conducted in a way that is explicit, transparent and reproducible (Figure 9.1) [ Table found in book chapter ]. The most enduring and reliable systematic reviews, notably those undertaken by the Cochrane Collaboration (discussed later in this chapter), are regularly updated to incorporate new evidence (Greenhalgh, 2020, p. 117, Chapter 9).
• A PRISMA assessment of the reporting quality of systematic reviews of nursing published in the Cochrane Library and paper-based journals by Juxia Zhang et al. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) was released as a standard of reporting systematic reviewers (SRs). However, not all SRs adhere completely to this standard. This study aimed to evaluate the reporting quality of SRs published in the Cochrane Library and paper-based journals (Zhang et al., 2019, Abstract).

Cochrane [Username]. (2016, Jan 27). What are systematic reviews? YouTube. https://www.youtube.com/watch?v=egJlW4vkb1Y

Davies, A. (2019). Carrying out systematic literature reviews: An introduction. British Journal of Nursing , 28 (15), 1008–1014. https://doi-org.ezproxy.simmons.edu/10.12968/bjon.2019.28.15.1008

Greenhalgh, T. (2019). Papers that summarize other papers (systematic reviews and meta-analyses). In How to read a Paper : The basics of evidence-based medicine and healthcare . (Sixth ed., pp. 117-136). Wiley Blackwell.

Holly, C. (2017). Systematic review. In J. Fitzpatrick (Ed.), Encyclopedia of nursing research (4th ed.). Springer Publishing Company. Credo Reference.

Zhang, J., Han, L., Shields, L., Tian, J., & Wang, J. (2019). A PRISMA assessment of the reporting quality of systematic reviews of nursing published in the Cochrane Library and paper-based journals. Medicine , 98 (49), e18099. https://doi.org/10.1097/MD.0000000000018099

• << Previous: Start
• Next: Meta-Analyses >>
• Last Updated: Nov 3, 2023 1:19 PM
• URL: https://simmons.libguides.com/systematic-reviews

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Hiatal hernias revisited—a systematic review of definitions, classifications, and applications, 1. introduction, 4. discussion, 5. conclusions, author contributions, institutional review board statement, informed consent statement, data availability statement, conflicts of interest.

• Akerlund, A.I. Hernia diaphragmatica hiatus oesophagei vom anatomischen und rontgenologischen gesichtspunkt. Acta Radiol. 1926 , 6 , 3–22. [ Google Scholar ] [ CrossRef ]
• Guthrie, D.; Jones, F.H. The frequency and diagnosis of hiatal hernia. Ann. Surg. 1940 , 111 , 971–978. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Harrington, S.W. The Surgical Treatment of the More Common Type of Diaphragmatic Hernia: Report of 404 Cases. Ann. Surg. 1945 , 122 , 546–568. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Stapleton, J.G. Oesophageal hiatus hernia. Can. Med. Assoc. J. 1947 , 57 , 13–16. [ Google Scholar ] [ PubMed ] [ PubMed Central ]
• Trueman, K.R. Diagnosis and treatment of para-oesophageal hiatus hernia. Can. Med. Assoc. J. 1947 , 56 , 149–153. [ Google Scholar ] [ PubMed ] [ PubMed Central ]
• Allison, P.R. Peptic ulcer of the oesophagus. Thorax 1948 , 3 , 20–42. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Olsen, A.M.; Harrington, S.W. Esophageal hiatal hernias of the short esophagus type; etiologic and therapeutic considerations. J. Thorac. Surg. 1948 , 17 , 189–209. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Merendino, K.A.; Varco, R.L.; Wangensteen, O.H. Displacement of the Esophagus into a New Diaphragmatic Orifice in the Repair of Paraesophageal and Esophageal Hiatus Hernia. Ann. Surg. 1949 , 129 , 185–197. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Stephens, H.B. The problem of the acquired short esophagus; report of 18 patients. Calif. Med. 1949 , 71 , 385–390. [ Google Scholar ] [ PubMed ] [ PubMed Central ]
• Kaplan, S. Oesophageal hiatus hernia. A clinical study of 45 cases. Postgrad. Med. J. 1951 , 27 , 165–177. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Sweet, R.H. Esophageal hiatus hernia of the diaphragm; the anatomical characteristics, technic of repair, and results of treatment in 111 consecutive cases. Ann. Surg. 1952 , 135 , 1–13. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Barrett, N.R. Hiatus hernia. Proc. R. Soc. Med. 1952 , 45 , 279–286. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Macdougall, J.T.; Abbott, A.C. Sliding hiatus hernia; a review of cases. Can. Med. Assoc. J. 1957 , 77 , 752–756. [ Google Scholar ] [ PubMed ] [ PubMed Central ]
• Barrett, N.R. Hiatus hernia. Br. Med. J. 1960 , 2 , 247–252. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Collis, J.L. A review of surgical results in hiatus hernia. Thorax 1961 , 16 , 114–119. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Paulson, D.L.M.; Shaw, R.R.M.; Kee, J.L.M. Esophageal hiatal diaphragmatic hernia and its complications. Ann. Surg. 1962 , 155 , 957–970. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Borgeskov, S.; Pedersen, O.T.; Frederiksen, T. Oesophageal Hiatal Hernia: A Radiological Follow-up. Thorax 1964 , 19 , 327–331. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Edwards, D.A.W.; Phillips, S.F.; Rowlands, E.N. Clinical and Radiological Results of Repair of Hiatus Hernia. Br. Med. J. 1964 , 2 , 714–718. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Brindley, G.V. Surgical Treatment of Complications of Esopliageal Hiatal Hernia. Ann. Surg. 1965 , 161 , 723–731. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Clagett, O.T. Present concepts regarding the surgical treatment of oesophageal hiatal hernia. Ann. R. Coll. Surg. Engl. 1966 , 38 , 195–209. [ Google Scholar ] [ PubMed ] [ PubMed Central ]
• Krupp, S.; Rossetti, M. Surgical treatment of hiatal hernias by fundoplication and gastropexy (Nissen repair). Ann. Surg. 1966 , 164 , 927–934. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Zellos, S. Surgical treatment of hiatal hernia, with particular reference to the transthoracic subdiaphragmatic approach. Thorax 1966 , 21 , 295–304. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Skinner, D.B.; Belsey, R.H. Surgical management of esophageal reflux and hiatus hernia. Long-term results with 1,030 patients. J. Thorac. Cardiovasc. Surg. 1967 , 53 , 33–54. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Hill, L.D. An effective operation for hiatal hernia: An eight year appraisal. Ann. Surg. 1967 , 166 , 681–692. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Nissen, R.; Pfeifer, K. Aktuelle Probleme in der Chirurgie: Zwerchfellhernien ; Verlag Hans Huber Bern: Bern, Switzerland, 1967; pp. 31–72. [ Google Scholar ]
• Borrie, J. Surgical treatment of hiatal hernia: A 10-year survey. Thorax 1967 , 22 , 344–350. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Hamelmann, H. Hiatal hernias in adults. Langenbecks Arch. Chir. 1968 , 322 , 361–370. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Allison, P.R. Hiatus hernia: (a 20-year retrospective survey). Ann. Surg. 1973 , 178 , 273–276. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Hoffman, E.; Sumner, M.C. A clinical and radiological review of 204 hiatal hernia operations. Thorax 1973 , 28 , 379–385. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Mokka, R.E.; Laitinen, S.; Punto, L.; Kairaluoma, M.I.; Pokela, R.; Kärkölä, P.; Huttunen, R.; Larmi, T.K. Hiatal hernia repair. Ann. Chir. Gynaecol. 1976 , 65 , 369–375. [ Google Scholar ] [ PubMed ]
• Jacobsson, S.I. Hiatal hernia. Follow-up of a ten-year material. Acta Chir. Scand. Suppl. 1976 , 464 , 1–28. [ Google Scholar ] [ PubMed ]
• Maher, J.W.; Hollenbeck, J.I.; Woodward, E.R. An analysis of recurrent esophagitis following posterior gastropexy. Ann. Surg. 1978 , 187 , 227–230. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Rossetti, M. Indication et tactique chirurgicales dans le reflux et la hernie hiatale [Surgical indications and tactics in reflux and hiatal hernia]. Rev. Med. Suisse Rom. 1982 , 102 , 753–756. (In French) [ Google Scholar ] [ PubMed ]
• Pearson, F.G.; Cooper, J.D.; Ilves, R.; Todd, T.R.; Jamieson, W.R. Massive hiatal hernia with incarceration: A report of 53 cases. Ann. Thorac. Surg. 1983 , 35 , 45–51. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Hallissey, M.T.; Ratliff, D.A.; Temple, J.G. Paraoesophageal hiatus hernia: Surgery for all ages. Ann. R. Coll. Surg. Engl. 1992 , 74 , 23–25. [ Google Scholar ] [ PubMed ] [ PubMed Central ]
• Kuster, G.G.; Gilroy, S. Laparoscopic technique for repair of paraesophageal hiatal hernias. J. Laparoendosc. Surg. 1993 , 3 , 331–338. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Menguy, R. Le traitement chirugical des hernies hiatales par roulement avec volvulus intrathoracique de la totalité de l’estomac [Surgical treatment of paraesophageal hiatal hernia with total intrathoracic volvulus of the stomach]. Chirurgie 1994 , 120 , 439–442; discussion 442–443. (In French) [ Google Scholar ] [ PubMed ]
• Jamieson, G.G.; Watson, D.I.; Britten-Jones, R.; Mitchell, P.C.; Anvari, M. Laparoscopic Nissen fundoplication. Ann. Surg. 1994 , 220 , 137–145. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Oddsdottir, M.; Franco, A.L.; Laycock, W.S.; Waring, J.P.; Hunter, J.G. Laparoscopic repair of paraesophageal hernia. New access, old technique. Surg. Endosc. 1995 , 9 , 164–168. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Myers, G.A.; Harms, B.A.; Starling, J.R. Management of paraesophageal hernia with a selective approach to antireflux surgery. Am. J. Surg. 1995 , 170 , 375–380. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Hill, L.D.; Kozarek, R.A.; Kraemer, S.J.; Aye, R.W.; Mercer, C.D.; Low, D.E.; Pope, C.E., 2nd. The gastroesophageal flap valve: In vitro and in vivo observations. Gastrointest. Endosc. 1996 , 44 , 541–547. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Perdikis, G.; Hinder, R.A.; Filipi, C.J.; Walenz, T.; McBride, P.J.; Smith, S.L.; Katada, N.; Klingler, P.J. Laparoscopic paraesophageal hernia repair. Arch. Surg. 1997 , 132 , 586–589; discussion 590–591. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Edye, M.B.; Canin-Endres, J.; Gattorno, F.; Salky, B.A. Durability of laparoscopic repair of paraesophageal hernia. Ann. Surg. 1998 , 228 , 528–535. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Altorki, N.K.; Yankelevitz, D.; Skinner, D.B. Massive hiatal hernias: The anatomic basis of repair. J. Thorac. Cardiovasc. Surg. 1998 , 115 , 828–835. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Maziak, D.E.; Todd, T.R.; Pearson, F.G. Massive hiatus hernia: Evaluation and surgical management. J. Thorac. Cardiovasc. Surg. 1998 , 115 , 53–60; discussion 61–62. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Wu, J.S.; Dunnegan, D.L.; Soper, N.J. Clinical and radiologic assessment of laparoscopic paraesophageal hernia repair. Surg. Endosc. 1999 , 13 , 497–502. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Swanstrom, L.L.; Jobe, B.A.; Kinzie, L.R.; Horvath, K.D. Esophageal motility and outcomes following laparoscopic paraesophageal hernia repair and fundoplication. Am. J. Surg. 1999 , 177 , 359–363. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Luketich, J.D.; Raja, S.; Fernando, H.C.; Campbell, W.; Christie, N.A.; Buenaventura, P.O.; Weigel, T.L.; Keenan, R.J.; Schauer, P.R. Laparoscopic repair of giant paraesophageal hernia: 100 consecutive cases. Ann. Surg. 2000 , 232 , 608–618. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Dahlberg, P.S.; Deschamps, C.; Miller, D.L.; Allen, M.S.; Nichols, F.C.; Pairolero, P.C. Laparoscopic repair of large paraesophageal hiatal hernia. Ann. Thorac. Surg. 2001 , 72 , 1125–1129. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Champion, J.K.; Rock, D. Laparoscopic mesh cruroplasty for large paraesophageal hernias. Surg. Endosc. 2003 , 17 , 551–553. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Leeder, P.C.; Smith, G.; Dehn, T.C. Laparoscopic management of large paraesophageal hiatal hernia. Surg. Endosc. 2003 , 17 , 1372–1375. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Patel, H.J.; Tan, B.B.; Yee, J.; Orringer, M.B.; Iannettoni, M.D. A 25-year experience with open primary transthoracic repair of paraesophageal hiatal hernia. J. Thorac. Cardiovasc. Surg. 2004 , 127 , 843–849. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Andujar, J.J.; Papasavas, P.K.; Birdas, T.; Robke, J.; Raftopoulos, Y.; Gagné, D.J.; Caushaj, P.F.; Landreneau, R.J.; Keenan, R.J. Laparoscopic repair of large paraesophageal hernia is associated with a low incidence of recurrence and reoperation. Surg. Endosc. 2004 , 18 , 444–447. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Targarona, E.M.; Novell, J.; Vela, S.; Cerdán, G.; Bendahan, G.; Torrubia, S.; Kobus, C.; Rebasa, P.; Balague, C.; Garriga, J.; et al. Mid term analysis of safety and quality of life after the laparoscopic repair of paraesophageal hiatal hernia. Surg. Endosc. 2004 , 18 , 1045–1050. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Zaninotto, G.; Portale, G.; Costantini, M.; Rizzetto, C.; Guirroli, E.; Ceolin, M.; Salvador, R.; Rampado, S.; Prandin, O.; Ruol, A.; et al. Long-term results (6–10 years) of laparoscopic fundoplication. J. Gastrointest. Surg. 2007 , 11 , 1138–1145. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Boushey, R.P.; Moloo, H.; Burpee, S.; Schlachta, C.M.; Poulin, E.C.; Haggar, F.; Trottier, D.C.; Mamazza, J. Laparoscopic repair of paraesophageal hernias: A Canadian experience. Can. J. Surg. 2008 , 51 , 355–360. [ Google Scholar ] [ PubMed ] [ PubMed Central ]
• Luketich, J.D.; Nason, K.S.; Christie, N.A.; Pennathur, A.; Jobe, B.A.; Landreneau, R.J.; Schuchert, M.J. Outcomes after a decade of laparoscopic giant paraesophageal hernia repair. J. Thorac. Cardiovasc. Surg. 2010 , 139 , 395–404, 404.e1. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Mittal, S.K.; Bikhchandani, J.; Gurney, O.; Yano, F.; Lee, T. Outcomes after repair of the intrathoracic stomach: Objective follow-up of up to 5 years. Surg. Endosc. 2011 , 25 , 556–566. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Pallabazzer, G.; Santi, S.; Parise, P.; Solito, B.; Giusti, P.; Rossi, M. Giant hiatal hernias: Direct hiatus closure has an acceptable recurrence rate. Updates Surg. 2011 , 63 , 75–81. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Carrott, P.W.; Hong, J.; Kuppusamy, M.; Kirtland, S.; Koehler, R.P.; Low, D.E. Repair of giant paraesophageal hernias routinely produces improvement in respiratory function. J. Thorac. Cardiovasc. Surg. 2012 , 143 , 398–404. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Lugaresi, M.; Mattioli, S.; Aramini, B.; D’Ovidio, F.; Di Simone, M.P.; Perrone, O. The frequency of true short oesophagus in type II-IV hiatal hernia. Eur. J. Cardiothorac. Surg. 2013 , 43 , e30–e36. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Bjelović, M.; Babic, T.; Gunjić, D.; Veselinović, M.; Spica, B. Laparoscopic repair of hiatal hernias: Experience after 200 consecutive cases. Srp. Arh. Celok. Lek. 2014 , 142 , 424–430. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Latzko, M.; Borao, F.; Squillaro, A.; Mansson, J.; Barker, W.; Baker, T. Laparoscopic repair of paraesophageal hernias. JSLS J. Soc. Laparosc. Robot. Surg. 2014 , 18 , e2014.00009. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Chang, C.G.; Thackeray, L. Laparoscopic Hiatal Hernia Repair in 221 Patients: Outcomes and Experience. JSLS J. Soc. Laparosc. Robot. Surg. 2016 , 20 , e2015.00104. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Stringham, J.R.; Phillips, J.V.; McMurry, T.L.; Lambert, D.L.; Jones, D.R.; Isbell, J.M.; Lau, C.L.; Kozower, B.D. Prospective study of giant paraesophageal hernia repair with 1-year follow-up. J. Thorac. Cardiovasc. Surg. 2017 , 154 , 743–751. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Mertens, A.C.; Tolboom, R.C.; Zavrtanik, H.; Draaisma, W.A.; Broeders, I.A.M.J. Morbidity and mortality in complex robot-assisted hiatal hernia surgery: 7-year experience in a high-volume center. Surg. Endosc. 2019 , 33 , 2152–2161. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Shea, B.; Boyan, W.; Decker, J.; Almagno, V.; Binenbaum, S.; Matharoo, G.; Squillaro, A.; Borao, F. Emergent Repair of Paraesophageal Hernias and the Argument for Elective Repair. JSLS J. Soc. Laparosc. Robot. Surg. 2019 , 23 , e2019.00015. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Dara, V.; Croo, A.; Peirsman, A.; Pattyn, P. Necessity of fundoplication and mesh in the repair of the different types of paraesophageal hernia. Acta Gastroenterol. Belg. 2019 , 82 , 251–256. [ Google Scholar ] [ PubMed ]
• Sorial, R.K.; Ali, M.; Kaneva, P.; Fiore, J.F., Jr.; Vassiliou, M.; Fried, G.M.; Feldman, L.S.; Ferri, L.E.; Lee, L.; Mueller, C.L. Modern era surgical outcomes of elective and emergency giant paraesophageal hernia repair at a high-volume referral center. Surg. Endosc. 2020 , 34 , 284–289. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Fuchs, K.H.; Breithaupt, W.; Varga, G.; Babic, B.; Schulz, T.; Meining, A. Primary laparoscopic fundoplication in selected patients with gastroesophageal reflux disease. Dis. Esophagus 2022 , 35 , doab032. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Rodríguez-Luna, M.R.; Pizzicannella, M.; Fiorillo, C.; Almuttawa, A.; Lapergola, A.; Mutter, D.; Marrescaux, J.; Dallemagne, B.; Perretta, S. Impact of surgical repair on type IV paraesophageal hernias (PEHs). Surg. Endosc. 2022 , 36 , 5467–5475. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• D’Elia, M.A.; Ahmadi, N.; Jarrar, A.; Neville, A.; Mamazza, J. Paraesophageal hernia repair in elderly patients: Outcomes from a 10-year retrospective study. Can. J. Surg. 2022 , 65 , E121–E127. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Amundson, J.R.; Kuchta, K.; Wu, H.; VanDruff, V.N.; Haggerty, S.P.; Linn, J.; Ujiki, M.B. A 13-year experience with biologic and biosynthetic absorbable mesh reinforced laparoscopic paraesophageal hernia repair. Surg. Endosc. 2023 , 37 , 7271–7279. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Salvador, R.; Vittori, A.; Capovilla, G.; Riccio, F.; Nezi, G.; Forattini, F.; Provenzano, L.; Nicoletti, L.; Moletta, L.; Costantini, A.; et al. Antireflux Surgery’s Lifespan: 20 Years After Laparoscopic Fundoplication. J. Gastrointest. Surg. 2023 , 27 , 2325–2335. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Giulini, L.; Razia, D.; Latorre-Rodríguez, A.R.; Shacker, M.; Csucska, M.; Mittal, S.K. Surgical Repair of Large Hiatal Hernias: Insight from a High-Volume Center. J. Gastrointest. Surg. 2023 , 27 , 2308–2315. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Furnée, E.J.; Draaisma, W.A.; Gooszen, H.G.; Hazebroek, E.J.; Smout, A.J.; Broeders, I.A. Tailored or routine addition of an antireflux fundoplication in laparoscopic large hiatal hernia repair: A comparative cohort study. World J. Surg. 2011 , 35 , 78–84. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Dallemagne, B.; Kohnen, L.; Perretta, S.; Weerts, J.; Markiewicz, S.; Jehaes, C. Laparoscopic repair of paraesophageal hernia. Long-term follow-up reveals good clinical outcome despite high radiological recurrence rate. Ann. Surg. 2011 , 253 , 291–296. [ Google Scholar ] [ CrossRef ]
• Stylopoulos, N.; Rattner, D.W. The history of hiatal hernia surgery: From Bowditch to laparoscopy. Ann. Surg. 2005 , 241 , 185–193. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Gyawali, C.P.; Kahrilas, P.J.; Savarino, E.; Zerbib, F.; Mion, F.; Smout, A.J.P.M.; Vaezi, M.; Sifrim, D.; Fox, M.R.; Vela, M.F.; et al. Modern diagnosis of GERD: The Lyon Consensus. Gut 2018 , 67 , 1351–1362. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Kahrilas, P.J.; Kim, H.C.; Pandolfino, J.E. Approaches to the diagnosis and grading of hiatal hernia. Best. Pract. Res. Clin. Gastroenterol. 2008 , 22 , 601–616. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• DeMeester, T.R. Etiology and Natural History of Gastroesophageal Reflux Disease and Predictors of progressive Disease. In Shackelford’s Surgery of the Alimentary Tract , 8th ed.; Yeo, C.J., DeMeester, S.R., Fadden, D.W.M., Eds.; Elsevier: Amsterdam, The Netherlands, 2019; pp. 204–220. [ Google Scholar ]
• Fuchs, K.H.; DeMeester, T.R.; Otte, F.; Broderick, R.C.; Breithaupt, W.; Varga, G.; Musial, F. Severity of GERD and disease progression. Dis. Esophagus 2021 , 34 , doab006. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• DeMeester, T.R.; Johnson, L.F.; Joseph, G.J.; Toscano, M.S.; Hall, A.W.; Skinner, D.B. Patterns of gastroesophageal reflux in health and disease. Ann. Surg. 1976 , 184 , 459–470. [ Google Scholar ] [ CrossRef ]
• Costantini, M.; Crookes, P.F.; Bremner, R.M.; Hoeft, S.F.; Ehsan, A.; Peters, J.H.; Bremner, C.G.; DeMeester, T.R. Value of physiologic assessment of foregut symptoms in a surgical practice. Surgery 1993 , 114 , 780–786. [ Google Scholar ] [ PubMed ]
• Tack, J.; Caenepeel, P.; Arts, J.; Lee, K.J.; Sifrim, D.; Janssens, J. Prevalence of acid reflux functional dyspepsia and its association with symptom profile. Gut 2005 , 54 , 1370–1376. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Rydberg, L.; Ruth, M.; Abrahamsson, H.; Lundell, L. Tailoring antireflux surgery: A randomized clinical trial. World J. Surg. 1999 , 23 , 612–618. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Fibbe, C.; Layer, P.; Keller, J.; Strate, U.; Emmermann, A.; Zornig, C. Esophageal motility in reflux disease before and after fundoplication: A prospective, randomized, clinical, and manometric study. Gastroenterology 2001 , 121 , 5–14. [ Google Scholar ] [ CrossRef ]
• Dallemagne, B.; Weertz, J.; Markiewicz, S.; Dewandre, J.M.; Wahlen, C.; Monami, B.; Jehaes, C. Clinical results of laparoscopic fundoplication ten years after surgery. Surg. Endosc. 2006 , 20 , 159–165. [ Google Scholar ] [ CrossRef ]
• Frantzides, C.T.; Madan, A.K.; Carlson, M.A.; Stavropoulos, G.P. A prospective, randomized trial of laparoscopic polytetrafluoroethylene (PTFE) patch repair vs simple cruroplasty for large hiatal hernia. Arch. Surg. 2002 , 137 , 649–652. [ Google Scholar ] [ CrossRef ]
• Granderath, F.A.; Schweiger, U.M.; Kamolz, T.; Pasiut, M.; Haas, C.F.; Pointner, R. Laparoscopic antireflux surgery with routine mesh-hiatoplasty in the treatment of gastroesophageal reflux disease. J. Gastrointest. Surg. 2002 , 6 , 347–353. [ Google Scholar ] [ CrossRef ]
• Bonavina, L.; DeMeester, T.; Fockens, P.; Dunn, D.; Saino, G.; Bona, D.; Lipham, J.; Bemelman, W.; Ganz, R.A. Laparoscopic sphincter augmentation device eliminates reflux symptoms and normalizes esophageal acid exposure: One- and 2-year results of a feasibility trial. Ann. Surg. 2010 , 252 , 857–862. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Jobe, B.A.; Richter, J.E.; Hoppo, T.; Peters, J.H.; Bell, R.; Dengler, W.C.; DeVault, K.; Fass, R.; Gyawali, C.P.; Kahrilas, P.J.; et al. Preoperative diagnostic workup before antireflux surgery: An evidence and experience-based consensus of the Esophageal Diagnostic Advisory Panel. J. Am. Coll. Surg. 2013 , 217 , 586–597. [ Google Scholar ] [ CrossRef ]
• Linke, G.R.; Gehrig, T.; Hogg, L.V.; Göhl, A.; Kenngott, H.; Schäfer, F.; Fischer, L.; Gutt, C.N.; Müller-Stich, B.P. Laparoscopic mesh-augmented hiatoplasty without fundoplication as a method to treat large hiatal hernias. Surg. Today 2014 , 44 , 820–826. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• DeMeester, S.R. Laparoscopic paraesophageal hernia repair: Critical steps and adjunct techniques to minimize recurrence. Surg. Laparosc. Endosc. Percutan Tech. 2013 , 23 , 429–435. [ Google Scholar ] [ CrossRef ]
• Gerdes, S.; Schoppmann, S.F.; Bonavina, L.; Boyle, N.; Müller-Stich, B.P.; Gutschow, C.A.; Hiatus Hernia Delphi Collaborative Group. Management of paraesophageal hiatus hernia: Recommendations following a European expert Delphi consensus. Surg. Endosc. 2023 , 37 , 4555–4565. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Fuchs, K.H.; Babic, B.; Breithaupt, W.; Dallemagne, B.; Fingerhut, A.; Furnee, E.; Granderath, F.; Horvath, O.P.; Kardos, P.; Pointner, R.; et al. EAES recommendations for the management of Gastroesophageal reflux Disease. Surg. Endosc. 2014 , 28 , 1753–1773. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Tolone, S.; Savarino, E.; Zaninotto, G.; Gyawali, C.P.; Frazzoni, M.; de Bortoli, N.; Frazzoni, L.; Del Genio, G.; Bodini, G.; Furnari, M.; et al. High-resolution manometry is superior to endoscopy and radiology in assessing and grading sliding hiatal hernia: A comparison with surgical in vivo evaluation. United Eur. Gastroenterol. J. 2018 , 6 , 981–989. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Granderath, F.A.; Schweiger, U.M.; Pointner, R. Laparoscopic antireflux surgery: Tailoring the hiatal closure to the size of hiatal surface area. Surg. Endosc. 2007 , 21 , 542–548. [ Google Scholar ] [ CrossRef ]
• Grubnik, V.V.; Malynovskyy, A.V. Laparoscopic repair of hiatal hernias: New classification supported by long-term results. Surg. Endosc. 2013 , 27 , 4337–4346. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Köckerling, F.; Zarras, K.; Adolf, D.; Kraft, B.; Jacob, D.; Weyhe, D.; Schug-Pass, C. What Is the Reality of Hiatal Hernia Management? A Registry Analysis. Front. Surg. 2020 , 7 , 584196. [ Google Scholar ] [ CrossRef ] [ PubMed ] [ PubMed Central ]
• Kafetzis, I.; Fuchs, K.H.; Sodmann, P.; Troya, J.; Zoller, W.; Meining, A.; Hann, A. Efficient artificial intelligence-based assessment of the gastroesophageal valve with Hill classification through active learning. Sci. Rep. 2024 , 14 , 18825. [ Google Scholar ] [ CrossRef ] [ PubMed ]

Click here to enlarge figure

Share and Cite

Fuchs, K.H.; Kafetzis, I.; Hann, A.; Meining, A. Hiatal Hernias Revisited—A Systematic Review of Definitions, Classifications, and Applications. Life 2024 , 14 , 1145. https://doi.org/10.3390/life14091145

Fuchs KH, Kafetzis I, Hann A, Meining A. Hiatal Hernias Revisited—A Systematic Review of Definitions, Classifications, and Applications. Life . 2024; 14(9):1145. https://doi.org/10.3390/life14091145

Fuchs, Karl Hermann, Ioannis Kafetzis, Alexander Hann, and Alexander Meining. 2024. "Hiatal Hernias Revisited—A Systematic Review of Definitions, Classifications, and Applications" Life 14, no. 9: 1145. https://doi.org/10.3390/life14091145

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

• Open access
• Published: 17 September 2024

Mapping health-related quality of life of children and families receiving pediatric invasive home mechanical ventilation: a scoping review protocol

• Keisha White Makinde   ORCID: orcid.org/0000-0003-2643-1303 1 ,
• Maysara Mitchell 1 ,
• Alexandra F. Merz 1 &
• Michael Youssef 1  

Systematic Reviews volume  13 , Article number:  236 ( 2024 ) Cite this article

Metrics details

Children utilizing invasive home mechanical ventilation (administered via tracheostomy tube) receive intensive care at home without the support of trained staff typically present in an intensive care unit; within the context of worsening home nursing shortages, much of the 24/7 care burden falls to families which are likely under supported. Prior reviews have explored the quality of life of children receiving various forms of mechanical ventilation, without addressing the impact on the family. Additionally, the literature inconsistently differentiates the unique experience of families with children using invasive home mechanical ventilation from non-invasive, which has lower morbidity and mortality and requires less nursing care in the home. Therefore, our study aims to explore and map the existing literature regarding the impact of invasive home mechanical ventilation on the child and family’s quality of life. Identified gaps will inform future research focused on improving the family quality of life of children with invasive home mechanical ventilation.

Five databases will be searched using keywords and controlled vocabulary to identify relevant studies: Ovid Medline, Embase, Scopus, and Cochrane Library. English language studies will meet inclusion criteria if they include primary research studies of children or families of children utilizing invasive home mechanical ventilation at home and assess quality of life. Children and young adults aged 0–25 years will be included. We exclude studies of hospitalized children, studies focused solely on healthcare professional experiences or clinical outcomes, and those focused on the period surrounding discharge from admission for tracheostomy placement. Two independent reviewers will screen studies at the title/abstract and full-text levels. Two independent reviewers will extract data from relevant studies. Disagreements will be resolved by an independent third reviewer. A targeted grey literature search will be performed utilizing ProQuest, clinicaltrials.gov, WHO trial registry, Google Scholar, and professional societies. Findings will be presented in tables and figures along with a narrative summary.

This scoping review seeks to map the literature and provide a descriptive report of the health-related quality of life of children using invasive home mechanical ventilation and their families.

Registration

Open Science Framework https://doi.org/10.17605/OSF.IO/6GB84

Date of Registration: November 29, 2023.

Peer Review reports

The number of children with medical complexity who utilize medical technology is growing [ 1 , 2 , 3 ]. This has resulted in increased survival and longer lifespan [ 4 , 5 ]—two quality metrics frequently used to assess the quality of healthcare received. Yet, stakeholders, payors, and patient advocates have highlighted the importance of quality of life as a critical metric that should also be used to determine the success or failure of a healthcare intervention [ 6 , 7 , 8 ]. Despite this, there has been a lag in the collection of quality of life indicators and integration into metrics for high-quality healthcare, particularly for new pediatric technologies [ 7 , 9 ]. This scoping review protocol focuses on quality of life in regards to a medical technology which provides the highest level of medical life support available in the home setting—pediatric invasive home mechanical ventilation (HMV).

Home mechanical ventilation is a vital intervention for children experiencing chronic respiratory failure, promoting respiratory stability and enhanced longevity in the comfort of a child’s home. HMV has the potential to benefit children with respiratory conditions by improving alveolar ventilation, alleviating symptoms associated with chronic respiratory failure, improving blood gases, reducing morbidity and mortality, and enhancing the child’s quality of life [ 10 ]. HMV can be administered either invasively or noninvasively. Non-invasive HMV (for instance bilevel positive airway pressure (BIPAP)) provides a lower amount of breathing support and is administered via mask. Conversely, invasive ventilation provides a much higher amount of breathing support and is administered via a surgically placed tracheostomy tube which connects to the breathing machine. Originally conceived for pediatric patients with isolated spinal injury or neuromuscular conditions 11, over time the use of invasive HMV has expanded to treat children with more complex diseases and multiple comorbidities. Typical pediatric patients utilizing invasive HMV may have primary lung diseases such as chronic lung disease of prematurity [ 12 , 13 ], underlying genetic conditions such as congenital central hypoventilation syndrome [ 14 ], and cardiopulmonary disease [ 4 , 13 , 15 ]. Invasive HMV is associated with a higher risk for morbidity, frequent and high acuity readmission, and higher mortality compared with non-invasive ventilation [ 4 , 16 ]. Any sudden loss of tracheostomy patency (e.g., mucus plug, accidental tracheostomy decannulation) or loss of ventilator function (e.g., loss of power in the home, unaddressed ventilator alarms) would be imminently life threatening. Due to the tenuousness of maintaining a patent airway, the American Thoracic Society guidelines recommends that optimal care for children with invasive home mechanical ventilation requires 24/7 hands-on nursing care in the home, two trained family caregivers living in the child’s household, along with monitoring equipment and regular multidisciplinary clinic visits [ 8 ]. In reality, the extensive and unrelenting nursing shortages throughout the country [ 17 , 18 ] mean that much of the in-home care falls to parents and families creating a significant burden for which they are underprepared.

Parents of children who utilize HMV are expected to provide extensive and ongoing care for their child—administering medications, managing medical emergencies that may arise, and calling off from work when home nursing services are inaccessible. There are few studies that examine the impacts on the family for children with invasive HMV [ 19 , 20 ]. On preliminary search, available studies focus on the parent [ 19 , 21 ] with little information on siblings or how the family’s functioning is impacted. Studies of HMV (including both invasive and non-invasive) highlight that a parent’s assumption of the primary caregiver role for their ventilator-dependent child can exacerbate financial burdens [ 9 , 10 ], sleep deprivation [ 10 ], and anxiety, impacting the overall family quality of life [ 11 ]. Chan et al. and Wang and Barnard interviewed parents of children who utilize ventilators at home and expose the significant strain on personal and romantic relationships, lack of friends and supports, frequent needs to transition between parent and caregiver, and constant worry that the child might suddenly die [ 22 , 23 ]. Further complicating the situation, nearly half of parents of children with any medical technology use (ranging from nebulizers and glucose monitors to ventilators) endorse having a need for respite care within the past year, yet only half of those needing respite had their needs met, often due to lack of availability or cost [ 24 ]. Considering these are the lived experiences of families of mostly children with non-invasive ventilation, it is imperative to understand more about the family quality of life of children utilizing invasive home mechanical ventilation given its increased demands for nursing, family caregiving, and higher morbidity and mortality.

Health-related quality of life (HRQOL) is a multidimensional concept described by many scholars which can be thought of as an individual’s outlook or perspective on life and its resultant satisfaction (or dissatisfaction) given the presence of a medical condition. This perspective is grounded in the context of the individual’s culture and value systems which are related to their goals, expectations, and concerns [ 25 ]. For the purpose of this review, we focus on the health-related quality of life of children, adolescents, and young adults with invasive home mechanical ventilation. For this protocol, we focus on the following domains [ 26 , 27 ]: (1) physical functioning, (2) psychological functioning, (3) social functioning, (4) cognitive functioning, and (5) general well-being.

We use the theory of health-related family quality of life (HRFQOL) as coined by Radina et al. [ 28 ] as the unifying theory for this review; the theory (see Fig.  1 ) defines HRFQOL as the intersection between the patient’s health-related quality of life and the family’s quality of life. Health-related family quality of life (HRFQOL) comprises 3 concepts: emotional closeness, family sense of coherence, and family functioning (see Fig.  2 ). For this review, we focus on the concepts and sub-concepts delineated by Radina et al. (see Fig.  2 ).

Diagram of domains of quality of life

Conceptual model of the theory of health-related family quality of life

Family can have many definitions. For the purposes of this review, we draw upon the definition of family quality of life offered by Park et al. as “people who think of themselves as part of the family, whether related by blood or marriage or not, and who support and care for each other on a regular basis” [ 29 ].

Tracheostomy is a surgical airway management procedure whereby an incision is made in the trachea to divert the passage of air for breathing. Tracheostomy is used interchangeably with tracheotomy for the purposes of this review. Patients with a tracheostomy may breathe independently or with assistance of a ventilator. For this review, we focus on patients with tracheostomy who utilize mechanical ventilation.

Mechanical ventilation is a type of assisted breathing whereby a medical device (i.e., ventilator) is used to fully or partially provide artificial ventilation. Practically, the support can be positive pressure ventilation (pressure-supported ventilation or bilevel positive airway pressure (BiPAP) or continuous positive airway pressure (CPAP). For this review, we focus on any level of mechanical ventilation that is administered through a tracheostomy. Patients with isolated oxygen use without ventilator support will be excluded. Invasive mechanical ventilation refers to ventilation delivered through a breathing tube—an endotracheal tube or tracheostomy tube. For the purpose of this review, we focus solely on invasive mechanical ventilation delivered via tracheostomy tube.

We define home as a location where the patient primarily lives with family. We exclude long-term care facilities in the definition of home for this study.

A preliminary search of PROSPERO, MEDLINE, the Cochrane Database of Systematic Reviews, and JBI Evidence Synthesis was conducted, and no current or in-progress scoping reviews or systematic reviews on this specific topic of interest. Mattson et al. [ 30 ] recently published a scoping review focused on quality of life of children with home mechanical ventilation; although informative, this review does not differentiate the experience of children living with invasive mechanical ventilation and their families. Our scoping review differs in two key ways. First, we focus on the family’s quality of life instead of the child’s quality of life. Secondly, we spotlight the experiences of children with invasive home mechanical ventilation, given their increased medically fragility, requisite home nursing needs, and higher risk of morbidity and mortality. Additionally, our review is strengthened by the use of a framework definition of health-related family quality of life from Radina et al. [ 28 ]which extends our focus beyond the challenges faced by parents to include the impact on the entire family unity, including siblings and extended family. By utilizing this approach and definition, we have preliminarily identified additional studies meeting our criteria which were not included in the review conducted by Mattson and colleagues, underscoring the differences in our search and screening approaches.

The existing literature predominantly focuses on mortality rates and medical outcomes of children utilizing HMV, with limited attention to the vital issue of family quality of life; though it is significantly impacted when intensive medical care is introduced in the home environment [ 31 ]. In addition to the few studies available, even fewer focus on family-level quality of life indicators or experiences of families with medically complex children. While previous scoping reviews have explored these concepts separately, no scoping review has reviewed both the health-related quality of life of the pediatric patient as well as the family quality of life of the overall family unit. Additionally, we aim to provide an updated overview of the literature, given the most recent scoping reviews assessed studies through 2020 [ 30 , 32 ].

The primary objective of this scoping review is to comprehensively map the existing literature on pediatric invasive mechanical ventilation in the home environment to understand the child’s health-related quality of life (HRQOL) and the health-related family quality of life (HRFQOL).

Authors will utilize the JBI Manual for Evidence Synthesis Chapter 11 entitled “Scoping Reviews” as a guideline for rigorous procedures [ 33 ]. Authors will utilize the Preferred Reporting Items for Systematic Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) as a guideline for reporting the findings of the scoping review [ 34 ].

This review will consider primary studies that include pediatric patients (0–25 years) who utilize invasive HMV, as well as the experiences of their family members. Furthermore, this review will consider studies that explore (1) the relationship between pediatric invasive HMV and health-related family quality of life (HRFQOL), (2) the relationship between pediatric invasive HMV and health-related quality of life (HRQOL) among children, adolescents, and young adults and their families, and (3) studies that address the experiences of children who utilize home mechanical ventilation with a tracheostomy. Studies must be published in English from 2004 to 2024 to meet inclusion criteria. This review will consider studies that explore the presence of invasive mechanical ventilation in the home environment. Studies that address the presence of invasive mechanical ventilation in the healthcare setting or within long-term care facilities will not be included. This scoping review will consider quantitative, qualitative, and mixed methods study designs for inclusion.

The search strategy will aim to locate published primary studies and grey literature, excluding reviews, and text and opinion papers. An initial limited search of the literature was undertaken by the medical librarian to identify studies on the topic. The text words contained in the titles and abstracts of relevant studies, and the index terms used to describe the studies, were used to develop an initial search strategy for Embase. The search strategy, including all identified keywords and index terms, will be adapted for each included information source. The initial search strategy will be piloted and adapted in partnership with the medical librarian to develop the final search strategy (see Appendix 1). Studies published in an English publication between 2004 and 2024 will be included. The databases to be searched include Ovid Medline 1946-, Embase.com 1947-, Scopus 1823-, and Cochrane Library 1996-.

Following the search, all identified records will be collated and uploaded into Endnote v.21 (Clarivate Analytics, PA, USA) and Covidence (Veritas Health Innovation, Melbourne, Australia), and duplicates removed. Covidence will be used to screen and manage the results of the scoping review to optimize collaboration and thoroughness among the research team. Pilot testing of source selection will be conducted prior to screening. The primary investigator will select a random sample ( n  = 20) of studies, which will be independently reviewed by all reviewers. Reviewers will screen the titles and abstracts using the inclusion and exclusion criteria provided in Covidence. After all pilot studies have been reviewed, reviewers will meet to discuss inter-rater reliability. If reliability is > 75%, reviewers will proceed with Screening. If reliability does not reach 75%, reviewers will have an in-depth discussion regarding discrepancies. Inclusion and exclusion criteria will be modified to meet newly shared understanding. Then reviewers will separately pilot the revised criteria. The formal screening will proceed once 75% agreement is reached.

During screening, titles and abstracts will be assessed by 2 independent reviewers against the inclusion criteria. Reviewers 1 and 2 will independently assess the title and abstract of each article using Covidence’s title and abstract screening feature. If there is disagreement among Reviewers 1 and 2, Reviewer 3—the primary investigator—will review the title and abstract to make the final determination on the article’s eligibility. To ensure fidelity to protocol, the primary investigator will provide oversight and review a random selection of screenings completed by Reviewers 1 and 2 to confirm adherence to review protocol. Potentially relevant papers will be retrieved in full and imported into Covidence.

During full-text review, the full text of selected citations will be assessed in detail against the inclusion criteria by 2 independent reviewers. Reasons for exclusion of full-text papers that do not meet the inclusion criteria will be recorded and reported in Covidence. Any disagreements that arise between the reviewers at each stage of the selection process will be resolved through discussion or with a third reviewer. The results of the search will be reported in full in the final scoping review and presented in a PRISMA flow diagram [ 21 ]. To ensure fidelity to protocol, the primary investigator will provide oversight and review a random selection of full texts completed by Reviewers 1 and 2 to confirm adherence to review protocol.

Several sources will be used to inform the grey literature search, which will be an adaptation of methods described by Godin et al. 35 . Utilizing keywords identified previously, we will search ProQuest Dissertations & Theses, National Library of Medicine clinical trials registry (clinicaltrials.gov), World Health Organization International Clinical Trials Registry Platform (trialsearch.who.int), Google Scholar, and relevant professional societies to identify clinical guidelines, dissertations and thesis, reports, and other findings from organizations that fit the inclusion criteria for this scoping review. The intention remains to identify and map the scientific literature and consensus statements from reputable sources while excluding sources with individualistic viewpoints, such as social media and blog postings.

Prior to the start of data extraction, the data extraction instrument will be pilot tested on 3 sources to ensure all relevant results are consistently extracted. Each reviewer will read the pilot studies and extract data using the extraction instrument in Covidence. Team members will then meet to discuss discrepancies, offer clarification, and make any modifications to the extraction instrument. Extraction will begin once consensus on the extraction instrument is reached.

Data will be extracted from papers included in the scoping review by 2 independent reviewers using a data extraction tool developed by the reviewers (see Appendix 2). The data extracted will include specific details about the population, concept, context, methods, and key findings, relevant to the review question. A draft extraction tool is provided (see Appendix 2). The draft data extraction tool will be modified and revised as necessary during the process of extracting data from each included paper. Modifications will be detailed in the full scoping review. Any disagreements that arise between the reviewers will be resolved through discussion or with a third reviewer. Authors of papers will be contacted to request missing or additional data, where required.

We will first present a flow diagram of our scoping review methodology including the study selection process. Extracted data will be analyzed using figures and tables, frequency counts of concepts, populations, and study characteristics. Then, we will utilize a table to present an overview of study characteristics including year, country, participant characteristics, and methodology. Lastly, we will present in a table an overview of themes and concepts elicited in the included studies.

This scoping review has begun and is in the data selection phase at submission. Inclusion and exclusion criteria were revised following screening. Notably, we exclude conference abstracts; although these are often published in peer-reviewed journals, they many times do not include thorough details of the data and results to allow us to extract findings with confidence that they represent the true and full findings of the primary research. Additionally, conference abstracts often presented preliminary results which may have changed following the abstract acceptance. We conducted a search using the abstract first author to determine if a manuscript was available; none of our conference abstracts had corresponding manuscripts and thus were excluded. We also had several clinical trial registrations returned in our search protocol; these registrations were excluded from review; however, the authors contacted the clinical trial primary investigator to ascertain if the study had concluded and if a published manuscript was available; if manuscripts become available in the process, we will add them to title/abstract screening and perform screening consistent with the full protocol.

Availability of data and materials

Data sharing is not applicable to this article as no datasets were generated or analyzed during the current study.

Abbreviations

Bilevel positive airway pressure

Continuous positive airway pressure

Health-related quality of life

Health-related family quality of life

Home mechanical ventilation

Preferred Reporting Items for Systematic Meta-Analyses extension for Scoping Reviews

Bogetz JF, Munjapara V, Henderson CM, et al. Home mechanical ventilation for children with severe neurological impairment: parents’ perspectives on clinician counselling. Dev Med Child Neurol. 2022;64(7):840–6.

PubMed   Google Scholar  

Jose D, Parameswaran N. Advances in management of respiratory failure in children. Indian J Pediatr. 2023;90(5):470–80. https://doi.org/10.1007/s12098-023-04559-z .

Chen IL, Chen HL. New developments in neonatal respiratory management. Pediatr Neonatol. 2022;63(4):341–7. https://doi.org/10.1016/j.pedneo.2022.02.002 .

Amin R, Verma R, Bai YQ, et al. Incidence and mortality of children receiving home mechanical ventilation. Pediatrics. 2023;151(4): e2022059898. https://doi.org/10.1542/peds.2022-059898 .

Van Den Oudenrijn LV: G M: Veldhoen, E. Forty years of home mechanical ventilation in children. Eur Resp J. 2019;54. https://doi.org/10.1183/13993003.congress-2019.PA699

World Health Organization Programme on Mental Health. WHOQOL User Manual . Division of mental health and prevention of substance abuse; 1998. Accessed July 15, 2024. https://www.who.int/toolkits/whoqol

Gettel CJ, Suter LG, Bagshaw K, et al. Patient-reported outcome-based performance measures in alternative payment models: current use, implementation barriers, and principles to succeed. Value in Health. 2024;27(2):199–205. https://doi.org/10.1016/j.jval.2023.10.017 .

Sterni LM, Collaco JM, Baker CD, et al. An official American Thoracic Society clinical practice guideline: pediatric chronic home invasive ventilation. Am J Respir Crit Care Med. 2016;193(8):e16–35. https://doi.org/10.1164/rccm.201602-0276ST .

PubMed   PubMed Central   Google Scholar  

Coleman CL, Morrison M, Perkins SK, Brosco JP, Schor EL. Quality of life and well-being for children and youth with special health care needs and their families: a vision for the future. Pediatrics. 2022;149(Supplement 7):e2021056150G. https://doi.org/10.1542/peds.2021-056150G .

Kwak S. Home mechanical ventilation in children with chronic respiratory failure: a narrative review. J Yeungnam Med Sci. 2023;40(2):123–35. https://doi.org/10.12701/jyms.2022.00227 .

Frates RC, Splaingard ML, Smith EO, Harrison GM. Outcome of home mechanical ventilation in children. J Pediatr. 1985;106(5):850–6. https://doi.org/10.1016/S0022-3476(85)80372-3 .

Upadhyay K, Vallarino DA, Talati AJ. Outcomes of neonates with tracheostomy secondary to bronchopulmonary dysplasia. BMC Pediatr. 2020;20(1):414. https://doi.org/10.1186/s12887-020-02324-1 .

Akangire G, Taylor JB, McAnany S, et al. Respiratory, growth, and survival outcomes of infants with tracheostomy and ventilator dependence. Pediatr Res. 2021;90(2):381–9. https://doi.org/10.1038/s41390-020-01183-x .

Evers-Bikker EE, de Weerd W, Wijkstra PJ, Corel L, Verweij LP, Vosse B a. H. Characteristics and outcomes in children with congenital central hypoventilation syndrome on long-term mechanical ventilation in the Netherlands. Eur J Pediatr. Published online November 25, 2023. https://doi.org/10.1007/s00431-023-05339-9

Edwards JD, Kun SS, Keens TG. Outcomes and causes of death in children on home mechanical ventilation via tracheostomy: an institutional and literature review. J Pediatr. 2010;157(6):955-959.e2. https://doi.org/10.1016/j.jpeds.2010.06.012 .

Amin RS, Fitton CM. Tracheostomy and home ventilation in children. Semin Neonatol. 2003;8(2):127–35. https://doi.org/10.1016/S1084-2756(02)00220-8 .

Zhavoronka M, Custer B, Neal A, Schweitzer J, Bombardieri M. How to ease the nursing shortage in America .; 2022. Accessed August 6, 2023. https://www.americanprogress.org/article/how-to-ease-the-nursing-shortage-in-america/

National Conference of State Legislatures. Addressing nursing shortages: options for states .; 2022. Accessed August 6, 2023. https://www.ncsl.org/health/addressing-nursing-shortages-options-for-states

Ozcan G, Zirek F, Tekin MN, et al. Psychosocial factors affecting the quality of life of parents who have children with home mechanical ventilation. Pediatr Pulmonol. Published online December 13, 2023. https://doi.org/10.1002/ppul.26799

Vo HH, Mercer AH, Jabre NA, Henderson CM, Boss RD, Wilfond BS. Parent perspectives on the child experience of pediatric home ventilation via tracheostomy. Hosp Pediatr. 2023;13(12):1124–33. https://doi.org/10.1542/hpeds.2023-007217 .

Israelsson-Skogsberg Å, Persson C, Markström A, Hedén L. Children with home mechanical ventilation-parents’ health-related quality of life, family functioning and sleep. Acta Paediatr. 2020;109(9):1807–14. https://doi.org/10.1111/apa.15177 .

Chan YH, Lim CZR, Bautista D, Malhotra R, Østbye T. The health and well-being of caregivers of technologically dependent children. Glob Pediatr Health. 2019;6:2333794X18823000. https://doi.org/10.1177/2333794X18823000 .

Wang KWK, Barnard A. Caregivers’ experiences at home with a ventilator-dependent child. Qual Health Res. 2008;18(4):501–8. https://doi.org/10.1177/1049732307306185 .

Sobotka SA, Lynch E, Quinn MT, Awadalla SS, Agrawal RK, Peek ME. Unmet respite needs of children with medical technology dependence. Clin Pediatr (Phila). 2019;58(11–12):1175–86. https://doi.org/10.1177/0009922819870251 .

WHOQOL - Measuring Quality of Life| The World Health Organization. Accessed November 13, 2023. https://www.who.int/tools/whoqol

Revicki D. Health-related quality of life in the evaluation of medical therapy for chronic illness. J Fam Pract. 1989;29(4). Accessed November 13, 2023. https://www.mdedge.com/familymedicine/article/179863/health-related-quality-life-evaluation-medical-therapy-chronic-illness

Quittner AL, Romero SL, Kimberg CI, Blackwell LS, Cruz I. Chronic Illness. In: Brown BB, Prinstein MJ, eds. Encyclopedia of adolescence. Academic Press; 2011:91–99. https://doi.org/10.1016/B978-0-12-373951-3.00105-8

Radina ME. Toward a theory of health-related family quality of life. J Fam Theory Rev. 2013;5(1):35–50. https://doi.org/10.1111/jftr.12001 .

Google Scholar  

Park J, Hoffman L, Marquis J, et al. Toward assessing family outcomes of service delivery: validation of a family quality of life survey. J Intellect Disabil Res. 2003;47(4–5):367–84. https://doi.org/10.1046/j.1365-2788.2003.00497.x .

CAS   PubMed   Google Scholar  

Mattson J, Lunnelie J, Löfholm T, Andersson ES, Aune RE, Björling G. Quality of life in children with home mechanical ventilation - a scoping review. SAGE Open Nursing. 2022;8:1–12.

Seear M, Kapur A, Wensley D, Morrison K, Behroozi A. The quality of life of home-ventilated children and their primary caregivers plus the associated social and economic burdens: a prospective study. Arch Dis Child. 2016;101(7):620–7. https://doi.org/10.1136/archdischild-2015-309796 .

Falkson S, Knecht C, Hellmers C, Metzing S. The perspective of families with a ventilator-dependent child at home. A literature review J Pediatric Nursing. 2017;36:213–24. https://doi.org/10.1016/j.pedn.2017.06.021 .

JBI Manual for evidence synthesis. JBI. 2020. https://doi.org/10.46658/JBIMES-20-01 .

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73. https://doi.org/10.7326/M18-0850 .

Godin K, Stapleton J, Kirkpatrick SI, Hanning RM, Leatherdale ST. Applying systematic review search methods to the grey literature: a case study examining guidelines for school-based breakfast programs in Canada. Syst Rev. 2015;4:138. https://doi.org/10.1186/s13643-015-0125-0 .

Download references

Acknowledgements

Not applicable.

Author information

Authors and affiliations.

Washington University in St. Louis School of Medicine, 660 South Euclid Avenue, MSC 8116-0043-08, Saint Louis, MO 63110-1010, USA

Keisha White Makinde, Maysara Mitchell, Alexandra F. Merz & Michael Youssef

You can also search for this author in PubMed   Google Scholar

Contributions

Keisha White Makinde participated in the conception and design of this protocol, drafting and revision of manuscript, and approval of final version. Maysara Mitchell participated in the conception and design of this protocol, drafting and revision of manuscript, and approval of final version. Alexandra Merz participated in the design of this protocol, revision of manuscript, and approval of final version. Michael Youssef participated in the design of this protocol, revision of manuscript, and approval of final version.

Authors’ information

Dr. KWM is a pediatric hospice and palliative medicine physician-scientist interested in understanding racial and ethnic health disparities and promoting health equity in patients facing serious illness. Dr. KWM has extensive educational background focused on the biopsychosocial experiences of individuals with illness, including a bachelor’s degree in Sociology and Medicine, Health, and Society and Master of Public Health. As an attending physician, Dr. KWM cares for children with serious illness including children who utilize home mechanical ventilation seen at St. Louis Children’s Hospital with a focus on interdisciplinary, family-centered, holistic care.

Corresponding author

Correspondence to Keisha White Makinde .

Ethics declarations

Ethics approval and consent to participate.

Not applicable, no human subjects.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix 1 Search strategy

Search conducted on March 8, 2024.

Appendix 2 Data extraction instrument

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

White Makinde, K., Mitchell, M., Merz, A.F. et al. Mapping health-related quality of life of children and families receiving pediatric invasive home mechanical ventilation: a scoping review protocol. Syst Rev 13 , 236 (2024). https://doi.org/10.1186/s13643-024-02658-2

Download citation

Received : 21 December 2023

Accepted : 06 September 2024

Published : 17 September 2024

DOI : https://doi.org/10.1186/s13643-024-02658-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Invasive mechanical ventilation
• Quality of life

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

A Guide to Writing a Qualitative Systematic Review Protocol to Enhance Evidence-Based Practice in Nursing and Health Care

Affiliations.

• 1 PhD candidate, School of Nursing and Midwifey, Monash University, and Clinical Nurse Specialist, Adult and Pediatric Intensive Care Unit, Monash Health, Melbourne, Victoria, Australia.
• 2 Lecturer, School of Nursing and Midwifery, Monash University, Melbourne, Victoria, Australia.
• 3 Senior Lecturer, School of Nursing and Midwifery, Monash University, Melbourne, Victoria, Australia.
• PMID: 26790142
• DOI: 10.1111/wvn.12134

Background: The qualitative systematic review is a rapidly developing area of nursing research. In order to present trustworthy, high-quality recommendations, such reviews should be based on a review protocol to minimize bias and enhance transparency and reproducibility. Although there are a number of resources available to guide researchers in developing a quantitative review protocol, very few resources exist for qualitative reviews.

Aims: To guide researchers through the process of developing a qualitative systematic review protocol, using an example review question.

Methodology: The key elements required in a systematic review protocol are discussed, with a focus on application to qualitative reviews: Development of a research question; formulation of key search terms and strategies; designing a multistage review process; critical appraisal of qualitative literature; development of data extraction techniques; and data synthesis. The paper highlights important considerations during the protocol development process, and uses a previously developed review question as a working example.

Implications for research: This paper will assist novice researchers in developing a qualitative systematic review protocol. By providing a worked example of a protocol, the paper encourages the development of review protocols, enhancing the trustworthiness and value of the completed qualitative systematic review findings.

Linking evidence to action: Qualitative systematic reviews should be based on well planned, peer reviewed protocols to enhance the trustworthiness of results and thus their usefulness in clinical practice. Protocols should outline, in detail, the processes which will be used to undertake the review, including key search terms, inclusion and exclusion criteria, and the methods used for critical appraisal, data extraction and data analysis to facilitate transparency of the review process. Additionally, journals should encourage and support the publication of review protocols, and should require reference to a protocol prior to publication of the review results.

Keywords: guidelines; meta synthesis; qualitative; systematic review protocol.

© 2016 Sigma Theta Tau International.

PubMed Disclaimer

Similar articles

• How has the impact of 'care pathway technologies' on service integration in stroke care been measured and what is the strength of the evidence to support their effectiveness in this respect? Allen D, Rixson L. Allen D, et al. Int J Evid Based Healthc. 2008 Mar;6(1):78-110. doi: 10.1111/j.1744-1609.2007.00098.x. Int J Evid Based Healthc. 2008. PMID: 21631815
• Procedures and methods of benefit assessments for medicines in Germany. Bekkering GE, Kleijnen J. Bekkering GE, et al. Eur J Health Econ. 2008 Nov;9 Suppl 1:5-29. doi: 10.1007/s10198-008-0122-5. Eur J Health Econ. 2008. PMID: 18987905
• [Procedures and methods of benefit assessments for medicines in Germany]. Bekkering GE, Kleijnen J. Bekkering GE, et al. Dtsch Med Wochenschr. 2008 Dec;133 Suppl 7:S225-46. doi: 10.1055/s-0028-1100954. Epub 2008 Nov 25. Dtsch Med Wochenschr. 2008. PMID: 19034813 German.
• Evidence-based medicine, systematic reviews, and guidelines in interventional pain management, part I: introduction and general considerations. Manchikanti L. Manchikanti L. Pain Physician. 2008 Mar-Apr;11(2):161-86. Pain Physician. 2008. PMID: 18354710 Review.
• An example of the use of systematic reviews to answer an effectiveness question. Forbes DA. Forbes DA. West J Nurs Res. 2003 Mar;25(2):179-92. doi: 10.1177/0193945902250036. West J Nurs Res. 2003. PMID: 12666642 Review.
• Patients' experiences with musculoskeletal spinal pain: A qualitative systematic review protocol. El Chamaa A, Kowalski K, Parikh P, Rushton A. El Chamaa A, et al. PLoS One. 2024 Aug 8;19(8):e0306993. doi: 10.1371/journal.pone.0306993. eCollection 2024. PLoS One. 2024. PMID: 39116059 Free PMC article.
• Physical Activity Interventions in People with Diabetes: A Systematic Review of The Qualitative Evidence. Vilafranca-Cartagena M, Bonet-Augè A, Colillas-Malet E, Puiggrós-Binefa A, Tort-Nasarre G. Vilafranca-Cartagena M, et al. Healthcare (Basel). 2024 Jul 9;12(14):1373. doi: 10.3390/healthcare12141373. Healthcare (Basel). 2024. PMID: 39057516 Free PMC article. Review.
• Telemedicine in Advanced Kidney Disease and Kidney Transplant: A Qualitative Meta-Analysis of Studies of Patient Perspectives. Manko CD, Apple BJ, Chang AR, Romagnoli KM, Johannes BL. Manko CD, et al. Kidney Med. 2024 May 24;6(7):100849. doi: 10.1016/j.xkme.2024.100849. eCollection 2024 Jul. Kidney Med. 2024. PMID: 39040545 Free PMC article.
• Voices of Wisdom: Geriatric Interviews on Self-Management of Type 2 Diabetes in the United States-A Systematic Review and Metasynthesis. Lo DF, Gawash A, Shah KP, Emanuel J, Goodwin B, Shamilov DD, Kumar G, Jean N, White CP. Lo DF, et al. J Diabetes Res. 2024 Jul 13;2024:2673742. doi: 10.1155/2024/2673742. eCollection 2024. J Diabetes Res. 2024. PMID: 39035684 Free PMC article. Review.
• Factors affecting implementation of mindfulness in hospital settings: A qualitative meta-synthesis of healthcare professionals' experiences. Knudsen RK, Skovbjerg S, Pedersen EL, Nielsen CL, Storkholm MH, Timmermann C. Knudsen RK, et al. Int J Nurs Stud Adv. 2024 Mar 27;6:100192. doi: 10.1016/j.ijnsa.2024.100192. eCollection 2024 Jun. Int J Nurs Stud Adv. 2024. PMID: 38746813 Free PMC article. Review.
• Search in MeSH

LinkOut - more resources

Full text sources.

• Ovid Technologies, Inc.

Other Literature Sources

• scite Smart Citations

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

The PMC website is updating on October 15, 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• BMC Med Res Methodol

Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks

Marina krnic martinic.

1 Department of Otorhinolaryngology, University Hospital Split, Split, Croatia

Dawid Pieper

2 Institute for Research in Operative Medicine (IFOM), Witten/Herdecke University, Cologne, Germany

Angelina Glatt

Livia puljak.

3 Center for Evidence-Based Medicine and Health Care, Catholic University of Croatia, Ilica 242, 10000 Zagreb, Croatia

Associated Data

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

A standard or consensus definition of a systematic review does not exist. Therefore, if there is no definition about a systematic review in secondary studies that analyse them or the definition is too broad, inappropriate studies might be included in such evidence synthesis. The aim of this study was to analyse the definition of a systematic review (SR) in health care literature, elements of the definitions that are used and to propose a starting point for an explicit and non-ambiguous SR definition.

We included overviews of systematic reviews (OSRs), meta-epidemiological studies and epidemiology textbooks. We extracted the definitions of SRs, as well as the inclusion and exclusion criteria that could indicate which definition of a SR the authors used. We extracted individual elements of SR definitions, categorised and quantified them.

Among the 535 analysed sources of information, 188 (35%) provided a definition of a SR. The most commonly used reference points for the definitions of SRs were Cochrane and the PRISMA statement. We found 188 different elements of SR definitions and divided them into 14 categories. The highest number of SR definition elements was found in categories related to searching ( N  = 51), analysis/synthesis ( N  = 23), overall methods ( N  = 22), quality/bias/appraisal/validity ( N  = 22) and aim/question ( N  = 13). The same five categories were also the most commonly used combination of categories in the SR definitions.

Currently used definitions of SRs are vague and ambiguous, often using terms such as clear, explicit and systematic, without further elaboration. In this manuscript we propose a more specific definition of a systematic review, with the ultimate aim of motivating the research community to establish a clear and unambiguous definition of this type of research.

In 1990, the term evidence-based medicine (EBM) was coined [ 1 ]. It was hailed as a new approach for teaching and practising clinical medicine [ 2 ], incorporating “the best available external clinical evidence from a systematic search” [ 3 ]. When it comes to the best available evidence about treatment, randomised controlled trials (RCTs) and a systematic review (SR)/meta-analysis are considered the “gold standard” [ 1 ].

The EBM movement has been widely adopted, and evidence syntheses are regularly used to support clinical guidelines and recommendations for practice. However, it has been suggested that the EBM might be a movement in crisis [ 4 ], as there is “too much evidence” [ 4 ]. A study published in 2016 indicated that more than 8000 systematic reviews were being indexed annually in MEDLINE, corresponding to a three-fold increase over the last decade [ 5 ]. A search conducted in October 2019 showed that more than 15,000 studies published in 2018 were marked with a systematic review tag in PubMed.

Furthermore, some SRs might actually be misleading, redundant and conflicted [ 6 ]. A recent overview of systematic reviews found 12 systematic reviews and two major guidelines about thrombolytic therapy for pulmonary embolism published within less than 2 years. The results of those evidence syntheses were discordant, and the benefit-to-risk ratio was elusive [ 7 ]. Inclusion and exclusion criteria played a part in the origin of the discordant results [ 7 ].

Just as different inclusion and exclusion criteria might be a problem when conducting a systematic review, the same can happen in overviews of systematic reviews (OSRs) or other types of studies analysing systematic reviews, where results will depend upon inclusion criteria. The problem here is that a standard or consensus definition of a systematic review does not exist.

For example, in a study that reported about the increasing popularity of SRs, Page et al. [ 5 ] used the PRISMA-P explanation of a SR [ 8 ]. Using a definition when searching for SRs is important because there are studies that may call themselves SRs but are not SRs; we can only speculate that authors use a descriptor SR to label their studies because they are not aware of what a systematic review is, or because systematic reviews are considered to be a higher standard of review.

Therefore, if there is no definition about a systematic review in secondary studies that analyse them or the definition is too broad, inappropriate studies might be included in such evidence synthesis. The aim of this study was to explore and analyse the definition of a systematic review (SR) in health care literature, the elements of definitions that are used and propose a starting point for a new, explicit SR definition.

This was a methodological study, for which we developed a protocol a priori. The study protocol is available from the corresponding author on request.

Included studies

We aimed to collect definitions of systematic reviews in the health care literature. As numerous collections of SRs have already been published in the past, we relied on existing resources. We used three different sources: i) OSRs about healthcare interventions, ii) studies that have analysed the methodological quality of systematic reviews and iii) relevant textbooks/internet sources that define systematic reviews.

We included OSRs and methodological studies identified previously by Pieper et al. [ 9 , 10 ]. We used a validated filter for retrieval [ 11 ]. Additionally, we searched for EBM-related and epidemiology handbooks published in English or German. There was no systematic search for the handbooks. We compiled a list of relevant handbooks known to us, using the same methodological approach as described by other authors in similar projects [ 12 ].

Furthermore, we searched Google Scholar between the 24th January, 2018 and the 7th February, 2018 using the following search phrases: “definition of a systematic review”, “definition of systematic review”, “definition of the systematic review”, “defined a systematic review”, “defined the systematic review”, “systematic review was defined”. Those phrases were used to search any part of the manuscript – without any restrictions. We analysed the first 50 search results for each search phrase if there were more than 50 search phrases retrieved for a phrase. We excluded duplicate manuscripts found by searching multiple sources before starting the analysis.

Data extraction analysis

We piloted a data extraction form in Microsoft Excel on a sample of ten manuscripts. Two authors piloted the data extraction form (LP, MKM). Furthermore, based on the advice of a third author (DP), the form was further refined. Thus, in an iterative process between the authors, the form was modified where necessary to avoid any misunderstandings or later disagreements.

We extracted the following information: i) whether the analysed literature sources reported a definition of a systematic review and ii) inclusion and exclusion criteria defining systematic reviews. We extracted the relevant exclusion criteria when they had explicit statements about studies that were not included because certain aspects of them were not considered to be characteristics of a systematic review.

When we found a definition or inclusion/exclusion criteria defining systematic reviews, the text was extracted verbatim. Subsequently, from those definitions and the inclusion/exclusion criteria, we extracted elements of a systematic review definition. The definition elements were defined as distinct methodological components and their attributes. Elements described with similar adjectives were not combined; instead, we presented all unique elements separately in order to present a wide variety of adjectives and attributes used in the definitions of SRs. We did not use an a priori defined list of those elements; instead, we presented elements that we found in the analysed sources of information and we kept expanding the list of elements as we found new variations of the elements of the SR definition.

One of the elements of a definition we used was the presence of a meta-analysis (MA), but only if the authors explicitly indicated that the MA was considered as defining characteristic of a SR. For example, in a study published in 2013, Aziz wrote explicitly that SRs without MA were not included because “these were not considered SRs” [ 13 ].

Extracted individual elements of the SR definition were then categorised into groups. For example, if a SR definition was: “systematic search”, “reproducible search” or “keywords searched”, those elements were sorted into a category called “Search”. The process of forming categorisations was iterative between the authors until we reached a consensus about the categories that will be used.

We extracted reference(s) for a definition of a systematic review or inclusion criteria referring to the systematic review, if available. We recorded the 2017 Journal Citation Reports (JCR) Journal Impact Factor (JIF) of a journal from the Web of Science. We hypothesised that manuscripts published in journals with a higher JIF would have a higher prevalence of SR definitions, due to the higher reporting standards.

For all data, one author (MKM) extracted data and the second author (LP) verified the extractions. Furthermore, one author (LP) categorised the definition elements and the second author verified the categorisations (MKM). Any discrepancies in opinion were resolved via discussion.

For the analysis of definitions from textbooks and Internet sources, we extracted the definitions verbatim and indicated the field from which the definition came from, such as medicine, psychology and social sciences. During the analyses of the textbooks, if the definition in the text was supplemented with a table, we treated this as one source of information and the extracted elements of the SR definition from both text and table. One author extracted data and the second author verified the extractions from the textbooks and Internet sources.

Descriptive statistics, including frequencies and percentages, were used to describe the categories of elements of a systematic review definition/inclusion criteria. We also analysed the frequency of each category by counting the categories of elements that were used in each source. If at least one element was used in a certain category, we considered that this category of elements was present in the information source. We expressed the JIF as the mean and standard deviation (M ± SD), we used a t-test to analyse the difference in the JIR between the information sources with and without a SR definition. For the analyses, we used the MedCalc statistical software, v 15.2.1 (©MedCalc Software bvba, Ostend, Belgium). The statistical significance was set at P  < 0.05.

Search results

After searching for OSRs and methodological studies, from the 347 identified full texts, we included 308 studies. We excluded 39 studies because 31 were duplicates and an additional eight manuscripts were excluded because they were written in Chinese or did not fit our inclusion criteria (commentaries, traditional narrative reviews, reviews of an unspecified type of reviews or analysed rapid reviews).

By searching Google Scholar we found 531 hits. Based on the limits we set, analysing 50 hits per search phrase, we analysed a total of 238 bibliographic records from Google Scholar. After removing the duplicates that we already had in the first cohort of the included studies, we included the remaining 200 manuscripts from this cohort of studies. Additionally, we analysed 27 textbooks. In total, we analysed 535 sources of information: 508 manuscripts from peer-reviewed journals and 27 from textbooks.

Prevalence of definitions of SRs

Among the 535 analysed sources of information, 188 (35%) defined what they consider to be a systematic review, 62 (18%) had an inclusion criteria in the methods that allowed us to extract information about what the authors considered to be a systematic review and 59 (18%) had exclusion criteria that we used as well for determining the authors’ definition of a SR. Some sources of information had both a definition of a SR and/or inclusion/exclusion criteria; in total there were 226 sources of information from which we could extract information related to the authors’ definition of a SR.

Among the 508 manuscripts, we found a JIF for 401 manuscripts, of which 113 had a SR definition, and 288 did not. Journals that did not provide SR a definition had a higher JIF (4.4 ± 5.1) than those with a definition (3.7 ± 4.5), but this difference was not significant ( P  = 0.099).

Organisations, databases and checklists used as a reference for SR definition

Many of the analysed sources explicitly mentioned relevant organisations, checklists and databases for defining what they considered to be a SR, some of the analysed sources of information only provided literature references to support their definitions or inclusion/exclusion criteria.

Explicit mentions of the names of the organisations, checklists, databases associated with a definition of SRs or criteria for the inclusion of SRs were found in 43 out of 535 (8%) analysed sources of information. Those were Cochrane ( N  = 24), the PRISMA statement ( N  = 13), criteria of Database of Reviews of Effect (DARE) ( N  = 5), National Institute for Health and Care Excellence (NICE) ( N  = 3), NHS Centre for Reviews and Dissemination (N = 3), Campbell collaboration (N = 2) National Health and Medical Research Council ( N  = 1), QUOROM (QUality Of Reporting of Meta-analyses) recommendations (N = 1), Guidelines from Agency for Healthcare Research and Quality (AHRQ) (N = 1), Institute of Medicine (IOM) (N = 1) and author Andy Oxman (N = 1), referred to as the “Oxman criteria“. Cochrane was mentioned most commonly, either as a reference to a whole organisation, the Cochrane Handbook for Systematic Reviews of Interventions or a specific Cochrane entity: the Dutch Cochrane Centre in one source of information. Details about the definitions and references provided in those 43 studies are shown in Additional file 1 : Table S1. The most commonly used supporting references in those studies were the manuscripts by Moher et al. and Liberati et al. describing the PRISMA statement, the PRISMA-P checklist, and the Cochrane Handbook (Additional file 1 : Table S1).

The most commonly used literature references that were used to support the statements provided in the definitions of SRs or the inclusion/exclusion criteria were also manuscripts describing the PRISMA statement and Cochrane Handbook (Additional file 1 : Table S2).

Elements of systematic review definitions

After analysing all the definitions of SRs and the inclusion/exclusion criteria for SRs, we extracted 188 individual elements of a SR definition; we categorised them into the following 14 categories: self-identification, indexing, aim/question, overall methods, search, identification of studies, selection of studies, study eligibility, data extraction, quality/bias/appraisal/validity, analysis/synthesis, describing included studies, reporting and unclear (Table ​ (Table1 1 ).

Categories and elements of systematic review definition found in health care literature; percentage calculated from 226 sources of information that had a SR definition, or inclusion/exclusion criteria that could be used for extracting individual elements of SR definition

Elements were sorted according to those categories (Table ​ (Table1). 1 ). The highest number of SR definition elements was found in categories related to searching ( N  = 51), analysis/synthesis ( N  = 23), overall methods ( N  = 22), quality/bias/appraisal/validity (N = 22) and aim/question ( N  = 13) (Table ​ (Table1 1 ).

Categories of systematic review elements

Among the 226 sources of information that had a SR definition or inclusion/exclusion criteria that could be used for extracting individual elements of a SR definition, 59 used only one category, 62 used two categories, while 105 used from three to ten categories of the SR definition elements. When we looked at the combinations that were used, none of the combinations of various categories was used more than ten times. The most commonly used combination of SR definition categories was used in nine of the manuscripts/books, and it used the following five categories: i) aim/research question, ii) search, iii) study eligibility, iv) quality, bias, appraisal, validity and v) analysis/synthesis. However, those nine manuscripts had different wording of the SR definition, as shown in Additional file 2 : Table S3; they did not use one consistent definition.

The same five categories were the most commonly used SR definition categories in our sample of information sources, with the following frequencies: i) search ( N  = 122), ii) aim/research question ( N  = 93), iii) analysis/synthesis ( N  = 90), iv) study eligibility ( N  = 89) and v) quality, bias, appraisal, validity ( N  = 81).

We found that authors of manuscripts and textbooks use various definitions of systematic reviews; in 535 sources of information, we found 188 different elements of a SR definition. The most commonly used categories of SR definition elements were related to searching, analysis/synthesis, overall methods, quality/bias/appraisal/validity and aim/question. The most commonly used reference resources were the Cochrane and PRISMA statement [ 14 , 15 ].

However, as our study showed, there is no uniformly used definition of a SR. We analysed various sources of information, including overviews of SRs and methodological studies about SRs because those studies included SRs and we expected that therefore they should provide a definition of a SR. Our expectations were not met; as we found that one-third of those information sources used an explicit definition of a SR. In another one-third of the information sources, we found either inclusion or exclusion criteria, from which we could deduce what they consider to be, or not to be, a SR.

Also, we found that journals that did not provide SR definition had a higher JIF than those with a definition, but this difference was not significant. This finding was not in line with our hypothesis, and it shows that in this respect journals with higher JIF did not have higher expectations from authors in terms of transparent reporting about what was considered to be a SR.

When extracting the elements of SR definitions, we tried to be as detailed as possible, to capture various terminology used in those definitions. We found many variations of similar concepts, but also many vague terms. Such vague terms were frequently reflected in the usage of the word systematic, such as: “systematic methods”, “systematic approach”, “systematic search”, “systematic synthesis”, “systematic analysis” and “systematic presentation”, without actually explaining what systematic means. We also found two expressions that were completely unclear about what the authors consider to be a SR, including “ Reviews were included if they were systematic ” and “ It was apparent in the text that a systematic review had been undertaken ”.

There were ten elements of a SR definition that used the type and number of sources that were searched in a SR, as an element of the SR definition. It has been suggested previously that a minimum number and types of sources should define SRs because searching only one database may not be universally considered a systematic search [ 16 ].

It could be argued that our categorisation was too detailed, as some of our categories of SR definition elements sound similar, for example, categories search, selection of studies, identification of studies and study eligibility. We left those categories as they were on purpose, because it may not be perfectly obvious what the difference between them is; for example, the term selection of studies in the Cochrane reviews is reserved for the description of the screening of abstracts and full texts, but it is unclear whether all authors use this term in the same context. Furthermore, it is unclear whether the identification of studies refers to searching, screening or eligibility, i.e. the inclusion/exclusion criteria. Because of this ambiguity, we chose to present more detailed categories.

The most commonly used individual five categories of the SR elements were also used as the most common combination of elements in the analysed sources of information, but only nine manuscripts used this combination of the five elements. Those five categories of elements are also included in the definition of SRs from the Cochrane Handbook [ 14 ].

In section 1.2.2 of the Cochrane Handbook, titled What is a systematic review?, the following definition can be found [quote]: “ A systematic review attempts to collate all empirical evidence that fits the pre-specified eligibility criteria in order to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimising bias, thus providing more reliable findings from which conclusions can be drawn and decisions made (Antman 1992, Oxman 1993). The key characteristics of a systematic review are: a clearly stated set of objectives with pre-defined eligibility criteria for the studies; an explicit, reproducible methodology; a systematic search that attempts to identify all the studies that would meet the eligibility criteria; an assessment of the validity of the findings of the included studies, for example through the assessment of the risk of bias; and a systematic presentation, and synthesis, of the characteristics and findings of the included studies” [ 14 ].

Also, Cochrane was the most commonly mentioned organisation in the definitions of SRs; 13% of the manuscripts/textbooks mentioned Cochrane as a source of the SR definition. Therefore, one could argue that the Cochrane’s definition could be used as a formal definition of what a SR is. However, the Cochrane’s definition is also vague, as it is unclear what it means “ explicit, systematic methods” or “explicit, reproducible methodology”. Someone can explicitly describe the methodology that is not adequate. This inadequate methodology may also be reproducible, but that does not mean that it is good. Furthermore, the Cochrane definition of a SR repeatedly uses the adjective “systematic”, without explaining what the meaning of systematic is.

Two references used in Cochrane’s definition of a SR are those of Antman et al. [ 17 ] and Oxman et al. [ 18 ]. We also analysed which references were used to support the definitions of SRs in the manuscripts and textbooks; we found that the authors most commonly referred to the PRISMA statement [ 15 ] and Cochrane Handbook. However, the definition of SRs from the PRISMA statement manuscripts also uses vague terms such as clearly, systematic and explicit, without going into details of what they entail [ 15 ].

The research community would benefit from having a very specific definition of a SR. The five most commonly used SR definition elements that we identified could be used to create a more elaborate and unambiguous definition of a SR. We believe that the international research community should create an unambiguous SR definition; we hope that this study will be a starting point in that direction. As a first step, we suggest starting with the following template:

A systematic review is a review that reports or includes the following:

• i) research question
• ii) sources that were searched, with a reproducible search strategy (naming of databases, naming of search platforms/engines, search date and complete search strategy)
• iii) inclusion and exclusion criteria
• iv) selection (screening) methods
• v) critically appraises and reports the quality/risk of bias of the included studies
• vi) information about data analysis and synthesis that allows the reproducibility of the results

Some of those elements are mentioned in the SR definition from the Cochrane Handbook [ 14 ], as shown above, but the Cochrane’s definition still leaves a lot of ambiguity in several aspects. Those elements should be more specific in future. For example, which details should the clinical question report, how many databases/sources should be searched to be considered systematic, whether key methodological aspects (screening of titles and abstracts, screening of full texts, data extraction and risk of bias assessment) should be done by two authors independently or done by one author and verified by another. The naming of the databases is important for ensuring transparency and reproducibility, which should be features of a systematic approach. Those and other considerations should be taken into account in further efforts to clarify what exactly makes a SR.

Information presented in this manuscript could help inform a consensus meeting or a similar gathering where interested SR researchers could contribute to standardising a SR definition. A similar approach was recently suggested for the definition of a predatory journal. Cobey et al. have conducted a scoping review in which they summarised the literature on predatory journals, described its epidemiological characteristics and extracted empirical descriptions of the potential characteristics of predatory journals. In their conclusions, they informed readers that the results will be shared with attendees that will attend a stakeholder meeting seeking to develop a standardised definition for what constitutes a predatory journal [ 19 ].

One limitation of our study could be the use of information sources published within a certain period of time. However, this type of work, which relies on analysis of published literature, usually suffers from a time lag. Each new update of the search results in new literature sources to analyse, and time lag appears again by the time analysis is completed.

Furthermore, in our approach, we analysed both expressions that appeared to be definitions of SRs and the characteristics of SRs eligible for inclusion. It may be considered that the inclusion criteria for SRs are not eligible elements to define what a SR is. However, we considered that the eligibility and inclusion criteria which describe SRs would be useful in our analysis, as we seldom found explicit statements about the definition of a SR. We consider that the range of descriptors we found indicates a very rich vocabulary used by authors who are defining or searching for SRs and that our approach is an adequate starting point towards building a future consensus definition of a systematic review. Likewise, it could be argued that we are mixing a definition of a SR with measures of the quality of a SR. However, in the absence of an existing definition, we believe that we should assess all the descriptors used for SRs and report them explicitly and transparently, then readers can see for themselves that some of those may overlap with quality descriptors. Also, for searching Google Scholar we used a limited number of phrases. Google used to include details for searching the advanced interface, which is no longer available but this search information could be available from other sites (mostly libraries) which we did not utilize.

Our analysis is also limited by the fact that we only focused on the definitions, while we acknowledge that some relevant information might also be found in the explanatory text to the definition, if available.

The majority of manuscripts that include SRs actually do not provide a definition of what they consider to be a SR. The most commonly used reference sources of a SR definition use vague and ambiguous terms. We propose a new definition of a systematic review, which is open for further commenting and elaboration, with the aim of motivating the research community to create a more specific definition of this type of research.

Supplementary information

Acknowledgments.

We are grateful to Dr. Svjetlana Dosenovic for critical reading of the manuscript. The manuscript was revised by a native English speaker from the company Proof-Reading-Service.com Ltd., Hertfordshire, UK.

Abbreviations

Authors’ contributions

Study design: LP, DP. Data extraction: MKM, AG. Data analysis and interpretation: MKM, AG, DP, LP. Writing the first draft of the manuscript: MKM, LP. Revisions of the manuscript for important intellectual content: MKM, AG, DP, LP. Final approval of the manuscript: MKM, AG, DP, LP. Agree to be accountable for all aspects of the work: MKM, AG, DP, LP. Guarantor: LP.

No external funding.

Availability of data and materials

Ethics approval and consent to participate.

This study involved only analysis of data from published scientific literature; we did not collect any participant data.

Consent for publication

Not applicable.

Competing interests

Livia Puljak is the section editor of the BMC Medical Research Methodology, but she was not involved in handling of this manuscript in any way. The remaining authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Contributor Information

Marina Krnic Martinic, Email: moc.liamg@iksnidodaniram .

Dawid Pieper, Email: [email protected] .

Angelina Glatt, Email: [email protected] .

Livia Puljak, Phone: +385-1-370-66-00, Email: [email protected] , Email: [email protected] .

Supplementary information accompanies this paper at 10.1186/s12874-019-0855-0.