systematic literature review level of evidence

Systematic Reviews

• Levels of Evidence
• Evidence Pyramid
• Joanna Briggs Institute

The evidence pyramid is often used to illustrate the development of evidence. At the base of the pyramid is animal research and laboratory studies – this is where ideas are first developed. As you progress up the pyramid the amount of information available decreases in volume, but increases in relevance to the clinical setting.

Meta Analysis  – systematic review that uses quantitative methods to synthesize and summarize the results.

Systematic Review  – summary of the medical literature that uses explicit methods to perform a comprehensive literature search and critical appraisal of individual studies and that uses appropriate st atistical techniques to combine these valid studies.

Randomized Controlled Trial – Participants are randomly allocated into an experimental group or a control group and followed over time for the variables/outcomes of interest.

Cohort Study – Involves identification of two groups (cohorts) of patients, one which received the exposure of interest, and one which did not, and following these cohorts forward for the outcome of interest.

Case Control Study – study which involves identifying patients who have the outcome of interest (cases) and patients without the same outcome (controls), and looking back to see if they had the exposure of interest.

Case Series   – report on a series of patients with an outcome of interest. No control group is involved.

• Levels of Evidence from The Centre for Evidence-Based Medicine
• The JBI Model of Evidence Based Healthcare
• How to Use the Evidence: Assessment and Application of Scientific Evidence From the National Health and Medical Research Council (NHMRC) of Australia. Book must be downloaded; not available to read online.

When searching for evidence to answer clinical questions, aim to identify the highest level of available evidence. Evidence hierarchies can help you strategically identify which resources to use for finding evidence, as well as which search results are most likely to be "best".                                             

Image source: Evidence-Based Practice: Study Design from Duke University Medical Center Library & Archives. This work is licensed under a Creativ e Commons Attribution-ShareAlike 4.0 International License .

The hierarchy of evidence (also known as the evidence-based pyramid) is depicted as a triangular representation of the levels of evidence with the strongest evidence at the top which progresses down through evidence with decreasing strength. At the top of the pyramid are research syntheses, such as Meta-Analyses and Systematic Reviews, the strongest forms of evidence. Below research syntheses are primary research studies progressing from experimental studies, such as Randomized Controlled Trials, to observational studies, such as Cohort Studies, Case-Control Studies, Cross-Sectional Studies, Case Series, and Case Reports. Non-Human Animal Studies and Laboratory Studies occupy the lowest level of evidence at the base of the pyramid.

• << Previous: What is a Systematic Review?
• Next: Locating Systematic Reviews >>
• Getting Started
• What is a Systematic Review?
• Locating Systematic Reviews
• Searching Systematically
• Developing Answerable Questions
• Identifying Synonyms & Related Terms
• Using Truncation and Wildcards
• Identifying Search Limits/Exclusion Criteria
• Keyword vs. Subject Searching
• Where to Search
• Search Filters
• Sensitivity vs. Precision
• Core Databases
• Other Databases
• Clinical Trial Registries
• Conference Presentations
• Databases Indexing Grey Literature
• Web Searching
• Handsearching
• Citation Indexes
• Documenting the Search Process
• Managing your Review

Research Support

• Last Updated: Aug 14, 2024 11:07 AM
• URL: https://guides.library.ucdavis.edu/systematic-reviews

Systematic Reviews: Levels of evidence and study design

Levels of evidence.

"Levels of Evidence" tables have been developed which outline and grade the best evidence. However, the review question will determine the choice of study design.

Secondary sources provide analysis, synthesis, interpretation and evaluation of primary works. Secondary sources are not evidence, but rather provide a commentary on and discussion of evidence. e.g. systematic review

Primary sources contain the original data and analysis from research studies. No outside evaluation or interpretation is provided. An example of a primary literature source is a peer-reviewed research article. Other primary sources include preprints, theses, reports and conference proceedings.

Levels of evidence for primary sources fall into the following broad categories of study designs   (listed from highest to lowest):

• Experimental : RTC's (Randomised Control Trials)
• Quasi-experimental studies (Non-randomised control studies, Before-and-after study, Interrupted time series)
• Observational studies (Cohort study, Case-control study, Case series) 

Based on information from Centre for Reviews and Dissemination. (2009). Systematic reviews: CRD's guidance for undertaking reviews in health care. Retrieved from http://www.york.ac.uk/inst/crd/index_guidance.htm

Hierarchy of Evidence Pyramid

"Levels of Evidence" are often represented in as a pyramid, with the highest level of evidence at the top:

Types of Study Design

The following definitions are adapted from the Glossary in " Systematic reviews: CRD's Guidance for Undertaking Reviews in Health Care " , Centre for Reviews and Dissemination, University of York :

• Systematic Review The application of strategies that limit bias in the assembly, critical appraisal, and synthesis of all relevant studies on a specific topic and research question. 
• Meta-analysis A systematic review which uses quantitative methods to summarise the results
• Randomized control clinical trial (RCT) A group of patients is randomised into an experimental group and a control group. These groups are followed up for the variables/outcomes of interest.
• Cohort study Involves the identification of two groups (cohorts) of patients, one which did receive the exposure of interest, and one which did not, and following these cohorts forward for the outcome of interest.
• Case-control study Involves identifying patients who have the outcome of interest (cases) and control patients without the same outcome, and looking to see if they had the exposure of interest.
• Critically appraised topic A short summary of an article from the literature, created to answer a specific clinical question.

EBM and Study Design

• << Previous: SR protocol
• Next: Searching for systematic reviews >>
• Getting started
• Types of reviews
• Formulate the question
• SR protocol
• Levels of evidence and study design
• Searching for systematic reviews
• Search strategies
• Subject databases
• Keeping up to date/Alerts
• Trial registers
• Conference proceedings
• Critical appraisal
• Documenting and reporting
• Managing search results
• Statistical methods
• Journal information/publishing
• Contact a librarian
• Last Updated: Aug 30, 2024 3:56 PM
• URL: https://ecu.au.libguides.com/systematic-reviews

Edith Cowan University acknowledges and respects the Noongar people, who are the traditional custodians of the land upon which its campuses stand and its programs operate. In particular ECU pays its respects to the Elders, past and present, of the Noongar people, and embrace their culture, wisdom and knowledge.

• Evidence-Based Medicine
• Finding the Evidence
• eJournals for EBM

Levels of Evidence

• JAMA Users' Guides
• Tutorials (Learning EBM)
• Web Resources

Resources That Rate The Evidence

• ACP Smart Medicine
• Agency for Healthcare Research and Quality
• Clinical Evidence
• Cochrane Library
• Health Services/Technology Assessment Texts (HSTAT)
• PDQ® Cancer Information Summaries from NCI
• Trip Database

Critically Appraised Individual Articles

• Evidence-Based Complementary and Alternative Medicine
• Evidence-Based Dentistry
• Evidence-Based Nursing
• Journal of Evidence-Based Dental Practice

Grades of Recommendation

Critically-appraised individual articles and synopses include:

Filtered evidence:

• Level I: Evidence from a systematic review of all relevant randomized controlled trials.
• Level II: Evidence from a meta-analysis of all relevant randomized controlled trials.
• Level III: Evidence from evidence summaries developed from systematic reviews
• Level IV: Evidence from guidelines developed from systematic reviews
• Level V: Evidence from meta-syntheses of a group of descriptive or qualitative studies
• Level VI: Evidence from evidence summaries of individual studies
• Level VII: Evidence from one properly designed randomized controlled trial

Unfiltered evidence:

• Level VIII: Evidence from nonrandomized controlled clinical trials, nonrandomized clinical trials, cohort studies, case series, case reports, and individual qualitative studies.
• Level IX: Evidence from opinion of authorities and/or reports of expert committee

Two things to remember:

1. Studies in which randomization occurs represent a higher level of evidence than those in which subject selection is not random.

2. Controlled studies carry a higher level of evidence than those in which control groups are not used.

Strength of Recommendation Taxonomy (SORT)

• SORT The American Academy of Family Physicians uses the Strength of Recommendation Taxonomy (SORT) to label key recommendations in clinical review articles. In general, only key recommendations are given a Strength-of-Recommendation grade. Grades are assigned on the basis of the quality and consistency of available evidence.
• << Previous: eJournals for EBM
• Next: JAMA Users' Guides >>
• Last Updated: Aug 16, 2024 4:11 PM
• URL: https://guides.library.stonybrook.edu/evidence-based-medicine
• Request a Class
• Hours & Locations
• Ask a Librarian
• Special Collections
• Library Faculty & Staff

Library Administration: 631.632.7100

• Stony Brook Home
• Campus Maps
• Web Accessibility Information
• Accessibility Barrier Report Form

Comments or Suggestions? | Library Webmaster

Except where otherwise noted, this work by SBU Libraries is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License .

Welcome to the new OASIS website! We have academic skills, library skills, math and statistics support, and writing resources all together in one new home.

• Walden University
• Faculty Portal

Evidence-Based Research: Levels of Evidence Pyramid

Introduction.

One way to organize the different types of evidence involved in evidence-based practice research is the levels of evidence pyramid. The pyramid includes a variety of evidence types and levels.

• systematic reviews
• critically-appraised topics
• critically-appraised individual articles
• randomized controlled trials
• cohort studies
• case-controlled studies, case series, and case reports
• Background information, expert opinion

Levels of evidence pyramid

The levels of evidence pyramid provides a way to visualize both the quality of evidence and the amount of evidence available. For example, systematic reviews are at the top of the pyramid, meaning they are both the highest level of evidence and the least common. As you go down the pyramid, the amount of evidence will increase as the quality of the evidence decreases.

Text alternative for Levels of Evidence Pyramid diagram

EBM Pyramid and EBM Page Generator, copyright 2006 Trustees of Dartmouth College and Yale University. All Rights Reserved. Produced by Jan Glover, David Izzo, Karen Odato and Lei Wang.

Filtered Resources

Filtered resources appraise the quality of studies and often make recommendations for practice. The main types of filtered resources in evidence-based practice are:

Scroll down the page to the Systematic reviews , Critically-appraised topics , and Critically-appraised individual articles sections for links to resources where you can find each of these types of filtered information.

Systematic reviews

Authors of a systematic review ask a specific clinical question, perform a comprehensive literature review, eliminate the poorly done studies, and attempt to make practice recommendations based on the well-done studies. Systematic reviews include only experimental, or quantitative, studies, and often include only randomized controlled trials.

You can find systematic reviews in these filtered databases :

• Cochrane Database of Systematic Reviews Cochrane systematic reviews are considered the gold standard for systematic reviews. This database contains both systematic reviews and review protocols. To find only systematic reviews, select Cochrane Reviews in the Document Type box.
• JBI EBP Database (formerly Joanna Briggs Institute EBP Database) This database includes systematic reviews, evidence summaries, and best practice information sheets. To find only systematic reviews, click on Limits and then select Systematic Reviews in the Publication Types box. To see how to use the limit and find full text, please see our Joanna Briggs Institute Search Help page .

You can also find systematic reviews in this unfiltered database :

To learn more about finding systematic reviews, please see our guide:

• Filtered Resources: Systematic Reviews

Critically-appraised topics

Authors of critically-appraised topics evaluate and synthesize multiple research studies. Critically-appraised topics are like short systematic reviews focused on a particular topic.

You can find critically-appraised topics in these resources:

• Annual Reviews This collection offers comprehensive, timely collections of critical reviews written by leading scientists. To find reviews on your topic, use the search box in the upper-right corner.
• Guideline Central This free database offers quick-reference guideline summaries organized by a new non-profit initiative which will aim to fill the gap left by the sudden closure of AHRQ’s National Guideline Clearinghouse (NGC).
• JBI EBP Database (formerly Joanna Briggs Institute EBP Database) To find critically-appraised topics in JBI, click on Limits and then select Evidence Summaries from the Publication Types box. To see how to use the limit and find full text, please see our Joanna Briggs Institute Search Help page .
• National Institute for Health and Care Excellence (NICE) Evidence-based recommendations for health and care in England.
• Filtered Resources: Critically-Appraised Topics

Critically-appraised individual articles

Authors of critically-appraised individual articles evaluate and synopsize individual research studies.

You can find critically-appraised individual articles in these resources:

• EvidenceAlerts Quality articles from over 120 clinical journals are selected by research staff and then rated for clinical relevance and interest by an international group of physicians. Note: You must create a free account to search EvidenceAlerts.
• ACP Journal Club This journal publishes reviews of research on the care of adults and adolescents. You can either browse this journal or use the Search within this publication feature.
• Evidence-Based Nursing This journal reviews research studies that are relevant to best nursing practice. You can either browse individual issues or use the search box in the upper-right corner.

To learn more about finding critically-appraised individual articles, please see our guide:

• Filtered Resources: Critically-Appraised Individual Articles

Unfiltered resources

You may not always be able to find information on your topic in the filtered literature. When this happens, you'll need to search the primary or unfiltered literature. Keep in mind that with unfiltered resources, you take on the role of reviewing what you find to make sure it is valid and reliable.

Note: You can also find systematic reviews and other filtered resources in these unfiltered databases.

The Levels of Evidence Pyramid includes unfiltered study types in this order of evidence from higher to lower:

You can search for each of these types of evidence in the following databases:

TRIP database

Background information & expert opinion.

Background information and expert opinions are not necessarily backed by research studies. They include point-of-care resources, textbooks, conference proceedings, etc.

• Family Physicians Inquiries Network: Clinical Inquiries Provide the ideal answers to clinical questions using a structured search, critical appraisal, authoritative recommendations, clinical perspective, and rigorous peer review. Clinical Inquiries deliver best evidence for point-of-care use.
• Harrison, T. R., & Fauci, A. S. (2009). Harrison's Manual of Medicine . New York: McGraw-Hill Professional. Contains the clinical portions of Harrison's Principles of Internal Medicine .
• Lippincott manual of nursing practice (8th ed.). (2006). Philadelphia, PA: Lippincott Williams & Wilkins. Provides background information on clinical nursing practice.
• Medscape: Drugs & Diseases An open-access, point-of-care medical reference that includes clinical information from top physicians and pharmacists in the United States and worldwide.
• Virginia Henderson Global Nursing e-Repository An open-access repository that contains works by nurses and is sponsored by Sigma Theta Tau International, the Honor Society of Nursing. Note: This resource contains both expert opinion and evidence-based practice articles.
• Previous Page: Phrasing Research Questions
• Next Page: Evidence Types
• Office of Student Disability Services

Walden Resources

Departments.

• Academic Residencies
• Academic Skills
• Career Planning and Development
• Customer Care Team
• Field Experience
• Military Services
• Student Success Advising
• Writing Skills

Centers and Offices

• Center for Social Change
• Office of Academic Support and Instructional Services
• Office of Degree Acceleration
• Office of Research and Doctoral Services
• Office of Student Affairs

Student Resources

• Doctoral Writing Assessment
• Form & Style Review
• Quick Answers
• ScholarWorks
• SKIL Courses and Workshops
• Walden Bookstore
• Walden Catalog & Student Handbook
• Student Safety/Title IX
• Legal & Consumer Information
• Website Terms and Conditions
• Cookie Policy
• Accessibility
• Accreditation
• State Authorization
• Net Price Calculator
• Cost of Attendance
• Contact Walden

Walden University is a member of Adtalem Global Education, Inc. www.adtalem.com Walden University is certified to operate by SCHEV © 2024 Walden University LLC. All rights reserved.

Nursing-Johns Hopkins Evidence-Based Practice Model

Jhebp model for levels of evidence, jhebp levels of evidence overview.

• Levels I, II and III

Evidence-Based Practice (EBP) uses a rating system to appraise evidence (usually a research study published as a journal article). The level of evidence corresponds to the research study design. Scientific research is considered to be the strongest form of evidence and recommendations from the strongest form of evidence will most likely lead to the best practices. The strength of evidence can vary from study to study based on the methods used and the quality of reporting by the researchers. You will want to seek the highest level of evidence available on your topic (Dang et al., 2022, p. 130).

The Johns Hopkins EBP model uses 3 ratings for the level of scientific research evidence 

• true experimental (level I)
• quasi-experimental (level II)
• nonexperimental (level III) 

The level determination is based on the research meeting the study design requirements  (Dang et al., 2022, p. 146-7).

You will use the Research Appraisal Tool (Appendix E) along with the Evidence Level and Quality Guide (Appendix D) to analyze and  appraise research studies . (Tools linked below.)

N onresearch evidence is covered in Levels IV and V.

• Evidence Level and Quality Guide (Appendix D)
• Research Evidence Appraisal Tool (Appendix E)

Level I Experimental study

randomized controlled trial (RCT)

Systematic review of RCTs, with or without meta-analysis

Level II Quasi-experimental Study

Systematic review of a combination of RCTs and quasi-experimental, or quasi-experimental studies only, with or without meta-analysis.

Level III Non-experimental study

Systematic review of a combination of RCTs, quasi-experimental and non-experimental, or non-experimental studies only, with or without meta-analysis.

Qualitative study or systematic review, with or without meta-analysis

Level IV Opinion of respected authorities and/or nationally recognized expert committees/consensus panels based on scientific evidence.

Clinical practice guidelines

Consensus panels

Level V Based on experiential and non-research evidence.

Literature reviews

Quality improvement, program, or financial evaluation

Case reports

Opinion of nationally recognized expert(s) based on experiential evidence

These flow charts can also help you detemine the level of evidence throigh a series of questions.

Single Quantitative Research Study

Summary/Reviews 

These charts are a part of the Research Evidence Appraisal Tool (Appendix E) document.

Dang, D., Dearholt, S., Bissett, K., Ascenzi, J., & Whalen, M. (2022). Johns Hopkins evidence-based practice for nurses and healthcare professionals: Model and guidelines. 4th ed. Sigma Theta Tau International

• << Previous: Start Here
• Next: Levels I, II and III >>
• Last Updated: Aug 15, 2024 1:14 PM
• URL: https://bradley.libguides.com/jhebp

Systematic Review Process: best practices

Levels of evidence.

• Formulate your search question
• Translate Search Strategies
• Locate systematic reviews & create a protocol
• Data collection: select your sources
• Why Grey Literature is important
• Screening and Data management
• KU Systematic Reviews

Hierarchy of evidence pyramid

The pyramidal shape qualitatively integrates the amount of evidence generally available from each type of study design and the strength of evidence expected from indicated designs.  Study designs in ascending levels of the pyramid generally exhibit increased quality of evidence and reduced risk of bias.

Understand the different levels of evidence

Meta Analysis  - systematic review that uses quantitative methods to synthesize and summarize the results.

Systematic Review  - summary of the medical literature that uses explicit methods to perform a comprehensive literature search and critical appraisal of individual studies and that uses appropriate statistical techniques to combine these valid studies.

Randomised Controlled Trial  - Participants are randomly allocated into an experimental group or a control group and followed over time for the variables/outcomes of interest.

Cohort Study  - Involves identification of two groups (cohorts) of patients, one which received the exposure of interest, and one which did not, and following these cohorts forward for the outcome of interest.

Case Control Study  - study which involves identifying patients who have the outcome of interest (cases) and patients without the same outcome (controls), and looking back to see if they had the exposure of interest.

Case Series  - report on a series of patients with an outcome of interest. No control group is involved.  (Definitions from CEBM)

Scholarly publications

The Joanna Briggs Institute Reviewers’ Manual 2015 Methodology for JBI Scoping Reviews

Clarifying differences between review designs and methods

Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach

A scoping review of scoping reviews: advancing the approach and enhancing the consistency

• << Previous: Welcome
• Next: Structure your search strategy >>

Suna Kıraç Library on Social Media

Koç University Suna Kıraç Library Rumelifeneri Yolu, 34450, Sarıyer-İstanbul T:+90-212 338 13 17 F:+90-212 338 13 21 [email protected]

Evidence-Based Practice in Health

• Introduction
• PICO Framework and the Question Statement
• Types of Clinical Question
• Hierarchy of Evidence

The Evidence Hierarchy: What is the "Best Evidence"?

Systematic reviews versus primary studies: what's best, systematic reviews and narrative reviews: what's the difference, filtered versus unfiltered information, the cochrane library.

• Selecting a Resource
• Searching PubMed
• Module 3: Appraise
• Module 4: Apply
• Module 5: Audit
• Reference Shelf

What is "the best available evidence"?  The hierarchy of evidence is a core principal of Evidence-Based Practice (EBP) and attempts to address this question.  The evidence higherarchy allows you to take a top-down approach to locating the best evidence whereby you first search for a recent well-conducted systematic review and if that is not available, then move down to the next level of evidence to answer your question.

EBP hierarchies rank study types based on the rigour (strength and precision) of their research methods.  Different hierarchies exist for different question types, and even experts may disagree on the exact rank of information in the evidence hierarchies.  The following image represents the hierarchy of evidence provided by the National Health and Medical Research Council (NHMRC). 1

Most experts agree that the higher up the hierarchy the study design is positioned, the more rigorous the methodology and hence the more likely it is that the study design can minimise the effect of bias on the results of the study.  In most evidence hierachies current, well designed systematic reviews and meta-analyses are at the top of the pyramid, and expert opinion and anecdotal experience are at the bottom. 2

Systematic Reviews and Meta Analyses

Well done systematic reviews, with or without an included meta-analysis, are generally considered to provide the best evidence for all question types as they are based on the findings of multiple studies that were identified in comprehensive, systematic literature searches.  However, the position of systematic reviews at the top of the evidence hierarchy is not an absolute.  For example:

• The process of a rigorous systematic review can take years to complete and findings can therefore be superseded by more recent evidence.
• The methodological rigor and strength of findings must be appraised by the reader before being applied to patients.
• A large, well conducted Randomised Controlled Trial (RCT) may provide more convincing evidence than a systematic review of smaller RCTs. 4

Primary Studies

If a current, well designed systematic review is not available, go to primary studies to answer your question. The best research designs for a primary study varies depending on the question type.  The table below lists optimal study methodologies for the main types of questions.

Note that the Clinical Queries filter available in some databases such as PubMed and CINAHL matches the question type to studies with appropriate research designs. When searching primary literature, look first for reports of clinical trials that used the best research designs. Remember as you search, though, that the best available evidence may not come from the optimal study type. For example, if treatment effects found in well designed cohort studies are sufficiently large and consistent, those cohort studies may provide more convincing evidence than the findings of a weaker RCT.

What is a Systematic Review?

A systematic review synthesises the results from all available studies in a particular area, and provides a thorough analysis of the results, strengths and weaknesses of the collated studies.  A systematic review has several qualities:

• It addresses a focused, clearly formulated question.
• It uses systematic and explicit methods:

                  a. to identify, select and critically appraise relevant research, and                   b. to collect and analyse data from the studies that are included in the review

Systematic reviews may or may not include a meta-analysis used to summarise and analyse the statistical results of included studies. This requires the studies to have the same outcome measure.

What is a Narrative Review?

Narrative reviews (often just called Reviews) are opinion with selective illustrations from the literature.  They do not qualify as adequate evidence to answer clinical questions.  Rather than answering a specific clinical question, they provide an overview of the research landscape on a given topic and so maybe useful for background information.  Narrative reviews usually lack systematic search protocols or explicit criteria for selecting and appraising evidence and are threfore very prone to bias. 5

Filtered information appraises the quality of a study and recommend its application in practice.  The critical appraisal of the individual articles has already been done for you—which is a great time saver.  Because the critical appraisal has been completed, filtered literature is appropriate to use for clinical decision-making at the point-of-care. In addition to saving time, filtered literature will often provide a more definitive answer than individual research reports.  Examples of filtered resources include, Cochrane Database of Systematic Reviews , BMJ Clincial Evidence , and ACP Journal Club .

Unfiltered information are original research studies that have not yet been synthesized or aggregated. As such, they are the more difficult to read, interpret, and apply to practice.  Examples of unfiltered resources include, CINAHL , EMBASE , Medline , and PubMe d . 3

The Cochrane Collaboration is an international voluntary organization that prepares, maintains and promotes the accessibility of systematic reviews of the effects of healthcare. 

The Cochrane Library is a database from the Cochrane Collaboration that allows simultaneous searching of six EBP databases.  Cochrane Reviews are systematic reviews authored by members of the Cochrane Collaboration and available via The Cochrane Database of Systematic Reviews .  They are widely recognised as the gold standard in systematic reviews due to the rigorous methodology used. 

Abstracts of completed Cochrane Reviews are freely available through PubMed and Meta-Search engines such as TRIP database. 

National access to the Cochrane Library is provided by the Australian Government via the National Health and Medical Research Council (NHMRC).

1. National Health and Medical Research Council. (2009). [Hierarchy of Evidence] . Retrieved 2 July, 2014 from: https://www.nhmrc.gov.au/

2. Hoffman, T., Bennett, S., & Del Mar, C. (2013). Evidence-Based Practice: Across the Health Professions (2nd ed.). Chatswood, NSW: Elsevier.

3. Kendall, S. (2008). Evidence-based resources simplified. Canadian Family Physician , 54, 241-243

4. Davidson, M., & Iles, R. (2013). Evidence-based practice in therapeutic health care. In, Liamputtong, P. (ed.). Research Methods in Health: Foundations for Evidence-Based Practice (2nd ed.). South Melbourne: Oxford University Press.

5. Cook, D., Mulrow, C., & Haynes, R. (1997). Systematic reviews: synthesis of best evidence for clinical decisions. Annals of Internal Medicine , 126, 376–80.

• << Previous: Types of Clinical Question
• Next: Module 2: Acquire >>
• Last Updated: Jul 24, 2023 4:08 PM
• URL: https://canberra.libguides.com/evidence

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• BMJ Journals

You are here

• Volume 21, Issue 4
• New evidence pyramid
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• M Hassan Murad ,
• Mouaz Alsawas ,
• http://orcid.org/0000-0001-5481-696X Fares Alahdab
• Rochester, Minnesota , USA
• Correspondence to : Dr M Hassan Murad, Evidence-based Practice Center, Mayo Clinic, Rochester, MN 55905, USA; murad.mohammad{at}mayo.edu

https://doi.org/10.1136/ebmed-2016-110401

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

• EDUCATION & TRAINING (see Medical Education & Training)
• EPIDEMIOLOGY
• GENERAL MEDICINE (see Internal Medicine)

The first and earliest principle of evidence-based medicine indicated that a hierarchy of evidence exists. Not all evidence is the same. This principle became well known in the early 1990s as practising physicians learnt basic clinical epidemiology skills and started to appraise and apply evidence to their practice. Since evidence was described as a hierarchy, a compelling rationale for a pyramid was made. Evidence-based healthcare practitioners became familiar with this pyramid when reading the literature, applying evidence or teaching students.

Various versions of the evidence pyramid have been described, but all of them focused on showing weaker study designs in the bottom (basic science and case series), followed by case–control and cohort studies in the middle, then randomised controlled trials (RCTs), and at the very top, systematic reviews and meta-analysis. This description is intuitive and likely correct in many instances. The placement of systematic reviews at the top had undergone several alterations in interpretations, but was still thought of as an item in a hierarchy. 1 Most versions of the pyramid clearly represented a hierarchy of internal validity (risk of bias). Some versions incorporated external validity (applicability) in the pyramid by either placing N-1 trials above RCTs (because their results are most applicable to individual patients 2 ) or by separating internal and external validity. 3

Another version (the 6S pyramid) was also developed to describe the sources of evidence that can be used by evidence-based medicine (EBM) practitioners for answering foreground questions, showing a hierarchy ranging from studies, synopses, synthesis, synopses of synthesis, summaries and systems. 4 This hierarchy may imply some sort of increasing validity and applicability although its main purpose is to emphasise that the lower sources of evidence in the hierarchy are least preferred in practice because they require more expertise and time to identify, appraise and apply.

The traditional pyramid was deemed too simplistic at times, thus the importance of leaving room for argument and counterargument for the methodological merit of different designs has been emphasised. 5 Other barriers challenged the placement of systematic reviews and meta-analyses at the top of the pyramid. For instance, heterogeneity (clinical, methodological or statistical) is an inherent limitation of meta-analyses that can be minimised or explained but never eliminated. 6 The methodological intricacies and dilemmas of systematic reviews could potentially result in uncertainty and error. 7 One evaluation of 163 meta-analyses demonstrated that the estimation of treatment outcomes differed substantially depending on the analytical strategy being used. 7 Therefore, we suggest, in this perspective, two visual modifications to the pyramid to illustrate two contemporary methodological principles ( figure 1 ). We provide the rationale and an example for each modification.

• Download figure
• Open in new tab
• Download powerpoint

The proposed new evidence-based medicine pyramid. (A) The traditional pyramid. (B) Revising the pyramid: (1) lines separating the study designs become wavy (Grading of Recommendations Assessment, Development and Evaluation), (2) systematic reviews are ‘chopped off’ the pyramid. (C) The revised pyramid: systematic reviews are a lens through which evidence is viewed (applied).

Rationale for modification 1

In the early 2000s, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group developed a framework in which the certainty in evidence was based on numerous factors and not solely on study design which challenges the pyramid concept. 8 Study design alone appears to be insufficient on its own as a surrogate for risk of bias. Certain methodological limitations of a study, imprecision, inconsistency and indirectness, were factors independent from study design and can affect the quality of evidence derived from any study design. For example, a meta-analysis of RCTs evaluating intensive glycaemic control in non-critically ill hospitalised patients showed a non-significant reduction in mortality (relative risk of 0.95 (95% CI 0.72 to 1.25) 9 ). Allocation concealment and blinding were not adequate in most trials. The quality of this evidence is rated down due to the methodological imitations of the trials and imprecision (wide CI that includes substantial benefit and harm). Hence, despite the fact of having five RCTs, such evidence should not be rated high in any pyramid. The quality of evidence can also be rated up. For example, we are quite certain about the benefits of hip replacement in a patient with disabling hip osteoarthritis. Although not tested in RCTs, the quality of this evidence is rated up despite the study design (non-randomised observational studies). 10

Rationale for modification 2

Another challenge to the notion of having systematic reviews on the top of the evidence pyramid relates to the framework presented in the Journal of the American Medical Association User's Guide on systematic reviews and meta-analysis. The Guide presented a two-step approach in which the credibility of the process of a systematic review is evaluated first (comprehensive literature search, rigorous study selection process, etc). If the systematic review was deemed sufficiently credible, then a second step takes place in which we evaluate the certainty in evidence based on the GRADE approach. 11 In other words, a meta-analysis of well-conducted RCTs at low risk of bias cannot be equated with a meta-analysis of observational studies at higher risk of bias. For example, a meta-analysis of 112 surgical case series showed that in patients with thoracic aortic transection, the mortality rate was significantly lower in patients who underwent endovascular repair, followed by open repair and non-operative management (9%, 19% and 46%, respectively, p<0.01). Clearly, this meta-analysis should not be on top of the pyramid similar to a meta-analysis of RCTs. After all, the evidence remains consistent of non-randomised studies and likely subject to numerous confounders.

Therefore, the second modification to the pyramid is to remove systematic reviews from the top of the pyramid and use them as a lens through which other types of studies should be seen (ie, appraised and applied). The systematic review (the process of selecting the studies) and meta-analysis (the statistical aggregation that produces a single effect size) are tools to consume and apply the evidence by stakeholders.

Implications and limitations

Changing how systematic reviews and meta-analyses are perceived by stakeholders (patients, clinicians and stakeholders) has important implications. For example, the American Heart Association considers evidence derived from meta-analyses to have a level ‘A’ (ie, warrants the most confidence). Re-evaluation of evidence using GRADE shows that level ‘A’ evidence could have been high, moderate, low or of very low quality. 12 The quality of evidence drives the strength of recommendation, which is one of the last translational steps of research, most proximal to patient care.

One of the limitations of all ‘pyramids’ and depictions of evidence hierarchy relates to the underpinning of such schemas. The construct of internal validity may have varying definitions, or be understood differently among evidence consumers. A limitation of considering systematic review and meta-analyses as tools to consume evidence may undermine their role in new discovery (eg, identifying a new side effect that was not demonstrated in individual studies 13 ).

This pyramid can be also used as a teaching tool. EBM teachers can compare it to the existing pyramids to explain how certainty in the evidence (also called quality of evidence) is evaluated. It can be used to teach how evidence-based practitioners can appraise and apply systematic reviews in practice, and to demonstrate the evolution in EBM thinking and the modern understanding of certainty in evidence.

• Leibovici L
• Agoritsas T ,
• Vandvik P ,
• Neumann I , et al
• ↵ Resources for Evidence-Based Practice: The 6S Pyramid. Secondary Resources for Evidence-Based Practice: The 6S Pyramid Feb 18, 2016 4:58 PM. http://hsl.mcmaster.libguides.com/ebm
• Vandenbroucke JP
• Berlin JA ,
• Dechartres A ,
• Altman DG ,
• Trinquart L , et al
• Guyatt GH ,
• Vist GE , et al
• Coburn JA ,
• Coto-Yglesias F , et al
• Sultan S , et al
• Montori VM ,
• Ioannidis JP , et al
• Altayar O ,
• Bennett M , et al
• Nissen SE ,

Contributors MHM conceived the idea and drafted the manuscript. FA helped draft the manuscript and designed the new pyramid. MA and NA helped draft the manuscript.

Competing interests None declared.

Provenance and peer review Not commissioned; externally peer reviewed.

Linked Articles

• Editorial Pyramids are guides not rules: the evolution of the evidence pyramid Terrence Shaneyfelt BMJ Evidence-Based Medicine 2016; 21 121-122 Published Online First: 12 Jul 2016. doi: 10.1136/ebmed-2016-110498
• Perspective EBHC pyramid 5.0 for accessing preappraised evidence and guidance Brian S Alper R Brian Haynes BMJ Evidence-Based Medicine 2016; 21 123-125 Published Online First: 20 Jun 2016. doi: 10.1136/ebmed-2016-110447

Read the full text or download the PDF:

Nursing - Systematic Reviews: Levels of Evidence

• Levels of Evidence
• Meta-Analyses
• Definitions
• Citation Search
• Write & Cite
• Give Feedback

"How would I use the 6S Model while taking care of a patient?" .cls-1{fill:#fff;stroke:#79a13f;stroke-miterlimit:10;stroke-width:5px;}.cls-2{fill:#79a13f;} The 6S Model is designed to work from the top down, starting with Systems - also referred to as computerized decision support systems (CDSSs). DiCenso et al. describes that, “an evidence-based clinical information system integrates and concisely summarizes all relevant and important research evidence about a clinical problem, is updated as new research evidence becomes available, and automatically links (through an electronic medical record) a specific patient’s circumstances to the relevant information” (2009). Systematic reviews lead up to this type of bio-available level of evidence.

What are systematic reviews, polit–beck evidence hierarchy/levels of evidence scale for therapy questions.

"Figure 2.2 [in context of book] shows our eight-level evidence hierarchy for Therapy/intervention questions. This hierarchy ranks sources of evidence with respect the readiness of an intervention to be put to use in practice" (Polit & Beck, 2021, p. 28). Levels are ranked on risk of bias - level one being the least bias, level eight being the most biased. There are several types of levels of evidence scales designed for answering different questions. "An evidence hierarchy for Prognosis questions, for example, is different from the hierarchy for Therapy questions" (p. 29).

Advantages of Levels of Evidence Scales

"Through controls imposed by manipulation, comparison, and randomization, alternative explanations can be discredited. It is because of this strength that meta-analyses of RCTs, which integrate evidence from multiple experiments, are at the pinnacle of the evidence hierarchies for Therapy questions" (p. 188).

"Tip: Traditional evidence hierarchies or level of evidence scales (e.g., Figure 2.2), rank evidence sources almost exclusively based on the risk of internal validity threats" (p. 217).

Systematic reviews can provide researchers with knowledge that prior evidence shows. This can help clarify established efficacy of a treatment without unnecessary and thus unethical research. Greenhalgh (2019) illustrates this citing Dean Fergusson and colleagues (2005) systematic review on a clinical surgical topic (p. 128).

Limits of Levels of Evidence Scales

Regarding the importance of real-world clinical practice settings, and the conflicting tradeoffs between internal and external validity, Polit and Beck (2021) write, "the first (and most prevalent) approach is to emphasize one and sacrifice another. Most often, it is external validity that is sacrificed. For example, external validity is not even considered in ranking evidence in level of evidence scales" (p. 221). ... From an EBP perspective, it is important to remember that drawing inferences about causal relationships relies not only on how high up on the evidence hierarchy a study is (Figure 2.2), but also, for any given level of the hierarchy, how successful the researcher was in managing study validity and balancing competing validity demands" (p. 222).

Polit and Beck note Levin (2014) that an evidence hierarchy "is not meant to provide a quality rating for evidence retrieved in the search for an answer" (p. 6), and as the Oxford Center for Evidence-Based Medicine concurs that evidence scales are, 'NOT intended to provide you with a definitive judgment about the quality of the evidence. There will inevitably be cases where "lower-level" evidence...will provide stronger than a "higher level" study (Howick et al., 2011, p.2)'" (p. 30).

Level of evidence (e.g., Figure 2.2) + Quality of evidence = Strength of evidence .

The 6S Model of Levels of Evidence

"The 6S hierarchy does not imply a gradient of evidence in terms of quality , but rather in terms of ease in retrieving relevant evidence to address a clinical question. At all levels, the evidence should be assessed for quality and relevance" (Polit & Beck, 2021, p. 24, Tip box).

The 6S Pyramid proposes a structure of quantitative evidence where articles that include pre-appraised and pre-synthesized studies are located at the top of the hierarchy (McMaster U., n.d.).

It can help to consider the level of evidence that a document represents, for example, a scientific article that summarizes and analyses many similar articles may provide more insight than the conclusion of a single research article. This is not to say that summaries can not be flawed, nor does it suggest that rare case studies should be ignored. The aim of health research is the well-being of all people, therefore it is important to use current evidence in light of patient preferences negotiated with clinical expertise.

Other Gradings in Levels of Evidence

While it is accepted that the strongest evidence is derived from meta-analyses, various evidence grading systems exist. for example: The Johns Hopkins Nursing Evidence-Based Practice model ranks evidence from level I to level V, as follows (Seben et al., 2010): Level I: Meta-analysis of randomized clinical trials (RCTs); experimental studies; RCTs Level II: Quasi-experimental studies Level III: Non-experimental or qualitative studies Level IV: Opinions of nationally recognized experts based on research evidence or an expert consensus panel Level V: Opinions of individual experts based on non-research evidence (e.g., case studies, literature reviews, organizational experience, and personal experience) The American Association of Critical-Care Nurses (AACN) evidence level system , updated in 2009, ranks evidence as follows (Armola et al., 2009): Level A: Meta-analysis of multiple controlled studies or meta-synthesis of qualitative studies with results that consistently support a specific action, intervention, or treatment Level B: Well-designed, controlled randomized or non-randomized studies with results that consistently support a specific action, intervention, or treatment Level C: Qualitative, descriptive, or correlational studies, integrative or systematic reviews, or RCTs with inconsistent results Level D: Peer-reviewed professional organizational standards, with clinical studies to support recommendations Level E: Theory-based evidence from expert opinion or multiple case reports Level M: Manufacturers’ recommendations (2017)

EBM Pyramid and EBM Page Generator

Unfiltered are resources that are primary sources describing original research. Randomized controlled trials, cohort studies, case-controlled studies, and case series/reports are considered unfiltered information.

Filtered are resources that are secondary sources which summarize and analyze the available evidence. They evaluate the quality of individual studies and often provide recommendations for practice. Systematic reviews, critically-appraised topics, and critically-appraised individual articles are considered filtered information.

Armola, R. R., Bourgault, A. M., Halm, M. A., Board, R. M., Bucher, L., Harrington, L., ... Medina, J. (2009). AACN levels of evidence. What's new? Critical Care Nurse , 29 (4), 70-73. doi:10.4037/ccn2009969

DiCenso, A., Bayley, L., & Haynes, R. B. (2009). Accessing pre-appraised evidence: Fine-tuning the 5S model into a 6S model. BMJ Evidence-Based Nursing , 12 (4) https://ebn.bmj.com/content/12/4/99.2.short

Fergusson, D., Glass, K. C., Hutton, B., & Shapiro, S. (2005). Randomized controlled trials of Aprotinin in cardiac surgery: Could clinical equipoise have stopped the bleeding?. Clinical Trials , 2 (3), 218-232.

Glover, J., Izzo, D., Odato, K. & Wang, L. (2008). Evidence-based mental health resources . EBM Pyramid and EBM Page Generator. Copyright 2008. All Rights Reserved. Retrieved April 28, 2020 from https://web.archive.org/web/20200219181415/http://www.dartmouth.edu/~biomed/resources.htmld/guides/ebm_psych_resources.html Note. Document removed from host. Old link used with the WayBack Machine of the Internet Archive to retrieve the original webpage on 2/10/21 http://www.dartmouth.edu/~biomed/resources.htmld/guides/ebm_psych_resources.html

Greenhalgh, T. (2019). How to read a paper: The basics of evidence-based medicine and healthcare . (Sixth ed.). Wiley Blackwell.

Haynes, R. B. (2001). Of studies, syntheses, synopses, and systems: The “4S” evolution of services for finding current best evidence. BMJ Evidence-Based Medicine , 6 (2), 36-38.

Haynes, R. B. (2006). Of studies, syntheses, synopses, summaries, and systems: the “5S” evolution of information services for evidence-based healthcare decisions. BMJ Evidence-Based Medicine , 11 (6), 162-164.

McMaster University (n.d.). 6S Search Pyramid Tool https://www.nccmt.ca/capacity-development/6s-search-pyramid

Polit, D., & Beck, C. (2019). Nursing research: Generating and assessing evidence for nursing practice . Wolters Kluwer Health.

Schub, E., Walsh, K. & Pravikoff D. (Ed.) (2017). Evidence-based nursing practice: Implementing [Skill Set]. Nursing Reference Center Plus

Seben, S., March, K. S., & Pugh, L. C. (2010). Evidence-based practice: The forum approach. American Nurse Today , 5 (11), 32-34.

• Systematic Review from the Encyclopedia of Nursing Research by Cheryl Holly Systematic reviews provide reliable evidential summaries of past research for the busy practitioner. By pooling results from multiple studies, findings are based on multiple populations, conditions, and circumstances. The pooled results of many small and large studies have more precise, powerful, and convincing conclusions (Holly, Salmond, & Saimbert, 2016) [ references in article ]. This scholarly synthesis of research findings and other evidence forms the foundation for evidence-based practice allowing the practitioner to make up-to-date decisions.

Standards & Guides

• Cochrane Handbook for Systematic Reviews of Interventions The Cochrane Handbook for Systematic Reviews of Interventions is the official guide that describes in detail the process of preparing and maintaining Cochrane systematic reviews on the effects of healthcare interventions.
• Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) PRISMA is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. PRISMA focuses on the reporting of reviews evaluating randomized trials, but can also be used as a basis for reporting systematic reviews of other types of research, particularly evaluations of interventions.
• Systematic Reviews by The Centre for Reviews and Dissemination "The guidance has been written for those with an understanding of health research but who are new to systematic reviews; those with some experience but who want to learn more; and for commissioners. We hope that experienced systematic reviewers will also find this guidance of value; for example when planning a review in an area that is unfamiliar or with an expanded scope. This guidance might also be useful to those who need to evaluate the quality of systematic reviews, including, for example, anyone with responsibility for implementing systematic review findings" (CRD, 2009, p. vi, "Who should use this guide")

• Carrying out systematic literature reviews: An introduction by Alan Davies Systematic reviews provide a synthesis of evidence for a specific topic of interest, summarising the results of multiple studies to aid in clinical decisions and resource allocation. They remain among the best forms of evidence, and reduce the bias inherent in other methods. A solid understanding of the systematic review process can be of benefit to nurses that carry out such reviews, and for those who make decisions based on them. An overview of the main steps involved in carrying out a systematic review is presented, including some of the common tools and frameworks utilised in this area. This should provide a good starting point for those that are considering embarking on such work, and to aid readers of such reviews in their understanding of the main review components, in order to appraise the quality of a review that may be used to inform subsequent clinical decision making (Davies, 2019, Abstract)
• Papers that summarize other papers (systematic reviews and meta-analyses) by Trisha Greenhalgh ... a systematic review is an overview of primary studies that: contains a statement of objectives, sources and methods; has been conducted in a way that is explicit, transparent and reproducible (Figure 9.1) [ Table found in book chapter ]. The most enduring and reliable systematic reviews, notably those undertaken by the Cochrane Collaboration (discussed later in this chapter), are regularly updated to incorporate new evidence (Greenhalgh, 2020, p. 117, Chapter 9).
• A PRISMA assessment of the reporting quality of systematic reviews of nursing published in the Cochrane Library and paper-based journals by Juxia Zhang et al. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) was released as a standard of reporting systematic reviewers (SRs). However, not all SRs adhere completely to this standard. This study aimed to evaluate the reporting quality of SRs published in the Cochrane Library and paper-based journals (Zhang et al., 2019, Abstract).

Cochrane [Username]. (2016, Jan 27). What are systematic reviews? YouTube. https://www.youtube.com/watch?v=egJlW4vkb1Y

Davies, A. (2019). Carrying out systematic literature reviews: An introduction. British Journal of Nursing , 28 (15), 1008–1014. https://doi-org.ezproxy.simmons.edu/10.12968/bjon.2019.28.15.1008

Greenhalgh, T. (2019). Papers that summarize other papers (systematic reviews and meta-analyses). In How to read a Paper : The basics of evidence-based medicine and healthcare . (Sixth ed., pp. 117-136). Wiley Blackwell.

Holly, C. (2017). Systematic review. In J. Fitzpatrick (Ed.), Encyclopedia of nursing research (4th ed.). Springer Publishing Company. Credo Reference.

Zhang, J., Han, L., Shields, L., Tian, J., & Wang, J. (2019). A PRISMA assessment of the reporting quality of systematic reviews of nursing published in the Cochrane Library and paper-based journals. Medicine , 98 (49), e18099. https://doi.org/10.1097/MD.0000000000018099

• << Previous: Start
• Next: Meta-Analyses >>
• Last Updated: Nov 3, 2023 1:19 PM
• URL: https://simmons.libguides.com/systematic-reviews

Evidence Based Practice: Study Designs & Evidence Levels

• Databases to Search
• EBP Resources
• Study Designs & Evidence Levels
• How Do I...

Introduction

This section reviews some research definitions and provides commonly used evidence tables.

Levels of Evidence Johns Hopkins Nursing Evidence Based Practice

Dang, D., & Dearholt, S. (2017). Johns Hopkins nursing evidence-based practice: model and guidelines. 3rd ed. Indianapolis, IN: Sigma Theta Tau International. www.hopkinsmedicine.org/evidence-based-practice/ijhn_2017_ebp.html

Identifying the Study Design

The type of study can generally be figured out by looking at three issues:

Q1. What was the aim of the study?

• To simply describe a population (PO questions)  = descriptive
• To quantify the relationship between factors (PICO questions)  =  analytic.

Q2. If analytic, was the intervention randomly allocated?

• Yes?  =  RCT 
• No? = Observational study  

For an observational study, the main type will then depend on the timing of the measurement of outcome, so our third question is:

Q3. When were the outcomes determined?

• Some time after the exposure or intervention? = Cohort study ('prospective study')
• At the same time as the exposure or intervention? = Cross sectional study or survey
• Before the exposure was determined? = Case-control study ('retrospective study' based on recall of the exposure)

Centre for Evidence-Based Medicine (CEBM)

Definitions of Study Types

Case report / Case series:  A report on a series of patients with an outcome of interest. No control group is involved.

Case control study:  A study which involves identifying patients who have the outcome of interest (cases) and patients without the same outcome (controls), and looking back to see if they had the exposure of interest.

Cohort study:  Involves identification of two groups (cohorts) of patients, one which received the exposure of interest, and one which did not, and following these cohorts forward for the outcome of interest.

Randomized controlled clinical trial:  Participants are randomly allocated into an experimental group or a control group and followed over time for the variables/outcomes of interest.

Systematic review:  A summary of the medical literature that uses explicit methods to perform a comprehensive literature search and critical appraisal of individual studies and that uses appropriate statistical techniques to combine these valid studies.

Meta-analysis:  A systematic review that uses quantitative methods to synthesize and summarize the results.

Meta-synthesis: A systematic approach to the analysis of data across qualitative studies. -- EJ Erwin, MJ Brotherson, JA Summers. Understanding Qualitative Meta-synthesis. Issues and Opportunities in Early Childhood Intervention Research, 33(3) 186-200 .

Cross sectional study:  The observation of a defined population at a single point in time or time interval. Exposure and outcome are determined simultaneously.

Prospective, blind comparison to a gold standard:  Studies that show the efficacy of a diagnostic test are also called prospective, blind comparison to a gold standard study. This is a controlled trial that looks at patients with varying degrees of an illness and administers both diagnostic tests — the test under investigation and the “gold standard” test — to all of the patients in the study group. The sensitivity and specificity of the new test are compared to that of the gold standard to determine potential usefulness.

Qualitative research:  answers a wide variety of questions related to human responses to actual or potential health problems.The purpose of qualitative research is to describe, explore and explain the health-related phenomena being studied.

Retrospective cohort:  follows the same direction of inquiry as a cohort study.  Subjects begin with the presence or absence of an exposure or risk factor and are followed until the outcome of interest is observed.  However, this study design uses information that has been collected in the past and kept in files or databases.  Patients are identified for exposure or non-exposures and the data is followed forward to an effect or outcome of interest.

(Adapted from CEBM's Glossary and Duke Libraries' Intro to Evidence-Based Practice )

American Association of Critical Care Nursing-- Levels of Evidence

Level A   Meta-analysis of multiple controlled studies or meta-synthesis of qualitative studies with results that consistently support a specific action, intervention or treatment

Level B  Well designed controlled studies, both randomized and nonrandomized, with results that consistently support a specific action, intervention, or treatment

Level C   Qualitative studies, descriptive or correlational studies, integrative reviews, systematic reviews, or randomized controlled trials with inconsistent results

Level D Peer-reviewed professional organizational standards, with clinical studies to support recommendations

Level E Theory-based evidence from expert opinion or multiple case reports

Level M  Manufacturers’ recommendations only  

Armola RR, Bourgault AM, Halm MA, Board RM, Bucher L, Harrington L, Heafey CA, Lee R, Shellner PK, Medina J. (2009) AACN levels of evidence: what's new ?  J.Crit Care Nurse. Aug;29(4):70-3.

Flow Chart of Study Designs

Figure: Flow chart of different types of studies (Q1, 2, and 3 refer to the three questions below  in "Identifying the Study Design" box.) Centre for Evidence-Based Medicine (CEBM)

What is a "Confidence Interval (CI)"?

A confidence interval (CI) can be used to show within which interval the population's mean score will probably fall. Most researchers use a CI of 95%. By using a CI of 95%, researchers accept there is a 5% chance they have made the wrong decision in treatment. Therefore, if 0 falls within the agreed CI, it can be concluded that there is no significant difference between the two treatments. When 0 lies outside the CI, researchers will conclude that there is a statistically significant difference.

Halfens, R. G., & Meijers, J. M. (2013). Back to basics: an introduction to statistics.  Journal Of Wound Care ,  22 (5), 248-251.

What is a "p-value?"

Categorical (nominal) tests This category of tests can be used when the dependent, or outcome, variable is categorical (nominal), such as the dif­ference between two wound treatments and the healing of the wound (healed versus non­healed). One of the most used tests in this category is the chi­squared test (χ2). The chi­squared statistic is calculated by comparing the differences between the observed and the expected frequencies. The expected frequencies are the frequencies that would be found if there was no relationship between the two variables. 

Based on the calculated χ2 statistic, a probability (p ­value) is given, which indicates the probability that the two means are not different from each other. Researchers are often satisfied if the probability is 5% or less, which means that the researchers would conclude that for p < 0.05, there is a significant difference. A p ­value ≥ 0.05 suggests that there is no significant difference between the means.

Halfens, R. G., & Meijers, J. M. (2013). Back to basics: an introduction to statistics. Journal Of Wound Care, 22(5), 248-251.

• << Previous: EBP Resources
• Next: How Do I... >>
• Last Updated: Sep 9, 2024 1:01 PM
• URL: https://mcw.libguides.com/evidencebasedpractice

MCW Libraries 8701 Watertown Plank Road Milwaukee, WI 53226 (414) 955-8300

Contact Us Locations & Hours Send Us Your Comments

Systematic Reviews

• The Research Question
• Inclusion and Exclusion Criteria
• Original Studies
• Translating
• Deduplication
• Project Management Tools
• Useful Resources
• What is not a systematic review?

Cochrane resources

Cochrane Handbook for Systematic Reviews of Interventions

• Introduction to Systematic Reviews

Here is a PowerPoint presentation that provides a brief overview of Systematic Reviews   Texas Medical Center Library

What is a systematic review?

A systematic review attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a research question. 1  Systematic Reviews are research projects that provide new insight on a topic and are designed to minimize bias. The project creates accessible research that examines relevant literature, which aids decision makers by aggregating information in a systematic way. Methodological transparency, along with its systematic approach and project reproducibility, are key elements of a systematic review.

1. Taken from Lasserson TJ, Thomas J, Higgins JPT. Chapter 1: Starting a review. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors).  Cochrane Handbook for Systematic Reviews of Interventions  version 6.1 (updated September 2020). Cochrane, 2020. Available from  www.training.cochrane.org/handbook

Components of a Systematic Review

Key elements of a systematic review include :.

• A specific and well-formulated question
• A reproducible methodology intended to avoid bias 
• Multiple databases searched for the review's data
• Specified and predefined inclusion and exclusion criteria 
• Multiple reviewers of the literature 
• Study assessments conducted in a standardized way with definitive methodology
• Adherence to a standardized reporting guideline, such as PRISMA

Systematic reviews can have an impact on the development of public health policies and on resource allocation decisions. They can inform clinical practices and implement evidence-based interventions for diseases and illnesses. Moreover, systematic reviews can compare benefits and harms of treatment options.

The systematic review process has been developed to minimize bias   and ensure transparency. Methods should be adequately documented so that they can be replicated. The integrity of a systematic review is based on its transparency and reproducibility of the methods used for the review. 

There are many resources on how to conduct, organize, and publish a systematic review. This guide is by no means exhaustive; its aim is to provide a starting place for understanding the core of what a systematic review is and how to conduct one.

What does it take to do a systematic review?

Time :  On average, systematic reviews can require up to 18 months of preparation. 

A team:  A systematic review can't be done alone! You need to work with subject experts to clarify issues related to the topic; librarians to develop comprehensive search strategies and identify appropriate databases; reviewers to screen abstracts and read the full text; a statistician who can assist with data analysis; and a project leader to coordinate the team and movement of data.

A clearly defined question : A clearly defined research question can help clarify the key concepts of a systematic review and explain the rationale for the review. It is recommended to use a framework (e.g. the PICO framework) to identify key concepts of the question.

A written protocol :  The protocol should outline the study methodology. The protocol should include the rationale for the systematic review; the research question broken into PICO components; explicit inclusion/exclusion criteria; relevant known literature on the research question; preliminary search terms and databases to be used; intended data abstraction/data management tools; and other components that may be unique to register the protocol. 

A registered protocol :   A few recommendations are  PROSPERO , an International Prospective Register of Systematic Reviews; Cochrane; and the Agency for Healthcare Research and Quality. Registering a protocol is important because it reduces duplication of effort and promotes transparency. 

Inclusion/exclusion criteria: Inclusion/exclusion criteria can help researchers define the terms of the investigation. These will include  the predefined question; study types; study-analysis criteria (i.e. criteria for reporting bias within studies); and quantitative methods to be used for any statistical analysis. 

Comprehensive literature searches :  Identify appropriate databases and conduct comprehensive and detailed literature searches that can be documented and duplicated. Cochrane recommends that 3+ different databases be used to conduct the searches. A strategy must be developed and then translated across the multiple pre-specified databases, preferably by an information specialist.  

Citation management:  You should have working knowledge of EndNote or another citation management system that will be accessible to the research team to help manage citations retrieved from literature searches.

Follow reporting guidelines :  Use appropriate guidelines for reporting your review for publication.

Time -  requires about 18 months of preparation .

  The suggested timeline for a Cochrane review is: 

• Preparation of protocol  1 – 2 months
• Searches for published and unpublished studies  3-8 months
• Pilot test of eligibility criteria  2-3 months
• Inclusion assessments  3-8 months
• Pilot test of ‘Risk of bias’ assessment  3 months
• Validity assessments  3-10 months
• Pilot test of data collection  3 months
• Data collection  3-10 months
• Data entry  3-10 months
• Follow up of missing information  5-11 months
• Analysis  8-10 months
• Preparation of review report  1-11 months
• Keeping the review up-to-date  12 months
• Next: Services >>
• Last Updated: Jul 29, 2024 2:41 PM
• URL: https://libguides.sph.uth.tmc.edu/SystematicReviews

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

Genome-wide association studies on periodontitis: A systematic review

Roles Conceptualization, Data curation, Formal analysis, Investigation, Project administration, Software, Validation, Visualization, Writing – original draft, Writing – review & editing

Affiliation School of Dentistry, University of Leeds, Leeds, United Kingdom

Roles Methodology, Supervision, Validation, Writing – review & editing

Affiliation School of Medicine, University of Leeds, Leeds, United Kingdom

Roles Data curation, Investigation, Methodology, Writing – review & editing

Affiliation Wolfson Institute of Population Health, Queen Mary, University of London, London, United Kingdom

Roles Funding acquisition, Methodology, Resources, Supervision, Validation, Writing – review & editing

Affiliation Leeds Institute of Medical Research, School of Medicine, University of Leeds, Leeds, United Kingdom

Roles Funding acquisition, Supervision, Writing – review & editing

Affiliation School of Psychology, University of Leeds, Leeds, United Kingdom

Roles Validation, Writing – review & editing

Affiliation Centre for Host Microbial Interactions, Faculty of Dentistry Oral and Craniofacial Sciences, King’s College London, London, United Kingdom

Roles Data curation, Formal analysis, Methodology, Project administration

Roles Funding acquisition, Supervision

Roles Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Writing – review & editing

Roles Conceptualization, Funding acquisition, Investigation, Methodology, Project administration, Resources, Software, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

Affiliation Oral Clinical Research Unit, Faculty of Dentistry Oral and Craniofacial Sciences, King’s College London, London, United Kingdom

• Chenyi Gao, 
• Mark Iles, 
• Harriet Larvin, 
• David Timothy Bishop, 
• David Bunce, 
• Mark Ide, 
• Fanyiwen Sun, 
• Susan Pavitt, 
• Jianhua Wu, 

• Published: September 6, 2024
• https://doi.org/10.1371/journal.pone.0306983
• Reader Comments

This study aims to systematically review the existing literature and critically appraise the evidence of genome-wide association studies (GWAS) on periodontitis. This study also aims to synthesise the findings of genetic risk variants of periodontitis from included GWAS.

A systematic search was conducted on PubMed, GWAS Catalog, MEDLINE, GLOBAL HEALTH and EMBASE via Ovid for GWAS on periodontitis. Only studies exploring single-nucleotide polymorphisms(SNPs) associated with periodontitis were eligible for inclusion. The quality of the GWAS was assessed using the Q-genie tool. Information such as study population, ethnicity, genomic data source, phenotypic characteristics(definition of periodontitis), and GWAS methods(quality control, analysis stages) were extracted. SNPs that reached conventional or suggestive GWAS significance level(5e-8 or 5e-06) were extracted and synthesized.

A total of 15 good-quality GWAS on periodontitis were included (Q-genie scores ranged from 38–50). There were huge heterogeneities among studies. There were 11 identified risk SNPs (rs242016, rs242014, rs10491972, rs242002, rs2978951, rs2738058, rs4284742, rs729876, rs149133391, rs1537415, rs12461706) at conventional GWAS significant level ( p<5x10 -8 ), and 41 at suggestive level ( p<5x10 -6 ), but no common SNPs were found between studies. Three SNPs (rs4284742 [G], rs11084095 [A], rs12461706 [T]) from three large studies were from the same gene region–SIGLEC5.

GWAS of periodontitis showed high heterogeneity of methodology used and provided limited SNPs statistics, making identifying reliable risk SNPs challenging. A clear guidance in dental research with requirement of expectation to make GWAS statistics available to other investigators are needed.

Citation: Gao C, Iles M, Larvin H, Bishop DT, Bunce D, Ide M, et al. (2024) Genome-wide association studies on periodontitis: A systematic review. PLoS ONE 19(9): e0306983. https://doi.org/10.1371/journal.pone.0306983

Editor: Gaetano Isola, University of Catania: Universita degli Studi di Catania, ITALY

Received: May 1, 2024; Accepted: June 26, 2024; Published: September 6, 2024

Copyright: © 2024 Gao et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the paper and its Supporting Information files.

Funding: “Chenyi Gao is supported by the external fund from the Alzheimer’s Society (Alzheimer’s Society Heather Corrie Ph.D. studentship: 546 (AS-PhD-19b-012)). Jianhua Wu and Harriet Larvin are supported by internal fund by Barts Charity (MGU0504). There was no additional external funding received for this study. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript”.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Periodontitis is a common disease affecting the tissues supporting and surrounding the teeth [ 1 ]. This disease is a major cause of tooth loss and is associated with a range of multiple long-term chronic conditions (e.g., diabetes [ 2 , 3 ], cardiovascular diseases [ 4 , 5 ], and cognitive impairments [ 6 – 9 ], which can negatively impact quality of life and increase the risk of mortality [ 10 ]. Because of the effect of periodontitis, periodontal treatment is critical to the patients and several recent periodontal treatment approaches has found more effectively reduce periodontitis [ 11 ] and also the inflammation mediator (C-reactive protein) that participated in both periodontitis and other systemic disease (i.e., cardiovascular disease) [ 12 ]. Despite of evolving periodontal treatments, how do we define the periodontitis is vitally important in either investigating the pathology but also in following treatment approaches. Although there have been consensus reports on universally accepted periodontitis definition such as for the 1999 classification of periodontal diseases [ 13 – 15 ]; and then later the 2018 AAP/EFP classification of periodontal and peri-implant diseases and conditions [ 16 ]), the actual definition of periodontitis employed in dental studies shows considerable heterogeneity.

Understanding of the pathology and aetiology of periodontitis is fundamental to improve the treatment approaches of periodontitis. Based on the current understanding, the pathology and aetiology of periodontitis is complex and multifactorial, including microorganism pathogens, environmental factors, lifestyle behaviours (such as nutrition, oral hygiene, and smoking) [ 2 ], epigenetic factors [ 17 ] and genetic factors [ 2 , 18 ]. By focusing on the genetic impact on periodontitis, a recent systematic review on heritability of periodontitis suggested 7%-38% across different study designs (e.g., twin study, other family study, or genome-wide association study (GWAS)) [ 19 ]. GWAS comprehensively investigates the association between a trait or diseases and hundreds of thousands of genetic variants (most commonly single nucleotide polymorphisms, SNPs) across the genome [ 20 ]; the technique is considered agnostic in terms of not depending on any aspect of the disease biology. GWAS as a technique aims to identify SNPs statistically associated with the trait or disease of interest, so called genetic risk variants or loci. The statistical technique involves comparing the allele frequency differences between cases (with the trait in question) and controls (persons without the trait of interest). The GWA approach has been applied to most common diseases since technology allowed the conduct of such studies in 2005 [ 21 ]. Since then, this GWA technique has been applied to oral diseases including periodontitis, contributing to finding more SNPs/genes associated with periodontitis, complementing genetic studies based on biological mechanisms proposed to influence the likelihood of periodontitis.

Current GWAS of periodontitis face many challenges, most notably limited sample size, population stratification, variation in methodologies applied, and use of non-consensus definitions of periodontitis, that complicate interpretation of the consistency of the results [ 20 ]. Even though there are some reviews [ 17 , 22 , 23 ] on the genetic aspects of periodontitis, there have been limited attempts to systematically evaluate GWAS studies of periodontitis. To date, there is only one systematic review on the heritability of periodontitis [ 19 ], and one descriptive review of periodontitis [ 23 ].

Since more GWAS on periodontitis have been published recently, a systematic evaluation and, ideally meta-analysis, of the available evidence would improve the understanding of the genetic mechanisms of periodontitis, address the current research gap, and contribute to better design and analysis of future GWAS. The aim of this systematic review on periodontitis is to: 1) critically appraise the evidence of GWAS of periodontitis. 2) Synthesise the findings and results by summarizing SNPs identified by high quality GWAS, and 3) meta-analyse appropriate studies/SNPs if summary statistics of GWAS were available.

Materials and methods

The study was to systematically review the GWAS that explored the genetic risk factors associated with periodontitis in the general population. This study is registered at the PROSPERO platform (ID: CRD42023456388). The main amendment in this manuscript compares to the protocol is that the meta-analysis was not performed due to appropriate data’s unavailability.

Search strategy

This systematic review was conducted according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement 2020 [ 24 ], as well as the guideline for performing systematic review for gene association studies [ 25 ]. The search was conducted on PubMed, the GWAS catalog, the ScienceDirect, the Embase, Global Health, Medline via Ovid to ensure the broad coverage of GWAS. The details search strategies in each database were summarised in S1-S4 Tables in S1 File . The search term including periodontitis (e.g., periodontal disease, periodontitis, periodon*) and GWAS terms (e.g., GWA, GWAS, genome wide association, whole genome association, WGA, WGAS). The “*” sign is to indicate any words start with “periodon”.

PEO : The P opulation of this study is human participants. The E xposure of this study is the genetic variants (SNPs). The O utcome of this study is ‘periodontitis’.

The search was restricted to the period 2005–2023 to limit the irrelevant genetic study, since GWAS study were not conducted prior to 2006. The search was completed on 31 Aug 2023.

Study selection

Strict eligibility criteria were developed by the four authors through detailed discussion to ensure relevant study inclusion:

• Study design is a genome-wide association study on human population including all ethnicities.
• Study disease/primary outcome/disease phenotype is of any form of periodontitis, including clinically diagnosed periodontitis with any diagnosis criteria, or self-reported. We do not specify the heritability, power, allele frequency at the stage of the search but will synthesize such information at the data extraction.

Exclusion criteria are as below:

• The study is a genetic/candidate gene study but not a whole genome wide scan for risk/protective SNPs of periodontitis.
• The study used oral pathogen or gingivitis as a proxy for periodontitis. Oral pathogens and gingivitis were not considered as periodontitis as they did not directly imply the presence of periodontitis.

Search results were downloaded and imported to Covidence (link: https://get.covidence.org/literature-review?campaignid=18165361407&adgroupid=138405766537&gclid=EAIaIQobChMIwfHnroSs_AIVTLDtCh10zAAEEAAYASAAEgJP_fD_BwE ) for study screening and eligibility assessment. First, duplication screening and removal is automatically done by COVIDENCE. Then, a three stage of eligibility assessment performed, and this consists of: initial abstract screening, full text screening and conflict resolve. Two authors (CG, JK) processed the abstract screening to narrow down the number of studies to be include in full text screening and eligibility assessment where all study might potentially include a GWA analysis on periodontitis is marked as “include” or “Maybe”. All “include” and “Maybe” marked study were screened for its full text to confirm its eligibility where GWAS on periodontitis or study include GWA analysis on periodontitis were marked as “include” only. Papers where the two authors had differing opinions regarding inclusion or exclusion were then screened by two more authors (HL, FS) and discussed by all four authors for the final decision. The eligibility of papers was confirmed before data extraction and quality assessment. A data extraction form was developed and based on the study [ 26 ]. The information of SNP location and chromosome, and the nearest gene was recorded from dbSNP ( https://www.ncbi.nlm.nih.gov/snp/ ) based on the Genome Reference Consortium (GRC) released "build 37" of the human genome (GRCh37) version to keep the consistency of the SNP location.

Data extraction

The steps for identifying genome-wide associated significant SNPs vary across studies, and one or two or three steps were applied among the included GWAS: 1) Conducting statistical test for association on one dataset only—discovery stage; 2) Conducting meta-analysis on (an)other dataset(s)—meta-analysis stage, optional; or 3) Further seeking an independent replication / validation using other dataset(s), optional. Sometimes step 2 and 3 can swop [ 20 ].

In this systematic review, no study was excluded by including replication or meta-analysis stage or not, but SNPs with p<5x10 -6 were extracted from the final stage of each identified study. SNPs with conventional GWAS significance ( p<5x10 -8 ) were also highlighted. For example, some studies contained discovery, validation/replication, and meta-analysis stage, so we extracted only the SNPs that were significant at the meta-analysis stage. If a study performed GWAS association analysis at discovery stage only, significant SNPs from discovery stage were extracted. The essential statistics and information were extracted on Excel sheets, including SNP ID, Chromosome and position, Odds Ratio or Coefficient Beta, Standard error or confidence interval, Nearest Gene, risk allele, ethnicity, sample size, quality control procedure, periodontitis definition etc. However, due to the unavailability of full list summary statistics from almost all studies (except Shungin et al. 2019), the further meta-analysis cannot be performed. Additional information (i.e, participants number, age, ethnicity, participants inclusion/exclusion criteria, data resource, type of periodontitis, clinical phenotype: measurements & definitions, study design: GWAS stage included, quality control during analysis, GWAS significant threshold, statistical model in GWAS) on methodology were also extracted.

Quality assessment

The Q-Genie tool, a validated tool developed by McMaster University for rating the quality of genetic association studies [ 27 , 28 ], for GWAS quality assessment was used. This tool includes 11 assessment areas including rationale, outcome classification, comparison groups, exposure, source of bias, power analysis, statistical methods used, test of assumptions and inferences, and conclusion, with each area scored max 5, making the total highest score 55. All papers were assessed by two authors in parallel, and any deviation between individual ratings was discussed and validated by a third author.

In addition to the Q-Genie tool, we also checked the quality control procedure in each included paper following the guidance of GWAS quality control (i.e., call rate, HWE, MAF, relatedness, population stratification, heterozygosity rate, sex mismatch) [ 20 , 29 , 30 ]. We have also extracted the genomic inflation, reference alleles and allele frequency from the 1000 Genomes allele frequency table from dbSNP for checking the inflation reported and comparing the reported alleles and allele frequency [ 31 ]. Additional information such as covariates controlled in the association analysis model and imputation quality check were also extracted from each study.

After the systematic search in GWAS catalog and PubMed, there were 16 papers from GWAS catalog, 202 papers from PubMed, 491 papers from EMBASE, Global Health and Medline Via Ovid, and 38 papers from ScienceDirect retrieved. After removal of duplicate studies and screening the abstract, 88 papers’ full text were assessed for eligibility. Of these, 15 papers were deemed eligible [ 32 – 46 ]. All included studies were published between 2010 and 2023. Fig 1 displays the PRISMA flow chart, and PRISMA 2020 checklist is provided as supplement (S7 Table in S1 File and S8 Table in S1 Checklist ).

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

https://doi.org/10.1371/journal.pone.0306983.g001

The Q-Genie tool was applied to assess the quality of each eligible study. Overall, the studies have satisfactory quality (scores ranged from 38–50, Table 1 ). However, the quality varies regarding the classification of the outcome, description of comparison groups, whether the study is adequately powered, statistical methods, description of the test and inferences (scores varied from 2 to 5 in each area among included GWAS). It is worth noting that many studies have insufficient sample size, and only five studies [ 34 , 37 , 39 , 44 , 45 ] each had a total sample over 10,000 ( Table 2 ).

https://doi.org/10.1371/journal.pone.0306983.t001

https://doi.org/10.1371/journal.pone.0306983.t002

For quality control procedure performed, all included studies have had or reported quality control steps taken but varied from 3 steps reported to 11 steps reported ( Table 3 ). There were 5 studies did not report the genomic inflation score or linkage disequilibrium score regression intercept for inflation check. The full details of the quality control step taken in each study can be viewed in S5 Table in S2 File . Most included SNPs have similar effect allele or minor allele frequency reported but few SNPs with mismatch allele as the 1000 Genomes project were also noted (e.g., rs4242220, rs12969041, rs2027756) and effect allele or minor allele from one study was not reported ( Table 4 ).

https://doi.org/10.1371/journal.pone.0306983.t003

https://doi.org/10.1371/journal.pone.0306983.t004

Data synthesis

Table 2 summarized every study and more detailed study characteristics can be viewed in the S6 Table in S2 File . The majority (n = 13) of studies included populations from Europe (German, Finish, Italian, Spanish, Dutch, Finnish, Turkish) and America (European American, American African, Caucasian, non -Hispanic Caucasian, Hispanic and Latino), with seven studies included mixed ethnicity population and six studies included only European or non-Hispanic Caucasian. There were also two studies included samples from East Asia only (Korean and Japanese [ 33 , 37 ].

Out of these 15 studies, there are six studies focused on the chronic periodontitis, two studies focused on aggressive periodontitis, one study focused on apical periodontitis and one focused on periodontal pocketing. The rest of 5 studies were interested in periodontitis regardless of periodontitis type or including multiple types of periodontitis. The definition and measurement of periodontitis were also varied with most of 15 studies employing full-mouth dental examination performed by trained examiner or dentists and some studies also including radiographs. Eight studies utilised or incorporate different versions of criteria from the Centres for Disease Control and Prevention and American Academy of Periodontology (CDC-AAP) definition [ 47 – 51 ] in which one study measured two sites per tooth [ 33 ], two studies measured four sites per tooth [ 32 , 46 ], two studies measured six sites per tooth [ 42 , 44 ] and three studies did not specify. Some studies also included radiographs such as x ray. The detailed study characteristic can be viewed in Table 2 .

The majority of studies analysed periodontitis as a binary phenotype (case/control), while five studies analysed periodontitis as a continuous variable or included linear regression as part of their analysis to investigate the risk SNPs for periodontitis related traits [ 32 , 38 , 44 – 46 ]. Two studies had no GWAS significant SNPs found nor the SNPs reaching our lowered suggestive significance threshold for SNP inclusion [ 43 , 46 ]. Therefore, these two studies were excluded in the next step of SNPs extraction in Table 3 . Except for Shungin et al. 2019, no studies have provided a full list of summary statistic (total sample size, number of cases, number of controls, odds ratios, 95% confidence interval or standard errors, p-value) but only reported top signal SNPs. It is also noted that some datasets were used in multiple studies (e.g. ARIC data was used in [ 42 , 44 , 45 ].

Table 3 reported all the top signal SNPs extracted from the remaining 13 studies with full details, where 11 risk SNPs (rs242016, rs242014, rs10491972, rs242002, rs2978951, rs2738058, rs4284742, rs729876, rs149133391, rs1537415, rs12461706) were associated with periodontitis at conventional GWAS significance level ( p<5x10 -8 ), plus 41 SNPs that reached the suggestive level of significance (p<5x10 -6 ). Although there are no common SNPs reported across study, three large-scale study reported three genome-wide significant SNPs (i.e., rs4284742 effect allele [G], rs11084095 [A], rs12461706 [T]) from the gene SIGLEC5 [ 34 , 39 , 45 ].

This systematic review has identified, critically evaluated, and synthesized genetics evidence from 15 publicly available GWAS studies on periodontitis, published between 2010 and 2023. The majority of studies had good quality, but 10 out of 15 were small studies with a sample size of less than 10,000. A total of 11 SNPs (rs242016, rs242014, rs10491972, rs242002, rs2978951, rs2738058, rs4284742, rs729876, rs149133391, rs1537415, rs12461706) at genome-wide conventional significant level ( p<5x10 -8 ) and 41 SNPs at suggestive level ( p<5x10 -6 ) were associated with the risk of having periodontitis. In which, three SNPs from three large studies (i.e., rs4284742 [G], rs11084095 [A], rs12461706 [T]) were reported in a same gene—SIGLEC5.

This systematic review has identified several risk variants associated with periodontitis from existing GWAS studies. However, there was huge heterogeneity among study designs and methodologies: sample sizes varied from hundreds to hundred thousand (with the proportion of cases varied from 7.2% to 73.4%, Table 2 ), ethnicity and population differences, especially, the periodontitis measurements and definitions which may have impact on the GWAS results. The periodontal measurements in the included studies varied from self-reported questionnaire, clinical examination to radiographs, full month examination to half mouth examination, and even studies using same criteria (e.g., CDC-AAP) employed measurements varied from on six sites per tooth to two sites per tooth. Although half mouth examination and fewer sites measurement could lead to more efficient measurement processes, risk of misclassification remains. By utilising questionnaire in measurements, self-reporting bias may also contribute to either underestimation or overestimation of the number of cases. For example, Shimizu et al. 2015 recruited and measured controls separately using self-reported health questionnaire, which may cause underestimation of cases from control participants and may contribute to reduced power to identify genome-wide significant SNPs. Although it is unclear to what extent these heterogeneities in periodontitis definition and measurements would lead to heterogeneity in the GWAS results across studies, future GWAS on periodontitis could benefit from more detailed measurements on periodontal conditions and use of standardised classification criteria. The 2018 periodontal status classification is advocated for use, and proposals have been made to further refine the use of current periodontitis classification to enhance epidemiological data collection and analysis [ 52 ]. In addition, more GWAS studies in different ethnic groups, with larger sample sizes and considering covariates are also important for observing the potential for ethnic differences on GWAS results or susceptibility of periodontitis and observe genome-wide significant SNPs.

Heterogeneity in the methods and results reporting was also noted. The current 15 studies incorporated multiple study designs such as inclusion of multiple GWA stages (e.g., 8 studies included discovery and replications). Meanwhile it has been noted that the replication stage had relatively small sample size than discovery stage with five studies included replication stage that have had more than 1,000 participants. However, insufficient sample size in the replication stage could lead to both type I and type II error in the replication results [ 53 ]. Meanwhile, it is also important to replicate the results from the combined analysis stage (meta-analysis of discovery and validation stage), which is also missing in the included studies. Future study could utilise better approaches to assess the reproducibility such as the meta-analysis model-based assessment [ 53 ]. In terms of results reporting, several included studies did not report the coefficient beta or odds ratio, the standard error or confidence interval, and effect allele of the reported SNPs. This missing information leads to difficulty during data synthesis. Genomic inflation scores were also missing in few studies and not all studies reported or not conducted all quality control steps selected from GWAS quality control guidance. A standardized guideline and consensus on GWAS reporting may be needed to uniform the GWAS report.

To date, there is no guideline in PRISMA on how to perform systematic review on GWAS studies, except Winkler 2014 suggested protocol for genome-wide association meta-analyses in terms of the quality control and conduction of such analyses, requiring the availability of full SNPs and associated statistics reported, allele frequencies, and population stratification [ 31 ]. Analytic tools like METAL might estimate the pooled effect for overlapping samples, but it needs genome-wide data to perform appropriate estimation. In addition, a standard and more detailed and widely acceptable quality assessment tool for GWAS is also needed. Although Q-genie tool has been used to assess validity and reliability here, it was designed for genetic association studies, not particularly for GWAS. An assessment tool specifically for GWAS with clear guidance on scoring would be beneficial not only for the assessors but also for the readers to better understand quality assessment criteria.

Form the included 15 studies, three studies based on populations of German, Dutch, European American, Turkish and Asian with sample size > 10,000 participants were commonly discovered three unique SNPs in the gene SIGLEC5, where the effect alleles of all three SNPs has been reported for their protective effect on periodontitis [ 34 , 39 , 45 ]. A recent study has investigated the three genome-wide significant SNPs in the region of SIGLEC5 and shown an impact on SIGLEC5 expression indicating that SIGLEC5 is indeed the target gene for the signal [ 54 ]. SIGLEC5 codes for sialic acid-binding Ig-like lectins as a transmembrane inhibitory receptor and is responsible for binding sialic acids and sialic acid-containing glycan ligands. It is expressed in cells in the innate immune system and plays a role in inflammation regulation, both in infection and wound healing [ 55 ]. They observed SNP rs11084095 at SIGLEC5 can influence ERG binding and enhancer activity [ 54 ], where ERG is important for endothelial homeostasis, including acute response to injury and repair of the endothelium [ 56 ]. Meanwhile, the SNP rs12461706 was found in complete linkage disequilibrium with rs11084095. In addition, SNP rs4284742 has been shown to affect MAFB binding affinity, with the common allele enhancing the binding affinity compared to the alternative allele [ 54 ]. MAFB is suggested to be associated with the activation of SIGLEC5 expression and contribute to early-onset periodontitis. Further investigation of SIGLEC5 in periodontitis pathologies and intervention targeting the biological pathway underpinned by SIGLEC5 may contributes to both aetiology understanding and disease treatment [ 57 ]. In addition to these three SNPs from the same gene region, the rest of the 8 SNPs may also contribute to the periodontitis, however, further GWAS replication with larger sample size may be needed.

The three SNPs in SIGLEC5, two of them were found from both aggressive and chronic periodontitis and one were found from mixed definition defined periodontitis (i.e., including both self-reported and also clinical definition defined periodontitis). This suggested that the SIGLEC5 and the three SNPs may play fundamental roles in all types of periodontitis instead of some specific periodontitis and investigation on SIGLEC 5 may contributes to all type of periodontitis treatment and common pathology understanding. According to the GWAS catalog, most of GWAS significant SNPs found on 15 studies were reported only for periodontitis, except rs2738058 was also found in kidney diseases in Chinese population (i.e., IGA glomerulonephritis) [ 58 ]. Meanwhile, several mapped genes of these GWAS significant SNPs were also found significant in IGA glomerulonephritis (e.g., DEFA9P and DEFA10P [ 58 ]), neuropsychological conditions (e.g., TMF1P1 [ 59 ]), despite of reporting uncommon SNP. These may suggest somewhat genetic similarity between periodontitis and these conditions but further investigation on their relationship with periodontitis still needed.

In comparison to the previous systematic review on the heritability of periodontitis [ 19 ], our focus lies more heavily on the methodology utilised in GWAS and synthesis of results. Moreover, comparing with the review article of genetics of periodontitis by Shaddox et al. 2021, we employed a systematic approach and included a greater number of studies than the prior review. It is noted that 6 out of 8 studies that used chronic periodontitis as disease phenotype did not meet the required sample size of >10K cases for such disease with low heritability, except the studies Munz et al. 2017 and Munz et al. 2019. For the 8 studies that used aggressive periodontitis with high heritability as disease definition, smaller sample size is usually acceptable but no studies showed any common risk variants, indicating a potential of false positive results.

The findings that NO common SNPs were consistently reported through all the included studies highlighted a significant level of heterogeneity in the results obtained from GWAS of periodontitis. Given this lack of repeatability in GWAS finding, any identified genetic variants must be interpretated with caution. Furthermore, we observed a reluctance within the dental research community to share GWAS results. Only one study (Shungin et al. 2019) provided a comprehensive list of SNPs statistics from their GWAS of periodontitis, while the remaining studies offered only a limited number of top-signaled SNPs, with some providing incomplete statistics such as lacking odds ratios or 95% confidence intervals. When we attempted to obtain full SNP statistics from these studies, they either declined or did not respond, underscoring a significant transparency issue in current dental research practices. In many common diseases within the medical field, guidelines exist mandating GWAS data sharing as a standard practice expected by funders and publishers. We call for similar guidelines to be established in dental research, requiring the sharing of GWAS statistics from published work to facilitate advancement within the field.

The current study has several strengths, such as summarizing existing risk variants in the literature and discussed the study design of current GWAS on periodontitis, and presenting each study clearly with its database used, population, and GWAS testing method used. However, there are also some limitations to consider. Publication bias should be kept in mind while interpreting the results, as only publicly available GWAS studies were included in the review [ 26 ]. Additionally, potential biases from the selected studies may exist, such as those from the periodontitis definition, and variation of methodology applied (e.g. number of covariates adjusted in the association analysis). The current study failed to obtain access to full GWAS summary statistics to perform meta-analysis which could contribute to resolve small sample size in many studies and detect risk variants in combined sample. For example, SNPs with consistently border-line association with periodontitis could have been missed, which may have in theory become statistically significant in meta-analysis if most studies had provided a full list of statistics for all SNPs. In addition, since the majority of included studies sampled were from white population (European, European American, Caucasian etc.) and only few of them included Latino, Black, Asian or mixed ethnicity, it is difficult to draw conclusion based on each ethnicity group. More studies investigating Black and Asian populations could help to understand the underlying ethnicity difference of periodontitis, and future GWAS should also provide summary statistics in repository such as GWAS catalog to facilitate further meta-analysis.

Conclusions

To conclude, our systematic review of 15 GWAS studies on periodontitis identified 11 SNPs were at genome-wide significance p<5x10 -8 level. Variants near or in the gene region SIGLEC5 were reported most frequently (i.e. in three large scale studies) for its potential role on periodontitis. These results imply potential therapeutic targets pathway underlined by the SIGLEC5. Further investigation on this gene could contributed to the periodontitis treatment approach design. However, the heterogeneity on study design, study sample size and target population between studies has been noted. To improve our understanding of periodontitis and support the development of effective treatment options, more high-quality and homogeneous methodology used in GWAS studies are needed. These studies should use standardized periodontitis definitions and assessment tools, have larger sample sizes, and include different ethnicities. Data repository of GWAS results should be made available so that further meta-analysis can be possible, especially in dental research, to ensure research transparency and reproducibility. These efforts will contribute to greater understanding of this oral disease and ultimately benefit public health.

Supporting information

S1 checklist. prisma checklist, s8 table..

https://doi.org/10.1371/journal.pone.0306983.s001

S1 File. S1-S4, S7 Tables.

https://doi.org/10.1371/journal.pone.0306983.s002

S2 File. S5 and S6 Tables.

https://doi.org/10.1371/journal.pone.0306983.s003

• View Article
• PubMed/NCBI
• Google Scholar

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Author Biographies
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

A systematic review of meta-analyses on the impact of formative assessment on k-12 students’ learning: toward sustainable quality education.

1. Introduction

2. materials and methods, 2.1. literature search strategy, 2.2. selection criteria, 2.3. data extraction, 2.4. methodological quality assessment, 2.5. robustness of the results-based recommendations, 2.6. data interpretation, 3.1. search results, 3.2. characteristics of the meta-analyses, 3.3. methodological quality assessment, 3.4. robustness of the results-based recommendations, 3.5. effect of formative assessment in key areas of learning, 3.5.1. effects of formative assessment on learning in steam-related subjects, 3.5.2. effect of formative assessment on learning literacy, 3.6. effect of computer-based formative assessment on learning, 3.7. effect on affective learning, 3.8. effect of formative assessment on learning within settings, 3.8.1. k-12 school students, 3.8.2. primary school students, 3.8.3. middle school students, 3.8.4. secondary school students, 3.9. other influencing variables, 3.9.1. differentiation, 3.9.2. professional development, 3.9.3. teacher and/or student-directed, 4. discussion, 4.1. generalizability of the results, 4.2. quality of the included meta-analyses, 4.3. formative assessment practices enhance learning, 4.4. type of intervention, 4.4.1. student centered, 4.4.2. student-directed formative assessment according to school setting, 4.4.3. student-directed formative assessment and subject area, 4.4.4. teacher-directed approaches, 4.5. effect of formative assessment within various contexts, 4.5.1. formative assessment in steam-related subjects, 4.5.2. formative assessment using computers, 4.5.3. formative assessment and literacy, 4.6. overall effect of formative assessment, 4.7. strengths and limitations, 5. conclusions, supplementary materials, author contributions, institutional review board statement, informed consent statement, data availability statement, acknowledgments, conflicts of interest.

• Lee, H.; Chung, H.; Zhang, Y.; Abedi, J.; Warschauer, M. The Effectiveness and Features of Formative Assessment in US K-12 Education: A Systematic Review. Appl. Meas. Educ. 2020 , 33 , 124–140. [ Google Scholar ] [ CrossRef ]
• McCallum, S.; Milner, M.M. The effectiveness of formative assessment: Student views and staff reflections. Assess. Eval. High. Educ. 2021 , 46 , 1–16. [ Google Scholar ] [ CrossRef ]
• Clark, I. Formative Assessment: Assessment Is for Self-regulated Learning. Educ. Psychol. Rev. 2012 , 24 , 205–249. [ Google Scholar ] [ CrossRef ]
• Dayal, H. How Teachers use Formative Assessment Strategies during Teaching: Evidence from the Classroom. Aust. J. Teach. Educ. 2021 , 46 , 1–21. [ Google Scholar ] [ CrossRef ]
• Näsström, G.; Andersson, C.; Granberg, C.; Palm, T.; Palmberg, B. Changes in Student Motivation and Teacher Decision Making When Implementing a Formative Assessment Practice. Front. Educ. 2021 , 6 , 616216. [ Google Scholar ] [ CrossRef ]
• Cowie, B.; Moreland, J. Leveraging disciplinary practices to support students’ active participation in formative assessment. Assess. Educ. Princ. Policy Pract. 2015 , 22 , 247–264. [ Google Scholar ] [ CrossRef ]
• Cowie, B.; Khoo, E. An Ecological Approach to Understanding Assessment for Learning in Support of Student Writing Achievement. Front. Educ. 2018 , 3 , 11. [ Google Scholar ] [ CrossRef ]
• Black, P.; Wiliam, D. Developing the theory of formative assessment. Educ. Assess. Eval. Account. 2009 , 21 , 5–31. [ Google Scholar ] [ CrossRef ]
• Xuan, Q.; Cheung, A.; Sun, D. The effectiveness of formative assessment for enhancing reading achievement in K-12 classrooms: A meta-analysis. Front. Psychol. 2022 , 13 , 990196. [ Google Scholar ] [ CrossRef ]
• Hornby, G.; Greaves, D. Essential Evidence-Based Teaching Strategies: Ensuring Optimal Academic Achievement for Students ; Springer: Berlin/Heidelberg, Germany, 2022. [ Google Scholar ]
• Andersson, C.; Palm, T. Characteristics of improved formative assessment practice. Educ. Inq. 2017 , 8 , 104–122. [ Google Scholar ] [ CrossRef ]
• Granberg, C.; Palm, T.; Palmberg, B. A case study of a formative assessment practice and the effects on students’ self-regulated learning. Stud. Educ. Eval. 2021 , 68 , 100955. [ Google Scholar ] [ CrossRef ]
• Ozan, C.; Kincal, R. The Effects of Formative Assessment on Academic Achievement, Attitudes toward the Lesson, and Self-Regulation Skills. Educ. Sci. Theory Pract. 2018 , 18 , 85–118. [ Google Scholar ] [ CrossRef ]
• Johnson, C.C.; Sondergeld, T.A.; Walton, J.B. A Study of the Implementation of Formative Assessment in Three Large Urban Districts. Am. Educ. Res. J. 2019 , 56 , 2408–2438. [ Google Scholar ] [ CrossRef ]
• Darling-Hammond, L.; Flook, L.; Cook-Harvey, C.; Barron, B.; Osher, D. Implications for educational practice of the science of learning and development. Appl. Dev. Sci. 2020 , 24 , 97–140. [ Google Scholar ] [ CrossRef ]
• Birenbaum, M.; DeLuca, C.; Earl, L.; Heritage, M.; Klenowski, V.; Looney, A.; Smith, K.; Timperley, H.; Volante, L.; Wyatt-Smith, C. International trends in the implementation of assessment for learning: Implications for policy and practice. Policy Futures Educ. 2015 , 13 , 117–140. [ Google Scholar ] [ CrossRef ]
• Clark, I.D. Formative Assessment: Policy, Perspectives and Practice. Fla. J. Educ. Adm. Policy 2011 , 4 , 158–180. [ Google Scholar ]
• Gordon, E.; McGill, M.; Sands, D.; Kalinich, K.; Pellegrino, J.; Chatterji, M. Bringing formative classroom assessment to schools and making it count. Qual. Assur. Educ. 2014 , 22 , 339–352. [ Google Scholar ] [ CrossRef ]
• Oo, C.Z.; Alonzo, D.; Asih, R.; Pelobillo, G.; Lim, R.; San, N.M.H.; O’Neill, S. Implementing school-based assessment reforms to enhance student learning: A systematic review. Educ. Assess. Eval. Account. 2023 , 36 , 7–30. [ Google Scholar ] [ CrossRef ]
• Yan, Z.; Li, Z.; Panadero, E.; Yang, M.; Yang, L.; Lao, H. A systematic review on factors influencing teachers’ intentions and implementations regarding formative assessment. Assess. Educ. Princ. Policy Pract. 2021 , 28 , 228–260. [ Google Scholar ] [ CrossRef ]
• Lutovac, S.; Flores, M.A. Conceptions of assessment in pre-service teachers’ narratives of students’ failure. Camb. J. Educ. 2022 , 52 , 55–71. [ Google Scholar ] [ CrossRef ]
• Li, Z.; Yan, Z.; Chan, K.K.Y.; Zhan, Y.; Guo, W.Y. The role of a professional development program in improving primary teachers’ formative assessment literacy. Teach. Dev. 2023 , 27 , 447–467. [ Google Scholar ] [ CrossRef ]
• Ahmedi, V. Teachers' Attitudes and Practices Towards Formative Assessment in Primary Schools. J. Soc. Stud. Educ. Res. 2019 , 10 , 161–175. [ Google Scholar ]
• DeLuca, C.; Chapman-Chin, A.; Klinger, D.A. Toward a Teacher Professional Learning Continuum in Assessment for Learning. Educ. Assess. 2019 , 24 , 267–285. [ Google Scholar ] [ CrossRef ]
• Yan, Z.; King, R.B. Assessment is contagious: The social contagion of formative assessment practices and self-efficacy among teachers. Assess. Educ. Princ. Policy Pract. 2023 , 30 , 130–150. [ Google Scholar ] [ CrossRef ]
• Schütze, B.; Rakoczy, K.; Klieme, E.; Besser, M.; Leiss, D. Training effects on teachers’ feedback practice: The mediating function of feedback knowledge and the moderating role of self-efficacy. ZDM 2017 , 49 , 475–489. [ Google Scholar ] [ CrossRef ]
• Lobos, K.; Bustos, C.; Saez-Delgado, F.; Cobo-Rendon, R.; Bruna, C. Promoting ASC in the primary education classroom: The role of teacher training. Int. J. Sch. Educ. Psychol. 2023 , 11 , 233–244. [ Google Scholar ] [ CrossRef ]
• Hebbecker, K.; Souvignier, E. Formatives Assessment im Leseunterricht der Grundschule—Implementation und Wirksamkeit eines modularen, materialgestützten Konzepts. Z. Für Erzieh. 2018 , 21 , 735–765. [ Google Scholar ] [ CrossRef ]
• Aust, L.; Schütze, B.; Hochweber, J.; Souvignier, E. Effects of formative assessment on intrinsic motivation in primary school mathematics instruction. Eur. J. Psychol. Educ. 2023 , 39 , 1–24. [ Google Scholar ] [ CrossRef ]
• Panadero, E.; Jonsson, A.; Pinedo, L.; Fernández-Castilla, B. Effects of Rubrics on Academic Performance, Self-Regulated Learning, and self-Efficacy: A Meta-analytic Review. Educ. Psychol. Rev. 2023 , 35 , 113. [ Google Scholar ] [ CrossRef ]
• Hebbecker, K.; Förster, N.; Forthmann, B.; Souvignier, E. Data-based decision-making in schools: Examining the process and effects of teacher support. J. Educ. Psychol. 2022 , 114 , 1695–1721. [ Google Scholar ] [ CrossRef ]
• Luo, W.; Lim, S.Q.W. Perceived formative assessment and student motivational beliefs and self-regulation strategies: A multilevel analysis. Educ. Psychol. 2024 , 44 , 1–19. [ Google Scholar ] [ CrossRef ]
• Andrade, H.L. A Critical Review of Research on Student Self-Assessment. Front. Educ. 2019 , 4 , 87. [ Google Scholar ] [ CrossRef ]
• Bennett, R.E. Formative assessment: A critical review. Assess. Educ. Princ. Policy Pract. 2011 , 18 , 5–25. [ Google Scholar ] [ CrossRef ]
• Double, K.S.; McGrane, J.A.; Hopfenbeck, T.N. The Impact of Peer Assessment on Academic Performance: A Meta-analysis of Control Group Studies. Educ. Psychol. Rev. 2020 , 32 , 481–509. [ Google Scholar ] [ CrossRef ]
• Bolat, Y.I.; Taş, N. A meta-analysis on the effect of gamified-assessment tools’ on academic achievement in formal educational settings. Educ. Inf. Technol. 2023 , 28 , 5011–5039. [ Google Scholar ] [ CrossRef ]
• McMillan, J.H.; Venable, J.C.; Varier, D. Studies of the effect of formative assessment on student achievement: So much more is needed. Pract. Assess. Res. Eval. 2013 , 18 , 1–15. [ Google Scholar ]
• Aromataris, E.; Fernandez, R.; Godfrey, C.M.; Holly, C.; Khalil, H.; Tungpunkom, P. Summarizing systematic reviews: Methodological development, conduct and reporting of an umbrella review approach. JBI Evid. Implement. 2015 , 13 , 132–140. [ Google Scholar ] [ CrossRef ]
• Fusar-Poli, P.; Radua, J. Ten simple rules for conducting umbrella reviews. BMJ Ment. Health 2018 , 21 , 95–100. [ Google Scholar ] [ CrossRef ]
• Page, M.J.; McKenzie, J.E.; Bossuyt, P.M.; Boutron, I.; Hoffmann, T.C.; Mulrow, C.D.; Shamseer, L.; Tetzlaff, J.M.; Akl, E.A.; Brennan, S.E.; et al. The PRISMA 2020 statement: An updated guideline for reporting systematic reviews. BMJ 2021 , 74 , 790–799. [ Google Scholar ] [ CrossRef ]
• Shea, B.J.; Reeves, B.C.; Wells, G.; Thuku, M.; Hamel, C.; Moran, J.; Moher, D.; Tugwell, P.; Welch, V.; Kristjansson, E.; et al. AMSTAR 2: A critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 2017 , 358 , j4008. [ Google Scholar ] [ CrossRef ]
• Hariton, E.; Locascio, J.J. Randomised controlled trials—the gold standard for effectiveness research. BJOG Int. J. Obstet. Gynaecol. 2018 , 125 , 1716. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Barbosa, A.; Whiting, S.; Simmonds, P.; Scotini Moreno, R.; Mendes, R.; Breda, J. Physical Activity and Academic Achievement: An Umbrella Review. Int. J. Environ. Res. Public Health 2020 , 17 , 5972. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• O'Brien, K.M.; Barnes, C.; Yoong, S.; Campbell, E.; Wyse, R.; Delaney, T.; Brown, A.; Stacey, F.; Davies, L.; Lorien, S.; et al. School-Based Nutrition Interventions in Children Aged 6 to 18 Years: An Umbrella Review of Systematic Reviews. Nutrients 2021 , 13 , 4113. [ Google Scholar ] [ CrossRef ]
• Johnson, B.T.; MacDonald, H.V.; Bruneau, M.L., Jr.; Goldsby, T.U.; Brown, J.C.; Huedo-Medina, T.B.; Pescatello, L.S. Methodological quality of meta-analyses on the blood pressure response to exercise: A review. J. Hypertens. 2014 , 32 , 706–723. [ Google Scholar ] [ CrossRef ]
• Monasta, L.; Batty, G.D.; Cattaneo, A.; Lutje, V.; Ronfani, L.; Van Lenthe, F.J.; Brug, J. Early-life determinants of overweight and obesity: A review of systematic reviews. Obes. Rev. 2010 , 11 , 695–708. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Grgic, J.; Grgic, I.; Pickering, C.; Schoenfeld, B.J.; Bishop, D.J.; Pedisic, Z. Wake up and smell the coffee: Caffeine supplementation and exercise performance-an umbrella review of 21 published meta-analyses. Br. J. Sports Med. 2020 , 54 , 681–688. [ Google Scholar ] [ CrossRef ]
• Guyatt, G.; Oxman, A.D.; Akl, E.A.; Kunz, R.; Vist, G.; Brozek, J.; Norris, S.; Falck-Ytter, Y.; Glasziou, P.; deBeer, H.; et al. GRADE guidelines: 1. Introduction—GRADE evidence profiles and summary of findings tables. J. Clin. Epidemiol. 2011 , 64 , 383–394. [ Google Scholar ] [ CrossRef ]
• Cohen, J. Statistical Power Analysis for the Behavioral Sciences ; Routledge: London, UK, 2013. [ Google Scholar ]
• Hazra, A. Using the confidence interval confidently. J. Thorac. Dis. 2017 , 9 , 4125–4130. [ Google Scholar ] [ CrossRef ]
• Graham, S.; Hebert, M.; Harris, K.R. A Meta-Analysis Formative Assessment and Writing. Elem. Sch. J. 2015 , 115 , 523–547. [ Google Scholar ] [ CrossRef ]
• Kingston, N.; Nash, B. Formative Assessment: A Meta-Analysis and a Call for Research. Educ. Meas. Issues Pract. 2011 , 30 , 28–37. [ Google Scholar ] [ CrossRef ]
• Akbay, T.; Sevim-Cirak, N.; Erol, O. Re-Examining the Effect of Audience Response Systems on Learning Outcomes: Evidence from the Last Decade. Int. J. Hum. Comput. Interact. 2023 , 40 , 1–15. [ Google Scholar ] [ CrossRef ]
• Guo, L. The effects of self-monitoring on strategy use and academic performance: A meta-analysis. Int. J. Educ. Res. 2022 , 112 , 101939. [ Google Scholar ] [ CrossRef ]
• Jurado-Castro, J.M.; Vargas-Molina, S.; Gómez-Urquiza, J.L.; Benítez-Porres, J. Effectiveness of real-time classroom interactive competition on academic performance: A systematic review and meta-analysis. PeerJ Comput. Sci. 2023 , 9 , e1310. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Karaman, P. The Effect of Formative Assessment Practices on Student Learning: A Meta- Analysis Study. Int. J. Assess. Tools Educ. 2021 , 8 , 801–817. [ Google Scholar ] [ CrossRef ]
• Li, R. Investigating effects of computer-mediated feedback on L2 vocabulary learning. Comput. Educ. 2023 , 198 , 104763. [ Google Scholar ] [ CrossRef ]
• Van der Kleij, F.M.; Feskens, R.C.W.; Eggen, T.J.H.M. Effects of Feedback in a Computer-Based Learning Environment on Students’ Learning Outcomes:A Meta-Analysis. Rev. Educ. Res. 2015 , 85 , 475–511. [ Google Scholar ] [ CrossRef ]
• Zheng, L. The effectiveness of self-regulated learning scaffolds on academic performance in computer-based learning environments: A meta-analysis. Asia Pac. Educ. Rev. 2016 , 17 , 187–202. [ Google Scholar ] [ CrossRef ]
• Lövdén, M.; Fratiglioni, L.; Glymour, M.M.; Lindenberger, U.; Tucker-Drob, E.M. Education and Cognitive Functioning Across the Life Span. Psychol. Sci. Public Interest 2020 , 21 , 6–41. [ Google Scholar ] [ CrossRef ]
• Andrade, H.; Wang, X.; Du, Y.; Akawi, R. Rubric-Referenced Self-Assessment and Self-Efficacy for Writing. J. Educ. Res. 2009 , 102 , 287–302. [ Google Scholar ] [ CrossRef ]
• Genlott, A.A.; Grönlund, Å. Closing the gaps—Improving literacy and mathematics by ict-enhanced collaboration. Comput. Educ. 2016 , 99 , 68–80. [ Google Scholar ] [ CrossRef ]
• Quintana, D.S. From pre-registration to publication: A non-technical primer for conducting a meta-analysis to synthesize correlational data. Front. Psychol. 2015 , 6 , 1549. [ Google Scholar ] [ CrossRef ]
• Kolaski, K.; Logan, L.R.; Ioannidis, J.P.A. Guidance to best tools and practices for systematic reviews. Syst. Rev. 2023 , 12 , 96. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Lamberg, T.; Gillette-Koyen, L.; Moss, D. Supporting Teachers to Use Formative Assessment for Adaptive Decision Making. Math. Teach. Educ. 2020 , 8 , 37–58. [ Google Scholar ] [ CrossRef ]
• Schildkamp, K.; van der Kleij, F.M.; Heitink, M.C.; Kippers, W.B.; Veldkamp, B.P. Formative assessment: A systematic review of critical teacher prerequisites for classroom practice. Int. J. Educ. Res. 2020 , 103 , 101602. [ Google Scholar ] [ CrossRef ]
• Reid, R.; Trout, A.L.; Schartz, M. Self-regulation interventions for children with attention deficit/hyperactivity disorder. Except. Child. 2005 , 71 , 361–377. [ Google Scholar ]
• Brookhart, S. Appropriate Criteria: Key to Effective Rubrics. Front. Educ. 2018 , 3 , 1–12. [ Google Scholar ] [ CrossRef ]
• Reddy, Y.M.; Andrade, H. A review of rubric use in higher education. Assess. Eval. High. Educ. 2010 , 35 , 435–448. [ Google Scholar ] [ CrossRef ]
• Dignath, C.; van Ewijk, R.; Perels, F.; Fabriz, S. Let Learners Monitor the Learning Content and Their Learning Behavior! A Meta-analysis on the Effectiveness of Tools to Foster Monitoring. Educ. Psychol. Rev. 2023 , 35 , 62. [ Google Scholar ] [ CrossRef ]
• Graham, S. Changing How Writing Is Taught. Rev. Res. Educ. 2019 , 43 , 277–303. [ Google Scholar ] [ CrossRef ]
• Saravanan, A.; Palanisamy, L.; Aziz, A.A. Systematic review: Challenges in teaching writing skills for upper secondary in ESL classrooms and suggestions to overcome them. Malays. J. Soc. Sci. Humanit. 2021 , 6 , 262–275. [ Google Scholar ]
• Troia, G.A.; Olinghouse, N.G.; Mo, Y.; Hawkins, L.; Kopke, R.A.; Chen, A.; Wilson, J.; Stewart, K.A. Academic standards for writing: To what degree do standards signpost evidence-based instructional practices and interventions? Elem. Sch. J. 2015 , 116 , 291–321. [ Google Scholar ] [ CrossRef ]
• Coker, D.L.; Farley-Ripple, E.; Jackson, A.F.; Wen, H.; MacArthur, C.A.; Jennings, A.S. Writing instruction in first grade: An observational study. Read. Writ. 2016 , 29 , 793–832. [ Google Scholar ] [ CrossRef ]
• Graham, S.; Alves, R.A. Research and teaching writing. Read. Writ. 2021 , 34 , 1613–1621. [ Google Scholar ] [ CrossRef ]
• Zainuddin, Z.; Chu, S.K.W.; Shujahat, M.; Perera, C.J. The impact of gamification on learning and instruction: A systematic review of empirical evidence. Educ. Res. Rev. 2020 , 30 , 100326. [ Google Scholar ] [ CrossRef ]
• Wang, A.I. The wear out effect of a game-based student response system. Comput. Educ. 2015 , 82 , 217–227. [ Google Scholar ] [ CrossRef ]
• Radosevich, D.; Salomon, R.; Radosevich, D.; Kahn, P. Using Student Response Systems to Increase Motivation, Learning, and Knowledge Retention. Innov. J. Online Educ. 2008 , 5 , 7. [ Google Scholar ]
• Licorish, S.A.; Owen, H.E.; Daniel, B.; George, J.L. Students’ perception of Kahoot!’s influence on teaching and learning. Res. Pract. Technol. Enhanc. Learn. 2018 , 13 , 9. [ Google Scholar ] [ CrossRef ]
• Cowan, N. The Magical Mystery Four: How is Working Memory Capacity Limited, and Why? Curr. Dir. Psychol. Sci. 2010 , 19 , 51–57. [ Google Scholar ] [ CrossRef ]
• Chamizo-Nieto, M.T.; Arrivillaga, C.; Rey, L.; Extremera, N. The Role of Emotional Intelligence, the Teacher-Student Relationship, and Flourishing on Academic Performance in Adolescents: A Moderated Mediation Study. Front. Psychol. 2021 , 12 , 695067. [ Google Scholar ] [ CrossRef ]
• Smiderle, R.; Rigo, S.J.; Marques, L.B.; Peçanha de Miranda Coelho, J.A.; Jaques, P.A. The impact of gamification on students’ learning, engagement and behavior based on their personality traits. Smart Learn. Environ. 2020 , 7 , 3. [ Google Scholar ] [ CrossRef ]
• Bicen, H.; Kocakoyun, Ş. Perceptions of Students for Gamification Approach: Kahoot as a Case Study. Int. J. Emerg. Technol. Learn. 2018 , 13 , 72. [ Google Scholar ] [ CrossRef ]
• Tomasik, M.J.; Berger, S.; Moser, U. On the Development of a Computer-Based Tool for Formative Student Assessment: Epistemological, Methodological, and Practical Issues. Front. Psychol. 2018 , 9 , 2245. [ Google Scholar ] [ CrossRef ]
• van der Steen, J.; van Schilt-Mol, T.; van der Vleuten, C.; Joosten-ten Brinke, D. Designing Formative Assessment That Improves Teaching and Learning: What Can Be Learned from the Design Stories of Experienced Teachers? J. Form. Des. Learn. 2023 , 7 , 182–194. [ Google Scholar ] [ CrossRef ]
• Topping, K.J. Peer Assessment. Theory Into Pract. 2009 , 48 , 20–27. [ Google Scholar ] [ CrossRef ]
• Ismail, S.M.; Rahul, D.R.; Patra, I.; Rezvani, E. Formative vs. summative assessment: Impacts on academic motivation, attitude toward learning, test anxiety, and self-regulation skill. Lang. Test. Asia 2022 , 12 , 40. [ Google Scholar ] [ CrossRef ]
• Choi, G.J.; Kang, H. Introduction to Umbrella Reviews as a Useful Evidence-Based Practice. J. Lipid Atheroscler. 2023 , 12 , 3–11. [ Google Scholar ] [ CrossRef ]
• Lazaros, B.; Vanesa, B.; John, P.A.I. Conducting umbrella reviews. BMJ Med. 2022 , 1 , e000071. [ Google Scholar ] [ CrossRef ]
• Peugh, J.L. A practical guide to multilevel modeling. J. Sch. Psychol. 2010 , 48 , 85–112. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Sortwell, A.; Trimble, K.; Ferraz, R.; Geelan, D.R.; Hine, G.; Ramirez-Campillo, R.; Carter-Thuiller, B.; Gkintoni, E.; Xuan, Q. A Systematic Review of Meta-Analyses on the Impact of Formative Assessment on K-12 Students’ Learning: Toward Sustainable Quality Education. Sustainability 2024 , 16 , 7826. https://doi.org/10.3390/su16177826

Sortwell A, Trimble K, Ferraz R, Geelan DR, Hine G, Ramirez-Campillo R, Carter-Thuiller B, Gkintoni E, Xuan Q. A Systematic Review of Meta-Analyses on the Impact of Formative Assessment on K-12 Students’ Learning: Toward Sustainable Quality Education. Sustainability . 2024; 16(17):7826. https://doi.org/10.3390/su16177826

Sortwell, Andrew, Kevin Trimble, Ricardo Ferraz, David R. Geelan, Gregory Hine, Rodrigo Ramirez-Campillo, Bastian Carter-Thuiller, Evgenia Gkintoni, and Qianying Xuan. 2024. "A Systematic Review of Meta-Analyses on the Impact of Formative Assessment on K-12 Students’ Learning: Toward Sustainable Quality Education" Sustainability 16, no. 17: 7826. https://doi.org/10.3390/su16177826

Article Metrics

Article access statistics, supplementary material.

ZIP-Document (ZIP, 509 KiB)

Further Information

Mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

• Open access
• Published: 04 September 2024

Emergency pediatric patients and use of the pediatric assessment triangle tool (PAT): a scoping review

• Tore A. G. Tørisen 1 ,
• Julie M. Glanville 2 ,
• Andres F. Loaiza 3 , 4 &
• Julia Bidonde 5  

BMC Emergency Medicine volume  24 , Article number:  158 ( 2024 ) Cite this article

1 Altmetric

Metrics details

We conducted a scoping review of the evidence for the use of the Pediatric Assessment Triangle (PAT) tool in emergency pediatric patients, in hospital and prehospital settings. We focused on the psychometric properties of the PAT, the reported impact, the setting and circumstances for tool implementation in clinical practice, and the evidence on teaching the PAT.

We followed the Joanna Briggs Institute methodology for scoping reviews and registered the review protocol. We searched MEDLINE, PubMed Central, the Cochrane Library, Epistemonikos, Scopus, CINAHL, Grey literature report, Lens.org, and the web pages of selected emergency pediatrics organizations in August 2022. Two reviewers independently screened and extracted data from eligible articles.

Fifty-five publications were included. The evidence suggests that the PAT is a valid tool for prioritizing emergency pediatric patients, guiding the selection of interventions to be undertaken, and determining the level of care needed for the patient in both hospital and prehospital settings. The PAT is reported to be fast, practical, and useful potentially impacting overcrowded and understaff emergency services. Results highlighted the importance of instruction prior using the tool. The PAT is included in several curricula and textbooks about emergency pediatric care.

Conclusions

This scoping review suggests there is a growing volume of evidence on the use of the PAT to assess pediatric emergency patients, some of which might be amenable to a systematic review. Our review identified research gaps that may guide the planning of future research projects. Further research is warranted on the psychometric properties of the PAT to provide evidence on the tool’s quality and usefulness. The simplicity and accuracy of the tool should be considered in addressing the current healthcare shortages and overcrowding in emergency services.

Review registration: Open Science Framework; 2022. https://osf.io/vkd5h/

Peer Review reports

Emergency medical services (EMS) are crucial to emergency care systems providing effective emergency medical care to people in need [ 1 ]. The World Health Organization (WHO) Emergency Care System Framework [ 2 ] (see Additional file 1) notes that effective emergency care involves a coordinated and integrated system of care, including the provision of prehospital care, transportation, and emergency department (ED) services. The WHO framework emphasizes the importance of early recognition of health issues and the timely provision of appropriate interventions to reduce morbidity and decrease the incidence of death and illness. Pediatric emergencies, particularly acute injuries and illnesses, generate considerable numbers of ambulance calls and ED visits in developed countries [ 3 , 4 ].

There is a general understanding that lack of pediatric emergency flow (or crowding) may lead to adverse outcomes for the child. However, the prevalence of pediatric emergencies poses significant challenges to emergency healthcare providers [ 5 , 6 ]. In the UK, pediatric emergencies represent 5–10% of all emergencies [ 7 ] and in the USA, children represent 20% of ED patients [ 8 ]. Injuries are the leading cause of morbidity and mortality among children and adolescents [ 9 , 10 ].

Caring for critically ill or injured pediatric patients can be challenging for emergency healthcare workers (EHWs) [ 11 ]. Patients’ histories may be difficult to obtain if the patient cannot provide verbal information or has been found alone without a caregiver [ 12 ]. Taking vital signs can be difficult and may not provide accurate information due to normal age-based variations [ 12 ]. Furthermore, some EHWs may have not received training in pediatric emergencies, which can be stressful [ 13 ].

Despite these challenges, EHWs need to conduct a rapid and accurate assessment of the pediatric patient to deliver timely effective emergency treatment. EHWs also need to reassure patients and caregivers and bring order to potentially chaotic situations. EHWs who lack specialized training in pediatric emergencies may unintentionally exacerbate stressful situations [ 13 ]. Emergency pediatric training for healthcare professionals inside and outside of the hospital is essential to ensure the best outcomes for critically ill or injured pediatric patients [ 14 , 15 ].

Emergency triage involves quickly identifying patients who require medical attention to prioritize treatment efficiently for those in greatest need [ 14 ]. Triage tools such as the Manchester Triage System and the Emergency Severity Index are helpful [ 16 ]. The Paediatric Canadian Triage and Acuity Scale (PaedCTAS) was developed specifically for pediatric patients [ 17 ], using the Pediatric Assessment Triage (PAT) tool as the first step in assessing emergency patients. It includes the “general impression” stage using the PAT, primary assessment with the airway, breathing, circulation, disability, and exposure (ABCDE) approach [ 18 ], secondary assessment, diagnostic assessment, and reassessment.

The Pediatric Assessment Triangle (PAT)

The PAT is used to quickly identify critically ill or injured children needing immediate medical attention. It focuses on three presenting components (“arms”): appearance, work of breathing, and circulation (Fig. 1 ). It can be used in prehospital or hospital settings for efficient rapid assessment of the patient's level of consciousness, breathing, and circulation, without requiring hands-on assessment or equipment [ 5 , 19 ]. It can help identify key pathophysiological problems and whether urgent transport or resources are needed. The PAT assessment takes 30–60 s [ 5 , 19 ] and it can be performed remotely (a “through the room” assessment).

The Pediatric Assessment Triangle components (arms). Figure adapted from Fuchs S and McEvoy M [ 20 ]]

Scoping review aim and design

Give the current shortage of healthcare personnel worldwide, and overcrowding of emergency departments, gathering of the PAT’s evidence is essential. This review aimed to identify the available scientific evidence about the PAT and its use by EMS. Our objective was to complete a scoping review within the pre-and-hospital care to synthesize:

What are the psychometric properties of the PAT (e.g., validity, reliability, applicability)?

What are the reported impact(s) of the PAT? (e.g., improved triage, cost, better clinical outcomes)

What are the requirements or circumstances for PAT implementation in clinical practice?

What is the evidence on the value of teaching EMS workers about PAT?

We followed the Joanna Briggs Institute framework for scoping reviews [ 21 ]. The review protocol was registered [ 22 ]. The review is reported according to the PRISMA extension for scoping reviews [ 23 ] (Additional file 2).

Eligibility criteria

Eligible publications (Table  1 ) reported the use of the PAT with pediatric populations in prehospital, hospital or training settings. Eligible outcomes matched our specific aims as follows: 1) psychometric performance, 2) impact(s), 3) implementation of PAT utilization, and 4) evidence on teaching the PAT.

We searched MEDLINE (PubMed), PubMed Central (via LitSense), the Cochrane Library, Epistemonikos, Scopus and CINAHL, from 1995 to July 2022, to include publications before the introduction of the PAT in the curricula of Pediatric Education for Prehospital Professionals (PEPP) and Advanced Pediatric Life Support (APLS) in 2000 [ 24 ]. The database searches were conducted from 24 to 28 July 2022. Fourteen websites of organizations involved in policy making in emergency pediatrics were searched between 6 and 10 August 2022. We searched for unpublished (grey) literature using Grey Literature Report ( http://www.greylit.org/ ) and Lens.org ( https://www.lens.org/ ). Full searches are presented in additional file 3.

Study selection process

We deduplicated records in EndNote and conducted double independent screening (TT, AFL-B) in Covidence (Veritas) against the eligibility criteria (Table  1 ). Conflicts were resolved by consensus or arbitrated by a third reviewer (JB). Additional file 4 lists records excluded at full text with reasons. Records reporting the same study were grouped and we cite the earliest publication while presenting relevant data from any of the related publications.

Data collection process

Data were extracted from eligible studies into a Microsoft 365 Excel form which was piloted on a random sample of five included studies, and modified as required based on feedback from the team [ 22 ]. One reviewer (TT) completed data extraction and a second reviewer (AFL-B) verified the extracted data. Disagreements were resolved by consensus or arbitrated by a third reviewer (JB). Risk of bias was not assessed [ 21 ].

Knowledge user (KU)/patient engagement and methodological appraisal

We defined KU/ patient engagement as individuals who may be affected by the research findings. Since this review was time sensitive, we did not recruit knowledge users or patients.

We did not appraise methodological quality or risk of bias of the included articles, which is consistent with guidance on scoping review conduct.

The synthesis included quantitative (e.g. psychometric properties) and qualitative analyses (e.g. content analysis) of the components of the impact, implementation and teaching. A word cloud was drawn for the impact of the PAT using the online program WordClouds. The team members identified, coded, and charted relevant units of text from the articles using a framework established a priori as a guide. The framework was developed through team discussions upon reviewing the preliminary results. Data were grouped by question and overviews are provided using charts and tables generated using Microsoft 365 Excel.

Search results and publication characteristics

The searches identified 548 records (Fig. 2 ). Fifty-five publications were included (full citations listed in Additional file 5) of which three were books. Sixteen publications were in non-English languages, but with English abstracts, and of these we retrieved 14 full text publications (Spanish ( n  = 9), German ( n  = 2), French ( n  = 1), Turkish ( n  = 1), and assumed Taiwanese Mandarin ( n  = 1)). Of these, there were seven papers that described the psychometric properties of the PAT, 18 were about the PAT’s impact, 38 described implementation pros and cons, and 30 provided references to the PAT used in educational/training environments. The publication dates ranged from 1999 to 2022, representing 18 countries with the majority classified as "high income" (World Bank classification) [ 25 ] (see Additional file 6). Study designs were diverse: primary research ( n  = 27, 49.1%), secondary research ( n  = 4, 7.3%), and "other" ( n  = 24, 43.6%). We identified no randomized controlled trials, systematic reviews, or scoping reviews.

PRISMA flow chart

Psychometric properties

The seven papers reporting psychometric properties were as follows. Four studies (Table 2 ) reported sensitivity and specificity, measuring test accuracy [ 26 , 27 , 28 , 29 ], of which one study reported an area under the receiver operating characteristic curve (AUROCC) [ 29 ] and four studies reported likelihood ratios (LR) [ 26 , 27 , 28 , 30 ].

PAT sensitivity (Fig. 3 ) ranged from 77.4% to 97.3% (four studies) suggesting it can accurately identify a large proportion of patients with the targeted condition [ 26 , 27 , 28 , 29 ]. Specificity, measuring a test's ability to correctly identify patients without the condition, ranged from 22.9% to 99.15% (four studies) [ 26 , 27 , 28 , 29 ].

PAT sensitivity and specificity

One study evaluated the PAT’s validity and reliability [ 31 ] by collecting data for 157 patients triaged by a single trained observer and an “enfermera clasificadora” (classifying nurse). This single pair showed high inter-observer agreement in applying the PAT and no errors associated with polypnea, pre-existing pallor, or irritability.

Likelihood ratios (LR) measure a test’s diagnostic accuracy which are less likely to change with the prevalence of a disorder. A positive LR (LR +) indicates a positive test result is more likely in people with the condition and a negative LR (LR-) indicates that a negative test result is more likely in people without the condition of interest. One study reported LR + of 5.2 (95% CI 5–7.8) [ 26 ] with a statistically significant high odds ratio (OR 111, 95% CI 73–168.6; p  < 0.001), indicating the PAT has a high ability to correctly identify and classify initial severity of disease during triage. A second study reported a LR + of 7.7 (95% CI 5.9–9.1) [ 27 ]. A third study triaged 1002 children using the PAT, reporting a LR + of 0.12 (95% CI 0.06–0.25) for children deemed stable by the PAT ( n  = 200) [ 28 ]. This study’s results for categories of pathophysiology (respiratory distress, respiratory failure, shock, central nervous system/metabolic disorder, and cardiopulmonary failure) highlighted the need to consider the clinical scenario when interpreting the PAT in EMS. However, the moderate LR- value (0.22, 95% CI 0.18–0.26) indicated that the test is less able to correctly identify children who do not need urgent care. The study reported a LR- of 0.12 (95% CI 0.06–0.25) for children found to be stable by the PAT ( n  = 802) [ 28 ]. The LR- values for children with the five specified categories of pathophysiology suggest the PAT has relatively low LR for identifying respiratory distress and shock, indicating it is better at ruling out those conditions. However, the relatively high LR- for respiratory failure and cardiopulmonary failure suggests the PAT is less effective at ruling out those conditions.

One study (2017) found that abnormal PAT results were associated with an increased risk of admission to the hospital (OR 5.14, 95% CI 4.98–5.32; p  < 0.01) [ 30 ]. Abnormal appearance (OR 3.99, 95% CI 3.63–4.38) or having one or more components of the PAT (OR 14.99, 95% CI 11.99–18.74) were significantly associated with hospital admission [ 30 ]. The study identified adjusted age (OR 4.44, 95% CI 3.77–5.24; p  < 0.001) and triage (OR 1.78, 95% CI 1.72–1.84; p  < 0.001) as independent risk factors for intensive care unit admission and longer stays in the pediatric ED [ 30 ]. One study reported the PAT performed similarly to the Pediatric Early Warning Score (PEWS) (AUROCC 0.963 (PAT) and 0.966 (PEWS); x 2  = 0.10; p  = 0.74) [ 29 ].

Four studies reported high levels of reliability in PAT results [ 27 , 28 , 29 , 32 ]. One study reported 93.6% reliability (Kappa index 0.7, 95% CI 0.5–0.8) [ 29 ]. A second study found paramedics used the PAT highly consistently across its three arms (Kappa 0.93, 95% CI 0.91–0.95) [ 32 ] and the paramedics’ impression, completed using PAT on first contact with the patient, showed substantial agreement with the investigators’ retrospective chart review on diagnosis and disposition (Kappa 0.62, 95% CI 0.57–0.66) and categorization of stable versus unstable (Kappa 0.66, 95% CI 0.62–0.71). A third study reported substantial inter-rater reliability agreement on PAT scores ( n  = 1002, two pediatric emergency physicians and a pediatric nurse practitioner) (Fleiss' κ 0.7, p  < 0.001) [ 28 ]. A fourth study reported an agreement rate of 93.24% between the PAT and the condition of sick children [ 29 ].

Reported impacts of the PAT

Eighteen publications reported on impacts after PAT implementation; the word cloud of impact names is display in Fig.  4 . Terms most used were “triage –communication -vocabulary and care”.

The PAT reported impact

Impact reported were on mortality, safety, effectiveness of care, timeliness of care, triage, and communication [ 27 , 28 , 29 , 30 , 31 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 ]. Three studies showed the ability of the PAT to correctly assess critical cases (e.g. higher risk of mortality in patients with sepsis with an altered or unstable PAT) [ 33 , 34 , 36 ]. Two studies found that PAT helped to avoid unnecessary interventions or potential harm to patients [ 27 , 35 ]. One study reported that a normal PAT result did not exclude severe infections, and a proper examination was still necessary to diagnose emergency pediatric patients [ 33 ]. One study reported that the PAT was timely and rapid to apply (mean 32.4 s) [ 31 ] and two studies reported that the PAT was equally effective, but faster and easier to use, than the PEWS in predicting critical illness in pediatric patients [ 29 , 38 ].

Communication and documentation were another way the PAT’s impact were reported. The PAT’s “general impression” aided in care communication and helped prioritize management options. The specific vocabulary to describe a patient’s vital signs and physical findings allowed for easy documentation and transfer/flow of information between EHWs [ 27 , 28 , 37 ]. Two studies highlighted the power of a common vocabulary in EMS replacing subjective comments with specific assessments [ 27 , 28 ].

Studies offered insights into achieving optimal triage outcomes using the PAT. One study demonstrated the PAT’s usefulness when classifying non-urgent patients [ 40 ] and a second noted the importance of setting severity and prioritization criteria (1 to 5 depending on severity) and using the PAT to ensure proper attention [ 45 ].

Abnormal PAT findings helped to identify patients with a higher risk of hospitalization [ 30 ] and enabled earlier interventions for high-risk patients [ 42 ]. One study used the PAT for children experiencing secondary complications to hematopoietic cell transplantation [ 44 ] and reported that an unstable PAT, along with other factors, accurately predicted the need for admission (relative risk 3.4, 95% CI 2.6–4.6; p  < 0.001). A study investigated features of 17,243 cases referred from in-hospital areas to the pediatric ED (median age 42 months (range: 0–120)); 65% of transferred patients were PAT-assessed as stable [ 41 ]. One study assessed the PAT as a discriminator in the triage classification system and assessed the correlation between pathophysiological diagnosis and triage classification [ 31 ]. Four studies suggested the PAT was considered practical and helpful in identifying emergency pediatric patients in need of intervention and identifying the probable underlying cause of illness [ 26 , 28 , 38 , 46 ]. Treatment priorities were met in children with fever, and to a lesser extent for pain, respiratory distress, and oxygen needs.

One study concluded that an abnormal PAT and a more severe triage level (I-III) were independent factors in identifying asthmatic children requiring hospitalization and longer stays [ 43 ]. One study suggested that the PAT did not perform well for patients with anaphylaxis and as a result patients did not receive timely interventions [ 39 ].

We found no data for impacts on pediatric readmission, patient/caregiver experience, or provider burnout.

Setting and circumstances for PAT implementation

Ten studies evaluated pre-hospital triage using the PAT [ 6 , 20 , 27 , 30 , 38 , 47 , 48 , 49 , 50 ] and 28 evaluated hospital triage [ 24 , 26 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 , 36 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 ]. No studies reported PAT use in emergency call centers or telemedicine services. One study noted that the PAT may be implemented by midwives working in hospitals or prehospital settings [ 37 ]. A study of 391 admissions reported PAT was considered a useful triage tool in resource-poor hospitals [ 52 ].

Four studies recommended formal training on using the PAT as necessary for effective use [ 27 , 28 , 45 , 47 ]. One study ( n  = not reported) found that a low utilization rate for the PAT (patient report forms collected over a three-month period) following its introduction increased significantly following training in PAT use (12% vs 63.3%) [ 47 ]. After implementation, one study reported that the 30 emergency nurses involved preferred using the PAT over the PEWS when assessing emergency pediatric patients [ 29 ]. In a study of the Advanced Pediatric Life Support (APLS) course, attendees considered the systematic assessment approach incorporating the PAT crucial to their clinical practice, highlighting the importance of training prior implementation [ 54 ]. Studies acknowledged that applying the PAT with young infants (7–89 days old) was challenging [ 33 ], implementing the PAT requires skills, on-site senior emergency pediatric care providers, and a pediatric-friendly environment [ 59 ] and that the feasibility of the PAT is promising, but further research for “clinical validation” (not further defined) was needed [ 30 ].

We found no information about the implementation of PAT in clinical guidelines, requirements for recertification after PAT implementation, cost of implementation, or sustainability.

Teaching the PAT

Thirty studies presented data on teaching PAT to EHWs as follows: an early report suggested that the PAT was ideal for pediatric life support courses in all settings, based on its simplicity and reproducibility for both teachers and clinicians [ 60 ]. The PAT is included in one textbook of general emergency pediatrics [ 61 ] and two textbooks for emergency pediatric care in the prehospital environment [ 20 , 62 ]. Courses for EHWs on pediatric life support have incorporated the PAT for the “first impression” assessment, as well as training on the use of the PAT tool itself [ 29 , 30 , 63 ].

Methods for teaching the PAT tool included classroom-based, use of simulation, use of virtual reality and video for case training [ 54 , 64 , 65 ]. The PAT has been recommended as a teaching tool for the goal-directed management of shock in children [ 66 ].

The number of people who have received PAT training is unknown, but more than 170,000 EHWs had received formal training up to 2010 (worldwide) [ 63 ]. The numbers of EHWs trained in the studies ranged from 30 to 1520 [ 29 , 54 ].

Eighteen studies reported the care of emergency pediatric patients and provide insights into best practices for care which can, in turn, inform educational programs or be used to develop evidence-based protocols [ 30 , 37 , 48 , 49 , 50 , 56 , 57 , 59 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 ]. Four publications describe how emergency care providers use the PAT to assess emergency pediatric patients generally or with specific medical problems [ 30 , 49 , 59 , 67 ].

We identified 55 documents reporting the use of the PAT in hospital and pre-hospital emergency pediatric care. Research indicates that the PAT is a valid and reliable tool for evaluating emergency pediatric patients, prioritizing interventions, and determining the appropriate level of care. EHWs found the PAT is fast and practical, akin to the intuitive ‘gut feeling’ of experienced clinicians., but they should complete formal training before implementing the PAT. Several emergency pediatric care course curricula and key textbooks include the PAT.

We found only seven publications on the PAT’s psychometric properties, which suggest that the PAT has good sensitivity and some variability in specificity. The low research volume may reflect ethical challenges around research involving children, the unique and unpredictable nature of emergency situations, the impossibility of controlling all variables and difficulties in obtaining funding [ 77 ]. Research on psychometric properties can be expensive and funding for pediatric-focused psychometric research may not be a priority for research funders. The PAT’s ease of use may have contributed to its rapid adoption in practice before adequate psychometric testing was conducted and published. Implementing the PAT may still be challenging in terms of training or resistance to change [ 47 ]. Despite the challenges of research in the emergency setting, a third of the included studies reported positive impacts when using the PAT, suggesting its potential for triaging and improving patient outcomes in clinical settings which merits further investigation in an era of emergency department overcrowding and shortages of healthcare personnel.

Other tools are also used for emergency pediatric assessment (e.g., the Pediatric Glasgow Coma Scale, the PEWS, and the Pediatric Vital Sign Score) and each has its strengths and limitations. Choosing a tool depends on the specific circumstances and the healthcare provider's expertise. Based on the included comparative studies, the PAT is often favored for its simplicity, rapidity, and ease of use in remote or face-to-face emergency settings, since it does not require hands-on assessment or the use of specialized equipment. The available research and comparative studies merit further investigation.

Evidence was identified on training EHWs to use the PAT to assess accurately a child's appearance, work of breathing, and circulation. Proficiency is needed in using the tool and there is a need to use it regularly, to maintain their knowledge. While the PAT can provide a quick snapshot of a child's overall condition, it is only one part of a comprehensive assessment, and EHWs should use additional tools and techniques to assess a child's condition. Online courses, in-person workshops, and continuing education courses offered by professional organizations as well as guides or manuals with step-by-step instructions on how to use the PAT are all available. Healthcare providers who are considering preparing or updating their PAT training, perhaps using simulation-based approaches, should review these sources of evidence-based training [ 78 ].

The main challenges to PAT instruction noted to date are the limited provision of hands-on experience (i.e. real-life emergency situations), limited feedback on site to the EHW on their performance (to enable them to identify and correct areas of weakness in their assessment skills) and lack of standardization in the training programs. Skill decay is problematic as EHWs may forget the PAT steps without regular use. Re-certification requirements depend on the EHW’s professional organization and any employer’s certification requirements.

Although research evidence seems to show that the PAT is considered a valuable tool for rapid assessment of the status of a distressed patient, and its simplicity makes it easy to implement across a range of settings, we identified limited evidence on using the PAT in low-income settings [ 52 , 79 ]. Resource-limited settings may lack coordinated emergency systems including at the scene aid, a system of triage, emergency medical care and critical care [ 80 ]. In these situations, different approaches to pediatric assessments may be adopted, limited data may be recorded on the frequency and quality of PAT assessments [ 81 ] and access to PAT training may be limited. Workforce shortages can impact the availability of trained EHWs to provide PAT instruction. Despite the limited evidence, we anticipate that the PAT is still a feasible tool for EHWs with limited resources [ 52 ]. The PAT’s simplicity can be helpful in rural areas, remote communities, and resource-limited clinics. Based on evidence from this review, the PAT provides a practical and effective way for EHWs to assess children in emergency situations and make informed decisions about their care.

Limitations

This scoping review has limitations. Firstly, we focused on English language articles and there may be additional full text publications in non-English languages that might have provided information on low- and middle-income countries’ experiences of the PAT, its impact, or its psychometric properties. This scoping review was pragmatic, but a follow up review may identify additional studies in languages other than English. Secondly, the search for grey literature was conducted on 14 websites, was hampered by the varying quality (and sometimes absence) of website search engines and the list of websites was prepared by one author (TT). A full systematic review would ideally search a larger number of websites and other sources of grey literature to potentially identify further research, particularly for LMICs. and might have been enhanced by suggestions from experts in the field.

Options for a future systematic review and other areas of research

A full systematic review would likely focus on those research questions for which there are most data following the scoping review and would also include detailed data extraction as well as the grouping of studies by outcomes of interest to provide summaries of the evidence for each outcome. Scoping reviews typically do not conduct risk of bias assessments or evaluate publication bias. A future systematic review could include these steps to assess the strength and quality of the evidence for the use of the PAT.

Other areas for research identified are how the PAT affects pediatric readmissions, patient/caregiver experience, and provider burnout. This scoping review did not find evidence of implementation, that is requirements of recertification and costs or data on utilization for example use of the PAT by emergency call centers, assessments by videoconference or other telemedicine services. Evidence on the utilization of the PAT specific to different emergency transport services such as air medical services, disaster response, etc. was not found.

In summary, this scoping review shows that the PAT has been used in clinical settings for over 20 years. There is some evidence of its validity and reliability, impacts and that the tool is broadly accepted by EHWs. Although the PAT condenses years of experience into a practical and useful assessment suitable for use by less experienced personnel, the need for prior training and certification was highlighted. Although there are gaps in the literature, the evidence has increase in recent years. Scoping reviews are used to inform research agendas and identify implications for policy or practice. As such, psychometric tool data are imperative. Further research on impact and implementation is warranted, and in particular, there is a need to standardize the teaching of PAT teaching and its certification. The simplicity, friendliness and low resources requirement of the tool should be considered in addressing the current healthcare shortages and overcrowding in emergency services.

Availability of data and materials

All data generated or analysed during this study are included in this published article [and its additional information files].

Kobusingye OC, Hyder AA, Bishai D, Joshipura M, Hicks ER, Mock C. Emergency medical services. In: Jamison DT, Breman JG, Measham AR, et al, editors. Disease control priorities in developing countries 2nd ed. Washington, DC: International Bank for Reconstruction and Development / The World Bank; 2006.  https://www.ncbi.nlm.nih.gov/books/NBK11744/ .

World Health Organization (WHO). WHO Emergency care system framework infographics Geneva: World Health Organization; 2018. Available from: https://www.who.int/publications/i/item/who-emergency-care-system-framework . Updated 2 May 2018; cited 2023 11 April.

Andersen K, Mikkelsen S, Jørgensen G, Zwisler ST. Paediatric medical emergency calls to a Danish emergency medical dispatch centre: a retrospective, observational study. Scand J Trauma Resusc Emerg Med. 2018;26(1):2.

Article   PubMed   PubMed Central   Google Scholar  

Lee LK, Porter JJ, Mannix R, Rees CA, Schutzman SA, Fleegler EW, et al. Pediatric traumatic injury emergency department visits and management in US children’s hospitals from 2010 to 2019. Ann Emerg Med. 2022;79(3):279–87.

Article   PubMed   Google Scholar  

Fuchs S. The origins and evolution of emergency medical services for children. Pediatr Ann. 2021;50(4):e150–4.

Fuchs S, Yamamoto L, editors. APLS: the pediatric emergency medicine resource. 5th ed. Burlington: Jones & Bartlett Learning; 2012. p. 538.

Houston R, Pearson GA. Ambulance provision for children: a UK national survey. Emerg Med J. 2010;27(8):631–6.

McDermott KW, Stocks C, Freeman WJ. Overview of pediatric emergency department visits, 2015: Statistical Brief #242. Rockville: Agency for Healthcare Research and Quality; 2018.

Google Scholar  

Nesje E, Valoy NN, Kruger AJ, Uleberg O. Epidemiology of paediatric trauma in Norway: a single-trauma centre observational study. Int J Emerg Med. 2019;12(1):18.

United Nations Children's Fund (UNICEF). Levels and trends in child mortality. Report 2022. New York: UNICEF; 2023. Available from: https://data.unicef.org/resources/levels-and-trends-in-child-mortality/ .

Jeruzal JN, Boland LL, Frazer MS, Kamrud JW, Myers RN, Lick CJ, et al. Emergency medical services provider perspectives on pediatric calls: a qualitative study. Prehosp Emerg Care. 2019;23(4):501–9.

Nordén C, Hult K, Engström Å. Ambulance nurses’ experiences of nursing critically ill and injured children: a difficult aspect of ambulance nursing care. Int Emerg Nurs. 2014;22(2):75–80.

Hansen M, Meckler G, Dickinson C, Dickenson K, Jui J, Lambert W, et al. Children’s safety initiative: a national assessment of pediatric educational needs among emergency medical services providers. Prehosp Emerg Care. 2015;19(2):287–91.

Li J, Roosevelt G, McCabe K, Preotle J, Pereira F, Takayesu JK, et al. Critically ill pediatric case exposure during emergency medicine residency. J Emerg Med. 2020;59(2):278–85.

Ralston ME, Zaritsky AL. New opportunity to improve pediatric emergency preparedness: pediatric emergency assessment, recognition, and stabilization course. Pediatrics. 2009;123(2):578–80.

Roukema J, Steyerberg EW, van Meurs A, Ruige M, van der Lei J, Moll HA. Validity of the manchester triage system in paediatric emergency care. Emerg Med J. 2006;23(12):906.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Yates MT, Ishikawa T, Schneeberg A, Brussoni M. Pediatric Canadian Triage and Acuity Scale (PaedsCTAS) as a measure of injury severity. Int J Environ Res Public Health. 2016;13(7):659.

Thim T, Krarup NH, Grove EL, Rohde CV, Lofgren B. Initial assessment and treatment with the Airway, Breathing, Circulation, Disability, Exposure (ABCDE) approach. Int J Gen Med. 2012;5:117–21.

Shah MN. The formation of the emergency medical services system. Am J Public Health. 2006;96(3):414–23.

Fuchs S, McEvoy M, editors. Pediatric education for prehospital professionals. 4th ed. Burlington: Jones & Bartlett Learning; 2021. p. 490.

Peters MDJ, Godfrey C, McInerney P, Munn Z, Tricco AC, Khalil H. Chapter 11: scoping reviews. In: JBI manual for evidence synthesis. Joanna Briggs Institute; 2020. Available from: https://synthesismanual.jbi.global .

Tørisen TAG, Glanville J, Loaiza-Betancur AF, Bidonde J. Emergency pediatric patients and use of the pediatric assessment triangle (PAT) tool. Protocol for a scoping review. Charlottesville: Open Science Framework; 2022. https://www.osf.io/vkd5h .

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for Scoping Reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169(7):467–73.

Fernandez A, Benito J, Mintegi S. Is this child sick? Usefulness of the pediatric assessment triangle in emergency settings. J Pediatr (Rio J). 2017;93(Suppl 1):60–7.

World Bank. New World Bank country classifications by income level: 2022-2023. The World Bank Group; 202. Available from: https://blogs.worldbank.org/opendata/new-world-bank-country-classifications-income-level-2022-2023 .

Avilés-Martinez KI, López-Enríquez A, Luévanos-Velázquez A, Jiménez-Pérez BA, García-Armenta MB, Ceja-Moreno H, et al. Triage, priorization tools of pediatric emergency room. Acta Pediatr de Mex. 2016;37(1):4–16.

Gausche-Hill M, Eckstein M, Horeczko T, McGrath N, Kurobe A, Ullum L, et al. Paramedics accurately apply the pediatric assessment triangle to drive management. Prehosp Emerg Care. 2014;18(4):520–30.

Horeczko T, Enriquez B, McGrath NE, Gausche-Hill M, Lewis RJ. The pediatric assessment triangle: accuracy of its application by nurses in the triage of children. J Emerg Nurs. 2013;39(2):182–9.

Ma X, Liu Y, Du M, Ojo O, Huang L, Feng X, et al. The accuracy of the pediatric assessment triangle in assessing triage of critically ill patients in emergency pediatric department. Int Emerg Nurs. 2021;58.

Fernández A, Ares MI, Garcia S, Martinez-Indart L, Mintegi S, Benito J. The validity of the pediatric assessment triangle as the first step in the triage process in a pediatric emergency department. Pediatr Emerg Care. 2017;33(4):234–8.

Lugo S, Pavlicich V. Application of the pediatric assessment triangle to the triage classification system in an emergency department. Rev Bol Ped. 2014;53(2):88–93.

Fernández A, Pijoan JI, Ares MI, Mintegi S, Benito FJ. Canadian paediatric triage and acuity scale: assessment in a European pediatric emergency department. Emergencias. 2010;22(5):355–60.

Ecclesia FG, Alonso Cadenas JA, Gómez B, Gangoiti I, Hernández-Bou S, de la Torre EM. Late-onset group B streptococcus bacteremia evaluated in the pediatric emergency department and risk factors for severe infection. Pediatr Infect Dis J. 2022;41(6):455–9.

Gomez B, Hernandez-Bou S, Garcia-Garcia JJ, Mintegi S. Bacteremia in previously healthy children in emergency departments: clinical and microbiological characteristics and outcome. Eur J Clin Microbiol Infect Dis. 2015;34(3):453–60.

Article   CAS   PubMed   Google Scholar  

Sánchez IA, Cotanda CP, Casas MM, de la Maza VTS, Cubells CL. Profile of the child seen in the resuscitation room. Rev Esp Salud Publica. 2019;93.

Shiva GS, Kumar VS, Kumar PR, Subramanian SB. A study on the role of paediatric assessment triangle, clinical scoring and serum lactate in the management of septic shock in children. Int J Contemp Pediatr. 2019;6(5):2037.

Article   Google Scholar  

Macnab AJ. Objective assessment and communication of the physiologic status of the sick infant. Can J Midwif Res Pract. 2004;3(2):7–12.

Mierek C, Nacca N, Scott JM, Wojcik SM, D’Agostino J, Dougher K, et al. View from the door: making pediatric transport decisions based on first impressions. JEMS. 2010;35(7):68–9, 71, 3, 5, 7, 9, 81.

PubMed   Google Scholar  

Arroabarren E, Alvarez-Garcia J, Anda M, de Prada M, Ponce MC, Palacios M. Quality of the triage of children with anaphylaxis at the emergency department. Pediatr Emerg Care. 2021;37(1):17–22.

Alp EE, Dalgic N, Yilmaz V, Altuntas Y, Ozdemir HM. Evaluation of patients with suspicion of COVID-19 in pediatric emergency department. Sisli Etfal Hastan Tip Bul. 2021;55(2):179–87.

PubMed   PubMed Central   Google Scholar  

Derİnöz-Güleryüz O. In-hospital pediatric patient transfers to the pediatric emergency department. Cukurova Med J. 2022;47(1):332–40.

Kawai R, Nomura O, Tomobe Y, Morikawa Y, Miyata K, Sakakibara H, et al. Retrospective observational study indicates that the paediatric assessment triangle may suggest the severity of Kawasaki disease. Acta Paediatr. 2018;107(6):1049–54.

Paniagua N, Elosegi A, Duo I, Fernandez A, Mojica E, Martinez-Indart L, et al. Initial asthma severity assessment tools as predictors of hospitalization. J Emerg Med. 2017;53(1):10–7.

Alonso Cadenas JA, Corredor Andrés B, Andina Martínez D, et al. Characteristics and risk factors for admission in children undergoing hematopoietic cell transplantation in a pediatric emergency department. Authorea. 2021. https://doi.org/10.22541/au.163253914.42579466/v1 .

Rodríguez Borbolla FJ, Sancha Herrera ML, Ortiz Angulo E, Pulido PP. Implementación del sistema de clasificación en la Unidad de Urgencias Pediátricas del Hospital Marqués de Valdecilla. Fundacion de Enfermeria de Cantabria. 2013;2(9):26–31.

Suárez M, Jaime M. Utilidad del triángulo de evaluación pediátrica en un servicio de emergencia pediátrica. Boletin Medico de Postgrado. 2018;34(2):39–45.

Ogden K. The use of the paediatric assessment triangle in the management of the sick child. Emerg Med J. 2016;33(9):e4.

Romig LE. PREP for peds-patient physiology, rescuer responses, equipment, protocols. Size-up & approach tips for pediatric calls. JEMS. 2001;26(5):24–33.

CAS   PubMed   Google Scholar  

Horeczko T, Gausche-Hill M. The paediatric assessment triangle: a powerful tool for the prehospital provider. J Paramed Pract. 2011;3(1):20–5.

Walker A, Hanna A. Kids really are just small adults: utilizing the pediatric triangle with the classic ABCD approach to assess pediatric patients. Cureus. 2020;12(3): e7424.

Morilla L, Morel Z, Pavlicich V. Clinical characteristics of pediatric patients with COVID-19 in an emergency department. Pediatría (Asunción). 2020;47(3):124–31.

Akindolire AE, Tongo OO. Paediatric critical care needs assessment in a tertiary facility in a developing country. Niger J Paediatr. 2018;45(1):10–4.

Anitha GF, Velmurugan L, Sangareddi S, Nedunchelian K, Selvaraj V. Effectiveness of flow inflating device in providing Continuous Positive Airway Pressure for critically ill children in limited-resource settings: a prospective observational study. Indian J Crit Care Med. 2016;20(8):441–7.

Benito J, Luaces-Cubells C, Mintegi S, Manrique Martínez I, De la Torre EM, Miguez Navarro C, et al. Evaluation and impact of the “advanced pediatric life support” course in the care of pediatric emergencies in Spain. Pediatr Emerg Care. 2018;34(9):628–32.

Guerrero-Márquez G, Míguez-Navarro MC. The physiological diagnosis missing in the pediatric assessment triangle. Pediatr Emerg Care. 2021;37(11).

Jayashree M, Singhi SC. Initial assessment and triage in ER. Indian J Pediatr. 2011;78(9):1100–8.

Gonzalez Brabin A, Martín Rivada Á, Cabrero Hernández M, Cañedo Villarroya E. MIR clinical case. Make your diagnosis: a newborn with decreased intake and lethargy. Pediatr Integ. 2019;23(3):162–5.

Simon Junior H, Schvartsman C, Sukys GA, Farhat SCL. Pediatric emergency triage systems. Rev Paul Pediatr. 2022;41:e2021038.

Gehri M, Flubacher P, Chablaix C, Pediatrics Curchod P. The PAT: a simple and rapid tool for the assessment of the severely ill or injured child. Rev Med Suisse. 2011;3(277):64–5.

Dieckmann RA. New assessment model saves critical time in pediatric emergencies. AAP News. 1999;15(2):22.

Mendes M, McCormick T. Pediatric resuscitation. In: Rose E, editor. Pediatric emergencies: a practical, clinical guide. United kingdom: Oxford Oxford University Press; 2020. p. 67–74.

Chapter   Google Scholar  

Fuchs S. The special needs of children. In: Cone D, Brice JH, Delbridge TR, Myers JB, editors. Emergency medical services: clinical practice and systems oversight. 3rd ed. Hoboken: Wiley-Blackwell; 2021. p. 379–85.

Dieckmann RA, Brownstein D, Gausche-Hill M. The pediatric assessment triangle: a novel approach for the rapid evaluation of children. Pediatr Emerg Care. 2010;26(4):312–5.

Chiu Y-C, Liu S-Y, Yen T-A, Chen Y-Y, Yang C-W, Chu T-S, et al. Application of high-fidelity patient simulation in the teaching of pediatric primary assessment and management - is it feasible for medical students? J Med Educ. 2018;22(1):17–27.

Hansen M, Spiro DM. Teaching the pediatric assessment triangle using online video cases. Ann Emerg Med. 2013;62(5):S172.

Patten J. Goal-directed management of shock in children [thesis]. Zagreb: University of Zagreb School of Medicine; 2015.

Tagg A. Paediatric Assessment Triangle [internet]: Don't forget the bubbles; 2019 [updated 06/02/2023. Available from: https://dontforgetthebubbles.com/the-paediatric-assessment-triangle/ .

Furmick J, Malburg L, Leetch A. Pediatric airway management. Pediatr Emerg Med Rep. 2017;22(10):1–17.

Khouli M. Injuries in children and general principles of management. Mexican J Med Res ICSA. 2015;3(5). https://doi.org/10.29057/mjmr.v3i5.1835 .

Pérez LFT, Bouza MR, Valle AML, Hoyos JB, Vera CV. Emergency management: introduction. Revista Infancia y Salud. 2019;1(2). Available from: http://rinsad.uca.es/ojs3/index.php/rinsad/article/view/20 .

Rochat MK, Gehri M. Pediatric emergencies - the essential, briefly, for general practitioners. Ther Umsch. 2013;70(11):653–60.

Yock Corrales A, Starr M. Assessment of the unwell child. Aust Fam Physician. 2010;39(5):270–5.

Agbim CA, Wang NE, Lee M. Respiratory distress in pediatric patients. Pediatr Emerg Med Rep. 2018;23(4):41–55.

Güler E, Özkaya AK. Recognition of shock in children: review. Turkiye Klinikleri Pediatri. 2015;24(2):45–50.

Ramser M. The febrile child in respiratory distress. Praxis. 2017;106(4):201–7.

Singh A, Frenkel O. Evidence-based emergency management of the pediatric airway. Pediatr Emerg Med Pract. 2013;10(1):1–25.

Neumar RW, Blomkalns AL, Cairns CB, D’Onofrio G, Kuppermann N, Lewis RJ, et al. Emergency medicine research: 2030 strategic goals. Acad Emerg Med. 2022;29(2):241–51.

Grant D. The future of paediatric simulation. In: Cheng A, Grant V, editors. Comprehensive healthcare simulation: Pediatric Edition: Springer International Publishing. 2016. p. 401.

Hansoti B, Jenson A, Keefe D, De Ramirez SS, Anest T, Twomey M, et al. Reliability and validity of pediatric triage tools evaluated in low resource settings: a systematic review. BMC Pediatr. 2017;17(1):37.

Slusher T, Bjorklund A, Aanyu HT, Kiragu A, Philip C. The assessment, evaluation, and management of the critically ill child in resource-limited international settings. J Pediatr Intensive Care. 2017;6(1):66–76.

Muttalib F, González-Dambrauskas S, Lee JH, Steere M, Agulnik A, Murthy S, et al. Pediatric emergency and critical care resources and infrastructure in resource-limited settings: a multicountry survey. Crit Care Med. 2021;49(4):671–81.

Download references

Acknowledgements

This work was submitted as a thesis requirement for the Master in Pre-Hospital Critical Care, University of Stavanger, Norway in December 2022. We thank the Norwegian Institute of Public Health for supporting this publication open access fees.

This work had no financial support.

Author information

Authors and affiliations.

University of Stavanger, P.O. Box 8600, NO-4036, Stavanger, Norway

Tore A. G. Tørisen

Glanville.Info, 38 Moorgate, York, YO24 4HR, UK

Julie M. Glanville

Instituto Universitario de Educación Física, Universidad de Antioquia. , Medellín, Colombia

Andres F. Loaiza

Grupo de Investigación en Entrenamiento Deportivoy, Actividad Física Para La Salud (GIEDAF), Universidad Santo Tomás. , Tunja, Colombia

School of Rehabilitation Science, College of Medicine, University of Saskatchewan, 107 Wiggins Rd, Saskatoon, SK, S7N 5E5, Canada

Julia Bidonde

You can also search for this author in PubMed   Google Scholar

Contributions

Project conceptualization was done by TT, JB, JG, and AFL. TT and JB were primarily responsible for the methodology and resources, while TT, JB, and AFL carried out the data extraction and validation process, and TT led the analysis and synthesis. TT and JB wrote the original draft, with input and feedback from JG and AFL during the review and editing process. The visualization was primarily handled by TT and AFL. TT and JB supervised the project, and TT managed project administration. JB was responsible for software acquisition.

Corresponding author

Correspondence to Julia Bidonde .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

Tore A. G. Tørisen is an instructor and medical supervisor for courses in “Pediatric Education for Prehospital Professionals” (PEPP) and he is registered with the American Academy of Pediatrics (AAP). The remaining authors declare no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: appendix b. prisma-scr, additional file 2. appendix c. search strategy, additional file 3. appendix d. excluded studies with reasons for exclusion, additional file 4. appendix e. included studies, additional file 5. tables, additional file 6. appendix a – who emergency care system framework, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Tørisen, T.A.G., Glanville, J.M., Loaiza, A.F. et al. Emergency pediatric patients and use of the pediatric assessment triangle tool (PAT): a scoping review. BMC Emerg Med 24 , 158 (2024). https://doi.org/10.1186/s12873-024-01068-w

Download citation

Received : 19 July 2023

Accepted : 08 August 2024

Published : 04 September 2024

DOI : https://doi.org/10.1186/s12873-024-01068-w

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Pediatrics or paediatric
• Pediatric assessment triangle
• Children; emergency medicine; review

BMC Emergency Medicine

ISSN: 1471-227X

• General enquiries: [email protected]

• Open access
• Published: 03 September 2024

Effectiveness of lifestyle interventions for glycaemic control among adults with type 2 diabetes in West Africa: a systematic review and meta-analysis protocol

• Ellen Barnie Peprah   ORCID: orcid.org/0000-0002-7623-2030 1 ,
• Yasmin Jahan 2 ,
• Anthony Danso-Appiah 3 ,
• Abdul-Basit Abdul-Samed 1 ,
• Tolib Mirzoev 2 ,
• Edward Antwi 4 ,
• Dina Balabanova 2 &
• Irene Agyepong 1  

Systematic Reviews volume  13 , Article number:  226 ( 2024 ) Cite this article

6 Altmetric

Metrics details

Lifestyle interventions are key to the control of diabetes and the prevention of complications, especially when used with pharmacological interventions. This protocol aims to review the effectiveness of lifestyle interventions in relation to nutrition and physical activity within the West African region. This systematic review and meta-analysis seeks to understand which interventions for lifestyle modification are implemented for the control of diabetes in West Africa at the individual and community level, what evidence is available on their effectiveness in improving glycaemic control and why these interventions were effective.

We will review randomised control trials and quasi-experimental designs on interventions relating to physical activity and nutrition in West Africa. Language will be restricted to English and French as these are the most widely spoken languages in the region. No other filters will be applied. Searching will involve four electronic databases — PubMed, Scopus, Africa Journals Online and Cairn.info using natural-language phrases plus reference/citation checking.

Two reviewers will independently screen results according to titles and abstracts against the inclusion and exclusion criteria to identify eligible studies. Upon full-text review, all selected studies will be assessed using Cochrane’s Collaboration tool for assessing the risk of bias of a study and the ROBINS-I tool before data extraction. Evidence will be synthesised narratively and statistically where appropriate. We will conduct a meta-analysis when the interventions and contexts are similar enough for pooling and compare the treatment effects of the interventions in rural to urban settings and short term to long term wherever possible.

We anticipate finding a number of studies missed by previous reviews and providing evidence of the effectiveness of different nutrition and physical activity interventions within the context of West Africa. This knowledge will support practitioners and policymakers in the design of interventions that are fit for context and purpose within the West African region.

Systematic review registration

This systematic review has been registered in the International Prospective Register for Systematic Reviews — PROSPERO, with registration number CRD42023435116. All amendments to this protocol during the process of the review will be explained accordingly.

Peer Review reports

Diabetes is a chronic disease estimated to affect 537 million adults worldwide. According to the World Health Organization, 1.5 million deaths are directly attributable to diabetes annually, and this disproportionately affects populations in developing countries. This same population is often at higher risk of late diagnosis, poor clinical management and its associated microvascular and/or cardiovascular complications [ 1 ].

The West African region — home to 16 developing economies — is still reeling from the impact of the COVID-19 pandemic. The rapidly changing sociocultural environment, demographics and economic conditions further threaten to worsen the burden of noncommunicable diseases such as diabetes within the region [ 2 , 3 ]. With less than 7 years to meet the United Nations Sustainable Development Goal 3.4 target of reducing by a third premature mortality from noncommunicable diseases, greater investment in interventions that bridge the gaps in service delivery, programme design and policy implementation will be required.

The 2023 Standards of Care in Diabetes of the American Diabetes Association (ADA) names, among many others, physical activity and medical nutrition therapy as interventions to facilitate positive health behaviours to improve outcomes for diabetes [ 4 ]. Physical activity includes all movements that increase energy expenditure such as walking, housework, gardening, swimming, dancing, yoga, aerobic activities and resistance training. Exercise, on the other hand, is structured and tailored towards improving physical fitness. Interventions for physical activity and exercise are both recommended for better glycaemic control [ 5 ]. ADA recommends at least 150 min or more of moderate to vigorous exercise a week and encourages an increase in non-sedentary physical activity among people living with type 2 diabetes. The goal of interventions for nutrition therapy is to manage weight, achieve individual glycaemic control targets and prevent complications. ADA recommends that nutrition therapy and counselling, under the guidance of a registered dietician, is administered to patients with type 2 diabetes with emphasis on managing energy balance, dietary protein, carbohydrate and fat intake and alcohol consumption [ 4 ].

There is evidence available supporting the effectiveness of physical activity and nutrition interventions to achieve glycaemic control and improve overall cardiometabolic health in other populations [ 6 , 7 , 8 ]. However, there is not much evidence of its effectiveness in the West African population. Those that are documented in literature exist in fragmented, regional spaces, and the West African context could be easily lost in larger studies such as Sagastume et al. [ 9 ]. O’Donoghue and colleagues [ 10 ] reviewed randomised control trials on lifestyle interventions from low- and middle-income countries. However, the sheer geographical breadth of studies represented within the review, the diversity of populations, differences in health system structures and priorities [ 11 ] and cultural and socio-economic contexts included in the review pose a challenge to generalisation of study findings to the West African population. Also, controversies remain on what type of nutrition therapy or meal plans work best for people with diabetes [ 12 ], whether structured self-management education yields greater benefit for patients [ 13 ] and whether exercise, its duration or intensity has varying effect on glycaemic control in patients with diabetes [ 14 , 15 ]. The aforementioned present the need to assemble existing studies and synthesise what is known about their effectiveness. Knowledge of what exists would shape future interventions for diabetes control in West Africa.

This review will seek to address the following questions:

Which individual-level interventions for lifestyle modification are available for the control of type 2 diabetes in adults West Africa?

What is the effectiveness of the available individual level interventions for lifestyle modification in glycaemic control?

Which community-level interventions for lifestyle modification are implemented for the control of diabetes in West Africa?

What is the effectiveness of community level interventions for lifestyle modification in improving glycaemic control?

Which factors influence the effectiveness of glycaemic control interventions at the individual and community level?

Criteria for considering studies for this review

The Population, Intervention, Comparison, Outcome and Studies (PICOS) framework will be used in determining inclusion for the study.

Adults aged 18 years and older living in West Africa with previously or newly diagnosed type 2 diabetes. We will not consider type 1 diabetes and paediatric and gestational diabetes mellitus.

Intervention

All lifestyle interventions relating to physical activity and nutrition will be considered. Physical activity will include low, moderate and high intensity exercises. Non-sedentary everyday movement such as walking, gardening and housework will be considered so long as it is delivered in a regimen and has been measured. Interventions for nutrition will include vegetarian, low carbohydrate diet, low fat or plant-based diet. For the purpose of this review, interventions for alcohol reduction will be considered as a part of nutrition. The duration of intervention could be short-term interventions which we define as 3 months or less or long-term intervention which we define as greater than 3 months. We define individual-level interventions as those targeted at the individual patient, such as one-on-one counselling or structured education programmes delivered to an individual. Community-level interventions are those implemented at the broader community or population level, such as public awareness campaigns and community-based physical activity programmes. In all situations, interventions could be provider-led, and group-based or individually based activities will be considered in the review.

The control will be usual care or no intervention.

The primary outcome of interest to this review is glycaemic control as indicated by glycated haemoglobin (HBA1c) values. Despite objections to the preference of HBA1c for diagnosing diabetes by some researchers based on the cost and biological variation [ 16 ], it is generally regarded as a reliable metric for glycaemic improvement in clinical trials [ 17 ]. We will say an intervention improves glycaemic control when there is a clinically significant reduction of HbA1c of greater 5 mmol/mol or 0.5% of HbA1c from pre-intervention baseline [ 18 ]. If there is a non-clinically significant reduction in HbA1c of less than 5 mmol/mol or 0.5% of HbA1c, no reduction or an increase in HbA1c from pre-intervention baseline, we will say that intervention does not improve glucose control.

Eligible study designs will be limited to randomised control trials and quasi-experimental studies such as pretest and posttest study designs, nonequivalent control group designs and controlled observational studies that attempt to establish causal relationships between the intervention and the outcomes.

We will search four online databases (PubMed, Scopus, Africa Journals Online and Cairn.info) for articles published from 2000 to 31st August 2024 We will also search websites of relevant government agencies and non-governmental organisations such as PATH and Sante Diabete for programme reports, evaluations and relevant publications and clinical trial registries for ongoing or recently completed trials (summarised in Additional File 1, PRISMA_2020_Search flowchart.docx attached). In order not to miss any relevant study, we will also search through the reference list and bibliographies of included studies.

Search strategy

Search terms we will use include “diabetes”, “lifestyle modification”, “physical activity”, “nutrition” and their synonyms, and MESH terms. (Additional File 2, Search strategy.docx) detail the full search strategy and a sample search for PubMed. Language will be restricted to English and French as these are the most widely used for scholarly publications and reports within the region. No other filters will be applied. A search alert will be created to update on any new studies, while the search and screening process is ongoing.

Study selection and management

Two reviewers will independently screen search results according to titles and abstracts against the inclusion and exclusion criteria to identify eligible studies (see Additional file 3, Algorithm for Screening.docx). Duplicates and irrelevant titles and abstracts will be removed. A third reviewer will settle discrepancies through a consensus. A full-text review of all selected studies will then be conducted against the inclusion criteria to identify studies to be included for analysis. Search results will be managed using the Rayyan software platform to facilitate the screening process.

Study risk-of-bias assessment

All selected studies will be assessed using Cochrane’s Collaboration tool for assessing the risk of bias of a study. For the risk-of-bias assessment of non-randomised studies, we will use the ROBINS-I tool. Judging from quotes from the authors, two independent reviewers will rate studies as either low risk, high risk or unclear, and a third reviewer will settle discrepancies if there are any. A sensitivity analysis will be conducted to evaluate the impact of high-risk studies on the overall analysis before a decision to exclude studies will be done.

Data extraction and management

For each of the studies selected, the following data will be extracted independently on a data collection form in Microsoft Excel by two reviewers: (1) first author’s last name; (2) year of publication; (3) country; (4) study setting; (5) characteristics of participants, sample size and mean age; (6) type of intervention, frequency and duration; (7) characteristics of control group; (8) pre-intervention baseline HbA1c; (9) post-intervention HbA1c; (10) any other outcomes if reported; and (11) author’s conclusions. We will contact authors for missing data or to clarify data. We will first attempt to contact the corresponding authors of the included studies via email, providing a clear timeline for their response. If the authors do not respond within 4 weeks, we will send a follow-up email. If the authors do not respond, we will proceed with the data synthesis and clearly report the missing information and its potential impact on the overall findings in the limitations section of the review.

Strategy for data synthesis

We will estimate the effect of the intervention using the relative risk for the number achieving glycaemic control as our primary outcome. If other effect estimates are provided, we will convert between estimates where possible. Measures of precision will be at 95% confidence intervals which will be computed using the participants per treatment group rather than the number of intervention attempts. Study authors will be contacted if there is the need for further information or clarification about methods used in analysing results. If the author of selected articles cannot be reached for clarification, we will not report confidence intervals or p -values for which clarification is needed. When both pre-intervention baseline and endpoint measures are reported, endpoint measures and their standardised deviation will be used.

We will conduct a meta-analysis when the interventions and contexts are similar enough for pooling. Since heterogeneity is expected a priori due to age, sex and study setting, i.e. whether urban or rural, we will estimate the pooled treatment effect estimates and its 95% confidence interval controlling for these variables. Forest plots will be used to visualise the data and extent of heterogeneity among studies. We will conduct a sensitivity analysis to explore the influence of various factors on the effect size of only the primary outcome, that is glycaemic control. Any post hoc sensitivity analyses that may arise during the review process will be explained in the final report.

We will use a cluster-based analysis when analysing interventions at the community level. When both individual- and cluster-level factors are reported, we will use cluster-level data for our analysis taking into consideration their design effect. We intend to perform a thematic, qualitative analysis in determining the factors that influence the effectiveness of identified interventions at the community level.

We anticipate retrieving data about the West African context on the effectiveness of physical activity and nutrition interventions on improving glycaemic control in patients living with an established type 2 diabetes. This information will guide practitioners and policymakers to design interventions that are fit for context and purpose within West Africa and Africa, by extension.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

American Diabetes Association

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

Population, Intervention, Comparison, Outcome and Studies

Glycated haemoglobin

Glazier RH, Bajcar J, Kennie NR, Willson K. A systematic review of interventions to improve diabetes care in socially disadvantaged populations. Diabetes Care. 2006;29(7):1675–88.

Article   PubMed   Google Scholar  

Atun R, Davies JI, Gale EAM, Bärnighausen T, Beran D, Kengne AP, et al. Diabetes in sub-Saharan Africa: from clinical care to health policy. Lancet Diabetes Endocrinol. 2017;5(8):622–67.

Gordon Patti K, Kohli P. COVID’s impact on non-communicable diseases: what we do not know may hurt us. Curr Cardiol Rep. 2022;24(7):829–37.

Article   PubMed   PubMed Central   Google Scholar  

ElSayed NA, Aleppo G, Aroda VR, Bannuru RR, Brown FM, Bruemmer D, et al. 5. Facilitating positive health behaviors and well-being to improve health outcomes: standards of care in diabetes—2023. Diabetes Care. 2023;46(Supplement_1):S68-96.

Caspersen CJ, Powell KE, Christenson GM. Physical activity, exercise, and physical fitness: definitions and distinctions for health-related research. Public Health Rep Wash DC 1974. 1985;100(2):126–31.

CAS   Google Scholar  

Warburton DER, Nicol CW, Bredin SSD. Health benefits of physical activity: the evidence. CMAJ Can Med Assoc J. 2006;174(6):801–9.

Article   Google Scholar  

Greaves CJ, Sheppard KE, Abraham C, Hardeman W, Roden M, Evans PH, et al. Systematic review of reviews of intervention components associated with increased effectiveness in dietary and physical activity interventions. BMC Public Health. 2011;11:119.

Hamasaki H. Daily physical activity and type 2 diabetes: a review. World J Diabetes. 2016;7(12):243.

Sagastume D, Siero I, Mertens E, Cottam J, Colizzi C, Peñalvo JL. The effectiveness of lifestyle interventions on type 2 diabetes and gestational diabetes incidence and cardiometabolic outcomes: a systematic review and meta-analysis of evidence from low- and middle-income countries. eClinicalMedicine. 2022;53:101650.

O’Donoghue G, O’Sullivan C, Corridan I, Daly J, Finn R, Melvin K, et al. Lifestyle interventions to improve glycemic control in adults with type 2 diabetes living in low-and-middle income countries: a systematic review and meta-analysis of randomized controlled trials (RCTs). Int J Environ Res Public Health. 2021;18(12):6273.

Mounier-Jack S, Mayhew SH, Mays N. Integrated care: learning between high-income, and low- and middle-income country health systems. Health Policy Plan. 2017;32(Suppl 4):iv6-12.

5. Facilitating positive health behaviors and well-being to improve health outcomes: standards of care in diabetes—2023 | Diabetes Care | American Diabetes Association. Available from: https://diabetesjournals.org/care/article/46/Supplement_1/S68/148055/5-Facilitating-Positive-Health-Behaviors-and-Well . [cited 2023 May 8].

Lamptey R, Amoakoh-Coleman M, Barker MM, Iddi S, Hadjiconstantinou M, Davies M, et al. Change in glycaemic control with structured diabetes self-management education in urban low-resource settings: multicentre randomised trial of effectiveness. BMC Health Serv Res. 2023;23(1):199.

Çolak TK, Acar G, Dereli EE, Özgül B, Demirbüken İ, Alkaç Ç, et al. Association between the physical activity level and the quality of life of patients with type 2 diabetes mellitus. J Phys Ther Sci. 2016;28(1):142–7.

Munan M, Dyck RA, Houlder S, Yardley JE, Prado CM, Snydmiller G, et al. Does exercise timing affect 24-hour glucose concentrations in adults with type 2 diabetes? A follow up to the exercise-physical activity and diabetes glucose monitoring study. Can J Diabetes. 2020;44(8):711-718.e1.

Higgins T. HbA1c for screening and diagnosis of diabetes mellitus. Endocrine. 2013;43(2):266–73.

Article   PubMed   CAS   Google Scholar  

American Diabetes Association. 6. Glycemic targets: standards of medical care in diabetes—2021. Diabetes Care. 2020;44(Supplement_1):S73-84.

Lenters-Westra E, Schindhelm RK, Bilo HJG, Groenier KH, Slingerland RJ. Differences in interpretation of haemoglobin A1c values among diabetes care professionals. Neth J Med. 2014;72(9):462–6.

PubMed   CAS   Google Scholar  

Download references

Acknowledgements

This research was funded by the NIHR Global Health Research Centre for Non-Communicable Disease Control in West Africa using UK aid from the UK government to support global health research. The views expressed in this publication are those of the author(s) and not necessarily those of the NIHR or the UK government.

This review is funded by the National Institution for Health and Care Research (NIHR) with grant number NIHR203246. The funding body played no role in the development of this protocol.

Author information

Authors and affiliations.

Ghana College of Physicians and Surgeons, Accra, Ghana

Ellen Barnie Peprah, Abdul-Basit Abdul-Samed & Irene Agyepong

London School of Hygiene and Tropical Medicine, London, UK

Yasmin Jahan, Tolib Mirzoev & Dina Balabanova

University of Ghana, Accra, Ghana

Anthony Danso-Appiah

Ghana Health Service, Accra, Ghana

Edward Antwi

You can also search for this author in PubMed   Google Scholar

Contributions

EBP prepared the initial draft of the manuscript; all authors reviewed, provided feedback and approved this version of the protocol. EBP will be the guarantor of the review.

Corresponding author

Correspondence to Ellen Barnie Peprah .

Ethics declarations

Ethics approval and consent to participate.

This review is in connection with executing the protocol titled: “Strengthening Capacity for NCD Control in West Africa: Phase 1 Study – Deepening Understanding of Contextual Influences and Effective Pathways to Prevention, Diagnosis and Primary Care Management and Referral of NCD”. The protocol has received ethical clearance from the Ghana Health Service Ethics Review Committee (ERC) (Protocol ID No: GHS-ERC 013/02/23).

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

13643_2024_2555_moesm1_esm.docx.

Additional file 1: Figure 1. PRISMA 2020 flow diagram for new systematic reviews which included searches of databases, registers and other sources.

Additional file 2. Describes search concepts, includes a sample search for PubMed. Table 1. PubMed search strategy.

Additional file 3: figure 2. decision-making flowchart for screening., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Peprah, E.B., Jahan, Y., Danso-Appiah, A. et al. Effectiveness of lifestyle interventions for glycaemic control among adults with type 2 diabetes in West Africa: a systematic review and meta-analysis protocol. Syst Rev 13 , 226 (2024). https://doi.org/10.1186/s13643-024-02555-8

Download citation

Received : 28 October 2023

Accepted : 03 May 2024

Published : 03 September 2024

DOI : https://doi.org/10.1186/s13643-024-02555-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Type 2 diabetes
• Physical activity
• Diet modification
• West Africa
• Glycaemic control

Systematic Reviews

ISSN: 2046-4053

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Advertisement

Risk of incident venous thromboembolism in patients with atopic dermatitis: systematic analysis of the literature and meta-analysis

• Published: 06 September 2024

Cite this article

• Yifei Wang 1 , 2   na1 ,
• Zhiqiang Chen 3   na1 ,
• Ting He 1 , 2 ,
• Changzheng Huang 1 , 2 &
• Chen Shen   ORCID: orcid.org/0000-0001-8231-9303 1 , 2  

1 Altmetric

Previous studies have showed conflicting results regarding the association between atopic dermatitis (AD) and venous thromboembolism (VTE) events.

Based on our systematic review of 6 cohort studies, patients with AD have an increased risk of VTE events.

These findings provide key evidence-based estimates to inform decision-making that VTE is a comorbidity of AD.

Atopic dermatitis (AD) is a prevalent chronic inflammatory skin disease. While various inflammatory conditions have been linked to venous thromboembolism (VTE), the risk of VTE among patients with AD remains unclear. We sought to systematically review and meta-analyze population-based studies to determine the association between AD and incident VTE. A systematic review was performed of published studies in PubMed, Web of Science, Embase and Cochrane library from their inception to 27 May 2024. At least two reviewers conducted title/abstract, full-text review and data extraction. Cohort studies examining the association of AD with incident VTE were included. Quality of evidence was assessed using the Newcastle-Ottawa Scale. Six cohort studies, encompassing a total of 10,186,861 participants, were included. The meta-analysis revealed a significantly increased risk for incident VTE among AD patients (pooled hazard ratio (HR), 1.10; 95% CI, 1.00–1.21), with an incidence rate of VTE at 3.35 events per 1000 patient-years. Individual outcome analyses suggested that AD was associated with higher risks of deep vein thrombosis (pooled HR, 1.15; 95% CI, 1.04–1.27) but not pulmonary embolism (pooled HR, 0.99; 95% CI, 0.87–1.13). This systematic review and meta-analysis indicated an increased risk of incident VTE among patients with AD. Future studies are necessary to elucidate the underlying pathophysiology of the association between AD and VTE.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Risk of Venous Thromboembolism Among Patients with Atopic Dermatitis: A Cohort Study in a US Administrative Claims Database

Association of inflammatory skin diseases with venous thromboembolism in US adults

Controversial cardiovascular and hematologic comorbidities in atopic dermatitis, data availability.

The data of this study are available from the corresponding author upon reasonable request.

Langan SM, Irvine AD, Weidinger S (2020) Atopic dermatitis. Lancet 396(10247):345–360. https://doi.org/10.1016/s0140-6736(20)31286-1

Article   CAS   PubMed   Google Scholar  

Weidinger S et al (2018) Atopic dermatitis. Nat Rev Dis Primers 4(1). https://doi.org/10.1038/s41572-018-0001-z

Eichenfield LF et al (2014) Guidelines of care for the management of atopic dermatitis: Sect. 1. Diagnosis and assessment of atopic dermatitis. J Am Acad Dermatol 70(2):338–351. https://doi.org/10.1016/j.jaad.2013.10.010

Article   PubMed   Google Scholar  

Ascott A et al (2019) Atopic eczema and major cardiovascular outcomes: a systematic review and meta-analysis of population-based studies. J Allergy Clin Immunol 143(5):1821–1829. https://doi.org/10.1016/j.jaci.2018.11.030

Article   PubMed   PubMed Central   Google Scholar  

Silverwood RJ et al (2018) Severe and predominantly active atopic eczema in adulthood and long term risk of cardiovascular disease: population based cohort study. BMJ (clinical research ed. 361. https://doi.org/10.1136/bmj.k1786

Arshad N et al (2017) Recurrence and mortality after first venous thromboembolism in a large population-based cohort. J Thromb Haemostasis: JTH 15(2):295–303. https://doi.org/10.1111/jth.13587

Article   CAS   Google Scholar  

Cohen AT et al (2017) Epidemiology of first and recurrent venous thromboembolism in patients with active cancer. A population-based cohort study. Thromb Haemost 117(1):57–65. https://doi.org/10.1160/TH15-08-0686

Riva N, Donadini MP, Ageno W (2015) Epidemiology and pathophysiology of venous thromboembolism: similarities with atherothrombosis and the role of inflammation. Thromb Haemost 113(6):1176–1183. https://doi.org/10.1160/TH14-06-0563

Tamagawa-Mineoka R et al (2008) Elevated platelet activation in patients with atopic dermatitis and psoriasis: increased plasma levels of beta-thromboglobulin and platelet factor 4. Allergology International: Official J Japanese Soc Allergology 57(4):391–396. https://doi.org/10.2332/allergolint.O-08-537

Piazza G, Ridker PM (2015) Is venous thromboembolism a chronic inflammatory disease? Clin Chem 61(2):313–316. https://doi.org/10.1373/clinchem.2014.234088

Shaheen MS, Silverberg JI (2021) Association of inflammatory skin diseases with venous thromboembolism in US adults. Arch Dermatol Res 313(4):281–289. https://doi.org/10.1007/s00403-020-02099-6

Chen T-L et al (2022) Association of risk of incident venous thromboembolism with atopic dermatitis and treatment with Janus kinase inhibitors: a systematic review and Meta-analysis. JAMA Dermatology 158(11):1254–1261. https://doi.org/10.1001/jamadermatol.2022.3516

Merola JF et al (2024) Venous thromboembolism risk is lower in patients with atopic dermatitis than other immune-mediated inflammatory diseases: a retrospective, observational, comparative cohort study using US claims data. J Am Acad Dermatol 90(5):935–944. https://doi.org/10.1016/j.jaad.2023.12.027

Afonso J et al (2023) The perils of Misinterpreting and Misusing Publication Bias in Meta-analyses: an Education Review on Funnel plot-based methods. Sports Medicine (Auckland, N.Z.) https://doi.org/10.1007/s40279-023-01927-9

Meyers KJ et al (2021) Risk of venous thromboembolism among patients with atopic dermatitis: a Cohort Study in a US administrative claims database. Dermatology Therapy 11(3):1041–1052. https://doi.org/10.1007/s13555-021-00538-4

Zirpel H et al (2024) Atopic dermatitis is associated with an increased risk of cardiovascular diseases: a large-scale, propensity-matched US-based retrospective study. Clin Exp Dermatol. https://doi.org/10.1093/ced/llae164

Wan J et al (2023) Incidence of Cardiovascular Disease and venous thromboembolism in patients with atopic dermatitis. The Journal of Allergy and Clinical Immunology. Practice 11(10). https://doi.org/10.1016/j.jaip.2023.08.007

Warren RB et al (2023) The risk of venous thromboembolism in atopic dermatitis: a matched cohort analysis in UK primary care. Br J Dermatol 189(4):427–436. https://doi.org/10.1093/bjd/ljad212

Chen T-L et al (2023) Risk of venous thromboembolism among adults with atopic dermatitis. JAMA Dermatology 159(7):720–727. https://doi.org/10.1001/jamadermatol.2023.1300

Schneeweiss MC et al (2021) Incidence of venous thromboembolism in patients with dermatologist-diagnosed chronic inflammatory skin diseases. JAMA Dermatology 157(7):805–816. https://doi.org/10.1001/jamadermatol.2021.1570

Bello N et al (2023) Systematic Literature Review and Meta-analysis of venous thromboembolism events in systemic Lupus Erythematosus. Rheumatol Therapy 10(1). https://doi.org/10.1007/s40744-022-00513-1

Grainge MJ, West J, Card TR (2010) Venous thromboembolism during active disease and remission in inflammatory bowel disease: a cohort study. Lancet (London England) 375(9715):657–663. https://doi.org/10.1016/S0140-6736(09)61963-2

Chen T-L et al (2022) Association of Psoriasis With Incident venous thromboembolism and peripheral vascular disease: a systematic review and Meta-analysis. JAMA Dermatology 158(1):59–67. https://doi.org/10.1001/jamadermatol.2021.4918

Davis DMR et al (2022) American Academy of Dermatology Guidelines: awareness of comorbidities associated with atopic dermatitis in adults. J Am Acad Dermatol 86(6). https://www.jaad.org/article/S0190-9622(22)00080-9/fulltext

Brunner PM et al (2017) The atopic dermatitis blood signature is characterized by increases in inflammatory and cardiovascular risk proteins. Sci Rep 7(1):8707. https://doi.org/10.1038/s41598-017-09207-z

Article   CAS   PubMed   PubMed Central   Google Scholar  

Réau V et al (2021) Venous thrombosis and predictors of relapse in eosinophil-related diseases. Sci Rep 11(1):6388. https://doi.org/10.1038/s41598-021-85852-9

Nastałek M, Wojas-Pelc A, Undas A (2010) Plasma fibrin clot properties in atopic dermatitis: links between thrombosis and atopy. J Thromb Thrombolysis 30(2):121–126. https://doi.org/10.1007/s11239-010-0478-0

Wollenberg A et al (2020) Summary of efficacy, impact on work productivity and activity, and safety of baricitinib in moderateto-severe atopic dermatitis from two monotherapy phase III trials. British Journal of Dermatology, 183(SUPPL 1): p. 66. https://doi.org/10.1111/bjd.18968

Del Duca E et al (2023) Proteomic characterization of atopic dermatitis blood from infancy to adulthood. J Am Acad Dermatol 88(5):1083–1093. https://pubmed.ncbi.nlm.nih.gov/36773824/

Pavel AB et al (2020) The proteomic skin profile of moderate-to-severe atopic dermatitis patients shows an inflammatory signature. J Am Acad Dermatol 82(3):690–699. https://doi.org/10.1016/j.jaad.2019.10.039

He H et al (2021) Mild atopic dermatitis lacks systemic inflammation and shows reduced nonlesional skin abnormalities. J Allergy Clin Immunol 147(4):1369–1380. https://doi.org/10.1016/j.jaci.2020.08.041

Hedderson MM et al (2022) Rates of cardiovascular events among patients with moderate-to-severe atopic dermatitis in an integrated health care system: a retrospective cohort study. PLoS ONE 17(11):e. https://doi.org/10.1371/journal.pone.0277469

Silverberg JI et al (2020) Validation of five patient-reported outcomes for atopic dermatitis severity in adults. Br J Dermatol 182(1):104–111. https://doi.org/10.1111/bjd.18002

Download references

Acknowledgements

We thank Professor Adam J Adler for critically reading the manuscript.

This work was supported by the Fundamental Research Funds for the Central Universities, Grant/Award Numbers: YCJJ20230213.

Author information

Yifei Wang and Zhiqiang Chen contributed equally to this work.

Authors and Affiliations

Department of Dermatology, Union Hospital, Tongji Medical College, Huazhong University of Science and Technology, Wuhan, 430022, China

Yifei Wang, Ting He, Changzheng Huang & Chen Shen

Hubei Engineering Research Center for Skin Repair and Theranostics, Wuhan, 430022, China

Department of Vascular Surgery, Fuyang Hospital, Anhui Medical University, Fuyang, 236000, China

Zhiqiang Chen

You can also search for this author in PubMed   Google Scholar

Contributions

Data curation: Yifei Wang, Zhiqiang Chen, Ting He, Changzheng Huang, Chen Shen. Methodology: Yifei Wang, Zhiqiang Chen, Ting He, Changzheng Huang, Chen Shen. Formal analysis and investigation: Yifei Wang, Zhiqiang Chen, Ting He, Changzheng Huang, Chen Shen. Writing – original draft preparation: Yifei Wang, Zhiqiang Chen, Chen Shen, Changzheng Huang. Writing – review and editing: Yifei Wang, Chen Shen, Changzheng Huang. Funding acquisition: Yifei Wang. All authors read and approved the final version of the manuscript.

Corresponding authors

Correspondence to Changzheng Huang or Chen Shen .

Ethics declarations

Ethical approval.

The manuscript has been read and approved by all the authors. The requirements for authorship as stated earlier in this document have been met. All the authors believe that the manuscript represents honest.

Conflict of interest

The authors declare no conflict of interest regard- ing the publication of this paper.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Rights and permissions.

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Wang, Y., Chen, Z., He, T. et al. Risk of incident venous thromboembolism in patients with atopic dermatitis: systematic analysis of the literature and meta-analysis. J Thromb Thrombolysis (2024). https://doi.org/10.1007/s11239-024-03038-2

Download citation

Accepted : 21 August 2024

Published : 06 September 2024

DOI : https://doi.org/10.1007/s11239-024-03038-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Atopic dermatitis
• Atopic eczema
• Venous thromboembolism
• Deep vein thrombosis
• Pulmonary embolism
• Find a journal
• Publish with us
• Track your research

This paper is in the following e-collection/theme issue:

Published on 9.9.2024 in Vol 26 (2024)

This is a member publication of University of Bristol (Jisc)

Remote Patient Monitoring and Machine Learning in Acute Exacerbations of Chronic Obstructive Pulmonary Disease: Dual Systematic Literature Review and Narrative Synthesis

Authors of this article:

• Open access
• Published: 04 September 2024

Determinants of diarrhoeal diseases among under-five children in Africa (2013–2023): a comprehensive systematic review highlighting geographic variances, socioeconomic influences, and environmental factors

• Jember Azanaw 1 ,
• Asmamaw Malede 1 ,
• Hailemariam Feleke Yalew 1 &
• Eshetu Abera Worede 1  

BMC Public Health volume  24 , Article number:  2399 ( 2024 ) Cite this article

1 Altmetric

Metrics details

Diarrhea diseases continue to present a significant threat to the well-being of children under the age of five in Africa, thereby contributing substantially to both morbidity and mortality rates. The period spanning between January 2013 and December 2023 has witnessed persistent challenges in the fight against these diseases, thereby necessitating a thorough investigation into the factors that determine their occurrence. It is important to note that the burden of diarrhea diseases is not evenly distributed across the continent, with residence, socioeconomic, and environmental factors playing pivotal roles in shaping the prevalence and incidence rates. Consequently, this systematic review aimed to consolidate and analyze the existing body of literature on the determinants of diarrhea diseases among children under the age of five in Africa between January 2013 and December 2023.

The systematic review employed a rigorous methodological approach to examine the determinants of diarrhea diseases among children under the age of five in Africa between January 2013 and December 2023. A comprehensive search strategy was implemented, utilizing databases such as PubMed, Scopus, and Web of Science, and incorporating relevant keywords. The inclusion criteria focused on studies published within the specified timeframe, with a specific focus on the determinants of diarrhea disease among children under the age of five in Africa. The study selection process involved a two-stage screening, with independent reviewers evaluating titles, abstracts, and full texts to determine eligibility. The quality assessment, employing a standardized tool, ensured the inclusion of studies with robust methodologies. Data extraction encompassed key study details, including demographics, residence factors, socioeconomic influences, environmental variables, and intervention outcomes.

The search yielded a total of 12,580 articles across 25 African countries; however, only 97 of these articles met the inclusion criteria and were ultimately included in the systematic review. The systematic review revealed geographic and seasonal disparities in the prevalence of diarrhoeal diseases across different countries in Africa. Factors such as age-related vulnerabilities, gender disparities, maternal occupation, disposal of young children’s stools, and economic status were identified as significant determinants of the prevalence of diarrhea disease.

This systematic review provides a comprehensive understanding of the determinants of diarrhea diseases among children under the age of five in Africa between January 2013 and December 2023. The nuanced analysis of residence variations, socioeconomic influences, environmental factors, and intervention outcomes underscores the complex nature of this issue. The findings highlight the necessity for region-specific and context-sensitive interventions to address the unique challenges faced by diverse communities. This review serves as a valuable resource for policymakers, healthcare professionals, and researchers, guiding the development of evidence-based strategies aimed at reducing the burden of diarrhea diseases and improving child health outcomes in Africa.

Peer Review reports

Diarrhoeal diseases pose a persistent threat to the health of children under five, particularly in low- and middle-income countries [ 1 ]. In Africa, this challenge is notably severe, with an alarming number of reported cases annually [ 2 ]. According to the World Health Organization (WHO), diarrhoeal diseases account for a substantial proportion of child mortality across the continent [ 3 , 4 ]. This emphasizes the imperative need for comprehensive understanding and targeted interventions. Despite advancements in Africa, these diseases remain a significant public health challenge, disproportionately affecting vulnerable populations like under five children due to limited access to clean water, sanitation, and adequate healthcare resources [ 5 , 6 ].

Between January 2013 and December 2023, like other parts of the world, Africa underwent substantial socioeconomic, environmental, and healthcare transformations crucial for contributing to the reduction of diarrheal diseases [ 7 , 8 ]. The transition encompasses research efforts, the expansion of higher education institutions [ 9 ], enhanced water accessibility, the promotion of good hygiene practices, improved access to sanitation facilities, all supported by United Nations Children’s Fund (UNICEF), WHO and other nongovernmental organizations. [ 7 ] These transitions potentially influenced the determinants and prevalence of diarrhoeal diseases among children under five. However, rapid urbanization, sustainability problems, climate fluctuations, and varying healthcare infrastructures have shaped the disease landscape [ 10 ]. Disparities in economic development and resource access among African nations have resulted in differing disease burdens [ 11 ] including under-five children. Understanding the determinants of diarrhoeal diseases is fundamental for designing effective public health interventions a head of time [ 12 ]. Investigating multifaceted factors contributing to these diseases among under-five children in Africa holds significant importance, not only for healthcare providers and policymakers but also for global health initiatives [ 13 ].

The systematic review on diarrhoeal diseases will help understand crucial aspects of how these diseases affect under-five children in Africa. Evidences from various studies indicated the existence of common factors and national variations of diarrhoeal diseases due to socio-economic strata and environmental conditions difference. Moreover, a comprehensive examination of these determinants can shed light on the impact of public health policies and interventions on disease prevalence and outcomes over time [ 14 ]. Synthesizing findings from diverse studies was not only offer a comprehensive overview but also enable the identification of factors through time [ 15 ]. These factors could highlight areas needing further research attention or where interventions might be most impactful.

Through a systematic review between January 2013 and December 2023, this review aims to provide a comprehensive synthesis of current knowledge. It seeks to address existing gaps and offer insights critical for targeted interventions and policy formulation. The rationale for selecting the period between January 2013 and December 2023 for this systematic review is to capture the most recent and relevant data on the determinants of diarrheal diseases among under-five children in Africa. This timeframe allows for the inclusion of studies conducted after the implementation of several major public health initiatives and interventions aimed at improving child health and reducing diarrheal diseases in the region. By focusing on this period, the review aims to provide an up-to-date understanding of the current trends, emerging determinants, and the effectiveness of recent policies and interventions, ensuring that the findings are relevant to contemporary public health strategies and practices.

Search strategies

A systematic search was conducted across major academic databases, including PubMed, MEDLINE, Scopus, and Web of Science, utilizing a combination of keywords related to diarrhoeal diseases, under-five children, Africa, determinants, and interventions. Studies published between January 2013 and December 2023 were included to capture the most recent and relevant information. Eligibility criteria were defined to ensure the selection of high-quality studies, including peer-reviewed articles, systematic reviews, and meta-analyses.

A systematic search was conducted across major academic databases, including PubMed, Scopus, Web of Science, Embase, Google scholar, MEDLINE, and Cochrane Library utilizing a combination of keywords “diarrhea,” “Diarrhoea,” “under-five children,” “Africa,” “determinants,” “Risk Factors”, “Preschool Children”, “child health,” “socioeconomic factors,” and “water sanitation,” search terms used in collecting relevant articles. Authors use Boolean operators (AND, OR) to combine keywords and phrases for effective searches. That was (“Diarrhea“[Mesh]OR“Diarrhoeal Diseases“[Mesh]OR“Diarrhea”OR“Diarrhoea”AND (“Child“[Mesh]OR“Pediatrics“[Mesh]OR“Under-Five” OR“Children” AND (“Determinants” [Mesh]OR"RiskFactors“[Mesh]OR"Epidemiology“[Mesh]OR"Causality“[Mesh]OR “Determinants” OR “Risk Factors” OR “Causality” AND “Africa”. Only English language was used to filters out and retrieve relevant studies published within this specified timeframe.

Screening of eligible studies

Initial screening by titles and abstracts based on predefined inclusion criteria done by two team members (EA and JA) independently. Then disparities were resolved by discussion with other team member (AM) and agreements reached on the included articles for full texts screening. Exclude studies that clearly do not meet the scope of the review. Then we obtain and review the full texts of potentially relevant articles identified in the initial screening (by titles and abstracts) to assess their eligibility based on inclusion/exclusion criteria. The full text retrieved by other two reviewers (JA and EA) independently. Again, discrepancies were solved through discussion with other team member (HF). Then the search results were reported based on the Preferred Reporting Items for Systematic Review and Meta-analysis statement (PRISMA) guideline.

Inclusion and exclusion criteria

Articles any study design, done in African countries, focused at children under five, and on determinants of diarrhoeal diseases were included under this systematic review. Studies with inadequate or unclear methodologies, studies not focusing on determinants, reviews without original data, and studies not involving under-five children were excluded. Studies other than English language also were excluded.

Data extraction

Extraction of relevant data from included studies using a predefined template. Title, first author, country, publication year, study design, sample size, prevalence and study period were the data extracted from each study.

Study quality assessment

In this systematic review, quality evaluation involved scrutinizing the methodological rigor and risk of bias of the included studies. The Newcastle-Ottawa Scale was used to appraise the quality of each study. Two independent reviewers assessed various aspects of each study, such as the clarity of the research aims, appropriateness of the methodology and research design, recruitment strategy, data collection method, researcher-participant relationship, ethical considerations, data analysis, statement of findings, and overall value of the research [ 16 ]. Discrepancies encountered during the evaluation process were resolved through thorough discussion among the reviewers. If required, the perspective of a third reviewer was sought to ensure a comprehensive and unbiased resolution. The methodological quality of each study included in the analysis was meticulously assessed, employing a rating system that categorized studies as very good (9–10 points), good (7–8 points), satisfactory (5–6 points), or unsatisfactory (0–4 points). Then based on modified Newcastle-Ottawa Scale (NOS) specifically tailored for cross-sectional studies was utilized. Studies with a score of ≥ 7 out of 10 on this scale were deemed to have achieved high methodological quality [ 17 ]. Consequently, only studies falling within the categories of good and very good quality, as per the established rating criteria, were considered for inclusion in the final analysis. It is noteworthy that studies rated as very good quality, indicating a higher level of methodological rigor, were given special attention and were ultimately included in the conclusive analysis. This meticulous approach ensures that only studies meeting stringent quality standards contribute to the overall findings and conclusions of the research.

Synthesis of findings

The phase of data synthesis in the systematic review encompassed a meticulous and comprehensive procedure to amalgamate findings from a variety of studies pertaining to the factors that contribute to cases of diarrhea diseases among children under the age of five in Africa between January 2013 and December 2023. The qualitative synthesis furnished valuable insights into the contextual intricacies of these determinants, thereby illuminating the socioeconomic, breastfeeding and nutrient intake, and environmental aspects that exert an influence on diarrhea diseases. Consequently, a thematic analysis was conducted to identify recurring themes across the studies. This entailed extracting and categorizing data that pertained to similar determinants in order to facilitate a structured synthesis.

The approach used to estimate the overall pooled prevalence of diarrheal diseases involved conducting a systematic review of studies focusing on children under 5. The review compiled and synthesized prevalence from the data reported in each study, providing a comprehensive overview of diarrheal disease burden across different countries during the specified timeframe. The overall pooled prevalence and other analysis were subsequently estimated using Stata Version 17. In this review, the authors addressed heterogeneity by conducting subgroup analyses that grouped studies according to factors such as geographical variation, publication year, study season, study setting, sample size, and study design. This approach allowed to explore variations in effect size based on these characteristics.

Protocol registration

The review protocol was registered with the PROSPERO database through a registration number (PROSPERO- CRD42024500697).

Search results

By searching through different electronic websites, a total of 12,580 were identified. After screening and retrieving the systematic review finally encompassed a 97 pertinent studies published across African nations between January 2013 and December 2023 (Fig.  1 ).

PRISMA flow diagram for selection of studies in determinants of diarrhoea disease in Africa

Study characteristics

The studies included a range of sample sizes, from a maximum of 30,066 in Nigeria [ 18 ] to a minimum of 300 in North Sudan [ 19 ]. The overall 338,222 individuals in 25 African countries were included in the systematic review (Table  1 ; Fig.  2 ) The overall pooled prevalence of diarrhoeal diseases among under-five children in Africa is estimated to be 16.886% with 95% CI (16.747, 17.025) with the range of 7.500% from Nigeria [ 20 ] to 67.300% at South Africa [ 21 ] during this specified period.

African countries included in the systematic review of diarrhoeal disease determinants

Heterogeneity assessment

Due to the diversity in the time periods, geographic locations, sample sizes, season of study and study designs of the included studies, significant heterogeneity in prevalence of diarrheal disease was observed (Cochran’s Q Test p-value = 0.00, I² = 99.30%). Significant regional variations are comprehended, with East Africa (I² = 99.25%) and Southern Africa (I² = 99.89%) exhibiting the highest heterogeneity. These findings point to significant regional variations in the prevalence of diarrheal illnesses. According to the overall test for regional differences, which is highly significant (Q_b = 43.36, p  < 0.001), regional factors have a large impact on diarrheal disease estimates. Study settings may have an impact on prevalence estimates, according to the significant test for setting differences (Q_b = 17.95, p  < 0.001). Lastly, the analysis shows considerable variety by season, with spring having very little variability, minimal impact sizes, and mixed and summer seasons displaying high effect sizes and heterogeneity (Table  2 ).

Synthesized findings on determinants of diarrhoeal diseases among under-five children

Synthesizing the data from various investigations on diarrhea illnesses among children under the age of five in Africa between January 2013 and December 2023 revealed both similarities and differences. Twenty-three determinants consistently displayed connections with diarrhea diseases across multiple investigations, emphasizing their significant roles in disease incidence. Factors such as limited access to uncontaminated water, inadequate sanitation facilities, and low socio-economic status were recurring themes contributing to the prevalence of diarrhea diseases in this population.

However, the synthesis also exposed disparities among the investigations regarding the impacts of certain determinants. While most of the factors displayed consistent connections with diarrhea diseases across different circumstances, others demonstrated situation-specific effects. This variability underscores the intricate interplay of environmental, socio-economic, and cultural factors influencing the dynamics of diarrhea diseases in diverse African settings (Table  3 ).

The aim of this systematic review is to gain a comprehensive understanding of the persistent and predominant factors that contribute to diarrhea among children under the age of five in Africa. Factors such as age-related vulnerabilities, gender disparities, maternal occupation, the method of stool disposal by young children, economic influences, and environmental factors collectively contribute to the prevalence of diarrhea diseases. All these factors were themed after assessing from included studies. The economic status of households emerges as a critical determinant in the prevalence of diarrhoeal diseases among children under the age of five in Africa. This trend was consistently observed across 12 articles focusing on the wealth status of households [ 18 , 19 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 ]. Potential explanation for this correlation is that poverty, characterized by limited access to vital resources such as clean water, sanitation, and healthcare, functions as a significant risk factor [ 109 ]. The other reason could be families with lower income often encounter difficulties in providing adequate nutrition, maintaining hygiene, and ensuring timely medical care, thereby increasing children’s vulnerability to diarrhoeal infections. Economic constraints frequently impede access to healthcare services, including vaccination programs and medical treatment, amplifying the severity and duration of diarrhoeal episodes [ 110 ].

Another 25 studies suggests that maternal education significantly influences the prevalence of diarrhoeal diseases among children under the age of five in Africa [ 19 , 24 , 27 , 29 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 ]. It appears that a mother’s level of education is intricately linked to the overall well-being of her child, as educated mothers tend to adopt healthier practices. The other possible explanation for this is that well-educated mothers are more likely to possess knowledge about proper sanitation, hygiene, and nutrition, which are crucial elements in preventing diarrhoeal infections [ 111 ]. Their ability to understand and implement preventive healthcare measures, such as timely vaccinations and appropriate child feeding practices, contributes to reducing the risk of diarrhoeal diseases. Furthermore, maternal education in child health highlights the importance of investing in educational opportunities for women as a comprehensive strategy to improve the well-being of children under the age of five in Africa [ 112 ].

From the findings of five studies, the prevalence of diarrhea disease in children under the age of five in Africa is influenced by maternal occupation [ 20 , 45 , 56 , 58 , 59 ]. Maternal jobs, often tied to socio-economic status, can impact household living conditions and resource accessibility [ 113 ]. This indicated, maternal employment is intertwined with factors such as education and healthcare access, further influencing the susceptibility of under-five children to diarrhoeal diseases [ 114 ].

The age of the mother or caregiver emerges as a crucial determinant affecting diarrhea prevalence in young children in Africa, as revealed by findings from 12 studies included in this systematic review [ 20 , 33 , 35 , 41 , 42 , 43 , 50 , 52 , 53 , 54 , 55 , 56 , 57 ]. This variation could be explained by the unique challenges faced by younger mothers in childcare practices, potentially impacting hygiene routines and healthcare-seeking behavior [ 115 ]. Conversely, older caregivers may bring valuable experience but could encounter obstacles related to evolving childcare knowledge and changing health dynamics [ 116 ].

The evidence collected from 22 studies [ 20 , 25 , 27 , 29 , 30 , 31 , 36 , 40 , 41 , 42 , 48 , 54 , 56 , 60 , 61 , 62 , 63 , 64 , 65 ] highlights that the age of children under five is a crucial determinant affecting the prevalence of diarrhea in Africa, showcasing distinct patterns across various age groups [ 117 ]. Infants, especially those in their first year of life, are particularly vulnerable due to their developing immune systems and reliance on breastfeeding or formula feeding [ 118 ], exposing them to potential contamination from water sources or inadequate hygiene practices. While toddlers and preschoolers exhibit a certain level of resilience compared to infants, their exploratory behaviors still render them susceptible to contaminated environments. Additionally, interconnected factors such as weaning practices, nutritional status, and access to clean water and sanitation services contribute to shaping the age-specific burden of diarrhea [ 117 ].

The impact of gender on the prevalence of diarrhea among children under five in Africa is influenced by a complex interplay of biological, cultural, and societal factors, as indicated by findings from 11 research studies [ 19 , 33 , 41 , 42 , 49 , 56 , 58 , 66 , 67 , 68 ]. This might be due to differential care practices, nutritional disparities, and healthcare-seeking behavior may contribute to variations in diarrhea rates between boys and girls [ 119 , 120 ]. Moreover, societal norms and cultural expectations could differently influence access to sanitation facilities, exposure to environmental contaminants, and overall hygiene practices based on gender. This finding supported by the study conducted in India [ 121 ].

The prevalence of diarrhoeal diseases among children under the age of five is influenced by the number of children. 12 research consistently indicates that households with a higher number of children under five tend to experience elevated rates of diarrhoeal infections [ 20 , 25 , 27 , 29 , 30 , 31 , 36 , 40 , 41 , 42 , 48 , 54 , 56 , 60 , 61 , 62 , 63 , 64 , 65 ]. This correlation may be attributed to factors such as increased transmission opportunities within larger households, higher likelihood of shared exposure to contaminated environments, and potentially greater challenges in maintaining optimal hygiene practices. Additionally, the strain on resources in larger families, including difficulties in ensuring access to clean water, proper sanitation, and timely medical care, could contribute to the heightened susceptibility of children to diarrhoeal diseases.

Two studies conducted in Africa have explored the association between media exposure and the prevalence of diarrhea in this population [ 91 , 105 ]. The findings suggest that higher media exposure, particularly to health-related information through various channels, is associated with a potential decrease in the prevalence of diarrhea. This might be access to educational programs, public health campaigns, and information about proper hygiene practices through media platforms may contribute to improved knowledge and awareness among caregivers, leading to better preventive measures against diarrhoeal diseases [ 122 , 123 ].

Ensuring the adoption of optimal exclusive breastfeeding practices is essential in mitigating the prevalence of diarrhoeal diseases among children under five. This assertion is substantiated by 12 studies conducted in Africa [ 20 , 28 , 38 , 49 , 53 , 58 , 61 , 68 , 71 , 72 , 88 , 93 ]. The significance of this may stem from the fact that initiating breastfeeding within the first hour of birth and exclusively continuing it for the initial six months establishes a strong foundation for infants’ immune systems [ 124 ]. This, in turn, provides protection against various infections, including those caused by diarrhoeal pathogens [ 125 ]. The immunological components present in breast milk, such as antibodies and enzymes, play a crucial role in preventing and alleviating the impact of diarrhoeal illnesses [ 126 , 127 ].

Environmental factors exert effect on the prevalence of diarrhoeal diseases among children under five in Africa, carrying profound implications for public health [ 128 ]. The risk of diarrhoeal diseases among children under five is associated with unimproved toilet facilities and shared sanitation. Five studies revealed that the prevalence of diarrhea in children under the age of five in Africa is substantially impacted by insufficient access to proper toilet facilities and the prevalence of shared sanitation, highlighting these factors as critical contributors [ 30 , 59 , 80 , 100 , 103 ]. This can be attributed to the fact that, in many communities in developing countries, the absence of individual household toilets necessitates reliance on shared sanitation facilities, contributing to hygiene challenges and heightened disease transmission [ 129 , 130 ]. Shared facilities often lack proper maintenance, increasing the risk of fecal-oral contamination. Furthermore, the proximity of these shared toilet facilities to households may vary, impacting convenience and utilization rates. Inadequate access to toilet facilities, coupled with reliance on shared sanitation, escalates the risk of diarrhoeal diseases among young children, exposing them to contaminated surfaces or water sources [ 131 ].

The findings from eight studies put forward that the high prevalence of diarrhea among children under the age of five in Africa is notably exacerbated by the widespread practice of open defecation [ 24 , 34 , 35 , 38 , 52 , 58 , 93 , 103 ]. This is attributed to areas where inadequate sanitation is prevalent, open defecation becomes a common practice, leading to the contamination of water sources and the surrounding areas with fecal matter [ 132 ]. This combined impact of inadequate sanitation and open defecation presents a significant public health challenge, disproportionately affecting the under-five age group in Africa [ 133 ].

Based on findings from 35 studies, the choice of drinking water source has been identified as a determinant, with households relying on unimproved water sources experiencing higher disease prevalence [ 23 , 26 , 34 , 36 , 38 , 41 , 47 , 48 , 51 , 52 , 54 , 55 , 56 , 58 , 60 , 69 , 72 , 74 , 80 , 83 , 86 , 87 , 88 , 89 , 90 , 91 , 92 , 93 , 94 , 95 , 96 , 97 , 98 , 99 ]. This may be attributed to the microbial contamination of unimproved water sources by bacteria, viruses, and parasites, posing a significant health risk [ 134 , 135 ]. Consequently, ingesting pathogens through contaminated water can lead to gastrointestinal infections.

Conversely, insights from 13 studies emphasize that implementing water treatment at the household level in Africa is a crucial strategy to mitigate the incidence of diarrhea among children under five [ 23 , 34 , 49 , 52 , 55 , 59 , 67 , 68 , 74 , 82 , 90 , 100 , 101 ]. The rationale behind this is that employing point-of-use at household level water treatment methods, such as boiling, chlorination, solar disinfection or filtration, could significantly reduce the microbial contamination of drinking water [ 136 , 137 ]. Moreover, the integration of household water treatment aligns with broader efforts to improve water quality in resource-constrained settings where access to safe and clean water sources may be limited [ 138 , 139 ].

A review of seven studies underscores the association between improper disposal of the youngest child’s stools and an increased prevalence of diarrhea diseases [ 25 , 53 , 66 , 72 , 73 , 74 , 75 ]. This may be attributed to unhygienic practices, such as inadequate disposal of diapers or a lack of access to child-friendly sanitation facilities, further contributing to the spread of pathogens [ 140 , 141 ]. The consequences of these insufficient disposal practices are significant, elevating the risk of fecal-oral transmission and subsequent diarrhoeal infections among the vulnerable under-five Children.

Ineffective disposal practices of both liquid waste, as demonstrated by five studies [ 23 , 25 , 45 , 53 , 75 ], and solid waste, as evidenced by 16 studies [ 24 , 38 , 39 , 44 , 50 , 52 , 59 , 64 , 74 , 75 , 87 , 92 , 93 , 94 , 104 ], significantly contribute to the increased occurrence of diarrhea among children under the age of five. This may be due to insufficient sanitation, open defecation, and the pollution of water sources resulting from inadequate management of liquid and solid waste create environments that promote the transmission of diarrheal pathogens [ 142 ]. In addition, poorly handled solid waste, including actions like open dumping and burning, releases pollutants into the air and water, contaminating food and drinking water sources [ 143 ]. The sum of these unhygienic conditions significantly contribute to the prevalence of diarrhoeal diseases, posing a critical public health challenge for children under five in affected communities.

Based on findings from four studies, inadequate food handling and consumption practices emerge as noteworthy factors influencing the prevalence of diarrhoeal diseases among children under the age of four [ 31 , 34 , 61 , 99 ]. This could be attributed to caregivers’ insufficient hand washing, cross-contamination during food preparation, and the consumption of undercooked or contaminated foods, all of which contribute to the transmission of diarrhoeal pathogens [ 144 ]. Insufficient awareness regarding safe food handling practices, coupled with a lack of access to clean water for food preparation, intensifies diarrhoeal problem [ 143 ].

Residence determinants encompass a variety of contextual factors that differ across countries, shaping disease patterns and affecting healthcare accessibility. As highlighted by 20 studies [ 27 , 30 , 34 , 40 , 41 , 42 , 43 , 51 , 52 , 58 , 59 , 62 , 68 , 76 , 88 , 106 , 107 ], the urban-rural disparity in the factors influencing diarrhoeal diseases among children under the age of five in Africa between January 2013 and December 2023 underscores distinctive challenges and opportunities in these environments. This may be attributed to the fact that urban areas may enjoy enhanced access to sanitation infrastructure, healthcare services, and education, potentially leading to a reduction in the incidence of diarrhoeal diseases [ 122 ]. In contrast, rural areas often face constraints in accessing clean water sources, sanitation facilities, and healthcare, increasing vulnerability to diarrhoeal diseases.

The prevalence of diarrheal diseases among children under the age of five in Africa is closely linked to vaccination coverage. The consistent findings of nine studies suggest that increased vaccination coverage is strongly correlated with a significant decrease in the incidence of diarrheal diseases among this susceptible population [ 25 , 28 , 30 , 49 , 61 , 67 , 82 , 93 , 98 ].The possible explanation, immunizing vaccines that target specific pathogens, such as rotavirus and measles, play a crucial role in defending against severe diarrheal episodes, thereby reducing the risk of complications and potential fatalities [ 145 ].

According to eight studies, spatiotemporal variation in the occurrence of diarrheal diseases among under-five children in Africa reflects the dynamic interplay of geographic variation and temporal factors influencing disease patterns [ 18 , 41 , 42 , 96 , 106 , 108 ]. The prevalence of diarrheal diseases varies across regions due to differences in environmental conditions, access to clean water, sanitation facilities, and healthcare infrastructure [ 146 , 147 ]. Moreover, temporal variations of diarrhoeal disease may be attributed to seasonal changes, climate conditions impact water quality, hygiene practices, and disease transmission, leading to fluctuations in diarrhoeal diseases prevalence [ 148 ].

The first limitation of the review is publication bias not assessed. The second limitation is variations in study methodologies, and the reliance on available literature, which may not capture the full spectrum of determinants of diarrhoeal diseases among under-five children in Africa. Future research should prioritize longitudinal studies employing standardized methodologies, and explore emerging determinants, ultimately informing targeted interventions for reducing the burden of diarrhoeal diseases among under-five children in Africa. The third limitation is, since the majority of the included studies were conducted in Ethiopia, which may introduce bias due to the overrepresentation of studies from Ethiopia compared to others countries.

This systematic review provides a comprehensive understanding of the determinants of diarrhea diseases among children under the age of five in Africa between January 2013 and December 2023. The nuanced analysis of geographical variations, socioeconomic influences, environmental factors, and intervention outcomes underscores the complex nature of diarrhoeal disease. The findings highlight the necessity for region-specific and context-sensitive interventions to address the unique challenges faced by diverse communities. This review serves as a valuable resource for policymakers, healthcare professionals, and researchers, guiding the development of evidence-based strategies aimed at reducing the burden of diarrhea diseases and improving child health outcomes in Africa.

Data availability

This research was done using a publicly available dataset found at published works.

Fagbamigbe AF, Uthman AO, Ibisomi L. Hierarchical disentanglement of contextual from compositional risk factors of diarrhoea among under-five children in low-and middle-income countries. Sci Rep. 2021;11(1):8564.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Wareth G, et al. The perspective of antibiotic therapeutic challenges of brucellosis in the Middle East and North African countries: current situation and therapeutic management. Transbound Emerg Dis. 2022;69(5):e1253–68.

Article   CAS   PubMed   Google Scholar  

Ugboko HU et al. Childhood diarrhoeal diseases in developing countries. Heliyon, 2020;6(4).

Alijanzadeh Maliji B, et al. Role of the World Health Organization in Management of Gastrointestinal diseases caused by Contaminated Water in Children in the Middle East: a review article. J Pediatr Rev. 2023;11(1):59–66.

Article   Google Scholar  

Ezeneme EV et al. The influence of socioeconomic inequality on maternal and child health outcomes in Nigeria: A critical analysis. GSJ, 2023;11(10).

Rodrigues CM, Plotkin SA. Impact of vaccines; health, economic and social perspectives. Front Microbiol. 2020;11:1526.

Article   PubMed   PubMed Central   Google Scholar  

Poague KI, Blanford JI, Anthonj C. Water, sanitation and hygiene in schools in low-and middle-income countries: a systematic review and implications for the COVID-19 pandemic. Int J Environ Res Public Health. 2022;19(5):3124.

Silvestri G, et al. Transition management for improving the sustainability of WASH services in informal settlements in Sub-saharan Africa—An exploration. Sustainability. 2018;10(11):4052.

Daniels C, Gebhardt C. Higher education, science and research systems for transformative change in Africa–what role for innovation policy? 2021, SAGE Publications Sage UK: London, England. pp. 553–8.

Connolly C, Keil R, Ali SH. Extended urbanisation and the spatialities of infectious disease: demographic change, infrastructure and governance. Urban Stud. 2021;58(2):245–63.

Wei J, Rahim S, Wang S. Role of environmental degradation, institutional quality, and government health expenditures for human health: evidence from emerging seven countries. Front Public Health. 2022;10:870767.

Contreras JD, Eisenberg JN. Does basic sanitation prevent diarrhea? Contextualizing recent intervention trials through a historical lens. Int J Environ Res Public Health. 2020;17(1):230.

Obasohan PE. Investigating multiple overlaps in the determinants of risk factors of Anaemia, Malaria, and Malnutrition, and their Multimorbidity, among children aged 6 to 59 months in Nigeria. University of Sheffield; 2022.

Talic S et al. Effectiveness of public health measures in reducing the incidence of covid-19, SARS-CoV-2 transmission, and covid-19 mortality: systematic review and meta-analysis. BMJ, 2021;375.

Peters MD, et al. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Implement. 2021;19(1):3–10.

Article   PubMed   Google Scholar  

Norris JM, et al. A modified Newcastle-Ottawa scale for assessment of study quality in genetic urological research. Eur Urol. 2021;79(3):325–6.

Zang Y, Fan Y, Gao Z. Pretreatment C-reactive protein/albumin ratio for predicting overall survival in pancreatic cancer: a meta-analysis. Medicine, 2020;99(23).

Awoniyi FB, Neupane S. The socio-economic difference in the prevalence and treatment of diarrheal disease in children under five years across the geo-political zones in Nigeria. medRxiv. 2021. 2021.10. 30.21265706.

Elmanssury A, Elnadif D, Safa A. Prevalence of diarrhea and association with socio-demographic factors among children under five in Mayo Camp-Khartoum State Sudan. Pakistan J Med Health Sci. 2022;16(03):1100–1100.

Akinyemi YC. Spatial pattern and determinants of diarrhoea morbidity among under-five-aged children in Lagos State. Nigeria Cities Health. 2022;6(1):180–91.

Ntshangase SN, Ghuman S, Haffejee F. Diarrhoeal prevalence and handwashing practices of children attending early childhood development centres in KwaZulu-Natal, South Africa. Health SA Gesondheid (Online). 2022;27:1–8.

Google Scholar  

Girmay AM, et al. Associations of WHO/UNICEF Joint Monitoring Program (JMP) Water, Sanitation and Hygiene (WASH) Service Ladder service levels and sociodemographic factors with diarrhoeal disease among children under 5 years in Bishoftu town, Ethiopia: a cross-sectional study. BMJ open. 2023;13(7):e071296.

Soboksa NE, et al. Association between microbial water quality, sanitation and hygiene practices and childhood diarrhea in Kersa and Omo Nada districts of Jimma Zone, Ethiopia. PLoS ONE. 2020;15(2):e0229303.

Gebru T, Taha M, Kassahun W. Risk factors of diarrhoeal disease in under-five children among health extension model and non-model families in Sheko district rural community, Southwest Ethiopia: comparative cross-sectional study. BMC Public Health. 2014;14(1):1–6.

Alemayehu K, et al. Prevalence and determinants of diarrheal diseases among under-five children in Horo Guduru Wollega Zone, Oromia Region, Western Ethiopia: a community-based cross-sectional study. Can J Infect Dis Med Microbiol. 2021;2021:1–9.

Seidu R, et al. Risk factor analysis of diarrhoeal disease incidence in faecal sludge-applying farmers’ households in Tamale. Ghana J Water Sanitation Hygiene Dev. 2013;3(2):134–43.

Apanga PA, Kumbeni MT. Factors associated with diarrhoea and acute respiratory infection in children under-5 years old in Ghana: an analysis of a national cross-sectional survey. BMC Pediatr. 2021;21(1):1–8.

Getahun M, et al. Determinants of Diarrhea among under-five Children-Visiting Government Health Facilities in Nekemte Town, Western Ethiopia: unmatched Case Control Study. Appl Sci Res Periodicals. 2023;1(1):53–68.

Claudine U, et al. Association between sociodemographic factors and diarrhea in children under 5 years in Rwanda. Korean J Parasitol. 2021;59(1):61.

Oyedele O. An empirical analysis of the effects of Household demographics on Diarrhea Morbidity in Children aged 0 to 48 months in Namibia. Int J Social Determinants Health Health Serv. 2023;53(2):176–82.

Bitew BD, Getachew A, Azanaw J. Diarrhea Prevalence and Associated Factors among children in Azezo Sub-city, Northwest Ethiopia: A Community-based cross-sectional study. Am J Trop Med Hyg, 2023: p. tpmd220192–220192.

Mohammed S, Tilahun M, Tamiru D. Morbidity and associated factors of diarrheal diseases among under five children in Arba-Minch district, Southern Ethiopia, 2012. Sci J Public Health. 2013;1(2):102–6.

Kombat MY et al. Predictors of Diarrhea Among Children Under the Age of Five Years in Ghana: Multivariate analysis of data from the 2014 Ghana Demographic and Health Survey. 2023.

Melese M, et al. Prevalence of Diarrhea, Intestinal parasites, and Associated factors among under-five children in Dabat District, Northwest Ethiopia: Multicenter cross-sectional study. Environ Health Insights. 2023;17:11786302231174744.

Sinmegn Mihrete T, Asres Alemie G, Shimeka Teferra A. Determinants of childhood diarrhea among underfive children in Benishangul Gumuz regional state, north West Ethiopia. BMC Pediatr. 2014;14:1–9.

Woldu W, Bitew BD, Gizaw Z. Socioeconomic factors associated with diarrheal diseases among under-five children of the nomadic population in northeast Ethiopia. Trop Med Health. 2016;44(1):1–8.

Hashi A, Kumie A, Gasana J. Prevalence of diarrhoea and associated factors among under-five children in Jigjiga District, Somali Region, Eastern Ethiopia. Open J Prev Med. 2016;6(10):233–46.

Asfaha KF, et al. Determinants of childhood diarrhea in Medebay Zana District, Northwest Tigray, Ethiopia: a community based unmatched case–control study. BMC Pediatr. 2018;18:1–9.

Melese B, et al. Prevalence of diarrheal diseases and associated factors among under-five children in Dale District, Sidama Zone, Southern Ethiopia: a cross-sectional study. BMC Public Health. 2019;19(1):1–10.

Mbugua S et al. Determinants of diarrhea among young children under the age of five in Kenya, evidence from KDHS 2008-09. Afr Popul Stud, 2014: pp. 1046–56.

Afitiri A-R, et al. Cumulative effects of environmental factors on household childhood diarrhoea in Ghana. Water Pract Technol. 2020;15(4):1032–49.

Kombat MY, et al. Prevalence and predictors of diarrhea among children under five in Ghana. BMC Public Health. 2024;24(1):154.

Tampah-Naah AM. Maternal and child level factors associated with childhood (0–23 months) diarrhoea in Ghana: a pooled analysis of national representative datasets. Ghana J Dev Stud. 2019;16(2):157–77.

Guillaume DA, Justus OO, Ephantus KW. Factors influencing diarrheal prevalence among children under five years in Mathare Informal Settlement, Nairobi, Kenya. J Public Health Afr. 2020;11(1).

Nwaoha AF, Ohaeri CC, Amaechi EC. Prevalence of diarrhoea, and associated risk factors, in children aged 0–5 years, at two hospitals in Umuahia, Abia, Nigeria. UNED Res J. 2017;9(1):7–14.

Bah D et al. Prevalence and risk factors of diarrheal diseases in Sierra Leone, 2019: a cross-sectional study. Pan Afr Med J. 2022;41(1).

Davies K, et al. Reducing child mortality in Sudan by preventing diarrheal disease. Columbia Univ J Global Health. 2014;4(1):1–5.

Aderinwale O et al. Diarrhea and its Determinants in Under-five Children in Chad Republic. 2023.

William GB, Kariuki JG, Kerochi A. Risk factors for Acute Watery Diarrhea among the Under five in Bentiu Protection of Civilian’s site, Unity State, South Sudan. J Int Bus Innov Strategic Manage. 2023;7(1):1–17.

Mebrahtom S, Worku A, Gage DJ. The risk of water, sanitation and hygiene on diarrhea-related infant mortality in eastern Ethiopia: a population-based nested case-control. BMC Public Health. 2022;22(1):1–14.

Omona S, et al. Prevalence of diarrhoea and associated risk factors among children under five years old in Pader District, northern Uganda. BMC Infect Dis. 2020;20:1–9.

Workie GY, Akalu TY, Baraki AG. Environmental factors affecting childhood diarrheal disease among under-five children in Jamma district, South Wello Zone, Northeast Ethiopia. BMC Infect Dis. 2019;19:1–7.

Baye A, et al. Priorities for intervention to prevent diarrhea among children aged 0–23 months in northeastern Ethiopia: a matched case-control study. BMC Pediatr. 2021;21(1):1–11.

Nantege R, et al. Prevalence and factors associated with diarrheal diseases among children below five years in selected slum settlements in Entebbe municipality, Wakiso district, Uganda. BMC Pediatr. 2022;22(1):1–8.

Diouf K, et al. Diarrhoea prevalence in children under five years of age in rural Burundi: an assessment of social and behavioural factors at the household level. Global Health Action. 2014;7(1):24895.

Moon J et al. Risk factors of diarrhea of children under five in Malawi: based on Malawi demographic and Health Survey 2015–2016. J Global Health Sci. 2019;1(2).

Chari S, Mbonane TP, Van Wyk RH. Social and Environmental Determinants of Diarrheal Diseases among Children Under Five Years in Epworth Township, Harare. 2023.

Girmay AM et al. Associations of WHO/UNICEF Joint Monitoring Program (JMP) Water, Sanitation and Hygiene (WASH) Service Ladder service levels and sociodemographic factors with diarrhoeal disease among children under 5 years in Bishoftu town, Ethiopia: a cross-sectional study. BMJ open. 2023;13(7).

Thiam S, et al. Prevalence of diarrhoea and risk factors among children under five years old in Mbour, Senegal: a cross-sectional study. Infect Dis Poverty. 2017;6(04):43–54.

Rukambile E, et al. Determinants of diarrhoeal diseases and height-for-age z-scores in children under five years of age in rural central Tanzania. J Prev Med Hyg. 2020;61(3):E409.

PubMed   PubMed Central   Google Scholar  

Gizaw Z, Woldu W, Bitew BD. Child feeding practices and diarrheal disease among children less than two years of age of the nomadic people in Hadaleala District, Afar Region, Northeast Ethiopia. Int Breastfeed J. 2017;12:1–10.

Siziya S, Muula AS, Rudatsikira E. Correlates of diarrhoea among children below the age of 5 years in Sudan. Afr Health Sci. 2013;13(2):376–83.

CAS   PubMed   PubMed Central   Google Scholar  

Wasihun AG, et al. Risk factors for diarrhoea and malnutrition among children under the age of 5 years in the Tigray Region of Northern Ethiopia. PLoS ONE. 2018;13(11):e0207743.

Solomon ET, et al. Diarrheal morbidity and predisposing factors among children under 5 years of age in rural East Ethiopia. Trop Med Health. 2020;48(1):1–10.

Mansour A, et al. Modifiable diarrhoea risk factors in Egyptian children aged < 5 years. Epidemiol Infect. 2013;141(12):2547–59.

Soboksa NE. Associations between improved water supply and sanitation usage and childhood diarrhea in Ethiopia: an analysis of the 2016 demographic and health survey. Environ Health Insights. 2021;15:11786302211002552.

Nsubuga EJ, et al. Factors Associated with Acute Watery Diarrhea among children aged 0–59 months in Obongi District, Uganda, April 2022: a case–control study. Uganda; April, 2022.

Atari DO, Mkandawire P, Lukou YS. Determinants of co-occurrences of diarrhea and fever among children under five years of age in South Sudan. Afr Geographical Rev. 2023;42(5):617–33.

Natnael T, Lingerew M, Adane M. Prevalence of acute diarrhea and associated factors among children under five in semi-urban areas of northeastern Ethiopia. BMC Pediatr. 2021;21(1):290.

Tambe A, Nzefa L, Nicoline N. Childhood Diarrhea determinants in Sub-saharan Africa: A Cross Sectional Study of Tiko-Cameroon. Challenges. 2015;6:229–43.

Dagnew AB, et al. Prevalence of diarrhea and associated factors among under-five children in Bahir Dar City, Northwest Ethiopia, 2016: a cross-sectional study. BMC Infect Dis. 2019;19:1–7.

Getahun W, Adane M. Prevalence of acute diarrhea and water, sanitation, and hygiene (WASH) associated factors among children under five in Woldia Town, Amhara Region, northeastern Ethiopia. BMC Pediatr. 2021;21(1):1–15.

Fenta A, Alemu K, Angaw DA. Prevalence and associated factors of acute diarrhea among under-five children in Kamashi district, western Ethiopia: community-based study. BMC Pediatr. 2020;20:1–7.

Mernie G, Kloos H, Adane M. Prevalence of and factors associated with acute diarrhea among children under five in rural areas in Ethiopia with and without implementation of community-led total sanitation and hygiene. BMC Pediatr. 2022;22(1):1–16.

Hassan KE, et al. The impact of household hygiene on the risk of bacterial diarrhea among Egyptian children in rural areas, 2004–2007. J Infect Developing Ctries. 2014;8(12):1541–51.

Anteneh ZA, Andargie K, Tarekegn M. Prevalence and determinants of acute diarrhea among children younger than five years old in Jabithennan District, Northwest Ethiopia, 2014. BMC Public Health. 2017;17(1):1–8.

Bizuneh H, et al. Factors associated with diarrheal morbidity among under-five children in Jigjiga town, Somali Regional State, eastern Ethiopia: a cross-sectional study. BMC Pediatr. 2017;17:1–7.

Shine S, et al. Prevalence and associated factors of diarrhea among under-five children in Debre Berhan town, Ethiopia 2018: a cross sectional study. BMC Infect Dis. 2020;20:1–6.

Regassa W, Lemma S. Assessment of diarrheal disease prevalence and associated risk factors in children of 6–59 months old at Adama District rural Kebeles, eastern Ethiopia, January/2015. Ethiop J Health Sci. 2016;26(6):581–8.

Nwokoro UU et al. Water, sanitation and hygiene risk factors associated with diarrhoea morbidity in a rural community of Enugu, South East Nigeria. Pan Afr Med J. 2020;37(1).

Danquah L et al. Sanitation and hygiene practices in relation to childhood diarrhoea prevalence: the case of households with children under-five years in Ghana. 2014.

Nguyen TYC, et al. Diarrhoea among children aged under five years and risk factors in informal settlements: a cross-sectional study in Cape Town, South Africa. Int J Environ Res Public Health. 2021;18(11):6043.

Bennion N, et al. Association between WASH-related behaviors and knowledge with childhood diarrhea in Tanzania. Int J Environ Res Public Health. 2021;18(9):4681.

Rukambile E, Practices H, et al. Water Supply, Sanitation, and Childhood Diarrhoea in Resource-Poor settings of Rural Central Tanzania: a mixed-method study. East Afr Sci. 2023;5(1):48–66.

Moon J et al. Risk factors of diarrhea of children under five in Malawi: based on Malawi demographic and Health Survey. J Glob Health Sci. 2019;1(2).

Jayte M, Dahir M, Incidence of Acute Watery Diarrhea among Internally Displaced People in Burhakaba Camps, Bay Region, Somalia. medRxiv. 2023. 2023.09. 10.23295320.

Asgedom AA, et al. Unimproved water and sanitation contributes to childhood diarrhoea during the war in Tigray, Ethiopia: a community based assessment. Sci Rep. 2023;13(1):7800.

Yaya S, et al. Improving water, sanitation and hygiene practices, and housing quality to prevent diarrhea among under-five children in Nigeria. Trop Med Infect Disease. 2018;3(2):41.

Dharod JM, et al. Examination of the Cameroon DHS data to investigate how water access and sanitation services are related to diarrhea and nutrition among infants and toddlers in rural households. J Water Health. 2021;19(6):1030–8.

Getachew B, et al. Factors associated with acute diarrhea among children aged 0–59 months in Harar town, eastern Ethiopia. East Afr J Health Biomedical Sci. 2018;2(1):26–35.

Mulatu G, et al. Association of drinking water and environmental sanitation with diarrhea among under-five children: evidence from Kersa demographic and health surveillance site, eastern Ethiopia. Front Public Health. 2022;10:962108.

McClelland PH, et al. Improved water and waste management practices reduce Diarrhea risk in children under age five in rural Tanzania: a community-based, cross-sectional analysis. Int J Environ Res Public Health. 2022;19(7):4218.

Alemayehu M, Alemu T, Astatkie A. Prevalence and determinants of diarrhea among under-five children in Benna Tsemay district, south omo zone, southern Ethiopia: a community-based cross-sectional study in pastoralist and agropastoralist context. Adv Public Health. 2020;2020:1–11.

Bushen G, Merga H, Tessema F. Effects of community-led total sanitation and hygiene implementation on diarrheal diseases prevention in children less than five years of age in South Western Ethiopia: a quasi-experimental study. PLoS ONE. 2022;17(4):e0265804.

Pessoa Colombo V, et al. Spatial distributions of diarrheal cases in relation to Housing conditions in Informal settlements: a cross-sectional study in Abidjan, Côte d’Ivoire. J Urb Health. 2023;100(5):1074–86.

Asare EO, Warren JL, Pitzer VE. Spatiotemporal patterns of diarrhea incidence in Ghana and the impact of meteorological and socio-demographic factors. Front Epidemiol. 2022;2:871232.

Abu M, Codjoe SNA. Experience and future perceived risk of floods and diarrheal disease in urban poor communities in Accra, Ghana. Int J Environ Res Public Health. 2018;15(12):2830.

Birungi S et al. Prevalence and factors associated with diarrhea among children less than five years old in Kabarole district, Uganda. 2016.

Kapwata T, et al. Diarrhoeal disease in relation to possible household risk factors in South African villages. Int J Environ Res Public Health. 2018;15(8):1665.

Thiam S et al. Prevalence of diarrhoea and risk factors among children under five years old in Mbour, Senegal: a cross-sectional study. 2018.

Chilambwe M, Mulenga D, Siziya S. Diarrhoea prevalence in under-five children in two urban populations setting of Ndola, Zambia: an assessment of knowledge and attitude at the household level. J Infect Dis Ther. 2015;3(227):2332–0877.

Tambe AB, Nzefa DL, Nicoline NA. Childhood diarrhea determinants in sub-saharan Africa: a cross sectional study of Tiko-Cameroon. Challenges. 2015;6(2):229–43.

Pickering AJ, et al. Effect of a community-led sanitation intervention on child diarrhoea and child growth in rural Mali: a cluster-randomised controlled trial. Lancet Global Health. 2015;3(11):e701–11.

Daffe ML, et al. Water, sanitation, and hygiene access in Senegal and its impact on the occurrence of diarrhea in children under 5 years old. J Water Health. 2022;20(11):1654–67.

Birhan TA, et al. Prevalence of diarrheal disease and associated factors among under-five children in flood-prone settlements of Northwest Ethiopia: a cross-sectional community-based study. Front Pead. 2023;11:1056129.

Fagbamigbe AF, Morakinyo OM, Abatta E. Analysis of regional variations in influence of household and environmental characteristics on prevalence of diarrhoea among under-five children in Nigeria. Volume 7. Annals of Medical and Health Sciences Research| May-June; 2017. 3.

Lanyero H, et al. Antibiotic use among children under five years with diarrhea in rural communities of Gulu, northern Uganda: a cross-sectional study. BMC Public Health. 2021;21:1–9.

Raza O, et al. Exploring spatial dependencies in the prevalence of childhood diarrhea in Mozambique using global and local measures of spatial autocorrelation. Med J Islamic Repub Iran. 2020;34:59.

Obaideen K, et al. The role of wastewater treatment in achieving sustainable development goals (SDGs) and sustainability guideline. Energy Nexus. 2022;7:100112.

Mokomane M, et al. The global problem of childhood diarrhoeal diseases: emerging strategies in prevention and management. Therapeutic Adv Infect Disease. 2018;5(1):29–43.

Gathogo LW. Determinants of Home Management of Diarrhea among Care Givers of children below 5 years in Ngandu Location. Nyeri County: JKUAT-COHES; 2021.

Bliznashka L, et al. Associations between women’s empowerment and child development, growth, and nurturing care practices in sub-saharan Africa: a cross-sectional analysis of demographic and health survey data. PLoS Med. 2021;18(9):e1003781.

Guhn M, et al. Associations of birth factors and socio-economic status with indicators of early emotional development and mental health in childhood: a population-based linkage study. Child Psychiatry Hum Dev. 2020;51:80–93.

Sumampouw OJ, Nelwan JE, Rumayar AA. Socioeconomic factors associated with diarrhea among under-five children in Manado Coastal Area, Indonesia. J Global Infect Dis. 2019;11(4):140.

Anderson EW, White KM. This is what family does: the family experience of caring for serious illness. Am J Hospice Palliat Medicine®. 2018;35(2):348–54.

Larkin M, Henwood M, Milne A. Carer-related research and knowledge: findings from a scoping review. Health Soc Care Commun. 2019;27(1):55–67.

Momberg D, et al. Water, sanitation and hygiene (WASH) in sub-saharan Africa and associations with undernutrition, and governance in children under five years of age: a systematic review. J Dev Origins Health Disease. 2021;12(1):6–33.

Article   CAS   Google Scholar  

Dawod B, Marshall JS, Azad MB. Breastfeeding and the developmental origins of mucosal immunity: how human milk shapes the innate and adaptive mucosal immune systems. Curr Opin Gastroenterol. 2021;37(6):547–56.

Mahmud I et al. Gender disparity in care-seeking behaviours and treatment outcomes for dehydrating diarrhoea among under-5 children admitted to a diarrhoeal disease hospital in Bangladesh: an analysis of hospital-based surveillance data. BMJ open. 2020;10(9).

Jarman AF, et al. Sex and gender differences in acute pediatric diarrhea: a secondary analysis of the Dhaka study. J Epidemiol Global Health. 2018;8(1–2):42.

Mallick R, Mandal S, Chouhan P. Impact of sanitation and clean drinking water on the prevalence of diarrhea among the under-five children in India. Child Youth Serv Rev. 2020;118:105478.

Morse T, et al. Developing a contextually appropriate integrated hygiene intervention to achieve sustained reductions in diarrheal diseases. Sustainability. 2019;11(17):4656.

Billah SM et al. Bangladesh: a success case in combating childhood diarrhoea. J Global Health. 2019;9(2).

Camacho-Morales A, et al. Breastfeeding contributes to physiological immune programming in the newborn. Front Pead. 2021;9:744104.

Pérez-Escamilla R et al. Perspective: should exclusive breastfeeding still be recommended for 6 months? Advances in Nutrition, 2019. 10(6): pp. 931–43.

Atyeo C, Alter G. The multifaceted roles of breast milk antibodies. Cell. 2021;184(6):1486–99.

Davis EC, et al. Gut microbiome and breast-feeding: implications for early immune development. J Allergy Clin Immunol. 2022;150(3):523–34.

Robert E, et al. Environmental determinants of E. Coli, link with the diarrheal diseases, and indication of vulnerability criteria in tropical West Africa (Kapore, Burkina Faso). PLoS Negl Trop Dis. 2021;15(8):e0009634.

Kumwenda S. Challenges to hygiene improvement in developing countries. Volume 1. IntechOpen London, UK; 2019.

Osumanu IK, Kosoe EA, Ategeeng F. Determinants of open defecation in the Wa municipality of Ghana: empirical findings highlighting sociocultural and economic dynamics among households. Journal of environmental and public health, 2019. 2019.

Baker KK, et al. Fecal fingerprints of enteric pathogen contamination in public environments of Kisumu, Kenya, associated with human sanitation conditions and domestic animals. Volume 52. Environmental science & technology; 2018. pp. 10263–74. 18.

Jubril I. The Effect of Open Defecation Practice in Saki Town, Nigeria. 2022.

Belay DG, et al. Open defecation practice and its determinants among households in sub-saharan Africa: pooled prevalence and multilevel analysis of 33 sub-saharan Africa countries demographic and health survey. Trop Med Health. 2022;50(1):28.

Gwimbi P, George M, Ramphalile M. Bacterial contamination of drinking water sources in rural villages of Mohale Basin, Lesotho: exposures through neighbourhood sanitation and hygiene practices. Environ Health Prev Med. 2019;24:1–7.

Sun S, Han J. Open defecation and squat toilets, an overlooked risk of fecal transmission of COVID-19 and other pathogens in developing communities. Environ Chem Lett. 2021;19:787–95.

Treacy J. Drinking water treatment and challenges in developing countries. The relevance of hygiene to health in developing countries, 2019: pp. 55–77.

Malan A, Sharma HR. Assessment of drinking water quality and various household water treatment practices in rural areas of Northern India. Arab J Geosci. 2023;16(1):96.

Wandera EA, et al. Impact of integrated water, sanitation, hygiene, health and nutritional interventions on diarrhoea disease epidemiology and microbial quality of water in a resource-constrained setting in Kenya: a controlled intervention study. Tropical Med Int Health. 2022;27(8):669–77.

Akamanya R. Towards an advanced water service level: Microbial drinking water quality management at government schools and healthcare facilities in resource-constrained rural settings, Kabarole Uganda. Loughborough University.

Agestika L. The association of child feces disposal, and Water, Sanitation, and Hygiene (WASH) toward child health in the urban slum, Indonesia. Hokkaido University; 2019.

Bauza V, et al. Child defecation and feces disposal practices and determinants among households after a combined household-level piped water and sanitation intervention in rural Odisha, India. Am J Trop Med Hyg. 2019;100(4):1013.

Ozoh A, et al. Indiscriminate solid waste disposal and problems with water-polluted urban cities in Africa. J Coastal Zone Manage. 2021;24(S5):1000005.

Srivastava R. Solid waste management and its impact on the environment, in Handbook of research on environmental and human health impacts of plastic pollution. 2020, IGI Global. pp. 389–400.

Geda NR. Inequalities in child care practices and health outcomes in Ethiopia. University of Saskatchewan; 2021.

Gagneux-Brunon A, et al. Vaccines for healthcare-associated infections: present, future, and expectations. Expert Rev Vaccines. 2018;17(5):421–33.

Dimitrova A, et al. Precipitation variability and risk of infectious disease in children under 5 years for 32 countries: a global analysis using demographic and Health Survey data. Lancet Planet Health. 2022;6(2):e147–55.

Chaurasia H, Srivastava S, Singh JK. Does seasonal variation affect diarrhoea prevalence among children in India? An analysis based on spatial regression models. Child Youth Serv Rev. 2020;118:105453.

Aik J, Ong J, Ng L-C. The effects of climate variability and seasonal influence on diarrhoeal disease in the tropical city-state of Singapore–A time-series analysis. Int J Hyg Environ Health. 2020;227:113517.

Download references

There is no specific grant for doing this research.

Author information

Authors and affiliations.

Department of Environmental and Occupational Health and Safety, Institute of Public Health, College of Medicine and Health Sciences, University of Gondar, Gondar, Ethiopia

Jember Azanaw, Asmamaw Malede, Hailemariam Feleke Yalew & Eshetu Abera Worede

You can also search for this author in PubMed   Google Scholar

Contributions

Data curation: Jember Azanaw and Eshetu Abera Worede. Formal analysis: Jember Azanaw. Investigation: Jember Azanaw, Eshetu Abera Worede. Methodology: Jember Azanaw, Eshetu Abera Worede, Asmamaw Malede, and Hailemariam Feleke Yalew. Software: Jember Azanaw, Eshetu Abera Worede Validation: Jember Azanaw, Eshetu Abera Worede, WoredeAsmamaw Malede, and Hailemariam Feleke Yalew. Visualization: Jember Azanaw, Eshetu Abera Worede, Asmamaw Malede, and Hailemariam Feleke Yalew. Write-up – Jember Azanaw, Eshetu Abera Worede. Review & editing: Jember Azanaw, Eshetu Abera Worede, Hailemariam Feleke Yalew., and Asmamaw Malede.

Corresponding author

Correspondence to Jember Azanaw .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Patient and public involvement.

There was no direct involvement of patients or public.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, supplementary material 4, supplementary material 5, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Azanaw, J., Malede, A., Yalew, H.F. et al. Determinants of diarrhoeal diseases among under-five children in Africa (2013–2023): a comprehensive systematic review highlighting geographic variances, socioeconomic influences, and environmental factors. BMC Public Health 24 , 2399 (2024). https://doi.org/10.1186/s12889-024-19962-0

Download citation

Received : 22 February 2024

Accepted : 02 September 2024

Published : 04 September 2024

DOI : https://doi.org/10.1186/s12889-024-19962-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Diarrhoeal diseases
• Determinants
• Risk factors

BMC Public Health

ISSN: 1471-2458

• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Front Vet Sci

Levels of Evidence, Quality Assessment, and Risk of Bias: Evaluating the Internal Validity of Primary Research

Jan m. sargeant.

1 Department of Population Medicine, Ontario Veterinary College, University of Guelph, Guelph, ON, Canada

Marnie L. Brennan

2 Centre for Evidence-Based Veterinary Medicine, School of Veterinary Medicine and Science, University of Nottingham, Sutton Bonington Campus, Loughborough, United Kingdom

Annette M. O'Connor

3 Department of Large Animal Clinical Sciences, College of Veterinary Medicine, Michigan State University, East Lansing, MI, United States

Clinical decisions in human and veterinary medicine should be based on the best available evidence. The results of primary research are an important component of that evidence base. Regardless of whether assessing studies for clinical case management, developing clinical practice guidelines, or performing systematic reviews, evidence from primary research should be evaluated for internal validity i.e., whether the results are free from bias (reflect the truth). Three broad approaches to evaluating internal validity are available: evaluating the potential for bias in a body of literature based on the study designs employed (levels of evidence), evaluating whether key study design features associated with the potential for bias were employed (quality assessment), and applying a judgement as to whether design elements of a study were likely to result in biased results given the specific context of the study (risk of bias assessment). The level of evidence framework for assessing internal validity assumes that internal validity can be determined based on the study design alone, and thus makes the strongest assumptions. Risk of bias assessments involve an evaluation of the potential for bias in the context of a specific study, and thus involve the least assumptions about internal validity. Quality assessment sits somewhere between the assumptions of these two. Because risk of bias assessment involves the least assumptions, this approach should be used to assess internal validity where possible. However, risk of bias instruments are not available for all study designs, some clinical questions may be addressed using multiple study designs, and some instruments that include an evaluation of internal validity also include additional components (e.g., evaluation of comprehensiveness of reporting, assessments of feasibility or an evaluation of external validity). Therefore, it may be necessary to embed questions related to risk of bias within existing quality assessment instruments. In this article, we overview the approaches to evaluating internal validity, highlight the current complexities, and propose ideas for approaching assessments of internal validity.

Introduction

Every day in clinical practice, veterinary professionals need to make decisions ranging from a decision as to whether (or not) to use an intervention or to apply a diagnostic test, to decisions about the overall management of complex clinical conditions. Increasingly, it is expected that clinical decisions are evidence-based. Evidence-based veterinary medicine incorporates clinician experience, client preferences, animal needs, and scientific evidence when making clinical decisions ( 1 ). In this approach, scientific evidence is obtained from relevant research. When research-based evidence does not exist, other sources of evidence, such as expert opinion may need to be used. Traditional narrative reviews provide an overview of a topic, and thus may be an attractive way of quickly acquiring knowledge for making clinical decisions. However, narrative reviews generally do not provide information on the identification and selection of the primary research being summarized (if any), the methodological quality of the studies, or the magnitude of the expected effect ( 2 , 3 ).

Formal methods have been developed to systematically identify, select, and synthesize the available evidence to assist veterinary professionals in evidence-based decision-making. These include critically appraised topics (CATs) ( 4 ), systematic review and meta-analysis (SR-MA) ( 5 – 7 ), and clinical practice guidelines ( 8 ) (see Box 1 for a short overview of these methods). These evidence synthesis approaches have different purposes which results in different processes and endpoints, but each includes an assessment of the internal validity of the research used. Critical appraisal of an individual study also includes an evaluation of internal validity, in addition to an evaluation of feasibility and generalizability ( 10 ). The evaluation of internal validity is the focus of this article. Understanding the different ways internal validity can be assessed, and the assumptions associated with these approaches, is necessary for researchers evaluating internal validity, and for veterinary professionals to assess studies for integration of evidence into practice.

Overview of synthesis methods used in veterinary practice and research.

Systematic review, meta-analysis, and network meta-analysis: Systematic review is a structured methodology for identifying, selecting and evaluating all relevant research to address a structured question, which may relate to descriptive characteristics such as prevalence, etiology, efficacy of interventions, or diagnostic test accuracy ( 5 ). Meta-analysis is the statistical combination of results from multiple studies. For addressing questions on intervention efficacy, meta-analysis provides an overall effect size for pairwise comparisons between two intervention groups. Network meta-analysis allows an estimation of the comparative efficacy across all available intervention options ( 6 ), which may provide more relevant information for veterinary professionals when there are multiple intervention options available. However, systematic reviews with pairwise meta-analysis or network meta-analysis require that a body of research exists that can be synthesized to address a clinical question and can also be resource and time intensive to conduct. Therefore, there are many clinical questions for which formally synthesized research summaries do not exist.

Critically appraised topics: Critically appraised topics (CATs) use the same principles as systematic reviews to address clinical questions but employ a more rapid approach, particularly in relation to the screening and summation of the evidence. They were designed to be employed by clinicians as a way of rapidly gathering and interpreting evidence on clinical questions relating to specific cases ( 4 ). Therefore, there is a greater risk that research addressing the question may be missed. However, in the absence of a well conducted systematic review or meta-analysis, CATs can provide a faster evaluation of research addressing a clinical question and can be undertaken by veterinary professionals who may have fewer resources and potentially less methodological or statistical expertise, particularly if they are freely available and accessible.

Clinical practice guidelines: Veterinary professionals often are involved in the management of complex clinical conditions, where an array of questions need to be addressed, including those related to etiology, prognosis, diagnostic test accuracy, and intervention efficacy. Clinical practice guidelines are intended to assist healthcare professionals in assessing more than one aspect of case approach, including appropriate prevention, diagnosis, treatment, or clinical management of diseases, disorders, and other health conditions ( 9 ). Although there are differences in the methods among authors and institutions, the key elements of guideline development include the establishment of a multidisciplinary working group to develop the guidelines, the involvement of appropriate stakeholders, identification of the topic area, systematic searches for research evidence, assessment of the internal validity of studies comprising the evidence base, a process for drafting recommendations, and ongoing review and updating of the guidelines as new evidence becomes available ( 8 ).

Internal validity refers to the extent to which the study results reflect the true state of nature (i.e., whether the effect size estimated in a study is free from systematic error, also called bias) ( 11 ). Although there are a large number of named biases ( 12 ), for studies that assess interventions or risk factors, the biases can be categorized into three broad types of bias: selection bias, information bias, and confounding ( 13 ). Selection bias impacts the effect size if, compared to the source population, the exposure or intervention groups differ in the distribution of factors associated with the outcome at the time the study population is selected, or if differential loss to follow up between groups occurs during the study. In case-control studies, selection bias occurs if cases or controls are selected based on criteria that are related to the exposure of interest. Information bias occurs when there are errors in measuring the exposure or intervention, or the outcome, or both. Finally, confounding is a mixing of effects that occurs when a variable (the confounder) that is independently associated with both the exposure and the outcome is not properly controlled. When confounding is not controlled, the estimate of the relationship between the exposure and the outcome will be biased.

There are several terms used to describe the approaches to assessing internal validity of primary research studies, including evidence hierarchies and levels of evidence, quality assessment, and risk of bias assessment. The use of these terms may be confusing, and it is not uncommon for some of these terms to be used interchangeably ( 14 , 15 ). Also, authors may mislabel the approaches and some evaluation tools (instruments) available for assessing internal validity may include additional components, such as those related to comprehensiveness (quality) of reporting, feasibility of applying an intervention, or external validity. Finally, some instruments may use the approach as a label for the instrument [e.g., Cochrane's risk of bias tool ( 16 ), which is an instrument that employs a risk of bias approach] and other instruments may not include the approach in the instrument name [e.g., the Jadad scale ( 17 ), which employs a quality assessment approach]. In an evaluation of the comprehensiveness of reporting in animal health systematic reviews (SRs), Sargeant et al., ( 18 )found that a range of instruments involving all three approaches had been used for assessing the internal validity of primary research studies. Although a large number of instruments are available, the approaches within each instrument used to assess internal validity can be grouped into three broad categories: based on study design, based on the presence or absence of design features, or based on a judgement about bias in the context of the study. These categories generally correspond to levels of evidence, quality assessment, and risk of bias, respectively. Therefore, our objective was to review these approaches to assessing internal validity as distinct entities and to describe the assumptions associated with each approach. Although we provide examples of specific instruments that include an evaluation of internal validity, our focus is on the approaches, rather than the tools. We discuss advances in the use of these approaches to assessing internal validity in human healthcare and propose a process for veterinary medicine for selecting the approach with the least assumptions as appropriate to the clinical question, the purpose of the assessment, and the research found that addresses the question of interest. The target audience for this article is individuals who assess internal validity of studies, individuals who develop instruments that include items related to the assessment of internal validity, and those who use evidence synthesis products created by others, such as systematic reviews or clinical practice guidelines.

Evaluating Internal Validity by Study Design: Levels of Evidence

Levels of evidence is an approach to evaluating the internal validity of a body of evidence, based on the potential for bias which is inherent to the employed study designs that were used to address the clinical question. The concept behind levels of evidence is that there is a hierarchy of study designs, with different study designs having different potential for bias. The way evidence hierarchies are used is based on either the name of the design or the description of the design. Readers of a study look for this information, then determine the design and assign a level of evidence. No further differentiation of methodological features or judgment is conducted.

Evidence hierarchies were initially introduced in 1979 by the Canadian Task Force on the Periodic Health Examination ( 19 ), with further development into an evidence pyramid by David Sackett in 1989 ( 20 ). A pyramid shaped figure commonly is used to illustrate the hierarchy of study designs for evaluating the efficacy of an intervention under realistic-use conditions (owned animals, as opposed to experimental settings), with the potential for bias decreasing from the base to the top of the pyramid ( Figure 1 ). Thus, study designs on the top of the pyramid represent those with inherently lower risk of bias compared to study designs lower on the hierarchy. The pyramid shape acknowledges that the quantity of research tends to decrease in the higher levels of evidence (for instance, there will be a larger volume of randomized controlled trials (RCTs) compared to SR-MA). Suggested modifications to the evidence pyramid for veterinary intervention studies include dividing RCTs into those conducted under realistic-use conditions vs. those conducted in nonrealistic-use conditions (e.g., research facility) ( 21 ), the inclusion of challenge trials (where disease outcomes are deliberately induced) below RCTs in the pyramid ( 21 , 22 ), and increasing the interpretability of the concept for students by displaying the hierarchy as a staircase rather than a pyramid ( 23 ).

Illustration of an evidence pyramid hierarchy for addressing intervention studies in veterinary medicine. SR, systematic review; MA, meta-analysis; RCT, randomized controlled trial.

The concept of evaluating the potential for bias in an individual study based on the study design can be extended to an evaluation of the potential for bias in a body of literature. This approach for evaluating the internal validity of a body of literature is referred to as “levels of evidence”. The approach is applied by identifying research (or other evidence) that pertains to the clinical question, determining the study design used for each of the studies, and then assigning each study to a level of evidence based on that design. For instance, a framework for levels of evidence in veterinary clinical nutrition has been proposed by Roudebush et al. ( 24 ). In this framework, level 1 evidence corresponds to at least 1 appropriately designed RCT in the target species with natural disease development, level 2 evidence would correspond to RCTs in laboratory settings with natural disease development, level 3 evidence would be obtained from non-randomized trials, deliberate disease induction trials, analytical observational studies or case series, and level 4 evidence would correspond to expert opinion, descriptive studies, studies in other species, or pathophysiological justification. Therefore, if the clinical question involves interventions, and the evidence found to address the question consists of 2 RCTs, 3 case-control studies, and 3 case series, the evidence would be designated as “level 1 evidence” because study designs with the highest evidentiary level in the available research consisted of RCTs. If all available evidence was from expert opinion, the body of research would comprise “level 4” evidence. This evidence would represent the best available evidence to inform decision-making at the time the assessment was made, although the overall level assigned would change as higher evidentiary level information becomes available.

The levels of evidence approach may be perceived as a quick and easy approach to assessing internal validity because it requires only a knowledge of the study design employed and not the individual features of a study that may or may not be associated with the potential for bias. However, that ease of use is based on very strong assumptions: 1) that study design maps directly to bias, 2) that authors always correctly label study designs, and 3) that authors execute and report study designs appropriately. The approach also pertains to a body of evidence, implying that there are multiple comparable studies available to address the question of interest.

An important critique of levels of evidence is that the approach focuses on the study design, rather than the actual design features that were used or the context of the study. Thus, although this framework illustrates the inherent potential for bias of the different study designs, it does not provide a consideration of the methodological rigor with which any specific individual study was conducted ( 25 ). For instance, although a well-conducted cohort study may be less biased than a poorly executed RCT, this nuance is not captured by a levels of evidence approach. Additionally, levels of evidence are based on the potential for confounding and selection biases, but there is no mechanism to evaluate the potential for information bias because this is linked to the outcome and the levels of evidence approach is based on features at the study, rather than outcome, level. For instance, RCTs provide a higher level of evidence compared to observational studies because random allocation to intervention groups minimizes the potential for confounding, and case-control studies provide a lower level of evidence than cohort studies because they are more prone to selection bias. However, a RCT that used a subjectively measured outcome would be assigned a higher level of evidence than a cohort study with an objective outcome, although the observational study may have a lower risk of information bias. Finally, studies may be mislabeled in terms of their study design; there is empirical evidence that this occurs in the veterinary literature ( 26 – 28 ). For example, studies labeled as case series in veterinary medicine frequently include a component corresponding to a cohort study design ( 27 ); these studies may be assigned an inappropriately low level of evidence if individuals classifying these studies rely on authors terminology rather than the complete design description to determine the design employed.

An additional consideration is that for questions related to aspects of clinical care other than selection of interventions, the framework and positioning of study designs included in Figure 1 may not be appropriate. Levels of evidence schema are available for other clinical questions, such as prognosis, diagnostic test accuracy, disease screening, and etiology ( 29 , 30 ).

Evaluating Internal Validity Based on Inclusion of Study Features Associated With Bias: Quality Assessment

As the name implies, quality assessment represents an evaluation of the quality of a primary research article. However, the term “quality” is difficult to specifically define in the context of evidence-based medicine, in that it does not appear to have been used consistently in the literature. The Merriam-Webster dictionary defines quality as “how good or bad something is” or “a high level of value or excellence” ( https://www.merriam-webster.com/dictionary/quality ). Quality generally is understood to be a multi-dimensional concept. While clear definitions are difficult to find in the research literature, the lay literature includes numerous treaties on the dimensions of quality. One example is the eight dimensions of quality delineated by David Gavin, which include performance, features, reliability, conformance, durability, serviceability, aesthetics, and perceived quality ( https://en.wikipedia.org/wiki/Eight_dimensions_of_quality ).

The findings from a review ( 31 ) identified that available instruments labeled as quality assessment tools varied in clarity and often involved more than just assessing internal validity. In addition to including an assessment of internal validity, quality assessment instruments also generally contain elements related to quality of reporting or an assessment of the inclusion of study features not directly related to bias, such as whether ethical approval was sought or whether the study participants were similar to those animals in the care of the individual doing the critique ( 14 , 31 – 33 ).

Quality assessment as an approach to evaluating internal validity involves an evaluation of the presence or absence of design features, i.e., a methodological checklist ( 14 , 15 ). For example, the Jadad scale ( 17 ) involves completing a checklist of whether the study was described as randomized, whether the study was described as double blind, and whether there was a description of withdrawals and dropouts, with points assigned for each category. Therefore, the Jadad scale uses a quality assessment approach to evaluating internal validity. In terms of assumptions, the quality assessment approach also makes strong assumptions, although these are less than those used in levels of evidence assessments. Instead of mapping bias to the study design, quality assessment maps bias to a design feature i.e., if a trial was randomized, it is assumed to be “good quality” and if the trial was not randomized the assumption is that it is “poor quality”. The same process is followed for additional study aspects, such a blinding or losses to follow-up, and an overall assessment of quality is then based on how the study 'performs' against these questions.

Quality assessment also considers more than just confounding and selection bias as components of internal validity. The inclusion of blinding as a design feature of interest illustrates this. Blinding as a design feature is intended to reduce the potential for differential care as a source of confounding bias (blinding of caregivers) or may be intended to reduce the potential for information bias (blinding of outcome assessors). Conducting a quality assessment is more complicated and time-consuming than evaluating levels of evidence because the presence or absence of the specific design features needs to be identified and validated within the study report. However, the approach requires only that the person evaluating internal validity can identify whether (or not) a design feature was used. Therefore, this approach requires more technical expertise that the levels of evidence approach, but less than the risk of bias approach.

Evaluating Internal Validity by Making Contextualized Judgements on Potential Occurrence of Bias: Risk of Bias Assessment

Risk of bias assessments have been developed specifically for evaluating the potential for elements of the design or conduct employed within a study to lead to a biased effect size ( 34 , 35 ). The components of risk of bias assessments are selected based on empirical evidence of their association with estimates of effect sizes ( 24 , 32 ). The way risk of bias assessments work is that individuals evaluating a study for internal validity answer a series of signaling questions about the presence or absence of design features followed by a judgment about the potential for the use of the design feature to lead to a biased estimate in the context of the specific study. A conclusion is then reached about potential for bias based on all evaluated design features in the context of the study. Thus, a risk of bias assessment makes fewer assumptions about the link between study design and design features compared to quality assessment. For instance, a quality assessment for an RCT would include an evaluation as to whether blinding of outcome assessors occurred, whereas a risk of bias assessment would involve an evaluation not only as to whether blinding was used, but also a judgement as to whether a lack of blinding of outcome assessors would be likely to lead to a biased estimate given the context of the study and the outcome measures used. Thus, a RCT that did not include blinding of outcome assessors might be rated as poor on a quality assessment but might not be a concern in a risk of bias assessment if the outcomes were measured objectively, precluding the likelihood that the estimate would be biased by a knowledge of the intervention group when classifying the outcomes. Because of the necessity of making a judgement about the potential that bias is associated with design features in the context of a specific topic area, this approach requires the highest level of knowledge of study design and bias. The risk of bias approach also generally is conducted at the outcome level, rather than at the study level. For instance, an unblinded RCT of interventions to treat lameness might be considered to have a high risk of bias if the outcome was assessed by owners (a subjective outcome) but not if the outcome was assessed by force plate measurement (an objective outcome). For a level of evidence assessment, the assessment of internal validity would be high quality because the trial was an RCT. For quality assessment, the study may be considered poor quality because it was unblinded, but the overall judgement would be dependent on a number of other study design flaws identified. Finally, in a risk of bias assessment, the study would likely be low risk of bias for the objective outcome and high risk of bias for the subjective outcome if blinding was not used.

Some components of a risk of bias assessment are the same as those included in a quality assessment approach (e.g., an assessment of randomization, allocation concealment, and blinding could be included in both). However, the way the assessment is done differs, with quality assessments generally involving present/absent judgements as opposed to assessments as to whether the risk of bias is likely or not. Hartling et al. ( 14 ) applied two instruments using a quality assessment approach and one instrument using a risk of bias approach to a sample of 163 trials and found that there was low correlation between quality assessment and risk of bias approaches when comparing the assessment of internal validity.

Although the critical elements for risk of bias are well described for RCTs in human healthcare and to a large extent in veterinary RCTs, these elements are not as well described for non-randomized trials and observational studies where allocation to groups is not under the control of the investigator. There are some risk of bias tools available for assessing risk of bias in non-randomized studies, such as ROBINS-I ( 36 ). However, ROBINS-I has been criticized for being challenging to use and for having low reliability, particularly amongst less experienced raters ( 37 , 38 ). A review and critique of approaches to risk of bias assessment for observational studies is available ( 39 ). It is anticipated that risk of bias tools for observational study designs, including studies related to questions of prognosis and causation, will continue to evolve as new instruments are developed and validated.

Historical Contexts and Comparisons of Internal Validity Assessment Approaches

Currently, the available approaches to assessing internal validity tend to be used for different applications. Levels of evidence have previously been used for creating evidence-based recommendations or clinical practices guidelines ( 30 , 40 , 41 ), where it is anticipated that multiple study designs may have been used to address the clinical question(s) of interest. Both quality assessment and risk of bias assessment approaches have been used as a component of systematic reviews with meta-analysis or network meta-analysis, as the intended product of these reviews is to summarize a single parameter (such as incidence or prevalence) or a summary effect size (such as a risk ratio, odds ratio, or hazard ratio) where it is desired that the estimate is unbiased. Often, that estimate is derived from studies with the same study design or a narrow range of study designs from high levels in the evidence hierarchy for the research question type. Therefore, the focus is on a specific parameter estimate based on multiple studies, rather than a descriptive summary of the evidentiary strength of those studies.

However, the different approaches are not necessarily mutually exclusive, but are nested within each other based on assumptions, and the methodology and use of the different approaches has evolved over time. As previously described, a criticism of the use of levels of evidence is that the potential for bias is based on the study design that was employed, rather than the methodological rigor of a specific study ( 42 ). For this reason, many frameworks for levels of evidence included wording such as “appropriately designed” ( 24 ) or “well designed” ( 41 )for the study designs, although the criteria for determining whether a study was designed and executed with rigor generally is not described. A lack of transparency for the criteria for evaluating internal validity of studies within an evidence level is problematic for individuals wishing to use the results. An example of the evolution toward more transparent considerations of internal validity of individual studies within a levels of evidence framework is seen in the progression of the Australian National Health and Medical Research Council (NHMRC) system for evaluating evidence in the development of clinical practice guidelines. The designation of levels of evidence in this framework originally was based on levels of evidence, with descriptors such as “properly-designed” or “well-designed” included for each type of study design ( 40 ). A concern with this approach was that the framework was not designed to address the strength of evidence from individual studies within each evidence level ( 43 ). Therefore, the framework was modified to include the use of risk of bias evaluations of individual studies within each evidence level. The combined use of levels of evidence and risk of bias assessment of studies within each level of evidence now forms the “evidence base” component of the NHMRC's FORM framework for the development of evidence-based clinical guidelines ( 44 ).

Another example of the evolution of approaches to assessing internal validity is from the Cochrane Back review group, who conduct systematic reviews of neck and back pain. The initial methods guidelines, published in 1997, recommended that a quality assessment be performed on each included study, with each item in the quality assessment tool scored based on whether the authors reported their use ( 45 ). Updated methods guidelines were published in 2003 ( 46 ). The framework for levels of evidence in this guidance was restricted to a consideration of randomized controlled trials and non-randomized controlled clinical trials, as these were considered the study designs that potentially were appropriate to address research questions in this content area. In the updated guidelines, the recommendations for the assessment of internal validity moved to a risk of bias approach, where judgements were made on whether the characteristics of each study were likely to lead to biased study results. In the 2003 methods guidelines, levels of evidence were recommended as an approach to qualitative analysis rather than the use of “vote counting” (summing the number of studies where a positive or negative outcome was reported). The guidelines were again updated in 2009 ( 47 ). In this version, the assessment of the internal validity of individual studies explicitly employed a risk of bias approach. It was further recommended that the use of evidence levels as a component of a qualitative synthesis be replaced with a formal rating of the quality of the evidence for each of the included outcomes. It was recommended that review authors use the GRADE approach for this component. The GRADE approach explicitly includes a consideration of the risk of bias across all studies included in the review, as well as an assessment of the consistency of results across studies, the directness of the evidence to the review question, the precision in the effect size estimate, and the potential for publication bias ( 48 ).

The examples from the human medical literature illustrate that assessment of internal validity need not be static, and that modifications to our approach to assessing internal validity can strengthen the evidence base for clinical decision making. When developing or using tools which include an evaluation of internal validity, the assessment of internal validity should use the approach with the least assumptions about bias. This implies that the risk of bias approach, where context specific judgements are made related to the potential for bias, is the preferred approach for assessing internal validity. The risk of bias approach is well developed for RCTs. Therefore, when RCTs are included in the evidence available to address a clinical question, a risk of bias assessment approach should be used. When evaluating internal validity as a component of a SR-MA, the Cochrane ROB2.0 tool ( 16 ) could be used for this purpose. Modifications to this tool have been proposed for evaluating trials in livestock trials ( 49 – 51 ). For critical appraisal instruments for RCTs, where additional components such as feasibility and external validity are a desired component, the questions or items within the instrument that are specific to assessing internal validity still could follow a risk of bias approach by specifically requiring a judgement on the potential for bias. Similarly, the use of questions or items requiring a judgement on the potential for bias also could be used for evaluation of RCTs included in clinical practice guidelines when RCTs are present in the evidence base.

However, there are circumstances where these recommendations may not be appropriate or sufficient, such as for observational studies where risk of bias assessment instruments do not formally exist, or where a variety of study designs have been identified that answer the clinical question (particularly non-intervention type questions). When observational studies are used as evidence, individuals assessing internal validity may wish to evaluate risks of bias for each study ad hoc by considering the specific risks of bias related to selection bias, information bias, and confounding in the context of the topic area. However, this approach requires considerable methodological expertise. Alternatively, a quality assessment approach could be used to evaluate internal validity for observational studies, recognizing that more assumptions related to the potential for bias are involved. As instruments for evaluating the risk of bias for observational studies are developed and validated, these could replace ad hoc or quality assessment approaches.

For situations where the evidence base includes multiple study types, such as clinical practice guidelines, the use of levels of evidence may be useful for framing the potential for bias inherent in the studies identified to address the clinical questions. However, within each evidence level, there still is a need to evaluate the internal validity of each study. The proposed approach for situations where RCTs and observational studies are included in the evidence base was described in the preceding paragraphs. For lower levels of evidence, such as case series, textbooks and narrative reviews, and expert opinion, levels of evidence could be used to emphasize that these types of evidence have high potential for bias based on their design.

Broader Considerations

It should be noted that although this article has focused on approaches to evaluating internal validity of studies, this is only one component of the assessment of evidence. Critical appraisal, CATs, SR-MA, and clinical practice guidelines explicitly incorporate other aspects of decision-making, including a consideration of the magnitude and precision of an intervention effect or the potential clinical impact, the consistency of the research results across studies, the applicability (external validity and feasibility) of the research results, and the directness of the evidence to a clinical situation (for instance, whether the study populations are similar to those in a practice setting). However, a discussion of these components for decision-making is beyond the scope of the current study. The interested reader is referred to further details on the components used in evaluating evidence for CATs ( 4 ), for SR-MA using the GRADE approach ( 52 ), for network meta-analysis ( 53 ) and for clinical practice guidelines ( 8 , 44 ).

Author Contributions

JS drafted the manuscript. All authors contributed equally to the conceptualization of this work. All authors read and approved the final contents.

Partial funding support was obtained from the University of Guelph Research Leadership Chair (Sargeant).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.