critical literature review techniques

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

Don't submit your assignments before you do this

The academic proofreading tool has been trained on 1000s of academic texts. Making it the most accurate and reliable proofreading tool for students. Free citation check included.

Try for free

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved August 12, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, get unlimited documents corrected.

✔ Free APA citation check included ✔ Unlimited document corrections ✔ Specialized in correcting academic texts

Warning: The NCBI web site requires JavaScript to function. more...

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.

Handbook of eHealth Evaluation: An Evidence-based Approach [Internet].

Chapter 9 methods for literature reviews.

Guy Paré and Spyros Kitsiou .

9.1. Introduction

Literature reviews play a critical role in scholarship because science remains, first and foremost, a cumulative endeavour ( vom Brocke et al., 2009 ). As in any academic discipline, rigorous knowledge syntheses are becoming indispensable in keeping up with an exponentially growing eHealth literature, assisting practitioners, academics, and graduate students in finding, evaluating, and synthesizing the contents of many empirical and conceptual papers. Among other methods, literature reviews are essential for: (a) identifying what has been written on a subject or topic; (b) determining the extent to which a specific research area reveals any interpretable trends or patterns; (c) aggregating empirical findings related to a narrow research question to support evidence-based practice; (d) generating new frameworks and theories; and (e) identifying topics or questions requiring more investigation ( Paré, Trudel, Jaana, & Kitsiou, 2015 ).

Literature reviews can take two major forms. The most prevalent one is the “literature review” or “background” section within a journal paper or a chapter in a graduate thesis. This section synthesizes the extant literature and usually identifies the gaps in knowledge that the empirical study addresses ( Sylvester, Tate, & Johnstone, 2013 ). It may also provide a theoretical foundation for the proposed study, substantiate the presence of the research problem, justify the research as one that contributes something new to the cumulated knowledge, or validate the methods and approaches for the proposed study ( Hart, 1998 ; Levy & Ellis, 2006 ).

The second form of literature review, which is the focus of this chapter, constitutes an original and valuable work of research in and of itself ( Paré et al., 2015 ). Rather than providing a base for a researcher’s own work, it creates a solid starting point for all members of the community interested in a particular area or topic ( Mulrow, 1987 ). The so-called “review article” is a journal-length paper which has an overarching purpose to synthesize the literature in a field, without collecting or analyzing any primary data ( Green, Johnson, & Adams, 2006 ).

When appropriately conducted, review articles represent powerful information sources for practitioners looking for state-of-the art evidence to guide their decision-making and work practices ( Paré et al., 2015 ). Further, high-quality reviews become frequently cited pieces of work which researchers seek out as a first clear outline of the literature when undertaking empirical studies ( Cooper, 1988 ; Rowe, 2014 ). Scholars who track and gauge the impact of articles have found that review papers are cited and downloaded more often than any other type of published article ( Cronin, Ryan, & Coughlan, 2008 ; Montori, Wilczynski, Morgan, Haynes, & Hedges, 2003 ; Patsopoulos, Analatos, & Ioannidis, 2005 ). The reason for their popularity may be the fact that reading the review enables one to have an overview, if not a detailed knowledge of the area in question, as well as references to the most useful primary sources ( Cronin et al., 2008 ). Although they are not easy to conduct, the commitment to complete a review article provides a tremendous service to one’s academic community ( Paré et al., 2015 ; Petticrew & Roberts, 2006 ). Most, if not all, peer-reviewed journals in the fields of medical informatics publish review articles of some type.

The main objectives of this chapter are fourfold: (a) to provide an overview of the major steps and activities involved in conducting a stand-alone literature review; (b) to describe and contrast the different types of review articles that can contribute to the eHealth knowledge base; (c) to illustrate each review type with one or two examples from the eHealth literature; and (d) to provide a series of recommendations for prospective authors of review articles in this domain.

9.2. Overview of the Literature Review Process and Steps

As explained in Templier and Paré (2015) , there are six generic steps involved in conducting a review article:

• formulating the research question(s) and objective(s),
• searching the extant literature,
• screening for inclusion,
• assessing the quality of primary studies,
• extracting data, and
• analyzing data.

Although these steps are presented here in sequential order, one must keep in mind that the review process can be iterative and that many activities can be initiated during the planning stage and later refined during subsequent phases ( Finfgeld-Connett & Johnson, 2013 ; Kitchenham & Charters, 2007 ).

Formulating the research question(s) and objective(s): As a first step, members of the review team must appropriately justify the need for the review itself ( Petticrew & Roberts, 2006 ), identify the review’s main objective(s) ( Okoli & Schabram, 2010 ), and define the concepts or variables at the heart of their synthesis ( Cooper & Hedges, 2009 ; Webster & Watson, 2002 ). Importantly, they also need to articulate the research question(s) they propose to investigate ( Kitchenham & Charters, 2007 ). In this regard, we concur with Jesson, Matheson, and Lacey (2011) that clearly articulated research questions are key ingredients that guide the entire review methodology; they underscore the type of information that is needed, inform the search for and selection of relevant literature, and guide or orient the subsequent analysis. Searching the extant literature: The next step consists of searching the literature and making decisions about the suitability of material to be considered in the review ( Cooper, 1988 ). There exist three main coverage strategies. First, exhaustive coverage means an effort is made to be as comprehensive as possible in order to ensure that all relevant studies, published and unpublished, are included in the review and, thus, conclusions are based on this all-inclusive knowledge base. The second type of coverage consists of presenting materials that are representative of most other works in a given field or area. Often authors who adopt this strategy will search for relevant articles in a small number of top-tier journals in a field ( Paré et al., 2015 ). In the third strategy, the review team concentrates on prior works that have been central or pivotal to a particular topic. This may include empirical studies or conceptual papers that initiated a line of investigation, changed how problems or questions were framed, introduced new methods or concepts, or engendered important debate ( Cooper, 1988 ). Screening for inclusion: The following step consists of evaluating the applicability of the material identified in the preceding step ( Levy & Ellis, 2006 ; vom Brocke et al., 2009 ). Once a group of potential studies has been identified, members of the review team must screen them to determine their relevance ( Petticrew & Roberts, 2006 ). A set of predetermined rules provides a basis for including or excluding certain studies. This exercise requires a significant investment on the part of researchers, who must ensure enhanced objectivity and avoid biases or mistakes. As discussed later in this chapter, for certain types of reviews there must be at least two independent reviewers involved in the screening process and a procedure to resolve disagreements must also be in place ( Liberati et al., 2009 ; Shea et al., 2009 ). Assessing the quality of primary studies: In addition to screening material for inclusion, members of the review team may need to assess the scientific quality of the selected studies, that is, appraise the rigour of the research design and methods. Such formal assessment, which is usually conducted independently by at least two coders, helps members of the review team refine which studies to include in the final sample, determine whether or not the differences in quality may affect their conclusions, or guide how they analyze the data and interpret the findings ( Petticrew & Roberts, 2006 ). Ascribing quality scores to each primary study or considering through domain-based evaluations which study components have or have not been designed and executed appropriately makes it possible to reflect on the extent to which the selected study addresses possible biases and maximizes validity ( Shea et al., 2009 ). Extracting data: The following step involves gathering or extracting applicable information from each primary study included in the sample and deciding what is relevant to the problem of interest ( Cooper & Hedges, 2009 ). Indeed, the type of data that should be recorded mainly depends on the initial research questions ( Okoli & Schabram, 2010 ). However, important information may also be gathered about how, when, where and by whom the primary study was conducted, the research design and methods, or qualitative/quantitative results ( Cooper & Hedges, 2009 ). Analyzing and synthesizing data : As a final step, members of the review team must collate, summarize, aggregate, organize, and compare the evidence extracted from the included studies. The extracted data must be presented in a meaningful way that suggests a new contribution to the extant literature ( Jesson et al., 2011 ). Webster and Watson (2002) warn researchers that literature reviews should be much more than lists of papers and should provide a coherent lens to make sense of extant knowledge on a given topic. There exist several methods and techniques for synthesizing quantitative (e.g., frequency analysis, meta-analysis) and qualitative (e.g., grounded theory, narrative analysis, meta-ethnography) evidence ( Dixon-Woods, Agarwal, Jones, Young, & Sutton, 2005 ; Thomas & Harden, 2008 ).

9.3. Types of Review Articles and Brief Illustrations

EHealth researchers have at their disposal a number of approaches and methods for making sense out of existing literature, all with the purpose of casting current research findings into historical contexts or explaining contradictions that might exist among a set of primary research studies conducted on a particular topic. Our classification scheme is largely inspired from Paré and colleagues’ (2015) typology. Below we present and illustrate those review types that we feel are central to the growth and development of the eHealth domain.

9.3.1. Narrative Reviews

The narrative review is the “traditional” way of reviewing the extant literature and is skewed towards a qualitative interpretation of prior knowledge ( Sylvester et al., 2013 ). Put simply, a narrative review attempts to summarize or synthesize what has been written on a particular topic but does not seek generalization or cumulative knowledge from what is reviewed ( Davies, 2000 ; Green et al., 2006 ). Instead, the review team often undertakes the task of accumulating and synthesizing the literature to demonstrate the value of a particular point of view ( Baumeister & Leary, 1997 ). As such, reviewers may selectively ignore or limit the attention paid to certain studies in order to make a point. In this rather unsystematic approach, the selection of information from primary articles is subjective, lacks explicit criteria for inclusion and can lead to biased interpretations or inferences ( Green et al., 2006 ). There are several narrative reviews in the particular eHealth domain, as in all fields, which follow such an unstructured approach ( Silva et al., 2015 ; Paul et al., 2015 ).

Despite these criticisms, this type of review can be very useful in gathering together a volume of literature in a specific subject area and synthesizing it. As mentioned above, its primary purpose is to provide the reader with a comprehensive background for understanding current knowledge and highlighting the significance of new research ( Cronin et al., 2008 ). Faculty like to use narrative reviews in the classroom because they are often more up to date than textbooks, provide a single source for students to reference, and expose students to peer-reviewed literature ( Green et al., 2006 ). For researchers, narrative reviews can inspire research ideas by identifying gaps or inconsistencies in a body of knowledge, thus helping researchers to determine research questions or formulate hypotheses. Importantly, narrative reviews can also be used as educational articles to bring practitioners up to date with certain topics of issues ( Green et al., 2006 ).

Recently, there have been several efforts to introduce more rigour in narrative reviews that will elucidate common pitfalls and bring changes into their publication standards. Information systems researchers, among others, have contributed to advancing knowledge on how to structure a “traditional” review. For instance, Levy and Ellis (2006) proposed a generic framework for conducting such reviews. Their model follows the systematic data processing approach comprised of three steps, namely: (a) literature search and screening; (b) data extraction and analysis; and (c) writing the literature review. They provide detailed and very helpful instructions on how to conduct each step of the review process. As another methodological contribution, vom Brocke et al. (2009) offered a series of guidelines for conducting literature reviews, with a particular focus on how to search and extract the relevant body of knowledge. Last, Bandara, Miskon, and Fielt (2011) proposed a structured, predefined and tool-supported method to identify primary studies within a feasible scope, extract relevant content from identified articles, synthesize and analyze the findings, and effectively write and present the results of the literature review. We highly recommend that prospective authors of narrative reviews consult these useful sources before embarking on their work.

Darlow and Wen (2015) provide a good example of a highly structured narrative review in the eHealth field. These authors synthesized published articles that describe the development process of mobile health ( m-health ) interventions for patients’ cancer care self-management. As in most narrative reviews, the scope of the research questions being investigated is broad: (a) how development of these systems are carried out; (b) which methods are used to investigate these systems; and (c) what conclusions can be drawn as a result of the development of these systems. To provide clear answers to these questions, a literature search was conducted on six electronic databases and Google Scholar . The search was performed using several terms and free text words, combining them in an appropriate manner. Four inclusion and three exclusion criteria were utilized during the screening process. Both authors independently reviewed each of the identified articles to determine eligibility and extract study information. A flow diagram shows the number of studies identified, screened, and included or excluded at each stage of study selection. In terms of contributions, this review provides a series of practical recommendations for m-health intervention development.

9.3.2. Descriptive or Mapping Reviews

The primary goal of a descriptive review is to determine the extent to which a body of knowledge in a particular research topic reveals any interpretable pattern or trend with respect to pre-existing propositions, theories, methodologies or findings ( King & He, 2005 ; Paré et al., 2015 ). In contrast with narrative reviews, descriptive reviews follow a systematic and transparent procedure, including searching, screening and classifying studies ( Petersen, Vakkalanka, & Kuzniarz, 2015 ). Indeed, structured search methods are used to form a representative sample of a larger group of published works ( Paré et al., 2015 ). Further, authors of descriptive reviews extract from each study certain characteristics of interest, such as publication year, research methods, data collection techniques, and direction or strength of research outcomes (e.g., positive, negative, or non-significant) in the form of frequency analysis to produce quantitative results ( Sylvester et al., 2013 ). In essence, each study included in a descriptive review is treated as the unit of analysis and the published literature as a whole provides a database from which the authors attempt to identify any interpretable trends or draw overall conclusions about the merits of existing conceptualizations, propositions, methods or findings ( Paré et al., 2015 ). In doing so, a descriptive review may claim that its findings represent the state of the art in a particular domain ( King & He, 2005 ).

In the fields of health sciences and medical informatics, reviews that focus on examining the range, nature and evolution of a topic area are described by Anderson, Allen, Peckham, and Goodwin (2008) as mapping reviews . Like descriptive reviews, the research questions are generic and usually relate to publication patterns and trends. There is no preconceived plan to systematically review all of the literature although this can be done. Instead, researchers often present studies that are representative of most works published in a particular area and they consider a specific time frame to be mapped.

An example of this approach in the eHealth domain is offered by DeShazo, Lavallie, and Wolf (2009). The purpose of this descriptive or mapping review was to characterize publication trends in the medical informatics literature over a 20-year period (1987 to 2006). To achieve this ambitious objective, the authors performed a bibliometric analysis of medical informatics citations indexed in medline using publication trends, journal frequencies, impact factors, Medical Subject Headings (MeSH) term frequencies, and characteristics of citations. Findings revealed that there were over 77,000 medical informatics articles published during the covered period in numerous journals and that the average annual growth rate was 12%. The MeSH term analysis also suggested a strong interdisciplinary trend. Finally, average impact scores increased over time with two notable growth periods. Overall, patterns in research outputs that seem to characterize the historic trends and current components of the field of medical informatics suggest it may be a maturing discipline (DeShazo et al., 2009).

9.3.3. Scoping Reviews

Scoping reviews attempt to provide an initial indication of the potential size and nature of the extant literature on an emergent topic (Arksey & O’Malley, 2005; Daudt, van Mossel, & Scott, 2013 ; Levac, Colquhoun, & O’Brien, 2010). A scoping review may be conducted to examine the extent, range and nature of research activities in a particular area, determine the value of undertaking a full systematic review (discussed next), or identify research gaps in the extant literature ( Paré et al., 2015 ). In line with their main objective, scoping reviews usually conclude with the presentation of a detailed research agenda for future works along with potential implications for both practice and research.

Unlike narrative and descriptive reviews, the whole point of scoping the field is to be as comprehensive as possible, including grey literature (Arksey & O’Malley, 2005). Inclusion and exclusion criteria must be established to help researchers eliminate studies that are not aligned with the research questions. It is also recommended that at least two independent coders review abstracts yielded from the search strategy and then the full articles for study selection ( Daudt et al., 2013 ). The synthesized evidence from content or thematic analysis is relatively easy to present in tabular form (Arksey & O’Malley, 2005; Thomas & Harden, 2008 ).

One of the most highly cited scoping reviews in the eHealth domain was published by Archer, Fevrier-Thomas, Lokker, McKibbon, and Straus (2011) . These authors reviewed the existing literature on personal health record ( phr ) systems including design, functionality, implementation, applications, outcomes, and benefits. Seven databases were searched from 1985 to March 2010. Several search terms relating to phr s were used during this process. Two authors independently screened titles and abstracts to determine inclusion status. A second screen of full-text articles, again by two independent members of the research team, ensured that the studies described phr s. All in all, 130 articles met the criteria and their data were extracted manually into a database. The authors concluded that although there is a large amount of survey, observational, cohort/panel, and anecdotal evidence of phr benefits and satisfaction for patients, more research is needed to evaluate the results of phr implementations. Their in-depth analysis of the literature signalled that there is little solid evidence from randomized controlled trials or other studies through the use of phr s. Hence, they suggested that more research is needed that addresses the current lack of understanding of optimal functionality and usability of these systems, and how they can play a beneficial role in supporting patient self-management ( Archer et al., 2011 ).

9.3.4. Forms of Aggregative Reviews

Healthcare providers, practitioners, and policy-makers are nowadays overwhelmed with large volumes of information, including research-based evidence from numerous clinical trials and evaluation studies, assessing the effectiveness of health information technologies and interventions ( Ammenwerth & de Keizer, 2004 ; Deshazo et al., 2009 ). It is unrealistic to expect that all these disparate actors will have the time, skills, and necessary resources to identify the available evidence in the area of their expertise and consider it when making decisions. Systematic reviews that involve the rigorous application of scientific strategies aimed at limiting subjectivity and bias (i.e., systematic and random errors) can respond to this challenge.

Systematic reviews attempt to aggregate, appraise, and synthesize in a single source all empirical evidence that meet a set of previously specified eligibility criteria in order to answer a clearly formulated and often narrow research question on a particular topic of interest to support evidence-based practice ( Liberati et al., 2009 ). They adhere closely to explicit scientific principles ( Liberati et al., 2009 ) and rigorous methodological guidelines (Higgins & Green, 2008) aimed at reducing random and systematic errors that can lead to deviations from the truth in results or inferences. The use of explicit methods allows systematic reviews to aggregate a large body of research evidence, assess whether effects or relationships are in the same direction and of the same general magnitude, explain possible inconsistencies between study results, and determine the strength of the overall evidence for every outcome of interest based on the quality of included studies and the general consistency among them ( Cook, Mulrow, & Haynes, 1997 ). The main procedures of a systematic review involve:

• Formulating a review question and developing a search strategy based on explicit inclusion criteria for the identification of eligible studies (usually described in the context of a detailed review protocol).
• Searching for eligible studies using multiple databases and information sources, including grey literature sources, without any language restrictions.
• Selecting studies, extracting data, and assessing risk of bias in a duplicate manner using two independent reviewers to avoid random or systematic errors in the process.
• Analyzing data using quantitative or qualitative methods.
• Presenting results in summary of findings tables.
• Interpreting results and drawing conclusions.

Many systematic reviews, but not all, use statistical methods to combine the results of independent studies into a single quantitative estimate or summary effect size. Known as meta-analyses , these reviews use specific data extraction and statistical techniques (e.g., network, frequentist, or Bayesian meta-analyses) to calculate from each study by outcome of interest an effect size along with a confidence interval that reflects the degree of uncertainty behind the point estimate of effect ( Borenstein, Hedges, Higgins, & Rothstein, 2009 ; Deeks, Higgins, & Altman, 2008 ). Subsequently, they use fixed or random-effects analysis models to combine the results of the included studies, assess statistical heterogeneity, and calculate a weighted average of the effect estimates from the different studies, taking into account their sample sizes. The summary effect size is a value that reflects the average magnitude of the intervention effect for a particular outcome of interest or, more generally, the strength of a relationship between two variables across all studies included in the systematic review. By statistically combining data from multiple studies, meta-analyses can create more precise and reliable estimates of intervention effects than those derived from individual studies alone, when these are examined independently as discrete sources of information.

The review by Gurol-Urganci, de Jongh, Vodopivec-Jamsek, Atun, and Car (2013) on the effects of mobile phone messaging reminders for attendance at healthcare appointments is an illustrative example of a high-quality systematic review with meta-analysis. Missed appointments are a major cause of inefficiency in healthcare delivery with substantial monetary costs to health systems. These authors sought to assess whether mobile phone-based appointment reminders delivered through Short Message Service ( sms ) or Multimedia Messaging Service ( mms ) are effective in improving rates of patient attendance and reducing overall costs. To this end, they conducted a comprehensive search on multiple databases using highly sensitive search strategies without language or publication-type restrictions to identify all rct s that are eligible for inclusion. In order to minimize the risk of omitting eligible studies not captured by the original search, they supplemented all electronic searches with manual screening of trial registers and references contained in the included studies. Study selection, data extraction, and risk of bias assessments were performed inde­­pen­dently by two coders using standardized methods to ensure consistency and to eliminate potential errors. Findings from eight rct s involving 6,615 participants were pooled into meta-analyses to calculate the magnitude of effects that mobile text message reminders have on the rate of attendance at healthcare appointments compared to no reminders and phone call reminders.

Meta-analyses are regarded as powerful tools for deriving meaningful conclusions. However, there are situations in which it is neither reasonable nor appropriate to pool studies together using meta-analytic methods simply because there is extensive clinical heterogeneity between the included studies or variation in measurement tools, comparisons, or outcomes of interest. In these cases, systematic reviews can use qualitative synthesis methods such as vote counting, content analysis, classification schemes and tabulations, as an alternative approach to narratively synthesize the results of the independent studies included in the review. This form of review is known as qualitative systematic review.

A rigorous example of one such review in the eHealth domain is presented by Mickan, Atherton, Roberts, Heneghan, and Tilson (2014) on the use of handheld computers by healthcare professionals and their impact on access to information and clinical decision-making. In line with the methodological guide­lines for systematic reviews, these authors: (a) developed and registered with prospero ( www.crd.york.ac.uk/ prospero / ) an a priori review protocol; (b) conducted comprehensive searches for eligible studies using multiple databases and other supplementary strategies (e.g., forward searches); and (c) subsequently carried out study selection, data extraction, and risk of bias assessments in a duplicate manner to eliminate potential errors in the review process. Heterogeneity between the included studies in terms of reported outcomes and measures precluded the use of meta-analytic methods. To this end, the authors resorted to using narrative analysis and synthesis to describe the effectiveness of handheld computers on accessing information for clinical knowledge, adherence to safety and clinical quality guidelines, and diagnostic decision-making.

In recent years, the number of systematic reviews in the field of health informatics has increased considerably. Systematic reviews with discordant findings can cause great confusion and make it difficult for decision-makers to interpret the review-level evidence ( Moher, 2013 ). Therefore, there is a growing need for appraisal and synthesis of prior systematic reviews to ensure that decision-making is constantly informed by the best available accumulated evidence. Umbrella reviews , also known as overviews of systematic reviews, are tertiary types of evidence synthesis that aim to accomplish this; that is, they aim to compare and contrast findings from multiple systematic reviews and meta-analyses ( Becker & Oxman, 2008 ). Umbrella reviews generally adhere to the same principles and rigorous methodological guidelines used in systematic reviews. However, the unit of analysis in umbrella reviews is the systematic review rather than the primary study ( Becker & Oxman, 2008 ). Unlike systematic reviews that have a narrow focus of inquiry, umbrella reviews focus on broader research topics for which there are several potential interventions ( Smith, Devane, Begley, & Clarke, 2011 ). A recent umbrella review on the effects of home telemonitoring interventions for patients with heart failure critically appraised, compared, and synthesized evidence from 15 systematic reviews to investigate which types of home telemonitoring technologies and forms of interventions are more effective in reducing mortality and hospital admissions ( Kitsiou, Paré, & Jaana, 2015 ).

9.3.5. Realist Reviews

Realist reviews are theory-driven interpretative reviews developed to inform, enhance, or supplement conventional systematic reviews by making sense of heterogeneous evidence about complex interventions applied in diverse contexts in a way that informs policy decision-making ( Greenhalgh, Wong, Westhorp, & Pawson, 2011 ). They originated from criticisms of positivist systematic reviews which centre on their “simplistic” underlying assumptions ( Oates, 2011 ). As explained above, systematic reviews seek to identify causation. Such logic is appropriate for fields like medicine and education where findings of randomized controlled trials can be aggregated to see whether a new treatment or intervention does improve outcomes. However, many argue that it is not possible to establish such direct causal links between interventions and outcomes in fields such as social policy, management, and information systems where for any intervention there is unlikely to be a regular or consistent outcome ( Oates, 2011 ; Pawson, 2006 ; Rousseau, Manning, & Denyer, 2008 ).

To circumvent these limitations, Pawson, Greenhalgh, Harvey, and Walshe (2005) have proposed a new approach for synthesizing knowledge that seeks to unpack the mechanism of how “complex interventions” work in particular contexts. The basic research question — what works? — which is usually associated with systematic reviews changes to: what is it about this intervention that works, for whom, in what circumstances, in what respects and why? Realist reviews have no particular preference for either quantitative or qualitative evidence. As a theory-building approach, a realist review usually starts by articulating likely underlying mechanisms and then scrutinizes available evidence to find out whether and where these mechanisms are applicable ( Shepperd et al., 2009 ). Primary studies found in the extant literature are viewed as case studies which can test and modify the initial theories ( Rousseau et al., 2008 ).

The main objective pursued in the realist review conducted by Otte-Trojel, de Bont, Rundall, and van de Klundert (2014) was to examine how patient portals contribute to health service delivery and patient outcomes. The specific goals were to investigate how outcomes are produced and, most importantly, how variations in outcomes can be explained. The research team started with an exploratory review of background documents and research studies to identify ways in which patient portals may contribute to health service delivery and patient outcomes. The authors identified six main ways which represent “educated guesses” to be tested against the data in the evaluation studies. These studies were identified through a formal and systematic search in four databases between 2003 and 2013. Two members of the research team selected the articles using a pre-established list of inclusion and exclusion criteria and following a two-step procedure. The authors then extracted data from the selected articles and created several tables, one for each outcome category. They organized information to bring forward those mechanisms where patient portals contribute to outcomes and the variation in outcomes across different contexts.

9.3.6. Critical Reviews

Lastly, critical reviews aim to provide a critical evaluation and interpretive analysis of existing literature on a particular topic of interest to reveal strengths, weaknesses, contradictions, controversies, inconsistencies, and/or other important issues with respect to theories, hypotheses, research methods or results ( Baumeister & Leary, 1997 ; Kirkevold, 1997 ). Unlike other review types, critical reviews attempt to take a reflective account of the research that has been done in a particular area of interest, and assess its credibility by using appraisal instruments or critical interpretive methods. In this way, critical reviews attempt to constructively inform other scholars about the weaknesses of prior research and strengthen knowledge development by giving focus and direction to studies for further improvement ( Kirkevold, 1997 ).

Kitsiou, Paré, and Jaana (2013) provide an example of a critical review that assessed the methodological quality of prior systematic reviews of home telemonitoring studies for chronic patients. The authors conducted a comprehensive search on multiple databases to identify eligible reviews and subsequently used a validated instrument to conduct an in-depth quality appraisal. Results indicate that the majority of systematic reviews in this particular area suffer from important methodological flaws and biases that impair their internal validity and limit their usefulness for clinical and decision-making purposes. To this end, they provide a number of recommendations to strengthen knowledge development towards improving the design and execution of future reviews on home telemonitoring.

9.4. Summary

Table 9.1 outlines the main types of literature reviews that were described in the previous sub-sections and summarizes the main characteristics that distinguish one review type from another. It also includes key references to methodological guidelines and useful sources that can be used by eHealth scholars and researchers for planning and developing reviews.

Typology of Literature Reviews (adapted from Paré et al., 2015).

As shown in Table 9.1 , each review type addresses different kinds of research questions or objectives, which subsequently define and dictate the methods and approaches that need to be used to achieve the overarching goal(s) of the review. For example, in the case of narrative reviews, there is greater flexibility in searching and synthesizing articles ( Green et al., 2006 ). Researchers are often relatively free to use a diversity of approaches to search, identify, and select relevant scientific articles, describe their operational characteristics, present how the individual studies fit together, and formulate conclusions. On the other hand, systematic reviews are characterized by their high level of systematicity, rigour, and use of explicit methods, based on an “a priori” review plan that aims to minimize bias in the analysis and synthesis process (Higgins & Green, 2008). Some reviews are exploratory in nature (e.g., scoping/mapping reviews), whereas others may be conducted to discover patterns (e.g., descriptive reviews) or involve a synthesis approach that may include the critical analysis of prior research ( Paré et al., 2015 ). Hence, in order to select the most appropriate type of review, it is critical to know before embarking on a review project, why the research synthesis is conducted and what type of methods are best aligned with the pursued goals.

9.5. Concluding Remarks

In light of the increased use of evidence-based practice and research generating stronger evidence ( Grady et al., 2011 ; Lyden et al., 2013 ), review articles have become essential tools for summarizing, synthesizing, integrating or critically appraising prior knowledge in the eHealth field. As mentioned earlier, when rigorously conducted review articles represent powerful information sources for eHealth scholars and practitioners looking for state-of-the-art evidence. The typology of literature reviews we used herein will allow eHealth researchers, graduate students and practitioners to gain a better understanding of the similarities and differences between review types.

We must stress that this classification scheme does not privilege any specific type of review as being of higher quality than another ( Paré et al., 2015 ). As explained above, each type of review has its own strengths and limitations. Having said that, we realize that the methodological rigour of any review — be it qualitative, quantitative or mixed — is a critical aspect that should be considered seriously by prospective authors. In the present context, the notion of rigour refers to the reliability and validity of the review process described in section 9.2. For one thing, reliability is related to the reproducibility of the review process and steps, which is facilitated by a comprehensive documentation of the literature search process, extraction, coding and analysis performed in the review. Whether the search is comprehensive or not, whether it involves a methodical approach for data extraction and synthesis or not, it is important that the review documents in an explicit and transparent manner the steps and approach that were used in the process of its development. Next, validity characterizes the degree to which the review process was conducted appropriately. It goes beyond documentation and reflects decisions related to the selection of the sources, the search terms used, the period of time covered, the articles selected in the search, and the application of backward and forward searches ( vom Brocke et al., 2009 ). In short, the rigour of any review article is reflected by the explicitness of its methods (i.e., transparency) and the soundness of the approach used. We refer those interested in the concepts of rigour and quality to the work of Templier and Paré (2015) which offers a detailed set of methodological guidelines for conducting and evaluating various types of review articles.

To conclude, our main objective in this chapter was to demystify the various types of literature reviews that are central to the continuous development of the eHealth field. It is our hope that our descriptive account will serve as a valuable source for those conducting, evaluating or using reviews in this important and growing domain.

• Ammenwerth E., de Keizer N. An inventory of evaluation studies of information technology in health care. Trends in evaluation research, 1982-2002. International Journal of Medical Informatics. 2004; 44 (1):44–56. [ PubMed : 15778794 ]
• Anderson S., Allen P., Peckham S., Goodwin N. Asking the right questions: scoping studies in the commissioning of research on the organisation and delivery of health services. Health Research Policy and Systems. 2008; 6 (7):1–12. [ PMC free article : PMC2500008 ] [ PubMed : 18613961 ] [ CrossRef ]
• Archer N., Fevrier-Thomas U., Lokker C., McKibbon K. A., Straus S.E. Personal health records: a scoping review. Journal of American Medical Informatics Association. 2011; 18 (4):515–522. [ PMC free article : PMC3128401 ] [ PubMed : 21672914 ]
• Arksey H., O’Malley L. Scoping studies: towards a methodological framework. International Journal of Social Research Methodology. 2005; 8 (1):19–32.
• A systematic, tool-supported method for conducting literature reviews in information systems. Paper presented at the Proceedings of the 19th European Conference on Information Systems ( ecis 2011); June 9 to 11; Helsinki, Finland. 2011.
• Baumeister R. F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1 (3):311–320.
• Becker L. A., Oxman A.D. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Overviews of reviews; pp. 607–631.
• Borenstein M., Hedges L., Higgins J., Rothstein H. Introduction to meta-analysis. Hoboken, nj : John Wiley & Sons Inc; 2009.
• Cook D. J., Mulrow C. D., Haynes B. Systematic reviews: Synthesis of best evidence for clinical decisions. Annals of Internal Medicine. 1997; 126 (5):376–380. [ PubMed : 9054282 ]
• Cooper H., Hedges L.V. In: The handbook of research synthesis and meta-analysis. 2nd ed. Cooper H., Hedges L. V., Valentine J. C., editors. New York: Russell Sage Foundation; 2009. Research synthesis as a scientific process; pp. 3–17.
• Cooper H. M. Organizing knowledge syntheses: A taxonomy of literature reviews. Knowledge in Society. 1988; 1 (1):104–126.
• Cronin P., Ryan F., Coughlan M. Undertaking a literature review: a step-by-step approach. British Journal of Nursing. 2008; 17 (1):38–43. [ PubMed : 18399395 ]
• Darlow S., Wen K.Y. Development testing of mobile health interventions for cancer patient self-management: A review. Health Informatics Journal. 2015 (online before print). [ PubMed : 25916831 ] [ CrossRef ]
• Daudt H. M., van Mossel C., Scott S.J. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. bmc Medical Research Methodology. 2013; 13 :48. [ PMC free article : PMC3614526 ] [ PubMed : 23522333 ] [ CrossRef ]
• Davies P. The relevance of systematic reviews to educational policy and practice. Oxford Review of Education. 2000; 26 (3-4):365–378.
• Deeks J. J., Higgins J. P. T., Altman D.G. In: Cochrane handbook for systematic reviews of interventions. Higgins J. P. T., Green S., editors. Hoboken, nj : John Wiley & Sons, Ltd; 2008. Analysing data and undertaking meta-analyses; pp. 243–296.
• Deshazo J. P., Lavallie D. L., Wolf F.M. Publication trends in the medical informatics literature: 20 years of “Medical Informatics” in mesh . bmc Medical Informatics and Decision Making. 2009; 9 :7. [ PMC free article : PMC2652453 ] [ PubMed : 19159472 ] [ CrossRef ]
• Dixon-Woods M., Agarwal S., Jones D., Young B., Sutton A. Synthesising qualitative and quantitative evidence: a review of possible methods. Journal of Health Services Research and Policy. 2005; 10 (1):45–53. [ PubMed : 15667704 ]
• Finfgeld-Connett D., Johnson E.D. Literature search strategies for conducting knowledge-building and theory-generating qualitative systematic reviews. Journal of Advanced Nursing. 2013; 69 (1):194–204. [ PMC free article : PMC3424349 ] [ PubMed : 22591030 ]
• Grady B., Myers K. M., Nelson E. L., Belz N., Bennett L., Carnahan L. … Guidelines Working Group. Evidence-based practice for telemental health. Telemedicine Journal and E Health. 2011; 17 (2):131–148. [ PubMed : 21385026 ]
• Green B. N., Johnson C. D., Adams A. Writing narrative literature reviews for peer-reviewed journals: secrets of the trade. Journal of Chiropractic Medicine. 2006; 5 (3):101–117. [ PMC free article : PMC2647067 ] [ PubMed : 19674681 ]
• Greenhalgh T., Wong G., Westhorp G., Pawson R. Protocol–realist and meta-narrative evidence synthesis: evolving standards ( rameses ). bmc Medical Research Methodology. 2011; 11 :115. [ PMC free article : PMC3173389 ] [ PubMed : 21843376 ]
• Gurol-Urganci I., de Jongh T., Vodopivec-Jamsek V., Atun R., Car J. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database System Review. 2013; 12 cd 007458. [ PMC free article : PMC6485985 ] [ PubMed : 24310741 ] [ CrossRef ]
• Hart C. Doing a literature review: Releasing the social science research imagination. London: SAGE Publications; 1998.
• Higgins J. P. T., Green S., editors. Cochrane handbook for systematic reviews of interventions: Cochrane book series. Hoboken, nj : Wiley-Blackwell; 2008.
• Jesson J., Matheson L., Lacey F.M. Doing your literature review: traditional and systematic techniques. Los Angeles & London: SAGE Publications; 2011.
• King W. R., He J. Understanding the role and methods of meta-analysis in IS research. Communications of the Association for Information Systems. 2005; 16 :1.
• Kirkevold M. Integrative nursing research — an important strategy to further the development of nursing science and nursing practice. Journal of Advanced Nursing. 1997; 25 (5):977–984. [ PubMed : 9147203 ]
• Kitchenham B., Charters S. ebse Technical Report Version 2.3. Keele & Durham. uk : Keele University & University of Durham; 2007. Guidelines for performing systematic literature reviews in software engineering.
• Kitsiou S., Paré G., Jaana M. Systematic reviews and meta-analyses of home telemonitoring interventions for patients with chronic diseases: a critical assessment of their methodological quality. Journal of Medical Internet Research. 2013; 15 (7):e150. [ PMC free article : PMC3785977 ] [ PubMed : 23880072 ]
• Kitsiou S., Paré G., Jaana M. Effects of home telemonitoring interventions on patients with chronic heart failure: an overview of systematic reviews. Journal of Medical Internet Research. 2015; 17 (3):e63. [ PMC free article : PMC4376138 ] [ PubMed : 25768664 ]
• Levac D., Colquhoun H., O’Brien K. K. Scoping studies: advancing the methodology. Implementation Science. 2010; 5 (1):69. [ PMC free article : PMC2954944 ] [ PubMed : 20854677 ]
• Levy Y., Ellis T.J. A systems approach to conduct an effective literature review in support of information systems research. Informing Science. 2006; 9 :181–211.
• Liberati A., Altman D. G., Tetzlaff J., Mulrow C., Gøtzsche P. C., Ioannidis J. P. A. et al. Moher D. The prisma statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. Annals of Internal Medicine. 2009; 151 (4):W-65. [ PubMed : 19622512 ]
• Lyden J. R., Zickmund S. L., Bhargava T. D., Bryce C. L., Conroy M. B., Fischer G. S. et al. McTigue K. M. Implementing health information technology in a patient-centered manner: Patient experiences with an online evidence-based lifestyle intervention. Journal for Healthcare Quality. 2013; 35 (5):47–57. [ PubMed : 24004039 ]
• Mickan S., Atherton H., Roberts N. W., Heneghan C., Tilson J.K. Use of handheld computers in clinical practice: a systematic review. bmc Medical Informatics and Decision Making. 2014; 14 :56. [ PMC free article : PMC4099138 ] [ PubMed : 24998515 ]
• Moher D. The problem of duplicate systematic reviews. British Medical Journal. 2013; 347 (5040) [ PubMed : 23945367 ] [ CrossRef ]
• Montori V. M., Wilczynski N. L., Morgan D., Haynes R. B., Hedges T. Systematic reviews: a cross-sectional study of location and citation counts. bmc Medicine. 2003; 1 :2. [ PMC free article : PMC281591 ] [ PubMed : 14633274 ]
• Mulrow C. D. The medical review article: state of the science. Annals of Internal Medicine. 1987; 106 (3):485–488. [ PubMed : 3813259 ] [ CrossRef ]
• Evidence-based information systems: A decade later. Proceedings of the European Conference on Information Systems ; 2011. Retrieved from http://aisel ​.aisnet.org/cgi/viewcontent ​.cgi?article ​=1221&context ​=ecis2011 .
• Okoli C., Schabram K. A guide to conducting a systematic literature review of information systems research. ssrn Electronic Journal. 2010
• Otte-Trojel T., de Bont A., Rundall T. G., van de Klundert J. How outcomes are achieved through patient portals: a realist review. Journal of American Medical Informatics Association. 2014; 21 (4):751–757. [ PMC free article : PMC4078283 ] [ PubMed : 24503882 ]
• Paré G., Trudel M.-C., Jaana M., Kitsiou S. Synthesizing information systems knowledge: A typology of literature reviews. Information & Management. 2015; 52 (2):183–199.
• Patsopoulos N. A., Analatos A. A., Ioannidis J.P. A. Relative citation impact of various study designs in the health sciences. Journal of the American Medical Association. 2005; 293 (19):2362–2366. [ PubMed : 15900006 ]
• Paul M. M., Greene C. M., Newton-Dame R., Thorpe L. E., Perlman S. E., McVeigh K. H., Gourevitch M.N. The state of population health surveillance using electronic health records: A narrative review. Population Health Management. 2015; 18 (3):209–216. [ PubMed : 25608033 ]
• Pawson R. Evidence-based policy: a realist perspective. London: SAGE Publications; 2006.
• Pawson R., Greenhalgh T., Harvey G., Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. Journal of Health Services Research & Policy. 2005; 10 (Suppl 1):21–34. [ PubMed : 16053581 ]
• Petersen K., Vakkalanka S., Kuzniarz L. Guidelines for conducting systematic mapping studies in software engineering: An update. Information and Software Technology. 2015; 64 :1–18.
• Petticrew M., Roberts H. Systematic reviews in the social sciences: A practical guide. Malden, ma : Blackwell Publishing Co; 2006.
• Rousseau D. M., Manning J., Denyer D. Evidence in management and organizational science: Assembling the field’s full weight of scientific knowledge through syntheses. The Academy of Management Annals. 2008; 2 (1):475–515.
• Rowe F. What literature review is not: diversity, boundaries and recommendations. European Journal of Information Systems. 2014; 23 (3):241–255.
• Shea B. J., Hamel C., Wells G. A., Bouter L. M., Kristjansson E., Grimshaw J. et al. Boers M. amstar is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. Journal of Clinical Epidemiology. 2009; 62 (10):1013–1020. [ PubMed : 19230606 ]
• Shepperd S., Lewin S., Straus S., Clarke M., Eccles M. P., Fitzpatrick R. et al. Sheikh A. Can we systematically review studies that evaluate complex interventions? PLoS Medicine. 2009; 6 (8):e1000086. [ PMC free article : PMC2717209 ] [ PubMed : 19668360 ]
• Silva B. M., Rodrigues J. J., de la Torre Díez I., López-Coronado M., Saleem K. Mobile-health: A review of current state in 2015. Journal of Biomedical Informatics. 2015; 56 :265–272. [ PubMed : 26071682 ]
• Smith V., Devane D., Begley C., Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. bmc Medical Research Methodology. 2011; 11 (1):15. [ PMC free article : PMC3039637 ] [ PubMed : 21291558 ]
• Sylvester A., Tate M., Johnstone D. Beyond synthesis: re-presenting heterogeneous research literature. Behaviour & Information Technology. 2013; 32 (12):1199–1215.
• Templier M., Paré G. A framework for guiding and evaluating literature reviews. Communications of the Association for Information Systems. 2015; 37 (6):112–137.
• Thomas J., Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. bmc Medical Research Methodology. 2008; 8 (1):45. [ PMC free article : PMC2478656 ] [ PubMed : 18616818 ]
• Reconstructing the giant: on the importance of rigour in documenting the literature search process. Paper presented at the Proceedings of the 17th European Conference on Information Systems ( ecis 2009); Verona, Italy. 2009.
• Webster J., Watson R.T. Analyzing the past to prepare for the future: Writing a literature review. Management Information Systems Quarterly. 2002; 26 (2):11.
• Whitlock E. P., Lin J. S., Chou R., Shekelle P., Robinson K.A. Using existing systematic reviews in complex systematic reviews. Annals of Internal Medicine. 2008; 148 (10):776–782. [ PubMed : 18490690 ]

This publication is licensed under a Creative Commons License, Attribution-Noncommercial 4.0 International License (CC BY-NC 4.0): see https://creativecommons.org/licenses/by-nc/4.0/

• Cite this Page Paré G, Kitsiou S. Chapter 9 Methods for Literature Reviews. In: Lau F, Kuziemsky C, editors. Handbook of eHealth Evaluation: An Evidence-based Approach [Internet]. Victoria (BC): University of Victoria; 2017 Feb 27.
• PDF version of this title (4.5M)
• Disable Glossary Links

In this Page

• Introduction
• Overview of the Literature Review Process and Steps
• Types of Review Articles and Brief Illustrations
• Concluding Remarks

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Recent Activity

• Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Ev... Chapter 9 Methods for Literature Reviews - Handbook of eHealth Evaluation: An Evidence-based Approach

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

• UConn Library
• Literature Review: The What, Why and How-to Guide
• Introduction

Literature Review: The What, Why and How-to Guide — Introduction

• Getting Started
• How to Pick a Topic
• Strategies to Find Sources
• Evaluating Sources & Lit. Reviews
• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings

What are Literature Reviews?

So, what is a literature review? "A literature review is an account of what has been published on a topic by accredited scholars and researchers. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries." Taylor, D.  The literature review: A few tips on conducting it . University of Toronto Health Sciences Writing Centre.

Goals of Literature Reviews

What are the goals of creating a Literature Review?  A literature could be written to accomplish different aims:

• To develop a theory or evaluate an existing theory
• To summarize the historical or existing state of a research topic
• Identify a problem in a field of research 

Baumeister, R. F., & Leary, M. R. (1997). Writing narrative literature reviews .  Review of General Psychology , 1 (3), 311-320.

What kinds of sources require a Literature Review?

• A research paper assigned in a course
• A thesis or dissertation
• A grant proposal
• An article intended for publication in a journal

All these instances require you to collect what has been written about your research topic so that you can demonstrate how your own research sheds new light on the topic.

Types of Literature Reviews

What kinds of literature reviews are written?

Narrative review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified. The review ends with a conclusion section which summarizes the findings regarding the state of the research of the specific study, the gaps identify and if applicable, explains how the author's research will address gaps identify in the review and expand the knowledge on the topic reviewed.

• Example : Predictors and Outcomes of U.S. Quality Maternity Leave: A Review and Conceptual Framework:  10.1177/08948453211037398  

Systematic review : "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139). Nelson, L. K. (2013). Research in Communication Sciences and Disorders . Plural Publishing.

• Example : The effect of leave policies on increasing fertility: a systematic review:  10.1057/s41599-022-01270-w

Meta-analysis : "Meta-analysis is a method of reviewing research findings in a quantitative fashion by transforming the data from individual studies into what is called an effect size and then pooling and analyzing this information. The basic goal in meta-analysis is to explain why different outcomes have occurred in different studies." (p. 197). Roberts, M. C., & Ilardi, S. S. (2003). Handbook of Research Methods in Clinical Psychology . Blackwell Publishing.

• Example : Employment Instability and Fertility in Europe: A Meta-Analysis:  10.1215/00703370-9164737

Meta-synthesis : "Qualitative meta-synthesis is a type of qualitative study that uses as data the findings from other qualitative studies linked by the same or related topic." (p.312). Zimmer, L. (2006). Qualitative meta-synthesis: A question of dialoguing with texts .  Journal of Advanced Nursing , 53 (3), 311-318.

• Example : Women’s perspectives on career successes and barriers: A qualitative meta-synthesis:  10.1177/05390184221113735

Literature Reviews in the Health Sciences

• UConn Health subject guide on systematic reviews Explanation of the different review types used in health sciences literature as well as tools to help you find the right review type
• << Previous: Getting Started
• Next: How to Pick a Topic >>
• Last Updated: Sep 21, 2022 2:16 PM
• URL: https://guides.lib.uconn.edu/literaturereview

• University of Texas Libraries

Literature Reviews

Steps in the literature review process.

• What is a literature review?
• Define your research question
• Determine inclusion and exclusion criteria
• Choose databases and search
• Review Results
• Synthesize Results
• Analyze Results
• Librarian Support
• Artificial Intelligence (AI) Tools
• You may need to some exploratory searching of the literature to get a sense of scope, to determine whether you need to narrow or broaden your focus
• Identify databases that provide the most relevant sources, and identify relevant terms (controlled vocabularies) to add to your search strategy
• Finalize your research question
• Think about relevant dates, geographies (and languages), methods, and conflicting points of view
• Conduct searches in the published literature via the identified databases
• Check to see if this topic has been covered in other discipline's databases
• Examine the citations of on-point articles for keywords, authors, and previous research (via references) and cited reference searching.
• Save your search results in a citation management tool (such as Zotero, Mendeley or EndNote)
• De-duplicate your search results
• Make sure that you've found the seminal pieces -- they have been cited many times, and their work is considered foundational 
• Check with your professor or a librarian to make sure your search has been comprehensive
• Evaluate the strengths and weaknesses of individual sources and evaluate for bias, methodologies, and thoroughness
• Group your results in to an organizational structure that will support why your research needs to be done, or that provides the answer to your research question  
• Develop your conclusions
• Are there gaps in the literature?
• Where has significant research taken place, and who has done it?
• Is there consensus or debate on this topic?
• Which methodological approaches work best?
• For example: Background, Current Practices, Critics and Proponents, Where/How this study will fit in 
• Organize your citations and focus on your research question and pertinent studies
• Compile your bibliography

Note: The first four steps are the best points at which to contact a librarian. Your librarian can help you determine the best databases to use for your topic, assess scope, and formulate a search strategy.

Videos Tutorials about Literature Reviews

This 4.5 minute video from Academic Education Materials has a Creative Commons License and a British narrator.

Recommended Reading

• Last Updated: Aug 13, 2024 1:52 PM
• URL: https://guides.lib.utexas.edu/literaturereviews

• Locations and Hours
• UCLA Library
• Research Guides
• Biomedical Library Guides

Systematic Reviews

• Types of Literature Reviews

What Makes a Systematic Review Different from Other Types of Reviews?

• Planning Your Systematic Review
• Database Searching
• Creating the Search
• Search Filters and Hedges
• Grey Literature
• Managing and Appraising Results
• Further Resources

Reproduced from Grant, M. J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91–108. doi:10.1111/j.1471-1842.2009.00848.x

• << Previous: Home
• Next: Planning Your Systematic Review >>
• Last Updated: Jul 23, 2024 3:40 PM
• URL: https://guides.library.ucla.edu/systematicreviews

What is a Literature Review? How to Write It (with Examples)

A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship, demonstrating your understanding of the topic and showing how your work contributes to the ongoing conversation in the field. Learning how to write a literature review is a critical tool for successful research. Your ability to summarize and synthesize prior research pertaining to a certain topic demonstrates your grasp on the topic of study, and assists in the learning process. 

Table of Contents

• What is the purpose of literature review? 
• a. Habitat Loss and Species Extinction: 
• b. Range Shifts and Phenological Changes: 
• c. Ocean Acidification and Coral Reefs: 
• d. Adaptive Strategies and Conservation Efforts: 

How to write a good literature review 

• Choose a Topic and Define the Research Question: 
• Decide on the Scope of Your Review: 
• Select Databases for Searches: 
• Conduct Searches and Keep Track: 
• Review the Literature: 
• Organize and Write Your Literature Review: 
• How to write a literature review faster with Paperpal? 
• Frequently asked questions 

What is a literature review?

A well-conducted literature review demonstrates the researcher’s familiarity with the existing literature, establishes the context for their own research, and contributes to scholarly conversations on the topic. One of the purposes of a literature review is also to help researchers avoid duplicating previous work and ensure that their research is informed by and builds upon the existing body of knowledge.

What is the purpose of literature review?

A literature review serves several important purposes within academic and research contexts. Here are some key objectives and functions of a literature review: 2  

1. Contextualizing the Research Problem: The literature review provides a background and context for the research problem under investigation. It helps to situate the study within the existing body of knowledge. 

2. Identifying Gaps in Knowledge: By identifying gaps, contradictions, or areas requiring further research, the researcher can shape the research question and justify the significance of the study. This is crucial for ensuring that the new research contributes something novel to the field. 

Find academic papers related to your research topic faster. Try Research on Paperpal  

3. Understanding Theoretical and Conceptual Frameworks: Literature reviews help researchers gain an understanding of the theoretical and conceptual frameworks used in previous studies. This aids in the development of a theoretical framework for the current research. 

4. Providing Methodological Insights: Another purpose of literature reviews is that it allows researchers to learn about the methodologies employed in previous studies. This can help in choosing appropriate research methods for the current study and avoiding pitfalls that others may have encountered. 

5. Establishing Credibility: A well-conducted literature review demonstrates the researcher’s familiarity with existing scholarship, establishing their credibility and expertise in the field. It also helps in building a solid foundation for the new research. 

6. Informing Hypotheses or Research Questions: The literature review guides the formulation of hypotheses or research questions by highlighting relevant findings and areas of uncertainty in existing literature. 

Literature review example

Let’s delve deeper with a literature review example: Let’s say your literature review is about the impact of climate change on biodiversity. You might format your literature review into sections such as the effects of climate change on habitat loss and species extinction, phenological changes, and marine biodiversity. Each section would then summarize and analyze relevant studies in those areas, highlighting key findings and identifying gaps in the research. The review would conclude by emphasizing the need for further research on specific aspects of the relationship between climate change and biodiversity. The following literature review template provides a glimpse into the recommended literature review structure and content, demonstrating how research findings are organized around specific themes within a broader topic. 

Literature Review on Climate Change Impacts on Biodiversity:

Climate change is a global phenomenon with far-reaching consequences, including significant impacts on biodiversity. This literature review synthesizes key findings from various studies: 

a. Habitat Loss and Species Extinction:

Climate change-induced alterations in temperature and precipitation patterns contribute to habitat loss, affecting numerous species (Thomas et al., 2004). The review discusses how these changes increase the risk of extinction, particularly for species with specific habitat requirements. 

b. Range Shifts and Phenological Changes:

Observations of range shifts and changes in the timing of biological events (phenology) are documented in response to changing climatic conditions (Parmesan & Yohe, 2003). These shifts affect ecosystems and may lead to mismatches between species and their resources. 

c. Ocean Acidification and Coral Reefs:

The review explores the impact of climate change on marine biodiversity, emphasizing ocean acidification’s threat to coral reefs (Hoegh-Guldberg et al., 2007). Changes in pH levels negatively affect coral calcification, disrupting the delicate balance of marine ecosystems. 

d. Adaptive Strategies and Conservation Efforts:

Recognizing the urgency of the situation, the literature review discusses various adaptive strategies adopted by species and conservation efforts aimed at mitigating the impacts of climate change on biodiversity (Hannah et al., 2007). It emphasizes the importance of interdisciplinary approaches for effective conservation planning. 

Strengthen your literature review with factual insights. Try Research on Paperpal for free!    

Writing a literature review involves summarizing and synthesizing existing research on a particular topic. A good literature review format should include the following elements. 

Introduction: The introduction sets the stage for your literature review, providing context and introducing the main focus of your review. 

• Opening Statement: Begin with a general statement about the broader topic and its significance in the field. 
• Scope and Purpose: Clearly define the scope of your literature review. Explain the specific research question or objective you aim to address. 
• Organizational Framework: Briefly outline the structure of your literature review, indicating how you will categorize and discuss the existing research. 
• Significance of the Study: Highlight why your literature review is important and how it contributes to the understanding of the chosen topic. 
• Thesis Statement: Conclude the introduction with a concise thesis statement that outlines the main argument or perspective you will develop in the body of the literature review. 

Body: The body of the literature review is where you provide a comprehensive analysis of existing literature, grouping studies based on themes, methodologies, or other relevant criteria. 

• Organize by Theme or Concept: Group studies that share common themes, concepts, or methodologies. Discuss each theme or concept in detail, summarizing key findings and identifying gaps or areas of disagreement. 
• Critical Analysis: Evaluate the strengths and weaknesses of each study. Discuss the methodologies used, the quality of evidence, and the overall contribution of each work to the understanding of the topic. 
• Synthesis of Findings: Synthesize the information from different studies to highlight trends, patterns, or areas of consensus in the literature. 
• Identification of Gaps: Discuss any gaps or limitations in the existing research and explain how your review contributes to filling these gaps. 
• Transition between Sections: Provide smooth transitions between different themes or concepts to maintain the flow of your literature review. 

Write and Cite as you go with Paperpal Research. Start now for free.   

Conclusion: The conclusion of your literature review should summarize the main findings, highlight the contributions of the review, and suggest avenues for future research. 

• Summary of Key Findings: Recap the main findings from the literature and restate how they contribute to your research question or objective. 
• Contributions to the Field: Discuss the overall contribution of your literature review to the existing knowledge in the field. 
• Implications and Applications: Explore the practical implications of the findings and suggest how they might impact future research or practice. 
• Recommendations for Future Research: Identify areas that require further investigation and propose potential directions for future research in the field. 
• Final Thoughts: Conclude with a final reflection on the importance of your literature review and its relevance to the broader academic community. 

Conducting a literature review

Conducting a literature review is an essential step in research that involves reviewing and analyzing existing literature on a specific topic. It’s important to know how to do a literature review effectively, so here are the steps to follow: 1  

Choose a Topic and Define the Research Question:

• Select a topic that is relevant to your field of study. 
• Clearly define your research question or objective. Determine what specific aspect of the topic do you want to explore? 

Decide on the Scope of Your Review:

• Determine the timeframe for your literature review. Are you focusing on recent developments, or do you want a historical overview? 
• Consider the geographical scope. Is your review global, or are you focusing on a specific region? 
• Define the inclusion and exclusion criteria. What types of sources will you include? Are there specific types of studies or publications you will exclude? 

Select Databases for Searches:

• Identify relevant databases for your field. Examples include PubMed, IEEE Xplore, Scopus, Web of Science, and Google Scholar. 
• Consider searching in library catalogs, institutional repositories, and specialized databases related to your topic. 

Conduct Searches and Keep Track:

• Develop a systematic search strategy using keywords, Boolean operators (AND, OR, NOT), and other search techniques. 
• Record and document your search strategy for transparency and replicability. 
• Keep track of the articles, including publication details, abstracts, and links. Use citation management tools like EndNote, Zotero, or Mendeley to organize your references. 

Review the Literature:

• Evaluate the relevance and quality of each source. Consider the methodology, sample size, and results of studies. 
• Organize the literature by themes or key concepts. Identify patterns, trends, and gaps in the existing research. 
• Summarize key findings and arguments from each source. Compare and contrast different perspectives. 
• Identify areas where there is a consensus in the literature and where there are conflicting opinions. 
• Provide critical analysis and synthesis of the literature. What are the strengths and weaknesses of existing research? 

Organize and Write Your Literature Review:

• Literature review outline should be based on themes, chronological order, or methodological approaches. 
• Write a clear and coherent narrative that synthesizes the information gathered. 
• Use proper citations for each source and ensure consistency in your citation style (APA, MLA, Chicago, etc.). 
• Conclude your literature review by summarizing key findings, identifying gaps, and suggesting areas for future research. 

Whether you’re exploring a new research field or finding new angles to develop an existing topic, sifting through hundreds of papers can take more time than you have to spare. But what if you could find science-backed insights with verified citations in seconds? That’s the power of Paperpal’s new Research feature!  

How to write a literature review faster with Paperpal?

Paperpal, an AI writing assistant, integrates powerful academic search capabilities within its writing platform. With the Research feature, you get 100% factual insights, with citations backed by 250M+ verified research articles, directly within your writing interface with the option to save relevant references in your Citation Library. By eliminating the need to switch tabs to find answers to all your research questions, Paperpal saves time and helps you stay focused on your writing.   

Here’s how to use the Research feature:  

• Ask a question: Get started with a new document on paperpal.com. Click on the “Research” feature and type your question in plain English. Paperpal will scour over 250 million research articles, including conference papers and preprints, to provide you with accurate insights and citations. 
• Review and Save: Paperpal summarizes the information, while citing sources and listing relevant reads. You can quickly scan the results to identify relevant references and save these directly to your built-in citations library for later access. 
• Cite with Confidence: Paperpal makes it easy to incorporate relevant citations and references into your writing, ensuring your arguments are well-supported by credible sources. This translates to a polished, well-researched literature review. 

The literature review sample and detailed advice on writing and conducting a review will help you produce a well-structured report. But remember that a good literature review is an ongoing process, and it may be necessary to revisit and update it as your research progresses. By combining effortless research with an easy citation process, Paperpal Research streamlines the literature review process and empowers you to write faster and with more confidence. Try Paperpal Research now and see for yourself.  

Frequently asked questions

A literature review is a critical and comprehensive analysis of existing literature (published and unpublished works) on a specific topic or research question and provides a synthesis of the current state of knowledge in a particular field. A well-conducted literature review is crucial for researchers to build upon existing knowledge, avoid duplication of efforts, and contribute to the advancement of their field. It also helps researchers situate their work within a broader context and facilitates the development of a sound theoretical and conceptual framework for their studies.

Literature review is a crucial component of research writing, providing a solid background for a research paper’s investigation. The aim is to keep professionals up to date by providing an understanding of ongoing developments within a specific field, including research methods, and experimental techniques used in that field, and present that knowledge in the form of a written report. Also, the depth and breadth of the literature review emphasizes the credibility of the scholar in his or her field.  

Before writing a literature review, it’s essential to undertake several preparatory steps to ensure that your review is well-researched, organized, and focused. This includes choosing a topic of general interest to you and doing exploratory research on that topic, writing an annotated bibliography, and noting major points, especially those that relate to the position you have taken on the topic. 

Literature reviews and academic research papers are essential components of scholarly work but serve different purposes within the academic realm. 3 A literature review aims to provide a foundation for understanding the current state of research on a particular topic, identify gaps or controversies, and lay the groundwork for future research. Therefore, it draws heavily from existing academic sources, including books, journal articles, and other scholarly publications. In contrast, an academic research paper aims to present new knowledge, contribute to the academic discourse, and advance the understanding of a specific research question. Therefore, it involves a mix of existing literature (in the introduction and literature review sections) and original data or findings obtained through research methods. 

Literature reviews are essential components of academic and research papers, and various strategies can be employed to conduct them effectively. If you want to know how to write a literature review for a research paper, here are four common approaches that are often used by researchers.  Chronological Review: This strategy involves organizing the literature based on the chronological order of publication. It helps to trace the development of a topic over time, showing how ideas, theories, and research have evolved.  Thematic Review: Thematic reviews focus on identifying and analyzing themes or topics that cut across different studies. Instead of organizing the literature chronologically, it is grouped by key themes or concepts, allowing for a comprehensive exploration of various aspects of the topic.  Methodological Review: This strategy involves organizing the literature based on the research methods employed in different studies. It helps to highlight the strengths and weaknesses of various methodologies and allows the reader to evaluate the reliability and validity of the research findings.  Theoretical Review: A theoretical review examines the literature based on the theoretical frameworks used in different studies. This approach helps to identify the key theories that have been applied to the topic and assess their contributions to the understanding of the subject.  It’s important to note that these strategies are not mutually exclusive, and a literature review may combine elements of more than one approach. The choice of strategy depends on the research question, the nature of the literature available, and the goals of the review. Additionally, other strategies, such as integrative reviews or systematic reviews, may be employed depending on the specific requirements of the research.

The literature review format can vary depending on the specific publication guidelines. However, there are some common elements and structures that are often followed. Here is a general guideline for the format of a literature review:  Introduction:   Provide an overview of the topic.  Define the scope and purpose of the literature review.  State the research question or objective.  Body:   Organize the literature by themes, concepts, or chronology.  Critically analyze and evaluate each source.  Discuss the strengths and weaknesses of the studies.  Highlight any methodological limitations or biases.  Identify patterns, connections, or contradictions in the existing research.  Conclusion:   Summarize the key points discussed in the literature review.  Highlight the research gap.  Address the research question or objective stated in the introduction.  Highlight the contributions of the review and suggest directions for future research.

Both annotated bibliographies and literature reviews involve the examination of scholarly sources. While annotated bibliographies focus on individual sources with brief annotations, literature reviews provide a more in-depth, integrated, and comprehensive analysis of existing literature on a specific topic. The key differences are as follows: 

References 

• Denney, A. S., & Tewksbury, R. (2013). How to write a literature review.  Journal of criminal justice education ,  24 (2), 218-234. 
• Pan, M. L. (2016).  Preparing literature reviews: Qualitative and quantitative approaches . Taylor & Francis. 
• Cantero, C. (2019). How to write a literature review.  San José State University Writing Center . 

Paperpal is an AI writing assistant that help academics write better, faster with real-time suggestions for in-depth language and grammar correction. Trained on millions of research manuscripts enhanced by professional academic editors, Paperpal delivers human precision at machine speed.  

Try it for free or upgrade to  Paperpal Prime , which unlocks unlimited access to premium features like academic translation, paraphrasing, contextual synonyms, consistency checks and more. It’s like always having a professional academic editor by your side! Go beyond limitations and experience the future of academic writing.  Get Paperpal Prime now at just US$19 a month!

Related Reads:

• Empirical Research: A Comprehensive Guide for Academics 
• How to Write a Scientific Paper in 10 Steps 
• How Long Should a Chapter Be?
• How to Use Paperpal to Generate Emails & Cover Letters?

6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Self-plagiarism in research: what it is and how to avoid it, you may also like, academic integrity vs academic dishonesty: types & examples, dissertation printing and binding | types & comparison , what is a dissertation preface definition and examples , the ai revolution: authors’ role in upholding academic..., the future of academia: how ai tools are..., how to write a research proposal: (with examples..., how to write your research paper in apa..., how to choose a dissertation topic, how to write a phd research proposal, how to write an academic paragraph (step-by-step guide).

• Academic Skills
• Reading, writing and referencing
• Literature reviews

Writing a literature review

Find out how to write a lit review.

What is a literature review ?

A literature review explores and evaluates the literature on a specific topic or question. It synthesises the contributions of the different authors, often to identify areas that need further exploration.

You may be required to write a literature review as a standalone document or part of a larger body of research, such as a thesis.

• The point of a standalone literature review is to demonstrate that you have read widely in your field and you understand the main arguments.
• As part of a thesis or research paper, the literature review defines your project by establishing how your work will extend or differ from previous work and what contribution it will make.

What are markers looking for?

In the best literature reviews, the writer:

• Has a clear understanding of key concepts within the topic.
• Clarifies important definitions and terminology.
• Covers the breadth of the specific topic.
• Critically discusses the ideas in the literature and evaluates how authors present them.
• Clearly indicates a research gap for future enquiry.

How do I write a literature review?

This video outlines a step by step approach to help you evaluate readings, organise ideas and write critically. It provides examples of how to connect, interpret and critique ideas to make sure your voice comes through strongly.

Tips for research, reading and writing

You may be given a specific question to research or broad topics which must be refined to a question that can be reasonably addressed in the time and word limit available.

Use your early reading to help you determine and refine your topic.

• Too much literature? You probably need to narrow your scope. Try to identify a more specific issue of interest.
• Not enough literature? Your topic may be too specific and needs to be broader.

Start with readings suggested by your lecturers or supervisors. Then, do your own research - the best place to go is the Library Website .

You can also use the Library Guides or speak to a librarian to identify the most useful databases for you and to learn how to search for sources effectively and efficiently.

Cover the field

Make sure your literature search covers a broad range of views and information relevant to your topic. Focussing on a narrow selection of sources may result in a lack of depth. You are not expected to cover all research and scholarly opinions on your topic, but you need to identify and include important viewpoints. A quality literature review examines and evaluates different viewpoints based on the evidence presented, rather than providing only material that reinforces a bias.

Use reading strategies

Survey, skim and scan to find the most relevant articles, and the most relevant parts of those articles. These can be re-read more closely later when you have acquired an overview of your topic.

Take notes as you read

This helps to organise and develop your thoughts. Record your own reactions to the text in your notes, perhaps in a separate column. These notes can form the basis of your critical evaluation of the text. Record any facts, opinions or direct quotes that are likely to be useful to your review, noting the page numbers, author and year.

Stop reading when you have enough

This depends on the word count required of this literature review. A review of one thousand words can only cover the major ideas and probably less than ten references. Longer reviews that form part of a large research paper will include more than fifty. Your tutor or supervisor should be able to suggest a suitable number.

As you read, ask yourself these questions:

• Have I answered my question without any obvious gaps?
• Have I read this before? Are there any new related issues coming up as I search the literature?
• Have I found multiple references which cover the same material or just enough to prove agreement?

There are many possible ways to organise the material. For example:

• chronologically
• by theoretical perspective
• from most to least important
• by issue or theme

It is important to remember that you are not merely cataloguing or describing the literature you read. Therefore, you need to choose an organisation that will enable you to compare the various authors' treatment of ideas. This is often best achieved by organising thematically, or grouping ideas into sets of common issues tackled in the various texts. These themes will form the basis of the different threads that are the focus of your study.

A standalone literature review

A standalone literature review is structured much like an academic essay.

• Introduction - establish the context for your topic and outline your main contentions about the literature
• Main body - explain and support these inferences in the main body
• Conclusion - summarise your main points and restate the contention.

The main difference between an essay and this kind of literature review is that an essay focuses on a topic and uses the literature as a support for the arguments. In a standalone literature review, the literature itself is the topic of discussion and evaluation. This means you evaluate and discuss not only the informational content but the quality of the author’s handling of the content.

A literature review as part of a larger research paper?

As part of a larger research paper, the literature review may take many forms, depending on your discipline, your topic and the logic of your research. Traditionally, in empirical research, the literature review is included in the introduction, or a standalone chapter immediately following the introduction. For other forms of research, you may need to engage more extensively with the literature and thus, the literature review may spread over more than one chapter, or even be distributed throughout the thesis.

Start writing early. Writing will clarify your thinking on the topic and reveal any gaps in information and logic. If your ideas change, sections and paragraphs can be reworked to change your contentions or include extra information.

Similarly, draft an overall plan for your review as soon as you are ready, but be prepared to rework sections of it to reflect your developing argument.

The most important thing to remember is that you are writing a review . That means you must move past describing what other authors have written by connecting, interpreting and critiquing their ideas and presenting your own analysis and interpretation.

Looking for one-on-one advice?

Get tailored advice from an Academic Skills Adviser by booking an Individual appointment, or get quick feedback from one of our Academic Writing Mentors via email through our Writing advice service.

Go to Student appointments

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Jun 20, 2024 9:08 AM
• URL: https://guides.library.yale.edu/YSNDoctoral

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

Writing a Critical Review of Literature: A Practical Guide for English Graduate Students

Global Language Review

As an integral part of dissertations and theses, research scholars in different disciplines require to write a comprehensive chapter on &quot;literature review&quot; that establishes the conceptual and theoretical foundations of an empirical research study. This, however, poses an intellectual challenge to produce a critical review of the published research on a given topic. Therefore, this paper addresses the students problems of writing the literature review in a thesis or dissertation at the graduate and postgraduate levels. It explains the process and steps of reviewing literature for a thesis chapter. These steps include; a) critical reading and note-taking, b) writing a summary of the reviewed literature, c) organization of literature review, and d) the use of a synthesis matrix. The last part of the paper offers suggestions on how to write critically and make the researcher&#39;s voice explicit in the chapter.

Related Papers

Scientific Research Publishing: Creative Education

Dr. Qais Faryadi

Literature writing is a skill that every PhD candidate must procure to communicate his or her research findings clearly. The main objective of this paper is to facilitate the literature writing process so that PhD candidates under- stand what PhD literature is and are able to write their PhD literature cor- rectly and scientifically. The methodology used in this research is a descrip- tive method as it deliberates and defines the various parts of literature writing process and elucidates the how to do of it in a very unpretentious and under- standing language. As thus, this paper summarizes the various steps of litera- ture writing to pilot the PhD students so that the task of PhD literature writ- ing process becomes adaptable and less discouraging. This research is a useful roadmap especially for students of the social science studies. Additionally, in this paper, literature writing techniques, procedures and important strategies are enlightened in a simple manner. This paper adopts a how-to approach when discussing a variety of relevant topics, such as literature review intro- duction, types of literature review, advantages of literature reviews, objective of literature review, literature review template, and important check lists about literature review are discussed. This paper has 5 parts, such as Intro- duction, Literature Review, Methodology, Results and Conclusion. The lit- erature review chapter is discussed in this paper. I will discuss the rest as a se- ries in the future. Keywords Thesis Writing Process, Literature Review, PhD, Social Science, Research Methodology

Mohammed I S

Literature review and writing form the basis of every academic research and writing, and it is most significant and indispensable to every academic research work. Its systematic process of writing has, however, been mysterious, complex, messy and boring, especially to inexperienced researchers and postgraduate students. This study explored the mysteries and ease with academic literature, writing and review. The study used secondary source to gather data and for the analysis, and found that academic literature writing and review comprise of different patterns and systems, dependent upon the nature and character of the research, the writing in contexts and its specific objectives; there are different types of literature and writing in academics, and while no one way is universally accepted by all at the same time, different approaches are required for different types of review and writings. The difficulty in understanding, reviewing and writing of literature mainly emanates from failure right from the inception to clearly identify what precisely the reviewer wants and how to go about looking for it in a systematic and comprehensive manner. Reviewing and writing of academic literature is a herculean task and for it to be successful there must be focus, specific objectives, adequate and timely provision and access to relevant materials. With proper understanding, it can be mastered and made easy. The study is essential for academics and post graduate students who must undergo literature review and writing at varying stages, especially at critical, stipulated and limited times.

Auxiliadora Padilha

HUMANUS DISCOURSE

Humanus Discourse

The importance of literature review in academic writing of different categories, levels, and purposes cannot be overemphasized. The literature review establishes both the relevance and justifies why new research is relevant. It is through a literature review that a gap would be established, and which the new research would fix. Once the literature review sits properly in the research work, the objectives/research questions naturally fall into their proper perspective. Invariably, other chapters of the research work would be impacted as well. In most instances, scanning through literature also provides you with the need and justification for your research and may also well leave a hint for further research. Literature review in most instances exposes a researcher to the right methodology to use. The literature review is the nucleus of a research work that might when gotten right spotlights a work and can as well derail a research work when done wrongly. This paper seeks to unveil the practical guides to writing a literature review, from purpose, and components to tips. It follows through the exposition of secondary literature. It exposes the challenges in writing a literature review and at the same time recommended tips that when followed will impact the writing of the literature review.

John Schostak

The literature is a multiplicity of voices. With each voice agendas emerge. Each text is itself a framing of voices and their agendas, shaped to present a debate slanted towards a conclusion. Within that debate can often be detected the friends, the strangers, the guests, the hosts and the enemies that are entertained by the writer. So, there is a problem. It is that whilst acts of framing bring and impose order, those very processes of ordering and categorisation select and edit so that some things are chosen to be foregrounded, others to be background and yet others to be excluded. In the writing task, agenda setting and framing pin possibilities and options down to what is regarded as 'realistic', 'plausible', 'do-able', 'true'. However, there has to be a moment when the literature appears like the vertigo experienced over a sheer and endless drop. Engagement with the literature is the essential step in widening out, indeed seeing the limitless possibilities for open debate with a public extending over centuries, even millennia. Making a voice map of the public space of debate is a way of trying to locate what is at stake in adopting a given way of framing the world and its agendas. Getting a sense of the historical development of major debates, discovering the tributaries, the dead-ends, the forgotten, the overlooked is all a part of the gradual sense of knowing where you are, where you stand, in relation to others. In particular, who claims to know what and why? What kinds of arguments are being made, and why? What are the assumptions at the back of explanations and theories? What happens if the assumptions are challenged or changed? From a review of the literature it is possible to sketch and fill out the details of the problematic, that is, the knot of problems, issues, concerns, interests that each of the voices in the literature have historically addressed. In determining how they address their chosen problems, the outlines of their methodologies can be formulated. Then it is a question of what is at stake expressed by each voice in the choices they make in exploring, examining and forming their conclusions using their chosen methodologies in relation to the problems they address. Which voices have they included in their own reviews of the debates, which have they excluded and why? By asking such questions as these a literature review then can be designed specifically to increase the power of a given argument, set of findings, recommendations and conclusions that have implications for action.

Tatam Chiway , Abdullah Ramdhani , Muhammad Ali Ramdhani

Mahendra Budhathoki

Literature review or research synthesis is an essential component in research field. Novice and student researchers usually take it as a required burden in research, and present haphazardly under sub-topics in research. There is the problem of application and correlating LR with their studies. The main purpose of this paper is to present introduction of LR/research synthesis, its functions and methods in research. LR/research synthesis consists of searching relevant literature, discussing the findings and evidence, correlating the individual studies, interpreting critically, and synthesizing them to build an argument for future research. It is a review article based on qualitative research, but not based on primary data. This paper contributes to answer the questions of writing a LR or synthesis paper, and becomes a useful reference material to novice and student researchers of higher education.

Nicolao Buenaventura

QUEST JOURNALS

Literature review and writing form the basis of research to which it is indispensable. Its systematic process however, remains mysterious, complex and problematic especially to postgraduate students most of whom undertake research for the first time at graduate level. This paper explored the challenges, strengths and mysteries with which literature review and writing was undertaken by graduate students at both master's and Doctoral levels. The paper used primary sources to gather data from graduate students' theses and proposals. Data from those sources revealed how literature review and writing showed different patterns depending on the nature of the research, and the specific objectives of the study. Similarly, different approaches were found to be suitable to different research contexts and methods. The challenges in writing and reviewing literature mainly springs from the failure to clearly define the research problem which propels clarity in the presentation of literature.

Publications

Cherley C Du Plessis

The ability to conduct an explicit and robust literature review by students, scholars or scientists is critical in producing excellent journal articles, academic theses, academic dissertations or working papers. A literature review is an evaluation of existing research works on a specific academic topic, theme or subject to identify gaps and propose future research agenda. Many postgraduate students in higher education institutions lack the necessary skills and understanding to conduct in-depth literature reviews. This may lead to the presentation of incorrect, false or biased inferences in their theses or dissertations. This study offers scientific knowledge on how literature reviews in different fields of study could be conducted to mitigate against biased inferences such as unscientific analogies and baseless recommendations. The literature review is presented as a process that involves several activities including searching, identifying, reading, summarising, compiling, analysing, interpreting and referencing. We hope this article serves as reference material to improve the academic rigour in the literature review chapters of postgraduate students' theses or dissertations. This article prompts established scholars to explore more innovative ways through which scientific literature reviews can be conducted to identify gaps (empirical, knowledge, theoretical, methodological, application and population gap) and propose a future research agenda.

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

RELATED PAPERS

Cogent Education

Zahra Shahsavar

Gloria Latham

InSITE Conference

shardul pandya

KANNANAYAKAL RAJAN

Review of General Psychology

Roy Baumeister

Mansoor Ahmed Khan

Raquel M T Lothringer , DIANA M WAIGANDT

Gavin Mount

dania azira

tahir desta

Sadruddin Qutoshi

tecnico emergencias

Mario Ivanov

shayal kumar

Dr. Diwakar Regmi

Oladokun Segun

Andrew Johnson

Mary Hanrahan

Advances in Social Sciences Research Journal

Rita Akele Twumasi

International Journal on Studies in English Language and Literature

JOSEPHINE DANIELS

Markéta Gregorová

International Journal of Modern Languages and Applied Linguistics

PROFESSOR DR ANEALKA AZIZ HUSSIN

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

• Privacy Policy

Home » Literature Review – Types Writing Guide and Examples

Literature Review – Types Writing Guide and Examples

Table of Contents

Literature Review

Definition:

A literature review is a comprehensive and critical analysis of the existing literature on a particular topic or research question. It involves identifying, evaluating, and synthesizing relevant literature, including scholarly articles, books, and other sources, to provide a summary and critical assessment of what is known about the topic.

Types of Literature Review

Types of Literature Review are as follows:

• Narrative literature review : This type of review involves a comprehensive summary and critical analysis of the available literature on a particular topic or research question. It is often used as an introductory section of a research paper.
• Systematic literature review: This is a rigorous and structured review that follows a pre-defined protocol to identify, evaluate, and synthesize all relevant studies on a specific research question. It is often used in evidence-based practice and systematic reviews.
• Meta-analysis: This is a quantitative review that uses statistical methods to combine data from multiple studies to derive a summary effect size. It provides a more precise estimate of the overall effect than any individual study.
• Scoping review: This is a preliminary review that aims to map the existing literature on a broad topic area to identify research gaps and areas for further investigation.
• Critical literature review : This type of review evaluates the strengths and weaknesses of the existing literature on a particular topic or research question. It aims to provide a critical analysis of the literature and identify areas where further research is needed.
• Conceptual literature review: This review synthesizes and integrates theories and concepts from multiple sources to provide a new perspective on a particular topic. It aims to provide a theoretical framework for understanding a particular research question.
• Rapid literature review: This is a quick review that provides a snapshot of the current state of knowledge on a specific research question or topic. It is often used when time and resources are limited.
• Thematic literature review : This review identifies and analyzes common themes and patterns across a body of literature on a particular topic. It aims to provide a comprehensive overview of the literature and identify key themes and concepts.
• Realist literature review: This review is often used in social science research and aims to identify how and why certain interventions work in certain contexts. It takes into account the context and complexities of real-world situations.
• State-of-the-art literature review : This type of review provides an overview of the current state of knowledge in a particular field, highlighting the most recent and relevant research. It is often used in fields where knowledge is rapidly evolving, such as technology or medicine.
• Integrative literature review: This type of review synthesizes and integrates findings from multiple studies on a particular topic to identify patterns, themes, and gaps in the literature. It aims to provide a comprehensive understanding of the current state of knowledge on a particular topic.
• Umbrella literature review : This review is used to provide a broad overview of a large and diverse body of literature on a particular topic. It aims to identify common themes and patterns across different areas of research.
• Historical literature review: This type of review examines the historical development of research on a particular topic or research question. It aims to provide a historical context for understanding the current state of knowledge on a particular topic.
• Problem-oriented literature review : This review focuses on a specific problem or issue and examines the literature to identify potential solutions or interventions. It aims to provide practical recommendations for addressing a particular problem or issue.
• Mixed-methods literature review : This type of review combines quantitative and qualitative methods to synthesize and analyze the available literature on a particular topic. It aims to provide a more comprehensive understanding of the research question by combining different types of evidence.

Parts of Literature Review

Parts of a literature review are as follows:

Introduction

The introduction of a literature review typically provides background information on the research topic and why it is important. It outlines the objectives of the review, the research question or hypothesis, and the scope of the review.

Literature Search

This section outlines the search strategy and databases used to identify relevant literature. The search terms used, inclusion and exclusion criteria, and any limitations of the search are described.

Literature Analysis

The literature analysis is the main body of the literature review. This section summarizes and synthesizes the literature that is relevant to the research question or hypothesis. The review should be organized thematically, chronologically, or by methodology, depending on the research objectives.

Critical Evaluation

Critical evaluation involves assessing the quality and validity of the literature. This includes evaluating the reliability and validity of the studies reviewed, the methodology used, and the strength of the evidence.

The conclusion of the literature review should summarize the main findings, identify any gaps in the literature, and suggest areas for future research. It should also reiterate the importance of the research question or hypothesis and the contribution of the literature review to the overall research project.

The references list includes all the sources cited in the literature review, and follows a specific referencing style (e.g., APA, MLA, Harvard).

How to write Literature Review

Here are some steps to follow when writing a literature review:

• Define your research question or topic : Before starting your literature review, it is essential to define your research question or topic. This will help you identify relevant literature and determine the scope of your review.
• Conduct a comprehensive search: Use databases and search engines to find relevant literature. Look for peer-reviewed articles, books, and other academic sources that are relevant to your research question or topic.
• Evaluate the sources: Once you have found potential sources, evaluate them critically to determine their relevance, credibility, and quality. Look for recent publications, reputable authors, and reliable sources of data and evidence.
• Organize your sources: Group the sources by theme, method, or research question. This will help you identify similarities and differences among the literature, and provide a structure for your literature review.
• Analyze and synthesize the literature : Analyze each source in depth, identifying the key findings, methodologies, and conclusions. Then, synthesize the information from the sources, identifying patterns and themes in the literature.
• Write the literature review : Start with an introduction that provides an overview of the topic and the purpose of the literature review. Then, organize the literature according to your chosen structure, and analyze and synthesize the sources. Finally, provide a conclusion that summarizes the key findings of the literature review, identifies gaps in knowledge, and suggests areas for future research.
• Edit and proofread: Once you have written your literature review, edit and proofread it carefully to ensure that it is well-organized, clear, and concise.

Examples of Literature Review

Here’s an example of how a literature review can be conducted for a thesis on the topic of “ The Impact of Social Media on Teenagers’ Mental Health”:

• Start by identifying the key terms related to your research topic. In this case, the key terms are “social media,” “teenagers,” and “mental health.”
• Use academic databases like Google Scholar, JSTOR, or PubMed to search for relevant articles, books, and other publications. Use these keywords in your search to narrow down your results.
• Evaluate the sources you find to determine if they are relevant to your research question. You may want to consider the publication date, author’s credentials, and the journal or book publisher.
• Begin reading and taking notes on each source, paying attention to key findings, methodologies used, and any gaps in the research.
• Organize your findings into themes or categories. For example, you might categorize your sources into those that examine the impact of social media on self-esteem, those that explore the effects of cyberbullying, and those that investigate the relationship between social media use and depression.
• Synthesize your findings by summarizing the key themes and highlighting any gaps or inconsistencies in the research. Identify areas where further research is needed.
• Use your literature review to inform your research questions and hypotheses for your thesis.

For example, after conducting a literature review on the impact of social media on teenagers’ mental health, a thesis might look like this:

“Using a mixed-methods approach, this study aims to investigate the relationship between social media use and mental health outcomes in teenagers. Specifically, the study will examine the effects of cyberbullying, social comparison, and excessive social media use on self-esteem, anxiety, and depression. Through an analysis of survey data and qualitative interviews with teenagers, the study will provide insight into the complex relationship between social media use and mental health outcomes, and identify strategies for promoting positive mental health outcomes in young people.”

Reference: Smith, J., Jones, M., & Lee, S. (2019). The effects of social media use on adolescent mental health: A systematic review. Journal of Adolescent Health, 65(2), 154-165. doi:10.1016/j.jadohealth.2019.03.024

Reference Example: Author, A. A., Author, B. B., & Author, C. C. (Year). Title of article. Title of Journal, volume number(issue number), page range. doi:0000000/000000000000 or URL

Applications of Literature Review

some applications of literature review in different fields:

• Social Sciences: In social sciences, literature reviews are used to identify gaps in existing research, to develop research questions, and to provide a theoretical framework for research. Literature reviews are commonly used in fields such as sociology, psychology, anthropology, and political science.
• Natural Sciences: In natural sciences, literature reviews are used to summarize and evaluate the current state of knowledge in a particular field or subfield. Literature reviews can help researchers identify areas where more research is needed and provide insights into the latest developments in a particular field. Fields such as biology, chemistry, and physics commonly use literature reviews.
• Health Sciences: In health sciences, literature reviews are used to evaluate the effectiveness of treatments, identify best practices, and determine areas where more research is needed. Literature reviews are commonly used in fields such as medicine, nursing, and public health.
• Humanities: In humanities, literature reviews are used to identify gaps in existing knowledge, develop new interpretations of texts or cultural artifacts, and provide a theoretical framework for research. Literature reviews are commonly used in fields such as history, literary studies, and philosophy.

Role of Literature Review in Research

Here are some applications of literature review in research:

• Identifying Research Gaps : Literature review helps researchers identify gaps in existing research and literature related to their research question. This allows them to develop new research questions and hypotheses to fill those gaps.
• Developing Theoretical Framework: Literature review helps researchers develop a theoretical framework for their research. By analyzing and synthesizing existing literature, researchers can identify the key concepts, theories, and models that are relevant to their research.
• Selecting Research Methods : Literature review helps researchers select appropriate research methods and techniques based on previous research. It also helps researchers to identify potential biases or limitations of certain methods and techniques.
• Data Collection and Analysis: Literature review helps researchers in data collection and analysis by providing a foundation for the development of data collection instruments and methods. It also helps researchers to identify relevant data sources and identify potential data analysis techniques.
• Communicating Results: Literature review helps researchers to communicate their results effectively by providing a context for their research. It also helps to justify the significance of their findings in relation to existing research and literature.

Purpose of Literature Review

Some of the specific purposes of a literature review are as follows:

• To provide context: A literature review helps to provide context for your research by situating it within the broader body of literature on the topic.
• To identify gaps and inconsistencies: A literature review helps to identify areas where further research is needed or where there are inconsistencies in the existing literature.
• To synthesize information: A literature review helps to synthesize the information from multiple sources and present a coherent and comprehensive picture of the current state of knowledge on the topic.
• To identify key concepts and theories : A literature review helps to identify key concepts and theories that are relevant to your research question and provide a theoretical framework for your study.
• To inform research design: A literature review can inform the design of your research study by identifying appropriate research methods, data sources, and research questions.

Characteristics of Literature Review

Some Characteristics of Literature Review are as follows:

• Identifying gaps in knowledge: A literature review helps to identify gaps in the existing knowledge and research on a specific topic or research question. By analyzing and synthesizing the literature, you can identify areas where further research is needed and where new insights can be gained.
• Establishing the significance of your research: A literature review helps to establish the significance of your own research by placing it in the context of existing research. By demonstrating the relevance of your research to the existing literature, you can establish its importance and value.
• Informing research design and methodology : A literature review helps to inform research design and methodology by identifying the most appropriate research methods, techniques, and instruments. By reviewing the literature, you can identify the strengths and limitations of different research methods and techniques, and select the most appropriate ones for your own research.
• Supporting arguments and claims: A literature review provides evidence to support arguments and claims made in academic writing. By citing and analyzing the literature, you can provide a solid foundation for your own arguments and claims.
• I dentifying potential collaborators and mentors: A literature review can help identify potential collaborators and mentors by identifying researchers and practitioners who are working on related topics or using similar methods. By building relationships with these individuals, you can gain valuable insights and support for your own research and practice.
• Keeping up-to-date with the latest research : A literature review helps to keep you up-to-date with the latest research on a specific topic or research question. By regularly reviewing the literature, you can stay informed about the latest findings and developments in your field.

Advantages of Literature Review

There are several advantages to conducting a literature review as part of a research project, including:

• Establishing the significance of the research : A literature review helps to establish the significance of the research by demonstrating the gap or problem in the existing literature that the study aims to address.
• Identifying key concepts and theories: A literature review can help to identify key concepts and theories that are relevant to the research question, and provide a theoretical framework for the study.
• Supporting the research methodology : A literature review can inform the research methodology by identifying appropriate research methods, data sources, and research questions.
• Providing a comprehensive overview of the literature : A literature review provides a comprehensive overview of the current state of knowledge on a topic, allowing the researcher to identify key themes, debates, and areas of agreement or disagreement.
• Identifying potential research questions: A literature review can help to identify potential research questions and areas for further investigation.
• Avoiding duplication of research: A literature review can help to avoid duplication of research by identifying what has already been done on a topic, and what remains to be done.
• Enhancing the credibility of the research : A literature review helps to enhance the credibility of the research by demonstrating the researcher’s knowledge of the existing literature and their ability to situate their research within a broader context.

Limitations of Literature Review

Limitations of Literature Review are as follows:

• Limited scope : Literature reviews can only cover the existing literature on a particular topic, which may be limited in scope or depth.
• Publication bias : Literature reviews may be influenced by publication bias, which occurs when researchers are more likely to publish positive results than negative ones. This can lead to an incomplete or biased picture of the literature.
• Quality of sources : The quality of the literature reviewed can vary widely, and not all sources may be reliable or valid.
• Time-limited: Literature reviews can become quickly outdated as new research is published, making it difficult to keep up with the latest developments in a field.
• Subjective interpretation : Literature reviews can be subjective, and the interpretation of the findings can vary depending on the researcher’s perspective or bias.
• Lack of original data : Literature reviews do not generate new data, but rather rely on the analysis of existing studies.
• Risk of plagiarism: It is important to ensure that literature reviews do not inadvertently contain plagiarism, which can occur when researchers use the work of others without proper attribution.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Implications in Research – Types, Examples and...

Ethical Considerations – Types, Examples and...

Context of the Study – Writing Guide and Examples

Future Research – Thesis Guide

Table of Contents – Types, Formats, Examples

Institutional Review Board – Application Sample...

Different types of literature review techniques followed in a research

The purpose of a literature review is to summarise crucial previous research done on a topic. It is an integral part of the research. From the literature review, the researcher finds out what is lacking in an existing area of research. There are several types of literature review techniques that can be followed in a thesis. They depend on the type of research and the purpose of the study. This article explains the characteristics and fundamental differences in the literature review techniques.

Narrative or traditional literature review technique

This technique of reviewing the literature is the most popular among thesis scholars, and also an important part of any study (Xiao and Watson, 2019). It helps to establish the conceptual and theoretical framework of thesis research. The aim is to identify the studies that have a problem of interest. It does not have a research problem and strategy. It can be used only when there is a topic of interest. The steps in this type of literature review technique include defining the audience, topic and a search for literature. It involves being critical and finding a logical structure. It describes and appraises the articles.

However, the method for selecting articles may not be described. The questions are broader, and the evaluation is variable. Narrative or traditional literature review technique can be of three further types:

• A general literature review is the crucial dimension of current knowledge of a topic.
• A historical literature review is based on the examination of research throughout a period. Generally, it starts with the first time a theory or topic has emerged.
• The methodological literature review describes the way a research design or methods are described.

Systematic literature review technique

A systematic literature review technique helps to identify and appraise the investigation. It is done to get an answer to the formulated question. It aims at determining the appropriate response to the research problem. This sounds similar to the narrative literature review technique but there are a few fundamental differences such as:

• The research questions are specific.
• The study selection is based on criteria.
• The evaluation style is critical and rigorous.
• The inferences are usually evidence-based.
• There is a specific format for presenting the review and it usually involves tables.

A systematic review is used to draw decision-making. Therefore it cannot be a part of all types of theses. The systematic literature review technique is majorly used in the field of science, psychology, medicine, and social sciences. There should be certain principles to be kept in mind while following the systemic review technique. These are clarity, transparency, equality, focus, and accessibility in the representation of results so that chances of bias are minimised.

Critical review technique

The critical review technique is more than a summary of the current studies. It is a detailed discussion of a topic or a study involving various views (Eaton, 2018). The critical review technique is the exercise of careful thinking considering strengths and weaknesses. A critical review requires seeking information and reviewing literature effectively. The aim is to conduct a critical assessment of the focus area of research. The essential characteristic is that it makes a reasoned judgment. A well-made critical review reveals the suitability and relevance of a problem. In addition to this, it weighs the study’s significance.

This type of review technique is used when the study needs to be explored critically. It makes an accurate and precise conclusion about the problem. The critical review is presented to make an unbiased, critical analysis. It involves strong and convincing opinions and arguments. The reasoning is vital in this type of review technique.

Theoretical framework review technique

The purpose of a theoretical framework in a thesis is to support the study with relevant theories. When a researcher explains the theory underpinning the idea, it makes the study stronger and more reliable. There may be many theories supporting a single idea. It is important to choose the one that best explains the main concepts and depicts the relation between them. A robust theoretical framework in the research helps explain, interpret, and generalize findings (Frederiksen et. al . 2018). It can increase the success rate of the research. Theories and concepts also signify the understanding of the researcher. In a way, it also acts as the blueprint of the study.

A theoretical framework is particularly essential in subjects like social sciences, economics and consumer psychology. In these subjects, new developments are built on existing theories that constantly evolve but do not become obsolete. Such theses require the identification of at least two theories. Diagrammatic presentation of theories is recommended as they help the reader to visualise better.

Differences in the different literature review techniques

No matter which type of literature review is chosen for a study, it must be written with a proper structure and format. Emotional phrasing and unjustified claims should be avoided. Moreover, the researcher should not use irrelevant content and non-scholarly sources. The focus must be on choosing the right type of review and finding a logical structure.

• Cooper, C., Booth, A., Varley-Campbell, J., Britten, N. and Garside, R., 2018. Defining the process to literature searching in systematic reviews: a literature review of guidance and supporting studies.  BMC medical research methodology ,  18 (1), pp.1-14.
• Dodgson, J.E., 2021. Critical analysis: The often-missing step in conducting literature review research.  Journal of Human Lactation ,  37 (1), pp.27-32.
• Eaton, S.E., 2018. Educational research literature reviews: Understanding the hierarchy of sources.
• Frederiksen, L., Phelps, S.F. and Kimmons, R., 2018. What is a Literature Review?.  Rapid Academic Writing .
• Hart, C., 2018. Doing a literature review: Releasing the research imagination.
• Snyder, H., 2019. Literature review as a research methodology: An overview and guidelines.  Journal of business research ,  104 , pp.333-339.
• Xiao, Y. and Watson, M., 2019. Guidance on conducting a systematic literature review.  Journal of Planning Education and Research ,  39 (1), pp.93-112.
• Priya Chetty
• Kinjal Aacha

I am a management graduate with specialisation in Marketing and Finance. I have over 12 years' experience in research and analysis. This includes fundamental and applied research in the domains of management and social sciences. I am well versed with academic research principles. Over the years i have developed a mastery in different types of data analysis on different applications like SPSS, Amos, and NVIVO. My expertise lies in inferring the findings and creating actionable strategies based on them. 

Over the past decade I have also built a profile as a researcher on Project Guru's Knowledge Tank division. I have penned over 200 articles that have earned me 400+ citations so far. My Google Scholar profile can be accessed here . 

I now consult university faculty through Faculty Development Programs (FDPs) on the latest developments in the field of research. I also guide individual researchers on how they can commercialise their inventions or research findings. Other developments im actively involved in at Project Guru include strengthening the "Publish" division as a bridge between industry and academia by bringing together experienced research persons, learners, and practitioners to collaboratively work on a common goal. 

I want to work as an academic writer in business management, marketing, and human resource management to develop my knowledge base further and utilizing it.

• Click to share on Twitter (Opens in new window)
• Click to share on Facebook (Opens in new window)
• Click to share on LinkedIn (Opens in new window)
• Click to share on WhatsApp (Opens in new window)
• Click to share on Telegram (Opens in new window)

Notify me of follow-up comments by email.

proofreading

• Systematic review
• Open access
• Published: 07 August 2024

Models and frameworks for assessing the implementation of clinical practice guidelines: a systematic review

• Nicole Freitas de Mello   ORCID: orcid.org/0000-0002-5228-6691 1 , 2 ,
• Sarah Nascimento Silva   ORCID: orcid.org/0000-0002-1087-9819 3 ,
• Dalila Fernandes Gomes   ORCID: orcid.org/0000-0002-2864-0806 1 , 2 ,
• Juliana da Motta Girardi   ORCID: orcid.org/0000-0002-7547-7722 4 &
• Jorge Otávio Maia Barreto   ORCID: orcid.org/0000-0002-7648-0472 2 , 4  

Implementation Science volume  19 , Article number:  59 ( 2024 ) Cite this article

679 Accesses

6 Altmetric

Metrics details

The implementation of clinical practice guidelines (CPGs) is a cyclical process in which the evaluation stage can facilitate continuous improvement. Implementation science has utilized theoretical approaches, such as models and frameworks, to understand and address this process. This article aims to provide a comprehensive overview of the models and frameworks used to assess the implementation of CPGs.

A systematic review was conducted following the Cochrane methodology, with adaptations to the "selection process" due to the unique nature of this review. The findings were reported following PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) reporting guidelines. Electronic databases were searched from their inception until May 15, 2023. A predetermined strategy and manual searches were conducted to identify relevant documents from health institutions worldwide. Eligible studies presented models and frameworks for assessing the implementation of CPGs. Information on the characteristics of the documents, the context in which the models were used (specific objectives, level of use, type of health service, target group), and the characteristics of each model or framework (name, domain evaluated, and model limitations) were extracted. The domains of the models were analyzed according to the key constructs: strategies, context, outcomes, fidelity, adaptation, sustainability, process, and intervention. A subgroup analysis was performed grouping models and frameworks according to their levels of use (clinical, organizational, and policy) and type of health service (community, ambulatorial, hospital, institutional). The JBI’s critical appraisal tools were utilized by two independent researchers to assess the trustworthiness, relevance, and results of the included studies.

Database searches yielded 14,395 studies, of which 80 full texts were reviewed. Eight studies were included in the data analysis and four methodological guidelines were additionally included from the manual search. The risk of bias in the studies was considered non-critical for the results of this systematic review. A total of ten models/frameworks for assessing the implementation of CPGs were found. The level of use was mainly policy, the most common type of health service was institutional, and the major target group was professionals directly involved in clinical practice. The evaluated domains differed between the models and there were also differences in their conceptualization. All the models addressed the domain "Context", especially at the micro level (8/12), followed by the multilevel (7/12). The domains "Outcome" (9/12), "Intervention" (8/12), "Strategies" (7/12), and "Process" (5/12) were frequently addressed, while "Sustainability" was found only in one study, and "Fidelity/Adaptation" was not observed.

Conclusions

The use of models and frameworks for assessing the implementation of CPGs is still incipient. This systematic review may help stakeholders choose or adapt the most appropriate model or framework to assess CPGs implementation based on their specific health context.

Trial registration

PROSPERO (International Prospective Register of Systematic Reviews) registration number: CRD42022335884. Registered on June 7, 2022.

Peer Review reports

Contributions to the literature

Although the number of theoretical approaches has grown in recent years, there are still important gaps to be explored in the use of models and frameworks to assess the implementation of clinical practice guidelines (CPGs). This systematic review aims to contribute knowledge to overcome these gaps.

Despite the great advances in implementation science, evaluating the implementation of CPGs remains a challenge, and models and frameworks could support improvements in this field.

This study demonstrates that the available models and frameworks do not cover all characteristics and domains necessary for a complete evaluation of CPGs implementation.

The presented findings contribute to the field of implementation science, encouraging debate on choices and adaptations of models and frameworks for implementation research and evaluation.

Substantial investments have been made in clinical research and development in recent decades, increasing the medical knowledge base and the availability of health technologies [ 1 ]. The use of clinical practice guidelines (CPGs) has increased worldwide to guide best health practices and to maximize healthcare investments. A CPG can be defined as "any formal statements systematically developed to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances" [ 2 ] and has the potential to improve patient care by promoting interventions of proven benefit and discouraging ineffective interventions. Furthermore, they can promote efficiency in resource allocation and provide support for managers and health professionals in decision-making [ 3 , 4 ].

However, having a quality CPG does not guarantee that the expected health benefits will be obtained. In fact, putting these devices to use still presents a challenge for most health services across distinct levels of government. In addition to the development of guidelines with high methodological rigor, those recommendations need to be available to their users; these recommendations involve the diffusion and dissemination stages, and they need to be used in clinical practice (implemented), which usually requires behavioral changes and appropriate resources and infrastructure. All these stages involve an iterative and complex process called implementation, which is defined as the process of putting new practices within a setting into use [ 5 , 6 ].

Implementation is a cyclical process, and the evaluation is one of its key stages, which allows continuous improvement of CPGs development and implementation strategies. It consists of verifying whether clinical practice is being performed as recommended (process evaluation or formative evaluation) and whether the expected results and impact are being reached (summative evaluation) [ 7 , 8 , 9 ]. Although the importance of the implementation evaluation stage has been recognized, research on how these guidelines are implemented is scarce [ 10 ]. This paper focused on the process of assessing CPGs implementation.

To understand and improve this complex process, implementation science provides a systematic set of principles and methods to integrate research findings and other evidence-based practices into routine practice and improve the quality and effectiveness of health services and care [ 11 ]. The field of implementation science uses theoretical approaches that have varying degrees of specificity based on the current state of knowledge and are structured based on theories, models, and frameworks [ 5 , 12 , 13 ]. A "Model" is defined as "a simplified depiction of a more complex world with relatively precise assumptions about cause and effect", and a "framework" is defined as "a broad set of constructs that organize concepts and data descriptively without specifying causal relationships" [ 9 ]. Although these concepts are distinct, in this paper, their use will be interchangeable, as they are typically like checklists of factors relevant to various aspects of implementation.

There are a variety of theoretical approaches available in implementation science [ 5 , 14 ], which can make choosing the most appropriate challenging [ 5 ]. Some models and frameworks have been categorized as "evaluation models" by providing a structure for evaluating implementation endeavors [ 15 ], even though theoretical approaches from other categories can also be applied for evaluation purposes because they specify concepts and constructs that may be operationalized and measured [ 13 ]. Two frameworks that can specify implementation aspects that should be evaluated as part of intervention studies are RE-AIM (Reach, Effectiveness, Adoption, Implementation, Maintenance) [ 16 ] and PRECEDE-PROCEED (Predisposing, Reinforcing and Enabling Constructs in Educational Diagnosis and Evaluation-Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development) [ 17 ]. Although the number of theoretical approaches has grown in recent years, the use of models and frameworks to evaluate the implementation of guidelines still seems to be a challenge.

This article aims to provide a complete map of the models and frameworks applied to assess the implementation of CPGs. The aim is also to subside debate and choices on models and frameworks for the research and evaluation of the implementation processes of CPGs and thus to facilitate the continued development of the field of implementation as well as to contribute to healthcare policy and practice.

A systematic review was conducted following the Cochrane methodology [ 18 ], with adaptations to the "selection process" due to the unique nature of this review (details can be found in the respective section). The review protocol was registered in PROSPERO (registration number: CRD42022335884) on June 7, 2022. This report adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines [ 19 ] and a completed checklist is provided in Additional File 1.

Eligibility criteria

The SDMO approach (Types of Studies, Types of Data, Types of Methods, Outcomes) [ 20 ] was utilized in this systematic review, outlined as follows:

Types of studies

All types of studies were considered for inclusion, as the assessment of CPG implementation can benefit from a diverse range of study designs, including randomized clinical trials/experimental studies, scale/tool development, systematic reviews, opinion pieces, qualitative studies, peer-reviewed articles, books, reports, and unpublished theses.

Studies were categorized based on their methodological designs, which guided the synthesis, risk of bias assessment, and presentation of results.

Study protocols and conference abstracts were excluded due to insufficient information for this review.

Types of data

Studies that evaluated the implementation of CPGs either independently or as part of a multifaceted intervention.

Guidelines for evaluating CPG implementation.

Inclusion of CPGs related to any context, clinical area, intervention, and patient characteristics.

No restrictions were placed on publication date or language.

Exclusion criteria

General guidelines were excluded, as this review focused on 'models for evaluating clinical practice guidelines implementation' rather than the guidelines themselves.

Studies that focused solely on implementation determinants as barriers and enablers were excluded, as this review aimed to explore comprehensive models/frameworks.

Studies evaluating programs and policies were excluded.

Studies that only assessed implementation strategies (isolated actions) rather than the implementation process itself were excluded.

Studies that focused solely on the impact or results of implementation (summative evaluation) were excluded.

Types of methods

Not applicable.

All potential models or frameworks for assessing the implementation of CPG (evaluation models/frameworks), as well as their characteristics: name; specific objectives; levels of use (clinical, organizational, and policy); health system (public, private, or both); type of health service (community, ambulatorial, hospital, institutional, homecare); domains or outcomes evaluated; type of recommendation evaluated; context; limitations of the model.

Model was defined as "a deliberated simplification of a phenomenon on a specific aspect" [ 21 ].

Framework was defined as "structure, overview outline, system, or plan consisting of various descriptive categories" [ 21 ].

Models or frameworks used solely for the CPG development, dissemination, or implementation phase.

Models/frameworks used solely for assessment processes other than implementation, such as for the development or dissemination phase.

Data sources and literature search

The systematic search was conducted on July 31, 2022 (and updated on May 15, 2023) in the following electronic databases: MEDLINE/PubMed, Centre for Reviews and Dissemination (CRD), the Cochrane Library, Cumulative Index to Nursing and Allied Health Literature (CINAHL), EMBASE, Epistemonikos, Global Health, Health Systems Evidence, PDQ-Evidence, PsycINFO, Rx for Change (Canadian Agency for Drugs and Technologies in Health, CADTH), Scopus, Web of Science and Virtual Health Library (VHL). The Google Scholar database was used for the manual selection of studies (first 10 pages).

Additionally, hand searches were performed on the lists of references included in the systematic reviews and citations of the included studies, as well as on the websites of institutions working on CPGs development and implementation: Guidelines International Networks (GIN), National Institute for Health and Care Excellence (NICE; United Kingdom), World Health Organization (WHO), Centers for Disease Control and Prevention (CDC; USA), Institute of Medicine (IOM; USA), Australian Department of Health and Aged Care (ADH), Healthcare Improvement Scotland (SIGN), National Health and Medical Research Council (NHMRC; Australia), Queensland Health, The Joanna Briggs Institute (JBI), Ministry of Health and Social Policy of Spain, Ministry of Health of Brazil and Capes Theses and Dissertations Catalog.

The search strategy combined terms related to "clinical practice guidelines" (practice guidelines, practice guidelines as topic, clinical protocols), "implementation", "assessment" (assessment, evaluation), and "models, framework". The free term "monitoring" was not used because it was regularly related to clinical monitoring and not to implementation monitoring. The search strategies adapted for the electronic databases are presented in an additional file (see Additional file 2).

Study selection process

The results of the literature search from scientific databases, excluding the CRD database, were imported into Mendeley Reference Management software to remove duplicates. They were then transferred to the Rayyan platform ( https://rayyan.qcri.org ) [ 22 ] for the screening process. Initially, studies related to the "assessment of implementation of the CPG" were selected. The titles were first screened independently by two pairs of reviewers (first selection: four reviewers, NM, JB, SS, and JG; update: a pair of reviewers, NM and DG). The title screening was broad, including all potentially relevant studies on CPG and the implementation process. Following that, the abstracts were independently screened by the same group of reviewers. The abstract screening was more focused, specifically selecting studies that addressed CPG and the evaluation of the implementation process. In the next step, full-text articles were reviewed independently by a pair of reviewers (NM, DG) to identify those that explicitly presented "models" or "frameworks" for assessing the implementation of the CPG. Disagreements regarding the eligibility of studies were resolved through discussion and consensus, and by a third reviewer (JB) when necessary. One reviewer (NM) conducted manual searches, and the inclusion of documents was discussed with the other reviewers.

Risk of bias assessment of studies

The selected studies were independently classified and evaluated according to their methodological designs by two investigators (NM and JG). This review employed JBI’s critical appraisal tools to assess the trustworthiness, relevance and results of the included studies [ 23 ] and these tools are presented in additional files (see Additional file 3 and Additional file 4). Disagreements were resolved by consensus or consultation with the other reviewers. Methodological guidelines and noncomparative and before–after studies were not evaluated because JBI does not have specific tools for assessing these types of documents. Although the studies were assessed for quality, they were not excluded on this basis.

Data extraction

The data was independently extracted by two reviewers (NM, DG) using a Microsoft Excel spreadsheet. Discrepancies were discussed and resolved by consensus. The following information was extracted:

Document characteristics : author; year of publication; title; study design; instrument of evaluation; country; guideline context;

Usage context of the models : specific objectives; level of use (clinical, organizational, and policy); type of health service (community, ambulatorial, hospital, institutional); target group (guideline developers, clinicians; health professionals; health-policy decision-makers; health-care organizations; service managers);

Model and framework characteristics : name, domain evaluated, and model limitations.

The set of information to be extracted, shown in the systematic review protocol, was adjusted to improve the organization of the analysis.

The "level of use" refers to the scope of the model used. "Clinical" was considered when the evaluation focused on individual practices, "organizational" when practices were within a health service institution, and "policy" when the evaluation was more systemic and covered different health services or institutions.

The "type of health service" indicated the category of health service where the model/framework was used (or can be used) to assess the implementation of the CPG, related to the complexity of healthcare. "Community" is related to primary health care; "ambulatorial" is related to secondary health care; "hospital" is related to tertiary health care; and "institutional" represented models/frameworks not specific to a particular type of health service.

The "target group" included stakeholders related to the use of the model/framework for evaluating the implementation of the CPG, such as clinicians, health professionals, guideline developers, health policy-makers, health organizations, and service managers.

The category "health system" (public, private, or both) mentioned in the systematic review protocol was not found in the literature obtained and was removed as an extraction variable. Similarly, the variables "type of recommendation evaluated" and "context" were grouped because the same information was included in the "guideline context" section of the study.

Some selected documents presented models or frameworks recognized by the scientific field, including some that were validated. However, some studies adapted the model to this context. Therefore, the domain analysis covered all models or frameworks domains evaluated by (or suggested for evaluation by) the document analyzed.

Data analysis and synthesis

The results were tabulated using narrative synthesis with an aggregative approach, without meta-analysis, aiming to summarize the documents descriptively for the organization, description, interpretation and explanation of the study findings [ 24 , 25 ].

The model/framework domains evaluated in each document were studied according to Nilsen et al.’s constructs: "strategies", "context", "outcomes", "fidelity", "adaptation" and "sustainability". For this study, "strategies" were described as structured and planned initiatives used to enhance the implementation of clinical practice [ 26 ].

The definition of "context" varies in the literature. Despite that, this review considered it as the set of circumstances or factors surrounding a particular implementation effort, such as organizational support, financial resources, social relations and support, leadership, and organizational culture [ 26 , 27 ]. The domain "context" was subdivided according to the level of health care into "micro" (individual perspective), "meso" (organizational perspective), "macro" (systemic perspective), and "multiple" (when there is an issue involving more than one level of health care).

The "outcomes" domain was related to the results of the implementation process (unlike clinical outcomes) and was stratified according to the following constructs: acceptability, appropriateness, feasibility, adoption, cost, and penetration. All these concepts align with the definitions of Proctor et al. (2011), although we decided to separate "fidelity" and "sustainability" as independent domains similar to Nilsen [ 26 , 28 ].

"Fidelity" and "adaptation" were considered the same domain, as they are complementary pieces of the same issue. In this study, implementation fidelity refers to how closely guidelines are followed as intended by their developers or designers. On the other hand, adaptation involves making changes to the content or delivery of a guideline to better fit the needs of a specific context. The "sustainability" domain was defined as evaluations about the continuation or permanence over time of the CPG implementation.

Additionally, the domain "process" was utilized to address issues related to the implementation process itself, rather than focusing solely on the outcomes of the implementation process, as done by Wang et al. [ 14 ]. Furthermore, the "intervention" domain was introduced to distinguish aspects related to the CPG characteristics that can impact its implementation, such as the complexity of the recommendation.

A subgroup analysis was performed with models and frameworks categorized based on their levels of use (clinical, organizational, and policy) and the type of health service (community, ambulatorial, hospital, institutional) associated with the CPG. The goal is to assist stakeholders (politicians, clinicians, researchers, or others) in selecting the most suitable model for evaluating CPG implementation based on their specific health context.

Search results

Database searches yielded 26,011 studies, of which 107 full texts were reviewed. During the full-text review, 99 articles were excluded: 41 studies did not mention a model or framework for assessing the implementation of the CPG, 31 studies evaluated only implementation strategies (isolated actions) rather than the implementation process itself, and 27 articles were not related to the implementation assessment. Therefore, eight studies were included in the data analysis. The updated search did not reveal additional relevant studies. The main reason for study exclusion was that they did not use models or frameworks to assess CPG implementation. Additionally, four methodological guidelines were included from the manual search (Fig.  1 ).

PRISMA diagram. Acronyms: ADH—Australian Department of Health, CINAHL—Cumulative Index to Nursing and Allied Health Literature, CDC—Centers for Disease Control and Prevention, CRD—Centre for Reviews and Dissemination, GIN—Guidelines International Networks, HSE—Health Systems Evidence, IOM—Institute of Medicine, JBI—The Joanna Briggs Institute, MHB—Ministry of Health of Brazil, NICE—National Institute for Health and Care Excellence, NHMRC—National Health and Medical Research Council, MSPS – Ministerio de Sanidad Y Política Social (Spain), SIGN—Scottish Intercollegiate Guidelines Network, VHL – Virtual Health Library, WHO—World Health Organization. Legend: Reason A –The study evaluated only implementation strategies (isolated actions) rather than the implementation process itself. Reason B – The study did not mention a model or framework for assessing the implementation of the intervention. Reason C – The study was not related to the implementation assessment. Adapted from Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;372:n71. https://doi.org/10.1136/bmj.n71 . For more information, visit:

According to the JBI’s critical appraisal tools, the overall assessment of the studies indicates their acceptance for the systematic review.

The cross-sectional studies lacked clear information regarding "confounding factors" or "strategies to address confounding factors". This was understandable given the nature of the study, where such details are not typically included. However, the reviewers did not find this lack of information to be critical, allowing the studies to be included in the review. The results of this methodological quality assessment can be found in an additional file (see Additional file 5).

In the qualitative studies, there was some ambiguity regarding the questions: "Is there a statement locating the researcher culturally or theoretically?" and "Is the influence of the researcher on the research, and vice versa, addressed?". However, the reviewers decided to include the studies and deemed the methodological quality sufficient for the analysis in this article, based on the other information analyzed. The results of this methodological quality assessment can be found in an additional file (see Additional file 6).

Documents characteristics (Table  1 )

The documents were directed to several continents: Australia/Oceania (4/12) [ 31 , 33 , 36 , 37 ], North America (4/12 [ 30 , 32 , 38 , 39 ], Europe (2/12 [ 29 , 35 ] and Asia (2/12) [ 34 , 40 ]. The types of documents were classified as cross-sectional studies (4/12) [ 29 , 32 , 34 , 38 ], methodological guidelines (4/12) [ 33 , 35 , 36 , 37 ], mixed methods studies (3/12) [ 30 , 31 , 39 ] or noncomparative studies (1/12) [ 40 ]. In terms of the instrument of evaluation, most of the documents used a survey/questionnaire (6/12) [ 29 , 30 , 31 , 32 , 34 , 38 ], while three (3/12) used qualitative instruments (interviews, group discussions) [ 30 , 31 , 39 ], one used a checklist [ 37 ], one used an audit [ 33 ] and three (3/12) did not define a specific instrument to measure [ 35 , 36 , 40 ].

Considering the clinical areas covered, most studies evaluated the implementation of nonspecific (general) clinical areas [ 29 , 33 , 35 , 36 , 37 , 40 ]. However, some studies focused on specific clinical contexts, such as mental health [ 32 , 38 ], oncology [ 39 ], fall prevention [ 31 ], spinal cord injury [ 30 ], and sexually transmitted infections [ 34 ].

Usage context of the models (Table  1 )

Specific objectives.

All the studies highlighted the purpose of guiding the process of evaluating the implementation of CPGs, even if they evaluated CPGs from generic or different clinical areas.

Levels of use

The most common level of use of the models/frameworks identified to assess the implementation of CPGs was policy (6/12) [ 33 , 35 , 36 , 37 , 39 , 40 ]. In this level, the model is used in a systematic way to evaluate all the processes involved in CPGs implementation and is primarily related to methodological guidelines. This was followed by the organizational level of use (5/12) [ 30 , 31 , 32 , 38 , 39 ], where the model is used to evaluate the implementation of CPGs in a specific institution, considering its specific environment. Finally, the clinical level of use (2/12) [ 29 , 34 ] focuses on individual practice and the factors that can influence the implementation of CPGs by professionals.

Type of health service

Institutional services were predominant (5/12) [ 33 , 35 , 36 , 37 , 40 ] and included methodological guidelines and a study of model development and validation. Hospitals were the second most common type of health service (4/12) [ 29 , 30 , 31 , 34 ], followed by ambulatorial (2/12) [ 32 , 34 ] and community health services (1/12) [ 32 ]. Two studies did not specify which type of health service the assessment addressed [ 38 , 39 ].

Target group

The focus of the target group was professionals directly involved in clinical practice (6/12) [ 29 , 31 , 32 , 34 , 38 , 40 ], namely, health professionals and clinicians. Other less related stakeholders included guideline developers (2/12) [ 39 , 40 ], health policy decision makers (1/12) [ 39 ], and healthcare organizations (1/12) [ 39 ]. The target group was not defined in the methodological guidelines, although all the mentioned stakeholders could be related to these documents.

Model and framework characteristics

Models and frameworks for assessing the implementation of cpgs.

The Consolidated Framework for Implementation Research (CFIR) [ 31 , 38 ] and the Promoting Action on Research Implementation in Health Systems (PARiHS) framework [ 29 , 30 ] were the most commonly employed frameworks within the selected documents. The other models mentioned were: Goal commitment and implementation of practice guidelines framework [ 32 ]; Guideline to identify key indicators [ 35 ]; Guideline implementation checklist [ 37 ]; Guideline implementation evaluation tool [ 40 ]; JBI Implementation Framework [ 33 ]; Reach, effectiveness, adoption, implementation and maintenance (RE-AIM) framework [ 34 ]; The Guideline Implementability Framework [ 39 ] and an unnamed model [ 36 ].

Domains evaluated

The number of domains evaluated (or suggested for evaluation) by the documents varied between three and five, with the majority focusing on three domains. All the models addressed the domain "context", with a particular emphasis on the micro level of the health care context (8/12) [ 29 , 31 , 34 , 35 , 36 , 37 , 38 , 39 ], followed by the multilevel (7/12) [ 29 , 31 , 32 , 33 , 38 , 39 , 40 ], meso level (4/12) [ 30 , 35 , 39 , 40 ] and macro level (2/12) [ 37 , 39 ]. The "Outcome" domain was evaluated in nine models. Within this domain, the most frequently evaluated subdomain was "adoption" (6/12) [ 29 , 32 , 34 , 35 , 36 , 37 ], followed by "acceptability" (4/12) [ 30 , 32 , 35 , 39 ], "appropriateness" (3/12) [ 32 , 34 , 36 ], "feasibility" (3/12) [ 29 , 32 , 36 ], "cost" (1/12) [ 35 ] and "penetration" (1/12) [ 34 ]. Regarding the other domains, "Intervention" (8/12) [ 29 , 31 , 34 , 35 , 36 , 38 , 39 , 40 ], "Strategies" (7/12) [ 29 , 30 , 33 , 35 , 36 , 37 , 40 ] and "Process" (5/12) [ 29 , 31 , 32 , 33 , 38 ] were frequently addressed in the models, while "Sustainability" (1/12) [ 34 ] was only found in one model, and "Fidelity/Adaptation" was not observed. The domains presented by the models and frameworks and evaluated in the documents are shown in Table  2 .

Limitations of the models

Only two documents mentioned limitations in the use of the model or frameworks. These two studies reported limitations in the use of CFIR: "is complex and cumbersome and requires tailoring of the key variables to the specific context", and "this framework should be supplemented with other important factors and local features to achieve a sound basis for the planning and realization of an ongoing project" [ 31 , 38 ]. Limitations in the use of other models or frameworks are not reported.

Subgroup analysis

Following the subgroup analysis (Table  3 ), five different models/frameworks were utilized at the policy level by institutional health services. These included the Guideline Implementation Evaluation Tool [ 40 ], the NHMRC tool (model name not defined) [ 36 ], the JBI Implementation Framework + GRiP [ 33 ], Guideline to identify key indicators [ 35 ], and the Guideline implementation checklist [ 37 ]. Additionally, the "Guideline Implementability Framework" [ 39 ] was implemented at the policy level without restrictions based on the type of health service. Regarding the organizational level, the models used varied depending on the type of service. The "Goal commitment and implementation of practice guidelines framework" [ 32 ] was applied in community and ambulatory health services, while "PARiHS" [ 29 , 30 ] and "CFIR" [ 31 , 38 ] were utilized in hospitals. In contexts where the type of health service was not defined, "CFIR" [ 31 , 38 ] and "The Guideline Implementability Framework" [ 39 ] were employed. Lastly, at the clinical level, "RE-AIM" [ 34 ] was utilized in ambulatory and hospital services, and PARiHS [ 29 , 30 ] was specifically used in hospital services.

Key findings

This systematic review identified 10 models/ frameworks used to assess the implementation of CPGs in various health system contexts. These documents shared similar objectives in utilizing models and frameworks for assessment. The primary level of use was policy, the most common type of health service was institutional, and the main target group of the documents was professionals directly involved in clinical practice. The models and frameworks presented varied analytical domains, with sometimes divergent concepts used in these domains. This study is innovative in its emphasis on the evaluation stage of CPG implementation and in summarizing aspects and domains aimed at the practical application of these models.

The small number of documents contrasts with studies that present an extensive range of models and frameworks available in implementation science. The findings suggest that the use of models and frameworks to evaluate the implementation of CPGs is still in its early stages. Among the selected documents, there was a predominance of cross-sectional studies and methodological guidelines, which strongly influenced how the implementation evaluation was conducted. This was primarily done through surveys/questionnaires, qualitative methods (interviews, group discussions), and non-specific measurement instruments. Regarding the subject areas evaluated, most studies focused on a general clinical area, while others explored different clinical areas. This suggests that the evaluation of CPG implementation has been carried out in various contexts.

The models were chosen independently of the categories proposed in the literature, with their usage categorized for purposes other than implementation evaluation, as is the case with CFIR and PARiHS. This practice was described by Nilsen et al. who suggested that models and frameworks from other categories can also be applied for evaluation purposes because they specify concepts and constructs that may be operationalized and measured [ 14 , 15 , 42 , 43 ].

The results highlight the increased use of models and frameworks in evaluation processes at the policy level and institutional environments, followed by the organizational level in hospital settings. This finding contradicts a review that reported the policy level as an area that was not as well studied [ 44 ]. The use of different models at the institutional level is also emphasized in the subgroup analysis. This may suggest that the greater the impact (social, financial/economic, and organizational) of implementing CPGs, the greater the interest and need to establish well-defined and robust processes. In this context, the evaluation stage stands out as crucial, and the investment of resources and efforts to structure this stage becomes even more advantageous [ 10 , 45 ]. Two studies (16,7%) evaluated the implementation of CPGs at the individual level (clinical level). These studies stand out for their potential to analyze variations in clinical practice in greater depth.

In contrast to the level of use and type of health service most strongly indicated in the documents, with systemic approaches, the target group most observed was professionals directly involved in clinical practice. This suggests an emphasis on evaluating individual behaviors. This same emphasis is observed in the analysis of the models, in which there is a predominance of evaluating the micro level of the health context and the "adoption" subdomain, in contrast with the sub-use of domains such as "cost" and "process". Cassetti et al. observed the same phenomenon in their review, in which studies evaluating the implementation of CPGs mainly adopted a behavioral change approach to tackle those issues, without considering the influence of wider social determinants of health [ 10 ]. However, the literature widely reiterates that multiple factors impact the implementation of CPGs, and different actions are required to make them effective [ 6 , 46 , 47 ]. As a result, there is enormous potential for the development and adaptation of models and frameworks aimed at more systemic evaluation processes that consider institutional and organizational aspects.

In analyzing the model domains, most models focused on evaluating only some aspects of implementation (three domains). All models evaluated the "context", highlighting its significant influence on implementation [ 9 , 26 ]. Context is an essential effect modifier for providing research evidence to guide decisions on implementation strategies [ 48 ]. Contextualizing a guideline involves integrating research or other evidence into a specific circumstance [ 49 ]. The analysis of this domain was adjusted to include all possible contextual aspects, even if they were initially allocated to other domains. Some contextual aspects presented by the models vary in comprehensiveness, such as the assessment of the "timing and nature of stakeholder engagement" [ 39 ], which includes individual engagement by healthcare professionals and organizational involvement in CPG implementation. While the importance of context is universally recognized, its conceptualization and interpretation differ across studies and models. This divergence is also evident in other domains, consistent with existing literature [ 14 ]. Efforts to address this conceptual divergence in implementation science are ongoing, but further research and development are needed in this field [ 26 ].

The main subdomain evaluated was "adoption" within the outcome domain. This may be attributed to the ease of accessing information on the adoption of the CPG, whether through computerized system records, patient records, or self-reports from healthcare professionals or patients themselves. The "acceptability" subdomain pertains to the perception among implementation stakeholders that a particular CPG is agreeable, palatable or satisfactory. On the other hand, "appropriateness" encompasses the perceived fit, relevance or compatibility of the CPG for a specific practice setting, provider, or consumer, or its perceived fit to address a particular issue or problem [ 26 ]. Both subdomains are subjective and rely on stakeholders' interpretations and perceptions of the issue being analyzed, making them susceptible to reporting biases. Moreover, obtaining this information requires direct consultation with stakeholders, which can be challenging for some evaluation processes, particularly in institutional contexts.

The evaluation of the subdomains "feasibility" (the extent to which a CPG can be successfully used or carried out within a given agency or setting), "cost" (the cost impact of an implementation effort), and "penetration" (the extent to which an intervention or treatment is integrated within a service setting and its subsystems) [ 26 ] was rarely observed in the documents. This may be related to the greater complexity of obtaining information on these aspects, as they involve cross-cutting and multifactorial issues. In other words, it would be difficult to gather this information during evaluations with health practitioners as the target group. This highlights the need for evaluation processes of CPGs implementation involving multiple stakeholders, even if the evaluation is adjusted for each of these groups.

Although the models do not establish the "intervention" domain, we thought it pertinent in this study to delimit the issues that are intrinsic to CPGs, such as methodological quality or clarity in establishing recommendations. These issues were quite common in the models evaluated but were considered in other domains (e.g., in "context"). Studies have reported the importance of evaluating these issues intrinsic to CPGs [ 47 , 50 ] and their influence on the implementation process [ 51 ].

The models explicitly present the "strategies" domain, and its evaluation was usually included in the assessments. This is likely due to the expansion of scientific and practical studies in implementation science that involve theoretical approaches to the development and application of interventions to improve the implementation of evidence-based practices. However, these interventions themselves are not guaranteed to be effective, as reported in a previous review that showed unclear results indicating that the strategies had affected successful implementation [ 52 ]. Furthermore, model domains end up not covering all the complexity surrounding the strategies and their development and implementation process. For example, the ‘Guideline implementation evaluation tool’ evaluates whether guideline developers have designed and provided auxiliary tools to promote the implementation of guidelines [ 40 ], but this does not mean that these tools would work as expected.

The "process" domain was identified in the CFIR [ 31 , 38 ], JBI/GRiP [ 33 ], and PARiHS [ 29 ] frameworks. While it may be included in other domains of analysis, its distinct separation is crucial for defining operational issues when assessing the implementation process, such as determining if and how the use of the mentioned CPG was evaluated [ 3 ]. Despite its presence in multiple models, there is still limited detail in the evaluation guidelines, which makes it difficult to operationalize the concept. Further research is needed to better define the "process" domain and its connections and boundaries with other domains.

The domain of "sustainability" was only observed in the RE-AIM framework, which is categorized as an evaluation framework [ 34 ]. In its acronym, the letter M stands for "maintenance" and corresponds to the assessment of whether the user maintains use, typically longer than 6 months. The presence of this domain highlights the need for continuous evaluation of CPGs implementation in the short, medium, and long term. Although the RE-AIM framework includes this domain, it was not used in the questionnaire developed in the study. One probable reason is that the evaluation of CPGs implementation is still conducted on a one-off basis and not as a continuous improvement process. Considering that changes in clinical practices are inherent over time, evaluating and monitoring changes throughout the duration of the CPG could be an important strategy for ensuring its implementation. This is an emerging field that requires additional investment and research.

The "Fidelity/Adaptation" domain was not observed in the models. These emerging concepts involve the extent to which a CPG is being conducted exactly as planned or whether it is undergoing adjustments and adaptations. Whether or not there is fidelity or adaptation in the implementation of CPGs does not presuppose greater or lesser effectiveness; after all, some adaptations may be necessary to implement general CPGs in specific contexts. The absence of this domain in all the models and frameworks may suggest that they are not relevant aspects for evaluating implementation or that there is a lack of knowledge of these complex concepts. This may suggest difficulty in expressing concepts in specific evaluative questions. However, further studies are warranted to determine the comprehensiveness of these concepts.

It is important to note the customization of the domains of analysis, with some domains presented in the models not being evaluated in the studies, while others were complementarily included. This can be seen in Jeong et al. [ 34 ], where the "intervention" domain in the evaluation with the RE-AIM framework reinforced the aim of theoretical approaches such as guiding the process and not determining norms. Despite this, few limitations were reported for the models, suggesting that the use of models in these studies reflects the application of these models to defined contexts without a deep critical analysis of their domains.

Limitations

This review has several limitations. First, only a few studies and methodological guidelines that explicitly present models and frameworks for assessing the implementation of CPGs have been found. This means that few alternative models could be analyzed and presented in this review. Second, this review adopted multiple analytical categories (e.g., level of use, health service, target group, and domains evaluated), whose terminology has varied enormously in the studies and documents selected, especially for the "domains evaluated" category. This difficulty in harmonizing the taxonomy used in the area has already been reported [ 26 ] and has significant potential to confuse. For this reason, studies and initiatives are needed to align understandings between concepts and, as far as possible, standardize them. Third, in some studies/documents, the information extracted was not clear about the analytical category. This required an in-depth interpretative process of the studies, which was conducted in pairs to avoid inappropriate interpretations.

Implications

This study contributes to the literature and clinical practice management by describing models and frameworks specifically used to assess the implementation of CPGs based on their level of use, type of health service, target group related to the CPG, and the evaluated domains. While there are existing reviews on the theories, frameworks, and models used in implementation science, this review addresses aspects not previously covered in the literature. This valuable information can assist stakeholders (such as politicians, clinicians, researchers, etc.) in selecting or adapting the most appropriate model to assess CPG implementation based on their health context. Furthermore, this study is expected to guide future research on developing or adapting models to assess the implementation of CPGs in various contexts.

The use of models and frameworks to evaluate the implementation remains a challenge. Studies should clearly state the level of model use, the type of health service evaluated, and the target group. The domains evaluated in these models may need adaptation to specific contexts. Nevertheless, utilizing models to assess CPGs implementation is crucial as they can guide a more thorough and systematic evaluation process, aiding in the continuous improvement of CPGs implementation. The findings of this systematic review offer valuable insights for stakeholders in selecting or adjusting models and frameworks for CPGs evaluation, supporting future theoretical advancements and research.

Availability of data and materials

Abbreviations.

Australian Department of Health and Aged Care

Canadian Agency for Drugs and Technologies in Health

Centers for Disease Control and

Consolidated Framework for Implementation Research

Cumulative Index to Nursing and Allied Health Literature

Clinical practice guideline

Centre for Reviews and Dissemination

Guidelines International Networks

Getting Research into Practice

Health Systems Evidence

Institute of Medicine

The Joanna Briggs Institute

Ministry of Health of Brazil

Ministerio de Sanidad y Política Social

National Health and Medical Research Council

National Institute for Health and Care Excellence

Promoting action on research implementation in health systems framework

Predisposing, Reinforcing and Enabling Constructs in Educational Diagnosis and Evaluation-Policy, Regulatory, and Organizational Constructs in Educational and Environmental Development

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

International Prospective Register of Systematic Reviews

Reach, effectiveness, adoption, implementation, and maintenance framework

Healthcare Improvement Scotland

United States of America

Virtual Health Library

World Health Organization

Medicine I of. Crossing the Quality Chasm: A New Health System for the 21st Century. 2001. Available from: http://www.nap.edu/catalog/10027 . Cited 2022 Sep 29.

Field MJ, Lohr KN. Clinical Practice Guidelines: Directions for a New Program. Washington DC: National Academy Press. 1990. Available from: https://www.nap.edu/read/1626/chapter/8 Cited 2020 Sep 2.

Dawson A, Henriksen B, Cortvriend P. Guideline Implementation in Standardized Office Workflows and Exam Types. J Prim Care Community Heal. 2019;10. Available from: https://pubmed.ncbi.nlm.nih.gov/30900500/ . Cited 2020 Jul 15.

Unverzagt S, Oemler M, Braun K, Klement A. Strategies for guideline implementation in primary care focusing on patients with cardiovascular disease: a systematic review. Fam Pract. 2014;31(3):247–66. Available from: https://academic.oup.com/fampra/article/31/3/247/608680 . Cited 2020 Nov 5.

Article   PubMed   Google Scholar  

Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10(1):1–13. Available from: https://implementationscience.biomedcentral.com/articles/10.1186/s13012-015-0242-0 . Cited 2022 May 1.

Article   Google Scholar  

Mangana F, Massaquoi LD, Moudachirou R, Harrison R, Kaluangila T, Mucinya G, et al. Impact of the implementation of new guidelines on the management of patients with HIV infection at an advanced HIV clinic in Kinshasa, Democratic Republic of Congo (DRC). BMC Infect Dis. 2020;20(1):N.PAG-N.PAG. Available from: https://search.ebscohost.com/login.aspx?direct=true&db=c8h&AN=146325052&amp .

Browman GP, Levine MN, Mohide EA, Hayward RSA, Pritchard KI, Gafni A, et al. The practice guidelines development cycle: a conceptual tool for practice guidelines development and implementation. 2016;13(2):502–12. https://doi.org/10.1200/JCO.1995.13.2.502 .

Killeen SL, Donnellan N, O’Reilly SL, Hanson MA, Rosser ML, Medina VP, et al. Using FIGO Nutrition Checklist counselling in pregnancy: A review to support healthcare professionals. Int J Gynecol Obstet. 2023;160(S1):10–21. Available from: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85146194829&doi=10.1002%2Fijgo.14539&partnerID=40&md5=d0f14e1f6d77d53e719986e6f434498f .

Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM. An introduction to implementation science for the non-specialist. BMC Psychol. 2015;3(1):1–12. Available from: https://bmcpsychology.biomedcentral.com/articles/10.1186/s40359-015-0089-9 . Cited 2020 Nov 5.

Cassetti V, M VLR, Pola-Garcia M, AM G, J JPC, L APDT, et al. An integrative review of the implementation of public health guidelines. Prev Med reports. 2022;29:101867. Available from: http://www.epistemonikos.org/documents/7ad499d8f0eecb964fc1e2c86b11450cbe792a39 .

Eccles MP, Mittman BS. Welcome to implementation science. Implementation Science BioMed Central. 2006. Available from: https://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-1-1 .

Damschroder LJ. Clarity out of chaos: Use of theory in implementation research. Psychiatry Res. 2020;1(283):112461.

Handley MA, Gorukanti A, Cattamanchi A. Strategies for implementing implementation science: a methodological overview. Emerg Med J. 2016;33(9):660–4. Available from: https://pubmed.ncbi.nlm.nih.gov/26893401/ . Cited 2022 Mar 7.

Wang Y, Wong ELY, Nilsen P, Chung VC ho, Tian Y, Yeoh EK. A scoping review of implementation science theories, models, and frameworks — an appraisal of purpose, characteristics, usability, applicability, and testability. Implement Sci. 2023;18(1):1–15. Available from: https://implementationscience.biomedcentral.com/articles/10.1186/s13012-023-01296-x . Cited 2024 Jan 22.

Moullin JC, Dickson KS, Stadnick NA, Albers B, Nilsen P, Broder-Fingert S, et al. Ten recommendations for using implementation frameworks in research and practice. Implement Sci Commun. 2020;1(1):1–12. Available from: https://implementationsciencecomms.biomedcentral.com/articles/10.1186/s43058-020-00023-7 . Cited 2022 May 20.

Glasgow RE, Vogt TM, Boles SM. *Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health. 1999;89(9):1322. Available from: /pmc/articles/PMC1508772/?report=abstract. Cited 2022 May 22.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Asada Y, Lin S, Siegel L, Kong A. Facilitators and Barriers to Implementation and Sustainability of Nutrition and Physical Activity Interventions in Early Childcare Settings: a Systematic Review. Prev Sci. 2023;24(1):64–83. Available from: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85139519721&doi=10.1007%2Fs11121-022-01436-7&partnerID=40&md5=b3c395fdd2b8235182eee518542ebf2b .

Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions. version 6. Cochrane; 2022. Available from: https://training.cochrane.org/handbook. Cited 2022 May 23.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372. Available from: https://www.bmj.com/content/372/bmj.n71 . Cited 2021 Nov 18.

M C, AD O, E P, JP H, S G. Appendix A: Guide to the contents of a Cochrane Methodology protocol and review. Higgins JP, Green S, eds Cochrane Handb Syst Rev Interv. 2011;Version 5.

Kislov R, Pope C, Martin GP, Wilson PM. Harnessing the power of theorising in implementation science. Implement Sci. 2019;14(1):1–8. Available from: https://implementationscience.biomedcentral.com/articles/10.1186/s13012-019-0957-4 . Cited 2024 Jan 22.

Ouzzani M, Hammady H, Fedorowicz Z, Elmagarmid A. Rayyan-a web and mobile app for systematic reviews. Syst Rev. 2016;5(1):1–10. Available from: https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-016-0384-4 . Cited 2022 May 20.

JBI. JBI’s Tools Assess Trust, Relevance & Results of Published Papers: Enhancing Evidence Synthesis. Available from: https://jbi.global/critical-appraisal-tools . Cited 2023 Jun 13.

Drisko JW. Qualitative research synthesis: An appreciative and critical introduction. Qual Soc Work. 2020;19(4):736–53.

Pope C, Mays N, Popay J. Synthesising qualitative and quantitative health evidence: A guide to methods. 2007. Available from: https://books.google.com.br/books?hl=pt-PT&lr=&id=L3fbE6oio8kC&oi=fnd&pg=PR6&dq=synthesizing+qualitative+and+quantitative+health+evidence&ots=sfELNUoZGq&sig=bQt5wt7sPKkf7hwKUvxq2Ek-p2Q#v=onepage&q=synthesizing=qualitative=and=quantitative=health=evidence& . Cited 2022 May 22.

Nilsen P, Birken SA, Edward Elgar Publishing. Handbook on implementation science. 542. Available from: https://www.e-elgar.com/shop/gbp/handbook-on-implementation-science-9781788975988.html . Cited 2023 Apr 15.

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implement Sci. 2009;4(1):1–15. Available from: https://implementationscience.biomedcentral.com/articles/10.1186/1748-5908-4-50 . Cited 2023 Jun 13.

Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76. Available from: https://pubmed.ncbi.nlm.nih.gov/20957426/ . Cited 2023 Jun 11.

Bahtsevani C, Willman A, Khalaf A, Östman M, Ostman M. Developing an instrument for evaluating implementation of clinical practice guidelines: a test-retest study. J Eval Clin Pract. 2008;14(5):839–46. Available from: https://search.ebscohost.com/login.aspx?direct=true&db=c8h&AN=105569473&amp . Cited 2023 Jan 18.

Balbale SN, Hill JN, Guihan M, Hogan TP, Cameron KA, Goldstein B, et al. Evaluating implementation of methicillin-resistant Staphylococcus aureus (MRSA) prevention guidelines in spinal cord injury centers using the PARIHS framework: a mixed methods study. Implement Sci. 2015;10(1):130. Available from: https://pubmed.ncbi.nlm.nih.gov/26353798/ . Cited 2023 Apr 3.

Article   PubMed   PubMed Central   Google Scholar  

Breimaier HE, Heckemann B, Halfens RJGG, Lohrmann C. The Consolidated Framework for Implementation Research (CFIR): a useful theoretical framework for guiding and evaluating a guideline implementation process in a hospital-based nursing practice. BMC Nurs. 2015;14(1):43. Available from: https://search.ebscohost.com/login.aspx?direct=true&db=c8h&AN=109221169&amp . Cited 2023 Apr 3.

Chou AF, Vaughn TE, McCoy KD, Doebbeling BN. Implementation of evidence-based practices: Applying a goal commitment framework. Health Care Manage Rev. 2011;36(1):4–17. Available from: https://pubmed.ncbi.nlm.nih.gov/21157225/ . Cited 2023 Apr 30.

Porritt K, McArthur A, Lockwood C, Munn Z. JBI Manual for Evidence Implementation. JBI Handbook for Evidence Implementation. JBI; 2020. Available from: https://jbi-global-wiki.refined.site/space/JHEI . Cited 2023 Apr 3.

Jeong HJJ, Jo HSS, Oh MKK, Oh HWW. Applying the RE-AIM Framework to Evaluate the Dissemination and Implementation of Clinical Practice Guidelines for Sexually Transmitted Infections. J Korean Med Sci. 2015;30(7):847–52. Available from: https://pubmed.ncbi.nlm.nih.gov/26130944/ . Cited 2023 Apr 3.

GPC G de trabajo sobre implementación de. Implementación de Guías de Práctica Clínica en el Sistema Nacional de Salud. Manual Metodológico. 2009. Available from: https://portal.guiasalud.es/wp-content/uploads/2019/01/manual_implementacion.pdf . Cited 2023 Apr 3.

Australia C of. A guide to the development, implementation and evaluation of clinical practice guidelines. National Health and Medical Research Council; 1998. Available from: https://www.health.qld.gov.au/__data/assets/pdf_file/0029/143696/nhmrc_clinprgde.pdf .

Health Q. Guideline implementation checklist Translating evidence into best clinical practice. 2022.

Google Scholar  

Quittner AL, Abbott J, Hussain S, Ong T, Uluer A, Hempstead S, et al. Integration of mental health screening and treatment into cystic fibrosis clinics: Evaluation of initial implementation in 84 programs across the United States. Pediatr Pulmonol. 2020;55(11):2995–3004. Available from: https://www.embase.com/search/results?subaction=viewrecord&id=L2005630887&from=export . Cited 2023 Apr 3.

Urquhart R, Woodside H, Kendell C, Porter GA. Examining the implementation of clinical practice guidelines for the management of adult cancers: A mixed methods study. J Eval Clin Pract. 2019;25(4):656–63. Available from: https://search.ebscohost.com/login.aspx?direct=true&db=c8h&AN=137375535&amp . Cited 2023 Apr 3.

Yinghui J, Zhihui Z, Canran H, Flute Y, Yunyun W, Siyu Y, et al. Development and validation for evaluation of an evaluation tool for guideline implementation. Chinese J Evidence-Based Med. 2022;22(1):111–9. Available from: https://www.embase.com/search/results?subaction=viewrecord&id=L2016924877&from=export .

Breimaier HE, Halfens RJG, Lohrmann C. Effectiveness of multifaceted and tailored strategies to implement a fall-prevention guideline into acute care nursing practice: a before-and-after, mixed-method study using a participatory action research approach. BMC Nurs. 2015;14(1):18. Available from: https://search.ebscohost.com/login.aspx?direct=true&db=c8h&AN=103220991&amp .

Lai J, Maher L, Li C, Zhou C, Alelayan H, Fu J, et al. Translation and cross-cultural adaptation of the National Health Service Sustainability Model to the Chinese healthcare context. BMC Nurs. 2023;22(1). Available from: https://www.scopus.com/inward/record.uri?eid=2-s2.0-85153237164&doi=10.1186%2Fs12912-023-01293-x&partnerID=40&md5=0857c3163d25ce85e01363fc3a668654 .

Zhao J, Li X, Yan L, Yu Y, Hu J, Li SA, et al. The use of theories, frameworks, or models in knowledge translation studies in healthcare settings in China: a scoping review protocol. Syst Rev. 2021;10(1):13. Available from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7792291 .

Tabak RG, Khoong EC, Chambers DA, Brownson RC. Bridging research and practice: models for dissemination and implementation research. Am J Prev Med. 2012;43(3):337–50. Available from: https://pubmed.ncbi.nlm.nih.gov/22898128/ . Cited 2023 Apr 4.

Phulkerd S, Lawrence M, Vandevijvere S, Sacks G, Worsley A, Tangcharoensathien V. A review of methods and tools to assess the implementation of government policies to create healthy food environments for preventing obesity and diet-related non-communicable diseases. Implement Sci. 2016;11(1):1–13. Available from: https://implementationscience.biomedcentral.com/articles/10.1186/s13012-016-0379-5 . Cited 2022 May 1.

Buss PM, Pellegrini FA. A Saúde e seus Determinantes Sociais. PHYSIS Rev Saúde Coletiva. 2007;17(1):77–93.

Pereira VC, Silva SN, Carvalho VKSS, Zanghelini F, Barreto JOMM. Strategies for the implementation of clinical practice guidelines in public health: an overview of systematic reviews. Heal Res Policy Syst. 2022;20(1):13. Available from: https://health-policy-systems.biomedcentral.com/articles/10.1186/s12961-022-00815-4 . Cited 2022 Feb 21.

Grimshaw J, Eccles M, Tetroe J. Implementing clinical guidelines: current evidence and future implications. J Contin Educ Health Prof. 2004;24 Suppl 1:S31-7. Available from: https://pubmed.ncbi.nlm.nih.gov/15712775/ . Cited 2021 Nov 9.

Lotfi T, Stevens A, Akl EA, Falavigna M, Kredo T, Mathew JL, et al. Getting trustworthy guidelines into the hands of decision-makers and supporting their consideration of contextual factors for implementation globally: recommendation mapping of COVID-19 guidelines. J Clin Epidemiol. 2021;135:182–6. Available from: https://pubmed.ncbi.nlm.nih.gov/33836255/ . Cited 2024 Jan 25.

Lenzer J. Why we can’t trust clinical guidelines. BMJ. 2013;346(7913). Available from: https://pubmed.ncbi.nlm.nih.gov/23771225/ . Cited 2024 Jan 25.

Molino C de GRC, Ribeiro E, Romano-Lieber NS, Stein AT, de Melo DO. Methodological quality and transparency of clinical practice guidelines for the pharmacological treatment of non-communicable diseases using the AGREE II instrument: A systematic review protocol. Syst Rev. 2017;6(1):1–6. Available from: https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-017-0621-5 . Cited 2024 Jan 25.

Albers B, Mildon R, Lyon AR, Shlonsky A. Implementation frameworks in child, youth and family services – Results from a scoping review. Child Youth Serv Rev. 2017;1(81):101–16.

Download references

Acknowledgements

Not applicable

This study is supported by the Fundação de Apoio à Pesquisa do Distrito Federal (FAPDF). FAPDF Award Term (TOA) nº 44/2024—FAPDF/SUCTI/COOBE (SEI/GDF – Process 00193–00000404/2024–22). The content in this article is solely the responsibility of the authors and does not necessarily represent the official views of the FAPDF.

Author information

Authors and affiliations.

Department of Management and Incorporation of Health Technologies, Ministry of Health of Brazil, Brasília, Federal District, 70058-900, Brazil

Nicole Freitas de Mello & Dalila Fernandes Gomes

Postgraduate Program in Public Health, FS, University of Brasília (UnB), Brasília, Federal District, 70910-900, Brazil

Nicole Freitas de Mello, Dalila Fernandes Gomes & Jorge Otávio Maia Barreto

René Rachou Institute, Oswaldo Cruz Foundation, Belo Horizonte, Minas Gerais, 30190-002, Brazil

Sarah Nascimento Silva

Oswaldo Cruz Foundation - Brasília, Brasília, Federal District, 70904-130, Brazil

Juliana da Motta Girardi & Jorge Otávio Maia Barreto

You can also search for this author in PubMed   Google Scholar

Contributions

NFM and JOMB conceived the idea and the protocol for this study. NFM conducted the literature search. NFM, SNS, JMG and JOMB conducted the data collection with advice and consensus gathering from JOMB. The NFM and JMG assessed the quality of the studies. NFM and DFG conducted the data extraction. NFM performed the analysis and synthesis of the results with advice and consensus gathering from JOMB. NFM drafted the manuscript. JOMB critically revised the first version of the manuscript. All the authors revised and approved the submitted version.

Corresponding author

Correspondence to Nicole Freitas de Mello .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

13012_2024_1389_moesm1_esm.docx.

Additional file 1: PRISMA checklist. Description of data: Completed PRISMA checklist used for reporting the results of this systematic review.

Additional file 2: Literature search. Description of data: The search strategies adapted for the electronic databases.

13012_2024_1389_moesm3_esm.doc.

Additional file 3: JBI’s critical appraisal tools for cross-sectional studies. Description of data: JBI’s critical appraisal tools to assess the trustworthiness, relevance, and results of the included studies. This is specific for cross-sectional studies.

13012_2024_1389_MOESM4_ESM.doc

Additional file 4: JBI’s critical appraisal tools for qualitative studies. Description of data: JBI’s critical appraisal tools to assess the trustworthiness, relevance, and results of the included studies. This is specific for qualitative studies.

13012_2024_1389_MOESM5_ESM.doc

Additional file 5: Methodological quality assessment results for cross-sectional studies. Description of data: Methodological quality assessment results for cross-sectional studies using JBI’s critical appraisal tools.

13012_2024_1389_MOESM6_ESM.doc

Additional file 6: Methodological quality assessment results for the qualitative studies. Description of data: Methodological quality assessment results for qualitative studies using JBI’s critical appraisal tools.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Freitas de Mello, N., Nascimento Silva, S., Gomes, D.F. et al. Models and frameworks for assessing the implementation of clinical practice guidelines: a systematic review. Implementation Sci 19 , 59 (2024). https://doi.org/10.1186/s13012-024-01389-1

Download citation

Received : 06 February 2024

Accepted : 01 August 2024

Published : 07 August 2024

DOI : https://doi.org/10.1186/s13012-024-01389-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Implementation
• Practice guideline
• Evidence-Based Practice
• Implementation science

Implementation Science

ISSN: 1748-5908

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Systematic literature review of gender equity and social inclusion in primary education for teachers in Tanzania: assessing status and future directions

• Open access
• Published: 13 August 2024
• Volume 3 , article number  122 , ( 2024 )

Cite this article

You have full access to this open access article

• Henry Nkya 1 &
• Isack Kibona 2  

Gender equity and social inclusion (GESI) are crucial for creating inclusive and equitable educational environments in primary schools. This systematic literature review aimed to interpret and synthesize the findings of previous studies on GESI interventions and programs in primary schools in Tanzania, identified gaps in the knowledge, and provided recommendations for policy and practice. A systematic literature review search identified 22 relevant studies that met the inclusion criteria. The studies conducted between 2010 and 2021, and the sample sizes of participants were above 50. More than 50% of the studies were conducted in rural areas and used a quasi-experimental design. The interventions evaluation included teacher training, community engagement, and curriculum reform. The systematic literature review employed statistical methods to measure effect sizes and employed traditional univariate systematic literature review to synthesize the results. A table summarizing the literature that met the inclusion criteria was created to ensure transparency and clarity in the data coding process. The systematic literature review found a positive effect of GESI interventions on various outcomes, including improved academic performance, reduced gender-based violence, and increased social inclusion. However, variations in effect sizes and study designs across the studies were noted. Several gaps were identified, such as the lack of long-term follow-up and the need for more rigorous study designs. The implications of the findings for policy and practice in promoting GESI in primary schools in Tanzania were discussed, and recommendations for future research were provided. This systematic literature review highlighted the importance of addressing GESI in primary school education in Tanzania and underscored the critical role of teachers in promoting these values. It calls for targeted interventions, policy enhancements, and further research to bridge the gaps identified in the literature.

Explore related subjects

• Artificial Intelligence

Avoid common mistakes on your manuscript.

1 Introduction

GESI are critical components of education that ensure equitable access to education for all individuals, regardless of their gender, socioeconomic status, ethnicity, or other backgrounds. In Tanzania, GESI has become a significant concern, particularly in primary schools, where gender and social inequalities often lead to disparities in educational outcomes.

Research has shown that girls are more likely to face barriers in education than boys, including poverty, early marriages, and cultural bias that prioritize boys’ education over girls [ 28 ]. Furthermore, children from marginalized groups, such as children with disabilities, children from ethnic minority groups, and children from low-income families, often experience unequal access to quality education [ 38 ].

Addressing GESI issues in primary schools is crucial for ensuring that all children have access to quality education, which is essential for their personal development and future success. GESI initiatives can promote equity and inclusion in schools and create an environment where all children feel valued and supported [ 13 ].

Addressing gender stereotypes in teacher education programs can play a vital role in promoting GESI in primary schools [ 32 ]. Similarly, Okkolin et al. [ 27 ] suggest that interventions that address GESI can improve educational outcomes for girls and marginalized groups.

Overall, promoting GESI in primary schools is essential for creating a more equitable and inclusive education system that benefits all children [ 2 ]. It requires a concerted effort from policymakers, educators, parents, and communities to work together to create a learning environment that is supportive, respectful, and inclusive for all children.

1.1 Theoretical framework

This study is guided by the Social Justice Theory, which emphasizes the need for equitable treatment, opportunities, and outcomes for all individuals, particularly those from marginalized and disadvantaged backgrounds [ 10 ]. This framework is crucial in understanding the components of GESI and their impact on educational outcomes. The Social Justice Theory aligns with the goals of GESI by promoting fairness and the elimination of disparities in education [ 1 ].

1.1.1 Components of GESI

The key components of GESI in this study include [ 24 ]:

Gender equity: ensuring that girls and boys have equal access to education and opportunities.

Social inclusion: creating an inclusive environment where all students, regardless of their backgrounds, can participate and succeed.

Teacher training: educating teachers on gender-sensitive and inclusive teaching practices.

Community engagement: involving communities in promoting GESI.

Curricula reform: developing and implementing curricula that address GESI issues.

1.2 Justification for focusing on Tanzania

Tanzania provides a unique context for examining GESI due to its diverse population and the significant challenges it faces in achieving GESI in education [ 18 ]. Despite efforts to promote GESI, disparities persist, making it an important area of study to identify effective interventions and inform policy and practice.

1.3 Rationale for conducting a systematic literature review

A Systematic literature review is an essential tool for synthesizing research findings from different studies and summarizing the overall effect size of an intervention or variable of interest [ 34 ]. Conducting a systematic literature review on GESI in primary school education is critical for providing an overview of the existing research and identifying gaps that need to be addressed in future research. It also helps establish the overall effect of interventions aimed at promoting GESI in primary schools in Tanzania [ 9 ]. The results of the Systematic literature review can inform policies and practices aimed at promoting GESI in primary school education, thereby improving learning outcomes for all children, regardless of their gender, social, and economic backgrounds.

By addressing the GESI issues and synthesizing the existing literature, this systematic literature review aims to contribute to a more equitable and inclusive educational environment in Tanzania [ 22 ].

1.4 Research objectives

To identify the state of GESI in primary schools. This objective aims to provide a comprehensive understanding of how GESI issues manifest in primary schools, considering various social and educational contexts.

To number factors that contribute to gender GESI in primary schools. This shall allow informed decisions on the effort to contain the issues of GESI.

To synthesize the findings of previous studies on GESI in primary schools. This objective focuses on aggregating and interpreting the results of existing research to offer a clear and cohesive picture of what is known about GESI interventions and their effectiveness.

To identify gaps in the knowledge of GESI in primary schools. By evaluating the existing literature, this objective seeks to highlight areas where further research is needed, identifying shortcomings in study designs, populations, or intervention strategies.

To provide recommendations for improving GESI in primary schools. Based on the synthesis of previous studies and identified gaps, this objective aims to propose actionable strategies and policies to enhance GESI in primary education.

2 Methodology

Having set the study objectives, the search-strategy for the study involved conducting a comprehensive literature review of studies on GESI in primary schools. The search was conducted using electronic databases such as Google Scholar, JSTOR, and EBSCOhost. The search terms used were “gender equity,” “social inclusion,” “primary schools,” “Tanzania,” and “teachers.” Additionally, hand searching was conducted by reviewing the reference lists of identified studies to identify any relevant studies that may have been missed during the initial search.

Inclusion criteria:

The study must be conducted in primary schools.

The study must focus on gender equity and/or social inclusion in education.

The study must involve teachers as the primary participants or focus on the teacher’s role in promoting GESI.

The study must be published in English between 2010 and 2022.

Exclusion criteria:

Studies conducted outside Tanzania.

Studies not related to gender equity and/or social inclusion in education.

Studies not involving teachers or not focusing on the teacher’s role in promoting GESI.

Studies published before 2010 or after 2022.

The search process was conducted by two independent reviewers to ensure the accuracy and completeness of the search results. The reviewers screened the titles and abstracts of the identified studies for relevance and then reviewed the full text of potentially relevant studies. Any discrepancies between the reviewers were resolved through discussion and consensus. Reviewers made necessary steps to ensure a justified systematic review. Overall, the Authors reviewed 22 papers considered to have met the set criteria.

2.1 Choice of the effect size measure and analytical methods

The effect size measure used in this study was generated by statistical tools, making it suitable for systematics review that synthesize findings across multiple studies. For similar research questions, the study employed traditional univariate meta-analysis. This method was chosen because it is suitable for synthesizing the results of multiple studies that investigate similar research questions. Traditional univariate meta-analysis allows for the calculation of an overall effect size, providing a comprehensive summary of the impact of GESI interventions across different studies.

2.2 Choice of software

We used R software, specifically the ‘metafor’ package, for our analysis. This software was selected due to its robustness and versatility in conducting analytical procedures. The ‘metafor’ package supports a wide range of meta-analytic models and methods, making it a comprehensive tool for this type of analysis.

2.3 Coding of effect sizes

Table 1 summarizes the literature included that meets the inclusion criteria. This table includes information such as study design, sample size, effect sizes, and any other relevant variables. This step ensures transparency and clarity in the data coding process.

3 Results and analysis

The layout of the manuscript has been organized accordingly, so that headings and subheadings clearly demarcates each step of the systematic literature review process.

3.1 Status of GESI in primary schools in Tanzania

3.1.1 persistent gender disparities.

One of the major findings in this study was that gender disparities in primary education persist in Tanzania. This was evident in the lower enrollment and completion rates for girls in primary schools compared to boys [ 36 ]. Girls are less likely to attend school than boys, with enrollment rates lower for girls at both the primary and secondary levels. Additionally, girls are more likely to drop out of school due to various reasons, including early marriage, household responsibilities, and financial constraints [ 5 ]. These disparities highlight the ongoing challenges faced by girls in accessing and completing primary education.

3.1.2 Cultural and societal beliefs

Several studies have identified cultural and societal beliefs as a major factor contributing to gender disparities in primary education. In many Tanzanian communities, girls are expected to prioritize domestic responsibilities over their education, which can lead to low enrollment rates and high drop-out rates [ 39 ]. Furthermore, gender-based violence and sexual harassment are prevalent in schools, with girls facing discrimination and harassment from both male students and teachers [ 4 ]. These issues underscore the need for targeted interventions to create a safer and more supportive educational environment for girls.

Furthermore, Losioki and Mdee [ 12 ] found that gender stereotypes perpetuated in teacher education programs in Tanzania, which can affect the ability of teachers to create a gender-equitable and socially inclusive classroom environment. Teachers may unconsciously reinforce gender stereotypes in the classroom, leading to further marginalization of girls and other vulnerable groups.

3.1.3 Underrepresented minorities

In addition, limited access to education for children with disabilities or those from low-income families and marginalized communities can perpetuate social inequalities in primary schools [ 30 ]. These students often face significant barriers, including inadequate school facilities, lack of appropriate learning materials, and insufficient support services, which hinder their educational progress.

3.2 Strategies addressing the challenge

Despite these challenges, there have been government efforts to improve GESI in primary schools. The government of Tanzania has committed to providing equal access to education for all children, regardless of gender, ethnicity, or socio-economic status. The government has implemented policies such as free primary education and affirmative action programs to promote equal access to education for all children, regardless of gender or social status [ 15 , 26 ]. These initiatives aim to reduce financial barriers to education and encourage the enrollment and retention of girls and children from marginalized groups. This includes initiatives such as the Tanzania Education Sector Development Plan (ESDP) and the Primary Education Development Program (PEDP) [ 6 , 16 ]. These programs aim to address systemic barriers in education and promote inclusive practices in schools. The government also is open to collaborate with external forces like international interventions, community development agencies and NGO to work toward enhancing GESI. Some Strategies Addressing GESI Challenges. For instance, projects that focus on community engagement and parental involvement have shown positive impacts in changing attitudes towards girls’ education and promoting inclusive practices [ 17 ].

3.2.1 International and community-based programs

In recent years, there have been an increase in programs and initiatives aimed at promoting GESI in primary education. For example, the “Let Girls Learn” program, launched by the US government in partnership with the Tanzanian government, aimed to increase access to education for girls and reduce gender disparities in education [ 7 ]. Similarly, the Tanzania Gender Networking Programme (TGNP) has been working to promote GESI in education through community mobilization, advocacy, and capacity building [ 14 ].

3.2.2 Interventions with recorded impact

Previous studies identified several approaches that have been successful in improving GESI in primary schools. Among others, at least two are discussed. One such approach is the use of gender-responsive pedagogy, which involves incorporating gender-sensitive teaching practices and materials into the classroom [ 17 ]. This method helps create a more inclusive learning environment that acknowledges and addresses the different needs of boys and girls. Another effective intervention is the provision of sanitary pads and menstrual hygiene education to girls, which has been shown to improve school attendance and reduce drop-out rates [ 35 ]. By addressing menstrual hygiene needs, schools can help ensure that girls do not miss out on education due to a lack of resources or stigma associated with menstruation.

3.2.3 Intervention recommendations

GESI are essential components of a quality education system, and there is a need to address the persistent gender disparities in primary education. While cultural and societal beliefs continue to be major barriers, efforts to improve GESI through government policies and initiatives, as well as community-based programs, showed promise. The use of gender-responsive pedagogy and the provision of menstrual hygiene education and supplies were promising approaches that showed positive results [ 21 ]. However, more research and investment are needed to ensure that all children have access to primary education. Continued collaboration between the government, NGOs, and communities is essential to sustain and expand these efforts, ensuring that all students can benefit from a supportive and equitable educational environment [ 29 ].

Overall, there is still much work to be done to ensure GESI in primary schools [ 33 ]. It will require continued efforts and collaboration from the government, educators, and communities to address cultural and traditional beliefs, promote teacher education that challenges gender stereotypes, and provide equal access to education for all children. Policymakers must prioritize the allocation of resources to support GESI initiatives and ensure that schools are equipped to meet the diverse needs of all students [ 3 ].

By addressing these systemic issues, Tanzania can make significant strides towards achieving an inclusive and equitable education system that benefits all children, irrespective of their gender or socioeconomic background. Continued research and monitoring are essential to evaluate the effectiveness of existing interventions and identify new strategies to overcome persistent challenges in promoting GESI in primary education [ 31 ].

3.2.4 Gaps in the knowledge about GESI in primary schools

While the literature have provided valuable insights into the state of GESI in primary schools in Tanzania, several gaps in the knowledge still need to be addressed.

One major gap is the lack of research on the experiences of marginalized groups, including children with disabilities and those from low-income households. Studies have shown that these groups face significant barriers to accessing education and are often excluded from educational opportunities. For example, a study by Mwaijande [ 20 ] found that children with disabilities faced challenges such as lack of access to assistive devices and negative attitudes from teachers and other students. Similarly, research by Pak et al. [ 30 ] and Thomas and Rugambwa [ 36 ] revealed that children from poor families often struggle to pay school fees and may not have access to basic learning materials.

Another gap in the Tanzanian knowledge is the lack of research on the experiences of female teachers in primary schools. While studies have examined gender stereotypes and biases among teacher education programs, Thomas and Rugambwa [ 36 ] stressed that there is limited research on the experiences of female teachers in the classroom. Research on female teachers could shed light on the ways in which gender intersects with other forms of marginalization, such as age and socioeconomic status.

Furthermore, there is a need for more research on effective interventions and strategies for promoting GESI in primary schools. While some studies have evaluated the impact of interventions such as teacher training programs [ 19 , 25 ] , more rigorous evaluations of these interventions are needed to determine their effectiveness and sustainability.

Additionally, there is a lack of longitudinal studies that follow the long-term impact of GESI interventions. Many studies focus on short-term outcomes, but understanding the lasting effects of interventions is crucial for developing sustainable policies and practices.

In summary, while previous research has provided valuable insights into GESI in primary schools, several gaps in the knowledge need to be addressed. Future research should focus on the experiences of marginalized groups, including children with disabilities and those from low-income households, as well as female teachers. Additionally, the study showed more need for more rigorous evaluations of interventions and strategies aimed at promoting GESI in primary schools. Longitudinal studies that assess the long-term impact of these interventions would also be beneficial.

3.3 Patterns observed across the studies

As observed in the study, there were some patterns and trends identified across the studies. Firstly, there was a consistent finding that gender disparities persist in primary schools, particularly in terms of access to education and academic achievement. Despite efforts to promote GESI, girls and marginalized groups continue to face significant barriers that hinder their educational progress.

Secondly, there was a growing recognition of the importance of addressing GESI in primary education, as evidenced by the increasing number of interventions and programs aimed at promoting these values. This trend indicates a positive shift towards acknowledging and addressing GESI issues within the education system.

Thirdly, the systematic literature review revealed that the role of teachers is critical in promoting GESI in primary schools. Teacher training and support are essential for equipping educators with the skills and knowledge needed to foster an inclusive and equitable learning environment. Studies consistently highlighted the need for gender-sensitive pedagogy and teacher professional development programs.

Finally, there were some gaps in the current knowledge base, particularly with regard to the long-term impact of interventions and the effectiveness of different approaches to promoting GESI in primary education. While some interventions showed promising results, more research was needed to determine their sustainability and broader applicability.

By addressing these gaps and building on the patterns observed across studies, future research could contribute to a more comprehensive understanding of GESI in primary schools and inform the development of policies and practices to promote equity and inclusion for all students.

To sum up, analysis revealed that GESI interventions have a positive effect on various outcomes such as academic performance, reduced gender-based violence, and increased social inclusion. However, variations in effect sizes and study designs were observed across the studies. The studies included in the systematic literature review used various designs, such as randomized controlled trials (RCTs) and quasi-experimental designs, which contributed to the diversity in effect sizes.

4 Discussion

GESI is a critical components of a better-quality education system over otherwise. In Tanzania, primary education is the foundation for future academic and professional success [ 23 ], making it essential to ensure that all students, regardless of gender or social status, have access to an inclusive and equitable education. Previous studies explored the state of GESI in primary schools and identified areas for improvement.

The findings of the study highlighted the state of GESI in primary schools. The analysis of some 10 included studies revealed that significant disparities in access to education and academic performance among genders persist, with girls being more disadvantaged. Additionally, children from marginalized backgrounds, such as those from low-income families or those with disabilities, face substantial barriers to education.

To sum up, the study suggests a holistic approach involving teachers, schools, communities, and policymakers. Thus, multifaceted approach is necessary to create a more inclusive and equitable education system. Therefore, Recommendations include:

Providing comprehensive teacher training on gender-sensitive teaching methods.

Implementing community-based initiatives to address social and cultural barriers.

Developing policies and programs prioritizing marginalized students’ needs.

4.1 Implications of the study

Overall, the systematic literature review provided important insights into the state of GESI in primary schools. While progress has been made, significant challenges remain. Continued efforts and investments are necessary to promote a more equitable and inclusive education system. Future research should address the identified gaps and build on the promising interventions highlighted in this study. Based on the evidence synthesized, it is clear that targeted interventions are necessary to address the barriers that girls and other marginalized groups face in accessing and completing primary education. The study has the following recommendations on policy and practice and the areas for future research.

4.1.1 Addressing school issues related to GESI

Teacher training: policies should mandate comprehensive training for teachers on gender-sensitive teaching practices. Educators need to be equipped with the skills and knowledge to foster an inclusive classroom environment that supports both boys and girls. This includes understanding how to address and counteract gender stereotypes and biases.

Providing resources: schools should be equipped with resources to support girls’ education. This includes the provision of sanitary pads, access to clean and safe gender-segregated toilets, and gender-sensitive teaching materials. These resources are essential in reducing barriers to attendance and participation for girls.

Reviewing curricula: the school curriculum should be reviewed and revised to promote GESI. Curricula should reflect the diversity of Tanzanian society and challenge existing gender stereotypes. Including content that promotes GESI will help inculcate these values in students from a young age.

4.1.2 Addressing structural and socio-economic barriers

Financial support: there should be policies to provide financial support to families who cannot afford school fees. This can include scholarships, free school meals, and other financial incentives that alleviate the economic burden on families and keep girls in school.

Cultural norms and attitudes: interventions must focus on changing cultural norms and attitudes that limit girls’ access to education. Community engagement and awareness campaigns are crucial in shifting perceptions and promoting the value of girls’ education. Programs should aim to involve parents and community leaders in promoting gender equity.

Reducing gender-based violence: schools should implement strict policies against gender-based violence and harassment. Providing a safe and supportive environment is crucial for retaining girls in school. Support services for victims of violence and harassment should be readily available.

4.1.3 Promoting girls’ participation and leadership

Extracurricular activities: schools should create opportunities for girls to engage in extracurricular activities. Programs such as sports, arts, and clubs can enhance girls’ skills and confidence, providing a platform for them to express themselves and develop leadership qualities.

Leadership training: providing leadership training for girls to support their involvement in decision-making processes within schools and communities is essential. This training can empower girls to take active roles in their schools and communities, fostering a sense of agency and leadership.

4.1.4 Comprehensive and integrated approach

Involving multiple stakeholders: a comprehensive approach to promoting GESI should involve multiple stakeholders, including the government, civil society, and communities. Collaboration among these groups is essential for creating a supportive environment for GESI.

Evidence-based interventions: policies and practices should be guided by evidence-based interventions tailored to the specific needs and contexts of different regions and populations. Utilizing data and research to inform practices ensures that efforts are effective and impactful.

Monitoring and evaluation: continuous monitoring and evaluation of interventions are necessary to assess their effectiveness and make necessary adjustments. This helps in ensuring the sustainability and scalability of successful initiatives.

The study highlights the importance of a comprehensive and integrated approach to promoting GESI in primary schools. It underscores the need for targeted interventions, policy enhancements, and continued efforts to address the persistent barriers that girls and marginalized groups face. By implementing these recommendations, Tanzania can make significant strides towards achieving a more inclusive and equitable education system for all children.

4.2 Areas for future research

Future research and policy efforts should focus on sustaining and scaling successful interventions, ensuring that all children, regardless of gender or socio-economic background, have access to quality education. Future research should address these gaps:

Experiences of marginalized groups: more high-quality research is needed on the experiences of marginalized groups, including children with disabilities and those from low-income households.

Female teachers: investigate the experiences of female teachers in primary schools to understand how gender intersects with other forms of marginalization, such as age and socioeconomic status.

Effectiveness of interventions: conduct more rigorous evaluations of specific interventions and strategies for promoting GESI, including long-term impact studies.

Intersectionality: explore the intersectionality of factors such as gender, socioeconomic status, and ethnicity to provide a more comprehensive

5 Conclusion

GESI is crucial for improving access to education, ensuring equal opportunities, and promoting positive social outcomes. Teachers play a critical role in promoting these values and must receive appropriate training and support to create inclusive learning environments. Policymakers and education leaders must prioritize efforts to address GESI in primary schools, including investing in research to understand the factors contributing to gender and social equality and identifying effective strategies for promoting GESI.

The systematic literature review examined the state of GESI in primary schools and revealed significant challenges, particularly in terms of teacher training and the implementation of policies and programs. The review highlighted persistent gender disparities and the barriers faced by marginalized groups, such as children with disabilities and those from low-income families.

The findings suggest that targeted interventions are needed to address these barriers, recommended interventions include:

Increasing access to education: efforts to increase access to education for marginalized groups, such as scholarships and school feeding programs.

Policy development: implementing policies that address gender-based violence and discrimination.

Community engagement: involving multiple stakeholders, including government, civil society, and communities, in promoting GESI.

Develop and implement teacher training programs: focus on GESI principles, awareness of gender biases, strategies for promoting inclusivity, and the use of gender-sensitive teaching materials.

Develop and implement gender-sensitive curricula: address gender biases and stereotypes across all subject areas.

Strengthen policies and regulations: enforce policies that promote GESI in school governance, teacher recruitment, and student enrollment.

Increase participation of girls: provide incentives for girls to attend school, such as scholarships and school feeding programs, and improve school infrastructure.

The study provides crucial insights into the state of GESI in primary schools and underscores the need for coordinated and sustained efforts to address these challenges. By implementing the recommended strategies and involving all stakeholders, Tanzania can ensure that all children have access to quality primary education that promotes GESI.

Data availability

No datasets were generated or analysed during the current study.

Adipat S, Chotikapanich R. Sustainable development goal 4: an education goal to achieve equitable quality education. Acad J Interdiscip Stud. 2022;11(6):174–83.

Article   Google Scholar  

Cavicchioni V, Motivans A. Monitoring educational disparities in less developed countries. In: In pursuit of equity in education: using international indicators to compare equity policies. New York: Springer; 2001. p. 217–40.

Google Scholar  

Clancy J, Barnett A, Cecelski E, Pachauri S, Dutta S, Oparaocha S, Kooijman A. Gender in the transition to sustainable energy for all: from evidence to inclusive policies. 2019.

Colclough C, Rose P, Tembon M. Gender inequalities in primary schooling: the roles of poverty and adverse cultural practice. Int J Educ Dev. 2000;20(1):5–27.

Esteves M. Gender equality in education: a challenge for policy makers. Int J Soc Sci. 2018;4(2):893–905.

Fenech M, Skattebol J. Supporting the inclusion of low-income families in early childhood education: an exploration of approaches through a social justice lens. Int J Incl Educ. 2021;25(9):1042–60.

Frank A. Understanding the “success” of an all girls’ boarding school in rural Tanzania: perspectives of graduates, teachers, and administrators, PhD thesis. The Florida State University; 2019.

Group WB. Malawi systematic country diagnostic: breaking the cycle of low growth and slow poverty reduction. Washington, DC: World Bank; 2018.

Book   Google Scholar  

Guthridge M, Kirkman M, Penovic T, Giummarra MJ. Promoting gender equality: a systematic review of interventions. Soc Justice Res. 2022;35(3):318–43.

Kaur B. Equity and social justice in teaching and teacher education. Teach Teach Educ. 2012;28(4):485–92.

Lokina RB, Nyoni J, Kahyarara G. Social policy, gender and labour in Tanzania. Dar es Salaam: Economic and Social Research Foundation (ESRF); 2016.

Losioki BE, Mdee HK. The contribution of the hidden curriculum to gender inequality in teaching and learning materials: experiences from Tanzania. Asian J Educ Train. 2023;9(2):54–8.

Lovell E. Gender equality, social inclusion and resilience in Malawi. In: Building resilience and adapting to climate change. 2021.

Makulilo AB, Bakari M. Building a transformative feminist movement for women empowerment in Tanzania: the role of the Tanzania gender networking programme (TGNP-Mtandao). Afr Rev. 2021;13(2):155–74.

Malelu AM. Institutional factors influencing career advancement of women faculty: a case of, PhD thesis. Kenyatta University; 2015.

Mashala YL. The impact of the implementation of free education policy on secondary education in Tanzania. Int J Acad Multidiscip Res. 2019;3(1):6–14.

Mhewa, M. M., Bhalalusesa, E. P., & Kafanabo, E. (2021). Secondary school teachers’ understanding of gender-responsive pedagogy in bridging inequalities of students’ learning in tanzania. Papers in Education and Development, 38(2).

Mohun R, Biswas S, Jacobson J, Sajjad F. Infrastructure: a game changer for women’s economic empowerment. Background paper. UN Secretary-General’s High-Level Panel on Women’s Economic Empowerment; 2016.

Mondal S, Joe W, Akhauri S, Sinha I, Thakur P, Kumar V, Kumar T, Pradhan N, Kumar A. Delivering PACE++ curriculum in community settings: impact of TARA intervention on gender attitudes and dietary practices among adolescent girls in Bihar, India. PLoS ONE. 2023;18(11): e0293941.

Mwaijande VT. Access to education and assistive devices for children with physical disabilities in Tanzania, Master’s thesis. Oslo and Akershus University College; 2014.

Mwakabenga RJ, Komba SC. Gender inequalities in pedagogical classroom practice: what influence do teachers make? J Educ Humanit Sci. 2021;10(3):66–82.

Nazneen S, Cole N. Literature review on socially inclusive budgeting. 2018.

Ndijuye LG, Mligo IR, Machumu MAM. Early childhood education in Tanzania: views and beliefs of stakeholders on its status and development. Global Educ Rev. 2020;7(3):22–39.

Nelly S. Gender equality and social inclusion (GESI) in village development. Legal Brief. 2021;10(2):245–52.

Nkya HE, Bimbiga I. Unlocking potential: the positive impact of in-service training on science and mathematics teachers teaching strategies. Res Humanit Soc Sci. 2023. https://doi.org/10.7176/RHSS/13-16-04 .

Nyoni WP, He C, Yusuph ML. Sustainable interventions in enhancing gender parity in senior leadership positions in higher education in Tanzania. J Educ Pract. 2017;8(13):44–54.

Okkolin M-A, Lehtomäki E, Bhalalusesa E. The successful education sector development in Tanzania—comment on gender balance and inclusive education. Gend Educ. 2010;22(1):63–71.

Omari CK, Mbilinyi DA. Born to be less equal: the predicament of the girl child in Tanzania. In: Gender, family and work in Tanzania. London: Routledge; 2018. p. 292–314.

Chapter   Google Scholar  

Opini B, Onditi H. Education for all and students with disabilities in Tanzanian primary schools: challenges and successes. Int J Educ Stud. 2016;3(2):65–76.

Pak K, Desimone LM, Parsons A. An integrative approach to professional development to support college-and career-readiness standards. Educ Policy Anal Arch. 2020;28(111): n111.

Palmary I. Back2School gender mainstreaming guidelines. 2024.

Prasetyo P, Azwardi A, Kistanti N. Gender equality and social inclusion (GESI) and institutions as key drivers of green entrepreneurship. Int J Data Netw Sci. 2023;7(1):391–8.

Shelley J. Identifying and overcoming barriers to gender equality in Tanzanian schools: educators’ reflections. Int J Pedagog Innov New Technol. 2019;6(1):9–27.

Siddaway AP, Wood AM, Hedges LV. How to do a systematic review: a best practice guide for conducting and reporting narrative reviews, meta-analyses, and meta-syntheses. Annu Rev Psychol. 2019;70(1):747–70.

Stoilova D, Cai R, Aguilar-Gomez S, Batzer NH, Nyanza EC, Benshaul-Tolonen A. Biological, material and socio-cultural constraints to effective menstrual hygiene management among secondary school students in Tanzania. PLOS Global Public Health. 2022;2(3): e0000110.

Thomas MA, Rugambwa A. Equity, power, and capabilities: constructions of gender in a Tanzanian secondary school. Fem Form. 2011;23(3):153–75.

Tieng’o EWB. Community perception on public primary schools: implications for sustainable fee free basic education in Rorya district, Tanzania. East Afr J Educ Soc Sci. 2019;1(1):32–47.

Wapling L. Inclusive education and children with disabilities: quality education for all in low and middle income countries. 2016. https://eajess.ac.tz/2020/05/26/community-perception-on-public-primary-schools-implications-for-sustainable-fee-free-basic-education-in-rorya-district-tanzania/ .

Zacharia L. Factors causing gender inequality in education in Tanzania: a case of Korogwe district secondary schools, PhD thesis. The Open University of Tanzania; 2014.

Download references

Author information

Authors and affiliations.

Tanzania Institute of Accountancy, Mwanza, Tanzania

Mbeya University of Science and Technology, Mbeya, Tanzania

Isack Kibona

You can also search for this author in PubMed   Google Scholar

Contributions

H.E was collecting the literatures and read and write major parts I.K was good on drafting conclusion and analysis part. But we work hand on hand together.

Corresponding author

Correspondence to Henry Nkya .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Nkya, H., Kibona, I. Systematic literature review of gender equity and social inclusion in primary education for teachers in Tanzania: assessing status and future directions. Discov Educ 3 , 122 (2024). https://doi.org/10.1007/s44217-024-00221-8

Download citation

Received : 26 March 2024

Accepted : 02 August 2024

Published : 13 August 2024

DOI : https://doi.org/10.1007/s44217-024-00221-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Equitable education
• Inclusive education
• Targeted intervention
• Find a journal
• Publish with us
• Track your research

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Usage of gamification techniques in software engineering education and training: a systematic review.

1. Introduction

2. research methodology, 2.1. prisma methodology.

• Population: the group or individuals targeted by the intervention.
• Intervention: the specific intervention or exposure being investigated.
• Comparison: the control or comparison group, if applicable.
• Outcome: the outcomes or effects measured in this study.

2.2. Research Questions Definition

• RQ1: What is the publication trend in the area of gamification applied to SEET? This question investigates the trend in publication quantity and the structure of publication venues, which are useful for understanding the progression of this topic.
• RQ2: In which areas of software engineering is gamification used? This question aims to identify the key areas of study and their contributions to the scientific community.
• RQ3: What are the analyzed application areas? This question explores the benefits of using gamification, considering its impact on learner engagement and performance.
• RQ4: What contribution does gamification offer when it is applied to SEET? This question examines the specific contribution and integration of gamification into educational practices.
• RQ5: On which continents is gamification mostly analyzed? This question aims to identify the continents that are most interested into gamification applied to SEET.
• RQ6: What are the advantages and disadvantages of gamification when applied to SEET? This question seeks to understand the the pros and cons to evaluate the success of gamification in educational settings.

2.3. Paper Selection

• Population-related search terms: “Software Engineering education”, “training”.
• Intervention-related search terms: “gamification”, “game-based learning”.

2.4. Inclusion and Exclusion Criteria

• Studies published between 2015 and 2023.
• Studies written in English.
• Studies published in peer-reviewed journals or conference proceedings.
• Studies focused on the application of gamification in software engineering education or training.
• Studies that present empirical evidence or substantial theoretical contributions.
• Studies for which the full text is not available (e.g., article not available online or DOI not found or not readable without subscriptions): 364 studies.
• Secondary or tertiary studies (e.g., reviews or surveys): 175 studies.
• Studies where gamification is not the main focus but is only mentioned: 284 studies.

3. Background

• Game mechanics refer to the rules and feedback systems that drive gameplay, such as scoring, levels, and rewards.
• Game dynamics involve the emotions and behaviors induced by the mechanics, such as competition, collaboration, and achievement.
• Aesthetics pertain to the overall look and feel of the gamified experience, which can enhance its appeal and immersion.

Application of Gamification in Software Engineering Education

4. results and discussion, 4.1. rq1: what is the publication trend in the area of gamification applied to seet, 4.2. rq2: in which areas of software engineering is gamification used, 4.3. rq3: what are the analyzed application areas.

• Proposal: This category includes studies that introduce new concepts, frameworks, or methodologies for applying gamification in SEET. We identified 10 papers that primarily focus on theoretical foundations and suggest innovative approaches to integrating gamification into educational contexts.
• Analysis: In this category, 10 studies provide detailed examinations of existing gamification techniques and their impacts on learning outcomes.
• Implementation/Tool: The largest category, with 24 studies, focuses on the practical aspects of implementing gamification. These papers describe the development and deployment of specific tools, platforms, or software that incorporate gamification elements into SEET. They often include case studies or reports on pilot projects.
• Validation: Comprising 17 studies, this category includes empirical research that evaluates the effectiveness of gamification through experiments, surveys, or longitudinal studies. These papers provide evidence-based insights into how gamification influences student engagement, motivation, and learning outcomes.
• Other: The remaining seven studies cover various other aspects of gamification that do not fit neatly into the above categories. This includes research on the broader impacts of gamification, such as its effects on educational policy, its role in lifelong learning, and interdisciplinary applications.

4.4. RQ4: What Contribution Does Gamification Offer When It Is Applied to SEET?

4.5. rq5: in which continents is gamification mostly analyzed, 4.6. rq6: what are the advantages and disadvantages of gamification when applied to seet.

• Difficulty in measuring performance improvements;
• Increased workload for educators [ 75 ];
• Lack of digital platforms to implement gamified techniques [ 36 ];
• Challenges in engaging all students;
• Difficulty for some students to understand the gamification method;
• Lack of appreciation for the method by some students;
• Difficulty for students in gaining human feedback, for platforms in which gamification is used alongside Artificial Intelligence [ 25 ];
• Insufficient knowledge of gamification approaches;
• Limited time and interest: students can sometimes become annoyed or disengaged by gamification elements if they find them distracting [ 90 ];
• Scarcity of materials and resources [ 75 ];
• Ensuring proper use of gamification by students.

5. Comparison with Other Review Papers

6. conclusions, author contributions, conflicts of interest, abbreviations.

• Malhotra, R.; Massoudi, M.; Jindal, R. An Innovative Approach: Coupling Project-Based Learning and Game-Based Learning Approach in Teaching Software Engineering Course. In Proceedings of the 2020 IEEE International Conference on Technology, Engineering, Management for Societal Impact Using Marketing, Entrepreneurship and Talent (TEMSMET), Bengaluru, India, 10 December 2020; pp. 1–5. [ Google Scholar ] [ CrossRef ]
• Kim, M.K.; Kim, S.M. Dynamic learner engagement in a wiki-enhanced writing course. J. Comput. High. Educ. 2020 , 32 , 582–606. [ Google Scholar ] [ CrossRef ]
• Ng, D.T.K.; Xinyu, C.; Leung, J.K.L.; Chu, S.K.W. Fostering students’ AI literacy development through educational games: AI knowledge, affective and cognitive engagement. J. Comput. Assist. Learn. 2024; online version of record before inclusion in an issue . [ Google Scholar ] [ CrossRef ]
• Magioli Sereno, M.; Ang, H.B. The impact of gamification on training, work engagement, and job satisfaction in banking. Int. J. Train. Dev. 2024 , 28 , 362–384. [ Google Scholar ] [ CrossRef ]
• Mongiello, M.; Nocera, F.; Parchitelli, A.; Patrono, L.; Rametta, P.; Riccardi, L.; Sergi, I. A smart iot-aware system for crisis scenario management. J. Commun. Softw. Syst. 2018 , 14 , 91–98. [ Google Scholar ] [ CrossRef ]
• Cavalera, G.; Rosito, R.C.; Lacasa, V.; Mongiello, M.; Nocera, F.; Patrono, L.; Sergi, I. An innovative smart system based on IoT technologies for fire and danger situations. In Proceedings of the 2019 4th International Conference on Smart and Sustainable Technologies (SpliTech), Split, Croatia, 18–21 June 2019; pp. 1–6. [ Google Scholar ] [ CrossRef ]
• Moher, D.; Liberati, A.; Tetzlaff, J.; Altman, D.G. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Int. J. Surg. 2010 , 8 , 336–341. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Petersen, K.; Feldt, R.; Mujtaba, S.; Mattsson, M. Systematic mapping studies in software engineering. In Proceedings of the 12th International Conference on Evaluation and Assessment in Software Engineering (EASE), Bari, Italy, 26–27 June 2008; BCS Learning & Development: Swindon, UK, 2008. [ Google Scholar ]
• Haddaway, N.R.; Page, M.J.; Pritchard, C.C.; McGuinness, L.A. PRISMA2020: An R package and Shiny app for producing PRISMA 2020-compliant flow diagrams, with interactivity for optimised digital transparency and Open Synthesis. Campbell Syst. Rev. 2022 , 18 , e1230. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Tsunoda, M.; Yumoto, H. Applying Gamification and Posing to Software Development. In Proceedings of the 2018 25th Asia-Pacific Software Engineering Conference (APSEC), Nara, Japan, 4–7 December 2018; pp. 638–642. [ Google Scholar ] [ CrossRef ]
• Takbiri, Y.; Amini, A.; Bastanfard, A. A Structured Gamification Approach for Improving Children’s Performance in Online Learning Platforms. In Proceedings of the 2019 5th Iranian Conference on Signal Processing and Intelligent Systems (ICSPIS), Shahrood, Iran, 18–19 December 2019; pp. 1–6. [ Google Scholar ] [ CrossRef ]
• Molins-Ruano, P.; Jurado, F.; Rodríguez, P.; Atrio, S.; Sacha, G.M. An Approach to Gamify an Adaptive Questionnaire Environment. In Proceedings of the 2016 IEEE Global Engineering Education Conference (EDUCON), Abu Dhabi, United Arab Emirates, 10–13 April 2016; pp. 1129–1133. [ Google Scholar ] [ CrossRef ]
• Ortega-Arranz, A.; Kalz, M.; Martínez-Monés, A. Creating Engaging Experiences in MOOCs through In-Course Redeemable Rewards. In Proceedings of the 2018 IEEE Global Engineering Education Conference (EDUCON), Canary Islands, Spain, 17–20 April 2018; pp. 1875–1882. [ Google Scholar ] [ CrossRef ]
• Monteiro, R.H.B.; Oliveira, S.R.B.; De Almeida Souza, M.R. A Standard Framework for Gamification Evaluation in Education and Training of Software Engineering: An Evaluation from a Proof of Concept. In Proceedings of the 2021 IEEE Frontiers in Education Conference (FIE), Lincoln, NE, USA, 13–16 October 2021; pp. 1–7. [ Google Scholar ] [ CrossRef ]
• Nistor, G.C.; Iacob, A. The advantages of gamification and game-based learning and their benefits in the development of education. In Proceedings of the International Scientific Conference eLearning and Software for Education, Bucharest, Romania, 19–20 April 2018; “Carol I” National Defence University: București, Romania, 2018; Volume 1, pp. 308–312. [ Google Scholar ]
• Fiore, M.; Gattullo, M.; Mongiello, M. First Steps in Constructing an AI-Powered Digital Twin Teacher: Harnessing Large Language Models in a Metaverse Classroom. In Proceedings of the 2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), Orlando, FL, USA, 16–21 March 2024; pp. 939–940. [ Google Scholar ] [ CrossRef ]
• Limantara, N.; Gaol, F.L.; Prabowo, H. Mechanics, dynamics, and aesthetics Framework on gamification at university. In Proceedings of the 2020 International Conference on Informatics, Multimedia, Cyber and Information System (ICIMCIS), Jakarta, Indonesia, 19–20 November 2020; pp. 34–39. [ Google Scholar ]
• Strmecki, D.; Bernik, A.; Radosevic, D. Gamification in E-Learning: Introducing Gamified Design Elements into E-Learning Systems. J. Comput. Sci. 2015 , 11 , 1108–1117. [ Google Scholar ] [ CrossRef ]
• Zichermann, G.; Cunningham, C. Gamification by Design: Implementing Game Mechanics in Web and Mobile Apps ; O’Reilly Media, Inc.: Sebastopol, CA, USA, 2011. [ Google Scholar ]
• Alsawaier, R.S. The effect of gamification on motivation and engagement. Int. J. Inf. Learn. Technol. 2018 , 35 , 56–79. [ Google Scholar ] [ CrossRef ]
• Apandi, A.M. Gamification meets mobile learning: Soft-skills enhancement. In Research Anthology on Developments in Gamification and Game-Based Learning ; IGI Global: Hershey, PA, USA, 2022; pp. 1280–1299. [ Google Scholar ]
• Kabilan, M.K.; Annamalai, N.; Chuah, K.M. Practices, purposes and challenges in integrating gamification using technology: A mixed-methods study on university academics. Educ. Inf. Technol. 2023 , 28 , 14249–14281. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Mongiello, M.; Pelliccione, P.; Siancalepore, M. Ac-contract: Run-time verification of context-aware systems. SEAMS 2015 , 15 , 106–115. [ Google Scholar ]
• Bistarelli, S.; Di Noia, T.; Mongiello, M.; Nocera, F. Pronto: An ontology driven business process mining tool. Procedia Comput. Sci. 2017 , 112 , 306–315. [ Google Scholar ] [ CrossRef ]
• Fulcini, T.; Torchiano, M. Is ChatGPT Capable of Crafting Gamification Strategies for Software Engineering Tasks? In Proceedings of the 2nd International Workshop on Gamification in Software Development, Verification, and Validation, San Francisco, CA, USA, 4 December 2023; pp. 22–28. [ Google Scholar ] [ CrossRef ]
• Ren, W.; Barrett, S. An Empirical Investigation on the Benefits of Gamification in Communication within University Development Teams. Comput. Appl. Eng. Educ. 2023 , 31 , 1808–1822. [ Google Scholar ] [ CrossRef ]
• Miljanovic, M.A.; Bradbury, J.S. Robobug: A serious game for learning debugging techniques. In Proceedings of the 2017 ACM Conference on International Computing Education Research, Tacoma, WA, USA, 18–20 August 2017; pp. 93–100. [ Google Scholar ]
• Navarro, E.O.; van der Hoek, A. SIMSE: An Interactive Simulation Game for Software Engineering Education. In Proceedings of the CATE, Kauai, HI, USA, 16–18 August 2004; Volume 1, pp. 12–17. [ Google Scholar ]
• Esper, S.; Foster, S.R.; Griswold, W.G. CodeSpells: Embodying the metaphor of wizardry for programming. In Proceedings of the 18th ACM Conference on Innovation and Technology in Computer Science Education, Canterbury, UK, 1–3 July 2013; pp. 249–254. [ Google Scholar ]
• Arif, R.F.; Rosyid, H.A.; Pujianto, U. Design and Implementation of Interactive Coding with Gamification for Web Programming Subject for Vocational High School Students. In Proceedings of the 2019 International Conference on Electrical, Electronics and Information Engineering (ICEEIE), Bali, Indonesia, 3–4 October 2019; Volume 6, pp. 177–182. [ Google Scholar ] [ CrossRef ]
• Rahim, R.H.A.; Tanalol, S.H.; Ismail, R.; Baharum, A.; Rahim, E.A.; Noor, N.A.M. Development of Gamification Linear Algebra Application Using Storytelling. In Proceedings of the 2019 International Conference on Information and Communication Technology Convergence (ICTC), Jeju Island, Republic of Korea, 16–18 October 2019; pp. 133–137. [ Google Scholar ] [ CrossRef ]
• Hajarian, M.; Diaz, P. Effective Gamification: A Guideline for Gamification Workshop of WEEF-GEDC 2021 Madrid Conference. In Proceedings of the 2021 World Engineering Education Forum/Global Engineering Deans Council (WEEF/GEDC), Madrid, Spain, 15–18 November 2021; pp. 506–510. [ Google Scholar ] [ CrossRef ]
• Iquira, D.; Galarza, M.; Sharhorodska, O. Enhancing Software Engineering Courses with a Mobile Gamified Platform: Results of a Mixed Approach. In Proceedings of the 2021 XVI Latin American Conference on Learning Technologies (LACLO), Arequipa, Peru, 19–21 October 2021; pp. 534–537. [ Google Scholar ] [ CrossRef ]
• Robledo-Rella, V.; de Lourdes Quezada Batalla, M.; Ramírez-de-Arellano, J.M.; Acosta, R.D.S. Gam-Mate: Gamification Applied to an Undergrad Discrete Math Course. In Proceedings of the 2022 10th International Conference on Information and Education Technology (ICIET), Matsue, Japan, 9–11 April 2022; pp. 135–139. [ Google Scholar ] [ CrossRef ]
• Gomes Fernandes Matsubara, P.; Lima Corrêa Da Silva, C. Game Elements in a Software Engineering Study Group: A Case Study. In Proceedings of the 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering Education and Training Track (ICSE-SEET), Buenos Aires, Argentina, 20–28 May 2017; pp. 160–169. [ Google Scholar ] [ CrossRef ]
• Rodrigues, P.; Souza, M.; Figueiredo, E. Games and Gamification in Software Engineering Education: A Survey with Educators. In Proceedings of the 2018 IEEE Frontiers in Education Conference (FIE), San Jose, CA, USA, 3–6 October 2018; pp. 1–9. [ Google Scholar ] [ CrossRef ]
• Quinde, C.P.; Paredes, R.I.; Maldonado, S.A.; Guerrero, J.S.; Toro, M.F.V. Gamification as a Didactic Strategy in a Digital Literancy: Case Study for Incacerated Individuals. In Proceedings of the 2018 IEEE Global Engineering Education Conference (EDUCON), Canary Islands, Spain, 17–20 April 2018; pp. 1314–1319. [ Google Scholar ] [ CrossRef ]
• John, I.; Fertig, T. Gamification for Software Engineering Students—An Experience Report. In Proceedings of the 2022 IEEE Global Engineering Education Conference (EDUCON), Tunis, Tunisia, 28–31 March 2022; pp. 1942–1947. [ Google Scholar ] [ CrossRef ]
• Ivanova, G.; Kozov, V.; Zlatarov, P. Gamification in Software Engineering Education. In Proceedings of the 2019 42nd International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), Opatija, Croatia, 20–24 May 2019; pp. 1445–1450. [ Google Scholar ] [ CrossRef ]
• Gasca-Hurtado, G.P.; Gómez-Álvarez, M.C.; Hincapié, J.A.; Zepeda, V.V. Gamification of an Educational Environment in Software Engineering: Case Study for Digital Accessibility of People with Disabilities. IEEE Rev. Iberoam. Tecnol. Aprendiz. 2021 , 16 , 382–392. [ Google Scholar ] [ CrossRef ]
• Carreño-León, M.; Sandoval-Bringas, A.; Álvarez-Rodríguez, F.; Camacho-González, Y. Gamification Technique for Teaching Programming. In Proceedings of the 2018 IEEE Global Engineering Education Conference (EDUCON), Canary Islands, Spain, 17–20 April 2018; pp. 2009–2014. [ Google Scholar ] [ CrossRef ]
• Sherif, E.; Liu, A.; Nguyen, B.; Lerner, S.; Griswold, W.G. Gamification to Aid the Learning of Test Coverage Concepts. In Proceedings of the 2020 IEEE 32nd Conference on Software Engineering Education and Training (CSEE&T), Munich, Germany, 9–12 November 2020; pp. 1–5. [ Google Scholar ] [ CrossRef ]
• Norsanto, D.; Rosmansyah, Y. Gamified Mobile Micro-Learning Framework: A Case Study of Civil Service Management Learning. In Proceedings of the 2018 International Conference on Information and Communications Technology (ICOIACT), Yogyakarta, Indonesia, 6–7 March 2018; pp. 146–151. [ Google Scholar ] [ CrossRef ]
• Call, M.W.; Fox, E.; Sprint, G. Gamifying Software Engineering Tools to Motivate Computer Science Students to Start and Finish Programming Assignments Earlier. IEEE Trans. Educ. 2021 , 64 , 423–431. [ Google Scholar ] [ CrossRef ]
• Trinidad, M.; Calderón, A.; Ruiz, M. GoRace: A Multi-Context and Narrative-Based Gamification Suite to Overcome Gamification Technological Challenges. IEEE Access 2021 , 9 , 65882–65905. [ Google Scholar ] [ CrossRef ]
• Prasetya, W.; Leek, C.; Melkonian, O.; ten Tusscher, J.; van Bergen, J.; Everink, J.; van der Klis, T.; Meijerink, R.; Oosenbrug, R.; Oostveen, J.; et al. Having Fun in Learning Formal Specifications. In Proceedings of the 2019 IEEE/ACM 41st International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET), Montreal, QC, Canada, 25–31 May 2019; pp. 192–196. [ Google Scholar ] [ CrossRef ]
• Bucchiarone, A.; Cicchetti, A.; Bassanelli, S.; Marconi, A. How to Merge Gamification Efforts for Programming and Modelling: A Tool Implementation Perspective. In Proceedings of the 2021 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C), Fukuoka, Japan, 10–15 October 2021; pp. 721–726. [ Google Scholar ] [ CrossRef ]
• Lema Moreta, L.; Gamboa, A.C.; Palacios, M.G. Implementing a Gamified Application for a Risk Management Course. In Proceedings of the 2016 IEEE Ecuador Technical Chapters Meeting (ETCM), Guayaquil, Ecuador, 12–14 October 2016; pp. 1–6. [ Google Scholar ] [ CrossRef ]
• Ouhbi, S.; Pombo, N. Software Engineering Education: Challenges and Perspectives. In Proceedings of the 2020 IEEE Global Engineering Education Conference (EDUCON), Porto, Portugal, 27–30 April 2020; pp. 202–209. [ Google Scholar ] [ CrossRef ]
• Villagra, S.; De Benedetti, G.; Bruno, T.; Fernández, L.; Outeda, N. Teaching Software Engineering: An Active Learning Experience. In Proceedings of the 2020 IEEE Congreso Bienal de Argentina (ARGENCON), Resistencia, Argentina, 1–4 December 2020; pp. 1–6. [ Google Scholar ] [ CrossRef ]
• Moser, G.; Vallon, R.; Bernhart, M.; Grechenig, T. Teaching Software Quality Assurance with Gamification and Continuous Feedback Techniques. In Proceedings of the 2021 IEEE Global Engineering Education Conference (EDUCON), Vienna, Austria, 21–23 April 2021; pp. 505–509. [ Google Scholar ] [ CrossRef ]
• Rattadilok, P.; Roadknight, C.; Li, L. Teaching Students About Machine Learning Through a Gamified Approach. In Proceedings of the 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE), Wollongong, NSW, Australia, 4–7 December 2018; pp. 1011–1015. [ Google Scholar ] [ CrossRef ]
• Bucchiarone, A.; Cicchetti, A.; Loria, E.; Marconi, A. Towards a Framework to Assist Iterative and Adaptive Design in Gameful Systems. In Proceedings of the 2021 36th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW), Melbourne, Australia, 15–19 November 2021; pp. 78–84. [ Google Scholar ] [ CrossRef ]
• Ebert, C.; Vizcaino, A.; Grande, R. Unlock the Business Value of Gamification. IEEE Softw. 2022 , 39 , 15–22. [ Google Scholar ] [ CrossRef ]
• Maxim, B.R.; Brunvand, S.; Decker, A. Use of Role-Play and Gamification in a Software Project Course. In Proceedings of the 2017 IEEE Frontiers in Education Conference (FIE), Indianapolis, IN, USA, 18–21 October 2017; pp. 1–5. [ Google Scholar ] [ CrossRef ]
• Jiménez-Hernández, E.M.; Jiménez-Murillo, J.A.; Segura-Castruita, M.A.; González-Leal, I. Using a Serious Video Game to Support the Learning of Tree Traversals. In Proceedings of the 2021 9th International Conference in Software Engineering Research and Innovation (CONISOFT), San Diego, CA, USA, 25–29 October 2021; pp. 238–244. [ Google Scholar ] [ CrossRef ]
• Nagaria, B.; Evans, B.C.; Mann, A.; Arzoky, M. Using an Instant Visual and Text Based Feedback Tool to Teach Path Finding Algorithms: A Concept. In Proceedings of the 2021 Third International Workshop on Software Engineering Education for the Next Generation (SEENG), Virtual, 24 May 2021; pp. 11–15. [ Google Scholar ] [ CrossRef ]
• Margalit, O. Using Computer Programming Competition for Cyber Education. In Proceedings of the 2016 IEEE International Conference on Software Science, Technology and Engineering (SWSTE), Beer Sheva, Israel, 23–24 June 2016; pp. 104–107. [ Google Scholar ] [ CrossRef ]
• Stol, K.J.; Schaarschmidt, M.; Goldblit, S. Gamification in Software Engineering: The Mediating Role of Developer Engagement and Job Satisfaction. Empir. Softw. Eng. 2022 , 27 , 35. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Đambić, G.; Keščec, T.; Kučak, D. A Blended Learning with Gamification Approach for Teaching Programming Courses in Higher Education. In Proceedings of the 2021 44th International Convention on Information, Communication and Electronic Technology (MIPRO), Opatija, Croatia, 27 September–1 October 2021; pp. 843–847. [ Google Scholar ] [ CrossRef ]
• Mi, Q.; Keung, J.; Mei, X.; Xiao, Y.; Chan, W.K. A Gamification Technique for Motivating Students to Learn Code Readability in Software Engineering. In Proceedings of the 2018 International Symposium on Educational Technology (ISET), Osaka, Japan, 31 July–2 August 2018; pp. 250–254. [ Google Scholar ] [ CrossRef ]
• Skalka, J.; Drlík, M.; Obonya, J.; Cápay, M. Architecture Proposal for Micro-Learning Application for Learning and Teaching Programming Courses. In Proceedings of the 2020 IEEE Global Engineering Education Conference (EDUCON), Porto, Portugal, 27–30 April 2020; pp. 980–987. [ Google Scholar ] [ CrossRef ]
• Silvis-Cividjian, N. Awesome Bug Manifesto: Teaching an Engaging and Inspiring Course on Software Testing (Position Paper). In Proceedings of the 2021 Third International Workshop on Software Engineering Education for the Next Generation (SEENG), Madrid, Spain, 24 May 2021; pp. 16–20. [ Google Scholar ] [ CrossRef ]
• Makarova, I.; Pashkevich, A.; Shubenkova, K. Blended Learning Technologies in the Automotive Industry Specialists’ Training. In Proceedings of the 2018 32nd International Conference on Advanced Information Networking and Applications Workshops (WAINA), Krakow, Poland, 16–18 May 2018; pp. 319–324. [ Google Scholar ] [ CrossRef ]
• de Paula Porto, D.; de Jesus, G.M.; Ferrari, F.C.; Fabbri, S.C.P.F. Initiatives and Challenges of Using Gamification in Software Engineering: A Systematic Mapping. J. Syst. Softw. 2021 , 173 , 110870. [ Google Scholar ] [ CrossRef ]
• Vlahu-Gjorgievska, E.; Videnovik, M.; Trajkovik, V. Computational Thinking and Coding Subject in Primary Schools: Methodological Approach Based on Alternative Cooperative and Individual Learning Cycles. In Proceedings of the 2018 IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE), Wollongong, NSW, Australia, 4–7 December 2018; pp. 77–83. [ Google Scholar ] [ CrossRef ]
• Chan, Y.C.; Min Gan, C.; Lim, C.Y.; Hwa Tan, T.; Cao, Q.; Seow, C.K. Learning CS Subjects of Professional Software Development and Team Projects. In Proceedings of the 2022 IEEE International Conference on Teaching, Assessment and Learning for Engineering (TALE), Hung Hom, Hong Kong, 4–7 December 2022; pp. 71–77. [ Google Scholar ] [ CrossRef ]
• Figueiredo, J.; García-Peñalvo, F.J. Increasing Student Motivation in Computer Programming with Gamification. In Proceedings of the 2020 IEEE Global Engineering Education Conference (EDUCON), Porto, Portugal, 27–30 April 2020; pp. 997–1000. [ Google Scholar ] [ CrossRef ]
• Pratama, F.A.; Silitonga, R.M.; Jou, Y.T. Rimigs: The Impact of Gamification on Students’ Motivation and Performance in Programming Class. Indones. J. Electr. Eng. Comput. Sci. 2021 , 24 , 1789–1795. [ Google Scholar ] [ CrossRef ]
• Naik, N.; Jenkins, P. Relax, It’sa Game: Utilising Gamification in Learning Agile Scrum Software Development. In Proceedings of the 2019 IEEE Conference on Games (CoG), London, UK, 20–23 August 2019; pp. 1–4. [ Google Scholar ]
• Swacha, J.; Szydłowska, J. Does Gamification Make a Difference in Programming Education? Evaluating FGPE-Supported Learning Outcomes. Educ. Sci. 2023 , 13 , 984. [ Google Scholar ] [ CrossRef ]
• Sousa-Vieira, M.E.; López-Ardao, J.C.; Fernández-Veiga, M.; Rodríguez-Rubio, R.F. Study of the Impact of Social Learning and Gamification Methodologies on Learning Results in Higher Education. Comput. Appl. Eng. Educ. 2023 , 31 , 131–153. [ Google Scholar ] [ CrossRef ]
• Monteiro, R.; Souza, M.; Oliveira, S.; Soares, E. The Adoption of a Framework to Support the Evaluation of Gamification Strategies in Software Engineering Education. In Proceedings of the 14th International Conference on Computer Supported Education, Online, 22–24 April 2022; pp. 450–457. [ Google Scholar ] [ CrossRef ]
• Jusas, V.; Barisas, D.; Jančiukas, M. Game Elements towards More Sustainable Learning in Object-Oriented Programming Course. Sustainability 2022 , 14 , 2325. [ Google Scholar ] [ CrossRef ]
• Maher, Y.; Moussa, S.M.; Khalifa, M.E. Learners on Focus: Visualizing Analytics through an Integrated Model for Learning Analytics in Adaptive Gamified E-Learning. IEEE Access 2020 , 8 , 197597–197616. [ Google Scholar ] [ CrossRef ]
• Bachtiar, F.A.; Pradana, F.; Priyambadha, B.; Bastari, D.I. CoMa: Development of Gamification-based E-learning. In Proceedings of the 2018 10th International Conference on Information Technology and Electrical Engineering (ICITEE), Bali, Indonesia, 24–26 July 2018; pp. 1–6. [ Google Scholar ] [ CrossRef ]
• Laskowski, M. Implementing Gamification Techniques into University Study Path - A Case Study. In Proceedings of the 2015 IEEE Global Engineering Education Conference (EDUCON), Tallinn, Estonia, 18–20 March 2015; pp. 582–586. [ Google Scholar ] [ CrossRef ]
• Fuchs, M.; Wolff, C. Improving Programming Education through Gameful, Formative Feedback. In Proceedings of the 2016 IEEE Global Engineering Education Conference (EDUCON), Abu Dhabi, United Arab Emirates, 10–13 April 2016; pp. 860–867. [ Google Scholar ] [ CrossRef ]
• Bucchiarone, A.; Cooper, K.M.L.; Lin, D.; Melcer, E.F.; Sung, K. Games and Software Engineering: Engineering Fun, Inspiration, and Motivation. ACM SIGSOFT Softw. Eng. Notes 2023 , 48 , 85–89. [ Google Scholar ] [ CrossRef ]
• Bucchiarone, A.; Martorella, T.; Colombo, D.; Cicchetti, A.; Marconi, A. POLYGLOT for Gamified Education: Mixing Modelling and Programming Exercises. In Proceedings of the 2021 ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C), Fukuoka, Japan, 10–15 October 2021; pp. 605–609. [ Google Scholar ] [ CrossRef ]
• Poecze, F.; Tjoa, A.M. Meta-Analytical Considerations for Gamification in Higher Education: Existing Approaches and Future Research Agenda. In Proceedings of the 2020 4th International Conference on Informatics and Computational Sciences (ICICoS), Semarang, Indonesia, 10–11 November 2020; pp. 1–6. [ Google Scholar ] [ CrossRef ]
• Cabezas, I. On Combining Gamification Theory and ABET Criteria for Teaching and Learning Engineering. In Proceedings of the 2015 IEEE Frontiers in Education Conference (FIE), Washington, DC, USA, 21–24 October 2015; pp. 1–9. [ Google Scholar ] [ CrossRef ]
• Bucchiarone, A.; Cooper, K.M.L.; Lin, D.; Smith, A.; Wanick, V. Fostering Collaboration and Advancing Research in Software Engineering and Game Development for Serious Contexts. ACM SIGSOFT Softw. Eng. Notes 2023 , 48 , 46–50. [ Google Scholar ] [ CrossRef ]
• Ristov, S.; Ackovska, N.; Kirandziska, V. Positive Experience of the Project Gamification in the Microprocessors and Microcontrollers Course. In Proceedings of the 2015 IEEE Global Engineering Education Conference (EDUCON), Tallinn, Estonia, 18–20 March 2015; pp. 511–517. [ Google Scholar ] [ CrossRef ]
• Bernik, A.; Radošević, D.; Bubaš, G. Introducing Gamification into E-Learning University Courses. In Proceedings of the 2017 40th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), Opatija, Croatia, 22–26 May 2017; pp. 711–716. [ Google Scholar ] [ CrossRef ]
• Schäfer, U. Training Scrum with Gamification: Lessons Learned after Two Teaching Periods. In Proceedings of the 2017 IEEE Global Engineering Education Conference (EDUCON), Athens, Greece, 25–28 April 2017; pp. 754–761. [ Google Scholar ] [ CrossRef ]
• Petrov, E.; Mustafina, J.; Alloghani, M. Overview on Modern Serious Games for Regional and Minority Languages Promotion. In Proceedings of the 2017 10th International Conference on Developments in eSystems Engineering (DeSE), Paris, France, 14–16 June 2017; pp. 120–123. [ Google Scholar ] [ CrossRef ]
• Tsalikidis, K.; Pavlidis, G. jLegends: Online Game to Train Programming Skills. In Proceedings of the 2016 7th International Conference on Information, Intelligence, Systems & Applications (IISA), Chalkidiki, Greece, 13–15 July 2016; pp. 1–6. [ Google Scholar ] [ CrossRef ]
• Gamarra, M.; Dominguez, A.; Velazquez, J.; Páez, H. A Gamification Strategy in Engineering Education—A Case Study on Motivation and Engagement. Comput. Appl. Eng. Educ. 2022 , 30 , 472–482. [ Google Scholar ] [ CrossRef ]
• Kadar, R.; Wahab, N.A.; Othman, J.; Shamsuddin, M.; Mahlan, S.B. A study of difficulties in teaching and learning programming: A systematic literature review. Int. J. Acad. Res. Progress. Educ. Dev. 2021 , 10 , 591–605. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Fiore, M.; Mongiello, M. Using Peer Assessment Leveraging Large Language Models in Software Engineering Education. Int. J. Softw. Eng. Knowl. Eng. 2024 , 34 . [ Google Scholar ] [ CrossRef ]
• Dal Sasso, T.; Mocci, A.; Lanza, M.; Mastrodicasa, E. How to gamify software engineering. In Proceedings of the 2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER), Klagenfurt, Austria, 20–24 February 2017; pp. 261–271. [ Google Scholar ] [ CrossRef ]
• Pedreira, O.; García, F.; Brisaboa, N.; Piattini, M. Gamification in software engineering—A systematic mapping. Inf. Softw. Technol. 2015 , 57 , 157–168. [ Google Scholar ] [ CrossRef ]
• Barreto, C.F.; França, C. Gamification in software engineering: A literature review. In Proceedings of the 2021 IEEE/ACM 13th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE), Madrid, Spain, 20–21 May 2021; pp. 105–108. [ Google Scholar ]
• Ngandu, M.R.; Risinamhodzi, D.; Dzvapatsva, G.P.; Matobobo, C. Capturing student interest in software engineering through gamification: A systematic literature review. Discov. Educ. 2023 , 2 , 47. [ Google Scholar ] [ CrossRef ]
• Chamorro-Atalaya, O.; Morales-Romero, G.; Trinidad-Loli, N.; Caycho-Salas, B.; Guía-Altamirano, T.; Auqui-Ramos, E.; Rocca-Carvajal, Y.; Arones, M.; Arévalo-Tuesta, J.A.; Gonzales-Huaytahuilca, R. Gamification in engineering education during COVID-19: A systematic review on design considerations and success factors in its implementation. Int. J. Learn. Teach. Educ. Res. 2023 , 22 , 301–327. [ Google Scholar ] [ CrossRef ]
• Barbosa Monteiro, R.H.; de Almeida Souza, M.R.; Bezerra Oliveira, S.R.; dos Santos Portela, C.; de Cristo Lobato, C.E. The Diversity of Gamification Evaluation in the Software Engineering Education and Industry: Trends, Comparisons and Gaps. In Proceedings of the 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering Education and Training (ICSE-SEET), Virtual Event, 25–28 May 2021; pp. 154–164. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Di Nardo, V.; Fino, R.; Fiore, M.; Mignogna, G.; Mongiello, M.; Simeone, G. Usage of Gamification Techniques in Software Engineering Education and Training: A Systematic Review. Computers 2024 , 13 , 196. https://doi.org/10.3390/computers13080196

Di Nardo V, Fino R, Fiore M, Mignogna G, Mongiello M, Simeone G. Usage of Gamification Techniques in Software Engineering Education and Training: A Systematic Review. Computers . 2024; 13(8):196. https://doi.org/10.3390/computers13080196

Di Nardo, Vincenzo, Riccardo Fino, Marco Fiore, Giovanni Mignogna, Marina Mongiello, and Gaetano Simeone. 2024. "Usage of Gamification Techniques in Software Engineering Education and Training: A Systematic Review" Computers 13, no. 8: 196. https://doi.org/10.3390/computers13080196

Article Metrics

Further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals