importance of literature review in research

A Guide to Literature Reviews

Importance of a good literature review.

• Conducting the Literature Review
• Structure and Writing Style
• Types of Literature Reviews
• Citation Management Software This link opens in a new window
• Acknowledgements

A literature review is not only a summary of key sources, but  has an organizational pattern which combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

The purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].
• << Previous: Definition
• Next: Conducting the Literature Review >>
• Last Updated: Jul 3, 2024 3:13 PM
• URL: https://libguides.mcmaster.ca/litreview

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• How to Write a Literature Review | Guide, Examples, & Templates

How to Write a Literature Review | Guide, Examples, & Templates

Published on January 2, 2023 by Shona McCombes . Revised on September 11, 2023.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research that you can later apply to your paper, thesis, or dissertation topic .

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates, and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarize sources—it analyzes, synthesizes , and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Upload your document to correct all your mistakes in minutes

Table of contents

What is the purpose of a literature review, examples of literature reviews, step 1 – search for relevant literature, step 2 – evaluate and select sources, step 3 – identify themes, debates, and gaps, step 4 – outline your literature review’s structure, step 5 – write your literature review, free lecture slides, other interesting articles, frequently asked questions, introduction.

• Quick Run-through
• Step 1 & 2

When you write a thesis , dissertation , or research paper , you will likely have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and its scholarly context
• Develop a theoretical framework and methodology for your research
• Position your work in relation to other researchers and theorists
• Show how your research addresses a gap or contributes to a debate
• Evaluate the current state of research and demonstrate your knowledge of the scholarly debates around your topic.

Writing literature reviews is a particularly important skill if you want to apply for graduate school or pursue a career in research. We’ve written a step-by-step guide that you can follow below.

Don't submit your assignments before you do this

The academic proofreading tool has been trained on 1000s of academic texts. Making it the most accurate and reliable proofreading tool for students. Free citation check included.

Try for free

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research problem and questions .

Make a list of keywords

Start by creating a list of keywords related to your research question. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list as you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some useful databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can also use boolean operators to help narrow down your search.

Make sure to read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

You likely won’t be able to read absolutely everything that has been written on your topic, so it will be necessary to evaluate which sources are most relevant to your research question.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models, and methods?
• Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible , and make sure you read any landmark studies and major theories in your field of research.

You can use our template to summarize and evaluate sources you’re thinking about using. Click on either button below to download.

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It is important to keep track of your sources with citations to avoid plagiarism . It can be helpful to make an annotated bibliography , where you compile full citation information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

To begin organizing your literature review’s argument and structure, be sure you understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly visual platforms like Instagram and Snapchat—this is a gap that you could address in your own research.

There are various approaches to organizing the body of a literature review. Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarizing sources in order.

Try to analyze patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organize your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text , your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, you can follow these tips:

• Summarize and synthesize: give an overview of the main points of each source and combine them into a coherent whole
• Analyze and interpret: don’t just paraphrase other researchers — add your own interpretations where possible, discussing the significance of findings in relation to the literature as a whole
• Critically evaluate: mention the strengths and weaknesses of your sources
• Write in well-structured paragraphs: use transition words and topic sentences to draw connections, comparisons and contrasts

In the conclusion, you should summarize the key findings you have taken from the literature and emphasize their significance.

When you’ve finished writing and revising your literature review, don’t forget to proofread thoroughly before submitting. Not a language expert? Check out Scribbr’s professional proofreading services !

This article has been adapted into lecture slides that you can use to teach your students about writing a literature review.

Scribbr slides are free to use, customize, and distribute for educational purposes.

Open Google Slides Download PowerPoint

If you want to know more about the research process , methodology , research bias , or statistics , make sure to check out some of our other articles with explanations and examples.

• Sampling methods
• Simple random sampling
• Stratified sampling
• Cluster sampling
• Likert scales
• Reproducibility

 Statistics

• Null hypothesis
• Statistical power
• Probability distribution
• Effect size
• Poisson distribution

Research bias

• Optimism bias
• Cognitive bias
• Implicit bias
• Hawthorne effect
• Anchoring bias
• Explicit bias

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a thesis, dissertation , or research paper , in order to situate your work in relation to existing knowledge.

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarize yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your thesis or dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

A literature review is a survey of credible sources on a topic, often used in dissertations , theses, and research papers . Literature reviews give an overview of knowledge on a subject, helping you identify relevant theories and methods, as well as gaps in existing research. Literature reviews are set up similarly to other  academic texts , with an introduction , a main body, and a conclusion .

An  annotated bibliography is a list of  source references that has a short description (called an annotation ) for each of the sources. It is often assigned as part of the research process for a  paper .  

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

McCombes, S. (2023, September 11). How to Write a Literature Review | Guide, Examples, & Templates. Scribbr. Retrieved September 9, 2024, from https://www.scribbr.com/dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, what is a theoretical framework | guide to organizing, what is a research methodology | steps & tips, how to write a research proposal | examples & templates, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

Libraries | Research Guides

Literature reviews, what is a literature review, learning more about how to do a literature review.

• Planning the Review
• The Research Question
• Choosing Where to Search
• Organizing the Review
• Writing the Review

A literature review is a review and synthesis of existing research on a topic or research question. A literature review is meant to analyze the scholarly literature, make connections across writings and identify strengths, weaknesses, trends, and missing conversations. A literature review should address different aspects of a topic as it relates to your research question. A literature review goes beyond a description or summary of the literature you have read. 

• Sage Research Methods Core This link opens in a new window SAGE Research Methods supports research at all levels by providing material to guide users through every step of the research process. SAGE Research Methods is the ultimate methods library with more than 1000 books, reference works, journal articles, and instructional videos by world-leading academics from across the social sciences, including the largest collection of qualitative methods books available online from any scholarly publisher. – Publisher

• Next: Planning the Review >>
• Last Updated: Jul 8, 2024 11:22 AM
• URL: https://libguides.northwestern.edu/literaturereviews

Literature Review - what is a Literature Review, why it is important and how it is done

What are literature reviews, goals of literature reviews, types of literature reviews, about this guide/licence.

• Strategies to Find Sources
• Evaluating Literature Reviews and Sources
• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings
• Useful Resources

Help is Just a Click Away

Search our FAQ Knowledge base, ask a question, chat, send comments...

Go to LibAnswers

 What is a literature review? "A literature review is an account of what has been published on a topic by accredited scholars and researchers. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries. " - Quote from Taylor, D. (n.d) "The literature review: A few tips on conducting it"

Source NC State University Libraries. This video is published under a Creative Commons 3.0 BY-NC-SA US license.

What are the goals of creating a Literature Review?

• To develop a theory or evaluate an existing theory
• To summarize the historical or existing state of a research topic
• Identify a problem in a field of research 

- Baumeister, R.F. & Leary, M.R. (1997). "Writing narrative literature reviews," Review of General Psychology , 1(3), 311-320.

When do you need to write a Literature Review?

• When writing a prospectus or a thesis/dissertation
• When writing a research paper
• When writing a grant proposal

In all these cases you need to dedicate a chapter in these works to showcase what have been written about your research topic and to point out how your own research will shed a new light into these body of scholarship.

Literature reviews are also written as standalone articles as a way to survey a particular research topic in-depth. This type of literature reviews look at a topic from a historical perspective to see how the understanding of the topic have change through time.

What kinds of literature reviews are written?

• Narrative Review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified. The review ends with a conclusion section which summarizes the findings regarding the state of the research of the specific study, the gaps identify and if applicable, explains how the author's research will address gaps identify in the review and expand the knowledge on the topic reviewed.
• Book review essays/ Historiographical review essays : This is a type of review that focus on a small set of research books on a particular topic " to locate these books within current scholarship, critical methodologies, and approaches" in the field. - LARR
• Systematic review : "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139). Nelson, L.K. (2013). Research in Communication Sciences and Disorders . San Diego, CA: Plural Publishing.
• Meta-analysis : "Meta-analysis is a method of reviewing research findings in a quantitative fashion by transforming the data from individual studies into what is called an effect size and then pooling and analyzing this information. The basic goal in meta-analysis is to explain why different outcomes have occurred in different studies." (p. 197). Roberts, M.C. & Ilardi, S.S. (2003). Handbook of Research Methods in Clinical Psychology . Malden, MA: Blackwell Pub.
• Meta-synthesis : "Qualitative meta-synthesis is a type of qualitative study that uses as data the findings from other qualitative studies linked by the same or related topic." (p.312). Zimmer, L. (2006). "Qualitative meta-synthesis: A question of dialoguing with texts," Journal of Advanced Nursing , 53(3), 311-318.

Guide adapted from "Literature Review" , a guide developed by Marisol Ramos used under CC BY 4.0 /modified from original.

• Next: Strategies to Find Sources >>
• Last Updated: Jul 3, 2024 10:56 AM
• URL: https://lit.libguides.com/Literature-Review

The Library, Technological University of the Shannon: Midwest

Service update: Some parts of the Library’s website will be down for maintenance on August 11.

Secondary menu

• Log in to your Library account
• Hours and Maps
• Connect from Off Campus
• UC Berkeley Home

Search form

Conducting a literature review: why do a literature review, why do a literature review.

• How To Find "The Literature"
• Found it -- Now What?

Besides the obvious reason for students -- because it is assigned! -- a literature review helps you explore the research that has come before you, to see how your research question has (or has not) already been addressed.

You identify:

• core research in the field
• experts in the subject area
• methodology you may want to use (or avoid)
• gaps in knowledge -- or where your research would fit in

It Also Helps You:

• Publish and share your findings
• Justify requests for grants and other funding
• Identify best practices to inform practice
• Set wider context for a program evaluation
• Compile information to support community organizing

Great brief overview, from NCSU

Want To Know More?

• Next: How To Find "The Literature" >>
• Last Updated: Apr 25, 2024 1:10 PM
• URL: https://guides.lib.berkeley.edu/litreview

• University of Texas Libraries

Literature Reviews

• What is a literature review?
• Steps in the Literature Review Process
• Define your research question
• Determine inclusion and exclusion criteria
• Choose databases and search
• Review Results
• Synthesize Results
• Analyze Results
• Librarian Support
• Artificial Intelligence (AI) Tools

What is a Literature Review?

A literature or narrative review is a comprehensive review and analysis of the published literature on a specific topic or research question. The literature that is reviewed contains: books, articles, academic articles, conference proceedings, association papers, and dissertations. It contains the most pertinent studies and points to important past and current research and practices. It provides background and context, and shows how your research will contribute to the field. 

A literature review should: 

• Provide a comprehensive and updated review of the literature;
• Explain why this review has taken place;
• Articulate a position or hypothesis;
• Acknowledge and account for conflicting and corroborating points of view

From  S age Research Methods

Purpose of a Literature Review

A literature review can be written as an introduction to a study to:

• Demonstrate how a study fills a gap in research
• Compare a study with other research that's been done

Or it can be a separate work (a research article on its own) which:

• Organizes or describes a topic
• Describes variables within a particular issue/problem

Limitations of a Literature Review

Some of the limitations of a literature review are:

• It's a snapshot in time. Unlike other reviews, this one has beginning, a middle and an end. There may be future developments that could make your work less relevant.
• It may be too focused. Some niche studies may miss the bigger picture.
• It can be difficult to be comprehensive. There is no way to make sure all the literature on a topic was considered.
• It is easy to be biased if you stick to top tier journals. There may be other places where people are publishing exemplary research. Look to open access publications and conferences to reflect a more inclusive collection. Also, make sure to include opposing views (and not just supporting evidence).

Source: Grant, Maria J., and Andrew Booth. “A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies.” Health Information & Libraries Journal, vol. 26, no. 2, June 2009, pp. 91–108. Wiley Online Library, doi:10.1111/j.1471-1842.2009.00848.x.

Librarian Assistance

For help, please contact the librarian for your subject area.  We have a guide to library specialists by subject .

• Last Updated: Aug 26, 2024 5:59 AM
• URL: https://guides.lib.utexas.edu/literaturereviews

Research Methods

• Getting Started
• Literature Review Research
• Research Design
• Research Design By Discipline
• SAGE Research Methods
• Teaching with SAGE Research Methods

Literature Review

• What is a Literature Review?
• What is NOT a Literature Review?
• Purposes of a Literature Review
• Types of Literature Reviews
• Literature Reviews vs. Systematic Reviews
• Systematic vs. Meta-Analysis

Literature Review  is a comprehensive survey of the works published in a particular field of study or line of research, usually over a specific period of time, in the form of an in-depth, critical bibliographic essay or annotated list in which attention is drawn to the most significant works.

Also, we can define a literature review as the collected body of scholarly works related to a topic:

• Summarizes and analyzes previous research relevant to a topic
• Includes scholarly books and articles published in academic journals
• Can be an specific scholarly paper or a section in a research paper

The objective of a Literature Review is to find previous published scholarly works relevant to an specific topic

• Help gather ideas or information
• Keep up to date in current trends and findings
• Help develop new questions

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Helps focus your own research questions or problems
• Discovers relationships between research studies/ideas.
• Suggests unexplored ideas or populations
• Identifies major themes, concepts, and researchers on a topic.
• Tests assumptions; may help counter preconceived ideas and remove unconscious bias.
• Identifies critical gaps, points of disagreement, or potentially flawed methodology or theoretical approaches.
• Indicates potential directions for future research.

All content in this section is from Literature Review Research from Old Dominion University 

Keep in mind the following, a literature review is NOT:

Not an essay 

Not an annotated bibliography  in which you summarize each article that you have reviewed.  A literature review goes beyond basic summarizing to focus on the critical analysis of the reviewed works and their relationship to your research question.

Not a research paper   where you select resources to support one side of an issue versus another.  A lit review should explain and consider all sides of an argument in order to avoid bias, and areas of agreement and disagreement should be highlighted.

A literature review serves several purposes. For example, it

• provides thorough knowledge of previous studies; introduces seminal works.
• helps focus one’s own research topic.
• identifies a conceptual framework for one’s own research questions or problems; indicates potential directions for future research.
• suggests previously unused or underused methodologies, designs, quantitative and qualitative strategies.
• identifies gaps in previous studies; identifies flawed methodologies and/or theoretical approaches; avoids replication of mistakes.
• helps the researcher avoid repetition of earlier research.
• suggests unexplored populations.
• determines whether past studies agree or disagree; identifies controversy in the literature.
• tests assumptions; may help counter preconceived ideas and remove unconscious bias.

As Kennedy (2007) notes*, it is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the original studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally that become part of the lore of field. In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews.

Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are several approaches to how they can be done, depending upon the type of analysis underpinning your study. Listed below are definitions of types of literature reviews:

Argumentative Review      This form examines literature selectively in order to support or refute an argument, deeply imbedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to to make summary claims of the sort found in systematic reviews.

Integrative Review      Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication.

Historical Review      Few things rest in isolation from historical precedent. Historical reviews are focused on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review      A review does not always focus on what someone said [content], but how they said it [method of analysis]. This approach provides a framework of understanding at different levels (i.e. those of theory, substantive fields, research approaches and data collection and analysis techniques), enables researchers to draw on a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection and data analysis, and helps highlight many ethical issues which we should be aware of and consider as we go through our study.

Systematic Review      This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyse data from the studies that are included in the review. Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?"

Theoretical Review      The purpose of this form is to concretely examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review help establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

* Kennedy, Mary M. "Defining a Literature."  Educational Researcher  36 (April 2007): 139-147.

All content in this section is from The Literature Review created by Dr. Robert Larabee USC

Robinson, P. and Lowe, J. (2015),  Literature reviews vs systematic reviews.  Australian and New Zealand Journal of Public Health, 39: 103-103. doi: 10.1111/1753-6405.12393

What's in the name? The difference between a Systematic Review and a Literature Review, and why it matters . By Lynn Kysh from University of Southern California

Systematic review or meta-analysis?

A  systematic review  answers a defined research question by collecting and summarizing all empirical evidence that fits pre-specified eligibility criteria.

A  meta-analysis  is the use of statistical methods to summarize the results of these studies.

Systematic reviews, just like other research articles, can be of varying quality. They are a significant piece of work (the Centre for Reviews and Dissemination at York estimates that a team will take 9-24 months), and to be useful to other researchers and practitioners they should have:

• clearly stated objectives with pre-defined eligibility criteria for studies
• explicit, reproducible methodology
• a systematic search that attempts to identify all studies
• assessment of the validity of the findings of the included studies (e.g. risk of bias)
• systematic presentation, and synthesis, of the characteristics and findings of the included studies

Not all systematic reviews contain meta-analysis. 

Meta-analysis is the use of statistical methods to summarize the results of independent studies. By combining information from all relevant studies, meta-analysis can provide more precise estimates of the effects of health care than those derived from the individual studies included within a review.  More information on meta-analyses can be found in  Cochrane Handbook, Chapter 9 .

A meta-analysis goes beyond critique and integration and conducts secondary statistical analysis on the outcomes of similar studies.  It is a systematic review that uses quantitative methods to synthesize and summarize the results.

An advantage of a meta-analysis is the ability to be completely objective in evaluating research findings.  Not all topics, however, have sufficient research evidence to allow a meta-analysis to be conducted.  In that case, an integrative review is an appropriate strategy. 

Some of the content in this section is from Systematic reviews and meta-analyses: step by step guide created by Kate McAllister.

• << Previous: Getting Started
• Next: Research Design >>
• Last Updated: Jul 15, 2024 10:34 AM
• URL: https://guides.lib.udel.edu/researchmethods

• UConn Library
• Literature Review: The What, Why and How-to Guide
• Introduction

Literature Review: The What, Why and How-to Guide — Introduction

• Getting Started
• How to Pick a Topic
• Strategies to Find Sources
• Evaluating Sources & Lit. Reviews
• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings

What are Literature Reviews?

So, what is a literature review? "A literature review is an account of what has been published on a topic by accredited scholars and researchers. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries." Taylor, D.  The literature review: A few tips on conducting it . University of Toronto Health Sciences Writing Centre.

Goals of Literature Reviews

What are the goals of creating a Literature Review?  A literature could be written to accomplish different aims:

• To develop a theory or evaluate an existing theory
• To summarize the historical or existing state of a research topic
• Identify a problem in a field of research 

Baumeister, R. F., & Leary, M. R. (1997). Writing narrative literature reviews .  Review of General Psychology , 1 (3), 311-320.

What kinds of sources require a Literature Review?

• A research paper assigned in a course
• A thesis or dissertation
• A grant proposal
• An article intended for publication in a journal

All these instances require you to collect what has been written about your research topic so that you can demonstrate how your own research sheds new light on the topic.

Types of Literature Reviews

What kinds of literature reviews are written?

Narrative review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified. The review ends with a conclusion section which summarizes the findings regarding the state of the research of the specific study, the gaps identify and if applicable, explains how the author's research will address gaps identify in the review and expand the knowledge on the topic reviewed.

• Example : Predictors and Outcomes of U.S. Quality Maternity Leave: A Review and Conceptual Framework:  10.1177/08948453211037398  

Systematic review : "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139). Nelson, L. K. (2013). Research in Communication Sciences and Disorders . Plural Publishing.

• Example : The effect of leave policies on increasing fertility: a systematic review:  10.1057/s41599-022-01270-w

Meta-analysis : "Meta-analysis is a method of reviewing research findings in a quantitative fashion by transforming the data from individual studies into what is called an effect size and then pooling and analyzing this information. The basic goal in meta-analysis is to explain why different outcomes have occurred in different studies." (p. 197). Roberts, M. C., & Ilardi, S. S. (2003). Handbook of Research Methods in Clinical Psychology . Blackwell Publishing.

• Example : Employment Instability and Fertility in Europe: A Meta-Analysis:  10.1215/00703370-9164737

Meta-synthesis : "Qualitative meta-synthesis is a type of qualitative study that uses as data the findings from other qualitative studies linked by the same or related topic." (p.312). Zimmer, L. (2006). Qualitative meta-synthesis: A question of dialoguing with texts .  Journal of Advanced Nursing , 53 (3), 311-318.

• Example : Women’s perspectives on career successes and barriers: A qualitative meta-synthesis:  10.1177/05390184221113735

Literature Reviews in the Health Sciences

• UConn Health subject guide on systematic reviews Explanation of the different review types used in health sciences literature as well as tools to help you find the right review type
• << Previous: Getting Started
• Next: How to Pick a Topic >>
• Last Updated: Sep 21, 2022 2:16 PM
• URL: https://guides.lib.uconn.edu/literaturereview

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Jun 20, 2024 9:08 AM
• URL: https://guides.library.yale.edu/YSNDoctoral

Why is it important to do a literature review in research?

The importance of scientific communication in the healthcare industry

The Importance and Role of Biostatistics in Clinical Research

 “A substantive, thorough, sophisticated literature review is a precondition for doing substantive, thorough, sophisticated research”. Boote and Baile 2005

Authors of manuscripts treat writing a literature review as a routine work or a mere formality. But a seasoned one knows the purpose and importance of a well-written literature review.  Since it is one of the basic needs for researches at any level, they have to be done vigilantly. Only then the reader will know that the basics of research have not been neglected.

The aim of any literature review is to summarize and synthesize the arguments and ideas of existing knowledge in a particular field without adding any new contributions.   Being built on existing knowledge they help the researcher to even turn the wheels of the topic of research.  It is possible only with profound knowledge of what is wrong in the existing findings in detail to overpower them.  For other researches, the literature review gives the direction to be headed for its success. 

The common perception of literature review and reality:

As per the common belief, literature reviews are only a summary of the sources related to the research. And many authors of scientific manuscripts believe that they are only surveys of what are the researches are done on the chosen topic.  But on the contrary, it uses published information from pertinent and relevant sources like

• Scholarly books
• Scientific papers
• Latest studies in the field
• Established school of thoughts
• Relevant articles from renowned scientific journals

and many more for a field of study or theory or a particular problem to do the following:

• Summarize into a brief account of all information
• Synthesize the information by restructuring and reorganizing
• Critical evaluation of a concept or a school of thought or ideas
• Familiarize the authors to the extent of knowledge in the particular field
• Encapsulate
• Compare & contrast

By doing the above on the relevant information, it provides the reader of the scientific manuscript with the following for a better understanding of it:

• It establishes the authors’  in-depth understanding and knowledge of their field subject
• It gives the background of the research
• Portrays the scientific manuscript plan of examining the research result
• Illuminates on how the knowledge has changed within the field
• Highlights what has already been done in a particular field
• Information of the generally accepted facts, emerging and current state of the topic of research
• Identifies the research gap that is still unexplored or under-researched fields
• Demonstrates how the research fits within a larger field of study
• Provides an overview of the sources explored during the research of a particular topic

Importance of literature review in research:

The importance of literature review in scientific manuscripts can be condensed into an analytical feature to enable the multifold reach of its significance.  It adds value to the legitimacy of the research in many ways:

• Provides the interpretation of existing literature in light of updated developments in the field to help in establishing the consistency in knowledge and relevancy of existing materials
• It helps in calculating the impact of the latest information in the field by mapping their progress of knowledge.
• It brings out the dialects of contradictions between various thoughts within the field to establish facts
• The research gaps scrutinized initially are further explored to establish the latest facts of theories to add value to the field
• Indicates the current research place in the schema of a particular field
• Provides information for relevancy and coherency to check the research
• Apart from elucidating the continuance of knowledge, it also points out areas that require further investigation and thus aid as a starting point of any future research
• Justifies the research and sets up the research question
• Sets up a theoretical framework comprising the concepts and theories of the research upon which its success can be judged
• Helps to adopt a more appropriate methodology for the research by examining the strengths and weaknesses of existing research in the same field
• Increases the significance of the results by comparing it with the existing literature
• Provides a point of reference by writing the findings in the scientific manuscript
• Helps to get the due credit from the audience for having done the fact-finding and fact-checking mission in the scientific manuscripts
• The more the reference of relevant sources of it could increase more of its trustworthiness with the readers
• Helps to prevent plagiarism by tailoring and uniquely tweaking the scientific manuscript not to repeat other’s original idea
• By preventing plagiarism , it saves the scientific manuscript from rejection and thus also saves a lot of time and money
• Helps to evaluate, condense and synthesize gist in the author’s own words to sharpen the research focus
• Helps to compare and contrast to  show the originality and uniqueness of the research than that of the existing other researches
• Rationalizes the need for conducting the particular research in a specified field
• Helps to collect data accurately for allowing any new methodology of research than the existing ones
• Enables the readers of the manuscript to answer the following questions of its readers for its better chances for publication
• What do the researchers know?
• What do they not know?
• Is the scientific manuscript reliable and trustworthy?
• What are the knowledge gaps of the researcher?

22. It helps the readers to identify the following for further reading of the scientific manuscript:

• What has been already established, discredited and accepted in the particular field of research
• Areas of controversy and conflicts among different schools of thought
• Unsolved problems and issues in the connected field of research
• The emerging trends and approaches
• How the research extends, builds upon and leaves behind from the previous research

A profound literature review with many relevant sources of reference will enhance the chances of the scientific manuscript publication in renowned and reputed scientific journals .

References:

http://www.math.montana.edu/jobo/phdprep/phd6.pdf

journal Publishing services  |  Scientific Editing Services  |  Medical Writing Services  |  scientific research writing service  |  Scientific communication services

Related Topics:

Meta Analysis

Scientific Research Paper Writing

Medical Research Paper Writing

Scientific Communication in healthcare

pubrica academy

Related posts.

Importance Of Proofreading For Scientific Writing Methods and Significance

Statistical analyses of case-control studies

Selecting material (e.g. excipient, active pharmaceutical ingredient, packaging material) for drug development

Comments are closed.

Libraries & Cultural Resources

Research guides, research in education.

• Where to Start

Education Literature Review Guide

General literature review guides.

• Citation Resources
• Contact Us!

• << Previous: Where to Start
• Next: Citation Resources >>
• Last Updated: Sep 3, 2024 5:13 PM
• URL: https://libguides.ucalgary.ca/edresearch

Libraries & Cultural Resources

• 403.220.8895

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• Current issue
• Write for Us
• BMJ Journals

You are here

• Volume 19, Issue 1
• Reviewing the literature
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• Joanna Smith 1 ,
• Helen Noble 2
• 1 School of Healthcare, University of Leeds , Leeds , UK
• 2 School of Nursing and Midwifery, Queens's University Belfast , Belfast , UK
• Correspondence to Dr Joanna Smith , School of Healthcare, University of Leeds, Leeds LS2 9JT, UK; j.e.smith1{at}leeds.ac.uk

https://doi.org/10.1136/eb-2015-102252

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Implementing evidence into practice requires nurses to identify, critically appraise and synthesise research. This may require a comprehensive literature review: this article aims to outline the approaches and stages required and provides a working example of a published review.

Are there different approaches to undertaking a literature review?

What stages are required to undertake a literature review.

The rationale for the review should be established; consider why the review is important and relevant to patient care/safety or service delivery. For example, Noble et al 's 4 review sought to understand and make recommendations for practice and research in relation to dialysis refusal and withdrawal in patients with end-stage renal disease, an area of care previously poorly described. If appropriate, highlight relevant policies and theoretical perspectives that might guide the review. Once the key issues related to the topic, including the challenges encountered in clinical practice, have been identified formulate a clear question, and/or develop an aim and specific objectives. The type of review undertaken is influenced by the purpose of the review and resources available. However, the stages or methods used to undertake a review are similar across approaches and include:

Formulating clear inclusion and exclusion criteria, for example, patient groups, ages, conditions/treatments, sources of evidence/research designs;

Justifying data bases and years searched, and whether strategies including hand searching of journals, conference proceedings and research not indexed in data bases (grey literature) will be undertaken;

Developing search terms, the PICU (P: patient, problem or population; I: intervention; C: comparison; O: outcome) framework is a useful guide when developing search terms;

Developing search skills (eg, understanding Boolean Operators, in particular the use of AND/OR) and knowledge of how data bases index topics (eg, MeSH headings). Working with a librarian experienced in undertaking health searches is invaluable when developing a search.

Once studies are selected, the quality of the research/evidence requires evaluation. Using a quality appraisal tool, such as the Critical Appraisal Skills Programme (CASP) tools, 5 results in a structured approach to assessing the rigour of studies being reviewed. 3 Approaches to data synthesis for quantitative studies may include a meta-analysis (statistical analysis of data from multiple studies of similar designs that have addressed the same question), or findings can be reported descriptively. 6 Methods applicable for synthesising qualitative studies include meta-ethnography (themes and concepts from different studies are explored and brought together using approaches similar to qualitative data analysis methods), narrative summary, thematic analysis and content analysis. 7 Table 1 outlines the stages undertaken for a published review that summarised research about parents’ experiences of living with a child with a long-term condition. 8

• View inline

An example of rapid evidence assessment review

In summary, the type of literature review depends on the review purpose. For the novice reviewer undertaking a review can be a daunting and complex process; by following the stages outlined and being systematic a robust review is achievable. The importance of literature reviews should not be underestimated—they help summarise and make sense of an increasingly vast body of research promoting best evidence-based practice.

• ↵ Centre for Reviews and Dissemination . Guidance for undertaking reviews in health care . 3rd edn . York : CRD, York University , 2009 .
• ↵ Canadian Best Practices Portal. http://cbpp-pcpe.phac-aspc.gc.ca/interventions/selected-systematic-review-sites / ( accessed 7.8.2015 ).
• Bridges J , et al
• ↵ Critical Appraisal Skills Programme (CASP). http://www.casp-uk.net / ( accessed 7.8.2015 ).
• Dixon-Woods M ,
• Shaw R , et al
• Agarwal S ,
• Jones D , et al
• Cheater F ,

Twitter Follow Joanna Smith at @josmith175

Competing interests None declared.

Read the full text or download the PDF:

Biology 231 - Research Methods

• Verifying If An Article Is Peer-Reviewed
• Should I Trust Internet Sources?
• Suggested Databases for Finding Sources
• Critical Analysis
• Creating an Annotated Bibliography
• Creating a Literature Review
• Resources for Writing Position Papers
• Off-Campus Access

Ask A Librarian

chat Text: 1-308-210-3865 email Librarians by Subject Make an Appointment

Structure of an Evidence Matrix

The purpose of a literature review is to demonstrate your familiarity with existing research and how your proposed research fits within it.  It should consist of at least 4-5 peer-reviewed articles and provide an integration of ideas, concepts, theories and findings.

Constructing an Evidence Matrix will assist in the dissection of the articles being reviewed and provide a system that easily organizes common themes for later discussion.

Create a spreadsheet with the following fields: 

Themes are items of interest to you found within the article and are points of comparison between articles.

• Has the subject or style of research evolved over time?
• Are there any approaches or variables that have been consistently examined?  Do the articles ask the same questions?  Can you identify any gaps in research or inquiry?
• Are you aware of any specific common theoretical models used?

NOTE:  It is crucial that when summarizing any content that you appropriate paraphrase the text entered into your matrix. If not, use quotation marks for any words that are not your own and provide page numbers for your quotes.  This will help avoid plagiarism.

Writing a Literature Review

In a full-length paper or thesis the literature review tends to be 4-5 paragraphs long and explains how your original research topic fits into the existing body of scholarship on the topic. 

A basic format is as follows:

Paragraph #1.  Introduction of the research topic.  "The purpose of the study is to...".

Paragraph #2.  Describe how others have studied the subjects (Methods column).  Include any sampling techniques, strategies and limitations to research processes.

Paragraph #3.  Discuss a common theme from two or more articles (identified in evidence matrix).

Paragraph #4.   Discuss a common theme from two or more articles (identified in evidence matrix).

Paragraph #5.  Discuss the existing research identified in previous paragraphs, then your research topic and why it is important.  Give strong examples why your research is relevant and necessary to build a stronger understanding of the subject.

TIPS: 

• Do not summarize each article read in a separate paragraph.  The purpose here is to critically analyze them as a whole and reflect.
• When referencing an article in the literature, do not refer to it by its title.  Instead relate the article to the author and date.

Source: https://libguides.sonoma.edu/c.php?g=202672&p=1354272 (Downloaded 17AUG2022)

• << Previous: Creating an Annotated Bibliography
• Next: Resources for Writing Position Papers >>
• Last Updated: Sep 4, 2024 6:55 AM
• URL: https://guides.library.unk.edu/BIO231/ResearchMethods
• University of Nebraska Kearney
• Library Policies
• Accessibility
• Research Guides
• Citation Guides
• A to Z Databases
• Open Nebraska
• Interlibrary Loan

2508 11th Avenue, Kearney, NE 68849-2240

Circulation Desk:  308-865-8599 Main Office:  308-865-8535

  Ask A Librarian

Importance and Issues of Literature Review in Research

• November 2020
• Conference: 'Tackle a literature review' under the series 'Publish or Perish‘ organised by Mysore University Library and Mysore Librarians and Information Scientists Association (MyLISA) on November 30, 2013

• Indian Space Research Organization

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations
• TABINDA RANI, HAMIDA BIBI, ZARMINA BALOCH

• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

• Open access
• Published: 06 September 2024

A scoping review of health literacy in rare disorders: key issues and research directions

• Una Stenberg   ORCID: orcid.org/0000-0002-6007-8630 1 , 2 ,
• Lydia Westfal 1 ,
• Andreas Dybesland Rosenberger 3 ,
• Kristin Ørstavik 4 ,
• Maria Flink 5 ,
• Heidi Holmen 6 ,
• Silje Systad 7 ,
• Karl Fredrik Westermann 2 &
• Gry Velvin 8  

Orphanet Journal of Rare Diseases volume  19 , Article number:  328 ( 2024 ) Cite this article

Metrics details

The ability to find, understand, appraise and utilise health information is crucial among individuals living with rare disorders. The aim of this study was to give a comprehensive overview of the literature on health literacy in adult persons with rare disorders.

We applied a scoping review methodology and performed a systematic search in 2021 in bibliographic databases. Searches were conducted in Medline (Ovid), Embase (Ovid), PsycInfo (Ovid), CINAHL (ebsco), and ERIC (Ovid). References were sorted and evaluated for inclusion using EndNote and Covidence. This review was guided by the question “What are the characteristics of research on health literacy in rare disorders?”

The database searches yielded 75 eligible reports. A total of 6223 individuals with rare disorders were represented alongside 1707 caregivers. The reports in this review have included study participants representing a total of 80 different rare disorders with unique ORPHA and ICD-10 codes. The results revealed that persons with rare disorders often exhibit gaps in health literacy through a lack of knowledge and access to information related to self-management, their own diagnosis and health, as well as daily coping and social rights. In addition, the importance of aid and information from healthcare personnel and the significance of getting social support from others in the same situation were accentuated.

This review emphasizes the importance of reinforcing health literacy among persons with rare disorders through peer support and education. This is the first review to give a comprehensive and state-of-the-art overview of literature investigating health literacy among persons with rare disorders and offers a basis for further research.

Introduction

In Europe, a disorder is considered rare when it affects less than 1:2000 individuals [ 1 ]. According to current calculations, more than 7000 different rare disorders have been identified. However, it is plausible that the actual number may be as high as 10,000 [ 2 ]. Although each rare disorder affects a limited quantity of individuals, it is estimated that the combined prevalence of all rare diseases is 3,5–5,9% [ 3 ]. Accordingly, up to 36 million people residing in the European Union are living with a rare disease [ 4 ]. Out of the total rare disorders, 72% have a genetic aetiology, and 70% have childhood onset [ 3 ]. Whilst there is a large clinical diversity between the rare disorders, they tend to have some aspects in common; they are known for being chronic, complicated, mostly degenerating, and often disabling [ 5 ].

Persons with rare disorders face some unique challenges in accessing information on their diagnosis, which may lead to issues in making beneficial health choices regarding treatment and care [ 6 ]. A key issue with rare disorders is the lack of research in the field [ 7 , 8 ]. Insufficient evidence and knowledge on rare diseases in general pose challenges both for professionals and people with these diseases [ 8 ]. Due to healthcare professionals’ limited understanding of their rare disorder in general, as well as a lack of information provided, persons with rare disorders often need to search for health-related information themselves [ 6 ]. A systematic review published in 2017 aimed to provide an overview of adults` shared experience of living with a rare disorder, found that in 12 out of 21 reports, persons with rare disorders reported progressively becoming “experts” on their own diagnosis [ 9 ]. In some cases, those living with rare disorders possess more information about the condition than the healthcare professionals they encounter [ 10 ].

Healthcare systems are increasingly challenging to navigate [ 11 ]. Simultaneously, the healthcare services share prospective aims of prioritising digitization, enabling more home-based care, promoting shared decision-making, and ensuring equitable access to services [ 5 , 12 , 13 ]. Managing one’s health while dealing with a rare disorder and the responsibility of seeking information can be especially demanding due to the challenging standards set by the healthcare system [ 6 ].

Increased participation and responsibility for one’s own health impose a demand on the individual to have adequate health literacy. Health literacy pertains to individuals’ ability to manage the complex health requirements of today’s society and make informed decisions regarding health [ 14 ]. This includes understanding the factors that affect one’s health, addressing health challenges, and making appropriate health choices. There is a lack of consensus on the definition of health literacy, and multiple interpretations have been made [ 14 ]. A review by Sørensen et al. [ 15 ] identified as many as 17 different definitions of health literacy and created a working definition of health literacy by considering the contents of each interpretation. The inclusive definition according to Sørensen et al. is stated as follows:

“Health literacy is linked to literacy and entails people’s knowledge , motivation and competencies to access , understand , appraise , and apply health information in order to make judgments and take decisions in everyday life concerning healthcare , disease prevention and health promotion to maintain or improve quality of life during the life course.” (ref p. 3).

Along with the comprehensive definition, Sørensen et al. developed an integrated model of health literacy [ 15 ]. The model has been widely used to understand the complex interaction between individual skills and abilities related to health literacy, social and environmental factors, and health outcomes. The core elements of the model are four cognitive competencies; to access, understand, appraise, and apply health-related information. These four competencies allow a person to manoeuvre three identified domains on the health spectrum: healthcare, disease prevention, and health promotion. The model suggests that an individual’s ability to access and use health information is determined by their own skills, motivation, and knowledge as well as the social and environmental context they reside within. These conditions, accordingly, affect individuals’ ability to address their health and ultimately impact their health outcomes.

Sorensen’s model emphasises that components such as empowerment, health outcomes, and health behaviour are interlaced and connected to an individual’s health literacy. Enhancing the level of health literacy allows individuals to become more empowered and take charge of their health, participate in health-promoting behaviours, and gradually attain improved health outcomes [ 15 ]. Thus, participation and empowerment can give persons with rare disorders enhanced control over their own health and treatment, and increased involvement in decision-making processes that concern their health. This may lead to better health outcomes and elevated health-related quality of life, which remain crucial as persons with rare disorders report lower quality of life compared to those with more common chronic conditions [ 16 ]. They can feel stigmatised and marginalised in the healthcare system, and it can be challenging to find psychosocial support. Examining how to increase health literacy and empowerment for persons with rare disorders can therefore be an important and relevant direction for further research. Health literacy of individuals with rare disorders is an emerging field of research, and the literature is based on a wide range of study methodologies [ 7 , 8 ]. Hence, this scoping review aims to give a comprehensive overview of empirical reports (from primary research studies) investigating health literacy among persons with rare disorders as reported in the international literature, by identifying characteristics of definitions, study populations, methods and interventions.

Study design and research questions

The scoping review process described by Arksey and O`Malley [ 17 ] aims to: “(…) map rapidly the key concepts underpinning a research area and the main sources and types of evidence available and can be undertaken as a stand-alone project in their own right , especially where an area is complex or has not been reviewed comprehensively before. ” A scoping review methodology is also suitable for examining the extent, range, variety, and characteristics of evidence on a topic, but also to identify research gaps. This scoping review was conducted according to the five-stage framework by Arksey and O`Malley [ 17 ], enhanced by Levac [ 18 ] and Daudt [ 19 ] and reported according to the PRISMA Extension for Scoping Reviews [ 20 ] (shown in Additional file 1 ). A protocol for this review is available on request.

The aim of this review was to identify the characteristics of research on health literacy in rare disorders. The specific research questions were:

What are the characteristics of study populations?

When and where have reports on health literacy been carried out?

What are the characteristics of research questions used to investigate health literacy?

What are the characteristics of methods used to investigate health literacy?

What are the characteristics of assessment tools used to measure health literacy?

What are the characteristics of interventions that have been described in the reports?

How is health literacy defined or described in the reports?

How is access to health information and support for individuals with rare disorders described the reports?

Overarching participatory approach

The study group in this scoping review included one co-researcher, one with experiential knowledge trained in research methods, several experienced healthcare professionals in the field of rare disorders, working in clinical practice (specialized health care), and experienced researchers in health literacy and scoping review methodology. All members have been involved in all stages of the review process.

Eligibility criteria

This scoping review included primary research reports that investigated health literacy in adults with rare disorders. Reports were included if they had investigated the individual`s capacities, skills and motivation to make judgements and decisions in everyday life concerning healthcare, disease prevention and health promotion in persons with a rare disorder. While being 18 years of age or older was set as a search criterion, reports that included both adults and persons below 18 were not excluded. Empirical reports in English and Scandinavian languages published in peer-reviewed journals were included. All study designs were included. Dissertations, reports published in abstract form only, editorials, commentaries and duplicates were excluded.

Systematic searches

In the first stage, research questions were developed by the study group in a highly iterative process. We agreed to apply a broad variety of synonyms, conducting many and extensive pilot searches and simultaneously enhancing the search strategy, and clarify the criteria for inclusion and exclusion of reports. A senior academic librarian, in close collaboration with the first author, developed a systematic literature search using MeSH-terms and free search terms combining a comprehensive set of synonyms and terms for health literacy and rare disorders. Both the librarian and the researchers in the study group had experience with previous literature searches in the field of rare disorders. The literature searches complied with the PICO principles and applied a combination of “OR” within groups and “AND” between groups. Searches were conducted in Medline (Ovid), Embase (Ovid), PsycInfo (Ovid), CINAHL (ebsco), and ERIC (Ovid) for publications between 2010 and 2021. No other sources for literature were searched for this review. The complete search strategy is displayed in Additional file 2 .

Selection of publications

All titles and abstracts were reviewed by the first author (US) and one of the co-authors independently using the systematic review software Covidence (Veritas Health Innovation). Disagreements and conflicts were resolved through discussion with a third review author.

Data extraction

All data from the included reports were extracted according to study characteristics, participant characteristics included ORPHA and ICD-codes, description of interventions, methods, assessment tools, definitions and understanding of health literacy was collected using data extraction forms and reported separately for each study in evidence summaries (Supplementary Material 4 – 9 : Tables 2–6). A full reference list of included reports is presented in Additional file 3 . Extracted data is presented in a descriptive manner using text, tables and figures. All members of the study group participated in the data extraction. We did not attempt to contact the authors in this review process.

The search of the online databases resulted in 5999 reports when duplicates were removed. From these, 5794 were excluded because they did not fulfil the inclusion criteria. A total of 177 reports were downloaded in full text and read by two authors. Of these, 102 reports were excluded, leaving 75 to undergo analysis in this review (Fig. 1). All the included reports were in English language.

Prisma flow diagram

Characteristics of study populations

A total of 6223 persons with a rare disorder and 1707 caregivers were represented in the 75 included reports. About 70% of the reports were based on data from samples with less than 100 participants. About 15% of the reports were based on samples with more than 200 participants. Of the included participants in the reports, about 60% were female. Of the reports that reported the mean age of the participants, approximately 75% of the participants were between 30 and 50 years of age. About 15% of the reports had participants with a mean age over 55 years, and eleven reports had participants with a mean age under 25 years.

The reports in this review have included study participants representing a total of 80 different rare disorders with unique ORPHA and ICD-10 codes. A detailed description of diagnoses is given in Table 1 (Additional file 4 ). Five of the reports included participants across rare disorders but did not specify what type of disorders. Most of the rare disorders had been investigated in one or two reports, but a few disorders were investigated in several reports: different types of Hemophilia were investigated in 24/75 reports, Cystic Fibrosis in 14/75 reports, Huntington’s disease in 7/75 reports, Scleroderma in 4/75 reports and Myotonic dystrophy type 1, Neurofibromatosis type 1 and Spina bifida in 3/75 reports.

Where and when have reports on health literacy been carried out?

Of the 75 included reports, 21 were conducted in the USA, 11 in Canada and eight in the UK (see Table 2 for details in Additional file 9 ). The included reports were published between 2010 and 2021, 54/75 after 2016.

Characteristics of research questions

The research questions most frequently investigated among the included reports were related to assessments of experienced knowledge and different health- and/or psychosocial outcomes (31/75). The second most investigated research questions (27/75) were about persons with rare disorders’ views, experiences and understanding of their own condition, care, health information, management, transition process or peer support (see Table 6 for details in Additional file 8 ). In addition, 16 reports were conducted to evaluate an intervention aimed to improve or strengthen participants` knowledge, health literacy or coping, and therefore included in this review.

Characteristics of methodological design

Of the included reports, 28/75 applied a quantitative cross-sectional design to explore characteristics of patient groups in terms of knowledge and disease-related variables. Among the cross-sectional reports, both digital and paper-based surveys were used, and some gathered data through medical charts or personal interviews. In addition, fourteen reports applied an experimental design investigating either the feasibility or effects of specific interventions, mainly to increase knowledge or health literacy. Among the qualitative designs (28/75), individual interviews were frequently applied, less so focus groups. Most of the qualitative reports aimed to explore experiences and gain insight into the views of persons who are living with a rare disorder, for example, needs of information and support, barriers to care and communication with health care providers. To present the qualitative results, a thematic analysis approach was most frequently applied. A minority of reports (5/75) reported a mixed or multi-method approach, combining interviews and surveys (see Table 3 for more details, Additional file 5 ).

Characteristics of assessment tools

Five of the assessment tools measured health literacy specifically. However, 23 standardized assessment tools aimed to assess important aspects relevant to health literacy, such as self-management skills, coping and medication adherence. Table 4 provides an overview of the standardised assessment tools used to measure health outcomes (Additional file 6 ). Quality of life was the outcome assessed most frequently (10/75) and was most commonly assessed with SF36 (4/75). Seven reports examined anxiety levels, while six estimated depression. Hospital Anxiety and Depression Scale (HAD) was the most commonly utilized tool to assess anxiety and depression (3/75). Correspondingly, 27 study-specific assessment tools sought to achieve outcomes closely related to health literacy, including health information-seeking patterns, medication information sources and knowledge, attitude and behaviour towards their condition. For a more detailed review of study-specific assessment tools, see Table 5 (Additional file 7 ).

Characteristics of interventions

A total of 16/75 of the reports included interventions. Each intervention originated from a distinct study and had diverse characteristics in terms of study design, objectives, intended recipients, implementation settings, and delivery personnel, including healthcare professionals and peers. Additional information regarding this is provided in Table 6 (Additional file 8 ). The interventions encompassed both face-to-face approaches, such as individual sessions [ 21 , 22 , 23 , 24 , 25 , 26 , 27 ] and group-based patient education [ 22 , 26 , 28 , 29 , 30 , 31 , 32 , 33 , 34 ], and written information/online training [ 29 , 31 , 35 , 36 , 37 ]. The interventions took place in a variety of settings, including hospitals, clinics, and online platforms. The common thread between the interventions is that they all share the objective of enhancing patient outcomes and experiences through education, support, and empowerment. For example, they aim to improve knowledge, health literacy, and self-treatment skills, as well as to promote treatment adherence and reduce interruptions in care. In 9/16 interventions, the primary aim was to improve knowledge or understanding of the patient’s particular health condition or treatment. These nine interventions applied various components such as audiovisual materials, individualised training courses, or booklets. Out of those nine interventions, six demonstrated a significant ( p  < 0.05) improvement in knowledge of the targeted health condition or treatment [ 21 , 25 , 28 , 32 , 36 , 37 ].

Out of all interventions, 5/16 aimed predominantly at reducing psychiatric symptoms, such as depression, anxiety, and somatic symptom severity. Several interventions displayed positive effects on mental health, including improvements in emotional health, coping strategies, and quality of life [ 22 , 26 , 28 , 29 , 32 , 33 , 35 ]. Examples of such interventions included group counselling and group mindfulness training. The interventions were evaluated using methods such as self-report questionnaires, physiological measures, and clinical assessments. The outcomes measured included improvements in physical health, mental health, quality of life, and social support.

Description of health literacy

Only 6/75 reports described in the introduction how they defined health literacy [ 21 , 38 , 39 , 40 , 41 , 42 ]. Five of these reports were based on the understanding and definition of health literacy as the cognitive and social skills that determine the motivation and ability of individuals to gain access to, understand and use information in ways that promote and maintain good health (WHO). One of the reports defined health literacy as “the patients’ skills on reading, listening, analysing decisions making and applying these skills to the situation related to health monitoring and coordination for strategy plan in term of health promotion” [ 42 ].

Access to health information and support

Most of the reports included in this review investigated knowledge or understanding of one’s own health and diagnosis, and access to health information. Persons with rare disorders commonly lack information about:

Own diagnosis and health [ 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 ].

Self-management and daily coping [ 6 , 10 , 54 , 59 , 60 , 61 , 62 , 63 ].

Medication, treatment options and research-based recommendations [ 6 , 10 , 28 , 51 , 64 , 65 , 66 , 67 ].

Peer and professional support [ 53 , 54 , 55 ].

Clinical trials and research [ 53 , 54 , 55 ].

Sexual knowledge [ 68 , 69 , 70 , 71 ].

Behaviour and attitude [ 28 , 72 , 73 ].

Social rights [ 28 , 60 ].

Pregnancy and childbirth [ 51 , 60 ].

Ageing [ 71 ].

Navigation and coordination [ 23 ].

The most important sources of health information summarized among the included reports were physicians, the internet, patient organizations and spouse/partner [ 74 , 75 , 76 ]. Transitions in life can be challenging and generate new needs for information and care. Three of the reports investigated the transition process from paediatric to adult services [ 23 , 77 , 78 ]. Persons with rare disorders and their family caregivers call for health information on various aspects of the disease burden including medical research and treatment, coping strategies, management, symptoms and general knowledge about the disease [ 57 , 63 ].

Only a few reports investigated how persons with rare disorders are navigating in healthcare and their experiences of healthcare services. These reports found that many persons with rare disorders feel let down by the system- and lack trust in the standards of health care [ 54 , 79 , 80 , 81 ]. Several reports described the frustration among persons with rare disorders because of a lack of knowledge about diagnosis and medication by healthcare professionals [ 54 , 56 , 59 , 62 , 73 , 81 , 82 ] and concerns about poor communication and information provision [ 83 ].

Some of the reports described the experiences of persons with rare disorders concerning limited access to peer- and professional support, like specialized care, treatment plans and access to peer groups [ 34 , 53 , 61 , 62 , 84 , 85 , 86 ]. Persons with rare disorders missed the engagement in health care to assist in their management of the disease [ 85 ], and one report claimed that hospital visits could be reduced with more information [ 52 ].

Several reports have investigated peer support [ 6 , 22 , 28 , 44 , 50 , 81 , 82 , 87 , 88 , 89 ]. Persons with rare disorders who connected and interacted with fellow individuals with rare disorders reported great improvements in overall health, disease severity, motivation to take care of health, emotional well-being and satisfaction with their primary treating physician [ 66 , 69 ].

This scoping review identified 75 reports presenting data on rare disorders and aspects of health literacy, thereby providing valuable insight into the characteristics of research in the field of health literacy in individuals with rare disorders. A total of 6223 individuals with rare disorders and 1707 caregivers were included, and 80 different rare disorders were represented. Most of the studies were published after 2016, and were conducted in the USA, Canada and UK. The most frequently investigated research questions were related to different health- and psychosocial outcomes, understanding of own condition, health information and support, or concerning evaluation of an intervention. The reports used a variety of research methodologies, including qualitative, quantitative, and mixed methods approaches. Cross-sectional designs were frequently employed to depict patient characteristics, knowledge and health-related variables, and qualitative designs were commonly used to capture the perspectives of persons living with rare disorders. In total 23 standardized assessment tools and 27 study-specific assessment used in the reports. Only five assessment tools measured health literacy specifically. Some of the reports also assessed interventions to improve elements such as knowledge, health literacy and coping strategies. These interventions encompassed both face-to-face approaches, such as individual sessions and group-based patient education.

Only six reports had described how they defined health literacy. Five of these reports were based on the understanding and WHO-definition of health literacy as the cognitive and social skills that determine the motivation and ability of individuals to gain access to, understand and use information in ways that promote and maintain good health (WHO). Concerning access to health information and support, the results revealed that individuals with rare disorders often exhibit gaps in knowledge and access to information related to self-management, their own diagnosis and health, as well as daily coping and social rights. In addition, the importance of aid and information from healthcare personnel and the significance of getting social support from others in the same situation were accentuated.

A recurring issue identified among the reports was that individuals with rare disorders consistently encounter challenges in accessing information on their own health and diagnosis, self-managing and coping [ 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 ]. This observation has been established in previous research and can sometimes be ascribed to a lack of knowledge among healthcare personnel [ 6 , 9 , 10 ]. The understanding and appraisal of health information could pose difficulties since the information available on rare conditions often is complex and contains medical terminology that is challenging to comprehend. This particularly applies to those with cognitive impairments, which pertains to 44% of the rare disease population [ 90 ]. More than 7000 rare disorders are identified, and only 80 of these disorders are represented in this review. More than 50% of the included reports have included study participants with Haemophilia, Cystic Fibrosis and Huntington’s disease, which means that a range of different rare disorders have not been included in health literacy research. A majority of the interventions in this study focused on increasing knowledge and understanding of one’s own health and treatment. Acquiring the skills to apply health knowledge to everyday life efficiently can profoundly impact health outcomes and is especially important when it comes to self-management, such as adherence to medication and treatment [ 15 , 91 , 92 , 93 , 94 , 95 ].

Another possible challenge related to access to information and support is the often-large geographical distances between persons with rare disorders. This may result in difficulties when it comes to meeting or participating in peer-support groups in person [ 96 ]. Peer interactions appear particularly important in this population [ 6 , 53 , 54 , 55 ], and several of the perceived benefits of the interventions in this review were associated with the recognition, acceptance and companionship encountered within peer-support groups [ 16 , 28 , 32 , 35 ]. Interestingly, none of the included reports explored the potential benefits of online peer support, which has been found to be an effective supplement to in-person meetings in people with other disorders.

While there is reason to believe that health literacy has a significant impact on health outcomes, only one of the included reports investigated this possible correlation, finding that individuals who possessed adequate health literacy displayed more favourable health-related outcomes [ 38 ]. In that report, the authors observed that individuals who possessed adequate health literacy displayed more favourable health-related outcomes. None of the included reports explored health literacy across various types of rare disorders. One prominent finding across the reviewed reports is the shortage of accessible health information specifically targeted towards individuals with rare conditions. There is a need to investigate if there are structural or social barriers that limit access to information and support for the population. Furthermore, it would be valuable to examine the underlying factors that impact health literacy in persons with rare disorders, including the association between health literacy and socio-demographic variables, health status, self-efficacy and health-related quality of life. Another potential research topic could be to evaluate the success of interventions aiming at improving health literacy in persons with rare disorders and their caregivers.

To the best of our knowledge, only four previous reports have explicitly aimed to examine the levels of health literacy in persons with rare disorders [ 38 , 39 , 40 , 42 ]. Furthermore, the data does not provide enough information to say anything about relatives’ health literacy. Enhancing health literacy is known to be an enabler for improved empowerment and participation, which is associated with positive health outcomes [ 13 , 94 ]. Empowerment is especially important in the field of rare disorders, due to the unique challenges of low prevalence, limited knowledge and expertise, and compromised quality of life [ 94 ].

To achieve a better understanding of health literacy in rare disorders, we could benefit from the incorporation of different perspectives, including those of persons with rare disorders, their family members and healthcare providers. We need future research on how different dimensions of health literacy, and interventions aiming to strengthen health literacy, influence health outcomes according to health care, disease prevention and health promotion. We need to achieve a deeper understanding of how the personal determinants of health literacy, such as individual skills and motivation, interact with situational determinants, such as social and environmental factors, to shape health outcomes. To properly address the executive challenges faced by persons with rare disorders we need a greater understanding of health literacy in rare disorders [ 28 ]. The integrated model of health literacy [ 15 ] can serve as a tool to point us in the right direction when designing future research projects.

The strength of our work lies in providing a comprehensive overview of the reported findings from research on health literacy in rare disorders. We conducted an up-to-date systematic search in five databases without restrictions. Despite using an array of synonyms in database searches to maximise the identification of relevant reports, the search terms used are not exhaustive. Hence, some reports may not have been detected. To reduce the risk of selection bias, two authors independently assessed the abstracts and reports in full text according to the a priori eligibility criteria. Further, in line with the scoping review framework, we have not evaluated the methodological quality or risk of bias among the included reports. This may be seen as a limitation; however, the purpose of scoping reviews is to give an overview of the available research literature, characterise a research area and pinpoint gaps in knowledge that should be addressed in future systematic reviews.

This review has important implications for practice. Healthcare does not offer curative treatment options for most rare disorders, and several reports suggest the development of consensus recommendations for care. To optimise health and secure continuity of care several reports included in this review recommend formalisation of the transition process through the courses of illness and life. Moreover, the results reveal that some of the key challenges for persons with rare disorders are related to important aspects of health literacy, such as accessing, understanding, and applying health information. Our findings indicate a need for strengthened health literacy in the rare disease population, that could be accomplished by developing health communication strategies tailored to the needs and preferences of persons with rare conditions. Healthcare personnel can play a significant role in enhancing health literacy, which is an additional implication for practice. Health care personnel can achieve this by offering clear and understandable health-related information and encouraging an active dialogue between patients and professionals. Another way for healthcare personnel to assist persons with rare disorders is by offering them the support needed to accept, cope, and effectively manage their condition [ 97 ].

This scoping review consists of 75 reports presenting data on rare disorders and aspects of health literacy, thereby providing valuable insight into the characteristics of research in the field of health literacy in individuals with rare disorders. In total, 6223 individuals with rare disorders and 1707 caregivers were included, and 80 different rare disorders were represented. Most of the studies were published after 2016, and were conducted in the USA, Canada and UK.

The findings of this scoping review demonstrate that persons with rare disorders experience considerable gaps in knowledge and information, particularly in relation to their own diagnosis and health, treatment options, self-management and coping strategies. Moreover, the lack of diagnosis-specific knowledge and limited information provided by healthcare professionals are identified as a common concern among persons with rare disorders. Access to, and understanding, health information is key aspects of health literacy. Therefore, our results imply a need for increased awareness regarding the state of health literacy among individuals with rare disorders. The points of view expressed in this review offer valuable perspectives that can help health personnel in outlining the communicative strategy when caring for individuals with rare disorders.

This review provides a solid understanding block for future research into the emerging field of health literacy in rare disorders, by examining the challenges that persons with rare conditions encounter. Moreover, the findings enable us to develop a better understanding of the care and support persons with a rare disorder and their family members require.

These results pave the way for future research that looks to improve the healthcare experience of those with rare disorders and their caretakers and shed light on the importance of empowering the rare disease population through peer support, participation, education and increased health literacy. Future reports in this field are necessary to develop strategies and interventions that improve health literacy and enhance health outcomes and the quality of life for individuals with rare disorders.

Data availability

All data generated or analysed during this review are included in this published article (and its additional files).

EURORDIS. What is a rare disease? [Internet]. EURORDIS. 2022 [cited 2023 Jun 2]. https://www.eurordis.org/information-support/what-is-a-rare-disease/

Haendel M, Vasilevsky N, Unni D, Bologa C, Harris N, Rehm H, et al. How many rare diseases are there? Nat Rev Drug Discov. 2020;19:77–8.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Nguengang Wakap S, Lambert DM, Olry A, Rodwell C, Gueydan C, Lanneau V, et al. Estimating cumulative point prevalence of rare diseases: analysis of the Orphanet database. Eur J Hum Genet. 2020;28:165–73.

Article   PubMed   Google Scholar  

Rare diseases [Internet]. European Commission. [cited 2023 Jun 6]. https://health.ec.europa.eu/non-communicable-diseases/expert-group-public-health/rare-diseases_en

The Ministry of Health and Care Services. Nasjonal strategi for sjeldne diagnoser [Internet]. 2021 Aug. https://www.regjeringen.no/no/dokumenter/nasjonal-strategi-for-sjeldne-diagnoser/id2867121/

Stanarević Katavić S. Health information behaviour of rare disease patients: seeking, finding and sharing health information. Health Info Libr J. 2019;36:341–56.

Zhu Q, Nguyễn Ð-T, Sheils T, Alyea G, Sid E, Xu Y, et al. Scientific evidence based rare disease research discovery with research funding data in knowledge graph. Orphanet J Rare Dis. 2021;16:483.

Article   PubMed   PubMed Central   Google Scholar  

Stoller JK. The challenge of Rare diseases. Chest. 2018;153:1309–14.

von der Lippe C, Diesen PS, Feragen KB. Living with a rare disorder: a systematic review of the qualitative literature. Mol Genet Genomic Med. 2017;5:758–73.

Kesselheim AS, McGraw S, Thompson L, O’Keefe K, Gagne JJ. Development and use of new therapeutics for rare diseases: views from patients, caregivers, and advocates. Patient. 2015;8:75–84.

Kickbusch I, Pelikan J, Apfel F, Tsouros AD. Health literacy. The solid facts [Internet]. WHO Regional Office for Europe, Copenhagen; 2013. https://apps.who.int/iris/handle/10665/326432

Sør-Øst H. Regional utviklingsplan 2040 [Internet]. 2022 [cited 2023 May 25]. https://helse-sorost.no/regional-utviklingsplan-2040

Omsorgsdepartementet H-. og. Strategi for å øke helsekompetansen i befolkningen 2019–2023. 2019 [cited 2023 May 25]; https://www.regjeringen.no/no/dokumenter/strategi-for-a-oke-helsekompetansen-i-befolkningen-2019-2023/id2644707/

Kickbusch I, Maag D. Health Literacy [Internet]. International encyclopedia of public health. Elsevier; 2008. pp. 204–11. https://doi.org/10.1016/b978-012373960-5.00584-0

Sørensen K, Van den Broucke S, Fullam J, Doyle G, Pelikan J, Slonska Z et al. Health literacy and public health: a systematic review and integration of definitions and models. BMC Public Health [Internet]. 2012;12. https://doi.org/10.1186/1471-2458-12-80

Bogart KR, Irvin VL. Health-related quality of life among adults with diverse rare disorders. Orphanet J Rare Dis. 2017;12:177.

Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005;8:19–32.

Article   Google Scholar  

Levac D, Colquhoun H, O’Brien KK. Scoping studies: advancing the methodology. Implement Sci. 2010;5:69.

Daudt HML, van Mossel C, Scott SJ. Enhancing the scoping study methodology: a large, inter-professional team’s experience with Arksey and O’Malley’s framework. BMC Med Res Methodol. 2013;13:48.

Tricco AC, Lillie E, Zarin W, O’Brien KK, Colquhoun H, Levac D, et al. PRISMA Extension for scoping reviews (PRISMA-ScR): Checklist and Explanation. Ann Intern Med. 2018;169:467–73.

Bhatt N, Boggio L, Simpson ML. Using an educational intervention to assess and improve disease-specific knowledge and health literacy and numeracy in adolescents and young adults with haemophilia A and B. Haemophilia. 2021;27:229–36.

Article   CAS   PubMed   Google Scholar  

Bogart KR, Frandrup E, Locke T, Thompson H, Weber N, Yates J et al. Rare place where I feel normal: Perceptions of a social support conference among parents of and people with Moebius syndrome. Res Dev Disabil. 2017;64:143–51.

Chaudhry SR, Keaton M, Nasr SZ. Evaluation of a cystic fibrosis transition program from pediatric to adult care. Pediatr Pulmonol. 2013;48:658–65.

Dicianno BE, Lovelace J, Peele P, Fassinger C, Houck P, Bursic A, et al. Effectiveness of a Wellness Program for individuals with Spina Bifida and spinal cord Injury within an Integrated Delivery System. Arch Phys Med Rehabil. 2016;97:1969–78.

le Doré S, Grinda N, Ferré E, Roussel-Robert V, Frotscher B, Chamouni P, et al. The hemarthrosis-simulating knee model: a useful tool for individualized education in patients with hemophilia (GEFACET study). J Blood Med. 2021;12:133–8.

Ringqvist K, Borg K, Möller MC. Tolerability and psychological effects of a multimodal day-care rehabilitation program for persons with Huntington’s disease. J Rehabil Med. 2021;53:jrm00143.

van Balen EC, Krawczyk M, Gue D, Jackson S, Gouw SC, van der Bom JG, et al. Patient-centred care in haemophilia: patient perspectives on visualization and participation in decision-making. Haemophilia. 2019;25:938–45.

Chaleat-Valayer E, Amélie Z, Marie-Hélène B, Perretant I, Sandrine T. Therapeutic education program for patients with hypermobile Ehlers-Danlos syndrome: feasibility and satisfaction of the participants. Education thérapeutique Du patient -. Therapeutic Patient Educ. 2019;11:10202.

Hoefnagels JW, Fischer K, Bos RAT, Driessens MHE, Meijer SLA, Schutgens REG, et al. A feasibility study on two tailored interventions to improve adherence in adults with haemophilia. Pilot Feasibility Stud. 2020;6:189.

O’Mahar K, Holmbeck GN, Jandasek B, Zukerman J. A camp-based intervention targeting independence among individuals with spina bifida. J Pediatr Psychol. 2010;35:848–56.

Raphaelis S, Mayer H, Ott S, Hornung R, Senn B. Effects of Written Information and Counseling on illness-related uncertainty in Women with Vulvar Neoplasia. Oncol Nurs Forum. 2018;45:748–60.

PubMed   Google Scholar  

Rovira-Moreno E, Abuli A, Codina-Sola M, Valenzuela I, Serra-Juhe C, Cuscó I, et al. Beyond the disease itself: a cross-cutting educational initiative for patients and families with rare diseases. J Genet Couns. 2021;30:693–700.

Stubberud J, Langenbahn D, Levine B, Stanghelle J, Schanke A-K. Emotional health and coping in spina bifida after goal management training: a randomized controlled trial. Rehabil Psychol. 2015;60:1–16.

Delisle VC, Gumuchian ST, Pelaez S, Malcarne VL, El-Baalbaki G, Körner A, et al. Reasons for non-participation in scleroderma support groups. Clin Exp Rheumatol. 2016;34(Suppl 100):56–62.

Depping MK, Uhlenbusch N, Härter M, Schramm C, Löwe B. Efficacy of a brief, peer-delivered self-management intervention for patients with Rare Chronic diseases: a Randomized Clinical Trial. JAMA Psychiatry. 2021;78:607–15.

Mulders G, de Wee EM, Vahedi Nikbakht-Van, de Sande MCVM, Kruip MJHA, Elfrink EJ, Leebeek FWG. E-learning improves knowledge and practical skills in haemophilia patients on home treatment: a randomized controlled trial. Haemophilia. 2012;18:693–8.

Smolich L, Charen K, Sherman SL. Health knowledge of women with a fragile X premutation: improving understanding with targeted educational material. J Genet Couns. 2020;29:983–91.

Jackson AD, Kirwan L, Gibney S, Jeleniewska P, Fletcher G, Doyle G. Associations between health literacy and patient outcomes in adolescents and young adults with cystic fibrosis. Eur J Public Health. 2020;30:112–8.

Merker VL, McDannold S, Riklin E, Talaei-Khoei M, Sheridan MR, Jordan JT, et al. Health literacy assessment in adults with neurofibromatosis: electronic and short-form measurement using FCCHL and Health LiTT. J Neurooncol. 2018;136:335–42.

Riklin E, Talaei-Khoei M, Merker VL, Sheridan MR, Jordan JT, Plotkin SR, et al. First report of factors associated with satisfaction in patients with neurofibromatosis. Am J Med Genet A. 2017;173:671–7.

LaDonna KA, Ghavanini AA, Venance SL. Truths and misinformation: a qualitative exploration of myotonic dystrophy. Can J Neurol Sci. 2015;42:187–94.

Parvizi MM, Lankarani KB, Handjani F, Ghahramani S, Parvizi Z, Rousta S. Health literacy in patients with epidermolysis bullosa in Iran. J Educ Health Promot. 2017;6:105.

Laberge L, Prévost C, Perron M, Mathieu J, Auclair J, Gaudreault M, et al. Clinical and genetic knowledge and attitudes of patients with myotonic dystrophy type 1. Public Health Genomics. 2010;13:424–30.

Rosnau K, Hashmi SS, Northrup H, Slopis J, Noblin S, Ashfaq M. Knowledge and self-esteem of individuals with neurofibromatosis type 1 (NF1). J Genet Couns. 2017;26:620–7.

Ioannou L, Massie J, Collins V, McClaren B, Delatycki MB. Population-based genetic screening for cystic fibrosis: attitudes and outcomes. Public Health Genomics. 2010;13:449–56.

Lewis KL, John B, Condren M, Carter SM. Evaluation of medication-related self-care skills in patients with cystic fibrosis. J Pediatr Pharmacol Ther. 2016;21:502–11.

PubMed   PubMed Central   Google Scholar  

Lindvall K, Colstrup L, Loogna K, Wollter I, Grönhaug S. Knowledge of disease and adherence in adult patients with haemophilia. Haemophilia. 2010;16:592–6.

Lonabaugh KP, O’Neal KS, McIntosh H, Condren M. Cystic fibrosis-related education: are we meeting patient and caregiver expectations? Patient Educ Couns. 2018;101:1865–70.

Mälstam E, Bensing S, Asaba E. Everyday managing and living with autoimmune Addison’s disease: exploring experiences using photovoice methods. Scand J Occup Ther. 2018;25:358–70.

Naik H, Shenbagam S, Go AM, Balwani M. Psychosocial issues in erythropoietic protoporphyria - the perspective of parents, children, and young adults: a qualitative study. Mol Genet Metab. 2019;128:314–9.

Takeuchi T, Muraoka K, Yamada M, Nishio Y, Hozumi I. Living with idiopathic basal ganglia calcification 3: a qualitative study describing the lives and illness of people diagnosed with a rare neurological disease. Springerplus. 2016;5:1713.

De la Corte-Rodriguez H, Rodriguez-Merchan EC, Alvarez-Roman T, Martin-Salces M, Garcia-Barcenilla S, Jimenez-Yuste V. Health education and empowerment in adult patients with haemophilia. Expert Rev Hematol. 2019;12:989–95.

Braisch U, Martinez-Horta S, MacDonald M, Orth M. Important but not enough - information about HD related topics and peer and professional support for young adults from HD families. J Huntingtons Dis. 2016;5:379–87.

Bryson B, Bogart K, Atwood M, Fraser K, Locke T, Pugh K, et al. Navigating the unknown: a content analysis of the unique challenges faced by adults with rare diseases. J Health Psychol. 2021;26:623–35.

Coathup V, Teare HJA, Minari J, Yoshizawa G, Kaye J, Takahashi MP, et al. Using digital technologies to engage with medical research: views of myotonic dystrophy patients in Japan. BMC Med Ethics. 2016;17:51.

Etchegary H. Healthcare experiences of families affected by Huntington disease: need for improved care. Chronic Illn. 2011;7:225–38.

Mohan R, Radhakrishnan N, Varadarajan M, Anand S. Assessing the current knowledge, attitude and behaviour of adolescents and young adults living with haemophilia. Haemophilia. 2021;27:e180–6.

Garrino L, Picco E, Finiguerra I, Rossi D, Simone P, Roccatello D. Living with and treating rare diseases: experiences of patients and professional health care providers. Qual Health Res. 2015;25:636–51.

Mooney J, Poland F, Spalding N, Scott DGI. In One Ear and Out the Other–Its a Lot to Take in’: A Qualitative Study Exploring the Informational Needs of Patients with ANCA-Associated Vasculitis. Musculoskeletal [Internet]. 2013; https://onlinelibrary.wiley.com/doi/abs/ https://doi.org/10.1002/msc.1030

David V, Feldman D, Danner-Boucher I, Rhun AL, Guyomarch B, Ravilly S, et al. Identifying the educational needs of lung transplant recipients with cystic fibrosis. Prog Transpl. 2015;25:18–25.

Gumuchian ST, Peláez S, Delisle VC, Carrier M-E, Jewett LR, El-Baalbaki G, et al. Understanding coping strategies among people living with scleroderma: a focus group study. Disabil Rehabil. 2018;40:3012–21.

Arya S, Wilton P, Page D, Boma-Fischer L, Floros G, Winikoff R, et al. They don’t really take my bleeds seriously: barriers to care for women with inherited bleeding disorders. J Thromb Haemost. 2021;19:1506–14.

Kurtz NS, Cote C, Heatwole C, Gagnon C, Youssof S. Patient-reported disease burden in oculopharyngeal muscular dystrophy. Muscle Nerve. 2019;60:724–31.

Pakhale S, Baron J, Armstrong M, Tasca G, Gaudet E, Aaron SD, et al. Lost in translation? How adults living with cystic fibrosis understand treatment recommendations from their healthcare providers, and the impact on adherence to therapy. Patient Educ Couns. 2016;99:1319–24.

Shepherd LM, Tahrani AA, Inman C, Arlt W, Carrick-Sen DM. Exploration of knowledge and understanding in patients with primary adrenal insufficiency: a mixed methods study. BMC Endocr Disord. 2017;17:47.

Arran N, Craufurd D, Simpson J. Illness perceptions, coping styles and psychological distress in adults with Huntington’s disease. Psychol Health Med. 2014;19:169–79.

Dellon EP, Helms SW, Hailey CE, Shay R, Carney SD, Schmidt HJ, et al. Exploring knowledge and perceptions of palliative care to inform integration of palliative care education into cystic fibrosis care. Pediatr Pulmonol. 2018;53:1218–24.

Shoshan L, Ben-Zvi D, Meyer S, Katz-Leurer M. Sexuality in relation to independence in daily functions among young people with spina bifida living in Israel. Rehabil Nurs. 2012;37:11–7. quiz 17–8.

Walsh MB, Charen K, Shubeck L, McConkie-Rosell A, Ali N, Bellcross C, et al. Men with an FMR1 premutation and their health education needs. J Genet Couns. 2021;30:1156–67.

Sylvain C, Lamothe L, Berthiaume Y, Rabasa-Lhoret R. How patients’ representations of cystic fibrosis-related diabetes inform their health behaviours. Psychol Health. 2016;31:1129–44.

Arnold E, Lane S, Webert KE, Chan A, Walker I, Tufts J, et al. What should men living with haemophilia need to know? The perspectives of Canadian men with haemophilia. Haemophilia. 2014;20:219–25.

Torres-Ortuño A, Cuesta-Barriuso R, Nieto-Munuera J, Galindo-Piñana P, López-Pina J-A. The behaviour and perception of illness: modulating variables of adherence in patients with haemophilia. Vox Sang [Internet]. 2018; https://doi.org/10.1111/vox.12669

Keyte R, Egan H, Nash EF, Regan A, Jackson C, Mantzios M. An exploration into experiences and attitudes regarding risky health behaviours in an adult cystic fibrosis population. Psychol Health Med. 2020;25:1013–9.

Carpenter DM, DeVellis RF, Hogan SL, Fisher EB, DeVellis BM, Jordan JM. Use and perceived credibility of medication information sources for patients with a rare illness: differences by gender. J Health Commun. 2011;16:629–42.

Dwyer AA, Quinton R, Morin D, Pitteloud N. Identifying the unmet health needs of patients with congenital hypogonadotropic hypogonadism using a web-based needs assessment: implications for online interventions and peer-to-peer support. Orphanet J Rare Dis. 2014;9:83.

Litzkendorf S, Frank M, Babac A, Rosenfeldt D, Schauer F, Hartz T, et al. Use and importance of different information sources among patients with rare diseases and their relatives over time: a qualitative study. BMC Public Health. 2020;20:860.

Molster C, Urwin D, Di Pietro L, Fookes M, Petrie D, van der Laan S, et al. Survey of healthcare experiences of Australian adults living with rare diseases. Orphanet J Rare Dis. 2016;11:30.

Lindsay S, Fellin M, Cruickshank H, McPherson A, Maxwell J. Youth and parents’ experiences of a new inter-agency transition model for spina bifida compared to youth who did not take part in the model. Disabil Health J. 2016;9:705–12.

Skirton H, Williams JK, Jackson Barnette J, Paulsen JS. Huntington disease: families’ experiences of healthcare services. J Adv Nurs. 2010;66:500–10.

Domaradzki J. Family caregivers’ experiences with healthcare services–a case of Huntington’s disease. Psychiatr Pol. 2016;50:375–91.

Katavic SS, Tanackovic SF, Badurina B. Illness perception and information behaviour of patients with rare chronic diseases. Inflamm Res [Internet]. 2016 [cited 2023 Jun 14];21. https://doi.org/10.1111/hir.12261

Socha Hernandez AV, Deeks LS, Shield AJ. Understanding medication safety and Charcot-Marie-tooth disease: a patient perspective. Int J Clin Pharm. 2020;42:1507–14.

Akanuwe JNA, Laparidou D, Curtis F, Jackson J, Hodgson TL, Siriwardena AN. Exploring the experiences of having Guillain-Barré syndrome: a qualitative interview study. Health Expect. 2020;23:1338–49.

Foley G, Timonen V, Hardiman O. Understanding psycho-social processes underpinning engagement with services in motor neurone disease: a qualitative study. Palliat Med. 2014;28:318–25.

Borghi L, Moreschi C, Toscano A, Comber P, Vegni E, The PKU. ME study: a qualitative exploration, through co-creative sessions, of attitudes and experience of the disease among adults with phenylketonuria in Italy. Mol Genet Metabolism Rep. 2020;23:100585.

CAS   Google Scholar  

Kazmerski TM, Gmelin T, Slocum B, Borrero S, Miller E. Attitudes and decision making related to pregnancy among Young women with cystic fibrosis. Matern Child Health J. 2017;21:818–24.

Flewelling KD, Sellers DE, Sawicki GS, Robinson WM, Dill EJ. Social support is associated with fewer reported symptoms and decreased treatment burden in adults with cystic fibrosis. J Cyst Fibros. 2019;18:572–6.

Kirk S, Milnes L. An exploration of how young people and parents use online support in the context of living with cystic fibrosis. Health Expect. 2016;19:309–21.

Salvatore V, Gilstrap A, Williams KR, Thorat S, Stevenson M, Gwosdow AR, et al. Evaluating the impact of peer support and connection on the quality of life of patients with familial chylomicronemia syndrome. Expert Opin Orphan Drugs. 2018;6:497–505.

Article   CAS   Google Scholar  

López-Bastida J, Oliva-Moreno J, Linertová R, Serrano-Aguilar P. Social/economic costs and health-related quality of life in patients with rare diseases in Europe. Eur J Health Economics: HEPAC: Health Econ Prev care. 2016;17(Suppl 1):1–5.

Sørensen K, Van den Broucke S, Pelikan JM, Fullam J, Doyle G, Slonska Z et al. Measuring health literacy in populations: illuminating the design and development process of the European Health Literacy Survey Questionnaire (HLS-EU-Q). BMC Public Health [Internet]. 2013;13. https://doi.org/10.1186/1471-2458-13-948

Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Crotty K. Low health literacy and health outcomes: an updated systematic review. Ann Intern Med. 2011;155:97–107.

Geboers B, Reijneveld SA, Jansen CJM, de Winter AF. Health Literacy Is Associated With Health Behaviors and Social factors among older adults: results from the LifeLines Cohort Study. J Health Commun. 2016;21:45–53.

Mackey LM, Doody C, Werner EL, Fullen B. Self-management skills in Chronic Disease Management: what role does health literacy have? Med Decis Mak. 2016;36:741–59.

Paasche-Orlow MK, Wolf MS. The causal pathways linking health literacy to health outcomes. Am J Health Behav. 2007;31:19–26.

de Vrueh R, de Baekelandt Erf JMH. Background paper 6.19. Rare diseases. Geneva: World Health Organization; 2013.

Google Scholar  

Uhlenbusch N, Löwe B, Härter M, Schramm C, Weiler-Normann C, Depping MK. Depression and anxiety in patients with different rare chronic diseases: a cross-sectional study. PLoS ONE. 2019;14:e0211343.

Download references

Acknowledgements

The authors want to thank leaders and colleagues. A special thank goes to librarian Hilde Iren Flaatten who has conducted systematic searches after literature and Anne Siri Albrigtsen for important contributions to this review. Several of the authors are members of Euro-NMD.

This work received funding from Norwegian National Advisory Unit on Rare Disorders. In addition the work was performed as part of regular activities at Frambu Resource Centre for Rare Disorders and Norwegian National Advisory Unit on Learning and Mastery in Health, Oslo University Hospital.

Author information

Authors and affiliations.

Frambu Resource Center for Rare Disorders, Sandbakkveien 18, Siggerud, 1404, Norway

Una Stenberg & Lydia Westfal

Norwegian National Advisory Unit on Learning and Mastery in Health, Oslo University Hospital, Postboks 4959 Nydalen, Oslo, 0424, Norway

Una Stenberg & Karl Fredrik Westermann

National Neuromuscular Centre Norway, University Hospital of North-Norway, Hansine Hansens vei 37, Tromsø, 9019, Norway

Andreas Dybesland Rosenberger

Section for Rare Neuromuscular Disorders and Unit for Congenital and Hereditary Neuromuscular Disorders (EMAN), Department of Neurology, Oslo University Hospital, Postboks 4950 Nydalen, Oslo, 0424, Norway

Kristin Ørstavik

Department of Neurobiology, Care Sciences and Society, Karolinska Institutet, Solnavägen 1, Solna, 171 77, Sweden

Maria Flink

Oslo Metropolitan University, Postbox 4, St. Olavs place, Oslo, N-0130, Norway

Heidi Holmen

National Centre for Rare Epilepsy-Related Disorders, Department of Rare Disorders, Division of Paediatric and Adolescent Medicine, Oslo University Hospital, Postboks, 4950 Nydalen, 0424, Oslo, Norway

Silje Systad

TRS Resource Centre for Rare Diseases, Sunnaas Rehabilitation Hospital, Bjørnemyrveien 11, 1453, Bjørnemyr, Norway

You can also search for this author in PubMed   Google Scholar

Contributions

All the authors (US, GV, SS, KØ, KFW, HH, ADR, MF) except LW have contributed in the conception, design and analysis of data and interpretation of data. US has led all the phases of the review, and LW has contributed in the analysis and interpretation of data, and in writing the manuscript. All authors have read and approved the final manuscript.

Corresponding author

Correspondence to Una Stenberg .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, supplementary material 4, supplementary material 5, supplementary material 6, supplementary material 7, supplementary material 8, supplementary material 9, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Stenberg, U., Westfal, L., Dybesland Rosenberger, A. et al. A scoping review of health literacy in rare disorders: key issues and research directions. Orphanet J Rare Dis 19 , 328 (2024). https://doi.org/10.1186/s13023-024-03332-5

Download citation

Received : 25 October 2023

Accepted : 21 August 2024

Published : 06 September 2024

DOI : https://doi.org/10.1186/s13023-024-03332-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Health literacy
• Rare disorders
• Rare disease
• Rare genetic disorder
• Rare developmental defect
• Integrative model of health literacy
• Scoping review

Orphanet Journal of Rare Diseases

ISSN: 1750-1172

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

• Systematic review
• Open access
• Published: 11 September 2024

Evaluation of research co-design in health: a systematic overview of reviews and development of a framework

• Sanne Peters   ORCID: orcid.org/0000-0001-6235-1752 1 ,
• Lisa Guccione 2 , 3 ,
• Jill Francis 1 , 2 , 3 , 4 ,
• Stephanie Best 1 , 2 , 3 , 5 ,
• Emma Tavender 6 , 7 ,
• Janet Curran 8 , 9 ,
• Katie Davies 10 ,
• Stephanie Rowe 1 , 8 ,
• Victoria J. Palmer 11 &
• Marlena Klaic 1  

Implementation Science volume  19 , Article number:  63 ( 2024 ) Cite this article

13 Altmetric

Metrics details

Co-design with consumers and healthcare professionals is widely used in applied health research. While this approach appears to be ethically the right thing to do, a rigorous evaluation of its process and impact is frequently missing. Evaluation of research co-design is important to identify areas of improvement in the methods and processes, as well as to determine whether research co-design leads to better outcomes. We aimed to build on current literature to develop a framework to assist researchers with the evaluation of co-design processes and impacts.

A multifaceted, iterative approach, including three steps, was undertaken to develop a Co-design Evaluation Framework: 1) A systematic overview of reviews; 2) Stakeholder panel meetings to discuss and debate findings from the overview of reviews and 3) Consensus meeting with stakeholder panel. The systematic overview of reviews included relevant papers published between 2000 and 2022. OVID (Medline, Embase, PsycINFO), EBSCOhost (Cinahl) and the Cochrane Database of Systematic reviews were searched for papers that reported co-design evaluation or outcomes in health research. Extracted data was inductively analysed and evaluation themes were identified. Review findings were presented to a stakeholder panel, including consumers, healthcare professionals and researchers, to interpret and critique. A consensus meeting, including a nominal group technique, was applied to agree upon the Co-design Evaluation Framework.

A total of 51 reviews were included in the systematic overview of reviews. Fifteen evaluation themes were identified and grouped into the following seven clusters: People (within co-design group), group processes, research processes, co-design context, people (outside co-design group), system and sustainment. If evaluation methods were mentioned, they mainly included qualitative data, informal consumer feedback and researchers’ reflections. The Co-Design Evaluation Framework used a tree metaphor to represent the processes and people in the co-design group (below-ground), underpinning system- and people-level outcomes beyond the co-design group (above-ground). To evaluate research co-design, researchers may wish to consider any or all components in the tree.

Conclusions

The Co-Design Evaluation Framework has been collaboratively developed with various stakeholders to be used prospectively (planning for evaluation), concurrently (making adjustments during the co-design process) and retrospectively (reviewing past co-design efforts to inform future activities).

Peer Review reports

Contributions to the literature

While stakeholder engagement in research seems ethically the right thing to do, a rigorous evaluation of its process and outcomes is frequently missing.

Fifteen evaluation themes were identified in the literature, of which research process , cognitive and emotional factors were the most frequently reported.

The Co-design Evaluation Framework can assist researchers with research co-design evaluation and provide guidance regarding what and when to evaluate.

The framework can be used prospectively, concurrently, and retrospectively to make improvements to existing and future research co-design projects.

Introduction

Lots of money is wasted in health research that does not lead to meaningful benefits for end-users, such as healthcare professionals and consumers [ 1 , 2 , 3 ]. One contributor to this waste is that research often focusses on questions and outcomes that are of limited importance to end-users [ 4 , 5 ]. Engaging relevant people in research co-design has increased in order to respond to this issue. There is a lack of consensus in the literature on the definition and processes involved in undertaking a co-design approach. For the purposes of this review, we define research co-design as meaningful end-user engagement that occurs across any stage of the research process , from the research planning phase to dissemination of research findings [ 6 ]. Meaningful end-user engagement refers to an explicit and measurable responsibility, such as contributing to writing a study proposal [ 6 ]. The variety of research co-design methods can be seen as a continuum ranging from limited involvement, such as consulting with end-users, to the much higher effort research approaches in which end-users and researchers aim for equal decision-making power and responsibility across the entire research process [ 6 ]. Irrespective of the intensity of involvement, it is generally recommended that a co-design approach should be based on several important principles such as equity, inclusion and shared ownership [ 7 ].

Over time, increasing attention has been given to research co-design [ 6 , 8 ]. Funding bodies encourage its use and it is recommended in the updated UK MRC framework on developing and evaluating complex interventions [ 9 ]. End-user engagement has an Equator reporting checklist [ 10 ] and related work has been reported by key organisations, such as the James Lind Alliance in the UK ( www.jla.nihr.ac.uk ), Patient Centered Outcomes Research Institute in the US ( www.pcori.org ) and Canadian Institutes of Health Research ( https://cihrirsc.gc.ca/e/41592.html ). In addition, peer reviewed publications involving co-design have risen from 173 per year in 2000 to 2617 in 2022 (PubMed), suggesting a growing importance in research activities.

Engaging end-users in the health research process is arguably the right thing to do, but the processes and outcomes of co-design have rarely been evaluated in a rigorous way [ 6 ]. Existing anecdotal evidence suggests that research co-design can benefit researchers, end-users and lead to more robust research processes [ 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 ]. Both researchers and end-users have reported positive experiences of engaging in the co-design process. Potential benefits include a better understanding of community needs, more applicable research questions, designs and materials and improved trust between the researchers and end-users. Several reviews on conducting research co-design have concluded that co-design can be feasible, though predominantly used in the early phases of research, for example formulating research questions and developing a study protocol [ 6 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 ]. However, these reviews highlighted that engagement of end-users in the research process required extra time and funding and had the risk of becoming tokenistic [ 6 , 11 , 12 , 13 , 14 , 15 , 16 , 17 , 18 , 19 ].

The use of resources in co-design studies might need to be justified to the funder as well as its impacts. A rigorous evaluation of research co-design processes and outcomes is needed to identify areas of potential improvement and to determine the impact of research co-design. Several overviews of reviews on research co-design have been published but with no or limited focus on evaluation [ 20 , 21 , 22 , 23 ]. Moreover, current literature provides little guidance around how and what to evaluate, and which outcomes are key.

This study thus had two aims:

To conduct a systematic overview of reviews to identify evaluation methods and process and outcome variables reported in the published health research co-design literature.

To develop a framework to assist researchers with the evaluation of co-design processes and impacts.

This project used a multifaceted, iterative approach to develop a Co-design Evaluation Framework. It consisted of the following steps: 1) A systematic overview of reviews; 2) Stakeholder panel meetings to discuss and debate findings from the overview of reviews and 3) Consensus meeting with stakeholder panel. The reporting checklist for overviews of reviews was applied in Additional file 1 [ 24 ].

Step 1: A systematic overview of reviews

We conducted a systematic overview of reviews [ 25 ], reviewing literature reviews rather than primary studies, to investigate the following question: What is known in the published literature about the evaluation of research co-design in health research? The protocol of our systematic overview of reviews was published in the PROSPERO database (CRD42022355338).

Sub questions:

What has been co-designed and what were the objectives of the co-design process?

Who was involved and what was the level of involvement?

What methods were used to evaluate the co-design processes and outcomes?

What was evaluated (outcome and process measures) and at what timepoint (for example concurrently, or after, the co-design process)?

Was a co-design evaluation framework used to guide evaluation?

Search strategy

We searched OVID (Medline, Embase, PsycINFO), EBSCOhost (Cinahl) and the Cochrane Database of Systematic reviews on the 11th of October 2022 for literature reviews that reported co-design evaluation or outcomes in health research. The search strategy was based on previous reviews on co-design [ 6 , 14 , 26 ] and refined with the assistance of a research librarian and the research team (search terms in Additional file 2). Papers published from January 2000 to September 2022 were identified and retrieved by one author (SP).

Study selection

Database records were imported into EndNote X9 (The EndNote Team, Philadelphia, 2013) and duplicates removed. We managed the study selection process in the software program Covidence (Veritas Health Innovation, Melbourne, Australia). Two independent reviewers (SP, MK or LG) screened the titles and abstracts of all studies against the eligibility criteria (Table  1 ). Discrepancies were resolved through discussion or with a third reviewer (either SP, MK or LG, depending on which 2 reviewers disagreed). If there was insufficient information in the abstract to decide about eligibility, the paper was retained to the full-text screening phase. Full-text versions of studies not excluded at the title and abstract screening phase were retrieved and independently screened by two reviewers (SP, MK or LG) against eligibility criteria. Disagreements were resolved through discussion, or with a third reviewer, and recorded in Covidence.

Data extraction of included papers was conducted by one of three reviewers (SP, MK or LG). A second reviewer checked a random sample of 20% of all extracted data (LG or SP). Disagreements were resolved through regular discussion. Data were extracted using an excel spreadsheet developed by the research team and included review characteristics (such as references, type of review, number of included studies, review aim), details about the co-design process (such as who was involved in the co-design, which topics the co-design focused on, what research phase(s) the co-design covered, in which research phase the co-design took place and what the end-users’ level of involvement was) and details about the co-design evaluation (what outcomes were reported, methods of data collection, who the participants of the evaluation were, the timepoint of evaluation, whether an evaluation framework was used or developed and conclusions about co-design evaluation).

Types of end-users’ involvement were categorised into four groups based on the categories proposed by Hughes et al. (2018): 1. Targeted consultation; 2. Embedded consultation; 3. Collaboration and co-production and 4. User-led research, see Table  2 .

Data extraction and analysis took place in three iterative phases (Fig.  1 ), with each phase containing one third of the included studies. Each phase of data extraction and analysis was followed by stakeholder panel meetings (see step 2 below). This stepwise approach enabled a form of triangulation wherein themes that emerged through each phase were discussed with the stakeholder panel and incorporated both retrospectively (re-coding data in the prior phase) and prospectively (coding new data in the next phase).

Iterative phases in the process of the Co-design evaluation framework development

All reported outcomes of research co-design in the first phase (one third of all data) were inductively coded into themes, according to the principles of thematic analysis [ 28 ]. Two researchers (SP and MK) double coded 10% of all data and reached consensus through discussion. Given that consensus was high, one researcher (SP) continued the coding while having frequent discussions and reviews within the research team. In phase 2 (also one third of all data), deductive coding was based on the themes identified in the first round. Data of the first phase were re-coded, if new codes emerged during the stakeholder panel meeting. The same process took place for the third phase.

Step 2: Stakeholder panel meetings to discuss and debate findings from the overview of reviews

Results from step 1 were presented to the stakeholder panel to interpret and critique the review findings. The panel consisted of ten people, including a mix of consumers, healthcare professionals and researchers. Stakeholders were selected for their experience or expertise in research co-design. The number of meetings was not pre-determined, rather, it was informed by the outcomes from step 1. The number of stakeholders in each meeting ranged from six to ten.

A core group from the broader stakeholder panel (SP, MK, LG, JF) with a breadth of research experience and methodological expertise discussed the themes arising from both steps 1 and 2 and considered various ways of presenting them. Multiple design options were considered and preliminary frameworks were developed. Following discussion with the stakeholder panel, it was agreed that the evaluation themes could be grouped into several clusters to make the framework more comprehensible. The grouping of evaluation themes into clusters was informed by reported proposed associations between evaluation themes in the literature as well as the stakeholder panel’s co-design experience and expertise. Evaluation themes as well as clusters were agreed upon during the stakeholder panel meetings.

Step 3: Consensus meeting with stakeholder panel

The consensus meeting included the same stakeholder panel as in step 2. The meeting was informed by a modified Nominal Group Technique (NGT). The NGT is a structured process for obtaining information and reaching consensus with a target group who have some association or experience with the topic [ 29 ]. Various adaptations of the NGT have been used and additional pre-meeting information has been suggested to enable more time for participants to consider their contribution to the topic [ 30 ]. The modified NGT utilised in this study contained the following: (i) identification of group members to include experts with depth and diverse experiences. They were purposively identified at the start of this study for their expertise or experience in research co-design and included: a patient consumer, a clinician, three clinician researchers and six researchers with backgrounds in behavioural sciences, psychology, education, applied ethics and participatory design. All authors on this paper were invited by e-mail to attend an online meeting; (ii) provision of information prior to the group meeting included findings of the overview of reviews, a draft framework and objectives of the meeting. Five authors with extensive research co-design experience were asked to prepare a case example of one of their co-design projects for sharing at the group meeting. The intention of this exercise was to discuss the fit between a real-world example and the proposed framework; (iii) hybrid meeting facilitated by two researchers (SP & JF) who have experience in facilitating consensus meetings. Following presentation of the meeting materials, including the preliminary framework, group members were invited to silently consider the preliminary framework and generate ideas and critiques; iv) participants sharing their ideas and critiques; v) clarification process where group members shared their co-design example project and discussed the fit with components of the initial framework, and vi) silent voting and/or agreement on the framework via a personal email to one of the researchers (SP).

Step 1: Systematic overview of reviews

The database searches identified a total of 8912 papers. After removing 3016 duplicates and screening 5896 titles and abstracts, 148 full texts were sought for retrieval. Sixteen were not retrieved as they were not available in English ( n  = 2) or full-text was not available ( n  = 14). Of the remaining 132 papers assessed for eligibility, 81 were excluded. The final number of papers included in this overview of reviews was 51 (See Fig.  2 ).

PRISMA flow chart (based on [ 31 ]) of overview of reviews

Characteristics of the included studies

Of the 51 included reviews [ 11 , 12 , 14 , 32 , 33 , 34 , 35 , 36 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 , 46 , 47 , 48 , 49 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 , 68 , 69 , 70 , 71 , 72 , 73 , 74 , 75 , 76 , 77 , 78 , 79 ], 17 were systematic reviews, 12 were scoping reviews, 14 did not report the type or method of review, three were narrative reviews, two were qualitative evidence synthesis, another two were a structured literature search and one was a realist review. The number of studies included in the reviews ranged from 7 to 260. Nineteen reviews focused on co-design with specific populations, for example older people, people with intellectual disabilities, people living with dementia and 32 reviews included co-design with a range of end-users. The co-design focused in most cases on a mix of topics ( n  = 31). Some reviews were specifically about one clinical topic, for example critical care or dementia. In ten cases, the clinical topics were not reported. Co-design took place during multiple research phases. Thirty-six reviews covered co-design in agenda/priority setting, 36 in study design, 30 in data collection, 25 in data analysis and 27 in dissemination. With regards to the research translation continuum, most of the co-design was reported in practice and community-based research ( n  = 32), three reviews were conducted in basic research and 11 in human research. The types of end-users’ involvement in co-design ranged from targeted consultation ( n  = 14) to embedded consultation ( n  = 20), collaboration and co-production ( n  = 14) to end-user- led research ( n  = 6), including papers covering multiple types of involvement. Seventeen papers did not report the types of involvement. The reported co-design included a variety of time commitments, from a minimum of a one-off 60-min meeting to multiple meetings over multiple years. Twenty-seven reviews did not report details about the end-users’ types of involvement.

Identified evaluation themes

Fifteen evaluation themes were identified and were arranged into two higher level groups: 1. within the co-design team and 2. broader than co-design team (Table  3 ). The themes related to the first group (within the co-design team) included: Structure and composition of the co-design group, contextual enablers/barriers, interrelationships between group members, decision making process, emotional factors, cognitive factors, value proposition, level/ quality of engagement, research process, health outcomes for co-design group and sustainment of the co-design team or activities. The themes within the second group (broader than co-design team) included: Healthcare professional-level outcomes, healthcare system level outcomes, organisational level outcomes and patient and community outcomes.

The research process was the most frequently reported evaluation theme in the reviews ( n  = 44, 86% of reviews), followed by cognitive factors ( n  = 35, 69%) and emotional factors ( n  = 34, 67%) (Table  4 ). Due to variability in reporting practices, it was not possible to specify the number of primary studies that reported specific evaluation themes. Evaluation methods for the themes were not reported in the majority of reviews ( n  = 43, 84%). If evaluation methods were mentioned, they were mainly based on qualitative data, including interviews, focus groups, field notes, document reviews and observations (see overview with references in Additional file 3). Survey data was mentioned in three reviews. Many reviews reported informal evaluation based on participant experiences (e.g. informal feedback), reflection meetings, narrative reflections and authors’ hypotheses (Additional file 3). The timing of the evaluation was only mentioned in two papers: 1. Before and after the co-design activities and 2. Post co-design activities. One paper suggested that continuous evaluation might be helpful to improve the co-design process (Additional file 3).

The systematic overview of reviews found that some authors reported proposed positive associations between evaluation themes (Table  5 ). The most frequently reported proposed association was between level/quality of engagement and emotional factors ( n  = 5, 10%). However, these proposed associations did not seem to have any empirical evidence and evaluation methods were not reported.

All evaluation themes were grouped into the following clusters (Table  6 ): People (within co-design group), group processes, research processes, co-design context, people (outside co-design group), system and sustainment.

Only one paper reported the evaluation in connection to the research phases (Agenda/priority setting, study design, data collection, data analysis and dissemination). This paper reported the following outcomes for the following research phases [ 58 ]:

Agenda/priority setting: Research process; Level/quality of engagement; Cognitive factors; Attributes of the co-design group; Interrelationships between group members; Sustainment of the co-design team or activities; Patient and community outcomes.

Study design: Attributes of the co-design group; Interrelationships between group members; Level/quality of engagement; Cognitive factors; Emotional factors; Research process.

The various research phases in which consumers could be involved, as well as the clusters of evaluation themes, informed the design of the co-design evaluation framework.

Two main options were voted on and discussed within the stakeholder panel. The two main options can be found in Additional file 4. Draft 2 was the prefered option as it was perceived as more dynamic than draft 1, representing a clearer interplay between the two contexts. The stakeholder panel suggested a few edits to the draft, such as the inclusion of bi-directional arrows in the tree trunk and a vertical arrow from underground to above ground with the label ‘impact’.

The final version of the Co-design Evaluation framework is presented in Fig.  3 .

Research Co-design evaluation framework

Figure  3 presents co-design evaluation as the below-ground and above-ground structures of a tree. The tree metaphor presents the processes and people in the co-design group (below-ground) as the basis for system- and people-level outcomes beyond the co-design group (above-ground). To evaluate research co-design, researchers may wish to consider any or all components in this Figure. These evaluation components relate to the methods, processes, and outcomes of consumer involvement in research.

The context within the co-design group (the roots of the tree) consists of the people, group processes and research processes, with various evaluation themes (dot points) related to them, as well as contextual barriers and enablers that relate to situational aspects that might enable or hinder consumer engagement. The context outside the co-design group, i.e., the wider community (the branches and leaves of the tree), comprises people who were not involved in the research co-design process, the system-level and sustainment-related outcomes. These above ground groups are potential beneficiaries or targets of the co-design activities.

The arrows in the middle of the trunk represent the potential mutual influence of the two contexts, suggesting that an iterative approach to evaluation might be beneficial. For example, when deciding the composition of the co-design group, it may be important to have an appropriate representation of the people most impacted by the problem issue or topic at hand. Or, if a co-designed healthcare intervention does not achieve the desired outcomes in the wider context, the co-design group might consider potential ways to improve the intervention or how it was delivered. Evaluation of a research co-design process might start with the foundations (the roots of the tree) and progress to above ground (the tree grows and might develop fruit). Yet, depending on the aim of the evaluation, a focus on one of the two contexts, either below or above ground, might be appropriate.

Which, and how many, components are appropriate to evaluate depends on the nature of the co-design approach and the key questions of the evaluation. For example, if a co-design approach is used in the very early stages of a research program, perhaps to identify priorities or to articulate a research question, then 'below' the ground components are key. While a randomised study comparing the effects of a co-designed intervention versus a researcher-designed intervention might only consider 'above' the ground components.

The white boxes on the right-hand side of Fig.  3 indicate the research phases, from agenda/priority setting to dissemination, in which consumers can and should be involved. This co-design evaluation framework may be applied at any phase of the research process or applied iteratively with a view to improving future co-design activities.

This systematic overview of reviews aimed to build on current literature and develop a framework to assist researchers with the evaluation of research co-design. Fifty-one included reviews reported on fifteen evaluation themes, which were grouped into the following clusters: People (within co-design group), group processes, research processes, co-design context, people (outside co-design group), system and sustainment. Most reviews did not report measurement methods for the evaluation themes. If methods were mentioned, they mostly included qualitative data, informal consumer feedback and researchers’ reflections. This finding strengthens our argument that a framework may be helpful in supporting methodologically robust studies to assess co-design processes and impacts. The Co-Design Evaluation Framework has adopted a tree metaphor. It presents the processes and people in the co-design group (below-ground) as the underpinning system- and people-level outcomes beyond the co-design group (above-ground). To evaluate stakeholder involvement in research, researchers may wish to consider any or all components in the tree. Which, and how many, components are appropriate to evaluate depends on the nature of the co-design approach and the key questions that stakeholders aim to address. Nonetheless, it will be important that evaluations delineate what parts of the research project have incorporated a co-design approach.

The Equator reporting checklist for Research Co-Design, GRIPP2, provides researchers with a series of concepts that should be considered and reported on when incorporating patient and public involvement in research [ 10 ]. These concepts include, but are not limited to, methods of involving patients and the public in research and intensity of engagement. The Co-Design Evaluation Framework is not intended as a replacement for the GRIPP2, rather, it can be used prospectively to inform development of the co-design project or retropsectively to inform completion of the GRIPP2. Table 7 provides hypothetical examples of research questions that co-design evaluation projects might address. The framework could be used at multiple points within co-design projects, including prospectively (planning for evaluation before the co-design process has started), concurrently ( incorporating improvements during the co-design process) and retrospectively (reviewing past co-design efforts to inform future projects).

Our systematic overview of reviews identified multiple evaluation themes. Some of these overlapped with reported values associated with public involvement in research [ 80 ], community engagement measures [ 15 ] and reported impacts of patient and public involvement in research, as described by others [ 16 , 81 , 82 ]. The added value of our systematic overview of reviews is that we went beyond a list of items and took it one step further by looking at evaluation themes, potential associations between evaluation themes, clusters of evaluation themes and ultimately developed a framework to assist others with research co-design evaluation.

Some reviews in our overview of reviews proposed potential associations between evaluation themes. Yet, these proposed associations were not empirically tested. One of the included studies [ 58 ] proposed conditions and mechanisms involved in co-design processes and outcomes related to diabetes research. Although it is a promising starting point, this should be further explored. A realist evaluation including other research topics and other approaches, such as the use of logic models, which was also recognised in the updated MRC framework [ 9 ], might help to build on explorations of included mechanisms of action [ 83 ] and give insight into how core ingredients contribute to certain co-design processes and outcomes. As recognised by others [ 6 , 84 ], the reporting practice of research co-design in the literature could be improved as details about context, mechanisms and expected outcomes are frequently missing. This will also help us to gain a better understanding of what works for whom, why, how and in which circumstances.

The lack of a consistent definition of co-design makes it challenging to identify and synthesise the literature, as recognised by others [ 6 ]. Given that there are so many different terms used in the literature, there is a risk that we might have missed some relevant papers in our overview of reviews. Nevertheless, we tried to capture as many as possible synonyms of co-design in our search terms. The absence of quality assessment of included studies in our overview of reviews can be seen as a limitation. However, our overview of reviews did not aim to assess existing literature on the co-design process, but rather focused on what to evaluate, how and when. We did note whether the reported evaluation themes were based on empirical evidence or authors’ opinions. Primary studies reported in the included reviews were not individually reviewed as this was outside the scope of this paper. A strength in our methods was the cyclical process undertaken between steps 1 and 2. Analysis of the data extracted from the overview was refined over three phases following rigorous discussions with a diverse and experienced stakeholder panel. It was a strength of our project that a mix of stakeholders were involved, including consumers, healthcare professionals and researchers.

Stakeholders are frequently engaged in research but if research co-design processes and outcomes are not evaluated, there will be limited learning from past experiences. Evaluation is essential to make refinements during existing projects and improve future co-design activities. It is also critical for ensuring commitments to the underpinning values of c-odesign are embedded within activities.

A systematic review of all primary studies within the included reviews of this overview of reviews, would allow greater depth relating to the practicalities of how to evaluate certain themes. It would lead to a better understanding of existing measures and methods and which evaluation areas need further development. Future research should also focus on whether co-design leads to better outcomes than no co-design (only researcher-driven research). To our knowledge, this has not been explored yet. Moreover, future research could gain better insight into the mechanisms of change within co-design and explore potential associations between evaluation themes for example, those proposed in the included reviews between level/quality of engagement and emotional factors.

We followed a systematic, iterative approach to develop a Co-Design Evaluation Framework that can be applied to various phases of the research co-design process. Testing of the utility of the framework is an important next step. We propose that the framework could be used at multiple points within co-design projects, including prospectively (planning for evaluation before the co-design process has started), concurrently (to incorporate improvements during the co-design process) and retrospectively (reviewing past co-design efforts to inform future projects).

Availability of data and materials

All data generated during this study are included either within the text or as a supplementary file.

Abbreviations

Medical Research Council

Guidance for Reporting Involvement of Patients and the Public

Chalmers I, Glasziou P. Systematic reviews and research waste. Lancet. 2016;387(10014):122–3.

Article   PubMed   Google Scholar  

Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet. 2009;374(9683):86–9.

Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 2014;383(9913):267–76.

Ioannidis JP. Why Most Clinical Research Is Not Useful. PLoS Med. 2016;13(6):e1002049.

Article   PubMed   PubMed Central   Google Scholar  

Oliver S. Patient involvement in setting research agendas. Eur J Gastroenterol Hepatol. 2006;18(9):935–8.

Slattery P, Saeri AK, Bragge P. Research co-design in health: a rapid overview of reviews. Health Res Policy Syst. 2020;18(1):17.

Ní Shé É, Harrison R. Mitigating unintended consequences of co-design in health care. Health Expect. 2021;24(5):1551–6.

Peters S, Sukumar K, Blanchard S, Ramasamy A, Malinowski J, Ginex P, et al. Trends in guideline implementation: an updated scoping review. Implement Sci. 2022;17:50.

Skivington K, Matthews L, Simpson SA, Craig P, Baird J, Blazeby JM, et al. A new framework for developing and evaluating complex interventions: update of Medical Research Council guidance. BMJ. 2021;374:n2061.

Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. BMJ. 2017;358:j3453.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Domecq JP, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, et al. Patient engagement in research: a systematic review. BMC Health Serv Res. 2014;14:89.

Manafo E, Petermann L, Mason-Lai P, Vandall-Walker V. Patient engagement in Canada: a scoping review of the “how” and “what” of patient engagement in health research. Health Res Policy Syst. 2018;16(1):5.

Fergusson D, Monfaredi Z, Pussegoda K, Garritty C, Lyddiatt A, Shea B, et al. The prevalence of patient engagement in published trials: a systematic review. Res Involv Engagem. 2018;4:17.

Vat LE, Finlay T, Jan Schuitmaker-Warnaar T, Fahy N, Robinson P, Boudes M, et al. Evaluating the “return on patient engagement initiatives” in medicines research and development: A literature review. Health Expect. 2020;23(1):5–18.

Luger TM, Hamilton AB, True G. Measuring Community-Engaged Research Contexts, Processes, and Outcomes: A Mapping Review. Milbank Q. 2020;98(2):493–553.

Modigh A, Sampaio F, Moberg L, Fredriksson M. The impact of patient and public involvement in health research versus healthcare: A scoping review of reviews. Health Policy. 2021;125(9):1208–21.

Clavel N, Paquette J, Dumez V, Del Grande C, Ghadiri DPS, Pomey MP, et al. Patient engagement in care: A scoping review of recently validated tools assessing patients’ and healthcare professionals’ preferences and experience. Health Expect. 2021;24(6):1924–35.

Newman B, Joseph K, Chauhan A, Seale H, Li J, Manias E, et al. Do patient engagement interventions work for all patients? A systematic review and realist synthesis of interventions to enhance patient safety. Health Expect. 2021;24:1905 No Pagination Specified.

Lowe D, Ryan R, Schonfeld L, Merner B, Walsh L, Graham-Wisener L, et al. Effects of consumers and health providers working in partnership on health services planning, delivery and evaluation. Cochrane Database Syst Rev. 2021;9:CD013373.

PubMed   Google Scholar  

Price A, Albarqouni L, Kirkpatrick J, Clarke M, Liew SM, Roberts N, et al. Patient and public involvement in the design of clinical trials: An overview of systematic reviews. J Eval Clin Pract. 2018;24(1):240–53.

Sarrami-Foroushani P, Travaglia J, Debono D, Braithwaite J. Implementing strategies in consumer and community engagement in health care: results of a large-scale, scoping meta-review. BMC Health Serv Res. 2014;14:402.

Abrams R, Park S, Wong G, Rastogi J, Boylan A-M, Tierney S, et al. Lost in reviews: Looking for the involvement of stakeholders, patients, public and other non-researcher contributors in realist reviews. Research Synthesis Methods. 2021;12(2):239–47.

Zych MM, Berta WB, Gagliardi AR. Conceptualising the initiation of researcher and research user partnerships: a meta-narrative review. Health Res Policy Syst. 2020;18(1):24.

Gates M, Gates A, Pieper D, Fernandes RM, Tricco AC, Moher D, et al. Reporting guideline for overviews of reviews of healthcare interventions: development of the PRIOR statement. BMJ. 2022;378:e070849.

Pollock A, Campbell P, Brunton G, Hunt H, Estcourt L. Selecting and implementing overview methods: implications from five exemplar overviews. Syst Rev. 2017;6(1):145.

Greenhalgh T, Hinton L, Finlay T, Macfarlane A, Fahy N, Clyde B, et al. Frameworks for supporting patient and public involvement in research: Systematic review and co-design pilot. Health Expectations: An International Journal of Public Participation in Health Care & Health Policy. 2019;22(4):785–801.

Article   Google Scholar  

Hughes M, Duffy C. Public involvement in health and social sciences research: A concept analysis. Health Expectations: An International Journal of Public Participation in Health Care & Health Policy. 2018;21(6):1183–90.

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2008;3(2):77–101.

Waggoner J, Carline JD, Durning SJ. Is There a Consensus on Consensus Methodology? Descriptions and Recommendations for Future Consensus Research. Acad Med. 2016;91(5):663–8.

Harvey N, Holmes CA. Nominal group technique: an effective method for obtaining group consensus. Int J Nurs Pract. 2012;18(2):188–94.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71.

Baldwin JN, Napier S, Neville S, Clair VAWS. Impacts of older people’s patient and public involvement in health and social care research: a systematic review. Age Ageing. 2018;47(6):801–9.

Bench S, Eassom E, Poursanidou K. The nature and extent of service user involvement in critical care research and quality improvement: A scoping review of the literature. Int J Consum Stud. 2018;42(2):217–31.

Bethell J, Commisso E, Rostad HM, Puts M, Babineau J, Grinbergs-Saull A, et al. Patient engagement in research related to dementia: a scoping review. Dementia. 2018;17(8):944–75.

Brett J, Staniszewska S, Mockford C, Herron-Marx S, Hughes J, Tysall C, et al. A systematic review of the impact of patient and public involvement on service users, researchers and communities. The Patient: Patient-Centered Outcomes Research. 2014;7(4):387–95.

Di Lorito C, Birt L, Poland F, Csipke E, Gove D, Diaz-Ponce A, et al. A synthesis of the evidence on peer research with potentially vulnerable adults: how this relates to dementia. Int J Geriatr Psychiatry. 2017;32(1):58–67.

Di Lorito C, Bosco A, Birt L, Hassiotis A. Co-research with adults with intellectual disability: A systematic review. J Appl Res Intellect Disabil. 2018;31(5):669–86.

Fox G, Fergusson DA, Daham Z, Youssef M, Foster M, Poole E, et al. Patient engagement in preclinical laboratory research: A scoping review. EBioMedicine. 2021;70:103484.

Frankena TK, Naaldenberg J, Cardol M, Linehan C, van Schrojenstein Lantman-de Valk H. Active involvement of people with intellectual disabilities in health research - a structured literature review. Res Dev Disabil. 2015;45–46:271–83.

Fudge N, Wolfe CD, McKevitt C. Involving older people in health research. Age Ageing. 2007;36(5):492–500.

Article   CAS   PubMed   Google Scholar  

George AS, Mehra V, Scott K, Sriram V. Community participation in health systems research: A systematic review assessing the state of research, the nature of interventions involved and the features of engagement with communities. PLoS One. 2015;10(10):ArtID e0141091.

Legare F, Boivin A, van der Weijden T, Pakenham C, Burgers J, Legare J, et al. Patient and public involvement in clinical practice guidelines: A knowledge synthesis of existing programs. Med Decis Making. 2011;31(6):E45–74.

McCarron TL, Clement F, Rasiah J, Moran C, Moffat K, Gonzalez A, et al. Patients as partners in health research: a scoping review. Health Expect. 2021;24:1378 No Pagination Specified.

Miller J, Knott V, Wilson C, Roder D. A review of community engagement in cancer control studies among indigenous people of Australia, New Zealand, Canada and the USA. Eur J Cancer Care. 2012;21(3):283–95.

Article   CAS   Google Scholar  

Shen S, Doyle-Thomas KAR, Beesley L, Karmali A, Williams L, Tanel N, et al. How and why should we engage parents as co-researchers in health research? A scoping review of current practices. Health Expect. 2017;20(4):543–54.

Velvin G, Hartman T, Bathen T. Patient involvement in rare diseases research: a scoping review of the literature and mixed method evaluation of Norwegian researchers’ experiences and perceptions. Orphanet J Rare Dis. 2022;17(1):212.

Wiles LK, Kay D, Luker JA, Worley A, Austin J, Ball A, et al. Consumer engagement in health care policy, research and services: A systematic review and meta-analysis of methods and effects. PLoS One. 2022;17:e0261808.no pagination

Cook N, Siddiqi N, Twiddy M, Kenyon R. Patient and public involvement in health research in low and middle-income countries: a systematic review. BMJ Open. 2019;9(5):e026514.

Chambers E, Gardiner C, Thompson J, Seymour J. Patient and carer involvement in palliative care research: An integrative qualitative evidence synthesis review. Palliat Med. 2019;33(8):969–84.

Brett J, Staniszewska S, Mockford C, Herron-Marx S, Hughes J, Tysall C, et al. Mapping the impact of patient and public involvement on health and social care research: a systematic review. Health Expect. 2014;17(5):637–50.

Boaz A, Hanney S, Jones T, Soper B. Does the engagement of clinicians and organisations in research improve healthcare performance: a three-stage review. BMJ Open. 2015;5(12):e009415.

Boivin A, L’Esperance A, Gauvin FP, Dumez V, Macaulay AC, Lehoux P, et al. Patient and public engagement in research and health system decision making: A systematic review of evaluation tools. Health Expect. 2018;21(6):1075–84.

Anderst A, Conroy K, Fairbrother G, Hallam L, McPhail A, Taylor V. Engaging consumers in health research: a narrative review. Aust Health Rev. 2020;44(5):806–13.

Arnstein L, Wadsworth AC, Yamamoto BA, Stephens R, Sehmi K, Jones R, et al. Patient involvement in preparing health research peer-reviewed publications or results summaries: a systematic review and evidence-based recommendations. Res Involv Engagem. 2020;6:34.

Becerril-Montekio V, Garcia-Bello LA, Torres-Pereda P, Alcalde-Rabanal J, Reveiz L, Langlois EV. Collaboration between health system decision makers and professional researchers to coproduce knowledge, a scoping review. Int J Health Plann Manage. 2022;28:28.

Google Scholar  

Bird M, Ouellette C, Whitmore C, Li L, Nair K, McGillion MH, et al. Preparing for patient partnership: A scoping review of patient partner engagement and evaluation in research. Health Expect. 2020;23:523 No Pagination Specified.

Dawson S, Campbell SM, Giles SJ, Morris RL, Cheraghi-Sohi S. Black and minority ethnic group involvement in health and social care research: A systematic review. Health Expect. 2018;21(1):3–22.

Harris J, Haltbakk J, Dunning T, Austrheim G, Kirkevold M, Johnson M, et al. How patient and community involvement in diabetes research influences health outcomes: A realist review. Health Expectations: An International Journal of Public Participation in Health Care & Health Policy. 2019;22(5):907–20.

Hubbard G, Kidd L, Donaghy E. Involving people affected by cancer in research: a review of literature. Eur J Cancer Care. 2008;17(3):233–44.

Hubbard G, Kidd L, Donaghy E, McDonald C, Kearney N. A review of literature about involving people affected by cancer in research, policy and planning and practice. Patient Educ Couns. 2007;65(1):21–33.

Jones EL, Williams-Yesson BA, Hackett RC, Staniszewska SH, Evans D, Francis NK. Quality of reporting on patient and public involvement within surgical research: a systematic review. Ann Surg. 2015;261(2):243–50.

Drahota A, Meza RD, Brikho B, Naaf M, Estabillo JA, Gomez ED, et al. Community-Academic Partnerships: A Systematic Review of the State of the Literature and Recommendations for Future Research. Milbank Q. 2016;94(1):163–214.

Forsythe LP, Szydlowski V, Murad MH, Ip S, Wang Z, Elraiyah TA, et al. A systematic review of approaches for engaging patients for research on rare diseases. J Gen Intern Med. 2014;29(Suppl 3):788–800.

Article   PubMed Central   Google Scholar  

Lander J, Hainz T, Hirschberg I, Strech D. Current practice of public involvement activities in biomedical research and innovation: a systematic qualitative review. PLoS One [Electronic Resource]. 2014;9(12):e113274.

Lee DJ, Avulova S, Conwill R, Barocas DA. Patient engagement in the design and execution of urologic oncology research. Urol Oncol. 2017;35(9):552–8.

Malterud K, Elvbakken KT. Patients participating as co-researchers in health research: A systematic review of outcomes and experiences. Scandinavian Journal of Public Health. 2020;48(6):617–28.

Miah J, Dawes P, Edwards S, Leroi I, Starling B, Parsons S. Patient and public involvement in dementia research in the European Union: a scoping review. BMC geriatr. 2019;19(1):220.

Nilsen ES, Myrhaug HT, Johansen M, Oliver S, Oxman AD. Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material. Cochrane Database Syst Rev. 2006;2006(3):CD004563.

PubMed   PubMed Central   Google Scholar  

Oliver S, Clarke-Jones L, Rees R, Milne R, Buchanan P, Gabbay J, et al. Involving consumers in research and development agenda setting for the NHS: developing an evidence-based approach. Health Technol Assess. 2004;8(15):1-148 III-IV.

Orlowski SK, Lawn S, Venning A, Winsall M, Jones GM, Wyld K, et al. Participatory Research as One Piece of the Puzzle: A Systematic Review of Consumer Involvement in Design of Technology-Based Youth Mental Health and Well-Being Interventions. JMIR Hum Factors. 2015;2(2):e12.

Pii KH, Schou LH, Piil K, Jarden M. Current trends in patient and public involvement in cancer research: A systematic review. Health Expectations: An International Journal of Public Participation in Health Care & Health Policy. 2019;22(1):3–20.

Sandoval JA, Lucero J, Oetzel J, Avila M, Belone L, Mau M, et al. Process and outcome constructs for evaluating community-based participatory research projects: a matrix of existing measures. Health Educ Res. 2012;27(4):680–90.

Sangill C, Buus N, Hybholt L, Berring LL. Service user’s actual involvement in mental health research practices: A scoping review. Int J Ment Health Nurs. 2019;28(4):798–815.

Schelven F, Boeije H, Marien V, Rademakers J. Patient and public involvement of young people with a chronic condition in projects in health and social care: A scoping review. Health Expect. 2020;23:789 No Pagination Specified.

Schilling I, Gerhardus A. Methods for Involving Older People in Health Research-A Review of the Literature. International Journal of Environmental Research & Public Health [Electronic Resource]. 2017;14(12):29.

Shippee ND, Domecq Garces JP, Prutsky Lopez GJ, Wang Z, Elraiyah TA, Nabhan M, et al. Patient and service user engagement in research: a systematic review and synthesized framework. Health Expect. 2015;18(5):1151–66.

Vaughn LM, Jacquez F, Lindquist-Grantz R, Parsons A, Melink K. Immigrants as research partners: A review of immigrants in community-based participatory research (CBPR). J Immigr Minor Health. 2017;19(6):1457–68.

Walmsley J, Strnadova I, Johnson K. The added value of inclusive research. J Appl Res Intellect Disabil. 2018;31(5):751–9.

Weschke S, Franzen DL, Sierawska AK, Bonde LS, Strech D. Schorr SG. Reporting of patient involvement: A mixed-methods analysis of current practice in health research publications. medRxiv; 2022. p. 21.

Gradinger F, Britten N, Wyatt K, Froggatt K, Gibson A, Jacoby A, et al. Values associated with public involvement in health and social care research: A narrative review. Health Expectations: An International Journal of Public Participation in Health Care & Health Policy. 2015;18(5):661–75.

Hoekstra F, Mrklas KJ, Khan M, McKay RC, Vis-Dunbar M, Sibley KM, et al. A review of reviews on principles, strategies, outcomes and impacts of research partnerships approaches: A first step in synthesising the research partnership literature. Health Res Pol Syst. 2020;18(1):51 no pagination.

Stallings SC, Boyer AP, Joosten YA, Novak LL, Richmond A, Vaughn YC, et al. A taxonomy of impacts on clinical and translational research from community stakeholder engagement. Health Expect. 2019;22(4):731–42.

Grindell C, Coates E, Croot L, O’Cathain A. The use of co-production, co-design and co-creation to mobilise knowledge in the management of health conditions: a systematic review. BMC Health Serv Res. 2022;22(1):877.

Staley K. “Is it worth doing?” Measuring the impact of patient and public involvement in research. Res Involv Engagem. 2015;1:6.

Download references

Acknowledgements

The authors would like to thank the graphic designers, Jenni Quinn and Kevin Calthorpe, for their work on Fig. 3 .

Not applicable.

Author information

Authors and affiliations.

School of Health Sciences, The University of Melbourne, Melbourne, Australia

Sanne Peters, Jill Francis, Stephanie Best, Stephanie Rowe & Marlena Klaic

Department of Health Services Research, Peter MacCallum Cancer Centre, Melbourne, Australia

Lisa Guccione, Jill Francis & Stephanie Best

Sir Peter MacCallum Department of Oncology, Faculty of Medicine, Dentistry and Health Sciences, The University of Melbourne, Melbourne, Australia

Centre for Implementation Research, Ottawa Hospital Research Institute, Ottawa, Canada

Jill Francis

Victorian Comprehensive Cancer Centre, Melbourne, VIC, Australia

Stephanie Best

Emergency Research, Murdoch Children’s Research Institute, Melbourne, Australia

Emma Tavender

Department of Critical Care, The University of Melbourne , Melbourne, Australia

School of Nursing, Faculty of Health, Ottawa, Canada

Janet Curran & Stephanie Rowe

Emergency Medicine, Faculty of Medicine, Ottawa, Canada

Janet Curran

Neurological Rehabilitation Group Mount Waverley, Mount Waverley, Australia

Katie Davies

The ALIVE National Centre for Mental Health Research Translation, The University of Melbourne, Melbourne, Australia

Victoria J. Palmer

You can also search for this author in PubMed   Google Scholar

Contributions

SP coordinated the authorship team, completed the systematic literature searches, synthesis of data, framework design and substantial writing. MK and LG were the second reviewers for the systematic overview of reviews. MK, LG and JF assisted with framework design.  SP, LG, JF, SB, ET, JC, KD, SR, VP and MK participated in the stakeholder meetings and the consensus process. All authors commented on drafts and approved the final submitted version of the manuscript.

Corresponding author

Correspondence to Sanne Peters .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., supplementary material 4., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Peters, S., Guccione, L., Francis, J. et al. Evaluation of research co-design in health: a systematic overview of reviews and development of a framework. Implementation Sci 19 , 63 (2024). https://doi.org/10.1186/s13012-024-01394-4

Download citation

Received : 01 April 2024

Accepted : 31 August 2024

Published : 11 September 2024

DOI : https://doi.org/10.1186/s13012-024-01394-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Research co-design
• Stakeholder involvement
• End-user engagement
• Consumer participation
• Outcome measures

Implementation Science

ISSN: 1748-5908

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 09 September 2024

Navigating post-pandemic challenges through institutional research networks and talent management

• Muhammad Zada   ORCID: orcid.org/0000-0003-0466-4229 1 , 2 ,
• Imran Saeed 3 ,
• Jawad Khan   ORCID: orcid.org/0000-0002-6673-7617 4 &
• Shagufta Zada 5 , 6  

Humanities and Social Sciences Communications volume  11 , Article number:  1164 ( 2024 ) Cite this article

Metrics details

• Business and management

Institutions actively seek global talent to foster innovation in the contemporary landscape of scientific research, education, and technological progress. The COVID-19 pandemic underscored the importance of international collaboration as researchers and academicians faced limitations in accessing labs and conducting research experiments. This study uses a research collaboration system to examine the relationship between organizational intellectual capital (Human and structural Capital) and team scientific and technological performance. Further, this study underscores the moderating role of top management support. Using a time-lagged study design, data were collected from 363 participants in academic and research institutions. The results show a positive relationship between organizational intellectual capital (Human and structural Capital) and team scientific and technological performance using a research collaboration system. Moreover, top management support positively moderates the study’s hypothesized relationships. The study’s findings contribute significantly to existing knowledge in this field, with implications for academia, researchers, and government focused on technology transmission, talent management, research creative collaboration, supporting innovation, scientific research, technological progress, and preparing for future challenges.

Similar content being viewed by others

Towards understanding the characteristics of successful and unsuccessful collaborations: a case-based team science study

Remote collaboration fuses fewer breakthrough ideas

Interpersonal relationships drive successful team science: an exemplary case-based study

Introduction.

Global talent management and the talent hunt within research and educational institutions have become extensively discussed topics in international human resource management (HRM) (Al et al., 2022 ). Global talent management is intricately connected to the notion of finding, managing, and facilitating the fetch of research, skills, techniques, and knowledge among team members and progress in education and technology (Kwok, 2022 ; Sommer et al., 2017 ). This topic assumes a greater position when it is looked at through the lens of research, academicians, and educational institutions serving as a means of achieving scientific and technological advancement and performance (Kaliannan et al., 2023 ; Patnaik et al., 2022 ). Effective knowledge management and transfer occur between teams engaged in cross-border research collaborations (Davenport et al., 2002 ; Fasi, 2022 ). Effective team management, global talent recruitment, and the exchange of scientific knowledge across national boundaries face different challenges due to the swift growth of economic and political fanaticism. This is particularly evident in advanced economies that rely heavily on knowledge-based industries (Vaiman et al., 2018 ). Research and educational sectors are encountering significant challenges in effectively hunting and managing international talent, particularly in the aftermath of the COVID-19 pandemic, during which approximately half of the global workforce faced the possibility of job loss (Almeida et al., 2020 ; Radhamani et al., 2021 ). Due to the implementation of lockdown measures by governments, many research intuitions are facing significant issues, and the pandemic has changed the situation; work was stuck, and scientists around the globe are thinking to be prepared for this kind of situation, which is possible through the use scientific research collaboration platforms. These platforms serve as a means to exchange research and knowledge, which is crucial in the talent hunt and management (Haak-Saheem, 2020 ). In the situation above, wherein limitations exist regarding the exchange of research and knowledge within the institutions, it becomes imperative for the top management of institutions to incentivize employees to engage the team in knowledge sharing actively and achieve team-level scientific and technological advancement. It can be achieved by implementing a research collaboration system that facilitates knowledge exchange and contributes to effective talent hunt and management (Haider et al., 2022 ; Xu et al., 2024 ).

A research collaboration network is a tool for scientific and technological advancement and talent management encompassing various processes and practices to facilitate the sharing, integration, translation, and transformation of scientific knowledge (Biondi & Russo, 2022 ). During and after the COVID-19 era characterized by travel restrictions, research networking platforms serve as valuable tools for students and researchers located in variance regions to engage in the exchange of research knowledge and achieve team-level scientific and technological advancement (Yang et al., 2024 ). Enhancing intellectual capital (IC) within the organizations is imperative within this framework (Pellegrini et al., 2022 ; Vătămănescu et al., 2023 ). Intellectual capital (IC) is the intangible assets owned by an organization that has the potential to generate value (Stewart, 1991 ). An organization’s intellectual capital (IC) includes human and structural capital (Marinelli et al., 2022 ). According to Vătămănescu et al. ( 2023 ), the organization can effectively manage the skills and abilities of its team members across different countries by properly utilizing both human and structural capital and establishing a strong research collaboration system with the help of top management support. This capability remains intact even during and after the COVID-19 pandemic. This study emphasizes the importance of talent hunt and management within research and educational institutions in the post-COVID-19 pandemic because of every country’s following implementation of lockdown measures. Our study focuses on the implication of facilitating the exchange of research, knowledge, and techniques among team members during and after this period. The effective way to share research expertise and techniques in such a scenario is through a research collaboration network (O’Dwyer et al., 2023 ).

While previous research has extensively explored talent management in various industries (Al Ariss, Cascio, & Paauwe, 2014 ; Susanto, Sawitri, Ali, & Rony, 2023 ), a noticeable gap exists in the body of knowledge regarding the discussion of global talent acquisition and management within research and academic institutions, particularly within volatile environments and about scientific and technological advancements (Harsch & Festing, 2020 ). The objective of this research is to fill this research gap.1) To investigate the strategies of how research and educational institutions hunt and manage gobble talent. 2)To analyze the impact of human and structural capital and team scientific and technological performance using a research collaboration system. 3) To examine the moderating effect of top management support on the IC to use the research collation network among institution research teams and scientific and technological performance.

In addition, current research contributes significantly to the literature by elucidating the pivotal role of organizational intellectual capital in strengthening scientific and technological performance through research collaborative networks. This study advances our grip on how internal resources drive innovation and research outcomes by empirically demonstrating the positive association between human and structural capital and team-level scientific and technological performance. Furthermore, the current study highlights the moderating effect of top management support, suggesting that management commitment can amplify the benefits of intellectual capital (human and structural capital). These results show a subtle perspective on how organizations can influence their intellectual assets to foster higher levels of productivity and innovation. The study’s theoretical contributions lie in integrating resource-based views and organizational theory with performance metrics, while its practical implications provide actionable insights for institutions aiming to optimize their intellectual resources and management practices. This research also sets the stage for future inquiries into the dynamics of intellectual capital and management support in various collaborative contexts.

Research theories, literature review, and hypotheses development

Research theories.

The focus of the current study pertains to the challenges surrounding talent management within institutions during and after the COVID-19 pandemic(Fernandes et al., 2023 ). Global talent management is intently linked to the objective of enhancing the intellectual capital of the organization (Zada et al., 2023 ). Considering the COVID-19 pandemic, which raised much more attention toward scientific and technological advancement, the academic sector has noticed an observable shift towards utilizing research collaboration platforms to share scientific knowledge effectively and achieve scientific and technological performance. Intellectual capital encompasses five distinct resource categories, as identified by Roos and Roos ( 1997 ), comprising three immaterial and two touchable resources. Intangible resources such as human capital, structural capital, and customer capital are complemented by tangible resources, encompassing monetary and physical assets. Global talent management encompasses human and structural capital management (Felin & Hesterly, 2007 ). The enhancement of talent management capabilities within the institution can be achieved by cultivating institution-specific competencies in both human and structural capital (Al Ariss et al., 2014 ). This concept lines up with the theoretical background of the resource-based view (RBV) theory presented by Barney ( 1991 ). According to this theory, organizations should prioritize examining their core resources to recognize valuable assets, competencies, and capabilities that can contribute to attaining a sustainable competitive advantage (Barney, 1991 ).

During and after the COVID-19 scenario, virtual platforms are utilized by institutions to engage students and staff abroad in research and knowledge exchange, which is part of global talent management. Staff possessing adequate knowledge repositories will likely participate in knowledge exchange activities. Therefore, organizations must improve their internal resources to enhance talent management, as per the fundamental principle of the RBV theory (Barney, 1991 ). Enhancing internal resources entails strengthening an organization’s human capital, which refers to its staff’s scientific research and technical skills and knowledge and structural capital. Strengthening these two resources can facilitate the institution in effectively sharing knowledge through a research collaboration platform, consequently enhancing their global talent management endeavors and contributing to the team’s scientific and technological performance.

In this research, we also utilize institutional theory (Oliver, 1997 ) and Scott ( 2008 ) as a framework to examine the utilization of research collaboration social platforms by faculty of institutions. Our focus is on exchanging research and technical knowledge within the climate of global talent management during and after the COVID-19 epidemic. According to Scott ( 2008 ), “Institutional theory is a widely recognized theoretical framework emphasizing rational myths, isomorphism, and legitimacy (p. 78)”. For electronic data interchange, the theory has been utilized in technology adoption research (Damsgaard, Lyytinen ( 2001 )) and educational institutes (J. et al., 2007 ). In the pandemic situation, institutional theory provides researchers with a framework to analyze the motivations of employees within institutions to engage in teams to achieve team-level scientific and technological performance through a research collaboration system. According to institutional theory, organizations should utilize a research collaboration network to ensure that their staff do not need to compromise their established norms, values, and expectations. During the COVID-19 pandemic, numerous countries implemented limitations on international movement as a preventive measure. Consequently, there has been a growing identification of the potential importance of utilizing an institutional research collaboration platform for facilitating the online exchange of knowledge, skills, research techniques, and global talent management among employees of institutions operating across various countries. The active support of staff by the top management of an institution can play a key role in expediting the implementation of social networks for research collaboration within the institution (Zada et al., 2023 ).

Literature review

An institution’s scientific and technological advancement is contingent upon optimal resource utilization (Muñoz et al., 2022 ). Global talent hunt and management encompasses utilizing information and communication technologies (ICT) to provide a way for the exchange of research knowledge and techniques, thereby enabling the implementation of knowledge-based strategies (Muñoz et al., 2022 ). In a high research-level turbulent environment, it becomes imperative to effectively manage human capital (HUC) to facilitate the appropriate exchange of research knowledge and techniques (Salamzadeh, Tajpour, Hosseini, & Brahmi, 2023 ). Research shows that transferring research knowledge and techniques across national boundaries, exchanging best practices, and cultivating faculty skills are crucial factors in maintaining competitiveness (Farahian, Parhamnia, & Maleki, 2022 ; Shao & Ariss, 2020 ).

It is widely acknowledged in scholarly literature that there is a prevailing belief among individuals that talent possesses movability and that research knowledge and techniques can be readily transferred (Bakhsh et al., 2022 ; Council, 2012 ). However, it is essential to note that the matter is more complex than it may initially appear (Biondi & Russo, 2022 ). The proliferation of political and economic nationalism in developed knowledge-based economies poses a significant risk to exchanging research knowledge and techniques among faculty members in research and educational institutions worldwide (Arocena & Sutz, 2021 ). During and after COVID-19, knowledge transfer can be effectively facilitated by utilizing a research collaboration network platform (Duan & Li, 2023 ; Sulaiman et al., 2022 ). This circumstance is noticeable within the domain of international research and development, wherein academic professionals have the opportunity to utilize research collaboration platforms as a means of disseminating valuable research knowledge and techniques to their counterparts in various nations (Jain et al., 2022 ).

The scientific and technological advancement of institutions linked by intuition research and development level and research and development depend on the intuition’s quality of research, knowledge, and management (Anshari & Hamdan, 2022 ). However, there is a need to enhance the research team’s capacity to learn and transfer research knowledge and techniques effectively. Research suggests that institutional human capital (HUC) is critical in managing existing resources and hunting international talent, particularly after the COVID-19 pandemic (Sigala, Ren, Li, & Dioko, 2023 ). Human capital refers to the combined implicit and crystal clear knowledge of employees within an institution and their techniques and capabilities to effectively apply this knowledge to achieve scientific and technological advancements (Al-Tit et al., 2022 ). According to Baron and Armstrong ( 2007 ) Human capital refers to the abilities, knowledge, techniques, skills, and expertise of individuals, particularly research team members, that are relevant to the current task.

Furthermore, HUC encompasses the scope of individuals who can contribute to this reservoir of research knowledge, techniques, and expertise through individual learning. As the literature shows, the concept of IC encompasses the inclusion of structural capital (STC), which requires fortification through the implementation of a proper global talent acquisition and management system (Pak et al., 2023 ; Phan et al., 2020 ). STC encompasses various mechanisms to enhance an institution’s performance and productivity (Barpanda, 2021 ). STC is extensively acknowledged as an expedited framework for HUC, as discussed by Bontis ( 1998 ) and further explored by Gogan, Duran, and Draghici ( 2015 ). During and after the COVID-19 epidemic, a practical approach to global talent management involves leveraging research collaboration network platforms to facilitate knowledge exchange among research teams (Arslan et al., 2021 ). However, the crucial involvement of top management support is imperative to effectively manage talent by utilizing research collaboration network platforms for knowledge transfer (Zada et al., 2023 ). Nevertheless, the existing body of knowledge needs to adequately explore the topic of talent management about knowledge transfer on research collaboration platforms, particularly in the context of institution-active management support (Tan & Md. Noor, 2013 ).

Conceptual model and research hypothesis

By analyzing pertinent literature and theoretical frameworks, we have identified the factors influencing staff intention in research and academic institutions to utilize research collaboration networks after the COVID-19 pandemic and achieve scientific and technical performance. This study aims to explain the determinants. Additionally, this study has considered the potential influence of top management support as a moderator on the associations between education and research institution staff intention on IC to utilize research collaboration platforms in the post-COVID-19 era and predictors. Through this discourse, we shall generate several hypotheses to serve as the basis for constructing a conceptual model (see Fig. 1 ).

Relationships between study variables: human capital, structural capital, top management support, and team scientific and technological performance. Source: authors’ development.

Human capital and team scientific and technological performance

According to Dess and Picken ( 2000 ), HUC encompasses individuals’ capabilities, knowledge, skills, research techniques, and experience, including staff and supervisors, relevant to the specific task. Human capital also refers to the ability to pay to this reservoir of knowledge, techniques, and expertize through individual learning (Dess & Picken, 2000 ). HUC refers to the combinations of characteristics staff possess, including but not limited to research proficiency, technical aptitude, business acumen, process comprehension, and other similar competencies (Kallmuenzer et al., 2021 ). The HUC is considered an institutional repository of knowledge, as Bontis and Fitz‐enz ( 2002 ) indicated, with its employees serving as representatives. The concept of HUC refers to the combined abilities, research proficiency, and competencies that individuals possess to address and resolve operational challenges within an institutional setting (Barpanda, 2021 ; Yang & Xiangming, 2024 ). The human capital possessed by institutions includes crucial attributes that allow organizations to acquire significant internal resources that are valuable, difficult to replicate, scarce, and cannot be substituted. It aligns with the theoretical framework of the RBV theory, as suggested by Barney ( 1991 ). IC is extensively recognized as a main factor in revitalizing organizational strategy and promoting creativity and innovation. It is crucial to enable organizations to acquire and effectively disseminate knowledge among their employees, contribute to talent management endeavors, and achieve scientific and technological performance (Alrowwad et al., 2020 ; He et al., 2023 ). Human capital is linked to intrinsic aptitude, cognitive capabilities, creative problem-solving, exceptional talent, and the capacity for originality (Bontis & Fitz‐enz, 2002 ). In talent management, there is a focus on enhancing scientific and technological performance and development. According to Shao and Ariss ( 2020 ), HUC is expected to strengthen employee motivation to utilize research collaboration networks for scientific knowledge-sharing endeavors. Based on these arguments, we proposed that.

Hypothesis 1 Human capital (HUC) positively impacts team scientific and technological performance using a research collaboration system.

Structural capital and team scientific and technological Performance

According to Mehralian, Nazari, and Ghasemzadeh ( 2018 ) structural capital (STC) encompasses an organization’s formalized knowledge assets. It consists of the structures and mechanisms employed by the institution to enhance its talent management endeavors. The concept of STC is integrated within the framework of institutions’ programs, laboratory settings, and databases (Cavicchi & Vagnoni, 2017 ). The significance of an organization’s structural capital as an internal tangible asset that bolsters its human capital has been recognized by scholars such as Secundo, Massaro, Dumay, and Bagnoli ( 2018 ), and This concept also lines up with the RBV theory (J. Barney, 1991 ). The strategic assets of an organization encompass its capabilities, organizational culture, patents, and trademarks (Gogan et al., 2015 ).

Furthermore, Birasnav, Mittal, and Dalpati ( 2019 ) Suggested that these strategic assets promote high-level organizational performance, commonly called STC. Literature shows that STC encompasses an organization’s collective expertise and essential knowledge that remains intact even when employees depart (Alrowwad et al., 2020 ; Mehralian et al., 2018 ; Sarwar & Mustafa, 2023 ). The institution’s socialization, training, and development process facilitates the transfer of scientific research knowledge, skills, and expertise to its team (Arocena & Sutz, 2021 ; Marchiori et al., 2022 ). The STC is broadly recognized as having important potential and is a highly productive resource for generating great value. STC motivates its team member to share expertise with their counterparts at subordinate organizations by utilizing an institution’s research collaboration network and achieving team-level scientific and technological performance. This method remains effective even in challenging environments where traditional means of data collection, face-to-face meetings, and travel are not feasible (Secundo et al., 2016 ). In light of the above literature and theory, we propose the following hypothesis.

Hypothesis 2: Structural capital (STC) positively impacts team scientific and technological performance using a research collaboration system.

Top management support as a moderator

If the relationship between two constructs is not constant, the existence of a third construct can potentially affect this relationship by enhancing or diminishing its strength. In certain cases, the impact of a third construct can adjust the trajectory of the relationship between two variables. The variable in question is commonly called the “moderating variable.” According to Zada et al. ( 2023 ), top management support to leaders efficiently encourages team members within institutions to share research scientific knowledge with their counterparts in different countries through international research collaboration systems. Similarly, another study shows that the active endorsement of the top management significantly affects the development of direct associations, thereby influencing the team and organization’s overall performance (Biondi & Russo, 2022 ; Phuong et al., 2024 ). Different studies have confirmed that top management support is crucial in fostering a conducive knowledge-sharing environment by offering necessary resources (Ali et al., 2021 ; Lee et al., 2016 ; Zada et al., 2023 ). During and after the COVID-19 epidemic, numerous nations implemented nonessential travel restrictions and lockdown measures. In the given context, utilizing a research collaboration system would effectively facilitate the exchange of research, skills, and knowledge among staff belonging to various subsidiaries of an institution (Rådberg & Löfsten, 2024 ; Rasheed et al., 2024 ). However, it is common for researchers to exhibit resistance to adopting a novel research technique, often citing various justifications for their reluctance. To address the initial hesitance of employees at subsidiary institutes towards utilizing research collaborative networking within the institute, top management must employ strategies that foster motivation, encouragement, and incentives. These measures help create an atmosphere where team members feel empowered to engage with the new system freely. Institutional theory asserts that top management support is crucial for aligning talent management with institutional norms. Human and structural capital, pivotal within the institutional framework, contributes to an institution’s capacity to attract and retain talent, enhancing legitimacy. Adaptation to scientific and technological advancements is imperative for international institutional competitiveness, as institutional theory dictates (Oliver, 1997 ). Grounded on the above discussion, we have hypothesized.

Hypothesis 3a : Top management support moderates the relationship between human capital (HUC) and team scientific and technological performance. Specifically, this relationship will be stronger for those with higher top management support and weaker for those with lower top management support.

Hypothesis 3b : Top management support moderates the relationship between structural capital (STC) and team scientific and technological performance through the use of research collaboration network platforms. Specifically, this relationship will be stronger for those with higher top management support and weaker for those with lower top management support.

Methods data and sample

Sample and procedures.

To test the proposed model, we collected data from respondents in China’s research and academic sector in three phases to mitigate standard method variance (Podsakoff, MacKenzie, Lee, & Podsakoff, 2003 ). In the first phase (T1-phase), respondents rated human capital, structural capital, and demographic information. After one month, respondents rated the team’s scientific and technological performance in the second phase (T2-phase). Following another one-month interval, respondents were asked to rate top management support in the third phase (T3-phase). In the first phase, after contacting 450 respondents, we received 417 usable questionnaires (92.66%). In the second phase, we received 403 usable questionnaires. In the third phase, we received 363 usable questionnaires (90.07%), constituting our final sample for interpreting the results. The sample comprises 63.4% male and 36.6% female respondents. The age distribution of the final sample was as follows: 25–30 years old (6.6%), 31–35 years old (57%), 36–40 years old (19.8%), and above 40 years old (16.5%). Regarding respondents’ experience, 45.7% had 1–5 years, 39.4% had 6–10 years, 11.3% had 11–15 years, and 3.6% had over 16 years. According to the respondents’ levels of education, 4.1% had completed bachelor’s degrees, 11.6% had earned master’s degrees, 78.8% were doctorate (PhD) scholars, and 5.5% were postdoctoral and above.

Measurement

To measure the variables, the current study adopted a questionnaire from previous literature, and age, gender, education, and experience were used as control variables. A five-point Likert scale was used (1 = strongly disagree to 5 = strongly agree). Human capital (HUC) was measured through an eight-item scale adopted by Kim, Atwater, Patel, and Smither ( 2016 ). The sample item is “The extent to which human capital of research and development department is competitive regarding team performance”. The self-reported scale developed by Nezam, Ataffar, Isfahani, and Shahin ( 2013 ) was adopted to measure structural capital. The scale consists of seven items. The sample scale item is “My organization emphasizes IT investment.” In order to measure top management support, a six-item scale was developed by Singh, Gupta, Busso, and Kamboj ( 2021 ), was adopted, and sample item includes “Sufficient incentives were provided by top management (TM) for achieving scientific and technological performance.” Finlay, the self-reported scale developed by Gonzalez-Mulé, Courtright, DeGeest, Seong, and Hong ( 2016 ) was adopted to gauge team scientific and technological performance and scales items are four. The sample item is “This team achieves its goals.”

Assessment of measurement model

In the process of employing AMOS for analysis, the initial step encompasses an assessment of the model to determine the strength and validity of the study variables. The evaluation of variable reliability conventionally revolves around two key aspects, which are indicator scale reliability and internal reliability. More precisely, indicator reliability is deemed to be recognized when factor loadings exceed the threshold of 0.60. In parallel, internal consistency reliability is substantiated by the attainment of values exceeding 0.70 for both Cronbach’s alpha and composite reliability, aligning with well-established and recognized guidelines (Ringle et al., 2020 ).

To gauge the reliability of construct indicators, we utilized two key metrics which are composite reliability (CR) and average variance extracted (AVE). The CR values for all variables were notably high, exceeding 0.70 and falling within the range of 0.882 to 0.955. This signifies a robust level of reliability for the indicators within each construct. Furthermore, the AVE values, which indicate convergent validity, exceeded the minimum threshold of 0.50, with each construct value varying from 0.608 to 0.653, thus affirming the presence of adequate convergent validity.

In addition to assessing convergent validity, we also examined discriminant validity by scrutinizing the cross-loadings of indicators on the corresponding variables and the squared correlations between constructs and AVE values. Our findings indicated that all measures exhibited notably stronger loadings on their intended constructs, thereby underscoring the measurement model’s discriminant validity.

Discriminant validity was recognized by observing average variance extracted (AVE) values that exceeded the squared correlations between constructs, as indicated in Table 1 . In conjunction with the Composite Reliability (CR) and AVE values, an additional discriminant validity assessment was conducted through a Heterotrait-Monotrait Ratio (HTMT) analysis. This analysis entailed a comparison of inter-construct correlations against a predefined upper threshold of 0.85. The results demonstrated that all HTMT values remained significantly below this threshold, affirming satisfactory discriminant validity for each variable (Henseler et al., 2015 ). Every HTMT value recorded was situated beneath the specified threshold, thereby supplying supplementary confirmation regarding the constructs’ discriminant validity. In summary, the results of the outer model assessment indicate that the variables showcased commendable levels of reliability and validity, with the discriminant validity being suitably and convincingly established.

Moreover, correlation Table 2 shows that human capital is significantly and positively correlated with structural capital ( r  = 0.594**), TMS ( r  = 0.456 **), and STP ( r  = 0.517**). Structural capital is also significantly and positively correlated with TMS ( r  = 0.893**) and STP ( r  = 0.853**). Furthermore, TMS is significantly and positively correlated with STP (0.859**).

Confirmatory factor analysis (CFA)

A comprehensive confirmatory factor analysis was estimated by employing the software AMOS version 24 to validate the distinctiveness of the variables. CFA shows the fitness of the hypothesized four factors model, including human capital, structural capital, top management support, and team scientific and technological performance, as delineated in Table 3 ; the results show that the hypothesized four-factor model shows fit and excellent alternative models. Consequently, The study variables demonstrate validity and reliability, which makes the dimension model appropriate for conducting a structural path analysis, as advocated by Hair, Page, and Brunsveld ( 2019 ).

Hypotheses testing

This study used the bootstrapping approach, which involves 5,000 bootstrap samples to test the proposed study model and assess the significance and strength of the structural correlations. Using this approach, bias-corrected confidence intervals and p-values were generated in accordance with Streukens and Leroi-Werelds ( 2016 ) guidelines. First, we did an analysis that entailed checking the path coefficients and their connected significance. The findings, as shown in Table 4 , validate Hypothesis 1, revealing a positive correlation between HUC and STP ( β  = 0.476, p  < 0.001). Additionally, the finding validates Hypothesis 2, highlighting a positive association between structural capital and STP ( β  = 0.877, p  < 0.001). For the moderation analysis, we utilized confidence intervals that do not encompass zero, per the guidelines that Preacher and Hayes ( 2008 ) recommended.

In our analysis, we found support for Hypothesis 3a, which posited that top management support (TMS) moderates the relationship between human capital (HUC) and team scientific and technological performance (STP). The results in Table 4 showed that the moderating role, more precisely, the interaction between HUC and TMS, was substantial and positive ( β  = −0.131, p  = 0.001). These results suggest that TMS enhances the positive association between HUC and STP, as shown in Fig. 2 . Consequently, we draw the conclusion that our data substantiates hypothesis 3a. Furthermore, Hypothesis 3b posited that TMS moderates the relationship between STC and STP. The results indicate that TMS moderates the association between STC and STP ( β  = −0.141, p  = 0.001, as presented in Table 4 and Fig. 3 ).

The moderating effect of top management support (TMS) on the relationship between human capital (HUC) and team scientific and technological performance (STP). Source: authors’ development.

The moderating effect of top management support (TMS) on the relationship between structural capital (SUC) and team scientific and technological performance (STP). Source: authors’ development.

The current study highlights the importance of research and academic institutions effectively enhancing their scientific and technological capabilities to manage their global talent within an international research collaboration framework and meet future challenges. Additionally, it underscores the need for these institutions to facilitate scientific knowledge exchange among their employees and counterparts in different countries. The enhancement of talent management through the exchange of scientific research knowledge can be most effectively accomplished by utilizing a collaborative research system between educational and research institutions (Shofiyyah et al., 2023 ), particularly in the context of the COVID-19 landscape. This study has confirmed that enhancing the higher education and research institutions’ human capital (HUC) and structural capital (STC) could attract and maintain global talent management and lead to more effective scientific and technological progress. The findings indicate that the utilization of human capital (HUC) has a significant and positive effect on scientific and technological term performance (STP) (Hypothesis 1), which is consistent with previous research (Habert & Huc, 2010 ). This study has additionally demonstrated that the implementation of s tructural capital (STC) has a significant and positive effect on team scientific and technological performance (STP), as indicated by hypothesis 2, which is also supported by the previous studies finding in different ways (Sobaih et al., 2022 ). This study has also shown that top management support moderates the association between human capital (HUC) and team scientific and technological performance hypothesis 3a and the association between structural capital (STC) and team scientific and technological performance hypothesis 3b. These hypotheses have garnered support from previous studies’ findings in different domains (Chatterjee et al., 2022 ). The study’s empirical findings also confirm the substantial moderating influence exerted by top management support on the relationships between HUC and STP described in hypothesis 3a and STC and STP described in hypothesis 3b, as evidenced by the results presented in Table 4 . Additionally, graphical representations are conducted to investigate the impacts on hypotheses 3a and 3b resulting from the application of high-top management support (TMS) and weak TMS.

The effect of high-top management support (TMS) and weak TMS on Hypothesis 3a is depicted in Fig. 2 . The solid line illustrates the effects of robust TMS on Hypothesis 3a, while the dashed line shows the effects of weak TMS on Hypothesis 3a. The graphic description validates that, as human capital (HUC) increases, team scientific and technological performance (STP) is more pronounced when influenced by robust TMS than weak TMS. This is evidenced by the steeper slope of the solid line in comparison to the dashed line. This finding suggests that employees within the research and academic sectors are more likely to utilize research collaboration networks when influenced by HUC and receive strong support from the organization’s top management.

The graph in Fig. 3 shows the impact of solid top management support (TMS) and weak TMS on Hypothesis 3b. The dotted lines continuous on the graph correspond to the effects of robust TMS and weak TMS, respectively. Figure 3 illustrates that, with increasing top management support (TMS), scientific and technological performance (STP) increase is more significant for robust TMS than weak TMS. This is evident from the steeper slope of the continuous line compared to the slope of the dotted line. This finding suggests that employees within universities and institutes are more likely to engage in research collaboration systems when they receive strong support from top management despite enhanced structural support.

Theoretical contribution

The current study makes significant contributions to the existing body of knowledge by exploring the intricate dynamics between organizational intellectual capital and team performance within scientific and technological research, especially during the unprecedented times brought about by the COVID-19 pandemic. Through its detailed examination of human and structural capital, alongside the moderating impact of top management support, the study provides a multi-faceted understanding of how these factors interact to enhance team outcomes.

This research enriches the literature on intellectual capital by providing empirical evidence on the positive association between HUC and STC and team performance. HUC, which includes employees’ skills, knowledge, and expertise, is a critical driver of innovation and productivity (Lenihan et al., 2019 ). The study highlights how a team’s collective intelligence and capabilities can lead to superior scientific and technological outputs. This finding aligns with and extends previous research that underscores the importance of skilled HR in achieving organizational success (Luo et al., 2023 ; Salamzadeh et al., 2023 ). Structural capital, encompassing organizational processes, databases, and intellectual property, contributes significantly to team performance(Ling, 2013 ). The study illustrates how well-established structures and systems facilitate knowledge sharing, streamline research processes, and ultimately boost the efficiency and effectiveness of research teams. This aspect of the findings adds depth to the existing literature by demonstrating the tangible benefits of investing in robust organizational infrastructure to support research activities.

Another essential contribution of this study is integrating a research collaboration network as a facilitating factor. This network, including digital platforms and tools that enable seamless communication and collaboration among researchers, has become increasingly vital in remote work and global collaboration (Mitchell, 2023 ). By examining how these systems leverage HUC and STC to enhance team performance, the study provides a practical understanding of the mechanisms through which technology can facilitate team scientific and technological performance.

One of the most novel contributions of this study is its emphasis on the moderating role of top management support. The findings suggest that when top management actively supports research initiatives, provides required resources, and fosters innovation, the positive effects of human and structural capital on team performance are amplified (Zada et al., 2023 ). This aspect of the study addresses a gap in the literature by highlighting the critical influence of top management on the success of intellectual capital investments. It underscores the importance of managerial involvement and strategic vision in driving research excellence and team scientific and technological performance.

Practical implications

The practical implications of the current study are weightage for organizations aiming to enhance their research and innovation capabilities and boost their scientific and technical progress. Organizations should prioritize recruiting, training, and retaining highly skilled and trained researchers and professionals globally. This can be achieved through targeted hiring practices, offering competitive compensation and retention, providing continuous professional development opportunities, and developing proper research collaboration networks. Organizations can leverage their expertize to drive innovative research and technological advancements by nurturing a global, talented workforce. Investing in robust organizational structures, processes, and systems is critical (Joseph & Gaba, 2020 ). This includes developing comprehensive databases, implementing efficient research processes, securing intellectual property, and strengthening collaborations. These factors support efficient knowledge sharing and streamline research activities, leading to higher productivity and quality research outcomes (Azeem et al., 2021 ). Organizations should ensure that their infrastructure is adaptable and can support remote and collaborative work environments.

The current study emphasizes the importance of digital platforms and tools facilitating research collaboration. Organizations should adopt advanced research collaboration networks that enable seamless communication, data sharing, and talent management. These systems are particularly crucial in a globalized research environment where team members may be geographically dispersed. Investing in such technology can significantly enhance research projects’ productivity in a sustainable way (Susanto et al., 2023 ). Top Management plays a vital role in the success of research initiatives and contributes to scientific and technological performance. Top management should actively support research teams by providing required resources, setting clear strategic directions, and fostering a culture of innovation. This includes allocating budgets for organizational research and development, encouraging cross-border collaboration, recognizing and rewarding research achievements, and enhancing overall performance. Effective Management ensures that the intellectual capital within the organization is fully utilized and aligned with organizational developmental goals (Paoloni et al., 2020 ). Organizations should create a working atmosphere that encourages research, creativity, and innovation. This can be done by establishing innovation labs, promoting interdisciplinary research, recruiting international talents, sharing research scholars, and encouraging the sharing of ideas across different departments globally. A research-oriented culture that supports innovation can inspire researchers to pursue groundbreaking work and contribute to the organization’s competitive edge.

Limitations and future research direction

The research presents numerous theoretical and practical implications; however, it has. The potential limitation of common method bias could impact the findings of this study. This concern arises because the data for the study variables were obtained from a single source and relied on self-report measures (Podsakoff, 2003 ). Therefore, it is recommended that future studies be conducted longitudinally to gain additional insights into organizations’ potential to enhance efficiency. Furthermore, it is essential to note that the sample size for this study was limited to 363 respondents who were deemed usable. These respondents were drawn from only ten research and academic institutions explicitly targeting the education and research sector.

Consequently, this restricted sample size may hinder the generalizability of the findings. Future researchers may employ a larger sample size and implement a more systematic approach to the organization to enhance the comprehensiveness and generalizability of findings in the context of global talent management and scientific and technological advancement. Furthermore, in future investigations, researchers may explore alternative boundary conditions to ascertain whether additional factors could enhance the model’s efficacy.

Numerous academic studies have emphasized the significance of examining talent management outcomes in global human resource management (HRM). The continuous international movement of highly qualified individuals is viewed as a driving force behind the development of new technologies, the dissemination of scientific findings, and the collaboration between institutions worldwide. Every organization strives to build a qualified and well-trained team, and the personnel department of the organization focuses on finding ways to transfer knowledge from experienced workers to new hires. This study uses a research collaboration system to examine the relationship between organizational intellectual capital (Human and structural Capital) and team scientific and technological performance. Further, this study underscores the moderating role of top management support. These findings offer a nuanced perspective on how organizations can leverage their intellectual assets to foster higher productivity and innovation, especially in emergencies.

Data availability

Due to respondents’ privacy concerns, data will not be publicly available. However, it can be made available by contacting the corresponding author at a reasonable request.

Al-Tit AA, Al-Ayed S, Alhammadi A, Hunitie M, Alsarayreh A, Albassam W (2022) The impact of employee development practices on human capital and social capital: the mediating contribution of knowledge management. J Open Innov 8(4):218

Article   Google Scholar  

Al Ariss A, Cascio WF, Paauwe J (2014) Talent management: current theories and future research directions. J World Bus 49(2):173–179

Al Jawali H, Darwish TK, Scullion H, Haak-Saheem W (2022) Talent management in the public sector: empirical evidence from the Emerging Economy of Dubai. Int J Hum Resour Manag 33(11):2256–2284

Ali M, Li Z, Khan S, Shah SJ, Ullah R (2021) Linking humble leadership and project success: the moderating role of top management support with mediation of team-building. Int J Manag Proj Bus 14(3):545–562

Almeida F, Santos JD, Monteiro JA (2020) The challenges and opportunities in the digitalization of companies in a post-COVID-19 World. IEEE Eng Manag Rev 48(3):97–103

Alrowwad AA, Abualoush SH, Masa’deh RE (2020) Innovation and intellectual capital as intermediary variables among transformational leadership, transactional leadership, and organizational performance. J Manag Dev 39(2):196–222

Anshari M, Hamdan M (2022) Understanding knowledge management and upskilling in Fourth Industrial Revolution: transformational shift and SECI model. VINE J Inf Knowl Manag Syst 52(3):373–393

Google Scholar  

Arocena R, Sutz J (2021) Universities and social innovation for global sustainable development as seen from the south. Technol Forecast Soc change 162:120399

Article   PubMed   Google Scholar  

Arslan A, Golgeci I, Khan Z, Al-Tabbaa O, Hurmelinna-Laukkanen P (2021) Adaptive learning in cross-sector collaboration during global emergency: conceptual insights in the context of COVID-19 pandemic. Multinatl Bus Rev 29(1):21–42

Azeem M, Ahmed M, Haider S, Sajjad M (2021) Expanding competitive advantage through organizational culture, knowledge sharing and organizational innovation. Technol Soc 66:101635

Bakhsh K, Hafeez M, Shahzad S, Naureen B, Faisal Farid M (2022) Effectiveness of digital game based learning strategy in higher educational perspectives. J Educ e-Learn Res 9(4):258–268

Barney J (1991) Firm resources and sustained competitive advantage. J Manag 17(1):99–120

Barney JB, Clark DN (2007) Resource-based theory: Creating and sustaining competitive advantage. Oup Oxford

Baron A, Armstrong M (2007) Human capital management: achieving added value through people. Kogan Page Publishers

Barpanda S (2021) Role of human and structural capital on performance through human resource practices in Indian microfinance institutions: a mediated moderation approach. Knowl Process Manag 28(2):165–180

Biondi L, Russo S (2022) Integrating strategic planning and performance management in universities: a multiple case-study analysis. J Manag Gov 26(2):417–448

Birasnav M, Mittal R, Dalpati A (2019) Integrating theories of strategic leadership, social exchange, and structural capital in the context of buyer–supplier relationship: an empirical study. Glob J Flex Syst Manag 20:219–236

Bontis N (1998) Intellectual capital: an exploratory study that develops measures and models. Manag Decis 36(2):63–76

Bontis N, Fitz‐enz J (2002) Intellectual capital ROI: a causal map of human capital antecedents and consequents. J Intellect Cap 3(3):223–247

Cavicchi C, Vagnoni E (2017) Does intellectual capital promote the shift of healthcare organizations towards sustainable development? Evidence from Italy. J Clean Prod 153:275–286

Chatterjee S, Chaudhuri R, Vrontis D (2022) Does remote work flexibility enhance organization performance? Moderating role of organization policy and top management support. J Bus Res 139:1501–1512

Council NR (2012) Education for life and work: developing transferable knowledge and skills in the 21st century. National Academies Press

Davenport S, Carr A, Bibby D (2002) Leveraging talent: spin–off strategy at industrial research. RD Manag 32(3):241–254

Dess GG, Picken JC (2000) Changing roles: Leadership in the 21st century. Organ Dyn 28(3):18–34

Duan W, Li C (2023) Be alert to dangers: collapse and avoidance strategies of platform ecosystems. J Bus Res 162:113869

Farahian M, Parhamnia F, Maleki N (2022) The mediating effect of knowledge sharing in the relationship between factors affecting knowledge sharing and reflective thinking: the case of English literature students during the COVID-19 crisis. Res Pract Technol Enhanc Learn 17(1):1–25

Fasi MA (2022) An overview on patenting trends and technology commercialization practices in the university Technology Transfer Offices in USA and China. World Pat Inf 68:102097

Felin T, Hesterly WS (2007) The knowledge-based view, nested heterogeneity, and new value creation: philosophical considerations on the locus of knowledge. Acad Manag Rev 32(1):195–218

Fernandes C, Veiga PM, Lobo CA, Raposo M (2023) Global talent management during the COVID‐19 pandemic? The Gods must be crazy! Thunderbird Int Bus Rev 65(1):9–19

Gogan LM, Duran DC, Draghici A (2015) Structural capital—a proposed measurement model. Procedia Econ Financ 23:1139–1146

Gonzalez-Mulé E, Courtright SH, DeGeest D, Seong J-Y, Hong D-S (2016) Channeled autonomy: the joint effects of autonomy and feedback on team performance through organizational goal clarity. J Manag 42(7):2018–2033

Haak-Saheem W (2020) Talent management in Covid-19 crisis: how Dubai manages and sustains its global talent pool. Asian Bus Manag 19:298–301

Habert B, Huc C (2010) Building together digital archives for research in social sciences and humanities. Soc Sci Inf 49(3):415–443

Haider SA, Akbar A, Tehseen S, Poulova P, Jaleel F (2022) The impact of responsible leadership on knowledge sharing behavior through the mediating role of person–organization fit and moderating role of higher educational institute culture. J Innov Knowl 7(4):100265

Hair JF, Page M, Brunsveld N (2019) Essentials of business research methods. Routledge

Harsch K, Festing M (2020) Dynamic talent management capabilities and organizational agility—a qualitative exploration. Hum Resour Manag 59(1):43–61

He S, Chen W, Wang K, Luo H, Wang F, Jiang W, Ding H (2023) Region generation and assessment network for occluded person re-identification. IEEE Trans Inf Forensic Secur 19:120–132

Henseler J, Ringle CM, Sarstedt M (2015) A new criterion for assessing discriminant validity in variance-based structural equation modeling. J Acad Mark Sci 43:115–135

Jain N, Thomas A, Gupta V, Ossorio M, Porcheddu D (2022) Stimulating CSR learning collaboration by the mentor universities with digital tools and technologies—an empirical study during the COVID-19 pandemic. Manag Decis 60(10):2824–2848

Joseph J, Gaba V (2020) Organizational structure, information processing, and decision-making: a retrospective and road map for research. Acad Manag Ann 14(1):267–302

Kaliannan M, Darmalinggam D, Dorasamy M, Abraham M (2023) Inclusive talent development as a key talent management approach: a systematic literature review. Hum Resour Manag Rev 33(1):100926

Kallmuenzer A, Baptista R, Kraus S, Ribeiro AS, Cheng C-F, Westhead P (2021) Entrepreneurs’ human capital resources and tourism firm sales growth: a fuzzy-set qualitative comparative analysis. Tour Manag Perspect 38:100801

Kim KY, Atwater L, Patel PC, Smither JW (2016) Multisource feedback, human capital, and the financial performance of organizations. J Appl Psychol 101(11):1569

Kwok L (2022) Labor shortage: a critical reflection and a call for industry-academia collaboration. Int J Contemp Hosp Manag 34(11):3929–3943

Lee J-C, Shiue Y-C, Chen C-Y (2016) Examining the impacts of organizational culture and top management support of knowledge sharing on the success of software process improvement. Comput Hum Behav 54:462–474

Lenihan H, McGuirk H, Murphy KR (2019) Driving innovation: public policy and human capital. Res policy 48(9):103791

Ling Y-H (2013) The influence of intellectual capital on organizational performance—knowledge management as moderator. Asia Pac J Manag 30(3):937–964

Luo J, Zhuo W, Xu B (2023) The bigger, the better? Optimal NGO size of human resources and governance quality of entrepreneurship in circular economy. Manag Decis (ahead-of-print) https://doi.org/10.1108/MD-03-2023-0325

Damsgaard J, Lyytinen K (2001) The role of intermediating institutions in the diffusion of electronic data interchange (EDI): how industry associations intervened in Denmark, Finland, and Hong Kong. Inf Soc 17(3):195–210

Marchiori DM, Rodrigues RG, Popadiuk S, Mainardes EW (2022) The relationship between human capital, information technology capability, innovativeness and organizational performance: an integrated approach. Technol Forecast Soc Change 177:121526

Marinelli L, Bartoloni S, Pascucci F, Gregori GL, Briamonte MF (2022) Genesis of an innovation-based entrepreneurial ecosystem: exploring the role of intellectual capital. J Intellect Cap 24(1):10–34

Mehralian G, Nazari JA, Ghasemzadeh P (2018) The effects of knowledge creation process on organizational performance using the BSC approach: the mediating role of intellectual capital. J Knowl Manag 22(4):802–823

Mitchell A (2023) Collaboration technology affordances from virtual collaboration in the time of COVID-19 and post-pandemic strategies. Inf Technol People 36(5):1982–2008

Muñoz JLR, Ojeda FM, Jurado DLA, Peña PFP, Carranza CPM, Berríos HQ, Vasquez-Pauca MJ (2022) Systematic review of adaptive learning technology for learning in higher education. Eurasia J Educ Res 98(98):221–233

Nezam MHK, Ataffar A, Isfahani AN, Shahin A (2013) The impact of structural capital on new product development performance effectiveness—-the mediating role of new product vision and competitive advantage. Int J Hum Resour Stud 3(4):281

O’Dwyer M, Filieri R, O’Malley L (2023) Establishing successful university–industry collaborations: barriers and enablers deconstructed. J Technol Transf 48(3):900–931

Oliver C (1997) Sustainable competitive advantage: combining institutional and resource‐based views. Strateg Manag J 18(9):697–713

Pak J, Heidarian Ghaleh H, Mehralian G (2023) How does human resource management balance exploration and exploitation? The differential effects of intellectual capital‐enhancing HR practices on ambidexterity and firm innovation. Human Resource Manag https://doi.org/10.1002/hrm.22180

Paoloni M, Coluccia D, Fontana S, Solimene S (2020) Knowledge management, intellectual capital and entrepreneurship: a structured literature review. J Knowl Manag 24(8):1797–1818

Patnaik S, Munjal S, Varma A, Sinha S (2022) Extending the resource-based view through the lens of the institution-based view: a longitudinal case study of an Indian higher educational institution. J Bus Res 147:124–141

Pellegrini L, Aloini D, Latronico L (2022) Open innovation and intellectual capital during emergency: evidence from a case study in telemedicine. Knowl Manag Res Pract 21(4), 765–776

Phan LT, Nguyen TV, Luong QC, Nguyen TV, Nguyen HT, Le HQ, Pham QD (2020) Importation and human-to-human transmission of a novel coronavirus in Vietnam. N Engl J Med 382(9):872–874

Article   PubMed   PubMed Central   Google Scholar  

Phuong QN, Le Ngoc M, Dong HT, Thao TLT, Tran T, Cac T (2024) Enhancing employment opportunities for people with disabilities in Vietnam: the role of vocational training and job placement centers. J Chin Hum Resour Manag 15(3):64–75

Podsakoff N (2003) Common method biases in behavioral research: a critical review of the literature and recommended remedies. J Appl Psychol 885(879):10.1037

Podsakoff PM, MacKenzie SB, Lee J-Y, Podsakoff NP (2003) Common method biases in behavioral research: a critical review of the literature and recommended remedies. J Appl Psychol 88(5):879

Preacher KJ, Hayes AF (2008) Asymptotic and resampling strategies for assessing and comparing indirect effects inmultiple mediator models. Behav Res Methods 40(3):879–891

Rådberg KK, Löfsten H (2024) The entrepreneurial university and development of large-scale research infrastructure: Exploring the emerging university function of collaboration and leadership. J Technol Transf 49(1):334–366

Radhamani R, Kumar D, Nizar N, Achuthan K, Nair B, Diwakar S (2021) What virtual laboratory usage tells us about laboratory skill education pre-and post-COVID-19: Focus on usage, behavior, intention and adoption. Educ Inf Technol 26(6):7477–7495

Rasheed MH, Khalid J, Ali A, Rasheed MS, Ali K (2024) Human resource analytics in the era of artificial intelligence: Leveraging knowledge towards organizational success in Pakistan. J Chin Hum Resour Manag 15:3–20

Ringle CM, Sarstedt M, Mitchell R, Gudergan SP (2020) Partial least squares structural equation modeling in HRM research. Int J Hum Resour Manag 31(12):1617–1643

Roos G, Roos J (1997) Measuring your company’s intellectual performance. Long Range Plan 30(3):413–426

Salamzadeh A, Tajpour M, Hosseini E, Brahmi MS (2023) Human capital and the performance of Iranian Digital Startups: the moderating role of knowledge sharing behaviour. Int J Public Sect Perform Manag 12(1-2):171–186

Sarwar A, Mustafa A (2023) Analysing the impact of green intellectual capital on environmental performance: the mediating role of green training and development. Technol Anal Strateg Manag 1–14. https://doi.org/10.1080/09537325.2023.2209205

Scott WR (2008) Institutions and organizations: ideas and interests. Sage

Secundo G, Dumay J, Schiuma G, Passiante G (2016) Managing intellectual capital through a collective intelligence approach: an integrated framework for universities. J Intellect Cap 17(2):298–319

Secundo G, Massaro M, Dumay J, Bagnoli C (2018) Intellectual capital management in the fourth stage of IC research: a critical case study in university settings. J Intellect Cap 19(1):157–177

Shao JJ, Ariss AA (2020) Knowledge transfer between self-initiated expatriates and their organizations: research propositions for managing SIEs. Int Bus Rev 29(1):101634

Shofiyyah NA, Komarudin TS, Hasan MSR (2023) Innovations in Islamic Education Management within the University Context: addressing challenges and exploring future prospects. Nidhomul Haq 8(2):193–209

Sigala M, Ren L, Li Z, Dioko LA (2023) Talent management in hospitality during the COVID-19 pandemic in Macao: a contingency approach. Int J Contemp Hosp Manag 35(8):2773–2792

Singh SK, Gupta S, Busso D, Kamboj S (2021) Top management knowledge value, knowledge sharing practices, open innovation and organizational performance. J Bus Res 128:788–798

Sobaih AEE, Hasanein A, Elshaer IA (2022) Higher education in and after COVID-19: the impact of using social network applications for e-learning on students’ academic performance. Sustainability 14(9):5195

Article   CAS   Google Scholar  

Sommer LP, Heidenreich S, Handrich M (2017) War for talents—how perceived organizational innovativeness affects employer attractiveness. RD Manag 47(2):299–310

Stewart T (1991) Brainpower: how intellectual capital is becoming America’s most valuable. Fortune

Streukens S, Leroi-Werelds S (2016) Bootstrapping and PLS-SEM: A step-by-step guide to get more out of your bootstrap results. Eur Manage J 34(6):618–632

Sulaiman F, Uden L, Eldy EF (2022) Online Learning in Higher Education Institution During COVID-19: A Review and the Way Forward. Paper presented at the International Workshop on Learning Technology for Education Challenges

Susanto P, Sawitri NN, Ali H, Rony ZT (2023) Employee performance and talent management impact increasing construction company productivity. Int J Psychol Health Sci 1(4):144–152

Tan CN-L, Md. Noor S (2013) Knowledge management enablers, knowledge sharing and research collaboration: a study of knowledge management at research universities in Malaysia. Asian J Technol Innov 21(2):251–276

Vaiman V, Sparrow P, Schuler R, Collings DG (2018) Macro talent management: a global perspective on managing talent in developed markets. Routledge

Vătămănescu E-M, Cegarra-Navarro J-G, Martínez-Martínez A, Dincă V-M, Dabija D-C (2023) Revisiting online academic networks within the COVID-19 pandemic–From the intellectual capital of knowledge networks towards institutional knowledge capitalization. J Intellect Cap 24(4):948–973

Wang Y, Lee L-H, Braud T, Hui P (2022) Re-shaping Post-COVID-19 teaching and learning: A blueprint of virtual-physical blended classrooms in the metaverse era. Paper presented at the 2022 IEEE 42nd International Conference on Distributed Computing Systems Workshops (ICDCSW)

Wang Z, Wang N, Liang H (2014) Knowledge sharing, intellectual capital and firm performance. Manag Decis 52(2):230–258

Xu A, Li Y, Donta PK (2024) Marketing decision model and consumer behavior prediction with deep learning. J Organ End Use Comput (JOEUC) 36(1):1–25

Yang G, Xiangming L (2024) Graduate socialization and anxiety: insights via hierarchical regression analysis and beyond. Stud High Educ 1–17. https://doi.org/10.1080/03075079.2024.2375563

Zada M, Khan J, Saeed I, Zada S, Jun ZY (2023) Linking public leadership with project management effectiveness: mediating role of goal clarity and moderating role of top management support. Heliyon 9(5)

Download references

Author information

Authors and affiliations.

School of Economics and Management, Hanjiang Normal University, Shiyan, 442000, China

Muhammad Zada

Facultad de Administración y Negocios, Universidad Autónoma de Chile, Santiago, 8320000, Chile

School of Law, Huazhong University of Science and Technology, Wuhan, Hubie, China

Imran Saeed

College of Management, Shenzhen University, Shenzhen, China

Department of Business Administration, Faculty of Management Sciences, Ilma University, Karachi, Pakistan

Shagufta Zada

Business School Henan University, Kaifeng, Henan, China

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization: Muhammad Zada and Imran Saeed. Methodology: Jawad Khan. Software: Shagufta Zada. Data collection: Muhammad Zada, Shagufta Zada and Jawad Khan. Formal analysis: Imran Saeed and Jawad Khan. Resources: Muhammad Zada. Writing original draft preparation: Muhammad Zada and Imran Saeed. Writing review and editing: Jawad Khan, Shagufta Zada. All authors have read and agreed to the published version of the paper.

Corresponding author

Correspondence to Muhammad Zada .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Ethical approval

The author sought and received ethical approval from the Research Ethical Committee School of Economics and Management at Hanjiang Normal University, China, with approval number 2023REC001, and the study complied with ethical standards.

Informed consent statement

Informed consent was obtained from all subjects involved in the study. All the participants were accessed with the support of the HR Department employed in China’s research and academia sector. Response Participants were provided with comprehensive information regarding the study’s purpose and procedures. Confidentiality and privacy were strictly implemented throughout the research process. Using the time lag data collection approach, we collected from 393 employees employed in China’s research and academic sector.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/ .

Reprints and permissions

About this article

Cite this article.

Zada, M., Saeed, I., Khan, J. et al. Navigating post-pandemic challenges through institutional research networks and talent management. Humanit Soc Sci Commun 11 , 1164 (2024). https://doi.org/10.1057/s41599-024-03697-9

Download citation

Received : 28 February 2024

Accepted : 30 August 2024

Published : 09 September 2024

DOI : https://doi.org/10.1057/s41599-024-03697-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

The importance of international and national publications for promotion and the impact of recruitment policies

• Published: 09 September 2024

Cite this article

• Lokman Tutuncu   ORCID: orcid.org/0000-0002-8653-9546 1 &
• Marco Seeber 2  

64 Accesses

Explore all metrics

Few studies explored whether publications in international peer-reviewed journals and publications in national journals, so-called local publications , affect academic promotion and whether their impact varies for different categories of scientists. Moreover, recruitment policies have mostly tried to incentivize international publications, but in recent years, some countries have promoted publications in national outlets, and the impact of such policies has not been explored. The goal of this article is to address these gaps considering all active Turkish full professors in Business and Economics ( n  = 1470) and the factors affecting the time to promotion. We found that international publications and, to a lesser extent, local publications predict significantly faster promotion, but with very different impact across categories of scientists. Moreover, local articles’ impact on time-to-promotion increased after the introduction of new regulations that encouraged publication in national journals, whereas international publications became completely irrelevant.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

Similar content being viewed by others

Faculty salary at korean universities: does publication matter.

Barriers to attracting the best researchers: perceptions of academics in economics and physics in three European countries

The Politics of Publications and Citations: A Cross Country Comparison

Data availability.

Data are collected from public sources and will be provided to the journal and reviewers upon request.

Raised to five articles as of March 2024.

We follow in this regard, the approach by Cruz-Castro and Sanz-Menendez ( 2010 ), see Sect. " Methods ".

Research universities are Ankara University, Atatürk, Boğaziçi, Çukurova, Dokuz Eylül, Ege, Erciyes, Fırat, Gazi (Ankara HBV), Gebze Teknik, Hacettepe, İstanbul, İstanbul-Cerrahpaşa, İstanbul Teknik, Karadeniz Teknik, Marmara, ODTÜ, and Uludağ University.

Aagard, K. (2015). How incentives trickle down: Local use of a national bibliometric indicator system. Science and Public Policy, 42 (5), 725–737. https://doi.org/10.1093/scipol/scu087

Article   Google Scholar  

Abalkina, A. (2023). Publication and collaboration anomalies in academic papers originating from a paper mill: Evidence from a Russia-based paper mill. Learned Publishing, 36 (4), 689–702. https://doi.org/10.1002/leap.1574

Abramo, G., D’Angelo, C. A., & Hladchenko, M. (2023). Assessing the effects of publication requirements for professorship on research performance and publishing behaviour of Ukrainian academics. Scientometrics, 128 , 4589–4609. https://doi.org/10.1007/s11192-023-04753-y

Afonso, A. (2016). Varieties of academic labor markets in Europe. PS. Political Science and Politics, 49 (4), 816–821. https://doi.org/10.1017/S1049096516001505

Akca, S., & Akbulut, M. (2018). Predatory journals in Turkey: An investigation through Beall’s list. Bilgi Dünyasi, 19 (2), 255–274. https://doi.org/10.15612/BD.2018.695

Altbach, P. G. (2009). Peripheries and centers: Research universities in developing countries. Asia Pacific Education Review, 10 , 15–27. https://doi.org/10.1007/s12564-009-9000-9

Baccini, A., & Petrovich, E. (2023). A global exploratory comparison of country self-citations 1996–2019. PLoS ONE, 18 (12). https://doi.org/10.1371/journal.pone.0294669

Belcher, D. D. (2007). Seeking acceptance in an English-only research world. Journal of Second Language Writing, 16 , 1–22. https://doi.org/10.1016/j.jslw.2006.12.001

Bell, J. G., & Seater, J. J. (1978). Publishing performance: Departmental and individual. Economic Inquiry, 16 (4), 599–615. https://doi.org/10.1111/j.1465-7295.1978.tb00530.x

Bernerth, J. B., & Aguinis, H. (2016). A critical review and best-practice recommendations for control variable usage. Personnel Pscyhology, 69 , 229–283. https://doi.org/10.1111/peps.12103

Burke, C. J. (1953). A brief note on one-tailed tests. Psychological Bulletin, 50 (5), 384–387. https://doi.org/10.1037/h0059627

Canagarajah, A. S. (1996). “Nondiscursive” requirements in academic publishing, material resources of periphery scholars, and the politics of knowledge production. Written Communication, 13 (4), 435–472. https://doi.org/10.1177/0741088396013004001

Çaylak, M. (2023). Doçentlikte yeni dönem: Nitelik esas alınacak. (New period in associate professorship: Quality will be the basis of promotion). Retrieved August 15, 2023, from https://www.trthaber.com/haber/egitim/docentlikte-yeni-donem-nitelik-esas-alinacak-787814.html

Çokgezen, M. (2018). Research performance of Turkish economists and economics departments: Another update and a review of the 2000s. Emerging Markets Finance and Trade, 55 (1), 133–149. https://doi.org/10.1080/1540496X.2018.1500891

Coleman, J. S. (1988). Social capital in the creation of human capital. American Journal of Sociology, 94 , 95–120.

Crewe, I. (1988). Reputation, research, and reality: The publication records of UK departments of politics. Scientometrics, 14 (3–4), 235–250. https://doi.org/10.1007/BF02020077

Cruz-Castro, L., & Sanz-Menendez, L. (2010). Mobility versus job stability: Assessing tenure and productivity outcomes. Research Policy, 39 , 27–38. https://doi.org/10.1016/j.respol.2009.11.008

Dawson, D., Morales, E., McKiernan, E. C., Schimanski, L. A., Niles, M. T., & Alperin, J. P. (2022). The role of collegiality in academic review, promotion, and tenure. PLoS ONE, 17 (4), e0265506. https://doi.org/10.1371/journal.pone.0265506

Demir, S. B. (2018). Pros and cons of the new financial support policy for Turkish researchers. Scientometrics, 116 , 2053–2068. https://doi.org/10.1007/s11192-018-2833-4

Dundar, H., & Lewis, D. R. (1998). Determinants of research productivity in higher education. Research in Higher Education, 39 (6), 607–631. https://doi.org/10.1023/A:1018705823763

Eysenck, H. J. (1960). The concept of statistical significance and the controversy about one-tailed tests. Psychological Review, 67 (4), 269–271. https://psycnet.apa.org/doi/10.1037/h0048412

Flowerdew, J. (2000). Discourse community, legitimate peripheral participation, and the nonnative–English-speaking scholar. TESOL Quarterly, 34 (1), 127–150. https://doi.org/10.2307/3588099

Freese, J. (2002). Least likely observations in regression models for categorical outcomes. Stata Journal, 2 (3), 296–300.

Grancay, M., Vveinhardt, J., & Sumilo, E. (2017). Publish or perish: How Central and Eastern European economists have dealt with the ever-increasing academic publishing requirements 2000–2015. Scientometrics, 111 , 1813–1837. https://doi.org/10.1007/s11192-017-2332-z

Guskov, A. E., Kosyakov, D. V., & Selivanova, I. V. (2018). Boosting research productivity in top Russian universities: The circumstances of breakthrough. Scientometrics, 117 , 1053–1080. https://doi.org/10.1007/s11192-018-2890-8

Harris, G., & Kaine, G. (1994). The determinants of research performance: A study of Australian university economists. Higher Education, 27 (2), 191–201. https://doi.org/10.1007/BF01384088

Heckman, J. J., & Moktan, S. (2020). Publishing and promotion in Economics: The tyranny of the top five. Journal of Economic Literature, 58 (2), 419–470. https://doi.org/10.1257/jel.20191574

Heng, K., Hamid, M. O., & Khan, A. (2020). Factors influencing academics’ research engagement and productivity: A developing countries perspective. Issues in Educational Research, 30 (3), 965–987.

Google Scholar  

Henriksen, D. (2018). What factors are associated with increasing co-authorship in the social sciences? A case study of Danish economics and political science. Scientometrics, 114 , 1395–1421. https://doi.org/10.1007/s11192-017-2635-0

Hesli, V. L., Lee, J. M., & Mitchell, S. M. (2012). Predicting rank attainment in political science: What else besides publications affects promotion? PS. Political Science and Politics, 45 (3), 475–492. https://doi.org/10.1017/S1049096512000364

Hicks, D. (1999). The difficulty of achieving full coverage of international social sciences literature and the bibliometric consequences. Scientometrics, 44 (2), 193–215.

Hicks, D. (2012). Performance-based university research funding systems. Research Policy, 41 (2), 251–326. https://doi.org/10.1016/j.respol.2011.09.007

Hladchenko, M., & Moed, H. F. (2021). The effect of publication traditions and requirements in research assessment and funding policies upon the use of national journals in 28 post-socialist countries. Journal of Informetrics, 15 (4), 101190. https://doi.org/10.1016/j.joi.2021.101190

Hladchenko, M., & Moed, H. F. (2021). National orientation of Ukrainian journals: Means-ends decoupling in a semi-peripheral state. Scientometrics, 126 , 2365–2389. https://doi.org/10.1007/s11192-020-03844-4

Hopwood, A. G. (2008). Changing pressures on the research process: On trying to research in an age when curiosity is not enough. European Accounting Review, 17 (1), 87–96. https://doi.org/10.1080/09638180701819998

Horta, H. (2013). Deepening our understanding of academic inbreeding effect on research information exchange and scientific output: New insights for academic based research. Higher Education, 65 (4), 487–510. https://doi.org/10.1007/s10734-012-9559-7

Horta, H., Veloso, F., & Grediaga, R. (2010). Navel gazing: Academic inbreeding and scientific productivity. Management Science, 56 (3), 414–429. https://doi.org/10.1287/mnsc.1090.1109

Howe-Walsh, L., & Turnbull, S. (2016). Barriers to women leaders in academia: Tales from science and technology. Studies in Higher Education, 41 (3), 415–428. https://doi.org/10.1080/03075079.2014.929102

Hoyle, R. H. (2023). Structural equation modeling: An overview. In R. H. Hoyle (Eds.) Handbook of Structural Equation Modeling (2 nd ed., pp. 3–16). Guildford Press.

Hutchinson, E. B., & Zivney, T. L. (1995). The publication profile of economists. Journal of Economic Education, 26 (1), 59–79. https://doi.org/10.2307/1183466

Iacobucci, D. (2009). Everything you want to know about SEM (structural equations modeling) but were afraid to ask. Journal of Consumer Psychology, 19 , 673–680.

Ingwersen, P., & Larsen, B. (2014). Influence of a performance indicator on Danish research production and citation impact 2000–12. Scientometrics, 101 , 1325–1344. https://doi.org/10.1007/s11192-014-1291-x

Ioannidis, J. P. A., Klavans, R., & Boyack, K. W. (2018). Thousands of scientists publish a paper every five days. Nature, 561 , 167–169. https://doi.org/10.1038/d41586-018-06185-8

Jöreskog K. G., & Sörbom, D. (1996). LISREL 8 user’s reference guide (2nd ed.). Scientific Software International.

Jungbauer-Gans, M., & Gross, C. (2013). Determinants of success in academic careers: Findings from the German academic labour market. Zeitschrift Für Soziologie, 42 (1), 74–92. https://doi.org/10.1515/zfsoz-2013-0106

Kallio, K., Kallio, T., Tienari, J., & Hyvönen, T. (2015). Ethos at stake: Performance management and academic work at universities. Human Relations, 69 (3), 685–709. https://doi.org/10.1177/001872671559680

Karadag, E. (2021). Academic (dis)qualification of Turkish rectors: Their career paths, H-index, and the number of articles and citations. Higher Education, 81 , 301–323. https://doi.org/10.1007/s10734-020-00542-1

Karadag, E., & Ciftci, S. K. (2022). Deepening the effects of the academic inbreeding: Its impact on individual and institutional research productivity.  Research in Higher Education, 63 , 1015–1036. https://doi.org/10.1007/s11162-021-09670-8

Karataş-Acer, E., & Güçlü, N. (2017). The expansion of higher education in Turkey: The rationales behind this expansion and its challenges. Yükseköğretim Dergisi, 7 (1), 28–38. https://doi.org/10.2399/yod.17.003

Kifor, C. V., Benedek, A. M., Sirbu, I., & Savescu, R. F. (2023). Institutional drivers of research productivity: A canonical multivariate analysis of Romanian public universities. Scientometrics, 128 , 2233–2258. https://doi.org/10.1007/s11192-023-04655-z

Kline, R. B. (2015). Principles and practice of structural equation modeling (4th ed.). Guildford Press.

Korytkowski, P., & Kulcyzcki, E. (2019). Examining how country-level science policy shapes publication patterns: The case of Poland. Scientometrics, 119 , 1519–1543. https://doi.org/10.1007/s11192-019-03092-1

Kosyakov, D., & Guskov, A. (2022). Reasons and consequences of changes in Russian research assessment policies. Scientometrics, 127 , 4609–4630. https://doi.org/10.1007/s11192-022-04469-5

Krammer, S. (2021). Research performance in business and management: The impact of team size and diversity. Academy of Management Annual Meeting Proceedings . https://doi.org/10.5465/AMBPP.2021.10981abstract

Kuzhabekova, A., & Ruby, A. (2018). Raising research productivity in a post-Soviet higher education system: A case from Central Asia. European Education, 50 (3), 266–282. https://doi.org/10.1080/10564934.2018.1444942

Kyvik, S. (1990). Age and scientific productivity. Differences between fields of learning. Higher Education, 19 , 37–55. https://doi.org/10.1007/BF00142022

Kyvik, S., & Reymert, I. (2017). Research collaboration in groups and networks: Differences across academic fields. Scientometrics, 113 , 951–967. https://doi.org/10.1007/s11192-017-2497-5

Larivière, V., Ni, C., Gingras, Y., Cronin, B., & Sugimoto, C. R. (2013). Bibliometrics: Global gender disparities in science. Nature News, 504 (7479), 211.

Levin, S. G., & Stephan, P. E. (1991). Research productivity over the life cycle: Evidence for academic scientists. American Economic Review, 81 (1), 114–132.

Levine, T., & Banas, J. (2002). One-tailed F-tests in communication research. Communication Monographs, 69 (2), 132–143. https://doi.org/10.1080/714041709

Lombardi, C. M., & Hurlbert, S. M. (2009). Misprescription and misuse of one-tailed tests. Austral Ecology, 34 (4), 447–468. https://doi.org/10.1111/j.1442-9993.2009.01946.x

Long, J. S., Allison, P. D., & McGinnis, R. (1993). Rank advancement in academic careers: Sex differences and the effects of productivity. American Sociological Review, 58 (5), 703–722. https://doi.org/10.2307/2096282

Lutter, M., & Schröder, M. (2016). Who becomes a tenured professor, and why? Panel data evidence from German sociology, 1980–2013. Research Policy, 45 , 999–1013. https://doi.org/10.1016/j.respol.2016.01.019

Ma, L., & Ladisch, M. (2019). Evaluation complacency or evaluation inertia? A study of evaluative metrics and research practices in Irish universities. Research Evaluation, 28 (3), 209–217. https://doi.org/10.1093/reseval/rvz008

Macháček, V., & Srholec, M. (2022). Predatory publishing in Scopus: Evidence on cross-country differences. Quantitative Science Studies, 3 (3), 859–887. https://doi.org/10.1162/qss_a_00213

Marusic, A., & Marusic, M. (1999). Small scientific journals from small countries: Breaking from a vicious circle of inadequacy. Croatian Medical Journal, 40 (4), 508–514.

Matumba, L., Maulidi, F., Balehegn, M., Abay, F., Salanje, G., Dzimbiri, L., & Kaunda, E. (2019). Blacklisting or whitelisting? Deterring faculty in developing countries from publishing in substandard journals. Journal of Scholarly Publishing, 50 (2), 83–95. https://doi.org/10.3138/jsp.50.2.01

Matveeva, N., Sterligov, I., & Yudkevich, M. (2021). The effect of Russian University Excellence Initiative on publications and collaboration patterns. Journal of Informetrics, 15 (1), 101110. https://doi.org/10.1016/j.joi.2020.101110

Merton, R. K. (1973). The normative structure of science. In N. W. Storer (Ed.), The sociology of science: theoretical and empirical investigations (pp. 265–278). University of Chicago Press.

Muller, M., Cowan, R., & Barnard, H. (2018). On the value of foreign PhDs in the developing world: Training versus selection effects in the case of South Africa. Research Policy, 47 (5), 886–900. https://doi.org/10.1016/j.respol.2018.02.013

Nazarovets, S. (2020). Controversial practice of rewarding for publications in national journals. Scientometrics, 124 , 813–818. https://doi.org/10.1007/s11192-020-03485-7

Nazarovets, S. (2022). Analysis of publications by authors of Ukrainian institutes in Scopus-delisted titles. Learned Publishing, 35 (4), 499–515. https://doi.org/10.1002/leap.1464

Nederhof, A. J. (2006). Bibliometric monitoring of research performance in the Social Sciences and Humanities: A review. Scientometrics, 66 (1), 81–100. https://doi.org/10.1007/s11192-006-0007-2

Nederhof, A. J., Zwaan, R. A., De Bruin, R. E., & Dekker, P. J. (1989). Assessing the usefulness of bibliometric indicators for the Humanities and the Social and Behavioral Sciences: A comparative study. Scientometrics, 15 (5–6), 423–435. https://doi.org/10.1007/BF02017063

Newsom, J. T. (2023). Longitudinal structural equation modeling: A comprehensive introduction (2nd ed.). Routledge.

Book   Google Scholar  

Newsom, J. T., Hachem, Z. A., Granger, A. M., & Denning, E. C. (2023). Where did I go wrong with my model? Ten tips for getting results in SEM. Structural Equation Modeling: A Multidisciplinary Journal, 30 (3), 491–500. https://doi.org/10.1080/10705511.2022.2101113

Nunkoo, R., & Ramkissoon, H. (2012). Structural equation modelling and regression analysis in tourism research. Current Issues in Tourism, 15 (8), 777–802. https://doi.org/10.1080/13683500.2011.641947

Önder, C., & Erdil, S. E. (2015). Aynı kurumsal beklentilere tabi aktörlerin farklılaşan davranışları: Öğretim üyelerinin bilimsel yayın üretkenliklerinin üniversite, bölüm ve birey düzeyindeki yordayıcıları. ODTÜ Gelişme Dergisi, 42 , 481–519.

Önder, Ç., & Erdil, S. (2017). Opportunities and opportunism; Publication outlet selection under pressure to increase research productivity. Research Evaluation, 26 (2), 66–77. https://doi.org/10.1093/reseval/rvx006

Osterloh, M. (2010). Governance by numbers. Does it really work in research? Analyse and Kritik, 32 (2), 267–283.

Rodgers, J. L. (2010). The epistemology of mathematical and statistical modeling: A quiet methodological revolution. American Psychologist, 65 , 1–12. https://doi.org/10.1037/a0018326

Salager-Meyer, F. (2008). Scientific publishing in developing countries: Challenges for the future. Journal of English for Academic Purposes, 7 , 121–132. https://doi.org/10.1016/j.jeap.2008.03.009

Sanz-Menendez, L., Cruz-Castro, L., & Alva, K. (2013). Time to tenure in Spanish universities: An event history analysis. PLoS ONE, 8 (10), e77028. https://doi.org/10.1371/journal.pone.0077028

Sasvari, P., Bakacsi, G., & Urbanovics, A. (2022). Scientific career tracks and publication performance – relationships discovered in the Hungarian academic promotion system. Heliyon, 8 , e09159. https://doi.org/10.1016/j.heliyon.2022.e09159

Scheidegger, F., Briviba, A., & Frey, B. S. (2023). Behind the curtains of academic publishing: Strategic responses of economists and business scholars. Scientometrics, 128 , 4765–4790. https://doi.org/10.1007/s11192-023-04772-9

Schröder, M., Lutter, M., & Habicht, I. M. (2021). Publishing, signaling, social capital, and gender: Determinants of becoming a tenured professor in German political science. PLoS ONE, 16 (1), e0243514. https://doi.org/10.1371/journal.pone.0243514

Schulze, G. G., Warning, S., & Wiermann, C. (2008). What and how long does it take to get tenure? The case of economics and business administration in Austria, Germany, and Switzerland. German Economic Review, 9 (4), 473–505. https://doi.org/10.1111/j.1468-0475.2008.00450.x

Seeber, M., Cattaneo, M., Meoli, M., & Malighetti, P. (2019). Self-citations as a strategic response to the use of metrics for career decisions. Research Policy, 48 (2), 478–491. https://doi.org/10.1016/j.respol.2017.12.004

Seeber, M., Debacker, N., Meoli, M., & Vandevelde, K. (2023). Exploring the effects of mobility and foreign nationality on internal career progression in universities. Higher Education, 85 , 1041–1081. https://doi.org/10.1007/s10734-022-00878-w

Taylor, S. W., Fender, B. F., & Burke, K. G. (2006). Unraveling the academic productivity of economists: The opportunity costs of teaching and service. Southern Economic Journal, 72 (4), 846–859. https://doi.org/10.2307/20111856

Teodorescu, D. (2000). Correlates of faculty publication productivity: A cross-national analysis. Higher Education, 39 , 201–222. https://doi.org/10.1023/A:1003901018634

Tien, F. F. (2007). To what degree does the promotion system rewards faculty research productivity? British Journal of Sociological Education, 28 (1), 105–123. https://doi.org/10.1080/01425690600996741

Tutuncu, L. (2023a). All-pervading insider bias alters review time in Turkish university journals. Scientometrics, 128 , 3743–3791. https://doi.org/10.1007/s11192-023-04724-3

Tutuncu, L. (2023b). Editorial board publication strategy and acceptance rates in Turkish national journals. Journal of Data and Information Science . Ahead-of-Print. https://doi.org/10.2478/jdis-2023-0019

Tutuncu, L. (2024). Gatekeepers or gatecrashers? The inside connection in editorial board publications of Turkish national journals. Scientometrics, 129 , 957–984.

Tutuncu, L., Yucedogru, R., & Sarisoy, I. (2022). Academic favoritism at work: Insider bias in Turkish national journals. Scientometrics, 127 , 2547–2576. https://doi.org/10.1007/s11192-022-04355-0

Ünal, A. F. (2017). Türkiye yükseköğretim alanında rakip kurumsal mantıklar: Akademik performans kriterlerinde çeşitlilik ve yayın üretkenliği üzerine etkisi. Amme İdaresi Dergisi, 50 (4), 83–114.

Uzun, A. (1998). A scientometric profile of Social Sciences research in Turkey. International Information & Library Review, 30 (3), 169–184. https://doi.org/10.1006/iilr.1998.0093

Uzuner, S. (2008). Multilingual scholars’ participation in core/global academic communities: A literature review. Journal of English for Academic Purposes, 7 (4), 250–263. https://doi.org/10.1016/j.jeap.2008.10.007

Van Arensbergen, P., Van der Weijden, I., & Van den Besselaar, P. (2012). Gender differences in scientific productivity: A persisting phenomenon? Scientometrics, 93 (3), 857–868.

Velema, T. A. (2012). The contingent nature of brain gain and brain circulation: Their foreign context and the impact of return scientists on the scientific community in their country of origin. Scientometrics, 93 , 893–913. https://doi.org/10.1007/s11192-012-0751-4

Weisshaar, K. (2017). Publish and perish? An assessment of gender gaps in promotion to tenure in academia. Social Forces, 96 (2), 529–560. https://doi.org/10.1093/sf/sox052

Winclawska, B. M. (1996). Polish Sociology Citation Index (principles for creation and the first results). Scientometrics, 35 , 387–391. https://doi.org/10.1007/BF02016909

Download references

Acknowledgements

The authors wish to thank Michele Meoli for his insightful feedback.

Author information

Authors and affiliations.

Faculty of Economics and Administrative Sciences, Zonguldak Bülent Ecevit University, Zonguldak, Turkey

Lokman Tutuncu

Department of Political Science and Management, University of Agder, Universitetsveien 25, 4630, Kristiansand, Norway

Marco Seeber

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Lokman Tutuncu .

Ethics declarations

Ethics approval and consent to participate.

Not applicable as the research does not involve human or animal subjects.

Consent for publication

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Table 7 , 8 , 9 and 10 .

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Tutuncu, L., Seeber, M. The importance of international and national publications for promotion and the impact of recruitment policies. High Educ (2024). https://doi.org/10.1007/s10734-024-01306-x

Download citation

Accepted : 02 September 2024

Published : 09 September 2024

DOI : https://doi.org/10.1007/s10734-024-01306-x

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Academic careers
• International and local publications
• Scientific mobility
• Promotion policies
• Full professor
• Research productivity
• Find a journal
• Publish with us
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Indian J Sex Transm Dis AIDS
• v.35(2); Jul-Dec 2014

Reviewing literature for research: Doing it the right way

Shital amin poojary.

Department of Dermatology, K J Somaiya Medical College, Mumbai, Maharashtra, India

Jimish Deepak Bagadia

In an era of information overload, it is important to know how to obtain the required information and also to ensure that it is reliable information. Hence, it is essential to understand how to perform a systematic literature search. This article focuses on reliable literature sources and how to make optimum use of these in dermatology and venereology.

INTRODUCTION

A thorough review of literature is not only essential for selecting research topics, but also enables the right applicability of a research project. Most importantly, a good literature search is the cornerstone of practice of evidence based medicine. Today, everything is available at the click of a mouse or at the tip of the fingertips (or the stylus). Google is often the Go-To search website, the supposed answer to all questions in the universe. However, the deluge of information available comes with its own set of problems; how much of it is actually reliable information? How much are the search results that the search string threw up actually relevant? Did we actually find what we were looking for? Lack of a systematic approach can lead to a literature review ending up as a time-consuming and at times frustrating process. Hence, whether it is for research projects, theses/dissertations, case studies/reports or mere wish to obtain information; knowing where to look, and more importantly, how to look, is of prime importance today.

Literature search

Fink has defined research literature review as a “systematic, explicit and reproducible method for identifying, evaluating, and synthesizing the existing body of completed and recorded work produced by researchers, scholars and practitioners.”[ 1 ]

Review of research literature can be summarized into a seven step process: (i) Selecting research questions/purpose of the literature review (ii) Selecting your sources (iii) Choosing search terms (iv) Running your search (v) Applying practical screening criteria (vi) Applying methodological screening criteria/quality appraisal (vii) Synthesizing the results.[ 1 ]

This article will primarily concentrate on refining techniques of literature search.

Sources for literature search are enumerated in Table 1 .

Sources for literature search

PubMed is currently the most widely used among these as it contains over 23 million citations for biomedical literature and has been made available free by National Center for Biotechnology Information (NCBI), U.S. National Library of Medicine. However, the availability of free full text articles depends on the sources. Use of options such as advanced search, medical subject headings (MeSH) terms, free full text, PubMed tutorials, and single citation matcher makes the database extremely user-friendly [ Figure 1 ]. It can also be accessed on the go through mobiles using “PubMed Mobile.” One can also create own account in NCBI to save searches and to use certain PubMed tools.

PubMed home page showing location of different tools which can be used for an efficient literature search

Tips for efficient use of PubMed search:[ 2 , 3 , 4 ]

Use of field and Boolean operators

When one searches using key words, all articles containing the words show up, many of which may not be related to the topic. Hence, the use of operators while searching makes the search more specific and less cumbersome. Operators are of two types: Field operators and Boolean operators, the latter enabling us to combine more than one concept, thereby making the search highly accurate. A few key operators that can be used in PubMed are shown in Tables ​ Tables2 2 and ​ and3 3 and illustrated in Figures ​ Figures2 2 and ​ and3 3 .

Field operators used in PubMed search

Boolean operators used in PubMed search

PubMed search results page showing articles on donovanosis using the field operator [TIAB]; it shows all articles which have the keyword “donovanosis” in either title or abstract of the article

PubMed search using Boolean operators ‘AND’, ‘NOT’; To search for articles on treatment of lepra reaction other than steroids, after clicking the option ‘Advanced search’ on the home page, one can build the search using ‘AND’ option for treatment and ‘NOT’ option for steroids to omit articles on steroid treatment in lepra reaction

Use of medical subject headings terms

These are very specific and standardized terms used by indexers to describe every article in PubMed and are added to the record of every article. A search using MeSH will show all articles about the topic (or keywords), but will not show articles only containing these keywords (these articles may be about an entirely different topic, but still may contain your keywords in another context in any part of the article). This will make your search more specific. Within the topic, specific subheadings can be added to the search builder to refine your search [ Figure 4 ]. For example, MeSH terms for treatment are therapy and therapeutics.

PubMed search using medical subject headings (MeSH) terms for management of gonorrhea. Click on MeSH database ( Figure 1 ) →In the MeSH search box type gonorrhea and click search. Under the MeSH term gonorrhea, there will be a list of subheadings; therapy, prevention and control, click the relevant check boxes and add to search builder →Click on search →All articles on therapy, prevention and control of gonorrhea will be displayed. Below the subheadings, there are two options: (1) Restrict to medical subject headings (MeSH) major topic and (2) do not include MeSH terms found below this term in the MeSH hierarchy. These can be used to further refine the search results so that only articles which are majorly about treatment of gonorrhea will be displayed

Two additional options can be used to further refine MeSH searches. These are located below the subheadings for a MeSH term: (1) Restrict to MeSH major topic; checking this box will retrieve articles which are majorly about the search term and are therefore, more focused and (2) Do not include MeSH terms found below this term in the MeSH hierarchy. This option will again give you more focused articles as it excludes the lower specific terms [ Figure 4 ].

Similar feature is available with Cochrane library (also called MeSH), EMBASE (known as EMTREE) and PsycINFO (Thesaurus of Psychological Index Terms).

Saving your searches

Any search that one has performed can be saved by using the ‘Send to’ option and can be saved as a simple word file [ Figure 5 ]. Alternatively, the ‘Save Search’ button (just below the search box) can be used. However, it is essential to set up an NCBI account and log in to NCBI for this. One can even choose to have E-mail updates of new articles in the topic of interest.

Saving PubMed searches. A simple option is to click on the dropdown box next to ‘Send to’ option and then choose among the options. It can be saved as a text or word file by choosing ‘File’ option. Another option is the “Save search” option below the search box but this will require logging into your National Center for Biotechnology Information account. This however allows you to set up alerts for E-mail updates for new articles

Single citation matcher

This is another important tool that helps to find the genuine original source of a particular research work (when few details are known about the title/author/publication date/place/journal) and cite the reference in the most correct manner [ Figure 6 ].

Single citation matcher: Click on “Single citation matcher” on PubMed Home page. Type available details of the required reference in the boxes to get the required citation

Full text articles

In any search clicking on the link “free full text” (if present) gives you free access to the article. In some instances, though the published article may not be available free, the author manuscript may be available free of charge. Furthermore, PubMed Central articles are available free of charge.

Managing filters

Filters can be used to refine a search according to type of article required or subjects of research. One can specify the type of article required such as clinical trial, reviews, free full text; these options are available on a typical search results page. Further specialized filters are available under “manage filters:” e.g., articles confined to certain age groups (properties option), “Links” to other databases, article specific to particular journals, etc. However, one needs to have an NCBI account and log in to access this option [ Figure 7 ].

Managing filters. Simple filters are available on the ‘search results’ page. One can choose type of article, e.g., clinical trial, reviews etc. Further options are available in the “Manage filters” option, but this requires logging into National Center for Biotechnology Information account

The Cochrane library

Although reviews are available in PubMed, for systematic reviews and meta-analysis, Cochrane library is a much better resource. The Cochrane library is a collection of full length systematic reviews, which can be accessed for free in India, thanks to Indian Council of Medical Research renewing the license up to 2016, benefitting users all over India. It is immensely helpful in finding detailed high quality research work done in a particular field/topic [ Figure 8 ].

Cochrane library is a useful resource for reliable, systematic reviews. One can choose the type of reviews required, including trials

An important tool that must be used while searching for research work is screening. Screening helps to improve the accuracy of search results. It is of two types: (1) Practical: To identify a broad range of potentially useful studies. Examples: Date of publication (last 5 years only; gives you most recent updates), participants or subjects (humans above 18 years), publication language (English only) (2) methodological: To identify best available studies (for example, excluding studies not involving control group or studies with only randomized control trials).

Selecting the right quality of literature is the key to successful research literature review. The quality can be estimated by what is known as “The Evidence Pyramid.” The level of evidence of references obtained from the aforementioned search tools are depicted in Figure 9 . Systematic reviews obtained from Cochrane library constitute level 1 evidence.

Evidence pyramid: Depicting the level of evidence of references obtained from the aforementioned search tools

Thus, a systematic literature review can help not only in setting up the basis of a good research with optimal use of available information, but also in practice of evidence-based medicine.

Source of Support: Nil.

Conflict of Interest: None declared.