qualitative and quantitative research google scholar

• Search Menu
• Sign in through your institution
• Advance articles
• Mini-reviews
• ESHRE Pages
• Editor's Choice
• Supplements
• Author Guidelines
• Submission Site
• Reasons to Publish
• Open Access
• Advertising and Corporate Services
• Advertising
• Reprints and ePrints
• Sponsored Supplements
• Branded Books
• Journals Career Network
• About Human Reproduction
• About the European Society of Human Reproduction and Embryology
• Editorial Board
• Self-Archiving Policy
• Dispatch Dates
• Contact ESHRE
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Introduction, when to use qualitative research, how to judge qualitative research, conclusions, authors' roles, conflict of interest.

• < Previous

Qualitative research methods: when to use them and how to judge them

• Article contents
• Figures & tables
• Supplementary Data

K. Hammarberg, M. Kirkman, S. de Lacey, Qualitative research methods: when to use them and how to judge them, Human Reproduction , Volume 31, Issue 3, March 2016, Pages 498–501, https://doi.org/10.1093/humrep/dev334

• Permissions Icon Permissions

In March 2015, an impressive set of guidelines for best practice on how to incorporate psychosocial care in routine infertility care was published by the ESHRE Psychology and Counselling Guideline Development Group ( ESHRE Psychology and Counselling Guideline Development Group, 2015 ). The authors report that the guidelines are based on a comprehensive review of the literature and we congratulate them on their meticulous compilation of evidence into a clinically useful document. However, when we read the methodology section, we were baffled and disappointed to find that evidence from research using qualitative methods was not included in the formulation of the guidelines. Despite stating that ‘qualitative research has significant value to assess the lived experience of infertility and fertility treatment’, the group excluded this body of evidence because qualitative research is ‘not generally hypothesis-driven and not objective/neutral, as the researcher puts him/herself in the position of the participant to understand how the world is from the person's perspective’.

Qualitative and quantitative research methods are often juxtaposed as representing two different world views. In quantitative circles, qualitative research is commonly viewed with suspicion and considered lightweight because it involves small samples which may not be representative of the broader population, it is seen as not objective, and the results are assessed as biased by the researchers' own experiences or opinions. In qualitative circles, quantitative research can be dismissed as over-simplifying individual experience in the cause of generalisation, failing to acknowledge researcher biases and expectations in research design, and requiring guesswork to understand the human meaning of aggregate data.

As social scientists who investigate psychosocial aspects of human reproduction, we use qualitative and quantitative methods, separately or together, depending on the research question. The crucial part is to know when to use what method.

The peer-review process is a pillar of scientific publishing. One of the important roles of reviewers is to assess the scientific rigour of the studies from which authors draw their conclusions. If rigour is lacking, the paper should not be published. As with research using quantitative methods, research using qualitative methods is home to the good, the bad and the ugly. It is essential that reviewers know the difference. Rejection letters are hard to take but more often than not they are based on legitimate critique. However, from time to time it is obvious that the reviewer has little grasp of what constitutes rigour or quality in qualitative research. The first author (K.H.) recently submitted a paper that reported findings from a qualitative study about fertility-related knowledge and information-seeking behaviour among people of reproductive age. In the rejection letter one of the reviewers (not from Human Reproduction ) lamented, ‘Even for a qualitative study, I would expect that some form of confidence interval and paired t-tables analysis, etc. be used to analyse the significance of results'. This comment reveals the reviewer's inappropriate application to qualitative research of criteria relevant only to quantitative research.

In this commentary, we give illustrative examples of questions most appropriately answered using qualitative methods and provide general advice about how to appraise the scientific rigour of qualitative studies. We hope this will help the journal's reviewers and readers appreciate the legitimate place of qualitative research and ensure we do not throw the baby out with the bath water by excluding or rejecting papers simply because they report the results of qualitative studies.

In psychosocial research, ‘quantitative’ research methods are appropriate when ‘factual’ data are required to answer the research question; when general or probability information is sought on opinions, attitudes, views, beliefs or preferences; when variables can be isolated and defined; when variables can be linked to form hypotheses before data collection; and when the question or problem is known, clear and unambiguous. Quantitative methods can reveal, for example, what percentage of the population supports assisted conception, their distribution by age, marital status, residential area and so on, as well as changes from one survey to the next ( Kovacs et al. , 2012 ); the number of donors and donor siblings located by parents of donor-conceived children ( Freeman et al. , 2009 ); and the relationship between the attitude of donor-conceived people to learning of their donor insemination conception and their family ‘type’ (one or two parents, lesbian or heterosexual parents; Beeson et al. , 2011 ).

In contrast, ‘qualitative’ methods are used to answer questions about experience, meaning and perspective, most often from the standpoint of the participant. These data are usually not amenable to counting or measuring. Qualitative research techniques include ‘small-group discussions’ for investigating beliefs, attitudes and concepts of normative behaviour; ‘semi-structured interviews’, to seek views on a focused topic or, with key informants, for background information or an institutional perspective; ‘in-depth interviews’ to understand a condition, experience, or event from a personal perspective; and ‘analysis of texts and documents’, such as government reports, media articles, websites or diaries, to learn about distributed or private knowledge.

Qualitative methods have been used to reveal, for example, potential problems in implementing a proposed trial of elective single embryo transfer, where small-group discussions enabled staff to explain their own resistance, leading to an amended approach ( Porter and Bhattacharya, 2005 ). Small-group discussions among assisted reproductive technology (ART) counsellors were used to investigate how the welfare principle is interpreted and practised by health professionals who must apply it in ART ( de Lacey et al. , 2015 ). When legislative change meant that gamete donors could seek identifying details of people conceived from their gametes, parents needed advice on how best to tell their children. Small-group discussions were convened to ask adolescents (not known to be donor-conceived) to reflect on how they would prefer to be told ( Kirkman et al. , 2007 ).

When a population cannot be identified, such as anonymous sperm donors from the 1980s, a qualitative approach with wide publicity can reach people who do not usually volunteer for research and reveal (for example) their attitudes to proposed legislation to remove anonymity with retrospective effect ( Hammarberg et al. , 2014 ). When researchers invite people to talk about their reflections on experience, they can sometimes learn more than they set out to discover. In describing their responses to proposed legislative change, participants also talked about people conceived as a result of their donations, demonstrating various constructions and expectations of relationships ( Kirkman et al. , 2014 ).

Interviews with parents in lesbian-parented families generated insight into the diverse meanings of the sperm donor in the creation and life of the family ( Wyverkens et al. , 2014 ). Oral and written interviews also revealed the embarrassment and ambivalence surrounding sperm donors evident in participants in donor-assisted conception ( Kirkman, 2004 ). The way in which parents conceptualise unused embryos and why they discard rather than donate was explored and understood via in-depth interviews, showing how and why the meaning of those embryos changed with parenthood ( de Lacey, 2005 ). In-depth interviews were also used to establish the intricate understanding by embryo donors and recipients of the meaning of embryo donation and the families built as a result ( Goedeke et al. , 2015 ).

It is possible to combine quantitative and qualitative methods, although great care should be taken to ensure that the theory behind each method is compatible and that the methods are being used for appropriate reasons. The two methods can be used sequentially (first a quantitative then a qualitative study or vice versa), where the first approach is used to facilitate the design of the second; they can be used in parallel as different approaches to the same question; or a dominant method may be enriched with a small component of an alternative method (such as qualitative interviews ‘nested’ in a large survey). It is important to note that free text in surveys represents qualitative data but does not constitute qualitative research. Qualitative and quantitative methods may be used together for corroboration (hoping for similar outcomes from both methods), elaboration (using qualitative data to explain or interpret quantitative data, or to demonstrate how the quantitative findings apply in particular cases), complementarity (where the qualitative and quantitative results differ but generate complementary insights) or contradiction (where qualitative and quantitative data lead to different conclusions). Each has its advantages and challenges ( Brannen, 2005 ).

Qualitative research is gaining increased momentum in the clinical setting and carries different criteria for evaluating its rigour or quality. Quantitative studies generally involve the systematic collection of data about a phenomenon, using standardized measures and statistical analysis. In contrast, qualitative studies involve the systematic collection, organization, description and interpretation of textual, verbal or visual data. The particular approach taken determines to a certain extent the criteria used for judging the quality of the report. However, research using qualitative methods can be evaluated ( Dixon-Woods et al. , 2006 ; Young et al. , 2014 ) and there are some generic guidelines for assessing qualitative research ( Kitto et al. , 2008 ).

Although the terms ‘reliability’ and ‘validity’ are contentious among qualitative researchers ( Lincoln and Guba, 1985 ) with some preferring ‘verification’, research integrity and robustness are as important in qualitative studies as they are in other forms of research. It is widely accepted that qualitative research should be ethical, important, intelligibly described, and use appropriate and rigorous methods ( Cohen and Crabtree, 2008 ). In research investigating data that can be counted or measured, replicability is essential. When other kinds of data are gathered in order to answer questions of personal or social meaning, we need to be able to capture real-life experiences, which cannot be identical from one person to the next. Furthermore, meaning is culturally determined and subject to evolutionary change. The way of explaining a phenomenon—such as what it means to use donated gametes—will vary, for example, according to the cultural significance of ‘blood’ or genes, interpretations of marital infidelity and religious constructs of sexual relationships and families. Culture may apply to a country, a community, or other actual or virtual group, and a person may be engaged at various levels of culture. In identifying meaning for members of a particular group, consistency may indeed be found from one research project to another. However, individuals within a cultural group may present different experiences and perceptions or transgress cultural expectations. That does not make them ‘wrong’ or invalidate the research. Rather, it offers insight into diversity and adds a piece to the puzzle to which other researchers also contribute.

In qualitative research the objective stance is obsolete, the researcher is the instrument, and ‘subjects’ become ‘participants’ who may contribute to data interpretation and analysis ( Denzin and Lincoln, 1998 ). Qualitative researchers defend the integrity of their work by different means: trustworthiness, credibility, applicability and consistency are the evaluative criteria ( Leininger, 1994 ).

Trustworthiness

A report of a qualitative study should contain the same robust procedural description as any other study. The purpose of the research, how it was conducted, procedural decisions, and details of data generation and management should be transparent and explicit. A reviewer should be able to follow the progression of events and decisions and understand their logic because there is adequate description, explanation and justification of the methodology and methods ( Kitto et al. , 2008 )

Credibility

Credibility is the criterion for evaluating the truth value or internal validity of qualitative research. A qualitative study is credible when its results, presented with adequate descriptions of context, are recognizable to people who share the experience and those who care for or treat them. As the instrument in qualitative research, the researcher defends its credibility through practices such as reflexivity (reflection on the influence of the researcher on the research), triangulation (where appropriate, answering the research question in several ways, such as through interviews, observation and documentary analysis) and substantial description of the interpretation process; verbatim quotations from the data are supplied to illustrate and support their interpretations ( Sandelowski, 1986 ). Where excerpts of data and interpretations are incongruent, the credibility of the study is in doubt.

Applicability

Applicability, or transferability of the research findings, is the criterion for evaluating external validity. A study is considered to meet the criterion of applicability when its findings can fit into contexts outside the study situation and when clinicians and researchers view the findings as meaningful and applicable in their own experiences.

Larger sample sizes do not produce greater applicability. Depth may be sacrificed to breadth or there may be too much data for adequate analysis. Sample sizes in qualitative research are typically small. The term ‘saturation’ is often used in reference to decisions about sample size in research using qualitative methods. Emerging from grounded theory, where filling theoretical categories is considered essential to the robustness of the developing theory, data saturation has been expanded to describe a situation where data tend towards repetition or where data cease to offer new directions and raise new questions ( Charmaz, 2005 ). However, the legitimacy of saturation as a generic marker of sampling adequacy has been questioned ( O'Reilly and Parker, 2013 ). Caution must be exercised to ensure that a commitment to saturation does not assume an ‘essence’ of an experience in which limited diversity is anticipated; each account is likely to be subtly different and each ‘sample’ will contribute to knowledge without telling the whole story. Increasingly, it is expected that researchers will report the kind of saturation they have applied and their criteria for recognising its achievement; an assessor will need to judge whether the choice is appropriate and consistent with the theoretical context within which the research has been conducted.

Sampling strategies are usually purposive, convenient, theoretical or snowballed. Maximum variation sampling may be used to seek representation of diverse perspectives on the topic. Homogeneous sampling may be used to recruit a group of participants with specified criteria. The threat of bias is irrelevant; participants are recruited and selected specifically because they can illuminate the phenomenon being studied. Rather than being predetermined by statistical power analysis, qualitative study samples are dependent on the nature of the data, the availability of participants and where those data take the investigator. Multiple data collections may also take place to obtain maximum insight into sensitive topics. For instance, the question of how decisions are made for embryo disposition may involve sampling within the patient group as well as from scientists, clinicians, counsellors and clinic administrators.

Consistency

Consistency, or dependability of the results, is the criterion for assessing reliability. This does not mean that the same result would necessarily be found in other contexts but that, given the same data, other researchers would find similar patterns. Researchers often seek maximum variation in the experience of a phenomenon, not only to illuminate it but also to discourage fulfilment of limited researcher expectations (for example, negative cases or instances that do not fit the emerging interpretation or theory should be actively sought and explored). Qualitative researchers sometimes describe the processes by which verification of the theoretical findings by another team member takes place ( Morse and Richards, 2002 ).

Research that uses qualitative methods is not, as it seems sometimes to be represented, the easy option, nor is it a collation of anecdotes. It usually involves a complex theoretical or philosophical framework. Rigorous analysis is conducted without the aid of straightforward mathematical rules. Researchers must demonstrate the validity of their analysis and conclusions, resulting in longer papers and occasional frustration with the word limits of appropriate journals. Nevertheless, we need the different kinds of evidence that is generated by qualitative methods. The experience of health, illness and medical intervention cannot always be counted and measured; researchers need to understand what they mean to individuals and groups. Knowledge gained from qualitative research methods can inform clinical practice, indicate how to support people living with chronic conditions and contribute to community education and awareness about people who are (for example) experiencing infertility or using assisted conception.

Each author drafted a section of the manuscript and the manuscript as a whole was reviewed and revised by all authors in consultation.

No external funding was either sought or obtained for this study.

The authors have no conflicts of interest to declare.

Beeson D , Jennings P , Kramer W . Offspring searching for their sperm donors: how family types shape the process . Hum Reprod 2011 ; 26 : 2415 – 2424 .

Google Scholar

Brannen J . Mixing methods: the entry of qualitative and quantitative approaches into the research process . Int J Soc Res Methodol 2005 ; 8 : 173 – 184 .

Charmaz K . Grounded Theory in the 21st century; applications for advancing social justice studies . In: Denzin NK , Lincoln YS (eds). The Sage Handbook of Qualitative Research . California : Sage Publications Inc. , 2005 .

Google Preview

Cohen D , Crabtree B . Evaluative criteria for qualitative research in health care: controversies and recommendations . Ann Fam Med 2008 ; 6 : 331 – 339 .

de Lacey S . Parent identity and ‘virtual’ children: why patients discard rather than donate unused embryos . Hum Reprod 2005 ; 20 : 1661 – 1669 .

de Lacey SL , Peterson K , McMillan J . Child interests in assisted reproductive technology: how is the welfare principle applied in practice? Hum Reprod 2015 ; 30 : 616 – 624 .

Denzin N , Lincoln Y . Entering the field of qualitative research . In: Denzin NK , Lincoln YS (eds). The Landscape of Qualitative Research: Theories and Issues . Thousand Oaks : Sage , 1998 , 1 – 34 .

Dixon-Woods M , Bonas S , Booth A , Jones DR , Miller T , Shaw RL , Smith JA , Young B . How can systematic reviews incorporate qualitative research? A critical perspective . Qual Res 2006 ; 6 : 27 – 44 .

ESHRE Psychology and Counselling Guideline Development Group . Routine Psychosocial Care in Infertility and Medically Assisted Reproduction: A Guide for Fertility Staff , 2015 . http://www.eshre.eu/Guidelines-and-Legal/Guidelines/Psychosocial-care-guideline.aspx .

Freeman T , Jadva V , Kramer W , Golombok S . Gamete donation: parents' experiences of searching for their child's donor siblings or donor . Hum Reprod 2009 ; 24 : 505 – 516 .

Goedeke S , Daniels K , Thorpe M , Du Preez E . Building extended families through embryo donation: the experiences of donors and recipients . Hum Reprod 2015 ; 30 : 2340 – 2350 .

Hammarberg K , Johnson L , Bourne K , Fisher J , Kirkman M . Proposed legislative change mandating retrospective release of identifying information: consultation with donors and Government response . Hum Reprod 2014 ; 29 : 286 – 292 .

Kirkman M . Saviours and satyrs: ambivalence in narrative meanings of sperm provision . Cult Health Sex 2004 ; 6 : 319 – 336 .

Kirkman M , Rosenthal D , Johnson L . Families working it out: adolescents' views on communicating about donor-assisted conception . Hum Reprod 2007 ; 22 : 2318 – 2324 .

Kirkman M , Bourne K , Fisher J , Johnson L , Hammarberg K . Gamete donors' expectations and experiences of contact with their donor offspring . Hum Reprod 2014 ; 29 : 731 – 738 .

Kitto S , Chesters J , Grbich C . Quality in qualitative research . Med J Aust 2008 ; 188 : 243 – 246 .

Kovacs GT , Morgan G , Levine M , McCrann J . The Australian community overwhelmingly approves IVF to treat subfertility, with increasing support over three decades . Aust N Z J Obstetr Gynaecol 2012 ; 52 : 302 – 304 .

Leininger M . Evaluation criteria and critique of qualitative research studies . In: Morse J (ed). Critical Issues in Qualitative Research Methods . Thousand Oaks : Sage , 1994 , 95 – 115 .

Lincoln YS , Guba EG . Naturalistic Inquiry . Newbury Park, CA : Sage Publications , 1985 .

Morse J , Richards L . Readme First for a Users Guide to Qualitative Methods . Thousand Oaks : Sage , 2002 .

O'Reilly M , Parker N . ‘Unsatisfactory saturation’: a critical exploration of the notion of saturated sample sizes in qualitative research . Qual Res 2013 ; 13 : 190 – 197 .

Porter M , Bhattacharya S . Investigation of staff and patients' opinions of a proposed trial of elective single embryo transfer . Hum Reprod 2005 ; 20 : 2523 – 2530 .

Sandelowski M . The problem of rigor in qualitative research . Adv Nurs Sci 1986 ; 8 : 27 – 37 .

Wyverkens E , Provoost V , Ravelingien A , De Sutter P , Pennings G , Buysse A . Beyond sperm cells: a qualitative study on constructed meanings of the sperm donor in lesbian families . Hum Reprod 2014 ; 29 : 1248 – 1254 .

Young K , Fisher J , Kirkman M . Women's experiences of endometriosis: a systematic review of qualitative research . J Fam Plann Reprod Health Care 2014 ; 41 : 225 – 234 .

• conflict of interest
• credibility
• qualitative research
• quantitative methods

Email alerts

Citing articles via.

• Recommend to your Library

Affiliations

• Online ISSN 1460-2350
• Copyright © 2024 European Society of Human Reproduction and Embryology
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.

Strengths and Limitations of Qualitative and Quantitative Research Methods

• September 2017
• 3(9):369-387

• Instituto Superior Politécnico Gaya
• This person is not on ResearchGate, or hasn't claimed this research yet.

Abstract and Figures

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

• Pepple, Grace Jamie
• Jerome Inemesit Emmanuel
• Roberto Robutti
• Nadav L. Sprague
• Isabel B. Fan
• Michelle Dandeneau
• Diana Hernández

• عامر شحاده شحاده
• انوار ابو هنود

• Θεογνωσία Κουνατίδου
• Αντώνης Λιοναράκης

• Chhoy Vannary

• José Augusto Monteiro

• C.R.Kothari
• Hamza Alshenqeeti
• Looi Theam Choy
• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

Qualitative and Quantitative Methods in Research

• Living reference work entry
• First Online: 10 August 2022
• Cite this living reference work entry

• Christina Mazzola Nicols 3  

56 Accesses

1 Definition

Research that aims to gather an in-depth understanding of human behavior and the factors contributing to the behavior. Frequent methods of qualitative data collection include observation, in-depth interviews, and focus groups. Words, pictures, or objects comprise the resulting data.

A qualitative research method used to understand behaviors in a natural setting. The researcher relies on their observations of the subject to collect and analyze data.

A qualitative method in which an interviewer directs a series of questions to the person he/she is interviewing, typically either in person or by telephone. As an interview progresses, questions tend to move from the general to the specific.

Interviews conducted in small groups of participants instead of individuals. Typically, a trained moderator leads a focused discussion among eight to ten participants over the course of 1–2 h. Focus groups can be conducted in...

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Institutional subscriptions

Further Readings

Douglas Evans, W. (2016). Social marketing research for global public health: Methods and technologies . Oxford Press.

Book   Google Scholar  

Hicks, N. J., & Nicols, C. M. (2016). Health industry communication: New media, new methods, new message (2nd ed.). Jones & Bartlett Learning.

Google Scholar  

Siegel, M., & Lotenberg, L. D. (2007). Marketing public health: Strategies to promote social change (2nd ed.). Jones and Bartlett Publishers.

Download references

Author information

Authors and affiliations.

Hager Sharp, Washington, DC, USA

Christina Mazzola Nicols

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Christina Mazzola Nicols .

Section Editor information

Milken Institute School of Public Health, George Washington University, Washington, D. C., USA

American University of Beirut, Beirut, Lebanon

Marco Bardus

Rights and permissions

Reprints and permissions

Copyright information

© 2021 The Author(s), under exclusive licence to Springer Nature Switzerland AG

About this entry

Cite this entry.

Nicols, C.M. (2021). Qualitative and Quantitative Methods in Research. In: The Palgrave Encyclopedia of Social Marketing. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-030-14449-4_154-1

Download citation

DOI : https://doi.org/10.1007/978-3-030-14449-4_154-1

Received : 11 May 2020

Accepted : 07 July 2021

Published : 10 August 2022

Publisher Name : Palgrave Macmillan, Cham

Print ISBN : 978-3-030-14449-4

Online ISBN : 978-3-030-14449-4

eBook Packages : Springer Reference Business and Management Reference Module Humanities and Social Sciences Reference Module Business, Economics and Social Sciences

• Publish with us

Policies and ethics

• Find a journal
• Track your research

New Scholar

• Tips for Publishing
• Publishing Process Overview
• Selecting a Journal Overview
• Conference Presentation or Poster Session Overview
• Predatory Publishers

Qualitative vs Quantitative Research

Types of empirical research, list of journals friendly to qualitative work.

• Understanding Research Impact This link opens in a new window

Reproducibility Librarian

Make an Appointment with Me

Do you want to meet with me in-person or online?

Quantitative Research is used to quantify the problem by way of generating numerical data or data that can be transformed into useable statistics. It is used to quantify attitudes, opinions, behaviors, and other defined variables – and generalize results from a larger sample population.

Qualitative Research is primarily exploratory research.  It is used to gain an understanding of underlying reasons, opinions, and motivations. This data is usually gathered using conversational methods such as interviews or focus groups.

Some journals and even some disciplines may have a preference as to what type of empirical research they wish to publish.  Some authors who have written an article that is primarily qualitative in nature, may seek out journals that are "qualitative research friendly." We have listed a few such journals below. 

• International Journal of Qualitative Methods
• Journal of Mixed Methods Research
• Quality & Quantity
• List of qualitative research journals compiled by Saint Louis University

You can also look through the last few issues of a journal to see if the articles they publish tend to be more qualitative or quantitative in nature.

• << Previous: Predatory Publishers
• Next: Understanding Research Impact >>
• Last Updated: Sep 17, 2024 1:22 PM
• URL: https://libguides.lib.rochester.edu/newscholar

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Journal Proposal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

The performance and qualitative evaluation of scientific work at research universities: a focus on the types of university and research.

1. Introduction

2. materials and methods, 3. literature review, 4.1. description of the research object and university research data analysis, 4.2. survey result analysis, 5. discussion, 6. conclusions, author contributions, institutional review board statement, informed consent statement, data availability statement, acknowledgments, conflicts of interest, appendix a. survey form for indicators for assessing the quality of the scientific research, appendix b. university research processes.

Click here to enlarge figure

• Etzkowitz, H. The Triple Helix University—Industry—Government Innovation in Action ; Routledge: New York, NY, USA, 2008; p. 225. ISBN 978-0415964500. [ Google Scholar ]
• Tang, H.H. The strategic role of world-class universities in regional innovation system: China’s Greater Bay Area and Hong Kong’s academic profession. Asian Educ. Dev. Stud. 2020 , 11 , 7–22. [ Google Scholar ] [ CrossRef ]
• National Research Council, USA. Research Universities and the Future of America: Ten Breakthrough Actions Vital to Our Nation’s Prosperity and Security (Report) ; National Research Council: Washington, DC, USA, 2012; p. 225. ISBN 978-0-309-25639-1.
• Powell, J.J.W.; Dusdal, J. The European Center of Science Productivity: Research Universities and Institutes in France, Germany, and the United Kingdom. Century Sci. Int. Perspect. Educ. Soc. 2017 , 33 , 55–83. [ Google Scholar ] [ CrossRef ]
• Intarakumnerd, P.; Goto, A. Role of public research institutes in national innovation systems in industrialized countries: The cases of Fraunhofer, NIST, CSIRO, AIST, and ITRI. Res. Policy 2018 , 47 , 1309–1320. [ Google Scholar ] [ CrossRef ]
• Vlasova, V.V.; Gokhberg, L.M.; Ditkovsky, K.A.; Kotsemir, M.N.; Kuznetsova, I.A.; Martynova, S.V.; Nesterenko, A.V.; Pakhomov, S.I.; Polyakova, V.V.; Ratay, T.V.; et al. Science Indicators: 2023: Statistical Collection ; National Research University Higher School of Economics: Moscow, Russia, 2023; p. 416. ISBN 978-5-7598-2765-8. (In Russian) [ Google Scholar ]
• Textor, C. Breakdown of R&D spending in China 2017–2022, by Entity. Available online: https://www.statista.com/statistics/1465556/research-and-development-expenditure-in-china-distribution-by-entity/ (accessed on 12 August 2024).
• Chen, K.; Zhang, C.; Feng, Z.; Zhang, Y.; Ning, L. Technology transfer systems and modes of national research institutes: Evidence from the Chinese Academy of Sciences. Res. Policy 2022 , 51 , 104471. [ Google Scholar ] [ CrossRef ]
• Vasilev, Y.; Vasileva, P.; Batova, O.; Tsvetkova, A. Assessment of Factors Influencing Educational Effectiveness in Higher Educational Institutions. Sustainability 2024 , 16 , 4886. [ Google Scholar ] [ CrossRef ]
• Ilyushin, Y.V.; Pervukhin, D.A.; Afanaseva, O.V. Application of the theory of systems with distributed parameters for mineral complex facilities management. ARPN J. Eng. Appl. Sci. 2019 , 14 , 3852–3864. [ Google Scholar ]
• Raupov, I.; Burkhanov, R.; Lutfullin, A.; Maksyutin, A.; Lebedev, A.; Safiullina, E. Experience in the Application of Hydrocarbon Optical Studies in Oil Field Development. Energies 2022 , 15 , 3626. [ Google Scholar ] [ CrossRef ]
• Rudnik, S.N.; Afanasev, V.G.; Samylovskaya, E.A. 250 years in the service of the Fatherland: Empress Catherine II Saint Petersburg Mining university in facts and figures. J. Min. Inst. 2023 , 263 , 810–830. [ Google Scholar ]
• Olcay, G.F.; Bulu, M. Is measuring the knowledge creation of universities possible? A review of university rankings. Technol. Forecast. Soc. Chang. 2017 , 123 , 153–160. [ Google Scholar ] [ CrossRef ]
• Lapinskas, A.; Makhova, L.; Zhidikov, V. Responsible resource wealth management in ensuring inclusive growth [Odpowiedzialne zarzdzanie zasobami w zapewnieniu wzrostu wczajcego]. Pol. J. Manag. Stud. 2021 , 23 , 288–304. [ Google Scholar ] [ CrossRef ]
• Peris-Ortiz, M.; García-Hurtado, D.; Román, A.P. Measuring knowledge exploration and exploitation in universities and the relationship with global ranking indicators. Eur. Res. Manag. Bus. Econ. 2023 , 29 , 100212. [ Google Scholar ] [ CrossRef ]
• Ponomariov, B.L.; Boardman, P.C. Influencing scientists’ collaboration and productivity patterns through new institutions: University research centers and scientific and technical human capital. Res. Policy 2010 , 39 , 613–624. [ Google Scholar ] [ CrossRef ]
• Isaeva, N.V.; Borisova, L.V. Comparative analysis of national policies for developing research universities’ campuses. Univ. Manag. Pract. Anal. 2013 , 6 , 74–87. (In Russian) [ Google Scholar ]
• Ponomarenko, T.V.; Marinina, O.A.; Nevskaya, M.A. Innovative learning methods in technical universities: The possibility of forming interdisciplinary competencies. Espacios 2019 , 40 , 1–10. Available online: http://www.revistaespacios.com/a19v40n41/19404116.html (accessed on 10 September 2024).
• Zhang, H.; Patton, D.; Kenney, M. Building global-class universities: Assessing the impact of the 985 Project. Res. Policy 2013 , 42 , 765–775. [ Google Scholar ] [ CrossRef ]
• Shang, J.; Zeng, M.; Zhang, G. Investigating the mentorship effect on the academic success of young scientists: An empirical study of the 985 project universities of China. J. Informetr. 2022 , 16 , 101285. [ Google Scholar ] [ CrossRef ]
• Chen, Y.; Pan, Y.; Liu, H.; Wu, X.; Deng, G. Efficiency analysis of Chinese universities with shared inputs: An aggregated two-stage network DEA approach. Socio-Econ. Plan. Sci. 2023 , 90 , 101728. [ Google Scholar ] [ CrossRef ]
• Mindeli, L.E. Financial Support for the Development of the Scientific and Technological Sphere ; Mindeli, L.E., Chernykh, S.I., Frolova, N.D., Todosiychuk, A.V., Fetisov, V.P., Eds.; Institute for Problems of Science Development of the Russian Academy of Sciences (IPRAN): Moscow, Russia, 2018; p. 215. ISBN 978-5-91294-125-2. (In Russian) [ Google Scholar ]
• Guba, K.S.; Slovogorodsky, N.A. “Publish or Perish” in Russian social sciences: Patterns of co-authorship in “predatory” and “pure” journals. Issues Educ. Educ. Stud. Mosc. 2022 , 4 , 80–106. (In Russian) [ Google Scholar ] [ CrossRef ]
• Slepykh, V.; Lovakov, A.; Yudkevich, M. Academic career after defending a PhD thesis on the example of four branches of Russian science. Issues Educ. Educ. Stud. Mosc. 2022 , 4 , 260–297. (In Russian) [ Google Scholar ] [ CrossRef ]
• Mohrman, K.; Ma, W.; Baker, D. The Research University in Transition: The Emerging Global Model. High. Educ. Policy 2008 , 21 , 5–27. [ Google Scholar ] [ CrossRef ]
• Altbach, P.G. The Road to Academic Excellence: The Making of World-Class Research Universities ; Altbach, P.G., Salmi, J., Eds.; World Bank: Washington, DC, USA, 2011; p. 381. ISBN 978-0-8213-9485-4. [ Google Scholar ] [ CrossRef ]
• Powell, J.J.W.; Fernandez, F.; Crist, J.T.; Dusdal, J.; Zhang, L.; Baker, D.P. Introduction: The Worldwide Triumph of the Research University and Globalizing Science. Century Sci. Int. Perspect. Educ. Soc. 2017 , 33 , 1–36. [ Google Scholar ] [ CrossRef ]
• Colina-Ysea, F.; Pantigoso-Leython, N.; Abad-Lezama, I.; Calla-Vásquez, K.; Chávez-Campó, S.; Sanabria-Boudri, F.M.; Soto-Rivera, C. Implementation of Hybrid Education in Peruvian Public Universities: The Challenges. Educ. Sci. 2024 , 14 , 419. [ Google Scholar ] [ CrossRef ]
• Fernandez, F.; Baker, D.P. Science Production in the United States: An Unexpected Synergy between Mass Higher Education and the Super Research University. Century Sci. Int. Perspect. Educ. Soc. 2021 , 33 , 85–111. [ Google Scholar ] [ CrossRef ]
• Jamil, S. The Challenge of Establishing World-Class Universities: Directions in Development ; World Bank: Washington, DC, USA, 2009; p. 115. ISBN 082-1378-767, 978-0821-3787-62. [ Google Scholar ] [ CrossRef ]
• Mudzakkir, M.; Sukoco, B.; Suwignjo, P. World-class Universities: Past and Future. Int. J. Educ. Manag. 2022 , 36 , 277–295. [ Google Scholar ] [ CrossRef ]
• Tian, L. Rethinking the global orientation of world-class universities from a comparative functional perspective. Int. J. Educ. Dev. 2023 , 96 , 102700. [ Google Scholar ] [ CrossRef ]
• Clark, B.R. Creating Entrepreneurial Universities: Organizational Pathways of Transformation ; Emerald Group Publishing Limited: London, UK, 1998; p. 200. ISBN 978-0 0804-3342-4. [ Google Scholar ]
• Etzkowitz, H. The Entrepreneurial University: Vision and Metrics. Ind. High. Educ. 2016 , 3 , 83–97. [ Google Scholar ] [ CrossRef ]
• Kazin, F.A.; Kondratiev, A.V. Development of the concept of an entrepreneurial university in Russian universities: New assessment tools. Univ. Upr. Prakt. Anal. Univ. Manag. Pract. Anal. 2022 , 26 , 18–41. (In Russian) [ Google Scholar ] [ CrossRef ]
• Berestov, A.V.; Guseva, A.I.; Kalashnik, V.M.; Kaminsky, V.I.; Kireev, S.V.; Sadchikov, S.M. The “national research university” project is a driver of Russian higher education. Vyss. Obraz. V Ross. High. Educ. Russ. 2020 , 29 , 22–34. (In Russian) [ Google Scholar ] [ CrossRef ]
• Matveeva, N.; Ferligoj, A. Scientific Collaboration in Russian Universities before and after the Excellence Initiative Project “5–100”. Scientometrics 2020 , 124 , 2383–2407. [ Google Scholar ] [ CrossRef ]
• Matveeva, N.; Sterligov, I.; Yudkevich, M. The Effect of Russian University Excellence Initiative on Publications and Collaboration Patterns. J. Informetr. 2021 , 15 , 101110. [ Google Scholar ] [ CrossRef ]
• Semenov, V.P.; Mikhailov Yu, I. Challenges and trends of quality management in the context of industrial and mineral resources economy. J. Min. Inst. 2017 , 226 , 497. [ Google Scholar ] [ CrossRef ]
• Litvinenko, V.S.; Bowbrick, I.; Naumov, I.A.; Zaitseva, Z. Global guidelines and requirements for professional competencies of natural resource extraction engineers: Implications for ESG principles and sustainable development goals. J. Clean. Prod. 2022 , 338 , 130530. [ Google Scholar ] [ CrossRef ]
• Semenova, T.; Martínez Santoyo, J.Y. Economic Strategy for Developing the Oil Industry in Mexico by Incorporating Environmental Factors. Sustainability 2024 , 16 , 36. [ Google Scholar ] [ CrossRef ]
• Rozhdestvensky, I.V.; Filimonov, A.V.; Khvorostyanaya, A.S. Methodology for assessing the readiness of higher educational institutions and scientific organizations for technology transfer. Innov. Innov. 2020 , 9 , 11–16. (In Russian) [ Google Scholar ] [ CrossRef ]
• Bakthavatchaalam, V.; Miles, M.; de Lourdes, M.-T.; Jose, S. Research productivity and academic dishonesty in a changing higher education landscape. On the example of technical universities in India (translated from English). Educ. Issues Educ. Stud. Mosc. 2021 , 2 , 126–151. (In Russian) [ Google Scholar ] [ CrossRef ]
• Aven, T. Perspectives on the nexus between good risk communication and high scientific risk analysis quality. Reliab. Eng. Syst. Saf. 2018 , 178 , 290–296. [ Google Scholar ] [ CrossRef ]
• Marinina, O.A.; Kirsanova, N.Y.; Nevskaya, M.A. Circular economy models in industry: Developing a conceptual framework. Energies 2022 , 15 , 9376–9386. [ Google Scholar ] [ CrossRef ]
• Siegel, D.; Bogers, M.; Jennings, D.; Xue, L. Technology transfer from national/federal labs and public research institutes: Managerial and policy implications. Res. Policy 2022 , 52 , 104646. [ Google Scholar ] [ CrossRef ]
• Snegirev, S.D.; Savelyev, V.Y. Quality management for scientific activities. Stand. I Kachestvo Stand. Qual. 2014 , 3 , 54–57. (In Russian) [ Google Scholar ]
• Leontyuk, S.M.; Vinogradova, A.A.; Silivanov, M.O. Fundamentals of ISO 9001:2015. J. Phys. Conf. Ser. 2019 , 1384 , 012068. [ Google Scholar ] [ CrossRef ]
• Wieczorek, O.; Muench, R. Academic capitalism and market thinking in higher education. In International Encyclopedia of Education , 4th ed.; Elsevier: Amsterdam, The Netherlands, 2023; pp. 37–47. [ Google Scholar ] [ CrossRef ]
• Overchenko, M.N.; Marinin, M.A.; Mozer, S.P. Quality improvement of mining specialists training on the basis of cooperation between Saint-Petersburg mining university and Orica company. J. Min. Inst. 2017 , 228 , 681. [ Google Scholar ] [ CrossRef ]
• Hernandez-Diaz, P.M.; Polanco, J.-A.; Escobar-Sierra, M. Building a measurement system of higher education performance: Evidence from a Latin-American country. Int. J. Qual. Reliab. Manag. 2021 , 38 , 1278–1300. [ Google Scholar ] [ CrossRef ]
• Zharova, A.; Karl, W.; Lessmann, H. Data-driven support for policy and decision-making in university research management: A case study from Germany. Eur. J. Oper. Res. 2023 , 308 , 353–368. [ Google Scholar ] [ CrossRef ]
• Lubango, L.M.; Pouris, A. Is patenting activity impeding the academic performance of South African University researchers? Technol. Soc. 2009 , 31 , 315–324. [ Google Scholar ] [ CrossRef ]
• de Jesus, C.S.; Cardoso, D.d.O.; de Souza, C.G. Motivational factors for patenting: A study of the Brazilian researchers profile. World Pat. Inf. 2023 , 75 , 102241. [ Google Scholar ] [ CrossRef ]
• Kiseleva, M.A. Development of an organizational and economic mechanism for managing the research activities of national research universities. Vestn. YURGTU NPI Bull. SRSTU NPI 2021 , 3 , 182–190. (In Russian) [ Google Scholar ] [ CrossRef ]
• Alakaleek, W.; Harb, Y.; Harb, A.A.; Shishany, A. The impact of entrepreneurship education: A study of entrepreneurial outcomes. Int. J. Manag. Educ. 2023 , 21 , 100800. [ Google Scholar ] [ CrossRef ]
• Rudko, V.A.; Gabdulkhakov, R.R.; Pyagay, I.N. Scientific and technical substantiation of the possibility for the organization of needle coke production in Russia. J. Min. Inst. 2023 , 263 , 795–809. Available online: https://pmi.spmi.ru/index.php/pmi/article/view/16246?setLocale=en_US (accessed on 10 September 2024).
• Gromyka, D.S.; Gogolinskii, K.V. Introduction of evaluation procedure of excavator bucket teeth into maintenance and repair: Prompts. MIAB. Mining Inf. Anal. Bull. 2023 , 8 , 94–111. (In Russian) [ Google Scholar ] [ CrossRef ]
• Michaud, J.; Turri, J. Values and credibility in science communication. Logos Epistem. 2018 , 9 , 199–214. [ Google Scholar ] [ CrossRef ]
• Oblova, I.S.; Gerasimova, I.G.; Goman, I.V. The scientific career through a gender lens: A contrastive analysis of the EU and Russia. Glob. J. Eng. Educ. 2022 , 24 , 21–27. [ Google Scholar ]
• Chiware, E.R.T.; Becker, D.A. Research trends and collaborations by applied science researchers in South African universities of technology: 2007–2017. J. Acad. Librariansh. 2018 , 44 , 468–476. [ Google Scholar ] [ CrossRef ]
• Palavesm, K.; Joorel, S. IRINS: Implementing a Research Information Management System in Indian Higher Education Institutions. Procedia Comput. Sci. 2022 , 211 , 238–245. [ Google Scholar ] [ CrossRef ]
• Litvinenko, V.S.; Petrov, E.I.; Vasilevskaya, D.V.; Yakovenko, A.V.; Naumov, I.A.; Ratnikov, M.A. Assessment of the role of the state in the management of mineral resources. J. Min. Inst. 2023 , 259 , 95–111. [ Google Scholar ] [ CrossRef ]
• Carillo, M.R.; Papagni, E.; Sapio, A. Do collaborations enhance the high-quality output of scientific institutions? Evidence from the Italian Research Assessment Exercise. J. Socio-Econ. 2013 , 47 , 25–36. [ Google Scholar ] [ CrossRef ]
• Chen, S.; Ren, S.; Cao, X. A comparison study of educational scientific collaboration in China and the USA. Phys. A Stat. Mech. Its Appl. 2021 , 585 , 126330. [ Google Scholar ] [ CrossRef ]
• Arpin, I.; Likhacheva, K.; Bretagnolle, V. Organising inter- and transdisciplinary research in practice. The case of the meta-organisation French LTSER platforms. Environ. Sci. Policy 2023 , 144 , 43–52. [ Google Scholar ] [ CrossRef ]
• Liew, M.S.; Tengku Shahdan, T.N.; Lim, E.S. Enablers in Enhancing the Relevancy of University-industry Collaboration. Procedia—Soc. Behav. Sci. 2013 , 93 , 1889–1896. [ Google Scholar ] [ CrossRef ]
• Tunca, F.; Kanat, Ö.N. Harmonization and Simplification Roles of Technology Transfer Offices for Effective University—Industry Collaboration Models. Procedia Comput. Sci. 2019 , 158 , 361–365. [ Google Scholar ] [ CrossRef ]
• Sciabolazza, V.L.; Vacca, R.; McCarty, C. Connecting the dots: Implementing and evaluating a network intervention to foster scientific collaboration and productivity. Soc. Netw. 2020 , 61 , 181–195. [ Google Scholar ] [ CrossRef ]
• Ovchinnikova, E.N.; Krotova, S.Y. Training mining engineers in the context of sustainable development: A moral and ethical aspect. Eur. J. Contemp. Educ. 2022 , 11 , 1192–1200. [ Google Scholar ] [ CrossRef ]
• Duryagin, V.; Nguyen Van, T.; Onegov, N.; Shamsutdinova, G. Investigation of the Selectivity of the Water Shutoff Technology. Energies 2023 , 16 , 366. [ Google Scholar ] [ CrossRef ]
• Mohamed, M.; Altinay, F.; Altinay, Z.; Dagli, G.; Altinay, M.; Soykurt, M. Validation of Instruments for the Improvement of Interprofessional Education through Educational Management: An Internet of Things (IoT)-Based Machine Learning Approach. Sustainability 2023 , 15 , 16577. [ Google Scholar ] [ CrossRef ]
• Tuan, N.A.; Hue, T.T.; Lien, L.T.; Van, L.H.; Nhung, H.T.T.; Dat, L.Q. Management factors influencing lecturers’ research productivity in Vietnam National University, Hanoi, Vietnam: A structural equation modeling analysis. Heliyon 2022 , 8 , e10510. [ Google Scholar ] [ CrossRef ]
• Akcay, B.; Benek, İ. Problem-Based Learning in Türkiye: A Systematic Literature Review of Research in Science Education. Educ. Sci. 2024 , 14 , 330. [ Google Scholar ] [ CrossRef ]
• Cherepovitsyn, A.E.; Tretyakov, N.A. Development of New System for Assessing the Applicability of Digital Projects in the Oil and Gas Sector. J. Min. Inst. 2023 , 262 , 628–642. Available online: https://pmi.spmi.ru/pmi/article/view/15795?setLocale=en_US (accessed on 10 September 2024).
• Murzo, Y.; Sveshnikova, S.; Chuvileva, N. Method of text content development in creation of professionally oriented online courses for oil and gas specialists. Int. J. Emerg. Technol. Learn. 2019 , 14 , 143–152. [ Google Scholar ] [ CrossRef ]
• Sveshnikova, S.A.; Skornyakova, E.R.; Troitskaya, M.A.; Rogova, I.S. Development of engineering students’ motivation and independent learning skills. Eur. J. Contemp. Educ. 2022 , 11 , 555–569. [ Google Scholar ] [ CrossRef ]
• Rijcke, S.D.; Wouters, P.F.; Rushforth, A.D.; Franssen, T.P.; Hammarfelt, B. Evaluation Practices and Effects of Indicator Use—A Literature Review. Res. Eval. 2016 , 25 , rvv038. [ Google Scholar ] [ CrossRef ]
• Cappelletti-Montano, B.; Columbu, S.; Montaldo, S.; Musio, M. New perspectives in bibliometric indicators: Moving from citations to citing authors. J. Informetr. 2021 , 15 , 101164. [ Google Scholar ] [ CrossRef ]
• García-Villar, C.; García-Santos, J.M. Bibliometric indicators to evaluate scientific activity. Radiología Engl. Ed. 2021 , 63 , 228–235. [ Google Scholar ] [ CrossRef ]
• Guskov, A.E.; Kosyakov, D.V. National factional account and assessment of scientific performance of organizations. Nauchnyye Tekhnicheskiye Bibl. Sci. Tech. Libr. 2020 , 1 , 15–42. (In Russian) [ Google Scholar ] [ CrossRef ]
• Khuram, S.; Rehman, C.A.; Nasir, N.; Elahi, N.S. A bibliometric analysis of quality assurance in higher education institutions: Implications for assessing university’s societal impact. Eval. Program Plan. 2023 , 99 , 102319. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Buehling, K. Changing research topic trends as an effect of publication rankings—The case of German economists and the Handelsblatt Ranking. J. Informetr. 2021 , 15 , 101199. [ Google Scholar ] [ CrossRef ]
• Kremcheev, E.A.; Kremcheeva, D.A. The content of the operation quality concept of the scientific and technical organization. Opcion 2019 , 35 , 3052–3066. [ Google Scholar ]
• Nyondo, D.W.; Langa, P.W. The development of research universities in Africa: Divergent views on relevance and experience. Issues Educ. Educ. Stud. Mosc. 2021 , 1 , 237–256. (In Russian) [ Google Scholar ] [ CrossRef ]
• Marozau, R.; Guerrero, M. Impacts of Universities in Different Stages of Economic Development. J. Knowl. Econ. 2016 , 12 , 1–21. [ Google Scholar ] [ CrossRef ]
• Fayomi, O.O.; Okokpujie, I.P.; Fayomi, O.S.I.; Udoye, N.E. An Overview of a Prolific University from Sustainable and Policy Perspective. Procedia Manuf. 2019 , 35 , 343–348. [ Google Scholar ] [ CrossRef ]
• Do, T.H.; Krott, M.; Böcher, M. Multiple traps of scientific knowledge transfer: Comparative case studies based on the RIU model from Vietnam, Germany, Indonesia, Japan, and Sweden. For. Policy Econ. 2020 , 114 , 102134. [ Google Scholar ] [ CrossRef ]
• See, K.F.; Ma, Z.; Tian, Y. Examining the efficiency of regional university technology transfer in China: A mixed-integer generalized data envelopment analysis framework. Technol. Forecast. Soc. Chang. 2023 , 197 , 122802. [ Google Scholar ] [ CrossRef ]
• Dusdal, J.; Zapp, M.; Marques, M.; Powell, J.J.W. Higher Education Organizations as Strategic Actors in Networks: Institutional and Relational Perspectives Meet Social Network Analysis. In Theory and Method in Higher Education Research ; Huisman, J., Tight, M., Eds.; Emerald Publishing Limited: Bingley, UK, 2021; Volume 7, pp. 55–73. [ Google Scholar ] [ CrossRef ]
• Silva, M.D.C.; de Mello, J.C.C.B.S.; Gomes, C.F.S.; Carlos, I.C. Efficiency analysis of scientific laboratories. Meta Aval. 2020 , 2 , 625–645. [ Google Scholar ] [ CrossRef ]
• Vinogradova, A.; Gogolinskii, K.; Umanskii, A.; Alekhnovich, V.; Tarasova, A.; Melnikova, A. Method of the Mechanical Properties Evaluation of Polyethylene Gas Pipelines with Portable Hardness Testers. Inventions 2022 , 7 , 125. [ Google Scholar ] [ CrossRef ]
• Chen, W.; Yan, Y. New components and combinations: The perspective of the internal collaboration networks of scientific teams. J. Informetr. 2023 , 17 , 101407. [ Google Scholar ] [ CrossRef ]
• Corcoran, A.W.; Hohwy, J.; Friston, K.J. Accelerating scientific progress through Bayesian adversarial collaboration. Neuron 2023 , 111 , 3505–3516. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Wu, L.; Yi, F.; Huang, Y. Toward scientific collaboration: A cost-benefit perspective. Res. Policy 2024 , 53 , 104943. [ Google Scholar ] [ CrossRef ]
• Ilyushin, Y.; Afanaseva, O. Spatial Distributed Control System Of Temperature Field: Synthesis And Modeling. ARPN J. Eng. Appl. Sci. 2021 , 16 , 1491–1506. [ Google Scholar ]
• Cossani, G.; Codoceo, L.; Caceres, H.; Tabilo, J. Technical efficiency in Chile’s higher education system: A comparison of rankings and accreditation. Eval. Program Plan. 2022 , 92 , 102058. [ Google Scholar ] [ CrossRef ]
• Marinin, M.A.; Marinina, O.A.; Rakhmanov, R.A. Methodological approach to assessing influence of blasted rock fragmentation on mining costs. Gorn. Zhurnal 2023 , 9 , 28–34. [ Google Scholar ] [ CrossRef ]
• Sutton, E. The increasing significance of impact within the Research Excellence Framework (REF). Radiography 2020 , 26 (Suppl. S2), S17–S19. [ Google Scholar ] [ CrossRef ]
• Basso, A.; di Tollo, G. Prediction of UK Research excellence framework assessment by the departmental h-index. Eur. J. Oper. Res. 2022 , 296 , 1036–1049. [ Google Scholar ] [ CrossRef ]
• Groen-Xu, M.; Bös, G.; Teixeira, P.A.; Voigt, T.; Knapp, B. Short-term incentives of research evaluations: Evidence from the UK Research Excellence Framework. Res. Policy 2023 , 52 , 104729. [ Google Scholar ] [ CrossRef ]
• Reddy, K.S.; Xie, E.; Tang, Q. Higher education, high-impact research, and world university rankings: A case of India and comparison with China. Pac. Sci. Rev. B Humanit. Soc. Sci. 2016 , 2 , 1–21. [ Google Scholar ] [ CrossRef ]
• Shima, K. Changing Science Production in Japan: The Expansion of Competitive Funds, Reduction of Block Grants, and Unsung Heroe. Century Sci. Int. Perspect. Educ. Soc. 2017 , 33 , 113–140. [ Google Scholar ] [ CrossRef ]
• Li, D. There is more than what meets the eye: University preparation for the socio-economic impact requirement in research assessment exercise 2020 in Hong Kong. Asian Educ. Dev. Stud. 2021 , 11 , 702–713. [ Google Scholar ] [ CrossRef ]
• Radushinsky, D.A.; Kremcheeva, D.A.; Smirnova, E.E. Problems of service quality management in the field of higher education of the economy of the Russian Federation and directions for their solution. Relacoes Int. No Mundo Atual 2023 , 6 , 33–54. [ Google Scholar ]
• Shi, Y.; Wang, D.; Zhang, Z. Categorical Evaluation of Scientific Research Efficiency in Chinese Universities: Basic and Applied Research. Sustainability 2022 , 14 , 4402. [ Google Scholar ] [ CrossRef ]
• Cheng, Z.; Xiao, T.; Chen, C.; Xiong, X. Evaluation of Scientific Research in Universities Based on the Idea of Education for Sustainable Development. Sustainability 2022 , 14 , 2474. [ Google Scholar ] [ CrossRef ]
• Hou, L.; Luo, J.; Pan, X. Research Topic Specialization of Universities in Information Science and Library Science and Its Impact on Inter-University Collaboration. Sustainability 2022 , 14 , 9000. [ Google Scholar ] [ CrossRef ]
• Elbawab, R. University Rankings and Goals: A Cluster Analysis. Economies 2022 , 10 , 209. [ Google Scholar ] [ CrossRef ]
• Kifor, C.V.; Olteanu, A.; Zerbes, M. Key Performance Indicators for Smart Energy Systems in Sustainable Universities. Energies 2023 , 16 , 1246. [ Google Scholar ] [ CrossRef ]
• Guironnet, J.P.; Peypoch, N. The geographical efficiency of education and research: The ranking of U.S. universities. Socio-Econ. Plan. Sci. 2018 , 62 , 44–55. [ Google Scholar ] [ CrossRef ]
• Ma, Z.; See, K.F.; Yu, M.M.; Zhao, C. Research efficiency analysis of China’s university faculty members: A modified meta-frontier DEA approach. Socio-Econ. Plan. Sci. 2021 , 76 , 100944. [ Google Scholar ] [ CrossRef ]
• Tavares, R.S.; Angulo-Meza, L.; Sant’Anna, A.P. A proposed multistage evaluation approach for Higher Education Institutions based on network Data envelopment analysis: A Brazilian experience. Eval. Program Plan. 2021 , 89 , 101984. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Le, M.H.; Afsharian, M.; Ahn, H. Inverse Frontier-based Benchmarking for Investigating the Efficiency and Achieving the Targets in the Vietnamese Education System. Omega 2021 , 103 , 102427. [ Google Scholar ] [ CrossRef ]
• Adot, E.; Akhmedova, A.; Alvelos, H.; Barbosa-Pereira, S.; Berbegal-Mirabent, J.; Cardoso, S.; Domingues, P.; Franceschini, F.; Gil-Doménech, D.; Machado, R.; et al. SMART-QUAL: A dashboard for quality measurement in higher education institutions. Qual. Reliab. Manag. 2023 , 40 , 1518–1539. [ Google Scholar ] [ CrossRef ]

Share and Cite

Radushinsky, D.A.; Zamyatin, E.O.; Radushinskaya, A.I.; Sytko, I.I.; Smirnova, E.E. The Performance and Qualitative Evaluation of Scientific Work at Research Universities: A Focus on the Types of University and Research. Sustainability 2024 , 16 , 8180. https://doi.org/10.3390/su16188180

Radushinsky DA, Zamyatin EO, Radushinskaya AI, Sytko II, Smirnova EE. The Performance and Qualitative Evaluation of Scientific Work at Research Universities: A Focus on the Types of University and Research. Sustainability . 2024; 16(18):8180. https://doi.org/10.3390/su16188180

Radushinsky, Dmitry A., Egor O. Zamyatin, Alexandra I. Radushinskaya, Ivan I. Sytko, and Ekaterina E. Smirnova. 2024. "The Performance and Qualitative Evaluation of Scientific Work at Research Universities: A Focus on the Types of University and Research" Sustainability 16, no. 18: 8180. https://doi.org/10.3390/su16188180

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

• Open access
• Published: 24 September 2024

Leaving the profession as a medical assistant: a qualitative study exploring the process, reasons and potential preventive measures

• Viola Mambrey 1 ,
• Annegret Dreher 1 &
• Adrian Loerbroks 1  

BMC Health Services Research volume  24 , Article number:  1111 ( 2024 ) Cite this article

Metrics details

Worldwide growing shortages among health care staff are observed. This also holds true for medical assistants in Germany. Medical assistants mainly work in outpatient care and are the first point of contact for patients while performing clinical and administrative tasks. We sought to explore profession turnover among medical assistants, that is, in terms of the underlying decision-making process, the reasons for leaving the medical assistant profession and potential retention measures from the perspective of former medical assistants.

For this qualitative study, we conducted semi-structured telephone interviews with 20 former medical assistants between August and November 2023. Eligible for participation were medical assistants who (i) were of legal age, (ii) completed medical assistant vocational training and ii) were formerly employed as a medical assistant, but currently employed in another profession. The interviews were recorded, transcribed verbatim and content-analyzed.

Former medical assistants expressed various, often interrelated reasons for leaving the profession. These were changes in priorities throughout their career (e.g., in terms of working hours and salary), a constant high workload, barriers to further training, poor career prospects, and poor interpersonal relationships particularly with supervisors, but also within the team and with patients as well as the perception of insufficient recognition by politics and society. Suggestions of former medical assistants to motivate medical assistants to stay in their profession included amongst others higher salaries, more flexible work structures, improved career prospects, and more recognition from supervisors, patients, and society.

Our study provides insights into the complex decision-making process underlying ultimate medical assistant profession turnover. In light of an already existing shortage of medical assistants, we suggest to further explore how the suggested interventions that aim at retention of working medical assistants can be implemented.

Peer Review reports

Introduction

Worldwide there is a growing shortage of health care workers [ 1 , 2 ]. Among the primary drivers are the demographic change in general populations, an ageing health care working population and a general increase in chronic diseases among the general population [ 3 , 4 ]. The staff shortage is aggravated by the high number of health care workers leaving their profession [ 5 ] and the COVID-19 pandemic seems to have exacerbated this trend [ 6 ]. Shortages of health care staff are associated with lower patient satisfaction, negative health outcomes for patients, and a decrease in job satisfaction among the remaining health care workers [ 7 ]. In addition, the consequences of turnover are the loss of knowledge, productivity and high cost (e.g., temporary replacements, overtime work, less productivity during staff induction) [ 7 , 8 ].

The majority of studies that have examined potential antecedents of profession turnover and turnover intention among health care staff rely on quantitative designs and thus on numerical data for analyses [ 7 , 9 , 10 , 11 , 12 , 13 ]. Compared to qualitative methods, such approaches are unable to capture the complex decision-making process that may underlie profession turnover [ 7 , 14 ]. Previous qualitative studies among health care workers, particularly nurses, often explored turnover intention or change of employers rather than entirely leaving one’s profession [ 15 ]. This is particularly relevant, as one’s intention to leave does not invariably translate into actual profession turnover [ 9 , 10 , 16 ]. So far, studies that profoundly explore the complex decision-making process and circumstances underlying the turnover of health care workers are limited [ 7 , 9 , 15 ]. The qualitative studies that explored reasons for profession turnover among former nurses reported these to include a high workload, poor interpersonal relations with supervisors and colleagues, limited career opportunities, as well as the associated poor mental health [ 5 , 17 , 18 , 19 , 20 ]. These studies, however, focused on former novice nurses or very young nurses [ 5 , 17 ], were restricted to inpatient care settings [ 18 , 19 , 20 ] and were carried out before the COVID-19 pandemic. The COVID-19 pandemic had major impacts on the working life of health care professionals [ 21 , 22 ].

We seek to address the abovementioned knowledge gaps by exploring profession turnover among medical assistants (MA) who represent the largest occupational group in outpatient care in Germany [ 23 ]. MA perform a wide range of tasks in the practice, including administrative tasks and medical procedures such as blood sampling, injections, wound care, X-rays, and laboratory diagnostics [ 24 ]. Outpatient care in Germany is generally characterized by work in small teams with the physician often being both the employer and supervisor. MA face a variety of job-specific stressors, such as poor practice organization, interpersonal stressors (e.g., poor collaboration, lack of social resources), or a strong dependence on the supervising physician [ 12 ]. There is a concern that the changes in working conditions in the health care system due to the COVID-19 pandemic will lead to increased turnover [ 6 , 25 ]. This would exacerbate the already emerging trend of MA to migrate to other professions [ 26 ]. A prior qualitative study from our group among MA explored the perceived changes of working-conditions associated with the COVID-19 pandemic [ 25 ]. MA shared that various stressors have emerged or exacerbated during that period such as an increased workload, changes in collaboration within the team, and an increase in demanding patients. These stressors were reported to elicit intentions to leave the profession [ 25 ]. Further, in a recent 4-year prospective cohort study among MA by our group we found in particular interpersonal factors, amongst others poor collaboration within the team, poor behavior of the supervisor to predict turnover from the profession [ 13 ]. While that quantitative approach provides data on the predictors of turnover, it fails to provide an in-depth understanding of the underlying decision-making process and the likely interrelationship of reasons for profession turnover among MA. Fostering this understanding is important to develop suitable interventions as part of a comprehensive health workforce policy to strengthen healthcare workers retention and to counteract growing staff shortages, in the case of our study the MA workforce [ 27 ]. To gain a profound understanding of how and why MA left their original profession we aimed to explore the decision-making process, the reasons and potential retention measures from the perspective of former MA.

Study design

We carried out a qualitative study building on semi-structured interviews [ 28 ]. This particular design was chosen because it allowed a comprehensive exploration of our understudied research topic [ 15 , 28 , 29 ]. The exploration of a broad spectrum of views and the detailed thematic analysis of the data were used to gain an in-depth understanding of the decision-making process, the circumstances and the reasons for turnover from the MA profession.

Study sample

We conducted semi-structured qualitative telephone interviews using a predefined topic guide. The completed checklist of consolidated criteria for reporting qualitative research (COREQ) were applied in writing this report (see checklist in Appendix 1 ) [ 30 ].

Participant’s recruitment and interview conduct

We employed convenience sampling. First, we invited former MA from an existing cohort of MA, that is, individuals who had reported employment as a MA at baseline (2016/2017) and reported to be currently employed, but not as a MA at follow up (2021) ( n  = 48) [ 31 ]. Second, MA were recruited through the professional organization of medical assistants in Germany (“Verband medizinischer Fachberufe e.V.”) which advertised our study on their social media profiles (Instagram, Facebook) on November 9th, 2023. Third, MA were recruited through personal contacts. Inclusion criteria were the legal minimum age (i.e., 18 years in Germany), completion of the MA vocational training, and that participants had formerly worked as a MA and were currently employed in another profession. The latter was important as we specifically wanted to explore the views of MA who had chosen to leave their profession (in contrast to MA in retirement or on parental leave) and would theoretically still be available for the MA workforce. Potential participants received written study information and provided written informed consent prior to participation. Recruitment and interviewing lasted from August 14 until November 28, 2023. The sample size was not determined a-prior, but was contingent upon thematic saturation. It has been suggested that saturation is, most often reached between 9 and 17 interviews in qualitative interview studies [ 32 ]. An external contractor transcribed the recorded interviews verbatim while omitting potentially de-anonymizing information from the transcripts. Interviews were conducted in German and quotes were translated into English by a research associate with a Master’s degree in English Studies. The ethics committee of the Medical Faculty of the University of Düsseldorf approved the study (study number 2023–2443).

Development of the topic guide

The topic guide was developed based on an extensive review of pertinent literature on turnover particularly in health care, which emphasized the significance of understanding the career decision-making process [ 7 , 10 , 11 ]. In order to achieve this understanding, relevant dimensions of the process need to be explored: the initial decision for the profession, the circumstances around the turnover, potential triggers or challenges regarding the leaving process, and general aspiration for one’s future career [ 7 , 10 , 11 ]. The topic guide was designed to explore the process of how MA exit from their profession, including the circumstances of quitting, criteria for the selection of the new workplace and potential starting points for measures to prevent MA turnover. The initial topic guide was discussed within the study team and revised until consensus was reached. VM, AD and AL are occupational health researchers and have experience in qualitative research, the development of topic guides and qualitative data analysis, including qualitative studies among the MA population (for the topic guide see Appendix 2 ) [ 25 , 33 ]. Moreover, in the qualitative interviews, we used 13 closed-ended questions to collect data on socio-demographics (i.e., sex, age, partnership, highest educational degree, recruitment channel), work-related information (i.e., years worked as a MA, time passed since having left the MA profession, former employer [i.e., general practitioner (GP), specialist, others], current employment in the health sector, and perspectives on the MA profession. The latter was assessed as follows: (a) likelihood of recommending younger people to take up the MA profession; (b) to what extent they would decide to take up the MA profession as of today; (c) probability of returning to the MA profession; and (d) satisfaction with their current job as compared to their job as a MA). Appendix 2 shows the final topic guide including the close ended questions.

Data analysis

We conducted our analysis building on Kuckartz’ content structuring content analysis using MAXQDA 2024 software (VERBI GmbH, Berlin, Germany) [ 34 ]. Categories were formed deductively according to the research questions. The main categories were then refined by inductively forming subcategories from the interview transcripts. The first five interviews were coded independently by VM and AD. The preliminary coding scheme was then compared and discrepancies were discussed until consensus was reached. The resulting coding scheme was then applied to code the remaining transcripts (VM). After the first full round of coding the coding scheme was reviewed by AL. A second round of coding was performed by VM with consideration of the suggestions.

Sample characteristics and descriptive results

In total, we conducted 20 interviews. Participant characteristics are shown in Table  1 . Most participants were recruited from our MA cohort study ( n  = 14). The interviews had a mean duration of 38.1 min ranging from 23 to 66 min. We interviewed 18 female participants and two male participants with a median age of 44.5 years ranging from 26 to 64 years. The participants had worked as a MA for a median of 22 years (min-max: 5–41 years) and the median number of months since the last exit from the MA profession was 40.5 (min-max: 10–132 months). The majority would somewhat to fully recommend the professional training as a MA to (young) people ( n  = 14), but more than half of the interviewed former MA would rather not or certainly not choose the MA profession again if they had to make that decision again ( n  = 11) (see Table  2 ). All of the participants reported that a return to the MA profession was unlikely or very unlikely.

The decision-making process: changing priorities and barriers to quitting

Entrance into the ma profession.

Participants described that their decision to take up professional MA training was often based on first-hand experiences with the profession through internships as a student or after graduating from high school, based on pragmatic reasons (e.g., a workplace close to one’s home, doing any kind of vocational training) or perceived as a coincidence (i.e., “it turned out that way”). Expectations prior to the training were to work in a medical profession, to work closely with people and to carry out administrative work. Some former MA remembered not having any expectations, which was often attributed to their young age and thus their inexperience at the start of training.

Barriers to quitting

Many former MA expressed that they enjoyed working as a MA, because of their personal interest in medical topics, the variety of tasks, and the close work with patients. Participants described, however, a discrepancy between these expectations and experiences towards and during their MA job at a young age and their personal needs and priorities in later working life (e.g., higher salary, work-life balance/family friendly working hours, better career prospects). Many emphasized that their decision to quit the MA profession was carefully considered and not made hastily. In fact, quitting was perceived to be often delayed due to personal reasons (e.g., low self-confidence with regard to one’s skill and career prospects, not seeing any alternative career path for oneself), financial reasons (e.g., being dependent on the salary as a single parent), a sense of loyalty towards colleagues and the employer/supervising physician. In addition, participants felt that their exit from the profession was delayed, because the physician attempted to keep MA down (compare verbatim quote Q1 below). Further, they felt ”emotionally blackmailed” by their supervising physician who took advantage of the loyalty of MA towards their colleagues (i.e., higher workload for colleagues if MA leaves due to staff shortage).

“I had to earn money. So, there was a total interdependency, and one was afraid, or one wasn’t confident enough to say: ‘I’ll find something new in another medical practice.’ It is especially your bosses who convey this feeling of dependency to you: ‘You’ll never find a new job if you quit now.’ This is really exhausting. And as a young woman, you actually believe that and then you’re too afraid to quit.” (Q1, ID 7123; for verbatim quotes see Appendix 3 ).

Some participants felt that changing the profession was like a natural progression in their career. By contrast, for others it felt like a difficult decision. The experienced working conditions as a MA made them feel that merely changing their employer (while continue working as a MA) would not have improved their situation. The mainly positive experiences they had made in the new profession and the disbelief that MA working conditions will improve soon, led most participants to believe that they would not return to the MA profession.

Reasons to exit from the MA profession

The former MA reported various reasons to quit their profession and emphasized that there were often multiple reasons (for an overview of the reasons for MA profession turnover see Appendix 4 ).

Constantly high workload

Many described that a constantly high workload was a key reason for their decision to quit. Different facets of such a high workload were mentioned: First, the reasons for a high workload were mentioned. These were, first, a high number of patients and the limited time to fulfill tasks and adequately care for patients. This made participants feel stressed and emotionally blunted (Q2, for verbatim quotes see Appendix 3 ). Moreover, the high number of patients made it necessary to triage patients on the phone according to the perceived severity of symptoms. Frequently, having to turn patients down on the phone or discussing with them how urgent their complaints were (not) was perceived as stressful (Q3). A second reason for the perceived increasing workload that contributed to quitting were increasing administrative tasks (e.g., accounting, documentation, data protection). Participants often felt that documentation was at the core of their work rather than working with patients.

“And eventually, it was not about the patients anymore, you just had to fill in more and more documents.” (Q4, ID 6912).

A third perceived contributor to the high workload were cumbersome and outdated processes (e.g., slow digitalization, faxing, necessity to print documents), which were experienced as time consuming, highly inefficient and frustrating, because time was perceived as scarce (Q5).

Aside the causes, participants also explained the consequences of the high workload. A high workload was perceived to result in long waiting times for the patients which was felt to translate into substantial dissatisfaction among patients and into working overtime for the MA. The perceived consequences of a persistent lack of time due to a high workload were (i) an inability to provide adequate patient care; (ii) lacking opportunities to expand medical knowledge during working hours (e.g., discussing patients with physicians); (iii) often not having a single break throughout the work day and (iv) feeling stressed. Work stress was felt to lead to physical and psychological complaints (e.g., back pain, burnout) and negatively influence the team atmosphere (Q6).

“It’s getting more and more. It’s not decreasing and it’s not stagnating either. On the contrary, requirements have increased more and more, but the salary certainly hasn’t. The way we treat each other in the team has also become less pleasant, because we all reach our limit at some point.” (Q6, ID 6734).

Long working days due to the high workload and the operating hours of physician offices with long midday breaks, often including (expected daily) overtime, in some cases even unpaid, were felt to be frustrating (Q7). These consequences of a high workload were mentioned as factors contributing to the decision to leave the MA profession.

Finally, concerning a high workload, former MA felt that there was a shortage of skilled MA which leads to a vicious cycle of - on the one hand – increased stress-induced absenteeism, which then reinforces staff shortage and increases the workload for the remaining MA (Q8, Q9). On the other hand, the lack of skilled MA was perceived to result in constant spontaneous changes in working shifts without the workload being adapted to the reduced workforce (e.g., rescheduling appointments), negatively affecting the private life (Q10, Q11).

Perceived barriers to further training and poor career prospects

Many former MA expressed that the limited further training and career prospects made them leave the MA profession (Q12). Participants felt that the missing career prospects had led to a sense of missing challenges and increasing boredom.

“I don’t want to spend another 20 years printing out prescriptions, talking to people on the phone and making appointments.” (Q13, ID 9162).

One former MA criticized that although MA could acquire skills through further training, they often did not have the legal permission to apply these skills. This means for example that they are not allowed to act as a formal training supervisor for MA in training. This is the sole legal responsibility of the physician in Germany. This was perceived as generally unfair and criticized as physicians do not have to complete any relevant training. Others felt the need to leave the MA profession as they perceived that the further training and novel skills could not be applied (Q14). On the one hand, they mentioned limited demand and thus restricted opportunities to apply their skills particularly in rural areas and/or small practices. On the other hand, the limited willingness of the employers to pay according to the acquired qualifications was mentioned, which was partly seen to be due to restricted financial leeway (i.e., small practices with limited revenue).

“The stagnation. In other words, I didn’t have the chance to have more responsibility, or earn more money anywhere.” (Q15, ID 9162).

Moreover, the costs for further training were often not or only partially covered by employers. Alternatively, coverage of fees was subject to conditions (e.g., continue working in the practice for five more years). Such conditions were not well received according to participants as further trainings were often only offered during their leisure time (i.e., Wednesday afternoon, weekends), were expensive in relation to the MA salary and were perceived to ultimately benefit the practice.

Interpersonal factors

A core theme that emerged from the interviews was the perceived behavior of supervisors towards MA as a reason for leaving the MA profession. This included insufficient recognition, poor support, exploiting dependency relationships and disrespectful/aggressive behavior. Many participants expressed to not have felt adequately appreciated by their supervisor(s). In this respect the salary was often alluded to which was perceived as insufficient and unfair in light of the responsibilities of MA, also in comparison to similar healthcare professions (e.g., nurses, nurse assistants) (Q16). They felt they were unable to financially support themselves (and their family) despite working full-time. Some MA only became fully aware of the low salary later in their working life.

“Back then, I couldn’t imagine that the money I was getting was very little. So, I didn’t realize that you can’t actually live off the salary of a medical assistant.” (Q17, ID 8982).

Participants frequently criticized that employers were not obliged to apply the collective wage agreement Footnote 1 . Many reported that the contracts were only “based on” the collective agreement, therefore not legally bound by it and often paid less. Further, even if strictly applied, wages according to the collective wage agreement were nonetheless perceived as insufficient as everyday “amenities” like owning a car or buying a pair of new shoes could not be afforded (Q18, Q19). The collective agreement stipulates that employees are assigned to different salary categories based on their qualifications and years of employment. Former MA said that they were not grouped correctly within the collective agreement according to their qualifications and years of experiences and respectively were paid less than they were entitled to which contributed to exit from the profession (Q20). Some former MA acknowledged that considering the profits that employers made under the current remuneration system applying to outpatient physicians, a higher salary for MA would be difficult to implement. Moreover, participants expressed that as long as other work-related factors were satisfactory, these seemed to compensate a salary that was perceived to be inadequate.

Former MA also shared that they felt that physician supervisors considered themselves as superior, that they felt that MAs’ performance and professional skills remained unrecognized and they were disappointed about the lack of their say within the practice.

“For me, it was mostly about the lack of recognition of what you were doing there whatsoever. […] It’s hectic, it’s chaotic and so on, but that has never really been an issue. For me, it has always been the lack of recognition, at least at the end.” (Q21, ID 8463).

The latter included, amongst others, that employers/supervising physicians decided without conferring with MA when to take their vacations. This implies that the practice remains closed during that period of time and MA, consequently, had to take their own vacation, too (i.e., leaving no remaining vacation days for individual vacation planning). Further, there was a perceived lack of say with regard to decisions on practice organization (e.g., communicating with MA how processes in their work areas can be optimized, instead of supervisors deciding alone) (Q22). In addition, a lack of support from supervisors for MA made them feel left alone and overwhelmed with the responsibility in certain situations (e.g., writing medication plans without supervision, training newly hired MA, implementing new measures), and lack of support for MA in front of patients was even perceived as degrading by one MA (Q23).

Some participants mentioned the strong dependency relationship between supervisors and MA as problematic. It resulted in the feeling that MA are kept down and feel to be at their supervisor`s mercy (e.g., things are promised during the job interview, but are revised after the start of work; not being given any responsibility; the range of tasks not corresponding to the MA training).

“[…] Well, in the practice […] our cleaner quit. And then my colleagues started cleaning the practice. And I was like: ‘Are you crazy?’ I would never do that. Not because I consider myself too good for cleaning. I’ve earned a lot of additional money with cleaning. But they hired me as a medical assistant and not as a cleaner, so there’s a limit there.” (Q24, ID3647).

They felt restricted in their capabilities to take action against these conditions, as supervisors are often also the employer and legal protection against dismissal is low in small companies with less than 10 employees. This motivated some former MA to change into the public service sector or to larger companies where employees’ rights are better protected.

Former MA alluded to overemphasized economic thinking of the employer as a further reason to quit. This included that employers wanted to limit the time of social interaction between MA and patients (e.g., on the telephone) due to economic reasons (Q25). Also, MA in training may be hired instead of trained MA to reduce staff costs. Participants felt that the focus was on economic efficiency rather than quality of patient care or wellbeing of the employees (Q26). They found this disappointing and perceived this as dissonant with their reasons as to why they chose the MA profession in the first place (Q27).

For some former MA, strong differences of opinions within the team and a constantly tensed team dynamic like bickering, lack of identification with MA colleagues, bullying, non-constructive team meetings were reasons for changing the profession (Q28).

Former MA frequently expressed that the patients’ pronounced (and increasing) demanding behavior (e.g., immediate receipt of prescriptions or appointments, unwillingness to wait, impudence, lack of consideration for other patients and work processes) sometimes combined with aggression, was perceived as stressful, frustrating and to strongly reduce work satisfaction (Q29).

“Expectations have become much higher in the last years. That means, they want something, and they want it right now. And I’m not talking about people who lost their arm, you know? […] Rather, these are people who need a prescription, […] They forget about it, but we have to do it right away. And then, you always have to ask yourself: ‘Alright, should I argue with that person or should I just do it?’ And then you try to make your position clear and tell them: ‘Please let us know at least one day in advance.’ Well, somehow you always have to discuss it. ‘Well, can’t you just quickly do it on the side?’” (Q30, ID5975).

Further, former MA discerned a lack of recognition from patients towards MA and the MA profession, amongst others not being perceived as medical personnel, questioning of MA professional competence (Q31).

External factors

Former MA also referred to external factors which influenced their decision to leave the profession. For instance, politics were mentioned in terms of the inadequate legal frameworks they provide which was perceived to ultimately influence MA work and/or working conditions (e.g., only simple tasks can be delegated to MA by the supervising physician, budgeting of health services by the statutory health insurances, which forces employers to think more economically and adversely influences salaries paid by them to MA). Moreover, former MA perceived a lack of recognition from society for the MA profession in general and in comparison to other health care professions such as nurses and physicians. For some former MA the feeling of low recognition by society and politics became very apparent during the COVID-19 pandemic. For instance, despite their exceptional commitment (e.g., being “COVID-19 experts” for patients, being constantly exposed to SARS-CoV-2 positive patients, working overtime), they did not receive COVID-19 financial bonuses like many other professions in health care in Germany or workers in other areas who worked from home (Q32, Q33).

“Well, people who were at home got it. (laughs) [note: single payments during the COVID-19 pandemic] And I’ve been there for ten hours instead of eight, and people giving me a hard time for ten hours and I simply didn’t get it.” (Q33, ID 5975).

Potential interventions

Former MA suggested several interventions that may help to motivate MA to stay in their profession. Table  3  provides an overview of the addressed actors (e.g., employers, fellow MA, policy makers), the interventions and examples of how former MA thought the interventions can take shape. Participants mentioned many potential interventions that directly address the specific reasons for quitting. These include, amongst others to reduce staff shortage, to increase recognition of MA, to increase the salary, to strengthen supervisor’s leadership skills, to create better career prospects, and to address demanding behavior of patients. Three interventions addressed aspects not specifically mentioned as a reason to quit, but were believed to be particularly feasible and effective for staff retention. The first addressed supervisors and suggested to provide more work flexibility to MA by offering working from home for administrative tasks. The second addressed politics by proposing to increase the quality of the MA training (e.g., oblige supervising physicians to teach certain learning content, differentiate according to medical specialty), which was perceived to indirectly strengthen the MA profession. The third intervention suggestion emphasized that the MA themselves should strengthen the MA profession by standing up for themselves more, engaging more in networking and encouraging each other to talk about the terms of their contracts.

To the best of our knowledge, this is the first study exploring the decision-making process and reasons for leaving the MA profession as well as potential intervention measures. Our findings suggest that MA leave their profession often due to a combination of several factors. We found that expectation towards one’s profession and working conditions at the start of the career change throughout one’s working life and expectations or needs in later working life are felt to remain unmet. The main reasons for leaving the MA profession mentioned were a persistently high workload, barriers to further training and career advancement, poor interpersonal relationships especially with supervisors, but also with patients and within the team as well as poor recognition by politics and society. In order to motivate MA to stay in their profession, participants suggested higher salaries, addressing staff shortage, lower economic pressure and increasing recognition from supervisors, patients, and society.

Comparison to prior research

The decision-making process.

Former MA perceived a discrepancy between the expectations they had when they chose their profession (e.g., to closely work with people and help them) and reality (i.e., no time to adequately care for patients) as well as the change of priorities throughout the career, (e.g., related to the salary, working times and career prospects). These findings are in line with those from a qualitative study that explored the decision process underlying turnover among nurses in Australia: that study found nurses to leave due to what the author referred to as “value images violations” indicating a mismatch of expectations vs. reality and “violation of trajectory and strategic images” indicating a change in priorities [ 35 ]. According to the author, nurses who perceived a mismatch of expectations vs. reality were usually older with more years of work experience, were in nursing as their career of deliberate choice and struggled tremendously with their decision to leave. By contrast, the decision process for those reporting changes in priorities was characterized as easy and fast [ 35 ]. As part of the decision process former nurses who obtained a university/college degree often reported a discrepancy between the nursing taught in class and how it is in practice after entering the profession [ 5 , 17 , 20 ]. This experienced discrepancy right after entering the profession was not mentioned by the participating former MA. This might be explained by the fact that MA training in Germany is based on a dual training program, with trainees mandatorily alternating between vocational school and work in a practice and thus already gaining professional experience as a MA [ 36 ].

Reasons for leaving the MA profession

Further, in line with our findings, other studies found that often a combination of factors lead to turnover and that single events trigger the decision to leave only infrequently [ 5 , 17 ]. Overall, it seems that many former MA perceived the low salary as a drawback of the profession. As long as other work-related factors were satisfactory, these seemed to compensate the salary. Once former MA perceived other factors were not rewarding anymore (e.g., career prospects, recognition, sufficient time for good quality patient care) these were often the tipping point leading to turnover. This highlights the complexity of the turnover decision process.

One of the core themes that emerged were high (and increasing) patient demands including rude and impatient behavior as reasons for turnover. In qualitative studies among former nurses, the demanding patient care itself [ 5 ], demanding behavior of patient relatives [ 18 ] and violence of patients [ 17 ] were reported as reasons for leaving the nursing profession. Only one study reported increased expectations of patients and their relatives as a factor perceived to affect turnover [ 37 ]. The authors of the latter study used open-ended questions to assess perceived antecedents of turnover among working hospital nurses in Australia. In a qualitative study from our group among MA in 2015/2016 on work-related stressors and resources we found that some experienced MA had perceived a change in patients’ attitudes and expectations towards more impatience [ 33 ]. During the COVID-19 pandemic, we carried out another qualitative study and observed that MA perceived a change (compared to the pre-COVID-19 era) in patients’ behavior towards demanding, selfish and aggressive behavior [ 25 ]. This might be due to patients becoming more consumer-oriented, with greater expectations of active involvement in their health care as well as greater emphasis on convenience in their health care experiences [ 38 ]. This might act as a barrier in the care of patients, if patients perceive healthcare as a service like any other (e.g., restaurant) [ 39 ]. MA most likely will not be able to comply with these heightened expectations given the current structures in the German health care system.

Qualitative studies on turnover among nurses from inpatient settings have suggested that a lack of support from the supervisor [ 19 ] as well as an occupational culture of hierarchy and discrimination are reasons for leaving the profession [ 15 ]. In a qualitative study among nurses from Canada who have left their position, the nurses expressed to be hesitant to voice concerns towards supervisors in fear of being penalized [ 19 ]. Further, a perceived lack of respect of physicians towards nurses was reported [ 19 ]. This is similar to the accounts of former MA in our study that emphasized strong hierarchical structures and feelings of subordinance towards supervising physicians to contribute to turnover. However, in addition to the lack of support from supervisors as well as the hierarchy, former MA also felt a strong relationship of dependency on their supervisors. The fact that the supervisor is often also the employer creates for some MA the perceived challenge that they constantly have to prove themselves as well as a feeling of restricted means to counter this condition due to a fear of punishment. The dependency relationship is enhanced as, employers in small companies (< 10 employees) – such as outpatient practices – face reduced legal hurdles in terminating employment in Germany. This also means that MA can be dismissed without any severance pay. Further, without a work council, there are often no formal contact persons who can mediate in the event of arguments. As a result, many MA working in outpatient care switch to inpatient care (e.g., offering higher salaries, more flexible working hours, and a work council) or leave the profession [ 40 ].

Despite the differences between MA and nurses in terms of their job profile and settings (i.e. outpatient vs. inpatient), former MA reported similar antecedents of turnover as nurses from different countries (e.g., high workload, limited career opportunities, interpersonal difficulties and a low salary) [ 5 , 17 , 18 , 19 , 20 , 37 ]. However, we found some factors, which seem to be more specific to outpatient care (e.g., dependency relationship between MA and supervisor), and factors that potentially became more prominent in recent years (e.g., demanding behavior from patients).

General situation of MA in Germany

Former MA criticized the economic thinking of employers, as in their opinion it contradicts their expectation of providing good patient care. Former MA in our study seemed to become aware that the practices are commercial enterprises though. This might be due to the close cooperation with the employers, the small company structures and because MA are also responsible for the accounting [ 24 ]. MA felt that their salary is too low. Notably, general physicians, too, seem to perceive the opportunities to paying higher salaries to MA to be limited by the current remuneration system applying to outpatient physicians [ 41 ]. Former MA also believed though that the remuneration of the practices was high and, together with the lifestyle of the employers (e.g., expensive vacations, cars), this created feelings of injustice. The disappointment seemed to prevail specifically during the COVID-19 pandemic as MA perceived their contributions were high and did not translate into bonuses or adaptations of salaries [ 25 , 42 ]. Likewise, the focus on being an economic enterprise seemed to elicit so-called “moral distress” among former MA, as they felt they have too little time to adequately care for the patients. Moral distress describes a state in which one knows what would be the ethically right thing to do, but feels prevented from acting accordingly [ 43 ]. Moral distress has been hypothesized to be experienced to a higher degree among more experienced nurses [ 44 ] and so far, has not been specifically studied among MA.

Qualitative vs. quantitative approaches to turnover

In a four-year prospective quantitative study from our group, we found psychosocial working conditions particularly reflecting interpersonal relations (i.e. poor collaboration, lack of social resources and poor leadership behavior) rather than work demands and resources (e.g., high workload, low job control, poor practice organization) to predict turnover among MA [ 13 ]. Those findings are consistent with the results of this qualitative study. However, at the same time, it becomes evident that the quantitative assessment is limited in its ability to uncover the complex underlying decision-making process and the multi-layered reasons and their interaction of MA turnover that emerged from this qualitative study (e.g., not a generally high workload itself, but a perceived constantly increased workload being important for the decision of turnover; low salary being compensated by otherwise good working conditions, however, a change in those leading to turnover). Moreover, in the qualitative interview’s themes emerged, like e.g., high patient demands, moral distress, which were not captured by the quantitative assessment and could thus not be examined as potential determinants of MA turnover.

Potential intervention measures

In this study, former MA suggested several potential interventions addressing different actors. The proposed interventions addressing politics and unspecific actors are to be implemented on a macro level (i.e., societal) and potentially require longer implementation periods. These are in line with the “global strategy for human resources for health: Workforce 2030” by the World Health Organization [ 27 ]. That strategy proposes the implementation of policies on country level to shape health labor markets by addressing education and further training of health workers, the retention of health workers, the inefficiencies of productivity such as pronounced administrative tasks, and the promotion of better working environments [ 27 ]. The measures proposed to the supervisors are located on a micro level (i.e., individuals, one-on-one interactions) or meso level (i.e., organizational level) and might prove to be easier and faster to implement than macro level interventions. A qualitative study from our group explored work-related intervention needs of MA and how these could be addressed according to GPs. In terms of an increase of expressed recognition and strengthening of leadership skills, GP perceived deficits however, only among other GPs and not themselves [ 41 ]. Many GPs did not perceive that they could improve their expressed recognition towards their MA. At the same time though, the GPs acknowledged that it is their responsibility to improve leadership skills and proposed participation in leadership courses. In contrast, in our study, former MA suggested several intervention areas that the supervisor could address easily, amongst others to strengthen team cohesion; to support work independency of MA, to allow working from home (compare Table  3 ). Further, former MA proposed that supervisors should actively express recognition for MA by showing interest in the MA and acknowledging MA work and medical expertise as well as include MA in decision-making processes (e.g., accept suggestions for change/improvements, planning vacation times together). It thus seems that the expression of recognition could be achieved more easily by supervisors than they assume in order to reduce the likelihood that MA leave the practice and the profession as a whole (e.g., acknowledge MA work compared to taking a leadership skill course). This includes an open dialog with MA on what MA perceive as recognition. However, it needs to be mentioned that employers are limited in their scope of action to address the understaffing and corresponding high workload of MA due to the already existing shortage of MA [ 45 ].

Methodological considerations

A strength of this study is that we included participants who actually left the MA profession rather than changed employer or merely had the intention leave. We interviewed both female ( n  = 18) and male former MA ( n  = 2) (98% of the general MA profession is female) [ 46 ]. Further, we included former MA from different (practice) settings (e.g., GP, specialist, rehabilitation center) as well as a broad range of years of work experience (ranging from 5 to 41 years), thereby likely exploring a large scope of potential views.

Nevertheless, this study has some limitations. First, we cannot rule out a recall bias. Most of the former MA left the profession three years ago at the time of the interviews (median = 40.5 months). This time lag could have influenced the participants’ perception of their decision process and of their reasons for leaving [ 47 ]. In addition, the experiences made in the new job might have changed the retrospective perception on their former MA profession. However, this time lag might also have been beneficial, as some former MA reported that they needed some time to be able to fully reflect on their process of leaving (e.g., MA realizing that they have been kept down by their employer).

We applied a convenient sample approach and a wide recruitment strategy. However, we did not recruit any participants with a low educational background. This might be due to a lower participation rate of people with a lower socio-economic status in health studies [ 48 ]. It could also imply that in particular MA with a higher level of education might be more likely to leave the MA profession potentially because they feel a higher need for career prospects and perceive to have more alternatives on the job market [ 49 ]. Further, we recruited one participant through personal contacts, as recruitment was initially difficult. However, the interviewer and the respective participant did not know each other personally. According to the analyzing author the participant’s responses were in line with those of the other participants and did not indicate any notable deviations. In addition, in terms of a potential selection bias, we excluded MA in training and did not particularly recruit MA who just finished training. Therefore, this study mainly covered perceptions of former MA with several years of work experience. Generational aspects as well as differences of current life phases (e.g., finishing further training, settling down, family planning, children being out of the house) might influence the constellation and weighting of the reasons for leaving [ 50 ]. Hence, future research should further focus on MA in sensitive periods of their life, ideally applying a life course approach [ 51 ] (e.g. novices on the job or those in the phases of family planning or parenthood). This would allow to follow MA over a long period of time and particularly gain insights on decision making regarding turnover during sensitive periods. It would additionally allow to explore potential differences in the priorities of reasons for leaving the MA profession as well as adequate retention measures.

This study elucidates the complex process that underlies the decision to leave the MA profession and explores the reasons as well as potential interventions. Reasons for leaving the MA profession were, for instance, a persistently high workload, limited career prospects and poor interpersonal relationships with supervisors and patients. In light of an already existing shortage of MA it seems necessary to counteract this trend. Former MA suggested several potential measures addressing relevant actors (e.g., supervisors, politics, MA themselves), such as to increase the salary, improve work processes, improve career prospects, increase scope of tasks and increase recognition from supervisors, patients and society.

Availability of data and materials

Data cannot be shared publicly because the transcripts contain highly sensitive information (e.g. conflicts with employers and colleagues, mental health). The ethics committee of the medical faculty of Düsseldorf would like to share the data on request only. The category system of qualitative content analysis is available from the corresponding author on reasonable request.

The collective wage agreement for MA is an agreement on the rights and obligations of employees and employers (e.g., salaries, special payments, working hours and vacation entitlement). It is negotiated between the Working Group for the Regulation of Working Conditions of Medical Assistants (AAA) representing outpatient physicians and the Association of Medical Professions (VmF) representing MA. Only if the employer and the MA are a member of the respective association (i.e., AAA or VmF), is the employer obliged to apply the collective agreement. However, membership to the AAA and VmF are voluntary. According to the 2024 collective agreement, the average hourly wage for MA is €16.63 per hour. In comparison, the minimum wage is €12.41 in Germany in 2024.

Health at a Glance. Europe 2022: state of Health in EU Cycle. Paris: OECD Publishing; 2022. (Health at a Glance: Europe).

Google Scholar  

Buchan J, Aiken L. Solving nursing shortages: a common priority. J Clin Nurs. 2008;17(24):3262–8.

Article   PubMed   PubMed Central   Google Scholar  

Kroezen M, Dussault G, Craveiro I, Dieleman M, Jansen C, Buchan J, et al. Recruitment and retention of health professionals across Europe: a literature review and multiple case study research. Health Policy (New York). 2015;119:1517–28.

Article   Google Scholar  

Buchan J, Twigg D, Dussault G, Duffield C, Stone PW. Policies to sustain the nursing workforce: an international perspective. Int Nurs Rev. 2015;62(2):162–70.

Article   CAS   PubMed   Google Scholar  

Flinkman M, Isopahkala-Bouret U, Salanterä S. Young registered nurses’ intention to leave the Profession and Professional turnover in early Career: a qualitative case study. Int Sch Res Notices. 2013;1:916061.

Tolksdorf KH, Tischler U, Heinrichs K. Correlates of turnover intention among nursing staff in the COVID-19 pandemic: a systematic review. BMC Nurs. 2022;21(1):1–17.

Hayes LJ, O’Brien-Pallas L, Duffield C, Shamian J, Buchan J, Hughes F, et al. Nurse turnover: a literature review–an update. Int J Nurs Stud. 2012;49(7):887–905.

Article   PubMed   Google Scholar  

Jones CB. Revisiting nurse turnover costs. J Nurs Adm. 2008;38(1):11–8.

Nei D, Snyder LA, Litwiller BJ. Promoting retention of nurses: a meta-Analytic examination of causes of nurse turnover. Health Care Manage Rev. 2015;40(3):237–53.

Hom PW, Mitchell TR, Lee TW, Griffeth RW. Reviewing employee turnover: focusing on proximal withdrawal States and an expanded criterion. Psychol Bull. 2012;138(5):831–58.

Halter M, Boiko O, Pelone F, Beighton C, Harris R, Gale J, et al. The determinants and consequences of adult nursing staff turnover: a systematic review of systematic reviews. BMC Health Serv Res. 2017;17(1):1–20.

Schmidt-Stiedenroth K, Mambrey V, Dreher A, Loerbroks A. Psychosocial working conditions and mental health among medical assistants in Germany: a scoping review. BMC Public Health. 2024;24(1):716.

Mambrey V, Li J, Loerbroks A. Working conditions predict turnover among medical assistants. Dtsch Arztebl Int. 2024;121(10):340–1.

PubMed   PubMed Central   Google Scholar  

Takase M. A concept analysis of turnover intention: implications for nursing management. Collegian. 2010;17(1):3–12.

Bahlman-van Ooijen W, Malfait S, Huisman-de Waal G, Hafsteinsdóttir TB. Nurses’ motivations to leave the nursing profession: a qualitative meta-aggregation. J Adv Nurs. 2023;79(12):4455–71.

Griffeth RW, Hom PW, Gaertner S. A meta-analysis of antecedents and correlates of employee turnover: update, moderator tests, and research implications for the next millennium. J Manag. 2000;26(3):463–88.

Kox JHAM, Groenewoud JH, Bakker EJM, Bierma-Zeinstra SMA, Runhaar J, Miedema HS, et al. Reasons why Dutch novice nurses leave nursing: A qualitative approach. Nurse Educ Pract. 2020;47:102848. https://doi.org/10.1016/j.nepr.2020.102848 .

Hung MSY, Lam SKK. Antecedents and contextual factors affecting occupational turnover among registered nurses in public hospitals in Hong Kong: a qualitative descriptive study. Int J Environ Res Public Health. 2020;17(11):3834.

Hayward D, Bungay V, Wolff AC, Macdonald V. A qualitative study of experienced nurses’ voluntary turnover: learning from their perspectives. J Clin Nurs. 2016;25(9–10):1336–45.

Zhu J, Rodgers S, Melia KM. A qualitative exploration of nurses leaving nursing practice in China. Nurs Open. 2015;2(1):3–13.

Poon YSR, Lin YP, Griffiths P, Yong KK, Seah B, Liaw SY. A global overview of healthcare workers’ turnover intention amid COVID-19 pandemic: a systematic review with future directions. Hum Resour Health. 2022;20(1):1–18.

Umbetkulova S, Kanderzhanova A, Foster F, Stolyarova V, Cobb-Zygadlo D. Mental health changes in healthcare workers during COVID-19 pandemic: a systematic review of longitudinal studies. Eval Health Prof. 2024;47(1):11–20.

Federal Statistical Office (Destatis). [GENESIS database: health workforce: Germany, years, employment, age groups, health professions] Datenbank GENESIS: Gesundheitspersonal: Deutschland, Jahre, Beschäftigungsverhältnis, Altersgruppen, Berufe im Gesundheitswesen. Wiesbaden, Germany; 2019.

National Association of Statutory Health Insurance Physicians, National Association of Statutory Health Insurance Funds. [Agreement on the delegation of medical services to non-physician staff in outpatient contract medical care pursuant to § 28 para. 1 sentence 3 SGB V] Vereinbarung über die Delegation ärztlicher Leistungen an nichtärztliches Personal in Der Ambulanten Ver. 2015;11. https://www.kbv.de/media/sp/24_Delegation.pdf .

Dreher A, Mambrey V, Loerbroks A. Changes of working conditions and job-related challenges due to the SARS-CoV-2 pandemic for medical assistants in general practices in Germany: a qualitative study. BMC Prim Care. 2022;23(1):1–14.

Deutscher Ärzteverlag GmbH. [Shortage of skilled workers could lead to reduction of office hours] Fachkräftemangel könnte zur Reduzierung von Sprechstundenzeiten führen. 2022. https://www.aerzteblatt.de/nachrichten/132505/Fachkraeftemangel-koennte-zur-Reduzierung-von-Sprechstundenzeiten-fuehren .

World Health Organization. Global strategy on human resources for health: workforce 2030. Geneva: World Health Organization; 2016.

Kuckartz U. Qualitative text analysis: a guide to methods, practice and using software. Los Angeles: Sage; 2014.

Book   Google Scholar  

Mayring P. Qualitative inhaltsanalyse: Grundlagen Und Techniken. 12th ed. Weinheim: Beltz; 2015.

Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.

Mambrey V, Angerer P, Loerbroks A. Psychosocial working conditions as determinants of concerns to have made important medical errors and possible intermediate factors of this association among medical assistants – a cohort study. BMC Health Serv Res. 2022;22(1):1–11.

Hennink M, Kaiser BN. Sample sizes for saturation in qualitative research: a systematic review of empirical tests. Soc Sci Med. 2022;292:114523.

Vu-Eickmann P, Loerbroks A. Psychosocial working conditions of physician assistants: results from a qualitative study on occupational stress, resources, possible approaches to prevention and intervention needs. Z fur Evidenz Fortbild Und Qualitat Im Gesundheitswesen. 2017;126:43–51.

Kuckartz U. Einführung in die computergestützte Analyse qualitativer Daten. 3., aktualisierte Auflage. Wiesbaden: VS Verlag für Sozialwissenschaften; 2010.

Cheung J. The decision process of leaving nursing. Aust Health Rev. 2004;28(3):340–8.

Brook J, Aitken L, Webb R, MacLaren J, Salmon D. Characteristics of successful interventions to reduce turnover and increase retention of early career nurses: a systematic review. Int J Nurs Stud. 2019;91:47–59.

Dawson AJ, Stasa H, Roche MA, Homer CSE, Duffield C. Nursing churn and turnover in Australian hospitals: nurses perceptions and suggestions for supportive strategies. BMC Nurs. 2014;13(1):1–10.

Shrank WH. Primary Care Practice Transformation and the rise of Consumerism. J Gen Intern Med. 2017;32(4):387–91.

Hoff T. The challenges of consumerism for primary care physicians. Am J Manag Care. 2020;26(1):e1–3.

Heimann S, North Rhine Association of Statutory Health Insurance Physicians. 2021. [Fachkräftemangel: Praxen fehlen qualifizierte MFA]. https://www.kvno.de/meta-navigation/suche/news/nachricht/fachkraeftemangel .

Scharf J, Vu-Eickmann P, Angerer P, Müller A, Loerbroks A. Work-related intervention needs of medical assistants and how to potentially address them according to supervising general practitioners: a qualitative study. Int J Environ Res Public Health. 2022;19(3):1359.

Siegrist J, Starke D, Chandola T, Godin I, Marmot M, Niedhammer I, et al. The measurement of effort-reward imbalance at work: European comparisons. Soc Sci Med. 2004;58(8):1483–99.

Morley G, Ives J, Bradbury-Jones C, Irvine F. What is ‘moral distress’? A narrative synthesis of the literature. Nurs Ethics. 2019;26(3):646–62.

Epstein E, Delgado S. Understanding and addressing Moral Distress. Online J Issues Nurs. 2010;15(3):1–12.

Oettel J, Wolf R, Zschille M, Leibner M. Personalsituation in Praxen der vertragsärztlichen und-psychotherapeutischen Versorgung. Zi-Paper 17/2021. Berlin, Germany; 2021.

Federal Employment Agency. (Statistics of the Federal Employment Agency. Employees subject to social insurance contributions in the occupational subgroup - medical assistants) Statistik der Bundesagentur für Arbeit: Sozialversicherungspflichtige Beschäftigte in der Berufsuntergrupp. 2019.

Colombo D, Suso-Ribera C, Fernández-Álvarez J, Cipresso P, Garcia-Palacios A, Riva G, et al. Affect Recall Bias: Being Resilient Distorting Real. 2020;44:906–18.

Demarest S, Van der Heyden J, Charafeddine R, Tafforeau J, Van Oyen H, Van Hal G. Socio-economic differences in participation of households in a Belgian national health survey. Eur J Public Health. 2013;23(6):981–5.

Carless SA, Arnup JL. A longitudinal study of the determinants and outcomes of career change. J Vocat Behav. 2011;78(1):80–91.

Roth C, Wensing M, Breckner A, Mahler C, Krug K, Berger S. Keeping nurses in nursing: a qualitative study of German nurses’ perceptions of push and pull factors to leave or stay in the profession. BMC Nurs. 2022;21(1):1–11.

Kuh D, Ben-Shlomo Y, Lynch J, Hallqvist J, Power C. Life course epidemiology. J Epidemiol Commun Health. 2003;57(10):778.

Article   CAS   Google Scholar  

Download references

Acknowledgements

This study was funded by the Berufsgenossenschaft für Gesundheitsdienst und Wohlfahrtspfelge (BGW) [Employer’s Liability Insurance Association for Health Services and Welfare] (funding number: 1590) which is gratefully acknowledged. We thank Louisa Scheepers for her comments on the topic guide and Lisa Guthardt for translating the verbatim quotes.

This study was funded by the Berufsgenossenschaft für Gesundheitsdienst und Wohlfahrtspflege (BGW) (funding number: 1590).

Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and affiliations.

Institute of Occupational, Social and Environmental Medicine, Centre for Health and Society, Faculty of Medicine and University hospital Düsseldorf, Heinrich Heine University Düsseldorf, Universitätsstr. 1, 40225, Düsseldorf, Germany

Viola Mambrey, Annegret Dreher & Adrian Loerbroks

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization of the research question: AL, VM; Study Design: AL, VM; Data Collection: VM; Data analysis: VM, AD; Interpretation of findings: VM, AL, Writing—original draft preparation: VM; Writing—review and editing: AL, AD. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Adrian Loerbroks .

Ethics declarations

Ethics approval and consent to participate.

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethics Committee of Medical Faculty of the Heinrich-Heine-University of Düsseldorf, Germany (ethic registration number 2023–2443). All participants received written study information and provided written informed consent.

Consent for publication

Not applicable.

Competing interests

Adrian Loerbroks has presented findings related to the health and working conditions of medical assistants at meetings or workshops of professional associations or companies (i.e., ABF-Synergie GmbH) and has received honoraria. The other authors declare no conflict of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. 

Supplementary Information

Supplementary material 1., supplementary material 2., supplementary material 3., supplementary material 4., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Mambrey, V., Dreher, A. & Loerbroks, A. Leaving the profession as a medical assistant: a qualitative study exploring the process, reasons and potential preventive measures. BMC Health Serv Res 24 , 1111 (2024). https://doi.org/10.1186/s12913-024-11607-7

Download citation

Received : 22 May 2024

Accepted : 18 September 2024

Published : 24 September 2024

DOI : https://doi.org/10.1186/s12913-024-11607-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Career exit
• Health personnel
• Leaving the profession
• Medical assistant
• Qualitative study

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]

Loading metrics

Open Access

Peer-reviewed

Research Article

Multilevel factors drive child exposure to enteric pathogens in animal feces: A qualitative study in northwestern coastal Ecuador

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Project administration, Resources, Software, Supervision, Visualization, Writing – original draft

Affiliations Department of Population Health Sciences, Georgia State University School of Public Health, Atlanta, Georgia, United States of America, Gangarosa Department of Environmental Health, Emory University Rollins School of Public Health, Atlanta, Georgia, United States of America

Roles Data curation, Formal analysis, Investigation, Methodology, Project administration, Validation, Writing – review & editing

Affiliation Universidad Técnica Luis Vargas Torres de Esmeraldas, Esmeraldas, Ecuador

Roles Formal analysis, Validation, Writing – review & editing

Affiliation Hubert Department of Global Health, Emory University Rollins School of Public Health, Atlanta, Georgia, United States of America

Affiliation Gangarosa Department of Environmental Health, Emory University Rollins School of Public Health, Atlanta, Georgia, United States of America

Roles Conceptualization, Methodology, Supervision, Writing – review & editing

Affiliation Department of Behavioral, Social, and Health Education Sciences, Emory University Rollins School of Public Health, Atlanta, Georgia, United States of America

Roles Conceptualization, Supervision, Writing – review & editing

Affiliation Department of Anthropology, Trinity College, Hartford, Connecticut, United States of America

Roles Conceptualization, Funding acquisition, Supervision, Writing – review & editing

Affiliation Department of Epidemiology, University of Michigan School of Public Health, Ann Arbor, Michigan, United States of America

Affiliation Rutgers Global Health Institute and Department of Biostatistics and Epidemiology, Rutgers School of Public Health, Piscataway, New Jersey, United States of America

* E-mail: [email protected]

Affiliation Department of Environmental and Occupational Health Sciences, University of Washington School of Public Health, Seattle, Washington, United States of America

Roles Conceptualization, Funding acquisition, Methodology, Supervision, Writing – review & editing

Affiliations Gangarosa Department of Environmental Health, Emory University Rollins School of Public Health, Atlanta, Georgia, United States of America, Hubert Department of Global Health, Emory University Rollins School of Public Health, Atlanta, Georgia, United States of America, Department of Behavioral, Social, and Health Education Sciences, Emory University Rollins School of Public Health, Atlanta, Georgia, United States of America

• April M. Ballard, 
• Betty Corozo Angulo, 
• Nicholas Laramee, 
• Jayden Pace Gallagher, 
• Regine Haardörfer, 
• Matthew C. Freeman, 
• James Trostle, 
• Joseph N. S. Eisenberg, 
• Gwenyth O. Lee, 

• Published: September 18, 2024
• https://doi.org/10.1371/journal.pgph.0003604
• Peer Review
• Reader Comments

Exposure to animal feces and associated enteric pathogens poses significant risks to child health. However, public health strategies to mitigate enteric infections among children largely aim to reduce exposure to human feces, overlooking transmission pathways related to animal feces. In this study we examine if and how children are exposed to enteric pathogens in animal feces in northwestern coastal Ecuador. We conducted qualitative interviews with mothers of children aged 10–18 months that owned ( n = 32) and did not own ( n = 26) animals in urban and rural communities. Using thematic analysis, we identified community, household, and child behavioral factors that influence exposure. We also compared child exposure by household animal ownership. Our findings revealed myriad opportunities for young children to be exposed to enteric pathogens in many locations and from multiple animal sources, regardless of household animal ownership. Animal feces management practices (AFM) used by mothers, such as rinsing feces into ditches and throwing feces into surrounding areas, may increase environmental contamination outside their homes and in their communities. Unsafe AFM practices were similar to unsafe child feces management practices reported in other studies, including practices related to defecation location, feces removal and disposal, environmental contamination cleaning, and handwashing. Findings suggest that animal feces may contaminate the environment along similar pathways as human feces. Identification and incorporation of safe AFM practices, similar to those developed for child feces management, would 1) mitigate child exposure to enteric pathogens by reducing animal feces contamination in domestic and public spaces; and 2) enable an integrated approach to address enteric pathogen exposure pathways related to animal and child feces.

Citation: Ballard AM, Corozo Angulo B, Laramee N, Pace Gallagher J, Haardörfer R, Freeman MC, et al. (2024) Multilevel factors drive child exposure to enteric pathogens in animal feces: A qualitative study in northwestern coastal Ecuador. PLOS Glob Public Health 4(9): e0003604. https://doi.org/10.1371/journal.pgph.0003604

Editor: Muhammad Asaduzzaman, University of Oslo Faculty of Medicine: Universitetet i Oslo Det medisinske fakultet, NORWAY

Received: March 5, 2024; Accepted: August 26, 2024; Published: September 18, 2024

Copyright: © 2024 Ballard et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: All relevant data are within the paper, which constitutes the minimal dataset for the study. For more information, contact April Ballard ( [email protected] ).

Funding: This work is supported by the National Institutes of Health (R01AI137679 to JNSE and KL), which provided financial support to AMB, BCA, MCF, GOL, and BAC. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Exposure to enteric pathogens during childhood is associated with substantial disease burden. Enteric infections and diarrheal diseases are the fifth leading cause of death in children under age five globally [ 1 – 5 ]. Persistent exposure to enteric pathogens during childhood can result in recurrent infections and lifelong consequences, such as deficits in growth and cognitive development [ 6 – 12 ]. Children in low- and middle-income countries (LMICs) bear the greatest burden of enteric disease due to pervasive fecal contamination of domestic environments resulting from inequities in access to improved water, sanitation, and hygiene (WASH) [ 13 , 14 ].

Interrupting the principal fecal-oral transmission pathways is critical to preventing enteric infections and related adverse health outcomes. Transmission principally occurs when feces from an infected host contaminate fluids, food, fomites, fingers, fields, and flies, followed by human exposure to the contamination through ingestion. This process is often visually depicted as the ‘F-diagram’ [ 15 – 19 ]. The provision of WASH services is a well-established public health strategy to prevent transmission of enteric pathogens, typically by targeting exposure to human feces [ 14 , 16 , 20 , 21 ]. However, transmission of enteric pathogens from animal feces has been overlooked in most WASH programming to date [ 19 , 22 – 24 ], despite the fact that animals produce approximately four times as much feces as humans [ 25 ] and many enteric pathogens capable of infecting humans are found in animal feces (e.g., Campylobacter spp., Cryptosporidium spp., enteropathogenic E . coli ) [ 26 ].

Understanding the upstream causes of environmental fecal contamination and child exposure to enteric pathogens in animal feces will be critical to the development of effective mitigation strategies to integrate into WASH programming. Various community, household, and child practices and behaviors can play a key role in exacerbating or mitigating exposure to enteric pathogens in animal feces. Animal husbandry and feces management practices, which are determined by diverse household and community factors, can increase contamination of the environment [ 18 , 27 ]. Children are then exposed through their interactions with animals, the environment and objects [ 18 ]. Current evidence is minimal and insufficient for determining a generalizable set of behaviors that influence zoonotic exposures [ 18 , 19 , 24 , 28 ]. However, community- and household-level factors related to animal husbandry and feces management [ 18 , 23 , 24 , 27 , 29 ]. may be root causes of exposure. For example, animal feces may be abundant throughout the domestic environment, regardless of household-level animal ownership, because letting animals roam freely to forage for food is a community norm that is perceived as beneficial to animals and reduces the financial burden of animal feed [ 27 , 30 – 33 ].

To address these key knowledge gaps, we qualitatively characterize exposure to enteric pathogens in animal feces among children in northwestern coastal Ecuador, a high enteric pathogen transmission setting. Previous studies have estimated the two-week prevalence of diarrhea among children under age five to be about 9% and of enteropathogenic E . coli infections to be around 25% [ 34 ]. We explore opportunities for and factors that influence child exposure at multiple levels (e.g., community, household, individual), including multiple communities along an urban-rural gradient with a range of conditions to increase the applicability to other LMICs. We also examine how household animal ownership influences exposure opportunities, which can provide important insights for potential mitigation strategies.

Study design and setting

We conducted qualitative research to understand if, why, how, and to what extent children are exposed to enteric pathogens in animal feces. To examine how community- and household-level factors may influence exposure, we interviewed mothers who owned and did not own animals. These mothers were participating in the Enteropatógenos, Crecimiento, Microbioma, y Diarrea (ECoMiD) study [ 34 ], a prospective cohort study in which mother-child dyads are followed from pregnancy through the critical first 24 months of life to examine how environmental exposures impact child gut microbiome composition and development. This study is reported in accordance with the Consolidated Criteria for Reporting Qualitative Research (COREQ) [ 35 ]. The location(s) where each of the 32 COREQ items is reported can be found in S2 Checklist , and our reflexivity statement can be found in S1 Text .

We carried out this work in seven ECoMiD study communities, representing four levels of varying urbanicity and rurality: (1) Esmeraldas (hereafter referred to as the urban community); (2) Borbón (a smaller town serving as a commercial center); (3) three rural villages near Borbón accessible by road (the rural road communities); and (4) two rural villages near Borbón only accessible by boat (the rural river communities) ( Fig 1 ). The study area is primarily populated by Afro-Ecuadorians, with an increasing number of people of mixed race (mestizos) and a small number of Chachis, an indigenous group. Esmeraldas is an urban hub of the study area and capital of Esmeraldas Province, with a population of over 160,000 [ 36 ], It is densely populated and has the most access to WASH infrastructure, roads, and medical infrastructure. Borbón is a town in Esmeraldas Province located at the confluence of the Cayapas, Santiago, and Onzole Rivers (population: 7,700) [ 36 ]. Borbón has underdeveloped infrastructure for its size, and basic WASH infrastructure of variable quality [ 37 , 38 ]. We also conducted the study in three smaller communities with access to a road connecting them to Borbón and other trade hubs in the area, and two smaller, more remote with access to Borbón primarily via river. The rural road communities typically have more limited infrastructure, such as simple piped water systems, and the river communities predominantly rely on river water, wells, and rainwater [ 37 , 38 ].

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

The maps for this figure were created using ArcGIS Online [ 39 ], the Outline Map Basemap [ 40 ], and the World Countries Generalized layer [ 41 ].

https://doi.org/10.1371/journal.pgph.0003604.g001

Sample and participant selection

To examine how household-level factors may influence exposure, we enrolled two types of participants from the ECoMiD cohort: (1) mothers in households that owned at least one animal and (2) mothers in households that did not own animals. Our original study design called for 30 interviews with animal-owning mothers and 30 with non-animal-owning mothers, which was based on recommendations to conduct at least 16–24 in-depth interviews (IDIs) and to have a larger sample when studying complex topics [ 42 , 43 ]. Mothers were eligible if their child in the cohort was between 6–18 months old. This age range was selected because children become more mobile and active during this time, making them particularly susceptible to environmental exposures. We used quota sampling to ensure an equal number of mothers who did and did not have animals in each of the four levels of urbanicity, if possible. To capture variability, we included mothers that owned different types and numbers of animals. Local study staff facilitated recruitment in each community by calling cohort mothers who had a child between 6–18 months old to query their animal ownership status and interest in participating.

Data collection

Author BCA, who is a woman from Esmeraldas and has conducted qualitative research for more than 10 years, conducted go-along, semi-structured IDIs in Spanish from January 21 st to April 21 st , 2021. Go along IDIs enable simultaneous observation and interviewing as the interviewer and participant inhabit and engage with the spaces they are discussing [ 44 , 45 ], which was ideal for our study objective. To understand how children may be exposed to enteric pathogens in animal feces, we asked mothers about a typical day for them and their child. Probes queried details about animals, environmental conditions, behaviors, and seasonality because interviews were conducted during the rainy season. The interview concluded with questions about reasons for and benefits of animal ownership and intra-household decision-making related to animals and the child. During interviews, mothers introduced the interviewer to household animals and showed where the animal(s) lived and spent time, as relevant. Basic demographics, household characteristics, and the type and number of animals (if any) owned by households were collected via a short survey. The IDI guide and short survey are provided in S1 Data . Qualitative Data Collection Tools. Systematic debriefing sessions were held between author AMB and BCA throughout data collection using a standard set of questions [ 46 ] to ascertain emerging themes in the data and enhance our approach in real time.

Interviews were audio recorded with permission from mothers. The go-along portions of interviews were not typically audio recorded due to logistical challenges (e.g., loud background noises, issues with audio recording due to social distancing requirements), but the interviewer took photos and detailed field notes about observations and the information ascertained during this portion of the interview. The audio recorded portion of interviews lasted 27 minutes on average (range: 15–50 minutes). Recordings were transcribed and de-identified by an Ecuadorian, and then translated from Spanish to English verbatim by two other Ecuadorians. To standardize transcript formatting and obtain quality transcriptions and translations, we trained the translators on the research topic, interview content, conducting first-pass transcript reviewing while translating, and the goal of achieving meaning equivalence. Author AMB debriefed with translators after each of the initial five translations were completed and checked translations for accuracy. English translations were stored alongside the original Spanish transcripts, which allowed us to interact with the original and translated versions throughout analyses to conduct second-pass transcript reviews and to improve the rigor of our analyses. When mothers refused to be audio recorded ( n = 17), the interviewer took detailed notes and created a transcript using the interview guide immediately following the interview. Mothers received an assortment of food items (e.g., rice, beans) as compensation for their time.

Data analysis

To identify key themes in the data, we conducted thematic analysis using MaxQDA 2020 software (VERBI Software, Berlin, Germany). A codebook with deductive and inductive codes was developed iteratively throughout the analysis process using existing literature, transcript readings, and debriefing notes. To standardize our coding approach and ensure reliability, we double-coded two sets of five transcripts, cross-checking coding strategies and interpretation of data by each coder after each set. Subsequently, transcripts were double coded 10 at a time, after which coding agreement was checked to address inter-rater reliability issues. Then, the two coders systematically debriefed [ 46 ], resolved coding differences, and wrote memos on key themes. We did not calculate inter-rater agreement statistics to assess inter-rater reliability because coding was part of the process to discover themes, so agreement was not always the goal [ 47 ], and differences in coding style result in artificial low agreement [ 47 , 48 ].

We assessed code and meaning saturation throughout the coding process [ 42 , 43 , 49 ] by tracking the number of additional codes and code definition changes there were after each round of coding (i.e., every 10 transcripts). Code saturation was considered achieved when 90% of meaningful codes were identified and developed, which occurred after coding five transcripts in this study. Meaning saturation was considered met when 90% of core codes had fully developed characteristics, which occurred after coding 10 transcripts. After coding, segments from transcripts for each code and intersections of prominent codes were queried and memos were written. Queries, memoing, and debriefing were performed iteratively to explore, describe, compare, conceptualize, and explain key themes–using the social ecological model [ 50 ] as a sensitizing construct to inform our interpretation and organization of the results throughout the analysis process [ 51 ]. Mothers’ animal ownership status at the time of the interview was used to conduct comparative analyses. A description of major themes, along with their corresponding sub-themes and parent and child codes, is provided in S2 Data . Analytic Codes.

All participants provided written consent prior to data collection and received a copy of the consent form. Participants’ rights to skip questions and end interviews at any time were emphasized by the interviewer. Institutional Review Boards at Emory University (IRB # 00101202) and Universidad San Francisco de Quito (IRB # 2018-022M and 021-011M) approved all study procedures. Additional information regarding the ethical, cultural, and scientific considerations specific to inclusivity in global research is including in the Supporting Information ( S1 Checklist . Inclusivity Checklist).

Every child, regardless of household animal ownership, had opportunities to be exposed to enteric pathogens due to the ubiquity of animals and animal feces in the environment. Children had direct contact with animals and potentially came into contact with animal feces on surfaces, in environmental media, and on objects. Community norms and environmental factors and conditions influenced the quantity of animals and animal feces in the environment, as well as their proximity to children. Fig 2 summarizes these multi-level influences on potential child exposure that our data revealed, while Fig 3 provides a visual depiction of the influences across locations where children spend time daily.

https://doi.org/10.1371/journal.pgph.0003604.g002

https://doi.org/10.1371/journal.pgph.0003604.g003

The final sample consisted of 32 mothers in households with animals, and 28 without. Mothers were 28 years old on average (range: 19–47 years). Children were 10 to 18 months old and approximately half (52%, n = 30) were female. Our final sample did not include children between 6–10 months old because few children that age were enrolled in the cohort at the time of recruitment due to a pause in the study at the beginning of the COVID-19 pandemic. The type of water and sanitation access and animal ownership varied across communities. Sixty-six percent of households used water from an improved source for their child’s drinking water and 81% had improved sanitation facilities. Over half of mothers who did not own animals (58%, n = 15) at the time of the interview had previously owned animals. Additional demographic information for the total sample and by study site are presented in Table 1 .

https://doi.org/10.1371/journal.pgph.0003604.t001

Child behaviors

Children performed behaviors daily that may lead to exposure to enteric pathogens inside and outside of their household and in other locations (e.g., parks, playgrounds, outdoor gatherings) where animals and animal feces were present. Children spent much of the day inside their house, though most played outside near their household regularly. They also habitually spent time in other locations, including at relatives’ and neighbors’ houses, community parks or playgrounds, and outdoor gatherings (e.g., bingo). Some mothers reported that child behaviors and interactions at home were quite different than those in other locations. For example, one child played in a garden by the river outside their grandmother’s house, but largely played inside when they were at home.

“It is different [at her grandmother’s house] because she is not inside the house there…She goes to the river side and sits and observes, searches for stones, throws stones to the river, things like that.” -age 39, rural road community, non-animal owner

Children had direct contact with animals across multiple locations, regardless of household animal ownership. Most interacted with dogs (e.g., petting, grabbing, and playing with them) that spent substantial amounts of time outdoors and were owned by their household, their relatives, or their neighbors. Some children had contact with cats (e.g., grabbing, touching, and carrying them), though most did not because mothers stated that cats “carry disease” and “cause asthma.” A few mothers felt that contact with cats and dogs was beneficial for their child’s immune system.

“For my way of thinking, [animal contact] is so nothing will make her sick so that her body is adjusted to cats and dogs…so I tell her to touch them for her body’s reaction…” -age 25, urban community, non-animal owner

Children played in indoor and outdoor spaces that were regularly contaminated with feces from dogs, cats, and free-range household chickens (hereafter referred to as free-range chickens), though no mother reported their child to have direct contact with animal feces. A few mothers who owned production chickens and pigs to generate household income also reported the presence of feces from these animals outside their household. Production chickens and pigs were kept outside near households contained within pens or pigsties, which also contained their feces. Children commonly crawled and walked freely, often unsupervised, throughout their house in the mornings and afternoons while mothers performed chores and cared for their other children. Free-range chickens and dogs were more active and reportedly entered households during mornings and afternoons, indicating that children may be in the same space as animals and their feces unsupervised. During this time, children played with toys and objects that they threw on the ground repeatedly, increasing the likelihood that objects and children’s hands may become contaminated with animal feces and related enteric pathogens. For example, multiple mothers reported children playing on the bare floor with kitchen objects (e.g., pots, pans, spoons, glasses) that were later used for cooking and eating or to drink water. Other objects that children played with may have been contaminated with animal feces and related enteric pathogens because they were high-touch objects (e.g., television remotes, cell phones) or because of their functional purpose (e.g., a tool for cleaning).

“The shoes [are her favorite toy]. And…what she likes to grab the most is also here in the kitchen…she grabs the pans or she starts to play with the trays…She grabs the broom, she puts it down and starts sweeping.” -age 19, commercial center, non-animal owner

Children also played with toys, sticks, soil or mud, sand, rocks, and surface water outside near households and in public spaces. Some mothers reported toys becoming contaminated with animal feces when their child played with them outside.

Child environment

The child environment refers to the close surroundings and daily conditions in which children lived, had direct contact with, and impacted their proximity and potential exposure to animal feces. This included aspects and conditions of the locations where children spent time and the individuals in close contact with them. Mothers reported that various individuals’ contact with animals and animal feces, as well as features of the built environment where children spent time, influenced the proximity of children to animals and animal feces.

Others’ contact with animals and animal feces.

Mothers, siblings, extended family members, and other individuals who interacted with children had frequent contact with animals and/or animal feces, regardless of household animal ownership. Interactions with animals varied in intensity and included dogs, cats, free-range chickens, and pigs. Some mothers, extended family members, and other household visitors were reported to have intense contact with animals (e.g., raising chickens, bathing pigs).

“I keep [the dogs and cats] clean so that they don’t get fleas, ticks, or any of that. I wipe them down over there. They sleep in a dry place, and I keep the outside area where they poop clean. If I ever see a tick, they get an injection, or I wash them with chemicals.” -age 28, urban community, owner of two dogs and two cats

However, most mothers had less intense interactions with animals. For example, some walked and played with family members’ dogs and others fed leftover food to dogs, cats, and free-range chickens. Older siblings commonly had contact with and helped care for dogs and cats. Mothers found it more appropriate for their older children to interact with and care for animals.

Mothers’, siblings’, and extended family members’ contact with animal feces across multiple locations was common. Mothers and grandparents had contact with feces from dogs, free-range chickens, and unspecified animals while removing it from where children play at their household and other locations (e.g., relatives’ or neighbors’ households, parks, playgrounds, outdoor social gatherings). Some mothers used a general “we” when reporting feces removal and disposal practices (e.g., “We throw it out.”), suggesting that multiple household members have contact with animal feces and may contribute to contamination of children’s interpersonal environment. Handwashing after removing feces was not always mentioned and surfaces were inconsistently cleaned with soap or disinfectants (e.g., bleach)–suggesting that environmental fecal contamination may remain after feces are removed. Household members also inadvertently stepped or put toys in animal feces, which resulted in feces contamination inside households.

“…Animal feces are brought inside, especially from dogs, on children’s shoes…it can happen suddenly. There are remains [of animal feces outside] and children while playing at night do not see well and step in it and bring it in on their shoes.” -age 26, rural road communities, owner of two dogs and one cat

Rat feces were observed in some kitchens by the interviewer, though mothers did not discuss this.

Built environment features.

Household gaps, open doors, lack of fencing, and ground-level entrances influenced animals’ ability to be present and defecate inside and outside near households. Mothers reported that cats–owned by the family, owned by others, and strays–were the most common animals in and near houses. Cats spent time on roofs and were difficult to keep out of houses because they could enter through any gaps or openings, especially at night. Some mothers made cats leave immediately while others gave them food even if they did not own them. Cats that were allowed to remain indoors spent time on living room furniture near where children played, under or on dining room tables where food was consumed, and/or in kitchens where food was prepared.

“The cats come in and out of the house. They get under the bed, under the dining table. The dog also enters and leaves, but spends more time outside…The cats spend time in the kitchen, on the floor, under the dog…” -age 32, rural road communities, owner of one dog and two cats

Mothers reported that they rarely found cat feces outside near their household, regardless of cat ownership, because it was buried in dirt or sand. However, cat-owning households reported finding feces inside on occasion.

Dogs were present frequently, but primarily outside near households, rather than indoors. Most dog-owning mothers reported that their dog(s) entered their house briefly on a typical day, to be fed by and spend time with their owners and/or because household doors were left open. During those times, dogs sometimes defecated inside. Mothers who did not own dogs rarely reported dogs entering their house and defecating. However, it was common practice for owners to let their dogs roam freely during the day, which allowed dogs to defecate outside near households that did not own dogs and lacked a fence. As a result, dog feces were found outside in household entryways or yards in the mornings and afternoons daily, regardless of ownership. Most mothers found multiple piles of dog feces near their household daily. In response, some tried to prevent dogs from being near their household. Others stated that they would put leftover food outside to feed the dogs and avoid food waste, which encouraged the presence of dogs regardless of ownership status.

“I don’t allow [animals to get near the house] because they get used to it. There is a dog that knew that I put food out sometimes so as not to throw it away, but I don’t give it to him anymore because they get used to it.” -age 25, rural road communities, non-animal owner

Free-range chickens and their feces were found outside near many households throughout the morning. The lack of fencing around households allowed free-range chickens to roam from compound to compound and defecate near households that did and did not own animals. When household doors were left open, nearby chickens reportedly entered and defecated inside some households. A few mothers reported that the raised entrance to their house prevented free-range chickens from entering. For example, one mother’s house was raised up on stilts, making the house’s door approximately three meters off the ground. Most mothers did not actively deter the presence of free-range chickens, and some encouraged their presence by putting leftover food outside, similar to dogs.

At other locations where children spent time, the same features of the built environment (i.e., household gaps, open doors, lack of fencing, and ground-level entrances) allowed cats, dogs, and free-range chickens and their feces to be proximal to children. In some cases, the types of animals and animal feces that were present in other locations differed from the child’s household. For example, some relatives owned animals that were not present at the child’s household.

“…At my mom’s house, she has like nine dogs and like two cats. So [my child] is over there, and my sister brings her up so she spends time with the dogs and playing with my nephews that are also there.” -age 22, urban community, owner of one cat

Similar to trends in children’s homes, animals were often present and defecated near where children spent time, such as in relatives’ households and in parks. Most mothers observed dog feces outside others’ households, in the street where children played, and at parks. Some reported the presence of cat and free-range chicken feces in these other locations, though this was less common. Cats, dogs, and free-range chickens also entered inside relatives’ and neighbors’ households due to structural gaps, lack of fencing, and open doors. However, indoor fecal contamination in others’ households was similar to indoor contamination at children’s homes: infrequent and largely from shoes or objects.

“We always clean at my mother’s place because the children play outside a lot. As I say, I go to my mother’s house every day. Sometimes we sweep together… [the animal feces] are picked up, but I would say yes [feces are brought inside] because sometimes they step on it without noticing.” -age 27, rural road community, non-animal owner

Community norms

Community norms refer to sets of behaviors or practices that are widely accepted and expected within a community, including those related to collective responsibility, health and well-being, and environmental hygiene, among others. Norms that influenced child proximity to animals and animal feces pertained to the animal management and husbandry practices and feces management practices at a child’s household and throughout their community.

Animal management and husbandry.

It was common for animal owners in the study area to let their cats, dogs, and free-range chickens roam freely throughout communities. As described above, this practice allowed animals to move from compound to compound and defecate in or near many places where children spent time.

“The dog, cat, and chickens all spend time outside the house…The cat is in the brush. The chickens, like, they are being raised freely. They just walk through the town. They come back at night to sleep, and the cat does too. [The cat] practically just comes in at night.” -age 31, commercial center, owner of 15 free-range chickens, one dog, and one cat

Free-range practices were perceived to be healthier for the animals and helped offset the economic burden of feed by allowing animals to forage for food. Mothers reported that people who owned dogs let them roam free during the day specifically to urinate, defecate, and/or forage for food. Free-range chickens were also released during the day to forage for food, and then placed in enclosures inside or outside near households in the afternoon or at night for protection against predators and theft. Uniquely, cats were largely active at night and would leave their owners’ house in search of food. In contrast to cats, dogs, and free-range chickens, pigs and production chickens used to generate household income were contained within pens or pigsties, which prevented them from roaming throughout communities.

“[The pigs are raised] in the pigsty…because if we let them loose, they walk around in other people’s patios daringly and there are people that don’t like that. Or the pigs can get sick, so that’s why they live locked up there.” -age 35, rural river community, owner of four free-range chickens, four dogs, two pigs, and four cats

Animal feces management.

Mothers used multiple and varying animal feces management practices that contributed to the contamination of their household and the surrounding community environment. Practices depended on the type of animal feces, the animal’s defecation location, and other factors (e.g., availability of trash collection services), which are described in Table 2 . Management differences by animal type were related to the frequency of defecation, the different types and sizes of stool, and whether the animal buried their feces. For example, mothers rarely managed cat feces because cats buried their feces, whereas dog feces were abundant and a noticeable nuisance that mothers removed from their yard regularly. Mothers removed and disposed of dog feces in the trash when garbage collection services were available, and reported rinsing or throwing feces into the surrounding area when garbage collection was irregular, infrequent, or unavailable or if feces were dried out.

“The poop is collected with a shovel and thrown directly into the surrounding vegetation because sometimes [the dog feces] are already dry and I do not wait for the garbage cart to throw them away…” -age 31, commercial center, non-animal owner

https://doi.org/10.1371/journal.pgph.0003604.t002

Feces from free-range chickens, production chickens, and pigs were also regularly rinsed away using a hose or bucket of water or thrown into the surrounding area, potentially spreading fecal contamination rather than eliminating it.

Regardless of animal type, feces on floors inside the home or in soil outside near child domestic and play areas were removed quickly most of the time, in hopes of allowing children to play in feces-free environments. Removing feces from outdoor household and public play spaces was less common and more difficult due to free-range animals, resulting in child toys and shoes becoming contaminated with feces.

Natural environmental conditions

Natural environmental conditions refer to the atmospheric characteristics of a geographic area based on its climate and seasonal weather patterns, and include temperature and precipitation, among others. Extreme weather events, such as heavy rain and flooding that commonly occurred during the rainy season, influenced the number of animals present and their proximity to children. Some mothers reported that animals died from drowning during this time. Others reported bringing their dogs inside their house more often to escape bad weather. Additionally, the number of free-range chickens in communities and near children varied by season because of flooding.

“[During the winter], it rains too much. And when the river grows, it floods all the town and the houses are practically sunk…So we can’t raise [chickens] like that because they have nowhere to run…During the summer, we can raise them better, but now in the winter, it is not possible.” -age 25, rural river communities, non-animal owner

By qualitatively characterizing the interrelated community, household, and child factors that drive exposure to enteric pathogens in animal feces, we identified critical insights for the development of effective mitigation strategies. We found that animals and animal feces were ubiquitous–regardless of animal ownership–due to community- and household-level animal and feces management practices. Although 66% of households had access to improved drinking water sources and 81% had improved sanitation facilities, all mothers reported opportunities for their child to be exposed to animal feces, even though 45% of households did not own animals. These findings are in line with other studies showing that WASH interventions aiming to reduce enteric infections by targeting human feces alone likely overlook other significant sources of environmental fecal contamination and enteric pathogens [ 19 , 22 , 24 , 52 – 54 ]. The findings also highlight that focusing only on animal ownership and only on household environments provide insufficient information to identify the relative risk of child exposure to enteric pathogens in animal feces. Collectively, our results suggest that reducing enteric pathogen transmission will require integrated programming that targets both human and animal feces and addresses the multilevel, upstream drivers of environmental fecal contamination and child exposure. Below we highlight two key findings and their implications.

First, free-range animal management and husbandry practices at the community- and household-level resulted in the persistent presence of animals and fecal contamination in children’s environments, including at their household and at other locations (e.g., parks, playgrounds). Existing evidence clearly demonstrates that child household and play spaces can be contaminated with animal feces and associated enteric pathogens when free-range animals are present [ 27 , 32 , 55 – 57 ], and that proximity to animals and such contaminants increases the risk of enteric infections among children [ 29 , 31 , 52 – 54 , 58 – 60 ]. Our findings add to existing research by highlighting that child exposure to enteric pathogens in animal feces is not solely shaped by household practices but is also significantly shaped by the normative practices of others in their community. Findings further suggest that sole focus on the household environment provides inaccurate and/or incomplete data because other significant locations where children may be exposed to enteric pathogens in animal feces can be missed. Consequently, research and interventions exclusively targeting the household level may be insufficient to examine and reduce child exposure to animal feces.

Second, inadequate animal feces management (AFM) practices, which include behaviors beyond removal and disposal, contributed to fecal contamination of children’s environments. While animal feces near child domestic and play areas were removed the majority of the time, maternal handwashing was seldom discussed and surfaces were inconsistently cleaned with soap or disinfectants after removal of animal feces. In Bangladesh, maternal hand contamination with animal feces was near universal despite the reported use of tools to clean up animal feces [ 61 ] and in rural India, removing child feces without using tools was associated with increased hand contamination [ 62 , 63 ]. Findings from these studies suggest that handwashing after handling animal feces could help reduce child exposure to enteric pathogens, especially given studies showing strong correlations between animal feces and enteric pathogen contamination on caregiver and child hands in the same household [ 61 , 64 ]. Similarly, cleaning or disinfecting surfaces after removal could reduce exposure, as a study in rural India found that environmental fecal contamination remains even after child feces are removed [ 62 ]. The modalities by and locations where mothers reported disposing of animal feces in our study–such as rinsing feces into drains or ditches and throwing feces into surrounding vegetation–also have been shown to increase environmental contamination in studies on child feces. [ 62 , 63 , 65 ] and can intensify transmission through various pathways. As a result, inadequate AFM practices in one household could impact the environmental contamination and exposure of children in neighboring households.

Taken together, these findings suggest that integration of safe AFM practices with existing child feces management programming is an important area for future research. Research could enable identification of integrated exposure control approaches that capture the many enteric pathogen exposure pathways related to both animal and human feces. Research on child feces management suggests that unsafe practices along the feces management pathway–which includes defecation, feces removal and disposal, defecation location cleaning, anal cleansing, and handwashing–increase environmental contamination [ 62 , 63 , 65 , 66 ]. The AFM practices identified in this study are similar to child feces management practices reported elsewhere [ 62 , 63 , 65 , 66 ], and previously observed among mothers in the ECoMiD cohort [ 66 ]. Incorporation of safe practices along the AFM pathway, similar to those developed for child feces management, (e.g., remove feces using a tool, clean defecation location with soap and water) [ 65 ], may therefore be an effective, practical approach for intervening on the multiplicity of exposures related to various animal sources. To establish safe AFM practices, future research should assess unsafe practices and feces contamination along the pathway established for child feces using surveys, observation or spot checks, and environmental sampling. The established child feces management pathway can guide data collection at key points to validate the AFM pathway, with changes made as relevant.

These results demonstrate that multilevel, multisectoral interventions to mitigate child exposure to enteric pathogens in animal feces are needed. Existing interventions have overwhelmingly included a single-component and primarily focused on the individual child- or household-level (e.g., building and encouraging use of enclosures to contain animals and animal feces, providing and encouraging use of child playpens to minimize their contact with environments contaminated by animals, etc.) [ 18 , 67 ]. However, intervening on exposure to enteric pathogens in animal feces–like human-sourced enteric pathogen exposures–requires the disruption of the upstream causes of environmental fecal contamination and the multiple exposure pathways to diverse enteric pathogens. Approaches that target exposure factors at multiple levels should consider features of the natural and built environment, as well as aspects of community, household, and child practices and behaviors. Such approaches could be integrated into existing comprehensive WASH interventions that aim to mitigate exposure to human feces.

Strengths and limitations

This study used rigorous qualitative methods (i.e., analyzing verbatim transcripts, double coding, systematic debriefing, and assessment and achievement of meaning and code saturation) that strengthen the validity of findings [ 42 , 43 , 68 , 69 ]. It included participants from multiple communities along an urban-rural gradient with a range of conditions, increasing the generalizability of our findings. The sample sizes were however uneven across communities due to circumstances surrounding the COVID-19 pandemic. Still, we found that the overall sample size was sufficient for saturation [ 43 ]. Additionally, our final sample only included children between 10–18 months old, which could limit the applicability of our results to children aged 6–10 months. Lastly, reliance on mothers may have biased our findings because they were not always the main or sole caregiver on a typical day and could have provided incomplete or inaccurate information about their child. However, the use of go-along IDIs enabled simultaneous in-depth interviewing and observation of the child’s environment, which ascertained key details that were not reliant on maternal reporting. Reflections, observations, and analyses conducted by author BCA, an experienced local researcher, strengthened interpretation by providing insights that enhanced the credibility of community comparisons and information provided by mothers [ 42 , 68 ].

Conclusions

Current approaches to control enteric pathogens that center on individual- or household-level interventions are likely insufficient to address the multifaceted nature of exposure to enteric pathogens in animal feces. Future mitigation strategies should adopt a broader approach that considers the multilevel nature of exposure, including factors at the community, household, and child levels. Such strategies will need to distinguish between and address both the upstream causes of environmental contamination and more proximal causes of enteric pathogen exposure in their design to enable interventions to be more targeted and effective.

Supporting information

S1 checklist. inclusivity checklist..

https://doi.org/10.1371/journal.pgph.0003604.s001

S2 Checklist. COREQ checklist.

https://doi.org/10.1371/journal.pgph.0003604.s002

S1 Text. Reflexivity statement.

https://doi.org/10.1371/journal.pgph.0003604.s003

S1 Data. Qualitative data collection tools.

https://doi.org/10.1371/journal.pgph.0003604.s004

S2 Data. Analytic codes.

https://doi.org/10.1371/journal.pgph.0003604.s005

Acknowledgments

We would like to thank local field staff for their help with recruiting, logistics, and data collection for this project. We would also like to thank all the participants in Ecuador that made this project possible.

• View Article
• Google Scholar
• PubMed/NCBI
• 36. City Population: Republic of Ecuador, 2010 census 2010 [Available from: https://www.citypopulation.de/en/ecuador/ .
• 39. Esri. ArcGIS Online. https://www.arcgis.com/index.html .
• 40. Esri. Outline Map. 2021.
• 41. Esri. World Countries Generalized. 2023.
• 42. Hennink M, Hutter I, Bailey A. Qualitative Research Methods: SAGE Publications, Inc.; 2010 2019-12–07.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

The PMC website is updating on October 15, 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• BMJ Glob Health
• v.4(Suppl 1); 2019

Synthesising quantitative and qualitative evidence to inform guidelines on complex interventions: clarifying the purposes, designs and outlining some methods

1 School of Social Sciences, Bangor University, Wales, UK

Andrew Booth

2 School of Health and Related Research (ScHARR), University of Sheffield, Sheffield, UK

Graham Moore

3 School of Social Sciences, Cardiff University, Wales, UK

Kate Flemming

4 Department of Health Sciences, The University of York, York, UK

Özge Tunçalp

5 Department of Reproductive Health and Research including UNDP/UNFPA/UNICEF/WHO/World Bank Special Programme of Research, Development and Research Training in Human Reproduction (HRP), World Health Organization, Geneva, Switzerland

Elham Shakibazadeh

6 Department of Health Education and Promotion, School of Public Health, Tehran University of Medical Sciences, Tehran, Iran

Associated Data

bmjgh-2018-000893supp001.pdf

bmjgh-2018-000893supp002.pdf

bmjgh-2018-000893supp003.pdf

bmjgh-2018-000893supp005.pdf

bmjgh-2018-000893supp004.pdf

Guideline developers are increasingly dealing with more difficult decisions concerning whether to recommend complex interventions in complex and highly variable health systems. There is greater recognition that both quantitative and qualitative evidence can be combined in a mixed-method synthesis and that this can be helpful in understanding how complexity impacts on interventions in specific contexts. This paper aims to clarify the different purposes, review designs, questions, synthesis methods and opportunities to combine quantitative and qualitative evidence to explore the complexity of complex interventions and health systems. Three case studies of guidelines developed by WHO, which incorporated quantitative and qualitative evidence, are used to illustrate possible uses of mixed-method reviews and evidence. Additional examples of methods that can be used or may have potential for use in a guideline process are outlined. Consideration is given to the opportunities for potential integration of quantitative and qualitative evidence at different stages of the review and guideline process. Encouragement is given to guideline commissioners and developers and review authors to consider including quantitative and qualitative evidence. Recommendations are made concerning the future development of methods to better address questions in systematic reviews and guidelines that adopt a complexity perspective.

Summary box

• When combined in a mixed-method synthesis, quantitative and qualitative evidence can potentially contribute to understanding how complex interventions work and for whom, and how the complex health systems into which they are implemented respond and adapt.
• The different purposes and designs for combining quantitative and qualitative evidence in a mixed-method synthesis for a guideline process are described.
• Questions relevant to gaining an understanding of the complexity of complex interventions and the wider health systems within which they are implemented that can be addressed by mixed-method syntheses are presented.
• The practical methodological guidance in this paper is intended to help guideline producers and review authors commission and conduct mixed-method syntheses where appropriate.
• If more mixed-method syntheses are conducted, guideline developers will have greater opportunities to access this evidence to inform decision-making.

Introduction

Recognition has grown that while quantitative methods remain vital, they are usually insufficient to address complex health systems related research questions. 1 Quantitative methods rely on an ability to anticipate what must be measured in advance. Introducing change into a complex health system gives rise to emergent reactions, which cannot be fully predicted in advance. Emergent reactions can often only be understood through combining quantitative methods with a more flexible qualitative lens. 2 Adopting a more pluralist position enables a diverse range of research options to the researcher depending on the research question being investigated. 3–5 As a consequence, where a research study sits within the multitude of methods available is driven by the question being asked, rather than any particular methodological or philosophical stance. 6

Publication of guidance on designing complex intervention process evaluations and other works advocating mixed-methods approaches to intervention research have stimulated better quality evidence for synthesis. 1 7–13 Methods for synthesising qualitative 14 and mixed-method evidence have been developed or are in development. Mixed-method research and review definitions are outlined in box 1 .

Defining mixed-method research and reviews

Pluye and Hong 52 define mixed-methods research as “a research approach in which a researcher integrates (a) qualitative and quantitative research questions, (b) qualitative research methods* and quantitative research designs, (c) techniques for collecting and analyzing qualitative and quantitative evidence, and (d) qualitative findings and quantitative results”.A mixed-method synthesis can integrate quantitative, qualitative and mixed-method evidence or data from primary studies.† Mixed-method primary studies are usually disaggregated into quantitative and qualitative evidence and data for the purposes of synthesis. Thomas and Harden further define three ways in which reviews are mixed. 53

• The types of studies included and hence the type of findings to be synthesised (ie, qualitative/textual and quantitative/numerical).
• The types of synthesis method used (eg, statistical meta-analysis and qualitative synthesis).
• The mode of analysis: theory testing AND theory building.

*A qualitative study is one that uses qualitative methods of data collection and analysis to produce a narrative understanding of the phenomena of interest. Qualitative methods of data collection may include, for example, interviews, focus groups, observations and analysis of documents.

†The Cochrane Qualitative and Implementation Methods group coined the term ‘qualitative evidence synthesis’ to mean that the synthesis could also include qualitative data. For example, qualitative data from case studies, grey literature reports and open-ended questions from surveys. ‘Evidence’ and ‘data’ are used interchangeably in this paper.

This paper is one of a series that aims to explore the implications of complexity for systematic reviews and guideline development, commissioned by WHO. This paper is concerned with the methodological implications of including quantitative and qualitative evidence in mixed-method systematic reviews and guideline development for complex interventions. The guidance was developed through a process of bringing together experts in the field, literature searching and consensus building with end users (guideline developers, clinicians and reviewers). We clarify the different purposes, review designs, questions and synthesis methods that may be applicable to combine quantitative and qualitative evidence to explore the complexity of complex interventions and health systems. Three case studies of WHO guidelines that incorporated quantitative and qualitative evidence are used to illustrate possible uses of mixed-method reviews and mechanisms of integration ( table 1 , online supplementary files 1–3 ). Additional examples of methods that can be used or may have potential for use in a guideline process are outlined. Opportunities for potential integration of quantitative and qualitative evidence at different stages of the review and guideline process are presented. Specific considerations when using an evidence to decision framework such as the Developing and Evaluating Communication strategies to support Informed Decisions and practice based on Evidence (DECIDE) framework 15 or the new WHO-INTEGRATE evidence to decision framework 16 at the review design and evidence to decision stage are outlined. See online supplementary file 4 for an example of a health systems DECIDE framework and Rehfuess et al 16 for the new WHO-INTEGRATE framework. Encouragement is given to guideline commissioners and developers and review authors to consider including quantitative and qualitative evidence in guidelines of complex interventions that take a complexity perspective and health systems focus.

Designs and methods and their use or applicability in guidelines and systematic reviews taking a complexity perspective

Supplementary data

Taking a complexity perspective.

The first paper in this series 17 outlines aspects of complexity associated with complex interventions and health systems that can potentially be explored by different types of evidence, including synthesis of quantitative and qualitative evidence. Petticrew et al 17 distinguish between a complex interventions perspective and a complex systems perspective. A complex interventions perspective defines interventions as having “implicit conceptual boundaries, representing a flexible, but common set of practices, often linked by an explicit or implicit theory about how they work”. A complex systems perspective differs in that “ complexity arises from the relationships and interactions between a system’s agents (eg, people, or groups that interact with each other and their environment), and its context. A system perspective conceives the intervention as being part of the system, and emphasises changes and interconnections within the system itself”. Aspects of complexity associated with implementation of complex interventions in health systems that could potentially be addressed with a synthesis of quantitative and qualitative evidence are summarised in table 2 . Another paper in the series outlines criteria used in a new evidence to decision framework for making decisions about complex interventions implemented in complex systems, against which the need for quantitative and qualitative evidence can be mapped. 16 A further paper 18 that explores how context is dealt with in guidelines and reviews taking a complexity perspective also recommends using both quantitative and qualitative evidence to better understand context as a source of complexity. Mixed-method syntheses of quantitative and qualitative evidence can also help with understanding of whether there has been theory failure and or implementation failure. The Cochrane Qualitative and Implementation Methods Group provide additional guidance on exploring implementation and theory failure that can be adapted to address aspects of complexity of complex interventions when implemented in health systems. 19

Health-system complexity-related questions that a synthesis of quantitative and qualitative evidence could address (derived from Petticrew et al 17 )

It may not be apparent which aspects of complexity or which elements of the complex intervention or health system can be explored in a guideline process, or whether combining qualitative and quantitative evidence in a mixed-method synthesis will be useful, until the available evidence is scoped and mapped. 17 20 A more extensive lead in phase is typically required to scope the available evidence, engage with stakeholders and to refine the review parameters and questions that can then be mapped against potential review designs and methods of synthesis. 20 At the scoping stage, it is also common to decide on a theoretical perspective 21 or undertake further work to refine a theoretical perspective. 22 This is also the stage to begin articulating the programme theory of the complex intervention that may be further developed to refine an understanding of complexity and show how the intervention is implemented in and impacts on the wider health system. 17 23 24 In practice, this process can be lengthy, iterative and fluid with multiple revisions to the review scope, often developing and adapting a logic model 17 as the available evidence becomes known and the potential to incorporate different types of review designs and syntheses of quantitative and qualitative evidence becomes better understood. 25 Further questions, propositions or hypotheses may emerge as the reviews progress and therefore the protocols generally need to be developed iteratively over time rather than a priori.

Following a scoping exercise and definition of key questions, the next step in the guideline development process is to identify existing or commission new systematic reviews to locate and summarise the best available evidence in relation to each question. For example, case study 2, ‘Optimising health worker roles for maternal and newborn health through task shifting’, included quantitative reviews that did and did not take an additional complexity perspective, and qualitative evidence syntheses that were able to explain how specific elements of complexity impacted on intervention outcomes within the wider health system. Further understanding of health system complexity was facilitated through the conduct of additional country-level case studies that contributed to an overall understanding of what worked and what happened when lay health worker interventions were implemented. See table 1 online supplementary file 2 .

There are a few existing examples, which we draw on in this paper, but integrating quantitative and qualitative evidence in a mixed-method synthesis is relatively uncommon in a guideline process. Box 2 includes a set of key questions that guideline developers and review authors contemplating combining quantitative and qualitative evidence in mixed-methods design might ask. Subsequent sections provide more information and signposting to further reading to help address these key questions.

Key questions that guideline developers and review authors contemplating combining quantitative and qualitative evidence in a mixed-methods design might ask

Compound questions requiring both quantitative and qualitative evidence?

Questions requiring mixed-methods studies?

Separate quantitative and qualitative questions?

Separate quantitative and qualitative research studies?

Related quantitative and qualitative research studies?

Mixed-methods studies?

Quantitative unpublished data and/or qualitative unpublished data, eg, narrative survey data?

Throughout the review?

Following separate reviews?

At the question point?

At the synthesis point?

At the evidence to recommendations stage?

Or a combination?

Narrative synthesis or summary?

Quantitising approach, eg, frequency analysis?

Qualitising approach, eg, thematic synthesis?

Tabulation?

Logic model?

Conceptual model/framework?

Graphical approach?

• WHICH: Which mixed-method designs, methodologies and methods best fit into a guideline process to inform recommendations?

Complexity-related questions that a synthesis of quantitative and qualitative evidence can potentially address

Petticrew et al 17 define the different aspects of complexity and examples of complexity-related questions that can potentially be explored in guidelines and systematic reviews taking a complexity perspective. Relevant aspects of complexity outlined by Petticrew et al 17 are summarised in table 2 below, together with the corresponding questions that could be addressed in a synthesis combining qualitative and quantitative evidence. Importantly, the aspects of complexity and their associated concepts of interest have however yet to be translated fully in primary health research or systematic reviews. There are few known examples where selected complexity concepts have been used to analyse or reanalyse a primary intervention study. Most notable is Chandler et al 26 who specifically set out to identify and translate a set of relevant complexity theory concepts for application in health systems research. Chandler then reanalysed a trial process evaluation using selected complexity theory concepts to better understand the complex causal pathway in the health system that explains some aspects of complexity in table 2 .

Rehfeuss et al 16 also recommends upfront consideration of the WHO-INTEGRATE evidence to decision criteria when planning a guideline and formulating questions. The criteria reflect WHO norms and values and take account of a complexity perspective. The framework can be used by guideline development groups as a menu to decide which criteria to prioritise, and which study types and synthesis methods can be used to collect evidence for each criterion. Many of the criteria and their related questions can be addressed using a synthesis of quantitative and qualitative evidence: the balance of benefits and harms, human rights and sociocultural acceptability, health equity, societal implications and feasibility (see table 3 ). Similar aspects in the DECIDE framework 15 could also be addressed using synthesis of qualitative and quantitative evidence.

Integrate evidence to decision framework criteria, example questions and types of studies to potentially address these questions (derived from Rehfeuss et al 16 )

GIS, Geographical Information System; RCT, randomised controlled trial.

Questions as anchors or compasses

Questions can serve as an ‘anchor’ by articulating the specific aspects of complexity to be explored (eg, Is successful implementation of the intervention context dependent?). 27 Anchor questions such as “How does intervention x impact on socioeconomic inequalities in health behaviour/outcome x” are the kind of health system question that requires a synthesis of both quantitative and qualitative evidence and hence a mixed-method synthesis. Quantitative evidence can quantify the difference in effect, but does not answer the question of how . The ‘how’ question can be partly answered with quantitative and qualitative evidence. For example, quantitative evidence may reveal where socioeconomic status and inequality emerges in the health system (an emergent property) by exploring questions such as “ Does patterning emerge during uptake because fewer people from certain groups come into contact with an intervention in the first place? ” or “ are people from certain backgrounds more likely to drop out, or to maintain effects beyond an intervention differently? ” Qualitative evidence may help understand the reasons behind all of these mechanisms. Alternatively, questions can act as ‘compasses’ where a question sets out a starting point from which to explore further and to potentially ask further questions or develop propositions or hypotheses to explore through a complexity perspective (eg, What factors enhance or hinder implementation?). 27 Other papers in this series provide further guidance on developing questions for qualitative evidence syntheses and guidance on question formulation. 14 28

For anchor and compass questions, additional application of a theory (eg, complexity theory) can help focus evidence synthesis and presentation to explore and explain complexity issues. 17 21 Development of a review specific logic model(s) can help to further refine an initial understanding of any complexity-related issues of interest associated with a specific intervention, and if appropriate the health system or section of the health system within which to contextualise the review question and analyse data. 17 23–25 Specific tools are available to help clarify context and complex interventions. 17 18

If a complexity perspective, and certain criteria within evidence to decision frameworks, is deemed relevant and desirable by guideline developers, it is only possible to pursue a complexity perspective if the evidence is available. Careful scoping using knowledge maps or scoping reviews will help inform development of questions that are answerable with available evidence. 20 If evidence of effect is not available, then a different approach to develop questions leading to a more general narrative understanding of what happened when complex interventions were implemented in a health system will be required (such as in case study 3—risk communication guideline). This should not mean that the original questions developed for which no evidence was found when scoping the literature were not important. An important function of creating a knowledge map is also to identify gaps to inform a future research agenda.

Table 2 and online supplementary files 1–3 outline examples of questions in the three case studies, which were all ‘COMPASS’ questions for the qualitative evidence syntheses.

Types of integration and synthesis designs in mixed-method reviews

The shift towards integration of qualitative and quantitative evidence in primary research has, in recent years, begun to be mirrored within research synthesis. 29–31 The natural extension to undertaking quantitative or qualitative reviews has been the development of methods for integrating qualitative and quantitative evidence within reviews, and within the guideline process using evidence to decision-frameworks. Advocating the integration of quantitative and qualitative evidence assumes a complementarity between research methodologies, and a need for both types of evidence to inform policy and practice. Below, we briefly outline the current designs for integrating qualitative and quantitative evidence within a mixed-method review or synthesis.

One of the early approaches to integrating qualitative and quantitative evidence detailed by Sandelowski et al 32 advocated three basic review designs: segregated, integrated and contingent designs, which have been further developed by Heyvaert et al 33 ( box 3 ).

Segregated, integrated and contingent designs 32 33

Segregated design.

Conventional separate distinction between quantitative and qualitative approaches based on the assumption they are different entities and should be treated separately; can be distinguished from each other; their findings warrant separate analyses and syntheses. Ultimately, the separate synthesis results can themselves be synthesised.

Integrated design

The methodological differences between qualitative and quantitative studies are minimised as both are viewed as producing findings that can be readily synthesised into one another because they address the same research purposed and questions. Transformation involves either turning qualitative data into quantitative (quantitising) or quantitative findings are turned into qualitative (qualitising) to facilitate their integration.

Contingent design

Takes a cyclical approach to synthesis, with the findings from one synthesis informing the focus of the next synthesis, until all the research objectives have been addressed. Studies are not necessarily grouped and categorised as qualitative or quantitative.

A recent review of more than 400 systematic reviews 34 combining quantitative and qualitative evidence identified two main synthesis designs—convergent and sequential. In a convergent design, qualitative and quantitative evidence is collated and analysed in a parallel or complementary manner, whereas in a sequential synthesis, the collation and analysis of quantitative and qualitative evidence takes place in a sequence with one synthesis informing the other ( box 4 ). 6 These designs can be seen to build on the work of Sandelowski et al , 32 35 particularly in relation to the transformation of data from qualitative to quantitative (and vice versa) and the sequential synthesis design, with a cyclical approach to reviewing that evokes Sandelowski’s contingent design.

Convergent and sequential synthesis designs 34

Convergent synthesis design.

Qualitative and quantitative research is collected and analysed at the same time in a parallel or complementary manner. Integration can occur at three points:

a. Data-based convergent synthesis design

All included studies are analysed using the same methods and results presented together. As only one synthesis method is used, data transformation occurs (qualitised or quantised). Usually addressed one review question.

b. Results-based convergent synthesis design

Qualitative and quantitative data are analysed and presented separately but integrated using a further synthesis method; eg, narratively, tables, matrices or reanalysing evidence. The results of both syntheses are combined in a third synthesis. Usually addresses an overall review question with subquestions.

c. Parallel-results convergent synthesis design

Qualitative and quantitative data are analysed and presented separately with integration occurring in the interpretation of results in the discussion section. Usually addresses two or more complimentary review questions.

Sequential synthesis design

A two-phase approach, data collection and analysis of one type of evidence (eg, qualitative), occurs after and is informed by the collection and analysis of the other type (eg, quantitative). Usually addresses an overall question with subquestions with both syntheses complementing each other.

The three case studies ( table 1 , online supplementary files 1–3 ) illustrate the diverse combination of review designs and synthesis methods that were considered the most appropriate for specific guidelines.

Methods for conducting mixed-method reviews in the context of guidelines for complex interventions

In this section, we draw on examples where specific review designs and methods have been or can be used to explore selected aspects of complexity in guidelines or systematic reviews. We also identify other review methods that could potentially be used to explore aspects of complexity. Of particular note, we could not find any specific examples of systematic methods to synthesise highly diverse research designs as advocated by Petticrew et al 17 and summarised in tables 2 and 3 . For example, we could not find examples of methods to synthesise qualitative studies, case studies, quantitative longitudinal data, possibly historical data, effectiveness studies providing evidence of differential effects across different contexts, and system modelling studies (eg, agent-based modelling) to explore system adaptivity.

There are different ways that quantitative and qualitative evidence can be integrated into a review and then into a guideline development process. In practice, some methods enable integration of different types of evidence in a single synthesis, while in other methods, the single systematic review may include a series of stand-alone reviews or syntheses that are then combined in a cross-study synthesis. Table 1 provides an overview of the characteristics of different review designs and methods and guidance on their applicability for a guideline process. Designs and methods that have already been used in WHO guideline development are described in part A of the table. Part B outlines a design and method that can be used in a guideline process, and part C covers those that have the potential to integrate quantitative, qualitative and mixed-method evidence in a single review design (such as meta-narrative reviews and Bayesian syntheses), but their application in a guideline context has yet to be demonstrated.

Points of integration when integrating quantitative and qualitative evidence in guideline development

Depending on the review design (see boxes 3 and 4 ), integration can potentially take place at a review team and design level, and more commonly at several key points of the review or guideline process. The following sections outline potential points of integration and associated practical considerations when integrating quantitative and qualitative evidence in guideline development.

Review team level

In a guideline process, it is common for syntheses of quantitative and qualitative evidence to be done separately by different teams and then to integrate the evidence. A practical consideration relates to the organisation, composition and expertise of the review teams and ways of working. If the quantitative and qualitative reviews are being conducted separately and then brought together by the same team members, who are equally comfortable operating within both paradigms, then a consistent approach across both paradigms becomes possible. If, however, a team is being split between the quantitative and qualitative reviews, then the strengths of specialisation can be harnessed, for example, in quality assessment or synthesis. Optimally, at least one, if not more, of the team members should be involved in both quantitative and qualitative reviews to offer the possibility of making connexions throughout the review and not simply at re-agreed junctures. This mirrors O’Cathain’s conclusion that mixed-methods primary research tends to work only when there is a principal investigator who values and is able to oversee integration. 9 10 While the above decisions have been articulated in the context of two types of evidence, variously quantitative and qualitative, they equally apply when considering how to handle studies reporting a mixed-method study design, where data are usually disaggregated into quantitative and qualitative for the purposes of synthesis (see case study 3—risk communication in humanitarian disasters).

Question formulation

Clearly specified key question(s), derived from a scoping or consultation exercise, will make it clear if quantitative and qualitative evidence is required in a guideline development process and which aspects will be addressed by which types of evidence. For the remaining stages of the process, as documented below, a review team faces challenges as to whether to handle each type of evidence separately, regardless of whether sequentially or in parallel, with a view to joining the two products on completion or to attempt integration throughout the review process. In each case, the underlying choice is of efficiencies and potential comparability vs sensitivity to the underlying paradigm.

Once key questions are clearly defined, the guideline development group typically needs to consider whether to conduct a single sensitive search to address all potential subtopics (lumping) or whether to conduct specific searches for each subtopic (splitting). 36 A related consideration is whether to search separately for qualitative, quantitative and mixed-method evidence ‘streams’ or whether to conduct a single search and then identify specific study types at the subsequent sifting stage. These two considerations often mean a trade-off between a single search process involving very large numbers of records or a more protracted search process retrieving smaller numbers of records. Both approaches have advantages and choice may depend on the respective availability of resources for searching and sifting.

Screening and selecting studies

Closely related to decisions around searching are considerations relating to screening and selecting studies for inclusion in a systematic review. An important consideration here is whether the review team will screen records for all review types, regardless of their subsequent involvement (‘altruistic sifting’), or specialise in screening for the study type with which they are most familiar. The risk of missing relevant reports might be minimised by whole team screening for empirical reports in the first instance and then coding them for a specific quantitative, qualitative or mixed-methods report at a subsequent stage.

Assessment of methodological limitations in primary studies

Within a guideline process, review teams may be more limited in their choice of instruments to assess methodological limitations of primary studies as there are mandatory requirements to use the Cochrane risk of bias tool 37 to feed into Grading of Recommendations Assessment, Development and Evaluation (GRADE) 38 or to select from a small pool of qualitative appraisal instruments in order to apply GRADE; Confidence in the Evidence from Reviews of Qualitative Research (GRADE-CERQual) 39 to assess the overall certainty or confidence in findings. The Cochrane Qualitative and Implementation Methods Group has recently issued guidance on the selection of appraisal instruments and core assessment criteria. 40 The Mixed-Methods Appraisal Tool, which is currently undergoing further development, offers a single quality assessment instrument for quantitative, qualitative and mixed-methods studies. 41 Other options include using corresponding instruments from within the same ‘stable’, for example, using different Critical Appraisal Skills Programme instruments. 42 While using instruments developed by the same team or organisation may achieve a degree of epistemological consonance, benefits may come more from consistency of approach and reporting rather than from a shared view of quality. Alternatively, a more paradigm-sensitive approach would involve selecting the best instrument for each respective review while deferring challenges from later heterogeneity of reporting.

Data extraction

The way in which data and evidence are extracted from primary research studies for review will be influenced by the type of integrated synthesis being undertaken and the review purpose. Initially, decisions need to be made regarding the nature and type of data and evidence that are to be extracted from the included studies. Method-specific reporting guidelines 43 44 provide a good template as to what quantitative and qualitative data it is potentially possible to extract from different types of method-specific study reports, although in practice reporting quality varies. Online supplementary file 5 provides a hypothetical example of the different types of studies from which quantitative and qualitative evidence could potentially be extracted for synthesis.

The decisions around what data or evidence to extract will be guided by how ‘integrated’ the mixed-method review will be. For those reviews where the quantitative and qualitative findings of studies are synthesised separately and integrated at the point of findings (eg, segregated or contingent approaches or sequential synthesis design), separate data extraction approaches will likely be used.

Where integration occurs during the process of the review (eg, integrated approach or convergent synthesis design), an integrated approach to data extraction may be considered, depending on the purpose of the review. This may involve the use of a data extraction framework, the choice of which needs to be congruent with the approach to synthesis chosen for the review. 40 45 The integrative or theoretical framework may be decided on a priori if a pre-developed theoretical or conceptual framework is available in the literature. 27 The development of a framework may alternatively arise from the reading of the included studies, in relation to the purpose of the review, early in the process. The Cochrane Qualitative and Implementation Methods Group provide further guidance on extraction of qualitative data, including use of software. 40

Synthesis and integration

Relatively few synthesis methods start off being integrated from the beginning, and these methods have generally been subject to less testing and evaluation particularly in a guideline context (see table 1 ). A review design that started off being integrated from the beginning may be suitable for some guideline contexts (such as in case study 3—risk communication in humanitarian disasters—where there was little evidence of effect), but in general if there are sufficient trials then a separate systematic review and meta-analysis will be required for a guideline. Other papers in this series offer guidance on methods for synthesising quantitative 46 and qualitative evidence 14 in reviews that take a complexity perspective. Further guidance on integrating quantitative and qualitative evidence in a systematic review is provided by the Cochrane Qualitative and Implementation Methods Group. 19 27 29 40 47

Types of findings produced by specific methods

It is highly likely (unless there are well-designed process evaluations) that the primary studies may not themselves seek to address the complexity-related questions required for a guideline process. In which case, review authors will need to configure the available evidence and transform the evidence through the synthesis process to produce explanations, propositions and hypotheses (ie, findings) that were not obvious at primary study level. It is important that guideline commissioners, developers and review authors are aware that specific methods are intended to produce a type of finding with a specific purpose (such as developing new theory in the case of meta-ethnography). 48 Case study 1 (antenatal care guideline) provides an example of how a meta-ethnography was used to develop a new theory as an end product, 48 49 as well as framework synthesis which produced descriptive and explanatory findings that were more easily incorporated into the guideline process. 27 The definitions ( box 5 ) may be helpful when defining the different types of findings.

Different levels of findings

Descriptive findings —qualitative evidence-driven translated descriptive themes that do not move beyond the primary studies.

Explanatory findings —may either be at a descriptive or theoretical level. At the descriptive level, qualitative evidence is used to explain phenomena observed in quantitative results, such as why implementation failed in specific circumstances. At the theoretical level, the transformed and interpreted findings that go beyond the primary studies can be used to explain the descriptive findings. The latter description is generally the accepted definition in the wider qualitative community.

Hypothetical or theoretical finding —qualitative evidence-driven transformed themes (or lines of argument) that go beyond the primary studies. Although similar, Thomas and Harden 56 make a distinction in the purposes between two types of theoretical findings: analytical themes and the product of meta-ethnographies, third-order interpretations. 48

Analytical themes are a product of interrogating descriptive themes by placing the synthesis within an external theoretical framework (such as the review question and subquestions) and are considered more appropriate when a specific review question is being addressed (eg, in a guideline or to inform policy). 56

Third-order interpretations come from translating studies into one another while preserving the original context and are more appropriate when a body of literature is being explored in and of itself with broader or emergent review questions. 48

Bringing mixed-method evidence together in evidence to decision (EtD) frameworks

A critical element of guideline development is the formulation of recommendations by the Guideline Development Group, and EtD frameworks help to facilitate this process. 16 The EtD framework can also be used as a mechanism to integrate and display quantitative and qualitative evidence and findings mapped against the EtD framework domains with hyperlinks to more detailed evidence summaries from contributing reviews (see table 1 ). It is commonly the EtD framework that enables the findings of the separate quantitative and qualitative reviews to be brought together in a guideline process. Specific challenges when populating the DECIDE evidence to decision framework 15 were noted in case study 3 (risk communication in humanitarian disasters) as there was an absence of intervention effect data and the interventions to communicate public health risks were context specific and varied. These problems would not, however, have been addressed by substitution of the DECIDE framework with the new INTEGRATE 16 evidence to decision framework. A d ifferent type of EtD framework needs to be developed for reviews that do not include sufficient evidence of intervention effect.

Mixed-method review and synthesis methods are generally the least developed of all systematic review methods. It is acknowledged that methods for combining quantitative and qualitative evidence are generally poorly articulated. 29 50 There are however some fairly well-established methods for using qualitative evidence to explore aspects of complexity (such as contextual, implementation and outcome complexity), which can be combined with evidence of effect (see sections A and B of table 1 ). 14 There are good examples of systematic reviews that use these methods to combine quantitative and qualitative evidence, and examples of guideline recommendations that were informed by evidence from both quantitative and qualitative reviews (eg, case studies 1–3). With the exception of case study 3 (risk communication), the quantitative and qualitative reviews for these specific guidelines have been conducted separately, and the findings subsequently brought together in an EtD framework to inform recommendations.

Other mixed-method review designs have potential to contribute to understanding of complex interventions and to explore aspects of wider health systems complexity but have not been sufficiently developed and tested for this specific purpose, or used in a guideline process (section C of table 1 ). Some methods such as meta-narrative reviews also explore different questions to those usually asked in a guideline process. Methods for processing (eg, quality appraisal) and synthesising the highly diverse evidence suggested in tables 2 and 3 that are required to explore specific aspects of health systems complexity (such as system adaptivity) and to populate some sections of the INTEGRATE EtD framework remain underdeveloped or in need of development.

In addition to the required methodological development mentioned above, there is no GRADE approach 38 for assessing confidence in findings developed from combined quantitative and qualitative evidence. Another paper in this series outlines how to deal with complexity and grading different types of quantitative evidence, 51 and the GRADE CERQual approach for qualitative findings is described elsewhere, 39 but both these approaches are applied to method-specific and not mixed-method findings. An unofficial adaptation of GRADE was used in the risk communication guideline that reported mixed-method findings. Nor is there a reporting guideline for mixed-method reviews, 47 and for now reports will need to conform to the relevant reporting requirements of the respective method-specific guideline. There is a need to further adapt and test DECIDE, 15 WHO-INTEGRATE 16 and other types of evidence to decision frameworks to accommodate evidence from mixed-method syntheses which do not set out to determine the statistical effects of interventions and in circumstances where there are no trials.

When conducting quantitative and qualitative reviews that will subsequently be combined, there are specific considerations for managing and integrating the different types of evidence throughout the review process. We have summarised different options for combining qualitative and quantitative evidence in mixed-method syntheses that guideline developers and systematic reviewers can choose from, as well as outlining the opportunities to integrate evidence at different stages of the review and guideline development process.

Review commissioners, authors and guideline developers generally have less experience of combining qualitative and evidence in mixed-methods reviews. In particular, there is a relatively small group of reviewers who are skilled at undertaking fully integrated mixed-method reviews. Commissioning additional qualitative and mixed-method reviews creates an additional cost. Large complex mixed-method reviews generally take more time to complete. Careful consideration needs to be given as to which guidelines would benefit most from additional qualitative and mixed-method syntheses. More training is required to develop capacity and there is a need to develop processes for preparing the guideline panel to consider and use mixed-method evidence in their decision-making.

This paper has presented how qualitative and quantitative evidence, combined in mixed-method reviews, can help understand aspects of complex interventions and the systems within which they are implemented. There are further opportunities to use these methods, and to further develop the methods, to look more widely at additional aspects of complexity. There is a range of review designs and synthesis methods to choose from depending on the question being asked or the questions that may emerge during the conduct of the synthesis. Additional methods need to be developed (or existing methods further adapted) in order to synthesise the full range of diverse evidence that is desirable to explore the complexity-related questions when complex interventions are implemented into health systems. We encourage review commissioners and authors, and guideline developers to consider using mixed-methods reviews and synthesis in guidelines and to report on their usefulness in the guideline development process.

Handling editor: Soumyadeep Bhaumik

Contributors: JN, AB, GM, KF, ÖT and ES drafted the manuscript. All authors contributed to paper development and writing and agreed the final manuscript. Anayda Portela and Susan Norris from WHO managed the series. Helen Smith was series Editor. We thank all those who provided feedback on various iterations.

Funding: Funding provided by the World Health Organization Department of Maternal, Newborn, Child and Adolescent Health through grants received from the United States Agency for International Development and the Norwegian Agency for Development Cooperation.

Disclaimer: ÖT is a staff member of WHO. The author alone is responsible for the views expressed in this publication and they do not necessarily represent the decisions or policies of WHO.

Competing interests: No financial interests declared. JN, AB and ÖT have an intellectual interest in GRADE CERQual; and JN has an intellectual interest in the iCAT_SR tool.

Patient consent: Not required.

Provenance and peer review: Not commissioned; externally peer reviewed.

Data sharing statement: No additional data are available.

Supplemental material: This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Log in using your username and password

• Search More Search for this keyword Advanced search
• Latest content
• For authors
• Browse by collection
• BMJ Journals

You are here

• Volume 14, Issue 9
• Child health-friendly neighbourhood: a qualitative study to explore the perspectives and experiences of experts and mothers of children under 6 years of age in Tehran, Iran
• Article Text
• Article info
• Citation Tools
• Rapid Responses
• Article metrics

• http://orcid.org/0000-0002-2800-8165 Parisa Akhbari 1 ,
• Nastaran Keshavarz-Mohammadi 2 ,
• Ali Ramezankhani 1
• 1 Department of Public Health , Shahid Beheshti University of Medical Sciences , Tehran , Iran (the Islamic Republic of)
• 2 Depertment of Public Health , Shahid Beheshti University of Medical Sciences , Tehran , Iran (the Islamic Republic of)
• Correspondence to Dr Nastaran Keshavarz-Mohammadi; n_keshavars{at}yahoo.com

Objectives Creating health-supportive environments is one of the key strategies for health promotion. The WHO launched the Healthy Cities Initiative which has inspired other international organisations to develop settings-based health initiatives, such as the Child Friendly Cities by UNICEF. Our study aimed to explore the perspectives and experiences of experts, city council staff and mothers of children under 6 years of age in the city of Tehran, Iran regarding child health-friendly neighbourhoods for children of this age group.

Design The purpose of this qualitative research was to investigate the viewpoints and experiences of mothers of children under 6 years old as well as professionals. Data were collected from January to July 2022 through semistructured, indepth interviews using an interview guide. Data were analysed using the directed content analysis method with MAXQDA V.2020 software.

Setting The study was conducted in Tehran, Iran.

Participants Participants were selected from three main groups: experts, mothers and city council staff. Participants were invited to take part using variation purposive sampling techniques.

Results Data analysis led to a definition of the concept of child health-friendly neighbourhoods for children under 6 years old, with 6 dimensions, 21 subdimensions and 80 characteristics. The six dimensions included the provision of neighbourhood green space, cultural centres, health centres, access to services, transport and security. The characteristics we identified had similarities and differences with UNICEF’s Child Friendly Cities.

Conclusion The concept of a child health-friendly neighbourhood for children under 6 years old is the result of a health-centred approach to a child-friendly city that provides a deeper understanding of the needs and services required to start a healthy life. This could contribute to further dialogue, research and actions to make all neighbourhoods a health-supportive environment as recommended by the Ottawa Charter for Health Promotion.

• Health & safety
• Child protection
• Health Education

Data availability statement

The data are not publicly available due to confidentiality of participants and data protection laws with respect to this study.

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/ .

https://doi.org/10.1136/bmjopen-2023-077167

Statistics from Altmetric.com

Request permissions.

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

The main strength of this study is that the needs of children were identified and investigated from the perspectives of experts, city council staff and mothers of children under 6 years of age.

To the best of our knowledge, this study is the first to examine the characteristics of a child health-friendly neighbourhood and the needs of young children in the Iranian context.

One potential limitation is that fathers and children were not included in the study.

Based on the conditions in each neighbourhood, the needs of children in each neighbourhood may be different.

Introduction

Acknowledging the role of social determinants of health and the birth of the field of health promotion in 1986 have created an important shift in understanding and thus strategies for protecting and promoting population health. 1 Based on the Ottawa Charter for Health Promotion, the creation of ‘health supportive environments’ is considered the third of the five main strategies for health promotion, before the development of individual health skills and provision of healthcare services. 2 Since then, the WHO has followed a health-promoting setting approach by launching its Healthy Cities Initiative in 1986. 2 3 Cities are an important place especially for children. It is estimated that by 2030 about 60% of urban residents will be under the age of 18. 4 For this reason, several initiatives were proposed to transform cities to healthier settings, such as the Child Friendly Cities initiated by UNICEF 4 5 and London’s 2017 Living Cities project, with a view to creating child-friendly cities and using an urban design approach that focuses on children and their health across the lifespan. 6 7 There is an increasing focus on taking children’s needs into account when designing cities and developing urban programmes and services more comprehensively. For example, the international initiative ‘City 95’ looked at cities from the perspectives of 3-year-old children with an average height of 95 cm. 8 9

A child-friendly city is one that prioritises the interests of children and their fundamental rights, including health, education and cultural opportunities, as well as safety and security, green spaces and play places. It refers to a city where children have a healthy start and have equal opportunities in life regardless of their ethnicity, religion, income, gender or abilities. 10 11 UNICEF proposed seven domains to explain the main indicators of a child-friendly city, namely playgrounds and green spaces, education and culture, health and children’s health, access to services, participation in decision-making, safety and ease of transport, and children’s security. 12

Although a child in the child-friendly city concept refers to an individual in the 0–18 years old age group, 13 most programmes and studies on child-friendly cities have focused on children 6–18 years old, 14–16 ignoring the needs of those under 6 years of age. Furthermore, the cities or neighbourhoods have mostly been studied from the perspectives of children’s social rights, focusing on urban planning, social rights areas 17–20 and the livability of the city or neighbourhoods for children. 21 There was limited consideration of a health-centred approach to developing our understanding of children’s needs. 22 Finally, the lack of specific indicators to measure the state of cities and neighbourhoods for children hinders the comprehensive implementation of a child-friendly city concept. 23 24 As context is a crucial factor in the implementation of a child-friendly city approach, the standards or criteria can vary, bearing in mind that they should reflect the conditions of the local context, institutions and organisations in each country. 22 Because they are among the most vulnerable, younger children may have demands that are different from older groups when it comes to neighbourhood features. 25 Besides, there is a limited number of studies that investigated the needs of children from the perspectives of mothers and relevant experts. Therefore, to fill this gap, especially within the Iranian context, a comprehensive mixed methods study was designed to develop a national Child Health-Friendly Neighborhood for Under 6 Years of Age Children (U6CHFN) and use it to explore the needs of young children under 6 years of age in Tehran. This article reports on the qualitative part of the study, which involved conducting semistructured interviews.

In June 2019, within the framework of a joint cooperation between the Ministry of Health and UNICEF to implement the global initiative ‘Child Friendly Cities’ in Iran, the first meeting of the National Child-Friendly City Coordination Committee was held at the Ministry of Interior of Iran. 12 cities, including Tehran, were selected to implement the Child-Friendly Cities framework. The population of children under the age of 6 in Tehran was 1 330 646 according to 2015 census. 26 Currently, Tehran and other selected cities are in the neighbourhood assessment phase. 27 For the purpose of this research, we define a child as someone who is less than 6 years old. Hence, the objective of this research was to investigate the viewpoints and encounters of specialists, municipal employees and mothers with children aged 6 and below in Tehran with regard to the creation of a child health-oriented community for this particular age group.

Study design and setting

This was a qualitative study. Qualitative data were analysed using the directed content analysis method, which allowed new themes to emerge from the data. 28 This research used the Child-Friendly Cities framework developed by UNICEF.

Study participants and sampling

Variation purposeful sampling techniques were used to recruit participants from the three main groups of experts, mothers and city council staff. Individuals with expertise and experience in infant psychology, urban planning and architecture, and health education and promotion comprised the experts. Tehran’s Child-Friendly Cities Secretariat employed the municipal personnel. Mothers were included in the research as representatives of children under the age of 6 as they are the person most responsible for their care. Experts were interviewed at their workplace, while mothers who accompanied their children in parks were interviewed in the park.

Before starting the interview sessions, an official letter from the university was provided to the participants. The participants (mothers and experts) were provided with an explanation of the study and consent was obtained for participation (mothers and experts) and for conducting and recording the interviews.

Data collection tools and techniques

Data were collected via semistructured, indepth interviews. The interviews were conducted face-to-face using an interview topic guide ( online supplemental file 1 ). The principal investigator (PA), who had previous experience in qualitative data collection, interviewing and interview calibration, and who was unknown to the participants, conducted the interviews.

Supplemental material

The subject of the interview was the opinion of the interviewees on the current state of their neighbourhoods and the changes that should be made to make neighbourhoods suitable and health-friendly to children under 6 years of age. The duration of the interview ranged from 20 to 60 min. Data saturation was verified by the research team after conducting 31 interviews. Nevertheless, three additional interviews were conducted to provide additional confirmation. In the end, no new data were discovered and the interview procedure was terminated. The study did not include any withdrawals from the participants. To improve the quality of data collection, the interviews were audio-recorded for verbatim transcription, with the consent of the participants. They were assured that their voices would be removed after the interviews. The interviewer (PA) recorded the participants’ perspectives, thoughts and feelings about the research topic and the interviews in a notebook. The researcher took notes during data collection when necessary ( online supplemental file 2 ).

Data analysis

The transcribed interviews were coded by the researcher (PA) shortly after each interview using directed content analysis techniques. UNICEF’s Child Friendly Cities framework, with six main themes, guided the data analysis. MAXQDA V.2020 software was used to facilitate the data analysis. The coding process was carried out according to Graneheim and Lundman. 29 First, the transcriptions were examined word by word. The texts were then divided into units of meaning and were coded. Codes denoting a single theme were placed in a category based on a continuous comparison of similarities, differences and appropriateness. The next step classified the categories and subcategories. The summary categories and the central concept of each category were then identified. Finally, the concepts were reviewed based on the description of the internal themes considering the whole data. 29

Graneheim and Lundman’s 29 indices, including credibility, dependability, confirmability and transferability, were applied to check for accuracy and the robustness of the data analysis. Various strategies were used to increase the credibility of the data, such as member checks, long-term engagement with the subject and data review by the research team. The reliability of the findings was verified through implementation of measures such as the coding and decoding method during data analysis and documentation of all stages from inception to conclusion. Methods such as sampling with maximum diversity, detailed descriptions of the findings and the participants, sampling, the time and place of data collection, and observer control were implemented. The transferability of the findings was increased by recording all steps and the decisions made during the study and by reviewing and confirming the data analysis report by colleagues .

Ethical consideration

Participants were provided with key information such as the purpose of study, the interview method, confidentiality of the information, the right to participate or withdraw and the procedure for storing the audio file. Furthermore, verbal informed consent to participate and audio-record the interviews was obtained from the participants (mothers and experts) at the beginning of each interview. A code was assigned to each participant to protect anonymity. In addition, the researcher kept the audio files of the interviews and were not shared with anyone.

Patient and public involvement

Patients and/or the public were not involved in the design, or conduct, or reporting or dissemination plans of this research.

The characteristics of the study participants are summarised in tables 1 and 2 .

• View inline

Demographic characteristics of the mothers

Demographic characteristics of the experts

Dimensions and characteristics of the U6CHFN

Analysing the qualitative interview data, the results were categorised into 6 dimensions, 21 subdimensions and 80 characteristics. The six main dimensions included neighbourhood green space, neighbourhood cultural centre, neighbourhood health centre, services in the neighbourhood, transport in the neighbourhood and security in the neighbourhood ( table 3 ).

Emerging key dimensions and subdimensions of child health-friendly neighbourhoods (from the data)

These six dimensions are explained in detail in the following sections:

Neighbourhood green space

Many participants believed that the presence of a park is one of the important dimensions of a child-friendly city, provided that the park has ‘playground space for children’, ‘ergonomic play equipment’, ‘attractive play equipment’, ‘creativity, art and culture in the park’ and ‘comfortable space’. They argued that the quantity and quality of these features and their use were unsatisfactory. For example, one of the mothers stated:

Part of the flooring (of the playground) is torn off, and my child once got his foot stuck on it, and fell down. They are all separated. The cracks are open. It is necessary that the floor of the park is healthy and soft. (Mother, Interview 8)

Another participant mentioned that:

Currently, playground equipment in many parks is not suitable for children of this age in terms of height. The size of children is small, and it is necessary for them to have play equipment and chairs and benches that are suitable for the size of children. (Child psychologist, Interview 22)

Neighbourhood cultural centre

Some interviewees believed that there was a need for community centres to offer educational and cultural activities that were age-appropriate, attractive and varied. Many educational topics were suggested, such as ‘art and creativity’, ‘self-care’, ‘life skills’ and ‘citizenship skills’, both from educational and recreational perspectives. Many families may have limited experience or skills in educating their children, and as a result some argued that self-care education should include protection from sexual assault. Consequently, professionals can significantly contribute to this effort. For example, one of the mothers discussed this point as follows:

One of the educational points that I think my child needs now is that I don’t know what to tell him about sexual issues, and I think it is necessary that a child educator who has studied and has experience in this field should teach him/her how to protect himself/herself from sexual abuse, education based on what they need to know, to be told so that they receive knowledge appropriate for their age. (Mother, Interview 14)

Another participant explained:

My son reached the age of middle school, and he is afraid to go to the neighborhood and to the city. He is afraid of riding the bus alone, if I was a child, I would have gone to school alone … I think it is necessary to teach them about the urban lifestyle and about commuting within the neighborhoods. From preschool onwards, it is necessary to learn how to cross the street, how to cross the crosswalk and traffic lights, learn how to get on and off the bus, so that later they can use it, they don’t have to be afraid anymore. (Urban planning and architecture expert, Interview 23)

Neighbourhood health centre

From the participants’ point of view, the neighbourhood health centre should provide ‘child health services’, ‘health education services’ and ‘child care’. Many participants were not satisfied with the child health services and health education. For example, one of the mothers said:

Health centers used to provide much better services for mothers and children, and education during pregnancy and about breastfeeding, but now they just fill in forms and do not provide any education. The health center should be a place to teach mothers how to breastfeed. how to sleep and bath the child, and to give advice and instructions to the father. (Mother, Interview 13)

Services in the neighbourhood

One of the characteristics of a child-friendly neighbourhood according to the participants was adequate access to children’s welfare services in the neighbourhood. They explained that, unfortunately, children’s welfare services are usually neglected and therefore need to be addressed. For example, one participant explained:

My child gets very upset and nervous when he comes to the shopping mall, because it is crowded, and he has to walk in the crowd. If there are baby strollers in all shopping malls, mothers can easily take the child with them, and the children even enjoy it. (Mother, Interview 13)

Several participants had suggestions for the creation of "children's entertainment and sports centers" in the neighborhood. These centres can be, for example, sports centres that have sports and swimming pools for children. For example, one participant pointed out:

Every neighborhood can have grass playgrounds for children, as well as indoor sports centers, for example, a children’s pool and they can play. (Urban planning and architecture expert, Interview 16)

Transport in the neighbourhood

The participants argued that the sidewalks, streets and generally the environment for transporting children in the neighbourhood should be ‘easy to use’, ‘attractive’, ‘educational’, ‘safe’ and ‘ergonomic’. One of the participating mothers said:

The most important problem I have is that I can’t take my baby out with the stroller, the sidewalks are full of stairs and up and down, you have to constantly lift and move the stroller with its weight, and when you take it on the street, cars honk and drive by at high speed. (Mother, Interview 13)

Another participant explained about the safety of shrubs or trees planted on the streets:

It is better to plant tall trees in front of houses or low annual flowers and plants. We have had many cases where a child who was crossing the sidewalk, the blade of these shrubs hit him in the face and injured him, or their leaf has got stuck in his clothes. (Public health specialist, Interview 22)

One participant highlighted the need for traffic signs to be ergonomically designed for children under the age of 6:

It is necessary that the green and safety light for the child and appropriate to the height of the child and the environment be considered from the origin which is the home of each child, to the destination where is a space designed for children, so that both drivers are required to obey the traffic law, and children feel safe from the presence of cars on the street to travel. (Urban planning and architecture expert, Interview 16)

Some participants argued that the frequency of the child's presence in the neighborhood could be increased by improving the "attractiveness of the child's walking route," for example by incorporating vegetation and painted walls. Some other participants argued that the presence of ‘child friendly urban furniture’ in the neighbourhood space that is based on the height and size of children under 6 years of age can facilitate the child’s passage in the neighbourhood as it allows them to rest when needed.

Security in the neighbourhood

Neighbourhood safety for children is an important dimension of a child health-friendly city that was frequently discussed by the participants. They discussed children's security from strangers and addicts in all areas of the neighborhood, especially in "parks," which are the main neighborhood spaces used by young children. For example, a participant explained:

Security is very important in educational spaces. In these environments, it is necessary that CCTV [closed-circuit television] is controlled and there is surveillance. This security leads to the reduction of anxiety and stress in the city and neighborhood, the space that has a camera and is monitored creates a sense of security for the presence of the child, and the family can trust that place. (Specialist in urban planning and architecture, Interview 27)

A mother stated:

I can’t allow my daughter to play alone even in front of our door. I have often seen drug addicts going to the street and exchanging drugs. (Mother, Interview 10)

The health of urban populations is directly affected by the ability of the urban environment to support the creation of safe neighbourhoods and healthy behaviours and habits during childhood. 30 Living in cities can expose children to risks and challenges, such as noise and air pollution, sedentary lifestyles, traffic hazards, crime, social isolation and disconnection from nature. However, it can also provide health-promoting resources and opportunities, such as education and healthcare. 30 31

There is a limited number of studies that have documented the effect of neighbourhoods on children’s health. 32 33 For instance, Dondi et al ’s 34 study demonstrated that exposure to toxic air pollutants during infancy and childhood can affect the growth and development of the respiratory system, nerves, glands and immune system. Furthermore, it has the potential to elevate the likelihood of developing cancer in the future. It is estimated that the 1.7 million fatalities among children under the age of 5 each year are a result of disregard for the health and environmental aspects in cities. 31 32 Therefore, cities should provide a platform to protect children’s health based on their needs. 30

One of the challenges to implementing the concept of a child-friendly city is the difficulty in accurately identifying the needs of children that the city can provide for, given the large scale of the city and with inadequate research. 35 We focused on the smaller scale of the neighbourhoods and consulted with stakeholders, particularly mothers, to gain a more comprehensive understanding of the needs of children living in urban areas. This approach can be considered a strength of our study. The present study investigated the health needs of children that can be addressed by the neighbourhood. The perspectives of three key stakeholders, namely experts, city council staff and mothers, were explored. A qualitative approach was employed to gain a deeper understanding of their experiences and perspectives. Previous studies in this field have mostly taken a quantitative approach. 36–40

The importance of the local environment in the health of children was underscored by the participants in this study, as indicated in the Results section. The participants suggested specific characteristics for various locations in the community that can support the preservation and promotion of health of children aged 0–6 years. Through data analysis, six dimensions were identified for a child-friendly neighbourhood, which closely resembled the seven dimensions of UNICEF’s Child Friendly Cities concept, 41 as presented in table 4 . However, a total of 80 child-friendly neighbourhood characteristics were developed, 23 of which were new items compared to the current UNICEF child-friendly city checklist. Some of these newly identified characteristics have been reported by other researchers 27 38 42 . However, there were few characteristics that were not reported in other studies, such as access to mother and child rooms in parks, sexual self-care education in child care facilities and child education centers, mental health screening for children, and child-friendly urban furniture.

Comparison among the identified domains of a child health-friendly neighbourhood for children under 6 years of age (from the data) and the UNICEF’s Child Friendly Cities framework 42

Security was a common concern among the participants, particularly among mothers. The discussion on neighbourhood security included parks, educational spaces and the surrounding area. Security has both objective and subjective dimensions. Objectively, it refers to the absence of danger and threats to a child’s life. 43 Subjective security, on the other hand, refers to an individual’s (here children or mothers) feeling of calmness and intimacy with a space, regardless of whether there is an actual hazard present. 44 According to other studies, children’s presence in a neighbourhood may be impacted by the perceived or actual security issues. 44–46 It is crucial to remember that in order to preserve objectivity, subjective assessments must be identified as such. 45 Therefore, it is essential to address the dark and hidden corners of the neighbourhood, provide proper lighting, install security cameras and if possible have security guards in the areas. 46

Although this study provided rich perspectives on the characteristics of a child health-friendly neighbourhood from three groups of stakeholders, due to lack of time we did not obtain the opinions of fathers and children. It should also be noted that, first, due to the qualitative nature of the study, only the typology of perspectives was provided rather than the prevalence of perspectives among the participants interviewed. In other words, the results cannot indicate what the main problems are that need to be addressed in the neighbourhoods. Also, the needs of children in each neighbourhood may be different. Quantitative research is therefore needed to identify priorities for future action. Due to the complexity, multidimensionality and subjectivity of the dimensions of health in the neighbourhood, we used a qualitative method. However, in order to assess and identify the existing needs of a neighbourhood, quantitative research is needed to evaluate these components so that we can identify the existing gaps for appropriate interventions.

Conclusions

The results of this study provide a deeper understanding of the health needs of children under 6 years of age that need to be met in order to provide a child-healthy neighbourhood. Thus, it contributes to further dialogue, research and intervention design to make neighbourhoods a health-supportive environment as recommended by the Ottawa Charter for Health Promotion. We suggest that in designing child-friendly cities and neighbourhoods, it is critical to take a comprehensive approach to the health needs of children of all ages, especially those aged 0–6. This involves providing secure and healthy settings and services in the neighbourhoods, as well as ensuring that natural and manmade infrastructure and services are responsive to children’s actual health requirements. To this end, this study highlighted the importance of access to neighbourhood green space, health services, transportation and security.

Ethics statements

Patient consent for publication.

Not required.

Ethics approval

The study was approved by the Ethics Committee at Shahid Beheshti University of Medical Sciences (IR.SBMU.PHNS.REC.1401.002).

Acknowledgments

This study is part of a PhD dissertation conducted at Shahid Beheshti Medical University. The research team appreciates all the participants: specialists and families. The authors would also like to thank Dr Panthea Hakimian, member of the Faculty of Architecture and Urban Planning, Shahid Beheshti University of Medical Sciences, for providing valuable comments. The authors are truly thankful to the reviewers whose comments and suggestions definitely made a remarkable improvement to the quality of this research paper.

• ↵ Public Health at the Crossroads: Achievements and Prospects. 2nd Ed . Cambridge : Cambridge University Press , 2004 : 303 .
• Phillipson C ,
• Williams S ,
• Kalache A ,
• Kickbusch I
• Greenbaum C
• Dehghan Mehrjerdi E
• Prihantini P ,
• Kurniawati W
• Saridar Masri S
• Agarwal MK ,
• Wilhelmsen T ,
• Roll-Hansen H , et al
• Graneheim UH ,
• Jin X , et al
• Zhao J , et al
• Mvoundza Ndjindji O ,
• Minto’o Rogombe S ,
• Mougola Bissiengou P , et al
• Carbone C ,
• Manieri E , et al
• Bechtel RB ,
• churchman A
• Kharazmi O ,
• Zarghani H ,
• Johari L , et al
• Carroll P ,
• Kearns R , et al
• Moayedfar S ,
• ↵ Child Friendly Cities Initiative (CFCI): Self-Assessment Tool for Local Authorities , Available : https://www-unicef-org.translate.goog/greece/en/documents/child-friendly-cities-initiative-cfci-self-assessment-tool-local-authorities [Accessed 19 Oct 2022 ].
• Building Better Cities with Young Children & Families
• Lucchesi ST ,
• Larranaga AM ,
• Ochoa JAA , et al
• Daneshvar Anbaran F ,
• Alizadeh K ,

Contributors All authors conceived and designed the study. PA was responsible for the interviews. PA and NK-M were responsible for data analysis. PA and NK-M were responsible for the initial draft and editing of the manuscript and approved the manuscript for submission. NK-M and AR revised the manuscript. All authors read and approved the final manuscript. NK-M is responsible for the overall content as the guarantor.

Funding This study was funded by Shahid Beheshti University of Medical Sciences.

Competing interests None declared.

Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.

Provenance and peer review Not commissioned; externally peer reviewed.

Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.

Read the full text or download the PDF: