sample qualitative research articles

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, generate accurate citations for free.

• Knowledge Base

Methodology

• What Is Qualitative Research? | Methods & Examples

What Is Qualitative Research? | Methods & Examples

Published on June 19, 2020 by Pritha Bhandari . Revised on September 5, 2024.

Qualitative research involves collecting and analyzing non-numerical data (e.g., text, video, or audio) to understand concepts, opinions, or experiences. It can be used to gather in-depth insights into a problem or generate new ideas for research.

Qualitative research is the opposite of quantitative research , which involves collecting and analyzing numerical data for statistical analysis.

Qualitative research is commonly used in the humanities and social sciences, in subjects such as anthropology, sociology, education, health sciences, history, etc.

• How does social media shape body image in teenagers?
• How do children and adults interpret healthy eating in the UK?
• What factors influence employee retention in a large organization?
• How is anxiety experienced around the world?
• How can teachers integrate social issues into science curriculums?

Table of contents

Approaches to qualitative research, qualitative research methods, qualitative data analysis, advantages of qualitative research, disadvantages of qualitative research, other interesting articles, frequently asked questions about qualitative research.

Qualitative research is used to understand how people experience the world. While there are many approaches to qualitative research, they tend to be flexible and focus on retaining rich meaning when interpreting data.

Common approaches include grounded theory, ethnography , action research , phenomenological research, and narrative research. They share some similarities, but emphasize different aims and perspectives.

Note that qualitative research is at risk for certain research biases including the Hawthorne effect , observer bias , recall bias , and social desirability bias . While not always totally avoidable, awareness of potential biases as you collect and analyze your data can prevent them from impacting your work too much.

Receive feedback on language, structure, and formatting

Professional editors proofread and edit your paper by focusing on:

• Academic style
• Vague sentences
• Style consistency

See an example

Each of the research approaches involve using one or more data collection methods . These are some of the most common qualitative methods:

• Observations: recording what you have seen, heard, or encountered in detailed field notes.
• Interviews:  personally asking people questions in one-on-one conversations.
• Focus groups: asking questions and generating discussion among a group of people.
• Surveys : distributing questionnaires with open-ended questions.
• Secondary research: collecting existing data in the form of texts, images, audio or video recordings, etc.
• You take field notes with observations and reflect on your own experiences of the company culture.
• You distribute open-ended surveys to employees across all the company’s offices by email to find out if the culture varies across locations.
• You conduct in-depth interviews with employees in your office to learn about their experiences and perspectives in greater detail.

Qualitative researchers often consider themselves “instruments” in research because all observations, interpretations and analyses are filtered through their own personal lens.

For this reason, when writing up your methodology for qualitative research, it’s important to reflect on your approach and to thoroughly explain the choices you made in collecting and analyzing the data.

Qualitative data can take the form of texts, photos, videos and audio. For example, you might be working with interview transcripts, survey responses, fieldnotes, or recordings from natural settings.

Most types of qualitative data analysis share the same five steps:

• Prepare and organize your data. This may mean transcribing interviews or typing up fieldnotes.
• Review and explore your data. Examine the data for patterns or repeated ideas that emerge.
• Develop a data coding system. Based on your initial ideas, establish a set of codes that you can apply to categorize your data.
• Assign codes to the data. For example, in qualitative survey analysis, this may mean going through each participant’s responses and tagging them with codes in a spreadsheet. As you go through your data, you can create new codes to add to your system if necessary.
• Identify recurring themes. Link codes together into cohesive, overarching themes.

There are several specific approaches to analyzing qualitative data. Although these methods share similar processes, they emphasize different concepts.

Qualitative research often tries to preserve the voice and perspective of participants and can be adjusted as new research questions arise. Qualitative research is good for:

• Flexibility

The data collection and analysis process can be adapted as new ideas or patterns emerge. They are not rigidly decided beforehand.

• Natural settings

Data collection occurs in real-world contexts or in naturalistic ways.

• Meaningful insights

Detailed descriptions of people’s experiences, feelings and perceptions can be used in designing, testing or improving systems or products.

• Generation of new ideas

Open-ended responses mean that researchers can uncover novel problems or opportunities that they wouldn’t have thought of otherwise.

Researchers must consider practical and theoretical limitations in analyzing and interpreting their data. Qualitative research suffers from:

• Unreliability

The real-world setting often makes qualitative research unreliable because of uncontrolled factors that affect the data.

• Subjectivity

Due to the researcher’s primary role in analyzing and interpreting data, qualitative research cannot be replicated . The researcher decides what is important and what is irrelevant in data analysis, so interpretations of the same data can vary greatly.

• Limited generalizability

Small samples are often used to gather detailed data about specific contexts. Despite rigorous analysis procedures, it is difficult to draw generalizable conclusions because the data may be biased and unrepresentative of the wider population .

• Labor-intensive

Although software can be used to manage and record large amounts of text, data analysis often has to be checked or performed manually.

If you want to know more about statistics , methodology , or research bias , make sure to check out some of our other articles with explanations and examples.

• Chi square goodness of fit test
• Degrees of freedom
• Null hypothesis
• Discourse analysis
• Control groups
• Mixed methods research
• Non-probability sampling
• Quantitative research
• Inclusion and exclusion criteria

Research bias

• Rosenthal effect
• Implicit bias
• Cognitive bias
• Selection bias
• Negativity bias
• Status quo bias

Quantitative research deals with numbers and statistics, while qualitative research deals with words and meanings.

Quantitative methods allow you to systematically measure variables and test hypotheses . Qualitative methods allow you to explore concepts and experiences in more detail.

There are five common approaches to qualitative research :

• Grounded theory involves collecting data in order to develop new theories.
• Ethnography involves immersing yourself in a group or organization to understand its culture.
• Narrative research involves interpreting stories to understand how people make sense of their experiences and perceptions.
• Phenomenological research involves investigating phenomena through people’s lived experiences.
• Action research links theory and practice in several cycles to drive innovative changes.

Data collection is the systematic process by which observations or measurements are gathered in research. It is used in many different contexts by academics, governments, businesses, and other organizations.

There are various approaches to qualitative data analysis , but they all share five steps in common:

• Prepare and organize your data.
• Review and explore your data.
• Develop a data coding system.
• Assign codes to the data.
• Identify recurring themes.

The specifics of each step depend on the focus of the analysis. Some common approaches include textual analysis , thematic analysis , and discourse analysis .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the “Cite this Scribbr article” button to automatically add the citation to our free Citation Generator.

Bhandari, P. (2024, September 05). What Is Qualitative Research? | Methods & Examples. Scribbr. Retrieved September 23, 2024, from https://www.scribbr.com/methodology/qualitative-research/

Is this article helpful?

Pritha Bhandari

Other students also liked, qualitative vs. quantitative research | differences, examples & methods, how to do thematic analysis | step-by-step guide & examples, "i thought ai proofreading was useless but..".

I've been using Scribbr for years now and I know it's a service that won't disappoint. It does a good job spotting mistakes”

Criteria for Good Qualitative Research: A Comprehensive Review

• Regular Article
• Open access
• Published: 18 September 2021
• Volume 31 , pages 679–689, ( 2022 )

Cite this article

You have full access to this open access article

• Drishti Yadav   ORCID: orcid.org/0000-0002-2974-0323 1  

104k Accesses

49 Citations

70 Altmetric

Explore all metrics

This review aims to synthesize a published set of evaluative criteria for good qualitative research. The aim is to shed light on existing standards for assessing the rigor of qualitative research encompassing a range of epistemological and ontological standpoints. Using a systematic search strategy, published journal articles that deliberate criteria for rigorous research were identified. Then, references of relevant articles were surveyed to find noteworthy, distinct, and well-defined pointers to good qualitative research. This review presents an investigative assessment of the pivotal features in qualitative research that can permit the readers to pass judgment on its quality and to condemn it as good research when objectively and adequately utilized. Overall, this review underlines the crux of qualitative research and accentuates the necessity to evaluate such research by the very tenets of its being. It also offers some prospects and recommendations to improve the quality of qualitative research. Based on the findings of this review, it is concluded that quality criteria are the aftereffect of socio-institutional procedures and existing paradigmatic conducts. Owing to the paradigmatic diversity of qualitative research, a single and specific set of quality criteria is neither feasible nor anticipated. Since qualitative research is not a cohesive discipline, researchers need to educate and familiarize themselves with applicable norms and decisive factors to evaluate qualitative research from within its theoretical and methodological framework of origin.

Similar content being viewed by others

Good Qualitative Research: Opening up the Debate

Beyond qualitative/quantitative structuralism: the positivist qualitative research and the paradigmatic disclaimer.

What is Qualitative in Research

Avoid common mistakes on your manuscript.

Introduction

“… It is important to regularly dialogue about what makes for good qualitative research” (Tracy, 2010 , p. 837)

To decide what represents good qualitative research is highly debatable. There are numerous methods that are contained within qualitative research and that are established on diverse philosophical perspectives. Bryman et al., ( 2008 , p. 262) suggest that “It is widely assumed that whereas quality criteria for quantitative research are well‐known and widely agreed, this is not the case for qualitative research.” Hence, the question “how to evaluate the quality of qualitative research” has been continuously debated. There are many areas of science and technology wherein these debates on the assessment of qualitative research have taken place. Examples include various areas of psychology: general psychology (Madill et al., 2000 ); counseling psychology (Morrow, 2005 ); and clinical psychology (Barker & Pistrang, 2005 ), and other disciplines of social sciences: social policy (Bryman et al., 2008 ); health research (Sparkes, 2001 ); business and management research (Johnson et al., 2006 ); information systems (Klein & Myers, 1999 ); and environmental studies (Reid & Gough, 2000 ). In the literature, these debates are enthused by the impression that the blanket application of criteria for good qualitative research developed around the positivist paradigm is improper. Such debates are based on the wide range of philosophical backgrounds within which qualitative research is conducted (e.g., Sandberg, 2000 ; Schwandt, 1996 ). The existence of methodological diversity led to the formulation of different sets of criteria applicable to qualitative research.

Among qualitative researchers, the dilemma of governing the measures to assess the quality of research is not a new phenomenon, especially when the virtuous triad of objectivity, reliability, and validity (Spencer et al., 2004 ) are not adequate. Occasionally, the criteria of quantitative research are used to evaluate qualitative research (Cohen & Crabtree, 2008 ; Lather, 2004 ). Indeed, Howe ( 2004 ) claims that the prevailing paradigm in educational research is scientifically based experimental research. Hypotheses and conjectures about the preeminence of quantitative research can weaken the worth and usefulness of qualitative research by neglecting the prominence of harmonizing match for purpose on research paradigm, the epistemological stance of the researcher, and the choice of methodology. Researchers have been reprimanded concerning this in “paradigmatic controversies, contradictions, and emerging confluences” (Lincoln & Guba, 2000 ).

In general, qualitative research tends to come from a very different paradigmatic stance and intrinsically demands distinctive and out-of-the-ordinary criteria for evaluating good research and varieties of research contributions that can be made. This review attempts to present a series of evaluative criteria for qualitative researchers, arguing that their choice of criteria needs to be compatible with the unique nature of the research in question (its methodology, aims, and assumptions). This review aims to assist researchers in identifying some of the indispensable features or markers of high-quality qualitative research. In a nutshell, the purpose of this systematic literature review is to analyze the existing knowledge on high-quality qualitative research and to verify the existence of research studies dealing with the critical assessment of qualitative research based on the concept of diverse paradigmatic stances. Contrary to the existing reviews, this review also suggests some critical directions to follow to improve the quality of qualitative research in different epistemological and ontological perspectives. This review is also intended to provide guidelines for the acceleration of future developments and dialogues among qualitative researchers in the context of assessing the qualitative research.

The rest of this review article is structured in the following fashion: Sect.  Methods describes the method followed for performing this review. Section Criteria for Evaluating Qualitative Studies provides a comprehensive description of the criteria for evaluating qualitative studies. This section is followed by a summary of the strategies to improve the quality of qualitative research in Sect.  Improving Quality: Strategies . Section  How to Assess the Quality of the Research Findings? provides details on how to assess the quality of the research findings. After that, some of the quality checklists (as tools to evaluate quality) are discussed in Sect.  Quality Checklists: Tools for Assessing the Quality . At last, the review ends with the concluding remarks presented in Sect.  Conclusions, Future Directions and Outlook . Some prospects in qualitative research for enhancing its quality and usefulness in the social and techno-scientific research community are also presented in Sect.  Conclusions, Future Directions and Outlook .

For this review, a comprehensive literature search was performed from many databases using generic search terms such as Qualitative Research , Criteria , etc . The following databases were chosen for the literature search based on the high number of results: IEEE Explore, ScienceDirect, PubMed, Google Scholar, and Web of Science. The following keywords (and their combinations using Boolean connectives OR/AND) were adopted for the literature search: qualitative research, criteria, quality, assessment, and validity. The synonyms for these keywords were collected and arranged in a logical structure (see Table 1 ). All publications in journals and conference proceedings later than 1950 till 2021 were considered for the search. Other articles extracted from the references of the papers identified in the electronic search were also included. A large number of publications on qualitative research were retrieved during the initial screening. Hence, to include the searches with the main focus on criteria for good qualitative research, an inclusion criterion was utilized in the search string.

From the selected databases, the search retrieved a total of 765 publications. Then, the duplicate records were removed. After that, based on the title and abstract, the remaining 426 publications were screened for their relevance by using the following inclusion and exclusion criteria (see Table 2 ). Publications focusing on evaluation criteria for good qualitative research were included, whereas those works which delivered theoretical concepts on qualitative research were excluded. Based on the screening and eligibility, 45 research articles were identified that offered explicit criteria for evaluating the quality of qualitative research and were found to be relevant to this review.

Figure  1 illustrates the complete review process in the form of PRISMA flow diagram. PRISMA, i.e., “preferred reporting items for systematic reviews and meta-analyses” is employed in systematic reviews to refine the quality of reporting.

PRISMA flow diagram illustrating the search and inclusion process. N represents the number of records

Criteria for Evaluating Qualitative Studies

Fundamental criteria: general research quality.

Various researchers have put forward criteria for evaluating qualitative research, which have been summarized in Table 3 . Also, the criteria outlined in Table 4 effectively deliver the various approaches to evaluate and assess the quality of qualitative work. The entries in Table 4 are based on Tracy’s “Eight big‐tent criteria for excellent qualitative research” (Tracy, 2010 ). Tracy argues that high-quality qualitative work should formulate criteria focusing on the worthiness, relevance, timeliness, significance, morality, and practicality of the research topic, and the ethical stance of the research itself. Researchers have also suggested a series of questions as guiding principles to assess the quality of a qualitative study (Mays & Pope, 2020 ). Nassaji ( 2020 ) argues that good qualitative research should be robust, well informed, and thoroughly documented.

Qualitative Research: Interpretive Paradigms

All qualitative researchers follow highly abstract principles which bring together beliefs about ontology, epistemology, and methodology. These beliefs govern how the researcher perceives and acts. The net, which encompasses the researcher’s epistemological, ontological, and methodological premises, is referred to as a paradigm, or an interpretive structure, a “Basic set of beliefs that guides action” (Guba, 1990 ). Four major interpretive paradigms structure the qualitative research: positivist and postpositivist, constructivist interpretive, critical (Marxist, emancipatory), and feminist poststructural. The complexity of these four abstract paradigms increases at the level of concrete, specific interpretive communities. Table 5 presents these paradigms and their assumptions, including their criteria for evaluating research, and the typical form that an interpretive or theoretical statement assumes in each paradigm. Moreover, for evaluating qualitative research, quantitative conceptualizations of reliability and validity are proven to be incompatible (Horsburgh, 2003 ). In addition, a series of questions have been put forward in the literature to assist a reviewer (who is proficient in qualitative methods) for meticulous assessment and endorsement of qualitative research (Morse, 2003 ). Hammersley ( 2007 ) also suggests that guiding principles for qualitative research are advantageous, but methodological pluralism should not be simply acknowledged for all qualitative approaches. Seale ( 1999 ) also points out the significance of methodological cognizance in research studies.

Table 5 reflects that criteria for assessing the quality of qualitative research are the aftermath of socio-institutional practices and existing paradigmatic standpoints. Owing to the paradigmatic diversity of qualitative research, a single set of quality criteria is neither possible nor desirable. Hence, the researchers must be reflexive about the criteria they use in the various roles they play within their research community.

Improving Quality: Strategies

Another critical question is “How can the qualitative researchers ensure that the abovementioned quality criteria can be met?” Lincoln and Guba ( 1986 ) delineated several strategies to intensify each criteria of trustworthiness. Other researchers (Merriam & Tisdell, 2016 ; Shenton, 2004 ) also presented such strategies. A brief description of these strategies is shown in Table 6 .

It is worth mentioning that generalizability is also an integral part of qualitative research (Hays & McKibben, 2021 ). In general, the guiding principle pertaining to generalizability speaks about inducing and comprehending knowledge to synthesize interpretive components of an underlying context. Table 7 summarizes the main metasynthesis steps required to ascertain generalizability in qualitative research.

Figure  2 reflects the crucial components of a conceptual framework and their contribution to decisions regarding research design, implementation, and applications of results to future thinking, study, and practice (Johnson et al., 2020 ). The synergy and interrelationship of these components signifies their role to different stances of a qualitative research study.

Essential elements of a conceptual framework

In a nutshell, to assess the rationale of a study, its conceptual framework and research question(s), quality criteria must take account of the following: lucid context for the problem statement in the introduction; well-articulated research problems and questions; precise conceptual framework; distinct research purpose; and clear presentation and investigation of the paradigms. These criteria would expedite the quality of qualitative research.

How to Assess the Quality of the Research Findings?

The inclusion of quotes or similar research data enhances the confirmability in the write-up of the findings. The use of expressions (for instance, “80% of all respondents agreed that” or “only one of the interviewees mentioned that”) may also quantify qualitative findings (Stenfors et al., 2020 ). On the other hand, the persuasive reason for “why this may not help in intensifying the research” has also been provided (Monrouxe & Rees, 2020 ). Further, the Discussion and Conclusion sections of an article also prove robust markers of high-quality qualitative research, as elucidated in Table 8 .

Quality Checklists: Tools for Assessing the Quality

Numerous checklists are available to speed up the assessment of the quality of qualitative research. However, if used uncritically and recklessly concerning the research context, these checklists may be counterproductive. I recommend that such lists and guiding principles may assist in pinpointing the markers of high-quality qualitative research. However, considering enormous variations in the authors’ theoretical and philosophical contexts, I would emphasize that high dependability on such checklists may say little about whether the findings can be applied in your setting. A combination of such checklists might be appropriate for novice researchers. Some of these checklists are listed below:

The most commonly used framework is Consolidated Criteria for Reporting Qualitative Research (COREQ) (Tong et al., 2007 ). This framework is recommended by some journals to be followed by the authors during article submission.

Standards for Reporting Qualitative Research (SRQR) is another checklist that has been created particularly for medical education (O’Brien et al., 2014 ).

Also, Tracy ( 2010 ) and Critical Appraisal Skills Programme (CASP, 2021 ) offer criteria for qualitative research relevant across methods and approaches.

Further, researchers have also outlined different criteria as hallmarks of high-quality qualitative research. For instance, the “Road Trip Checklist” (Epp & Otnes, 2021 ) provides a quick reference to specific questions to address different elements of high-quality qualitative research.

Conclusions, Future Directions, and Outlook

This work presents a broad review of the criteria for good qualitative research. In addition, this article presents an exploratory analysis of the essential elements in qualitative research that can enable the readers of qualitative work to judge it as good research when objectively and adequately utilized. In this review, some of the essential markers that indicate high-quality qualitative research have been highlighted. I scope them narrowly to achieve rigor in qualitative research and note that they do not completely cover the broader considerations necessary for high-quality research. This review points out that a universal and versatile one-size-fits-all guideline for evaluating the quality of qualitative research does not exist. In other words, this review also emphasizes the non-existence of a set of common guidelines among qualitative researchers. In unison, this review reinforces that each qualitative approach should be treated uniquely on account of its own distinctive features for different epistemological and disciplinary positions. Owing to the sensitivity of the worth of qualitative research towards the specific context and the type of paradigmatic stance, researchers should themselves analyze what approaches can be and must be tailored to ensemble the distinct characteristics of the phenomenon under investigation. Although this article does not assert to put forward a magic bullet and to provide a one-stop solution for dealing with dilemmas about how, why, or whether to evaluate the “goodness” of qualitative research, it offers a platform to assist the researchers in improving their qualitative studies. This work provides an assembly of concerns to reflect on, a series of questions to ask, and multiple sets of criteria to look at, when attempting to determine the quality of qualitative research. Overall, this review underlines the crux of qualitative research and accentuates the need to evaluate such research by the very tenets of its being. Bringing together the vital arguments and delineating the requirements that good qualitative research should satisfy, this review strives to equip the researchers as well as reviewers to make well-versed judgment about the worth and significance of the qualitative research under scrutiny. In a nutshell, a comprehensive portrayal of the research process (from the context of research to the research objectives, research questions and design, speculative foundations, and from approaches of collecting data to analyzing the results, to deriving inferences) frequently proliferates the quality of a qualitative research.

Prospects : A Road Ahead for Qualitative Research

Irrefutably, qualitative research is a vivacious and evolving discipline wherein different epistemological and disciplinary positions have their own characteristics and importance. In addition, not surprisingly, owing to the sprouting and varied features of qualitative research, no consensus has been pulled off till date. Researchers have reflected various concerns and proposed several recommendations for editors and reviewers on conducting reviews of critical qualitative research (Levitt et al., 2021 ; McGinley et al., 2021 ). Following are some prospects and a few recommendations put forward towards the maturation of qualitative research and its quality evaluation:

In general, most of the manuscript and grant reviewers are not qualitative experts. Hence, it is more likely that they would prefer to adopt a broad set of criteria. However, researchers and reviewers need to keep in mind that it is inappropriate to utilize the same approaches and conducts among all qualitative research. Therefore, future work needs to focus on educating researchers and reviewers about the criteria to evaluate qualitative research from within the suitable theoretical and methodological context.

There is an urgent need to refurbish and augment critical assessment of some well-known and widely accepted tools (including checklists such as COREQ, SRQR) to interrogate their applicability on different aspects (along with their epistemological ramifications).

Efforts should be made towards creating more space for creativity, experimentation, and a dialogue between the diverse traditions of qualitative research. This would potentially help to avoid the enforcement of one's own set of quality criteria on the work carried out by others.

Moreover, journal reviewers need to be aware of various methodological practices and philosophical debates.

It is pivotal to highlight the expressions and considerations of qualitative researchers and bring them into a more open and transparent dialogue about assessing qualitative research in techno-scientific, academic, sociocultural, and political rooms.

Frequent debates on the use of evaluative criteria are required to solve some potentially resolved issues (including the applicability of a single set of criteria in multi-disciplinary aspects). Such debates would not only benefit the group of qualitative researchers themselves, but primarily assist in augmenting the well-being and vivacity of the entire discipline.

To conclude, I speculate that the criteria, and my perspective, may transfer to other methods, approaches, and contexts. I hope that they spark dialog and debate – about criteria for excellent qualitative research and the underpinnings of the discipline more broadly – and, therefore, help improve the quality of a qualitative study. Further, I anticipate that this review will assist the researchers to contemplate on the quality of their own research, to substantiate research design and help the reviewers to review qualitative research for journals. On a final note, I pinpoint the need to formulate a framework (encompassing the prerequisites of a qualitative study) by the cohesive efforts of qualitative researchers of different disciplines with different theoretic-paradigmatic origins. I believe that tailoring such a framework (of guiding principles) paves the way for qualitative researchers to consolidate the status of qualitative research in the wide-ranging open science debate. Dialogue on this issue across different approaches is crucial for the impending prospects of socio-techno-educational research.

Amin, M. E. K., Nørgaard, L. S., Cavaco, A. M., Witry, M. J., Hillman, L., Cernasev, A., & Desselle, S. P. (2020). Establishing trustworthiness and authenticity in qualitative pharmacy research. Research in Social and Administrative Pharmacy, 16 (10), 1472–1482.

Article   Google Scholar  

Barker, C., & Pistrang, N. (2005). Quality criteria under methodological pluralism: Implications for conducting and evaluating research. American Journal of Community Psychology, 35 (3–4), 201–212.

Bryman, A., Becker, S., & Sempik, J. (2008). Quality criteria for quantitative, qualitative and mixed methods research: A view from social policy. International Journal of Social Research Methodology, 11 (4), 261–276.

Caelli, K., Ray, L., & Mill, J. (2003). ‘Clear as mud’: Toward greater clarity in generic qualitative research. International Journal of Qualitative Methods, 2 (2), 1–13.

CASP (2021). CASP checklists. Retrieved May 2021 from https://casp-uk.net/casp-tools-checklists/

Cohen, D. J., & Crabtree, B. F. (2008). Evaluative criteria for qualitative research in health care: Controversies and recommendations. The Annals of Family Medicine, 6 (4), 331–339.

Denzin, N. K., & Lincoln, Y. S. (2005). Introduction: The discipline and practice of qualitative research. In N. K. Denzin & Y. S. Lincoln (Eds.), The sage handbook of qualitative research (pp. 1–32). Sage Publications Ltd.

Google Scholar  

Elliott, R., Fischer, C. T., & Rennie, D. L. (1999). Evolving guidelines for publication of qualitative research studies in psychology and related fields. British Journal of Clinical Psychology, 38 (3), 215–229.

Epp, A. M., & Otnes, C. C. (2021). High-quality qualitative research: Getting into gear. Journal of Service Research . https://doi.org/10.1177/1094670520961445

Guba, E. G. (1990). The paradigm dialog. In Alternative paradigms conference, mar, 1989, Indiana u, school of education, San Francisco, ca, us . Sage Publications, Inc.

Hammersley, M. (2007). The issue of quality in qualitative research. International Journal of Research and Method in Education, 30 (3), 287–305.

Haven, T. L., Errington, T. M., Gleditsch, K. S., van Grootel, L., Jacobs, A. M., Kern, F. G., & Mokkink, L. B. (2020). Preregistering qualitative research: A Delphi study. International Journal of Qualitative Methods, 19 , 1609406920976417.

Hays, D. G., & McKibben, W. B. (2021). Promoting rigorous research: Generalizability and qualitative research. Journal of Counseling and Development, 99 (2), 178–188.

Horsburgh, D. (2003). Evaluation of qualitative research. Journal of Clinical Nursing, 12 (2), 307–312.

Howe, K. R. (2004). A critique of experimentalism. Qualitative Inquiry, 10 (1), 42–46.

Johnson, J. L., Adkins, D., & Chauvin, S. (2020). A review of the quality indicators of rigor in qualitative research. American Journal of Pharmaceutical Education, 84 (1), 7120.

Johnson, P., Buehring, A., Cassell, C., & Symon, G. (2006). Evaluating qualitative management research: Towards a contingent criteriology. International Journal of Management Reviews, 8 (3), 131–156.

Klein, H. K., & Myers, M. D. (1999). A set of principles for conducting and evaluating interpretive field studies in information systems. MIS Quarterly, 23 (1), 67–93.

Lather, P. (2004). This is your father’s paradigm: Government intrusion and the case of qualitative research in education. Qualitative Inquiry, 10 (1), 15–34.

Levitt, H. M., Morrill, Z., Collins, K. M., & Rizo, J. L. (2021). The methodological integrity of critical qualitative research: Principles to support design and research review. Journal of Counseling Psychology, 68 (3), 357.

Lincoln, Y. S., & Guba, E. G. (1986). But is it rigorous? Trustworthiness and authenticity in naturalistic evaluation. New Directions for Program Evaluation, 1986 (30), 73–84.

Lincoln, Y. S., & Guba, E. G. (2000). Paradigmatic controversies, contradictions and emerging confluences. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (2nd ed., pp. 163–188). Sage Publications.

Madill, A., Jordan, A., & Shirley, C. (2000). Objectivity and reliability in qualitative analysis: Realist, contextualist and radical constructionist epistemologies. British Journal of Psychology, 91 (1), 1–20.

Mays, N., & Pope, C. (2020). Quality in qualitative research. Qualitative Research in Health Care . https://doi.org/10.1002/9781119410867.ch15

McGinley, S., Wei, W., Zhang, L., & Zheng, Y. (2021). The state of qualitative research in hospitality: A 5-year review 2014 to 2019. Cornell Hospitality Quarterly, 62 (1), 8–20.

Merriam, S., & Tisdell, E. (2016). Qualitative research: A guide to design and implementation. San Francisco, US.

Meyer, M., & Dykes, J. (2019). Criteria for rigor in visualization design study. IEEE Transactions on Visualization and Computer Graphics, 26 (1), 87–97.

Monrouxe, L. V., & Rees, C. E. (2020). When I say… quantification in qualitative research. Medical Education, 54 (3), 186–187.

Morrow, S. L. (2005). Quality and trustworthiness in qualitative research in counseling psychology. Journal of Counseling Psychology, 52 (2), 250.

Morse, J. M. (2003). A review committee’s guide for evaluating qualitative proposals. Qualitative Health Research, 13 (6), 833–851.

Nassaji, H. (2020). Good qualitative research. Language Teaching Research, 24 (4), 427–431.

O’Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014). Standards for reporting qualitative research: A synthesis of recommendations. Academic Medicine, 89 (9), 1245–1251.

O’Connor, C., & Joffe, H. (2020). Intercoder reliability in qualitative research: Debates and practical guidelines. International Journal of Qualitative Methods, 19 , 1609406919899220.

Reid, A., & Gough, S. (2000). Guidelines for reporting and evaluating qualitative research: What are the alternatives? Environmental Education Research, 6 (1), 59–91.

Rocco, T. S. (2010). Criteria for evaluating qualitative studies. Human Resource Development International . https://doi.org/10.1080/13678868.2010.501959

Sandberg, J. (2000). Understanding human competence at work: An interpretative approach. Academy of Management Journal, 43 (1), 9–25.

Schwandt, T. A. (1996). Farewell to criteriology. Qualitative Inquiry, 2 (1), 58–72.

Seale, C. (1999). Quality in qualitative research. Qualitative Inquiry, 5 (4), 465–478.

Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22 (2), 63–75.

Sparkes, A. C. (2001). Myth 94: Qualitative health researchers will agree about validity. Qualitative Health Research, 11 (4), 538–552.

Spencer, L., Ritchie, J., Lewis, J., & Dillon, L. (2004). Quality in qualitative evaluation: A framework for assessing research evidence.

Stenfors, T., Kajamaa, A., & Bennett, D. (2020). How to assess the quality of qualitative research. The Clinical Teacher, 17 (6), 596–599.

Taylor, E. W., Beck, J., & Ainsworth, E. (2001). Publishing qualitative adult education research: A peer review perspective. Studies in the Education of Adults, 33 (2), 163–179.

Tong, A., Sainsbury, P., & Craig, J. (2007). Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. International Journal for Quality in Health Care, 19 (6), 349–357.

Tracy, S. J. (2010). Qualitative quality: Eight “big-tent” criteria for excellent qualitative research. Qualitative Inquiry, 16 (10), 837–851.

Download references

Open access funding provided by TU Wien (TUW).

Author information

Authors and affiliations.

Faculty of Informatics, Technische Universität Wien, 1040, Vienna, Austria

Drishti Yadav

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Drishti Yadav .

Ethics declarations

Conflict of interest.

The author declares no conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Yadav, D. Criteria for Good Qualitative Research: A Comprehensive Review. Asia-Pacific Edu Res 31 , 679–689 (2022). https://doi.org/10.1007/s40299-021-00619-0

Download citation

Accepted : 28 August 2021

Published : 18 September 2021

Issue Date : December 2022

DOI : https://doi.org/10.1007/s40299-021-00619-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Qualitative research
• Evaluative criteria
• Find a journal
• Publish with us
• Track your research

Qualitative Research Methods: A Practice-Oriented Introduction

• February 2023

• University of Hamburg

Abstract and Figures

Discover the world's research

• 25+ million members
• 160+ million publication pages
• 2.3+ billion citations

Supplementary resource (1)

• Simanga Phillip Lukhele
• LDM Lebeloane
• Nonzukiso Tyilo
• Thobeka Matshoba

• Kingsley Afful

• Int J Econ Bus Res

• Muhammad Yusry Affandy bin Md Isa
• Muhammad Syahmi bin Shakhruddin
• Maryuni Maryuni
• Sabarinah Prasetyo

• Arinza Justistio
• Amiruddin Amiruddin
• Junaidah Junaidah

• SCI JUSTICE
• Carlos Miguel Ibaviosa

• Firas Khatib

• Leon Festinger
• Kenneth L. Pike
• Recruit researchers
• Join for free
• Login Email Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google Welcome back! Please log in. Email · Hint Tip: Most researchers use their institutional email address as their ResearchGate login Password Forgot password? Keep me logged in Log in or Continue with Google No account? Sign up

18 Qualitative Research Examples

Chris Drew (PhD)

Dr. Chris Drew is the founder of the Helpful Professor. He holds a PhD in education and has published over 20 articles in scholarly journals. He is the former editor of the Journal of Learning Development in Higher Education. [Image Descriptor: Photo of Chris]

Learn about our Editorial Process

Qualitative research is an approach to scientific research that involves using observation to gather and analyze non-numerical, in-depth, and well-contextualized datasets.

It serves as an integral part of academic, professional, and even daily decision-making processes (Baxter & Jack, 2008).

Methods of qualitative research encompass a wide range of techniques, from in-depth personal encounters, like ethnographies (studying cultures in-depth) and autoethnographies (examining one’s own cultural experiences), to collection of diverse perspectives on topics through methods like interviewing focus groups (gatherings of individuals to discuss specific topics).

Qualitative Research Examples

1. ethnography.

Definition: Ethnography is a qualitative research design aimed at exploring cultural phenomena. Rooted in the discipline of anthropology , this research approach investigates the social interactions, behaviors, and perceptions within groups, communities, or organizations.

Ethnographic research is characterized by extended observation of the group, often through direct participation, in the participants’ environment. An ethnographer typically lives with the study group for extended periods, intricately observing their everyday lives (Khan, 2014).

It aims to present a complete, detailed and accurate picture of the observed social life, rituals, symbols, and values from the perspective of the study group.

Example of Ethnographic Research

Title: “ The Everyday Lives of Men: An Ethnographic Investigation of Young Adult Male Identity “

Citation: Evans, J. (2010). The Everyday Lives of Men: An Ethnographic Investigation of Young Adult Male Identity. Peter Lang.

Overview: This study by Evans (2010) provides a rich narrative of young adult male identity as experienced in everyday life. The author immersed himself among a group of young men, participating in their activities and cultivating a deep understanding of their lifestyle, values, and motivations. This research exemplified the ethnographic approach, revealing complexities of the subjects’ identities and societal roles, which could hardly be accessed through other qualitative research designs.

Read my Full Guide on Ethnography Here

2. Autoethnography

Definition: Autoethnography is an approach to qualitative research where the researcher uses their own personal experiences to extend the understanding of a certain group, culture, or setting. Essentially, it allows for the exploration of self within the context of social phenomena.

Unlike traditional ethnography, which focuses on the study of others, autoethnography turns the ethnographic gaze inward, allowing the researcher to use their personal experiences within a culture as rich qualitative data (Durham, 2019).

The objective is to critically appraise one’s personal experiences as they navigate and negotiate cultural, political, and social meanings. The researcher becomes both the observer and the participant, intertwining personal and cultural experiences in the research.

Example of Autoethnographic Research

Title: “ A Day In The Life Of An NHS Nurse “

Citation: Osben, J. (2019). A day in the life of a NHS nurse in 21st Century Britain: An auto-ethnography. The Journal of Autoethnography for Health & Social Care. 1(1).

Overview: This study presents an autoethnography of a day in the life of an NHS nurse (who, of course, is also the researcher). The author uses the research to achieve reflexivity, with the researcher concluding: “Scrutinising my practice and situating it within a wider contextual backdrop has compelled me to significantly increase my level of scrutiny into the driving forces that influence my practice.”

Read my Full Guide on Autoethnography Here

3. Semi-Structured Interviews

Definition: Semi-structured interviews stand as one of the most frequently used methods in qualitative research. These interviews are planned and utilize a set of pre-established questions, but also allow for the interviewer to steer the conversation in other directions based on the responses given by the interviewee.

In semi-structured interviews, the interviewer prepares a guide that outlines the focal points of the discussion. However, the interview is flexible, allowing for more in-depth probing if the interviewer deems it necessary (Qu, & Dumay, 2011). This style of interviewing strikes a balance between structured ones which might limit the discussion, and unstructured ones, which could lack focus.

Example of Semi-Structured Interview Research

Title: “ Factors influencing adherence to cancer treatment in older adults with cancer: a systematic review “

Citation: Puts, M., et al. (2014). Factors influencing adherence to cancer treatment in older adults with cancer: a systematic review. Annals of oncology, 25 (3), 564-577.

Overview: Puts et al. (2014) executed an extensive systematic review in which they conducted semi-structured interviews with older adults suffering from cancer to examine the factors influencing their adherence to cancer treatment. The findings suggested that various factors, including side effects, faith in healthcare professionals, and social support have substantial impacts on treatment adherence. This research demonstrates how semi-structured interviews can provide rich and profound insights into the subjective experiences of patients.

4. Focus Groups

Definition: Focus groups are a qualitative research method that involves organized discussion with a selected group of individuals to gain their perspectives on a specific concept, product, or phenomenon. Typically, these discussions are guided by a moderator.

During a focus group session, the moderator has a list of questions or topics to discuss, and participants are encouraged to interact with each other (Morgan, 2010). This interactivity can stimulate more information and provide a broader understanding of the issue under scrutiny. The open format allows participants to ask questions and respond freely, offering invaluable insights into attitudes, experiences, and group norms.

Example of Focus Group Research

Title: “ Perspectives of Older Adults on Aging Well: A Focus Group Study “

Citation: Halaweh, H., Dahlin-Ivanoff, S., Svantesson, U., & Willén, C. (2018). Perspectives of older adults on aging well: a focus group study. Journal of aging research .

Overview: This study aimed to explore what older adults (aged 60 years and older) perceived to be ‘aging well’. The researchers identified three major themes from their focus group interviews: a sense of well-being, having good physical health, and preserving good mental health. The findings highlight the importance of factors such as positive emotions, social engagement, physical activity, healthy eating habits, and maintaining independence in promoting aging well among older adults.

5. Phenomenology

Definition: Phenomenology, a qualitative research method, involves the examination of lived experiences to gain an in-depth understanding of the essence or underlying meanings of a phenomenon.

The focus of phenomenology lies in meticulously describing participants’ conscious experiences related to the chosen phenomenon (Padilla-Díaz, 2015).

In a phenomenological study, the researcher collects detailed, first-hand perspectives of the participants, typically via in-depth interviews, and then uses various strategies to interpret and structure these experiences, ultimately revealing essential themes (Creswell, 2013). This approach focuses on the perspective of individuals experiencing the phenomenon, seeking to explore, clarify, and understand the meanings they attach to those experiences.

Example of Phenomenology Research

Title: “ A phenomenological approach to experiences with technology: current state, promise, and future directions for research ”

Citation: Cilesiz, S. (2011). A phenomenological approach to experiences with technology: Current state, promise, and future directions for research. Educational Technology Research and Development, 59 , 487-510.

Overview: A phenomenological approach to experiences with technology by Sebnem Cilesiz represents a good starting point for formulating a phenomenological study. With its focus on the ‘essence of experience’, this piece presents methodological, reliability, validity, and data analysis techniques that phenomenologists use to explain how people experience technology in their everyday lives.

6. Grounded Theory

Definition: Grounded theory is a systematic methodology in qualitative research that typically applies inductive reasoning . The primary aim is to develop a theoretical explanation or framework for a process, action, or interaction grounded in, and arising from, empirical data (Birks & Mills, 2015).

In grounded theory, data collection and analysis work together in a recursive process. The researcher collects data, analyses it, and then collects more data based on the evolving understanding of the research context. This ongoing process continues until a comprehensive theory that represents the data and the associated phenomenon emerges – a point known as theoretical saturation (Charmaz, 2014).

Example of Grounded Theory Research

Title: “ Student Engagement in High School Classrooms from the Perspective of Flow Theory “

Citation: Shernoff, D. J., Csikszentmihalyi, M., Shneider, B., & Shernoff, E. S. (2003). Student engagement in high school classrooms from the perspective of flow theory. School Psychology Quarterly, 18 (2), 158–176.

Overview: Shernoff and colleagues (2003) used grounded theory to explore student engagement in high school classrooms. The researchers collected data through student self-reports, interviews, and observations. Key findings revealed that academic challenge, student autonomy, and teacher support emerged as the most significant factors influencing students’ engagement, demonstrating how grounded theory can illuminate complex dynamics within real-world contexts.

7. Narrative Research

Definition: Narrative research is a qualitative research method dedicated to storytelling and understanding how individuals experience the world. It focuses on studying an individual’s life and experiences as narrated by that individual (Polkinghorne, 2013).

In narrative research, the researcher collects data through methods such as interviews, observations , and document analysis. The emphasis is on the stories told by participants – narratives that reflect their experiences, thoughts, and feelings.

These stories are then interpreted by the researcher, who attempts to understand the meaning the participant attributes to these experiences (Josselson, 2011).

Example of Narrative Research

Title: “Narrative Structures and the Language of the Self”

Citation: McAdams, D. P., Josselson, R., & Lieblich, A. (2006). Identity and story: Creating self in narrative . American Psychological Association.

Overview: In this innovative study, McAdams et al. (2006) employed narrative research to explore how individuals construct their identities through the stories they tell about themselves. By examining personal narratives, the researchers discerned patterns associated with characters, motivations, conflicts, and resolutions, contributing valuable insights about the relationship between narrative and individual identity.

8. Case Study Research

Definition: Case study research is a qualitative research method that involves an in-depth investigation of a single instance or event: a case. These ‘cases’ can range from individuals, groups, or entities to specific projects, programs, or strategies (Creswell, 2013).

The case study method typically uses multiple sources of information for comprehensive contextual analysis. It aims to explore and understand the complexity and uniqueness of a particular case in a real-world context (Merriam & Tisdell, 2015). This investigation could result in a detailed description of the case, a process for its development, or an exploration of a related issue or problem.

Example of Case Study Research

Title: “ Teacher’s Role in Fostering Preschoolers’ Computational Thinking: An Exploratory Case Study “

Citation: Wang, X. C., Choi, Y., Benson, K., Eggleston, C., & Weber, D. (2021). Teacher’s role in fostering preschoolers’ computational thinking: An exploratory case study. Early Education and Development , 32 (1), 26-48.

Overview: This study investigates the role of teachers in promoting computational thinking skills in preschoolers. The study utilized a qualitative case study methodology to examine the computational thinking scaffolding strategies employed by a teacher interacting with three preschoolers in a small group setting. The findings highlight the importance of teachers’ guidance in fostering computational thinking practices such as problem reformulation/decomposition, systematic testing, and debugging.

Read about some Famous Case Studies in Psychology Here

9. Participant Observation

Definition: Participant observation has the researcher immerse themselves in a group or community setting to observe the behavior of its members. It is similar to ethnography, but generally, the researcher isn’t embedded for a long period of time.

The researcher, being a participant, engages in daily activities, interactions, and events as a way of conducting a detailed study of a particular social phenomenon (Kawulich, 2005).

The method involves long-term engagement in the field, maintaining detailed records of observed events, informal interviews, direct participation, and reflexivity. This approach allows for a holistic view of the participants’ lived experiences, behaviours, and interactions within their everyday environment (Dewalt, 2011).

Example of Participant Observation Research

Title: Conflict in the boardroom: a participant observation study of supervisory board dynamics

Citation: Heemskerk, E. M., Heemskerk, K., & Wats, M. M. (2017). Conflict in the boardroom: a participant observation study of supervisory board dynamics. Journal of Management & Governance , 21 , 233-263.

Overview: This study examined how conflicts within corporate boards affect their performance. The researchers used a participant observation method, where they actively engaged with 11 supervisory boards and observed their dynamics. They found that having a shared understanding of the board’s role called a common framework, improved performance by reducing relationship conflicts, encouraging task conflicts, and minimizing conflicts between the board and CEO.

10. Non-Participant Observation

Definition: Non-participant observation is a qualitative research method in which the researcher observes the phenomena of interest without actively participating in the situation, setting, or community being studied.

This method allows the researcher to maintain a position of distance, as they are solely an observer and not a participant in the activities being observed (Kawulich, 2005).

During non-participant observation, the researcher typically records field notes on the actions, interactions, and behaviors observed , focusing on specific aspects of the situation deemed relevant to the research question.

This could include verbal and nonverbal communication , activities, interactions, and environmental contexts (Angrosino, 2007). They could also use video or audio recordings or other methods to collect data.

Example of Non-Participant Observation Research

Title: Mental Health Nurses’ attitudes towards mental illness and recovery-oriented practice in acute inpatient psychiatric units: A non-participant observation study

Citation: Sreeram, A., Cross, W. M., & Townsin, L. (2023). Mental Health Nurses’ attitudes towards mental illness and recovery‐oriented practice in acute inpatient psychiatric units: A non‐participant observation study. International Journal of Mental Health Nursing .

Overview: This study investigated the attitudes of mental health nurses towards mental illness and recovery-oriented practice in acute inpatient psychiatric units. The researchers used a non-participant observation method, meaning they observed the nurses without directly participating in their activities. The findings shed light on the nurses’ perspectives and behaviors, providing valuable insights into their attitudes toward mental health and recovery-focused care in these settings.

11. Content Analysis

Definition: Content Analysis involves scrutinizing textual, visual, or spoken content to categorize and quantify information. The goal is to identify patterns, themes, biases, or other characteristics (Hsieh & Shannon, 2005).

Content Analysis is widely used in various disciplines for a multitude of purposes. Researchers typically use this method to distill large amounts of unstructured data, like interview transcripts, newspaper articles, or social media posts, into manageable and meaningful chunks.

When wielded appropriately, Content Analysis can illuminate the density and frequency of certain themes within a dataset, provide insights into how specific terms or concepts are applied contextually, and offer inferences about the meanings of their content and use (Duriau, Reger, & Pfarrer, 2007).

Example of Content Analysis

Title: Framing European politics: A content analysis of press and television news .

Citation: Semetko, H. A., & Valkenburg, P. M. (2000). Framing European politics: A content analysis of press and television news. Journal of Communication, 50 (2), 93-109.

Overview: This study analyzed press and television news articles about European politics using a method called content analysis. The researchers examined the prevalence of different “frames” in the news, which are ways of presenting information to shape audience perceptions. They found that the most common frames were attribution of responsibility, conflict, economic consequences, human interest, and morality.

Read my Full Guide on Content Analysis Here

12. Discourse Analysis

Definition: Discourse Analysis, a qualitative research method, interprets the meanings, functions, and coherence of certain languages in context.

Discourse analysis is typically understood through social constructionism, critical theory , and poststructuralism and used for understanding how language constructs social concepts (Cheek, 2004).

Discourse Analysis offers great breadth, providing tools to examine spoken or written language, often beyond the level of the sentence. It enables researchers to scrutinize how text and talk articulate social and political interactions and hierarchies.

Insight can be garnered from different conversations, institutional text, and media coverage to understand how topics are addressed or framed within a specific social context (Jorgensen & Phillips, 2002).

Example of Discourse Analysis

Title: The construction of teacher identities in educational policy documents: A critical discourse analysis

Citation: Thomas, S. (2005). The construction of teacher identities in educational policy documents: A critical discourse analysis. Critical Studies in Education, 46 (2), 25-44.

Overview: The author examines how an education policy in one state of Australia positions teacher professionalism and teacher identities. While there are competing discourses about professional identity, the policy framework privileges a  narrative that frames the ‘good’ teacher as one that accepts ever-tightening control and regulation over their professional practice.

Read my Full Guide on Discourse Analysis Here

13. Action Research

Definition: Action Research is a qualitative research technique that is employed to bring about change while simultaneously studying the process and results of that change.

This method involves a cyclical process of fact-finding, action, evaluation, and reflection (Greenwood & Levin, 2016).

Typically, Action Research is used in the fields of education, social sciences , and community development. The process isn’t just about resolving an issue but also developing knowledge that can be used in the future to address similar or related problems.

The researcher plays an active role in the research process, which is normally broken down into four steps: 

• developing a plan to improve what is currently being done
• implementing the plan
• observing the effects of the plan, and
• reflecting upon these effects (Smith, 2010).

Example of Action Research

Title: Using Digital Sandbox Gaming to Improve Creativity Within Boys’ Writing

Citation: Ellison, M., & Drew, C. (2020). Using digital sandbox gaming to improve creativity within boys’ writing. Journal of Research in Childhood Education , 34 (2), 277-287.

Overview: This was a research study one of my research students completed in his own classroom under my supervision. He implemented a digital game-based approach to literacy teaching with boys and interviewed his students to see if the use of games as stimuli for storytelling helped draw them into the learning experience.

Read my Full Guide on Action Research Here

14. Semiotic Analysis

Definition: Semiotic Analysis is a qualitative method of research that interprets signs and symbols in communication to understand sociocultural phenomena. It stems from semiotics, the study of signs and symbols and their use or interpretation (Chandler, 2017).

In a Semiotic Analysis, signs (anything that represents something else) are interpreted based on their significance and the role they play in representing ideas.

This type of research often involves the examination of images, sounds, and word choice to uncover the embedded sociocultural meanings. For example, an advertisement for a car might be studied to learn more about societal views on masculinity or success (Berger, 2010).

Example of Semiotic Research

Title: Shielding the learned body: a semiotic analysis of school badges in New South Wales, Australia

Citation: Symes, C. (2023). Shielding the learned body: a semiotic analysis of school badges in New South Wales, Australia. Semiotica , 2023 (250), 167-190.

Overview: This study examines school badges in New South Wales, Australia, and explores their significance through a semiotic analysis. The badges, which are part of the school’s visual identity, are seen as symbolic representations that convey meanings. The analysis reveals that these badges often draw on heraldic models, incorporating elements like colors, names, motifs, and mottoes that reflect local culture and history, thus connecting students to their national identity. Additionally, the study highlights how some schools have shifted from traditional badges to modern logos and slogans, reflecting a more business-oriented approach.

15. Qualitative Longitudinal Studies

Definition: Qualitative Longitudinal Studies are a research method that involves repeated observation of the same items over an extended period of time.

Unlike a snapshot perspective, this method aims to piece together individual histories and examine the influences and impacts of change (Neale, 2019).

Qualitative Longitudinal Studies provide an in-depth understanding of change as it happens, including changes in people’s lives, their perceptions, and their behaviors.

For instance, this method could be used to follow a group of students through their schooling years to understand the evolution of their learning behaviors and attitudes towards education (Saldaña, 2003).

Example of Qualitative Longitudinal Research

Title: Patient and caregiver perspectives on managing pain in advanced cancer: a qualitative longitudinal study

Citation: Hackett, J., Godfrey, M., & Bennett, M. I. (2016). Patient and caregiver perspectives on managing pain in advanced cancer: a qualitative longitudinal study.  Palliative medicine ,  30 (8), 711-719.

Overview: This article examines how patients and their caregivers manage pain in advanced cancer through a qualitative longitudinal study. The researchers interviewed patients and caregivers at two different time points and collected audio diaries to gain insights into their experiences, making this study longitudinal.

Read my Full Guide on Longitudinal Research Here

16. Open-Ended Surveys

Definition: Open-Ended Surveys are a type of qualitative research method where respondents provide answers in their own words. Unlike closed-ended surveys, which limit responses to predefined options, open-ended surveys allow for expansive and unsolicited explanations (Fink, 2013).

Open-ended surveys are commonly used in a range of fields, from market research to social studies. As they don’t force respondents into predefined response categories, these surveys help to draw out rich, detailed data that might uncover new variables or ideas.

For example, an open-ended survey might be used to understand customer opinions about a new product or service (Lavrakas, 2008).

Contrast this to a quantitative closed-ended survey, like a Likert scale, which could theoretically help us to come up with generalizable data but is restricted by the questions on the questionnaire, meaning new and surprising data and insights can’t emerge from the survey results in the same way.

Example of Open-Ended Survey Research

Title: Advantages and disadvantages of technology in relationships: Findings from an open-ended survey

Citation: Hertlein, K. M., & Ancheta, K. (2014). Advantages and disadvantages of technology in relationships: Findings from an open-ended survey.  The Qualitative Report ,  19 (11), 1-11.

Overview: This article examines the advantages and disadvantages of technology in couple relationships through an open-ended survey method. Researchers analyzed responses from 410 undergraduate students to understand how technology affects relationships. They found that technology can contribute to relationship development, management, and enhancement, but it can also create challenges such as distancing, lack of clarity, and impaired trust.

17. Naturalistic Observation

Definition: Naturalistic Observation is a type of qualitative research method that involves observing individuals in their natural environments without interference or manipulation by the researcher.

Naturalistic observation is often used when conducting research on behaviors that cannot be controlled or manipulated in a laboratory setting (Kawulich, 2005).

It is frequently used in the fields of psychology, sociology, and anthropology. For instance, to understand the social dynamics in a schoolyard, a researcher could spend time observing the children interact during their recess, noting their behaviors, interactions, and conflicts without imposing their presence on the children’s activities (Forsyth, 2010).

Example of Naturalistic Observation Research

Title: Dispositional mindfulness in daily life: A naturalistic observation study

Citation: Kaplan, D. M., Raison, C. L., Milek, A., Tackman, A. M., Pace, T. W., & Mehl, M. R. (2018). Dispositional mindfulness in daily life: A naturalistic observation study. PloS one , 13 (11), e0206029.

Overview: In this study, researchers conducted two studies: one exploring assumptions about mindfulness and behavior, and the other using naturalistic observation to examine actual behavioral manifestations of mindfulness. They found that trait mindfulness is associated with a heightened perceptual focus in conversations, suggesting that being mindful is expressed primarily through sharpened attention rather than observable behavioral or social differences.

Read my Full Guide on Naturalistic Observation Here

18. Photo-Elicitation

Definition: Photo-elicitation utilizes photographs as a means to trigger discussions and evoke responses during interviews. This strategy aids in bringing out topics of discussion that may not emerge through verbal prompting alone (Harper, 2002).

Traditionally, Photo-Elicitation has been useful in various fields such as education, psychology, and sociology. The method involves the researcher or participants taking photographs, which are then used as prompts for discussion.

For instance, a researcher studying urban environmental issues might invite participants to photograph areas in their neighborhood that they perceive as environmentally detrimental, and then discuss each photo in depth (Clark-Ibáñez, 2004).

Example of Photo-Elicitation Research

Title: Early adolescent food routines: A photo-elicitation study

Citation: Green, E. M., Spivak, C., & Dollahite, J. S. (2021). Early adolescent food routines: A photo-elicitation study. Appetite, 158 .

Overview: This study focused on early adolescents (ages 10-14) and their food routines. Researchers conducted in-depth interviews using a photo-elicitation approach, where participants took photos related to their food choices and experiences. Through analysis, the study identified various routines and three main themes: family, settings, and meals/foods consumed, revealing how early adolescents view and are influenced by their eating routines.

Features of Qualitative Research

Qualitative research is a research method focused on understanding the meaning individuals or groups attribute to a social or human problem (Creswell, 2013).

Some key features of this method include:

• Naturalistic Inquiry: Qualitative research happens in the natural setting of the phenomena, aiming to understand “real world” situations (Patton, 2015). This immersion in the field or subject allows the researcher to gather a deep understanding of the subject matter.
• Emphasis on Process: It aims to understand how events unfold over time rather than focusing solely on outcomes (Merriam & Tisdell, 2015). The process-oriented nature of qualitative research allows researchers to investigate sequences, timing, and changes.
• Interpretive: It involves interpreting and making sense of phenomena in terms of the meanings people assign to them (Denzin & Lincoln, 2011). This interpretive element allows for rich, nuanced insights into human behavior and experiences.
• Holistic Perspective: Qualitative research seeks to understand the whole phenomenon rather than focusing on individual components (Creswell, 2013). It emphasizes the complex interplay of factors, providing a richer, more nuanced view of the research subject.
• Prioritizes Depth over Breadth: Qualitative research favors depth of understanding over breadth, typically involving a smaller but more focused sample size (Hennink, Hutter, & Bailey, 2020). This enables detailed exploration of the phenomena of interest, often leading to rich and complex data.

Qualitative vs Quantitative Research

Qualitative research centers on exploring and understanding the meaning individuals or groups attribute to a social or human problem (Creswell, 2013).

It involves an in-depth approach to the subject matter, aiming to capture the richness and complexity of human experience.

Examples include conducting interviews, observing behaviors, or analyzing text and images.

There are strengths inherent in this approach. In its focus on understanding subjective experiences and interpretations, qualitative research can yield rich and detailed data that quantitative research may overlook (Denzin & Lincoln, 2011).

Additionally, qualitative research is adaptive, allowing the researcher to respond to new directions and insights as they emerge during the research process.

However, there are also limitations. Because of the interpretive nature of this research, findings may not be generalizable to a broader population (Marshall & Rossman, 2014). Well-designed quantitative research, on the other hand, can be generalizable.

Moreover, the reliability and validity of qualitative data can be challenging to establish due to its subjective nature, unlike quantitative research, which is ideally more objective.

Compare Qualitative and Quantitative Research Methodologies in This Guide Here

In conclusion, qualitative research methods provide distinctive ways to explore social phenomena and understand nuances that quantitative approaches might overlook. Each method, from Ethnography to Photo-Elicitation, presents its strengths and weaknesses but they all offer valuable means of investigating complex, real-world situations. The goal for the researcher is not to find a definitive tool, but to employ the method best suited for their research questions and the context at hand (Almalki, 2016). Above all, these methods underscore the richness of human experience and deepen our understanding of the world around us.

Angrosino, M. (2007). Doing ethnographic and observational research. Sage Publications.

Areni, C. S., & Kim, D. (1994). The influence of in-store lighting on consumers’ examination of merchandise in a wine store. International Journal of Research in Marketing, 11 (2), 117-125.

Barker, C., Pistrang, N., & Elliott, R. (2016). Research Methods in Clinical Psychology: An Introduction for Students and Practitioners. John Wiley & Sons.

Baxter, P. & Jack, S. (2008). Qualitative case study methodology: Study design and implementation for novice researchers. The Qualitative Report, 13 (4), 544-559.

Berger, A. A. (2010). The Objects of Affection: Semiotics and Consumer Culture. Palgrave Macmillan.

Bevan, M. T. (2014). A method of phenomenological interviewing. Qualitative health research, 24 (1), 136-144.

Birks, M., & Mills, J. (2015). Grounded theory: A practical guide . Sage Publications.

Bryman, A. (2015) . The SAGE Handbook of Qualitative Research. Sage Publications.

Chandler, D. (2017). Semiotics: The Basics. Routledge.

Charmaz, K. (2014). Constructing grounded theory. Sage Publications.

Cheek, J. (2004). At the margins? Discourse analysis and qualitative research. Qualitative Health Research, 14(8), 1140-1150.

Clark-Ibáñez, M. (2004). Framing the social world with photo-elicitation interviews. American Behavioral Scientist, 47(12), 1507-1527.

Creswell, J. W. (2013). Research Design: Qualitative, Quantitative and Mixed Methods Approaches. Sage Publications.

Creswell, J. W., & Creswell, J. D. (2017). Research design: Qualitative, quantitative, and mixed methods approaches. Sage publications.

Crowe, S., Cresswell, K., Robertson, A., Huby, G., Avery, A., & Sheikh, A. (2011). The case study approach. BMC Medical Research Methodology, 11(100), 1-9.

Denzin, N. K., & Lincoln, Y. S. (2011). The Sage Handbook of Qualitative Research. Sage.

Dewalt, K. M., & Dewalt, B. R. (2011). Participant observation: A guide for fieldworkers. Rowman Altamira.

Doody, O., Slevin, E., & Taggart, L. (2013). Focus group interviews in nursing research: part 1. British Journal of Nursing, 22(1), 16-19.

Durham, A. (2019). Autoethnography. In P. Atkinson (Ed.), Qualitative Research Methods. Oxford University Press.

Duriau, V. J., Reger, R. K., & Pfarrer, M. D. (2007). A content analysis of the content analysis literature in organization studies: Research themes, data sources, and methodological refinements. Organizational Research Methods, 10(1), 5-34.

Evans, J. (2010). The Everyday Lives of Men: An Ethnographic Investigation of Young Adult Male Identity. Peter Lang.

Farrall, S. (2006). What is qualitative longitudinal research? Papers in Social Research Methods, Qualitative Series, No.11, London School of Economics, Methodology Institute.

Fielding, J., & Fielding, N. (2008). Synergy and synthesis: integrating qualitative and quantitative data. The SAGE handbook of social research methods, 555-571.

Fink, A. (2013). How to conduct surveys: A step-by-step guide . SAGE.

Forsyth, D. R. (2010). Group Dynamics . Wadsworth Cengage Learning.

Fugard, A. J. B., & Potts, H. W. W. (2015). Supporting thinking on sample sizes for thematic analyses: A quantitative tool. International Journal of Social Research Methodology, 18 (6), 669–684.

Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory: Strategies for qualitative research. Aldine de Gruyter.

Gray, J. R., Grove, S. K., & Sutherland, S. (2017). Burns and Grove’s the Practice of Nursing Research E-Book: Appraisal, Synthesis, and Generation of Evidence. Elsevier Health Sciences.

Greenwood, D. J., & Levin, M. (2016). Introduction to action research: Social research for social change. SAGE.

Harper, D. (2002). Talking about pictures: A case for photo elicitation. Visual Studies, 17 (1), 13-26.

Heinonen, T. (2012). Making Sense of the Social: Human Sciences and the Narrative Turn. Rozenberg Publishers.

Heisley, D. D., & Levy, S. J. (1991). Autodriving: A photoelicitation technique. Journal of Consumer Research, 18 (3), 257-272.

Hennink, M. M., Hutter, I., & Bailey, A. (2020). Qualitative Research Methods . SAGE Publications Ltd.

Hsieh, H. F., & Shannon, S. E. (2005). Three Approaches to Qualitative Content Analysis. Qualitative Health Research, 15 (9), 1277–1288.

Jorgensen, D. L. (2015). Participant Observation. In Emerging Trends in the Social and Behavioral Sciences: An Interdisciplinary, Searchable, and Linkable Resource. John Wiley & Sons, Inc.

Jorgensen, M., & Phillips, L. (2002). Discourse Analysis as Theory and Method . SAGE.

Josselson, R. (2011). Narrative research: Constructing, deconstructing, and reconstructing story. In Five ways of doing qualitative analysis . Guilford Press.

Kawulich, B. B. (2005). Participant observation as a data collection method. Forum: Qualitative Social Research, 6 (2).

Khan, S. (2014). Qualitative Research Method: Grounded Theory. Journal of Basic and Clinical Pharmacy, 5 (4), 86-88.

Koshy, E., Koshy, V., & Waterman, H. (2010). Action Research in Healthcare . SAGE.

Krippendorff, K. (2013). Content Analysis: An Introduction to its Methodology. SAGE.

Lannon, J., & Cooper, P. (2012). Humanistic Advertising: A Holistic Cultural Perspective. International Journal of Advertising, 15 (2), 97–111.

Lavrakas, P. J. (2008). Encyclopedia of survey research methods. SAGE Publications.

Lieblich, A., Tuval-Mashiach, R., & Zilber, T. (2008). Narrative research: Reading, analysis and interpretation. Sage Publications.

Mackey, A., & Gass, S. M. (2015). Second language research: Methodology and design. Routledge.

Marshall, C., & Rossman, G. B. (2014). Designing qualitative research. Sage publications.

McAdams, D. P., Josselson, R., & Lieblich, A. (2006). Identity and story: Creating self in narrative. American Psychological Association.

Merriam, S. B., & Tisdell, E. J. (2015). Qualitative Research: A Guide to Design and Implementation. Jossey-Bass.

Mick, D. G. (1986). Consumer Research and Semiotics: Exploring the Morphology of Signs, Symbols, and Significance. Journal of Consumer Research, 13 (2), 196-213.

Morgan, D. L. (2010). Focus groups as qualitative research. Sage Publications.

Mulhall, A. (2003). In the field: notes on observation in qualitative research. Journal of Advanced Nursing, 41 (3), 306-313.

Neale, B. (2019). What is Qualitative Longitudinal Research? Bloomsbury Publishing.

Nolan, L. B., & Renderos, T. B. (2012). A focus group study on the influence of fatalism and religiosity on cancer risk perceptions in rural, eastern North Carolina. Journal of religion and health, 51 (1), 91-104.

Padilla-Díaz, M. (2015). Phenomenology in educational qualitative research: Philosophy as science or philosophical science? International Journal of Educational Excellence, 1 (2), 101-110.

Parker, I. (2014). Discourse dynamics: Critical analysis for social and individual psychology . Routledge.

Patton, M. Q. (2015). Qualitative research & evaluation methods: Integrating theory and practice . Sage Publications.

Polkinghorne, D. E. (2013). Narrative configuration in qualitative analysis. In Life history and narrative. Routledge.

Puts, M. T., Tapscott, B., Fitch, M., Howell, D., Monette, J., Wan-Chow-Wah, D., Krzyzanowska, M., Leighl, N. B., Springall, E., & Alibhai, S. (2014). Factors influencing adherence to cancer treatment in older adults with cancer: a systematic review. Annals of oncology, 25 (3), 564-577.

Qu, S. Q., & Dumay, J. (2011). The qualitative research interview . Qualitative research in accounting & management.

Ali, J., & Bhaskar, S. B. (2016). Basic statistical tools in research and data analysis. Indian Journal of Anaesthesia, 60 (9), 662–669.

Rosenbaum, M. S. (2017). Exploring the social supportive role of third places in consumers’ lives. Journal of Service Research, 20 (1), 26-42.

Saldaña, J. (2003). Longitudinal Qualitative Research: Analyzing Change Through Time . AltaMira Press.

Saldaña, J. (2014). The Coding Manual for Qualitative Researchers. SAGE.

Shernoff, D. J., Csikszentmihalyi, M., Shneider, B., & Shernoff, E. S. (2003). Student engagement in high school classrooms from the perspective of flow theory. School Psychology Quarterly, 18 (2), 158-176.

Smith, J. A. (2015). Qualitative Psychology: A Practical Guide to Research Methods . Sage Publications.

Smith, M. K. (2010). Action Research. The encyclopedia of informal education.

Sue, V. M., & Ritter, L. A. (2012). Conducting online surveys . SAGE Publications.

Van Auken, P. M., Frisvoll, S. J., & Stewart, S. I. (2010). Visualising community: using participant-driven photo-elicitation for research and application. Local Environment, 15 (4), 373-388.

Van Voorhis, F. L., & Morgan, B. L. (2007). Understanding Power and Rules of Thumb for Determining Sample Sizes. Tutorials in Quantitative Methods for Psychology, 3 (2), 43–50.

Wodak, R., & Meyer, M. (2015). Methods of Critical Discourse Analysis . SAGE.

Zuber-Skerritt, O. (2018). Action research for developing educational theories and practices . Routledge.

• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 10 Reasons you’re Perpetually Single
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 20 Montessori Toddler Bedrooms (Design Inspiration)
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 21 Montessori Homeschool Setups
• Chris Drew (PhD) https://helpfulprofessor.com/author/chris-drew-phd-2/ 101 Hidden Talents Examples

Leave a Comment Cancel Reply

Your email address will not be published. Required fields are marked *

• Search Close search
• Find a journal
• Search calls for papers
• Journal Suggester
• Open access publishing

We’re here to help

Find guidance on Author Services

Open access

A qualitative study of primary teachers’ classroom feedback rationales

• Cite this article
• https://doi.org/10.1080/00131881.2018.1451759

Introduction

Conclusions, disclosure statement.

• Full Article
• Figures & data
• Reprints & Permissions
• View PDF PDF

As part of teachers’ everyday classroom assessment practice, feedback can be seen as connected to the formative function of assessment, with the aim of helping students in their learning processes. Much research on teacher feedback focuses precisely on the feedback’s formative quality. However, in order to strengthen our understanding about the nature of teacher feedback, we also need to understand more about teachers’ rationales for giving feedback to their students, especially in primary school settings.

The present study aimed to explore and conceptualise primary school teachers’ rationales for giving students feedback.

Thirteen Swedish primary school teachers ( 10 women and 3 men) with 4 to 40 years of teaching experience working with students aged 7–9 years-old (grades 1–3), participated in the study. An open sampling procedure was adopted to recruit the teachers.

Design and methods

Data were collected using a semi-structured interview approach. We employed a constructivist grounded theory design for the coding and analysis of the transcribed data.

Analysis indicated that two main concerns emerged as regulating teachers’ assessment practices. These addressed what the teachers perceived as (1) students’ academic needs and (2) students’ behavioural and emotional needs. According to the findings, the teachers’ rationales for giving students feedback were based on those needs, and dependent on factors such as situation, relationships, time and effort. This resulted in a constant comparison and weighing of different needs by the teachers. Some needs were described as prioritised before others, which caused some rationales to be identified as taking precedence over others.

Discussion and conclusions

Based on a systematic analysis of – and thus grounded in – interview data from primary teachers, the current qualitative study offers a framework for surveying, understanding and discussing teacher feedback. Overall, the study showed how everyday practices of classroom assessment and classroom management overlapped, thus underlining the importance in teacher education of understanding classroom assessment, classroom management and the relationships between the two.

• Classroom assessment
• feedback rationales
• grounded theory
• interview study
• primary school

Teachers’ actions in social situations are, according to a symbolic interactionist approach (Blumer Citation 1969 ; Charon Citation 2007 ), affected by the teachers’ perspectives on the situation and the meaning that they derive from them. ‘Humans act in a world they define, and although there may actually be a reality out there, their definition is far more important for what they do’ (Charon Citation 2007 , 136). With reference to Charon ( Citation 2007 ), it is therefore imperative to understand teacher feedback from the perspective of the teacher. How teachers define the situation and rationalise what is needed are central to how they give feedback. It is, therefore, important to investigate this in order to understand more clearly the everyday teacher feedback situation in the classroom. When included in teachers’ dialogic repertoire and everyday classroom assessment practices, feedback can be seen as an important part of formative assessment. As such, feedback can then be discussed in terms of effectiveness, depending on its level of formative function (e.g. Gamlem and Munthe Citation 2014 ; Jonsson, Lundahl, and Holmgren Citation 2015 ; See, Gorard, and Siddiqui Citation 2016 ). It is evident that, no matter how teachers carry out classroom assessment, it will affect students’ learning (Popham Citation 2009 ) and self-esteem (Gipps Citation 1995 ). Hargreaves ( Citation 2011 ) found that teachers perceived feedback as most effective when there is trust in the teacher-student relationship, the feedback clearly relates to progress and criteria, and students fully understand the feedback. Research on feedback related to formative assessment has often drawn attention to how teacher feedback is associated with learning outcomes and academic achievement (Brookhart Citation 2012 ; Hattie and Timperley Citation 2007 ). There have been some large studies within the assessment for learning field, in the UK (Black et al. Citation 2003 ), in Scotland (Hayward Citation 2015 ) and in Norway (Hopfenbeck et al. Citation 2013 ). It has been noted that balancing a formative approach with accountability demands was something teachers found challenging (Hayward Citation 2015 ), and some teachers felt that developing their formative practices required a change in view on teaching (Black et al. Citation 2003 ).

Studies based on classroom observation offer the opportunity to analyse, in detail, different dimensions of teacher feedback. Hargreaves ( Citation 2014 ) identifies and describes categories of teachers’ autonomy-promoting feedback, noting how teacher feedback can potentially support children’s development as independent learners. Elsewhere, Boistrup ( Citation 2015 ) construed feedback discourses including on the one hand, ‘do it quick and do it right’ – where the feedback focus was on matters such as correct answers or the number of tasks accomplished by the student – and, on the other, ‘reasoning takes time’, which was characterised by advocating a slower pace and a focus on content and processes within the school subject. Some of the classroom assessment described in the literature is informal, implicit and often non-verbal (Jordan and Putz Citation 2004 ), as is much of the feedback communicated in dialogues between teacher and student (Ruiz-Primo Citation 2011 ). Research on feedback is not, though, restricted to aspects of academic performance. Studies of the link between feedback and student behaviour go back at least as far as White ( Citation 1975 ), who found teacher approval as most common for addressing instructional behaviour and almost non-existent for addressing managerial behaviour. Later research on feedback on behaviour often highlighted approval and disapproval (e.g. Swinson and Knight Citation 2007 ), and specifically the use of praise (e.g. Brophy Citation 1981 ; Chalk and Bizo Citation 2004 ). Indeed, Torrance and Pryor ( Citation 1998 ) described teachers’ everyday assessment practices as including assessment on behavioural performances as well as academic achievements. Tunstall and Gsipps ( Citation 1996 ), in their typology, described assessment-related feedback as being about classroom and individual management as well as performance, mastery and learning orientation.

There are various definitions of the term ‘feedback’. In the present study, we adopt a broad understanding of feedback, including what Hattie and Timperley ( Citation 2007 ) label ‘feed back’ (response on performances), ‘feed up’ (goal-oriented feedback) and ‘feed forward’ (feedback about the next step). Feedback is defined as interpersonal, communicating responses to performances, academic and behavioural.

The present study: aims and purpose

From the existing literature, it is evident that much has been discussed about how feedback is communicated, how feedback can be enhanced, different feedback focuses and the effects of interventions. However, research specifically concerning how teachers conceptualise feedback (e.g. Brown, Harris, and Harnett Citation 2012 ), and, hence, studies focussing on teachers’ feedback rationales that give teachers the opportunity to discuss their own understanding of and perspectives on feedback in their own words, seem limited.

In line with a symbolic interactionist framework (Blumer Citation 1969 ; Charon Citation 2007 ), we viewed it as essential to study the teachers’ ways of reasoning about their classroom feedback to their students, in order to enhance our knowledge about teacher feedback in the everyday classroom. We adopted a grounded theory methodology, since its explorative approach is especially suitable for studying topics where studies are limited. We asked teachers to rationalise the feedback strategies they use in classroom interactions with students, and then analysed their narratives. The findings offer a framework for further analyses of and reflections on teacher rationales for giving students feedback, both for researchers and teachers.

Interviews were carried out and grounded theory analysis was conducted in order to examine, qualitatively, the teachers’ rationales for giving students feedback. As Morcom ( Citation 2014 ) stated, ‘qualitative research methodology endeavours to understand the world of the participant by situating the researcher with all their values and assumptions in that world’ (21).

Ethical considerations

We have carefully followed the ethical principles and guidelines as stated by The Swedish Research Council ( Citation 2017 ). We obtained informed consent from all participants in the study. They were also informed that they had the option of withdrawing from the study at any time. In order to ensure confidentiality for the participants, all participants, including people mentioned in interviews, and schools and locations, have been given fictitious names in transcriptions, field notes and publications.

Participants

Thirteen teachers ( 10 women and 3 men) with 4 to 40 years of teaching, working in Swedish primary schools with students aged 7–9 years-old (grades 1–3), participated in the study. An open sampling procedure was adopted to recruit the teachers, as it ‘seeks to maximise variations in experiences and descriptions using participants from contrasting milieus and backgrounds’ (Hallberg Citation 2006 , 143). In line with this quotation, we wanted to discover and categorise phenomena common among participants from a broad range of contexts. The participating teachers taught in 11 different schools, in 8 different municipalities. Some worked in larger inner-city schools and others in small rural schools, in schools in low and mixed socio-economical areas, in schools with ethnically mixed student groups and schools with almost only students from non-ethnic minority backgrounds. In this way, the data were based on a diverse student population: socio-economically, ethnically and socio-geographically.

Data collection

Interviews were conducted by the first author. Field notes, classroom observations and classroom recordings from four of the teachers’ classrooms, analysed in a previous study (Eriksson, Boistrup, and Thornberg Citation 2017 ), served as a source in the process of constructing interview questions. The four teachers from Eriksson, Boistrup, and Thornberg ( Citation 2017 ) were interviewed first; two of them together, the others individually. During analysis, new questions arose, which led to additional interviews with three of the teachers. These interviews were supplemented by interviews with nine additional teachers. The interviews were conducted in Swedish. In total, 15 semi-structured interviews were conducted, with an average length of 55 min. Coding and analysis, carried out in parallel with the data collection, resulted in additional interview questions and re-interviews in order to examine the constructed codes and categories (this iterative procedure is described as theoretical sampling in grounded theory, see Charmaz Citation 2014 ).

In the interviews, the teachers were asked: in what situations they perceived that they gave students feedback, how they described their feedback in terms of aim, focus, strategies and priorities, and what things they saw as affecting their feedback. The interviewer used probing and follow-up questions (e.g. ‘How come?’, ‘What do you mean?’ and ‘Tell me more’) and took a non-judgemental approach (e.g. Kvale and Brinkmann Citation 2009 ). In the interviews with the first four teachers, some questions were situated, linked to classroom feedback that had been observed. Each interview was recorded and transcribed verbatim. Quotations from the transcribed data presented in the findings have been translated from Swedish into English by the authors.

Data analysis

The data analysis was guided by grounded theory methods (Charmaz Citation 2014 ; Glaser and Strauss Citation 1967 ). Analysis was accomplished by initial and focused coding (Charmaz Citation 2014 ), theoretical coding (Glaser Citation 1978 ), constant comparisons, theoretical sampling and memo writing (Charmaz Citation 2014 ; Glaser Citation 1978 ; Glaser and Strauss Citation 1967 ). While adopting grounded theory, we tried to encounter the data with what can be described as an open mind rather than an empty head (Dey Citation 1993 ). This meant that we initially tried to put what we already knew aside, while staying close to data during the analysis. In this way, we could construct codes that were not only derived from data, but were also realised in wordings that were authentic in relation to the data. In a later part of analysis, comparisons were made with previous research. Thus, constant comparison was made within the interview data, and between codes and categories.

In line with a constructivist grounded theory approach, symbolic interactionism was used as an open-ended theoretical perspective combined with curiosity and openness (see Charmaz Citation 2014 ). Accordingly, definition of situation was used in terms of how the participants interpret a situation they are in which influences their actions (Thomas and Thomas and Thomas Citation 1928 ); main concern addresses what the participants are mainly occupied with (Glaser Citation 1978 ) in relation to feedback; and rationales were regarded as the participants’ motives or logical explanations for acting in different ways (Alvarado Citation 2003 ). We also adopted various categories of teacher feedback strategies, including deliberating , i.e. promoting explorative thinking and discussions; mirroring , i.e. a neutral, non-evaluating response, merely confirming having noticed a strategy; and steering , i.e. communicating that there is a desirable strategy or answer, developed by Eriksson, Boistrup, and Thornberg ( Citation 2017 ) as sensitising concepts (cf. Blumer Citation 1969 ). In line with a constructivist grounded theory approach, we have used the literature as a possible source of inspiration, creative associations, critical reflections and multiple lenses (Thornberg Citation 2012 ).

After six interviews, we had constructed a set of focused codes, which were further elaborated by the three interviews that followed, and strengthened and confirmed by the last six interviews, ensuring saturation and trustworthiness. The different categories of feedback rationales occurred in all participating teachers’ descriptions, with slight variations in frequency. Of course, it must be recognised that the particular schools that the teachers worked at might have affected how frequently different feedback rationales were used. That was, however, not the study’s focus, as we were interested in describing the common feedback rationales the teachers used.

From the data analysis, we concluded that teachers’ feedback practices depended on what they interpreted as perceived needs . We defined perceived needs as any need that the teachers described that they had to address with feedback, in the classroom situation. Thus, it seemed that teachers’ main concern in their feedback practice was to meet perceived needs in the ongoing classroom situation. In line with grounded theory, the terms used to describe the feedback rationales were constructed from the data as such. Consequently, they are not terms already established within the research field. In this way, the findings clearly address participating teachers ’ rationales, providing authenticity in the sense of a grounded theory approach.

Two patterns of general perceived needs for giving feedback

Two recurrent patterns of general perceived needs for giving feedback to students were constructed and coded as two main categories representing different kinds of needs: (a) students’ academic needs and (b) students’ behavioural and emotional needs. Both categories of needs could be perceived by teachers to be at an individual level as well as group level. They evoked a variety of feedback rationales that influenced and guided teachers’ feedback practice, as well as the choice of necessary feedback strategies.

Isabelle has extremely low self-esteem, although she is one of the best in the class in algebra. But she has no belief in herself. She also has really low status in the group. So, I am struggling with what kind of support might help her. (Alex)

Because a prototype is presented as a set of typical characteristics of a category, membership of a category was a question of the degree of family resemblance to the prototype rather than sharing the full set of common features (Dey Citation 1999 ). Therefore, overlaps between categories were possible in our analysis, which reflected greater sensitivity to the nuanced complexity of teachers’ feedback rationales. As mentioned above, teachers’ feedback rationales were analysed into the two main categories of student needs depending on how the needs were emphasised by the teachers in relation to the various feedback rationales. The excerpts chosen to illustrate a certain category of teachers’ feedback rationales may come from a single or just a few teachers, but are broadly representative of the participants as a group. Although all categories were present in all teacher interviews, sometimes a category would be clearly evident and at other times it maybe linked to other categories, which made some excerpts more suitable than others for exemplification purposes.

Pattern 1: addressing academic needs

Academic needs were related to students’ skills, knowledge, learning process and the ability to sustain concentration on school subject activities. This main concern addressed what kind of feedback would best fit the group, or individual students, in order to address and optimise academic learning and activities. The teachers’ perceptions of their students’ knowledge tended to set parameters for how they planned their lessons and teaching, including their rationales for feedback.

Feedback rationale (1): academic encouragement

When using the feedback rationale identified as academic encouragement , the teacher viewed the student as someone in need of support and encouragement in order to progress. This rationale was mostly associated with giving praise (e.g. ‘For those who are unconfident, you have to give praise for every sentence or letter’ (Alex)), and ‘mirroring’, which, as explained earlier, refers to a neutral, non-evaluating response, confirming that the teacher has noticed what kind of strategy was used or answer the student has given (e.g. ‘They need to feel noticed in order to push themselves into doing more’ (Chris)). Thus, the teachers expressed an encouraging approach in which they recognised and showed care more frequently to those students who needed it the most by praising and encouraging their academic achievements regarding the amount of work done and its quality. As one teacher, Tove, explained, ‘They would need someone beside them. Someone who confirms that I [the student] am doing things correctly, or someone that gives a slight push’.

They need to feel that there is someone there to help them. / – / Without, for that matter, coddling them, telling them that ‘yes but, well it’s no big deal, but it’s a pity that you find it difficult’. But rather that ‘we can do it, if we decide we can do it, but it will require hard work.’(Mio)

Feedback rationale (2): individualising

meeting the students and giving them feedback related to where they are in their learning process. / – / Pointing out what is good and then ‘Well, how can you proceed?’ / – / And then they can discover it by themselves. (Robin)

I can’t tell Tomas that he has done a good job, if it is not good based on his [work], You can’t compare Tomas with someone else [another student]. You must compare Tomas with what he has achieved earlier. (Toni)

Feedback rationale (3): peer learning modelling

When adopting the rationale peer learning modelling for giving feedback, the teacher recognised desirable knowledge and skills among the students. They did this by highlighting a student’s or a group of students’ performance or learning activity as a model for the others, or for a specific student – in other words, drawing students’ attention to those who performed well or better, in the hope that this might lead them to adopt the model themselves, thereby learning from their peers. As Kirsten explained, ‘they need to get some sort of model. Otherwise they are supposed to guess what it might look like’.

The rationale was also evident when teachers merely underlined the advantages of reviewing others’ work and working with peers. As Alex put it, it was important ‘To change focus from me [the teacher] as the only one who knows’; Pim similarly observed, ‘Because it’s not as if it [the feedback] always has to come from me’. Mio explained the advantages of using peer learning modelling in this way: ‘You may need different ways of saying something. And then it might be the way the peer said it that made them learn quicker’. The teachers argued that, by seeing peers as ‘knowers’, the students would, to greater extent, seek help from one another.

Furthermore, the teachers considered peer learning modelling as a possible source of inspiration and motivation. For instance, Toni stated, ‘Hopefully they became a bit inspired by each other to continue working on their texts’, and added that the students seemed to like reading their texts to their peers. Kirsten, in turn, described the students as more motivated to do a task ‘if they know that it has a recipient’. Thus, peer learning modelling was considered to be stimulating and helpful to both the observer and the model.

Feedback rationale (4): task controlling

If you lose grip of the group during a conversation, and it [the discussion] takes off in a different direction–, I think it is about how the group works as a group. With some groups, you can have those deliberating conversations and stick to your intentional idea. (Alex)

Feedback rationale (5): classwork flowing

So, then the feedback perhaps might not be so well thought out. You might not have the time, or take the time to reflect for yourself, or be that accurate and specific and calmly make time to say what you want to say to the child you are standing beside.

The need to be time-efficient, by giving short responses to as many students as possible, both demotivated and dissuaded the teachers from using the more time-consuming and meta-cognitive supportive ‘deliberating’ as feedback strategy. Furthermore, if teachers in such classroom situations adopted deliberating feedback by posing a question that invited the student to reflect, it risked not being followed up by the teacher, who needed to leave them in order to help someone else, and then might not have time to return.

Feedback rationale (6): time-for-reasoning

It does not become this, swoosh (illustrating a constant running between students by waving the arms back and forth), that I have to go there, and then I have to be there, and then I need to go there. Instead, I can sit down with a student and discuss. (Chris)

I find it insufficient merely to look at [the answer]; you have to listen to how they reason. Not just look at–, it is like–, to complete those tasks on time, that is one way. But you want to know how they think too in order to make a correct assessment, I think. Not only that you–, you can’t just move numbers, you have to think–, the students’ strategies for reaching the answer. (Toni)

Pattern 2: addressing behavioural and emotional needs

Behavioural and emotional needs refer to students’ needs related to developing and maintaining desirable behaviour, emotional well-being and adjustment, such as: positive self-esteem, appropriate social norms, positive classroom climate, feeling included in a group or class.

Feedback rationale (1): need-for-order

When it came to the main concern of students’ behavioural and emotional needs, different kinds of need - for - order rationales were emphasised by the teachers. It sometimes had to do with creating order, where different rationales for doing so were highlighted in the teachers’ statements. At other times, it was about maintaining order. In the data analysis, three different need - for - order rationales were constructed. These were termed (a) fire-fighting, (b) peer order modelling and (c) without-order-no-learning.

(a) Fire - fighting

The metaphorical term fire - fighting was used for preventing disorder, or maintaining a rather fragile order in situations where potential disorder was anticipated. Taking action to maintain classroom order was then considered necessary. The teachers explained it by emphasising the need to be one step ahead in order to maintain order in the classroom. This rationale could take the form of the teacher preparing specific students beforehand in terms of what the lesson will be about; for example, ‘Then I’ll have to catch them on the way in. Tell them what’s going to happen now’ (Pim). This need - for - order rationale was, however, mostly described as the teacher needing to be there the moment a student has finished a task, to give feedback directly on the performance and directing the student towards the next task. Vanja said that ‘Sometimes it’s about whether I should decide that it becomes chaotic, or shall I decide to give these children help quickly so that they get started and it doesn’t become chaotic’. The teachers stressed that being one step ahead of some students, helping them before others, might not be fair. But, the thinking was, since chaos was avoided, everyone benefitted from it. Alex, Robin and Pim all described it metaphorically as ‘it’s sometimes all about extinguishing fires’. This rationale also comprised teachers noticing and managing small cases of disorder just after they appeared, preventing them from escalating into wider disorder, perhaps involving more students, and becoming more difficult to control. The teachers described a need for having so-called ‘eyes in the back of the head’. This connects with Kounin’s ( Citation 1970 ) reference to teachers’ ‘with-it-ness’ in the classroom, which thus leads to being alert to signs that they may need to act on.

(b) Peer order modelling

If I give a student a response, like ‘Well, how nice that you show me that you learnt how to raise your hand and be quiet and wait for your turn. Good! That’s why you’re getting to speak now.’ Then I can see how some other student, who is sitting there, talking when they are supposed to be quiet and listening, reacts like ‘Oh!’ (Robin)

(c)Without - order - no - learning

This rationale accentuated the need - for - order as something that had to be fulfilled in order for the students to learn, and was often communicated as a joint class project: ‘Let’s just put down everything and try to make this work, so it doesn’t have to be like this’ (Mio), and, ‘It’s hard for anyone to concentrate on work if the sound level in the classroom is this high’ (Robin). The teachers emphasised the wider possibilities of giving more ‘deliberating’ feedback, promoting explorative discussions and thinking through dialogue when there was order in the classroom, which was confirmed in the classroom observations. When there was disorder in the classroom, a considerable part of the teachers’ focus and feedback efforts were on creating order. Hence, the teachers considered classroom order to be the superior need, because a failure to address this behavioural need at group (classroom) level would severely block their ability to address students’ academic needs as well as other behavioural and emotional needs.

Feedback rationale (2): Caring

There are some that I know that I praise intentionally. Mia is such a girl–, that I need to lift–, and Clara, I have to lift her too. With her, I try to remember to be friendly, extra kind, if it–, and highlighting her answers [during gatherings] as good. (Alex)

The relationships between teachers’ feedback rationales

Depending on how different needs were weighted in relation to each other, and consequently which rationales the teacher decided to act upon, the teacher could choose different feedback strategies. In ongoing classroom situations, teachers had to take into account and coordinate various feedback rationales based on their ongoing assessment of the present academic, emotional and behavioural needs. In this process, there was a constant interplay between feedback rationales, in which without - order - no - learning , classwork flowing , and individualising seemed to be the three primary rationales. The coordination of these three rationales guided teachers in their adoption of the other feedback rationales.

Meeting every student’s individual needs in the moment

The teachers were deeply concerned with teaching, supporting and giving feedback based on students’ individual needs. They described striving to be able to meet all students’ individual needs at the same time. The focus was not limited to academic or behavioural and emotional needs, but rather to all needs identified at a specific moment. Yet, in alignment with the teaching task, this predominantly concerned students’ academic needs. The teachers acted upon striving for this using different rationales: which ones were used depended on which needs were identified. When there was order in the classroom and everyone seemed to be working, there were few acute needs identified. Fulfilling those needs was then seen as rather easily manageable. When there were several hands raised at the same time, the striving for individualising took shape by means of the teacher helping students in pairs or groups, by simultaneously switching between two students, or by rushing between students in order to provide short, specific feedback.

Through analysis, it became apparent that the rationale individualising informed and guided the feedback rationale academic encouragement . The teachers expressed their belief that students performing poorly were in need of more frequent support, while students performing well were not. Individual differences regarding the amount and frequency of feedback needed was a central aspect of teachers’ attempts to meet all students’ needs at the same time. Whereas individualising focused on particular individual needs, time - for - reasoning was guided by a more general idea of pedagogical dialogue as something that helped every student. In fact, according to the teachers, a combination of these rationales – i.e. individualised time - for - reasoning – emerged in the analysis as ‘the gold standard’ rationale of classroom feedback. However, too often it was too difficult to fulfil because of the need to construct classroom order: there being too many needs at the same time, and a lack of time. Teachers’ attempts to meet all students’ individual needs manifested itself in the sense that there were a few main rationales, and at the same time, the rationales that seemed to be needed at that moment were the ones used and seen as equally important when opportunities arose and compromises were few.

However, the teachers drew attention to the difficulty of meeting all students’ individual needs at the same time by emphasising a lack of time and resources and feelings of inadequacy. When teachers found themselves in situations where they perceived too many student needs at the same time, the primary feedback rationale individualising came into conflict with the primary feedback rationale classwork flowing . The perceived problems of not meeting all students’ needs in the moment were: lack of learning – taking into account the students who had to wait too long to get feedback – and the risk of off-task behaviour and classroom disorder. These were problems that, in turn, evoked the primary feedback rationale without - order - no - learning that motivated the teachers to adopt classroom flowing in the moment before individualising . Thus, the rationale classroom flowing seemed to be often driven by both without - order - no - learning and attempts to meet all student needs in the moment.

Superior and subordinate needs

Classroom management can be defined as ‘establishing and maintaining order in a group-based educational system whose goals include student learning as well as social and emotional growth’ (Emmer and Sabornie Citation 2015 , 8). In line with such classroom management literature, teachers were strongly focussed on creating and maintaining classroom order that was conducive to learning: the very core of the feedback rationale without - order - no - learning . Analysis indicated that, although this primary feedback is a need - for - order rationale addressing behavioural needs (counteracting disruptive behaviour and promoting appropriate behaviour in the classroom), a clear category overlap exists. The fulfilment of the behavioural need - for - order was assumed to be a necessary condition for the fulfilment of academic needs. The ideal was a kind of classroom order in which all students were engaged in on-task behaviour: in other words, involved in academic engagement and learning.

The primary feedback rationale without - order - no - learning motivated teachers to adopt classwork flowing and fire - fighting as feedback rationales in order to prevent disorder and off-task behaviour. Thus, they constituted efforts to maintain students’ attention and focus on on-task behaviour in the classroom. The without - order - no - learning rationale also motivated the peer order modelling based on social learning. In addition, when teachers encountered classroom disorder, or perceived in other ways a strong need for constructing classroom order conducive to learning (e.g. interpreted that order was fragile), the without - order - no - learning rationale appeared to overrule feedback rationales such as individualising , academic encouragement , time - for - reasoning , which were aimed at addressing academic needs. Similarly, if the three primary feedback rationales without - order - no - learning , classwork flowing and individualising were perceived to be in conflict with each other in an actual classroom situation, need - for - order rationales and trying to keep all students focused on academic tasks in the moment took precedence over rationales concerning other needs. However, although the without - order - no - learning rationale was a superior rationale concerning behavioural needs, it was in alignment with the teaching task predominantly concerning students’ academic needs, since its aim was to create a better learning environment.

This study explored and conceptualised primary school teachers’ rationales for giving students feedback. Our broad approach was based on a symbolic interactionist framework (Blumer Citation 1969 ; Charon Citation 2007 ), where knowledge about teachers’ perspectives is crucial in order to better understand their actions in everyday situation, such as feedback.

We examined primary teachers’ main concerns and rationales about their everyday classroom feedback practice. Analysis of their reports allowed us to generate a set of feedback rationales. Among these, three feedback rationales were considered to be the most significant: without - order - no - learning , classwork flowing and individualising . The teachers’ main concerns were to address academic needs as well as emotional and behavioural needs. The findings contribute to the field by offering a framework for the further analysis of teacher rationales in relation to feedback.

The reported feedback rationales highlighted teachers’ awareness of the impact that feedback may have on self-esteem (cf. Gipps Citation 1995 ). This awareness was also articulated in their desire to be receptive as to how students interpreted and made use of feedback. This receptiveness was often described as most easily fulfilled when there was order in the classroom, creating opportunity to use rationales such as individualising and time - for - reasoning . At the same time, feedback rationales addressing emotional needs were less described and less elaborated on by the teachers as compared with feedback rationales addressing academic and behavioural needs. Thus, the teachers appeared to be more concerned with academic learning and externalising behavioural problems in the classroom than with internalising emotional problems among the students (cf. Lane et al. Citation 2004 ). In line with Lane et al., our study suggests a need for more elaborated awareness and feedback rationales among teachers regarding students’ emotional needs beyond low self-esteem, since research has found academic achievement to be negatively associated with negative emotions and internalising problems, and positively linked to positive emotions (for a review, see Valiente, Swanson, and Eisenberg Citation 2012 ).

In the current study, we identified an overlap between classroom assessment and classroom management in respect of teachers’ main concerns and rationales for classroom feedback. We found that without - order - no - learning was the dominant feedback rationale that tended to overrule a range of other rationales when classroom order was fragile or had already become disorderly. Order and on-task behaviour was seen as the norm by the teachers, and was understood as the basic classroom condition for learning. It was assumed that making room for an ideal feedback meant focusing upon academic needs and rationales such as time - for - reasoning and individualising . Classroom management has to be understood as integrated with, and part of, classroom instruction (cf. Muijs and Reynolds Citation 2005 ). The current study demonstrates how this integration may happen, through its analysis of primary teachers’ rationales for their everyday classroom feedback.

It is noteworthy that in the rationale classwork flowing , the teacher gave short, specific responses and often tended to use ‘steering’ feedback. This was a time-efficient but instructionally scanty approach that was assumed to cope with a lack of time and a large amount of student need at the same time, and construct classroom order conducive for learning (or at least on-task behaviour). This can be compared to the rationale time - for reasoning , which the teachers described as providing for communication on processes such as inquiring and reasoning, but also as something that was not often achievable. This tension between rationales is similar to the two assessment discourses ‘do it quick and do it right’, and ‘reasoning takes time’ reported by Boistrup ( Citation 2015 ). Hence, although teachers in the current study had very good intentions of giving individualised, deliberative, meta-cognitive supporting and autonomy-promoting feedback (cf. Hargreaves Citation 2014 ), the findings indicated that sometimes teachers could not give feedback in accordance with their primary intentions, since they had different needs to take into consideration. In addition, constructing classroom order might, in particular, overrule such intentions in the actual classroom situation.

Limitations and implications for future research and professional development

In this small scale, qualitative study, the small sample size, of course, limits the transferability of these findings. Nonetheless, possible future research on a larger scale should include teachers at other school levels as well as teachers from different countries. Another important limitation is that the findings do not provide information on how teachers actually act on their feedback rationales, but a constructed conceptualisation of how teachers describe their feedback rationales and how they act on them. Students are represented in the study as described by the teachers. Students’ need for feedback as presented is, thus, necessarily an image of how teachers perceive them. Teacher feedback is, particularly from a symbolic interactionist view, a complex interactional pattern between teachers and students, meaning that it is crucial to examine students’ perspectives to better understand teacher feedback. However, social research is always an issue of, and limited to, perspectives (Charon Citation 2007 ). Larsson ( Citation 2009 ) talks about ‘generalisation through recognition of patterns’ in which the reader judges the generalisability. In addition, Glaser ( Citation 1978 ) argues that findings are not a fixed end-point, but are constantly open for modification as new data are gathered. In accordance with constructivist grounded theory research (Charmaz Citation 2014 ), we do not claim to offer an exact picture – but, rather, an interpretative portrayal of the phenomenon studied.

Despite the limitations and cautions considering generalisability mentioned above, we suggest that the findings have implications for teachers and teacher education. The study contributes to the literature as well as to teachers and teacher educators by presenting a systematic conceptualisation of teachers’ rationales for giving students feedback in the classroom. Based on systematic analysis of – and thus grounded in – qualitative interview data with primary teachers, the current study offers a framework for surveying, understanding and discussing teacher feedback. It uses some explicit concepts that maybe helpfully included in teachers’ professional language, in terms of their teaching practices of classroom assessment and feedback. Furthermore, and in line with Torrance and Pryor ( Citation 1998 ) and Tunstall and Gsipps ( Citation 1996 ), our findings showed that the everyday practices of classroom assessment and classroom management overlapped, which, in turn, underlines the importance of both classroom assessment training (cf. Black and Wiliam Citation 2009 ) and classroom management training (cf. Dicke et al. Citation 2015 ) in teacher education. In order to support professional development, teachers and those involved in teacher development may find it valuable to examine the overlaps and integration between them with reference to the rationales and feedback strategies set out and discussed in our study.

No potential conflict of interest was reported by the authors.

• Alvarado, José Luis . 2003. “Teachers’ Rationale and Procedures Used during Lesson Script Deviations.” Behavioral Disorders 28 (4): 370–385 http://www.jstor.org/stable/23889206 .   Web of Science ® Google Scholar
• Bandura, Albert . 1977. Social Learning Theory . Englewood Cliffs, NJ: Prentice Hall.   Google Scholar
• Black, Paul , and Dylan Wiliam . 2009. “Assessment and Classroom Learning.” Assessment in Education 5 ( 1 ): 7–74. doi:10.1080/0969595980050102.   Google Scholar
• Black, Paul , Christine Harrison , Clare Lee , Bethan Marshall , and Dylan Wiliam . 2003. Assessment for Learning: Putting It into Practice . Maidenhead: Open University Press.   Google Scholar
• Blumer, Herbert . 1969. Symbolic Interactionism . Berkeley: University of California Press.   Google Scholar
• Boistrup, Lisa Björklund . 2015. “Governing through Implicit and Explicit Assessment Acts: Multimodality in Mathematics Classrooms.” In Negotiating Spaces for Literacy Learning: Multimodality and Governmentality , edited by Mary Hamilton , Rachel Heydon , Kathryn Hibbert and Roz Stooke , 131–148. London: Bloomsbury Publishing.   Google Scholar
• Brookhart, Susan M. 2012. “Teacher Feedback in Formative Classroom Assessment.” In Leading Student Assessment: Studies in Educational Leadership, 15 , edited by Charles F. Webber and Judy L. Lupart , 225–239. Dordrecht: Springer.   Google Scholar
• Brophy, Jere . 1981. “Teacher Praise: A Functional Analysis.” Review of Educational Research 51 ( 1 ): 5–32. doi:10.3102/00346543051001005.   Web of Science ® Google Scholar
• Brown, Gavin T. L. , Lois R. Harris , and Jennifer Harnett . 2012. “Teacher Beliefs about Feedback within an Assessment for Learning Environment: Endorsement of Improved Learning over Student Well-Being.” Teaching and Teacher Education 28 ( 7 ): 968–978. doi:10.1016/j.tate.2012.05.003.   Web of Science ® Google Scholar
• Chalk, Karen , and Lewis A. Bizo . 2004. “Specific Praise Improves on‐Task Behaviour and Numeracy Enjoyment: A Study of Year Four Pupils Engaged in the Numeracy Hour.” Educational Psychology in Practice 20 ( 4 ): 335–351. doi:10.1080/0266736042000314277.   Google Scholar
• Charmaz, Kathy . 2014. Constructing Grounded Theory . 2nd ed. London: Sage.   Google Scholar
• Charon, Joel M. 2007. Symbolic Interactionism: An Introduction, an Interpretation, an Integration . 9th ed. Upper Saddle River, NJ: Prentice Hall.   Google Scholar
• Dey, Ian . 1993. Qualitative Data Analysis: A User-Friendly Guide for Social Scientists . London: Routledge.10.4324/9780203412497   Google Scholar
• Dey, Ian . 1999. Grounding Grounded Theory: Guidelines for Qualitative Inquiry . San Diego, CA : Academic Press.10.1016/B978-012214640-4/50011-5   Google Scholar
• Dicke, Theresa , Jill Elling , A. Annett Schmeck , and Detlev Leutner . 2015. “ Reducing Reality Shock: The Effects of Classroom Management Skills Training on Beginning Teachers .” Teaching and Teacher Education 48 : 1–12. doi:10.1016/j.tate.2015.01.013.   Web of Science ® Google Scholar
• Emmer, Edmund T. , and Edward J. Sabornie , eds. 2015. Handbook of Classroom Management . 2nd ed. New York: Routledge.   Google Scholar
• Eriksson, Elisabeth , Lisa Björklund Boistrup , and Robert Thornberg . 2017. “ A Categorisation of Teacher Feedback in the Classroom: A Field Study on Feedback Based on Routine Classroom Assessment in Primary School .” Research Papers in Education 32 ( 3 ): 316–332. doi:10.1080/02671522.2016.1225787.   Web of Science ® Google Scholar
• Gamlem, Siv M. , and Elaine Munthe . 2014. “Mapping the Quality of Feedback to Support Students’ Learning in Lower Secondary Classrooms.” Cambridge Journal of Education 44 ( 1 ): 75–92. doi:10.1080/0305764X.2013.855171.   Web of Science ® Google Scholar
• Gipps, Caroline . 1995. Beyond Testing. towards a Theory of Educational Assessment . London: The Falmer Press.   Google Scholar
• Glaser, Barney . 1978. Theoretical Sensitivity: Advances in the Methodology of Grounded Theory . Mill Valley, CA: Sociological Press.   Google Scholar
• Glaser, Barney , and Anselm Strauss . 1967. The Discovery of Grounded Theory: Strategies for Qualitative Research . New Brunswick, NJ: Aldine Transaction.   Google Scholar
• Hallberg, Lillemor R.-M. 2006. “The ‘Core Category’ of Grounded Theory: Making Constant Comparisons.” International Journal of Qualitative Studies on Health and Well-Being 1 ( 3 ): 141–148. doi:10.1080/17482620600858399.   Google Scholar
• Hargreaves, Eleaonore . 2011. “Teachers’ Feedback to Pupils: ‘like So Many Bottles Thrown out to Sea?’.” In Assessment Reform in Education – Policy and Practice , edited by Rita Berry and Bob Adamson , 121–133. Dordrecht: Springer. doi:10.1007/978-94-007-0729-0_9.   Google Scholar
• Hargreaves, Eleanore . 2014. “The Practice of Promoting Primary Pupils’ Autonomy: Examples of Teacher Feedback.” Educational Research 56 ( 3 ): 295–309. doi:10.1080/00131881.2014.934554.   Web of Science ® Google Scholar
• Hattie, John , and Helen Timperley . 2007. “The Power of Feedback.” Review of Educational Research 77 ( 1 ): 81–112. doi:10.3102/003465430298487.   Web of Science ® Google Scholar
• Hayward, Louise . 2015. “Assessment is Learning: The Preposition Vanishes.” Assessment in Education: Principles, Policy and Practice 22 ( 1 ): 27–43. doi:10.1080/0969594X.2014.984656.   Web of Science ® Google Scholar
• Hopfenbeck, Therese , Astrid Tolo , Teresa Florez , and Yasmine El Masri . 2013. Balancing Trust and Accountability? The Assessment for Learning Programme in Norway . Paris: OECD. doi:10.1787/5k3txnpqlsnn-en.   Google Scholar
• Jonsson, Anders , Christian Lundahl , and Anders Holmgren . 2015. “Evaluating a Large-Scale Implementation of Assessment for Learning in Sweden.” Assessment in Education: Principles, Policy & Practice 22 ( 1 ): 104–121. doi:10.1080/0969594X.2014.970612.   Web of Science ® Google Scholar
• Jordan, Brigitte , and Peter Putz . 2004. “Assessment as Practice: Notes on Measures, Tests, and Targets.” Human Organization 63 ( 3 ): 346–358. doi:10.17730/humo.63.3.yj2w5y9tmblc422k.   Web of Science ® Google Scholar
• Kounin, Jacob S. 1970. Discipline and Group Management in Classrooms . New York: Holt, Rineheart & Winston.   Google Scholar
• Kvale, Steinar , and Svend Brinkmann . 2009. Interviews: Learning the Craft of Qualitative Research Interviewing . London: Sage.   Google Scholar
• Lane, Kathleen L. , Melinda R. Pierson , E. Jemma Robertson , and Annette Little . 2004. “Teachers’ Views of Prereferral Interventions: Perceptions of and Recommendations for Implementation Support.” Education & Treatment of Children 27 (4): 420–439.   Google Scholar
• Larsson, Staffan . 2009. “A Pluralist View of Generalization in Qualitative Research.” International Journal of Research & Method in Education 32 ( 1 ): 25–38. doi:10.1080/17437270902759931.   Google Scholar
• Morcom, Veronica . 2014. “Scaffolding Social and Emotional Learning in an Elementary Classroom Community: A Sociocultural Perspective.” International Journal of Educational Research 67 : 18–29. doi:10.1016/j.ijer.2014.04.002.   Web of Science ® Google Scholar
• Muijs, Daniel , and David Reynolds . 2005. Effective Teaching: Evidence and Practice . 2nd ed. London: Sage.   Google Scholar
• Popham, W. James . 2009. “Assessment Literacy for Teachers: Faddish or Fundamental?” Theory into Practice 48 : 4–11. doi:10.1080/00405840802577536.   Web of Science ® Google Scholar
• Ruiz-Primo, Maria Araceli . 2011. “Informal Formative Assessment: The Role of Instructional Dialogues in Assessing Students’ Learning.” Studies in Educational Evaluation 37 ( 1 ): 15–24. doi:10.1016/j.stueduc.2011.04.003.   Google Scholar
• See, Beng Huat , Stephen Gorard , and Nadia Siddiqui . 2016. “Teachers’ Use of Research Evidence in Practice: A Pilot Study of Feedback to Enhance Learning.” Educational Research 58 ( 1 ): 56–72. doi:10.1080/00131881.2015.1117798.   Web of Science ® Google Scholar
• Swinson, Jeremy , and R. Knight . 2007. “Teacher Verbal Feedback Directed toward Secondary Students with Challenging Behaviour and Its Relationship to Their Behaviour.” Educational Psychology in Practice 23 ( 3 ): 241–255. doi:10.1080/02667360701507327.   Google Scholar
• The Swedish Research Council . 2017. CODEX: Regler och riktinjer för forskning [CODEX: Rules and guidelines for research]. http://www.codex.vr.se .   Google Scholar
• Thomas, William I. , and Dorothy Swaine Thomas . 1928. The Child in America . New York: Knopf.   Google Scholar
• Thornberg, Robert . 2012. “Informed Grounded Theory.” Scandinavian Journal of Educational Research 56 ( 3 ): 243–259. doi:10.1080/00313831.2011.581686.   Web of Science ® Google Scholar
• Torrance, Harry , and John Pryor . 1998. Investigating Formative Assessment: Teaching, Learning and Assessment in the Classroom . Buckingham: Open University Press.   Google Scholar
• Tunstall, Pat , and Caroline Gsipps . 1996. “Teacher Feedback to Young Children in Formative Assessment: A Typology.” British Educational Research Journal 22 ( 4 ): 389– 404 . doi:10.1080/0141192960220402.   Google Scholar
• Valiente, Carlos , Jodi Swanson , and Nancy Eisenberg . 2012. “Linking Students’ Emotions and Academic Achievement: When and Why Emotions Matter.” Child Development Perspectives 6 ( 2 ): 129–135. doi:10.1111/j.1750-8606.2011.00192.x.   PubMed Web of Science ® Google Scholar
• White, Mary Alice . 1975. “Natural Rates of Teacher Approval and Disapproval in the Classroom.” Journal of Applied Behavior Analysis 8 ( 4 ): 367–372. doi:10.1901/jaba.1975.8-367.   PubMed Web of Science ® Google Scholar
• Back to Top

Related research

People also read lists articles that other readers of this article have read.

Recommended articles lists articles that we recommend and is powered by our AI driven recommendation engine.

Cited by lists all citing articles based on Crossref citations. Articles with the Crossref icon will open in a new tab.

• People also read
• Recommended articles

To cite this article:

Download citation, your download is now in progress and you may close this window.

• Choose new content alerts to be informed about new research of interest to you
• Easy remote access to your institution's subscriptions on any device, from any location
• Save your searches and schedule alerts to send you new results
• Export your search results into a .csv file to support your research

Login or register to access this feature

Register now or learn more

• Research article
• Open access
• Published: 21 November 2018

Characterising and justifying sample size sufficiency in interview-based studies: systematic analysis of qualitative health research over a 15-year period

• Konstantina Vasileiou   ORCID: orcid.org/0000-0001-5047-3920 1 ,
• Julie Barnett 1 ,
• Susan Thorpe 2 &
• Terry Young 3  

BMC Medical Research Methodology volume  18 , Article number:  148 ( 2018 ) Cite this article

764k Accesses

1307 Citations

184 Altmetric

Metrics details

Choosing a suitable sample size in qualitative research is an area of conceptual debate and practical uncertainty. That sample size principles, guidelines and tools have been developed to enable researchers to set, and justify the acceptability of, their sample size is an indication that the issue constitutes an important marker of the quality of qualitative research. Nevertheless, research shows that sample size sufficiency reporting is often poor, if not absent, across a range of disciplinary fields.

A systematic analysis of single-interview-per-participant designs within three health-related journals from the disciplines of psychology, sociology and medicine, over a 15-year period, was conducted to examine whether and how sample sizes were justified and how sample size was characterised and discussed by authors. Data pertinent to sample size were extracted and analysed using qualitative and quantitative analytic techniques.

Our findings demonstrate that provision of sample size justifications in qualitative health research is limited; is not contingent on the number of interviews; and relates to the journal of publication. Defence of sample size was most frequently supported across all three journals with reference to the principle of saturation and to pragmatic considerations. Qualitative sample sizes were predominantly – and often without justification – characterised as insufficient (i.e., ‘small’) and discussed in the context of study limitations. Sample size insufficiency was seen to threaten the validity and generalizability of studies’ results, with the latter being frequently conceived in nomothetic terms.

Conclusions

We recommend, firstly, that qualitative health researchers be more transparent about evaluations of their sample size sufficiency, situating these within broader and more encompassing assessments of data adequacy . Secondly, we invite researchers critically to consider how saturation parameters found in prior methodological studies and sample size community norms might best inform, and apply to, their own project and encourage that data adequacy is best appraised with reference to features that are intrinsic to the study at hand. Finally, those reviewing papers have a vital role in supporting and encouraging transparent study-specific reporting.

Peer Review reports

Sample adequacy in qualitative inquiry pertains to the appropriateness of the sample composition and size . It is an important consideration in evaluations of the quality and trustworthiness of much qualitative research [ 1 ] and is implicated – particularly for research that is situated within a post-positivist tradition and retains a degree of commitment to realist ontological premises – in appraisals of validity and generalizability [ 2 , 3 , 4 , 5 ].

Samples in qualitative research tend to be small in order to support the depth of case-oriented analysis that is fundamental to this mode of inquiry [ 5 ]. Additionally, qualitative samples are purposive, that is, selected by virtue of their capacity to provide richly-textured information, relevant to the phenomenon under investigation. As a result, purposive sampling [ 6 , 7 ] – as opposed to probability sampling employed in quantitative research – selects ‘information-rich’ cases [ 8 ]. Indeed, recent research demonstrates the greater efficiency of purposive sampling compared to random sampling in qualitative studies [ 9 ], supporting related assertions long put forward by qualitative methodologists.

Sample size in qualitative research has been the subject of enduring discussions [ 4 , 10 , 11 ]. Whilst the quantitative research community has established relatively straightforward statistics-based rules to set sample sizes precisely, the intricacies of qualitative sample size determination and assessment arise from the methodological, theoretical, epistemological, and ideological pluralism that characterises qualitative inquiry (for a discussion focused on the discipline of psychology see [ 12 ]). This mitigates against clear-cut guidelines, invariably applied. Despite these challenges, various conceptual developments have sought to address this issue, with guidance and principles [ 4 , 10 , 11 , 13 , 14 , 15 , 16 , 17 , 18 , 19 , 20 ], and more recently, an evidence-based approach to sample size determination seeks to ground the discussion empirically [ 21 , 22 , 23 , 24 , 25 , 26 , 27 , 28 , 29 , 30 , 31 , 32 , 33 , 34 , 35 ].

Focusing on single-interview-per-participant qualitative designs, the present study aims to further contribute to the dialogue of sample size in qualitative research by offering empirical evidence around justification practices associated with sample size. We next review the existing conceptual and empirical literature on sample size determination.

Sample size in qualitative research: Conceptual developments and empirical investigations

Qualitative research experts argue that there is no straightforward answer to the question of ‘how many’ and that sample size is contingent on a number of factors relating to epistemological, methodological and practical issues [ 36 ]. Sandelowski [ 4 ] recommends that qualitative sample sizes are large enough to allow the unfolding of a ‘new and richly textured understanding’ of the phenomenon under study, but small enough so that the ‘deep, case-oriented analysis’ (p. 183) of qualitative data is not precluded. Morse [ 11 ] posits that the more useable data are collected from each person, the fewer participants are needed. She invites researchers to take into account parameters, such as the scope of study, the nature of topic (i.e. complexity, accessibility), the quality of data, and the study design. Indeed, the level of structure of questions in qualitative interviewing has been found to influence the richness of data generated [ 37 ], and so, requires attention; empirical research shows that open questions, which are asked later on in the interview, tend to produce richer data [ 37 ].

Beyond such guidance, specific numerical recommendations have also been proffered, often based on experts’ experience of qualitative research. For example, Green and Thorogood [ 38 ] maintain that the experience of most qualitative researchers conducting an interview-based study with a fairly specific research question is that little new information is generated after interviewing 20 people or so belonging to one analytically relevant participant ‘category’ (pp. 102–104). Ritchie et al. [ 39 ] suggest that studies employing individual interviews conduct no more than 50 interviews so that researchers are able to manage the complexity of the analytic task. Similarly, Britten [ 40 ] notes that large interview studies will often comprise of 50 to 60 people. Experts have also offered numerical guidelines tailored to different theoretical and methodological traditions and specific research approaches, e.g. grounded theory, phenomenology [ 11 , 41 ]. More recently, a quantitative tool was proposed [ 42 ] to support a priori sample size determination based on estimates of the prevalence of themes in the population. Nevertheless, this more formulaic approach raised criticisms relating to assumptions about the conceptual [ 43 ] and ontological status of ‘themes’ [ 44 ] and the linearity ascribed to the processes of sampling, data collection and data analysis [ 45 ].

In terms of principles, Lincoln and Guba [ 17 ] proposed that sample size determination be guided by the criterion of informational redundancy , that is, sampling can be terminated when no new information is elicited by sampling more units. Following the logic of informational comprehensiveness Malterud et al. [ 18 ] introduced the concept of information power as a pragmatic guiding principle, suggesting that the more information power the sample provides, the smaller the sample size needs to be, and vice versa.

Undoubtedly, the most widely used principle for determining sample size and evaluating its sufficiency is that of saturation . The notion of saturation originates in grounded theory [ 15 ] – a qualitative methodological approach explicitly concerned with empirically-derived theory development – and is inextricably linked to theoretical sampling. Theoretical sampling describes an iterative process of data collection, data analysis and theory development whereby data collection is governed by emerging theory rather than predefined characteristics of the population. Grounded theory saturation (often called theoretical saturation) concerns the theoretical categories – as opposed to data – that are being developed and becomes evident when ‘gathering fresh data no longer sparks new theoretical insights, nor reveals new properties of your core theoretical categories’ [ 46 p. 113]. Saturation in grounded theory, therefore, does not equate to the more common focus on data repetition and moves beyond a singular focus on sample size as the justification of sampling adequacy [ 46 , 47 ]. Sample size in grounded theory cannot be determined a priori as it is contingent on the evolving theoretical categories.

Saturation – often under the terms of ‘data’ or ‘thematic’ saturation – has diffused into several qualitative communities beyond its origins in grounded theory. Alongside the expansion of its meaning, being variously equated with ‘no new data’, ‘no new themes’, and ‘no new codes’, saturation has emerged as the ‘gold standard’ in qualitative inquiry [ 2 , 26 ]. Nevertheless, and as Morse [ 48 ] asserts, whilst saturation is the most frequently invoked ‘guarantee of qualitative rigor’, ‘it is the one we know least about’ (p. 587). Certainly researchers caution that saturation is less applicable to, or appropriate for, particular types of qualitative research (e.g. conversation analysis, [ 49 ]; phenomenological research, [ 50 ]) whilst others reject the concept altogether [ 19 , 51 ].

Methodological studies in this area aim to provide guidance about saturation and develop a practical application of processes that ‘operationalise’ and evidence saturation. Guest, Bunce, and Johnson [ 26 ] analysed 60 interviews and found that saturation of themes was reached by the twelfth interview. They noted that their sample was relatively homogeneous, their research aims focused, so studies of more heterogeneous samples and with a broader scope would be likely to need a larger size to achieve saturation. Extending the enquiry to multi-site, cross-cultural research, Hagaman and Wutich [ 28 ] showed that sample sizes of 20 to 40 interviews were required to achieve data saturation of meta-themes that cut across research sites. In a theory-driven content analysis, Francis et al. [ 25 ] reached data saturation at the 17th interview for all their pre-determined theoretical constructs. The authors further proposed two main principles upon which specification of saturation be based: (a) researchers should a priori specify an initial analysis sample (e.g. 10 interviews) which will be used for the first round of analysis and (b) a stopping criterion , that is, a number of interviews (e.g. 3) that needs to be further conducted, the analysis of which will not yield any new themes or ideas. For greater transparency, Francis et al. [ 25 ] recommend that researchers present cumulative frequency graphs supporting their judgment that saturation was achieved. A comparative method for themes saturation (CoMeTS) has also been suggested [ 23 ] whereby the findings of each new interview are compared with those that have already emerged and if it does not yield any new theme, the ‘saturated terrain’ is assumed to have been established. Because the order in which interviews are analysed can influence saturation thresholds depending on the richness of the data, Constantinou et al. [ 23 ] recommend reordering and re-analysing interviews to confirm saturation. Hennink, Kaiser and Marconi’s [ 29 ] methodological study sheds further light on the problem of specifying and demonstrating saturation. Their analysis of interview data showed that code saturation (i.e. the point at which no additional issues are identified) was achieved at 9 interviews, but meaning saturation (i.e. the point at which no further dimensions, nuances, or insights of issues are identified) required 16–24 interviews. Although breadth can be achieved relatively soon, especially for high-prevalence and concrete codes, depth requires additional data, especially for codes of a more conceptual nature.

Critiquing the concept of saturation, Nelson [ 19 ] proposes five conceptual depth criteria in grounded theory projects to assess the robustness of the developing theory: (a) theoretical concepts should be supported by a wide range of evidence drawn from the data; (b) be demonstrably part of a network of inter-connected concepts; (c) demonstrate subtlety; (d) resonate with existing literature; and (e) can be successfully submitted to tests of external validity.

Other work has sought to examine practices of sample size reporting and sufficiency assessment across a range of disciplinary fields and research domains, from nutrition [ 34 ] and health education [ 32 ], to education and the health sciences [ 22 , 27 ], information systems [ 30 ], organisation and workplace studies [ 33 ], human computer interaction [ 21 ], and accounting studies [ 24 ]. Others investigated PhD qualitative studies [ 31 ] and grounded theory studies [ 35 ]. Incomplete and imprecise sample size reporting is commonly pinpointed by these investigations whilst assessment and justifications of sample size sufficiency are even more sporadic.

Sobal [ 34 ] examined the sample size of qualitative studies published in the Journal of Nutrition Education over a period of 30 years. Studies that employed individual interviews ( n  = 30) had an average sample size of 45 individuals and none of these explicitly reported whether their sample size sought and/or attained saturation. A minority of articles discussed how sample-related limitations (with the latter most often concerning the type of sample, rather than the size) limited generalizability. A further systematic analysis [ 32 ] of health education research over 20 years demonstrated that interview-based studies averaged 104 participants (range 2 to 720 interviewees). However, 40% did not report the number of participants. An examination of 83 qualitative interview studies in leading information systems journals [ 30 ] indicated little defence of sample sizes on the basis of recommendations by qualitative methodologists, prior relevant work, or the criterion of saturation. Rather, sample size seemed to correlate with factors such as the journal of publication or the region of study (US vs Europe vs Asia). These results led the authors to call for more rigor in determining and reporting sample size in qualitative information systems research and to recommend optimal sample size ranges for grounded theory (i.e. 20–30 interviews) and single case (i.e. 15–30 interviews) projects.

Similarly, fewer than 10% of articles in organisation and workplace studies provided a sample size justification relating to existing recommendations by methodologists, prior relevant work, or saturation [ 33 ], whilst only 17% of focus groups studies in health-related journals provided an explanation of sample size (i.e. number of focus groups), with saturation being the most frequently invoked argument, followed by published sample size recommendations and practical reasons [ 22 ]. The notion of saturation was also invoked by 11 out of the 51 most highly cited studies that Guetterman [ 27 ] reviewed in the fields of education and health sciences, of which six were grounded theory studies, four phenomenological and one a narrative inquiry. Finally, analysing 641 interview-based articles in accounting, Dai et al. [ 24 ] called for more rigor since a significant minority of studies did not report precise sample size.

Despite increasing attention to rigor in qualitative research (e.g. [ 52 ]) and more extensive methodological and analytical disclosures that seek to validate qualitative work [ 24 ], sample size reporting and sufficiency assessment remain inconsistent and partial, if not absent, across a range of research domains.

Objectives of the present study

The present study sought to enrich existing systematic analyses of the customs and practices of sample size reporting and justification by focusing on qualitative research relating to health. Additionally, this study attempted to expand previous empirical investigations by examining how qualitative sample sizes are characterised and discussed in academic narratives. Qualitative health research is an inter-disciplinary field that due to its affiliation with medical sciences, often faces views and positions reflective of a quantitative ethos. Thus qualitative health research constitutes an emblematic case that may help to unfold underlying philosophical and methodological differences across the scientific community that are crystallised in considerations of sample size. The present research, therefore, incorporates a comparative element on the basis of three different disciplines engaging with qualitative health research: medicine, psychology, and sociology. We chose to focus our analysis on single-per-participant-interview designs as this not only presents a popular and widespread methodological choice in qualitative health research, but also as the method where consideration of sample size – defined as the number of interviewees – is particularly salient.

Study design

A structured search for articles reporting cross-sectional, interview-based qualitative studies was carried out and eligible reports were systematically reviewed and analysed employing both quantitative and qualitative analytic techniques.

We selected journals which (a) follow a peer review process, (b) are considered high quality and influential in their field as reflected in journal metrics, and (c) are receptive to, and publish, qualitative research (Additional File  1 presents the journals’ editorial positions in relation to qualitative research and sample considerations where available). Three health-related journals were chosen, each representing a different disciplinary field; the British Medical Journal (BMJ) representing medicine, the British Journal of Health Psychology (BJHP) representing psychology, and the Sociology of Health & Illness (SHI) representing sociology.

Search strategy to identify studies

Employing the search function of each individual journal, we used the terms ‘interview*’ AND ‘qualitative’ and limited the results to articles published between 1 January 2003 and 22 September 2017 (i.e. a 15-year review period).

Eligibility criteria

To be eligible for inclusion in the review, the article had to report a cross-sectional study design. Longitudinal studies were thus excluded whilst studies conducted within a broader research programme (e.g. interview studies nested in a trial, as part of a broader ethnography, as part of a longitudinal research) were included if they reported only single-time qualitative interviews. The method of data collection had to be individual, synchronous qualitative interviews (i.e. group interviews, structured interviews and e-mail interviews over a period of time were excluded), and the data had to be analysed qualitatively (i.e. studies that quantified their qualitative data were excluded). Mixed method studies and articles reporting more than one qualitative method of data collection (e.g. individual interviews and focus groups) were excluded. Figure  1 , a PRISMA flow diagram [ 53 ], shows the number of: articles obtained from the searches and screened; papers assessed for eligibility; and articles included in the review (Additional File  2 provides the full list of articles included in the review and their unique identifying code – e.g. BMJ01, BJHP02, SHI03). One review author (KV) assessed the eligibility of all papers identified from the searches. When in doubt, discussions about retaining or excluding articles were held between KV and JB in regular meetings, and decisions were jointly made.

PRISMA flow diagram

Data extraction and analysis

A data extraction form was developed (see Additional File  3 ) recording three areas of information: (a) information about the article (e.g. authors, title, journal, year of publication etc.); (b) information about the aims of the study, the sample size and any justification for this, the participant characteristics, the sampling technique and any sample-related observations or comments made by the authors; and (c) information about the method or technique(s) of data analysis, the number of researchers involved in the analysis, the potential use of software, and any discussion around epistemological considerations. The Abstract, Methods and Discussion (and/or Conclusion) sections of each article were examined by one author (KV) who extracted all the relevant information. This was directly copied from the articles and, when appropriate, comments, notes and initial thoughts were written down.

To examine the kinds of sample size justifications provided by articles, an inductive content analysis [ 54 ] was initially conducted. On the basis of this analysis, the categories that expressed qualitatively different sample size justifications were developed.

We also extracted or coded quantitative data regarding the following aspects:

Journal and year of publication

Number of interviews

Number of participants

Presence of sample size justification(s) (Yes/No)

Presence of a particular sample size justification category (Yes/No), and

Number of sample size justifications provided

Descriptive and inferential statistical analyses were used to explore these data.

A thematic analysis [ 55 ] was then performed on all scientific narratives that discussed or commented on the sample size of the study. These narratives were evident both in papers that justified their sample size and those that did not. To identify these narratives, in addition to the methods sections, the discussion sections of the reviewed articles were also examined and relevant data were extracted and analysed.

In total, 214 articles – 21 in the BMJ, 53 in the BJHP and 140 in the SHI – were eligible for inclusion in the review. Table  1 provides basic information about the sample sizes – measured in number of interviews – of the studies reviewed across the three journals. Figure  2 depicts the number of eligible articles published each year per journal.

The publication of qualitative studies in the BMJ was significantly reduced from 2012 onwards and this appears to coincide with the initiation of the BMJ Open to which qualitative studies were possibly directed.

Pairwise comparisons following a significant Kruskal-Wallis Footnote 2 test indicated that the studies published in the BJHP had significantly ( p  < .001) smaller samples sizes than those published either in the BMJ or the SHI. Sample sizes of BMJ and SHI articles did not differ significantly from each other.

Sample size justifications: Results from the quantitative and qualitative content analysis

Ten (47.6%) of the 21 BMJ studies, 26 (49.1%) of the 53 BJHP papers and 24 (17.1%) of the 140 SHI articles provided some sort of sample size justification. As shown in Table  2 , the majority of articles which justified their sample size provided one justification (70% of articles); fourteen studies (25%) provided two distinct justifications; one study (1.7%) gave three justifications and two studies (3.3%) expressed four distinct justifications.

There was no association between the number of interviews (i.e. sample size) conducted and the provision of a justification (rpb = .054, p  = .433). Within journals, Mann-Whitney tests indicated that sample sizes of ‘justifying’ and ‘non-justifying’ articles in the BMJ and SHI did not differ significantly from each other. In the BJHP, ‘justifying’ articles ( Mean rank  = 31.3) had significantly larger sample sizes than ‘non-justifying’ studies ( Mean rank  = 22.7; U = 237.000, p  < .05).

There was a significant association between the journal a paper was published in and the provision of a justification (χ 2 (2) = 23.83, p  < .001). BJHP studies provided a sample size justification significantly more often than would be expected ( z  = 2.9); SHI studies significantly less often ( z  = − 2.4). If an article was published in the BJHP, the odds of providing a justification were 4.8 times higher than if published in the SHI. Similarly if published in the BMJ, the odds of a study justifying its sample size were 4.5 times higher than in the SHI.

The qualitative content analysis of the scientific narratives identified eleven different sample size justifications. These are described below and illustrated with excerpts from relevant articles. By way of a summary, the frequency with which these were deployed across the three journals is indicated in Table  3 .

Saturation was the most commonly invoked principle (55.4% of all justifications) deployed by studies across all three journals to justify the sufficiency of their sample size. In the BMJ, two studies claimed that they achieved data saturation (BMJ17; BMJ18) and one article referred descriptively to achieving saturation without explicitly using the term (BMJ13). Interestingly, BMJ13 included data in the analysis beyond the point of saturation in search of ‘unusual/deviant observations’ and with a view to establishing findings consistency.

Thirty three women were approached to take part in the interview study. Twenty seven agreed and 21 (aged 21–64, median 40) were interviewed before data saturation was reached (one tape failure meant that 20 interviews were available for analysis). (BMJ17). No new topics were identified following analysis of approximately two thirds of the interviews; however, all interviews were coded in order to develop a better understanding of how characteristic the views and reported behaviours were, and also to collect further examples of unusual/deviant observations. (BMJ13).

Two articles reported pre-determining their sample size with a view to achieving data saturation (BMJ08 – see extract in section In line with existing research ; BMJ15 – see extract in section Pragmatic considerations ) without further specifying if this was achieved. One paper claimed theoretical saturation (BMJ06) conceived as being when “no further recurring themes emerging from the analysis” whilst another study argued that although the analytic categories were highly saturated, it was not possible to determine whether theoretical saturation had been achieved (BMJ04). One article (BMJ18) cited a reference to support its position on saturation.

In the BJHP, six articles claimed that they achieved data saturation (BJHP21; BJHP32; BJHP39; BJHP48; BJHP49; BJHP52) and one article stated that, given their sample size and the guidelines for achieving data saturation, it anticipated that saturation would be attained (BJHP50).

Recruitment continued until data saturation was reached, defined as the point at which no new themes emerged. (BJHP48). It has previously been recommended that qualitative studies require a minimum sample size of at least 12 to reach data saturation (Clarke & Braun, 2013; Fugard & Potts, 2014; Guest, Bunce, & Johnson, 2006) Therefore, a sample of 13 was deemed sufficient for the qualitative analysis and scale of this study. (BJHP50).

Two studies argued that they achieved thematic saturation (BJHP28 – see extract in section Sample size guidelines ; BJHP31) and one (BJHP30) article, explicitly concerned with theory development and deploying theoretical sampling, claimed both theoretical and data saturation.

The final sample size was determined by thematic saturation, the point at which new data appears to no longer contribute to the findings due to repetition of themes and comments by participants (Morse, 1995). At this point, data generation was terminated. (BJHP31).

Five studies argued that they achieved (BJHP05; BJHP33; BJHP40; BJHP13 – see extract in section Pragmatic considerations ) or anticipated (BJHP46) saturation without any further specification of the term. BJHP17 referred descriptively to a state of achieved saturation without specifically using the term. Saturation of coding , but not saturation of themes, was claimed to have been reached by one article (BJHP18). Two articles explicitly stated that they did not achieve saturation; instead claiming a level of theme completeness (BJHP27) or that themes being replicated (BJHP53) were arguments for sufficiency of their sample size.

Furthermore, data collection ceased on pragmatic grounds rather than at the point when saturation point was reached. Despite this, although nuances within sub-themes were still emerging towards the end of data analysis, the themes themselves were being replicated indicating a level of completeness. (BJHP27).

Finally, one article criticised and explicitly renounced the notion of data saturation claiming that, on the contrary, the criterion of theoretical sufficiency determined its sample size (BJHP16).

According to the original Grounded Theory texts, data collection should continue until there are no new discoveries ( i.e. , ‘data saturation’; Glaser & Strauss, 1967). However, recent revisions of this process have discussed how it is rare that data collection is an exhaustive process and researchers should rely on how well their data are able to create a sufficient theoretical account or ‘theoretical sufficiency’ (Dey, 1999). For this study, it was decided that theoretical sufficiency would guide recruitment, rather than looking for data saturation. (BJHP16).

Ten out of the 20 BJHP articles that employed the argument of saturation used one or more citations relating to this principle.

In the SHI, one article (SHI01) claimed that it achieved category saturation based on authors’ judgment.

This number was not fixed in advance, but was guided by the sampling strategy and the judgement, based on the analysis of the data, of the point at which ‘category saturation’ was achieved. (SHI01).

Three articles described a state of achieved saturation without using the term or specifying what sort of saturation they had achieved (i.e. data, theoretical, thematic saturation) (SHI04; SHI13; SHI30) whilst another four articles explicitly stated that they achieved saturation (SHI100; SHI125; SHI136; SHI137). Two papers stated that they achieved data saturation (SHI73 – see extract in section Sample size guidelines ; SHI113), two claimed theoretical saturation (SHI78; SHI115) and two referred to achieving thematic saturation (SHI87; SHI139) or to saturated themes (SHI29; SHI50).

Recruitment and analysis ceased once theoretical saturation was reached in the categories described below (Lincoln and Guba 1985). (SHI115). The respondents’ quotes drawn on below were chosen as representative, and illustrate saturated themes. (SHI50).

One article stated that thematic saturation was anticipated with its sample size (SHI94). Briefly referring to the difficulty in pinpointing achievement of theoretical saturation, SHI32 (see extract in section Richness and volume of data ) defended the sufficiency of its sample size on the basis of “the high degree of consensus [that] had begun to emerge among those interviewed”, suggesting that information from interviews was being replicated. Finally, SHI112 (see extract in section Further sampling to check findings consistency ) argued that it achieved saturation of discursive patterns . Seven of the 19 SHI articles cited references to support their position on saturation (see Additional File  4 for the full list of citations used by articles to support their position on saturation across the three journals).

Overall, it is clear that the concept of saturation encompassed a wide range of variants expressed in terms such as saturation, data saturation, thematic saturation, theoretical saturation, category saturation, saturation of coding, saturation of discursive themes, theme completeness. It is noteworthy, however, that although these various claims were sometimes supported with reference to the literature, they were not evidenced in relation to the study at hand.

Pragmatic considerations

The determination of sample size on the basis of pragmatic considerations was the second most frequently invoked argument (9.6% of all justifications) appearing in all three journals. In the BMJ, one article (BMJ15) appealed to pragmatic reasons, relating to time constraints and the difficulty to access certain study populations, to justify the determination of its sample size.

On the basis of the researchers’ previous experience and the literature, [30, 31] we estimated that recruitment of 15–20 patients at each site would achieve data saturation when data from each site were analysed separately. We set a target of seven to 10 caregivers per site because of time constraints and the anticipated difficulty of accessing caregivers at some home based care services. This gave a target sample of 75–100 patients and 35–50 caregivers overall. (BMJ15).

In the BJHP, four articles mentioned pragmatic considerations relating to time or financial constraints (BJHP27 – see extract in section Saturation ; BJHP53), the participant response rate (BJHP13), and the fixed (and thus limited) size of the participant pool from which interviewees were sampled (BJHP18).

We had aimed to continue interviewing until we had reached saturation, a point whereby further data collection would yield no further themes. In practice, the number of individuals volunteering to participate dictated when recruitment into the study ceased (15 young people, 15 parents). Nonetheless, by the last few interviews, significant repetition of concepts was occurring, suggesting ample sampling. (BJHP13).

Finally, three SHI articles explained their sample size with reference to practical aspects: time constraints and project manageability (SHI56), limited availability of respondents and project resources (SHI131), and time constraints (SHI113).

The size of the sample was largely determined by the availability of respondents and resources to complete the study. Its composition reflected, as far as practicable, our interest in how contextual factors (for example, gender relations and ethnicity) mediated the illness experience. (SHI131).

Qualities of the analysis

This sample size justification (8.4% of all justifications) was mainly employed by BJHP articles and referred to an intensive, idiographic and/or latently focused analysis, i.e. that moved beyond description. More specifically, six articles defended their sample size on the basis of an intensive analysis of transcripts and/or the idiographic focus of the study/analysis. Four of these papers (BJHP02; BJHP19; BJHP24; BJHP47) adopted an Interpretative Phenomenological Analysis (IPA) approach.

The current study employed a sample of 10 in keeping with the aim of exploring each participant’s account (Smith et al. , 1999). (BJHP19).

BJHP47 explicitly renounced the notion of saturation within an IPA approach. The other two BJHP articles conducted thematic analysis (BJHP34; BJHP38). The level of analysis – i.e. latent as opposed to a more superficial descriptive analysis – was also invoked as a justification by BJHP38 alongside the argument of an intensive analysis of individual transcripts

The resulting sample size was at the lower end of the range of sample sizes employed in thematic analysis (Braun & Clarke, 2013). This was in order to enable significant reflection, dialogue, and time on each transcript and was in line with the more latent level of analysis employed, to identify underlying ideas, rather than a more superficial descriptive analysis (Braun & Clarke, 2006). (BJHP38).

Finally, one BMJ paper (BMJ21) defended its sample size with reference to the complexity of the analytic task.

We stopped recruitment when we reached 30–35 interviews, owing to the depth and duration of interviews, richness of data, and complexity of the analytical task. (BMJ21).

Meet sampling requirements

Meeting sampling requirements (7.2% of all justifications) was another argument employed by two BMJ and four SHI articles to explain their sample size. Achieving maximum variation sampling in terms of specific interviewee characteristics determined and explained the sample size of two BMJ studies (BMJ02; BMJ16 – see extract in section Meet research design requirements ).

Recruitment continued until sampling frame requirements were met for diversity in age, sex, ethnicity, frequency of attendance, and health status. (BMJ02).

Regarding the SHI articles, two papers explained their numbers on the basis of their sampling strategy (SHI01- see extract in section Saturation ; SHI23) whilst sampling requirements that would help attain sample heterogeneity in terms of a particular characteristic of interest was cited by one paper (SHI127).

The combination of matching the recruitment sites for the quantitative research and the additional purposive criteria led to 104 phase 2 interviews (Internet (OLC): 21; Internet (FTF): 20); Gyms (FTF): 23; HIV testing (FTF): 20; HIV treatment (FTF): 20.) (SHI23). Of the fifty interviews conducted, thirty were translated from Spanish into English. These thirty, from which we draw our findings, were chosen for translation based on heterogeneity in depressive symptomology and educational attainment. (SHI127).

Finally, the pre-determination of sample size on the basis of sampling requirements was stated by one article though this was not used to justify the number of interviews (SHI10).

Sample size guidelines

Five BJHP articles (BJHP28; BJHP38 – see extract in section Qualities of the analysis ; BJHP46; BJHP47; BJHP50 – see extract in section Saturation ) and one SHI paper (SHI73) relied on citing existing sample size guidelines or norms within research traditions to determine and subsequently defend their sample size (7.2% of all justifications).

Sample size guidelines suggested a range between 20 and 30 interviews to be adequate (Creswell, 1998). Interviewer and note taker agreed that thematic saturation, the point at which no new concepts emerge from subsequent interviews (Patton, 2002), was achieved following completion of 20 interviews. (BJHP28). Interviewing continued until we deemed data saturation to have been reached (the point at which no new themes were emerging). Researchers have proposed 30 as an approximate or working number of interviews at which one could expect to be reaching theoretical saturation when using a semi-structured interview approach (Morse 2000), although this can vary depending on the heterogeneity of respondents interviewed and complexity of the issues explored. (SHI73).

In line with existing research

Sample sizes of published literature in the area of the subject matter under investigation (3.5% of all justifications) were used by 2 BMJ articles as guidance and a precedent for determining and defending their own sample size (BMJ08; BMJ15 – see extract in section Pragmatic considerations ).

We drew participants from a list of prisoners who were scheduled for release each week, sampling them until we reached the target of 35 cases, with a view to achieving data saturation within the scope of the study and sufficient follow-up interviews and in line with recent studies [8–10]. (BMJ08).

Similarly, BJHP38 (see extract in section Qualities of the analysis ) claimed that its sample size was within the range of sample sizes of published studies that use its analytic approach.

Richness and volume of data

BMJ21 (see extract in section Qualities of the analysis ) and SHI32 referred to the richness, detailed nature, and volume of data collected (2.3% of all justifications) to justify the sufficiency of their sample size.

Although there were more potential interviewees from those contacted by postcode selection, it was decided to stop recruitment after the 10th interview and focus on analysis of this sample. The material collected was considerable and, given the focused nature of the study, extremely detailed. Moreover, a high degree of consensus had begun to emerge among those interviewed, and while it is always difficult to judge at what point ‘theoretical saturation’ has been reached, or how many interviews would be required to uncover exception(s), it was felt the number was sufficient to satisfy the aims of this small in-depth investigation (Strauss and Corbin 1990). (SHI32).

Meet research design requirements

Determination of sample size so that it is in line with, and serves the requirements of, the research design (2.3% of all justifications) that the study adopted was another justification used by 2 BMJ papers (BMJ16; BMJ08 – see extract in section In line with existing research ).

We aimed for diverse, maximum variation samples [20] totalling 80 respondents from different social backgrounds and ethnic groups and those bereaved due to different types of suicide and traumatic death. We could have interviewed a smaller sample at different points in time (a qualitative longitudinal study) but chose instead to seek a broad range of experiences by interviewing those bereaved many years ago and others bereaved more recently; those bereaved in different circumstances and with different relations to the deceased; and people who lived in different parts of the UK; with different support systems and coroners’ procedures (see Tables 1 and 2 for more details). (BMJ16).

Researchers’ previous experience

The researchers’ previous experience (possibly referring to experience with qualitative research) was invoked by BMJ15 (see extract in section Pragmatic considerations ) as a justification for the determination of sample size.

Nature of study

One BJHP paper argued that the sample size was appropriate for the exploratory nature of the study (BJHP38).

A sample of eight participants was deemed appropriate because of the exploratory nature of this research and the focus on identifying underlying ideas about the topic. (BJHP38).

Further sampling to check findings consistency

Finally, SHI112 argued that once it had achieved saturation of discursive patterns, further sampling was decided and conducted to check for consistency of the findings.

Within each of the age-stratified groups, interviews were randomly sampled until saturation of discursive patterns was achieved. This resulted in a sample of 67 interviews. Once this sample had been analysed, one further interview from each age-stratified group was randomly chosen to check for consistency of the findings. Using this approach it was possible to more carefully explore children’s discourse about the ‘I’, agency, relationality and power in the thematic areas, revealing the subtle discursive variations described in this article. (SHI112).

Thematic analysis of passages discussing sample size

This analysis resulted in two overarching thematic areas; the first concerned the variation in the characterisation of sample size sufficiency, and the second related to the perceived threats deriving from sample size insufficiency.

Characterisations of sample size sufficiency

The analysis showed that there were three main characterisations of the sample size in the articles that provided relevant comments and discussion: (a) the vast majority of these qualitative studies ( n  = 42) considered their sample size as ‘small’ and this was seen and discussed as a limitation; only two articles viewed their small sample size as desirable and appropriate (b) a minority of articles ( n  = 4) proclaimed that their achieved sample size was ‘sufficient’; and (c) finally, a small group of studies ( n  = 5) characterised their sample size as ‘large’. Whilst achieving a ‘large’ sample size was sometimes viewed positively because it led to richer results, there were also occasions when a large sample size was problematic rather than desirable.

‘Small’ but why and for whom?

A number of articles which characterised their sample size as ‘small’ did so against an implicit or explicit quantitative framework of reference. Interestingly, three studies that claimed to have achieved data saturation or ‘theoretical sufficiency’ with their sample size, discussed or noted as a limitation in their discussion their ‘small’ sample size, raising the question of why, or for whom, the sample size was considered small given that the qualitative criterion of saturation had been satisfied.

The current study has a number of limitations. The sample size was small (n = 11) and, however, large enough for no new themes to emerge. (BJHP39). The study has two principal limitations. The first of these relates to the small number of respondents who took part in the study. (SHI73).

Other articles appeared to accept and acknowledge that their sample was flawed because of its small size (as well as other compositional ‘deficits’ e.g. non-representativeness, biases, self-selection) or anticipated that they might be criticized for their small sample size. It seemed that the imagined audience – perhaps reviewer or reader – was one inclined to hold the tenets of quantitative research, and certainly one to whom it was important to indicate the recognition that small samples were likely to be problematic. That one’s sample might be thought small was often construed as a limitation couched in a discourse of regret or apology.

Very occasionally, the articulation of the small size as a limitation was explicitly aligned against an espoused positivist framework and quantitative research.

This study has some limitations. Firstly, the 100 incidents sample represents a small number of the total number of serious incidents that occurs every year. 26 We sent out a nationwide invitation and do not know why more people did not volunteer for the study. Our lack of epidemiological knowledge about healthcare incidents, however, means that determining an appropriate sample size continues to be difficult. (BMJ20).

Indicative of an apparent oscillation of qualitative researchers between the different requirements and protocols demarcating the quantitative and qualitative worlds, there were a few instances of articles which briefly recognised their ‘small’ sample size as a limitation, but then defended their study on more qualitative grounds, such as their ability and success at capturing the complexity of experience and delving into the idiographic, and at generating particularly rich data.

This research, while limited in size, has sought to capture some of the complexity attached to men’s attitudes and experiences concerning incomes and material circumstances. (SHI35). Our numbers are small because negotiating access to social networks was slow and labour intensive, but our methods generated exceptionally rich data. (BMJ21). This study could be criticised for using a small and unrepresentative sample. Given that older adults have been ignored in the research concerning suntanning, fair-skinned older adults are the most likely to experience skin cancer, and women privilege appearance over health when it comes to sunbathing practices, our study offers depth and richness of data in a demographic group much in need of research attention. (SHI57).

‘Good enough’ sample sizes

Only four articles expressed some degree of confidence that their achieved sample size was sufficient. For example, SHI139, in line with the justification of thematic saturation that it offered, expressed trust in its sample size sufficiency despite the poor response rate. Similarly, BJHP04, which did not provide a sample size justification, argued that it targeted a larger sample size in order to eventually recruit a sufficient number of interviewees, due to anticipated low response rate.

Twenty-three people with type I diabetes from the target population of 133 ( i.e. 17.3%) consented to participate but four did not then respond to further contacts (total N = 19). The relatively low response rate was anticipated, due to the busy life-styles of young people in the age range, the geographical constraints, and the time required to participate in a semi-structured interview, so a larger target sample allowed a sufficient number of participants to be recruited. (BJHP04).

Two other articles (BJHP35; SHI32) linked the claimed sufficiency to the scope (i.e. ‘small, in-depth investigation’), aims and nature (i.e. ‘exploratory’) of their studies, thus anchoring their numbers to the particular context of their research. Nevertheless, claims of sample size sufficiency were sometimes undermined when they were juxtaposed with an acknowledgement that a larger sample size would be more scientifically productive.

Although our sample size was sufficient for this exploratory study, a more diverse sample including participants with lower socioeconomic status and more ethnic variation would be informative. A larger sample could also ensure inclusion of a more representative range of apps operating on a wider range of platforms. (BJHP35).

‘Large’ sample sizes - Promise or peril?

Three articles (BMJ13; BJHP05; BJHP48) which all provided the justification of saturation, characterised their sample size as ‘large’ and narrated this oversufficiency in positive terms as it allowed richer data and findings and enhanced the potential for generalisation. The type of generalisation aspired to (BJHP48) was not further specified however.

This study used rich data provided by a relatively large sample of expert informants on an important but under-researched topic. (BMJ13). Qualitative research provides a unique opportunity to understand a clinical problem from the patient’s perspective. This study had a large diverse sample, recruited through a range of locations and used in-depth interviews which enhance the richness and generalizability of the results. (BJHP48).

And whilst a ‘large’ sample size was endorsed and valued by some qualitative researchers, within the psychological tradition of IPA, a ‘large’ sample size was counter-normative and therefore needed to be justified. Four BJHP studies, all adopting IPA, expressed the appropriateness or desirability of ‘small’ sample sizes (BJHP41; BJHP45) or hastened to explain why they included a larger than typical sample size (BJHP32; BJHP47). For example, BJHP32 below provides a rationale for how an IPA study can accommodate a large sample size and how this was indeed suitable for the purposes of the particular research. To strengthen the explanation for choosing a non-normative sample size, previous IPA research citing a similar sample size approach is used as a precedent.

Small scale IPA studies allow in-depth analysis which would not be possible with larger samples (Smith et al. , 2009). (BJHP41). Although IPA generally involves intense scrutiny of a small number of transcripts, it was decided to recruit a larger diverse sample as this is the first qualitative study of this population in the United Kingdom (as far as we know) and we wanted to gain an overview. Indeed, Smith, Flowers, and Larkin (2009) agree that IPA is suitable for larger groups. However, the emphasis changes from an in-depth individualistic analysis to one in which common themes from shared experiences of a group of people can be elicited and used to understand the network of relationships between themes that emerge from the interviews. This large-scale format of IPA has been used by other researchers in the field of false-positive research. Baillie, Smith, Hewison, and Mason (2000) conducted an IPA study, with 24 participants, of ultrasound screening for chromosomal abnormality; they found that this larger number of participants enabled them to produce a more refined and cohesive account. (BJHP32).

The IPA articles found in the BJHP were the only instances where a ‘small’ sample size was advocated and a ‘large’ sample size problematized and defended. These IPA studies illustrate that the characterisation of sample size sufficiency can be a function of researchers’ theoretical and epistemological commitments rather than the result of an ‘objective’ sample size assessment.

Threats from sample size insufficiency

As shown above, the majority of articles that commented on their sample size, simultaneously characterized it as small and problematic. On those occasions that authors did not simply cite their ‘small’ sample size as a study limitation but rather continued and provided an account of how and why a small sample size was problematic, two important scientific qualities of the research seemed to be threatened: the generalizability and validity of results.

Generalizability

Those who characterised their sample as ‘small’ connected this to the limited potential for generalization of the results. Other features related to the sample – often some kind of compositional particularity – were also linked to limited potential for generalisation. Though not always explicitly articulated to what form of generalisation the articles referred to (see BJHP09), generalisation was mostly conceived in nomothetic terms, that is, it concerned the potential to draw inferences from the sample to the broader study population (‘representational generalisation’ – see BJHP31) and less often to other populations or cultures.

It must be noted that samples are small and whilst in both groups the majority of those women eligible participated, generalizability cannot be assumed. (BJHP09). The study’s limitations should be acknowledged: Data are presented from interviews with a relatively small group of participants, and thus, the views are not necessarily generalizable to all patients and clinicians. In particular, patients were only recruited from secondary care services where COFP diagnoses are typically confirmed. The sample therefore is unlikely to represent the full spectrum of patients, particularly those who are not referred to, or who have been discharged from dental services. (BJHP31).

Without explicitly using the term generalisation, two SHI articles noted how their ‘small’ sample size imposed limits on ‘the extent that we can extrapolate from these participants’ accounts’ (SHI114) or to the possibility ‘to draw far-reaching conclusions from the results’ (SHI124).

Interestingly, only a minority of articles alluded to, or invoked, a type of generalisation that is aligned with qualitative research, that is, idiographic generalisation (i.e. generalisation that can be made from and about cases [ 5 ]). These articles, all published in the discipline of sociology, defended their findings in terms of the possibility of drawing logical and conceptual inferences to other contexts and of generating understanding that has the potential to advance knowledge, despite their ‘small’ size. One article (SHI139) clearly contrasted nomothetic (statistical) generalisation to idiographic generalisation, arguing that the lack of statistical generalizability does not nullify the ability of qualitative research to still be relevant beyond the sample studied.

Further, these data do not need to be statistically generalisable for us to draw inferences that may advance medicalisation analyses (Charmaz 2014). These data may be seen as an opportunity to generate further hypotheses and are a unique application of the medicalisation framework. (SHI139). Although a small-scale qualitative study related to school counselling, this analysis can be usefully regarded as a case study of the successful utilisation of mental health-related resources by adolescents. As many of the issues explored are of relevance to mental health stigma more generally, it may also provide insights into adult engagement in services. It shows how a sociological analysis, which uses positioning theory to examine how people negotiate, partially accept and simultaneously resist stigmatisation in relation to mental health concerns, can contribute to an elucidation of the social processes and narrative constructions which may maintain as well as bridge the mental health service gap. (SHI103).

Only one article (SHI30) used the term transferability to argue for the potential of wider relevance of the results which was thought to be more the product of the composition of the sample (i.e. diverse sample), rather than the sample size.

The second major concern that arose from a ‘small’ sample size pertained to the internal validity of findings (i.e. here the term is used to denote the ‘truth’ or credibility of research findings). Authors expressed uncertainty about the degree of confidence in particular aspects or patterns of their results, primarily those that concerned some form of differentiation on the basis of relevant participant characteristics.

The information source preferred seemed to vary according to parents’ education; however, the sample size is too small to draw conclusions about such patterns. (SHI80). Although our numbers were too small to demonstrate gender differences with any certainty, it does seem that the biomedical and erotic scripts may be more common in the accounts of men and the relational script more common in the accounts of women. (SHI81).

In other instances, articles expressed uncertainty about whether their results accounted for the full spectrum and variation of the phenomenon under investigation. In other words, a ‘small’ sample size (alongside compositional ‘deficits’ such as a not statistically representative sample) was seen to threaten the ‘content validity’ of the results which in turn led to constructions of the study conclusions as tentative.

Data collection ceased on pragmatic grounds rather than when no new information appeared to be obtained ( i.e. , saturation point). As such, care should be taken not to overstate the findings. Whilst the themes from the initial interviews seemed to be replicated in the later interviews, further interviews may have identified additional themes or provided more nuanced explanations. (BJHP53). …it should be acknowledged that this study was based on a small sample of self-selected couples in enduring marriages who were not broadly representative of the population. Thus, participants may not be representative of couples that experience postnatal PTSD. It is therefore unlikely that all the key themes have been identified and explored. For example, couples who were excluded from the study because the male partner declined to participate may have been experiencing greater interpersonal difficulties. (BJHP03).

In other instances, articles attempted to preserve a degree of credibility of their results, despite the recognition that the sample size was ‘small’. Clarity and sharpness of emerging themes and alignment with previous relevant work were the arguments employed to warrant the validity of the results.

This study focused on British Chinese carers of patients with affective disorders, using a qualitative methodology to synthesise the sociocultural representations of illness within this community. Despite the small sample size, clear themes emerged from the narratives that were sufficient for this exploratory investigation. (SHI98).

The present study sought to examine how qualitative sample sizes in health-related research are characterised and justified. In line with previous studies [ 22 , 30 , 33 , 34 ] the findings demonstrate that reporting of sample size sufficiency is limited; just over 50% of articles in the BMJ and BJHP and 82% in the SHI did not provide any sample size justification. Providing a sample size justification was not related to the number of interviews conducted, but it was associated with the journal that the article was published in, indicating the influence of disciplinary or publishing norms, also reported in prior research [ 30 ]. This lack of transparency about sample size sufficiency is problematic given that most qualitative researchers would agree that it is an important marker of quality [ 56 , 57 ]. Moreover, and with the rise of qualitative research in social sciences, efforts to synthesise existing evidence and assess its quality are obstructed by poor reporting [ 58 , 59 ].

When authors justified their sample size, our findings indicate that sufficiency was mostly appraised with reference to features that were intrinsic to the study, in agreement with general advice on sample size determination [ 4 , 11 , 36 ]. The principle of saturation was the most commonly invoked argument [ 22 ] accounting for 55% of all justifications. A wide range of variants of saturation was evident corroborating the proliferation of the meaning of the term [ 49 ] and reflecting different underlying conceptualisations or models of saturation [ 20 ]. Nevertheless, claims of saturation were never substantiated in relation to procedures conducted in the study itself, endorsing similar observations in the literature [ 25 , 30 , 47 ]. Claims of saturation were sometimes supported with citations of other literature, suggesting a removal of the concept away from the characteristics of the study at hand. Pragmatic considerations, such as resource constraints or participant response rate and availability, was the second most frequently used argument accounting for approximately 10% of justifications and another 23% of justifications also represented intrinsic-to-the-study characteristics (i.e. qualities of the analysis, meeting sampling or research design requirements, richness and volume of the data obtained, nature of study, further sampling to check findings consistency).

Only, 12% of mentions of sample size justification pertained to arguments that were external to the study at hand, in the form of existing sample size guidelines and prior research that sets precedents. Whilst community norms and prior research can establish useful rules of thumb for estimating sample sizes [ 60 ] – and reveal what sizes are more likely to be acceptable within research communities – researchers should avoid adopting these norms uncritically, especially when such guidelines [e.g. 30 , 35 ], might be based on research that does not provide adequate evidence of sample size sufficiency. Similarly, whilst methodological research that seeks to demonstrate the achievement of saturation is invaluable since it explicates the parameters upon which saturation is contingent and indicates when a research project is likely to require a smaller or a larger sample [e.g. 29 ], specific numbers at which saturation was achieved within these projects cannot be routinely extrapolated for other projects. We concur with existing views [ 11 , 36 ] that the consideration of the characteristics of the study at hand, such as the epistemological and theoretical approach, the nature of the phenomenon under investigation, the aims and scope of the study, the quality and richness of data, or the researcher’s experience and skills of conducting qualitative research, should be the primary guide in determining sample size and assessing its sufficiency.

Moreover, although numbers in qualitative research are not unimportant [ 61 ], sample size should not be considered alone but be embedded in the more encompassing examination of data adequacy [ 56 , 57 ]. Erickson’s [ 62 ] dimensions of ‘evidentiary adequacy’ are useful here. He explains the concept in terms of adequate amounts of evidence, adequate variety in kinds of evidence, adequate interpretive status of evidence, adequate disconfirming evidence, and adequate discrepant case analysis. All dimensions might not be relevant across all qualitative research designs, but this illustrates the thickness of the concept of data adequacy, taking it beyond sample size.

The present research also demonstrated that sample sizes were commonly seen as ‘small’ and insufficient and discussed as limitation. Often unjustified (and in two cases incongruent with their own claims of saturation) these findings imply that sample size in qualitative health research is often adversely judged (or expected to be judged) against an implicit, yet omnipresent, quasi-quantitative standpoint. Indeed there were a few instances in our data where authors appeared, possibly in response to reviewers, to resist to some sort of quantification of their results. This implicit reference point became more apparent when authors discussed the threats deriving from an insufficient sample size. Whilst the concerns about internal validity might be legitimate to the extent that qualitative research projects, which are broadly related to realism, are set to examine phenomena in sufficient breadth and depth, the concerns around generalizability revealed a conceptualisation that is not compatible with purposive sampling. The limited potential for generalisation, as a result of a small sample size, was often discussed in nomothetic, statistical terms. Only occasionally was analytic or idiographic generalisation invoked to warrant the value of the study’s findings [ 5 , 17 ].

Strengths and limitations of the present study

We note, first, the limited number of health-related journals reviewed, so that only a ‘snapshot’ of qualitative health research has been captured. Examining additional disciplines (e.g. nursing sciences) as well as inter-disciplinary journals would add to the findings of this analysis. Nevertheless, our study is the first to provide some comparative insights on the basis of disciplines that are differently attached to the legacy of positivism and analysed literature published over a lengthy period of time (15 years). Guetterman [ 27 ] also examined health-related literature but this analysis was restricted to 26 most highly cited articles published over a period of five years whilst Carlsen and Glenton’s [ 22 ] study concentrated on focus groups health research. Moreover, although it was our intention to examine sample size justification in relation to the epistemological and theoretical positions of articles, this proved to be challenging largely due to absence of relevant information, or the difficulty into discerning clearly articles’ positions [ 63 ] and classifying them under specific approaches (e.g. studies often combined elements from different theoretical and epistemological traditions). We believe that such an analysis would yield useful insights as it links the methodological issue of sample size to the broader philosophical stance of the research. Despite these limitations, the analysis of the characterisation of sample size and of the threats seen to accrue from insufficient sample size, enriches our understanding of sample size (in)sufficiency argumentation by linking it to other features of the research. As the peer-review process becomes increasingly public, future research could usefully examine how reporting around sample size sufficiency and data adequacy might be influenced by the interactions between authors and reviewers.

The past decade has seen a growing appetite in qualitative research for an evidence-based approach to sample size determination and to evaluations of the sufficiency of sample size. Despite the conceptual and methodological developments in the area, the findings of the present study confirm previous studies in concluding that appraisals of sample size sufficiency are either absent or poorly substantiated. To ensure and maintain high quality research that will encourage greater appreciation of qualitative work in health-related sciences [ 64 ], we argue that qualitative researchers should be more transparent and thorough in their evaluation of sample size as part of their appraisal of data adequacy. We would encourage the practice of appraising sample size sufficiency with close reference to the study at hand and would thus caution against responding to the growing methodological research in this area with a decontextualised application of sample size numerical guidelines, norms and principles. Although researchers might find sample size community norms serve as useful rules of thumb, we recommend methodological knowledge is used to critically consider how saturation and other parameters that affect sample size sufficiency pertain to the specifics of the particular project. Those reviewing papers have a vital role in encouraging transparent study-specific reporting. The review process should support authors to exercise nuanced judgments in decisions about sample size determination in the context of the range of factors that influence sample size sufficiency and the specifics of a particular study. In light of the growing methodological evidence in the area, transparent presentation of such evidence-based judgement is crucial and in time should surely obviate the seemingly routine practice of citing the ‘small’ size of qualitative samples among the study limitations.

A non-parametric test of difference for independent samples was performed since the variable number of interviews violated assumptions of normality according to the standardized scores of skewness and kurtosis (BMJ: z skewness = 3.23, z kurtosis = 1.52; BJHP: z skewness = 4.73, z kurtosis = 4.85; SHI: z skewness = 12.04, z kurtosis = 21.72) and the Shapiro-Wilk test of normality ( p  < .001).

Abbreviations

British Journal of Health Psychology

British Medical Journal

Interpretative Phenomenological Analysis

Sociology of Health & Illness

Spencer L, Ritchie J, Lewis J, Dillon L. Quality in qualitative evaluation: a framework for assessing research evidence. National Centre for Social Research 2003 https://www.heacademy.ac.uk/system/files/166_policy_hub_a_quality_framework.pdf Accessed 11 May 2018.

Fusch PI, Ness LR. Are we there yet? Data saturation in qualitative research Qual Rep. 2015;20(9):1408–16.

Google Scholar  

Robinson OC. Sampling in interview-based qualitative research: a theoretical and practical guide. Qual Res Psychol. 2014;11(1):25–41.

Article   Google Scholar  

Sandelowski M. Sample size in qualitative research. Res Nurs Health. 1995;18(2):179–83.

Article   CAS   Google Scholar  

Sandelowski M. One is the liveliest number: the case orientation of qualitative research. Res Nurs Health. 1996;19(6):525–9.

Luborsky MR, Rubinstein RL. Sampling in qualitative research: rationale, issues. and methods Res Aging. 1995;17(1):89–113.

Marshall MN. Sampling for qualitative research. Fam Pract. 1996;13(6):522–6.

Patton MQ. Qualitative evaluation and research methods. 2nd ed. Newbury Park, CA: Sage; 1990.

van Rijnsoever FJ. (I Can’t get no) saturation: a simulation and guidelines for sample sizes in qualitative research. PLoS One. 2017;12(7):e0181689.

Morse JM. The significance of saturation. Qual Health Res. 1995;5(2):147–9.

Morse JM. Determining sample size. Qual Health Res. 2000;10(1):3–5.

Gergen KJ, Josselson R, Freeman M. The promises of qualitative inquiry. Am Psychol. 2015;70(1):1–9.

Borsci S, Macredie RD, Barnett J, Martin J, Kuljis J, Young T. Reviewing and extending the five-user assumption: a grounded procedure for interaction evaluation. ACM Trans Comput Hum Interact. 2013;20(5):29.

Borsci S, Macredie RD, Martin JL, Young T. How many testers are needed to assure the usability of medical devices? Expert Rev Med Devices. 2014;11(5):513–25.

Glaser BG, Strauss AL. The discovery of grounded theory: strategies for qualitative research. Chicago, IL: Aldine; 1967.

Kerr C, Nixon A, Wild D. Assessing and demonstrating data saturation in qualitative inquiry supporting patient-reported outcomes research. Expert Rev Pharmacoecon Outcomes Res. 2010;10(3):269–81.

Lincoln YS, Guba EG. Naturalistic inquiry. London: Sage; 1985.

Book   Google Scholar  

Malterud K, Siersma VD, Guassora AD. Sample size in qualitative interview studies: guided by information power. Qual Health Res. 2015;26:1753–60.

Nelson J. Using conceptual depth criteria: addressing the challenge of reaching saturation in qualitative research. Qual Res. 2017;17(5):554–70.

Saunders B, Sim J, Kingstone T, Baker S, Waterfield J, Bartlam B, et al. Saturation in qualitative research: exploring its conceptualization and operationalization. Qual Quant. 2017. https://doi.org/10.1007/s11135-017-0574-8 .

Caine K. Local standards for sample size at CHI. In Proceedings of the 2016 CHI conference on human factors in computing systems. 2016;981–992. ACM.

Carlsen B, Glenton C. What about N? A methodological study of sample-size reporting in focus group studies. BMC Med Res Methodol. 2011;11(1):26.

Constantinou CS, Georgiou M, Perdikogianni M. A comparative method for themes saturation (CoMeTS) in qualitative interviews. Qual Res. 2017;17(5):571–88.

Dai NT, Free C, Gendron Y. Interview-based research in accounting 2000–2014: a review. November 2016. https://ssrn.com/abstract=2711022 or https://doi.org/10.2139/ssrn.2711022 . Accessed 17 May 2018.

Francis JJ, Johnston M, Robertson C, Glidewell L, Entwistle V, Eccles MP, et al. What is an adequate sample size? Operationalising data saturation for theory-based interview studies. Psychol Health. 2010;25(10):1229–45.

Guest G, Bunce A, Johnson L. How many interviews are enough? An experiment with data saturation and variability. Field Methods. 2006;18(1):59–82.

Guetterman TC. Descriptions of sampling practices within five approaches to qualitative research in education and the health sciences. Forum Qual Soc Res. 2015;16(2):25. http://nbn-resolving.de/urn:nbn:de:0114-fqs1502256 . Accessed 17 May 2018.

Hagaman AK, Wutich A. How many interviews are enough to identify metathemes in multisited and cross-cultural research? Another perspective on guest, bunce, and Johnson’s (2006) landmark study. Field Methods. 2017;29(1):23–41.

Hennink MM, Kaiser BN, Marconi VC. Code saturation versus meaning saturation: how many interviews are enough? Qual Health Res. 2017;27(4):591–608.

Marshall B, Cardon P, Poddar A, Fontenot R. Does sample size matter in qualitative research?: a review of qualitative interviews in IS research. J Comput Inform Syst. 2013;54(1):11–22.

Mason M. Sample size and saturation in PhD studies using qualitative interviews. Forum Qual Soc Res 2010;11(3):8. http://nbn-resolving.de/urn:nbn:de:0114-fqs100387 . Accessed 17 May 2018.

Safman RM, Sobal J. Qualitative sample extensiveness in health education research. Health Educ Behav. 2004;31(1):9–21.

Saunders MN, Townsend K. Reporting and justifying the number of interview participants in organization and workplace research. Br J Manag. 2016;27(4):836–52.

Sobal J. 2001. Sample extensiveness in qualitative nutrition education research. J Nutr Educ. 2001;33(4):184–92.

Thomson SB. 2010. Sample size and grounded theory. JOAAG. 2010;5(1). http://www.joaag.com/uploads/5_1__Research_Note_1_Thomson.pdf . Accessed 17 May 2018.

Baker SE, Edwards R. How many qualitative interviews is enough?: expert voices and early career reflections on sampling and cases in qualitative research. National Centre for Research Methods Review Paper. 2012; http://eprints.ncrm.ac.uk/2273/4/how_many_interviews.pdf . Accessed 17 May 2018.

Ogden J, Cornwell D. The role of topic, interviewee, and question in predicting rich interview data in the field of health research. Sociol Health Illn. 2010;32(7):1059–71.

Green J, Thorogood N. Qualitative methods for health research. London: Sage; 2004.

Ritchie J, Lewis J, Elam G. Designing and selecting samples. In: Ritchie J, Lewis J, editors. Qualitative research practice: a guide for social science students and researchers. London: Sage; 2003. p. 77–108.

Britten N. Qualitative research: qualitative interviews in medical research. BMJ. 1995;311(6999):251–3.

Creswell JW. Qualitative inquiry and research design: choosing among five approaches. 2nd ed. London: Sage; 2007.

Fugard AJ, Potts HW. Supporting thinking on sample sizes for thematic analyses: a quantitative tool. Int J Soc Res Methodol. 2015;18(6):669–84.

Emmel N. Themes, variables, and the limits to calculating sample size in qualitative research: a response to Fugard and Potts. Int J Soc Res Methodol. 2015;18(6):685–6.

Braun V, Clarke V. (Mis) conceptualising themes, thematic analysis, and other problems with Fugard and Potts’ (2015) sample-size tool for thematic analysis. Int J Soc Res Methodol. 2016;19(6):739–43.

Hammersley M. Sampling and thematic analysis: a response to Fugard and Potts. Int J Soc Res Methodol. 2015;18(6):687–8.

Charmaz K. Constructing grounded theory: a practical guide through qualitative analysis. London: Sage; 2006.

Bowen GA. Naturalistic inquiry and the saturation concept: a research note. Qual Res. 2008;8(1):137–52.

Morse JM. Data were saturated. Qual Health Res. 2015;25(5):587–8.

O’Reilly M, Parker N. ‘Unsatisfactory saturation’: a critical exploration of the notion of saturated sample sizes in qualitative research. Qual Res. 2013;13(2):190–7.

Manen M, Higgins I, Riet P. A conversation with max van Manen on phenomenology in its original sense. Nurs Health Sci. 2016;18(1):4–7.

Dey I. Grounding grounded theory. San Francisco, CA: Academic Press; 1999.

Hays DG, Wood C, Dahl H, Kirk-Jenkins A. Methodological rigor in journal of counseling & development qualitative research articles: a 15-year review. J Couns Dev. 2016;94(2):172–83.

Moher D, Liberati A, Tetzlaff J, Altman DG, Prisma Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009; 6(7): e1000097.

Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res. 2005;15(9):1277–88.

Boyatzis RE. Transforming qualitative information: thematic analysis and code development. Thousand Oaks, CA: Sage; 1998.

Levitt HM, Motulsky SL, Wertz FJ, Morrow SL, Ponterotto JG. Recommendations for designing and reviewing qualitative research in psychology: promoting methodological integrity. Qual Psychol. 2017;4(1):2–22.

Morrow SL. Quality and trustworthiness in qualitative research in counseling psychology. J Couns Psychol. 2005;52(2):250–60.

Barroso J, Sandelowski M. Sample reporting in qualitative studies of women with HIV infection. Field Methods. 2003;15(4):386–404.

Glenton C, Carlsen B, Lewin S, Munthe-Kaas H, Colvin CJ, Tunçalp Ö, et al. Applying GRADE-CERQual to qualitative evidence synthesis findings—paper 5: how to assess adequacy of data. Implement Sci. 2018;13(Suppl 1):14.

Onwuegbuzie AJ. Leech NL. A call for qualitative power analyses. Qual Quant. 2007;41(1):105–21.

Sandelowski M. Real qualitative researchers do not count: the use of numbers in qualitative research. Res Nurs Health. 2001;24(3):230–40.

Erickson F. Qualitative methods in research on teaching. In: Wittrock M, editor. Handbook of research on teaching. 3rd ed. New York: Macmillan; 1986. p. 119–61.

Bradbury-Jones C, Taylor J, Herber O. How theory is used and articulated in qualitative research: development of a new typology. Soc Sci Med. 2014;120:135–41.

Greenhalgh T, Annandale E, Ashcroft R, Barlow J, Black N, Bleakley A, et al. An open letter to the BMJ editors on qualitative research. BMJ. 2016;i563:352.

Download references

Acknowledgments

We would like to thank Dr. Paula Smith and Katharine Lee for their comments on a previous draft of this paper as well as Natalie Ann Mitchell and Meron Teferra for assisting us with data extraction.

This research was initially conceived of and partly conducted with financial support from the Multidisciplinary Assessment of Technology Centre for Healthcare (MATCH) programme (EP/F063822/1 and EP/G012393/1). The research continued and was completed independent of any support. The funding body did not have any role in the study design, the collection, analysis and interpretation of the data, in the writing of the paper, and in the decision to submit the manuscript for publication. The views expressed are those of the authors alone.

Availability of data and materials

Supporting data can be accessed in the original publications. Additional File 2 lists all eligible studies that were included in the present analysis.

Author information

Authors and affiliations.

Department of Psychology, University of Bath, Building 10 West, Claverton Down, Bath, BA2 7AY, UK

Konstantina Vasileiou & Julie Barnett

School of Psychology, Newcastle University, Ridley Building 1, Queen Victoria Road, Newcastle upon Tyne, NE1 7RU, UK

Susan Thorpe

Department of Computer Science, Brunel University London, Wilfred Brown Building 108, Uxbridge, UB8 3PH, UK

Terry Young

You can also search for this author in PubMed   Google Scholar

Contributions

JB and TY conceived the study; KV, JB, and TY designed the study; KV identified the articles and extracted the data; KV and JB assessed eligibility of articles; KV, JB, ST, and TY contributed to the analysis of the data, discussed the findings and early drafts of the paper; KV developed the final manuscript; KV, JB, ST, and TY read and approved the manuscript.

Corresponding author

Correspondence to Konstantina Vasileiou .

Ethics declarations

Ethics approval and consent to participate.

Not applicable.

Consent for publication

Competing interests.

Terry Young is an academic who undertakes research and occasional consultancy in the areas of health technology assessment, information systems, and service design. He is unaware of any direct conflict of interest with respect to this paper. All other authors have no competing interests to declare.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional Files

Additional file 1:.

Editorial positions on qualitative research and sample considerations (where available). (DOCX 12 kb)

Additional File 2:

List of eligible articles included in the review ( N  = 214). (DOCX 38 kb)

Additional File 3:

Data Extraction Form. (DOCX 15 kb)

Additional File 4:

Citations used by articles to support their position on saturation. (DOCX 14 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Cite this article.

Vasileiou, K., Barnett, J., Thorpe, S. et al. Characterising and justifying sample size sufficiency in interview-based studies: systematic analysis of qualitative health research over a 15-year period. BMC Med Res Methodol 18 , 148 (2018). https://doi.org/10.1186/s12874-018-0594-7

Download citation

Received : 22 May 2018

Accepted : 29 October 2018

Published : 21 November 2018

DOI : https://doi.org/10.1186/s12874-018-0594-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Sample size
• Sample size justification
• Sample size characterisation
• Data adequacy
• Qualitative health research
• Qualitative interviews
• Systematic analysis

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

ORIGINAL RESEARCH article

A qualitative study of health workers' experiences during early surges in the covid-19 pandemic in the u.s.: implications for ongoing occupational health challenges.

• Department of Health Promotion and Policy, School of Public Health and Health Sciences, University of Massachusetts, Amherst, MA, United States

Background: Health workers (HWs) have faced significant threats to physical and psychological health during the COVID-19 pandemic. The recent surges associated with the spread of the delta variant in the U.S., coupled with political resistance to effective public health mitigation strategies, indicate that the risks experienced early in the pandemic are not likely to abate soon. This study sought to better understand the experiences, thoughts, concerns, and recommendations of HWs during one of the first major surges in the U.S. and to explore how these experiences might inform efforts to mitigate potential ongoing COVID-related negative health and psychological impacts on HWs.

Methods: HWs were recruited using a multi-faceted approach tailored to public health mitigation guidelines. Semi-structured interviews were conducted via video conference with front line HWs, support staff, and opioid use disorder service organization providers between April 1 and July 9, 2020 using the Social-Ecological Model as a framework. Interviews were audio-recorded and professionally transcribed; transcripts were analyzed inductively and deductively using thematic analytic methods, generating major themes and subthemes.

Results: A total of 22 HWs participated in the study; 14 were female; 3 identified as a member of a racial or ethnic minority population. Major themes identified included: (1) Institutions, Infrastructure, and the Pandemic; (2) Working Under Fire; (3) The Political Becomes Personal and (4) Hope. Themes and subthemes explicated the ways in which phenomena at personal, interpersonal, community, organizational, and societal levels affected HWs experiences and suggested potential mechanisms through which negative effects on HW mental health and health may be mitigated.

Conclusions: Previous global infectious disease epidemics have had profound negative effects on HWs' health and mental health. This study suggests the potential for similar negative impacts that may be exacerbated by the U.S.'s current sociopolitical milieu. Efforts to systematically describe and quantify these effects and to intervene to mitigate them are warranted.

Introduction

As of September 9, 2021, COVID-19 had sickened more than 222 million people globally and killed more than 4.6 million ( 1 ). The U.S. currently has one of the highest number of cases per 100,000 people in the world and despite having only 4% of the world population, has accounted for an estimated 14% of deaths worldwide ( 2 ). New York and Massachusetts experienced major surges in COVID-19 cases in the early days of the U.S. pandemic when little was known about the virus's infectivity, its range of clinical manifestations, or effective treatments. The U.S.'s lack of public health infrastructure and the absence of a coordinated federal government response left state governments and health care systems struggling to procure personal protective equipment (PPE), reliable diagnostic tests, and testing supplies. Health systems and health workers (HWs) had to rapidly develop and test new clinical protocols and learn how to function in the face of great uncertainty. These and other challenges placed a substantial strain on health systems, service organizations, and their HWs. Failure to achieve widespread acceptance of effective mitigation strategies such as masking, social distancing, and vaccination, has led to multiple subsequent surges, extending the strain on health systems, service organizations, and HWs.

Studies of previous global infectious disease epidemics showed significant health and mental health consequences for HWs ( 3 – 6 ). A highly publicized suicide of a New York emergency medicine physician on April 26, 2020 was an early warning sign that the COVID-19 pandemic might also have serious consequences for HWs' mental health ( 7 ). Although there is currently no systematic approach to calculating the number of excess deaths among HWs related to COVID-19 in the U.S., a report by the National Nurses Union in September 2020 suggested that the excess HW deaths due to COVID-19 was more than 1,700 ( 8 ). The Kaiser Health Network, in partnership with The Guardian, has been tracking the number of COVID-19-related HW deaths, estimating ~3,176 excess deaths by mid-January 2021 ( 9 ), a number believed to be a substantial undercount due to the lack of robust tracking systems. Some of the earliest studies of the mental health impacts of the COVID-19 pandemic on HWs took place in countries such as China that experienced high infection rates in January and February, 2021. Relying primarily on survey data, these international studies suggested that HWs were experiencing increased rates of depression, anxiety, post-traumatic stress, and burnout ( 10 – 15 ). U.S.-based studies on the impacts of COVID-19 on HWs have largely consisted of surveys focused on mental health and have also demonstrated increased rates of depression, anxiety, and substance use ( 16 , 17 ).

Prior studies of HWs during the COVID-19 pandemic have provided important descriptions of the potential negative impacts of the pandemic on HWs' mental health. However, a more nuanced understanding of HWs' experiences during the pandemic, including the potential ways in which institutions and social phenomena may affect their experiences, is needed to be able to decrease the occupational risks experienced by HWs for approaching 2 years. This qualitative study aimed to address this gap in knowledge through interviews with U.S. HWs' across multiple health sectors during the earliest surges in the U.S.

Theoretical Framework

Although there were no empiric data on the effects of the COVID-19 pandemic on HWs when this study was undertaken, the SARS epidemics in China and Hong Kong suggested that HWs could be at risk for pandemic-related mental health sequelae ( 3 – 6 , 18 ). The existence of the prior experiences provided a rationale for using both inductive and deductive approaches to studying HWs' COVID-19 experiences. The Social-Ecological Model (SEM) served as the theoretical framework for the study ( 19 ) and data collection and analysis were structured to allow for new theory to emerge ( 20 ). We selected the SEM because it posits that human development and behavior are influenced by nested individual, interpersonal, community, organizational and broader societal levels of influence. The model has been used extensively in public health and health care as an explanatory model and as an intervention framework. Part of the model's applicability for this study lies in its reflection of the nuances of complex systems of human health. The earliest days of the COVID-19 pandemic suggested that geopolitical, national and state governments, health care systems, community and individual factors would shape and be shaped by the pandemic. The SEM's inclusion of interrelated micro-, meso-, and macro-level social constructions makes it an apt theoretical framework for exploring nuances of HW's experiences with the COVID-19 pandemic.

Study Population and Recruitment

HWs participated in semi-structured interviews conducted via Zoom between April 1 and July 9, 2020. HWs were defined broadly as nurses, medical assistants, clerical staff, janitorial and food service staff, social workers, physicians, pharmacists and pharmacy technologists, psychologists, and community-based substance use service providers to be able to capture a broad range of experiences. A multimodal approach to recruitment was used in response to restrictions related to COVID-19 and anticipation that health workers might have limited availability. The first wave of recruitment included a convenience sample of HWs who were known professionally or personally by research team members and who worked in states experiencing current or recent surges. In the second wave of recruitment, leaders of a community-based coalition of organizations that provide services for people with opioid use disorder (OUD) in Massachusetts partnered with the research team to send an invitation to members on its list serv. One of the 16 people recruited during the first phase ultimately was unable to participate in an interview because they were too busy. Seven were recruited in the second wave; the total number on the list serv was not known. All interviewees were also asked to suggest additional HWs to interview (snowball sampling) ( 21 ) and three of the participants were recruited through this method. A letter of invitation that included the purpose of the study, details of what would be asked of participants, and contact information for the investigative team was sent via email with an attached consent form that was reviewed with participants at the start of interview sessions. Zoom interviews were scheduled at a mutually agreed upon time. The study was approved by the University of Massachusetts's Institutional Review Board.

A semi-structured interview guide was developed and pilot-tested for clarity and completeness. Question development was guided by the five SEM levels (individual, interpersonal, community, organizational, societal). The interview guide consisted of open-ended questions with probes related to key areas of interest, including the personal effects of social distancing/quarantine, family and community effects, organizational factors related to the participants' experiences, trusted sources of information, opinions regarding government responses, and positive impacts of the pandemic. The interview guide was amended in an iterative process to add probes as new concepts emerged during interviews. Interviews were conducted by SG, who has extensive experience with qualitative research methods ( 22 – 28 ) including studies in which she trained and supervised students and research assistants, as she did for the current study (KW, MF, NP, EC, and KC), and KC. Interviews were conducted via Zoom using audio and video in compliance with social distancing and travel restrictions in Massachusetts at the time. Interviews were conducted until data saturation was reached, defined as no new concepts emerging over three consecutive interviews, and achieved after ~16 interviews. Between 20 and 30 interviews were estimated to be needed to achieve data saturation based on the homogeneity of work setting among participants and the narrow focus of the study. This estimate was based on recommendations in methodological texts and papers as well as publications of similar studies.in journals with high impact factors ( 29 – 32 ). Interviews were audio-recorded and professionally transcribed verbatim and field notes were taken during the interviews.

Interview transcripts were analyzed using thematic analysis, applying a validated rapid analytic technique ( 33 ). This approach was utilized because it has been identified as an important method for research questions that address rapidly changing health and public health risks, such as those presented by the COVID-19 pandemic. Four members of the investigative team (KC, MF, EC, NP) reviewed a subset of the transcripts to familiarize themselves with the interview content. SG generated a template of broad themes derived from the interview questions for the first phase of the analysis; an open category was included on the template to allow for inclusion of concepts that may have fallen outside of either the structured portion of the template or the theoretical framework. Using a deductive approach, analysts each read a subset of the transcripts and identified key concepts which were entered on a separate form for each transcript. With supporting quotations. SG independently analyzed one of each of the other analysts' transcripts to assess agreement; differences in key concepts were resolved through discussion with the full team. In Phase 2 of the analysis, SG identified unifying themes across the key concepts identified in Phase 1 and applied an inductive approach to theorize connections between SEM levels. These unifying themes and connections were discussed with the full team and revised based on discussion, resulting in a comprehensive set of major themes and subthemes. A summary of themes was sent to participants for review and comment (member checking). During the analysis, reflexivity was considered and discussed. Considerations included that SG is a HW (primary care pediatrician and internist), a parent of school-aged children, and teaches an undergraduate-level elective on the U.S. opioid epidemic. EC, MF, NP, and KW had SG as a professor in an elective course in the spring semester of 2020, KC was a master's student advisee of SG's, and MD is an undergraduate research assistant working with SG.

Of the 22 HWs interviewed, 14 identified as female; 1 as Black, 2 as South Asian, and 18 as Caucasian/white; 7 were physicians and 6 worked in organizations serving people with opioid use disorder (OUD) ( Table 1 ). Concepts pertinent to the study's aims were identified at all SEM levels. Major themes are organized beginning with the outermost context of the SEM (Societal), such as government and state responses, and moving to the innermost (Interpersonal and Personal), such as impact on families, with a final broad theme. The interconnectedness of the SEM levels means that subthemes for each major theme often touch on multiple levels. Major themes included: (1) Institutions, Infrastructure, and the Pandemic (Societal level); (2) Working Under Fire (Organizational, Community, Interpersonal, and Personal levels); (3) The Political Becomes Personal (Interpersonal and Personal levels); and (4) Hope (Societal, Organizational, and Community levels). Major themes and subthemes are described in detail below with supporting quotations. Additional quotations are located in Table 2 . Participants' roles in the health care system and a unique study identifier are included in parentheses after quotations; additional details were not attributed to quotations to protect participant confidentiality.

Table 1 . Participant demographic characteristics.

Table 2 . Selected supplementary illustrative quotations.

Theme 1: Institutions, Infrastructure, and the Pandemic

Many HWs commented on the role of institutions and infrastructure in combatting the pandemic. The phenomena described in this theme were located primarily at the SEM Societal level. Subthemes included: (1) Federal, and State Government Responses: Coordination Confusion; (2) The Syndemic of Misinformation; and (3) Rules of the (Capitalist) Game. Included in these subthemes are perspectives on how international and domestic political and public health institutions shaped the early arc of the pandemic, and how institutions' responses shaped HWs' and others' perceptions of the institutions. Some HWs discussed the role of social media and the press in the pandemic response. Many HWs felt the lack of public health infrastructure impacted HWs' experiences early in the course of the pandemic and reflected on institutional failures in response to the pandemic.

Federal and State Government Responses: Coordination Confusion

A small number of HWs felt that the problems the U.S. was experiencing in its pandemic response were inevitable:

“ I don't know that this could have been avoided… I don't really have a lot of criticism of the government right now .” (Nurse practitioner, RJ3)

Most HWs felt that the federal government's response to the pandemic was inadequate and a major contributor to the rapidly worsening state of the pandemic in the U.S.

“ ... it seems pretty clear that it's [federal government response] not been coordinated… not been systematic. It's been seemingly random, at times brutal, at times… unjust… lacking compassion…It's frustrating to see… states are having to design their own disaster plans, because the power of the federal government is to align disaster response across states… instead, we've got a hodgepodge.” (Hospitalist, SG1)

One HW with knowledge of the federal government's disaster preparedness programs was puzzled as to why prior plans for such a pandemic were not being implemented.

“ It has been very… hard for me to understand… what came of all the [preparedness] programs... and whether or not they [agencies responsible for programs] were allowed to [be involved in decision-making], during what most people would call the Superbowl [of pandemic preparedness]. We've been talking about it for decades… We have a disinterested health care system. You have a totally underfunded public health system… the knowledge around the pandemic… exists in the military and the National Security Council. It does not exist in health departments.” (ED physician/administrator, SG2)

Some HWs noted and worried about the loss of trust in the federal government, including agencies such as the Centers for Disease Control and Prevention (CDC).

“ ... the erosion of faith in federal leadership and the ability to believe what the CDC was saying…, to believe that there would be an organized response… it is unconscionable and it's impossible for me to get my head around the damage that it has done forever.” (ED physician, SG3)

In comparison, although some HWs also felt their state governments could have instituted lockdowns sooner, most felt that their governors and state governments had stepped into the leadership void to provide clear, science-based, public health messaging and leadership.

The Syndemic of Misinformation

“ … we've got this, like, false information pandemic.” ( ED physician/researcher, SG6 )

Some HWs felt that it was difficult to obtain reliable up-to-date information about the pandemic due to the rapidly evolving state of knowledge about COVID-19. Others felt the volume of rapidly disseminated misinformation was problematic.

“ We sort of have this infodemic…the big examples...are…ACE inhibitors and Ibuprofen… and the hemoglobin hijacking theory... All these things start and then they get amplified by social media.” ( ED physician/researcher, SG6)

HWs described some of the reasons they did not trust certain sources of information.

“ When you have some big personalities… telling people that this is the way to do things and if you're not… you're wrong and you are killing people… when you start… speaking in absolutes… I start to… lose… respect for you as an authority… I worry that their concern is about public image and not so much getting the right answer .” (ED physician, SG8)

Social media was generally seen as unreliable because of the lack of its lack of scientific rigor.

“ I use Twitter professionally... and I actually shut it off about two or three weeks into COVID because I was...going to bed [and] I'm looking at Twitter and reading about … this person died and that person died... it was good for me to absorb...what was going on. But now I just shut Twitter off…because I'm like, ‘This is not helpful to me'… it was too anecdotal.” (Psychiatrist/researcher, SG9)

Many physicians relied on peer-reviewed journals and their medical societies for up-to-date accurate information. However, several also expressed concern that the quality of the studies published and the lack of understanding about “preprints” (papers reporting results of studies that have not been peer reviewed) were problematic.

“ It's been an interesting time in research because part of the [misinformation] problem are these preprint servers… they're not peer reviewed… I see many go straight through to other journals without significant changes even though there should be… it's a reminder that research is very much flawed…” (ED physician/researcher, SG6)

Rules of the (Capitalist) Game

One HW contrasted health care and public health systems' financial models and cultures, in the context of trying to understand the federal government's response.

“ Public health just doesn't really have a business model. It has a budget. It comes to them from tax dollars,...or grant…and health care does not think like that… within public offices, especially at the federal level, the … the understanding of how corporate health care is, is shockingly absent. Public sector folks come from public health departments. They're like do-gooders... and I say ‘Have you ever been inside a hospital... ever… had to cut budgets because you wanted to maximize margins to recruit a high-powered neurosurgeon?' and they're like, ‘What are you talking about?”' (ED physician/administrator, SG2)

Some HWs felt that the U.S.'s capitalist political economy was not structured to mount an effective response.

“… because there was no coordinated federal response, we were bidding against other hospitals, bidding up the price of PPE... in this super dysfunctional way… there are times when capitalism really doesn't work and this is one of them.” (Hospitalist, SG1)

“ One thing that bothered me was that we had this big meeting… where he had all the different CEOs of different companies come on and talk about what they were going to do. It felt like such a capitalist take .” (Pharmacy technician, EC3)

Another HW felt, similarly, that the U.S. response was what one would expect based on institutional structures.

“ Don't hate the player, hate the game – I see most of us, all of us, as just responding to the rules of engagement and the incentives that are outlined for us, right?... You can bellyache about how the rules of the game are not what you wish they were or you can try to rewrite the rules of the game.” (ED physician/administrator, SG2)

Theme 2: Working Under Fire

HWs discussed changes in their work since the surges began and the work-related challenges that they faced. Subthemes included: (1) The Myth of the Health Worker Hero; (2) Changes in Clinical Practice; (3) Organizational Leadership; and (4) Work Related Worries, Fears, and Loss. This major theme described HWs experiences that illustrated the toll the pandemic, government response, and citizens' behaviors was taking.

The Myth of the Health Worker Hero

A few physicians expressed frustration with society's framing of HWs as “heroes” and the unfair expectations they felt society had of HWs:

“ I'm gonna say that this [public applauding of HWs] has been seen as a positive… I think it's a really big negative. This whole… hero worship thing. I think it's nice that people are appreciating their nurses and their doctors and their health care providers…but I also think that this is sort of the problem… we just expect people [HWs] to fix things. We want a hero to come with their superpowers and just make it all better… we were telling people to stay home if you're sick… just wash your hands… and we got to a point, nobody wanted to listen and we have this whole, consequence because of it and now, we want our [HW] heroes to come in and clean up our mess. It just doesn't work that way. It just doesn't.” (ED physician, SG8)

“ I'm thinking about this one tweet. Early on, a nurse said she was taking a break from her job because she had an underlying health condition… People were attacking her, being like, oh you signed up for this. Yes, she signed up to care for people but under the caveat that she'd be provided the proper PPE and she wouldn't have to risk her own life .” (Pharmacy technician, EC3)

Some HWs also were puzzled, frustrated, or upset by factions of the public claiming the pandemic was not “real” and that people were not taking the pandemic seriously.

“ To see others not treating it [COVID] as a threat is disrespectful to not only… my family [who are HWs] , but other workers and essential workers .” (Pharmacy technician EC3)

Changes in Clinical Practice

“ It's completely disrupted our normal way of practicing medicine.” (ED physician, NP1)

When HWs talked about their experiences at work, much of the discussion centered on positive aspects of their experience and solidarity among coworkers. There were recommendations for continuing some of the changes in clinical practice and hope that they would result in long-lasting improvements.

“ [I] think that our teamwork in the ED is always fantastic… techs, nurses and EMS…but right now…we have a super appreciation because you're in a PPE room… and you have to do everything … to kind of spare people [co-workers] having to come in… I think that culture of collaboration, which we typically have been good at… is even enhanced and I would like to see that go forward.” (ED physician/researcher, SG6)

Some also discussed support from the community.

“ ... I know that a lot of local universities as well as hotels have been offering their spaces for health care workers that don't want to stay at their homes… to prevent transmission .” (Pharmacy tech, EC3)

Some HWs welcomed the increased use of telemedicine and the flexibility it provided,

“ ...[telemedicine] takes away...all these other barriers… people have huge transportation issues and can't get to their appointments or they don't feel like rolling out of bed and leaving their house to take… a half-hour bus ride to come here to see their therapist. They can actually have a conversation with their therapist while they're lying in bed. I mean people who are depressed, who are agoraphobic, who have anxiety leaving the home, … [telemedicine] takes away all of that.” (OUD treatment services manager, KC1)

while others felt telemedicine presented new challenges and potential inequities in care delivery.

Organizational Leadership

Several HWs discussed how the response of leadership in their hospitals or organizations affected their experience with the pandemic surge. Some comments were positive,

“ The director is a very passionate but very direct person, a very practical person. Her supervision style has completely changed through this to be much more in tune with how the employees are doing mentally, much more in tune with our self-care, telling us it's okay to say no .” (OUD treatment coordinator, KC7)

and some were negative.

“ They [workplace supervisors] are like, ‘… you're not gonna wear N95 masks. You don't need to wear it.' Really? Okay. Tell that to the nurse who didn't wear an N95 mask and ended up with COVID.” (Hospital-based social worker, MF2)

“… they [administrators] were… mandating things that weren't feasible and forcing employees to … not do something because it wasn't really possible … [or] be insubordinate in order to actually do their job. … people who work in administration don't want to defer to people who do the actual work .” (OUD treatment coordinator, KC3)

Work-Related Worries, Fear, and Loss

HWs shared a broad range of work-related concerns in relation to the pandemic. Categories within this subtheme included health disparities and vulnerable populations, foregone health care, loss of personal connection with patients, loss of trust in the health care system, financial concerns, and provider burnout.

Worry about the pandemic's impact on people in vulnerable populations and the associated emerging racial/ethnic and socioeconomic disparities in who was contracting and dying from COVID-19 was among the most commonly discussed concerns.

“ … you got a white collar worker who can still work at home…it's still stressful… but nobody knows what other people are dealing with… I feel like it's going to be very easy to really not know how much other people are struggling…I think it has the potential to really worsen inequality in the country, but… sort of quietly and invisibly worsen it.” (ED physician/researcher, SG7)

“ I… think it's really challenging for our homeless population… I spoke to someone [known to ED staff and homeless] at the beginning of everything, and he [said], ‘That's okay [that public buildings with restrooms were closed], I'm just not going to eat so that I don't have to use the bathroom.”' (ED physician/researcher, SG7)

Many HWs also worried about patients forgoing health care due to fear of contracting COVID-19 if they went to a health care facility.

“ I see that a lot of people aren't coming to the hospital for preventive things. There's been a spike in people experiencing strokes and heart attacks because people aren't getting the treatment they need… there is that fear of going to the emergency room ... people are going to go months, if not years, without getting their proper dental checkups or primary care checkups or eye checkups – that's going to create years and years of damage for people. That's going to create more strain later on .” (Pharmacy technician, EC3)

Some HWs spoke about the diminished opportunity for personal connection when interacting with patients while gowned, masked and gloved.

“ … one more drastic difference in my day-to-day life is that my ability to connect with patients when I am wearing a ridiculous amount of gear and they cannot see my face and I cannot see their face is terrible.” (ED physician/researcher, SG7)

Some HWs also expressed concern about loss of trust in health care systems due to pandemic-related rumors and misinformation.

“ …it did feel…sad… that there were people saying that if you had coronavirus, that we would not do CPR [at hospital x]… EMS had heard this and stopped bringing patients to our hospital for a short period of time… I wonder what's gonna happen with people and their trust in the medical community. Are they gonna feel like the hospital is super dangerous and then never ever come back?...the effects of this are gonna linger …”(ED physician, SG8)

Concern for the financial stability of health care systems made some HWs worry about job security.

“ The hospital isn't doing well [financially]. It wasn't doing well before this…I think there's going to be a lot of people being laid off…” (Medical records staff, MF5)

Finally, some HWs also worried about the long-term mental health effects working in the pandemic conditions would have on HWs, including the possibility of having to decide on allocation of scarce resources such as ventilators.

“ ... moral distress… we talk about this in the emergency room, with situations where you know what should be done and you can't do it... if you're in a place where there's not enough ventilators and you want to put someone [on a ventilator but can't unless you take someone else off]...I think that that sort of thing is going to affect health care workers all over the country . (ED physician, SG8)

Theme 3: The Political Becomes Personal

HWs described both positive and negative impacts of the pandemic on their personal lives, noting often that personal effects were closely related to work effects. Subthemes for this major theme included: (1) Health Workers as Unintended Threats to the Public's Health; (2) An Emotional Toll; (3) Family: Disruptions and Silver Linings. These subthemes were largely related to Personal and Interpersonal SEM levels. Several physicians prefaced their discussion of how the pandemic was affecting them with comments about awareness of their socioeconomic privilege.

Health Workers as Unintended Threats to the Public's Health

Although the services HWs provided were often crucial to fighting the pandemic, some worried that their high risk of contracting COVID-19 made them a risk to patients and public health. This concern was compounded by the lack of reliable, rapid testing in the U.S. early in the pandemic. This may have contributed to the moral hazard experienced by health workers, whose work is intended to improve health and wellness and cause no harm.

“ I'm concerned with being able to get tested because I guess the fear is, jeez, what if I have the virus but I'm asymptomatic and I give it to someone else?” (Chiropractor, EB2)

An Emotional Toll

Many HWs described ways in which their experiences as a HW during the pandemic were negatively affecting their physical and emotional health and the impact of their work on the mental health of their children.

“ … my [own] sobriety has probably never been shakier than it has been during this time.… I … drove by a package store for 45 minutes. Back and forth having conversations in my head, ‘Who will know? What does it matter?' Luckily, I was up to the task, and it was just … a waste of gas and time, but I can definitely understand people with less momentum [with recovery] struggling even harder, because I wasn't reaching out and asking for help. I haven't been going to in-person meetings, so I don't get to see these people and let them know how I'm doing either.” (OUD treatment coordinator, KC7)

“ [my] 10 year old [has] told me many nights she just has a hard time turning her brain off because she's worried about people… an adjustment is having these big conversations with her, grownup conversations…people are dying.” (Nurse practitioner, RJ3)

Family: Disruptions and Silver Linings

The personal impacts that HWs described were often closely related to their family situation. For example, all public schools and many private schools in New York and Massachusetts had abruptly transitioned to on-line remote learning in mid-March and most daycares closed at this time, meaning HWs who were also parents were faced with needing to care for young children and supervise older children doing remote schooling while working.

In addition to worrying about putting the public at risk because of HWs' high risk of exposure to COVID-19, HWs also worried more about the risk they put their families at than the risk to themselves.

“ I know the risk associated with being an emergency physician but I signed up for those risks… that's what I do, that's part of my job. My family didn't… it's an emotional toll for all of us with a concern that we may bring this home to our family members .” (ED physician, NP1)

Some HWs also found that lockdown had positive aspects for their family.

“… it's like if you have a closet and... you're thinking of throwing things out of your closet ‘cause you wanna simplify things. But...if you have something that you bought and… that stuff is really nice and you used to wear it. You're like, you don't wanna get rid of it'cause you bought it and it's hard to let it go. But when you come into an empty closet, you could just buy what you wanna buy. You fill it with what… you're gonna wear and what you wanna have. So now it's like someone just came in and… cleaned out the closet. And now I can… add stuff back as I want ‘cause I'm not… trying to empty out a really full closet .” (Psychiatrist/researcher, SG9)

Others found it difficult to get work done while juggling home responsibilities and that increased work demands reduced the time they spent with their families.

“ I've always been kind of a workaholic but… week two of March through …the first week of April, I worked at least 18 hours a day… minimum… I'm actually spending the least amount of time with my kids… [than I have ever spent] in their whole lives...” (ED physician/researcher, SG6)

Some HWs found themselves questioning the importance of some of the non-clinical aspects of their work and the ways in which they had previously structured their days, with some hoping to preserve some of the slow down once lockdowns and social distancing requirements ended.

“ …probably for at least a month I was very scatterbrained and I was just like… first of all, who cares about [focus of HW's research] right now, you know?... Before this [interview] I had two meetings, one with primary care and one with surgeons about [research].... I was like ‘I do not give a shit'… it's not relevant right now…maybe it will be relevant again.” (ED physician/researcher, SG7)

Theme 4: Hope

In addition to silver linings discussed by HWs, some also described hopes that the pandemic could result in broader positive societal changes. The concepts related to this theme centered on Societal, Organizational and Community SEM levels.

“ I feel like if this country can use the pandemic to… [institute] paid sick leave, to pass policies that should have existed and should have been in place long ago… that could be a lovely silver lining…that we say, ‘Oh, we actually need to take care of everyone with these things rather than every man for themselves.”' (ED physician/researcher, SG7)

Some HWs hoped that the chaotic and fragmented response by the government and the health care system might advance discussions about addressing problems, big and small, in the health care system.

“ I am hopeful that the health care system will… in light of the PPE issue… think more about waste…like the amount of stuff we throw out that could be reusable .” (ED physician/researcher, SG7)

Interpretations in Relation to the Social Ecological Model

“ Man is an animal suspended in webs of significance he himself has spun.” – Clifford Geertz ( 34 )

The major themes and subthemes identified can be interpreted as a web of interactions involving the SEM levels ( Figure 1 ). In the U.S., historical social, cultural, and political phenomena have generated a strongly individualistic society with a largely unregulated capitalist political economy ( 35 ). The health care and public health systems that developed within these socio-political boundaries are largely siloed and divorced from each other ( 36 ). As one HW noted, these two systems in general, and unregulated capitalism in particular, are poorly designed for addressing pandemic challenges. The absence of effective federal leadership may have made it all the more challenging to overcome the limitations of the U.S.'s sociopolitical system design, or as one HW framed it, the “Rules of the Game”. Despite the obstacles these “Rules” can present, the thoughts and experiences HWs participating in this study described suggested how actions taking place at organizational, personal, and interpersonal levels may mitigate the effects of failures at societal and structural levels.

Figure 1 . Positive and negative influences on Health worker health across social ecological levels.

The COVID-19 pandemic's rapidly changing landscape has presented political, public health, medical, community, and individual challenges. Understanding the acute and long-term occupational health effects on HWs will likely require a diverse set of research methodologies. This study was one of the first to take an in-depth qualitative exploration of HWs experiences, feelings, and perceptions during the first surges of COVID-19 infections in the U.S. While much has been learned about the clinical and epidemiological aspects of the COVID-19 virus since data for this study were collected, narrative data such as reported in this study provide the context needed to address the social complexities of the pandemic ( 37 ). The U.S. continues to lack a coordinated federal response and public health mitigation strategies such as mask-wearing and vaccination have become deeply politicized, meaning that HWs' heavy workloads, isolation, anxiety, grief, and death are likely to continue for some time. As the virus spreads unabated across the nation and overwhelms health care systems, the narratives shared by the HWs who participated in this study offer insights into the potential short- and long-term impacts the pandemic may have on HWs' physical and mental health as well as potential approaches to mitigating risk. The themes identified in this study illuminate ways in which institutional and infrastructural failures have likely played a role in the U.S.'s pandemic experience but also showed sources of community, interpersonal and personal resilience, resourcefulness, and hope.

HWs in the U.S. have long had some of the highest rates of job-related stress, burnout, and suicide ( 38 , 39 ). Prior infectious disease epidemics, such as SARS and Ebola, demonstrated the disparate mortality, physical, and mental health effects such epidemics can have on HWs ( 40 ). The COVID-19 pandemic is affecting HWs across the globe on a scale not seen for more than a century. In the U.S., poor public health preparedness, lack of a coordinated national response, and failure of the federal government to act swiftly using the best scientific data available may have put HWs at even greater risk for physical and mental health sequelae than they might have otherwise experienced. A study led by the National Nurses Study and the Kaiser Foundation published in a special report by the National Academies of Science, Engineering, and Medicine described how the nation's lack of a uniform system to collect, collate and report illnesses and deaths of HWs related to COVID-19 impairs the ability to accurately monitor and develop interventions to mitigate HWs' risks ( 41 ). The report calls for a national system to track not only deaths directly due to work-related COVID-19 infection but collateral deaths, such as suicides due to the fatigue, stress, and burnout, and mental health morbidity. The report notes that accuracy of reporting improved significantly for nursing homes after the Centers for Medicaid and Medicare Services (CMS) implemented a new reporting policy in May of 2020 that included penalties for failure to comply, demonstrating that accurate collection of these data is possible. The results of the current study suggest that development of a robust system to rapidly track the effects of the pandemic and identification of best practices to mitigate the pandemic effects on HWs should be a national priority. These suggestions and other interventions suggested by this study, are consistent with the World Health Organization's Maintaining essential health services: operational guidance for the COVID-19 context , which outlines 10 operational strategies for maintaining essential health services, which involves protection of health workers' physical and mental health ( 42 ).

The themes identified in this study raise questions about the role of HWs in society and HWs' ethical responsibilities. While some HWs felt that the work-related risks they were experiencing were part of “what they signed up for,” others questioned whether their responsibility extended to putting their lives and their families' at risk. This questioning may have been magnified by the perceived lack of support for HWs evidenced by the federal government undermining public health messages about mitigation, failure to help procure adequate PPE, and large portions of the population electing not to wear masks, practice social distancing, or, once vaccines became available, to be vaccinated. Although nurses and other staff have unionized in some regions of the country, a large portion of health workers in the U.S are not part of a labor union. Working conditions during COVID have renewed interest in health worker organization ( 43 , 44 ). Physicians historically have not been thought of as workers requiring labor protection, but as employment arrangements have shifted so that more physicians are employed by hospital systems than working in private practice, the pandemic experience may raise questions as to whether there may be a role for more extensive labor organization in the future across all HW roles.

Even in the face of major stressors, themes of HW professionalism, caring, and hope emerged. A narrative review of resilience strategies to manage psychological distress among health care workers during the COVID-19 pandemic published in June 2020 suggested several approaches that tie to the current study's themes ( 45 ). For example, organizational justice and organizational strategies, including staff feedback sessions and demonstrating support for workers, link to the Organizational Leadership subtheme in the current study. It remains unclear what the widespread disregard for HWs' and their families' health and mental health may mean for sustaining an adequate workforce in some health professions or how to make clear the toll the behaviors that suggest lack of regard has had.

Limitations

The results of this study should be considered in the context of its limitations. First, one of the goals of the study was to interview HWs in the midst of the first surges in the country. Because the duration of surges in Massachusetts and New York could not be predicted at the time and it was not certain other surges would follow, we relied on a convenience plus snowball sampling approach to recruit participants. This allowed us to recruit HWs quickly and include HWs with a diversity of roles in health care. This also meant that the HWs who were interviewed by someone they knew may have been more or less willing to share controversial or critical thoughts but the criticisms and difficult topics discussed by HWs suggest familiarity may have facilitated openness. The majority of HWs interviewed were white; interviews with Black, Latinx and otherwise socially marginalized HWs as well as HWs in the lowest paying HW jobs may have generated additional themes or alternative views of the themes identified. Interviewers were affiliated with a university in a state with a tradition of liberal politics. Although political affiliation data were not collected, the political divisions in the country may have meant that different perspectives may be obtained in states that experienced surges later in the course of the pandemic with differing majority political views. Some questions in the interview guide were tailored to the earliest days of COVID-19 spread and may have less relevance at the current stage of the pandemic. Although the interviews for this study were conducted in the beginning of the pandemic, the mental health and physical consequences affecting HWs are likely to continue and potentially worsen as infection and death rates continue to climb.

Conclusions

This study of U.S. HWs experiences in the early days of the COVID-19 pandemic generated important narrative insights into the unique physical and psychological risks to HWs and similarities to risks identified in prior serious respiratory viral infection epidemics such as SARS-CoV. Although little of the response to the pandemic to date has involved a coordinated effort at the federal or other level, it is of urgent importance that the health and well-being of HWs be protected. The potential need for change at multiple levels of the SEM that were suggested by this study that could be tested in a large representative sample of health workers. For example, at the Societal level, combining the national databases tracking health worker infection rates, morbidity and mortality, coordinating efforts to implement evidence-based protective interventions in health care settings while also trying to understand address the forces that have reduced concern for the collective good and addressing the problematic capacity for emergency response are high level needs. At the Organizational level, management training and guidelines for rapid assembly and performance of an incident command center may help support the coordination needed to protect health workers. Finally, making mental health care support and options for family support more readily accessible and affordable could potentially offer better support at the individual level.

Data Availability Statement

Raw data will be made available on reasonable request with any data that may risk loss of confidentiality redacted.

Ethics Statement

The studies involving human participants were reviewed and approved by University of Massachusetts, Amherst. The Ethics Committee waived the requirement of written informed consent for participation.

Author Contributions

SG conceived of the study, trained team members, interviewed participants, led the analysis, and drafted the manuscript. KC, MF, EC, and NP made intellectual contributions to study development, interviewed participants, participated in the analysis, contributed to manuscript review and editing, and approved the final version. MD and KW made intellectual contributions to study development, participated in the analysis, contributed to manuscript review and editing, and approved the final version. All authors contributed to the article and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The authors would like to thank the study participants for giving generously of their time and Cherry Sullivan and Michele Farry for connecting us to coalition members and their thoughtful suggestions on topical areas of interest to their community.

1. Pettersson H, Manley B, Hern S. CNN. Tracking Covid-19's global spread . CNN. Available online at: https://www.cnn.com/interactive/2020/health/coronavirus-maps-and-cases (accessed September 9, 2021).

Google Scholar

2. Covid World Map: Tracking the Global Outbreak - The New York Times. Available online at: https://www.nytimes.com/interactive/2020/world/coronavirus-maps.html?pageType=LegacyCollection&collectionName=Maps+and+Trackers&label=Maps+and+Trackers&module=hub_Band&region=inline&template=storyline_band_recirc (accessed November 17, 2020).

3. Chua SE, Cheung V, Cheung C, McAlonan GM, Wong JW, Cheung EP, et al. Psychological effects of the sars outbreak in hong kong on high-risk health care workers. Can J Psychiatry. (2004) 49:391–3. doi: 10.1177/070674370404900609

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Chan-Yeung M. Severe Acute Respiratory Syndrome (SARS) and Healthcare Workers. Int J Occup Environ Health. (2004) 10:421–7. doi: 10.1179/oeh.2004.10.4.421

5. Lee AM, Wong JG, McAlonan GM, Cheung V, Cheung C, Sham PC, et al. Stress and Psychological Distress among SARS Survivors 1 Year after the Outbreak. Can J Psychiatry. (2007) 52:233–40. doi: 10.1177/070674370705200405

6. Bai Y, Li Y-C, Lin C-C, Chen J-Y, Chue C-M, Chou P. Survey of Stress Reactions Among Health Care Workers Involved With the SARS Outbreak. Psychiatric Serv. (2004) 55:1055–7. doi: 10.1176/appi.ps.55.9.1055

7. Top ER. Doctor Who Treated Coronavirus Patients Dies by Suicide - The New York Times . Available online at: https://www.nytimes.com/2020/04/27/nyregion/new-york-city-doctor-suicide-coronavirus.html (accessed February 16, 2022).

8. Sins Sins of Omission How Government Failures to Track Covid-19 Data Have Led to More Than 1 700 Health Care Worker Deaths Jeopardize Public Health . Available online at: https://www.nationalnursesunited.org/sites/default/files/nnu/graphics/documents/0920_Covid19_SinsOfOmission_Data_Report.pdf (accessed February 16, 2022).

9. Lawrence Lawrence TS of K TG, Christina Jewett O'Hagan M, Ungar L, Bailey M, Ridderbusch K, Aleccia J, Gee A. Lost on the Frontline. Kaiser Health News. (2020). Available online at: https://khn.org/news/lost-on-the-frontline-health-care-worker-death-toll-covid19-coronavirus/ (accessed February 16, 2022).

10. Nyashanu M, Pfende F, Ekpenyong M. Exploring the challenges faced by frontline workers in health and social care amid the COVID-19 pandemic: experiences of frontline workers in the English Midlands region, UK. J Interprof Care . (2020) 34:655–61. doi: 10.1080/13561820.2020.1792425

11. Sheraton M, Deo N, Dutt T, Surani S, Hall-Flavin D, Kashyap R. Psychological effects of the COVID 19 pandemic on healthcare workers globally: a systematic review. Psychiatry Res . (2020) 292:113360. doi: 10.1016/j.psychres.2020.113360

12. Eftekhar Ardebili M, Naserbakht M, Bernstein C, Alazmani-Noodeh F, Hakimi H, Ranjbar H. Healthcare providers experience of working during the COVID-19 pandemic: a qualitative study. Am J Infect Control . (2020) 49:547–54. doi: 10.1016/j.ajic.2020.10.001

13. Liu Q, Luo D, Haase JE, Guo Q, Wang XQ, Liu S, et al. The experiences of health-care providers during the COVID-19 crisis in China: a qualitative study. Lancet Glob Health. (2020) 8:e790–8. doi: 10.1016/S2214-109X(20)30204-7

14. Cabarkapa S, Nadjidai SE, Murgier J, Ng CH. The psychological impact of COVID-19 and other viral epidemics on frontline healthcare workers and ways to address it: a rapid systematic review. Brain Behav Immun Health . (2020) 8:100144. doi: 10.1016/j.bbih.2020.100144

15. Shreffler J, Petrey J, Huecker M. The Impact of COVID-19 on Healthcare Worker Wellness: a scoping review. West J Emerg Med. (2020) 21:1059–66. doi: 10.5811/westjem.2020.7.48684

16. Hennein R, Lowe S. A hybrid inductive-abductive analysis of health workers' experiences and wellbeing during the COVID-19 pandemic in the United States. PLoS ONE . (2020) 15:e0240646. doi: 10.1371/journal.pone.0240646

17. Krok-Schoen JL, Pisegna JL, BrintzenhofeSzoc K, MacKenzie AR, Canin B, Plotkin E, et al. Experiences of healthcare providers of older adults with cancer during the COVID-19 pandemic. J Geriatr Oncol. (2020) 12:190–5. doi: 10.1016/j.jgo.2020.09.021

18. Wu P, Fang Y, Guan Z, Fan B, Kong J, Yao Z, et al. The Psychological Impact of the SARS Epidemic on Hospital Employees in China: Exposure, Risk Perception, and Altruistic Acceptance of Risk. Can J Psychiatry. (2009) 54:302–11. doi: 10.1177/070674370905400504

19. McLeroy KR, Bibeau D, Steckler A, Glanz K. An ecological perspective on health promotion programs. Health Educ Q. (1988) 15:351–77. doi: 10.1177/109019818801500401

20. Abductive reasoning and qualitative research - Lipscomb - 2012 - Nursing Philosophy - Wiley Online Library. Available online at: https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1466-769X.2011.00532.x (accessed February 16, 2022).

21. Doing Qualitative Research . 2nd ed. Thousand Oaks, Calif: Sage Publications. (1999). p. 406.

22. Goff SL, Lagu T, Pekow PS, Hannon NS, Hinchey KL, Jackowitz TA, et al. A qualitative analysis of hospital leaders' opinions about publicly reported measures of health care quality. Jt Comm J Qual Patient Saf. (2015) 41:169–76. doi: 10.1016/S1553-7250(15)41022-0

23. Goff SL, Mazor KM, Priya A, Pekow PS, Lindenauer PK. Characteristics of high performing primary care pediatric practices: a qualitative study. Acad Pediatr . (2019) 20:267–74. doi: 10.1016/j.acap.2019.04.005

24. Goff SL, Mazor KM, Meterko V, Dodd K, Sabin J. Patients' beliefs and preferences regarding doctors' medication recommendations. J Gen Intern Med. (2008) 23:236–41. doi: 10.1007/s11606-007-0470-3

25. Goff SL, Kleppel R, Lindenauer PK, Rothberg MB. Hospital workers' perceptions of waste: a qualitative study involving photo-elicitation. BMJ Qual Saf. (2013) 22:826–35. doi: 10.1136/bmjqs-2012-001683

26. Goff SL, Eneanya ND, Feinberg R, Germain MJ, Marr L, Berzoff J, et al. Advance care planning: a qualitative study of dialysis patients and families. Clin J Am Soc Nephrol . (2015) 10:390–400. doi: 10.2215/CJN.07490714

27. Mann E, Goff SL, Colon-Cartagena W, Bellantonio S, Rothberg MB. Do-not-hospitalize orders for individuals with advanced dementia: healthcare proxies' perspectives. J Am Geriatr Soc. (2013) 61:1568–73. doi: 10.1111/jgs.12406

28. Moyer LJ, Carbone ET, Anliker JA, Goff SL. The Massachusetts BMI letter: a qualitative study of responses from parents of obese children. Patient Educ Couns. (2014) 94:210–7. doi: 10.1016/j.pec.2013.10.016

29. Lagrotteria A, Swinton M, Simon J, King S, Boryski G, Ma IWY, et al. Clinicians' Perspectives After Implementation of the Serious Illness Care Program: A Qualitative Study. JAMA Network Open. (2021) 4:e2121517. doi: 10.1001/jamanetworkopen.2021.21517

30. Patton MQ,. Qualitative Research & Evaluation Methods. SAGE Publications Inc. (2022). Available online at: https://us.sagepub.com/en-us/nam/qualitative-research-evaluation-methods/book232962 (accessed February 16, 2022).

31. Vasileiou K, Barnett J, Thorpe S, Young T. Characterising and justifying sample size sufficiency in interview-based studies: systematic analysis of qualitative health research over a 15-year period. BMC Med Res Methodol. (2018) 18:148. doi: 10.1186/s12874-018-0594-7

32. Vrecko S. Everyday drug diversions: A qualitative study of the illicit exchange and non-medical use of prescription stimulants on a university campus. Soc Sci Med. (2015) 131:297–304. doi: 10.1016/j.socscimed.2014.10.016

33. Gale RC, Wu J, Erhardt T, Bounthavong M, Reardon CM, Damschroder LJ, et al. Comparison of rapid vs in-depth qualitative analytic methods from a process evaluation of academic detailing in the Veterans Health Administration. Implementat Sci. (2019) 14:11. doi: 10.1186/s13012-019-0853-y

34. Geertz C. The Interpretation of Cultures: Selected Essays . New York: Basic Books (1973).

35. Turner BS. Individualism, capitalism and the dominant culture: A note on the debate. Aust N Z J Sociol. (1988) 24:47–64. doi: 10.1177/144078338802400103 (accessed February 16, 2022).

CrossRef Full Text | Google Scholar

36. Bradley EH, Taylor, LA,. The American Health Care Paradox Why Spending More is Getting Us Less. New York: PublicAffairs (2013). Available online at: http://public.eblib.com/choice/publicfullrecord.aspx?p=1319701 (accessed February 16, 2022).

37. Gubrium A, Gubrium E. Narrative complexity in the time of COVID-19. Lancet. (2021) 397:2244–5. doi: 10.1016/S0140-6736(21)01287-3

38. Shanafelt TD, Hasan O, Dyrbye LN, Sinsky C, Satele D, Sloan J, et al. Changes in Burnout and Satisfaction With Work-Life Balance in Physicians and the General US Working Population Between 2011 and 2014. Mayo Clin Proc. (2015) 90:1600–13. doi: 10.1016/j.mayocp.2015.08.023

39. Dyrbye LN, Shanafelt TD. Physician Burnout: A Potential Threat to Successful Health Care Reform. JAMA. (2011) 305:2009–10. doi: 10.1001/jama.2011.652

40. Evans DK, Goldstein M, Popova A. Health-care worker mortality and the legacy of the Ebola epidemic. Lancet Glob Health. (2015) 3:e439–40. doi: 10.1016/S2214-109X(15)00065-0

41. Rapid Expert Consultation on Understanding Causes of Health Care Worker Deaths Due to the COVID-19 Pandemic (2020). Washington, D.: National Academies of Sciences, Engineering, and Medicine (2020).

42. Maintaining Essential Health Services: Operational Guidance for the COVID-19 Context Interim Guidance 1 June 2020. Available online at: https://www.who.int/publications-detail-redirect/WHO-2019-nCoV-essential_health_services-2020.2 (accessed February 1, 2022).

43. Healthcare Workers Are Organizing Like Their Lives Depend On It. In These Times . Available online at: https://inthesetimes.com/article/healthcare-workers-labor-unions-national-nurses-united-nurses-united-ppe-shortages (accessed February 16, 2022).

44. For Health Care Workers The The Pandemic Is Fueling Renewed Interest In Unions. NPR.org . Available online at: https://www.npr.org/sections/health-shots/2021/01/11/955128562/for-health-care-workers-the-pandemic-is-fueling-renewed-interest-in-unions (accessed February 16, 2022).

45. Heath C, Sommerfield A, Ungern-Sternberg BS. Resilience strategies to manage psychological distress among healthcare workers during the COVID-19 pandemic: a narrative review. Anaesthesia. (2020) 75:1364–71. doi: 10.1111/anae.15180

Keywords: health worker, COVID-19, occupational health and safety, qualitative, mitigation

Citation: Goff SL, Wallace K, Putnam N, Fernandes M, Chow E, DaCosta M and Clary K (2022) A Qualitative Study of Health Workers' Experiences During Early Surges in the COVID-19 Pandemic in the U.S.: Implications for Ongoing Occupational Health Challenges. Front. Public Health 10:780711. doi: 10.3389/fpubh.2022.780711

Received: 21 September 2021; Accepted: 04 February 2022; Published: 15 March 2022.

Reviewed by:

Copyright © 2022 Goff, Wallace, Putnam, Fernandes, Chow, DaCosta and Clary. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

*Correspondence: Sarah L. Goff, sgoff@umass.edu

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

The top 10 journal articles

This year, APA’s 89 journals published more than 4,000 articles. Here are the most downloaded to date.

By Lea Winerman

December 2018, Vol 49, No. 11

Print version: page 36

1: Journal Article Reporting Standards for Qualitative Research in Psychology

This American Psychologist open-access article lays out—for the first time—journal article reporting standards for qualitative research in psychology (Levitt, H.M., et al., Vol. 73, No. 1). The voluntary guidelines are designed to help authors communicate their work clearly, accurately and transparently. Developed by a working group of the APA Publications and Communications Board, the new standards describe what should be included in a qualitative research report, as well as in qualitative meta-analyses and mixed-methods research reports. They cover a range of qualitative traditions, methods and reporting styles. The article presents these standards and their rationale, details the ways they differ from quantitative research reporting standards and describes how they can be used by authors as well as by reviewers and editors. DOI: 10.1037/amp0000151

2: The Relationship Between Frequency of Instagram Use, Exposure to Idealized Images, and Psychological Well-Being in Women

Frequent use of the social media photo-sharing app Instagram could contribute to negative psychological outcomes in women, suggests this study in Psychology of Popular Media Culture (Sherlock, M., & Wagstaff, D.L., advance online publication). Researchers surveyed 119 women, ages 18 to 35, about their Instagram use, mental health outcomes and self-perceptions. On average, more Instagram use was correlated with more depressive symptoms, lower self-esteem, more general and physical appearance anxiety, and more body dissatisfaction. In a follow-up experiment, the researchers showed women beauty, fitness or travel images from Instagram. Participants who saw the beauty and fitness images rated their own attractiveness lower than a control group that saw no images. DOI: 10.1037/ppm0000182

3: Journal Article Reporting Standards for Quantitative Research in Psychology

This open-access article in American Psychologist lays out new journal article reporting standards for quantitative research in APA journals (Appelbaum, M., et al., Vol. 73, No. 1). The new standards are voluntary guidelines for authors and reviewers, developed by a task force of APA’s Publications and Communications Board. The recommendations include dividing the hypotheses, analyses and conclusions sections into primary, secondary and exploratory groupings to enhance understanding and reproducibility. The standards also offer modules for authors reporting on N-of-1 designs, replications, clinical trials, longitudinal studies and observational studies, structural equation modeling and Bayesian analysis. DOI: 10.1037/amp0000191

4: The Effects of Sleep Deprivation on Item and Associative Recognition Memory

Sleep deprivation degrades different kinds of memory in the same way, finds this study in the Journal of Experimental Psychology: Learning, Memory, and Cognition (Ratcliff, R., & Van Dongen, H., Vol. 44, No. 2). Researchers assigned 26 participants to either a sleep-deprivation group or a control group. Before and after 57 hours of sleep deprivation, the participants did two memory tests in which they were shown word pairs and asked to recognize whether a word was on the pairs list (item recognition) or whether two words were studied in the same pair (associative recognition). Using a diffusion decision model, they found that sleep deprivation, unlike aging-related memory decline, reduced the quality of the information stored in memory for both tests to the same degree. DOI: 10.1037/xlm0000452

5: Do the Associations of Parenting Styles with Behavior Problems and Academic Achievement Vary by Culture?

Children with authoritative (high-warmth, high-control) parents have fewer behavior problems and better academic achievement compared with children of authoritarian (low-warmth, high-control) parents, and that association generally holds up across different countries and cultural groups, finds this meta-analysis in Cultural Diversity & Ethnic Minority Psychology (Pinquart, M., & Kauser, R., Vol. 24, No. 1). Researchers analyzed the results of 428 studies of parenting styles, with data on nearly 350,000 children from 52 countries. They found more similarities than differences in children’s responses to different parenting styles across ethnic groups and geographic regions. Authoritative parenting was associated with at least one positive outcome and authoritarian parenting was associated with at least one negative outcome in all regions. Overall, the association between parenting style and child outcomes was weaker in countries with more individualistic cultures. DOI: 10.1037/cdp0000149

6: Social Media Behavior, Toxic Masculinity and Depression

Men who adhere to standards of "toxic masculinity" are more likely to engage in negative behaviors on social media and are also more likely to suffer from depression, and these variables are intertwined in nuanced ways, according to a study in Psychology of Men & Masculinity (Parent, M.C., et al., advance online publication). In an online survey with 402 men, ages 18 to 74, researchers measured three areas: participants’ beliefs in toxic masculinity (sexism, heterosexism and competitiveness); their symptoms of depression; and their social media behavior, such as how often they posted positive or negative comments about things they saw online. Overall, the researchers found that men who endorsed "toxic masculinity" ideals reported more negative online behaviors and that negative online behaviors were associated with depression. DOI: 10.1037/men0000156

7: Prevention of Relapse in Major Depressive Disorder With Either Mindfulness-Based Cognitive Therapy or Cognitive Therapy

Mindfulness-based cognitive therapy (MBCT) and cognitive therapy (CT) are equally effective ways to prevent patients from relapsing into depression, finds this article in the Journal of Consulting and Clinical Psychology (Farb, N., et al., Vol. 86, No. 2). In the randomized trial, registered at ClinicalTrials.gov , 166 patients in remission from major depressive disorder were assigned to an eight-week session of either MBCT or CT. Researchers then followed the patients for two years, checking in on their depression symptoms every three months. Overall, relapse rates did not differ between the two treatment groups (18 out of 84 patients in the CT group and 18 out of 82 in the MBCT group), nor did the average time to relapse. DOI: 10.1037/ccp0000266

8: What Do Undergraduates Learn About Human Intelligence?

Many psychology textbooks contain inaccurate and incomplete information about intelligence, finds this analysis in the open-access, open-data journal  Archives of Scientific Psychology  (Warne, R.T., et al., Vol. 6, No. 1). By examining 29 of the most popular introductory psychology textbooks, researchers found that 79.3 percent contained inaccurate statements in their sections about intelligence and 79.3 percent contained logical fallacies. The five most commonly taught topics were IQ (93.1 percent), Gardner’s multiple intelligences (93.1 percent), Spearman’s g (93.1 percent), Sternberg’s triarchic theory (89.7 percent) and how intelligence is measured (82.8 percent), but few texts discussed the relative lack of empirical evidence for some of these theories. The authors note the limitations of the study, including the choice of standards for accuracy and the inherent subjectivity required for some of the data collection process.  DOI: 10.1037/arc0000038

9: Bullying Victimization and Student Engagement in Schools

Students at schools with less bullying and more positive atmospheres are more engaged with their schoolwork and school communities, finds this study in School Psychology Quarterly (Yang, C., et al., Vol. 33, No. 1). Researchers surveyed nearly 26,000 Delaware public school students in fourth through 12th grade about how often they had been the victims of bullying, as well as their perceptions of their schools’ climate, including teacher-student relationships, student-student relationships, fairness of rules, clarity of expectations, school safety and respect for diversity. Students also took a survey that assessed their levels of emotional and cognitive-behavioral engagement in their schools, including how happy they felt at school and how committed they were to their schoolwork. After controlling for student and school demographic factors including gender, race/ethnicity and socioeconomic status, a positive school climate was associated with higher student engagement across all grades. DOI: 10.1037/spq0000250

10: Emotion Regulation Therapy for Generalized Anxiety Disorder With and Without Co-Occurring Depression

Emotion regulation therapy (ERT) is an effective treatment for generalized anxiety disorder, with or without co-occurring major depression, finds this study in the Journal of Consulting and Clinical Psychology (Mennin, D.S., et al., Vol. 86, No. 3). ERT uses principles from cognitive-behavioral therapy and affect science to teach patients to identify, accept and manage their emotions and to use this awareness to guide their thinking and behavior. Researchers assigned 53 patients with anxiety (23 of whom also had depression) to be treated with ERT or to be part of a control group awaiting treatment. After 20 weeks, patients in the treatment group showed statistically and clinically significant improvements in anxiety and depression symptoms—including functional impairment, quality of life, worry and rumination—compared with the control group. DOI: 10.1037/ccp0000289

Letters to the Editor

• Send us a letter

• Open access
• Published: 27 May 2020

How to use and assess qualitative research methods

• Loraine Busetto   ORCID: orcid.org/0000-0002-9228-7875 1 ,
• Wolfgang Wick 1 , 2 &
• Christoph Gumbinger 1  

Neurological Research and Practice volume  2 , Article number:  14 ( 2020 ) Cite this article

783k Accesses

375 Citations

90 Altmetric

Metrics details

This paper aims to provide an overview of the use and assessment of qualitative research methods in the health sciences. Qualitative research can be defined as the study of the nature of phenomena and is especially appropriate for answering questions of why something is (not) observed, assessing complex multi-component interventions, and focussing on intervention improvement. The most common methods of data collection are document study, (non-) participant observations, semi-structured interviews and focus groups. For data analysis, field-notes and audio-recordings are transcribed into protocols and transcripts, and coded using qualitative data management software. Criteria such as checklists, reflexivity, sampling strategies, piloting, co-coding, member-checking and stakeholder involvement can be used to enhance and assess the quality of the research conducted. Using qualitative in addition to quantitative designs will equip us with better tools to address a greater range of research problems, and to fill in blind spots in current neurological research and practice.

The aim of this paper is to provide an overview of qualitative research methods, including hands-on information on how they can be used, reported and assessed. This article is intended for beginning qualitative researchers in the health sciences as well as experienced quantitative researchers who wish to broaden their understanding of qualitative research.

What is qualitative research?

Qualitative research is defined as “the study of the nature of phenomena”, including “their quality, different manifestations, the context in which they appear or the perspectives from which they can be perceived” , but excluding “their range, frequency and place in an objectively determined chain of cause and effect” [ 1 ]. This formal definition can be complemented with a more pragmatic rule of thumb: qualitative research generally includes data in form of words rather than numbers [ 2 ].

Why conduct qualitative research?

Because some research questions cannot be answered using (only) quantitative methods. For example, one Australian study addressed the issue of why patients from Aboriginal communities often present late or not at all to specialist services offered by tertiary care hospitals. Using qualitative interviews with patients and staff, it found one of the most significant access barriers to be transportation problems, including some towns and communities simply not having a bus service to the hospital [ 3 ]. A quantitative study could have measured the number of patients over time or even looked at possible explanatory factors – but only those previously known or suspected to be of relevance. To discover reasons for observed patterns, especially the invisible or surprising ones, qualitative designs are needed.

While qualitative research is common in other fields, it is still relatively underrepresented in health services research. The latter field is more traditionally rooted in the evidence-based-medicine paradigm, as seen in " research that involves testing the effectiveness of various strategies to achieve changes in clinical practice, preferably applying randomised controlled trial study designs (...) " [ 4 ]. This focus on quantitative research and specifically randomised controlled trials (RCT) is visible in the idea of a hierarchy of research evidence which assumes that some research designs are objectively better than others, and that choosing a "lesser" design is only acceptable when the better ones are not practically or ethically feasible [ 5 , 6 ]. Others, however, argue that an objective hierarchy does not exist, and that, instead, the research design and methods should be chosen to fit the specific research question at hand – "questions before methods" [ 2 , 7 , 8 , 9 ]. This means that even when an RCT is possible, some research problems require a different design that is better suited to addressing them. Arguing in JAMA, Berwick uses the example of rapid response teams in hospitals, which he describes as " a complex, multicomponent intervention – essentially a process of social change" susceptible to a range of different context factors including leadership or organisation history. According to him, "[in] such complex terrain, the RCT is an impoverished way to learn. Critics who use it as a truth standard in this context are incorrect" [ 8 ] . Instead of limiting oneself to RCTs, Berwick recommends embracing a wider range of methods , including qualitative ones, which for "these specific applications, (...) are not compromises in learning how to improve; they are superior" [ 8 ].

Research problems that can be approached particularly well using qualitative methods include assessing complex multi-component interventions or systems (of change), addressing questions beyond “what works”, towards “what works for whom when, how and why”, and focussing on intervention improvement rather than accreditation [ 7 , 9 , 10 , 11 , 12 ]. Using qualitative methods can also help shed light on the “softer” side of medical treatment. For example, while quantitative trials can measure the costs and benefits of neuro-oncological treatment in terms of survival rates or adverse effects, qualitative research can help provide a better understanding of patient or caregiver stress, visibility of illness or out-of-pocket expenses.

How to conduct qualitative research?

Given that qualitative research is characterised by flexibility, openness and responsivity to context, the steps of data collection and analysis are not as separate and consecutive as they tend to be in quantitative research [ 13 , 14 ]. As Fossey puts it : “sampling, data collection, analysis and interpretation are related to each other in a cyclical (iterative) manner, rather than following one after another in a stepwise approach” [ 15 ]. The researcher can make educated decisions with regard to the choice of method, how they are implemented, and to which and how many units they are applied [ 13 ]. As shown in Fig.  1 , this can involve several back-and-forth steps between data collection and analysis where new insights and experiences can lead to adaption and expansion of the original plan. Some insights may also necessitate a revision of the research question and/or the research design as a whole. The process ends when saturation is achieved, i.e. when no relevant new information can be found (see also below: sampling and saturation). For reasons of transparency, it is essential for all decisions as well as the underlying reasoning to be well-documented.

Iterative research process

While it is not always explicitly addressed, qualitative methods reflect a different underlying research paradigm than quantitative research (e.g. constructivism or interpretivism as opposed to positivism). The choice of methods can be based on the respective underlying substantive theory or theoretical framework used by the researcher [ 2 ].

Data collection

The methods of qualitative data collection most commonly used in health research are document study, observations, semi-structured interviews and focus groups [ 1 , 14 , 16 , 17 ].

Document study

Document study (also called document analysis) refers to the review by the researcher of written materials [ 14 ]. These can include personal and non-personal documents such as archives, annual reports, guidelines, policy documents, diaries or letters.

Observations

Observations are particularly useful to gain insights into a certain setting and actual behaviour – as opposed to reported behaviour or opinions [ 13 ]. Qualitative observations can be either participant or non-participant in nature. In participant observations, the observer is part of the observed setting, for example a nurse working in an intensive care unit [ 18 ]. In non-participant observations, the observer is “on the outside looking in”, i.e. present in but not part of the situation, trying not to influence the setting by their presence. Observations can be planned (e.g. for 3 h during the day or night shift) or ad hoc (e.g. as soon as a stroke patient arrives at the emergency room). During the observation, the observer takes notes on everything or certain pre-determined parts of what is happening around them, for example focusing on physician-patient interactions or communication between different professional groups. Written notes can be taken during or after the observations, depending on feasibility (which is usually lower during participant observations) and acceptability (e.g. when the observer is perceived to be judging the observed). Afterwards, these field notes are transcribed into observation protocols. If more than one observer was involved, field notes are taken independently, but notes can be consolidated into one protocol after discussions. Advantages of conducting observations include minimising the distance between the researcher and the researched, the potential discovery of topics that the researcher did not realise were relevant and gaining deeper insights into the real-world dimensions of the research problem at hand [ 18 ].

Semi-structured interviews

Hijmans & Kuyper describe qualitative interviews as “an exchange with an informal character, a conversation with a goal” [ 19 ]. Interviews are used to gain insights into a person’s subjective experiences, opinions and motivations – as opposed to facts or behaviours [ 13 ]. Interviews can be distinguished by the degree to which they are structured (i.e. a questionnaire), open (e.g. free conversation or autobiographical interviews) or semi-structured [ 2 , 13 ]. Semi-structured interviews are characterized by open-ended questions and the use of an interview guide (or topic guide/list) in which the broad areas of interest, sometimes including sub-questions, are defined [ 19 ]. The pre-defined topics in the interview guide can be derived from the literature, previous research or a preliminary method of data collection, e.g. document study or observations. The topic list is usually adapted and improved at the start of the data collection process as the interviewer learns more about the field [ 20 ]. Across interviews the focus on the different (blocks of) questions may differ and some questions may be skipped altogether (e.g. if the interviewee is not able or willing to answer the questions or for concerns about the total length of the interview) [ 20 ]. Qualitative interviews are usually not conducted in written format as it impedes on the interactive component of the method [ 20 ]. In comparison to written surveys, qualitative interviews have the advantage of being interactive and allowing for unexpected topics to emerge and to be taken up by the researcher. This can also help overcome a provider or researcher-centred bias often found in written surveys, which by nature, can only measure what is already known or expected to be of relevance to the researcher. Interviews can be audio- or video-taped; but sometimes it is only feasible or acceptable for the interviewer to take written notes [ 14 , 16 , 20 ].

Focus groups

Focus groups are group interviews to explore participants’ expertise and experiences, including explorations of how and why people behave in certain ways [ 1 ]. Focus groups usually consist of 6–8 people and are led by an experienced moderator following a topic guide or “script” [ 21 ]. They can involve an observer who takes note of the non-verbal aspects of the situation, possibly using an observation guide [ 21 ]. Depending on researchers’ and participants’ preferences, the discussions can be audio- or video-taped and transcribed afterwards [ 21 ]. Focus groups are useful for bringing together homogeneous (to a lesser extent heterogeneous) groups of participants with relevant expertise and experience on a given topic on which they can share detailed information [ 21 ]. Focus groups are a relatively easy, fast and inexpensive method to gain access to information on interactions in a given group, i.e. “the sharing and comparing” among participants [ 21 ]. Disadvantages include less control over the process and a lesser extent to which each individual may participate. Moreover, focus group moderators need experience, as do those tasked with the analysis of the resulting data. Focus groups can be less appropriate for discussing sensitive topics that participants might be reluctant to disclose in a group setting [ 13 ]. Moreover, attention must be paid to the emergence of “groupthink” as well as possible power dynamics within the group, e.g. when patients are awed or intimidated by health professionals.

Choosing the “right” method

As explained above, the school of thought underlying qualitative research assumes no objective hierarchy of evidence and methods. This means that each choice of single or combined methods has to be based on the research question that needs to be answered and a critical assessment with regard to whether or to what extent the chosen method can accomplish this – i.e. the “fit” between question and method [ 14 ]. It is necessary for these decisions to be documented when they are being made, and to be critically discussed when reporting methods and results.

Let us assume that our research aim is to examine the (clinical) processes around acute endovascular treatment (EVT), from the patient’s arrival at the emergency room to recanalization, with the aim to identify possible causes for delay and/or other causes for sub-optimal treatment outcome. As a first step, we could conduct a document study of the relevant standard operating procedures (SOPs) for this phase of care – are they up-to-date and in line with current guidelines? Do they contain any mistakes, irregularities or uncertainties that could cause delays or other problems? Regardless of the answers to these questions, the results have to be interpreted based on what they are: a written outline of what care processes in this hospital should look like. If we want to know what they actually look like in practice, we can conduct observations of the processes described in the SOPs. These results can (and should) be analysed in themselves, but also in comparison to the results of the document analysis, especially as regards relevant discrepancies. Do the SOPs outline specific tests for which no equipment can be observed or tasks to be performed by specialized nurses who are not present during the observation? It might also be possible that the written SOP is outdated, but the actual care provided is in line with current best practice. In order to find out why these discrepancies exist, it can be useful to conduct interviews. Are the physicians simply not aware of the SOPs (because their existence is limited to the hospital’s intranet) or do they actively disagree with them or does the infrastructure make it impossible to provide the care as described? Another rationale for adding interviews is that some situations (or all of their possible variations for different patient groups or the day, night or weekend shift) cannot practically or ethically be observed. In this case, it is possible to ask those involved to report on their actions – being aware that this is not the same as the actual observation. A senior physician’s or hospital manager’s description of certain situations might differ from a nurse’s or junior physician’s one, maybe because they intentionally misrepresent facts or maybe because different aspects of the process are visible or important to them. In some cases, it can also be relevant to consider to whom the interviewee is disclosing this information – someone they trust, someone they are otherwise not connected to, or someone they suspect or are aware of being in a potentially “dangerous” power relationship to them. Lastly, a focus group could be conducted with representatives of the relevant professional groups to explore how and why exactly they provide care around EVT. The discussion might reveal discrepancies (between SOPs and actual care or between different physicians) and motivations to the researchers as well as to the focus group members that they might not have been aware of themselves. For the focus group to deliver relevant information, attention has to be paid to its composition and conduct, for example, to make sure that all participants feel safe to disclose sensitive or potentially problematic information or that the discussion is not dominated by (senior) physicians only. The resulting combination of data collection methods is shown in Fig.  2 .

Possible combination of data collection methods

Attributions for icons: “Book” by Serhii Smirnov, “Interview” by Adrien Coquet, FR, “Magnifying Glass” by anggun, ID, “Business communication” by Vectors Market; all from the Noun Project

The combination of multiple data source as described for this example can be referred to as “triangulation”, in which multiple measurements are carried out from different angles to achieve a more comprehensive understanding of the phenomenon under study [ 22 , 23 ].

Data analysis

To analyse the data collected through observations, interviews and focus groups these need to be transcribed into protocols and transcripts (see Fig.  3 ). Interviews and focus groups can be transcribed verbatim , with or without annotations for behaviour (e.g. laughing, crying, pausing) and with or without phonetic transcription of dialects and filler words, depending on what is expected or known to be relevant for the analysis. In the next step, the protocols and transcripts are coded , that is, marked (or tagged, labelled) with one or more short descriptors of the content of a sentence or paragraph [ 2 , 15 , 23 ]. Jansen describes coding as “connecting the raw data with “theoretical” terms” [ 20 ]. In a more practical sense, coding makes raw data sortable. This makes it possible to extract and examine all segments describing, say, a tele-neurology consultation from multiple data sources (e.g. SOPs, emergency room observations, staff and patient interview). In a process of synthesis and abstraction, the codes are then grouped, summarised and/or categorised [ 15 , 20 ]. The end product of the coding or analysis process is a descriptive theory of the behavioural pattern under investigation [ 20 ]. The coding process is performed using qualitative data management software, the most common ones being InVivo, MaxQDA and Atlas.ti. It should be noted that these are data management tools which support the analysis performed by the researcher(s) [ 14 ].

From data collection to data analysis

Attributions for icons: see Fig. 2 , also “Speech to text” by Trevor Dsouza, “Field Notes” by Mike O’Brien, US, “Voice Record” by ProSymbols, US, “Inspection” by Made, AU, and “Cloud” by Graphic Tigers; all from the Noun Project

How to report qualitative research?

Protocols of qualitative research can be published separately and in advance of the study results. However, the aim is not the same as in RCT protocols, i.e. to pre-define and set in stone the research questions and primary or secondary endpoints. Rather, it is a way to describe the research methods in detail, which might not be possible in the results paper given journals’ word limits. Qualitative research papers are usually longer than their quantitative counterparts to allow for deep understanding and so-called “thick description”. In the methods section, the focus is on transparency of the methods used, including why, how and by whom they were implemented in the specific study setting, so as to enable a discussion of whether and how this may have influenced data collection, analysis and interpretation. The results section usually starts with a paragraph outlining the main findings, followed by more detailed descriptions of, for example, the commonalities, discrepancies or exceptions per category [ 20 ]. Here it is important to support main findings by relevant quotations, which may add information, context, emphasis or real-life examples [ 20 , 23 ]. It is subject to debate in the field whether it is relevant to state the exact number or percentage of respondents supporting a certain statement (e.g. “Five interviewees expressed negative feelings towards XYZ”) [ 21 ].

How to combine qualitative with quantitative research?

Qualitative methods can be combined with other methods in multi- or mixed methods designs, which “[employ] two or more different methods [ …] within the same study or research program rather than confining the research to one single method” [ 24 ]. Reasons for combining methods can be diverse, including triangulation for corroboration of findings, complementarity for illustration and clarification of results, expansion to extend the breadth and range of the study, explanation of (unexpected) results generated with one method with the help of another, or offsetting the weakness of one method with the strength of another [ 1 , 17 , 24 , 25 , 26 ]. The resulting designs can be classified according to when, why and how the different quantitative and/or qualitative data strands are combined. The three most common types of mixed method designs are the convergent parallel design , the explanatory sequential design and the exploratory sequential design. The designs with examples are shown in Fig.  4 .

Three common mixed methods designs

In the convergent parallel design, a qualitative study is conducted in parallel to and independently of a quantitative study, and the results of both studies are compared and combined at the stage of interpretation of results. Using the above example of EVT provision, this could entail setting up a quantitative EVT registry to measure process times and patient outcomes in parallel to conducting the qualitative research outlined above, and then comparing results. Amongst other things, this would make it possible to assess whether interview respondents’ subjective impressions of patients receiving good care match modified Rankin Scores at follow-up, or whether observed delays in care provision are exceptions or the rule when compared to door-to-needle times as documented in the registry. In the explanatory sequential design, a quantitative study is carried out first, followed by a qualitative study to help explain the results from the quantitative study. This would be an appropriate design if the registry alone had revealed relevant delays in door-to-needle times and the qualitative study would be used to understand where and why these occurred, and how they could be improved. In the exploratory design, the qualitative study is carried out first and its results help informing and building the quantitative study in the next step [ 26 ]. If the qualitative study around EVT provision had shown a high level of dissatisfaction among the staff members involved, a quantitative questionnaire investigating staff satisfaction could be set up in the next step, informed by the qualitative study on which topics dissatisfaction had been expressed. Amongst other things, the questionnaire design would make it possible to widen the reach of the research to more respondents from different (types of) hospitals, regions, countries or settings, and to conduct sub-group analyses for different professional groups.

How to assess qualitative research?

A variety of assessment criteria and lists have been developed for qualitative research, ranging in their focus and comprehensiveness [ 14 , 17 , 27 ]. However, none of these has been elevated to the “gold standard” in the field. In the following, we therefore focus on a set of commonly used assessment criteria that, from a practical standpoint, a researcher can look for when assessing a qualitative research report or paper.

Assessors should check the authors’ use of and adherence to the relevant reporting checklists (e.g. Standards for Reporting Qualitative Research (SRQR)) to make sure all items that are relevant for this type of research are addressed [ 23 , 28 ]. Discussions of quantitative measures in addition to or instead of these qualitative measures can be a sign of lower quality of the research (paper). Providing and adhering to a checklist for qualitative research contributes to an important quality criterion for qualitative research, namely transparency [ 15 , 17 , 23 ].

Reflexivity

While methodological transparency and complete reporting is relevant for all types of research, some additional criteria must be taken into account for qualitative research. This includes what is called reflexivity, i.e. sensitivity to the relationship between the researcher and the researched, including how contact was established and maintained, or the background and experience of the researcher(s) involved in data collection and analysis. Depending on the research question and population to be researched this can be limited to professional experience, but it may also include gender, age or ethnicity [ 17 , 27 ]. These details are relevant because in qualitative research, as opposed to quantitative research, the researcher as a person cannot be isolated from the research process [ 23 ]. It may influence the conversation when an interviewed patient speaks to an interviewer who is a physician, or when an interviewee is asked to discuss a gynaecological procedure with a male interviewer, and therefore the reader must be made aware of these details [ 19 ].

Sampling and saturation

The aim of qualitative sampling is for all variants of the objects of observation that are deemed relevant for the study to be present in the sample “ to see the issue and its meanings from as many angles as possible” [ 1 , 16 , 19 , 20 , 27 ] , and to ensure “information-richness [ 15 ]. An iterative sampling approach is advised, in which data collection (e.g. five interviews) is followed by data analysis, followed by more data collection to find variants that are lacking in the current sample. This process continues until no new (relevant) information can be found and further sampling becomes redundant – which is called saturation [ 1 , 15 ] . In other words: qualitative data collection finds its end point not a priori , but when the research team determines that saturation has been reached [ 29 , 30 ].

This is also the reason why most qualitative studies use deliberate instead of random sampling strategies. This is generally referred to as “ purposive sampling” , in which researchers pre-define which types of participants or cases they need to include so as to cover all variations that are expected to be of relevance, based on the literature, previous experience or theory (i.e. theoretical sampling) [ 14 , 20 ]. Other types of purposive sampling include (but are not limited to) maximum variation sampling, critical case sampling or extreme or deviant case sampling [ 2 ]. In the above EVT example, a purposive sample could include all relevant professional groups and/or all relevant stakeholders (patients, relatives) and/or all relevant times of observation (day, night and weekend shift).

Assessors of qualitative research should check whether the considerations underlying the sampling strategy were sound and whether or how researchers tried to adapt and improve their strategies in stepwise or cyclical approaches between data collection and analysis to achieve saturation [ 14 ].

Good qualitative research is iterative in nature, i.e. it goes back and forth between data collection and analysis, revising and improving the approach where necessary. One example of this are pilot interviews, where different aspects of the interview (especially the interview guide, but also, for example, the site of the interview or whether the interview can be audio-recorded) are tested with a small number of respondents, evaluated and revised [ 19 ]. In doing so, the interviewer learns which wording or types of questions work best, or which is the best length of an interview with patients who have trouble concentrating for an extended time. Of course, the same reasoning applies to observations or focus groups which can also be piloted.

Ideally, coding should be performed by at least two researchers, especially at the beginning of the coding process when a common approach must be defined, including the establishment of a useful coding list (or tree), and when a common meaning of individual codes must be established [ 23 ]. An initial sub-set or all transcripts can be coded independently by the coders and then compared and consolidated after regular discussions in the research team. This is to make sure that codes are applied consistently to the research data.

Member checking

Member checking, also called respondent validation , refers to the practice of checking back with study respondents to see if the research is in line with their views [ 14 , 27 ]. This can happen after data collection or analysis or when first results are available [ 23 ]. For example, interviewees can be provided with (summaries of) their transcripts and asked whether they believe this to be a complete representation of their views or whether they would like to clarify or elaborate on their responses [ 17 ]. Respondents’ feedback on these issues then becomes part of the data collection and analysis [ 27 ].

Stakeholder involvement

In those niches where qualitative approaches have been able to evolve and grow, a new trend has seen the inclusion of patients and their representatives not only as study participants (i.e. “members”, see above) but as consultants to and active participants in the broader research process [ 31 , 32 , 33 ]. The underlying assumption is that patients and other stakeholders hold unique perspectives and experiences that add value beyond their own single story, making the research more relevant and beneficial to researchers, study participants and (future) patients alike [ 34 , 35 ]. Using the example of patients on or nearing dialysis, a recent scoping review found that 80% of clinical research did not address the top 10 research priorities identified by patients and caregivers [ 32 , 36 ]. In this sense, the involvement of the relevant stakeholders, especially patients and relatives, is increasingly being seen as a quality indicator in and of itself.

How not to assess qualitative research

The above overview does not include certain items that are routine in assessments of quantitative research. What follows is a non-exhaustive, non-representative, experience-based list of the quantitative criteria often applied to the assessment of qualitative research, as well as an explanation of the limited usefulness of these endeavours.

Protocol adherence

Given the openness and flexibility of qualitative research, it should not be assessed by how well it adheres to pre-determined and fixed strategies – in other words: its rigidity. Instead, the assessor should look for signs of adaptation and refinement based on lessons learned from earlier steps in the research process.

Sample size

For the reasons explained above, qualitative research does not require specific sample sizes, nor does it require that the sample size be determined a priori [ 1 , 14 , 27 , 37 , 38 , 39 ]. Sample size can only be a useful quality indicator when related to the research purpose, the chosen methodology and the composition of the sample, i.e. who was included and why.

Randomisation

While some authors argue that randomisation can be used in qualitative research, this is not commonly the case, as neither its feasibility nor its necessity or usefulness has been convincingly established for qualitative research [ 13 , 27 ]. Relevant disadvantages include the negative impact of a too large sample size as well as the possibility (or probability) of selecting “ quiet, uncooperative or inarticulate individuals ” [ 17 ]. Qualitative studies do not use control groups, either.

Interrater reliability, variability and other “objectivity checks”

The concept of “interrater reliability” is sometimes used in qualitative research to assess to which extent the coding approach overlaps between the two co-coders. However, it is not clear what this measure tells us about the quality of the analysis [ 23 ]. This means that these scores can be included in qualitative research reports, preferably with some additional information on what the score means for the analysis, but it is not a requirement. Relatedly, it is not relevant for the quality or “objectivity” of qualitative research to separate those who recruited the study participants and collected and analysed the data. Experiences even show that it might be better to have the same person or team perform all of these tasks [ 20 ]. First, when researchers introduce themselves during recruitment this can enhance trust when the interview takes place days or weeks later with the same researcher. Second, when the audio-recording is transcribed for analysis, the researcher conducting the interviews will usually remember the interviewee and the specific interview situation during data analysis. This might be helpful in providing additional context information for interpretation of data, e.g. on whether something might have been meant as a joke [ 18 ].

Not being quantitative research

Being qualitative research instead of quantitative research should not be used as an assessment criterion if it is used irrespectively of the research problem at hand. Similarly, qualitative research should not be required to be combined with quantitative research per se – unless mixed methods research is judged as inherently better than single-method research. In this case, the same criterion should be applied for quantitative studies without a qualitative component.

The main take-away points of this paper are summarised in Table 1 . We aimed to show that, if conducted well, qualitative research can answer specific research questions that cannot to be adequately answered using (only) quantitative designs. Seeing qualitative and quantitative methods as equal will help us become more aware and critical of the “fit” between the research problem and our chosen methods: I can conduct an RCT to determine the reasons for transportation delays of acute stroke patients – but should I? It also provides us with a greater range of tools to tackle a greater range of research problems more appropriately and successfully, filling in the blind spots on one half of the methodological spectrum to better address the whole complexity of neurological research and practice.

Availability of data and materials

Not applicable.

Abbreviations

Endovascular treatment

Randomised Controlled Trial

Standard Operating Procedure

Standards for Reporting Qualitative Research

Philipsen, H., & Vernooij-Dassen, M. (2007). Kwalitatief onderzoek: nuttig, onmisbaar en uitdagend. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [Qualitative research: useful, indispensable and challenging. In: Qualitative research: Practical methods for medical practice (pp. 5–12). Houten: Bohn Stafleu van Loghum.

Chapter   Google Scholar  

Punch, K. F. (2013). Introduction to social research: Quantitative and qualitative approaches . London: Sage.

Kelly, J., Dwyer, J., Willis, E., & Pekarsky, B. (2014). Travelling to the city for hospital care: Access factors in country aboriginal patient journeys. Australian Journal of Rural Health, 22 (3), 109–113.

Article   Google Scholar  

Nilsen, P., Ståhl, C., Roback, K., & Cairney, P. (2013). Never the twain shall meet? - a comparison of implementation science and policy implementation research. Implementation Science, 8 (1), 1–12.

Howick J, Chalmers I, Glasziou, P., Greenhalgh, T., Heneghan, C., Liberati, A., Moschetti, I., Phillips, B., & Thornton, H. (2011). The 2011 Oxford CEBM evidence levels of evidence (introductory document) . Oxford Center for Evidence Based Medicine. https://www.cebm.net/2011/06/2011-oxford-cebm-levels-evidence-introductory-document/ .

Eakin, J. M. (2016). Educating critical qualitative health researchers in the land of the randomized controlled trial. Qualitative Inquiry, 22 (2), 107–118.

May, A., & Mathijssen, J. (2015). Alternatieven voor RCT bij de evaluatie van effectiviteit van interventies!? Eindrapportage. In Alternatives for RCTs in the evaluation of effectiveness of interventions!? Final report .

Google Scholar  

Berwick, D. M. (2008). The science of improvement. Journal of the American Medical Association, 299 (10), 1182–1184.

Article   CAS   Google Scholar  

Christ, T. W. (2014). Scientific-based research and randomized controlled trials, the “gold” standard? Alternative paradigms and mixed methodologies. Qualitative Inquiry, 20 (1), 72–80.

Lamont, T., Barber, N., Jd, P., Fulop, N., Garfield-Birkbeck, S., Lilford, R., Mear, L., Raine, R., & Fitzpatrick, R. (2016). New approaches to evaluating complex health and care systems. BMJ, 352:i154.

Drabble, S. J., & O’Cathain, A. (2015). Moving from Randomized Controlled Trials to Mixed Methods Intervention Evaluation. In S. Hesse-Biber & R. B. Johnson (Eds.), The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry (pp. 406–425). London: Oxford University Press.

Chambers, D. A., Glasgow, R. E., & Stange, K. C. (2013). The dynamic sustainability framework: Addressing the paradox of sustainment amid ongoing change. Implementation Science : IS, 8 , 117.

Hak, T. (2007). Waarnemingsmethoden in kwalitatief onderzoek. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [Observation methods in qualitative research] (pp. 13–25). Houten: Bohn Stafleu van Loghum.

Russell, C. K., & Gregory, D. M. (2003). Evaluation of qualitative research studies. Evidence Based Nursing, 6 (2), 36–40.

Fossey, E., Harvey, C., McDermott, F., & Davidson, L. (2002). Understanding and evaluating qualitative research. Australian and New Zealand Journal of Psychiatry, 36 , 717–732.

Yanow, D. (2000). Conducting interpretive policy analysis (Vol. 47). Thousand Oaks: Sage University Papers Series on Qualitative Research Methods.

Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22 , 63–75.

van der Geest, S. (2006). Participeren in ziekte en zorg: meer over kwalitatief onderzoek. Huisarts en Wetenschap, 49 (4), 283–287.

Hijmans, E., & Kuyper, M. (2007). Het halfopen interview als onderzoeksmethode. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [The half-open interview as research method (pp. 43–51). Houten: Bohn Stafleu van Loghum.

Jansen, H. (2007). Systematiek en toepassing van de kwalitatieve survey. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [Systematics and implementation of the qualitative survey (pp. 27–41). Houten: Bohn Stafleu van Loghum.

Pv, R., & Peremans, L. (2007). Exploreren met focusgroepgesprekken: de ‘stem’ van de groep onder de loep. In L. PLBJ & H. TCo (Eds.), Kwalitatief onderzoek: Praktische methoden voor de medische praktijk . [Exploring with focus group conversations: the “voice” of the group under the magnifying glass (pp. 53–64). Houten: Bohn Stafleu van Loghum.

Carter, N., Bryant-Lukosius, D., DiCenso, A., Blythe, J., & Neville, A. J. (2014). The use of triangulation in qualitative research. Oncology Nursing Forum, 41 (5), 545–547.

Boeije H: Analyseren in kwalitatief onderzoek: Denken en doen, [Analysis in qualitative research: Thinking and doing] vol. Den Haag Boom Lemma uitgevers; 2012.

Hunter, A., & Brewer, J. (2015). Designing Multimethod Research. In S. Hesse-Biber & R. B. Johnson (Eds.), The Oxford Handbook of Multimethod and Mixed Methods Research Inquiry (pp. 185–205). London: Oxford University Press.

Archibald, M. M., Radil, A. I., Zhang, X., & Hanson, W. E. (2015). Current mixed methods practices in qualitative research: A content analysis of leading journals. International Journal of Qualitative Methods, 14 (2), 5–33.

Creswell, J. W., & Plano Clark, V. L. (2011). Choosing a Mixed Methods Design. In Designing and Conducting Mixed Methods Research . Thousand Oaks: SAGE Publications.

Mays, N., & Pope, C. (2000). Assessing quality in qualitative research. BMJ, 320 (7226), 50–52.

O'Brien, B. C., Harris, I. B., Beckman, T. J., Reed, D. A., & Cook, D. A. (2014). Standards for reporting qualitative research: A synthesis of recommendations. Academic Medicine : Journal of the Association of American Medical Colleges, 89 (9), 1245–1251.

Saunders, B., Sim, J., Kingstone, T., Baker, S., Waterfield, J., Bartlam, B., Burroughs, H., & Jinks, C. (2018). Saturation in qualitative research: Exploring its conceptualization and operationalization. Quality and Quantity, 52 (4), 1893–1907.

Moser, A., & Korstjens, I. (2018). Series: Practical guidance to qualitative research. Part 3: Sampling, data collection and analysis. European Journal of General Practice, 24 (1), 9–18.

Marlett, N., Shklarov, S., Marshall, D., Santana, M. J., & Wasylak, T. (2015). Building new roles and relationships in research: A model of patient engagement research. Quality of Life Research : an international journal of quality of life aspects of treatment, care and rehabilitation, 24 (5), 1057–1067.

Demian, M. N., Lam, N. N., Mac-Way, F., Sapir-Pichhadze, R., & Fernandez, N. (2017). Opportunities for engaging patients in kidney research. Canadian Journal of Kidney Health and Disease, 4 , 2054358117703070–2054358117703070.

Noyes, J., McLaughlin, L., Morgan, K., Roberts, A., Stephens, M., Bourne, J., Houlston, M., Houlston, J., Thomas, S., Rhys, R. G., et al. (2019). Designing a co-productive study to overcome known methodological challenges in organ donation research with bereaved family members. Health Expectations . 22(4):824–35.

Piil, K., Jarden, M., & Pii, K. H. (2019). Research agenda for life-threatening cancer. European Journal Cancer Care (Engl), 28 (1), e12935.

Hofmann, D., Ibrahim, F., Rose, D., Scott, D. L., Cope, A., Wykes, T., & Lempp, H. (2015). Expectations of new treatment in rheumatoid arthritis: Developing a patient-generated questionnaire. Health Expectations : an international journal of public participation in health care and health policy, 18 (5), 995–1008.

Jun, M., Manns, B., Laupacis, A., Manns, L., Rehal, B., Crowe, S., & Hemmelgarn, B. R. (2015). Assessing the extent to which current clinical research is consistent with patient priorities: A scoping review using a case study in patients on or nearing dialysis. Canadian Journal of Kidney Health and Disease, 2 , 35.

Elsie Baker, S., & Edwards, R. (2012). How many qualitative interviews is enough? In National Centre for Research Methods Review Paper . National Centre for Research Methods. http://eprints.ncrm.ac.uk/2273/4/how_many_interviews.pdf .

Sandelowski, M. (1995). Sample size in qualitative research. Research in Nursing & Health, 18 (2), 179–183.

Sim, J., Saunders, B., Waterfield, J., & Kingstone, T. (2018). Can sample size in qualitative research be determined a priori? International Journal of Social Research Methodology, 21 (5), 619–634.

Download references

Acknowledgements

no external funding.

Author information

Authors and affiliations.

Department of Neurology, Heidelberg University Hospital, Im Neuenheimer Feld 400, 69120, Heidelberg, Germany

Loraine Busetto, Wolfgang Wick & Christoph Gumbinger

Clinical Cooperation Unit Neuro-Oncology, German Cancer Research Center, Heidelberg, Germany

Wolfgang Wick

You can also search for this author in PubMed   Google Scholar

Contributions

LB drafted the manuscript; WW and CG revised the manuscript; all authors approved the final versions.

Corresponding author

Correspondence to Loraine Busetto .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Busetto, L., Wick, W. & Gumbinger, C. How to use and assess qualitative research methods. Neurol. Res. Pract. 2 , 14 (2020). https://doi.org/10.1186/s42466-020-00059-z

Download citation

Received : 30 January 2020

Accepted : 22 April 2020

Published : 27 May 2020

DOI : https://doi.org/10.1186/s42466-020-00059-z

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Qualitative research
• Mixed methods
• Quality assessment

Neurological Research and Practice

ISSN: 2524-3489

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

The PMC website is updating on October 15, 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• J Korean Med Sci
• v.37(16); 2022 Apr 25

A Practical Guide to Writing Quantitative and Qualitative Research Questions and Hypotheses in Scholarly Articles

Edward barroga.

1 Department of General Education, Graduate School of Nursing Science, St. Luke’s International University, Tokyo, Japan.

Glafera Janet Matanguihan

2 Department of Biological Sciences, Messiah University, Mechanicsburg, PA, USA.

The development of research questions and the subsequent hypotheses are prerequisites to defining the main research purpose and specific objectives of a study. Consequently, these objectives determine the study design and research outcome. The development of research questions is a process based on knowledge of current trends, cutting-edge studies, and technological advances in the research field. Excellent research questions are focused and require a comprehensive literature search and in-depth understanding of the problem being investigated. Initially, research questions may be written as descriptive questions which could be developed into inferential questions. These questions must be specific and concise to provide a clear foundation for developing hypotheses. Hypotheses are more formal predictions about the research outcomes. These specify the possible results that may or may not be expected regarding the relationship between groups. Thus, research questions and hypotheses clarify the main purpose and specific objectives of the study, which in turn dictate the design of the study, its direction, and outcome. Studies developed from good research questions and hypotheses will have trustworthy outcomes with wide-ranging social and health implications.

INTRODUCTION

Scientific research is usually initiated by posing evidenced-based research questions which are then explicitly restated as hypotheses. 1 , 2 The hypotheses provide directions to guide the study, solutions, explanations, and expected results. 3 , 4 Both research questions and hypotheses are essentially formulated based on conventional theories and real-world processes, which allow the inception of novel studies and the ethical testing of ideas. 5 , 6

It is crucial to have knowledge of both quantitative and qualitative research 2 as both types of research involve writing research questions and hypotheses. 7 However, these crucial elements of research are sometimes overlooked; if not overlooked, then framed without the forethought and meticulous attention it needs. Planning and careful consideration are needed when developing quantitative or qualitative research, particularly when conceptualizing research questions and hypotheses. 4

There is a continuing need to support researchers in the creation of innovative research questions and hypotheses, as well as for journal articles that carefully review these elements. 1 When research questions and hypotheses are not carefully thought of, unethical studies and poor outcomes usually ensue. Carefully formulated research questions and hypotheses define well-founded objectives, which in turn determine the appropriate design, course, and outcome of the study. This article then aims to discuss in detail the various aspects of crafting research questions and hypotheses, with the goal of guiding researchers as they develop their own. Examples from the authors and peer-reviewed scientific articles in the healthcare field are provided to illustrate key points.

DEFINITIONS AND RELATIONSHIP OF RESEARCH QUESTIONS AND HYPOTHESES

A research question is what a study aims to answer after data analysis and interpretation. The answer is written in length in the discussion section of the paper. Thus, the research question gives a preview of the different parts and variables of the study meant to address the problem posed in the research question. 1 An excellent research question clarifies the research writing while facilitating understanding of the research topic, objective, scope, and limitations of the study. 5

On the other hand, a research hypothesis is an educated statement of an expected outcome. This statement is based on background research and current knowledge. 8 , 9 The research hypothesis makes a specific prediction about a new phenomenon 10 or a formal statement on the expected relationship between an independent variable and a dependent variable. 3 , 11 It provides a tentative answer to the research question to be tested or explored. 4

Hypotheses employ reasoning to predict a theory-based outcome. 10 These can also be developed from theories by focusing on components of theories that have not yet been observed. 10 The validity of hypotheses is often based on the testability of the prediction made in a reproducible experiment. 8

Conversely, hypotheses can also be rephrased as research questions. Several hypotheses based on existing theories and knowledge may be needed to answer a research question. Developing ethical research questions and hypotheses creates a research design that has logical relationships among variables. These relationships serve as a solid foundation for the conduct of the study. 4 , 11 Haphazardly constructed research questions can result in poorly formulated hypotheses and improper study designs, leading to unreliable results. Thus, the formulations of relevant research questions and verifiable hypotheses are crucial when beginning research. 12

CHARACTERISTICS OF GOOD RESEARCH QUESTIONS AND HYPOTHESES

Excellent research questions are specific and focused. These integrate collective data and observations to confirm or refute the subsequent hypotheses. Well-constructed hypotheses are based on previous reports and verify the research context. These are realistic, in-depth, sufficiently complex, and reproducible. More importantly, these hypotheses can be addressed and tested. 13

There are several characteristics of well-developed hypotheses. Good hypotheses are 1) empirically testable 7 , 10 , 11 , 13 ; 2) backed by preliminary evidence 9 ; 3) testable by ethical research 7 , 9 ; 4) based on original ideas 9 ; 5) have evidenced-based logical reasoning 10 ; and 6) can be predicted. 11 Good hypotheses can infer ethical and positive implications, indicating the presence of a relationship or effect relevant to the research theme. 7 , 11 These are initially developed from a general theory and branch into specific hypotheses by deductive reasoning. In the absence of a theory to base the hypotheses, inductive reasoning based on specific observations or findings form more general hypotheses. 10

TYPES OF RESEARCH QUESTIONS AND HYPOTHESES

Research questions and hypotheses are developed according to the type of research, which can be broadly classified into quantitative and qualitative research. We provide a summary of the types of research questions and hypotheses under quantitative and qualitative research categories in Table 1 .

Research questions in quantitative research

In quantitative research, research questions inquire about the relationships among variables being investigated and are usually framed at the start of the study. These are precise and typically linked to the subject population, dependent and independent variables, and research design. 1 Research questions may also attempt to describe the behavior of a population in relation to one or more variables, or describe the characteristics of variables to be measured ( descriptive research questions ). 1 , 5 , 14 These questions may also aim to discover differences between groups within the context of an outcome variable ( comparative research questions ), 1 , 5 , 14 or elucidate trends and interactions among variables ( relationship research questions ). 1 , 5 We provide examples of descriptive, comparative, and relationship research questions in quantitative research in Table 2 .

Hypotheses in quantitative research

In quantitative research, hypotheses predict the expected relationships among variables. 15 Relationships among variables that can be predicted include 1) between a single dependent variable and a single independent variable ( simple hypothesis ) or 2) between two or more independent and dependent variables ( complex hypothesis ). 4 , 11 Hypotheses may also specify the expected direction to be followed and imply an intellectual commitment to a particular outcome ( directional hypothesis ) 4 . On the other hand, hypotheses may not predict the exact direction and are used in the absence of a theory, or when findings contradict previous studies ( non-directional hypothesis ). 4 In addition, hypotheses can 1) define interdependency between variables ( associative hypothesis ), 4 2) propose an effect on the dependent variable from manipulation of the independent variable ( causal hypothesis ), 4 3) state a negative relationship between two variables ( null hypothesis ), 4 , 11 , 15 4) replace the working hypothesis if rejected ( alternative hypothesis ), 15 explain the relationship of phenomena to possibly generate a theory ( working hypothesis ), 11 5) involve quantifiable variables that can be tested statistically ( statistical hypothesis ), 11 6) or express a relationship whose interlinks can be verified logically ( logical hypothesis ). 11 We provide examples of simple, complex, directional, non-directional, associative, causal, null, alternative, working, statistical, and logical hypotheses in quantitative research, as well as the definition of quantitative hypothesis-testing research in Table 3 .

Research questions in qualitative research

Unlike research questions in quantitative research, research questions in qualitative research are usually continuously reviewed and reformulated. The central question and associated subquestions are stated more than the hypotheses. 15 The central question broadly explores a complex set of factors surrounding the central phenomenon, aiming to present the varied perspectives of participants. 15

There are varied goals for which qualitative research questions are developed. These questions can function in several ways, such as to 1) identify and describe existing conditions ( contextual research question s); 2) describe a phenomenon ( descriptive research questions ); 3) assess the effectiveness of existing methods, protocols, theories, or procedures ( evaluation research questions ); 4) examine a phenomenon or analyze the reasons or relationships between subjects or phenomena ( explanatory research questions ); or 5) focus on unknown aspects of a particular topic ( exploratory research questions ). 5 In addition, some qualitative research questions provide new ideas for the development of theories and actions ( generative research questions ) or advance specific ideologies of a position ( ideological research questions ). 1 Other qualitative research questions may build on a body of existing literature and become working guidelines ( ethnographic research questions ). Research questions may also be broadly stated without specific reference to the existing literature or a typology of questions ( phenomenological research questions ), may be directed towards generating a theory of some process ( grounded theory questions ), or may address a description of the case and the emerging themes ( qualitative case study questions ). 15 We provide examples of contextual, descriptive, evaluation, explanatory, exploratory, generative, ideological, ethnographic, phenomenological, grounded theory, and qualitative case study research questions in qualitative research in Table 4 , and the definition of qualitative hypothesis-generating research in Table 5 .

Qualitative studies usually pose at least one central research question and several subquestions starting with How or What . These research questions use exploratory verbs such as explore or describe . These also focus on one central phenomenon of interest, and may mention the participants and research site. 15

Hypotheses in qualitative research

Hypotheses in qualitative research are stated in the form of a clear statement concerning the problem to be investigated. Unlike in quantitative research where hypotheses are usually developed to be tested, qualitative research can lead to both hypothesis-testing and hypothesis-generating outcomes. 2 When studies require both quantitative and qualitative research questions, this suggests an integrative process between both research methods wherein a single mixed-methods research question can be developed. 1

FRAMEWORKS FOR DEVELOPING RESEARCH QUESTIONS AND HYPOTHESES

Research questions followed by hypotheses should be developed before the start of the study. 1 , 12 , 14 It is crucial to develop feasible research questions on a topic that is interesting to both the researcher and the scientific community. This can be achieved by a meticulous review of previous and current studies to establish a novel topic. Specific areas are subsequently focused on to generate ethical research questions. The relevance of the research questions is evaluated in terms of clarity of the resulting data, specificity of the methodology, objectivity of the outcome, depth of the research, and impact of the study. 1 , 5 These aspects constitute the FINER criteria (i.e., Feasible, Interesting, Novel, Ethical, and Relevant). 1 Clarity and effectiveness are achieved if research questions meet the FINER criteria. In addition to the FINER criteria, Ratan et al. described focus, complexity, novelty, feasibility, and measurability for evaluating the effectiveness of research questions. 14

The PICOT and PEO frameworks are also used when developing research questions. 1 The following elements are addressed in these frameworks, PICOT: P-population/patients/problem, I-intervention or indicator being studied, C-comparison group, O-outcome of interest, and T-timeframe of the study; PEO: P-population being studied, E-exposure to preexisting conditions, and O-outcome of interest. 1 Research questions are also considered good if these meet the “FINERMAPS” framework: Feasible, Interesting, Novel, Ethical, Relevant, Manageable, Appropriate, Potential value/publishable, and Systematic. 14

As we indicated earlier, research questions and hypotheses that are not carefully formulated result in unethical studies or poor outcomes. To illustrate this, we provide some examples of ambiguous research question and hypotheses that result in unclear and weak research objectives in quantitative research ( Table 6 ) 16 and qualitative research ( Table 7 ) 17 , and how to transform these ambiguous research question(s) and hypothesis(es) into clear and good statements.

a These statements were composed for comparison and illustrative purposes only.

b These statements are direct quotes from Higashihara and Horiuchi. 16

a This statement is a direct quote from Shimoda et al. 17

The other statements were composed for comparison and illustrative purposes only.

CONSTRUCTING RESEARCH QUESTIONS AND HYPOTHESES

To construct effective research questions and hypotheses, it is very important to 1) clarify the background and 2) identify the research problem at the outset of the research, within a specific timeframe. 9 Then, 3) review or conduct preliminary research to collect all available knowledge about the possible research questions by studying theories and previous studies. 18 Afterwards, 4) construct research questions to investigate the research problem. Identify variables to be accessed from the research questions 4 and make operational definitions of constructs from the research problem and questions. Thereafter, 5) construct specific deductive or inductive predictions in the form of hypotheses. 4 Finally, 6) state the study aims . This general flow for constructing effective research questions and hypotheses prior to conducting research is shown in Fig. 1 .

Research questions are used more frequently in qualitative research than objectives or hypotheses. 3 These questions seek to discover, understand, explore or describe experiences by asking “What” or “How.” The questions are open-ended to elicit a description rather than to relate variables or compare groups. The questions are continually reviewed, reformulated, and changed during the qualitative study. 3 Research questions are also used more frequently in survey projects than hypotheses in experiments in quantitative research to compare variables and their relationships.

Hypotheses are constructed based on the variables identified and as an if-then statement, following the template, ‘If a specific action is taken, then a certain outcome is expected.’ At this stage, some ideas regarding expectations from the research to be conducted must be drawn. 18 Then, the variables to be manipulated (independent) and influenced (dependent) are defined. 4 Thereafter, the hypothesis is stated and refined, and reproducible data tailored to the hypothesis are identified, collected, and analyzed. 4 The hypotheses must be testable and specific, 18 and should describe the variables and their relationships, the specific group being studied, and the predicted research outcome. 18 Hypotheses construction involves a testable proposition to be deduced from theory, and independent and dependent variables to be separated and measured separately. 3 Therefore, good hypotheses must be based on good research questions constructed at the start of a study or trial. 12

In summary, research questions are constructed after establishing the background of the study. Hypotheses are then developed based on the research questions. Thus, it is crucial to have excellent research questions to generate superior hypotheses. In turn, these would determine the research objectives and the design of the study, and ultimately, the outcome of the research. 12 Algorithms for building research questions and hypotheses are shown in Fig. 2 for quantitative research and in Fig. 3 for qualitative research.

EXAMPLES OF RESEARCH QUESTIONS FROM PUBLISHED ARTICLES

• EXAMPLE 1. Descriptive research question (quantitative research)
• - Presents research variables to be assessed (distinct phenotypes and subphenotypes)
• “BACKGROUND: Since COVID-19 was identified, its clinical and biological heterogeneity has been recognized. Identifying COVID-19 phenotypes might help guide basic, clinical, and translational research efforts.
• RESEARCH QUESTION: Does the clinical spectrum of patients with COVID-19 contain distinct phenotypes and subphenotypes? ” 19
• EXAMPLE 2. Relationship research question (quantitative research)
• - Shows interactions between dependent variable (static postural control) and independent variable (peripheral visual field loss)
• “Background: Integration of visual, vestibular, and proprioceptive sensations contributes to postural control. People with peripheral visual field loss have serious postural instability. However, the directional specificity of postural stability and sensory reweighting caused by gradual peripheral visual field loss remain unclear.
• Research question: What are the effects of peripheral visual field loss on static postural control ?” 20
• EXAMPLE 3. Comparative research question (quantitative research)
• - Clarifies the difference among groups with an outcome variable (patients enrolled in COMPERA with moderate PH or severe PH in COPD) and another group without the outcome variable (patients with idiopathic pulmonary arterial hypertension (IPAH))
• “BACKGROUND: Pulmonary hypertension (PH) in COPD is a poorly investigated clinical condition.
• RESEARCH QUESTION: Which factors determine the outcome of PH in COPD?
• STUDY DESIGN AND METHODS: We analyzed the characteristics and outcome of patients enrolled in the Comparative, Prospective Registry of Newly Initiated Therapies for Pulmonary Hypertension (COMPERA) with moderate or severe PH in COPD as defined during the 6th PH World Symposium who received medical therapy for PH and compared them with patients with idiopathic pulmonary arterial hypertension (IPAH) .” 21
• EXAMPLE 4. Exploratory research question (qualitative research)
• - Explores areas that have not been fully investigated (perspectives of families and children who receive care in clinic-based child obesity treatment) to have a deeper understanding of the research problem
• “Problem: Interventions for children with obesity lead to only modest improvements in BMI and long-term outcomes, and data are limited on the perspectives of families of children with obesity in clinic-based treatment. This scoping review seeks to answer the question: What is known about the perspectives of families and children who receive care in clinic-based child obesity treatment? This review aims to explore the scope of perspectives reported by families of children with obesity who have received individualized outpatient clinic-based obesity treatment.” 22
• EXAMPLE 5. Relationship research question (quantitative research)
• - Defines interactions between dependent variable (use of ankle strategies) and independent variable (changes in muscle tone)
• “Background: To maintain an upright standing posture against external disturbances, the human body mainly employs two types of postural control strategies: “ankle strategy” and “hip strategy.” While it has been reported that the magnitude of the disturbance alters the use of postural control strategies, it has not been elucidated how the level of muscle tone, one of the crucial parameters of bodily function, determines the use of each strategy. We have previously confirmed using forward dynamics simulations of human musculoskeletal models that an increased muscle tone promotes the use of ankle strategies. The objective of the present study was to experimentally evaluate a hypothesis: an increased muscle tone promotes the use of ankle strategies. Research question: Do changes in the muscle tone affect the use of ankle strategies ?” 23

EXAMPLES OF HYPOTHESES IN PUBLISHED ARTICLES

• EXAMPLE 1. Working hypothesis (quantitative research)
• - A hypothesis that is initially accepted for further research to produce a feasible theory
• “As fever may have benefit in shortening the duration of viral illness, it is plausible to hypothesize that the antipyretic efficacy of ibuprofen may be hindering the benefits of a fever response when taken during the early stages of COVID-19 illness .” 24
• “In conclusion, it is plausible to hypothesize that the antipyretic efficacy of ibuprofen may be hindering the benefits of a fever response . The difference in perceived safety of these agents in COVID-19 illness could be related to the more potent efficacy to reduce fever with ibuprofen compared to acetaminophen. Compelling data on the benefit of fever warrant further research and review to determine when to treat or withhold ibuprofen for early stage fever for COVID-19 and other related viral illnesses .” 24
• EXAMPLE 2. Exploratory hypothesis (qualitative research)
• - Explores particular areas deeper to clarify subjective experience and develop a formal hypothesis potentially testable in a future quantitative approach
• “We hypothesized that when thinking about a past experience of help-seeking, a self distancing prompt would cause increased help-seeking intentions and more favorable help-seeking outcome expectations .” 25
• “Conclusion
• Although a priori hypotheses were not supported, further research is warranted as results indicate the potential for using self-distancing approaches to increasing help-seeking among some people with depressive symptomatology.” 25
• EXAMPLE 3. Hypothesis-generating research to establish a framework for hypothesis testing (qualitative research)
• “We hypothesize that compassionate care is beneficial for patients (better outcomes), healthcare systems and payers (lower costs), and healthcare providers (lower burnout). ” 26
• Compassionomics is the branch of knowledge and scientific study of the effects of compassionate healthcare. Our main hypotheses are that compassionate healthcare is beneficial for (1) patients, by improving clinical outcomes, (2) healthcare systems and payers, by supporting financial sustainability, and (3) HCPs, by lowering burnout and promoting resilience and well-being. The purpose of this paper is to establish a scientific framework for testing the hypotheses above . If these hypotheses are confirmed through rigorous research, compassionomics will belong in the science of evidence-based medicine, with major implications for all healthcare domains.” 26
• EXAMPLE 4. Statistical hypothesis (quantitative research)
• - An assumption is made about the relationship among several population characteristics ( gender differences in sociodemographic and clinical characteristics of adults with ADHD ). Validity is tested by statistical experiment or analysis ( chi-square test, Students t-test, and logistic regression analysis)
• “Our research investigated gender differences in sociodemographic and clinical characteristics of adults with ADHD in a Japanese clinical sample. Due to unique Japanese cultural ideals and expectations of women's behavior that are in opposition to ADHD symptoms, we hypothesized that women with ADHD experience more difficulties and present more dysfunctions than men . We tested the following hypotheses: first, women with ADHD have more comorbidities than men with ADHD; second, women with ADHD experience more social hardships than men, such as having less full-time employment and being more likely to be divorced.” 27
• “Statistical Analysis
• ( text omitted ) Between-gender comparisons were made using the chi-squared test for categorical variables and Students t-test for continuous variables…( text omitted ). A logistic regression analysis was performed for employment status, marital status, and comorbidity to evaluate the independent effects of gender on these dependent variables.” 27

EXAMPLES OF HYPOTHESIS AS WRITTEN IN PUBLISHED ARTICLES IN RELATION TO OTHER PARTS

• EXAMPLE 1. Background, hypotheses, and aims are provided
• “Pregnant women need skilled care during pregnancy and childbirth, but that skilled care is often delayed in some countries …( text omitted ). The focused antenatal care (FANC) model of WHO recommends that nurses provide information or counseling to all pregnant women …( text omitted ). Job aids are visual support materials that provide the right kind of information using graphics and words in a simple and yet effective manner. When nurses are not highly trained or have many work details to attend to, these job aids can serve as a content reminder for the nurses and can be used for educating their patients (Jennings, Yebadokpo, Affo, & Agbogbe, 2010) ( text omitted ). Importantly, additional evidence is needed to confirm how job aids can further improve the quality of ANC counseling by health workers in maternal care …( text omitted )” 28
• “ This has led us to hypothesize that the quality of ANC counseling would be better if supported by job aids. Consequently, a better quality of ANC counseling is expected to produce higher levels of awareness concerning the danger signs of pregnancy and a more favorable impression of the caring behavior of nurses .” 28
• “This study aimed to examine the differences in the responses of pregnant women to a job aid-supported intervention during ANC visit in terms of 1) their understanding of the danger signs of pregnancy and 2) their impression of the caring behaviors of nurses to pregnant women in rural Tanzania.” 28
• EXAMPLE 2. Background, hypotheses, and aims are provided
• “We conducted a two-arm randomized controlled trial (RCT) to evaluate and compare changes in salivary cortisol and oxytocin levels of first-time pregnant women between experimental and control groups. The women in the experimental group touched and held an infant for 30 min (experimental intervention protocol), whereas those in the control group watched a DVD movie of an infant (control intervention protocol). The primary outcome was salivary cortisol level and the secondary outcome was salivary oxytocin level.” 29
• “ We hypothesize that at 30 min after touching and holding an infant, the salivary cortisol level will significantly decrease and the salivary oxytocin level will increase in the experimental group compared with the control group .” 29
• EXAMPLE 3. Background, aim, and hypothesis are provided
• “In countries where the maternal mortality ratio remains high, antenatal education to increase Birth Preparedness and Complication Readiness (BPCR) is considered one of the top priorities [1]. BPCR includes birth plans during the antenatal period, such as the birthplace, birth attendant, transportation, health facility for complications, expenses, and birth materials, as well as family coordination to achieve such birth plans. In Tanzania, although increasing, only about half of all pregnant women attend an antenatal clinic more than four times [4]. Moreover, the information provided during antenatal care (ANC) is insufficient. In the resource-poor settings, antenatal group education is a potential approach because of the limited time for individual counseling at antenatal clinics.” 30
• “This study aimed to evaluate an antenatal group education program among pregnant women and their families with respect to birth-preparedness and maternal and infant outcomes in rural villages of Tanzania.” 30
• “ The study hypothesis was if Tanzanian pregnant women and their families received a family-oriented antenatal group education, they would (1) have a higher level of BPCR, (2) attend antenatal clinic four or more times, (3) give birth in a health facility, (4) have less complications of women at birth, and (5) have less complications and deaths of infants than those who did not receive the education .” 30

Research questions and hypotheses are crucial components to any type of research, whether quantitative or qualitative. These questions should be developed at the very beginning of the study. Excellent research questions lead to superior hypotheses, which, like a compass, set the direction of research, and can often determine the successful conduct of the study. Many research studies have floundered because the development of research questions and subsequent hypotheses was not given the thought and meticulous attention needed. The development of research questions and hypotheses is an iterative process based on extensive knowledge of the literature and insightful grasp of the knowledge gap. Focused, concise, and specific research questions provide a strong foundation for constructing hypotheses which serve as formal predictions about the research outcomes. Research questions and hypotheses are crucial elements of research that should not be overlooked. They should be carefully thought of and constructed when planning research. This avoids unethical studies and poor outcomes by defining well-founded objectives that determine the design, course, and outcome of the study.

Disclosure: The authors have no potential conflicts of interest to disclose.

Author Contributions:

• Conceptualization: Barroga E, Matanguihan GJ.
• Methodology: Barroga E, Matanguihan GJ.
• Writing - original draft: Barroga E, Matanguihan GJ.
• Writing - review & editing: Barroga E, Matanguihan GJ.

Top Marketing Research Articles Related to Qualitative Research

Learn all about qualitative research. Quirks.com is the largest source of marketing research information.

Search Results

More Filters

Loading filters...

Article Seeing through various lenses: The power of diverse perspectives in marketing research Karen Schofield | September 19, 2024

Article How to protect marketing research from fraud in the age of AI Arjun S | September 17, 2024

Article How to use laddering to uncover the emotions surrounding B2B jobs to be done Tracy Tuten | September 1, 2024

Article Enhancing the virtual backroom experience Sofia Rojo del Busto | September 1, 2024

Article Q Report respondents opine on AI, 
pain points and future plans Joseph Rydholm | September 1, 2024

Article Quirk's Time Capsule – June 2002 Quirk's Staff | September 1, 2024

Article Before You Go: 10 minutes with Lorin Drake, Publix Super Markets Emily C. Koenig | September 1, 2024

Sponsored Article InsightFarm: Co-Creation: AI Prompt Approaches that Work! Kelley Styring | September 1, 2024

Article The top trends that are shaping the marketing research industry Mary Kay Evans | September 5, 2024

Article Work and Play: Juggling, joggling and seeing patterns Nancy Cox | August 20, 2024

Article Free on-demand marketing research insights and learning sessions Quirk's Staff | August 20, 2024

Article Are consumers really irrational? Ruchira Jain | August 20, 2024

Article Creating concrete designs: How attribute mapping revolutionizes the product development cycle Elspeth Bradley | August 19, 2024

Article Integrating conversational AI into qualitative research processes Siddish Reddy | August 14, 2024

Sponsored Article The smart, fun and caring women of MHN 2024 Murray Hill National Staff | July 1, 2024