tenny 2022 qualitative research

Qualitative Research: Characteristics, Design, Methods & Examples

Lauren McCall

MSc Health Psychology Graduate

MSc, Health Psychology, University of Nottingham

Lauren obtained an MSc in Health Psychology from The University of Nottingham with a distinction classification.

Learn about our Editorial Process

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

Qualitative research is a type of research methodology that focuses on gathering and analyzing non-numerical data to gain a deeper understanding of human behavior, experiences, and perspectives.

It aims to explore the “why” and “how” of a phenomenon rather than the “what,” “where,” and “when” typically addressed by quantitative research.

Unlike quantitative research, which focuses on gathering and analyzing numerical data for statistical analysis, qualitative research involves researchers interpreting data to identify themes, patterns, and meanings.

Qualitative research can be used to:

• Gain deep contextual understandings of the subjective social reality of individuals
• To answer questions about experience and meaning from the participant’s perspective
• To design hypotheses, theory must be researched using qualitative methods to determine what is important before research can begin. 

Examples of qualitative research questions include: 

• How does stress influence young adults’ behavior?
• What factors influence students’ school attendance rates in developed countries?
• How do adults interpret binge drinking in the UK?
• What are the psychological impacts of cervical cancer screening in women?
• How can mental health lessons be integrated into the school curriculum? 

Characteristics 

Naturalistic setting.

Individuals are studied in their natural setting to gain a deeper understanding of how people experience the world. This enables the researcher to understand a phenomenon close to how participants experience it. 

Naturalistic settings provide valuable contextual information to help researchers better understand and interpret the data they collect.

The environment, social interactions, and cultural factors can all influence behavior and experiences, and these elements are more easily observed in real-world settings.

Reality is socially constructed

Qualitative research aims to understand how participants make meaning of their experiences – individually or in social contexts. It assumes there is no objective reality and that the social world is interpreted (Yilmaz, 2013). 

The primacy of subject matter 

The primary aim of qualitative research is to understand the perspectives, experiences, and beliefs of individuals who have experienced the phenomenon selected for research rather than the average experiences of groups of people (Minichiello, 1990).

An in-depth understanding is attained since qualitative techniques allow participants to freely disclose their experiences, thoughts, and feelings without constraint (Tenny et al., 2022). 

Variables are complex, interwoven, and difficult to measure

Factors such as experiences, behaviors, and attitudes are complex and interwoven, so they cannot be reduced to isolated variables , making them difficult to measure quantitatively.

However, a qualitative approach enables participants to describe what, why, or how they were thinking/ feeling during a phenomenon being studied (Yilmaz, 2013). 

Emic (insider’s point of view)

The phenomenon being studied is centered on the participants’ point of view (Minichiello, 1990).

Emic is used to describe how participants interact, communicate, and behave in the research setting (Scarduzio, 2017).

Interpretive analysis

In qualitative research, interpretive analysis is crucial in making sense of the collected data.

This process involves examining the raw data, such as interview transcripts, field notes, or documents, and identifying the underlying themes, patterns, and meanings that emerge from the participants’ experiences and perspectives.

Collecting Qualitative Data

There are four main research design methods used to collect qualitative data: observations, interviews,  focus groups, and ethnography.

Observations

This method involves watching and recording phenomena as they occur in nature. Observation can be divided into two types: participant and non-participant observation.

In participant observation, the researcher actively participates in the situation/events being observed.

In non-participant observation, the researcher is not an active part of the observation and tries not to influence the behaviors they are observing (Busetto et al., 2020). 

Observations can be covert (participants are unaware that a researcher is observing them) or overt (participants are aware of the researcher’s presence and know they are being observed).

However, awareness of an observer’s presence may influence participants’ behavior. 

Interviews give researchers a window into the world of a participant by seeking their account of an event, situation, or phenomenon. They are usually conducted on a one-to-one basis and can be distinguished according to the level at which they are structured (Punch, 2013). 

Structured interviews involve predetermined questions and sequences to ensure replicability and comparability. However, they are unable to explore emerging issues.

Informal interviews consist of spontaneous, casual conversations which are closer to the truth of a phenomenon. However, information is gathered using quick notes made by the researcher and is therefore subject to recall bias. 

Semi-structured interviews have a flexible structure, phrasing, and placement so emerging issues can be explored (Denny & Weckesser, 2022).

The use of probing questions and clarification can lead to a detailed understanding, but semi-structured interviews can be time-consuming and subject to interviewer bias. 

Focus groups 

Similar to interviews, focus groups elicit a rich and detailed account of an experience. However, focus groups are more dynamic since participants with shared characteristics construct this account together (Denny & Weckesser, 2022).

A shared narrative is built between participants to capture a group experience shaped by a shared context. 

The researcher takes on the role of a moderator, who will establish ground rules and guide the discussion by following a topic guide to focus the group discussions.

Typically, focus groups have 4-10 participants as a discussion can be difficult to facilitate with more than this, and this number allows everyone the time to speak.

Ethnography

Ethnography is a methodology used to study a group of people’s behaviors and social interactions in their environment (Reeves et al., 2008).

Data are collected using methods such as observations, field notes, or structured/ unstructured interviews.

The aim of ethnography is to provide detailed, holistic insights into people’s behavior and perspectives within their natural setting. In order to achieve this, researchers immerse themselves in a community or organization. 

Due to the flexibility and real-world focus of ethnography, researchers are able to gather an in-depth, nuanced understanding of people’s experiences, knowledge and perspectives that are influenced by culture and society.

In order to develop a representative picture of a particular culture/ context, researchers must conduct extensive field work. 

This can be time-consuming as researchers may need to immerse themselves into a community/ culture for a few days, or possibly a few years.

Qualitative Data Analysis Methods

Different methods can be used for analyzing qualitative data. The researcher chooses based on the objectives of their study. 

The researcher plays a key role in the interpretation of data, making decisions about the coding, theming, decontextualizing, and recontextualizing of data (Starks & Trinidad, 2007). 

Grounded theory

Grounded theory is a qualitative method specifically designed to inductively generate theory from data. It was developed by Glaser and Strauss in 1967 (Glaser & Strauss, 2017).

This methodology aims to develop theories (rather than test hypotheses) that explain a social process, action, or interaction (Petty et al., 2012). To inform the developing theory, data collection and analysis run simultaneously. 

There are three key types of coding used in grounded theory: initial (open), intermediate (axial), and advanced (selective) coding. 

Throughout the analysis, memos should be created to document methodological and theoretical ideas about the data. Data should be collected and analyzed until data saturation is reached and a theory is developed. 

Content analysis

Content analysis was first used in the early twentieth century to analyze textual materials such as newspapers and political speeches.

Content analysis is a research method used to identify and analyze the presence and patterns of themes, concepts, or words in data (Vaismoradi et al., 2013). 

This research method can be used to analyze data in different formats, which can be written, oral, or visual. 

The goal of content analysis is to develop themes that capture the underlying meanings of data (Schreier, 2012). 

Qualitative content analysis can be used to validate existing theories, support the development of new models and theories, and provide in-depth descriptions of particular settings or experiences.

The following six steps provide a guideline for how to conduct qualitative content analysis.

• Define a Research Question : To start content analysis, a clear research question should be developed.
• Identify and Collect Data : Establish the inclusion criteria for your data. Find the relevant sources to analyze.
• Define the Unit or Theme of Analysis : Categorize the content into themes. Themes can be a word, phrase, or sentence.
• Develop Rules for Coding your Data : Define a set of coding rules to ensure that all data are coded consistently.
• Code the Data : Follow the coding rules to categorize data into themes.
• Analyze the Results and Draw Conclusions : Examine the data to identify patterns and draw conclusions in relation to your research question.

Discourse analysis

Discourse analysis is a research method used to study written/ spoken language in relation to its social context (Wood & Kroger, 2000).

In discourse analysis, the researcher interprets details of language materials and the context in which it is situated.

Discourse analysis aims to understand the functions of language (how language is used in real life) and how meaning is conveyed by language in different contexts. Researchers use discourse analysis to investigate social groups and how language is used to achieve specific communication goals.

Different methods of discourse analysis can be used depending on the aims and objectives of a study. However, the following steps provide a guideline on how to conduct discourse analysis.

• Define the Research Question : Develop a relevant research question to frame the analysis.
• Gather Data and Establish the Context : Collect research materials (e.g., interview transcripts, documents). Gather factual details and review the literature to construct a theory about the social and historical context of your study.
• Analyze the Content : Closely examine various components of the text, such as the vocabulary, sentences, paragraphs, and structure of the text. Identify patterns relevant to the research question to create codes, then group these into themes.
• Review the Results : Reflect on the findings to examine the function of the language, and the meaning and context of the discourse. 

Thematic analysis

Thematic analysis is a method used to identify, interpret, and report patterns in data, such as commonalities or contrasts. 

Although the origin of thematic analysis can be traced back to the early twentieth century, understanding and clarity of thematic analysis is attributed to Braun and Clarke (2006).

Thematic analysis aims to develop themes (patterns of meaning) across a dataset to address a research question. 

In thematic analysis, qualitative data is gathered using techniques such as interviews, focus groups, and questionnaires. Audio recordings are transcribed. The dataset is then explored and interpreted by a researcher to identify patterns. 

This occurs through the rigorous process of data familiarisation, coding, theme development, and revision. These identified patterns provide a summary of the dataset and can be used to address a research question.

Themes are developed by exploring the implicit and explicit meanings within the data. Two different approaches are used to generate themes: inductive and deductive. 

An inductive approach allows themes to emerge from the data. In contrast, a deductive approach uses existing theories or knowledge to apply preconceived ideas to the data.

Phases of Thematic Analysis

Braun and Clarke (2006) provide a guide of the six phases of thematic analysis. These phases can be applied flexibly to fit research questions and data. 

Template analysis

Template analysis refers to a specific method of thematic analysis which uses hierarchical coding (Brooks et al., 2014).

Template analysis is used to analyze textual data, for example, interview transcripts or open-ended responses on a written questionnaire.

To conduct template analysis, a coding template must be developed (usually from a subset of the data) and subsequently revised and refined. This template represents the themes identified by researchers as important in the dataset. 

Codes are ordered hierarchically within the template, with the highest-level codes demonstrating overarching themes in the data and lower-level codes representing constituent themes with a narrower focus.

A guideline for the main procedural steps for conducting template analysis is outlined below.

• Familiarization with the Data : Read (and reread) the dataset in full. Engage, reflect, and take notes on data that may be relevant to the research question.
• Preliminary Coding : Identify initial codes using guidance from the a priori codes, identified before the analysis as likely to be beneficial and relevant to the analysis.
• Organize Themes : Organize themes into meaningful clusters. Consider the relationships between the themes both within and between clusters.
• Produce an Initial Template : Develop an initial template. This may be based on a subset of the data.
• Apply and Develop the Template : Apply the initial template to further data and make any necessary modifications. Refinements of the template may include adding themes, removing themes, or changing the scope/title of themes. 
• Finalize Template : Finalize the template, then apply it to the entire dataset. 

Frame analysis

Frame analysis is a comparative form of thematic analysis which systematically analyzes data using a matrix output.

Ritchie and Spencer (1994) developed this set of techniques to analyze qualitative data in applied policy research. Frame analysis aims to generate theory from data.

Frame analysis encourages researchers to organize and manage their data using summarization.

This results in a flexible and unique matrix output, in which individual participants (or cases) are represented by rows and themes are represented by columns. 

Each intersecting cell is used to summarize findings relating to the corresponding participant and theme.

Frame analysis has five distinct phases which are interrelated, forming a methodical and rigorous framework.

• Familiarization with the Data : Familiarize yourself with all the transcripts. Immerse yourself in the details of each transcript and start to note recurring themes.
• Develop a Theoretical Framework : Identify recurrent/ important themes and add them to a chart. Provide a framework/ structure for the analysis.
• Indexing : Apply the framework systematically to the entire study data.
• Summarize Data in Analytical Framework : Reduce the data into brief summaries of participants’ accounts.
• Mapping and Interpretation : Compare themes and subthemes and check against the original transcripts. Group the data into categories and provide an explanation for them.

Preventing Bias in Qualitative Research

To evaluate qualitative studies, the CASP (Critical Appraisal Skills Programme) checklist for qualitative studies can be used to ensure all aspects of a study have been considered (CASP, 2018).

The quality of research can be enhanced and assessed using criteria such as checklists, reflexivity, co-coding, and member-checking. 

Co-coding 

Relying on only one researcher to interpret rich and complex data may risk key insights and alternative viewpoints being missed. Therefore, coding is often performed by multiple researchers.

A common strategy must be defined at the beginning of the coding process  (Busetto et al., 2020). This includes establishing a useful coding list and finding a common definition of individual codes.

Transcripts are initially coded independently by researchers and then compared and consolidated to minimize error or bias and to bring confirmation of findings. 

Member checking

Member checking (or respondent validation) involves checking back with participants to see if the research resonates with their experiences (Russell & Gregory, 2003).

Data can be returned to participants after data collection or when results are first available. For example, participants may be provided with their interview transcript and asked to verify whether this is a complete and accurate representation of their views.

Participants may then clarify or elaborate on their responses to ensure they align with their views (Shenton, 2004).

This feedback becomes part of data collection and ensures accurate descriptions/ interpretations of phenomena (Mays & Pope, 2000). 

Reflexivity in qualitative research

Reflexivity typically involves examining your own judgments, practices, and belief systems during data collection and analysis. It aims to identify any personal beliefs which may affect the research. 

Reflexivity is essential in qualitative research to ensure methodological transparency and complete reporting. This enables readers to understand how the interaction between the researcher and participant shapes the data.

Depending on the research question and population being researched, factors that need to be considered include the experience of the researcher, how the contact was established and maintained, age, gender, and ethnicity.

These details are important because, in qualitative research, the researcher is a dynamic part of the research process and actively influences the outcome of the research (Boeije, 2014). 

Reflexivity Example

Who you are and your characteristics influence how you collect and analyze data. Here is an example of a reflexivity statement for research on smoking. I am a 30-year-old white female from a middle-class background. I live in the southwest of England and have been educated to master’s level. I have been involved in two research projects on oral health. I have never smoked, but I have witnessed how smoking can cause ill health from my volunteering in a smoking cessation clinic. My research aspirations are to help to develop interventions to help smokers quit.

Establishing Trustworthiness in Qualitative Research

Trustworthiness is a concept used to assess the quality and rigor of qualitative research. Four criteria are used to assess a study’s trustworthiness: credibility, transferability, dependability, and confirmability.

1. Credibility in Qualitative Research

Credibility refers to how accurately the results represent the reality and viewpoints of the participants.

To establish credibility in research, participants’ views and the researcher’s representation of their views need to align (Tobin & Begley, 2004).

To increase the credibility of findings, researchers may use data source triangulation, investigator triangulation, peer debriefing, or member checking (Lincoln & Guba, 1985). 

2. Transferability in Qualitative Research

Transferability refers to how generalizable the findings are: whether the findings may be applied to another context, setting, or group (Tobin & Begley, 2004).

Transferability can be enhanced by giving thorough and in-depth descriptions of the research setting, sample, and methods (Nowell et al., 2017). 

3. Dependability in Qualitative Research

Dependability is the extent to which the study could be replicated under similar conditions and the findings would be consistent.

Researchers can establish dependability using methods such as audit trails so readers can see the research process is logical and traceable (Koch, 1994).

4. Confirmability in Qualitative Research

Confirmability is concerned with establishing that there is a clear link between the researcher’s interpretations/ findings and the data.

Researchers can achieve confirmability by demonstrating how conclusions and interpretations were arrived at (Nowell et al., 2017).

This enables readers to understand the reasoning behind the decisions made. 

Audit Trails in Qualitative Research

An audit trail provides evidence of the decisions made by the researcher regarding theory, research design, and data collection, as well as the steps they have chosen to manage, analyze, and report data. 

The researcher must provide a clear rationale to demonstrate how conclusions were reached in their study.

A clear description of the research path must be provided to enable readers to trace through the researcher’s logic (Halpren, 1983).

Researchers should maintain records of the raw data, field notes, transcripts, and a reflective journal in order to provide a clear audit trail. 

Discovery of unexpected data

Open-ended questions in qualitative research mean the researcher can probe an interview topic and enable the participant to elaborate on responses in an unrestricted manner.

This allows unexpected data to emerge, which can lead to further research into that topic. 

The exploratory nature of qualitative research helps generate hypotheses that can be tested quantitatively (Busetto et al., 2020).

Flexibility

Data collection and analysis can be modified and adapted to take the research in a different direction if new ideas or patterns emerge in the data.

This enables researchers to investigate new opportunities while firmly maintaining their research goals. 

Naturalistic settings

The behaviors of participants are recorded in real-world settings. Studies that use real-world settings have high ecological validity since participants behave more authentically. 

Limitations

Time-consuming .

Qualitative research results in large amounts of data which often need to be transcribed and analyzed manually.

Even when software is used, transcription can be inaccurate, and using software for analysis can result in many codes which need to be condensed into themes. 

Subjectivity 

The researcher has an integral role in collecting and interpreting qualitative data. Therefore, the conclusions reached are from their perspective and experience.

Consequently, interpretations of data from another researcher may vary greatly. 

Limited generalizability

The aim of qualitative research is to provide a detailed, contextualized understanding of an aspect of the human experience from a relatively small sample size.

Despite rigorous analysis procedures, conclusions drawn cannot be generalized to the wider population since data may be biased or unrepresentative.

Therefore, results are only applicable to a small group of the population. 

While individual qualitative studies are often limited in their generalizability due to factors such as sample size and context, metasynthesis enables researchers to synthesize findings from multiple studies, potentially leading to more generalizable conclusions.

By integrating findings from studies conducted in diverse settings and with different populations, metasynthesis can provide broader insights into the phenomenon of interest.

Extraneous variables

Qualitative research is often conducted in real-world settings. This may cause results to be unreliable since extraneous variables may affect the data, for example:

• Situational variables : different environmental conditions may influence participants’ behavior in a study. The random variation in factors (such as noise or lighting) may be difficult to control in real-world settings.
• Participant characteristics : this includes any characteristics that may influence how a participant answers/ behaves in a study. This may include a participant’s mood, gender, age, ethnicity, sexual identity, IQ, etc.
• Experimenter effect : experimenter effect refers to how a researcher’s unintentional influence can change the outcome of a study. This occurs when (i) their interactions with participants unintentionally change participants’ behaviors or (ii) due to errors in observation, interpretation, or analysis. 

What sample size should qualitative research be?

The sample size for qualitative studies has been recommended to include a minimum of 12 participants to reach data saturation (Braun, 2013).

Are surveys qualitative or quantitative?

Surveys can be used to gather information from a sample qualitatively or quantitatively. Qualitative surveys use open-ended questions to gather detailed information from a large sample using free text responses.

The use of open-ended questions allows for unrestricted responses where participants use their own words, enabling the collection of more in-depth information than closed-ended questions.

In contrast, quantitative surveys consist of closed-ended questions with multiple-choice answer options. Quantitative surveys are ideal to gather a statistical representation of a population.

What are the ethical considerations of qualitative research?

Before conducting a study, you must think about any risks that could occur and take steps to prevent them. Participant Protection : Researchers must protect participants from physical and mental harm. This means you must not embarrass, frighten, offend, or harm participants. Transparency : Researchers are obligated to clearly communicate how they will collect, store, analyze, use, and share the data. Confidentiality : You need to consider how to maintain the confidentiality and anonymity of participants’ data.

What is triangulation in qualitative research?

Triangulation refers to the use of several approaches in a study to comprehensively understand phenomena. This method helps to increase the validity and credibility of research findings. 

Types of triangulation include method triangulation (using multiple methods to gather data); investigator triangulation (multiple researchers for collecting/ analyzing data), theory triangulation (comparing several theoretical perspectives to explain a phenomenon), and data source triangulation (using data from various times, locations, and people; Carter et al., 2014).

Why is qualitative research important?

Qualitative research allows researchers to describe and explain the social world. The exploratory nature of qualitative research helps to generate hypotheses that can then be tested quantitatively.

In qualitative research, participants are able to express their thoughts, experiences, and feelings without constraint.

Additionally, researchers are able to follow up on participants’ answers in real-time, generating valuable discussion around a topic. This enables researchers to gain a nuanced understanding of phenomena which is difficult to attain using quantitative methods.

What is coding data in qualitative research?

Coding data is a qualitative data analysis strategy in which a section of text is assigned with a label that describes its content.

These labels may be words or phrases which represent important (and recurring) patterns in the data.

This process enables researchers to identify related content across the dataset. Codes can then be used to group similar types of data to generate themes.

What is the difference between qualitative and quantitative research?

Qualitative research involves the collection and analysis of non-numerical data in order to understand experiences and meanings from the participant’s perspective.

This can provide rich, in-depth insights on complicated phenomena. Qualitative data may be collected using interviews, focus groups, or observations.

In contrast, quantitative research involves the collection and analysis of numerical data to measure the frequency, magnitude, or relationships of variables. This can provide objective and reliable evidence that can be generalized to the wider population.

Quantitative data may be collected using closed-ended questionnaires or experiments.

What is trustworthiness in qualitative research?

Trustworthiness is a concept used to assess the quality and rigor of qualitative research. Four criteria are used to assess a study’s trustworthiness: credibility, transferability, dependability, and confirmability. 

Credibility refers to how accurately the results represent the reality and viewpoints of the participants. Transferability refers to whether the findings may be applied to another context, setting, or group.

Dependability is the extent to which the findings are consistent and reliable. Confirmability refers to the objectivity of findings (not influenced by the bias or assumptions of researchers).

What is data saturation in qualitative research?

Data saturation is a methodological principle used to guide the sample size of a qualitative research study.

Data saturation is proposed as a necessary methodological component in qualitative research (Saunders et al., 2018) as it is a vital criterion for discontinuing data collection and/or analysis. 

The intention of data saturation is to find “no new data, no new themes, no new coding, and ability to replicate the study” (Guest et al., 2006). Therefore, enough data has been gathered to make conclusions.

Why is sampling in qualitative research important?

In quantitative research, large sample sizes are used to provide statistically significant quantitative estimates.

This is because quantitative research aims to provide generalizable conclusions that represent populations.

However, the aim of sampling in qualitative research is to gather data that will help the researcher understand the depth, complexity, variation, or context of a phenomenon. The small sample sizes in qualitative studies support the depth of case-oriented analysis.

What is narrative analysis?

Narrative analysis is a qualitative research method used to understand how individuals create stories from their personal experiences.

There is an emphasis on understanding the context in which a narrative is constructed, recognizing the influence of historical, cultural, and social factors on storytelling.

Researchers can use different methods together to explore a research question.

Some narrative researchers focus on the content of what is said, using thematic narrative analysis, while others focus on the structure, such as holistic-form or categorical-form structural narrative analysis. Others focus on how the narrative is produced and performed.

Boeije, H. (2014). Analysis in qualitative research. Sage.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative research in psychology , 3 (2), 77-101. https://doi.org/10.1191/1478088706qp063oa

Brooks, J., McCluskey, S., Turley, E., & King, N. (2014). The utility of template analysis in qualitative psychology research. Qualitative Research in Psychology , 12 (2), 202–222. https://doi.org/10.1080/14780887.2014.955224

Busetto, L., Wick, W., & Gumbinger, C. (2020). How to use and assess qualitative research methods. Neurological research and practice , 2 (1), 14-14. https://doi.org/10.1186/s42466-020-00059-z 

Carter, N., Bryant-Lukosius, D., DiCenso, A., Blythe, J., & Neville, A. J. (2014). The use of triangulation in qualitative research. Oncology nursing forum , 41 (5), 545–547. https://doi.org/10.1188/14.ONF.545-547

Critical Appraisal Skills Programme. (2018). CASP Checklist: 10 questions to help you make sense of a Qualitative research. https://casp-uk.net/images/checklist/documents/CASP-Qualitative-Studies-Checklist/CASP-Qualitative-Checklist-2018_fillable_form.pdf Accessed: March 15 2023

Clarke, V., & Braun, V. (2013). Successful qualitative research: A practical guide for beginners. Successful Qualitative Research , 1-400.

Denny, E., & Weckesser, A. (2022). How to do qualitative research?: Qualitative research methods. BJOG : an international journal of obstetrics and gynaecology , 129 (7), 1166-1167. https://doi.org/10.1111/1471-0528.17150 

Glaser, B. G., & Strauss, A. L. (2017). The discovery of grounded theory. The Discovery of Grounded Theory , 1–18. https://doi.org/10.4324/9780203793206-1

Guest, G., Bunce, A., & Johnson, L. (2006). How many interviews are enough? An experiment with data saturation and variability. Field Methods, 18 (1), 59-82. doi:10.1177/1525822X05279903

Halpren, E. S. (1983). Auditing naturalistic inquiries: The development and application of a model (Unpublished doctoral dissertation). Indiana University, Bloomington.

Hammarberg, K., Kirkman, M., & de Lacey, S. (2016). Qualitative research methods: When to use them and how to judge them. Human Reproduction , 31 (3), 498–501. https://doi.org/10.1093/humrep/dev334

Koch, T. (1994). Establishing rigour in qualitative research: The decision trail. Journal of Advanced Nursing, 19, 976–986. doi:10.1111/ j.1365-2648.1994.tb01177.x

Lincoln, Y., & Guba, E. G. (1985). Naturalistic inquiry. Newbury Park, CA: Sage.

Mays, N., & Pope, C. (2000). Assessing quality in qualitative research. BMJ, 320(7226), 50–52.

Minichiello, V. (1990). In-Depth Interviewing: Researching People. Longman Cheshire.

Nowell, L. S., Norris, J. M., White, D. E., & Moules, N. J. (2017). Thematic Analysis: Striving to Meet the Trustworthiness Criteria. International Journal of Qualitative Methods, 16 (1). https://doi.org/10.1177/1609406917733847

Petty, N. J., Thomson, O. P., & Stew, G. (2012). Ready for a paradigm shift? part 2: Introducing qualitative research methodologies and methods. Manual Therapy , 17 (5), 378–384. https://doi.org/10.1016/j.math.2012.03.004

Punch, K. F. (2013). Introduction to social research: Quantitative and qualitative approaches. London: Sage

Reeves, S., Kuper, A., & Hodges, B. D. (2008). Qualitative research methodologies: Ethnography. BMJ , 337 (aug07 3). https://doi.org/10.1136/bmj.a1020

Russell, C. K., & Gregory, D. M. (2003). Evaluation of qualitative research studies. Evidence Based Nursing, 6 (2), 36–40.

Saunders, B., Sim, J., Kingstone, T., Baker, S., Waterfield, J., Bartlam, B., Burroughs, H., & Jinks, C. (2018). Saturation in qualitative research: exploring its conceptualization and operationalization. Quality & quantity , 52 (4), 1893–1907. https://doi.org/10.1007/s11135-017-0574-8

Scarduzio, J. A. (2017). Emic approach to qualitative research. The International Encyclopedia of Communication Research Methods, 1–2 . https://doi.org/10.1002/9781118901731.iecrm0082

Schreier, M. (2012). Qualitative content analysis in practice / Margrit Schreier.

Shenton, A. K. (2004). Strategies for ensuring trustworthiness in qualitative research projects. Education for Information, 22 , 63–75.

Starks, H., & Trinidad, S. B. (2007). Choose your method: a comparison of phenomenology, discourse analysis, and grounded theory. Qualitative health research , 17 (10), 1372–1380. https://doi.org/10.1177/1049732307307031

Tenny, S., Brannan, J. M., & Brannan, G. D. (2022). Qualitative Study. In StatPearls. StatPearls Publishing.

Tobin, G. A., & Begley, C. M. (2004). Methodological rigour within a qualitative framework. Journal of Advanced Nursing, 48, 388–396. doi:10.1111/j.1365-2648.2004.03207.x

Vaismoradi, M., Turunen, H., & Bondas, T. (2013). Content analysis and thematic analysis: Implications for conducting a qualitative descriptive study. Nursing & health sciences , 15 (3), 398-405. https://doi.org/10.1111/nhs.12048

Wood L. A., Kroger R. O. (2000). Doing discourse analysis: Methods for studying action in talk and text. Sage.

Yilmaz, K. (2013). Comparison of Quantitative and Qualitative Research Traditions: epistemological, theoretical, and methodological differences. European journal of education , 48 (2), 311-325. https://doi.org/10.1111/ejed.12014

OPINION article

How to construct an appropriate research design for the study of learner identity in blended learning.

• 1 International College, Fujian Agriculture and Forestry University, Fuzhou, China
• 2 School of Education, Shanghai International Studies University, Shanghai, China

1. Introduction

With English language becoming the norm for academic communication and knowledge exchange for scholars and students around the world, the focus of English as a foreign language (EFL) or English as a second language (ESL) has gradually shifted from English for general purposes (EGP) to English for academic purposes (EAP) ( Li, 2020 ; Maswana and Yamada, 2021 ). EAP aims to cultivate learners' academic English language ability as well as to broaden their knowledge of disciplinary culture ( Hyland and Hamp-Lyons, 2002 ; Jund, 2010 ; Douglas and Rosvold, 2018 ). Under this circumstance, learner identity in EAP education witnesses an increasing attention from researchers in the field (see Etherington, 2006 ; Charles and Pecorari, 2022 ). Some studies ( Lave and Wenger, 1991 ; Sinha, 1999 ; Coll Salvador and Falsafi, 2010 ; Delahunty et al., 2014 ) reveal that the acquisition of new knowledge has an impact on individual's construction of learner identity in a given context. Compared with massive studies on learner identity constructed in traditional face-to-face classrooms ( Trent, 2008 ; Wearmouth et al., 2011 ; Nasrollahi Shahri, 2018 ; Lan and Lan, 2022 ; Xu and Kim, 2022 ) or online learning ( Ginns and Ellis, 2007 ; Mayrberger and Linke, 2014 ; Kwon et al., 2021 ), to date, research studies on learner identity constructed with blended learning method seem scarce and insufficient, attaching to EAP courses in particular.

Besides, it appears that few studies have examined learner identity from the perspective of dynamic process in collaborative writing. To be more specific, collaborative writing is frequently employed as a learner-centered approach to engage students in English learning because of the limited class hours set for the course ( Chen et al., 2022 ; Lan and Lan, 2022 ). Abound with meaning of negotiation, the benefits of collaborative writing in facilitating interaction and development of basic language skills have been widely documented ( Storch, 2005 ; Kim, 2008 ; Wigglesworth and Storch, 2009 ; Shehadeh, 2011 ; Dobao, 2014 ; Ahmad, 2020 ; Anggraini et al., 2020 ). In addition, learner identity construction in collaborative writing has also captured researchers' attention (see Arnold et al., 2012 ; King, 2015 ; Brown and Pehrson, 2019 ; Chen et al., 2021 ). Among the scarce studies on learner identity, researchers mainly focus on the linguistic elements in learners' written products rather than the dynamic process to examine learner identity in collaborative writing.

As teachers and researchers, we show our concern about the learner identity in blended EAP education, and we read with great interest the newly-published article entitled “ Exploring learner identity in the blended learning context: A case study of collaborative writing ” by Chen et al. (2022) . For the sake of reader's convenience, we will refer it as “the article” hereafter. The article aimed to explore six Chinese university students' learner identities constructed in the online and offline collaborative writing sessions of an EAP course. It paid special attention to the construction of learner identity in relation to learners' engagement and interaction during the process of blended collaborative writing section. In our view, the article has bridged two above-mentioned gaps and contributed a conclusion with both interpretative and illuminating evidences, which demystified and specified the construction of learner identity and its influencing factors in a blended EAP course. We believe the article will provide researchers and practitioners with a new perspective of learner identity construction. Therefore, we would like to comment on the strengths and weaknesses of the article, especially focusing on the validity and reliability of research design, aiming to provide constructive suggestions for future relevant studies in the field. In order to better guide our analysis, we propose the following two research questions:

1. How did the article select participants in order to dig out more relevant data to address its research questions?

2. How did the article analyze the data to facilitate the in-depth exploration and understanding of the intricate phenomenon such as learner identity construction in relation to learners' engagement and interaction?

In our discussion process, we adopt content analysis method ( Bell et al., 2022 ) to examine the selection of participants, data analysis and research findings of the article. Concerning the data analysis process in our article, first, we read the article recursively and independently. Then, we coded and categorized the description and discussion texts relating to the selection of participants, data analysis and research findings of the article. For the sake of trustworthiness, we compared each other's coding, discussed how to reach a consensus on different coding between us, and finally modified them until agreement and consistency were maintained. We also sought feedbacks from two expert researchers to solicit their suggestions for the purpose of triangulation.

Before we analyze the article in detail, we think it is necessary to review and explicate a key term in the article, namely, learner identity in order to help better understand and interpret the article. Some researchers (see Sampson, 1978 ; Burke, 1980 ; Scotton, 1980 ; Heller, 1984 ) interpreted learner identity as fixed personalities, learning styles, and motivations, but recent studies of learner identity adopt a dynamic approach, thus making a sharp contrast to those in the early stage. Coll Salvador and Falsafi (2010) made a distinction between learner identity and learner identity process (LIP), in which learner identity was a fixed image separating from learning situations, while LIP was a process that emphasized the experience and adaption of learner identity to a particular learning context. Based on a post structural perspective, Norton (1997 , p. 410) defined identity as “how a person understands his or her relationship to the world, how that relationship is structured across time and space, and how the person understands possibilities for the future”. This definition was echoed by Flórez González (2018 ) who claimed identity was fluid, context-dependent, and context-producing, constructed in certain historical and cultural circumstances (see also Toohey, 2000 ; Pavlenko and Blackledge, 2004 ; Norton and Toohey, 2011 ; Nasrollahi Shahri, 2018 ; Lan and Lan, 2022 ).

In following sections, we will first provide a summary of the article, then discuss its advantages and limitations by addressing our two guiding questions, and finally summarize the implications of the article.

2. The study

Qualitative in nature, the article employed a case study approach to examine learner identity construction and its influencing factors in a blended learning context. Six non-English major male participants were selected from a comprehensive university based on an initial four-week observation due to their diverse engagement in learning activities.

In regard to the data collection, the article collected multiple sources of data derived from four collaborative writing sessions, including class observations, field notes, semi-structured interviews, history logs on the writing platform and the transcriptions of participants' offline group discussions for the purpose of triangulation.

With reference to the data analysis, the article examined the data that revealed the learner identity and their influencing factors inductively and deductively by classifying the data into online and offline categories. First, the data that expressed learner identity in offline collaborative learning sessions were analyzed by taking a discourse analysis method. Specifically, the article identified, coded and categorized participants' verbal characteristics in offline classroom discussions by drawing on Poupore (2016) analytical framework, followed with the statistics of frequencies of each participant's verbal characteristics. In this way, the article aimed to reveal learner identity separately. Second, the data that reported learner identities in online collaborative writing sessions were analyzed by adopting the framework of work load roles proposed by Arnold et al. (2012) . Be specific, the frequencies of participants' writing revisions which acted as a main reflection of participant' online engagement were counted to investigate the learner identities in online sessions. Finally, two rounds of semi-structured interviews were transcribed, coded and categorized to examine the factors influencing learner identity construction in online and offline sessions.

The article revealed three major findings based on the data analysis. First, the article demonstrated that the construction of learner identity in blended learning depended largely on specific learning activities and learning contexts, with more positive identities in offline sessions and negative ones in online sessions. Such divergence may be caused by teacher's different involvement or pedagogical guidance in two learning sessions. In addition, the article also disclosed that both individual and contextual factors had an impact on learner identity, in which individual factors intersected to affect learner identity construction in both learning sessions, while the impact of contextual factors changed according to different learning sessions. Finally, the article illustrated that the learner identity construction displayed different patterns in online and offline sessions, with some participants demonstrating consistency and others revealing changes. This finding further proved that individual's learner identity was constantly confirmed and reconstructed through LIPs.

In the end, the article provided implications for course design, pedagogical practice, and materials development in blended learning context, appealing that a careful course design and teacher's active involvement or guidance are essential for maintaining learners' positive learner identities and improving learning outcomes.

3. Discussion

In this part, we would like to make comments on the selection of participants, data analysis and research findings, aiming to examine the validity and reliability of the article for the sake of facilitating further research in this field. In particular, by discussing the group size in the article, we attempt to arouse researchers' attention to the intention, representativeness as well as transparency of the selection of participants. In addition, comments on the data analysis intend to raise researchers' concern about the importance of analysis unit, data analysis triangulation as well as transparency of analysis process. Finally, by comparing the findings in the article with those in previous researches, we appeal that the first priority should be given to the selection of theoretical framework.

3.1. Selection of participants

The article picked a focal group with six participants demonstrating diverse engagement in learning activities based on an initial four-week observation. Obviously, the article selected the focal group intentionally, attempting to explore the relationship between learners' interaction and learner identity construction. Nonetheless, one thing that puzzles us is the size of group, since the article did not explicate the reason why the group consisted of six participants. In our opinion, it would be better to make an explanation about it, as many studies have revealed that the size of group has a tremendous impact on participants' involvement in group discussion ( Mishra, 2016 ) and interaction in collaborative writing ( Arnold et al., 2012 ; Dobao and Blum, 2013 ; Dobao, 2014 ). Therefore, we argue that future study should include groups of different sizes to make a comparison in order to have a deep and detailed understanding of the impact of learners' interaction on learner identity. As Tenny et al. (2022) indicate the more representative the sample is to the expected research population, the more likely the researcher will take various factors at play into consideration. We believe that the selection of participants should be purposeful ( Sargeant, 2012 ) to saturate the data, and at the same time the details and processes involved in the selection should be elaborated for the purpose of transparency ( Oun and Bach, 2014 ).

3.2. Data analysis

As far as we are concerned, the data in the article seemed to be clearly classified into online and offline categories and analyzed inductively and deductively with peer debriefing adopted to ensure the trustworthiness of the data analysis. However, after taking a close look at the analysis process of learner identity in online collaborative writing, we perceive some problems regarding the selection of analysis unit, reliability of analysis framework and transparency of the analysis process. First, drawing on the work load roles put forward by Arnold et al. (2012) , the article utilized revision frequency as an analysis unit to identify learner identity without considering the type of revision, such as formal or meaning-based one. In reality, Arnold et al. (2012) discovered that learner identity transferred when different types of revision were taken into account, for learners made different efforts to revise different types of errors or problems based on their own perceived advantages and limitations. Therefore, we suggest that the type of revision should be a better choice for being an analysis unit, as data analysis aims to describe a phenomenon in detail in qualitative study ( Flick, 2014 ).

In addition, we are skeptical about the reliability of the analysis framework adopted in the article as we discover that the criterion used to examine work load roles in the article is inconsistent with that in Arnold et al. (2012) . Specifically, work load roles were judged by the workload of revision in Arnold et al.'s study, while it is determined by the frequency of revision in our commented article. As addressed by the authors themselves, the frequency of revision served as a proxy of students' online involvement ( Chen et al., 2022 , p. 6), which, in our opinion, is not equivalent to actual contributions of revision. Given the divergence mentioned above, we suggest that another analytic method be combined for researchers to eliminate bias and seek convergence among a variety of data to build up themes or categories ( Golafshani, 2003 ). In other words, it is suggested that triangulation of data analysis methods be employed, if necessary, to make up for the weakness of a single technique and enhance the interpretation and reliability of research findings ( Thurmond, 2001 ).

Finally, we find no clue as to how the learner identities are verified from the two rounds of semi-structured interviews for there is no description about the process of data analysis in this regard. In fact, the elaboration of analysis process is indispensable because data do not speak for themselves, it is the analyses and interpretations on the part of researchers that yield descriptive and causal inferences in qualitative study ( Moravcsik, 2020 ).

Given the major influence of data analysis on research findings ( Flick, 2014 ), we appeal that future research should give weight to the selection of analysis unit in that an appropriate unit is conducive to locate the data relevant to research questions ( Mezmir, 2020 ). In addition, researchers should make sure that data analysis method selected is suitable for the study. If necessary, another analysis method can be combined for triangulation ( Leech and Onwuegbuzie, 2007 ). Finally, the process of data analysis needs to be transparent and trustworthy ( O'Kane et al., 2021 ).

3.3. Research findings

With regard to the research findings of the article, on the one hand, the factors that influenced learners' engagement and interaction in the online collaborative writing sessions are distinct from those in previous researches. When we examine the findings carefully, we find that only the individual (learners' English ability, character and perception) and contextual factors (assigned roles and teacher's involvement) are disclosed. However, previous studies have found that various factors may affect learners' engagement and interaction, such as the genre of writing ( Reed et al., 1985 ), the type of task ( Li and Zhu, 2017 ) and computer-mediated contexts ( Wang, 2019 ). Moreover, different communication modes, for instance, using online conference or online editing software will elicit changes in learners' engagement and interaction in collaborative writing ( Aubrey, 2022 ). Based on the description in the article and our investigation of writing platform, we figure out that the very limited interaction in online collaborative writing can partly attribute to the communication mode of platform, on which participants communicate mainly through text messages asynchronously. As mentioned by one participant in interview, it was difficult for group members to communicate on the internet platform, and it was challenging for new revisers to comprehend the previous one's intention (see Chen et al., 2022 , p. 10). Therefore, future research could incorporate above-mentioned factors into online course design. This suggestion also echoes the finding that lack of a well-organized online learning session resulted in the transformation of a positive LIP into a negative one in blended learning (see Chen et al., 2022 , p. 11).

On the other hand, no macro-level factors related to learner identity construction were discovered. In fact, many macro-level factors, such as race and culture ( Kubota and Lin, 2009 ), societal power relations ( Norton, 2013 ), educational policy ( Hajar, 2017 ) have a significant influence on learner identity. We assume that the inconsistency of influencing factors can be ascribed to different focuses and organizations guided by theoretical frameworks adopted. It is obvious that the theoretical framework employed in the article mainly focused on learner identity in particular learning context. To the best of our knowledge, as a social being, learner identity is not only contextual constructed but also historically, culturally and politically situated ( Pavlenko and Blackledge, 2004 ). Therefore, an alternative, for instance, the framework proposed by Norton (1997) could be adopted to examine both micro and macro influencing factors. In fact, the selection of theoretical framework is vital because it is the base for the construction of knowledge ( Osanloo and Grant, 2016 ) and provides an anchor for analysis and interpretation of data as well as research findings ( Merriam and Tisdell, 2016 ). As a result, we argue that researchers should give their first priority to the selection of theoretical framework for it not only determines the focus, organization, exposure and hiding of meaning in the study, but also relates the study to previous scholarship and concept ( Collins and Stockton, 2018 ).

4. Conclusion

In summary, the article is a thought-provoking, well-explored, and illuminative piece. Firstly, it expounds the interplay of learner identity construction and learners' social interaction. In addition, the process of collaborative learning activities rather than the static writing product is examined. Finally, the patterns of learner identity construction across different learning sessions are revealed. All in all, the article provides readers with new insights into the complexity of learner identify and variety of influencing factors. We believe that, after reading the article, course administrators, teachers and students can have a better understanding of learner identity and factors that hinder or facilitate positive learner identity construction in blended learning context, which prompts them to make corresponding adjustments in their course design, teaching or learning respectively. Hence, we would like to recommend the article without any hesitation to more readers, particularly those who are keen on learner identity in blended learning.

Author contributions

XL and XZ selected the commented article together. XL drafted the opinion. XZ provided insights and valuable suggestions during her writing and helped revise the text. All authors contributed to the article and approved the submitted version.

This study was funded by College English Teaching and Research Projects of Foreign Language Teaching and Research Press, China (Grant No. KH200249A).

Acknowledgments

We gratefully acknowledge the research article of Jing Chen, Jie Tan, and Jun Lei, which provides us with a valuable source to write this commented article.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Ahmad, S. Z. (2020). Cloud-based collaborative writing to develop EFL students' writing quantity and quality. Int. Educ. Stud. 13, 51–64. doi: 10.5539/ies.v13n3p51

CrossRef Full Text | Google Scholar

Anggraini, R., Rozimela, Y., and Anwar, D. (2020). The effects of collaborative writing on EFL learners' writing skills and their perception of the strategy. J. Lang. Teach. Res. 11, 335–341. doi: 10.17507/jltr.1102.25

Arnold, N., Ducate, L., and Kost, C. (2012). Collaboration or cooperation? Analyzing group dynamics and revision processes in wikis. Calico J. 29, 431–448. doi: 10.11139/cj.29.3.431-448

Aubrey, S. (2022). Dynamic engagement in second language computer-mediated collaborative writing tasks: does communication mode matter? Stud. Second Lang. Learn. Teach. 12, 59–86. doi: 10.14746/ssllt.2022.12.1.4

Bell, E., Bryman, A., and Harley, B. (2022). Business Research Methods . Oxford: Oxford University Press.

Google Scholar

Brown, R., and Pehrson, S. (2019). Group Processes: Dynamics Within and Between Groups . New York, NY: John Wiley and Sons.

Burke, P. J. (1980). The self: Measurement requirements from an interactionist perspective. Soc. Psychol. Q. 1, 18–29. doi: 10.2307/3033745

Charles, M., and Pecorari, D. (2022). Introducing English for Academic Purposes. Beijing: Foreign Language Teaching and Research Press.

Chen, J., Tan, J., and Lei, J. (2022). Exploring learner identity in the blended learning context: a case study of collaborative writing. System 108, 102841. doi: 10.1016/j.system.2022.102841

Chen, J., Chen, J. Y., and Guo, K. (2021). Investigating student engagement in a blended academic English writing classroom. J. Foreign Lang. World 202, 28–36.

Coll Salvador, C., and Falsafi, L. (2010). Learner identity: An educational and analytical tool. Rev Educ. 353, 211–233.

Collins, C. S., and Stockton, C. M. (2018). The central role of theory in qualitative research. Int. J. Q. Methods 17, 1609406918797475. doi: 10.1177/1609406918797475

Delahunty, J., Verenikina, I., and Jones, P. (2014). Socio-emotional connections: identity, belonging and learning in online interactions. A literature review. Technol. Pedag. Educ. 23, 243–265. doi: 10.1080/1475939X.2013.813405

Dobao, A. F. (2014). Attention to form in collaborative writing tasks: comparing pair and small group interaction. Can. Modern Lang. Rev. 70, 158–187. doi: 10.3138/cmlr.1768

Dobao, A. F., and Blum, A. (2013). Collaborative writing in pairs and small groups: learners' attitudes and perceptions. System 41, 365–378. doi: 10.1016/j.system.2013.02.002

Douglas, S. R., and Rosvold, M. (2018). Intercultural communicative competence and English for academic purposes: a synthesis review of the scholarly literature. Can. J. Appl. Ling. 21, 23–42. doi: 10.7202/1050809ar

Etherington, S. (2006). Becoming a different kind of learner: a case study of a Japanese EAP learner. The East Asian Learner 2, 1–10.

Flick, U. (2014). Mapping the field. Handb. Q. Data Anal. 1, 3–18. doi: 10.4135/9781446282243.n1

Flórez González, A. (2018). Strengthening local identity by writing chronicles in the EFL classroom. Colombian Appl. Ling. J. 20, 195–208. doi: 10.14483/22487085.13121

Ginns, P., and Ellis, R. (2007). Quality in blended learning: Exploring the relationships between on-line and face-to-face teaching and learning. Internet Higher Educ. 10, 53–64. doi: 10.1016/j.iheduc.2006.10.003

Golafshani, N. (2003). Understanding reliability and validity in qualitative research. Q. Rep. 8, 597–607.

Hajar, A. (2017). Identity, investment and language learning strategies of two Syrian students in Syria and Britain. Lang. Cult. Curric. 30, 250–264. doi: 10.1080/07908318.2017.1317266

Heller, M. (1984). Language and ethnic identity in a Toronto French language school. Can. Ethnic Stud. 16, 1.

Hyland, K., and Hamp-Lyons, L. (2002). EAP: issues and directions. J. Eng. Acad. Purposes 1, 1–12. doi: 10.1016/S1475-1585(02)00002-4

Jund, A. (2010). Toward a pedagogy of intercultural understanding in teaching English for academic purposes. Electr. J. Eng. Second Lang. 14, 1–13.

Kim, Y. (2008). The contribution of collaborative and individual tasks to the acquisition of L2 vocabulary. Modern Lang. J. 92, 114–130. doi: 10.1111/j.1540-4781.2008.00690.x

King, B. W. (2015). Wikipedia writing as praxis: computer-mediated socialization of second-language writers. Lang. Learn. Technol. 19, 106–123.

Kubota, R., and Lin, A. M. (2009). Race, Culture, and Identities in Second Language Education: Exploring Critically Engaged Practice . New York, NY: Routledge.

Kwon, S., Kim, W., Bae, C., Cho, M., Lee, S., Dreamson, N., et al. (2021). The identity changes in online learning and teaching: instructors, learners, and learning management systems. Int. J. Educ. Technol. Higher Educ. 18, 1–18. doi: 10.1186/s41239-021-00304-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Lan, R., and Lan, C. (2022). Group learning and university students' EFL learner identity construction: a multiple case study in China. Open J. Modern Ling. 12, 399–413. doi: 10.4236/ojml.2022.124028

Lave, J., and Wenger, E. (1991). Situated Learning: Legitimate Peripheral Participation . Cambridge: Cambridge University Press.

Leech, N. L., and Onwuegbuzie, A. J. (2007). An array of qualitative data analysis tools: a call for data analysis triangulation. School Psychol. Q. 22, 557. doi: 10.1037/1045-3830.22.4.557

Li, Y. (2020). Educational Change Amongst English Language College Teachers in China: Transitioning From Teaching for General to Academic Purposes . Berlin: Springer Nature .

Li, M., and Zhu, W. (2017). Explaining dynamic interactions in wiki-based collaborative writing. Lang. Learn. Technol. 21, 96–120.

Maswana, S., and Yamada, H. (2021). Vocabulary and move analysis of high school essays from an EAP writing perspective. J. Asia TEFL 18, 891–903. doi: 10.18823/asiatefl.2021.18.3.9.891

Mayrberger, K., and Linke, F. (2014). “The relevance of participatory experience—a German perspective on participatory learning with social media at school”. in Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications , eds T. Amiel, and B. Wilson. (Chesapeake, VA: AACE) , 2179–2187.

Merriam, S. B., and Tisdell, E. J. (2016). Qualitative Research: A Guide to Design and Information . New York, NY: Wiley and Sons.

Mezmir, E. A. (2020). Qualitative data analysis: an overview of data reduction, data display, and interpretation. Res. Hum. Soc. Sci. 10, 15–27. doi: 10.7176/RHSS/10-21-02

Mishra, L. (2016). Focus group discussion in qualitative research. Techno Learn 6, 1. doi: 10.5958/2249-5223.2016.00001.2

Moravcsik, A. (2020). Transparency in Qualitative Research . London: SAGE Publications Limited.

Nasrollahi Shahri, M. N. (2018). Constructing a voice in english as a foreign language: identity and engagement. TESOL Q. 52, 85–109. doi: 10.1002/tesq.373

Norton, B. (1997). Language, identity, and the ownership of English. TESOL Q. 31, 409–429. doi: 10.2307/3587831

Norton, B. (2013). Identity and Language Learning Identity and Language Learning . New York, NY: Multilingual Matters.

Norton, B., and Toohey, K. (2011). Identity, language learning, and social change. Lang. Teach. 44, 412–446. doi: 10.1017/S0261444811000309

O'Kane, P., Smith, A., and Lerman, M. P. (2021). Building transparency and trustworthiness in inductive research through computer-aided qualitative data analysis software. Org. Res. Methods 24, 104–139. doi: 10.1177/1094428119865016

Osanloo, A., and Grant, C. (2016). Understanding, selecting, and integrating a theoretical framework in dissertation research: Creating the blueprint for your “house”. Admin. J. Educ. Prac. Res. 4, 12–26. doi: 10.5929/2014.4.2.9

Oun, M. A., and Bach, C. (2014). Qualitative research method summary. J. Multidis. Eng. Sci. Technol. 1, 252–258.

Pavlenko, A., and Blackledge, A. (2004). Negotiation of Identities in Multilingual Contexts . Clevedon: Multilingual Matters.

PubMed Abstract | Google Scholar

Poupore, G. (2016). Measuring group work dynamics and its relation with L2 learners' task motivation and language production. Lang. Teach. Res. 20, 719–740. doi: 10.1177/1362168815606162

Reed, W. M., Burton, J. K., and Kelly, P. P. (1985). The effects of writing ability and mode of discourse on cognitive capacity engagement. Res. Teach. Eng. 1, 283–297.

Sampson, E. E. (1978). Personality and the location of identity 1. J. Pers. 46, 552–568. doi: 10.1111/j.1467-6494.1978.tb01017.x

Sargeant, J. (2012). Qualitative research part II: participants, analysis, and quality assurance. J. Grad. Med. Educ. 4, 1–3. doi: 10.4300/JGME-D-11-00307.1

Scotton, C. M. (1980). Explaining linguistic choices as identity negotiations. Lang. Pergam. 1, 359–366. doi: 10.1016/B978-0-08-024696-3.50061-2

Shehadeh, A. (2011). Effects and student perceptions of collaborative writing in L2. J. Sec. Lang. Writing 20, 286–305. doi: 10.1016/j.jslw.2011.05.010

Sinha, C. (1999). “Situated selves: Learning to be a learner”. In Technological Resources for Learning , eds J. Bliss, R. Sãljõ, and P. Light (Oxford: Pergamon) , 32–48.

Storch, N. (2005). Collaborative writing: product, process, and students' reflections. J. Sec. Lang. Writing 14, 153–173. doi: 10.1016/j.jslw.2005.05.002

Tenny, S., Brannan, J. M., and Brannan, G. D. (2022). “Qualitative study”, in StatPearls . Treasure Island, FL: StatPearls Publishing. Available online at: https://europepmc.org/article/NBK/nbk470395

Thurmond, V. A. (2001). The point of triangulation. J. Nurs. Scholarship 33, 253–258. doi: 10.1111/j.1547-5069.2001.00253.x

Toohey, K. (2000). Learning English at School: Identity, Social Relations and Classroom Practice . Clevedon: Multilingual Matters.

Trent, J. (2008). Promoting investment by Chinese learners in classroom discourse: integrating content and language in the undergraduate classroom. J. Asian Pacific Commun. 18, 30–48. doi: 10.1075/japc.18.1.03tre

Wang, L. (2019). The impact of computer-mediated contexts on interaction pattern of ESL learners in collaborative writing. Technol. Pedag. Educ. 28, 547–562. doi: 10.1080/1475939X.2019.1674183

Wearmouth, J., Berryman, M., and Whittle, L. (2011). ‘Shoot for the moon!' Students' identities as writers in the context of the classroom. Br. J. Special Educ. 38, 92–99. doi: 10.1111/j.1467-8578.2011.00502.x

Wigglesworth, G., and Storch, N. (2009). Pair versus individual writing: effects on fluency, complexity and accuracy. Lang. Testing 26, 445–466. doi: 10.1177/0265532209104670

Xu, G., and Kim, J. (2022). Building and sustaining a group of Chinese EFL learners' imagined identities and agency. Sustainability 14, 4659. doi: 10.3390/su14084659

Keywords: learner identity, blended learning, online collaborative writing, research design, validity and reliability

Citation: Lian X and Zheng X (2023) How to construct an appropriate research design for the study of learner identity in blended learning? Front. Psychol. 14:1126605. doi: 10.3389/fpsyg.2023.1126605

Received: 18 December 2022; Accepted: 03 January 2023; Published: 19 January 2023.

Reviewed by:

Copyright © 2023 Lian and Zheng. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY) . The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.

† ORCID: Xiuping Lian orcid.org/0000-0001-9879-1098 Xinmin Zheng orcid.org/0000-0003-0989-9297

Disclaimer: All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article or claim that may be made by its manufacturer is not guaranteed or endorsed by the publisher.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed, evidence based medicine, affiliations.

• 1 University of Nebraska Medical Center
• 2 Penn Highlands Healthcare System
• PMID: 29262040
• Bookshelf ID: NBK470182

Evidence-based medicine (EBM) uses the scientific method to organize and apply current data to improve healthcare decisions. Thus, the best available science is combined with the healthcare professional's clinical experience and the patient's values to arrive at the best medical decision for the patient. There are 5 main steps for applying EBM to clinical practice :

Copyright © 2024, StatPearls Publishing LLC.

PubMed Disclaimer

Conflict of interest statement

Disclosure: Steven Tenny declares no relevant financial relationships with ineligible companies.

Disclosure: Matthew Varacallo declares no relevant financial relationships with ineligible companies.

• Definition/Introduction
• Issues of Concern
• Clinical Significance
• Nursing, Allied Health, and Interprofessional Team Interventions
• Review Questions

Similar articles

• Applying evidence-based medicine in general practice: a video-stimulated interview study on workplace-based observation. Welink LS, Van Roy K, Damoiseaux RAMJ, Suijker HA, Pype P, de Groot E, Bartelink MEL. Welink LS, et al. BMC Fam Pract. 2020 Jan 8;21(1):5. doi: 10.1186/s12875-019-1073-x. BMC Fam Pract. 2020. PMID: 31914934 Free PMC article.
• [Evidence-based Risk and Benefit Communication for Shared Decision Making]. Nakayama T. Nakayama T. Yakugaku Zasshi. 2018;138(3):331-334. doi: 10.1248/yakushi.17-00185-5. Yakugaku Zasshi. 2018. PMID: 29503424 Review. Japanese.
• [Neuropediatrician and evidence-based medicine]. González de Dios J. González de Dios J. Rev Neurol. 2008;47 Suppl 1:S75-95. Rev Neurol. 2008. PMID: 18767019 Spanish.
• Evidence-based toxicology: a comprehensive framework for causation. Guzelian PS, Victoroff MS, Halmes NC, James RC, Guzelian CP. Guzelian PS, et al. Hum Exp Toxicol. 2005 Apr;24(4):161-201. doi: 10.1191/0960327105ht517oa. Hum Exp Toxicol. 2005. PMID: 15957536 Review.
• [Evidence-based medicine: reality and illusions. Extension of epistemological reflexions]. Timio M, Antiseri D. Timio M, et al. Ital Heart J Suppl. 2000 Mar;1(3):411-4. Ital Heart J Suppl. 2000. PMID: 10815272 Italian.
• Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. 1996. Clin Orthop Relat Res. 2007 Feb;455:3-5. - PubMed
• Sackett DL, Straus SE. Finding and applying evidence during clinical rounds: the "evidence cart". JAMA. 1998 Oct 21;280(15):1336-8. - PubMed
• Sackett DL. Evidence-based medicine. Spine (Phila Pa 1976) 1998 May 15;23(10):1085-6. - PubMed
• Steves R, Hootman JM. Evidence-Based Medicine: What Is It and How Does It Apply to Athletic Training? J Athl Train. 2004 Mar;39(1):83-87. - PMC - PubMed
• Fernandez A, Sturmberg J, Lukersmith S, Madden R, Torkfar G, Colagiuri R, Salvador-Carulla L. Evidence-based medicine: is it a bridge too far? Health Res Policy Syst. 2015 Nov 06;13:66. - PMC - PubMed

Publication types

• Search in PubMed
• Search in MeSH
• Add to Search

Related information

• Cited in Books

LinkOut - more resources

Full text sources.

• NCBI Bookshelf

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

MASC LibGuide Resources

• Qualitative Research
• Case Studies
• SWOT Analysis
• Writing Standards for Graduate Level Education

Qualitative Research Resources

Articles on the definition and validity of qualitative research. .

• Chapter 1: Introduction. In Introduction to Qualitative Research Methods Hurst, A. (2023). Chapter 1: Introduction. In Introduction to Qualitative Research Methods . Oregon State University Pressbooks.
• Chapter 21: Conclusion. The Value of Qualitative Research Hurst, A. (2023). Chapter 21: Conclusion. The Value of Qualitative Research. In Introduction to Qualitative Research Methods . Oregon State University Pressbooks
• Qualitative Study Tenny, S., Brannan, J., & Brannan, G. (Last updated September, 18 2022). Qualitative Study. StatPearls. National Library of Medicine Note: From the National Library of Medicine but is general enough that it can apply to all fields.

Videos on qualitative research. 

• Qualitative Paradigm Buckler, S. (Academic). (2015).  Qualitative paradigm  [Video]. Sage Knowledge. 
• An Introduction to Qualitative Research Laurie, C., & Jensen, E. (Academics). (2017).   An introduction to qualitative research  [Video]. Sage Research Methods.
• Valerie Janesick Discusses Qualitative Research, Reflection & Writing Janesick, V. (Academic). (2017).  Valerie Janesick discusses qualitative research, reflection & writing  [Video]. Sage Research Methods. 
• An Introduction to Qualitative Data Analysis Jensen, E., & Laurie, C. (Academics). (2017).  An introduction to qualitative data analysis  [Video]. Sage Research Methods. 
• << Previous: Home
• Next: Case Studies >>
• Last Updated: Jul 15, 2024 11:27 AM
• URL: https://resources.nu.edu/c.php?g=1409821

© Copyright 2024 National University. All Rights Reserved.

Privacy Policy | Consumer Information

• UConn Library
• Scientific Research and Communication
• Qualitative Research: What is it?

Scientific Research and Communication — Qualitative Research: What is it?

• Essential Resources
• The Scientific Method
• Types of Scientific Papers
• Organization of a Scientific Paper
• Peer Review & Academic Journals
• Primary and Secondary Sources
• Scientific Information Literacy
• Critical Reading Methods
• Scientific Writing Guidebooks
• Science Literature Reviews
• Searching Strategies for Science Databases
• Engineering Career Exploration
• Quantitative Research: What Is It?
• AI Tools for Research
• Avoiding Plagiarism

What is qualitative research?

"Qualitative research is a type of research that explores and provides deeper insights into real-world problems. [1]  Instead of collecting numerical data points or intervene or introduce treatments just like in quantitative research, qualitative research helps generate hypotheses as well as further investigate and understand quantitative data."

"Qualitative research at its core, ask open-ended questions whose answers are not easily put into numbers such as ‘how’ and ‘why’. [2]  Due to the open-ended nature of the research questions at hand, qualitative research design is often not linear in the same way quantitative design is. [2]  One of the strengths of qualitative research is its ability to explain processes and patterns of human behavior that can be difficult to quantify. [3]  Phenomena such as experiences, attitudes, and behaviors can be difficult to accurately capture quantitatively, whereas a qualitative approach allows participants themselves to explain how, why, or what they were thinking, feeling, and experiencing at a certain time or during an event of interest."

• Qualitative Study - Steven Tenny; Grace D. Brannan; Janelle M. Brannan; Nancy C. Sharts-Hopko. This article details what qualitative research is, and some of the methodologies used.

Examples of Qualitative Research

• Quantitative vs Qualitative Chart Chart showing examples of quantitative vs. qualitative research.

EBooks on Qualitative Research Methodology

Physical Library Books

• << Previous: Engineering Career Exploration
• Next: Quantitative Research: What Is It? >>
• Last Updated: Jul 1, 2024 2:12 PM
• URL: https://guides.lib.uconn.edu/sciencecommunication

• Physician Physician Board Reviews Physician Associate Board Reviews CME Lifetime CME Free CME MATE and DEA Compliance
• Student USMLE Step 1 USMLE Step 2 USMLE Step 3 COMLEX Level 1 COMLEX Level 2 COMLEX Level 3 96 Medical School Exams Student Resource Center NCLEX - RN NCLEX - LPN/LVN/PN 24 Nursing Exams
• Nurse Practitioner APRN/NP Board Reviews CNS Certification Reviews CE - Nurse Practitioner FREE CE
• Nurse RN Certification Reviews CE - Nurse FREE CE
• Pharmacist Pharmacy Board Exam Prep CE - Pharmacist
• Allied Allied Health Exam Prep Dentist Exams CE - Social Worker CE - Dentist
• Point of Care
• Free CME/CE

Qualitative Study

Introduction.

Qualitative research is a type of research that explores and provides deeper insights into real-world problems. [1] Instead of collecting numerical data points or intervening or introducing treatments just like in quantitative research, qualitative research helps generate hypothenar to further investigate and understand quantitative data. Qualitative research gathers participants' experiences, perceptions, and behavior. It answers the hows and whys instead of how many or how much. It could be structured as a standalone study, purely relying on qualitative data, or part of mixed-methods research that combines qualitative and quantitative data. This review introduces the readers to some basic concepts, definitions, terminology, and applications of qualitative research.

Qualitative research, at its core, asks open-ended questions whose answers are not easily put into numbers, such as "how" and "why." [2] Due to the open-ended nature of the research questions, qualitative research design is often not linear like quantitative design. [2] One of the strengths of qualitative research is its ability to explain processes and patterns of human behavior that can be difficult to quantify. [3] Phenomena such as experiences, attitudes, and behaviors can be complex to capture accurately and quantitatively. In contrast, a qualitative approach allows participants themselves to explain how, why, or what they were thinking, feeling, and experiencing at a particular time or during an event of interest. Quantifying qualitative data certainly is possible, but at its core, qualitative data is looking for themes and patterns that can be difficult to quantify, and it is essential to ensure that the context and narrative of qualitative work are not lost by trying to quantify something that is not meant to be quantified.

However, while qualitative research is sometimes placed in opposition to quantitative research, where they are necessarily opposites and therefore "compete" against each other and the philosophical paradigms associated with each other, qualitative and quantitative work are neither necessarily opposites, nor are they incompatible. [4] While qualitative and quantitative approaches are different, they are not necessarily opposites and certainly not mutually exclusive. For instance, qualitative research can help expand and deepen understanding of data or results obtained from quantitative analysis. For example, say a quantitative analysis has determined a correlation between length of stay and level of patient satisfaction, but why does this correlation exist? This dual-focus scenario shows one way in which qualitative and quantitative research could be integrated.

Qualitative Research Approaches

Ethnography

Ethnography as a research design originates in social and cultural anthropology and involves the researcher being directly immersed in the participant’s environment. [2] Through this immersion, the ethnographer can use a variety of data collection techniques to produce a comprehensive account of the social phenomena that occurred during the research period. [2] That is to say, the researcher’s aim with ethnography is to immerse themselves into the research population and come out of it with accounts of actions, behaviors, events, etc, through the eyes of someone involved in the population. Direct involvement of the researcher with the target population is one benefit of ethnographic research because it can then be possible to find data that is otherwise very difficult to extract and record.

Grounded theory

Grounded Theory is the "generation of a theoretical model through the experience of observing a study population and developing a comparative analysis of their speech and behavior." [5] Unlike quantitative research, which is deductive and tests or verifies an existing theory, grounded theory research is inductive and, therefore, lends itself to research aimed at social interactions or experiences. [3] [2] In essence, Grounded Theory’s goal is to explain how and why an event occurs or how and why people might behave a certain way. Through observing the population, a researcher using the Grounded Theory approach can then develop a theory to explain the phenomena of interest.

Phenomenology

Phenomenology is the "study of the meaning of phenomena or the study of the particular.” [5] At first glance, it might seem that Grounded Theory and Phenomenology are pretty similar, but the differences can be seen upon careful examination. At its core, phenomenology looks to investigate experiences from the individual's perspective. [2] Phenomenology is essentially looking into the "lived experiences" of the participants and aims to examine how and why participants behaved a certain way from their perspective. Herein lies one of the main differences between Grounded Theory and Phenomenology. Grounded Theory aims to develop a theory for social phenomena through an examination of various data sources. In contrast, Phenomenology focuses on describing and explaining an event or phenomenon from the perspective of those who have experienced it.

Narrative research

One of qualitative research’s strengths lies in its ability to tell a story, often from the perspective of those directly involved in it. Reporting on qualitative research involves including details and descriptions of the setting involved and quotes from participants. This detail is called a "thick" or "rich" description and is a strength of qualitative research. Narrative research is rife with the possibilities of "thick" description as this approach weaves together a sequence of events, usually from just one or two individuals, hoping to create a cohesive story or narrative. [2] While it might seem like a waste of time to focus on such a specific, individual level, understanding one or two people’s narratives for an event or phenomenon can help to inform researchers about the influences that helped shape that narrative. The tension or conflict of differing narratives can be "opportunities for innovation." [2]

Research Paradigm

Research paradigms are the assumptions, norms, and standards underpinning different research approaches. Essentially, research paradigms are the "worldviews" that inform research. [4] It is valuable for qualitative and quantitative researchers to understand what paradigm they are working within because understanding the theoretical basis of research paradigms allows researchers to understand the strengths and weaknesses of the approach being used and adjust accordingly. Different paradigms have different ontologies and epistemologies. Ontology is defined as the "assumptions about the nature of reality,” whereas epistemology is defined as the "assumptions about the nature of knowledge" that inform researchers' work. [2] It is essential to understand the ontological and epistemological foundations of the research paradigm researchers are working within to allow for a complete understanding of the approach being used and the assumptions that underpin the approach as a whole. Further, researchers must understand their own ontological and epistemological assumptions about the world in general because their assumptions about the world will necessarily impact how they interact with research. A discussion of the research paradigm is not complete without describing positivist, postpositivist, and constructivist philosophies.

Positivist versus postpositivist

To further understand qualitative research, we must discuss positivist and postpositivist frameworks. Positivism is a philosophy that the scientific method can and should be applied to social and natural sciences. [4] Essentially, positivist thinking insists that the social sciences should use natural science methods in their research. It stems from positivist ontology, that there is an objective reality that exists that is wholly independent of our perception of the world as individuals. Quantitative research is rooted in positivist philosophy, which can be seen in the value it places on concepts such as causality, generalizability, and replicability.

Conversely, postpositivists argue that social reality can never be one hundred percent explained, but could be approximated. [4] Indeed, qualitative researchers have been insisting that there are “fundamental limits to the extent to which the methods and procedures of the natural sciences could be applied to the social world,” and therefore, postpositivist philosophy is often associated with qualitative research. [4] An example of positivist versus postpositivist values in research might be that positivist philosophies value hypothesis-testing, whereas postpositivist philosophies value the ability to formulate a substantive theory.

Constructivist

Constructivism is a subcategory of postpositivism. Most researchers invested in postpositivist research are also constructivist, meaning they think there is no objective external reality that exists but instead that reality is constructed. Constructivism is a theoretical lens that emphasizes the dynamic nature of our world. "Constructivism contends that individuals' views are directly influenced by their experiences, and it is these individual experiences and views that shape their perspective of reality.” [6]  constructivist thought focuses on how "reality" is not a fixed certainty and how experiences, interactions, and backgrounds give people a unique view of the world. Constructivism contends, unlike positivist views, that there is not necessarily an "objective"reality we all experience. This is the ‘relativist’ ontological view that reality and our world are dynamic and socially constructed. Therefore, qualitative scientific knowledge can be inductive as well as deductive.” [4]

So why is it important to understand the differences in assumptions that different philosophies and approaches to research have? Fundamentally, the assumptions underpinning the research tools a researcher selects provide an overall base for the assumptions the rest of the research will have. It can even change the role of the researchers. [2] For example, is the researcher an "objective" observer, such as in positivist quantitative work? Or is the researcher an active participant in the research, as in postpositivist qualitative work? Understanding the philosophical base of the study undertaken allows researchers to fully understand the implications of their work and their role within the research and reflect on their positionality and bias as it pertains to the research they are conducting.

Data Sampling 

The better the sample represents the intended study population, the more likely the researcher is to encompass the varying factors. The following are examples of participant sampling and selection: [7]

• Purposive sampling- selection based on the researcher’s rationale for being the most informative.
• Criterion sampling selection based on pre-identified factors.
• Convenience sampling- selection based on availability.
• Snowball sampling- the selection is by referral from other participants or people who know potential participants.
• Extreme case sampling- targeted selection of rare cases.
• Typical case sampling selection based on regular or average participants. 

Data Collection and Analysis

Qualitative research uses several techniques, including interviews, focus groups, and observation. [1] [2] [3] Interviews may be unstructured, with open-ended questions on a topic, and the interviewer adapts to the responses. Structured interviews have a predetermined number of questions that every participant is asked. It is usually one-on-one and appropriate for sensitive topics or topics needing an in-depth exploration. Focus groups are often held with 8-12 target participants and are used when group dynamics and collective views on a topic are desired. Researchers can be participant-observers to share the experiences of the subject or non-participants or detached observers.

While quantitative research design prescribes a controlled environment for data collection, qualitative data collection may be in a central location or the participants' environment, depending on the study goals and design. Qualitative research could amount to a large amount of data. Data is transcribed, which may then be coded manually or using computer-assisted qualitative data analysis software or CAQDAS such as ATLAS.ti or NVivo. [8] [9] [10]

After the coding process, qualitative research results could be in various formats. It could be a synthesis and interpretation presented with excerpts from the data. [11] Results could also be in the form of themes and theory or model development.

Dissemination

The healthcare team can use two reporting standards to standardize and facilitate the dissemination of qualitative research outcomes. The Consolidated Criteria for Reporting Qualitative Research or COREQ is a 32-item checklist for interviews and focus groups. [12] The Standards for Reporting Qualitative Research (SRQR) is a checklist covering a more comprehensive range of qualitative research. [13]

Applications

Many times, a research question will start with qualitative research. The qualitative research will help generate the research hypothesis, which can be tested with quantitative methods. After the data is collected and analyzed with quantitative methods, a set of qualitative methods can be used to dive deeper into the data to better understand what the numbers truly mean and their implications. The qualitative techniques can then help clarify the quantitative data and also help refine the hypothesis for future research. Furthermore, with qualitative research, researchers can explore poorly studied subjects with quantitative methods. These include opinions, individual actions, and social science research.

An excellent qualitative study design starts with a goal or objective. This should be clearly defined or stated. The target population needs to be specified. A method for obtaining information from the study population must be carefully detailed to ensure no omissions of part of the target population. A proper collection method should be selected that will help obtain the desired information without overly limiting the collected data because, often, the information sought is not well categorized or obtained. Finally, the design should ensure adequate methods for analyzing the data. An example may help better clarify some of the various aspects of qualitative research.

A researcher wants to decrease the number of teenagers who smoke in their community. The researcher could begin by asking current teen smokers why they started smoking through structured or unstructured interviews (qualitative research). The researcher can also get together a group of current teenage smokers and conduct a focus group to help brainstorm factors that may have prevented them from starting to smoke (qualitative research).

In this example, the researcher has used qualitative research methods (interviews and focus groups) to generate a list of ideas of why teens start to smoke and factors that may have prevented them from starting to smoke. Next, the researcher compiles this data. The research found that, hypothetically, peer pressure, health issues, cost, being considered "cool," and rebellious behavior all might increase or decrease the likelihood of teens starting to smoke.

The researcher creates a survey asking teen participants to rank how important each of the above factors is in either starting smoking (for current smokers) or not smoking (for current nonsmokers). This survey provides specific numbers (ranked importance of each factor) and is thus a quantitative research tool.

The researcher can use the survey results to focus efforts on the one or two highest-ranked factors. Let us say the researcher found that health was the primary factor that keeps teens from starting to smoke, and peer pressure was the primary factor that contributed to teens starting smoking. The researcher can go back to qualitative research methods to dive deeper into these for more information. The researcher wants to focus on keeping teens from starting to smoke, so they focus on the peer pressure aspect.

The researcher can conduct interviews and focus groups (qualitative research) about what types and forms of peer pressure are commonly encountered, where the peer pressure comes from, and where smoking starts. The researcher hypothetically finds that peer pressure often occurs after school at the local teen hangouts, mostly in the local park. The researcher also hypothetically finds that peer pressure comes from older, current smokers who provide the cigarettes.

The researcher could further explore this observation made at the local teen hangouts (qualitative research) and take notes regarding who is smoking, who is not, and what observable factors are at play for peer pressure to smoke. The researcher finds a local park where many local teenagers hang out and sees that the smokers tend to hang out in a shady, overgrown area of the park. The researcher notes that smoking teenagers buy their cigarettes from a local convenience store adjacent to the park, where the clerk does not check identification before selling cigarettes. These observations fall under qualitative research.

If the researcher returns to the park and counts how many individuals smoke in each region, this numerical data would be quantitative research. Based on the researcher's efforts thus far, they conclude that local teen smoking and teenagers who start to smoke may decrease if there are fewer overgrown areas of the park and the local convenience store does not sell cigarettes to underage individuals.

The researcher could try to have the parks department reassess the shady areas to make them less conducive to smokers or identify how to limit the sales of cigarettes to underage individuals by the convenience store. The researcher would then cycle back to qualitative methods of asking at-risk populations their perceptions of the changes and what factors are still at play, and quantitative research that includes teen smoking rates in the community and the incidence of new teen smokers, among others. [14] [15]

Register For Free And Read The Full Article

Learn more about a subscription to statpearls point-of-care.

Qualitative research functions as a standalone research design or combined with quantitative research to enhance our understanding of the world. Qualitative research uses techniques including structured and unstructured interviews, focus groups, and participant observation not only to help generate hypotheses that can be more rigorously tested with quantitative research but also to help researchers delve deeper into the quantitative research numbers, understand what they mean, and understand what the implications are. Qualitative research allows researchers to understand what is going on, especially when things are not easily categorized. [16]

Issues of Concern

As discussed in the sections above, quantitative and qualitative work differ in many ways, including the evaluation criteria. There are four well-established criteria for evaluating quantitative data: internal validity, external validity, reliability, and objectivity. Credibility, transferability, dependability, and confirmability are the correlating concepts in qualitative research. [4] [11] The corresponding quantitative and qualitative concepts can be seen below, with the quantitative concept on the left and the qualitative concept on the right:

• Internal validity: Credibility
• External validity: Transferability
• Reliability: Dependability
• Objectivity: Confirmability

In conducting qualitative research, ensuring these concepts are satisfied and well thought out can mitigate potential issues from arising. For example, just as a researcher will ensure that their quantitative study is internally valid, qualitative researchers should ensure that their work has credibility. 

Indicators such as triangulation and peer examination can help evaluate the credibility of qualitative work.

• Triangulation: Triangulation involves using multiple data collection methods to increase the likelihood of getting a reliable and accurate result. In our above magic example, the result would be more reliable if we interviewed the magician, backstage hand, and the person who "vanished." In qualitative research, triangulation can include telephone surveys, in-person surveys, focus groups, and interviews and surveying an adequate cross-section of the target demographic.
• Peer examination: A peer can review results to ensure the data is consistent with the findings.

A "thick" or "rich" description can be used to evaluate the transferability of qualitative research, whereas an indicator such as an audit trail might help evaluate the dependability and confirmability.

• Thick or rich description:  This is a detailed and thorough description of details, the setting, and quotes from participants in the research. [5] Thick descriptions will include a detailed explanation of how the study was conducted. Thick descriptions are detailed enough to allow readers to draw conclusions and interpret the data, which can help with transferability and replicability.
• Audit trail: An audit trail provides a documented set of steps of how the participants were selected and the data was collected. The original information records should also be kept (eg, surveys, notes, recordings).

One issue of concern that qualitative researchers should consider is observation bias. Here are a few examples:

• Hawthorne effect: The effect is the change in participant behavior when they know they are being observed. Suppose a researcher wanted to identify factors that contribute to employee theft and tell the employees they will watch them to see what factors affect employee theft. In that case, one would suspect employee behavior would change when they know they are being protected.
• Observer-expectancy effect: Some participants change their behavior or responses to satisfy the researcher's desired effect. This happens unconsciously for the participant, so it is essential to eliminate or limit the transmission of the researcher's views.
• Artificial scenario effect: Some qualitative research occurs in contrived scenarios with preset goals. In such situations, the information may not be accurate because of the artificial nature of the scenario. The preset goals may limit the qualitative information obtained.

Clinical Significance

Qualitative or quantitative research helps healthcare providers understand patients and the impact and challenges of the care they deliver. Qualitative research provides an opportunity to generate and refine hypotheses and delve deeper into the data generated by quantitative research. Qualitative research is not an island apart from quantitative research but an integral part of research methods to understand the world around us. [17]

Enhancing Healthcare Team Outcomes

Qualitative research is essential for all healthcare team members as all are affected by qualitative research. Qualitative research may help develop a theory or a model for health research that can be further explored by quantitative research. Much of the qualitative research data acquisition is completed by numerous team members, including social workers, scientists, nurses, etc. Within each area of the medical field, there is copious ongoing qualitative research, including physician-patient interactions, nursing-patient interactions, patient-environment interactions, healthcare team function, patient information delivery, etc. 

Moser A, Korstjens I. Series: Practical guidance to qualitative research. Part 1: Introduction. The European journal of general practice. 2017 Dec:23(1):271-273. doi: 10.1080/13814788.2017.1375093. Epub     [PubMed PMID: 29185831]

Cleland JA. The qualitative orientation in medical education research. Korean journal of medical education. 2017 Jun:29(2):61-71. doi: 10.3946/kjme.2017.53. Epub 2017 May 29     [PubMed PMID: 28597869]

Foley G, Timonen V. Using Grounded Theory Method to Capture and Analyze Health Care Experiences. Health services research. 2015 Aug:50(4):1195-210. doi: 10.1111/1475-6773.12275. Epub 2014 Dec 18     [PubMed PMID: 25523315]

Effect of hypophysectomy on cAMP changes in rat adrenal medulla evoked by catecholamines and carbamylcholine., Otten U,Mueller RA,Thoenen H,, Naunyn-Schmiedeberg's archives of pharmacology, 1975     [PubMed PMID: 10591278]

Huston P, Rowan M. Qualitative studies. Their role in medical research. Canadian family physician Medecin de famille canadien. 1998 Nov:44():2453-8     [PubMed PMID: 9839063]

Corner EJ, Murray EJ, Brett SJ. Qualitative, grounded theory exploration of patients' experience of early mobilisation, rehabilitation and recovery after critical illness. BMJ open. 2019 Feb 24:9(2):e026348. doi: 10.1136/bmjopen-2018-026348. Epub 2019 Feb 24     [PubMed PMID: 30804034]

Moser A, Korstjens I. Series: Practical guidance to qualitative research. Part 3: Sampling, data collection and analysis. The European journal of general practice. 2018 Dec:24(1):9-18. doi: 10.1080/13814788.2017.1375091. Epub 2017 Dec 4     [PubMed PMID: 29199486]

Houghton C,Murphy K,Meehan B,Thomas J,Brooker D,Casey D, From screening to synthesis: using nvivo to enhance transparency in qualitative evidence synthesis. Journal of clinical nursing. 2017 Mar     [PubMed PMID: 27324875]

Soratto J, Pires DEP, Friese S. Thematic content analysis using ATLAS.ti software: Potentialities for researchs in health. Revista brasileira de enfermagem. 2020:73(3):e20190250. doi: 10.1590/0034-7167-2019-0250. Epub 2020 Apr 22     [PubMed PMID: 32321144]

Zamawe FC. The Implication of Using NVivo Software in Qualitative Data Analysis: Evidence-Based Reflections. Malawi medical journal : the journal of Medical Association of Malawi. 2015 Mar:27(1):13-5     [PubMed PMID: 26137192]

Korstjens I, Moser A. Series: Practical guidance to qualitative research. Part 4: Trustworthiness and publishing. The European journal of general practice. 2018 Dec:24(1):120-124. doi: 10.1080/13814788.2017.1375092. Epub 2017 Dec 5     [PubMed PMID: 29202616]

Tong A,Sainsbury P,Craig J, Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. International journal for quality in health care : journal of the International Society for Quality in Health Care. 2007 Dec     [PubMed PMID: 17872937]

O'Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research: a synthesis of recommendations. Academic medicine : journal of the Association of American Medical Colleges. 2014 Sep:89(9):1245-51. doi: 10.1097/ACM.0000000000000388. Epub     [PubMed PMID: 24979285]

Palermo C, King O, Brock T, Brown T, Crampton P, Hall H, Macaulay J, Morphet J, Mundy M, Oliaro L, Paynter S, Williams B, Wright C, E Rees C. Setting priorities for health education research: A mixed methods study. Medical teacher. 2019 Sep:41(9):1029-1038. doi: 10.1080/0142159X.2019.1612520. Epub 2019 May 29     [PubMed PMID: 31141390]

Tscholl DW, Handschin L, Rössler J, Weiss M, Spahn DR, Nöthiger CB. It's not you, it's the design - common problems with patient monitoring reported by anesthesiologists: a mixed qualitative and quantitative study. BMC anesthesiology. 2019 May 28:19(1):87. doi: 10.1186/s12871-019-0757-z. Epub 2019 May 28     [PubMed PMID: 31138143]

Grossoehme DH, Overview of qualitative research. Journal of health care chaplaincy. 2014;     [PubMed PMID: 24926897]

Delaney H, Devane D, Hunter A, Hennessy M, Parker A, Murphy L, Cronin P, Smith V. Limited evidence exists on the effectiveness of education and training interventions on trial recruitment; a systematic review. Journal of clinical epidemiology. 2019 Sep:113():75-82. doi: 10.1016/j.jclinepi.2019.05.013. Epub 2019 May 22     [PubMed PMID: 31128220]

Use the mouse wheel to zoom in and out, click and drag to pan the image

Warning: The NCBI web site requires JavaScript to function. more...

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing; 2024 Jan-.

StatPearls [Internet].

Human subjects research design.

Marlon L. Bayot ; Grace D. Brannan ; Janelle M. Brannan ; Steven Tenny .

Affiliations

Last Update: August 14, 2023 .

• Definition/Introduction

Human subjects research is a heavily regulated type of research, hence this paper will start with two critical definitions. The US Department of Health and Human Services (HHS) Code of Federal Regulations, 45 CFR 46, provides the following definitions: [1]  “A living individual about whom an investigator (whether professional or student) conducting research:

• Obtains information or biospecimens through intervention or interaction with the individual, and uses, studies, or analyzes the information or biospecimens; or
• Obtains, uses, studies, analyzes, or generates identifiable private information or identifiable biospecimens."

Research means “a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge.” Human subjects research is at the intersection of these two federal definitions and must obtain Institutional Review Board approval before starting, regardless of the type of design involved. The topic of a human research study varies and can include building a theory or hypothesis, determining patient satisfaction, or testing a medication, tool, device, process, or health intervention, to name a few. 

Research studies are classified into a qualitative study, a quantitative study, or a combination of both, called a mixed-methods study. [2] [3]  Qualitative studies gather non-numerical data, whereas quantitative research involves collecting numerical data. Other classifications of research studies exist depending on the purpose and utility of the study, [4]  examples include health systems research and operational research. [5] This review will be limited to the most common quantitative and qualitative research designs.

Quantitative Research

A research study can be done to describe variables and/or to determine the association of test and outcome variables regarding the research topic. [1] Quantitative research studies also subdivide into either interventional studies or non-interventional (observational) studies.  For interventional research studies, the researcher performs some intervention or manipulation of one or more groups in the research study and compares the outcomes to the other groups to help analyze the variables of interest. It may or may not be randomized, although a randomized controlled trial is considered a gold standard, as randomization of patients into the treatment groups reduce bias. Interventional studies apply to medical drugs, biologics, and devices.

For observational or non-interventional research studies, the investigator gathers data for identified variables of interest without any intervention or outside influence by the investigator on the groups under study. Cohort, cross-sectional, and case-control are the common types. [2]

A cohort study involves longitudinally following a group or groups of population with certain known exposures to determine who develops certain diseases or illnesses. This type of study could establish causal relationships between exposure and outcomes such as illness. [2] A cross-sectional study deals with a population at a given point in time as opposed to longitudinally and could provide information such as prevalence. Case-control studies compare populations with and without the exposure to determine if an illness will develop and at what rate in either group. A classic example is comparing smokers and non-smokers to determine which group develops lung cancer.

Qualitative Research

Qualitative research aims to answer the more open-ended questions that arise during the research process. Rather than trying to answer quantitative ‘how much’ or ‘how many’-type questions, qualitative research seeks to answer ‘how’ and ‘why’ questions. [3] Qualitative research often aims to understand and explain why or how a phenomenon is the way it is in order to provide insights and explanations of real-life problems and experiences. [4] Qualitative research can be used alone, in conjunction with quantitative research in mixed methods research, or as a way to explain the findings of a quantitative study because a quantitative study might show that there is a correlation between two things, but a qualitative study could then tell why that correlation exists, and not just that it does indeed exist.

There are many approaches used for qualitative research. Some of the most common are ethnography, grounded theory, phenomenology, and narrative research. [3] Ethnography is an approach that involves the researcher to be immersed in their participant’s environment, and through this immersion, collect insight into the actions, behaviors, and events that could aid them in their research. [4]  Grounded theory is an approach where the researcher observes the population of interest in order to develop a theory that explains the topic of interest. [3] Phenomenology as an approach emphasizes the importance of the ‘lived experience’ for explaining phenomena. [4] Grounded theory and phenomenology are similar, but grounded theory focuses on observation as a whole to create a theory, whereas phenomenology focuses on the perspective of participants themselves to explain why or how something happens. Lastly, narrative research showcases one of qualitative research’s strengths, the ability to tell a story. When research includes the perspective of the individuals involved, it can create robust theory-building because it takes into account the real-life implications and impacts of phenomena in a way that quantitative research often lacks. Data for qualitative research is collected in many ways, including interviews, focus groups, case studies, and medical record reviews.

Mixed Methods Research

In some cases, a combination of both qualitative and quantitative methods, or what is called a mixed-methods research is performed. Mixed methods approaches that combine qualitative and quantitative research can allow for hypothesis generation and hypothesis testing to help try to answer questions in a more well-rounded way. This is usually done to get the benefits of both numerical and non-numerical information to answer the research questions on hand. For example, a cross-sectional study found that young teens are vaping at a high rate. For further elucidation of the reasons why these teens vape, a subsequent focus group could be performed. 

• Issues of Concern

One of the primary concerns in doing research is the identification and formulation of the research problem (i.e., research question). [5]  The research problem should be ethical, researchable, significant, and feasible. In medicine, the goal of the research is not only to add relevant findings to the scientific body of knowledge but also to provide a beneficial, useful contribution to stakeholders, particularly the patients.

The second area of concern for research studies is selecting the correct research study to perform.  Many times descriptive and qualitative research must first take place to produce a robust, significant, and feasible research hypothesis for later quantitative research methods. [6]   Additionally, different research study types have different levels of strength and risk of bias as delineated in the hierarchy of research study designs. [7]  

• Clinical Significance

The significance of research studies and its findings collectively support both clinical and public health needs. The discovery of new medicines and new treatment modalities for specific diseases is possible using randomized clinical (control) trials, more commonly termed as RCTs. [8] Public health, both as medical and social science, can choose from a wide range of qualitative studies, descriptive, analytic, community-based trials [9] , and operations researches, among others, to explore and describe the characteristics of certain groups of populations and its associations to the disease process or a particular health intervention, yielding findings that will inform policymakers and stakeholders.

In clinical settings, case studies and case series can be used by clinicians, surgeons, and other clinical specialists to scientifically document and describe the occurrence of rare diseases. [10]  Researchers can perform studies to determine the association of exposure variables or risk factors in rare diseases or cohort studies to investigate rare exposure variables present in the study population. Meanwhile, studies such as meta-analysis and systematic review are good choices for researchers who want to summarize the results of previous research findings, in quantitative and qualitative means, respectively. [11] [12] Mixed methods are employed to combine and exhaust the utility of the research type or study design combinations (e.g., quantitative and qualitative studies). [13]

Research studies can be both simple and complex; thus, they can be performed in several ways, which must be consistently systematic and scientific. The acquisition of new research findings will eventually find utility in the application of evidence-based medicine (EBM). [14] Research studies must be carried out within the walls of medical ethics, free of bias, and primarily geared towards the welfare of our patients rather than just merely the expedition of science. [15]

• Review Questions
• Access free multiple choice questions on this topic.
• Comment on this article.

Disclosure: Marlon Bayot declares no relevant financial relationships with ineligible companies.

Disclosure: Grace Brannan declares no relevant financial relationships with ineligible companies.

Disclosure: Janelle Brannan declares no relevant financial relationships with ineligible companies.

Disclosure: Steven Tenny declares no relevant financial relationships with ineligible companies.

This book is distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) ( http://creativecommons.org/licenses/by-nc-nd/4.0/ ), which permits others to distribute the work, provided that the article is not altered or used commercially. You are not required to obtain permission to distribute this article, provided that you credit the author and journal.

• Cite this Page Bayot ML, Brannan GD, Brannan JM, et al. Human Subjects Research Design. [Updated 2023 Aug 14]. In: StatPearls [Internet]. Treasure Island (FL): StatPearls Publishing; 2024 Jan-.

In this Page

Bulk download.

• Bulk download StatPearls data from FTP

Related information

• PMC PubMed Central citations
• PubMed Links to PubMed

Similar articles in PubMed

• Currently effective Indian Health Service eligibility regulations. Indian Health Service, HHS. Republication of currently effective Indian Health Service eligibility regulations. [Fed Regist. 1999] Currently effective Indian Health Service eligibility regulations. Indian Health Service, HHS. Republication of currently effective Indian Health Service eligibility regulations. . Fed Regist. 1999 Oct 28; 64(208):58318-22.
• Exemption of certain research and demonstration projects from regulations for protection of human research subjects--Office of the Secretary, HHS. Final rule. [Fed Regist. 1983] Exemption of certain research and demonstration projects from regulations for protection of human research subjects--Office of the Secretary, HHS. Final rule. . Fed Regist. 1983 Mar 4; 48(44):9266-70.
• American Society of Clinical Oncology policy statement: oversight of clinical research. [J Clin Oncol. 2003] American Society of Clinical Oncology policy statement: oversight of clinical research. American Society of Clinical Oncology. J Clin Oncol. 2003 Jun 15; 21(12):2377-86. Epub 2003 Apr 29.
• Review How to Determine if a Project Is Human Subjects Research, a Quality Improvement Project, or Both. [Ochsner J. 2020] Review How to Determine if a Project Is Human Subjects Research, a Quality Improvement Project, or Both. Bass PF 3rd, Maloy JW. Ochsner J. 2020 Spring; 20(1):56-61.
• Review Understanding Food and Drug Administration regulatory requirements for an investigational device exemption for sponsor-investigators. [J Investig Med. 2012] Review Understanding Food and Drug Administration regulatory requirements for an investigational device exemption for sponsor-investigators. Holbein ME, Berglund JP. J Investig Med. 2012 Oct; 60(7):987-94.

Recent Activity

• Human Subjects Research Design - StatPearls Human Subjects Research Design - StatPearls

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

What does “urgency” mean when prioritizing cancer treatment? Results from a qualitative study with German oncologists and other experts during the COVID-19 pandemic

• Open access
• Published: 15 July 2024
• Volume 150 , article number  352 , ( 2024 )

Cite this article

You have full access to this open access article

• Sabine Sommerlatte   ORCID: orcid.org/0000-0001-6239-4349 1 ,
• Helene Hense 2 ,
• Stephan Nadolny 1 , 3 ,
• Anna-Lena Kraeft 4 ,
• Celine Lugnier 4 ,
• Jochen Schmitt 2 ,
• Olaf Schoffer 2 ,
• Anke Reinacher-Schick 4 &
• Jan Schildmann 1  

Cancer care in Germany during the COVID-19 pandemic was affected by resource scarcity and the necessity to prioritize medical measures. This study explores ethical criteria for prioritization and their application in cancer practices from the perspective of German oncologists and other experts.

We conducted fourteen semi-structured interviews with German oncologists between February and July 2021 and fed findings of interviews and additional data on prioritizing cancer care into four structured group discussions, in January and February 2022, with 22 experts from medicine, nursing, law, ethics, health services research and health insurance. Interviews and group discussions were digitally recorded, transcribed verbatim and analyzed using qualitative content analysis.

Narratives of the participants focus on “urgency” as most acceptable criterion for prioritization in cancer care. Patients who are considered curable and those with a high level of suffering, were given a high degree of “urgency.” However, further analysis indicates that the “urgency” criterion needs to be further distinguished according to at least three different dimensions: “urgency” to (1) prevent imminent harm to life, (2) prevent future harm to life and (3) alleviate suffering. In addition, “urgency” is modulated by the “success,” which can be reached by means of an intervention, and the “likelihood” of reaching that success.

Our analysis indicates that while “urgency” is a well-established criterion, its operationalization in the context of oncology is challenging. We argue that combined conceptual and clinical analyses are necessary for a sound application of the “urgency” criterion to prioritization in cancer care.

Avoid common mistakes on your manuscript.

The COVID-19 pandemic has fueled public and scientific attention on and debates about the equitable distribution of health resources (Emanuel et al. 2020 ; Marckmann et al. 2020 ; Trudeau et al. 2020 ; Wilkinson 2020 ). Although the debate focused initially on the care of COVID-19 patients and the distribution of intensive care resources, it quickly became evident that areas not primarily involved in the care of COVID-19 patients, such as cancer care, were also affected by limitations. Such limitations in oncology included a temporary reduction of cancer surgeries and a decline in screening and follow-up care (American Association for Cancer Research 2022 ; Eckford et al. 2022 ; Mazidimoradi et al. 2022 , 2023 ; Oba et al. 2020 ; Reinacher-Schick et al. 2023 ; Rückher et al. 2022 ). The reservation of supply capacity for expected or actual corona patients (Lambertini et al. 2020 ; Leung et al. 2020 ; Reynolds et al. 2020 ; Stevens 2020 ) and the shortage of staff, due to, for example, infections with Sars-CoV-2 as well as other burdens (Lim et al. 2022 ; Schug et al. 2022 ; Sommerlatte et al. 2023 ; Tabur et al. 2022 ), were among the reasons for these changes.

In case of resource scarcity, any prioritization should be based on transparent and comprehensibly justified criteria and must take into account the supply reality of a country (Emanuel et al. 2020 ; ZEKO 2000 ). Accordingly, various national and international professional societies and other author groups have published recommendations for the prioritization of (cancer) care (American College of Surgeons 2020 ; Curigliano et al. 2020 ; Emanuel et al. 2020 ; Hanna et al. 2020 ; Marckmann et al. 2020 ; Marron et al. 2020 ; Meyfroidt et al. 2020 ; ÖGARI 2020 ; SAMW 2021 ). The well-established ethical criterion of “urgency” regarding allocation in case of scarcity was named as a principle to guide prioritization in many of these guidelines (American College of Surgeons 2020 ; Curigliano et al. 2020 ; Meyfroidt et al. 2020 ; SAMW 2021 ). However, while some recommendations mention “urgency” as an abstract criterion, other guidelines provide concrete priority lists without explaining how “urgency” was understood and translated into suggested rankings to prioritize certain diagnostic or therapeutic measures. This study aims to explore ethical criteria for prioritization and their application in cancer practices during the COVID-19 pandemic from the perspective of German oncologists and other experts.

Based on combined empirical-ethical analysis (see methods section), we focus on the operationalization of “urgency” in oncology and complementary criteria such as “success” and “likelihood of success”, which should guide allocation decisions. We argue that unlike in intensive care medicine, where “urgency” usually refers to the necessity to act in a timely manner in order to prevent imminent death and “likelihood of success” refers to the likelihood of surviving intensive care (Meyfroidt et al. 2020 ; Pugh et al. 2021 ; Marckmann et al. 2020 ; Bognar 2024 ), in oncology various therapeutic goals such as cure, prolongation of life and alleviation of suffering must be taken into account in the application of these criteria in prioritization decisions.

The following presentation of the methodology and results follows the consolidated criteria for reporting qualitative research (COREQ) (Tong et al. 2007 ). The completed COREQ checklist is shown in Online Resource 1. Additional information concerning the credentials, occupation and gender of the researchers involved in this study is provided in Online Resource 2.

The data collection took place in two stages: (1) qualitative semi-structured interviews and (2) group discussions.

Qualitative interviews

We recruited a convenience sample of oncologists via the mailing list of members of the Working Group for Medical Oncology of the German Cancer Society (Arbeitsgemeinschaft Internistische Onkologie, n  = 929). In addition, oncologists were contacted directly by the research team. All participants received written study information and informed consent form. Inclusion criteria were medical activity in a field of cancer medicine and consent to participate in the study. Exclusion criteria were lack of knowledge of the German language and lack of capacity to consent.

Group discussions

We applied the sampling strategy of a criterion-guided purposive selection in group discussions, using professional qualification as a criterion in order to capture the different perspectives of relevant stakeholders and experts and, thus, obtain as much variability of positions as possible (Akremi 2022 ). Purposive sampling achieves variance in participant characteristics and a heterogeneous sample (Schreier 2020 ).

The research team generated a list of relevant stakeholders from the fields of medicine, nursing, ethics, law, health services research, as well as health insurance and patient representatives. The experts received an invitation to participate in the group discussions via email and, if interested, written study information and an informed consent form. The inclusion criterion was proven expertise in the respective specialist area. Exclusion criteria were lack of knowledge of the German language and lack of capacity to consent.

Data collection

Based on a selective literature review and discussions within the research team, we developed an interview guideline on ethical challenges in dealing with scarce resources and prioritization criteria during the pandemic (Table  1 ). Fourteen qualitative semi-structured interviews were conducted via telephone (So, JS) Footnote 1 between February and July 2021. The first two interviews were pilot interviews.

We conducted four structured group discussions online on January 20, February 2, 7 and 15, 2022. They were each led by a moderator (JS) and supported by a co-moderator (HH, So) Footnote 2 . Two researchers took notes on important discussion points (HH, So) (Pohontsch et al. 2018 ). ARS Footnote 3 was a participant observer in group discussion 1. The group discussions were structured according to three distinct topics: (1) diagnostics, screening and follow-up care, (2) tumor surgery and system therapy, and (3) psychosocial, and general and specialized palliative care.

Based on the analysis of quantitative and qualitative data collected beforehand by the CancerCOVID consortium (AIO and DGHO 2022 ), we created a PowerPoint presentation with key findings on the care of patients with colorectal and pancreatic cancer and verbatim quotes from the interviews (Table  2 ) to serve as stimuli for the group discussions. We focused on these two tumor entities because they represent the scientific focus of the interdisciplinary CancerCOVID research network, within the framework of which the group discussions were conducted (Lugnier et al. 2024 ). Due to their comparatively high incidence among both men and women, these entities are well suited as examples (Sung et al. 2021 ). Furthermore, we developed questions on criteria and possible justifications of these criteria for prioritizing (1) diagnostic procedures, (2) tumor surgery and systemic therapies, and (3) psychosocial and palliative care in the context of a current or future pandemic to stimulate discussions at the beginning of each topic (Table  2 ). Stimuli and questions were pilot tested within the research team and with other researchers at the Institute for History and Ethics of Medicine at Martin Luther University Halle-Wittenberg.

Both interviews and group discussions were digitally recorded as audio files and subsequently transcribed and anonymized during the process (Dresing et al. 2015 ).

Data analysis

First, a preliminary analysis of the interviews was carried out in preparation for the group discussions. The second step involved analyzing the overall results of the interviews and group discussions. Analysis was based on the method of qualitative content analysis according to Kuckartz ( 2018 ). It includes the following 7 phases:

Initiatory work with the text, memo writing, case summaries.

Developing the main categories.

Coding the data with the main categories.

Inductive formation of subcategories.

Coding the data with the subcategories.

Simple and complex analyses (e.g. visualizations, tabular case summaries, relationships between the subcategories of a main category).

Writing up the results, documenting the procedure (Kuckartz 2018 ).

At the center of the method is the coding of the text using a category system. Thematically similar data sections were assigned so-called “codes” and summarized into superordinate main categories. First, the material was roughly coded using deductively generated codes, which were based on the questions in the interview guide and the topics of the group discussions. Further analysis was carried out using inductive coding of the material. Representative quotes were selected to illustrate the categories (printed in italics in the results section). Quotations were translated from German to English using DeepL software and checked by a native English speaker. Data coding and analysis of (sub)categories was performed using MAXQDA 2022. Additionally, we used pinboards to visualize the (sub)categories and relationships between them in order to refine the (sub)categories. Five researchers from the fields of medical ethics, medicine, nursing science, and health services research were involved in the analysis (So, HH, SN, JS, SD) Footnote 4 . The analysis of the group discussions focused on topic no. 2 (tumor surgery and system therapy), since difficult prioritization decisions and prioritization criteria were discussed in the interviews, particularly in relation to oncological therapies.

Empirical-ethical analysis was based on a consultative approach, according to which the study participants fed into the normative analysis by means of exploring their views and experiences (Davies et al. 2015 ). Examples for such an approach are “Reflexive Balancing” or ”Symbiotic Empirical Ethics” as proposed by Ives ( 2014 ) and Frith ( 2012 ). In line with core principles of this approach our methodology comprises the following overarching steps, which do run in an iterative process: setting out the circumstances and exploring morally relevant aspects of practice, specifying theories and principles, using ethical theory as a tool of analysis, theory building, and making normative judgements (Frith 2012 ).”

Participants

We conducted and analyzed fourteen interviews, including two pilot interviews. Seven physicians were approached directly, 7 were recruited via mailing list. Each interview lasted between 12 and 54 min. The characteristics of the sample are shown in Table  3 . The four group discussions each lasted between 72 and 76 min. Twenty-two experts from medicine, nursing, law, ethics, health services research and health insurance participated (Table  4 ).

Main categories are shown in Table  5 . Footnote 5 Regarding (material) prioritization criteria in oncology, “urgency” and “(likelihood of) success” were at the center of both the interviews and the group discussions. Our analysis shows that both criteria have several dimensions when being considered in the context of cancer care. In the following, we present the identified dimensions first selected based on the narratives of the research participants and illustrate these dimensions using representative quotes as examples. We use pseudonyms for interviews (INT 1–14) and group discussions (GD 1–4) and identify the citations by means of abbreviations for the respective position (pos.) or page (p.).

“Urgency” to prevent imminent harm to life

Some experts cite imminent harm to life, i.e. imminent death, as the greatest harm possible and a category on which the “urgency” of a therapeutic measure crucially depends. The patient with the smallest time window of opportunity to avert imminent death is considered most urgent.

And as long as we get there with the criterion of urgency, I think there is a clear order of priority, i.e. the person who would die next is treated first in order to prevent them from dying, to avert the greatest harm. (GD 4, p. 24, ethicist)

“Urgency” to prevent future harm to life

Furthermore, the threat of future harm to life, in the sense of a worsening of prognosis, was deemed relevant to determine the “urgency” of a therapeutic measure. Two types of future harm to life were discussed: a shortening of life expectancy in general, and particularly, that a disease which is deemed curable at the present stage will not be curable after postponement of a treatment.

Then, for example, we have acute leukemia, which is a curable disease, even a well curable disease, but which urgently requires rapid initiation of treatment. We know very clearly that the prognosis will be worse if we wait unnecessarily long to start treatment. (INT 6, pos. 31, physician) If surgery has already taken place, it is important to ensure that systemic therapy, if it is necessary, is given within a certain time frame, because we know, also from analyzing data, that if chemotherapy, e.g. for colon carcinoma, is postponed for longer than 3 weeks, survival is actually significantly worse. (GD 2, p. 20, expert from health services research)

“Urgency” to alleviate suffering

The alleviation of suffering was another dimension of “urgency” discussed in the interviews and group discussions next to imminent or future harm to life. There is widespread agreement that patients with symptoms, such as severe pain, require urgent treatment due to the high level of suffering and deterioration of quality of life, even though the symptoms might not indicate an immediately life-threatening condition.

[…] Particularly in oncology/palliative care, there may well be emergencies that require immediate treatment, intracranial pressure or something like that, and in order to maintain the quality of life as far as possible, very urgent pain therapy or something like that […]. (GD 1, p. 26, physician)

“(Likelihood of) success”

To heal or not to heal ‒ “(likelihood of) success” in the context of different treatment goals.

In addition to the different dimensions of “urgency,” we found that the criterion of “urgency” was modulated by that of “success” and “likelihood of success.” In the following and based on the narratives in interviews and group discussions, “success” is understood as the achievement of a therapeutic goal and the actual realization of a potential benefit of a therapeutic measure. “Likelihood of success” in the strict sense, describes the probability of achieving a specific therapeutic goal, such as a cure, prolongation of life or alleviation of suffering.

A strong consideration was expressed in both the interviews and group discussions to prioritize curative patients over those for whom “only” a prolongation of life but no cure can be achieved.

[…] so in the end mostly either curable patient versus non-curable patient, then you have to say that the decision was usually more in favor of the curable patient, or at least significantly more often. (INT 4, pos. 47, physician) […] I would like to add two points to the discussion, which are probably not entirely without controversy, namely that patients with a curative treatment approach naturally have a very high priority. And in particular the postponement of operations or multimodal therapy concepts for patients who can expect a curative approach is of course highly problematic. (GD 2, p. 20, physician)

In this context, the criteria of “success” and “likelihood of success” were sometimes conflated. Consequently, no probabilities were compared (e.g. 80% chance of cure vs. 20% chance of cure), but, instead, a higher probability of “success” was attributed to patients who might achieve a higher benefit in terms of the years of life gained or with a curative therapy approach per se.

So you would tend to take the one where you have the feeling that there is, how should I put it, a realistic, reasonable, perhaps even very good chance of cure? I mean, the difference is clear. It’s a different treatment goal, but this distinction between palliative and curative is a bit like saying that I’m going to take the person for whom I can actually do something curative and put the person I can only treat palliatively on the back burner. That goes a bit in that direction. I have no real prospect of success, so to speak. (GD 1, p. 23, law expert)

This study explored ethical criteria for prioritization and their operationalization in cancer care during the COVID-19 pandemic from the perspective of oncologists and other experts. The criteria of “urgency” and “(likelihood of) success” were at the center of both the interviews and group discussions. Those are well-established criteria which are familiar from other medical fields, such as organ transplantation and intensive care medicine (Bobbert and Ganten 2013 ; Gottlieb 2017 ; Marckmann et al. 2020 ; ÖGARI 2020; SAMW 2021) and, at first sight, there seems to be a similarity to the allocation debate in critical care medicine. However, “urgency” in debates about allocating intensive care resources is – often implicitly – equated with urgency to avert harm, in the sense of loss of life, since intensive care units generally treat patients whose vital or organ functions are in a life-threatening condition and the aim is to save their lives and enable them to continue living as independently as possible outside the intensive care unit (Meyfroidt et al. 2020 ; Neitzke et al. 2019 ; Pugh et al. 2021 ). “Likelihood of success” is usually defined as the probability of surviving intensive care and getting discharged (Marckmann et al. 2020 ; Bognar 2024 ). Our analyses of interviews and group discussions revealed that the operationalization of these criteria in the context of oncology is much more challenging, because different dimensions of harm and, thus, diverse corresponding therapy goals, such as a cure, lifetime prolongation and the alleviation of suffering, must be taken into account (Markman 1994 ; Mieras et al. 2021 ). In addition, possible harm (e.g. loss of lifetime because a cancer is no longer curable due to delayed treatment) may lie in the future and, due to the probabilistic nature of future events and outcomes, might be less certain than in the case of rationing ventilators in intensive care, where death is imminent (Han et al. 2011 ).

Two groups of cancer patients, those for whom a cure can be achieved (1) and ones with a high level of suffering (2), were given high “urgency” (and, thereby, priority) by the interviewees. This observation might be explained by the fact that the interviewees implicitly refer to the rule of rescue, which states that the rescue of people must take place without question and as a priority, and emphasizes the importance of live-saving measures (McKie and Richardson 2003 ; Schöne-Seifert and Friedrich 2013 ). In this context, Schöne-Seifert and Friedrich ( 2013 ) distinguish between two types of “urgency.” In the first case, a therapeutic measure must be taken quickly in order to be successful at all (e.g. stopping an arterial hemorrhage). This would correspond in the case of our interviewees in the oncological context, for example, to the initiation of curative chemotherapy which may have to be carried out quickly (albeit with different time windows) so that a cure, i.e. a rescue, can still be achieved. In the second case, there is “urgency” because the existing condition (e.g. severe pain) is unbearable, even if it does not immediately lead to death or severe irreversible damage (Schöne-Seifert and Friedrich 2013 ). In oncology, for example, this could be the case for urgent pain therapy.

Furthermore, the consideration to prioritize “curative patients” is in line with some recommendations, such as those of the American Society of Clinical Oncology referring to the principle of maximizing health benefits, which can be operationalized as most lives saved (Emanuel and Persad 2023 ; Marron et al. 2020 ). While we agree that curing cancer is a great good, we think that giving unrestricted and unquestioned priority to patients with a curative treatment goal is a shortcut, which might entail the risk of systematically disadvantaging those patients who are not considered curable but for whom there is a high chance of gaining a significant extension of life.

Based on our findings from interviews and group discussions and further conceptual and ethical analyses, we suggest considering the following points when operationalizing “urgency,” “likelihood of success” and the “benefit of a therapy” in cancer care.

Nuancing “urgency”

“ Urgency” refers to the necessity to avert significant harm in a timely manner (Schöne-Seifert and Friedrich 2013 ). According to our data, different qualities of harm (immediate or future harm to life as well as immediate suffering) must be taken into account in oncology. “Urgency” is, therefore, a gradual and multidimensional criterion and must be assessed individually regarding harm and respective time windows for averting the damage (Schöne-Seifert and Friedrich 2013 ). The “urgent first” maxim is justifiable if it does not mean unreasonable sacrifices for those postponed (Schöne-Seifert and Friedrich 2013 ). We argue that, if a cancer can still be cured with a high likelihood in two weeks, it may, in certain cases, be ethically justifiable to postpone treatment within this time frame and to prioritize, for example, an urgent need to relieve existing suffering in a palliative situation.

Distinguishing “success” from “likelihood of success”

It seems from some narratives that curative patients were given priority because the “likelihood of success” was conflated with “success” in the sense of the potential maximum benefit of a measure, i.e. curing the patient, while the likelihood to actually achieve that cure was hardly discussed. Our data suggest that it is necessary to clearly state what “curative” actually means in order to be able to make informed prioritization decisions in cancer care. We argue, that “curative,” from an ethical perspective, does not seem to be a decisional criterion if there is a patient for whom a cure is possible with a 20% chance, whereas for another “incurable” patient there might be a 90% probability that life could be significantly prolonged. Accordingly, it also seems to be important to identify those patients for whom a significant extension of life can still be achieved, rather than simply dividing patients into curable and non-curable. Doctors seem to be able to make a relatively accurate prognosis for patients who can still live for years (Orlovic et al. 2023 ).

Nuancing “success”

Various dimensions of “success,” in the sense of actual benefit achieved, namely: cure, prolonging life and improving quality of life, are important in oncology (Markman 1994 ; Mieras et al. 2021 ). While it does not appear to us to be ethically unproblematic per se to simply prioritize curative treatment measures, it can make sense the other way round to identify measures that only have a small benefit or are “futile” (ZEKO 2022 ), as these would also be the measures whose postponement would cause the least harm and for which the Rule of Rescue is not deemed to be applicable (Schöne-Seifert and Friedrich 2013 ). Some authors suggest prioritization/rationing based on thresholds. A distinction is made between thresholds with low utility and those with a low chance of success (Schöne-Seifert et al. 2012 ). Withholding a potential benefit when there is little chance of success (e.g. cure in 20%) seems more problematic to Schöne-Seifert et al. ( 2012 ) than withholding a very small benefit even with a maximum response, i.e. a high chance of success, since in the former case an extremely high benefit may be expected in individual cases. The frameworks of the American Society of Clinical Oncology and the European Society for Medical Oncology concerning the benefit of diagnostic and treatment interventions and cancer drugs may provide a good evidence-based starting point regarding the respective decisions about prioritization (Cherny et al. 2015 , 2017 ; Schildmann 2019 ; Schnipper et al. 2012 ).

Limitations

This study focusses on the experiences of experts. The patient perspective has therefore not been analyzed. The interviews were mainly conducted with doctors from the clinical sector. Overall, only participants from Germany were included in the study. Transferability of our empirical findings to other contexts is, therefore, limited. Additionally, there may be social desirability bias since the prioritization of medical measures is ethically controversial and socially relevant, and some of the participating experts were representatives of professional interest groups. In addition, the ethical analysis focused on the prioritization of therapeutic measures. The prioritization of diagnostics, prevention and psychosocial and supportive palliative care and the significance of these areas of care were therefore not considered and should be addressed in future research.

To the best of our knowledge, this is the first study that has explored and differentiated the well-established criterion of “urgency” in the context of cancer care based on the perspective of oncologists and other experts. According to German experts, “urgency” is acceptable for prioritizing therapeutic measures in oncology. However, the criterion must be operationalized in light of the different oncological treatment goals and regarding the maximum achievable “benefit” and “likelihood of success.” The results of this study have been incorporated into the development of an S1 guideline on prioritization in gastrointestinal tumors in the context of scarce resources (AIO and DGHO 2022; Lugnier et al. 2024 ).

Data availability

The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.

JS: Jan Schildmann; So: Sabine Sommerlatte.

HH: Helene Hense.

ARS: Anke Reinacher-Schick.

SD: Sophie Dahlke; SN: Stephan Nadolny.

As part of the iterative data analysis in the team, “urgency” and “likelihood of success” crystallized as central concepts that guided the group discussions. It also became evident, that the operationalization of these concepts in oncology differs significantly from other areas such as intensive care medicine. This does not seem to apply to the procedural and inadmissible criteria. We argue, that, for example, criteria such as transparency should apply to all areas of medicine in the same way. Prioritization according to religious belief should be equally inadmissible in both intensive care medicine and oncology. For this reason, the further analysis and presentation of the results focused on the categories of “urgency” and “likelihood of success”.

Akremi L (2022) Stichprobenziehung in Der Qualitativen Sozialforschung. In: Baur N, Blasius J (eds) Handbuch Methoden Der Empirischen Sozialforschung. Springer Fachmedien, Wiesbaden, pp 405–424. https://doi.org/10.1007/978-3-658-37985-8_26

Chapter   Google Scholar  

American Association for Cancer Research (2022) AACR Report on the Impact of COVID-19 on Cancer Research and Patient Care. https://www.AACR.org/COVIDReport . Accessed 24 May 2024

American College of Surgeons (2020) COVID 19: Elective Case Triage Guidelines for Surgical Care. https://www.facs.org/media/wfjhq0jw/guidance_for_triage_of_nonemergent_surgical_procedures.pdf . Accessed 8 March 2024

Arbeitsgemeinschaft Internistische Onkologie der DKG (AIO), Deutsche Gesellschaft für Hämatologie und Medizinische Onkologie (DGHO) (2023) S1-Leitlinie Priorisierung und Ressourcenallokation im Kontext der Pandemie. Empfehlungen für die Krebsversorgung am Beispiel gastrointestinaler Tumoren. AWMF online. https://register.awmf.org/de/leitlinien/detail/018-039 . Accessed 8 March 2024

Bobbert M, Ganten TM (2013) Liver allocation: urgency of need or prospect of success? Ethical considerations. Clin Transpl 27(S25):34–39. https://doi.org/10.1111/ctr.12154

Article   Google Scholar  

Bognar G (2024) A theory of triage. Bioethics 38(2):95–106. https://doi.org/10.1111/bioe.13242

Article   PubMed   Google Scholar  

Cherny NI, Sullivan R, Dafni U, Kerst JM, Sobrero A, Zielinski C, de Vries EG, Piccart MJ (2015) A standardised, generic, validated approach to stratify the magnitude of clinical benefit that can be anticipated from anti-cancer therapies: the European Society for Medical Oncology Magnitude of Clinical Benefit Scale (ESMO-MCBS). Ann Oncol 26(8):1547–1573. https://doi.org/10.1093/annonc/mdv249

Article   CAS   PubMed   Google Scholar  

Cherny NI, Dafni U, Bogaerts J, Latino NJ, Pentheroudakis G, Douillard JY, Tabernero J, Zielinski C, Piccart MJ, de Vries EGE (2017) ESMO-Magnitude of Clinical Benefit Scale version 1.1. Ann Oncol 28(10):2340–2366. https://doi.org/10.1093/annonc/mdx310

Curigliano G, Banerjee S, Cervantes A, Garassino MC, Garrido P, Girard N, Haanen J, Jordan K, Lordick F, Machiels JP, Michielin O, Peters S, Tabernero J, Douillard JY, Pentheroudakis G, Panel (2020) members Managing cancer patients during the COVID-19 pandemic: an ESMO multidisciplinary expert consensus. Ann Oncol 31(10):1320–1335. https://doi.org/10.1016/j.annonc.2020.07.010

Davies R, Ives J, Dunn M (2015) A systematic review of empirical bioethics methodologies. BMC Med Ethics 16:15. https://doi.org/10.1186/s12910-015-0010-3

Article   PubMed   PubMed Central   Google Scholar  

Dresing T, Pehl T, Schmieder C (2015) Manual (on) Transcription. Transcription Conventions, Software Guides and Practical Hints for Qualitative Researchers. 3 rd English Edition. Marburg. http://www.audiotranskription.de/english/transcription-practicalguide.htm . Accessed 23 May 2024

Eckford RD, Gaisser A, Arndt V, Baumann M, Kludt E, Mehlis K, Ubels J, Winkler EC, Weg-Remers S, Schlander M (2022) The COVID-19 pandemic and Cancer patients in Germany: impact on treatment, Follow-Up care and psychological Burden. Front Public Health 9:788598. https://doi.org/10.3389/fpubh.2021.788598

Emanuel EJ, Persad G (2023) The shared ethical framework to allocate scarce medical resources: a lesson from COVID-19. Lancet 401(10391):1892–1902. https://doi.org/10.1016/S0140-6736(23)00812-7

Emanuel EJ, Persad G, Upshur R, Thome B, Parker M, Glickman A, Zhang C, Boyle C, Smith M, Phillips JP (2020) Fair allocation of scarce medical resources in the time of Covid-19. N Engl J Med 382(21):2049–2055. https://doi.org/10.1056/NEJMsb2005114

Frith L (2012) Symbiotic empirical ethics: a practical methodology. Bioethics 26(4):198–206. https://doi.org/10.1111/j.1467-8519.2010.01843.x

Gottlieb J (2017) Lung allocation. J Thorac Dis 9(8):2670–2674. https://doi.org/10.21037/jtd.2017.07.83

Han PK, Klein WM, Arora NK (2011) Varieties of uncertainty in health care: a conceptual taxonomy. Med Decis Mak 31(6):828–838. https://doi.org/10.1177/0272989x11393976

Hanna TP, Evans GA, Booth CM (2020) Cancer, COVID-19 and the precautionary principle: prioritizing treatment during a global pandemic. Nat Rev Clin Oncol 17(5):268–270. https://doi.org/10.1038/s41571-020-0362-6

Article   CAS   PubMed   PubMed Central   Google Scholar  

Ives J (2014) A method of reflexive balancing in a pragmatic, interdisciplinary and reflexive bioethics. Bioethics 28(6):302–312. https://doi.org/10.1111/bioe.12018

Kuckartz (2018) Qualitative inhaltsanalyse. Methoden, Praxis, Computeruntertützung, 4th edn. Beltz Juventa, Weinheim

Google Scholar  

Lambertini M, Toss A, Passaro A, Criscitiello C, Cremolini C, Cardone C, Loupakis F, Viscardi G, Meattini I, Dieci MV, Ferrara R, Giusti R, Di Maio M (2020) Cancer care during the spread of coronavirus disease 2019 (COVID-19) in Italy: young oncologists’ perspective. ESMO Open 5(2):e000759. https://doi.org/10.1136/esmoopen-2020-000759

Leung MST, Lin SG, Chow J, Harky A (2020) COVID-19 and Oncology: Service transformation during pandemic. Cancer Med 9(19):7161–7171. https://doi.org/10.1002/cam4.3384

Lim KHJ, Murali K, Thorne E, Punie K, Kamposioras K, Oing C, O’Connor M, Élez E, Amaral T, Garrido P, Lambertini M, Devnani B, Westphalen CB, Morgan G, Haanen JBAG, Hardy C, Banerjee S (2022) The impact of COVID-19 on oncology professionals one year on: lessons learned from the ESMO Resilience Task Force survey series. ESMO Open 7(1):100374. https://doi.org/10.1016/j.esmoop.2021.100374

Lugnier C, Sommerlatte S, Attenberger U et al (2024) Prioritization and resource allocation in the context of the COVID-19 pandemic: recommendations for colorectal and pancreatic Cancer in Germany. Oncol Res Treat. 10.1159/000538171. Advance online publication https://doi.org/10.1159/000538171

Marckmann G, Neitzke G, Schildmann J, Michalsen A, Dutzmann J, Hartog C, Jöbges S, Knochel K, Michels G, Pin M, Riessen R, Rogge A, Taupitz J, Janssens U (2020) Decisions on the allocation of intensive care resources in the context of the COVID-19 pandemic: clinical and ethical recommendations of DIVI, DGINA, DGAI, DGIIN, DGNI, DGP, DGP and AEM. Med Klin Intensivmed Notfmed 115(Suppl 3):115–122. https://doi.org/10.1007/s00063-020-00709-9

Markman M (1994) Realistic goals of cancer therapy: effective and humane care. Cleve Clin J Med 61(6):468–472. https://doi.org/10.3949/ccjm.61.6.468

Marron JM, Joffe S, Jagsi R, Spence RA (2020) Ethics and resource scarcity: ASCO recommendations for the oncology community during the COVID-19 pandemic. J Clin Oncol 38(19):2201–2206. https://doi.org/10.1200/JCO.20.00960

Mazidimoradi A, Tiznobaik A, Salehiniya H (2022) Impact of the COVID-19 pandemic on colorectal cancer screening: a systematic review. J Gastrointest Cancer 53(3):730–744. https://doi.org/10.1007/s12029-021-00679-x

Mazidimoradi A, Hadavandsiri F, Momenimovahed Z, Salehiniya H (2023) Impact of the COVID-19 pandemic on colorectal cancer diagnosis and treatment: a systematic review. J Gastrointest Cancer 54(1):171–187. https://doi.org/10.1007/s12029-021-00752-5

McKie J, Richardson J (2003) The rule of rescue. Soc Sci Med 56:2407–2419. https://doi.org/10.1016/s0277-9536(02)00244-7

Meyfroidt G, Vlieghe E, Biston P, De Decker K, Wittebole X, Collin V, Depuydt P, Nguyen DN, Hermans G, Jorens P, Ledoux D, Taccone F, Devisch I (2020) Ethical principles concerning proportionality of critical care during the 2020 COVID-19 pandemic in Belgium: advice by the Belgian Society of Intensive Care Medicine – update 26-03-2020. https://www.aem-online.de/fileadmin/user_upload/COVID-19_-_Belgian_Society_of_Intensive_Care_Medicine__-ethical_final.pdf . Accessed 16 June 2024

Mieras A, Pasman HRW, Klop HT, Onwuteaka-Philipsen BD, Tarasevych S, Tiemessen MA, Becker-Commissaris A (2021) What goals do patients and oncologists have when starting Medical Treatment for metastatic Lung Cancer? Clin Lung Cancer 22(3):242–251e5. https://doi.org/10.1016/j.cllc.2020.06.014

Neitzke G, Burchardi H, Duttge G, Hartog C, Erchinger R, Gretenkort P, Michalsen A, Mohr M, Nauck F, Salomon F, Stopfkuchen H, Weiler N, Janssens U (2019) Limits to the appropriateness of intensive care: policy statement of the German Interdisciplinary Association of Intensive Care and Emergency Medicine (DIVI). Med Klin Intensivmed Notfmed 114(1):46–52. https://doi.org/10.1007/s00063-018-0514-y

Oba A, Stoop TF, Löhr M, Hackert T, Zyromski N, Nealon WH, Unno M, Schulick RD, Al-Musawi MH, Wu W, Zhao Y, Satoi S, Wolfgang CL, Abu Hilal M, Besselink MG, Del Chiaro M, European Consortium on Minimally Invasive Pancreatic Surgery, International Study Group on Cystic Tumors of the Pancreas (2020) ; Pancreas Club, European Pancreatic Club, Chinese Pancreatic Surgery Association, Study Group of Preoperative Therapy for Pancreatic Cancer, Study Group of Pancreatic Ductal Adenocarcinoma with Peritoneal Metastasis and Global survey on pancreatic surgery during the COVID-19 pandemic. Ann Surg 272(2):e87-e93. https://doi.org/10.1097/SLA.0000000000004006

Orlovic M, Droney J, Vickerstaff V, Rosling J, Bearne A, Powell M, Riley J, McFarlane P, Koffman J, Stone P (2023) Accuracy of clinical predictions of prognosis at the end-of-life: evidence from routinely collected data in urgent care records. BMC Palliat Care 22(1):51. https://doi.org/10.1186/s12904-023-01155-y

Österreichische Gesellschaft für Anästhesiologie, Reanimation und Intensivmedizin (ÖGARI) (2020) Allokation intensivmedizinischer Ressourcen aus Anlass der Covid-19-Pandemie. Klinisch-ethische Empfehlungen für Beginn, Durchführung und Beendigung von Intensivtherapie bei Covid-19-PatientInnen. https://www.oegari.at/web_files/cms_daten/covid-19_ressourcenallokation_gari-statement_v1.7_final_2020-03-17.pdf . Accessed 8 March 2024

Pohontsch NJ, Müller V, Brandner S, Karlheim C, Jünger S, Klindtworth K, Stamer M, Höfling-Engels N, Kleineke V, Brandt B, Xyländer M, Patzelt C, Meyer T (2018) Group discussions in health services research – part 1: introduction and deliberations on selection of method and planning. Gesundheitswesen 80(10):864–870. https://doi.org/10.1055/s-0043-104379

Pugh J, Wilkinson D, Palacios-Gonzalez C, Savulescu J (2021) Beyond individual triage: regional allocation of life-saving resources such as ventilators in public health emergencies. Health Care Anal 29(4):263–282. https://doi.org/10.1007/s10728-020-00427-5

Reinacher-Schick A, Ebert MP, Piso P, Hüppe D, Schmitt J, Schildmann J (2023) Effects of the pandemic on the care of patients with colorectal cancer. Dtsch Arztebl Int 120(33–34):545–552. https://doi.org/10.3238/arztebl.m2023.0139

Reynolds KL, Klempner SJ, Parikh A, Hochberg EP, Michaelson MD, Mooradian MJ, Lee RJ, Soumerai TE, Hobbs G, Piotrowska Z, Sykes DB, Farago AF, Bardia A, Rosovsky RP, Ryan DP (2020) The art of Oncology: COVID-19 era. Oncologist 25(11):997–1000. https://doi.org/10.1634/theoncologist.2020-0512 . g

Rückher J, Mangiapane S, Seufferlein T, Pflüger M, Wesselmann S (2022) Auswirkungen Der Covid- 19-Pandemie auf die onkologische Versorgung. In: Klauber J, Wasem J, Beivers A, Mostert C (eds) Krankenhaus-Report 2022. Patientenversorgung während Der Pandemie. Springer, Berlin/Heidelberg, pp 109–126. https://doi.org/10.1007/978-3-662-64685-4_1

Schildmann J (2019) Value and values in cancer care. Assessing the benefit of treatment in patients with advanced cancer. Oncol Res Treat 42(1–2):35–40. https://doi.org/10.1159/000496119

Schnipper LE, Smith TJ, Raghavan D, Blayney DW, Ganz PA, Mulvey TM, Wollins DS (2012) American Society of Clinical Oncology identifies five key opportunities to improve care and reduce costs: the top five list for oncology. J Clin Oncol 30(14):1715–1724. https://doi.org/10.1200/JCO.2012.42.8375

Schöne-Seifert B, Friedrich DR (2013) Priorisierung Nach Dringlichkeit? Kritische Überlegungen Zur rule of rescue. In: Schmitz-Luhn B, Bohmeier A (eds) Priorisierung in Der Medizin. Kriterien Im Dialog. Springer, Berlin, pp 109–121

Schöne-Seifert B, Friedrich DR, Diederich A (2012) Rationing health care by thresholds for clinical benefit and its acceptance by the German population. Z Evid Fortbild Qual Gesundhwes 106(6):426–434. https://doi.org/10.1016/j.zefq.2012.06.008

Schreier M (2020) Fallauswahl. In: Mey G, Mruck K (eds) Handbuch qualitative Forschung in Der Psychologie. Springer Fachmedien, Wiesbaden, pp 238–251

Schug C, Geiser F, Hiebel N, Beschoner P, Jerg-Bretzke L, Albus C, Weidner K, Morawa E, Erim Y (2022) Sick leave and intention to quit the job among nursing staff in German hospitals during the COVID-19 pandemic. Int J Environ Res Public Health 19:1947. https://doi.org/10.3390/ijerph19041947

Schweizerische Akademie der Medizinischen Wissenschaften (SAMW) (2021) Intensive care triage under exceptional resource scarcity. Guidance on the application of Sect. 9.3 of the SAMS Guidelines «Intensive-care interventions» (2013). 2021. https://www.samw.ch/de/Ethik/Themen-A-bis-Z/Intensivmedizin/Intensivmedizin-Triage.html . Accessed 8 March 2024

Sommerlatte S, Lugnier C, Schoffer O, Jahn P, Kraeft AL, Kourti E, Michl P, Reinacher-Schick A, Schmitt J, Birkner T, Schildmann J, Herpertz S (2023) Mental burden and moral distress among oncologists and oncology nurses in Germany during the third wave of the COVID-19 pandemic: a cross-sectional survey. J Cancer Res Clin Oncol 149(9):6211–6223. https://doi.org/10.1007/s00432-023-04580-x

Stevens S (2020) Urgent next steps on NHS response to COVID-19. London. National Health Service (NHS). https://www.england.nhs.uk/coronavirus/wp-content/uploads/sites/52/2020/03/urgent-next-steps-on-nhs-response-to-covid-19-letter-simon-stevens.pdf . Accessed 24 May 2024

Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, Bray F (2021) Global Cancer statistics 2020: GLOBOCAN estimates of incidence and Mortality Worldwide for 36 cancers in 185 countries. CA Cancer J Clin 71(3):209–249. https://doi.org/10.3322/caac.21660

Tabur A, Elkefi S, Emhan A, Mengenci C, Bez Y, Asan O (2022) Anxiety, burnout and depression, psychological well-being as predictor of healthcare professionals’ turnover during the COVID-19 pandemic: study in a pandemic hospital. Healthcare 10(3):525. https://doi.org/10.3390/healthcare10030525

Tong A, Sainsbury P, Craig J (2007) Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 19(6):349–357. https://doi.org/10.1093/intqhc/mzm042

Trudeau J, Zewde S-W, Jae-in M, Ramaphosa C, Sánchez Pérez-Castejón P, Lofven S, Fakhfakh E (2020) The international community must guarantee equal global access to a covid-19 vaccine. The Washington Post. https://www.washingtonpost.com/opinions/2020/07/15/international-community-must-guarantee-equal-global-access-covid-19-vaccine/ . Accessed 19 July 2023

Wilkinson D (2020) ICU triage in an impending crisis: uncertainty, pre-emption and preparation. J Med Ethics 46(5):287–288. https://doi.org/10.1136/medethics-2020-106226

Zentrale Kommission zur Wahrung ethischer Grundsätze in der Medizin und ihren Grenzgebieten (Zentrale Ethikkommission, ZEKO) (2022) Medical responsibility at the limits of the appropriateness of medical measures. On dealing with futility. https://www.zentrale-ethikkommission.de/fileadmin/user_upload/zentrale-ethikkommission/BAEK_Futility_englisch.pdf . Accessed June 7 2024

Zentrale Kommission zur Wahrung ethischer Grundsätze in der Medizin und ihren Grenzgebieten (Zentrale Ethikkommission, ZEKO) (2000) Prioritäten in Der Medizinischen Versorgung Im System Der Gesetzlichen Krankenversicherung (GKV): Müssen und können wir uns entscheiden? Dtsch Arztebl 97(15):A1017–1023

Download references

Acknowledgements

We would like to thank all study participants for participating in interviews and group discussions, all colleagues and student researchers taking part in the pretest, Sophie Dahlke and Jonathan Bay for support regarding the descriptive data and content analysis of interviews and group discussions and Philip Saunders for providing language assistance.

Open Access funding was enabled and organized by Projekt DEAL. This work was supported by the German Federal Ministry of Education and Research (funding code 01KI20521A-C).

Open Access funding enabled and organized by Projekt DEAL.

Author information

Authors and affiliations.

Institute for History and Ethics of Medicine, Interdisciplinary Center for Health Sciences, Medical Faculty, Martin Luther University Halle-Wittenberg, Magdeburger Str. 8, 06112, Halle (Saale), Germany

Sabine Sommerlatte, Stephan Nadolny & Jan Schildmann

Center for Evidence-Based Healthcare, Medical Faculty and University Hospital Carl Gustav Carus , TUD Dresden University of Technology, Dresden, Germany

Helene Hense, Jochen Schmitt & Olaf Schoffer

Institute for Educational and Health-Care Research in the Health Sector, Faculty of Health, Hochschule Bielefeld - University of Applied Sciences and Arts, Interaktion 1, 33619, Bielefeld, Germany

Stephan Nadolny

Department of Hematology, Oncology and Palliative Care, St. Josef Hospital, Ruhr University Bochum, Bochum, Germany

Anna-Lena Kraeft, Celine Lugnier & Anke Reinacher-Schick

You can also search for this author in PubMed   Google Scholar

Contributions

S.S.: Formal analysis (lead), investigation (equal), writing – original draft (lead), writing – review & editing (lead). H.H.: Formal analysis (equal), investigation (equal), writing – review & editing (equal). S.N.: Formal analysis (equal), writing – review & editing (equal). AL. K.: Resources (equal), writing – review & editing (equal). C.L.: Resources (equal), writing – review & editing (equal). Jo.S.: Funding acquisition (equal), supervision (equal), writing – review & editing (equal). O.S.: Funding acquisition (equal), writing – review & editing (equal). A.R-S.: Funding acquisition (equal), resources (equal), supervision (equal), writing – review & editing (equal). Ja. S.: Formal analysis (equal), funding acquisition (lead), investigation (equal), resources (equal), supervision (lead), writing – review & editing (equal). All authors reviewed the manuscript and approved the final version of the manuscript to be published and agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Corresponding author

Correspondence to Sabine Sommerlatte .

Ethics declarations

Ethics approval, consent to participate and study registration.

This study was conducted in accordance with the principles of the Declaration of Helsinki. All procedures were performed in compliance with relevant laws and institutional guidelines and have been approved by the Ethics Committee of the medical faculty of Martin Luther University Halle-Wittenberg on December 12, 2020 (reference number: 2020 − 209). All study participants provided informed consent for participation. This study was registered in the German Clinical Trials Register (trial number: DRKS00026043).

Competing interests

Unrelated to this study, Olaf Schoffer worked as a paid consultant for Novartis. The other authors have declared no conflicts of interest.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Sommerlatte, S., Hense, H., Nadolny, S. et al. What does “urgency” mean when prioritizing cancer treatment? Results from a qualitative study with German oncologists and other experts during the COVID-19 pandemic. J Cancer Res Clin Oncol 150 , 352 (2024). https://doi.org/10.1007/s00432-024-05863-7

Download citation

Received : 16 April 2024

Accepted : 19 June 2024

Published : 15 July 2024

DOI : https://doi.org/10.1007/s00432-024-05863-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Empirical bioethics
• Cancer care
• Resource allocation
• Prioritization criteria
• Find a journal
• Publish with us
• Track your research

This paper is in the following e-collection/theme issue:

Published on 15.7.2024 in Vol 26 (2024)

Factors Associated With Continuous Use of a Cancer Education Metaverse Platform: Mixed Methods Study

Authors of this article:

Original Paper

• Sunghak Kim 1 , PhD   ; 
• Timothy Jung 2 , PhD   ; 
• Dae Kyung Sohn 3 , MD, PhD   ; 
• Mina Suh 4 , MD, PhD   ; 
• Yoon Jung Chang 1 , MD, PhD  

1 National Cancer Survivorship Center, National Cancer Control Institute, National Cancer Center, Goyang, Republic of Korea

2 Faculty of Business and Law, Manchester Metropolitan University, Manchester, United Kingdom

3 Center for Colorectal Cancer, National Cancer Center, Goyang, Republic of Korea

4 Division of Cancer Early Detection, National Cancer Control Institute, National Cancer Center, Goyang, Republic of Korea

Corresponding Author:

Yoon Jung Chang, MD, PhD

National Cancer Survivorship Center

National Cancer Control Institute

National Cancer Center

323 Ilsan-ro

Ilsandong-gu

Goyang, 10408

Republic of Korea

Phone: 82 10 8729 5835

Email: [email protected]

Background: Early detection of cancer and provision of appropriate treatment can increase the cancer cure rate and reduce cancer-related deaths. Early detection requires improving the cancer screening quality of each medical institution and enhancing the capabilities of health professionals through tailored education in each field. However, during the COVID-19 pandemic, regional disparities in educational infrastructure emerged, and educational accessibility was restricted. The demand for remote cancer education services to address these issues has increased, and in this study, we considered medical metaverses as a potential means of meeting these needs. In 2022, we used Metaverse Educational Center, developed for the virtual training of health professionals, to train radiologic technologists remotely in mammography positioning.

Objective: This study aims to investigate the user experience of the Metaverse Educational Center subplatform and the factors associated with the intention for continuous use by focusing on cases of using the subplatform in a remote mammography positioning training project.

Methods: We conducted a multicenter, cross-sectional survey between July and December 2022. We performed a descriptive analysis to examine the Metaverse Educational Center user experience and a logistic regression analysis to clarify factors closely related to the intention to use the subplatform continuously. In addition, a supplementary open-ended question was used to obtain feedback from users to improve Metaverse Educational Center.

Results: Responses from 192 Korean participants (male participants: n=16, 8.3%; female participants: n=176, 91.7%) were analyzed. Most participants were satisfied with Metaverse Educational Center (178/192, 92.7%) and wanted to continue using the subplatform in the future (157/192, 81.8%). Less than half of the participants (85/192, 44.3%) had no difficulty in wearing the device. Logistic regression analysis results showed that intention for continuous use was associated with satisfaction (adjusted odds ratio 3.542, 95% CI 1.037-12.097; P =.04), immersion (adjusted odds ratio 2.803, 95% CI 1.201-6.539; P =.02), and no difficulty in wearing the device (adjusted odds ratio 2.020, 95% CI 1.004-4.062; P =.049). However, intention for continuous use was not associated with interest (adjusted odds ratio 0.736, 95% CI 0.303-1.789; P =.50) or perceived ease of use (adjusted odds ratio 1.284, 95% CI 0.614-2.685; P =.51). According to the qualitative feedback, Metaverse Educational Center was useful in cancer education, but the experience of wearing the device and the types and qualities of the content still need to be improved.

Conclusions: Our results demonstrate the positive user experience of Metaverse Educational Center by focusing on cases of using the subplatform in a remote mammography positioning training project. Our results also suggest that improving users’ satisfaction and immersion and ensuring the lack of difficulty in wearing the device may enhance their intention for continuous use of the subplatform.

Introduction

Breast cancer is one of the most common cancers affecting women. Survival rates can increase with early detection, which is linked to proper treatment [ 1 , 2 ]. Mammography is a widely accepted method for early detection of breast cancer that enables health professionals to detect tumors in patients with no signs or symptoms of the disease [ 3 , 4 ]. High-quality mammograms are essential for the successful early detection of breast cancer, and mammography positioning is one of the key elements in securing high-quality mammograms. Thus, mammography positioning training for radiologic technologists is crucial for improving the feasibility of successful early detection of breast cancer [ 5 ]. However, mammography education is challenging because of the lack of material resources [ 6 ]. One possible reason for the lack of material resources is that it is difficult to observe other people using mammography directly or to obtain realistic training resources. While other cancer screening education fields, such as endoscope cleaning and disinfection method training, are relatively easy to demonstrate externally, mammography positioning training is relatively difficult to demonstrate externally because of its association with personal privacy issues and sensitivity. In addition, the outbreak of infectious diseases such as COVID-19 and the disparity in cancer screening and diagnosis educational systems among medical institutions have led to a decline in access to excellent cancer education [ 7 , 8 ]. Overcoming these problems requires efficient digital health care services that enable comprehensive, systematic, and continuous remote cancer education. In particular, producing material resources for mammography positioning training using metaverse technology may help alleviate the challenges of the lack of material resources in the real world. Therefore, this study selected remote mammography positioning training in the metaverse subplatform as its research topic.

National Cancer Center (NCC) Korea’s Division of Cancer Early Detection runs a national cancer screening support project aimed at increasing the cure rate and reducing cancer-related mortality by inducing early cancer detection and treatment. The division leads regionally driven quality improvement activities of national cancer screening to enhance its accuracy and reliability and to perform regular training in each region to reduce regional disparities in the quality of national cancer screening [ 9 , 10 ]. The division provides various remote educational programs and content for training purposes, including non–face-to-face virtual reality (VR) educational videos for mammography positioning training. However, the previous way of providing VR educational videos to each regional cancer center (RCC) required significant manual effort during the delivery process. The division recognized the need for a digital platform to facilitate content delivery and management and continuous non–face-to-face education.

The need for a digital platform for the early detection of cancer aligned well with the Dr. Meta metaverse platform’s need for content variety. We developed a multipurpose metaverse digital cancer care platform, Dr. Meta, at NCC Korea in 2021, supported by the Republic of Korea’s Ministry of Science and ICT [ 11 ]. The Dr. Meta metaverse platform showed potential for successful cancer control; however, according to participants’ feedback, its weakness was the lack of content diversity [ 11 ]. In 2022, supported by the Republic of Korea’s Ministry of Health and Welfare, we upgraded the metaverse platform to improve the Dr. Meta user experience by embedding new educational VR content in the Metaverse Educational Center subplatform and adding a new VR Healing Theater subplatform under the Dr. Meta platform [ 12 ]. Especially, the Metaverse Educational Center subplatform, which helps users overcome physical constraints (eg, time and space) in education and conveniently receive training on demand, was developed to actively respond to the demand for non–face-to-face services in the post–COVID-19 era. The subplatform is equipped with multimedia so that users can upload and interact with various medical training materials, ranging from typical documents, images, videos, presentation slides, and webpages to 3D object data and VR content created using 360° video technology. In this interactive environment, users can directly control and interact with the subplatform’s interface, virtual objects, and spatial elements and be more immersed with the subplatform. Users can also look in any direction while playing the educational VR content and have immersive experiences in this subplatform [ 11 , 12 ]. As the previous studies show that immersion and interaction are associated with the intention to continuously use the VR services, it is anticipated that such improved immersion may positively affect the intention for continuous use of the subplatform [ 13 , 14 ]. However, improving the diversity of content in the subplatform was a challenge [ 11 ]. Consequently, we collaborated with the Division of Cancer Early Detection of NCC Korea in 2022 using Metaverse Educational Center to virtually educate radiologic technologists working at each RCC on mammography positioning.

Previous studies have shown that the metaverse can serve as a promising tool in cancer education [ 15 - 17 ]. However, to effectively use the tool, it is insufficient to merely know the positive relationship between the metaverse and cancer education. Identifying the factors that may affect the use of metaverse services is important for creating successful strategies for developing, upgrading, and operating them and maximizing their effectiveness. Numerous studies have explored possible factors that can impact the use of metaverse technology, frequently applying the technology acceptance model (TAM) to identify meaningful factors [ 18 - 20 ]. On the basis of the TAM, perceived usefulness and perceived ease of use of technology impact satisfaction and, subsequently, intention to use [ 21 ]. In this study, we considered TAM-based factors when developing a metaverse platform and evaluating the user experience. In the context of remote mammography positioning training for radiologic technologists, this study goes beyond the simple usability test of Metaverse Educational Center and examines the factors and relationships associated with using the subplatform for an advanced understanding of its use.

Study Objectives

This study investigated the user experience of the Metaverse Educational Center subplatform and the factors associated with the intention for continuous use by focusing on cases of using the subplatform in a remote mammography positioning training project. We believe that Metaverse Educational Center is an outstanding digital tool for cancer education. As there are other educational VR contents being produced for home-based, hospice, non–face-to-face practice training and endoscope cleaning and disinfection method training in the Metaverse Educational Center subplatform, this study can serve as evidence of this subplatform’s wide applicability across various training topics and target groups if positive outcomes are found. Furthermore, we expected that study outcomes suggest practical strategies for improving the quality of Metaverse Educational Center and facilitating its successful dissemination and implementation.

Study Design

After demonstrating the Metaverse Educational Center subplatform of the Dr. Meta metaverse platform to participants, we conducted a multicenter, cross-sectional survey to examine their experiences with the subplatform. Quantitative data were collected through Likert-scale survey questions, and qualitative data were collected through an open-ended question.

In 2022, we disseminated Dr. Meta to 12 RCCs (ie, Ajou University Hospital, Chonnam National University Hwasun Hospital, Chungbuk National University Hospital, Chungnam National University Hospital, Gachon University Gil Medical Center, Gyeongsang National University Hospital, Jeju National University Hospital, Jeonbuk National University Hospital, Kangwon National University Hospital, Kyungpook National University Chilgok Hospital, Pusan National University Hospital, and Ulsan University Hospital). On the basis of goal of positively contributing to the efficiency of national cancer screening through VR education, we collaborated with NCC Korea’s Division of Cancer Early Detection and installed the Division of Cancer Early Detection VR educational videos for mammography positioning training within our Metaverse Educational Center subplatform. After preparing for the launch of the remote mammography positioning training program, we conducted virtual sessions for radiologic technologists through the Metaverse Educational Center subplatform in cooperation with the Division of Cancer Early Detection. The survey was conducted after the completion of virtual sessions.

Study Participants

Metaverse Educational Center usability tests were conducted at 13 cancer centers (the NCC and 12 RCCs). Participants aged >19 years were recruited from each cancer center between July and December 2022. Inclusion criteria for the study were having experience in using Metaverse Educational Center and being aged >19 years. Exclusion criteria for the study were having experience in using the subplatform but being aged ≤19 years, having a severe physical or mental condition that prevented them from participating in the study, being unable to complete the self-reported survey, not consenting to the survey, or withdrawing consent during the study. The remote mammography positioning training program was designed to educate radiologic technologists, so most study participants were radiologic technologists. Doctors, nurses, and hospital administrative staff also participated in this study. Figure 1 shows images of participants using Metaverse Educational Center.

Ethical Considerations

The study was approved by the institutional review board of NCC Korea (IRB: NCC2022-0209). Only those who read the research information and voluntarily consented to participate in the survey were able to participate. Data used in this study were anonymous because participants were not asked for identifying information (eg, name and social security number). Users who participated in the study received an everyday item worth approximately ₩7000 (US $5) as a reward.

In the survey preparation phase, we asked platform users about their impressions of the initially prepared questionnaire. Their responses conveyed a preference to mitigate perceived burdens when participating in the survey after having professional virtual training through new metaverse technology. Therefore, we used theory-based but easy and short survey questions to facilitate participants’ understanding of the questions among diverse participant groups. The survey questions were adapted from previous studies examining the TAM in the context of metaverse services [ 11 , 12 , 22 ], translated into Korean, abbreviated, and modified to suit the purpose and context of the study. In addition, we adopted the viewpoint of separating the experience of technology use into hardware and software aspects [ 22 , 23 ]. Researchers with expertise in communication, metaverse, and health translated survey questions from English into Korean, followed by consensus to determine the most suitable translation. Subsequently, the questionnaire was reviewed by an expert proficient in both Korean and English, who was separate from the research team, to complete the final version. First, participants were asked to provide their sociodemographic information (age, sex, and position in the hospital). Next, they were asked to assess their individual differences (2 items: “I am usually interested in using new technologies or devices” and “I tend to not experience dizziness or motion sickness easily”) and user experiences (8 items: “I was generally satisfied with using this subplatform,” “This subplatform made me feel interested,” “This subplatform made me feel immersed,” “This subplatform was easy to use,” “There was no discomfort (eg, dizziness and nausea) in using this subplatform,” “There was no difficulty in wearing this subplatform device (ie, head-mounted display and controllers),” “There was no difficulty in operating this subplatform device (ie, head-mounted display and controllers),” and “I want to continue using this subplatform in the future”) relevant to the use of Metaverse Educational Center. Each item was scored on a 4-point Likert scale (1=“strongly disagree” to 4=“strongly agree”). Finally, participants were asked to provide their feedback to improve Metaverse Educational Center by answering the open-ended question (“If you have any suggestions or improvements, please write them in”). We used these 10 quantitative questions and the open-ended qualitative question to evaluate participants’ experiences with Metaverse Educational Center . The questionnaire used in this study is presented in Multimedia Appendix 1 .

Statistical Analysis

First, we conducted a descriptive analysis to examine how participants perceived their experiences using Metaverse Educational Center. The user experience was rated by obtaining the frequency and proportion of participants with positive opinions for each survey question. The number of participants with positive opinions after using Metaverse Educational Center was calculated by adding those who answered “agree” and “strongly agree.” Next, we performed a logistic regression analysis to identify the variables associated with the intention to continue using the subplatform. To create a binary dependent variable, we categorized participants’ reaction values to a survey question relevant to intention for continuous use into 2 groups (“strongly disagree”+“disagree” vs “agree”+“strongly agree”). Univariable logistic regression analyses were performed for each variable to obtain odds ratios with 95% CIs of intention for continuous use. A multiple logistic regression analysis was performed to identify factors related to this dichotomous dependent variable using only variables with statistical significance from the univariable logistic regression analyses. Finally, we qualitatively reviewed the feedback acquired from the open-ended question on the improvement of the subplatform. To identify and interpret the main themes and patterns within the feedback data with the researchers’ subjectivity, we conducted a reflexive thematic analysis by following the 6 phases of analysis (familiarizing yourself with the data set; coding; generating initial themes; developing and reviewing themes; refining, defining, and naming themes; and writing up) [ 24 - 26 ]. We read and reread the data to become familiar with its content. After that, we captured important data content and features relevant to the user experience of the subplatform and generated them as several codes. On the basis of collated codes, we generated and refined each theme in a recursive process and finalized its name to tell the best story of the data. We developed such a story into a discussion of the advantages and improvements of the subplatform and suggested future directions for upgrading the subplatform. The research team has multidisciplinary academic backgrounds in communication, metaverse, cancer prevention, cancer screening, and cancer treatment and has various experiences conducting digital health research. This researcher reflexivity may have influenced data analysis. In addition, we applied a content analysis approach to quantify the qualitative feedback data and assist the discussion derived from the reflexive thematic analysis. Data analysis was conducted using IBM SPSS Statistics V22.0 (IBM Corp).

Participant Characteristics

Overall, 196 individuals participated in the survey after using the Metaverse Educational Center subplatform, and data from 192 participants were analyzed. Among the 192 participants, 74 (38.5%), 69 (35.9%), 38 (19.8%), and 11 (5.7%) were aged 20 to 29, 30 to 39, 40 to 49, and 50 to 59 years, respectively. In addition, 16 (8.3%) participants were male, and 176 (91.7%) were female. Regarding position in the hospital, most participants (160/192, 83.3%) fell into the “Health professionals (other)” category. These results can be attributed to mammography positioning training for radiologic technologists using Metaverse Educational Center in collaboration with the Division of Cancer Early Detection of NCC Korea in 2022. Table 1 presents participants’ demographic characteristics.

a Percentages may not add up to 100% due to rounding.

b “Health professionals (other)” includes radiologic technologists, physical therapists, and pharmacists in this study.

c “Other” includes hospital administrative staff, educators, social workers, and researchers in this study.

Participants’ Experiences Using the Metaverse Educational Center Subplatform

We conducted a quantitative descriptive analysis to examine how participants perceived their experiences using Metaverse Educational Center. Our results showed that most participants (178/192, 92.7%) were satisfied with their user experience of Metaverse Educational Center. Most participants responded that Metaverse Educational Center was an interesting (184/192, 95.8%) and immersive (174/192, 90.6%) subplatform. A total of 138 (71.9%) participants reported that the subplatform was easy to use. Most participants (157/192, 81.8%) wanted to continue using the subplatform in the future. Table 2 presents the overall positive experiences of Metaverse Educational Center’s participants.

a Items are based on a 4-point scale (ranging from 1=“strongly disagree” to 4=“strongly agree”). The number of participants who had positive opinions after using Metaverse Educational Center was calculated by adding the number of participants who answered “agree” and “strongly agree.”

b Percentages may not add up to 100% due to rounding.

Factors Associated With Intention for Continuous Use of the Metaverse Educational Center Subplatform

We conducted a quantitative logistic regression analysis to identify the variables associated with the intention to continue using the subplatform. In univariable logistic regression analyses, a usual interest in using new technologies or devices (odds ratio 1.998, 95% CI 1.141-3.497; P =.02), satisfaction (odds ratio 8.079, 95% CI 3.284-19.875; P <.001), interest (odds ratio 3.000, 95% CI 1.570-5.731; P =.001), immersion (odds ratio 5.753, 95% CI 2.985-11.088; P <.001), perceived ease of use (odds ratio 2.973, 95% CI 1.741-5.077; P <.001), no discomfort in use (odds ratio 2.021, 95% CI 1.287-3.173; P =.002), and no difficulty in wearing the device (odds ratio 2.494, 95% CI 1.448-4.296; P =.001) were statistically different between participants not having intention for continuous use and participants having intention for continuous use. In the multiple logistic regression analysis, participants who were generally satisfied with using Metaverse Educational Center were more likely to intend to continue use the subplatform than those who were not (adjusted odds ratio 3.542, 95% CI 1.037-12.097; P =.04). When immersion increased by one unit, users were 2.803 times more likely to have intention for continuous use (adjusted odds ratio 2.803, 95% CI 1.201-6.539; P =.02). Having no difficulty in wearing the device was positively associated with intention for continuous use (adjusted odds ratio 2.020, 95% CI 1.004-4.062; P =.049). Table 3 presents the results of the univariable and multiple logistic regression analyses for each variable.

a Not applicable.

Feedback on the Metaverse Educational Center Subplatform

We conducted a qualitative feedback review to explore participants’ diverse opinions regarding the use of the subplatform. Participants answered the open-ended question “If you have any suggestions or improvements, please write them in.” We broke down the 145 participants’ responses at the sentence-unit level and collected 276 sentences. We then sorted these sentences into groups with homogeneous attributes and labeled them accordingly. Participants provided a wide range of feedback on the advantages and improvements of Metaverse Educational Center. Through the reflexive thematic analysis, we identified several important themes related to the subplatform’s advantages (eg, “immersion and concentration,” “usefulness,” and “satisfaction”) and understood how they are manifested in the data. We also identified various themes related to the subplatform’s improvements in hardware (eg, “heaviness of the device,” “difficulties in wearing the device,” and “difficulties in operating the device”) and software (eg, “deficiency of interactivity in the content,” “deficiency of content variety,” and “poor qualities of the images”) aspects and understood how they relate to one another. In addition, the frequency of each theme was calculated by applying the content analysis approach to complement the outcomes of the reflexive thematic analysis. Regarding the subplatform’s advantages (21 sentences), the most commonly reported label was “usefulness.” Regarding the subplatform’s improvements (255 sentences), the most commonly reported one was “deficiency of interactivity in the content.” “Deficiency of content variety” and “heaviness of the device” were other representative improvements mentioned by participants. Table 4 presents more information about the feedback on the participants’ experiences with Metaverse Educational Center.

Principal Findings

This study investigated the user experience of the Metaverse Educational Center subplatform and the factors associated with intention for continuous use by focusing on cases of using the subplatform in a remote mammography positioning training project. As securing a sufficient number of participants was difficult in the first study using the Dr. Meta metaverse platform [ 11 ], we were able to examine this metaverse platform’s user experience but were unable to reveal more detailed relationships among these factors. However, in this study, we secured more participants than in the first study of Dr. Meta use [ 11 ] by inviting new RCCs to our project and recruiting participants for an extended period. We also installed specific purpose-driven VR content produced for use in an actual cancer education project in cooperation with the Division of Cancer Early Detection of NCC Korea. On the basis of the increased sample size and new metaverse cancer education content, this study targeted the subplatform in an obvious context, complemented the methodology, and addressed its user experience in various ways.

Our results, focusing on cases of using Metaverse Educational Center in a remote mammography positioning training project, showed the positive user experience of the subplatform. Most users considered the subplatform satisfactory, interesting, and immersive. More than half of the users considered the subplatform easy to use. Although less than half of the users indicated positive opinions regarding the lack of difficulty in wearing this subplatform device, more than half of the users expressed positive opinions on the lack of discomfort in using the subplatform and lack of difficulty in operating its device. Finally, most users wanted to use the subplatform continuously. The positive user experience of the subplatform indicates that it can also be successfully used in another project of the Division of Cancer Early Detection (endoscope cleaning and disinfection method training). We intend to enhance the scalability of the subplatform and its use in the future by developing metaverse content covering various cancer education topics and mounting them into the subplatform.

Our study’s outcomes demonstrated factors related to the intention to continue using Metaverse Educational Center. According to the logistic regression analysis results, the associations between intention for continuous use of Metaverse Educational Center and satisfaction, immersion, and lack of difficulty in wearing the device were statistically significant. In previous studies, satisfaction, immersion, and lack of difficulty in wearing the device have been identified as factors directly or indirectly related to the intention to continuously use metaverse or other interactive media services [ 13 , 27 , 28 ]. Our results are consistent with those of previous studies. A higher level of satisfaction may increase the likelihood of intention for continuous use [ 22 , 27 ]. Users immersed in metaverse content may lose their self-consciousness, concentrate intensely on the content, and become satisfied with it [ 13 , 29 ]. Users may pursue an easy and relaxed experience of wearing the device, and, therefore, they are more likely to use the device if it is not difficult to wear continuously [ 28 , 30 , 31 ].

Conversely, the associations between the intention for continuous use of Metaverse Educational Center and interest, perceived ease of use, and lack of discomfort in use were not statistically significant. No associations of intention for continuous use with interest, perceived ease of use, or discomfort in use were unexpected because their associations (whether direct or indirect) have often been observed in previous studies [ 32 - 34 ]. These discrepancies may be due to different contextual characteristics. This study provided remote mammography positioning training to health professionals. Therefore, users may have had a strong motivation to learn skills, and the content may have been produced to deliver professional knowledge as a top priority. Entertainment, ease of use, and comfort may have weak relevance to the strong purposefulness of training; interest, perceived ease of use, and lack of discomfort in use may not be associated with the intention for continuous use of the educational metaverse platform. In addition, many younger individuals known to be familiar with the metaverse [ 35 ] and many radiologic technologists known to be familiar with complex medical devices [ 36 ] participated in this study. We assume that they tend to use emerging technologies to accomplish goals frequently; that is, they have high technology readiness [ 37 ]. However, contrary to the expectation of positive associations [ 38 , 39 ], neither age group nor position in the hospital showed a significant association with the intention for continuous use of the Metaverse Educational Center subplatform. Although younger participants may be familiar with metaverse services for purposes such as entertainment, they may not have been familiar with metaverse services for medical training. Furthermore, radiologic technologists may be familiar with the medical devices they usually handle but may be unfamiliar with metaverse devices, which can be considered relatively cutting-edge and perceived differently from other medical devices. This discussion implies that scrutinizing external variables such as the characteristics of the target period, target population group, and other situational contexts is vital when exploring factors associated with the intention for continuous use of metaverse services, including the Metaverse Educational Center subplatform. Various media theories, including the TAM, have evolved to include external variables for better explanatory power [ 40 ]. Therefore, subsequent research is needed to delve into the mixed results regarding the associations of intention for continuous use contingent on external variables to understand the reasons behind them.

Implications

Discovering the factors associated with the intention to continuously use the Metaverse Educational Center subplatform has both theoretical and practical implications. With regard to theoretical implications, knowledge of these statistically supported associations may trigger research into the mechanisms or conditions of associations. Applying mixed methods approaches directly ask participants deeper about quantitative results regarding which factors are associated, in what direction, and why it can be one of the future research directions. Such attempts to understand the patterns of association may lead to the development of a systematic theoretical framework. Regarding practical implications, information about factors related to the intention for continuous use of the subplatform can guide the development of more effective intervention strategies that encourage its continuous use. It will be feasible to save time and cost for upgrading the subplatform and promoting its user experience by prioritizing factors that are likely to affect intention for continuous use.

Furthermore, the feedback on Metaverse Educational Center–shaped discourse pointed to future directions for upgrading the subplatform. Regarding the subplatform’s advantages, the total number of relevant sentences was smaller than that of sentences related to the subplatform’s improvements because the question did not directly ask about the advantages. However, the usefulness of the subplatform was remarkable. As the usefulness of the Metaverse Educational Center subplatform is proven, it will be possible to increase the quantity and quality of educational content helpful in gaining knowledge with varying topics and formats and expand the range of use of the subplatform. Such upgrading strategies can facilitate dynamic virtual training and activate the use of the subplatform among various user groups. Regarding the subplatform’s improvements, the deficiency of interactivity in the content, deficiency of content variety, and heaviness of the device were the main weaknesses. Poor screen focus and image quality, which lead to negative visual experiences, were also notable weaknesses. Despite upgrading the Dr. Meta metaverse platform based on the results of the first study to use the platform [ 11 ], the results of this study indicate that further quantitative and qualitative enhancements are needed for the content and devices of the subplatform. To improve the usability of the subplatform, it is crucial to address these weaknesses by securing diverse interactive metaverse content, solving the device’s weight problem, and improving functions related to screen focus control and image quality.

In addition, the feedback substantially supported the results of the usability test. The outcomes of the descriptive and logistic regression analyses were similar to those of the responses to the feedback question. More than half of the users thought that the Metaverse Educational Center subplatform was satisfactory, interesting, immersive, and easy to use. Less than half of the users believed that there was no difficulty in wearing the subplatform device. The former was addressed in the feedback as an advantage, whereas the latter was mentioned as an improvement. Factors associated with the intention for continuous use of the subplatform (ie, satisfaction, immersion, and lack of difficulty in wearing the device) also appeared in the feedback. Beyond emphasizing quantitative results, the feedback also clarifies new elements that received little attention but might be related to the user experience with the subplatform. For example, improving the audio system, reducing the runtime of content, and optimizing the operation of content would attract users’ motivation to experience the subplatform.

The feedback also suggests that certain advantages and improvements may be closely related. For example, some participants found it easier to understand the educational content through immersion (eg, “The content was easily understandable due to high immersion” [immersion and concentration - ease of understanding]), and some were overall satisfied with the subplatform because the educational content was beneficial to them (eg, “I was satisfied with this subplatform because I could get helpful information” [usefulness - satisfaction]). Some participants complained of dizziness, fatigue, and distraction due to the lack of breaks, lengthy videos, and blurry image quality when receiving training through the subplatform (eg, “The educational video was too long, so it was unable to rest for a long time. Even the image quality was not sharp. I was dizzy, tired, and had decreased concentration after finishing the training” [poor qualities of the images; lengthy videos - dizziness, fatigue, and distraction]). Some also expressed that wearing the device was challenging due to its heavy weight (eg, “The device was heavy and easy to slip off, making it difficult to wear” [heaviness of the device - difficulties in wearing the device]). Some connected the difficulties in wearing the device with wearing glasses (eg, “It was hard to wear the device to the extent that I had to take off my glasses during the education session. To make matters worse, the poor quality of the educational content’s image disrupted my learning” [difficulties in wearing the device; there was no specific categorized label about wearing glasses]). Such feedback can serve as indirect evidence for inferring new potential factors, their relationships, and directions. Determining whether these ideas are statistically valid could also be a topic for future research. By probing the feedback, we could reconfirm the results obtained from statistical methods and deal with more profound levels of user experience statements that are seldom obtained from quantitative approaches.

Future Perspective

This study used a mixed methods design to comprehensively understand the usability of the Metaverse Educational Center subplatform while embracing diverse user experiences. Through quantitative approaches, this study suggests that the upgrade direction of improving factors associated with the intention for continuous use of the subplatform, that is, immersion and lack of difficulty in wearing the device, may effectively enhance the user experience and lead to future successful effects. In addition, through qualitative approaches, this study identified ideas for materializing this upgrade direction.

First, enhancing users’ perceived interactivity is an option to increase immersion. Studies have shown that perceived interactivity of new media, including metaverse and other interactive media, is positively associated with immersion [ 41 - 43 ]. In this study, although most participants thought that the subplatform was immersive, there was also considerable feedback indicating that participants wished for the educational content to be more interactive. Adding content scenarios and features that allow users to directly manipulate the 3D equipment needed for mammography or correct patients’ mammography positioning in a virtual environment through the switch buttons on the subplatform device would help increase perceived interactivity. The incorporation of participatory content that induces user engagement and sensory expansion through hands-on practice in a virtual space after theoretical education would also be beneficial. By improving interactive content in which users can directly control virtual spaces and digital objects relevant to mammography positioning training, it is possible to enhance the immersion of the subplatform beyond its current level and sustainably increase intention for continuous use.

Next, reducing the device weight is an option to decrease the difficulty in wearing the device. Previous studies have asserted that the heavy weight of a head-mounted display makes it difficult to wear and serves as a barrier preventing the activation of VR service [ 28 , 31 ]. In this study, less than half of the participants thought there was no difficulty in wearing the subplatform device, and there was considerable feedback claiming difficulties in wearing the device. Some feedback indicated that it was uncomfortable to wear the device because of its heavy weight, allowing us to infer the relationship between difficulties in wearing the device and its heaviness. Other feedback also mentioned that the subplatform device pressed on the nose or placed strain on the neck with fatigue, which seemed to be caused indirectly by the heaviness of the device. The difficulties in wearing the device and its heaviness issues were already acknowledged in the first Dr. Meta study [ 11 ]. Some improvements had been made using accessories to lighten the device weight; however, the results of this study revealed that further improvements are needed. If the difficulties in wearing the device cannot be adequately solved by improving the head-mounted displays, accessories, or peripheral devices, replacing them with entirely new devices could also be an option. If the disadvantages of a head-mounted display, such as difficulties in wearing it [ 28 ], outweigh its advantages, such as greater immersion by blocking additional visual inputs and other external stimuli [ 44 ], it may be more efficient to mitigate the disadvantages, even at the expense of some advantages. Overall engagement may slightly decrease if the subplatform is experienced on existing devices, such as desktops, laptops, cellphones, or tablets. However, the engagement inherent in the characteristics of metaverse content will be maintained, and difficulties in wearing the device will largely disappear. Most importantly, if users are given the choice of device to use for the subplatform by themselves, they can weigh the pros and cons and select it according to their needs and preferences. As a self-tailored experience, this may increase user satisfaction [ 45 ]. As difficulties in wearing the device may impair immersion [ 28 ], it would be beneficial to develop web and app modes and compatible modes to allow users to use the subplatform on familiar or mobile devices. Development of a new use model that can be run on different portable devices can alleviate difficulties in wearing the device, improve accessibility to the educational content, and complement the existing use model in diffusing the subplatform.

In this study, although not addressed by quantitative approaches, we obtained specific feedback related to the deficiency of content variety, following the first Dr. Meta study [ 11 ]. Participants wanted to receive not only mammography positioning training but also other mammography-related education through the Metaverse Educational Center subplatform and showed interest in cancer education on different topics; meeting these demands requires diverse scenarios, digital objects, and virtual environments. Applying artificial intelligence (AI) technology can effectively solve deficiencies in content variety. Scholars argue that embedding generative AI that creates images and 3D digital objects into the metaverse can save the time, energy, and costs associated with content development [ 46 , 47 ]. Moreover, using AI to collect and analyze metaverse use data can further advance service automation and personalization [ 48 , 49 ]. If AI combined with a metaverse captures and analyzes user experience and choice data, it can dynamically change virtual backgrounds and digital objects in real time to match user needs. Nonetheless, many hurdles remain in successfully integrating the metaverse and AI. The controversy related to copyright issues may limit the use of AI for content development in the metaverse. Legal deliberations and discussions are imperative to determine the ownership of copyrights concerning medical education content generated by AI, whether they reside in the final product or the source materials used for AI training, alongside their respective proportions. Data privacy violation is also a potential challenge for using AI to collect and analyze metaverse use data. Defining the scope of data use and establishing data protection methods are indispensable throughout the data collection, processing, and application phases [ 49 ]. It is essential to establish policies and ethics related to the use of metaverses and AI, and data security must be ensured. If these prerequisites are met, the industry will rapidly accelerate toward building data-driven metaverse systems using AI in the future. A successful combination of metaverses and AI may also appeal to participants to continuously use the subplatform.

Finally, the positive study outcomes in the user experience of the Metaverse Educational Center subplatform prove the extension of using this subplatform widely in not only various training topics and target groups but also different geographical locations. At the beginning stage of developing the Dr. Meta metaverse platform, we needed to consider regional disparities in medical resources and information and communication technologies to facilitate a multicenter cancer education environment using metaverse technology. However, we conducted research to demonstrate an example where multiple medical institutions could more actively use the metaverse by overcoming such physical constraints. We have established the Dr. Meta metaverse platform’s infrastructure and maintained its setting to disseminate and implement it in local communities. We aim to construct a more robust remote cancer education network with each RCC and other hospitals in multiple regions of the Republic of Korea by using the Metaverse Educational Center subplatform. In addition, we plan to develop an English version of Dr. Meta and intend to promote collaboration with foreign institutions. We anticipate that these future research directions will contribute to enhancing the generalizability of our study findings.

Limitations

Although this study presented positive assessments of the cancer education metaverse platform and outlined prospects for its future development, several limitations exist. First, our results may be specific to a particular context, topic, or population, making it challenging to generalize the results to the overall level of education through a medical metaverse. In this study, we used the subplatform only for the remote mammography positioning training of health professionals. Hence, it is difficult to anticipate whether contextual differences affect our results. For instance, if the purpose of cancer education is not to train health professionals but to teach cancer care methods to patients or cancer prevention methods to the general public, the factors associated with intention for continuous use might differ. Demographic information, such as sex and age distribution ratios, might also vary, potentially affecting research outcomes. Further studies are needed to confirm whether the results of this study can be reproduced in the general population or other specific population groups.

Second, as this study used cross-sectional data, not all observed relationships among variables are actual causalities but associations; therefore, careful interpretation of causal assumptions is necessary. Although mixed methods can supplement explanations of the possibilities of some causal relationships, some aspects still need to be addressed. In particular, with the current research design, it is difficult to provide valid reasons for the relationships showing inconsistent results compared with other existing studies, such as those regarding interest or perceived ease of use. To explain the relationships among factors and their mechanisms concretely, an experimental study or other study using different research methods capable of providing evidence of causality should follow.

Third, because this study mainly tested the usability of the cancer education metaverse platform, its effects remain unknown. By expanding the scope of research to encompass variables related to the educational effectiveness of the subplatform, an inclusive understanding of its impacts can be achieved. However, developing a topic-focused questionnaire rather than a universal usability questionnaire is required to accomplish this goal. Efforts to develop survey methods that can overcome physical constraints and can be easily used by each cancer center and to explore a systematically validated questionnaire suitable for educational topics are required. We will consider conducting this subplatform effects study by seeking research methods specialized in a target content topic for metaverse cancer education.

Conclusions

This study introduced a nationwide project in Korea using the Metaverse Educational Center subplatform of the Dr. Meta multidomain metaverse cancer care digital platform. The results of this study demonstrated the positive user experience of the subplatform. This study also identified variables that may influence the intention to use the subplatform continuously. We comprehensively described how to make effective plans for improving the user experience with the subplatform by considering how to enhance the evaluation of these variables. Although this study showing the subplatform’s positive user experience may fall short of demonstrating the full educational effectiveness of the subplatform, it may serve as evidence to expand the use of the subplatform and a catalyst for conducting future effectiveness research. Using this study as a starting point, if future research verifies the efficacy of the subplatform, it will be possible to argue the potential of the subplatform for successful remote cancer education more significantly. Moving forward, beyond mammography positioning training for health professionals, expanding the scope of this cancer education metaverse platform to include various topics and target populations may enable the establishment of a valuable system for successfully conducting remote cancer education. When operating educational intervention programs for at-risk patient populations with limited access to medical information or difficulty in hospital visits, the subplatform may play a key role in sharing knowledge. Moreover, it may provide a communication space that offers enjoyment and psychological support to younger patients who are relatively familiar with virtual spaces and new media (eg, adolescent and young adult patients). To bolster metaverse cancer education programs and amplify their effects in the future, it is necessary to explore the ways to secure their various contents and establish systems to promote their broad use.

Acknowledgments

The authors thank Ajou University Hospital, Chonnam National University Hwasun Hospital, Chungbuk National University Hospital, Chungnam National University Hospital, Gachon University Gil Medical Center, Gyeongsang National University Hospital, Jeju National University Hospital, Jeonbuk National University Hospital, Kangwon National University Hospital, Kyungpook National University Chilgok Hospital, Pusan National University Hospital, and Ulsan University Hospital for their collaboration in developing Dr. Meta and conducting the usability tests. This research was supported by grants from the Ministry of Health and Welfare of Korea (NCC 2261080-1) and National Cancer Center Korea (NCC-23F1620-2).

Data Availability

The data sets generated or analyzed in this study are available from the corresponding author on reasonable request.

Authors' Contributions

SK contributed to the conceptualization, data curation, formal analysis, investigation, methodology, project administration, writing of the original draft, and review and editing of the paper. TJ contributed to the supervision and review and editing of the paper. DKS contributed to the funding acquisition, investigation, and supervision. MS contributed to the resources and review and editing of the paper. YJC contributed to the conceptualization, funding acquisition, investigation, methodology, resources, supervision, project administration, writing of the original draft, and review and editing of the paper. All authors approved the final version of the study.

Conflicts of Interest

None declared.

The questionnaire used to evaluate participants’ experiences using Metaverse Educational Center.

• Breast cancer. Cleveland Clinic. URL: https://my.clevelandclinic.org/health/diseases/3986-breast-cancer [accessed 2023-12-11]
• Breast cancer. World Health Organization. Mar 13, 2024. URL: https://www.who.int/news-room/fact-sheets/detail/breast-cancer [accessed 2024-06-26]
• Breast cancer. Centers for Disease Control and Prevention. URL: https://www.cdc.gov/breast-cancer/index.html [accessed 2023-12-11]
• Mammograms. National Institutes of Health National Cancer Institute. URL: https:/​/www.​cancer.gov/​types/​breast/​mammograms-fact-sheet#how-are-screening-and-diagnostic-mammograms-different [accessed 2023-12-11]
• Zujić PV, Božanić A, Jurković S, Šegota D, Dujmić EG, Čandrlić B, et al. The role of self-evaluation and education of radiographers involved in a breast cancer screening program at Clinical Hospital Center Rijeka. Radiography (Lond). Nov 2021;27(4):1162-1165. [ CrossRef ] [ Medline ]
• Strøm B, Pires Jorge JA, Richli Meystre N, Henner A, Kukkes T, Metsälä E, et al. Challenges in mammography education and training today: the perspectives of radiography teachers/mentors and students in five European countries. Radiography (Lond). Feb 2018;24(1):41-46. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Frenk J, Chen LC, Chandran L, Groff EO, King R, Meleis A, et al. Challenges and opportunities for educating health professionals after the COVID-19 pandemic. Lancet. Oct 29, 2022;400(10362):1539-1556. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kenamond MC, Mourad WF, Randall ME, Kaushal A. No oncology patient left behind: challenges and solutions in rural radiation oncology. Lancet Reg Health Am. Sep 2022;13:100289. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Han KT, Jun JK, Im JS. National cancer control plan of the Korea: current status and the fourth plan (2021-2025). J Prev Med Public Health. May 2023;56(3):205-211. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Shin HY, Jang DH, Jun JK. A nationwide survey on the effectiveness of training on endoscope reprocessing within the national cancer screening program in Korea. Am J Infect Control. Aug 2021;49(8):1031-1035. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kim S, Jung T, Sohn DK, Chae Y, Kim YA, Kang SH, et al. The multidomain metaverse cancer care digital platform: development and usability study. JMIR Serious Games. Nov 30, 2023;11:e46242. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Kim S, Jung T, Sohn DK, Chang YJ. Enhancing cancer care services with the multidomain metaverse cancer care digital platform. In: Proceedings of the 8th International XR-Metaverse Conference 2023. 2023. Presented at: XR-Metaverse 2023; June 28-30, 2023; Las Vegas, NV. [ CrossRef ]
• Xie T, Zheng L, Liu G, Liu L. Exploring structural relations among computer self-efficacy, perceived immersion, and intention to use virtual reality training systems. Virtual Real. 2022;26(4):1725-1744. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Barrett AJ, Pack A, Quaid ED. Understanding learners’ acceptance of high-immersion virtual reality systems: insights from confirmatory and exploratory PLS-SEM analyses. Comput Educ. Aug 2021;169:104214. [ CrossRef ]
• Kok DL, Dushyanthen S, Peters G, Sapkaroski D, Barrett M, Sim J, et al. Virtual reality and augmented reality in radiation oncology education - a review and expert commentary. Tech Innov Patient Support Radiat Oncol. Dec 2022;24:25-31. [ FREE Full text ] [ CrossRef ] [ Medline ]
• van der Kruk SR, Zielinski R, MacDougall H, Hughes-Barton D, Gunn KM. Virtual reality as a patient education tool in healthcare: a scoping review. Patient Educ Couns. Jul 2022;105(7):1928-1942. [ CrossRef ] [ Medline ]
• Wang LJ, Casto B, Luh JY, Wang SJ. Virtual reality-based education for patients undergoing radiation therapy. J Cancer Educ. Jun 24, 2022;37(3):694-700. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Cabero-Almenara J, Llorente-Cejudo C, Palacios-Rodríguez A, Gallego-Pérez Ó. Degree of acceptance of virtual reality by health sciences students. Int J Environ Res Public Health. Apr 18, 2023;20(8):5571. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Tennant M, McGillivray J, Youssef GJ, McCarthy MC, Clark TJ. Feasibility, acceptability, and clinical implementation of an immersive virtual reality intervention to address psychological well-being in children and adolescents with cancer. J Pediatr Oncol Nurs. 2020;37(4):265-277. [ CrossRef ] [ Medline ]
• Wang EY, Kennedy KM, Zhang L, Qian D, Forbes T, Zuniga-Hernandez M, et al. Predicting pediatric healthcare provider use of virtual reality using a technology acceptance model. JAMIA Open. Oct 2023;6(3):ooad076. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Han JH, Sa HJ. Acceptance of and satisfaction with online educational classes through the technology acceptance model (TAM): the COVID-19 situation in Korea. Asia Pacific Educ Rev. Sep 15, 2021;23(3):403-415. [ CrossRef ]
• Jang HJ, Kim KH. Study on the influence of VR characteristics on user satisfaction and intention to use continuously -focusing on VR presence, user characteristics, and VR sickness-. J Korea Contents Assoc. 2018;18(5):420-431. [ FREE Full text ] [ CrossRef ]
• Lin S, Zimmer JC, Lee V. Decoupling software from hardware in technology acceptance research. J Comput Inf Syst. Dec 10, 2015;54(2):77-86. [ CrossRef ]
• Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. Jan 2006;3(2):77-101. [ CrossRef ]
• Braun V, Clarke V. Toward good practice in thematic analysis: avoiding common problems and be(com)ing a knowing researcher. Int J Transgend Health. Oct 25, 2022;24(1):1-6. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Doing reflexive TA. The University of Auckland. URL: https://www.thematicanalysis.net/doing-reflexive-ta/ [accessed 2024-06-09]
• Tran HT, Kim M. Factors influencing the continued intent to use virtual interactive platforms in Korean small- and medium-sized enterprises for remote and hybrid work. Sustainability. Jun 22, 2023;15(13):9972. [ CrossRef ]
• Choi J, Lee KK, Choi J. Determinants of user satisfaction with mobile VR headsets: the human factors approach by the user reviews analysis and product lab testing. Int J Content. 2019;15(1):1-9. [ CrossRef ]
• Guerra-Tamez CR. The impact of immersion through virtual reality in the learning experiences of art and design students: the mediating effect of the flow experience. Educ Sci. Feb 09, 2023;13(2):185. [ CrossRef ]
• Herz M, Rauschnabel PA. Understanding the diffusion of virtual reality glasses: the role of media, fashion and technology. Technol Forecast Soc Change. Jan 2019;138:228-242. [ CrossRef ]
• Yan Y, Chen K, Xie Y, Song Y, Liu Y. The effects of weight on comfort of virtual reality devices. In: Proceedings of the AHFE 2018 International Conference on Ergonomics in Design. 2018. Presented at: AHFE 2018; July 21-25, 2018; Orlando, FL. [ CrossRef ]
• Choi J, Moon H, Park M. What factors influence continuous usage intention of head-mounted display-based virtual reality content?: a cross-sectional survey. Korean J Women Health Nurs. Sep 2023;29(3):208-218. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Syed-Abdul S, Malwade S, Nursetyo AA, Sood M, Bhatia M, Barsasella D, et al. Virtual reality among the elderly: a usefulness and acceptance study from Taiwan. BMC Geriatr. Aug 19, 2019;19(1):223. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Garrido LE, Frías-Hiciano M, Moreno-Jiménez M, Cruz GN, García-Batista ZE, Guerra-Peña K, et al. Focusing on cybersickness: pervasiveness, latent trajectories, susceptibility, and effects on the virtual reality experience. Virtual Real. Mar 01, 2022;26(4):1347-1371. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Liu Q, Wang Y, Tang Q, Liu Z. Do you feel the same as I do? differences in virtual reality technology experience and acceptance between elderly adults and college students. Front Psychol. Sep 25, 2020;11:573673. [ FREE Full text ] [ CrossRef ] [ Medline ]
• Radiologic technologist. Cleveland Clinic. URL: https://my.clevelandclinic.org/health/articles/24667-radiologic-technologist [accessed 2024-02-07]
• Lin CH, Shih HY, Sher PJ. Integrating technology readiness into technology acceptance: the TRAM model. Psychol Market. May 24, 2007;24(7):641-657. [ CrossRef ]
• Liu Q, Wang Y, Yao MZ, Tang Q, Yang Y. The effects of viewing an uplifting 360-degree video on emotional well-being among elderly adults and college students under immersive virtual reality and smartphone conditions. Cyberpsychol Behav Soc Netw. Mar 2020;23(3):157-164. [ CrossRef ] [ Medline ]
• Adelana OP, Ayanwale MA, Ishola AM, Oladejo AI, Adewuyi HO. Exploring pre-service teachers’ intention to use virtual reality: a mixed method approach. Comput Educ X Reality. Dec 2023;3:100045. [ CrossRef ]
• Marangunić N, Granić A. Technology acceptance model: a literature review from 1986 to 2013. Univ Access Inf Soc. Feb 16, 2014;14(1):81-95. [ CrossRef ]
• Bae S, Jung TH, Moorhouse N, Suh M, Kwon O. The influence of mixed reality on satisfaction and brand loyalty in cultural heritage attractions: a brand equity perspective. Sustainability. Apr 08, 2020;12(7):2956. [ CrossRef ]
• Joo E, Yang J. How perceived interactivity affects consumers' shopping intentions in live stream commerce: roles of immersion, user gratification and product involvement. J Res Interact Market. Feb 09, 2023;17(5):754-772. [ CrossRef ]
• Rodríguez-Ardura I, Meseguer-Artola A. E-learning continuance: the impact of interactivity and the mediating role of imagery, presence and flow. Inf Manag. Jun 2016;53(4):504-516. [ CrossRef ]
• Skjoldborg NM, Bender PK, Jensen de López KM. The efficacy of head-mounted-display virtual reality intervention to improve life skills of individuals with autism spectrum disorders: a systematic review. Neuropsychiatr Dis Treat. Oct 2022;Volume 18:2295-2310. [ CrossRef ]
• Nguyen MH, Smets EM, Bol N, Loos EF, van Weert JC. How tailoring the mode of information presentation influences younger and older adults' satisfaction with health websites. J Health Commun. 2018;23(2):170-180. [ CrossRef ] [ Medline ]
• Lv Z. Generative artificial intelligence in the metaverse era. Cognit Robot. 2023;3:208-217. [ CrossRef ]
• Liu Y, Siau KL. Generative artificial intelligence and metaverse: future of work, future of society, and future of humanity. In: Proceedings of the First International Conference on AI-Generated Content. 2023. Presented at: AIGC 2023; August 25-26, 2023; Shanghai, China. [ CrossRef ]
• Huynh-The T, Pham QV, Pham XQ, Nguyen TT, Han Z, Kim DS. Artificial intelligence for the metaverse: a survey. Eng Appl Artif Intell. Jan 2023;117:105581. [ CrossRef ]
• Soliman MM, Ahmed E, Darwish A, Hassanien AE. Artificial intelligence powered Metaverse: analysis, challenges and future perspectives. Artif Intell Rev. Feb 05, 2024;57:36. [ CrossRef ]

Abbreviations

Edited by G Eysenbach, T de Azevedo Cardoso; submitted 26.02.24; peer-reviewed by S Yamane, T Thang; comments to author 15.03.24; revised version received 27.04.24; accepted 18.06.24; published 15.07.24.

©Sunghak Kim, Timothy Jung, Dae Kyung Sohn, Mina Suh, Yoon Jung Chang. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 15.07.2024.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

• Introduction
• Conclusions
• Article Information

eTable 1. Diagnosis Error Evaluation and Research (DEER) Taxonomy

eTable 2. Reliable Diagnosis Challenges (RDC) Taxonomy

eTable 3. Diagnostic Pitfalls Associated With Neurological Conditions

eTable 4. Diagnostic Pitfalls Associated With Breast Cancer

eTable 5. Generic Diagnostic Pitfalls Associated With Most Frequent Conditions

See More About

Sign up for emails based on your interests, select your interests.

Customize your JAMA Network experience by selecting one or more topics from the list below.

• Academic Medicine
• Acid Base, Electrolytes, Fluids
• Allergy and Clinical Immunology
• American Indian or Alaska Natives
• Anesthesiology
• Anticoagulation
• Art and Images in Psychiatry
• Artificial Intelligence
• Assisted Reproduction
• Bleeding and Transfusion
• Caring for the Critically Ill Patient
• Challenges in Clinical Electrocardiography
• Climate and Health
• Climate Change
• Clinical Challenge
• Clinical Decision Support
• Clinical Implications of Basic Neuroscience
• Clinical Pharmacy and Pharmacology
• Complementary and Alternative Medicine
• Consensus Statements
• Coronavirus (COVID-19)
• Critical Care Medicine
• Cultural Competency
• Dental Medicine
• Dermatology
• Diabetes and Endocrinology
• Diagnostic Test Interpretation
• Drug Development
• Electronic Health Records
• Emergency Medicine
• End of Life, Hospice, Palliative Care
• Environmental Health
• Equity, Diversity, and Inclusion
• Facial Plastic Surgery
• Gastroenterology and Hepatology
• Genetics and Genomics
• Genomics and Precision Health
• Global Health
• Guide to Statistics and Methods
• Hair Disorders
• Health Care Delivery Models
• Health Care Economics, Insurance, Payment
• Health Care Quality
• Health Care Reform
• Health Care Safety
• Health Care Workforce
• Health Disparities
• Health Inequities
• Health Policy
• Health Systems Science
• History of Medicine
• Hypertension
• Images in Neurology
• Implementation Science
• Infectious Diseases
• Innovations in Health Care Delivery
• JAMA Infographic
• Law and Medicine
• Leading Change
• Less is More
• LGBTQIA Medicine
• Lifestyle Behaviors
• Medical Coding
• Medical Devices and Equipment
• Medical Education
• Medical Education and Training
• Medical Journals and Publishing
• Mobile Health and Telemedicine
• Narrative Medicine
• Neuroscience and Psychiatry
• Notable Notes
• Nutrition, Obesity, Exercise
• Obstetrics and Gynecology
• Occupational Health
• Ophthalmology
• Orthopedics
• Otolaryngology
• Pain Medicine
• Palliative Care
• Pathology and Laboratory Medicine
• Patient Care
• Patient Information
• Performance Improvement
• Performance Measures
• Perioperative Care and Consultation
• Pharmacoeconomics
• Pharmacoepidemiology
• Pharmacogenetics
• Pharmacy and Clinical Pharmacology
• Physical Medicine and Rehabilitation
• Physical Therapy
• Physician Leadership
• Population Health
• Primary Care
• Professional Well-being
• Professionalism
• Psychiatry and Behavioral Health
• Public Health
• Pulmonary Medicine
• Regulatory Agencies
• Reproductive Health
• Research, Methods, Statistics
• Resuscitation
• Rheumatology
• Risk Management
• Scientific Discovery and the Future of Medicine
• Shared Decision Making and Communication
• Sleep Medicine
• Sports Medicine
• Stem Cell Transplantation
• Substance Use and Addiction Medicine
• Surgical Innovation
• Surgical Pearls
• Teachable Moment
• Technology and Finance
• The Art of JAMA
• The Arts and Medicine
• The Rational Clinical Examination
• Tobacco and e-Cigarettes
• Translational Medicine
• Trauma and Injury
• Treatment Adherence
• Ultrasonography
• Users' Guide to the Medical Literature
• Vaccination
• Venous Thromboembolism
• Veterans Health
• Women's Health
• Workflow and Process
• Wound Care, Infection, Healing

Get the latest research based on your areas of interest.

Others also liked.

• Download PDF
• X Facebook More LinkedIn

Schiff GD , Volodarskaya M , Ruan E, et al. Characteristics of Disease-Specific and Generic Diagnostic Pitfalls : A Qualitative Study . JAMA Netw Open. 2022;5(1):e2144531. doi:10.1001/jamanetworkopen.2021.44531

Manage citations:

© 2024

• Permissions

Characteristics of Disease-Specific and Generic Diagnostic Pitfalls : A Qualitative Study

• 1 Division of General Internal Medicine and Primary Care, Brigham and Women’s Hospital, Boston, Massachusetts
• 2 Center for Patient Safety Research and Practice, Brigham and Women’s Hospital, Boston, Massachusetts
• 3 Center for Primary Care, Harvard Medical School, Boston, Massachusetts
• 4 Department of Surgery, Rush University Medical Center, Chicago, Illinois
• 5 Department of Medicine, Montefiore Medical Center, Bronx, New York
• 6 Department of Internal Medicine, Kaiser Permanente, San Francisco, California
• 7 Department of Biomedical Informatics, Vanderbilt University, Nashville, Tennessee
• 8 Center for Innovations in Quality, Effectiveness and Safety, Michael E. DeBakey Veterans Affairs Medical Center and Baylor College of Medicine, Houston, Texas
• 9 Department of Medicine, Harvard Medical School, Boston, Massachusetts
• 10 Department of Biomedical Informatics, Columbia University, New York, New York

Question   Are there similarities among clinical situations associated with diagnostic errors?

Findings   This qualitative study identified 836 relevant cases among 4325 patient safety incident reports, 403 closed malpractice claims, 24 morbidity and mortality reports, and 355 focus group responses. From these, 661 disease-specific and 21 generic diagnostic “pitfalls” were identified.

Meaning   Diagnostic pitfalls represent a potentially useful construct that bridges cognitive and systems diagnosis error approaches because they can delineate and demonstrate recurrent patterns of diagnostic error.

Importance   Progress in understanding and preventing diagnostic errors has been modest. New approaches are needed to help clinicians anticipate and prevent such errors. Delineating recurring diagnostic pitfalls holds potential for conceptual and practical ways for improvement.

Objectives   To develop the construct and collect examples of “diagnostic pitfalls,” defined as clinical situations and scenarios vulnerable to errors that may lead to missed, delayed, or wrong diagnoses.

Design, Setting, and Participants   This qualitative study used data from January 1, 2004, to December 31, 2016, from retrospective analysis of diagnosis-related patient safety incident reports, closed malpractice claims, and ambulatory morbidity and mortality conferences, as well as specialty focus groups. Data analyses were conducted between January 1, 2017, and December 31, 2019.

Main Outcomes and Measures   From each data source, potential diagnostic error cases were identified, and the following information was extracted: erroneous and correct diagnoses, presenting signs and symptoms, and areas of breakdowns in the diagnostic process (using Diagnosis Error Evaluation and Research and Reliable Diagnosis Challenges taxonomies). From this compilation, examples were collected of disease-specific pitfalls; this list was used to conduct a qualitative analysis of emerging themes to derive a generic taxonomy of diagnostic pitfalls.

Results   A total of 836 relevant cases were identified among 4325 patient safety incident reports, 403 closed malpractice claims, 24 ambulatory morbidity and mortality conferences, and 355 focus groups responses. From these, 661 disease-specific diagnostic pitfalls were identified. A qualitative review of these disease-specific pitfalls identified 21 generic diagnostic pitfalls categories, which included mistaking one disease for another disease (eg, aortic dissection is misdiagnosed as acute myocardial infarction), failure to appreciate test result limitations, and atypical disease presentations.

Conclusions and Relevance   Recurring types of pitfalls were identified and collected from diagnostic error cases. Clinicians could benefit from knowledge of both disease-specific and generic cross-cutting pitfalls. Study findings can potentially inform educational and quality improvement efforts to anticipate and prevent future errors.

Diagnostic errors are the leading type of medical error reported by patients 1 and a leading cause for malpractice claims. 2 - 4 A study of malpractice claims in primary care in Massachusetts found that more than 70% of primary care claims reside in the diagnostic safety realm, and patients rank diagnostic errors as the leading type of medical error that they have experienced. 5 In 2015, the National Academy of Medicine issued a report, Improving Diagnosis in Health Care, highlighting the importance and causes of diagnostic errors and making recommendations for preventing and mitigating such errors. 6 However, despite increasing appreciation of diagnostic errors as a patient safety issue, progress in understanding and preventing diagnostic errors has been modest. 7 Unlike medication errors, which have been more successfully reduced with system-level and information technology–based interventions, there are no comparable single technical or educational fixes for diagnostic errors. 8 , 9

One central and recurring theme has centered around efforts to differentiate so-called cognitive errors from system errors. 6 , 10 , 11 However, emerging evidence suggests that most diagnostic errors are multifactorial, with these cognitive and system factors often overlapping and interacting. 12 , 13 We sought to develop a new approach—“diagnostic pitfalls”—that recognizes and bridges this overlap and that may potentially provide practical, disease-specific guidance to help clinicians and organizations consider, anticipate, identify, and mitigate what can go wrong in diagnosis.

The concept of diagnostic pitfalls is hardly a new notion. It was highlighted more than a century ago by Cabot in the article “Diagnostic Pitfalls Identified During a Study of Three Thousand Autopsies.” 14 Nonetheless, during the ensuing decades, the term has been variably and inconsistently used in the medical literature but rarely invoked in current diagnosis research. In this study, we aim to review a comprehensive series of diagnostic error cases and identify both diagnosis-specific and generic cross-cutting issues that frequently occur as pitfalls.

We define diagnostic pitfalls as clinical situations and scenarios that are vulnerable to errors that may lead to missed, delayed, or wrong diagnoses. This practical construct embraces both cognitive issues (eg, knowledge gaps, heuristics, and biases) and system factors (eg, communication breakdowns, disease presentation factors, and test limitations) associated with diagnostic error. We used a multisource approach to collect diagnostic error cases to delineate potential pitfalls. This study had 3 aims: (1) to develop and refine the construct of diagnostic pitfalls, (2) to collect a list of disease-specific examples of diagnostic pitfalls, and (3) to analyze these disease-specific pitfalls to create a taxonomy of generic types of diagnostic pitfalls occurring in primary care practice. Knowledge of such potential diagnostic pitfalls could help clinicians and organizations design educational efforts and safety nets to anticipate, prevent, and/or mitigate such errors.

We analyzed data from physicians and adult patients in ambulatory practices and academic medical centers across Massachusetts between January 1, 2004, and December 31, 2016. All cases involved diagnostic errors and data originated from (1) closed malpractice claims, (2) patient safety incident reports, (3) ambulatory morbidity and mortality conferences, or (4) specialty focus groups. For all data sources, we reviewed each case and abstracted key variables of interest. After aggregating abstracted data, we calculated summary statistics, conducted a qualitative thematic analysis of disease-specific challenges, and then iteratively derived generic diagnostic pitfalls. This study followed the Standards for Reporting Qualitative Research ( SRQR ) reporting guideline and was approved by the Mass General Brigham institutional review board.

For all data sources, a trained research assistant reviewed each clinical case in consultation with a general internist (G.D.S.) with expertise in diagnostic error. We abstracted the same key data elements from each source and recorded them in an Access (Microsoft Corp) database using a single standardized form with preset, drop-down fields and a free-text box to briefly summarize each case. Variables of interest included data source (eg, closed malpractice claim), erroneous (ie, initial) diagnosis, correct diagnosis, and presenting signs and symptoms. We also classified breakdowns in the diagnostic process using 2 complementary taxonomies, Diagnosis Error Evaluation and Research (DEER) 12 , 15 (eTable 1 in the Supplement ) and Reliable Diagnosis Challenges (RDC) 16 (eTable 2 in the Supplement ), assigning up to 3 DEER codes and 3 RDC codes for each case. DEER identifies what went wrong and situates where the failure occurred in the diagnostic process, whereas RDC identifies the general challenges complicating the diagnostic process and the potential reasons a mistake occurred.

We pooled data from all 4 sources and reviewed each case scenario for eligibility for qualitative coding. Eligibility was broadly defined as cases previously coded by the 2 malpractice insurers as diagnosis related as well as cases in the institutional databases or collected from specialty focus groups that could be considered diagnostic errors or delays.

We extracted 11 years of diagnosis-related patient safety incident reports (2004-2014) from a large academic medical center in eastern Massachusetts using RL Solutions, a commercial incident management solution. The Patient Safety/Risk Management department reviews every report on a daily basis and fills in fields that were left blank or required follow-up to clarify additional information.

We reviewed 5 years of closed malpractice claims involving adult primary care ambulatory clinics (ie, internal and family medicine) from 2 malpractice insurers, Controlled Risk Insurance Company (CRICO) and Coverys, which, when combined, insure more than 85% of all clinicians in Massachusetts. All cases were coded by insurers by professional medical coders with nursing experience. We restricted our review to all closed claims assigned a diagnostic error tag with an initial claim date between 2010 and 2014.

We collated information on all available primary care morbidity and mortality conferences at our academic medical center over the 11-year period between 2004 and 2014. We restricted cases to those involving diagnostic errors in the ambulatory setting of care. To obtain information, we solicited materials from departmental internal online repositories.

Between March 2015 and December 2016, we put together 6 one-hour focus groups with attending physicians and clinical fellows during regularly scheduled division meetings or educational conferences for the following specialties: (1) neurology, (2) gastroenterology, (3) dermatology, (4) pulmonary and critical care medicine, (5) rheumatology, and (6) oral medicine and dentistry. We targeted these as examples of medical specialties that we postulated were in a position to observe diagnostic errors and delays that occurred upstream in the course of patients’ primary care. After obtaining oral informed consent, we oriented the specialists to the problem of diagnostic errors and the concept of diagnostic pitfalls. Participants received a paper form to solicit up to 3 examples of diagnostic pitfalls, asking “What kind of disease-specific diagnostic pitfalls, errors, or mistakes do you most commonly observe primary care physicians make?” Participants were allowed 15 minutes to provide written examples; we then engaged them in a 30-minute discussion to discuss their examples. The written responses were collected, and the sessions were digitally recorded and transcribed.

Data analyses were performed between January 1, 2017, and December 31, 2019. We aggregated data abstracted from all sources and calculated descriptive statistics based on DEER and RDC coding. We also computed the most frequently missed or delayed diagnoses by disease and system and noted the most frequent signs or symptoms mentioned. Next, we performed a preliminary review of all cases to collate disease-specific examples that met our preestablished definition of diagnostic pitfalls. We then conducted an iterative thematic analysis of these disease-specific pitfalls to derive generic diagnostic pitfalls. We started by familiarizing ourselves with the data, using DEER and RDC taxonomies as preliminary codes to provide structure and describe the content, then searched for patterns or themes across cases, reviewed these themes until reaching saturation, and, finally, named and defined those broader themes. Analysis occurred iteratively to allow for regular, real-time identification and interpretation of recurrent patterns, organizing of themes, verification of accuracy and consistency of findings, and adjudication of any disagreement. 17 We assigned each case up to 3 generic diagnostic pitfalls based on this final list.

Our data sources included 4352 patient safety incident reports, 403 closed malpractice claims, 24 ambulatory morbidity and mortality rounds, and 355 focus group responses collected over 6 sessions among physicians from 6 specialties ( Figure 1 ). Focus groups ranged in size from 8 to 25 specialist physician participants. We reviewed each case for eligibility and identified 836 relevant diagnostic error cases comprising 75 of 4352 incident reports (2%), 396 of 403 closed claims (98%), 10 of 24 morbidity and mortality conferences (42%), and all 355 focus group responses.

Table 1 lists the top 10 most commonly missed or delayed diagnoses by condition and system (836 cases). The most frequent diagnoses were colorectal (38 [5%]), lung (36 [4%]), breast (20 [2%]), prostate (18 [2%]), and bladder (10 [1%]) cancers; myocardial infarction (20 [2%]); stroke (15 [2%]); sepsis (13 [2%]); pulmonary embolism (9 [1%]); and brain hemorrhage (8 [1%]). The most common diagnoses by system were oncology, neurology, and cardiology diagnoses (reflecting in large part the case contributions from the focus groups). Pain (abdominal, general, and chest), emesis, fever, headache, and altered mental status were the most frequent presenting signs and symptoms.

Based on DEER taxonomy coding of the diagnostic process from access or presentation through follow-up ( Figure 2 A), 63% of 1208 errors were “localized” to the testing (n = 503) and assessment (n = 260) phases. The most common DEER subcategories were failure in ordering the needed test (n = 214), failure to consider the correct diagnosis (n = 144), failure to or delay in follow-up of (abnormal) test results (n = 107), failure in weighing a critical piece of history data (n = 94), and failure to order or delay in ordering a referral (n = 83).

Coding using the RDC taxonomy ( Figure 2 B) revealed that 59% of 1041 errors were associated with testing challenges (n = 314) and challenging disease presentation (n = 305). The most frequent RDC subcategories assigned were test follow-up issues (n = 131), challenges in recognition of acuity or severity of illness (n = 82), test performance or interpretation (n = 69), masking or mimicking diagnosis (n = 64), and failure to diagnose the underlying cause (n = 62).

Most specialists were able to offer a number of specific examples of diagnosis failures that they considered to represent recurring pitfalls. These examples generally clustered into (1) more serious specialty diagnoses that were missed or delayed as a result of misdiagnosis upstream by the primary care physician; (2) instances of misdiagnosis or overdiagnosis in which patients were labeled as having a specialty diagnosis (eg, multiple sclerosis) that did not meet diagnostic criteria and/or where an alternate diagnosis (usually more benign, nonspecific, or psychiatric in nature) was more likely; and (3) generic pitfalls in clinical examination or testing relevant to that specialty (see eTable 3 in the Supplement for diagnostic pitfalls associated with neurologic conditions).

From this pool of eligible cases, we identified 661 disease-specific pitfalls. Illustrative disease-specific pitfalls included (1) misreading a lung mass as pneumonia on chest radiograph; (2) ordering screening, instead of diagnostic, mammogram in evaluation of breast lump (see eTable 4 in the Supplement for additional breast cancer examples); (3) attributing intermittent hematuria to urinary tract infection despite negative urine cultures, thereby missing bladder cancer; (4) misinterpreting facial flushing as rosacea, delaying diagnosis of carcinoid syndrome; and (5) underweighing the possibility of transient ischemic attack in a patient with bilateral neurologic symptoms.

Although our primary aim was to collect disease-specific pitfalls, we also sought to understand the different types of recurring cross-cutting generic issues that characterized these distinct pitfalls. Based on the patterns and themes noted among disease-specific pitfalls, we derived 21 generic diagnostic pitfalls ( Table 2 ) and grouped them using DEER categories. Prominent examples of these generic pitfalls include one disease misdiagnosed as another disease (eg, colon cancer erroneously diagnosed as hemorrhoids or celiac disease; bipolar disorder labeled or misdiagnosed as depression), failure to appreciate the limitations of a test or examination (eg, patient with breast lump and negative mammogram or ultrasonography result), atypical presentation (eg, Addison disease presenting with weight loss, cognitive difficulties, and fatigue), presuming chronic disease accounts for new symptoms (eg, attributing weight loss and cough to a patient’s chronic obstructive pulmonary disease, leading to delayed lung cancer diagnosis), and failure to monitor evolving symptoms (eg, normal results from cranial imagining shortly after head injury but chronic subdural hematoma later developed).

For the most frequent diagnoses (which were dominated by malpractice cases, given the larger sample of these cases), we illustrate the types of generic diagnostic pitfalls in eTable 5 in the Supplement . The relative frequency of the 5 leading types of generic pitfalls were failure to follow-up, not appreciating test limitations, mistaking one disease for another disease (eg, aortic dissection is misdiagnosed as acute myocardial infarction), failure to appreciate patient risk factors, and atypical disease presentation.

Using multiple sources, we identified a series of diagnostic error cases that we used to help characterize failures in the diagnostic process in primary care. We used these instances of opportunities to improve diagnoses to compile a list of disease-specific examples as well as create a generic taxonomy of the types of recurring scenarios and issues—vulnerabilities in the diagnostic process that we have termed diagnostic pitfalls . By exploring both traditional sources of diagnostic error cases, such as malpractice claims and organizational error reports, and more novel inputs, such as errors identified by specialists with patients referred by primary care physicians, we were able to compile a rich collection of cases to review for pitfalls.

We used 2 previously validated tools for classifying where in the diagnostic process (DEER) and why (RDC) these diagnoses were potentially challenging for clinicians. 15 , 16 Similar to prior studies, 15 we found that issues associated with diagnostic testing predominated (especially ordering [mainly failure to order], interpretation, and follow-up of abnormal results) and patient assessment (particularly in failure to consider a diagnosis, recognize disease severity or urgency, and recognize atypical disease presentation or one that mimicked or was masked by a competing diagnosis). As in earlier studies, many of these cases had overlapping and multifactorial issues, suggesting the need for a multifaceted approach to recognize and prevent such errors. 10 , 11 , 15

To make diagnoses more reliable, we need to enhance the ability of clinicians and systems to anticipate what can go wrong and build strategies to minimize vulnerabilities associated with these pitfalls. 18 Compiling a list of potential pitfalls represents an essential first step for additional interventions. For instance, in addition to educational interventions focused on clinicians anticipating these pitfalls, such pitfalls could inform the development of decision-support interventions that warn clinicians in real time to avoid these errors.

A key feature of high-reliability organizations is continual awareness and worry about what can go wrong. 19 - 22 In applying lessons from a retrospective review of errors, the distillation of recurring pitfalls can help frontline clinicians anticipate these pitfalls and thereby recognize and prevent future errors. 23 Diagnostic pitfalls can potentially fulfill such a role, warning of diagnosis-specific risks as well as providing a more generalized awareness of omnipresent vulnerabilities inherent in uncertain diagnoses ( Box ). 24 Nevertheless, because education and warnings are lower in the hierarchy of effectiveness of improvement interventions, operationalizing systems to enhance situational awareness and building safety nets to minimize such errors are also a necessity. 25

Potential Ways the “Pitfalls” Construct Can Be Useful

Facilitate efforts and approaches to reduce diagnostic errors.

More clinically oriented and engaging to clinicians than other diagnostic error approaches (cognitive biases, industrial CQI methods)

Lends itself to fewer defensive responses from clinicians—by showing recurring pitfalls, vulnerabilities, and errors others have made, clinicians feel less singled out when their own diagnoses are in error

Shining light on recurring pitfalls can provide motivation to stimulate clinicians, practices, and organizations to address them to prevent recurrences

Complements other heuristics’ didactics (eg, biases) by providing actual examples of recurring failures or vulnerabilities

Provides an interoperable framework for conversations with and across specialties regarding disease-specific diagnostic vulnerabilities

Provide practical levers to catalyze change

Provides an educational framework to communicate and impart experiences with misdiagnosis for particular symptoms or diagnoses for educating new or practicing clinicians

Inform new section of medical textbooks to supplement usual sections on epidemiology, pathophysiology, diagnosis, and treatment; this could involve creating a new pitfalls section listing common and important pitfalls to anticipate and avoid in diagnosis

Potential to facilitate electronic diagnostic decision support via context-aware warnings of lurking pitfalls (eg, easily confused diagnoses, test limitations, atypical presentations, and causes to consider); to avoid overalerting, the warnings could be in the background to be queried by clinicians (click Alt-P to activate) when pondering uncertain or dangerous clinical situations

Inform triggers to retrospectively search for past or potentially ongoing diagnostic errors in clinical databases (eg, search for women with breast lump referred for screening, rather than diagnostic, mammogram; search abnormal mammograms that are not followed up)

Ability to identify vulnerable situations to prioritize and improve process redesign to immunize systems and clinicians to prevent, detect, and mitigate errors (eg, design forcing functions to serve as “guard rails” against that pitfall)

Framework for collecting epidemiologic data on incidence, risk factors, and high-risk situations for disease-specific pitfalls

Patient tool to aid patients in reviewing list of pitfalls online to raise questions about potential pitfalls in their own diagnosis (eg, could my negative COVID-19 test result be a false-negative result?)

Abbreviation: CQI, continuous quality improvement.

For instance, beyond simply anticipating errors, it is critical to put in place mechanisms to prevent their occurrence or mitigate their harm. Our data identified multiple examples of well-known pitfalls, for example, misdiagnosing acute aortic dissection as acute myocardial infarction (a more common acute cardiac condition) or concluding that a woman who has a palpable breast lump and negative mammogram results has had breast cancer “ruled out” (eTable 4 in the Supplement ). This findings suggest that awareness of pitfalls needs to be supplemented with ways to remind clinicians in real time and with design mechanisms to provide forcing functions to potentially avoid such pitfalls (eg, mammography protocols requiring women undergoing screening mammography to fill out a form inquiring whether they had or are being referred for a breast lump).

Identifying disease-specific failures offers many ways for making progress in understanding and preventing diagnostic errors ( Box ). It holds the potential for guidance for a more granular understanding of what went wrong in the diagnostic process, a need highlighted by the National Quality Forum report on improving diagnostic quality and safety. 26 By focusing on specific clinical scenarios, this approach has the potential to better engage practicing physicians in ways that industrial-improvement language related to “systems redesign” may fail to speak to and spark their clinical imaginations. It also holds potential to better link work being done in the general diagnostic error realm with specialists and specialty societies and researchers, who are working in the silos of their respective diseases. 27 - 29 Generic pitfalls provide a framework for bridging the gap across specific diseases. Specific pitfalls can also be operationally defined to inform the development and deployment of electronic triggers to retrospectively examine and understand their occurrence and an institution’s vulnerability to these types of missteps. 30

This study has some strengths. Although it uses a convenience sample, our approach integrated high-quality data sources rich in examples of diagnostic error and included a wide range of diseases, which resulted in a substantial, broad set of diagnostic errors for analyses. To support uniform data collection, we used the same structured abstraction form across all data sources.

Nonetheless, we recognize that our findings should also be considered in light of potential limitations. Our approach is prone to potential bias in representativeness of diagnoses owing to underreporting and selective reporting of patient safety incidents, malpractice claims filed, morbidity and mortality cases presented, and the specialist physicians who participated in our focus groups. Although data sources spanned multiple specialties, the focus groups were a convenience sample of 6 specialties, and all information sources, except for closed malpractice claims, originated from a single academic medical center. We also recognize that coding cases using DEER and RDC taxonomies may be subject to reviewer bias and experience, although these were previously validated tools with good reliability and all cases were discussed and secondarily reviewed by a diagnosis error expert internist. Given that our data sources represent a convenience sample, some from older retrospective data, we were unable to estimate the true current prevalence of these diagnostic pitfalls. Accordingly, we only report frequencies and emphasize the general principles and themes identified. Some of these examples possibly risk overinterpretation if potentially rare causes or diagnoses were oversampled, risking “overdiagnosis” types of errors if uncritically applied. Thus, the implications for clinical practice merit cautious and careful consideration to avoid excessive worry or overdiagnosis (eg, in many cases, breast lumps, headaches, or rectal bleeding do not herald cancer). 31

Disease-specific and generic diagnostic pitfalls represent a potentially useful construct that bridges the gap across disease, system, and cognitive factors associated with diagnostic errors in medicine. We were able to identify and classify such pitfalls using cross-sectional lenses of locally reported cases, regional malpractice claims, and specialty expert input. Pitfalls can help illustrate specific and recurrent types of errors for specific diagnosis and clinical situations, as well as illustrate crosscutting themes that can be applied more broadly across diseases. Distillation of diagnostic pitfalls offers a number of benefits for making progress in understanding and preventing diagnostic errors and in better understanding what went wrong, how frequently the error occurs, where in the diagnostic process it occurs and why it occurs, and for providing clinically rich examples to show frontline clinicians situations that are predisposed to errors.

Accepted for Publication: November 29, 2021.

Published: January 21, 2022. doi:10.1001/jamanetworkopen.2021.44531

Open Access: This is an open access article distributed under the terms of the CC-BY License . © 2022 Schiff GD et al. JAMA Network Open .

Corresponding Author: Gordon D. Schiff, MD, Center for Patient Safety Research and Practice, Division of General Internal Medicine and Primary Care, Brigham and Women’s Hospital, Harvard Medical School, 1620 Tremont St, BC-3, Boston, MA 02120 ( [email protected] ).

Author Contributions: Dr Schiff and Mr Reyes Nieva had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis.

Concept and design: Schiff, Reyes Nieva.

Acquisition, analysis, or interpretation of data: All authors.

Drafting of the manuscript: Schiff, Reyes Nieva.

Critical revision of the manuscript for important intellectual content: All authors.

Statistical analysis: Schiff, Volodarskaya, Reyes Nieva.

Obtained funding: Schiff, Reyes Nieva.

Administrative, technical, or material support: Schiff, Volodarskaya, Lim, Reyes Nieva.

Supervision: Schiff, Reyes Nieva.

Conflict of Interest Disclosures: Dr Schiff reported receiving grants from Controlled Risk Insurance Company (CRICO) and the Gordon and Betty Moore Foundation during the conduct of the study. Dr Singh reported receiving grants from the Department of Veterans Affairs and the Agency for Healthcare Research and Quality; and serving in an advisory role for Leapfrog Group outside the submitted work. Dr Reyes Nieva reported receiving grants from CRICO/Risk Management Foundation of the Harvard Medical Institutions during the conduct of the study. No other disclosures were reported.

Funding/Support: This research was funded by the CRICO/Risk Management Foundation of the Harvard Medical Institutions. Dr Singh is partially supported by the Veterans Affairs Health Services Research and Development’s Center for Innovations in Quality, Effectiveness and Safety (CIN 13–413). Subsequent work on the pitfalls construct was funded by the Gordon and Betty Moore Foundation as part of the Brigham-Betsy Lehman Center PRIDE (Primary Care Improvement in Diagnostic Error) Project.

Role of the Funder/Sponsor: The funding sources had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

Disclaimer: The content is solely the responsibility of the authors and does not necessarily represent the official views of a funder or sponsor.

• Register for email alerts with links to free full-text articles
• Access PDFs of free articles
• Manage your interests
• Save searches and receive search alerts

Europe PMC requires Javascript to function effectively.

Either your web browser doesn't support Javascript or it is currently turned off. In the latter case, please turn on Javascript support in your web browser and reload this page.

• Open access
• Published: 11 July 2024

Challenges of pre-hospital emergency care at Addis Ababa Fire and Disaster Risk Management Commission, Addis Ababa, Ethiopia: a qualitative study

• Feleku Yimer Seid 1 ,
• Birhanu Chekol Gete 1 &
• Amanuel Sisay Endeshaw 2  

BMC Health Services Research volume  24 , Article number:  803 ( 2024 ) Cite this article

69 Accesses

Metrics details

A challenge to pre-hospital emergency care is any barrier or obstacle that impedes quality pre-hospital care or impacts community pre-hospital utilization. The Addis Ababa Fire and Disaster Risk Management Commission (AAFDRMC) provides pre-hospital emergency services in Addis Ababa, Ethiopia. These services operate under a government-funded organization that delivers free emergency services, including out-of-hospital medical care and transportation to the most appropriate health facility. This study aimed to assess the challenges of pre-hospital emergency care at the Addis Ababa Fire and Disaster Risk Management Commission in Addis Ababa, Ethiopia.

A qualitative descriptive study was conducted from November 20 to December 4, 2022. Data were collected through in-depth, semi-structured interviews with 21 experienced individuals in the field of pre-hospital emergency care, who were selected using purposeful sampling. A thematic analysis method was used to analyze the data.

This study includes twenty-one participants working at the Addis Ababa Fire and Disaster Risk Management Commission. Three major themes emerged. The themes that arose were the participants’ perspectives on the challenges of pre-hospital emergency care in Addis Ababa, Ethiopia.

Conclusion and recommendation

The Fire and Disaster Risk Management Commission faces numerous challenges in providing quality pre-hospital emergency care in Addis Ababa. Respondents stated that infrastructure, communication, and resources were the main causes of pre-hospital emergency care challenges. There has to be more focus on emergency management in light of infrastructure reform, planning, staff training, and education, recruiting additional professional power, improving communication, and making pre-hospital emergency care an independent organization in the city.

Peer Review reports

EMS (Emergency Medical Service) systems provide a community’s gateway to acute and emergency medical care for members of the public with time-sensitive conditions, critical illnesses, and injuries. [ 1 ] The actions taken to provide medical care are time-dependent, starting with the site of the injury (scene), ambulance transportation, and medical facility treatment. [ 2 ]

Pre-hospital emergency medical care has many challenges, including unpredictable patient profiles, emergency conditions, and care administration in a non-medical area. The process before medical interventions, such as justice, stigmatization, dangerous situations, and safe driving; the treatment process, such as triage, refusal of treatment or transport, and informed consent; and the end of life and care are all ethical aspects of pre-hospital emergency medicine challenges. [ 3 , 4 , 5 ]

Pre-hospital emergency transportation initiatives and interventions, which include different transportation modes and financial supply, have been implemented in different low and middle-income countries (LMICs) [ 5 , 6 ] by government and nongovernment or private organizations to improve emergency medical transportation and to tackle treatment delay challenges related to the absence effective mode of transportation in pre-hospital care. [ 7 ]

In recent years, low-income countries, including Ethiopia, have been suffering from a double burden of communicable and non-communicable diseases, necessitating emergency care for managing injury and chronic disease complications simultaneously. [ 8 , 9 ] Reports noted that implementing emergency care systems can reduce mortality and disability-adjusted life years (DALYs) by 45% and 35%, respectively, in LMICs. [ 10 ]

Pre-hospital emergency care services in LMICs are hampered by a variety of issues, such as distance, fuel costs, nonfunctioning ambulances, unaffordable ambulances, inaccessible ambulances, poorly managed ambulance systems, poor network or communications, lack of awareness of the programmed, use of traditional modes of transportation, perceived response times, and other misconceptions. [ 11 , 12 ]

The availability and efficacy of an adequate pre-hospital care system in Ethiopia are minimal. However, timely access to health facilities during emergency cases is a crucial element in emergency conditions, and using an appropriate pre-hospital transportation mode has contributed to the reduction of mortality by preventing a delay in treatment. [ 13 , 14 ] In Addis Ababa city, organizations such as the Red Cross, Addis Ababa Fire and Disaster Risk Management Commission (AAFDRMC), and other private institutions provide pre-hospital medical services for emergency patients or laboring pregnant women. [ 15 , 16 ] The AAFDRMC is responsible for providing pre-hospital emergency services in Addis Ababa, Ethiopia. These services operate under a government-funded organization that delivers free emergency services, including out-of-hospital medical care and transportation to the most appropriate health facility. The AAFDRMC has one central dispatch center for fire and pre-hospital services, eight ambulance stations, and about 32 ambulances. The authority provides free pre-hospital care, including scene-to-health facility and inter-facility transfers, with care providers being nurses with short-term pre-hospital patient care training. The AAFDRMC also aims to improve health facilities’ response capacity and preparedness to cope with the challenges at the time of a disaster, such as mass-casualty incidents (MCIs), by implementing emergency and disaster preparedness plans.

In Ethiopia and other Sub-Saharan countries, providing pre-hospital care is beset by various challenges that impede its effectiveness. [ 17 ] These include a dearth of Emergency Medical Technicians (EMTs) and certified professional paramedics, inadequately trained ambulance crews, and a suboptimal ambulance system. These challenges further complicate the referral process, leading to patients with emergency cases being transported via ineffective modes of transportation. Those results increase the likelihood of disability, increase the severity of pain, increase the failure rate of rescue, contribute to treatment delay, and result in high morbidity and mortality. [ 15 ]

Very few studies have been conducted on pre-hospital emergency care service delivery in Ethiopia. Understanding emergency care service delivery challenges is essential to monitor, evaluate, and implement programs to improve pre-hospital emergency care. This study was intended to explore the challenges of pre-hospital emergency medical care at the AAFDRMC in Addis Ababa, Ethiopia.

Materials and methods

Study design, area, and period.

An institutional-based qualitative descriptive study was conducted at the Addis Ababa Fire and Disaster Risk Management Commission (AAFDRMC), Addis Ababa, Ethiopia, from November 20 to December 4, 2022.

Addis Ababa is the capital city of Ethiopia. The city has over five million people across eleven sub-cities and over a hundred woredas. The city is located at the heart of the country, at an altitude ranging from 2,100 m at Akaki in the south to 3,000 m at Entoto Hill in the north. Pre-hospital medical service is given by the AAFDRMC, the Ethiopian Red Cross Society, the Addis Ababa branch (ERCS), and a few private, for-profit pre-hospital service-giving organizations, including Tebita Ambulance. The AAFDRMC was established in 1934 G.C. to prevent and control fire and related accidents. In 2008, it started to provide pre-hospital care to the community. It has more than 160 emergency care providers and nine stations, of which the main branch is in the Arada sub-city.

This study is reported per the Consolidated criteria for reporting qualitative research (COREQ) Checklists. [ 18 ]

Participant recruitment

All healthcare professionals, dispatch center workers, ambulance drivers, and case team managers working at AAFDRMC with experience of at least one year were eligible to be included in this study. Emergency healthcare providers who were unavailable during data collection and supportive staff who did not have direct involvement or contact with the emergency care process were excluded.

Participants were recruited using a maximum variation purposive sampling technique, which was used across the different groups of participants to provide a realistic perspective concerning the pre-hospital emergency care challenges. The endpoint for sample selection was information saturation, meaning that further data collection failed to provide additional information or new codes were not developed. The study’s participants were twenty-one AAFDRMC workers, all with at least a year prior experience in pre-hospital emergency care.

Data collection

Development of an interview guide.

Data were collected using in-depth, semi-structured interviews with participants regarding the process of EMS in the affected area. An interview guide was prepared after reviewing different literature and reviewed by experts in the field of pre-hospital emergency care. It was initially written in English, then translated into Amharic, and then back to English to ensure consistency. (Supplementary Information 1 )

Training of interviewers

In-depth interviews were conducted by two healthcare providers (emergency nurses) from St. Paul Hospital Millennium Medical College who have experience in pre-hospital medical care for more than a year. A one-day simulation-based training session regarding the interview guide and ethics of qualitative research interviews was given to the data collectors, and the research team closely supervised them.

Interview process

Interviews with the participants started with their experience regarding pre-hospital emergency care, and according to the interview guidelines, general open-ended questions were asked, for instance, “Describe an infrastructure problem when you participated in pre-hospital emergency response.” Then, depending on the context of the responses, the interviewer continued with probe questions such as “Could you explain more?”

The time and location of the interviews were arranged by agreement with the participants. A total of 21 in-depth interviews were conducted. The interviews varied in length from 31 to 62 min. Interviews were digitally recorded with the informed consent of participants, and the recordings were transcribed verbatim.

Study participants were informed of their right to withdraw from the study at any point. Participants were assigned pseudonym identifiers to ensure anonymity in the research report and any publications. Identifying information was removed from transcripts before analysis.

Transcription and translation of the data

The data collectors transcribed the audio recordings to the local language, Amharic. To minimize errors and ensure faithfulness to the original recordings, a second data collector reviewed a random selection of transcripts for accuracy To ensure the truthfulness of the transcription process, we presented the transcribed document to the first five participants to check for accuracy. Any discrepancies or missing information identified by the participants were addressed and corrected in the transcripts.

Following participant verification, the Amharic transcripts were translated into English by a professional translator experienced in qualitative research. The translator ensured a conceptually accurate rendering of the data while preserving the original meaning and intent of the participants’ words.

Data storage

Digital recordings were stored securely on encrypted storage devices. Transcripts were stored electronically on password-protected computers or with access restricted to the research team. Paper copies of transcripts were kept in locked cabinets.

Data analysis

We applied a thematic analysis method, described by Braun and Clarke, to identify, examine, and report patterns within the data to find themes. [ 19 ] Two researchers (FY and BC) performed coding, and the discrepancies raised were handled by discussion. Coding frames were generated after repeated line-by-line reading of the interview transcript. Generated codes and collated them into categories, further refined and organized into potential themes that directly corresponded to the pre-hospital emergency care challenges. Open Code 4.03 software was used in the coding process, enabling the application of clearly defined coding criteria, minimizing ambiguity and enhancing the reliability of the coding scheme.

Trustworthiness was ensured by incorporating various components at each stage of the analysis. Credibility was maintained by faithfully representing the participants’ words and perspectives in the final report. Dependability was established through a detailed description of the research methods, facilitating study replication. Conformability was achieved by assuming a follower role during interviews, allowing participants to shape the discussions, and seeking clarification when needed. Transferability was demonstrated by providing sufficient details about the study site, participants, and data collection methods, allowing readers to assess the potential applicability of the findings to other contexts.

Twenty-one people were interviewed, of which 67% (14/21) were females, consisting of 13 ambulance nurses, two dispatchers, two ambulance drivers, three case team managers, and one pre-hospital manager. The participants ranged in age from 26 to 59 years, averaging 42.5 (S.D. ±7.60) years. (Table  1 ).

Pre-hospital emergency medical care challenges were classified under three main themes, including challenges related to infrastructure, communication, and resources for pre-hospital care. (Supplementary Information 2 )

Theme 1: challenges relating to infrastructure

Difficulty to access was because of the geographical condition of the city, the lack of emergency roads in the city, poor road conditions and poor road networks, sharing roads between the public and emergency vehicles, increased travel distance (wide coverage area of branches), a lack of road signs, eroded terrain, and narrow roads. Seasonal difficulties were commonly reported, such as difficulty passing through roads during the rainy season, traffic crowds, and insufficient ambulances at a branch.

“…Especially around 12 hr. post meridian (pm) local time , it is difficult to arrive at the scene due to traffic congestion , a lack of emergency roads , and narrow roads; as a result , the patient receives emergency care for a longer period ”. p01 (Ambulance nurse) “…During the rainy season , roads around the city’s outskirts become very muddy or are destroyed by floods , making timely access to the scene difficult”. p07 (Ambulance driver) “…Due to the lack of a different route for ambulances and low community awareness regarding the use of an ambulance , it is difficult to reach the site quickly when an emergency call comes in.” p17 (Ambulance driver) “…The Kaity Fire and Emergency Rescue Branch has a large catchment area. Referral hospitals are located far from health facilities , and the institution has only two ambulances for its large catchment area , making it difficult to address many calls in the short time interval available. For example , the Trunesh Beijing Hospital is far from the Catholic and Bisrate Gabriel referral catchment areas; these health facilities and emergency patients deteriorate during the journey” p15 (Ambulance case team manager) “…The shortage of trained paramedic professionals and EMT staff , as well as the ineffective distribution of emergency pre-hospital centers , are major challenges that this center faces and must be addressed as soon as possible.” p20 (Ambulance case team manager)

Most participants also mentioned infrastructural problems within the Addis Ababa Fire and Risk Management Commission that can affect the proper delivery of pre-hospital care. The problem includes no appropriate ambulance entrance and standing area at local health facilities. Some hospital lifts are crowded and far from ambulance standing areas to transfer patients. Participants reported that the liaison of some hospital areas are hidden and far from the emergency room and labor and delivery rooms, and wheelchair or stretcher roads were poorly constructed.

“…When we arrive at the hospital , there is no ambulance standing area. The corridors are not conducive to riding stretchers… The liaison room is hidden and difficult to find… This causes a delay in handing over the patient on time”. p6 (Ambulance nurse)

In a fire and emergency rescue commission institution, infrastructure challenges include no appropriate standing place for ambulances, no shower room, no laundry service room, no appropriate pits to store wastes and no incineration places, no properly prepared area for ambulance washing, and no nursing station near the site of an ambulance standing place.

“… In our branch , there is no properly prepared ambulance standing area , and the ambulance is exposed to the sun at its standing area throughout the day , making the interior of the ambulance very hot. And after we leave the hospitals , cleaning the ambulance is difficult because there is no ambulance washing area , nurse’s shower room , or laundry service”. p13 (Ambulance Nurse)

Theme 2: challenges relating to communication

Based on the participants’ views, several challenges were pointed out regarding communication. Because the line is free to call, many people continuously call from the community for joking or fake calls. Misinformative incoming phone calls, such as calling ambulances for non-emergency cases, misinformation about the case, misinformation about the scene, not purely heard calls, and false callings were also common.

“… Many calls come in , especially at night , and when I answer them , most are dropped , or I do not know where they are from… Some are for making fun of and insulting others… Some make false calls , making the line busy for true callers”. p10 (Dispatcher)

Another participant from the dispatch center also stated regarding prank calls below

“One of the most common challenges is that some people call our emergency phone numbers and report false signs and symptoms , wasting the staff’s time and energy.” p12 (Dispatcher)

At the dispatch center, a communication gap identified was misinformation about the type of case, place of call, and catchment area. Since dispatchers are not health professionals and lack skills, they cannot correctly understand medical terms and not forward detailed information to ambulance nurses. The dispatch center forwarded the message to the ambulance nurse after holding it for many reasons; local health facilities and the community also did not correctly tell the status of the patient and type of case when they were communicating with the dispatch person or ambulance nurse, and large areas were not covered because there was no network (telecommunication problem) and there were limitations in radio communications.

“…Always , there is a conflict between the ambulance nurse and local health facility staff because , when they make the call , they do not forward the type of case and status of the patient correctly to the dispatch centers , and vital signs are not correctly recorded”. p5 (Ambulance nurse) “Most of the time , the dispatcher has incorrect information about the citation and type of call , and they transfer the call to the ambulance nurse very late… As a result of not being psychologically and materially prepared to handle emergency situations correctly”. p14 (Ambulance nurse)

Shortage of communication technology equipment was also one of the major issues within the institution. There are no GPS and GIS technologies in dispatch or ambulance to identify where the call came from and where the ambulance was reported in adequate communication radios, which are very old and outdated.

“…We frequently become perplexed when we are dispatched on a mission since Addis Ababa lacks a reliable GPS system. We are unsure of the best and closest route to take to reach the victims”. p03 (Ambulance case team manager) “…We only have one communication radio on each of our two operational ambulances. If a call comes from anywhere and one ambulance goes , the left ambulance cannot hear any call or communication from the dispatch center. At this time , we use our phone , which is very late for communication”. p8 (Ambulance nurse)

Regarding inter-liaison communication, there was poor or no clear communication between liaisons at health facilities about the case and status of the patient prior to the call to dispatch centers.

“…When we arrived at the hospital , there was little or no communication between the referring health facility and the catchment hospital… Emergency department staff are not voluntary in handing over patients , and we make calls to the command post… This prolongs the patient’s access to care”. p09 (Ambulance Nurse)

Theme 3: challenges relating to resources

Lack of medical commodities like a shortage of ambulances, modernized ambulances, insufficient portable oxygen, defective vital sign materials, insufficient emergency equipment, and, in some cases, a lack of emergency medication (refill problem) are frequently cited challenges.

“…When receiving emergency patients from a health facility , monitoring their vital signs is extremely difficult due to nonfunctional or a lack of vital sign monitoring materials. And no adequate emergency medication in the ambulance…” p04 (Ambulance nurse)

Regarding infection prevention, medical materials challenges include the lack of adequate personal protective equipment (PPE), such as gowns and inappropriate gown colors that do not show contamination. Whole covers, face shields, head covers, gloves, boots, eye goggles, and aprons were reported.

“…You do not receive fluid-proof PPE , such as eye goggles. Even if we are handling trauma cases and patients with highly contagious diseases… The majority of what is available to you are cloth uniforms…” p11 (Ambulance nurse)

Inadequate funding for personal protective equipment in pre-hospital emergency rooms has become a major issue.

“There is insufficient personal protective equipment due to insufficient funds allocated to this center. Given that all EMS personnel are at the forefront of the fight against the most infectious diseases such as tuberculosis , HIV/AIDS , and COVID-19 , and are in direct contact with patients , we should not be subjected to such stresses and challenges.” p21 (Ambulance nurse)

In respect to sanitization and hygiene materials, inadequate sanitizer, alcohol, and bleach were reported.

“…In pre-hospital emergencies , since cross-contamination is very high in the ambulance system , infection prevention activity is mandatory , but we do not get any alcohol for hand rubs. There is no place prepared to clean an ambulance and stretcher following a trauma arrival…” p02 (Ambulance nurse).

From the point of view of our participant, financial constraints were the most common problem which can hinder the service. There are not enough year budgets, the pre-hospital does not have its own budget, and any material for service is decided through a process or discussion that takes a long time. Sometimes the commission’s top executives ignore the requested budget or provide a very small budget for purchasing materials.

“…There is no independent budget for pre-hospital care… To buy emergency medication and materials , after many challenges and a long time , you get little response or no response at all since they ignore the pre-hospital service… and this makes it difficult to give quality pre-hospital emergency care…” p19 (Pre-hospital care manager).

According to the participants, human resource/staff-related challenges most commonly highlighted were only a few diploma/degree nurses, no physicians, no paramedic professionals, no training programs to keep educational levels up to date, no training center, no detailed training if new epidemics emerge, no library to read, no internet access, no procedure room to improve skills, no research center in the institution, lack of career development, and no risk payment.

“…I work as an ambulance nurse. I have six years of experience as a diploma nurse… there is the possibility of further education…The top managers decided that a diploma or level 4 nurse is enough to give pre-hospital care… no B.Sc. nurse , no trained paramedics at all… Despite the fact that there is an emergency , an epidemic case , and no training…” p18 (Ambulance nurse).

We work in a shifting program with no duty Monday through Friday. “ It is known that working in pre-hospital emergencies has a higher risk of infection and life-threatening injuries… However , there is still a payment risk…” p16 (Ambulance nurse) .

The adverse chain of management includes a lack of collaboration and participation with other organizations; every activity in pre-hospital emergency medical care is only controlled by the top managers of the commissioner, who are nonprofessionals, and middle and bottom managers have no independent decision-making activity.

“… Pre-hospital care is dependent on the commission , and this makes the pre-hospital service poorly developed and addresses community emergency calls easily” p19 (Pre-hospital care manager) .

The capacity of emergency care services is typically insufficient during severe emergencies and disasters, and local community resources are likely to be overwhelmed. [ 20 ] Emergency care may not be available, especially in the early hours following an emergency. [ 21 , 22 ] In this case, the public’s involvement in delivering first aid may be crucial. In response, some organizations around the world have recently offered civilian-based pre-hospital guidelines, public education, and exercises for communities. [ 23 ] Based on participants’ experiences, the results of this study showed that infrastructure plays a vital role in hospital and pre-hospital emergency medical services. In Addis Abeba City, infrastructure issues such as poor road construction and networks, geographic problems, a lack of emergency roads, seasonal problems, traffic jams, and narrow roads are frequently blamed for causing delays or making access to accident scenes or hospitals difficult. Other infrastructure issues with local health facility distance include inadequate ambulance parking, lift issues, and a lack of a liaison room. These factors have also been reported in previous studies in Iran. [ 2 , 24 ]

Focusing on the challenges relating to communication, an inappropriate telecommunication system, a communication gap or miscommunication, false calling, inadequate communication radios, being nonprofessional dispatchers, a lack of Geographic Information Systems (GIS) and Global Positioning Systems (GPS), and poor communication between and within facilities are important factors that affect it. Various studies have recommended several enhancements to improve communication and situational awareness in emergency response systems. These include modernizing infrastructures, implementing clear and standardized communication protocols, providing training programs for dispatchers, and incorporating GIS and (GPS). [ 24 , 25 , 26 , 27 , 28 ]

Regarding the challenges relating to resources for pre-hospital care, insufficient professional staffing, not enough ambulances and EMS centers, non-paramedic activities and a lack of motivation among some EMS personnel, a lack of resources at EMS dispatch centers, non-health professionals at the dispatch center, and insufficient nurses for substitution are essential barriers. This finding is consistent with the findings of another study. [ 24 , 25 , 29 , 30 ] Related studies have recognized the significant stress experienced by paramedics and EMTs, resulting in a lack of motivation to perform their duties effectively. To combat this issue, it has been suggested that investing in staff well-being through competitive salaries, providing opportunities for career development, and fostering a supportive work environment can help alleviate these challenges. [ 31 , 32 ]

The participants in this study indicated that pre-hospital emergency care was a neglected part of service in the Addis Ababa Fire and Emergency Rescue Commission. The organization’s structural management is entirely non-health professional, resulting in nonprofessional decisions that result in service flaws and poor response. This result is consistent with the study conducted in Gabon [ 3 ]. Establishing advisory boards with a strong representation of clinical professionals like paramedics, emergency physicians, and nurses, creating joint committees, improving communication channels, and considering qualified paramedics or emergency medical professionals in leadership positions is paramount to addressing the abovementioned challenges. [ 33 , 34 ] The participant also demonstrated challenge-related education, such as a lack of training and no opportunity for advancement, resulting in poor staff knowledge. According to the study participants, most Addis Ababa EMS providers have inadequate skills in handling new epidemic emergency cases. Past studies supported this result. [ 24 , 25 , 35 ] Emergency care cross-contamination in the ambulance system is very high, and infection prevention activity is mandatory. From many participants’ points of view, infection prevention and control factors, including lack of PPE, a uniform for a new employee, a shower room, an ambulance washing place, and adequate solutions and detergents, are the main challenges to preventing infection, which was supported by a study conducted in Gabon and Jimma. [ 3 , 35 ]

To the best of our knowledge, this is a novel study exploring the challenges of pre-hospital emergency medical care in Addis Ababa, Ethiopia, which will significantly impact plans to improve the quality of pre-hospital care. As a limitation of this study due to financial constraints, we did not conduct a focused group discussion, which might have provided a better understanding of pre-hospital medical care challenges.

Conclusion and recommendations

The findings of this study revealed that the pre-hospital emergency care provided by the Addis Ababa Fire and Disaster Risk Management Commission in previous emergencies was chaotic. It faces several challenges that limit its ability to provide quality emergency care, including traffic congestion, a lack of an emergency road, a significant communication gap, a lack of professional power, and a lack of medical materials.

The establishment of a formal pre-hospital care system and the establishment of pre-hospital emergency care as an independent organization in the city, investment in infrastructure and infrastructure reform, staff training and education, recruiting additional professional power, public awareness campaigns, and more widely available emergency medical training are all viable solutions to the current barriers to access. Further studies should also be conducted using a multi-center approach that includes the views of different stakeholder groups.

Data availability

Data and materials will be shared upon reasonable request.

Abbreviations

Addis Ababa Fire and Disaster Risk Management Commission

Emergency Medical Services

Emergency Medical Technician

Low- and Middle-Income Countries

Non-Governmental Organization

Mould-Millman N, Naidoo R, De Vries S, Stein C, Wallis L. AFEM Consensus Conference, 2013. AFEM out-of-hospital emergency care workgroup consensus paper: advancing out-of-hospital emergency care in Africa-advocacy and development. Afr J Emerg Med. 2014;4(2):90–5.

Article   Google Scholar  

Bigdeli M, Khorasani-Zavareh D, Mohammadi R. Pre-hospital care time intervals among victims of road traffic injuries in Iran. A cross-sectional study. BMC Public Health. 2010;10:1–7.

Bosson N, Redlener MA, Foltin GL, Raven MC, Foran MP, Wall SP. Barriers to utilization of pre-hospital emergency medical services among residents in Libreville, Gabon: a qualitative study: Obstacles à L’utilisation Des services d’urgence médicale préhospitaliers parmi les habitants de Libreville, Gabon: une étude qualitative. Afr J Emerg Med. 2013;3(4):172–7.

Erbay H. Some ethical issues in pre-hospital emergency medicine. Turkish J Emerg Med. 2014;14(4):193–8.

Kironji AG, Hodkinson P, de Ramirez SS, Anest T, Wallis L, Razzak J, et al. Identifying barriers for out of hospital emergency care in low and low-middle income countries: a systematic review. BMC Health Serv Res. 2018;18(1):1–20.

Pacagnella RC, Cecatti JG, Osis MJ, Souza JP. The role of delays in severe maternal morbidity and mortality: expanding the conceptual framework. Reprod Health Matters. 2012;20(39):155–63.

Article   PubMed   Google Scholar  

Kobusingye OC, Hyder AA, Bishai D, Hicks ER, Mock C, Joshipura M. Emergency medical systems in low-and middle-income countries: recommendations for action. Bull World Health Organ. 2005;83(8):626–31.

PubMed   PubMed Central   Google Scholar  

Jamison DT, Breman JG, Measham AR, Alleyne G, Claeson M, Evans DB et al. Disease control priorities in developing countries. 2006.

Denu ZA, Osman MY, Bisetegn TA, Biks GA, Gelaye KA. Barriers and opportunities of establishing an integrated pre-hospital emergency response system in North West Ethiopia: a qualitative study. Inj Prev. 2022;28(4):347–52.

Chang CY, Abujaber S, Reynolds TA, Camargo CA Jr., Obermeyer Z. Burden of emergency conditions and emergency care usage: new estimates from 40 countries. Emerg Med J. 2016;33(11):794–800.

Gebreegziabher A, Medhanyie AA, Meressa B, Hagazi M, Gessessew A. Determinants of ambulance service utilization among pregnant women in Mekelle City, Ethiopia: a case-control study. East Afr J Health Sci. 2019;1(1):88–99.

Google Scholar  

Mothers S, Life G, Emergency. Obstetric and Newborn Care Access and Availability.

Oguntunde O, Yusuf FM, Nyenwa J, Dauda DS, Salihu A, Sinai I. Emergency transport for obstetric emergencies: integrating community-level demand creation activities for improved access to maternal, newborn, and child health services in northern Nigeria. International journal of women’s health. 2018:773 – 82.

Godefay H, Kinsman J, Admasu K, Byass P. Can innovative ambulance transport avert pregnancy–related deaths? One–year operational assessment in Ethiopia. J Global Health. 2016;6(1).

Sultan M, Abebe Y, Tsadik AW, Ababa A, Yesus AG, Mould-Millman N-K. Trends and barriers of emergency medical service use in Addis Ababa; Ethiopia. BMC Emerg Med. 2019;19(1):1–8.

Sultan M, Abebe Y, Tsadik AW, Jennings CA, Mould-Millman N-K. Epidemiology of ambulance utilized patients in Addis Ababa, Ethiopia. BMC Health Serv Res. 2018;18:1–7.

Calvello E, Reynolds T, Hirshon JM, Buckle C, Moresky R, O’Neill J, et al. Emergency care in sub-saharan Africa: results of a consensus conference. Afr J Emerg Med. 2013;3(1):42–8.

Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.

Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Res Psychol. 2006;3(2):77–101.

Kirk MA, Deaton ML. Bringing order out of chaos: effective strategies for medical response to mass chemical exposure. Emerg Med Clin North Am. 2007;25(2):527–48.

Zhong S, Clark M, Hou X-Y, Zang Y, FitzGerald G. Progress and challenges of disaster health management in China: a scoping review. Global Health Action. 2014;7(1):24986.

Sheikhbardsiri H, Yarmohammadian MH, Khankeh H, Khademipour G, Moradian MJ, Rastegarfar B, et al. An operational exercise for disaster assessment and emergency preparedness in south of Iran. J Public Health Manage Pract. 2020;26(5):451–6.

Bobko JP, Harris WJ, Thomas S. The First Care Provider System: improving Community Resilience for unexpected disasters. Why civilians should be prepared to act in mass-trauma events. EMS World. 2016;45(3):32.

PubMed   Google Scholar  

Sorani M, Tourani S, Khankeh HR, Panahi S. Pre-hospital emergency medical services challenges in disaster; a qualitative study. Emergency. 2018;6(1).

Khorasani-Zavareh D, Mohammadi R, Bohm K. Factors influencing pre-hospital care time intervals in Iran: a qualitative study. J Injury Violence Res. 2018;10(2):83.

Moghadam SAB, Toorchi M, Heris YS. Knowledge and attitude of Iranian red crescent society volunteers in dealing with bioterrorist attacks. Emergency. 2016;4(1):16.

Lam RPK, Leung LP, Balsari S, Hsiao K-h, Newnham E, Patrick K, et al. Urban disaster preparedness of Hong Kong residents: a territory-wide survey. Int J Disaster risk Reduct. 2017;23:62–9.

Article   PubMed   PubMed Central   Google Scholar  

Pouraghaei M, Babaie J, Rad Saeed L. Challenges of Emergency Medical Services Response to Arasbaran Twin earthquakes; a content analysis. Arch Acad Emerg Med. 2022;10(1):e35.

Murphy A, Barrett M, Cronin J, McCoy S, Larkin P, Brenner M, et al. A qualitative study of the barriers to pre-hospital management of acute pain in children. Emerg Med J. 2014;31(6):493–8.

Alinia S, Khankeh H, Maddah SSB, Negarandeh R. Barriers of pre-hospital services in road traffic injuries in Tehran: the viewpoint of service providers. Int J Community Based Nurs Midwifery. 2015;3(4):272.

Kurth JD, Powell JR, Gage CB, Fauvel AD, Crowe RP, Cash RE, et al. Evaluating changes in the emergency medical services workforce: a preliminary multistate study. J Am Coll Emerg Physicians Open. 2023;4(3):e12975.

Parker L, Prior SJ, Van Dam PJ, Edwards DG. Altruism in Paramedicine: a scoping review. Healthc (Basel). 2022;10(9).

Haugland H, Rehn M, Klepstad P, Krüger A, Albrektsen GE, Berlac PA, et al. Developing quality indicators for physician-staffed emergency medical services: a consensus process. Scand J Trauma Resusc Emerg Med. 2017;25(1):14.

Bashiri A, Alizadeh Savareh B, Ghazisaeedi M. Promotion of pre-hospital emergency care through clinical decision support systems: opportunities and challenges. Clin Exp Emerg Med. 2019;6(4):288–96.

Ibro SA, Matthew L, Shemsi SS, Akuma AO, Haso TK, Ahmed AA, et al. Perceived use of health care service: barriers to access pre-hospital care in Jimma City. Oromia Region of Ethiopia; 2022.

Download references

Acknowledgements

We thank the employees of the Addis Ababa Fire and Disaster Risk Management Commission for their help throughout data collection.

This research received financial aid from Saint Paul’s Hospital Millennium Medical College. The funder has no role in the study design, analysis, and interpretation of data.

Author information

Authors and affiliations.

Department of Emergency and Critical Care, St. Paul Hospital Millennium Medical College, Addis Ababa, Ethiopia

Feleku Yimer Seid & Birhanu Chekol Gete

Department of Anesthesia, College of Medicine and Health Science, Bahir Dar University, Bahir Dar, Ethiopia

Amanuel Sisay Endeshaw

You can also search for this author in PubMed   Google Scholar

Contributions

F.Y designed the study, analysed the data and drafted the manuscript. B.C and A.S were involved in the design, analysis of the data, drafting of the manuscript and critically reviewing the article. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Amanuel Sisay Endeshaw .

Ethics declarations

Ethics approval and consent to participate.

The Institutional Review Board (IRB) of St. Paul’s Hospital Millennium Medical College, Addis Ababa, Ethiopia (Ref №: Pm 23/282) granted ethical approval to conduct this study. Permission to conduct this study was obtained from the Addis Ababa Fire and Disaster Risk Management Commission. All study participants gave written informed consent after being informed of the objective, advantages, and risks of participating. Information about the study subjects’ anonymity was kept confidential.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1: Interview guide

Supplementary material 2: coding tree for thematic analysis, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Seid, F.Y., Gete, B.C. & Endeshaw, A.S. Challenges of pre-hospital emergency care at Addis Ababa Fire and Disaster Risk Management Commission, Addis Ababa, Ethiopia: a qualitative study. BMC Health Serv Res 24 , 803 (2024). https://doi.org/10.1186/s12913-024-11292-6

Download citation

Received : 26 January 2024

Accepted : 08 July 2024

Published : 11 July 2024

DOI : https://doi.org/10.1186/s12913-024-11292-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Pre-hospital care
• Emergency care
• Dispatch center

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]