focus group discussion literature review

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Glob Qual Nurs Res
• v.3; Jan-Dec 2016

Methodological Aspects of Focus Groups in Health Research

Anja p. tausch.

1 GESIS–Leibniz Institute for the Social Sciences, Mannheim, Germany

Natalja Menold

Although focus groups are commonly used in health research to explore the perspectives of patients or health care professionals, few studies consider methodological aspects in this specific context. For this reason, we interviewed nine researchers who had conducted focus groups in the context of a project devoted to the development of an electronic personal health record. We performed qualitative content analysis on the interview data relating to recruitment, communication between the focus group participants, and appraisal of the focus group method. The interview data revealed aspects of the focus group method that are particularly relevant for health research and that should be considered in that context. They include, for example, the preferability of face-to-face recruitment, the necessity to allow participants in patient groups sufficient time to introduce themselves, and the use of methods such as participant-generated cards and prioritization.

Focus groups have been widely used in health research in recent years to explore the perspectives of patients and other groups in the health care system (e.g., Carr et al., 2003 ; Côté-Arsenault & Morrison-Beedy, 2005 ; Kitzinger, 2006 ). They are often included in mixed-methods studies to gain more information on how to construct questionnaires or interpret results ( Creswell & Plano Clark, 2007 ; Kroll, Neri, & Miller, 2005 ).

The fact that the group process helps people to identify and clarify their views is considered to be an important advantage of focus groups compared with individual interviews ( Kitzinger, 1995 ). The group functions as a promoter of synergy and spontaneity by encouraging the participants to comment, explain, disagree, and share their views. Thus, experiences are shared and opinions voiced that might not surface during individual interviews ( Carey, 1994 ; Stewart, Shamdasani, & Rook, 2007 ). Although focus groups allow participants to respond in their own words and to choose discussion topics themselves, they are not completely unstructured. Questions relating to the research topic are designed by the researchers and are used to guide the discussion ( Stewart et al., 2007 ). The degree of structure of the focus group depends on the openness of the research question(s). Hence, although it takes more time and effort to organize focus groups, and they cause greater logistical problems than individual interviews do, they might generate more ideas about, and yield deeper insights into, the problem under investigation ( Coenen, Stamm, Stucki, & Cieza, 2012 ; Kingry, Tiedje, & Friedman, 1990 ; Morgan, 2009 ).

Historically, focus groups were used mainly for market research before the method was adopted for application in qualitative research in the social sciences ( Morgan, 1996 ). The use of focus groups in health care research is even more recent. For this reason, methodological recommendations on using focus groups in the health care context are quite rare, and researchers rely mainly on general advice from the social sciences (e.g., Krueger, 1988 ; Morgan, 1993 ; Morgan & Krueger, 1998 ; Stewart et al., 2007 ). Even though focus groups have been used in a great variety of health research fields, such as patients’ treatments and perceptions in the context of specific illnesses (rheumatoid arthritis: for example, Feldthusen, Björk, Forsblad-d’Elia, & Mannerkorpi, 2013 ; cancer: for example, Gerber, Hamann, Rasco, Woodruff, & Lee, 2012 ; diabetes: for example, Nafees, Lloyd, Kennedy-Martin, & Hynd, 2006 ; heart failure: for example, Rasmusson et al., 2014 ), community health research (e.g., Daley et al., 2010 ; Rhodes, Hergenrather, Wilkin, Alegría-Ortega, & Montaño, 2006 ), or invention of new diagnostic or therapeutic methods (e.g., Vincent, Clark, Marquez Zimmer, & Sanchez, 2006 ), the method and its particular use in health research is rarely reflected. Methodological articles about the focus group method in health care journals mainly summarize general advice from the social sciences (e.g., Kingry et al., 1990 ; Kitzinger, 1995 , 2006 ), while field-specific aspects of the target groups (patients, doctors, other medical staff) and the research questions (not only sociological but often also medical or technical) are seldom addressed. Reports on participant recruitment and methods of conducting the focus groups are primarily episodic in nature (e.g., Coenen et al., 2012 ; Côté-Arsenault & Morrison-Beedy, 2005 ) and often focus on very specific aspects of the method (communication: for example, Lehoux, Poland, & Daudelin, 2006 ; activating methods: for example, Colucci, 2007 ) or aim at a comparison between face-to-face focus groups and other methods (individual interviews: for example, Coenen et al., 2012 ; telephone groups: for example, Frazier et al., 2010 ; Internet groups: for example, Nicholas et al., 2010 ). Thus, systematic reviews of factors influencing the results of focus groups as well as advantages, disadvantages, and pitfalls are missing. One consequence is that researchers might find it difficult to recruit enough participants or might be surprised by the communication styles of the target groups. Furthermore, in the tradition of classical clinical research, the group discussions might result in a question-and-answer situation or “resemble individual interviews done in group settings” ( Colucci, 2007 , p. 1,424), thereby missing out on the opportunity to use the group setting to activate all participants and to encourage a deeper elaboration of their ideas. Colucci, for example, proposed the use of exercises (e.g., activity-oriented questions) to focus the attention of the group on the core topic and to facilitate subsequent analyses.

Recommendations from the social sciences on using the focus group method can be subsumed under the following headings: subjects (target groups, composition of groups, recruitment), communication in the groups (discussion guide, moderator, moderating techniques), and analysis of focus groups (e.g., Morgan, 1993 ; Morgan & Krueger, 1998 ; Stewart et al., 2007 ). Specific requirements for health research can be identified in all three thematic fields: Recruitment might be facilitated by using registers of quality circles to recruit physicians or pharmacists, or by recruiting patients in outpatients departments. It might be hampered by heavy burdens on target groups—be they time burdens (e.g., clinical schedules, time-consuming therapy) or health constraints (e.g., physical fitness). With regard to communication in focus groups, finding suitable locations, identifying optimal group sizes, planning a good time line, as well as selecting suitable moderators (e.g., persons who are capable of translating medical terms into everyday language) might pose a challenge. The analysis of focus groups in health care research might also require special procedures because the focus group method is used to answer not only sociological research questions (e.g., related to the reconstruction of the perspectives of target groups) but also more specific research questions, such as user requirements with regard to written information or technical innovations.

The aim of our study was to gather more systematic methodological information for conducting focus groups in the context of health research in general and in the more specific context of the implementation of a technical innovation. To this end, we conducted interviews with focus group moderators about their experiences when planning and moderating focus groups. The groups in question were part of a research program aimed at developing and evaluating an electronic personal health record. We chose this program for several reasons: First, because it consisted of several subprojects devoted to different research topics related to the development of a personal electronic health record, it offered a variety of research content (cf. next section). Second, the focus groups were conducted to answer research questions of varying breadth, which can be regarded as typical of research in health care. Third, the focus groups comprised a variety of target groups—not only patients but also different types of health care professionals (general practitioners, independent specialists with different areas of specialization, hospital doctors, pharmacists, medical assistants, nursing staff).

In this article, we report the findings of these interviews in relation to the following questions: (a) What challenges associated with the characteristics of the target groups of health research (patients, physicians, other health care professionals) might be considered during the recruitment process? How should the specific research question relating to a technical innovation be taken into account during the recruitment process? (b) Should specific aspects of the communication styles of target groups be taken into account when planning and moderating focus groups in health care? Can additional challenges be identified in relation to the technical research question? and (c) How was the method appraised by the interviewees in their own research context?

Research Program and Description of Focus Groups

The “Information Technology for Patient-Centered Health Care” (INFOPAT) research program ( www.infopat.eu ) addresses the fact that, because patients with chronic conditions (e.g., colorectal cancer, type 2 diabetes) have complex health care needs, many personal health data are collected in different health care settings. The aim of the program is to develop and evaluate an electronic personal health record aimed at improving regional health care for chronically ill people and strengthening patients’ participation in their health care process. Subprojects are devoted, for example, to developing the personal electronic health record (Project Cluster 1), a medication platform (Project Cluster 2), and a case management system for chronically ill patients (Project Cluster 3). In the first, qualitative, phase, the researchers explored patients’ and health care professionals’ experiences with cross-sectoral health care and patient self-management, and their expectations regarding the advantages and disadvantages of a personal electronic health record. The information gathered in this phase of the program served as a basis for constructing a personal electronic health record prototype. This prototype was implemented as an intervention in a second, quantitative, phase dedicated to investigating the impact of such a record on a range of health care variables (e.g., self-management, health status, patient–doctor relationship, compliance). The University Hospital Heidelberg Ethics Committee approved the studies of the INFOPAT research program. All participants gave their written informed consent, and the participants’ anonymity and confidentiality were ensured throughout the studies according to the ethical standards of German Sociological Association. 1

Twenty-one focus groups were conducted during the qualitative phase of the program. Three groups consisted of colorectal cancer patients, four comprised type 2 diabetes patients, four were made up of physicians, three comprised physicians and pharmacists, four consisted of physicians and other health care professionals, and three consisted of other health care professionals (for more detailed information, see Tausch & Menold, 2015 ). Participants were recruited from urban and rural districts of the Rhine-Neckar region in Germany. Patients were approached in clinics, by their local general practitioners, or in self-help groups. Health care professionals were recruited in clinics, cooperating medical practices, and professional networks.

The focus groups took place at several locations at the National Center of Tumor Diseases (NCT) in Heidelberg, Germany, and the University of Heidelberg. The groups consisted of between four and seven participants and lasted between 1.5 and 2 hours. All focus groups were conducted by two researchers—a moderator and a co-moderator; a third researcher took notes. Semistructured discussion guides were used, and the groups were video- and audio recorded (cf., for example, Baudendistel et al., 2015 ; Kamradt et al., 2015 ). The researchers performed content analysis on the transcripts; the schema of categories was oriented toward the research questions. The focus groups addressed research questions of varying breadth, including, for example, individual health care experiences (comparatively broad), the expected impact of the record on the patient–doctor relationship (medium breadth), and technical requirements for such a personal health record (comparatively narrow). The variety of the research questions was important for our study because it proved to be of relevance for the interviewees’ appraisal of the usefulness of the focus group method.

Interviews With the Focus Group Moderators

We conducted qualitative interviews with nine of the 10 focus group moderators in the INFOPAT program (one moderator moved to a different department shortly after the completion of data collection and was not available for interview). The interviewees were aged between 30 and 54 years ( M age = 36 years; SD = 8.3 years). Their professions were health scientist, pharmacist, general practitioner, or medical ethicist. Their professional experience ranged from one to 23 years ( M = 7.1 years, SD = 7.7 years), and they had little or no previous experience of organizing and conducting focus groups. The moderators were interviewed in groups of one to three persons according to their project assignment (cf. Table 1 ).

Overview of Interviews and Interviewees.

The interviews lasted approximately 1 hour, and the interview questions were guided by the chronological order in which a focus group is organized and conducted (recruitment, preparation, moderation, methods) and by the utilization and usefulness of the results. We tape recorded the interviews, transcribed them verbatim, and performed qualitative content analysis on the transcripts ( Elo & Kyngäs, 2008 ; Mayring, 2015 ) with the help of the program MAXQDA 10.0.

The final system of categories 2 ( Tausch & Menold, 2015 ) consisted of two types of codes: All relevant text passages were coded with respect to the content of the statement. In addition, a second type of code was required if the statement related to a specific group of participants (e.g., patients, hospital doctors, men, women).

On the basis of the research questions, the contents of interview statements were classified into the three superordinate thematic categories: recruitment, communication in the focus groups, and appraisal of the focus group method. Consequently, the reporting of the results is structured according to three main topics.

Recruitment

Statements relating to the recruitment of the participants were sorted into the main categories “factors promoting participation”, “factors preventing participation”, and “general appraisal of the recruitment process”. Figure 1 shows the subcategories that were identified under these main categories. Because many of the statements referred only to patients or only to health care professionals (physicians, other health care professionals), the subcodes shown in Figure 1 are sorted by these two types of participants.

Factors relating to the recruitment process.

Factors relevant for all target groups

As the following interviewee statement shows, addressing potential participants face-to-face (rather than in writing) proved crucial for the success of recruitment in all target groups:

Well, a really good tip when recruiting patients is . . . to address the people yourself. Not to get someone else to do it who . . . has nothing to do with [the project], because ultimately you really do have to explain a lot of things, also directly to the patient. And then it’s always good if the person [who does the recruiting] is actually involved in the project. 3

In the case of the clinicians, being addressed by a superior was even more effective for their willingness to participate: “And then top down. If the nursing director asks me, then it’s not so easy to say no.”

Furthermore, a positive response was more often achieved if the groups were scheduled at convenient times for the addressees, and they only had to choose between several alternatives. Patients welcomed times contiguous with their therapies: “And many [of the patients] said: ‘Yes, maybe we can do it after my chemotherapy, on that day when I’m in the clinic anyway?’” Whereas medical assistants were given the opportunity to take part in the groups during working hours, general practitioners preferred evening appointments on less busy weekdays (e.g., Wednesdays and Fridays):

Well, what I found quite good was to suggest a day and a time. And we concentrated on the fact that practices are often closed on Wednesday afternoons. So that’s a relatively convenient day. And then evenings for the pharmacists from seven-thirty onwards.

Interest in the topic of the discussion, or at least in research in general, was an important variable for participation. Together with lack of time, it turned out to be the main reason why sampling plans could not be realized. Among patients, men were much more interested in discussing a technical innovation such as an electronic personal health record, while women—besides their lesser interest—often declined because of family responsibilities: “Well, I’d say a higher proportion of women said: ‘I have a lot to do at home, housework and with the children, therefore I can’t do it.’”

Family physicians, physicians from cooperating medical practices, and hospital doctors showed more interest in discussing an electronic personal health record than did medical specialists in private practice, who often saw no personal gain in such an innovation. For example, one interviewee stated,

Family physicians generally have a greater willingness [to engage with] this [health] record topic. They see . . . also a personal benefit for themselves. . . . or they simply think it might be of relevance to them or they are interested in the topic for other reasons. Some of them even approached us themselves and said, “Oh, that interests me and I’d like to take part.”

In addition, because of heavy workload, private practitioners were difficult to reach (e.g., by telephone). This also lowered the participation of this target group on the focus groups.

Factors relevant only for patients

Two other variables that influenced patients’ willingness to participate were mentioned in the interviews. First, because this target group consisted of cancer patients and diabetes patients with multimorbidity, poor physical fitness also prevented several addressees from participating in the groups. The inability to climb stairs, or the general inability to leave the house, made it impossible for them to reach the location where the groups took place: “[They] immediately replied: ‘Well, no, . . . that’s really too much for me,’ and unfortunately they could not, therefore, be included in the groups.” Furthermore, unstable physical fitness often led to high drop-out rates. The moderators of the focus groups therefore proposed that up to twice as many participants as required should be recruited: “And depending on the severity of the illness, you have to expect a drop-out rate of up to fifty percent. So, if you want to have four people, you should invite eight.”

Second, moderators reported that patients’ liking for, or dislike of, talking and discussing influenced their tendency to join the groups. Participating patients were generally described as talkative. For example: “And with patients, all in all, I had the feeling that those who agreed [to participate] were all people who liked talking, because those who did not like talking refused out of hand.” Patients who refused to participate often argued that they felt uncomfortable speaking in front of a group: “And the men, when they declined they often said: ‘No, group discussion is not for me! I don’t like talking in front of a group.’”

The researchers eventually succeeded in recruiting sufficient participants. However, they were not able to realize the sampling plans according to a certain proportion of male and female patients or types of physicians. “Well, we finally managed to fill up our groups, but only as many [participants] as necessary.” Comparing the different target groups, recruiting patients was described as easier than recruiting physicians: “And that was much easier insofar as you just had to go to the clinic and each day there were five or six patients whom you could address.” However, only 10% of the patients who were addressed agreed to participate. In the health care professional group, the recruitment rates ranged between 0% and 30%, depending on the subgroup. This can be demonstrated by the following interviewee utterance:

And in the private practitioner sector it was rather . . . . Well, we tried to recruit specialists in private practice, in other words internists, gastroenterologists, and oncologists. The success [rate proved to be] extremely poor. . . . Well, on the whole, the willingness to take part, the interest, is not there. Or, well they don’t give the reasons, but they say they don’t want to take part. So that was difficult and, yes, it didn’t go too well.

Communication in the Focus Groups

With regard to the communication in the focus groups, the moderators identified factors that influenced communication in a positive or negative way. In addition, we discussed a number of factors with them that are often described in the social science literature as problematic when conducting focus groups. However, the interviewees considered that some of these factors had not influenced communication in the focus groups conducted within the framework of the INFOPAT program. In our system of categories, we also coded whether the factors in question were related to (a) the setting or (b) the moderation of the focus groups (cf. Figure 2 ).

Influences on and characteristics of the focus group discussion.

Factors relating to the setting

As Figure 2 shows, communication was reported to be positively influenced by small group size, location, provision of food and beverages, and conducting the focus group without a break. In contrast to general recommendations on focus groups in the context of sociological research, the moderators in the INFOPAT program considered a smaller group size of between four and six participants to be ideal. With regard to location, the interviewees reported that, depending on the target group, different places were perceived as positive. Patients preferred locations inside the clinic because they were easy to reach and caused no additional effort. Furthermore, because these locations were familiar to them, they facilitated an atmosphere of security and ease, which was seen as an important prerequisite for an open and honest discussion. This is clear from the following quotation:

Well, the patient focus groups were all located at the clinic. We chose this location on purpose to make it easier for them, because they come to the clinic anyway for their therapy. And they know the place and they feel comfortable and in good hands.

By contrast, the clinician groups benefited from being located outside the clinic. In contrast to other common addressees of focus groups, these professionals were not only accustomed to participating in groups outside their familiar surroundings but also this location helped them to distance themselves from their professional duties and to engage more deeply in the discussion, as shown by the following quotation:

Yes, one was located at the O-Center. We chose this location on purpose so that the clinicians had to leave the hospital. It’s not too far, only a few yards away. But we wanted them to leave the clinic, and not to run back to the ward when they were called. And, well, I liked this location.

Food and beverages were welcome in all the groups and also helped to create a positive and trusting atmosphere. And finally, the interviewees found that it was better to omit the break, thereby avoiding the interruption of the ongoing discussion. This is reasonable considering the comparatively short duration of the focus group session (between 1.5 and 2 hours). Statements relating to a break might have been different in the case of longer focus group durations.

The interviewees reported that the size and temperature of the room and time pressure on the participants or the moderator had a negative impact on communication. Some of the focus groups in the project took place in midsummer and had to be held in rooms without blinds or air conditioning. The moderators of these groups had to work hard to maintain the participants’ (and their own) attention and concentration. Time pressure on the participants (e.g., the clinicians) led to an unwillingness to engage in active discussion and created a question-and-answer situation, as shown by the following statement:

And in one group of physicians . . . we never reached the point where they joined in fully. During the whole discussion they never completely arrived. And they had already cut the time short in advance. They were under so much time pressure that they were not able to discuss in an open manner.

Moderators reported that they, too, had experienced time pressure—namely, in situations where they did not have enough time to prepare the room and the recording devices. This had caused them to be nervous and stressed at the beginning of the discussion, which had negatively affected the mood of the participants, thereby rendering an honest and open discussion particularly difficult.

Factors relating to the moderation

Many of the positive factors reported by the interviewees have already been described for focus groups in general—for example, using open questions, directly addressing quiet participants, and handling the discussion guide in a flexible way. Furthermore, by showing interest in every statement, and by generating a feeling of security in every participant, moderators fostered a fruitful discussion:

I believe that another important point is that you are calm yourself. That you give the people the feeling “you can feel safe with me, you don’t have to worry that I will make fun of you . . . or that I won’t take you seriously.”

Interviewees also considered that building a bridge between the technical innovation under discussion (a web-based electronic personal health record) and everyday life (e.g., online banking) was an important factor in getting all participants to contribute to the discussion. As one interviewee noted,

We tried to anchor it in their everyday lives. And . . . the example that always worked was when we said: “Think of it as if it were a kind of online banking.” Everyone understands what online banking is. It’s about important data on the internet; they’re safe there somehow. I have my password. And people understood that. Well, it’s important to anchor it in their reality . . . because otherwise the topic is simply far too abstract.

In this context, the fact that the groups were moderated by the researchers themselves proved very helpful because they were able to answer all questions relating to the research topic. As the following quote shows, this was an important prerequisite for opinion formation on the part of participants:

Well, I think that a really important quality criterion . . . is that you have completely penetrated [the topic]. If you only know the process from the outside . . . and you then conduct the focus group about it. . . . Somewhere, at some stage, [one discussion] narrowly missed the point. . . . You simply have to be totally immersed in the topic, well, I believe that [someone who is totally immersed in the topic] is the ideal person for the job. And in our case the thinking was, okay, so I’m a doctor, but on balance it’s more important that both [moderators] are absolutely well informed because it’s a complex topic.

The more specific the research question was, the more useful the moderating strategy of inviting one participant after the other to express their opinion appeared to be. By using this strategy, the moderators ensured that every participant contributed to the discussion.

A point that was strongly emphasized by the interviewees was the duration of the round of introductions at the beginning of the focus group session. In the patient groups, introductions took much more time than the researchers had expected. Patients had a high need to express themselves and to tell the others about their illness and their experiences with the health system. Although this left less time to work through the topics in the discussion guide, the researchers came to realize that there were several good reasons not to limit these contributions: First, the introductions round proved important for helping the participants to “arrive” at the focus group, for creating a basis of trust, and for building up a sense of community among the participants. Second, the interviewees reported that, because many topics in the discussion guide (e.g., participants’ experiences with coordinating visits to different medical specialists) had already been brought up in the round of introductions, they did not have to be discussed further at a later stage:

And that is the crux of this general exchange of experiences at the beginning. Sure, it costs you a lot of time, but I almost think that if you don’t give them that time, you won’t get what you want from them, in the sense that you say: “I want to hear your frank opinion or attitude.” You don’t want them to simply answer you because they think that’s what you want to hear. You have to create an atmosphere in which they really forget where they are. I’m relatively convinced that you wouldn’t achieve that without such [a round of introductions].

The moderators’ experience in the physician groups was different. These groups benefited from having a rather short round of introductions. Giving participants too much time to introduce themselves meant that they presented their expertise rather than reporting their experiences. In contrast to the patient groups, this did not substantially contribute to the discussion of the research topics.

Depending on the context, status differences between the moderators and the participants, or among the participants, were appraised differently by interviewees. In one group comprising physicians and medical assistants, the moderators observed that status differences had a negative influence on communication. Very young female medical assistants, in particular, did not feel free to express their opinions in the presence of their superiors. By contrast, presumed differences in status between family doctors, hospital doctors, and medical specialists in private practice did not have any negative impact on communication. Nor did different forms of address (some participants in these groups were addressed by their first name and some by their last name, depending on the relationship between the moderator and the participants). Status differences between moderators (if medical doctors) and participants (patients) had an impact on communication when patients regarded doctors as an important source of information (e.g., about the meaning of their blood values) or as representatives of the health care system to whom complaints about the system should be addressed. The latter case was the subject of the following interview statement by a moderator who is a physician by profession:

And a lot [was said about] the kind of experiences they had had here at the NCT. And of course, when the patients have been treated here for many years—or even for not so many [years], but they have had many experiences—they sometimes reported at length. And I had the feeling that this had a bit of a feedback function, quite generally, for the NCT. Also the somehow frustrating experiences they had had, or a lot of things that had not gone that well in conversational exchanges [with the staff]. There was a relatively large amount of feedback that didn’t have a lot to do with the topic because I was, of course, involved as a senior physician and I am not an external researcher, but rather someone who is also seen as being jointly responsible, or at least as someone who can channel criticism.

Finally, because most of the moderators were not medical professionals, they did not experience the translation of medical or technical terms into everyday language as problematic. Rather, they automatically used terms that were also familiar to the participants.

Characteristics of the discussion

The factors described above resulted in focus group discussions that might be interpreted as characteristic of health research. The patient focus groups were characterized by a strong need to talk and a high need for information. In the health care professional focus groups, researchers experienced a greater variety of communication styles. Because of a lack of time, or because they falsely expected a question-and-answer situation, some groups demonstrated a low degree of willingness to engage in discussion:

Although, I believe that was partly due . . . well there was one [woman] who was very demanding; she wanted to know straight away: “Yes, what’s the issue here? What do I have to say to you?” Well, the three who came from the one practice, I think they really had the feeling that we would ask them questions and they would bravely answer them and then they could go home again. So, for them this principle that they were supposed to engage in a discussion, for them that was somehow a bit, I don’t know . . . disconcerting. . . . They really thought: “Okay, well we want to know now what this is all about. And they’ll ask us the questions and then we’ll say yes, no, don’t know, maybe. And then we’ll go home again.” Well, at least that was my impression.

Other groups, especially those consisting of different types of health care professionals (e.g., physicians with different areas of specialization, or physicians and pharmacists), were characterized by lively discussion and a great variety of opinions.

Appraisal of the Focus Group Method

We classified moderators’ statements relating to the appraisal of the focus group method into four main categories: “advantages of the method”, “disadvantages of the method”, “recommendations for other researchers in related research areas”, and “statements on how they used the results” (cf. Figure 3 ).

Appraisal of the focus group method.

The researchers reported that the focus group method yielded a rich blend of perspectives and opinions, brought forth, in particular, by the interaction between the participants:

But for this question and the topic, and for our lack of knowledge, that was . . . a lot of new information . . . and very many good ideas and critical remarks that you naturally read in the literature from time to time. But, let’s say, because of the complexity of the participants’ reactions and the weight they attached to things, it’s different than reading in a literature review that [this or that] could be taken into account.

The results of the focus groups further enriched the researchers’ work by relating it to everyday life: “Well, what was nice was that the topic was related to the participants’ lives. That people said: ‘Now the topic is important for me.’” Furthermore, the method yielded information about which aspects were most important and how the variety of opinions should be prioritized. This was achieved, in particular, by using participant-generated cards:

And with regard to prioritization, we incorporated it using participant-generated cards. We said: “Look: If you could develop this record now, what would be the three most important things that must absolutely be taken into consideration, from your point of view, no matter what they relate to.” And they wrote them down on the cards. And after that they were asked to carry out their own prioritization—that is, what was most important to them personally. One person wrote “data protection” first, while another [wrote] “sharing with my wife.” . . . That was good. . . . That helped a lot because it was simply clear once again what things were important to them.

In cases where concrete questions had to be answered or decisions had to be made, the interviewees also welcomed the opportunity to use structuring methods such as presentations, flip-charts, and participant-generated cards to obtain the relevant information:

. . . Well, the aim was that at the end we [would] have a set of requirements for the engineering [people]. And the engineering [people] don’t so much want to know about experiences and desires and barriers, but rather they want to know should the button be green or red and can you click on it. And that’s why I thought at the beginning it will be difficult with a focus group and an open discussion. Now, if you say that one can also interpret a focus group the way we did, partly with very specific questions and these participant-generated cards, then I think it is indeed possible to answer such questions as well.

Disadvantages

The main disadvantages of the focus group method were seen in the considerable organizational effort and expenditure of time involved. A question raised by some of the interviewees was whether comparable results could have been achieved using less time-consuming and organizationally demanding methods.

It’s true to say that you lose time. Well, you could implement [the innovation] straight away and see whether it’s better. Maybe, in this case you’re wrong and you just think it’s better or in any case not worse than before. You basically lose a year on this whole focus groups thing.

Moreover, in some cases, the discussion went in an unwanted direction and the moderators never fully succeeded in bringing the group back to the intended topics.

Furthermore, like many other medical research projects, INFOPAT included quite specific research questions. In this connection, the moderators emphasized that open focus group discussions would not have succeeded in answering those questions. Only by using methods such as participant-generated cards and prioritization was it possible to answer at least some of them. Nonetheless, some interviewees did not consider the focus group method to be really suitable for this type of research questions:

Of course we also have our engineers as counterparts who . . . need very specific requirements at some point. The question is whether such a focus group . . . . [It] can’t answer that in detail in this first stage. It’s simply not practicable.

Recommendations

As described under the “Communication in the Focus Groups” section above, the round of introductions in the patient groups lasted much longer than planned, thereby shortening the time available for other topics in the discussion guide. As a result, the moderators decided to choose a different thematic focus in each group so that every topic was discussed more deeply in at least one group.

What we usually did was to consider what hadn’t been addressed that much in the previous focus group. That [topic] was given more room in the next focus group because the guide, well it was quite a lot. You could have easily gone on discussing for another hour or two.

Using the results

On the whole, the researchers were satisfied with the number of groups that were conducted and the results that they yielded. They did not agree that more groups would have led to better, or different, results—with one possible exception, namely, in the case of specific target groups (e.g., migrants). Only one group had been composed of patients with a migrant background, and, as one interviewee stated, “I just thought, the patients with a migrant background . . . now that was [only] one group, it by no means covers the whole range.”

In cases where the results of the focus groups were perceived as not being concrete enough to proceed to the next research step (e.g., formulating a specification sheet for the construction of the electronic personal health record), the researchers planned to bring experts together in a roundtable format to make decisions on the basis of the priorities, agreements, and disagreements that had emerged from the focus groups. Following the construction of a prototype, they intended to conduct further focus groups to validate or adapt the usability of the electronic personal health record system.

Our analysis of interviews with focus group moderators yielded considerable insights into methodological aspects of conducting focus groups in health research. Our first research question related to characteristics of the target groups that should be considered during the recruitment process. We identified face-to-face contact as an important factor promoting focus group participation. The interviewees considered this type of contact to be better suited to answering target persons’ questions and explaining the method and aims of the focus groups. Moreover, they felt that addressees might find it more difficult to decline a face-to-face invitation than a written one. With regard to health care professionals, an invitation issued by a hierarchically higher person was most effective, even though ethical aspects should be considered in this case, and voluntary participation should nevertheless be ensured. Otherwise, the order to participate might prevent an atmosphere of open communication and might lead to a lower quantity or to more negative statements.

Furthermore, whereas physicians are usually accustomed to discussing topics with others, an important characteristic that influenced willingness to participate on the part of members of other target groups (other health care professionals, patients) was a liking for, or a dislike of, talking. Researchers might take account of this fact by explaining the method in more detail, by developing arguments to overcome fears, or, as suggested, for example, by Colucci (2007) , by convincing the addressees with other activities implemented in the focus groups. Other relevant personal characteristics—be they related to the research topic (e.g., technical interest in the case of an electronic innovation) or to the specific target group (e.g., physical fitness on the part of patients or lack of time on the part of health care professionals)—should be anticipated when planning recruitment. These characteristics might be taken into account by preparing arguments, providing incentives, giving thought to favorable dates and times, and choosing easily accessible locations. An interesting finding was that, depending on the target group, different locations were considered to have a positive influence on the discussion. Whereas locations inside the clinic were preferred in the case of the patient focus groups because of familiarity and easy accessibility, hospital doctors were more engaged in the discussion when the focus group site was located at least some yards away from their workplace.

Finally, the experience of our researchers that up to 50% of the patients had to cancel at short notice because of health problems does not appear to be uncommon in this research context. That overrecruitment is an effective strategy—particularly in health care research—has been reported by other authors (e.g., Coenen et al., 2012 ).

With our second research question, we focused on aspects of communication in the focus groups. The interviews revealed several factors specific to research topics and addressees of health care studies that influenced the discussions. Consequently, in addition to considering general recommendations regarding the organization and moderation of focus groups (e.g., choosing adequate rooms with a pleasant atmosphere, serving food and beverages, using open questions, showing interest in all contributions, and directly addressing quiet participants), these health care specific aspects should be taken into account. Relevant factors that should be addressed when moderating focus groups in this context are (a) the strong need to talk and the high need for information in the patient groups, (b) status differences between the participants or between the moderators and the participants, (c) the size of the focus group, and (d) the specificity of the topic of discussion. The interview data revealed that these factors influenced the discussions and thus the results achieved with the groups. In addition, the following four possibilities of addressing these factors were identified:

First, the moderators had to devote more time to the round of introductions in the patient groups, which served as a warm-up, created an atmosphere of fellowship and openness, and accommodated this target group’s strong need to talk. Second, with respect to status differences between the moderator and the participants, no definite recommendations can be derived from the interviews. The interviewees found that it was less favorable when the moderator was perceived not only in that role but also in other roles (e.g., physician), because this might hamper a goal-oriented discussion. However, they considered deep insight into the research topic on the part of the moderators to be beneficial, at least for certain research topics. Thus, one should carefully weigh up whether it is more advantageous or more disadvantageous when the group moderator is a physician. Interviewees considered status differences between participants to be disadvantageous only in one case, where—because of organizational constraints—medical assistants and their superiors joined the same focus group, which gave rise to some reticence on the part of the young assistants. Similar problems have been reported by other authors, for example, Côté-Arsenault and Morrison-Beedy (2005 ; see also Hollander, 2004 ). However, interviewees did not experience as problematic status differences between physicians with different areas of specialization.

Third, with respect to group size, interviewees found comparatively small focus groups appropriate to give all participants enough time to tell their stories. In contrast to social science research, where groups of between eight and 20 participants are recommended, our interviewees considered groups of between four and six persons to be optimal. This is in line with Côté-Arsenault and Morrison-Beedy (2005) , who recommended small groups for health research, especially when sensitive topics are discussed. Our interview data revealed that this recommendation might also be useful for other health research topics.

Fourth, with regard to the topic of the discussion, interviewees found it helpful to structure different phases of the discussion in different ways, depending on the specificity of the research questions. In contrast to social science research, certain types of research questions in health research require comparatively specific answers. Some of the focus groups in our study were aimed at collecting participants’ expectations regarding an electronic personal health record or—even more specifically—at developing a product specifications document. Conducting focus groups during the development of a technical innovation is a method that is being increasingly used in health care research. Hence, the experiences of the interviewees with regard to these aspects of their research might be relevant for many other research programs. For this type of research questions, it proved useful to include more structured parts in the discussion, for example, having certain questions answered by each participant in turn, or using methods such as participant-generated cards and prioritization. This made it easier to obtain the opinion of each participant and to cover as many concerns and expectations as possible. This finding is in line with recommendations by Colucci (2007) , who proposed the use of activity-oriented questions for health research topics as an enrichment of data collection and a means of making it easier to talk about sensitive and complex topics.

All the moderators found that their discussion guides contained too many questions and too many topics. This might have been due, at least partly, to a desire to determine all relevant aspects in advance—a tendency that might be typical of health research. However, Morgan (1995) also addressed this phenomenon in relation to social research in general: “A common error in focus group question guidelines is too much emphasis on what is of interest to the researcher and not enough emphasis on what is of interest to the participants” (p. 520).

With our third research question, we addressed the appraisal of the focus group method in the interviewees’ research context. Our results show that one should think carefully before using focus groups in the field of health research. The impression that they are quick and easy to conduct might be a misconception, especially in this research context. In fact, the appraisal of the method by the moderators revealed both advantages and disadvantages. The main advantages were the rich blend of perspectives and opinions obtained and the opportunity to have them prioritized by the target groups. For their research topics, the interviewees saw a further important advantage in the fact that they were able to relate their scientific research to everyday life, a point that might be of general importance for a number of research questions in health research, especially those that refer to new medical diagnostics or technical innovations.

The interviewees considered that the main disadvantages of focus groups were the substantial organizational effort and expenditure of time they required. They raised the question whether comparable results could have been achieved using less costly methods. Fortunately, we conducted our interviews with researchers from a research program aimed at answering research questions of different degrees of specificity. As a result, the moderators were able to compare the usefulness of focus groups for different types of research questions. Their statements revealed that they were satisfied with the results relating to more open research questions such as experiences with cross-sectoral health care. For more specific research questions, the interviewees valued the possibility of organizing the discussions in a more structured way and using methods that activated all participants (e.g., participant-generated cards, prioritizations). Nonetheless, they considered meetings of experts to be a necessary intermediate step, for example, on the way to a product specifications document. We recommend that, depending on the specificity of the results that are projected, consideration should be given to including such intermediate steps in the planning stage.

Limitations of the Study

Our analysis of the interviews with the focus group moderators revealed a number of methodological problems that typically occur when focus groups are used in a health research context and yielded recommendations on using such groups in this context. However, some limitations of the present study should also be discussed: First, we conducted our research with focus group moderators, all of whom worked in the same research program. Even though the INFOPAT program consists of several subprojects, they all deal to a greater or lesser extent with the advantages and disadvantages of an electronic support system (electronic personal health record). Furthermore, the moderators were mainly health scientists and had little or no experience with conducting focus groups. This might also have been specific for the research program in which our study was conducted. In other health care programs, focus groups might be moderated mainly by physicists or lay persons (e.g., in participatory health research). Consequently, had we also conducted interviews with focus group moderators from other research areas or included moderators with other professions or more focus group experience, this might have led to different results. However, our research project is rather typical for applied qualitative research in medical science when developing new technologies. Here, focus groups are used by the researchers to find out the potential requirements for the new technology. The researchers are often experts in a specific scientific topic and have no or only limited experience in conducting qualitative research in terms of focus groups. Therefore, our findings are of a particular importance for the researchers with little experiences in conducting focus groups, which can apply to every research, conducted first time. In addition, the little experience of our focus group moderators was a special advantage and strength of the study. More experienced moderators would have prevented some of the problems our moderators—as other unexperienced moderators—faced. As a result, the moderators would not have named these potential problems in the interviews and given no advice for preventing them.

Second, the study was conducted in Germany and thus represents problems and challenges of the German health care system. In other countries, physicians might have different work-shifts or there might be different possibilities in the health care system to reach the target groups. Therefore, more research on the methodology of focus groups in the context of the development of new technologies in health care in other countries and cultures with a consideration of additional relevant groups is needed.

Third, in our interviews, we focused mainly on the organization and conducting of focus groups. For two reasons, we did not address the aspect of data analysis: First, we conducted the interviews shortly after the focus groups had been completed, at a time when data analysis was still in progress. Second, analysis of qualitative data can be carried out in many different ways, depending on research questions and preferences of researchers, and some of the recommended methods are very complex. Had we discussed them in detail, it would have been too time-consuming in the interviews.

Concluding Remarks

Our results revealed a number of methodological challenges that might be typical of conducting focus groups in health research. We hope that our findings will be of use to researchers in similar research fields. Furthermore, we encourage other researchers who are interested in health research topics to gather more information about methodological aspects specific to this research field. Our results were achieved in the context of the development of a technical innovation. It might be interesting to endeavor to replicate them in other health care research projects dealing with technical innovations. Moreover, we would encourage researchers of other topics in health research to interview focus group moderators about their experiences in their specific research context. We hope that our results will serve as a useful basis for comparing results in different areas of health research.

Acknowledgments

We thank the focus group moderators in the INFOPAT program for their great willingness to share their experiences and for their openness during the interviews.

Author Biographies

Anja P. Tausch , PhD, is senior researcher at GESIS–Leibniz Institute for the Social Sciences, Mannheim, Germany.

Natalja Menold , PhD, is senior researcher and head of the Survey Instruments Unit at GESIS–Leibniz Institute for the Social Sciences, Mannheim, Germany.

1. http://www.soziologie.de/en/gsa/ethik-kommission/code-of-ethics.html , retrieved on 05/10/2015.

2. The language of the research project, focus groups, and interviews was German. The scheme was developed in German on the basis of the German text material from the transcribed interviews. The scheme and the citations were translated for the purpose of international publication by an experienced, qualified, and fully bilingual translator, whose mother tongue is English and who also has an MA in sociology from a German university. A German version of the full categorial system can be found in Tausch and Menold (2015) .

3. All citations included in this publication were translated from German.

Declaration of Conflicting Interests: The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding: The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by the German Federal Ministry of Education and Research (BMBF; FKZ 01KQ1003D).

• Baudendistel I., Winkler E., Kamradt M., Brophy S., Längst G., Eckrich F., . . .Ose D. (2015). The patients’ active role in managing a personal electronic health record: A qualitative analysis . Supportive Care in Cancer , 23 , 2613-2621. doi: 10.1007/s00520-015-2620-1 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Carey M. A. (1994). The group effect in focus groups: Planning, implementing, and interpreting focus group research . In Morse J. M. (Ed.), Critical issues in qualitative research methods (pp. 225–241). Thousand Oaks, CA: Sage. [ Google Scholar ]
• Carr A., Hewlett S., Hughes R., Mitchell H., Ryan S., Carr M., Kirwan J. (2003). Rheumatology outcomes: The patient’s perspective . The Journal of Rheumatology , 30 , 880–883. [ PubMed ] [ Google Scholar ]
• Coenen M., Stamm T., Stucki G., Cieza A. (2012). Individual interviews and focus groups in patients with rheumatoid arthritis: A comparison of two qualitative methods . Quality of Life Research , 21 , 359–370. [ PubMed ] [ Google Scholar ]
• Colucci E. (2007). “Focus groups can be fun”: The use of activity-oriented questions in focus group discussions . Qualitative Health Research , 17 , 1422–1433. [ PubMed ] [ Google Scholar ]
• Côté-Arsenault D., Morrison-Beedy D. (2005). Maintaining your focus in focus groups: Avoiding common mistakes . Research in Nursing & Health , 28 , 172–179. [ PubMed ] [ Google Scholar ]
• Creswell J. W., Plano Clark V. L. (2007). Designing and conducting mixed methods research . Thousand Oaks, CA: Sage. [ Google Scholar ]
• Daley C. M., James A. S., Ulrey E., Joseph S., Talawyma A., Choi W. S., . . .Coe M. K. (2010). Using focus groups in community-based participatory research: Challenges and resolutions . Qualitative Health Research , 20 , 697–706. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Elo S., Kyngäs H. (2008). The qualitative content analysis process . Journal of Advanced Nursing , 62 , 107–115. [ PubMed ] [ Google Scholar ]
• Feldthusen C., Björk M., Forsblad-d’Elia H., Mannerkorpi K. (2013). Perception, consequences, communication, and strategies for handling fatigue in persons with rheumatoid arthritis of working age—A focus group study . Clinical Rheumatology , 32 , 557–566. [ PubMed ] [ Google Scholar ]
• Frazier L. M., Miller V. A., Horbelt D. V., Delmore J. E., Miller B. E., Paschal A. M. (2010). Comparison of focus groups on cancer and employment conducted face to face or by telephone . Qualitative Health Research , 20 , 617–627. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Gerber D. E., Hamann H. A., Rasco D. W., Woodruff S., Lee S. J. C. (2012). Patient comprehension and attitudes toward maintenance chemotherapy for lung cancer . Patient Education and Counseling , 89 , 102–108. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Hollander J. A. (2004). The social contexts of focus groups . Journal of Contemporary Ethnography , 33 , 602–637. [ Google Scholar ]
• Kamradt M., Baudendistel I., Längst G., Kiel M., Eckrich F., Winkler E., Ose D. (2015). Collaboration and communication in colorectal cancer care: A qualitative study of the challenges experienced by patients and health care professionals . Family Practice , 32 , 686-696. doi: 10.1093/fampra/cmv069 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Kingry M. J., Tiedje L. B., Friedman L. L. (1990). Focus groups: A research technique for nursing . Nursing Research , 39 , 124–125. [ PubMed ] [ Google Scholar ]
• Kitzinger J. (1995). Qualitative research: Introducing focus groups . British Medical Journal , 311 , 299–302. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Kitzinger J. (2006). Chapter 3: Focus groups . In Pope C., Mays N. (Eds.), Qualitative research in health care (3rd ed., pp. 21–31). Malden, MA: Blackwell. [ Google Scholar ]
• Kroll T., Neri M. T., Miller K. (2005). Using mixed methods in disability and rehabilitation research . Rehabilitation Nursing , 30 , 106–113. [ PubMed ] [ Google Scholar ]
• Krueger R. A. (1988). Focus groups: A practical guide for applied research . Newbury Park, CA: Sage. [ Google Scholar ]
• Lehoux P., Poland B. D., Daudelin G. (2006). Focus group research and “the patient’s view.” Social Science & Medicine , 63 , 2091–2104. [ PubMed ] [ Google Scholar ]
• Mayring P. (2015). Qualitative Inhaltsanalyse [Qualitative content analysis] (12th ed.). Weinheim, Germany: Beltz. [ Google Scholar ]
• Morgan D. L. (Ed.). (1993). Successful focus groups: Advancing the state of the art . Newbury Park, CA: Sage. [ Google Scholar ]
• Morgan D. L. (1995). Why things (sometimes) go wrong in focus groups . Qualitative Health Research , 5 , 516–523. [ Google Scholar ]
• Morgan D. L. (1996). Focus groups . Annual review of sociology , 22 , 129–152. [ Google Scholar ]
• Morgan D. L. (2009). Focus groups as qualitative research (2nd ed.). Thousand Oaks, CA: Sage. [ Google Scholar ]
• Morgan D. L., Krueger R. A. (Eds.). (1998). Focus group kit . Thousand Oaks, CA: Sage. [ Google Scholar ]
• Nafees B., Lloyd A., Kennedy-Martin T., Hynd S. (2006). How diabetes and insulin therapy affects the lives of people with type 1 diabetes . European Diabetes Nursing , 3 , 92–97. [ Google Scholar ]
• Nicholas D. B., Lach L., King G., Scott M., Boydell K., Sawatzky B. J., . . .Young N. L. (2010). Contrasting internet and face-to-face focus groups for children with chronic health conditions: Outcomes and participant experiences . International Journal of Qualitative Methods , 9 , 105–121. [ Google Scholar ]
• Rasmusson K., Lappe D., Roberts C., Croasdell S., Meegan S., Budge D. (2014). Heart failure patient perspectives: Learning from focus groups to improve care . Heart & Lung , 43 , 382–383. [ Google Scholar ]
• Rhodes S. D., Hergenrather K. C., Wilkin A., Alegría-Ortega J., Montaño J. (2006). Preventing HIV infection among young immigrant Latino men: Results from focus groups using community-based participatory research . Journal of the National Medical Association , 98 , 564–573. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Stewart D. W., Shamdasani P. N., Rook D. W. (2007). Focus groups: Theory and practice (2nd ed.). Thousand Oaks, CA: Sage. [ Google Scholar ]
• Tausch A., Menold N. (2015). Methodische Aspekte der Durchführung von Fokusgruppen in der Gesundheitsforschung - Welche Anforderungen ergeben sich aufgrund der besonderen Zielgruppen und Fragestellungen? [Methodological aspects of conducting focus groups in health research - which requirements result from the particular target groups and research questions?]. GESIS Papers , 2015 ( 12 ). Retrieved from http://www.gesis.org/fileadmin/upload/forschung/publikationen/gesis_reihen/gesis_papers/GESIS-Papers_2015-12.pdf [ Google Scholar ]
• Vincent D., Clark L., Marquez Zimmer L., Sanchez J. (2006). Using focus groups to develop a culturally competent diabetes self-management program for Mexican Americans . The Diabetes Educator , 32 , 89–97. [ PubMed ] [ Google Scholar ]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Literature review: considerations in undertaking focus group research with culturally and linguistically diverse groups

Affiliation.

• 1 Centre for Applied Nursing Research, University of Western Sydney and Sydney South West Area Health Service, Liverpool, NSW, Australia.
• PMID: 17518876
• DOI: 10.1111/j.1365-2702.2006.01760.x

Aims: This integrated literature review seeks to identify the key considerations in conducting focus groups and discusses the specific considerations for focus group research with culturally and linguistically diverse groups.

Background: The focus group method is a technique of group interview that generates data through the opinions expressed by participants. Focus groups have become an increasingly popular method of data collection in health care research. Although focus groups have been used extensively with Western populations, they are a particularly useful tool for engaging culturally and linguistically diverse populations. The success of focus groups in this context is dependent upon the cultural competence of the research team and the research questions.

Methods: The electronic databases Medline, CINAHL, Embase, Psychlit and the Internet using the Google Scholar search engine were explored using the search terms 'focus group', 'cultural sensitivity', 'transcultural nursing', 'transcultural care', 'cultural diversity' and 'ethnic groups'. Hand searching of reference lists and relevant journals was also undertaken. English language articles were selected for the review if they discussed the following issues: (i) methodological implications of the focus group method; (ii) strengths and limitations of the focus group method; (iii) recommendations for researchers and (iv) use of the focus group in culturally and linguistically diverse groups. Conclusions were drawn from each of the articles and consensus regarding a summary of recommendations was derived from a minimum of two authors.

Results: Findings from this review revealed several key issues involving focus group implementation including recruitment, sample size, data collection, data analysis and use within multicultural populations. Strengths and limitations of the focus group method were also identified.

Conclusions: Focus groups are a useful tool to expand existing knowledge about service provision and identify consumer needs that will assist in the development of future intervention programmes, particularly within multicultural populations. Careful planning related to methodological and pragmatic issues are critical in deriving effective data and protecting participants.

Relevance to clinical practice: Focus groups can facilitate increased understanding of perspectives of culturally and linguistically diverse groups and thereby shape clinical practice to better meet the needs of these groups.

PubMed Disclaimer

Similar articles

• Using the Internet to conduct research with culturally diverse populations: challenges and opportunities. Suarez-Balcazar Y, Balcazar FE, Taylor-Ritzler T. Suarez-Balcazar Y, et al. Cultur Divers Ethnic Minor Psychol. 2009 Jan;15(1):96-104. doi: 10.1037/a0013179. Cultur Divers Ethnic Minor Psychol. 2009. PMID: 19209984
• Conducting culturally competent rural nursing research. Bushy A. Bushy A. Annu Rev Nurs Res. 2008;26:221-36. Annu Rev Nurs Res. 2008. PMID: 18709752 Review.
• Exploring community nurses' perceptions of life review in palliative care. Trueman I, Parker J. Trueman I, et al. J Clin Nurs. 2006 Feb;15(2):197-207. doi: 10.1111/j.1365-2702.2006.01244.x. J Clin Nurs. 2006. PMID: 16422737
• Focus group interviews: a methodology for socially sensitive research. Goss GL. Goss GL. Clin Excell Nurse Pract. 1998 Jan;2(1):30-4. Clin Excell Nurse Pract. 1998. PMID: 12675074 Review.
• Focus group interviews in health-care research. Clarke A. Clarke A. Prof Nurse. 1999 Mar;14(6):395-7. Prof Nurse. 1999. PMID: 10205536 Review.
• A lifestyle screening tool for young children in the community: needs and wishes of parents and youth healthcare professionals. Krijger A, Schiphof-Godart L, Lanting C, Elstgeest L, Raat H, Joosten K. Krijger A, et al. BMC Health Serv Res. 2024 May 3;24(1):584. doi: 10.1186/s12913-024-10997-y. BMC Health Serv Res. 2024. PMID: 38702743 Free PMC article.
• Assessing the mental health needs of Yazidi adolescents and young adults in an Iraqi Kurdi IDP Camp: a focus group study. Benoni R, Giacomelli C, Vegro G, Hamo F, Avesani R, Albi P, Gatta M, Moretti F. Benoni R, et al. Int J Equity Health. 2024 May 1;23(1):88. doi: 10.1186/s12939-024-02182-8. Int J Equity Health. 2024. PMID: 38693504 Free PMC article.
• Sikh and Muslim perspectives on kidney transplantation: phase 1 of the DiGiT project - a qualitative descriptive study. Brand S, Daga S, Mistry K, Morsy M, Bagul A, Hamer R, Malik S. Brand S, et al. BMJ Open. 2023 Dec 1;13(12):e059668. doi: 10.1136/bmjopen-2021-059668. BMJ Open. 2023. PMID: 38040423 Free PMC article. Clinical Trial.
• Beyond Information Provision: Analysis of the Roles of Structure and Agency in COVID-19 Vaccine Confidence in Ethnic Minority Communities. Poduval S, Kamal A, Martin S, Islam A, Kaviraj C, Gill P. Poduval S, et al. Int J Environ Res Public Health. 2023 Nov 1;20(21):7008. doi: 10.3390/ijerph20217008. Int J Environ Res Public Health. 2023. PMID: 37947565 Free PMC article.
• Understanding the mix of services for mental health care in urban DR Congo: a qualitative descriptive study. Mukala Mayoyo E, Criel B, Sow A, Coppieters Y, Chenge F. Mukala Mayoyo E, et al. BMC Health Serv Res. 2023 Nov 4;23(1):1206. doi: 10.1186/s12913-023-10219-x. BMC Health Serv Res. 2023. PMID: 37925407 Free PMC article.

Publication types

• Search in MeSH

Related information

• Cited in Books

LinkOut - more resources

Full text sources.

• Ovid Technologies, Inc.

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Published: 05 October 2018

Interviews and focus groups in qualitative research: an update for the digital age

• P. Gill 1 &
• J. Baillie 2  

British Dental Journal volume  225 ,  pages 668–672 ( 2018 ) Cite this article

30k Accesses

20 Altmetric

Metrics details

Highlights that qualitative research is used increasingly in dentistry. Interviews and focus groups remain the most common qualitative methods of data collection.

Suggests the advent of digital technologies has transformed how qualitative research can now be undertaken.

Suggests interviews and focus groups can offer significant, meaningful insight into participants' experiences, beliefs and perspectives, which can help to inform developments in dental practice.

Qualitative research is used increasingly in dentistry, due to its potential to provide meaningful, in-depth insights into participants' experiences, perspectives, beliefs and behaviours. These insights can subsequently help to inform developments in dental practice and further related research. The most common methods of data collection used in qualitative research are interviews and focus groups. While these are primarily conducted face-to-face, the ongoing evolution of digital technologies, such as video chat and online forums, has further transformed these methods of data collection. This paper therefore discusses interviews and focus groups in detail, outlines how they can be used in practice, how digital technologies can further inform the data collection process, and what these methods can offer dentistry.

You have full access to this article via your institution.

Similar content being viewed by others

Participatory action research

Determinants of behaviour and their efficacy as targets of behavioural change interventions

Interviews in the social sciences

Introduction.

Traditionally, research in dentistry has primarily been quantitative in nature. 1 However, in recent years, there has been a growing interest in qualitative research within the profession, due to its potential to further inform developments in practice, policy, education and training. Consequently, in 2008, the British Dental Journal (BDJ) published a four paper qualitative research series, 2 , 3 , 4 , 5 to help increase awareness and understanding of this particular methodological approach.

Since the papers were originally published, two scoping reviews have demonstrated the ongoing proliferation in the use of qualitative research within the field of oral healthcare. 1 , 6 To date, the original four paper series continue to be well cited and two of the main papers remain widely accessed among the BDJ readership. 2 , 3 The potential value of well-conducted qualitative research to evidence-based practice is now also widely recognised by service providers, policy makers, funding bodies and those who commission, support and use healthcare research.

Besides increasing standalone use, qualitative methods are now also routinely incorporated into larger mixed method study designs, such as clinical trials, as they can offer additional, meaningful insights into complex problems that simply could not be provided by quantitative methods alone. Qualitative methods can also be used to further facilitate in-depth understanding of important aspects of clinical trial processes, such as recruitment. For example, Ellis et al . investigated why edentulous older patients, dissatisfied with conventional dentures, decline implant treatment, despite its established efficacy, and frequently refuse to participate in related randomised clinical trials, even when financial constraints are removed. 7 Through the use of focus groups in Canada and the UK, the authors found that fears of pain and potential complications, along with perceived embarrassment, exacerbated by age, are common reasons why older patients typically refuse dental implants. 7

The last decade has also seen further developments in qualitative research, due to the ongoing evolution of digital technologies. These developments have transformed how researchers can access and share information, communicate and collaborate, recruit and engage participants, collect and analyse data and disseminate and translate research findings. 8 Where appropriate, such technologies are therefore capable of extending and enhancing how qualitative research is undertaken. 9 For example, it is now possible to collect qualitative data via instant messaging, email or online/video chat, using appropriate online platforms.

These innovative approaches to research are therefore cost-effective, convenient, reduce geographical constraints and are often useful for accessing 'hard to reach' participants (for example, those who are immobile or socially isolated). 8 , 9 However, digital technologies are still relatively new and constantly evolving and therefore present a variety of pragmatic and methodological challenges. Furthermore, given their very nature, their use in many qualitative studies and/or with certain participant groups may be inappropriate and should therefore always be carefully considered. While it is beyond the scope of this paper to provide a detailed explication regarding the use of digital technologies in qualitative research, insight is provided into how such technologies can be used to facilitate the data collection process in interviews and focus groups.

In light of such developments, it is perhaps therefore timely to update the main paper 3 of the original BDJ series. As with the previous publications, this paper has been purposely written in an accessible style, to enhance readability, particularly for those who are new to qualitative research. While the focus remains on the most common qualitative methods of data collection – interviews and focus groups – appropriate revisions have been made to provide a novel perspective, and should therefore be helpful to those who would like to know more about qualitative research. This paper specifically focuses on undertaking qualitative research with adult participants only.

Overview of qualitative research

Qualitative research is an approach that focuses on people and their experiences, behaviours and opinions. 10 , 11 The qualitative researcher seeks to answer questions of 'how' and 'why', providing detailed insight and understanding, 11 which quantitative methods cannot reach. 12 Within qualitative research, there are distinct methodologies influencing how the researcher approaches the research question, data collection and data analysis. 13 For example, phenomenological studies focus on the lived experience of individuals, explored through their description of the phenomenon. Ethnographic studies explore the culture of a group and typically involve the use of multiple methods to uncover the issues. 14

While methodology is the 'thinking tool', the methods are the 'doing tools'; 13 the ways in which data are collected and analysed. There are multiple qualitative data collection methods, including interviews, focus groups, observations, documentary analysis, participant diaries, photography and videography. Two of the most commonly used qualitative methods are interviews and focus groups, which are explored in this article. The data generated through these methods can be analysed in one of many ways, according to the methodological approach chosen. A common approach is thematic data analysis, involving the identification of themes and subthemes across the data set. Further information on approaches to qualitative data analysis has been discussed elsewhere. 1

Qualitative research is an evolving and adaptable approach, used by different disciplines for different purposes. Traditionally, qualitative data, specifically interviews, focus groups and observations, have been collected face-to-face with participants. In more recent years, digital technologies have contributed to the ongoing evolution of qualitative research. Digital technologies offer researchers different ways of recruiting participants and collecting data, and offer participants opportunities to be involved in research that is not necessarily face-to-face.

Research interviews are a fundamental qualitative research method 15 and are utilised across methodological approaches. Interviews enable the researcher to learn in depth about the perspectives, experiences, beliefs and motivations of the participant. 3 , 16 Examples include, exploring patients' perspectives of fear/anxiety triggers in dental treatment, 17 patients' experiences of oral health and diabetes, 18 and dental students' motivations for their choice of career. 19

Interviews may be structured, semi-structured or unstructured, 3 according to the purpose of the study, with less structured interviews facilitating a more in depth and flexible interviewing approach. 20 Structured interviews are similar to verbal questionnaires and are used if the researcher requires clarification on a topic; however they produce less in-depth data about a participant's experience. 3 Unstructured interviews may be used when little is known about a topic and involves the researcher asking an opening question; 3 the participant then leads the discussion. 20 Semi-structured interviews are commonly used in healthcare research, enabling the researcher to ask predetermined questions, 20 while ensuring the participant discusses issues they feel are important.

Interviews can be undertaken face-to-face or using digital methods when the researcher and participant are in different locations. Audio-recording the interview, with the consent of the participant, is essential for all interviews regardless of the medium as it enables accurate transcription; the process of turning the audio file into a word-for-word transcript. This transcript is the data, which the researcher then analyses according to the chosen approach.

Types of interview

Qualitative studies often utilise one-to-one, face-to-face interviews with research participants. This involves arranging a mutually convenient time and place to meet the participant, signing a consent form and audio-recording the interview. However, digital technologies have expanded the potential for interviews in research, enabling individuals to participate in qualitative research regardless of location.

Telephone interviews can be a useful alternative to face-to-face interviews and are commonly used in qualitative research. They enable participants from different geographical areas to participate and may be less onerous for participants than meeting a researcher in person. 15 A qualitative study explored patients' perspectives of dental implants and utilised telephone interviews due to the quality of the data that could be yielded. 21 The researcher needs to consider how they will audio record the interview, which can be facilitated by purchasing a recorder that connects directly to the telephone. One potential disadvantage of telephone interviews is the inability of the interviewer and researcher to see each other. This is resolved using software for audio and video calls online – such as Skype – to conduct interviews with participants in qualitative studies. Advantages of this approach include being able to see the participant if video calls are used, enabling observation of non-verbal communication, and the software can be free to use. However, participants are required to have a device and internet connection, as well as being computer literate, potentially limiting who can participate in the study. One qualitative study explored the role of dental hygienists in reducing oral health disparities in Canada. 22 The researcher conducted interviews using Skype, which enabled dental hygienists from across Canada to be interviewed within the research budget, accommodating the participants' schedules. 22

A less commonly used approach to qualitative interviews is the use of social virtual worlds. A qualitative study accessed a social virtual world – Second Life – to explore the health literacy skills of individuals who use social virtual worlds to access health information. 23 The researcher created an avatar and interview room, and undertook interviews with participants using voice and text methods. 23 This approach to recruitment and data collection enables individuals from diverse geographical locations to participate, while remaining anonymous if they wish. Furthermore, for interviews conducted using text methods, transcription of the interview is not required as the researcher can save the written conversation with the participant, with the participant's consent. However, the researcher and participant need to be familiar with how the social virtual world works to engage in an interview this way.

Conducting an interview

Ensuring informed consent before any interview is a fundamental aspect of the research process. Participants in research must be afforded autonomy and respect; consent should be informed and voluntary. 24 Individuals should have the opportunity to read an information sheet about the study, ask questions, understand how their data will be stored and used, and know that they are free to withdraw at any point without reprisal. The qualitative researcher should take written consent before undertaking the interview. In a face-to-face interview, this is straightforward: the researcher and participant both sign copies of the consent form, keeping one each. However, this approach is less straightforward when the researcher and participant do not meet in person. A recent protocol paper outlined an approach for taking consent for telephone interviews, which involved: audio recording the participant agreeing to each point on the consent form; the researcher signing the consent form and keeping a copy; and posting a copy to the participant. 25 This process could be replicated in other interview studies using digital methods.

There are advantages and disadvantages of using face-to-face and digital methods for research interviews. Ultimately, for both approaches, the quality of the interview is determined by the researcher. 16 Appropriate training and preparation are thus required. Healthcare professionals can use their interpersonal communication skills when undertaking a research interview, particularly questioning, listening and conversing. 3 However, the purpose of an interview is to gain information about the study topic, 26 rather than offering help and advice. 3 The researcher therefore needs to listen attentively to participants, enabling them to describe their experience without interruption. 3 The use of active listening skills also help to facilitate the interview. 14 Spradley outlined elements and strategies for research interviews, 27 which are a useful guide for qualitative researchers:

Greeting and explaining the project/interview

Asking descriptive (broad), structural (explore response to descriptive) and contrast (difference between) questions

Asymmetry between the researcher and participant talking

Expressing interest and cultural ignorance

Repeating, restating and incorporating the participant's words when asking questions

Creating hypothetical situations

Asking friendly questions

Knowing when to leave.

For semi-structured interviews, a topic guide (also called an interview schedule) is used to guide the content of the interview – an example of a topic guide is outlined in Box 1 . The topic guide, usually based on the research questions, existing literature and, for healthcare professionals, their clinical experience, is developed by the research team. The topic guide should include open ended questions that elicit in-depth information, and offer participants the opportunity to talk about issues important to them. This is vital in qualitative research where the researcher is interested in exploring the experiences and perspectives of participants. It can be useful for qualitative researchers to pilot the topic guide with the first participants, 10 to ensure the questions are relevant and understandable, and amending the questions if required.

Regardless of the medium of interview, the researcher must consider the setting of the interview. For face-to-face interviews, this could be in the participant's home, in an office or another mutually convenient location. A quiet location is preferable to promote confidentiality, enable the researcher and participant to concentrate on the conversation, and to facilitate accurate audio-recording of the interview. For interviews using digital methods the same principles apply: a quiet, private space where the researcher and participant feel comfortable and confident to participate in an interview.

Box 1: Example of a topic guide

Study focus: Parents' experiences of brushing their child's (aged 0–5) teeth

1. Can you tell me about your experience of cleaning your child's teeth?

How old was your child when you started cleaning their teeth?

Why did you start cleaning their teeth at that point?

How often do you brush their teeth?

What do you use to brush their teeth and why?

2. Could you explain how you find cleaning your child's teeth?

Do you find anything difficult?

What makes cleaning their teeth easier for you?

3. How has your experience of cleaning your child's teeth changed over time?

Has it become easier or harder?

Have you changed how often and how you clean their teeth? If so, why?

4. Could you describe how your child finds having their teeth cleaned?

What do they enjoy about having their teeth cleaned?

Is there anything they find upsetting about having their teeth cleaned?

5. Where do you look for information/advice about cleaning your child's teeth?

What did your health visitor tell you about cleaning your child's teeth? (If anything)

What has the dentist told you about caring for your child's teeth? (If visited)

Have any family members given you advice about how to clean your child's teeth? If so, what did they tell you? Did you follow their advice?

6. Is there anything else you would like to discuss about this?

Focus groups

A focus group is a moderated group discussion on a pre-defined topic, for research purposes. 28 , 29 While not aligned to a particular qualitative methodology (for example, grounded theory or phenomenology) as such, focus groups are used increasingly in healthcare research, as they are useful for exploring collective perspectives, attitudes, behaviours and experiences. Consequently, they can yield rich, in-depth data and illuminate agreement and inconsistencies 28 within and, where appropriate, between groups. Examples include public perceptions of dental implants and subsequent impact on help-seeking and decision making, 30 and general dental practitioners' views on patient safety in dentistry. 31

Focus groups can be used alone or in conjunction with other methods, such as interviews or observations, and can therefore help to confirm, extend or enrich understanding and provide alternative insights. 28 The social interaction between participants often results in lively discussion and can therefore facilitate the collection of rich, meaningful data. However, they are complex to organise and manage, due to the number of participants, and may also be inappropriate for exploring particularly sensitive issues that many participants may feel uncomfortable about discussing in a group environment.

Focus groups are primarily undertaken face-to-face but can now also be undertaken online, using appropriate technologies such as email, bulletin boards, online research communities, chat rooms, discussion forums, social media and video conferencing. 32 Using such technologies, data collection can also be synchronous (for example, online discussions in 'real time') or, unlike traditional face-to-face focus groups, asynchronous (for example, online/email discussions in 'non-real time'). While many of the fundamental principles of focus group research are the same, regardless of how they are conducted, a number of subtle nuances are associated with the online medium. 32 Some of which are discussed further in the following sections.

Focus group considerations

Some key considerations associated with face-to-face focus groups are: how many participants are required; should participants within each group know each other (or not) and how many focus groups are needed within a single study? These issues are much debated and there is no definitive answer. However, the number of focus groups required will largely depend on the topic area, the depth and breadth of data needed, the desired level of participation required 29 and the necessity (or not) for data saturation.

The optimum group size is around six to eight participants (excluding researchers) but can work effectively with between three and 14 participants. 3 If the group is too small, it may limit discussion, but if it is too large, it may become disorganised and difficult to manage. It is, however, prudent to over-recruit for a focus group by approximately two to three participants, to allow for potential non-attenders. For many researchers, particularly novice researchers, group size may also be informed by pragmatic considerations, such as the type of study, resources available and moderator experience. 28 Similar size and mix considerations exist for online focus groups. Typically, synchronous online focus groups will have around three to eight participants but, as the discussion does not happen simultaneously, asynchronous groups may have as many as 10–30 participants. 33

The topic area and potential group interaction should guide group composition considerations. Pre-existing groups, where participants know each other (for example, work colleagues) may be easier to recruit, have shared experiences and may enjoy a familiarity, which facilitates discussion and/or the ability to challenge each other courteously. 3 However, if there is a potential power imbalance within the group or if existing group norms and hierarchies may adversely affect the ability of participants to speak freely, then 'stranger groups' (that is, where participants do not already know each other) may be more appropriate. 34 , 35

Focus group management

Face-to-face focus groups should normally be conducted by two researchers; a moderator and an observer. 28 The moderator facilitates group discussion, while the observer typically monitors group dynamics, behaviours, non-verbal cues, seating arrangements and speaking order, which is essential for transcription and analysis. The same principles of informed consent, as discussed in the interview section, also apply to focus groups, regardless of medium. However, the consent process for online discussions will probably be managed somewhat differently. For example, while an appropriate participant information leaflet (and consent form) would still be required, the process is likely to be managed electronically (for example, via email) and would need to specifically address issues relating to technology (for example, anonymity and use, storage and access to online data). 32

The venue in which a face to face focus group is conducted should be of a suitable size, private, quiet, free from distractions and in a collectively convenient location. It should also be conducted at a time appropriate for participants, 28 as this is likely to promote attendance. As with interviews, the same ethical considerations apply (as discussed earlier). However, online focus groups may present additional ethical challenges associated with issues such as informed consent, appropriate access and secure data storage. Further guidance can be found elsewhere. 8 , 32

Before the focus group commences, the researchers should establish rapport with participants, as this will help to put them at ease and result in a more meaningful discussion. Consequently, researchers should introduce themselves, provide further clarity about the study and how the process will work in practice and outline the 'ground rules'. Ground rules are designed to assist, not hinder, group discussion and typically include: 3 , 28 , 29

Discussions within the group are confidential to the group

Only one person can speak at a time

All participants should have sufficient opportunity to contribute

There should be no unnecessary interruptions while someone is speaking

Everyone can be expected to be listened to and their views respected

Challenging contrary opinions is appropriate, but ridiculing is not.

Moderating a focus group requires considered management and good interpersonal skills to help guide the discussion and, where appropriate, keep it sufficiently focused. Avoid, therefore, participating, leading, expressing personal opinions or correcting participants' knowledge 3 , 28 as this may bias the process. A relaxed, interested demeanour will also help participants to feel comfortable and promote candid discourse. Moderators should also prevent the discussion being dominated by any one person, ensure differences of opinions are discussed fairly and, if required, encourage reticent participants to contribute. 3 Asking open questions, reflecting on significant issues, inviting further debate, probing responses accordingly, and seeking further clarification, as and where appropriate, will help to obtain sufficient depth and insight into the topic area.

Moderating online focus groups requires comparable skills, particularly if the discussion is synchronous, as the discussion may be dominated by those who can type proficiently. 36 It is therefore important that sufficient time and respect is accorded to those who may not be able to type as quickly. Asynchronous discussions are usually less problematic in this respect, as interactions are less instant. However, moderating an asynchronous discussion presents additional challenges, particularly if participants are geographically dispersed, as they may be online at different times. Consequently, the moderator will not always be present and the discussion may therefore need to occur over several days, which can be difficult to manage and facilitate and invariably requires considerable flexibility. 32 It is also worth recognising that establishing rapport with participants via online medium is often more challenging than via face-to-face and may therefore require additional time, skills, effort and consideration.

As with research interviews, focus groups should be guided by an appropriate interview schedule, as discussed earlier in the paper. For example, the schedule will usually be informed by the review of the literature and study aims, and will merely provide a topic guide to help inform subsequent discussions. To provide a verbatim account of the discussion, focus groups must be recorded, using an audio-recorder with a good quality multi-directional microphone. While videotaping is possible, some participants may find it obtrusive, 3 which may adversely affect group dynamics. The use (or not) of a video recorder, should therefore be carefully considered.

At the end of the focus group, a few minutes should be spent rounding up and reflecting on the discussion. 28 Depending on the topic area, it is possible that some participants may have revealed deeply personal issues and may therefore require further help and support, such as a constructive debrief or possibly even referral on to a relevant third party. It is also possible that some participants may feel that the discussion did not adequately reflect their views and, consequently, may no longer wish to be associated with the study. 28 Such occurrences are likely to be uncommon, but should they arise, it is important to further discuss any concerns and, if appropriate, offer them the opportunity to withdraw (including any data relating to them) from the study. Immediately after the discussion, researchers should compile notes regarding thoughts and ideas about the focus group, which can assist with data analysis and, if appropriate, any further data collection.

Qualitative research is increasingly being utilised within dental research to explore the experiences, perspectives, motivations and beliefs of participants. The contributions of qualitative research to evidence-based practice are increasingly being recognised, both as standalone research and as part of larger mixed-method studies, including clinical trials. Interviews and focus groups remain commonly used data collection methods in qualitative research, and with the advent of digital technologies, their utilisation continues to evolve. However, digital methods of qualitative data collection present additional methodological, ethical and practical considerations, but also potentially offer considerable flexibility to participants and researchers. Consequently, regardless of format, qualitative methods have significant potential to inform important areas of dental practice, policy and further related research.

Gussy M, Dickson-Swift V, Adams J . A scoping review of qualitative research in peer-reviewed dental publications. Int J Dent Hygiene 2013; 11 : 174–179.

Article   Google Scholar  

Burnard P, Gill P, Stewart K, Treasure E, Chadwick B . Analysing and presenting qualitative data. Br Dent J 2008; 204 : 429–432.

Gill P, Stewart K, Treasure E, Chadwick B . Methods of data collection in qualitative research: interviews and focus groups. Br Dent J 2008; 204 : 291–295.

Gill P, Stewart K, Treasure E, Chadwick B . Conducting qualitative interviews with school children in dental research. Br Dent J 2008; 204 : 371–374.

Stewart K, Gill P, Chadwick B, Treasure E . Qualitative research in dentistry. Br Dent J 2008; 204 : 235–239.

Masood M, Thaliath E, Bower E, Newton J . An appraisal of the quality of published qualitative dental research. Community Dent Oral Epidemiol 2011; 39 : 193–203.

Ellis J, Levine A, Bedos C et al. Refusal of implant supported mandibular overdentures by elderly patients. Gerodontology 2011; 28 : 62–68.

Macfarlane S, Bucknall T . Digital Technologies in Research. In Gerrish K, Lathlean J (editors) The Research Process in Nursing . 7th edition. pp. 71–86. Oxford: Wiley Blackwell; 2015.

Google Scholar  

Lee R, Fielding N, Blank G . Online Research Methods in the Social Sciences: An Editorial Introduction. In Fielding N, Lee R, Blank G (editors) The Sage Handbook of Online Research Methods . pp. 3–16. London: Sage Publications; 2016.

Creswell J . Qualitative inquiry and research design: Choosing among five designs . Thousand Oaks, CA: Sage, 1998.

Guest G, Namey E, Mitchell M . Qualitative research: Defining and designing In Guest G, Namey E, Mitchell M (editors) Collecting Qualitative Data: A Field Manual For Applied Research . pp. 1–40. London: Sage Publications, 2013.

Chapter   Google Scholar  

Pope C, Mays N . Qualitative research: Reaching the parts other methods cannot reach: an introduction to qualitative methods in health and health services research. BMJ 1995; 311 : 42–45.

Giddings L, Grant B . A Trojan Horse for positivism? A critique of mixed methods research. Adv Nurs Sci 2007; 30 : 52–60.

Hammersley M, Atkinson P . Ethnography: Principles in Practice . London: Routledge, 1995.

Oltmann S . Qualitative interviews: A methodological discussion of the interviewer and respondent contexts Forum Qualitative Sozialforschung/Forum: Qualitative Social Research. 2016; 17 : Art. 15.

Patton M . Qualitative Research and Evaluation Methods . Thousand Oaks, CA: Sage, 2002.

Wang M, Vinall-Collier K, Csikar J, Douglas G . A qualitative study of patients' views of techniques to reduce dental anxiety. J Dent 2017; 66 : 45–51.

Lindenmeyer A, Bowyer V, Roscoe J, Dale J, Sutcliffe P . Oral health awareness and care preferences in patients with diabetes: a qualitative study. Fam Pract 2013; 30 : 113–118.

Gallagher J, Clarke W, Wilson N . Understanding the motivation: a qualitative study of dental students' choice of professional career. Eur J Dent Educ 2008; 12 : 89–98.

Tod A . Interviewing. In Gerrish K, Lacey A (editors) The Research Process in Nursing . Oxford: Blackwell Publishing, 2006.

Grey E, Harcourt D, O'Sullivan D, Buchanan H, Kipatrick N . A qualitative study of patients' motivations and expectations for dental implants. Br Dent J 2013; 214 : 10.1038/sj.bdj.2012.1178.

Farmer J, Peressini S, Lawrence H . Exploring the role of the dental hygienist in reducing oral health disparities in Canada: A qualitative study. Int J Dent Hygiene 2017; 10.1111/idh.12276.

McElhinney E, Cheater F, Kidd L . Undertaking qualitative health research in social virtual worlds. J Adv Nurs 2013; 70 : 1267–1275.

Health Research Authority. UK Policy Framework for Health and Social Care Research. Available at https://www.hra.nhs.uk/planning-and-improving-research/policies-standards-legislation/uk-policy-framework-health-social-care-research/ (accessed September 2017).

Baillie J, Gill P, Courtenay P . Knowledge, understanding and experiences of peritonitis among patients, and their families, undertaking peritoneal dialysis: A mixed methods study protocol. J Adv Nurs 2017; 10.1111/jan.13400.

Kvale S . Interviews . Thousand Oaks (CA): Sage, 1996.

Spradley J . The Ethnographic Interview . New York: Holt, Rinehart and Winston, 1979.

Goodman C, Evans C . Focus Groups. In Gerrish K, Lathlean J (editors) The Research Process in Nursing . pp. 401–412. Oxford: Wiley Blackwell, 2015.

Shaha M, Wenzell J, Hill E . Planning and conducting focus group research with nurses. Nurse Res 2011; 18 : 77–87.

Wang G, Gao X, Edward C . Public perception of dental implants: a qualitative study. J Dent 2015; 43 : 798–805.

Bailey E . Contemporary views of dental practitioners' on patient safety. Br Dent J 2015; 219 : 535–540.

Abrams K, Gaiser T . Online Focus Groups. In Field N, Lee R, Blank G (editors) The Sage Handbook of Online Research Methods . pp. 435–450. London: Sage Publications, 2016.

Poynter R . The Handbook of Online and Social Media Research . West Sussex: John Wiley & Sons, 2010.

Kevern J, Webb C . Focus groups as a tool for critical social research in nurse education. Nurse Educ Today 2001; 21 : 323–333.

Kitzinger J, Barbour R . Introduction: The Challenge and Promise of Focus Groups. In Barbour R S K J (editor) Developing Focus Group Research . pp. 1–20. London: Sage Publications, 1999.

Krueger R, Casey M . Focus Groups: A Practical Guide for Applied Research. 4th ed. Thousand Oaks, California: SAGE; 2009.

Download references

Author information

Authors and affiliations.

Senior Lecturer (Adult Nursing), School of Healthcare Sciences, Cardiff University,

Lecturer (Adult Nursing) and RCBC Wales Postdoctoral Research Fellow, School of Healthcare Sciences, Cardiff University,

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to P. Gill .

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Gill, P., Baillie, J. Interviews and focus groups in qualitative research: an update for the digital age. Br Dent J 225 , 668–672 (2018). https://doi.org/10.1038/sj.bdj.2018.815

Download citation

Accepted : 02 July 2018

Published : 05 October 2018

Issue Date : 12 October 2018

DOI : https://doi.org/10.1038/sj.bdj.2018.815

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

• Research article
• Open access
• Published: 30 November 2020

Improving the use of focus group discussions in low income settings

• Pauline F. D. Scheelbeek   ORCID: orcid.org/0000-0002-6209-2284 1 ,
• Yashua A. Hamza 2 ,
• Joanna Schellenberg 3 &
• Zelee Hill 4  

BMC Medical Research Methodology volume  20 , Article number:  287 ( 2020 ) Cite this article

9909 Accesses

7 Altmetric

Metrics details

The quality of data obtained through Focus Group Discussions (FGDs) is highly dependent on appropriate design and facilitation. In low-income settings steep power gradients between researcher and participants, as well as conversational norms, could reduce the ability of participants to voice personal opinions. Activity-oriented exercises have been suggested as a way overcoming these challenges, however little evidence exists - to date - on their use in low-income settings. We selected six exercises for use in Ethiopia and Nigeria and report our experiences.

The six exercises (picture sorting, associative pictures, picture ranking, decision trees, predictive story-telling and provocative statements) were used in 32 maternal and new-born care themed FGDs conducted in Amhara and Southern Nations Nationalities and People’s Regions (Ethiopia) and Gombe State (Nigeria). Six facilitators and two supervisors who used these exercises were interviewed about their experiences. FGD verbatim transcripts and interview notes were analysed to explore methodological effectiveness and respondents’ experience. All data were coded in NVIVO using a deductive coding frame.

Facilitators and participants described the methods as ‘fun’ and ‘enjoyable’. The exercises yielded more in-depth and complete information than ‘normal’ FGDs, but facilitator’s probing skills and overall FGD group dynamics proved crucial in this success. Explaining and conducting the exercises increased FGD length. Data richness, participant reaction and understanding, and ease of facilitation varied by study site, exercise, and participant group. Overall, the exercises worked better in Nigeria than in Ethiopia. The provocative statement exercise was most difficult for participants to understand, the decision-tree most difficult to facilitate and the picture exercises most enjoyable. The story telling exercise took relatively little time, was well understood, yielded rich data and reduced social desirability bias.

The majority of the exercises proved successful tools in yielding richer and less biased information from FGDs and were experienced as fun and engaging. Tailoring of the exercises, as well as thorough training and selection of the facilitators, were pivotal in this success. The difference in the two countries shows that adequate piloting and adaptation is crucial, and that some exercises may not be adaptable to all settings.

Peer Review reports

Focus group discussions (FGDs) have been used in public health research since the 1990s [ 1 ]. They aim to explore participants’ experiences, beliefs, and attitudes by using group processes to stimulate responses and gain insights through participants exchanging views, and questioning and challenging each other [ 1 , 2 , 3 , 4 , 5 ]. They are often perceived as a cost and time efficient way of collecting information from multiple participants; and it is this efficiency, rather than their methodological strengths, that is often highlighted in the literature [ 2 ]. FGDs are commonly conducted as group interviews (hence losing the advantages that an FGD bring to data collection), with interaction between the facilitator and each participant in turn, rather than between participants [ 6 , 7 ]. As a result, the quality of the research they produce has been questioned [ 1 , 7 ].

Most of the literature on how to conduct FGDs comes from high income settings [ 7 , 8 ], and guidelines for low income settings have been criticized for focusing on practicalities, rather than on managing contextual challenges in the interactive process [ 9 ]. Challenges include steep power gradients between researcher and participants, and conversational norms, reducing the ability of participants to voice disagreements, or provide personal opinions [ 7 ]. We have used focus groups in Ghana for over 15 years, and have had difficulty making FGDs interactive, with both facilitators and participants seemingly more comfortable with group interviews. Over this time we noticed that the level of participant-participant interaction increased during activity oriented exercise, a similar experience reported by researchers in Tanzania [ 7 ].

Based on a review of the literature and our experiences in Ghana, we decided to trial the use of a variety of exercise. As well as encouraging interaction, we wanted to see if using exercises could improve the quality of our data around the drivers of behavior change. Understanding behaviors can be challenging as individuals can have difficulty explaining their own behaviors, may be unaware of some of the factors that influence their behavior, or there may be social desirability bias in reporting (e.g. [ 10 , 11 ]). These issues can lead to participants providing an unrealistic impression of their behaviors, giving superficial answers and being strongly influenced by probes [ 12 , 13 , 14 ]. Exercises may help reduce these problems by encouraging respondents to reflect on their behavior and by provoking gut reactions that uncover hidden reasons for behaviors and reduce social desirability bias. They can also focus participants on what ‘other’ people do, this may improve data quality as people are often better at explaining the behavior of others rather than their own, and because projecting their answers onto others may reduce social desirability bias [ 15 , 16 ].

Activity oriented exercise, such as sorting, story-telling and sentence completion, have been suggested as a way of improving focus groups for over two decades [ 6 ]. They are described as a means to make a group more enjoyable, increase interaction, reduce boredom, focus attention, increase reflexivity, and make sensitive topics less threatening [ 5 , 6 ]. Despite their potential advantage social scientists often rely on discussion questions, whilst exercises are more widely used in market research [ 9 ]. We found little written about the use of these exercises in low- income settings.

During the planning for a study in Ethiopia and Nigeria, we conducted a scoping literature review to identify exercises that could enhance our data collection. We searched PubMed/MEDLINE, Web of Science, Scopus, and Cochrane, screened relevant websites and expert forums for grey literature, conducted internet searches, and asked experts for relevant citations. We included literature from fields such as public health, sociology, market research, economics and criminology. We identified 36 exercises and generated a set of tables where we described each exercise, reflected on how it could be used in the study, on its advantages and disadvantages and on which biases it may help overcome. Based on these reflections we then scored each method as having high, medium or low potential for overcoming social desirability bias, improve enjoyment and yield rich in-depth data for each of the study behaviours in our study settings. These tables can be found in the supplementary files. We then met as a study team to review the tables and selected six exercises for use. This paper describes our experiences using these six methods.

The six exercises were used as part of a study on maternal and newborn care practices that explored the role that community health workers play in behaviour change [ 17 , 18 , 19 ]. The research, in Ethiopia and Nigeria, consisted of 49 interviews with recent mothers, 13 “friendship pair” interviews – in which mothers and a friend were interviewed together – and 32 FGDs with mothers, grandmothers, fathers and community health workers.

In Ethiopia, we conducted the study in four Woreda (districts) in Amhara and The Southern Nations, Nationalities and Peoples (SNNP) Regions. The Woreda were selected because they had relatively well functioning community health workers and were ‘typical’ – that is they had no unusual characteristics in terms of employment, accessibility, or population. Within each Woreda we selected a Kebele (smallest administrative division) that was reasonably accessible to the data collection team. The majority of respondents worked in agriculture and were typical for the area in terms of socio-economic class and education. The age of mothers in the FGDs ranged from 18 to 30 years old (mean = 26) and the mean reported number of children was 3.1 (range 1–7); for fathers and grandmothers age ranges were 28–45 years (mean = 36) and 36–75 years (mean = 55), with a mean reported number of children of 4 and 5 respectively. The median number of years in education was 3 for mothers and 5 for fathers, and of all interviewees 69% were Christian (predominantly Orthodox) and 31% Muslim. The interviewed community health workers had an average of 8.5 (Health Extension Workers - HEWs) and 1.9 (Health Development Army volunteers- HDAs) years of experience.

In Nigeria, we collected data from two Local Government Areas (LGAs) in Gombe State, a heterogenous state comprising multi-ethnic groups of which Fulani is the largest. One selected LGA was predominantly Christian and the other Muslim. Within each LGA, two communities were selected: the LGA headquarters and an accessible rural community that, for security reasons, would allow the researchers to be back in Gombe City before sunset. The age range of mothers in the FGDs ranged from 20 to 40 years of age (mean = 28) and the mean reported number of children was 3.9 (range 1–9); for fathers and grandmothers age ranges were 30–61 years (mean = 43) and 30–70 years (mean = 53), with a mean reported number of children of 7.7 and 5.8 respectively. Interviewees in Nigeria had usually received more years of education than those in Ethiopia: the median number of years in education was 6 for mothers and 12 for fathers. Over a third (37%) of all interviewees were Christians, whilst all others were Muslims. The interviewed frontline workers had an average of 2 (Federation of Muslim Women’s Associations in Nigeria Volunteers– FOMWANs) and 3.2 (Traditional Birth Attendants – TBAs) years of experience.

The FGD data were collected in 2015, by four interviewers in Ethiopia and six in Nigeria. The Ethiopian interviewers, 2 male and 2 female, had on average 5 years of experience in qualitative data collection and were skilled in facilitating focus group discussions. They were native speakers in the languages in which FGDs were conducted, apart from Silte, where we used translators. Cross-religious interviewing did not pose a problem, but focus groups with fathers were all conducted by male interviewers. The Nigerian interviewers, 3 male and 3 female, had varying experience in qualitative data collection (between 2 and 17 years), and were native speakers of the languages used in the FGDs. The more junior interviewers took the role of note takers, whilst the more senior interviewers facilitated the groups. Religion of facilitators and participants were matched, but interviewers facilitated FGDs with both sexes.

Facilitators and translators had 4 days of classroom training, including 2 days focusing on the activity-oriented exercises. They had 2 days of training on verbatim transcript writing, administration and data safety & confidentiality issues. During the training we explained the objectives of each exercise, reviewed the discussion guides and conducted role plays. This was followed by a two-day pilot test, where the semi-structured guides were adjusted, the exercises discussed and problems resolved. The FGDs consisted of 3–7 participants and were conducted in neutral locations such as community centres, the length of the FGDs are shown in Table  1 . Study aims were explained and written consent was obtained from each participant (or their designated proxy if the participant was illiterate) before the start of the FGDs. The development of the FGD topic guides has been described in detail elsewhere [ 17 , 18 , 19 ].

All FGDs were audio-recorded and transcribed by the interviewers in English within a week of data collection. During data collection, interviewers received regular feedback from senior researchers on their facilitation techniques and the use of the exercises. We also held a review meeting in the middle of data collection, to reflect on our findings and experiences and adjust data collection as needed.

Each exercise is described below along with their aims. Increasing interaction between participants was an aim of all the exercises and is not listed. The exercises aimed to be culturally acceptable, salient to participants and feasible to conduct. This included adjusting exercises where literacy was required.

Picture pile sorting

Participants were asked to work together to put seven pictures of newborn care practices into piles of those commonly practiced or not commonly practiced in their community. They were then asked to pile the same pictures into practices they felt were important or not important for the health of the baby. Finally, they were asked to pile the pictures into practices that that were promoted, or not promoted, by Community Health Workers (CHW). Facilitators probed on reasons for the classification and encouraged participants to provide details of the practices. The exercise aimed to focus the participants on the task rather than on the facilitator and increase reflexivity. Pictures for this exercise (as well as the associative picture and picture ranking exercises) were carefully selected with the help of facilitators, local government officials and community health workers. Where needed, pictures were taken in the local area to ensure they reflected the local context.

Associative pictures

Participants were shown a picture of a newborn care practice and asked to give their immediate reaction to the picture. Facilitators probed to understand the reaction. In total 5 pictures were shown. Their responses were used to facilitate further discussions within the group. By encouraging an immediate reaction, we aimed to capture the participants’ first emotions and responses to the pictures, with the aim of decreasing social desirability bias in their answers and uncovering hidden drivers of behaviour.

Picture ranking

Participants were shown pictures representing the immediate family, a neighbour and a CHW. They were asked to rank the people from most to least influential on newborn care practices. If two or more people had equal influence they could be placed next to each other. After ranking the pictures, the participants were asked about any ranking differences for specific care elements, such as bathing, feeding and thermal care. This exercise aimed to focus the participants on the task and increase reflexivity.

Decision tree (root-cause analysis)

A trunk of a tree, its main roots and sub roots were drawn on a large flipchart. The central problem was written down on the trunk of the tree. Participants were then asked to give underlying reasons for why the problem may occur. The sub roots were subsequently used to document “reasons for those reasons”. When applicable further subdivisions were made to determine the core of the problem. The exercise aimed to yield in-depth answers for complex questions by triggering the participants’ cognitive skills and encouraging them to think in a more comprehensive and reflexive way about the problem presented.

Predictive story telling

The facilitator read a fictitious, but realistic, story that needed to be finished by the participants. The story explored a dilemma new mothers may face when receiving conflicting advice from family members and community health workers. Once the first part of the story was completed, participants were given some new information and asked to say what would happen next. The exercise aimed to reduce embarrassment and social desirability bias, as participants can give their answers in the third person (i.e. as if they are talking about the fictitious person) whilst reflecting on their own practices.

Provocative statements (Q-statements)

A series of controversial statements were read out to the participants, who were encouraged to respond as soon as the facilitator finished the statement. As for the associative picture exercise, this exercise aimed to capture gut reactions, to reduce social desirability bias and uncover hidden drivers of behaviours.

The exercises described above were used in FGDs with mothers, grandmothers, fathers and community health workers. Each respondent group had a specially designed interview guide that included 3–4 of the selected methods as shown in Table 1 .

Our experiences with the exercises were evaluated in three ways: Interviews with and reflections of senior researchers who observed 16 of the 40 pilot and actual FGDs, and who supervised the data collectors; skype or phone interviews with eight of the 10 data collectors about their experiences and views on the exercises; and a review of all the FGD transcripts to explore how well the methods worked and how the respondents reacted to them. The skype or phone interviews were conducted in 2015, 4 months after the completion of data collection. Interviews were conducted in English by one of us (PS) using a semi structured guide which asked about overall impression of the exercises in the FGDs, advantages and disadvantages of each exercise, as well the ability of the exercise to reach its objective, such as yielding rich and valid data, overcoming social desirability bias, participant engagement, and enjoyment of the exercises for participants and facilitator. Furthermore, facilitators were asked whether they had any recommendations for improvement for each of the exercises. (Supplementary Files) Notes were taken during the interviews, which were written up as expanded notes immediately after the interview.

Data analysis was conducted by PS and ZH who met regularly during the analysis process to discuss the coding and emerging themes. Analysis started with multiple readings of the expanded notes and FGD transcripts to ensure familiarity. Analysis was conducted in Nvivo with nodes for each exercise, and was both deductive and inductive. The deductive themes were: data quality (sub themes were data depth/richness, interactions between participants and impact on social desirability bias); feasibility (sub themes were ease of facilitation, participant comprehension and impact on FGD length) and participant reaction (sub themes enjoyment).

Analysis of the FGD transcripts included coding all participant comments on the methods (e.g. we are enjoying this, I don’t understand this, this is taking too long) and any other reactions (e.g. laughter, silence, confusion, enthusiasm, distraction). For the sub-theme data depth/richness we coded examples of the length and depth of responses and for impact on social desirability bias examples of guarded responses, frank/open responses and any statement where respondents were critical or the services they had received. To put these findings in context we generated memos for each FGD on the overall group dynamics, the facilitators skills in probing and managing the group and anything that seemed to make a method work well or nor so well.

The skype or phone interviews and the FGDs were analyzed separately and the main themes and findings were then compared. This was bi-directional as we explored whether the findings from the interviews with study staff were reflected in the FGD data (e.g. if the staff said a method was challenging for participants could we see evidence of this in the FGD transcripts) as well as exploring if the study staff interviews could explain the results of the FGD analysis (e.g. if a method garnered very rich FGD data could the staff interviews help understand why?). This resulted in a set of final themes that reflected both the interview and the FGD transcript data.

The final coding tree was similar to the initial deductive themes that are described above. Inductive sub-themes emerged under ‘feasibility’ relating to issues of data processing (e.g. note taking, transcription and translation) and the constraints of physical space; and for ‘participant reaction’ relating to distraction and emotional responses. Below we present our overarching findings on the exercises followed by exercise-specific findings.

The facilitators felt that the exercises yielded more in-depth and complete information than the ‘normal’ FGDs they were used to. This was echoed by the principal investigator (ZH) who found the FGDs more discursive, and less of a group interview, than previous FGDs in the study settings. Both facilitators and participants describe the methods as ‘ fun’ and ‘ enjoyable’ :

‘Some of the participants we selected for the in-depth interviews [with no “exercises”] came out very disappointed: they heard from friends that it was something fun with pictures and drawings, but in their own interview only “normal” questions were asked’ [FGD facilitator- Nigeria] .

Data richness, participant reaction and ease of facilitation varied by study site, exercise, and to a lesser extent participant group. In relation to study site, overall the methods worked better in Nigeria than in Ethiopia, as they were better understood. The facilitators felt that participants were more open and interacted with each other more than in Ethiopia. This was also true for the non-exercise elements of the FGDs.

In both sites there was evidence of social desirability bias, which varied by FGD. Some FGDs resulted in frank and illuminating discussions and others in guarded and short responses – unsurprisingly the exercises worked best in the more open FGDs. Facilitators put the variation down to group dynamics, for example, the presence of a domineering and judgemental participant in one FGD appeared to stop participants opening up, while in another FGD a very open participant shared a personal story which prompted others in the group to be more open. As well as group dynamics, the depth of the data was linked to the ability of the facilitator to probe effectively without turning the FGD into a group interview.

The number of exercises used in the FGDs was relatively high (3–4), and took longer than we had anticipated, particularly in Ethiopia. Some facilitators felt that this was too many, and caused exhaustion towards the end of the FGDs for both participants and facilitators. We had many examples of participants having to leave, asking when the FGD would end and being distracted.

Respondent 2: Ar e we not going? Facilitator: We are only left with two questions. Respondent 2: In Christ’s sake Respondent 3: It is getting late [Mother FGD- Ethiopia]

Using the same methods across respondent groups, allowed us to compare and contrast findings.

On a practical level the note-takers and transcribers experienced difficulties in capturing the discussion when the exercises yielded excitement or heated discussion as multiple participants often talked at the same time or in quick succession. Where translators were used valuable discussions that occurred while exercises were being conducted were lost.

Our findings for each method are presented below and are summarized in Table  2 .

Associative picture, ranking & sorting exercises

The pictures were often greeted by participants with enthusiasm, and sometimes an emotional response: ‘poor woman … . ’ or ‘it [picture] gives me joy’ ; and participants enjoyed the exercises:

Facilitator: Okay, thank you, we will revisit some of these discussions later, but let me gather these pictures. Respondents: All clap. Respondent 1: We are enjoying this, we like this discussion with the pictures [clapping continues] [Grandmother FGD Nigeria].

Despite this enthusiasm the exercises were not always easily understood, and facilitators sometimes explained the exercise multiple times, had to reorient discussions, or had to sort or rank the pictures themselves using multiple prompts ‘who is the next influential … … and then who’ . This added to the length of the FGDs. Although there were some comprehension problems in Nigeria: ‘Can you explain it to us again’, these were significantly less than in Ethiopia. This is reflected in a greater facilitator- participant interaction during the exercises in Ethiopia, compared to greater participant-participant interaction in Nigeria.

The facilitators themselves described the exercises as ‘hit and miss’ , particularly the associative picture exercise, but felt they were a useful starting off exercise:

‘ Excitement in this round did not always lead to useful discussions [ … ] and quality of answers, but gave an initial overview. This is a great exercise at the beginning, as it could serve as some sort of ice-breaker. Or it could come back at several points during the interview to re-energise the participants’ [Facilitator Nigeria].

Overall these exercises yielded useful information about behaviours that were practiced, beliefs about these behaviours, advice given about the behaviours and key influencers. The level of discussion was linked to whether the topic of the picture was of a new or unusual behaviour, or whether there were variations in opinions about the practice of the behaviour. For example, the first quote below is for wrapping the baby after delivery, this is a widely accepted behaviour with little variation in beliefs. The second quote relates to skin to skin care, a new practice, where discussions of the topic were livelier and in depth. A lack of discussion did not mean the exercise did not work well, as it told us something about the normality of the behaviour:

Facilitator: Let me show you another picture. It shows a baby being wrapped after delivery. [ … ] What do you think about this? Participant 1: It is good Participant 2: It is good

All respondents (mothers) upon seeing a skin-to-skin picture: It is not practised! [laughter] Respondent 1: Are they naked? Facilitator: Yes. Respondent 1: [laughs] Respondent 2: [looks surprised by the picture] Respondent 1: We don’t see such thing before … ...

It was essential that the facilitator explained the picture before starting the exercise. Otherwise discussions deviated into the details of the picture, such as whether the person was from their ethnic group, was the baby circumcised, or that the mother looked tired. Another problem was our initial use of too many, and too similar, sorting and ranking exercises. This led to repetition in content, which frustrated the participants:

‘Respondent 1: We have already discussed on it. You are asking repeatedly. Facilitator: It is not a repeated question. Earlier we have discussed on the most important and less important behaviours. Now I am asking you about what the Health workers promote and encourage you to do and do not’ [Mother FGD- Ethiopia].

Other issues that came up less frequently were participants believing that we were looking for the correct answer rather than an opinion; a lack of space making the sorting and ranking difficult to do; and poor eyesight in the older respondents meaning that pictures in the associative picture exercise needed to be passed around rather than held up, which added substantially to the time the FGD took.

The root cause analysis was described by all facilitators as useful, but challenging. Overall, they provided rich information on the problems they explored, especially in the FGDs where the participants were open and felt able to be critical. In these cases, participants were eager to share their opinions and knowledge:

‘Facilitator: What are the reasons why women do not go and deliver at the health facility as desired? Respondent 1: Here I am [raises hand eagerly] Respondent 2: Here I am too, I have much to say [raises hand]. Facilitator: “ Good! Let us get the main reasons’ [community health workers FGD- Nigeria].

The depth of data was also linked to whether the participants perceived that the chosen topic was a problem – probably linked to their openness. For example, in one FGD participants repeatedly said there was no problem and all was being done well, yet in a second FGD participants shared their personal experiences of the problem, which provided insightful and rich data.

Not all participants understood the symbolic meaning of the tree trunk and the roots, and from the transcripts it appears that the drawings were not always used. Facilitators reported that writing the reasons for the problem on the tree root helped to structure the participants’ thoughts and deepen out emerging issue. Writing down the participants’ reasons was difficult for illiterate participants, and the facilitators tried to overcome this by reading the labels out loud and summarizing them every now and then.

Facilitators found it challenging to process all information to make an adequate and informative root diagram on the spot, especially as participants often gave answers in an unstructured manner. The exercise was regarded as the most demanding for facilitators, requiring a high level of analytical skills. Furthermore, some facilitators suggested that the tool worked better with more educated participants, such as the community health workers, who themselves had better analytic skills.

Predictive storytelling

Participants identified with the character and the situation in the story, and answered questions as if the character was living in their village. Some facilitators opened the story as if telling a folk tale which participants liked. With the help of facilitator probes the scenarios generated discussions and uncovered insights such as how families manage conflicting advice, and the level of trust in community health workers: “We were able to get interesting information on mechanisms of change. There were arguments for both sides of the story and they explored several scenarios” [Facilitator- Ethiopia]. The method was relatively quick to do in relation to the quantity of data captured.

Facilitators felt that focusing on a character reduced social desirability bias. For example, participants said the character may behave at odds with expected behavior, but then stressed that they themselves would behave differently: ‘She will listen to her sister’s advice, but as for me it is better if she follows the health worker advice’ [Mother FGD Ethiopia]. In most FGDs participants switched between the third and first person when talking about the scenarios, with the first person used to present a strong belief about a behavior, or to reflect on what they would do differently to the character: ‘NO … NO … NO … … . I do not want to do it like this’ [Grandmother FGD Ethiopia].

In Ethiopia participants frequently reported what the character ‘should’ do, rather than on what they ‘ would’ do. Answers tended to be shorter and less complex than in Nigeria, where discussions were often lively, humorous and more open:

‘Respondent 3: Her mother has right over her … she will follow the mother’s advice, she will be afraid of God’s wrath Respondent 2: but some Ladis [character] will not behave like this, the Ladi of nowadays [laughter], the rude Ladis of nowadays, who will look into our eyeballs and tell us that the doctors have told them this and that [imitating Ladi and shaking her body – everyone laughs] Respondent 1: They will not keep quiet and listen’ [Grandmother FGD Nigeria]

The switching between the first and the third persona, and between should and would, was an entry point for facilitator probes.

Provocative statements

The use of statements was often poorly understood by participants, but the exercise did work once participants understood the task. The exercise elicited some useful information, for example it uncovered differences in perceptions of volunteer workers compared to paid community workers, recent changes in who influences behaviors and it allowed us to gauge the strength of people’s opinions. The method did not always yield immediate gut reactions, but there were some successes- particularly in Nigeria: ‘People literally jumped up to give their answers – I had the feeling they really told me their gut feeling, the first thing that came to mind’ [ Facilitator- Nigeria].

Provocative statements worked best where there was a clear understanding that this was a statement to discuss, and not a fact or the facilitators view. It was the method that participants asked the facilitator to explain again most often, and where participants sometimes remained in silence: ‘This topic is difficult for us, please clarify’ [Community health worker FGD, Nigeria]. This occurred despite facilitators conducting practice rounds with humorous examples. The lack of understanding was attributed to confusion between a statement and someone’s opinion: ‘They see an “authoritative” figure and confuse the statement for their opinion. Sometimes they even said, “Is that really true?” whereas it was meant to be provocative’ [Facilitator- Ethiopia]. Faced with silence or confusion the facilitators had to explain the exercise again or rephrase the statement as a ‘normal’ question: ‘On many occasions, we had to convert Q-statements to questions. I think it is more a cultural thing: people are just not used to being confronted by provocative questions. It is either you are asking them a question, or you are telling them something’ [Facilitator-Ethiopia] .

Short and focused provocative statements were most suitable in both sites as they were easier to understand. We encountered some problems with translation and with the use of translators. The provocative statements were originally written in English and translated in Amharic and Hausa. For most statements, this did not pose any difficulties, but some statements were difficult to translate. This resulted in long descriptive read outs that were not suitable for use. In these cases, we developed alternative statements written directly in the local language. Where we used translators during the FGD we trained them around the use of provocative statements, but we still encountered problems: ‘The translator had difficulties in interpreting the statements properly, resulting in very long statements, that did not provoke any immediate response’ [Facilitator- Ethiopia].

The provocative statements were done at the end of the FGDs, and the depth and usefulness of responses was linked to how lively the participants were at the point, and whether they perceived that they had provided similar answers previously.

There was agreement across fieldworkers, supervisors and senior investigators that the exercises improved interaction between participants, enhanced data quality, and made the FGDs more enjoyable compared to previous experiences in the settings. We did not encounter a limitation identified in the literature of participants not liking the idea of playing ‘games’ or thinking the researchers were strange [ 5 , 6 ].

The methods were more successful in Nigeria than Ethiopia: this may be related to cultural differences in conversational norms and issues of positionality between facilitator and participants, exemplified by an Ethiopian facilitator describing that participants saw them as an ‘ authoritative figure’ . The potential for high levels of social desirability bias has been noted for Ethiopia given the political and cultural context [ 20 ], and although we found evidence that social desirability bias remained we found that some techniques reduced it, with the story telling approach for example allowing a projection of responses onto an ‘other’. In Ethiopia in-depth ethnographic methods could be a more useful tool to overcoming such bias than FGDs.

A strong influence of positionality on participants was also found in Tanzania, where exercises reduced its impact as participants focused less on the facilitator, with the best FGDs occurring when the facilitator removed themselves from the process altogether [ 7 ]. Given the importance of probes for generating rich data in our study, and in other studies in similar settings [ 21 ], we would not recommend the removal of the facilitator, but our findings highlight the importance of the selection and training of data collectors, and in-depth thinking about how they present themselves to the participants. We found that key to success of the exercises was, whether the facilitators had fully understood the aims and objectives of the discussions, mastered the content and process of the exercises and knew how to probe effectively. Some of the Nigerian FGDs were facilitated by one of the senior researchers (YH), and the impact on data quality of their greater experience in encouraging interaction between participants and their gentle but effective probing was clearly visible.

The predictive story telling was one of the most successful methods it was well understood, enjoyed by participants and facilitator, generated interaction and its projective nature seems to have gone some way to address social desirability bias. We found one study that reported on the use of stories in a low-income setting, albeit in in-depth interviews [ 21 ]. They also concluded that the method had been successful and increased the quality of their data, but faced the same challenge of respondents often reporting what should be done rather than what the character in the story would have done. We discovered that in some languages the differences between should and would were also difficult to translate. We think that one of the reasons the story was successful was because it was carefully tailored to the cultural context and resonated with the participants.

The provocative statements worked the least well with comprehension issues in both sites- despite facilitators using humorous examples. This meant that facilitators spent time explaining the exercise, lengthening the FGDs, and in Ethiopia facilitators often had to rephrase the statements as questions. We also found issues of translation and of using translators particularly problematic for this method, with statements written directly in the local language working best. Language issues are a common problem in FGDs in low income settings with the use of translators having a negative impact on group interaction and discussion, and where at all possible they should not be used [ 8 ].

We felt that the number of methods included in the FGDs and repetition of content hindered their effectiveness by making the FGDs too long and at times frustrating for the participant. Although the exercises were described as fun, towards the end of the FGDs participants were often anxious to leave. In Ethiopia the need to explain the exercises multiple times and the use of translators meant the FGDs were longer than we would have hoped. There is no recommendation on the number of exercises that should be included in FGDs, but the numbers should be influenced by the impact they have on the length of the FGDs, participants’ age and ability to express themselves [ 6 ].

This is – to our knowledge – the first study reporting on experience with a range of activity-oriented exercises in low-income settings. The study was however subject to some limitations. First, data collection focused on maternal and child care behaviors and the findings may not be transferable to studies accessing other types of behaviors. Second, participants were not asked directly about their opinion and experiences of participating in the FGDs, rather conclusions were drawn from transcripts. Third, the time and scope of the research did not allow for exploration of other methods and exercises beyond the six evaluated in this study: there are several other promising methods that should be explored for use in low-income settings. Finally, interviews with facilitators occurred by phone and skype, after completion of the FGD work in both countries and were conducted by a researcher who had supervised data collection rather than someone external to the study team.

Conclusions

The majority of our exercises– and in particular story-telling - proved successful tools in yielding rich and less biased information from FGDs. Participants and facilitators experienced the exercises as fun: they improved participation and engagement. However, the exercises adversely affected the length of the FGDs and in future we would use fewer exercises. Given the differences we noted between the two study sites, we recommend that all exercises are adequately piloted and adapted, or not used at all if they do not improve quality.

Availability of data and materials

The datasets generated during and/or analyzed during the current study are not publicly available due to issues of confidentiality and privacy. Although respondents’ names are not included in transcripts they do include place names and any names mentioned by respondents, for example, names of Community Health Workers. Restricted and summarized data are available from the corresponding author on reasonable request.

Abbreviations

Focus Group Discussion

Southern Nations, Nationalities and Peoples

Health Extension Worker

Health Development Army

Local Government Area

Federation of Muslim Women’s Associations in Nigeria Volunteer TBA

Traditional Birth Attendant

Community Health Worker

Willis K, Green J, Daly J, Williamson L, Bandyopadhyay M. Perils and possibilities: achieving best evidence from focus groups in public health research. Aust N Z J Public Health. 2009;33(2):131–6.

Article   Google Scholar  

Reed J, Payton VR. Focus groups: issues of analysis and interpretation. J Adv Nurs. 1997;26(4):765–71.

Article   CAS   Google Scholar  

Freeman T. ‘Best practice’in focus group research: making sense of different views. J Adv Nurs. 2006;56(5):491–7.

Kitzinger J. Qualitative research. Introducing focus groups. BMJ. 1995;311(7000):299.

Kitzinger J. The methodology of focus groups: the importance of interaction between research participants. Socio Health Illness. 1994;16(1):103–21.

Colucci E. “Focus groups can be fun”: the use of activity-oriented questions in focus group discussions. Qual Health Res. 2007;17(10):1422–33.

Jakobsen H. Focus groups and methodological rigour outside the minority world: making the method work to its strengths in Tanzania. Qual Res. 2012;12(2):111–30.

Hennink M. Emergent issues in International Focus Group Discussions. In: Hesse-Biber SN, Leavy P, editors. Handbook of Emergent Methods. New York: The Guilford Press; 2010. p. 189–205.

Google Scholar  

Morgan D, Fellow C, Guevara H. Emergent Approaches to Focus Group Research. In: Hesse-Biber SN, Leavy P, editors. Handbook of Emergent Methods. New York: The Guilford Press; 2010. p. 189–205.

Denzin NK, Lincoln YS. The landscape of qualitative research: theory and issues. London: Sage; 1998.

Polkinghorne DE. Language and meaning: data collection in qualitative research. J Couns Psychol. 2005;52(2):137.

Colucci E. On the use of focus groups in cross-cultural research. Cross-Cult Res. 2008:233–52.

Schilder K, Tomov T, Mladenova M, Mayeya J, Jenkins R, Gulbinat W, Manderscheid R, Baingana F, Whiteford H, Khandelval S. The appropriateness and use of focus group methodology across international mental health communities. Int Rev Psychiatr. 2004;16(1–2):24–30.

Krueger RA. Quality control in focus group research. Success Focus Groups. 1993:65–85.

Böhm G, Pfister H-R. How people explain their own and others’ behavior: a theory of lay causal explanations. Front Psychol. 2015;6(139):1–15.

Malle BF. The actor-observer asymmetry in attribution: a (surprising) meta-analysis. Psychol Bull. 2006;132(6):895.

Hill Z, Amare Y, Scheelbeek P, Schellenberg J. ‘ People have started to deliver in the facility these days’: a qualitative exploration of factors affecting facility delivery in Ethiopia. BMJ Open. 2019;9(6):e025516.

Hill Z, Scheelbeek P, Hamza Y, Amare Y, Schellenberg J. Are we using the right approach to change newborn care practices in the community? Qualitative evidence from Ethiopia and northern Nigeria. Glob Health. 2020;8(3):383–95.

Hill Z, Scheelbeek P, Schellenberg J, Hamza Y. “Everything is from god but it is always better to get to the hospital on time”: a qualitative study with community members to identify factors that influence facility delivery in Gombe state, Nigeria. Glob Health Action. 2020;13(1):1785735.

Ostebo MT, Cogburn MD, Mandani AS. The silencing of political context in health research in Ethiopia: why it should be a concern. Health Policy Plan. 2018;33(2):258–70.

Gourlay A, Mshana G, Birdthistle I, Bulugu G, Zaba B, Urassa M. Using vignettes in qualitative research to explore barriers and facilitating factors to the uptake of prevention of mother-to-child transmission services in rural Tanzania: a critical analysis. BMC Med Res Methodol. 2014;14(1):21.

Download references

Acknowledgements

Not applicable.

The study was funded by the Bill and Melinda Gates foundation as part of the IDEAS study. Grant number OPP1017031. The funders had no role in the study design, data collection, analysis, interpretation or write up of the manuscript.

Author information

Authors and affiliations.

Department of Population Health, London School of Hygiene & Tropical Medicine, London, UK

Pauline F. D. Scheelbeek

Childcare & Wellness Clinics, Abuja, Nigeria

Yashua A. Hamza

Department of Disease Control, London School of Hygiene & Tropical Medicine, London, UK

Joanna Schellenberg

University College London, Institute for Global Health, London, UK

You can also search for this author in PubMed   Google Scholar

Contributions

ZH, PS & YH developed the research concept, PS, ZH & YH developed the research methodology, PS & YH collected the data, PS & ZH analyzed and interpreted the data, PS & ZH wrote the manuscript, JS & YH commented on drafts of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Pauline F. D. Scheelbeek .

Ethics declarations

Ethics approval and consent to participate.

Study protocols were reviewed and approved by the London School of Hygiene & Tropical Medicine Ethics Committee, the Ministry of Science and Technology in Ethiopia, the National Research Ethics Committee in Nigeria and the Gombe State Government in Nigeria. Written consent was obtained from all participants; illiterate participants were asked to appoint a proxy to sign on their behalf. A copy of the consent forms can be found in the supplementary files.

All data were anonymized for publication.

Consent for publication

Competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1., additional file 2., additional file 3., additional file 4., additional file 5., additional file 6., additional file 7., additional file 8., additional file 9., additional file 10., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Scheelbeek, P.F.D., Hamza, Y.A., Schellenberg, J. et al. Improving the use of focus group discussions in low income settings. BMC Med Res Methodol 20 , 287 (2020). https://doi.org/10.1186/s12874-020-01168-8

Download citation

Received : 12 May 2020

Accepted : 19 November 2020

Published : 30 November 2020

DOI : https://doi.org/10.1186/s12874-020-01168-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Focus group discussions
• Qualitative data collection
• Low-income settings
• Activity-oriented exercises

BMC Medical Research Methodology

ISSN: 1471-2288

• General enquiries: [email protected]

The key success factors in focus group discussions with the elderly for novice researchers: a review

Journal of Health Research

ISSN : 2586-940X

Article publication date: 6 March 2020

Issue publication date: 7 August 2020

This study aimed to discuss the key success factors in focus group discussions among the elderly as a means of supporting novice researchers and also to share the experiences of novice researchers with focus group discussions.

Design/methodology/approach

This study used a literature review of previous studies that revealed three themes regarding the key success of focus group discussions with the elderly. Focusing on issues published between 2009 and 2019, four health-related databases, namely the Cumulative Index of Nursing and Allied Health Literature (CINAHL), ScienceDirect, PsycARTICLES and Thaijo were investigated using keyword searches, both individually and in combination, and the inclusion criteria used in selecting relevant articles as primary sources included research written in the English and Thai languages.

The literature review involved eight published articles related to this topic in the nursing field. Three themes for key successes were identified as follows: good planning and convenient organization, being accepting and flexible concerning their opinions and good management.

Originality/value

Focus group discussions with the elderly are complex and challenging for novice researchers. It is necessary that a novice researcher in the nursing field builds up the skills of a moderator in conducting focus group interviews if he or she wishes to obtain rigorous data.

• Focus group discussion
• Novice researcher

Muijeen, K. , Kongvattananon, P. and Somprasert, C. (2020), "The key success factors in focus group discussions with the elderly for novice researchers: a review", Journal of Health Research , Vol. 34 No. 4, pp. 359-371. https://doi.org/10.1108/JHR-05-2019-0114

Emerald Publishing Limited

Copyright © 2019, Kasorn Muijeen, Puangpaka Kongvattananon and Chomchuen Somprasert

Published in Journal of Health Research. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) license. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this license may be seen at http://creativecommons.org/licences/by/4.0/legalcode

Introduction

Qualitative research on the elderly population requires understanding on the part of the researcher perhaps more than that required for other age groups because these members of the population have had a great deal of life experience, and some are very knowledgeable. Some, however, have not been very successful in life. This range makes focus group discussions a challenge. Collecting the qualitative research data for the present study focused on methods that were consistent with the natural way of life of people in society. In Thai society, especially in Thai villages, people often talk in groups at leisure or while carrying out activities where they can carry on a conversation at the same time. Therefore, group discussion is a normal social behavior in communities or societies that are familiar with each other. Qualitative researchers sometimes use this activity as a data collection method [ 1 ]. Focus group discussion in qualitative research refers to a data collection method that focuses on discussing and exchanging opinions and experiences with the participants. Moreover, with this method, the researcher can observe the participants' interactions with each other and talk to them, thus covering the various issues that have emerged during the conversation and using them for further analysis [ 2 ].

Research on the elderly in Thailand has generally focused on illness and disability. However, for the past decade, this trend has been expanded and now, “successful or healthy aging” or “healthy elderly” are issues that are discussed [ 3 ]. The increasing number of elderly people in the world has resulted in the need to study more deeply the factors that influence the health of older people [ 4 ] and perhaps especially understand why some older people seem to adapt to the challenges of the aging process whilst others cannot despite interventions and policies targeted at them. Therefore, qualitative research on the elderly is often focused on certain aspects, such as health awareness or the health literacy of the elderly [ 5 ]. Focus group discussions with the elderly, on the other hand, are a popular method of collecting data for qualitative research purposes. However, focus groups represent a challenge for new researchers in terms of collecting qualitative data. Novice researchers are typically concerned about the process of focus groups with the elderly and need to understand the characteristics of this special group. The main responsibility of the moderator in this context is to create discussions that offer useful responses got from the participants [ 6 ]. Moreover, the novice researcher needs to be aware of these factors because the focus group discussion provides a quick approach to research which may require considerable time for analysis. Careful planning regarding the moderation of focus group discussions is the basis for successful qualitative research [ 7 ] in this area. When using the discussion group method, preparation has many aspects, and effort must be made so that the time and effort will not be wasted [ 8 ].

The processes of focus group discussions with the elderly are challenging for new researchers who need previous experience on conducting such discussions or previous experience with being a moderator, all of which are a part of the data collection method in qualitative research. Addressing issues such as good planning and organizing the participants and facilities, as well as training, to understand the characteristics of being a good moderator, as well as having an accepting attitude and being flexible are key to the success of group discussions for novice researchers. In addition, several studies have shown that success in conducting focus groups mainly derives from the characteristics of an effective facilitator or moderator. In addition, training and personal readiness are essential in order to enhance the moderator's skills [ 9 , 10 ].

This study has two main points of interest. The first is to understand the factors required in order to conduct focus group discussions amongst the elderly for novice researchers. The second is to describe the researcher's experience with others by sharing the experience of the following published project: “The Effects of Using a Knowledge-management Development Program on Knowledge Management Behaviors for the Health Promotion of Elderly Leaders in the Romyen Community in Thailand” [ 11 ]. The study's objective provides a discussion of the key success factors of focus group discussions with the elderly for novice researchers.

Selection criteria and search strategy

A systemic review was used on articles published between 2009 and 2019, using four health-related databases, namely the Cumulative Index of Nursing and Allied Health Literature (CINAHL), ScienceDirect, PsycARTICLES and Thaijo, which were investigated using keyword searches.

Systemic and critical reviews used the guiding framework based on the PRISMA 2009 flow diagram [ 12 ], which is composed of four steps as follows: identification, screening, eligibility, and included. From the database, 377 studies published from January 2009 to March 2019 were identified that could be adapted as a PRISMA 2009 flow diagram for the article selection process, as shown in Figure 1 . The process of searching began with step 1 as follows: identification using the keywords used within the database search, both individually and in combination, including the terms “key success,” “moderator,” “facilitator,” “conduct,” “focus group discussion with the elderly,” “focus group discussion with older adults,” “novice researchers” and “experience novice researchers.” Step 2 included the screening of articles from the inclusion criteria used in selecting relevant articles as primary sources. This was research written in the English and Thai languages, while the exclusion criteria were articles written in other languages and that were not related to the novice or new researcher as a moderator in focus group discussions with the elderly or focus group discussions with older individuals. Step 3 concerned eligibility, where the full text of the article was examined by checking the content or details of the investigation, as depicted in Figure 1 , for a total of 25 full articles repossessed and checked for eligibility. Step 4 included twenty-five eligible articles to be considered; however, sixteen articles did not have a moderator in the focus group. Finally, the nine remaining articles were qualitative and quantitative studies synthesized by the author, as shown in Figure 1 .

Study selection and data extraction

The data extracted from the studies were tabulated independently by the authors, with data including the study design, purpose and key content or findings of the studies. The findings or the key content were related to the factors connected with focus group discussions, such as the skill of the moderator and how to conduct a focus group discussion.

Ethical consideration

This study did not involve human participants but rather used a literature review of previous studies.

The main results were derived from information gathered from nine articles related to the key success factors regarding focus group discussions among the elderly with novice researchers. These nine articles were divided into two parts as follows: the first part concerned the research and literature related to focus group discussions, as can be seen in the seven articles presented in Table I ; and the second part concerned research that used focus group discussion strategies with the elderly, as shown in Table II .

The main messages from each paper were presented after a thorough analysis, as seen in Tables I and II . According to the information in Table I , the study [ 13 , 15 ] presented the steps of a focus group, including planning and good communication skills for conducting focus group discussions consistent with Doody [ 9 ], who mentioned the environment and duration of the focus group as being important factors. An effective moderator should be a good listener and possess good communication skills. Moreover, Wilson [ 16 ] described planning and the role of the moderator as being important as well. As indicated, the size of the focus group depends on the complexity of the topic and the skill of the moderator [ 14 ]. A skillful moderator needs to know the topic in depth or should be an expert in the subject of discussion [ 17 ]. Technology can also affect the focus group [ 18 ]. Table II [ 19 , 20 ] supports the characteristics of the elderly participant group and how to create a successful focus group discussion. According to the data, the relevant information concerning how to successfully complete a focus group discussion with the elderly can be summarized according to three key success factors as follows: 1) Good planning and event organization, 2) Be accepting and flexible concerning their opinions and 3) Have good management skills.

Good planning and event organization

The stage of good planning and event organization includes preparation, the presession and the session itself. These are important points for novice researchers in terms of conducting a successful focus group discussion. Good planning and event organization should cover the researcher's team, the participants, the environment and question creation in line with the objectives of the focus group. The participants should be contacted in advance and be made aware of the objectives of the activity. It is important to select suitable persons for the focus group. The researcher has to give careful attention to time and provide enough time to prepare. This covers the duration of the focus group, the selection of the participants, the size of the group, preparing the environment, conducting the focus group, supporting the participants during the interview and the participants’ role as a moderator. Wilson [ 16 ] recommends that the moderator reviews the literature on the topic, prepares to take field notes and prepares his or her role in advance. Questions should also be prepared in advance with the research team, considering the purpose of the study, and especially for the elderly, the questions should be relatively easy. In addition, in order to collect comparable data from all of the focus group discussion participants, a list of broad questions or predetermined probes can be developed as a guide for the moderators in order to facilitate the discussion among the participants. Some of the questions in a focus group discussion from Reichstadt's study [ 21 ] are, for example, “How would you define successful aging?” and “What are the necessary components of successful aging?” These questions can guide the moderator in conducting the group and will help him or her explore the research problem. In each focus group, the moderators can begin with these broad questions, with further questions asked based on the participants' responses, in order to elicit additional opinions from the group. This ensures that the participants are provided with an opportunity to discuss their personal attitudes and beliefs [ 1 ]. For the conclusion stage, the moderator should choose approximately five factors related to how to be successful in the focus group discussion, including the size of the focus group, the timing, the environment and location, the selection of participants and understanding the participants and focus group scripts and materials.

Size of the focus group

Variations exist in the literature regarding the optimum size of a focus group, depending on the complexity of the topic, the type of participants and the skills of the moderator. Most researchers suggest that the appropriate group size ranges from 4 to 12 participants [ 1 , 2 , 6 ]. A large group can generate more opinions and/or feelings about the topic, while a smaller group may be more beneficial when sensitive issues are concerned. Moreover, the number of participants in the focus group can depend on the participants having the same experience because some experiences take a long time to discuss and, for this reason, the group should be a small group [ 22 ]. Therefore, the size of a focus group can vary according to the research objective, can be flexible and should be large enough to create a group discussion. For example, most focus group discussions with the elderly in Thailand have 6–12 participants [ 19 – 21 ].

Time of the focus group

The period of time for conducting the focus group depends on the complexity of the topic, the number of questions asked and the size of the group. The duration of the focus group discussion should be in the range of 1 to 2 hrs and should not be longer than 2 hrs because this can affect the physical and mental health of the participants, especially among the elderly, who often have problems with their physical health and are tired from the effects of old age [ 21 ]. Barbour [ 2 ] recommends that the time for discussion be limited to 45 to 90 mins because outside this time period, the participants may lose their ideas and thoughts about the topic, or the session time would have been reached. However, a period of less than 1 hr may prevent the discussion of important issues in accordance with the research objectives. Therefore, it is recommended to use a flexible time period of one hr to no more than 2 hrs.

The environment and location

The environment and location are important as they can have a significant impact on the discussion of a focus group. The following have been suggested concerning the environment and location of a focus group discussion among the elderly: (1) the room should be of adequate size, lighting and temperature; (2) the seating should be arranged in a circle format to ensure that all the participants can see one another, and the ergonomic characteristics of the elderly should be considered; for example, chairs should not be too high or too low because most older people have problems with their knees; (3) the moderator should create a relaxing atmosphere, comfortable and nonthreatening, and the participants should not feel that the questions are too abstract, for example using academic language and (4) the location or setting should be a safe environment where disturbances from other people or external noise [ 14 , 15 ] are not present.

Selection of participants and understanding the participants

The groups can be homogeneous or heterogeneous, depending upon the purpose of the research; yet, attendees should have previous experience or prior knowledge of the topic. Homogeneous groups have similar characteristics, such as being from the same background, being close to the same age, of the same gender, social status or occupation and having experience with the topic as these factors may influence the group discussion. However, heterogeneous groups could be used for exploratory purposes where varieties of groups are preferred. Moreover, it is important to make sure that the focus group participants are not in conflict with one another or are incompatible. Pawi [ 17 ], for example, discussed the differences in the research outcomes obtained from homogeneous and heterogeneous groups, as well as the differences between groups of individuals that were familiar with each other or intimate and those that were strangers. Therefore, novice researchers need to understand all of the steps of the focus group discussion. The novice researcher should carefully select the participants in the group because this will help him or her achieve the research objectives and make it easier to control the focus group discussion. In the Thai context, the novice researcher can use respectful pronouns instead of calling people by their proper names, such as uncle and aunt, when they conduct a group. In addition, the novice researcher must try to find out which of the elderly participants have hearing problems, and while he or she is moderating a focus group discussion his or her speaking volume must not be too loud or too low, and questions may have to be repeated [ 19 , 20 ].

Focus group scripts and materials

Nagle and Williams [ 22 ] stated that the lack of important equipment or materials could interrupt or even stop the discussion from flowing smoothly, causing participants to feel uncomfortable and limiting the discussion. The moderator could prepare a script as an outline for the focus group discussion. The scripts for introducing the group discussion can include, for example, the following: (1) welcoming statements, including the process of self-introduction and group member introductions, with members providing general stories about themselves; (2) overviewing the topic; they can begin by explaining the purpose of the study and of the focus group; (3) stating the ground rules, reviewing the group discussion rules and other information that the participants need to know prior to the group interview and (4) beginning with the first question. It is also important that the questions in the scripts move from the general to the more specific or in order of importance. The research team should brainstorm in order to develop a list of questions based on the purposes of the research. These should be open-ended rather than yes–no questions, to generate more useful information from the group discussion.

O'Connor and Murphy [ 15 ] suggested that creating focus groups for the elderly has nine components that help to complete the group, which can be summarized as follows: (1) audiotapes or audio cassettes used to record the data; (2) participant information form and consent form; (3) labels for the participants and moderators; (4) a watch (5) coffee and bakery goods or drinks; (6) a sign-in consent form and a document listing the participants; (7) flip charts and markers and (8) notebooks, pens and audio cassettes that can be used for recording the data. Digital voice recorders are important because recording focus group discussions is not easy. Focus groups are notoriously difficult to manage and control, especially amongst the elderly, who may speak softly, which makes for poor recording [ 18 ].

Being accepting and flexible concerning their opinions

By recognizing the importance of the moderator role, the novice researcher should gain insight into the requirements of this role in order to promote his or her effectiveness. One's life and work experience may provide a level of preparation, but the development of additional skills can be carried out with the following: theoretical knowledge practice using open-ended questions and rewriting; being familiar with many group conversations; conducting a pilot study; talking to researchers that have had experience in using group discussion methods and participation in focus group discussions for the purpose of critical analysis. A novice researcher should understand that teamwork is important, and various techniques can be used to help solve the problems that may arise during the decision-making process because it is difficult for only one person to manage and conduct a complete focus group. Conducting a focus group is difficult for a novice researcher, especially regarding the data collection and the work process of the focus group. Accepting comments and being flexible are the challenges for moderators to improve their skills, including the following; good interpersonal skills, being good listeners and being non-judgmental and adaptable.

Good interpersonal skills

The interpersonal skills of the moderator are crucial to success in conducting a focus group. Shaha et al. [ 13 ] agree that interpersonal and group leadership skills are essential. At the beginning of the group interview, a moderator welcomes all participants and introduces himself or herself and the team. He or she then requests the participants to introduce themselves and to put on a name label. Then, he or she identifies the purpose of the interview, asks an introductory question as a warm-up to create a relaxing atmosphere and thanks them afterward for sharing their experience. Moreover, Krueger and Casey suggest that light refreshments and eating together tend to promote conversation, discussion and communication within the focus group. Nagle and Williams [ 22 ] suggest that the communication skills of the moderator also involve how to deal with interruptions when someone is providing an opinion. In this regard, the moderator should provide some signals for them to wait, explaining that they will be given an opportunity to speak next. In Thailand, most focus group discussions with the elderly are significant because of research carried out by the moderator and his or her relationship with the group [ 20 ]. If the relationship between the group participants and the moderator is good, insightful information can be received and the research questions can be answered. On the other hand, if the relationship is not good at the outset, the group operation will have obstacles.

Being good listeners

A moderator must be a good listener and a thinker at the same time. Curtis and Redmond [ 23 ] stated that a moderator is required to concentrate on the discussion and to observe the level of the participants' interaction during a group discussion. Packer–Muti [ 10 ] suggested that a moderator should listen attentively with sensitivity and empathy, and Mack et al. [ 24 ] pointed out the importance of reading the body language of the participants, including crying, which can be related to the content of the discussion, in which case the moderator must not join in the emotion or outwardly express sympathy. In addition, the moderators must listen intently while engaging with the other participants, without expressing emotions themselves. If a problem occurs in a dynamic group, the group of responders will respond accordingly, with the moderator warning the participants about the rules of participation in the focus group. However, the important thing to consider is that the participants are elderly and when they speak the moderator must actively listen.

Be non-judgmental and adaptable

The moderator must accept that the participants may be inclined to divert from the expected answers or thought patterns, in response to questions. Mack et al. [ 24 ] mentioned that a moderator should believe that all participants have ideas on the topic and that the moderator should be impartial with his or her general opinions. Nagle and Williams [ 22 ] recommend that a moderator avoid head nods or responses such as “yes,” “I agree” and “OK.” These responses convey acceptance of the answers, which can limit the findings and/or any new ideas or opinions that may come later from other participants in a group. A good moderator needs to have an adaptable and non-judgmental attitude with no partiality or bias toward any of the group participants.

Good management of group dynamics

During a focus group, the researcher recognizes the potential tension between participants with certain characteristics or attitudes. Unfortunately, occasionally, group dynamics cannot be identified at the beginning of a session. The moderator plays the important role of the manager in creating the appropriate group process in which all the participants can participate in the discussion and exchange ideas. However, some group dynamics cannot be predicted. There have been many suggestions for moderators concerning how to deal with disruptive group dynamics, such as countering dominating or disruptive group members and concerning shy or silent participants. However, the researcher's experience will help to resolve problems with disruptive members or barriers in the group dynamics. This study provides details of the experience of novice researchers in the implementation of focus group discussions for the elderly, as detailed below.

The experience of novice researchers in focus groups for the elderly

This section of the present paper explains theresearcher's experience with other researchers by sharing her direct experience from the project mentioned above. The feeling of the author during the first focus group amongst the elderly was one of excitement. The author strongly agrees that good planning and convenient event organization, being accepting and flexible and practicing good management of the group's dynamics are important factors to make the focus group a success for new researchers. However, in conducting that focus group, where the elderly expressed all of their opinions, the data were still incomplete in relation to the objectives, which will be discussed in greater detail in the next section.

As Thai society is developing into an aging population, the elderly population is an important group. Therefore, there has been a great deal of education about elderly groups, and it can be said that qualitative research has allowed researchers to find answers concerning the elderly in profound ways. Based on the literature review and the shared experiences of the author in conducting focus groups with the elderly, it can be concluded that the keys to success include all of the three steps discussed above because they represent the sequence of events that will occur in focus group discussions.

Good planning and convenient event organization represent the preparation process, and these are important for a successful group discussion. The experience of the author in the research project mentioned above can be described as follows: the size of the group from this research comprised five elderly participants in a homogenous group of females who had the same experience with knowledge management training projects. It was concluded that the research team should meet and prepare itself before beginning the focus group, including reading about and understanding the research objectives and sharing the duties for preparation in various steps [ 14 , 20 ].

The skill of being a good listener and being non-judgmental needs to be practiced, and more experience in conducting focus groups is required [ 20 , 21 ]. The experience of this author could provide an example of the skills of the moderator and suggests that creating a focus group with the elderly also requires building relationships rather than just using friendly questions. Moreover, being a good listener and being non-judgmental and adaptable are important examples from the author's experience, and the moderator would be advised to be a quick decision-maker, especially when the participants talk too much. The moderator should make conclusions and lead to the next topic, and he/she should invite other participants to share their opinions. An example can be seen in the words of a participant: “You must try to eat lemon when you wake up the morning; it will be good for the digestive system.” The moderator concluded the conversation as follows: “You think that lemons are good for the digestive system. The other participants agree or disagree? Why?” Good management of group dynamics is consistent with the role of the moderator [ 16 , 19 ].

In addition, the popularity of the Internet and technology including online or Facebook can facilitate the process of focus group discussions. For example, the researcher can use online methods or Facebook to make an appointment with the participants [ 25 ]. Some studies [ 18 , 26 ] have mentioned the benefits of new approaches, such as the use of online focus groups that give individuals that are difficult to reach the ability to participate and that give researchers access to a greater variety of participants. However, there are still limitations in using technology for the elderly in terms of preparing focus groups [ 27 ]. The novice researcher should be careful about using technology with elderly sample groups because they are often limited in their ability in using the Internet and technology.

Focus group discussions with the elderly are complex and challenging for novice researchers, and a variety of skills are required when conducting a focus group. Therefore, the novice qualitative researcher needs to have confidence in the skills discussed in this paper so that he or she can obtain rigorous data. A novice researcher should also understand the strengths and limitations of a focus group discussion with the elderly, as this can have an impact on how to lead the group in order to generate quality data.

PRISMA 2009 flow diagram adapted to examine “key success of focus group discussion with the elderly for novice researchers” (adapted from Moher, Liberati, Tetzlaffand Altman [ 12 ])

Research or literature related to focus group discussions

Research on focus group discussions with the elderly

1 Stalmeijer RE , McNaughton N , Van Mook WN . Using focus groups in medical education research: AMEEguide no. 91 . Med Teacher . 2014 Nov ; 36 ( 11 ): 923 - 939 , doi: 10.3109/0142159x.2014.917165 .

2 Barbour RS . Focus groups . In: Bourgeault I , Dingwall R , de Vries R , editors. The SAGE handbook of qualitative methods in health research . Los Angeles : Sage ; 2010 . 327 - 352 .

3 Kinsella KG , Velkoff VA . An aging world: 2001 , Washington, DC : US Government Printing Office ; 2001 .

4 World Health Organization [WHO] . Ageing and health . [updated 2018 Feb 5; cited 2019 April 1]. Available from: https://www.who.int/news-room/fact-sheets/detail/ageing-and-health .

5 Dilshad RM , Latif MI . Focus group interview as a tool for qualitative research: an analysis . Pakistan J Soc Sci . 2013 ; 33 ( 1 ): 191 - 198 .

6 Webb C , Doman M . Conducting focus groups: experiences from nursing research . Junctures J Themat Dialogue . 2008 ; 10 : 51 - 60 . [cited 2019 April 1]. Available from: https://junctures.org/index.php/junctures/article/view/49/394 .

7 Kidd PS , Parshall MB . Getting the focus and the group: enhancing analytical rigor in focus group research . Qual Health Res . 2000 May ; 10 ( 3 ): 293 - 308 , doi: 10.1177/104973200129118453 .

8 Jamieson L , Williams LM . Focus group methodology: explanatory notes for the novice nurse researcher . Contemp Nurs . 2003 Jun ; 14 ( 3 ): 271 - 280 , doi: 10.5172/conu.14.3.271 .

9 Doody O , Slevin E , Taggart L . Preparing for and conducting focus groups in nursing research: part 2 . Br J Nurs . 2013 Feb ; 22 ( 3 ): 170 - 173 , doi: 10.12968/bjon.2013.22.3.170 .

10 Packer-Muti B. Conducting a focus group . Qual Rep . 2010 ; 15 ( 4 ): 1023 - 1026 .

11 Othaganont P , Kongvattananon P . The effects of using a knowledge management development program on knowledge management behaviors for the health promotion of elderly leaders in the Romyen community: research project . Pathumthani : Faculty of Nursing, Thammasat University ; 2018 .

12 Moher D , Liberati A , Tetzlaff J , Altman DG . Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement . PLoS Med . 2009 Jul ; 6 ( 7 ): e1000097 . doi: 10.1371/journal.pmed.1000097 .

13 Shaha M , Wenzel J , Hill EE . Planning and conducting focus group research with nurses . Nurse Res . 2011 ; 18 ( 2 ): 77 - 87 , doi: 10.7748/nr2011.01.18.2.77.c8286 .

14 Jayasekara RS . Focus groups in nursing research: methodological perspectives . Nurs.Outlook . 2012 Nov-Dec ; 60 ( 6 ): 411 - 416 , doi: 10.1016/j.outlook.2012.02.001 .

15 O'Connor C , Murphy S . Novice researchers' reflection on conducting a focus group . J PerioperPract . 2009 May ; 19 ( 5 ): 143 - 147 , doi: 10.1177/175045890901900504 .

16 Wilson R , Slaughter SE , Forbes D , Hanson HM , Khadaroo RG . Experiences of a novice researcher conducting focus group interviews . International Journal of Nursing Student Scholarship . 2016 ; 3 : 1-11 .

17 Pawi Putit Z , Buncuan J . Challenges in conducting focus group: moderators' experiences in Nursing . Journal of Malaysia Nurses Association . 2010 ; 5 : 2 - 9 .

18 Stancanelli J . Conducting an online focus group . Qual Rep . 2010 ; 15 ( 3 ): 761 - 765 .

19 Gray RS , Pattaravanich U , Chamchan C , Prasartkul P . Perceived negative and positive impacts of redefining older persons in Thailand . Journal of Population and Social Studies . 2015 ; 23 ( 1 ): 73 - 85 .

20 Sutipan P , Intarakamhang U . Healthy lifestyle behavioral needs among the elderly with hypertension in Chiang Mai, Thailand , International Journal of Behavioral Science . 2017 ; 12 ( 1 ): 1 - 2 .

21 Reichstadt J , Depp CA , Palinkas LA , Folsom DP , Jeste DV . Building blocks of successful aging: a focus group study of older adults' perceived contributors to successful aging . Am J Geriatr Psychiatry , 2007 Mar ; 15 ( 3 ): 194 - 201 , doi: 10.1097/JGP.0b013e318030255f .

22 Nagle B , Williams N . Methodology brief: introduction to the focus group . Center for Assessment, Planning and Accountability ; 2013 . [cited 2019 April 2]. Available from: http://www.mmgconnect.com/projects/userfiles/file/focusgroupbrief.pdf .

23 Curtis E , Redmond R . Focus groups in nursing research , Nurse Res . 2007 ; 14 ( 2 ): 25 - 37 , doi: 10.7748/nr2007.01.14.2.25.c6019 .

24 Mack N , Woodsong C , MacQueen KM , Guest G and Namey E . Qualitative research methods: a data collector's field guide . North Carolina, US : Family Health International ; 2005 .

25 Watson M , Peacock S , Jones D . The analysis of interaction in online focus groups . Int J TherRehabil . 2006 ; 13 ( 12 ): 551 - 557 , doi: 10.12968/ijtr.2006.13.12.22471 .

26 Woodyatt CR , Finneran CA , Stephenson R . In-person versus online focus group discussions: a comparative analysis of data quality . Qual Health Res . 2016 May ; 26 ( 6 ): 741 - 749 , doi: 10.1177/1049732316631510 .

27 Thrul J , Belohlavek A , Hambrick D , Kaur M , Ramo DE . Conducting online focus groups on Facebook to inform health behavior change interventions: two case studies and lessons learned . Internet Interv . 2017 Sep ; 9 : 106 - 111 , doi: 10.1016/j.invent.2017.07.005 .

Corresponding author

Related articles, all feedback is valuable.

Please share your general feedback

Report an issue or find answers to frequently asked questions

Contact Customer Support

College & Research Libraries ( C&RL ) is the official, bi-monthly, online-only scholarly research journal of the Association of College & Research Libraries, a division of the American Library Association.

C&RL is now on Instragram! Follow us today.

María Evelia Emerson is Student Success Librarian and Assistant Professor at the University of Illinois Urbana-Champaign, email: [email protected] .

C&RL News

ALA JobLIST

Advertising Information

• Research is an Activity and a Subject of Study: A Proposed Metaconcept and Its Practical Application (76908 views)
• Information Code-Switching: A Study of Language Preferences in Academic Libraries (40088 views)
• Three Perspectives on Information Literacy in Academia: Talking to Librarians, Faculty, and Students (28581 views)

“But Where Are You Really From?”: Multiracial Students, Sense of Belonging, and Academic Libraries

María Evelia Emerson *

Academic libraries need to have a stronger understanding of how to best support multiracial college students. The purpose of this study was to learn if multiracial students viewed their academic library as a place that increased their sense of belonging. Through interviews and a focus group with multiracial undergraduate students at the University of Illinois Urbana-Champaign (UIUC), the author learned that the interviewed students viewed the library in a positive manner but did not feel like it increased their sense of belonging on campus. Suggestions on how to increase their sense of belonging from students are included in the discussion.

“In America, you don’t get to decide what race you are. It is decided for you.”

~ Chimamanda Ngozi Adichie, Americanah 1

Introduction 

In 1967, the Supreme Court officially legalized interracial marriage in the case of Loving vs. Commonwealth of Virginia . Although there were interracial couples and multiracial individuals well before 1967, the legalization of interracial marriage was certainly a contributing factor in American society becoming to become less monoracial—a trend that continues today. However, “multiracial” as an identity often remains unacknowledged. For example, it was not until the year 2000 that the US Census allowed citizens the option to select more than one racial category, 2 and there are still official documents today, in 2024, that restrict users to select only one race. However, the US Census shows a substantial increase in individuals identifying as “multiracial” (categorized as “two or more races” in the census). 3 In 2010, nine million people selected “multiracial,” while in 2020, the percentage increased by 276 percent, meaning 33.8 million people selected multiracial. 4

Although the multiracial identity is still not commonly recognized, individuals who identify as multiracial still experience daily reminders that much of society views them as outsiders. Questions and statements such as: “What are you?”; “Where are you from?”; and “I never saw you as…” are common experiences for people from multiracial backgrounds. These questions make it clear to the person on the receiving end that they are racially ambiguous and that others will decide what race they are for them. Niall Singh, an artist of mixed Indian and Scottish heritage, said in an interview with Mixed Messages (a weekly newsletter about the mixed-race experience) that, “often my perspective of how I see myself has been defined by white people, because they’re the ones who most frequently tell me what they think I look like.” 5   

The author of this article identifies as multiracial and believes there needs to be a better understanding and awareness of the multiracial population, particularly since more people are identifying with this demographic. The author conducted a series of one-on-one interviews, and also led a focus group of seven undergraduate students who self-identified as multiracial at the University of Illinois Urbana-Champaign (UIUC). The purpose of the interviews and focus group was to learn if the interviewed students viewed their academic library as a place that increased their sense of belonging at UIUC, and to learn more about ways academic libraries can support multiracial students.

For the purpose of this article, the term “multiracial” can be defined as a person who identifies with two or more racial groups. Other terms commonly used are biracial, mixed-race, and racially mixed. Although “multiracial” is the predominant term used throughout the article, other terms are used interchangeably. Sense of belonging, a term categorized as a social need in Abraham Maslow’s work “A Theory of Human Motivation,” 6 can be defined as a human emotional need to feel accepted and supported by members of a group; it contributes to one’s physical and mental well-being. 

The University of Illinois Urbana-Champaign is a public, land grant institution, and is in the twin cities of Urbana and Champaign in southeast Illinois. UIUC is the flagship institution of the University of Illinois system and is an R1 research university. The University Library is one of the largest academic libraries in the United States. The undergraduate student body makeup at UIUC in the fall of 2022 is 38.87 percent Caucasian, 21.73 percent Asian American, 13.49 percent Hispanic, 5.81 percent African American, .03 percent Native Hawaiian or Pacific Islander, .03 percent Native American, 15.2 percent International, and 1.3 percent unknown. 7  Starting in 2010, students at UIUC could select “multiracial” as their racial identity; in 2010, 560 (1.6%) undergraduate students selected multiracial for their racial demographic, and in 2022, the number increased to 1,233 (3.53%). 8 With the continual increase in students identifying as mixed-race, there is a need for academic libraries to have a better understanding of how to better support the students of this demographic.  

Literature Review 

Libraries have been viewed, or strive to be seen, as safe spaces for their users. The American Library Association has a “Code of Ethics” that, when practiced, strives to make libraries a safe space where censorship is resisted, information is accessible, and patron privacy is upheld, among other principles. In the summer of 2021, a ninth principle was added to the code, stating that libraries, “work to recognize and dismantle systemic and individual biases [and] to confront inequity and oppression.” 9 Libraries can contribute to fulfilling these principles in different ways, such as implementing different book displays, programs, initiatives, and resources that represent the different backgrounds and needs of library users. 10 While there is an abundance of literature that discusses different ways that libraries attempt to be more inclusive, there is a definite research gap about the role of academic libraries in their support of multiracial students, particularly regarding these students’ sense of belonging. 

The majority of literature involving mixed-raced populations references the groundbreaking and foundational work of Maria Root and Kristen Renn; the author drew upon these works in some of their interview and focus group questions for this study. Root, a leading expert on mixed-race people, has conducted research on multiracial identity and experiences, providing a spotlight on the complexity—and always evolving identity development—of mixed-race people, as well as the need for society to have a much stronger understanding of multiracial experiences. Through her work with mixed-race individuals, Root established that racial identity development for multiracial individuals is fluid instead of rigid. 11 Her “Bill of Rights for Racially Mixed People,” a document written to affirm mixed-race identity, also acknowledges this by stating that mixed-raced individuals have the right, “to identify…differently in different situations,” and “to change my identity over my lifetime—and more than once.” 12

Similar to Root, Renn has researched the racial identity development of biracial and mixed-race people, but focuses specifically on college students. 13 Renn drew on Urie Bronfenbrenner’s ecological theory of human development 14 for its flexibility of identity development, stating that, “…the attempt to explore the cumulative, interactive influences of overlapping social settings, some or all of which may be sending contradictory messages regarding racial identity and identification, calls for more theoretical flexibility than the racial identity development models…” 15

For the literature review, the author focused on how higher education institutions supported mixed-race students, and then looked at the role academic libraries play regarding a sense of belonging for students. Lastly, she looked at the literature to learn more about the programs and research conducted in academic libraries, so these institutions have a better understanding of the sense of belonging for multiracial students. 

Multiracial Students, Higher Education, and Sense of Belonging  

There is a large body of research that focuses on multiracial students in higher education. Many of these articles focus on the mental health and well-being of multiracial students. Michael T. West and Cara S. Maffini discussed how college is a normal time for students to explore their identity more, continuing that students also, “may begin to question and reflect on how their previous or current identity, which has been shaped by their past experiences (e.g., family, hometown), can be integrated with their new experiences.” 16 Some articles investigate ways to provide coping strategies and therapy exercises to mixed-race students, such as venting sessions to process their emotions that stem from the complexity of a multiracial identity. 17 Samuel D. Museus, Susan A. Lambe Sariñana, and Tasha Kawamata Ryan suggested that providing more learning opportunities on college campuses, “might simultaneously serve as a productive way to educate campus communities about multiracial issues and an effective coping strategy for mixed-race students.” 18 Other articles focus on a particular student demographic, such as Black and white individuals, 19 or students of Asian descent. 20

Research on multiracial students is especially prevalent in student affairs literature. 21 There is also literature on different experiences that multiracial students encountered in their college campuses. Haley K.M. Okamoto analyzed the challenges and identity development multiracial college students experienced, highlighting the need of work like this because, “while racial identity has certainly made its way into higher education curriculums, racial identity is still often conceptualized as singular and static.” 22 Some higher education institutions, such as Stanford University and Yale University, are starting to include the topic of multiracialism in their curriculum; however, this is still a field that is missing from most institutions.  

Academic Libraries and Student Sense of Belonging 

While many academic libraries aim to be seen as a safe place for students, they are not always successful in this endeavor. This is especially true for marginalized students since libraries have a history of whiteness in their profession, buildings, collections, policies, and treatment of users. Myrna Morales, Em Claire Knowles, and Chris Bourg argue that, “academic librarians are perhaps uniquely equipped and empowered to define and redefine systems of knowledge that convey ‘truths’ about what we know about the world and how that knowledge is organized and evaluated,” 23 yet diversity initiatives often fail to consider how their own policies, standards, and practices impact their students and colleagues.

The profession of librarianship is also an overwhelmingly white, female-dominated field; in 2021, over 83 percent of librarians identified as white, and 83.2 percent identified as women in 2020. 24 This lack of diversity in library staff negatively impacts students not only due to insufficient representation in the programs and collections, but also in the librarians themselves, who may serve as partners and mentors to students. Brook et al. discuss how the library centers whiteness in public services, stating that, “when students cannot see themselves and their values represented in the library, be that in the staff, the policies, the services, or the space, then it is easy to understand why they might not use the library.” 25 Lack of diversity in the library profession impacts BIPOC librarians as well as students. Recent library literature highlights the problems of working in predominantly white spaces and discusses steps to decenter whiteness in the library profession. 26

Academic libraries play a prominent role in student success, and can help students by providing support and a sense of community during their time in college. This is especially important since college is a time when students—especially underrepresented students—may experience imposter syndrome, which contributes to a lower retention rate. Imposter syndrome, originally called imposter phenomenon, is a term coined by psychologists Clance and Imes in their 1978 study, “The Imposter Phenomenon in High Achieving Women: Dynamics and Therapeutic Intervention.” 27 Clance and Imes researched how high achieving women often feel like they are not qualified to belong in a workplace or education setting; however, their study excluded many other groups, “namely women of color and people of various income levels, genders, and professional backgrounds.” 28 Tulshyan and Burey argue that this study overlooked how institutions and workplaces cultivate these feelings of inadequacy, and that, “leaders must create a culture for women and people of color that addresses systemic bias and racism.” 29

Focusing on academic libraries, Ramsey and Brown argue that these types of institutions can assist students in overcoming imposter syndrome by creating inclusive spaces to, “help enhance a sense of belonging in their students, which can counter the ‘outsider’ feelings inherent in imposter syndrome.” 30 Oliveira also presents the different ways that academic libraries contribute to student retention and discusses the themes of social learning spaces, library use, and information literacy, which can assist with the college transition process and increase student retention. 31

Academic Libraries and Multiracial Student Sense of Belonging

Academic libraries engage with students and attempt to promote diversity and inclusivity in different ways, such as displays, exhibits, relationships, and visibility of collections. 32 However, the majority of these initiatives and programs focus on underrepresented groups, typically through the lens of a singular identity perspective and experience. For example, Bucy’s article focused on the Native American student experience in the academic library, 33 and Couture et al. discussed their work with first-generation students. 34 While these groups are underrepresented and often misunderstood student populations whose experiences are essential to understand, much of the literature still presents these groups from a homogeneous perspective, which creates a barrier to understanding the complexity and intersectionality that occurs with many students.  

Although most library programs and incentives focus on homogeneous groups, some academic libraries have seen the importance of acknowledging and supporting multiracial students. The University of Michigan Library designed a library program specifically geared towards mixed-race students, including a faculty panel titled, “ Multiracial in a Monoracial World: Interraciality Informing Academic Work ,” and film screenings on mixed-race experiences. 35 There are also several articles that address some of the behind-the-scenes work that libraries can do to become more inclusive. Beall discussed creative ways that the Dewey Decimal Classification system and MARC records can become more inclusive of racially mixed people, 36 while Furner discussed using critical race theory as a way to evaluate bibliographic classification schemes. 37

Methodology 

The study, which was conducted during the spring semester of 2022, was designed in two parts: individual interviews with undergraduate students who self-identified as mixed-race, followed by a focus group of the same individuals. The questions asked in the one-on-one interviews were focused on the participant’s individual journey of their mixed-race identity, as well as their experiences in libraries, particularly the University of Illinois Library. The focus group questions pertained to how academic libraries can increase a sense of belonging among their multiracial students. Approval for the study was sent to the campus Institutional Review Board in November of 2021, and it obtained approval in December 2021.

Students qualified for the study if they were an undergraduate student at UIUC and identified with two or more races. The study was advertised through: the library’s social media accounts, email recruitment to the campus’ culture houses, emails to registered student organizations, and the weekly campus newsletter. As an incentive to participate, students received a $30 gift card to Amazon upon completing both the individual interview and the focus group. Funding was provided for up to twenty students to participate in the study. Since this was a qualitative study, the initial goal of twenty students (which is only 1.62 percent of the multiracial student population at UIUC) was chosen, as it is a manageable number of interviews to work through and sufficient for identifying common themes.

Interested students first had to complete a pre-screening questionnaire (Appendix A) to ensure that they qualified for the study. Thirteen students originally responded and completed the pre-screening form; eleven students qualified for the study. Those who did not qualify identified with only one race. Students were sent a consent form to sign; several students chose not to participate at that point in the process, resulting in the final number of participants being seven. Table 1 provides information about the racial background each of the students selected on the pre-screening questionnaire (Appendix A).

The seven students represented only .05 percent of the multiracial population at UIUC. Of the seven students, 71 percent selected the racial background of Asian and Caucasian from the pre-screening questionnaire. While the participating students represent a very small sample, themes were easily identified, and similar experiences were described from the individual conversations and focus group with the participants. There are also future plans to expand this study to see if the answers are representative of a larger population.

It is important to note that Hispanic, Latinx, and Spanish are not races, but ethnicities. People who identify as Hispanic, Latinx, or Spanish are defined by the U.S. census to be, “a person of Cuban, Mexican, Puerto Rican, South or Central American, or other Spanish culture or origin regardless of race.” 38 However, for the purpose of this study, students could select this as a race they identify with to align with university data as much as possible. 

One-on-one interviews and a focus group were chosen as the qualitative research methods for the study. The personal interviews allowed the author to learn more about individuals’ personal experiences and thoughts regarding their multiracial identity, while the focus group helped participants hear other students’ thoughts, and to share similarities or differences in those opinions. Focus groups also allowed participants to work off each other and brainstorm different ways that libraries could be more inclusive of their identities. Since the study was conducted during the COVID-19 pandemic, Zoom was chosen as the format for the interviews and focus group to ensure safety. Meeting via Zoom also allowed for more flexibility with scheduling around participating students’ classes, jobs, and extracurricular activities. Study participants also had the choice of having the camera on or off while in the interviews and focus group to help participants feel more at ease while engaging with the author. All participants decided to have their cameras off during the interviews and focus group. Before conducting the one-on-one interviews and focus groups, the author also consulted with some of her colleagues to learn more about best practices for leading these conversations.

Questions for both the one-on-one interviews and the focus group were designed in several different ways. First, the author drew from her own experiences as a multiracial individual and designed some questions that she felt would provide a foundation on how study participants viewed their mixed-race identities, as well as how others may perceive them and how that impacts their day-to-day lives. The author shared these questions with another multiracial colleague to discuss if they felt the questions would help provide information that the author hoped to learn about multiracial students and libraries. Second, the author designed questions that would provide foundational information about students’ usage of libraries to gain a better understanding of ways students utilize, spend time in, and view the library. Lastly, the author incorporated some of the work of Renn and Root into the interview questions, specifically questions number three and four (Appendix B). The author drew from Root’s Bill of Rights to design question three, which asked about being incorrectly identified as a mixed-race individual, as well as Renn’s work on racial identity development to craft question number four. 39 The question preparation for the interview and focus group was conducted over several afternoons, in addition to being revised a few times based on discussion with the author’s colleague, as well as the literature review.

The one-on-one interviews lasted between fifteen minutes to forty-five minutes; each student was asked the same set of questions, included in appendices B and C. Each interview was recorded with the student’s consent, and the audio recording was transcribed using the NVivo transcription service. After all the personal interviews were complete, a focus group with all seven participants took place over Zoom, moderated by the author. The focus group session was fifty minutes long, recorded with the participants’ consent, and transcribed using the same software as the personal interviews. While almost all the participants expressed their thoughts verbally, one student participated more frequently using the chat function in Zoom. At the end of the focus group, each participant was sent a $30 Amazon gift card to their university email account.

After the audio recordings of the personal interviews and focus group were transcribed, the author identified recurrent themes that occurred in the conversations. The transcriptions of the interviews and focus group—along with the chat transcript of the focus group—were printed and read several times in order to find recurring experiences, thoughts, and keywords from the study participants, which were then color-coded into categories. Quotes were also pulled from the interviews and focus group, and were color-coded into the appropriate category that supported the recognized themes. There were some outliers within the identified themes, which are included in the findings section of this article. The outliers, though few, are important to include since they highlight how everyone’s multiracial journeys are different, even if there is overlap within their experiences. An afternoon was set aside to conduct and complete the qualitative data coding process of the seven interviews and one focus group.

Themes of identity challenges, feelings of erasure, and the desire to find community, all emerged from the individual interviews and focus group. Participants discussed how these themes tied into their experiences on campus and in the academic library. Overall, students in this study viewed the University of Illinois Library as a neutral place on campus, which served as a location to study and to provide academic resources rather than a place that fostered a sense of belonging, community, and support. All the participants used the library at a minimum of a few hours a week, and none of them had experiences where they felt unwelcome in the University of Illinois Library. 

The author defined the themes from the study as follows:

• Identity challenges refers to both challenges experienced internally by mixed-race individuals (e.g. having to “decide” which race they are, or feelings of imposter syndrome), as well as challenges from others of the individual’s identity (e.g. “I don’t see you as…” or “Let me guess what you are.”).
• Feelings of erasure address the experiences many of the study participants expressed with feeling like their identities as multiracial individuals are rarely recognized as an identity in the same way that other groups are recognized. For example, while seeing many displays and resources on sexual orientations or homogenous races, they do not see the same recognition of multiracial identities.
• Finding community references the struggle many of the participants described of feeling like they do not belong in a specific community since their identity encompasses several different communities.

Identity Challenges

One of the questions in the individual interviews focused on their journey with their multiracial identity. All participants did have at least one time in their lives where they struggled with their mixed identity. Family relationships, growing up in a predominantly white neighborhood or schools, and misidentification of their identities all contributed to the confusion and difficulties they experienced in their multiracial identity journeys. One participant stated, “If you are multiracial then you are in a constant state of flux in between those ethnic distinctions.” 

Many participants discussed how people are often confused about how to view them. Several told stories about how it is common for people to guess what they “are.” While some of the students said that they understand people are curious and normally mean well, others expressed frustration. One student said, “sometimes it’s fine, but sometimes it’s kind of weird when they just keep guessing. Like, it’s not a game.” Another student remarked during the focus group that they wish others knew that “I am many things, right? …I just wish that people would respect the multifaceted nature and mind of that.”  

When asked about how the library did or did not contribute to their identity journey, many of the students did not feel like they had resources available to support them or provide more information about their identity. A few participants talked about a time in the library when they stumbled upon a book that told a story about a mixed-race individual, and they were excited to read it because they could hear about someone else’s multiracial identity journey. However, these were isolated incidents, and the students discussed how overall there is a lack of resources that could support or assist them in learning more about being multiracial. During the focus group, one student said, “I haven’t really seen many materials that do represent multiracial groups, and I think that would be great for increasing our sense of belonging.” Others agreed with the statement and talked about how more awareness of the topic and resources could help themselves and other students. 

Many of the study participants also described the feelings of imposter syndrome, or having to choose one race over another, on multiple occasions in their lives. However, one student discussed a positive experience of not needing to choose their race describing a non-UIUC library where the library was, “full of people from different backgrounds using different languages…we were all respecting one another…and it made me feel like I didn’t have to choose one of my races over the other.” This experience highlights the relief from not having to choose between races, and the potential support academic libraries can provide by having a more heterogeneous environment.

Despite the challenges and complexities that can come with identifying as mixed-race, all the participants said that they felt being multiracial made them more empathetic and understanding of people. They also felt like it helped them learn how to better navigate differences of opinions and backgrounds. One participant of the study said that being multiracial is, “definitely a blessing. It’s more like it turns into a blessing once you realize how to manage that in your life. If you want to make it a blessing, you can make it a blessing.” 

Feelings of Erasure

All the participants discussed how they felt that their multiracial identity is often ignored, or erased, both by institutions and people in their lives. The lack of multiracial options in surveys and forms, the erasure of their identity in conversations with friends and classmates (and sometimes even families), the absence of multiracial topics in educational settings, and the lack of acknowledgment of the multiracial identity in libraries were discussed in the personal interviews and focus group. One student talked about a time they felt ignored when they went to [Library] to find something to read, and found books organized by different race and sexual orientation identities: “But I didn’t see the mixed section, so that was just something that was like…I am that, it would have been cool to have at least a few books to look through because [multiracial] is an identity.”  

This anecdote ties into the recurring theme regarding the visibility of heritage and history months, as well as spotlighting authors from a variety of backgrounds. Study participants discussed seeing library displays promoting different books, authors, histories, or celebrations of specific groups, yet none for multiracial identities. Many of the participants acknowledged that there is no multiracial identity month to highlight, yet it is an identity that libraries can still put a spotlight on; doing so would help them feel more seen. One of the participants did not completely agree that mixed-race identities are erased, but instead were more misunderstood by their peers, remarking, “I think a lot of the time the way it [being multiracial] works is that it’s acknowledged but not necessarily respected.” This response was shared in the focus group and, upon hearing that, many of the other participants remarked on how they feel like the mixed-race experience is misunderstood by many; in an individual interview, a participant said that often people get confused about being multiracial because, “it’s like they only see one side of things.” Several remarked that they felt that with more resources in the library and discussions about the mixed-race experience, others could begin to have a better understanding of the complexity and varied experiences and journeys that occur with that identity. During the focus group, participants talked about when they felt most at home and safe; feelings of acceptance, respect, understanding, and acknowledgment of all parts of their racial background contributed to an increased sense of belonging. One participant said, “I feel most comfortable when people acknowledge both my Polish and Chinese side because I feel like it’s pretty easy…for like other people to kind of dismiss whichever culture or race that you look less like.” None of the participants identified their academic library (or any library) as a place where they felt at home. For participants, the library was viewed as a neutral building to in which to study, find resources for their classes, and receive research support. It was not seen as a support system, nor as campus resource that could contribute to their sense of belonging and/or understanding of their place on campus. 

Finding Community 

Very few students felt that they had a multiracial community; most participants felt alone in their identity, though some did actively seek out other multiracial individuals. Some students mentioned siblings who they could talk about their identity with, but others remarked on how their families ignored that part of their children’s identity. This would sometimes happen for different reasons; some participants discussed how their parents are immigrants and were told to “Americanize” their children, while others acknowledged family tensions due to race. One participant said they were, “sort of whitewashed by my dad because he, I don’t know, but he liked to promote my whiteness over anything else.”  

When asked about finding community in the library, the students commented on how they found library spaces that worked well for their academic needs, but they did not typically find people or resources that could relate to their mixed-race experience. Regarding library staff, some of the participants felt like they saw themselves represented by the library student workers, but other participants did not feel represented in that sense. As already mentioned, some of the participants reflected on the time they found a library resource about, or written by, someone with a mixed-race background. However, one student thought of a particular example with a book when asked about community and discussed their excitement over finding a book that was about a character with the same racial makeup as themselves. While this student may have not found a multiracial community in person, they did think of one instance in which a library resource provided a small community to them.

In both the interviews and the focus group, participants brought up other campus resources which could serve as possible places of community and/or increase a sense of belonging, such as culture houses and student organizations. However, several of the participants still struggled with finding a community within culture houses because they felt the houses did not relate to their multiracial experience, and better served students who identified with one racial background. Several students also discussed the various student organizations on campus that are for students from different racial and ethnic groups, but felt that one’s appearance and skin tone impacted whether a multiracial student could participate and feel welcomed in these groups. One student reflected on their experience in the focus group and said that, “even though my lived experiences are not necessarily indicative of my ethnic background, it still exists.” The focus group discussed whether there was a student organization for multiracial students; the majority of the students believed one did not exist on the campus, but several other students said there used to be one, but that it had disbanded for unknown reasons. 

The struggle of not belonging to any specific group was discussed by almost all the participants. One participant said, “I feel like even though I say I can connect to more people I still feel like there is some disconnect, and I don’t belong in a particular group.” This feeling was brought up several times regarding library displays, with one student noting, “all of these displays are for specific cultures and I’m just feeling like I didn’t really fit into any of those categories.” Several participants stated that, even among mixed-race individuals, it is rare to find someone else with the same exact background as you, or with the same lived experiences. However, the one commonality of the multiracial community is the shared experience of not belonging to one single racial group. One study participant said that, “even if you come from a different background, you still have this one shared identity of being multiracial, multicultural.” 

Discussion  

All the participants of this study use the UIUC libraries to support their academic needs. All of them spent at least a couple hours a week studying in the library, while others spent close to fifteen hours a week. Several of the participants used their hometown’s public libraries, and one student mentioned using their high school library. While the results of this study were not a negative reflection of the University of Illinois Library, or of libraries in general, it was discovered that the participating students did not view their academic library as a place to increase their sense of belonging at UIUC.

Overall, students described using libraries for research and scholarship, although several participants did go to the library to find leisure books to read. Similar to other experiences in their lives, participants did not feel like their multiracial identity was acknowledged within the library, especially in ways that other cultures and races have been acknowledged, such as through various displays and collections. Many of the participants talked about specific library displays. The library’s display for Black History Month frequently came up, and another student talked about a display that highlighted Arab women authors. While the students were not surprised about not finding a display about multiracialism, several expressed a desire to see that part of their identity acknowledged in that specific way. 

An unexpected perspective that emerged with several study participants was the belief that libraries are neutral. Because of this belief, many of the participants felt that the library did not have a responsibility to support their sense of belonging; one participant said, “I have no problems with them [libraries] because they take such a neutral stance on everything. And that’s fine…but if part of their goal was to be more inclusive, then, you know, I probably expect a bit more.” Another student remarked that, “libraries aren’t political places,” and many participants stated that, while they did not see their own representation in the library, they did not believe it was part of the library’s responsibilities to increase their sense of belonging. Several of the study participants said that they had a very neutral stance on the library, and many participants reflected on how they associate the library with a physical space to study in. These observations from the interviews and focus group are important because they show that the University of Illinois Library is not as inclusive as it—according to the library’s mission statement—strives to be. 40 Instead, the study participants believe that the mission of the library is to solely provide resources and services in an unbiased way. 

There has been a great deal of criticism of the idea that libraries are neutral institutions 41 because libraries uphold white supremacy; this relates back to the issue of whiteness in academic libraries, and how the systemic inequalities in library policies and structures need to be addressed for academic libraries to become more inclusive. While the University of Illinois Library mission statement has language that communicate the necessity and importance of supporting student needs through library collections and services, it does not include active language against neutrality in libraries. By adopting more active language against neutrality in its mission statement, as well as continuing to include more services and collection items that are more inclusive and accessible, the library may begin to change students’ perspective of the library from a neutral space to one that is more inclusive space.

Academic libraries are a place where students can grow not only through scholarly resources, but through the programs, space, staff, and exhibits offered. It is important for academic libraries to explore ways to reach out to, and connect with, students who have intersectional identities and who are seeking places and communities that make them feel less alone. 

Conclusion  

The author learned from the discussion with the participating students that, while they viewed the library in a positive manner, they did not feel like it increased their sense of belonging on campus. Instead, they viewed it as a physical space to study in, with several of them viewing libraries as neutral spaces whose purpose is meant only to provide academic support. While none of the students felt unwelcomed in the University of Illinois Library, they did acknowledge that they felt erasure of their multiracial identity in the library as they did with many other places on campus. 

With the population of the United States continuing to become more diverse, there needs to be more attention on, and awareness of, multiracial identities to combat the erasure of this demographic. Being multiracial brings about a wealth of complexities, including identity challenges and a lack of a sense of belonging. Mixed-race individuals have unique experiences that all contribute to their sense of self, but the complexity of this identity also allows for more empathy and support for others. 

Themes of identity challenges, erasure, and finding community all emerged from conversations with participants, and will be used in guiding questions and discussions for the next steps of this study. Additionally, the idea of academic libraries as neutral places will be explored more in-depth to gain a stronger understanding of how students view the purpose of academic libraries. This study is only a small sampling from one university, and it will next be expanded to explore the experiences of a larger sampling of multiracial students in higher education on a national level. With a larger sample size, more information, experiences, and reflections will be gathered, which will provide a stronger understanding and will continue the conversation of how academic libraries can best support this group of students and increase their sense of belonging on their campuses.

Acknowledgements 

The author wishes to acknowledge the Research and Publication Committee of the University of Illinois at Urbana-Champaign Library, which provided support for the completion of this research. Additionally, thank you to Emma Saito Lincoln for her help in constructing the questions designed for this study. Lastly, thank you to Jen-chien Yu for her help in learning about the interview process and focus groups.

Appendix A. Pre-screen Questionnaire

• University of Illinois email
• Black or African American
• Hispanic, Latinx, or Spanish
• Native American
• Hawaiian or other Pacific Islander
• Middle Eastern or Northern African
• Provide own answer

Appendix B. Individual Interview Questions

• What does it mean (to you) to be multiracial?
• Do you feel like you have a multiracial community, or do you feel alone? How so?
• Have people ever classified your race incorrectly? If so, how have you responded and how did it make you feel? Have you experienced identity issues as a multiracial person? If so, how? If not, what has helped you feel secure in your identity? 
• How much time do you normally spend in the library? Why do you like spending time there AND/OR what would make you spend more time there? 
• What has your library experience been like, not only at the University of Illinois Urbana-Champaign, but other libraries as well?
• Do you feel represented in the library? On campus? How so, or how not? 
• Can you tell me about a time when you really didn’t feel like you belonged at the library? 
• Was there a time or times where you felt like you were “seen” in the library? Like your identity was acknowledged in some way, directly or indirectly?Do you feel it’s important for multiracial experiences to be better understood? If so, why? If not, why not? 

Appendix C. Focus Group Questions 

• What are some ways that libraries can contribute toward increasing a sense of belonging on campus?You identify as multiracial students, but do you feel like your identity as a multiracial/biracial person is a valued identity? In particular at the University of Illinois Urbana-Champaign?  
• Have you ever felt “erased” in the library? As if your identity does not exist? If so, how? 
• What would you like to see more of in the library for this part of your identity to feel acknowledged? 
• What does it mean to feel ‘at home’ in a place?Do you feel like that feeling applies to your experience in the U of I libraries? How so? 
• Taking what you just described, in what ways is the library responsible for making you feel at home? 
• Which library do you feel most at home at and why? 
• What do you want others to know about your experience/identity? 

1. Chimamanda Ngozi Adichie, Americanah (New York: Anchor Books, 2014).

2. “Frequently Asked Questions (FAQs) about Race,” U.S. Census Bureau, last modified December 3, 2021. https://www.census.gov/topics/population/race/about/faq.html . 

3. “Improved Race and Ethnicity Measures Reveal U.S. Population is Much More Multiracial,” U.S. Census Bureau, last modified June 10, 2022, https://www.census.gov/library/stories/2021/08/improved-race-ethnicity-measures-reveal-united-states-population-much-more-multiracial.html .

4. U.S. Census Bureau, “Improved Race and Ethnicity.”

5. Isabella Silvers, “Niall Singh: ‘Often, How I See Myself Has Been Defined by White People,’” Mixed Messages , July 4, 2022, https://mixedmessages.substack.com/p/niall-singh-often-how-i-see-myself . 

6. A.H. Maslow, “A Theory of Human Motivation,” Psychological Review 50, no. 4 (1943): 370–396, https://doi.org/10.1037/h0054346 .  

7. “University of Illinois Student Enrollment,” Division of Management Information, last modified September 7, 2022, https://dmi.illinois.edu/stuenr/#ethgen .

8. Division of Management, “University of Illinois.”

9. “Professional Ethics,” American Library Association, last modified June 29, 2021, https://www.ala.org/tools/ethics .

10. Sabrina Thomas and Kacy J. Lovelace, “Combining Efforts: Libraries as Mental Health Safe Spaces,” College and Research Libraries News 80, no. 10 (2019): 546–48, https://doi.org/10.5860/crln.80.10.546 ; Lisa Gay-Milliken and Jeffrey DiScala, “Going beyond Book Displays: Providing Safe Spaces for LGBTQ Youth,” Knowledge Quest 48, no. 3 (2020): 10–12, http://www.library.illinois.edu.proxy2.library.illinois.edu/proxy/go.php?url=https://search-ebsco-host.com.proxy2.library.illinois.edu/login.aspx?direct=true&db=eric&AN=EJ1238793&site=eds-live&scope=site ; Stephanie Everett, “Visualizing the Silent Dialogue About Race: Diversity Outreach in an Academic Library,” Journal of Academic Librarianship 44, no. 4 (2018): 518–26, https://doi.org/10.1016/j.acalib.2018.04.002 ; Margaret K. Merga. “Libraries as Wellbeing Supportive Spaces in Contemporary Schools,” Journal of Library Administration 61, no. 6 (2021): 659–75,  https://doi.org/10.1080/01930826.2021.1947056 ; Kelsey Keyes. “Welcoming Spaces: Supporting Parenting Students at the Academic Library,” Journal of Academic Librarianship 43, no. 4 (2017): 319–28. . 

11. Maria P.P. Root, ed., The Multiracial Experience: Racial Borders as the New Frontier , (London: Sage, 1996).

12. Maria P.P. Root, “A Bill of Rights for Racially Mixed People,” Embracing Diversity, last modified December 10, 2018, https://www.safehousealliance.org/wp-content/uploads/2012/10/A-Bill-of-Rights-for-Racially-Mixed-People.pdf .  

13. Kristen A. Renn, “Patterns of Situational Identity Among Biracial and Multiracial College Students,” The Review of Higher Education 23, no. 4 (2000): 399–420, https://doi.org/10.1353/rhe.2000.0019 ; Kristen A. Renn, “Research on Biracial and Multiracial Identity Development: Overview and Synthesis,” New Directions for Student Services, no. 123 (2008): 13–21, https://doi.org/10.1002/ss.282 .

14. Urie Bronfenbrenner, The Ecology of Human Development: Experiments by Nature and Design (Cambridge, Mass: Harvard University Press, 1979).

15. Kristen A. Renn, “Understanding the Identities of Mixed-Race College Students through a Developmental Ecology Lens,” Journal of College Student Development 44, no. 3 (2003): 383–403, https://doi.org/10.1353/csd.2003.0032 .    

16. Michael T. West and Cara S. Maffini, “‘What Are You?’ Conceptualizing the Complexities of Bicultural and Biracial College Student Experiences,” Journal of College Counseling 22, no. 2 (2019): 164–178, https://doi.org/10.1002/jocc.12128 .

17. Derrick A. Paladino and Harrison Davis Jr., “Counseling and Outreach Strategies for Assisting Multiracial College Students,” Journal of College Student Psychotherapy 20, no. 3 (2006): 19–31, https://doi.org/10.1300/J035v20n03_03 .

18. Samuel D. Museus, Susan A. Lambe Sariñana, and Tasha Kawamata Ryan, “A Qualitative Examination of Multiracial Students’ Coping Responses to Experiences with Prejudice and Discrimination in College,” Journal of College Student Development 56, no. 4 (2015): 331–348,  https://psycnet.apa.org/doi/10.1353/csd.2015.0041 .

19. Elizabeth M. Lusk et al., “Biracial Identity and its Relation to Self-Esteem and Depression in Mixed Black/White Biracial Individuals,” Journal of Ethnic & Cultural Diversity in Social Work 19, no. 2: (2010):109–26, https://doi.org/10.1080/15313201003771783 . 

20. Shinwoo Choi et al., “Sense of Belonging, Racial Microaggressions, and Depressive Symptoms Among Students of Asian Descent in the United States,” Smith College Studies in Social Work 91, no. 2 (2021): 115–41, https://doi.org/10.1080/00377317.2021.1882922 . 

21. Nana Osei-Kofi, “Identity, Fluidity, and Groupism: The Construction of Multiraciality in Education Discourse,” Review of Education, Pedagogy, and Cultural Studies 34, no. 5 (2012): 245–257, https://doi.org/10.1080/10714413.2012.732782 ; Aurora Chang, “Multiracial Matters–Disrupting and Reinforcing the Racial Rubric in Educational Discourse,” Race Ethnicity and Education 19, no. 4 (2016): 706–30, https://doi.org/10.1080/13613324.2014.885427 ; Jackie Hyman, “The New Multiracial Student: Where Do We Start?,” The Vermont Connection 31, no. 1 (2010): 14, https://scholarworks.uvm.edu/tvc/vol31/iss1/14 ; Angela H. Kellogg and Debora L. Liddell, “‘Not Half but Double’: Exploring Critical Incidents in the Racial Identity of Multiracial College Students,” Journal of College Student Development 53, no. 4 (2012): 524–41, https://doi.org/10.1353/csd.2012.0054 ; Samuel D. Museus, April L. Yee, and Susan A. Lambe, “Multiracial in a Monoracial World: Student Stories of Racial Dissolution on the Colorblind Campus,” About Campus 16, no. 4 (2011): 20–25, https://doi.org/10.1002/abc.20070 .

22. Haley K.M. Okamoto, “Mixed Plate: Exploring Multiracial Student Identities within Higher Education,” PSU McNair Scholars Online Journal 14, no. 1 (2020): 1–44, https://doi.org/10.15760/mcnair.2020.14.1.4 .

23. Myrna Morale, Em Claire Knowles, and Chris Bourg, “Diversity, Social Justice, and the Future of Libraries,” portal: Libraries and the Academy 14, no. 3 (2014): 439–451, https://doi.org/10.1353/pla.2014.0017 .

24. “Library Professionals: Facts and Figures,” Department for Professional Employees, https://static1.squarespace.com/static/5d10ef48024ce300010f0f0c/t/60c24d4d903e4a17f8629234/1623346509173/Library+Workers+Facts+%26+Figures+2021+%282%29.pdf .

25. Freeda Brook, Dave Ellenwood, and Althea Eannace Lazzaro, “In Pursuit of Antiracist Social Justice: Denaturalizing Whiteness in the Academic Library,” Library Trends 64, no. 2 (2015): 246–84, https://doi.org/10.1353/lib.2015.0048 .

26. Isabel Espinal, Tonia Sutherland, and Charlotte Roh, “A Holistic Approach for Inclusive Librarianship: Decentering Whiteness in Our Profession,” Library Trends 67, no. 1 (2018): 147–62, https://doi.org/10.1353/lib.2018.0030 ; Jennifer Brown et al., “We Here: Speaking Our Truth,” Library Trends 67, no. 1 (2018): 163–81, https://doi.org/10.1353/lib.2018.0031 ; Emily A. Crist and Kelly Clark Keefe, “A Critical Phenomenology of Whiteness in Academic Libraries,” Journal of Academic Librarianship 48, no. 4 (2022): 102557, https://doi.org/10.1016/j.acalib.2022.102557 .

27. Pauline Rose Clance and Suzanne Imes, “The Imposter Phenomenon in High Achieving Women: Dynamics and Therapeutic Intervention,” Psychotherapy: Theory, Research & Practice 15, no. 3 (1978): 241–247, https://doi.org/10.1037/h0086006 .

28. Ruckia Tulshyan and Jodi-Ann Burey, “Stop Telling Women They Have Imposter Syndrome, ” Harvard Business Review , February 11, 2021, https://hbr.org/2021/02/stop-telling-women-they-have-imposter-syndrome .

29. Tulshyan, “Stop Telling.”

30. Elizabeth Ramsey and Deana Brown, “Feeling Like a Fraud: Helping Students Renegotiate Their Academic Identities,” College & Undergraduate Libraries 25, no. 1 (2018): 86–90, https://doi.org/10.1080/10691316.2017.1364080 .  

31. Silas M. Oliveira, “The Academic Library’s Role in Student Retention: A Review of the Literature,” Library Review 66, no. 4/5 (2017): 310–29, https://doi.org/10.1108/LR-12-2016-0102 .    

32. Everett, “Visualizing,” 518–26; Michelle M. Maloney, “Cultivating Community, Promoting Inclusivity: Collections as Fulcrum for Targeted Outreach,” New Library World 113, no. 5/6 (2012): 281–89; https://doi.org/10.1108/03074801211226364 ; Margaret Adeogun, “Increasing Patron’s Outreach and Engagement through Relationship Marketing: A Case Study,” Journal of Library Administration 62, no. 3 (2022): 359–75, https://doi.org/10.1080/01930826.2022.2043691 ; Paolo P. Gujilde, “Chapter 4: Moving beyond Buzzwords: Belonging in Library Collections,” Advances in Librarianship 48 (April 2021): 35–41, https://doi.org/10.1108/S0065-283020210000048004 .  

33. Rosalind Bucy, “Native American Student Experiences of the Academic Library,” College & Research Libraries 83, no. 3 (2022): 416–33, https://doi.org/10.5860/crl.83.3.416 .        

34. Juliann Couture et al., “‘We’re Gonna Figure This Out’: First-Generation Students and Academic Libraries,” Portal: Libraries and the Academy 21, no. 1 (2021): 127–47, https://doi.org/10.1353/pla.2021.0009 .   

35. Helen Look, “Multiracial Resources in a Monoracially-Organized Library World,” Urban Library Journal 22, no. 2 (2016): 1–6, http://www.library.illinois.edu.proxy2.library.illinois.edu/proxy/go.php?url=https://search-ebscohost-com.proxy2.library.illinois.edu/login.aspx?direct=true&db=lls&AN=119554073&site=eds-live&scope=site .

36. Julianne Beall, “Racially Mixed People, DDC Table 5 Ethnic and National Groups, and MARC 21 Bibliographic Format Field 083,” Cataloging & Classification Quarterly 47, no. 7 (2009): 657–70, https://doi.org/10.1080/01639370903112005 . 

37. Jonathan Furner, “Dewey Deracialized: A Critical Race-Theoretic Perspective,” KO Knowledge Organization 34, no. 3 (2007): 144–68, https://doi.org/10.5771/0943-7444-2007-3-144 . 

38. “About the Hispanic Population and its Origin,” U.S. Census , last modified April 12, 2022, https://www.census.gov/topics/population/hispanic-origin/about.html .

39. Maria P.P. Root,“A Bill of Rights for Racially Mixed People,” Embracing Diversity , last modified December 10, 2018, https://www.safehousealliance.org/wp-content/uploads/2012/10/A-Bill-of-Rights-for-Racially-Mixed-People.pdf ; Kristen A. Renn, “Research on Biracial and Multiracial Identity Development: Overview and Synthesis,” New Directions for Student Services no. 123 (2008): 13–21, https://doi.org/10.1002/ss.282 .

40. “Strategic Framework,” University Library, University of Illinois, last modified August 26, 2019, https://www.library.illinois.edu/geninfo/libraryinit/strategic-framework-2019/ .

41. J.A. Ferretti, “Neutrality is Polite Oppression: How Critical Librarianship and Pedagogy Principles Counter Neutral Narratives and Benefit the Profession, Critical Librarianship and Pedagogy Symposium (CLAPS), keynote presentation, Nov. 28, 2018, University of Arizona, Tucson, AZ; Anastasia Chiu, Fobazi M. Ettarh, and Jennifer A. Ferretti, “Not the Shark, But the Water: How Neutrality and Vocational Awe Intertwine to Uphold White Supremacy,” In Knowledge Justice: Disrupting Library and Information Studies through Critical Race Theory , eds, Sofia Y. Leung and Jorge R. López-McKnight (Cambridge: MIT Press, 2021), 49–71; Amelia N. Gibson et al., “Libraries on the Frontlines: Neutrality and Social Justice,” Equality, Diversity and Inclusion: An International Journal 36, no. 8 (2017): 751–66, https://doi.org/10.1108/EDI-11-2016-0100 ; Todd Honma. “Trippin’ Over the Color Line: The Invisibility of Race in Library and Information Studies,” InterActions: UCLA Journal of Education and Information Studies 1, no. 2 (2005), https://doi.org/10.5070/D412000540 . 

* María Evelia Emerson is Student Success Librarian and Assistant Professor at the University of Illinois Urbana-Champaign, email: [email protected] . ©2024 María Evelia Emerson, Attribution-NonCommercial ( https://creativecommons.org/licenses/by-nc/4.0/ ) CC BY-NC.

Article Views (Last 12 Months)

Contact ACRL for article usage statistics from 2010-April 2017.

Article Views (By Year/Month)

© 2024 Association of College and Research Libraries , a division of the American Library Association

Print ISSN: 0010-0870 | Online ISSN: 2150-6701

ALA Privacy Policy

ISSN: 2150-6701

• Open access
• Published: 02 July 2024

Family planning decision-making in relation to psychiatric disorders in women: a qualitative focus group study

• Shahenda A. I. H. Ahmad 1 ,
• Jorina Holtrop 1 ,
• Monique J. M. van den Eijnden 2 ,
• Nini H. Jonkman 3 ,
• Maria G. van Pampus 4 ,
• Odile A. van den Heuvel 5 , 6 , 7 ,
• Birit F. P. Broekman 1 , 5 , 8 &
• Noralie N. Schonewille 1 , 5 , 8  

Reproductive Health volume  21 , Article number:  96 ( 2024 ) Cite this article

93 Accesses

Metrics details

Recent studies revealed an elevated likelihood of unintended pregnancies among women with psychiatric disorders compared to their counterparts without such vulnerability. Despite the importance of understanding family planning decision-making in this group, qualitative inquiries are lacking. This study explored family planning decisions among women with psychiatric disorders.

Utilizing a qualitative approach, three focus group discussions were conducted with purposive sampling: women with a history of unintended pregnancies ( N  = 3), women without children ( N  = 5), and women with a history of intended pregnancies ( N  = 9), all of whom had self-reported psychiatric disorders. Using thematic framework analysis, we investigated the themes “Shadow of the past,” reflecting past experiences, and “Shadow of the future,” reflecting future imaginaries, building upon the existing “Narrative Framework.”

The Narrative Framework formed the foundation for understanding family planning among women with psychiatric disorders. The retrospective dimension of focus group discussions provided opportunities for reflective narratives on sensitive topics, revealing emotions of regret, grief and relief. Childhood trauma, adverse events, and inadequate parenting enriched the "Shadow of the past". The “Shadow of the present” was identified as a novel theme, addressing awareness of psychiatric disorders and emotions toward psychiatric stability. Social influences, stigma, and concerns about transmitting psychiatric disorders shaped future imaginaries in the shadow of the future.

Conclusions

This study enlightens how family planning decision-making in women with psychiatric disorders might be complex, as marked by the enduring impact of past experiences and societal influences in this sample. These nuanced insights underscore the necessity for tailored support for women with psychiatric disorders.

Plain English Summary

Recent studies show that women with psychiatric disorders are more likely to experience unintended pregnancies. However, the underlying reasons are not fully understood. Understanding those reasons is important to provide better healthcare. Our study explored how women with psychiatric disorders make decisions about family planning.

We had conversations with different groups of women—women with unintended pregnancies, women without children, and women with intended pregnancies—through focus group discussions. We partnered with the Dutch mental health organization MIND to capture diverse opinions. Key themes and categories in the discussions were identified and organized.

We found four main themes: "Shadow of the past" showed how past events, trauma, and lack of knowledge about parenting affect family planning. "Shadow of the present" revealed different feelings about family planning, the importance of the awareness of psychiatric disorders, and uncertainty about decisions. "Shadow of the future" included thoughts about becoming a mother, the impact of social influences, and concerns about passing on psychiatric disorders. "Reflections on the decision" showed how psychiatric disorders, experiences with motherhood, and feelings of regret, grief and relief had an influence on family planning decisions.

In conclusion, our study highlighted the complexity of family planning decisions for women with psychiatric disorders. Past experiences and societal influences, like stigma, play a big role. These insights show the need for personalized family planning support for women with psychiatric disorders.

Peer Review reports

Literature suggests that psychiatric disorders and family planning decision-making are related. Recent studies revealed that women with psychiatric disorders more often experience unintended pregnancies compared to counterparts without such vulnerability [ 1 , 2 , 3 ]. Moreover, childlessness is associated with having chronic illnesses, among which are psychiatric illnesses [ 4 , 5 ]. However, there is limited knowledge about contributing factors that shape family planning decision-making in women with psychiatric disorders [ 6 , 7 ].

Family planning decision-making defines the process through which individuals make choices about whether to have children, when to have them, and how many children to have [ 8 ]. It involves contemplating economic, social, cultural, and health-related factors [ 9 ]. Understanding family planning decision-making is fundamental for estimating the need for contraception, predicting reproductive patterns, and developing programs aimed at preventing unintended pregnancies [ 10 ]. The use of (emergency) contraceptives [ 11 ], improved accessibility to abortion services [ 12 ], and increased understanding of risk factors [ 11 , 13 , 14 ] have reduced unintended pregnancies. However, in 2010–2019, more than half of all pregnancies worldwide were still unintended [ 15 ].

Unintended pregnancies are particularly common among women with psychiatric disorders, reaching rates of up to 65% [ 5 , 16 ]. Aside from difficulties with planning, women with psychiatric disorders face elevated risks of psychiatric problems after pregnancy, leading to additional risks for adverse outcomes for both mothers and children [ 17 , 18 , 19 , 20 ]. The preconception phase is crucial for women planning to conceive by offering an opportunity to enhance nutrition and lifestyle choices to minimize maternal and child health risks [ 21 ]. Pregnancy planning is especially important for women with psychiatric disorders because they may need to make medication adjustments, take precautionary measures regarding the relapse of psychiatric disorders, and optimize mother–child attachment [ 17 , 18 , 19 , 20 ]. Indeed, women with unintended pregnancies encounter significant additional challenges due to the absence of the pregnancy planning phase, such as limited access to prenatal care, financial strain, and emotional stress [ 1 ].

Several frameworks exist for describing the factors that shape family planning decision-making [ 22 , 23 , 24 , 25 , 26 , 27 , 28 ]. However, most frameworks are past-driven and focus predominantly on cognitive factors [ 29 ]. We hypothesize that these frameworks are inadequate for capturing the uncertainty about the future faced by women with psychiatric disorders. The “Narrative Framework” provides a different perspective on family planning decision-making, particularly amid the amplified uncertainty and stress of the COVID-19 pandemic. It integrates past experiences, psychological predispositions, and socioeconomic factors, termed the “Shadow of the past,” alongside expectations, future imaginaries, and future narratives, termed the “Shadow of the future.” This framework captures decision-making processes by including these elements.

The aim of this qualitative study was to delve into the family planning experiences of women with psychiatric disorders (history of psychiatric disorder and/or current diagnosis). The “Narrative Framework” will provide a foundation for structuring the themes involved in the decision-making process [ 29 ]. The results of this study will contribute to knowledge about family planning decision-making in women with psychiatric disorders.

Materials and methods

Study design.

A qualitative study of experiences with family planning in women with self-reported psychiatric disorder(s) was conducted. We adopted a constructionist approach in which we acknowledged the role of society in shaping perspectives on family planning decision-making [ 30 ]. Therefore, focus group discussions were selected as the method of data collection because participants themselves can represent societal influences on each other’s perspectives.

Participant recruitment

Seventeen volunteers of the Dutch mental health umbrella organization MIND with self-reported psychiatric disorders were included. Participants were recruited from a sample of survey respondents ( n  = 378) from MIND [ 5 ]. The survey was conducted to collect quantitative and qualitative information about family planning. Participants were given the option to provide their email address if they wished to participate in a focus group discussion. Both men and women participated in the survey, but to address the current research question, only women were eligible ( n  = 17). Prior to participating in the focus groups, and after explanation of the study, all participants signed an informed consent form. Focus group discussion 1 ( n  = 3) consisted of women with a history of (initially) unintended pregnancies. Women who experienced an unintended pregnancy but did not remain pregnant (due to miscarriage or abortion) also participated in focus group discussion 1. Focus group discussion 2 ( n  = 5) consisted of women who did not have children and who had not been pregnant prior to participation. Focus group discussion 3 ( n  = 9) consisted of women with a history of intended pregnancies resulting in one or more children. Purposeful sampling created homogeneous focus group discussions regarding pregnancy intentions, ultimately benefiting the willingness of women to engage in discussions [ 31 ]. Due to the anonymous nature of the survey, it was unclear prior to the focus group discussions which psychiatric disorders the participants had been diagnosed with. Medical records were inaccessible; participants disclosed their psychiatric diagnoses during the focus group discussions. Figure  1 provides an overview of participant inclusion.

Flowchart of participant inclusion. Legend: this figure outlines the participant selection process for the study

Data collection and storage

Three focus group discussions were held at a central location in the Netherlands (organization MIND, Amersfoort, The Netherlands) between October and November 2021. The interview guide was created based on answers to an earlier survey and consisted of two main research questions, complemented by specific questions per focus group discussion: 1) How does your (history with) psychiatric disorder influence your desire for children? and 2) What is your experience discussing family planning with your mental health professional? The interview guide for each focus group is provided in Additional file 1 . The focus group discussions were held in Dutch and lasted between 120 and 155 min (median duration 123 min). A researcher with lived experience with perinatal mental health problems [ME] led the discussions, [NS] was present to observe and take field notes. Digital research data, including audio recordings and ad verbatim transcripts of the focus group discussions, were pseudonymized and stored in a password-protected file on a secure server of the hospital (OLVG). Paper consent forms are stored in a locked research cabinet of the same hospital. This process ensures that the data is securely stored, and that participants’ anonymity is protected throughout the study.

Data analysis

The focus group discussions were audio-recorded and transcribed ad-verbatim. The transcripts were converted to ATLAS.ti v9 for data analysis. Table 1 summarizes the steps performed during the analysis.

Methodological integrity

The research team, with backgrounds in psychiatry, obstetrics, neurosciences, and health behavior, ensured a foundation for conducting a nuanced and in-depth qualitative analysis of family planning decision-making, thereby incorporating triangulation. The epistemological approach was clearly stated and closely adhered to, which helped to align the research question with the applied methods. The conclusions were grounded in the evidence through the inclusion of quotations. Providing contextual information, such as the study setting and participant details, enhanced the comprehensibility of the results. Unlike a conventional consensus-reaching method, different interpretations were integrated into the findings to enrich the data analysis process. Utilizing methods of researcher reflexivity, such as memos and field notes, contributed to a reflexive and transparent analytical process. Reflexivity was considered throughout the process, acknowledging that [ME]’s background with lived experience with perinatal mental health problems may have influenced the discussions and interaction with participants, potentially fostering a more open and empathetic environment.

Information about the demographics of the participants ( n  = 17) is reported in Table  2 . Ages ranged between 24 and 70 years, with a median age of 57 years. All women had a Dutch background. The participants exhibited diverse occupational backgrounds, with six (35%) declaring themselves unfit for employment for reasons related to their psychiatric disorder(s). A history of pregnancy and psychiatric disorder(s) are described in Table  3 . Mood disorders were the predominant psychiatric disorder ( n  = 10), manifesting across all focus group discussions. Subsequently, trauma-related disorders ( n  = 8) and anxiety disorders ( n  = 6) were the most prevalent.

Our findings are presented within the context of an adapted version of the “Narrative Framework”. While the original framework highlights “ Shadow of the past ” and “ Shadow of the future ” as primary themes, our findings expanded this framework by incorporating two additional themes, namely “ Reflections on the decision ” and “ Shadow of the present ”, which were specifically tailored to our study population. This adaptation is illustrated in Fig.  2 . Each theme includes categories supported by quotations translated into English. Additional file 2 provides the original Dutch quotations.

Framework of family planning decision-making in women with psychiatric disorders. Legend: this figure shows the framework of family planning decision-making with part ( a ) illustrating the Narrative Framework and part ( b ) illustrating the adapted version tailored to women with psychiatric disorders. For each theme categories are listed on the right side of figure b. This figure is reproduced with permission from “Guetto R, Bazzani G, Vignoli D (2022) Narratives of the future and fertility decision-making in uncertain times. An application to the COVID-19 pandemic. Vienna Yearb Popul Res 20:223–260. https://www.jstor.org/stable/27222579 ”

Reflections on the decision

The retrospective nature of the focus group discussions allowed 17 women to delve into reflective narratives concerning sensitive topics regarding their psychiatric disorder(s) and family planning decision-making. The importance of these reflections, amplified by their emotionally charged nature, justifies the creation of a dedicated theme to them, as they were not yet addressed by the Narrative Framework [ 29 ]. Table 4  provides several quotations belonging to this theme.

Regret, grief and relief

Reflections on family planning decisions revealed a mix of emotions experienced by the participants. Participants without children often felt grief, yet some also found relief in their decision. This illustrates the complexity of emotions within individuals. The impact of participants' psychiatric disorder(s) on family planning was central in all focus group discussions. For some women, challenges in parenting due to personal circumstances brought feelings of grief and regret. However, positive reflections on motherhood also emerged, particularly from mothers who found fulfillment in motherhood despite initially unintended pregnancies.

Shadow of the past

Originally, the shadow of the past reflected the impact of life experiences, psychological predispositions, and socioeconomic factors on an individual’s decision about having children [ 29 ]. In our sample, this theme was enriched by the interplay between personal experiences in the women’s upbringing, the perceived parenting skills of their parents, and how those women respond to these experiences. Quotations are provided in Table  5 .

(Childhood) trauma and adverse life events

The participants of all focus group discussions spontaneously shared (childhood) trauma and adverse life events when asked about their family planning decision-making, revealing their perceived connection between these experiences and their family planning considerations. They believed that their upbringing impacted their decisions. Lived experience with trauma affected perspectives on family planning differently: some women were motivated to move past their childhood trauma by building a (better) family for themselves by having a child, while other women refrained from having children because of their adverse life events.

Inadequate parenting

This category showed how reflections on their own upbringing, specifically on their parents' parenting skills or the lack thereof, can influence women’s perspectives on having children. The decision to have children became a personal and reflective process shaped by the desire to replicate positive aspects of one's upbringing or breaking away from negative patterns and challenges.

Shadow of the present

In the Narrative Framework [ 29 ], (socioeconomic) status and personal perceptions bridged the gap between the shadow of the past and the shadow of the future. While socioeconomic factors such as financial considerations or the availability of a (suitable) partner contributed to the decision-making process of the participants, personal perceptions of their psychiatric disorder(s) were pertinent. This led to the extension of the framework with a novel theme: the shadow of the present. This theme incorporates categories relevant to our sample, including awareness of psychiatric disorder(s) and psychiatric stability, which were previously unaddressed in empirical models of family planning (Table  6 ).

Awareness of psychiatric disorder(s)

Awareness of psychiatric disorder(s) at the time of the decision was a recurrent theme among participants with children. Some participants mentioned the impact of not being aware of their psychiatric disorder at the time of the decision, indicating that they would have made different choices if they had been aware of the diagnosis earlier. For some of the participants, the diagnosis was liberating, explaining the challenges of motherhood.

Emotions toward psychiatric stability

Psychiatric stability at the time of decision-making was mentioned as one of the factors influencing choices. For some individuals, the stability of symptoms presented an opportunity to pursue parenthood, while for others, psychiatric stability did not hold the same level of deliberative weight. Diverse attitudes toward this issue highlight how some participants experienced resilience by learning from previous experiences, while others did not.

Perceptions of desire for children

A spectrum of diverse viewpoints on the desire for children was described as a complex array of thoughts, emotions, and perspectives that concurrently coexist, contributing to a fluctuating experience over time and giving rise to feelings of ambivalence and uncertainty. As one participant expressed “childbearing desire is not 100% yes or 100% no,” highlighting nuanced attitudes toward motherhood and childlessness. While ‘perceptions’ were originally emphasized as personal interpretations of past and current experiences [ 29 ] our participants argued that viewpoints on the desire for children change over time.

Opportunities and constraints for childbearing plans resemble the (socioeconomic) status element of the Narrative Framework [ 29 ]. This category includes several factors mentioned as reasons whether to have a child, including maternal age, financial stability or having a (stable) partner.

Shadow of the future

The shadow of the future emphasizes the importance of expectations and personal narratives in uncertain situations [ 29 ]. Our study shows how social influences, stigma around mental health, and uncertainty about passing on a condition can influence future imaginaries (Table  7 ).

Social influences

During the focus group discussions, the social system’s impact was heavily discussed. Many expressed frustrations with the lack of support from their social environment for their desire for children, leading to uncertainty about their decision. While personal visions of the future can be influenced by society, personal visions may also differ, thereby placing social influences in the shadow of the future [ 29 ]. Participants’ experiences with stigma due to their psychiatric disorder(s) often leaned toward deciding against having children.

Transmission of psychiatric disorder(s)

The transgenerational transmission of psychiatric disorder(s) to children was a key theme among participants, as discussed in all focus group discussions. Many participants were conscious of the risk of passing their condition to their offspring, which influenced their decisions against having children or causing regret if transmission occurred. Awareness of the challenges varied, with some participants doubting the possibility of breaking the transmission cycle, while others remained hopeful.

Prospect of motherhood

Insecurities about motherhood were deliberated. Participants felt incapable of raising a child for varying reasons, such as difficulty combining motherhood and having psychiatric symptoms. Other participants could rationalize this by referring to ‘other mothers’ without psychiatric disorder(s) who make parenting mistakes. Another participant stated that although her illness was heritable, it also aided her in supporting her child.

Key findings

This study has provided insights into family planning decision-making among women with psychiatric disorders by extending the Narrative Framework [ 29 ] with two themes. First, we dedicated a theme to reflections on decision-making, which encompasses emotions of grief, relief and regret. Second, we introduced the shadow of the present, which emphasized the impact of psychiatric disorders on decision-making by considering awareness of psychiatric disorders and psychiatric stability. Furthermore, the shadows of the past and future were broadened by integrating categories tailored to women with psychiatric disorders, including trauma, adverse life events, and social influences.

Interpretation in relation to literature

The retrospective nature of the focus group discussions allowed us to reflect on the participants’ family planning decisions, where the emotions of regret, grief and relief emerged. Regret over the delay in childbearing decisions has been described before in couples seeking fertility treatments [ 32 ]. Like our participants, voluntary childless women reported relief, feeling financially unburdened compared to their parenting peers, and enjoying various forms of freedom. However, they also faced stigmatization, and some expressed that their decision was influenced by their concern about potentially transmitting diseases to their children [ 33 ]. We hypothesize that cognitive dissonance, influenced by the type of psychiatric disorder, may contribute to regret in individuals as they grapple with conflicting thoughts and emotions [ 34 ].

Our study enhanced the shadow of the past with insights from 17 women with psychiatric disorder(s), shedding light on the impact of (childhood) trauma and adverse experiences on family planning decision-making. Previous research has shown an increased risk of unintended pregnancy in mothers with adverse childhood experiences [ 35 ]. Furthermore, women with unintended pregnancies reported more psychosocial problems [ 36 ]. Together with our findings, these findings imply that past experiences (related to psychiatric disorders) play a significant role in shaping family planning decisions and outcomes.

In the shadow of the present, we expanded upon the existing themes of (socioeconomic) status and perceptions [ 29 ]. Various enablers and constraints in the decision-making process surfaced, aligning with descriptions in other frameworks [ 22 , 23 , 25 , 26 ], and are therefore not unique to our population. The personal perceptions of our participants were portrayed as a complex array of emotions and thoughts, contributing to a fluctuating experience of family planning marked by ambivalence and uncertainty. Ambivalence toward motherhood in women with severe mental illness has been previously documented [ 37 ]. However, our study focused primarily on ambivalence in decision-making. Surprisingly, participants did not bring up the issue of psychoactive medication usage in relation to family planning. Although most psychotropic medication can be continued during pregnancy, some psychoactive medications can be teratogenic and should be used with caution [ 38 , 39 ]. Also, previous studies showed that (pregnant) women with psychiatric disorders contemplate their medication usage [ 40 ]. Overall, our findings suggest that family planning decision-making is more complex in women with psychiatric disorders than in those without psychiatric disorders, consistent with prior research [ 41 ]. A potential explanation lies in additional factors influencing the decision, such as awareness of the psychiatric disorder and psychiatric stability at the time of the decision.

The notion that social influences, including stigma, shape the shadow of the future through uncertainty is not limited to women with psychiatric disorders. A study on disabled women’s childbirth experiences revealed diverse reactions from their surroundings, leading to heightened fears and a sense of diminished control over their childbirth experiences [ 42 ]. Despite the difference in study populations, similar findings indicate a convergence in the experiences of women. While uncertainty about the future during the COVID-19 pandemic has been noted among the general population [ 28 , 29 ], we specifically examined uncertainty regarding stigma surrounding psychiatric disorders and their potential transmission. Participants' narratives may be influenced by maladaptive prospection seen in persons with psychiatric disorders such as depression and anxiety, which distorts future expectations [ 43 ]. Additionally, the potential of transmitting psychiatric disorders to their children might have intensified feelings of uncertainty about the future and thus made the decision-making process more challenging. This phenomenon is not novel and has been documented in various other hereditary diseases [ 44 , 45 ]. The actual extent of inheritance in psychiatric disorders significantly influences this dynamic. For instance, the estimated heritability for psychotic and neurodevelopmental disorders ranges from 74–85%, whereas for mood and anxiety disorders, it ranges between 37–58% [ 46 ]. The high heritability rate of these disorders aligns with the uncertainty as described in the narratives.

Strengths and limitations

This study provides a nuanced exploration of family planning decision-making in 17 women with psychiatric disorders. The transdiagnostic approach sheds light on overarching issues that were experienced. The use of focus group discussions captured societal dynamics and fostered an interactive environment for reflective perspectives [ 47 , 48 ]. Thematic framework analysis offered a structured examination of identified themes [ 29 ]. However, limitations include the small group of women with unintended pregnancies ( n  = 3) and the retrospective nature of reflections, potentially introducing recall bias [ 49 , 50 ]. Moreover, it is important to acknowledge the wide age range of participants, as this may affect the consistency of the data. We addressed the potential recall bias by incorporating the reflective nature of our framework and acknowledging its influence on the findings. The iterative process did not include respondent validation of the findings. To mitigate potential misinterpretations, we involved a researcher with lived experience with perinatal mental health problems.

Furthermore, the utilization of focus groups may restrict the depth of individual analyses. Although all our participants had been known with a psychiatric disorder according to the Diagnostic and Statistical Manual of Mental Disorders (DSM-5), we hypothesize that within the diverse array of disorders represented, each psychiatric disorder may have impacted family planning decisions in distinct ways. Additionally, the focus groups did not allow for an in-depth examination of the individual socioeconomic status of the women and how this influenced their reproductive desires. Given the established importance of socioeconomic status in the context of unintended pregnancies [ 14 ], it is crucial to consider this factor in individual sessions. Opting for individual interviews could provide a more comprehensive exploration.

Suggestions for future research

Future research could benefit from longitudinal and prospective study designs, allowing examination of family planning decision-making in women with psychiatric disorders considering the fluctuating aspect of family planning. Distinguishing between various psychiatric disorders and their unique impact on decision-making could provide a more nuanced understanding, possibly through individual in-depth interviews. As women with unintended pregnancies and psychiatric disorders may experience more challenges with family planning decision-making, it would be interesting to include these women in future research.

Our study sheds light on family planning decisions among women with psychiatric disorders. Like women without psychiatric disorders, past experiences, socioeconomic status, and perceptions on the desire for children shape decision-making. We found that traumatic events have a lasting impact on family planning choices. Stigma, uncertainty about parenting skills, and concerns about transmitting psychiatric disorder(s) contribute to ambivalence about having children. Feelings of regret, grief and relief regarding these decisions reflect the influence of psychiatric disorders. Our results emphasize that women with psychiatric disorders deserve support tailored to their needs, e.g. the possibility to discuss family planning at perinatal mental health facilities. Moreover, healthcare professionals could consider offering ongoing emotional support beyond the reproductive phase to those reflecting on their family planning decisions.

Availability of data and materials

The dataset(s) supporting the conclusions of this article are included within the article and the additional files. The original data are available upon reasonable request.

Enthoven CA, El MH, Koopman-Verhoeff ME, Jansen W, den Berg MPL, Sondeijker F, Hillegers MHJ, Bijma HH, Jansen PW. Clustering of characteristics associated with unplanned pregnancies: the generation R study. BMC Public Health. 2022;22(1):1957. https://doi.org/10.1186/s12889-022-14342-y .

Article   PubMed   PubMed Central   Google Scholar  

Schonewille NN, Rijkers N, Berenschot A, Lijmer JG, van den Heuvel OA, Broekman BFP. Psychiatric vulnerability and the risk for unintended pregnancies, a systematic review and meta-analysis. BMC Pregnancy Childb. 2022;22(1):153. https://doi.org/10.1186/s12884-022-04452-1 .

Article   CAS   Google Scholar  

Gariepy AM, Lundsberg LS, Miller D, Stanwood NL, Yonkers KA. Are pregnancy planning and pregnancy timing associated with maternal psychiatric illness, psychological distress and support during pregnancy? J Affect Disorders. 2016;205:87–94. https://doi.org/10.1016/j.jad.2016.06.058 .

Article   PubMed   Google Scholar  

Liu A, Akimova ET, Ding X, Jukarainen S, Vartiainen P, Kiiskinen T, Kuitunen S, Havulinna AS, Gissler M, Lombardi S, Fall T, Mills MC, Ganna A (2023) The relationship of early-life diseases with lifetime childlessness: Evidence from Finland and Sweden. MedRxiv. 2022.03.25.22272822. https://doi.org/10.1101/2022.03.25.22272822 .

Schonewille NN, van den Eijnden MJM, Jonkman NH, van Kempen AAMW, van Pampus MG, Goedhart FG, van den Heuvel OA, Broekman BFP. Experiences with Family Planning amongst Persons with Mental Health Problems: A Nationwide Patient Survey. Int J Environ Res Public Health. 2023;20(4):3070. https://doi.org/10.3390/ijerph20043070 .

Bagadia A, Nanjundaswamy MH, Ganjekar S, Thippeswamy H, Desai G, Chandra PS. Factors influencing decision-making around pregnancy among women with severe mental illness (SMI): A qualitative study. Int J Soc Psychiatrc. 2020;66(8):792–8. https://doi.org/10.1177/0020764020925104 .

Article   Google Scholar  

Becker T, Krumm S. Research on family planning issues in women with mental disorders. J Ment Health. 2006;15:383–5. https://doi.org/10.1080/09638230600801488 .

Santelli J, Rochat R, Hatfield-Timajchy K, Gilbert BC, Curtis K, Cabral R, Hirsch JS, Schieve L. The measurement and meaning of unintended pregnancy. Perspect Sex Repro H. 2003;35(2):94–101. https://doi.org/10.1363/3509403 .

Santelli J, Lindberg LD, Orr MG, Finer LB, Speizer I. Toward a multidimensional measure of pregnancy intentions: evidence from the United States. Stud Family Plann. 2009;40(2):87–100. https://doi.org/10.1111/j.1728-4465.2009.00192.x .

Klerman LV. The Intendedness of Pregnancy: A Concept in Transition. Matern Child Hlth J. 2000;4(3):155–62. https://doi.org/10.1023/A:1009534612388 .

Finer LB, Zolna MR. Declines in Unintended Pregnancy in the United States, 2008–2011. New Eng J Med. 2016;374(9):843–52. https://doi.org/10.1056/NEJMsa1506575 .

Article   CAS   PubMed   Google Scholar  

Bearak J, Popinchalk A, Alkema L, Sedgh G. Global, regional, and subregional trends in unintended pregnancy and its outcomes from 1990 to 2014: estimates from a Bayesian hierarchical model. Lancet Glob Health. 2018;6(4):e380–9. https://doi.org/10.1016/S2214-109X(18)30029-9 .

Goshu YA, Yitayew AE. Prevalence and determinant factors of unintended pregnancy among pregnant women attending antenatal clinics of Addis Zemen hospital. PLoS ONE. 2019;14(1):e0210206. https://doi.org/10.1371/journal.pone.0210206 .

Article   CAS   PubMed   PubMed Central   Google Scholar  

Thaxton L, Espey E. Family Planning American Style Redux: Unintended Pregnancy Improves, Barriers Remain. Obstet Gynecol Clin North Am. 2017;44(1):41–56. https://doi.org/10.1016/j.ogc.2016.11.001 .

Bearak J, Popinchalk A, Ganatra B, Moller AB, Tunçalp Ö, Beavin C, Kwok L, Alkema L. Unintended pregnancy and abortion by income, region, and the legal status of abortion: estimates from a comprehensive model for 1990–2019. Lancet Glob Health. 2020;8(9):e1152–61. https://doi.org/10.1016/S2214-109X(20)30315-6 .

Dossett EC, Castañeda-Cudney SL, Nguyen MT, Olgun M, Wang J, Myrick KJ, Hallmark L, Saks ER. Reproductive psychiatric advance directives: promoting autonomy for perinatal people with serious mental illness diagnoses. Arch Women Ment Hlth. 2023. https://doi.org/10.1007/s00737-023-01382-5 .

Kent A. Psychiatric disorders in pregnancy. Obstet Gynaecol Reprod Med. 2011;21(11):317–22. https://doi.org/10.1016/j.ogrm.2011.09.002 .

Miller L. Sexuality, reproduction, and family planning in women with schizophrenia. Schizophrenia Bull. 1997;23(4):623–35. https://doi.org/10.1093/schbul/23.4.623 .

Mohamed MA, Elhelbawy A, Khalid M, AbdAllatif LA, Lialy HE. Effects of bipolar disorder on maternal and fetal health during pregnancy: a systematic review. BMC Pregnancy Childb. 2023;23(1):617. https://doi.org/10.1186/s12884-023-05924-8 .

Viguera AC, Tondo L, Koukopoulos AE, Reginaldi D, Lepri B, Baldessarini RJ. Episodes of Mood Disorders in 2,252 Pregnancies and Postpartum Periods. Am J Psychiatry. 2011;168(11):1179–85. https://doi.org/10.1176/appi.ajp.2011.11010148 .

Stephenson J, Heslehurst N, Hall J, Schoenaker DAJM, Hutchinson J, Cade JE, Poston L, Barrett G, Crozier SR, Barker M, Kumaran K, Yajnik CS, Baird J, Mishra GD. Before the beginning: nutrition and lifestyle in the preconception period and its importance for future health. Lancet. 2018;391(10132):1830–41. https://doi.org/10.1016/S0140-6736(18)30311-8 .

Ajzen I. The theory of planned behavior. Organ Behav Hum. 1991;50(2):179–211. https://doi.org/10.1016/0749-5978(91)90020-T .

Ajzen I, Klobas J (2013) Fertility intentions: An approach based on the theory of planned behaviour. Demogr Res 29(8):203–232. https://www.demographic-research.org/special/16/8/.

Miller W. Personality traits and developmental experiences as antecedents of childbearing motivation. Demography. 1992;29(2):265–85.

Miller W. Childbearing motivations, desires, and intentions: a theoretical framework. Genet Soc Gen Psychol Monogr. 1994;120(2):223–58.

CAS   PubMed   Google Scholar  

Miller W, Severy L, Pasta D. A framework for modelling fertility motivation in couples. Pop Stud. 2004;58(2):193–205. https://doi.org/10.1080/0032472042000213712 .

Ganle JK, Apolot RR, Rugoho T, Sumankuuro J. ‘They are my future’: childbearing desires and motivations among women with disabilities in Ghana - implications for reproductive healthcare. Reprod Health. 2020;17(1):151. https://doi.org/10.1186/s12978-020-01000-y .

Vignoli D, Bazzani G, Guetto R, Minello A, Pirani E (2020) Uncertainty and Narratives of the Future: A Theoretical Framework for Contemporary Fertility. In: Schoen R (eds) Analyzing Contemporary Fertility, The Springer Series on Demographic Methods and Population Analysis, vol 51 Springer, Cham, pp 25–47.

Guetto R, Bazzani G, Vignoli D (2022) Narratives of the future and fertility decision-making in uncertain times. An application to the COVID-19 pandemic. Vienna Yearb Popul Res 20(1):223–260. https://econpapers.repec.org/RePEc:vid:yearbk:v:20:y:2022:i:1:oid:0x003d3ca9.

Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2006;3(2):77–101. https://doi.org/10.1191/1478088706qp063oa .

Palinkas LA, Horwitz SM, Green CA, Wisdom JP, Duan N, Hoagwood K. Purposeful Sampling for Qualitative Data Collection and Analysis in Mixed Method Implementation Research. Adm Policy Ment Health. 2015;42(5):533–44. https://doi.org/10.1007/s10488-013-0528-y .

Adachi T, Endo M, Ohashi K. Regret over the delay in childbearing decision negatively associates with life satisfaction among Japanese women and men seeking fertility treatment: A cross-sectional study. BMC Public Health. 2020;20(1):886. https://doi.org/10.1186/s12889-020-09025-5 .

Mollen D. Voluntarily Childfree Women: Experiences and Counseling Considerations. J Ment Health Couns. 2006;28(3):269–82. https://doi.org/10.17744/mehc.28.3.39w5h93mreb0mk4f .

Baumann T, Piesbergen C, Vant E, Tominschek I. Kognitive Dissonanz nach Entscheidungen und psychische Erkrankungen: Explorative Untersuchung von Patienten mit Zwangsstörung oder Depressionen. Psychotherapeut. 2014;59(2):130–7. https://doi.org/10.1007/s00278-013-0988-2 .

Testa A, Jackson DB, Ganson KT, Nagata JM. Maternal adverse childhood experiences and pregnancy intentions. Ann Epidemiol. 2021;64:47–52. https://doi.org/10.1016/j.annepidem.2021.09.011 .

Dijkstra CI, Dalmijn EW, Bolt SH, Groenhof F, Peters LL, Jansen DEMC. Women with unwanted pregnancies, their psychosocial problems, and contraceptive use in primary care in Northern Netherlands: insights from a primary care registry database. Fam Pract. 2023;40(5–6):648–54. https://doi.org/10.1093/fampra/cmad036 .

Banerjee D, Arasappa R, Chandra PS, Desai G. “It seemed like my fault for wanting to become a mother …”—Experiences and perceptions related to motherhood in women with severe mental illness. Asia-Pac Psychiat. 2022;14(4):e12519. https://doi.org/10.1111/appy.12519 .

Damkier P, Videbech P. The Safety of Second-Generation Antipsychotics During Pregnancy: A Clinically Focused Review. CNS Drugs. 2018;32(4):351–66. https://doi.org/10.1007/s40263-018-0517-5 .

Kautzky A, Slamanig R, Unger A, Höflich A. Neonatal outcome and adaption after in utero exposure to antidepressants: A systematic review and meta-analysis. Acta Psychiatr Scand. 2022;145(1):6–28. https://doi.org/10.1111/acps.13367 .

Stevens AWMM, Daggenvoorde TH, van der Klis SMD, Kupka RW, Goossens PJJ. Thoughts and Considerations of Women With Bipolar Disorder About Family Planning and Pregnancy: A Qualitative Study. J Am Psychiatr Nurses Assoc. 2018;24(2):118–26. https://doi.org/10.1177/1078390317711251 .

Ross NE, Webster TG, Tastenhoye CA, Hauspurg AK, Foust JE, Gopalan PR, Friedman SH. Reproductive Decision-Making Capacity in Women With Psychiatric Illness: A Systematic Review. J Acad Consult Liaison Psychiatry. 2022;63(1):61–70. https://doi.org/10.1016/j.jaclp.2021.08.007 .

Walsh-Gallagher D, Sinclair M, Conkey RM. The ambiguity of disabled women’s experiences of pregnancy, childbirth and motherhood: A phenomenological understanding. Midwifery. 2012;28(2):156–62. https://doi.org/10.1016/j.midw.2011.01.003 .

Seligman MEP, Railton P, Baumeister RF, Sripada C. Navigating Into the Future or Driven by the Past. Perspect Psychol Sci. 2013;8(2):119–41. https://doi.org/10.1177/1745691612474317 .

Severijns Y, de Die-Smulders CEM, Gültzow T, de Vries H, van Osch LADM. Hereditary diseases and child wish: exploring motives, considerations, and the (joint) decision-making process of genetically at-risk couples. J Community Genet. 2021;12(3):325–35. https://doi.org/10.1007/s12687-021-00510-x .

Fahy N, Rice C, Lahiri N, Desai R, Stott J. Genetic risk for Huntington Disease and reproductive decision-making: A systematic review. Clin Genet. 2023;104(2):147–62. https://doi.org/10.1111/cge.14345 .

Andreassen OA, Hindley GFL, Frei O, Smeland OB. New insights from the last decade of research in psychiatric genetics: discoveries, challenges and clinical implications. World Psychiatry. 2023;22(1):4–24. https://doi.org/10.1002/wps.21034 .

Krueger RA, Casey MA. Focus Groups: a practical guide for applied research. 4th ed. Thousand Oaks: SAGE Publications; 2009. p. 219. Available from: https://books.google.nl/books?id=ySKeSZeWywMC .

Tausch AP, Menold N. Methodological Aspects of Focus Groups in Health Research: Results of Qualitative Interviews With Focus Group Moderators. Glob Qual Nurs Res. 2016. https://doi.org/10.1177/2333393616630466 .

Guzzo KB, Hayford SR. Revisiting Retrospective Reporting of First-Birth Intendedness. Matern Child Hlth J. 2014;18(9):2141–7. https://doi.org/10.1007/s10995-014-1462-7 .

Chamberlin S, Njerenga S, Smith-Greenaway E, Yeatman S. Women’s Life Experiences and Shifting Reports of Pregnancy Planning. Matern Child Hlth J. 2022;26(8):1719–26. https://doi.org/10.1007/s10995-022-03447-w .

Download references

Acknowledgements

We express our gratitude to all the women who participated in the focus group discussions. Additionally, we extend our appreciation to Yente Dubbeldam for her contribution to data cleaning and analysis.

This research was funded by ZonMw, grant number 554002007.

Author information

Authors and affiliations.

Department of Psychiatry and Medical Psychology, OLVG, Oosterpark 9, Amsterdam, 1091 AC, The Netherlands

Shahenda A. I. H. Ahmad, Jorina Holtrop, Birit F. P. Broekman & Noralie N. Schonewille

Team Knowledge, Innovation and Research, MIND, Stationsplein 125, Amersfoort, 3818 LE, The Netherlands

Monique J. M. van den Eijnden

Department of Research and Epidemiology, OLVG, Oosterpark 9, Amsterdam, 1091 AC, The Netherlands

Nini H. Jonkman

Department of Gynecology and Obstetrics, OLVG, Oosterpark 9, Amsterdam, 1091 AC, The Netherlands

Maria G. van Pampus

Department of Psychiatry, Amsterdam UMC, Vrije Universiteit Amsterdam, Boelelaan 1117, Amsterdam, 1081 HV, The Netherlands

Odile A. van den Heuvel, Birit F. P. Broekman & Noralie N. Schonewille

Department of Anatomy and Neuroscience, Amsterdam UMC, Vrije Universiteit Amsterdam, Boelelaan 1117, Amsterdam, 1081 HV, The Netherlands

Odile A. van den Heuvel

Compulsivity, Impulsivity and Attention Program, Amsterdam Neuroscience, Amsterdam, The Netherlands

Amsterdam Public Health, Mental Health Program, Amsterdam UMC, Vrije Universiteit Amsterdam, Boelelaan 1117, Amsterdam, 1081 HV, The Netherlands

Birit F. P. Broekman & Noralie N. Schonewille

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization, funding, design and methodology, ME, BB and NS; focus group discussions, ME and NS; study and analytic design advice, NJ; formal analysis, all authors (SA, JH, ME, NJ, MP, OH, BB, NS); original draft preparation, SA and JH; review and editing, all authors (SA, JH, ME, NJ, MP, OH, BB, NS); visualization, SA; supervision, BB and NS.

Corresponding author

Correspondence to Noralie N. Schonewille .

Ethics declarations

Ethics approval and consent to participate.

The study was conducted in accordance with the Declaration of Helsinki and declared as not part of the Human Subjects Research Law by Medical Ethics Research Committee United Nieuwegein, protocol code W21.215, date of declaration 24–09-2021. Local approval was granted by OLVG Hospital. Written informed consent was obtained from all the subjects involved in the focus group discussions.

Consent for publication

All subjects involved in the focus groups have provided consent for publication.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1: this file provides the interview guides for the three focus group discussions., additional file 2: this file provides the original quotations for each theme per category in dutch language., rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Ahmad, S.A.I.H., Holtrop, J., van den Eijnden, M.J.M. et al. Family planning decision-making in relation to psychiatric disorders in women: a qualitative focus group study. Reprod Health 21 , 96 (2024). https://doi.org/10.1186/s12978-024-01836-8

Download citation

Received : 28 March 2024

Accepted : 24 June 2024

Published : 02 July 2024

DOI : https://doi.org/10.1186/s12978-024-01836-8

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Family planning
• Unintended pregnancies
• Psychiatric disorders
• Decision-making
• Qualitative research

Reproductive Health

ISSN: 1742-4755

• General enquiries: [email protected]

• Open access
• Published: 03 July 2024

The impact of evidence-based nursing leadership in healthcare settings: a mixed methods systematic review

• Maritta Välimäki 1 , 2 ,
• Shuang Hu 3 ,
• Tella Lantta 1 ,
• Kirsi Hipp 1 , 4 ,
• Jaakko Varpula 1 ,
• Jiarui Chen 3 ,
• Gaoming Liu 5 ,
• Yao Tang 3 ,
• Wenjun Chen 3 &
• Xianhong Li 3  

BMC Nursing volume  23 , Article number:  452 ( 2024 ) Cite this article

362 Accesses

Metrics details

The central component in impactful healthcare decisions is evidence. Understanding how nurse leaders use evidence in their own managerial decision making is still limited. This mixed methods systematic review aimed to examine how evidence is used to solve leadership problems and to describe the measured and perceived effects of evidence-based leadership on nurse leaders and their performance, organizational, and clinical outcomes.

We included articles using any type of research design. We referred nurses, nurse managers or other nursing staff working in a healthcare context when they attempt to influence the behavior of individuals or a group in an organization using an evidence-based approach. Seven databases were searched until 11 November 2021. JBI Critical Appraisal Checklist for Quasi-experimental studies, JBI Critical Appraisal Checklist for Case Series, Mixed Methods Appraisal Tool were used to evaluate the Risk of bias in quasi-experimental studies, case series, mixed methods studies, respectively. The JBI approach to mixed methods systematic reviews was followed, and a parallel-results convergent approach to synthesis and integration was adopted.

Thirty-one publications were eligible for the analysis: case series ( n  = 27), mixed methods studies ( n  = 3) and quasi-experimental studies ( n  = 1). All studies were included regardless of methodological quality. Leadership problems were related to the implementation of knowledge into practice, the quality of nursing care and the resource availability. Organizational data was used in 27 studies to understand leadership problems, scientific evidence from literature was sought in 26 studies, and stakeholders’ views were explored in 24 studies. Perceived and measured effects of evidence-based leadership focused on nurses’ performance, organizational outcomes, and clinical outcomes. Economic data were not available.

Conclusions

This is the first systematic review to examine how evidence is used to solve leadership problems and to describe its measured and perceived effects from different sites. Although a variety of perceptions and effects were identified on nurses’ performance as well as on organizational and clinical outcomes, available knowledge concerning evidence-based leadership is currently insufficient. Therefore, more high-quality research and clinical trial designs are still needed.

Trail registration

The study was registered (PROSPERO CRD42021259624).

Peer Review reports

Global health demands have set new roles for nurse leaders [ 1 ].Nurse leaders are referred to as nurses, nurse managers, or other nursing staff working in a healthcare context who attempt to influence the behavior of individuals or a group based on goals that are congruent with organizational goals [ 2 ]. They are seen as professionals “armed with data and evidence, and a commitment to mentorship and education”, and as a group in which “leaders innovate, transform, and achieve quality outcomes for patients, health care professionals, organizations, and communities” [ 3 ]. Effective leadership occurs when team members critically follow leaders and are motivated by a leader’s decisions based on the organization’s requests and targets [ 4 ]. On the other hand, problems caused by poor leadership may also occur, regarding staff relations, stress, sickness, or retention [ 5 ]. Therefore, leadership requires an understanding of different problems to be solved using synthesizing evidence from research, clinical expertise, and stakeholders’ preferences [ 6 , 7 ]. If based on evidence, leadership decisions, also referred as leadership decision making [ 8 ], could ensure adequate staffing [ 7 , 9 ] and to produce sufficient and cost-effective care [ 10 ]. However, nurse leaders still rely on their decision making on their personal [ 11 ] and professional experience [ 10 ] over research evidence, which can lead to deficiencies in the quality and safety of care delivery [ 12 , 13 , 14 ]. As all nurses should demonstrate leadership in their profession, their leadership competencies should be strengthened [ 15 ].

Evidence-informed decision-making, referred to as evidence appraisal and application, and evaluation of decisions [ 16 ], has been recognized as one of the core competencies for leaders [ 17 , 18 ]. The role of evidence in nurse leaders’ managerial decision making has been promoted by public authorities [ 19 , 20 , 21 ]. Evidence-based management, another concept related to evidence-based leadership, has been used as the potential to improve healthcare services [ 22 ]. It can guide nursing leaders, in developing working conditions, staff retention, implementation practices, strategic planning, patient care, and success of leadership [ 13 ]. Collins and Holton [ 23 ] in their systematic review and meta-analysis examined 83 studies regarding leadership development interventions. They found that leadership training can result in significant improvement in participants’ skills, especially in knowledge level, although the training effects varied across studies. Cummings et al. [ 24 ] reviewed 100 papers (93 studies) and concluded that participation in leadership interventions had a positive impact on the development of a variety of leadership styles. Clavijo-Chamorro et al. [ 25 ] in their review of 11 studies focused on leadership-related factors that facilitate evidence implementation: teamwork, organizational structures, and transformational leadership. The role of nurse managers was to facilitate evidence-based practices by transforming contexts to motivate the staff and move toward a shared vision of change.

As far as we are aware, however, only a few systematic reviews have focused on evidence-based leadership or related concepts in the healthcare context aiming to analyse how nurse leaders themselves uses evidence in the decision-making process. Young [ 26 ] targeted definitions and acceptance of evidence-based management (EBMgt) in healthcare while Hasanpoor et al. [ 22 ] identified facilitators and barriers, sources of evidence used, and the role of evidence in the process of decision making. Both these reviews concluded that EBMgt was of great importance but used limitedly in healthcare settings due to a lack of time, a lack of research management activities, and policy constraints. A review by Williams [ 27 ] showed that the usage of evidence to support management in decision making is marginal due to a shortage of relevant evidence. Fraser [ 28 ] in their review further indicated that the potential evidence-based knowledge is not used in decision making by leaders as effectively as it could be. Non-use of evidence occurs and leaders base their decisions mainly on single studies, real-world evidence, and experts’ opinions [ 29 ]. Systematic reviews and meta-analyses rarely provide evidence of management-related interventions [ 30 ]. Tate et al. [ 31 ] concluded based on their systematic review and meta-analysis that the ability of nurse leaders to use and critically appraise research evidence may influence the way policy is enacted and how resources and staff are used to meet certain objectives set by policy. This can further influence staff and workforce outcomes. It is therefore important that nurse leaders have the capacity and motivation to use the strongest evidence available to effect change and guide their decision making [ 27 ].

Despite of a growing body of evidence, we found only one review focusing on the impact of evidence-based knowledge. Geert et al. [ 32 ] reviewed literature from 2007 to 2016 to understand the elements of design, delivery, and evaluation of leadership development interventions that are the most reliably linked to outcomes at the level of the individual and the organization, and that are of most benefit to patients. The authors concluded that it is possible to improve individual-level outcomes among leaders, such as knowledge, motivation, skills, and behavior change using evidence-based approaches. Some of the most effective interventions included, for example, interactive workshops, coaching, action learning, and mentoring. However, these authors found limited research evidence describing how nurse leaders themselves use evidence to support their managerial decisions in nursing and what the outcomes are.

To fill the knowledge gap and compliment to existing knowledgebase, in this mixed methods review we aimed to (1) examine what leadership problems nurse leaders solve using an evidence-based approach and (2) how they use evidence to solve these problems. We also explored (3) the measured and (4) perceived effects of the evidence-based leadership approach in healthcare settings. Both qualitative and quantitative components of the effects of evidence-based leadership were examined to provide greater insights into the available literature [ 33 ]. Together with the evidence-based leadership approach, and its impact on nursing [ 34 , 35 ], this knowledge gained in this review can be used to inform clinical policy or organizational decisions [ 33 ]. The study is registered (PROSPERO CRD42021259624). The methods used in this review were specified in advance and documented in a priori in a published protocol [ 36 ]. Key terms of the review and the search terms are defined in Table  1 (population, intervention, comparison, outcomes, context, other).

In this review, we used a mixed methods approach [ 37 ]. A mixed methods systematic review was selected as this approach has the potential to produce direct relevance to policy makers and practitioners [ 38 ]. Johnson and Onwuegbuzie [ 39 ] have defined mixed methods research as “the class of research in which the researcher mixes or combines quantitative and qualitative research techniques, methods, approaches, concepts or language into a single study.” Therefore, we combined quantitative and narrative analysis to appraise and synthesize empirical evidence, and we held them as equally important in informing clinical policy or organizational decisions [ 34 ]. In this review, a comprehensive synthesis of quantitative and qualitative data was performed first and then discussed in discussion part (parallel-results convergent design) [ 40 ]. We hoped that different type of analysis approaches could complement each other and deeper picture of the topic in line with our research questions could be gained [ 34 ].

Inclusion and exclusion criteria

Inclusion and exclusion criteria of the study are described in Table  1 .

Search strategy

A three-step search strategy was utilized. First, an initial limited search with #MEDLINE was undertaken, followed by analysis of the words used in the title, abstract, and the article’s key index terms. Second, the search strategy, including identified keywords and index terms, was adapted for each included data base and a second search was undertaken on 11 November 2021. The full search strategy for each database is described in Additional file 1 . Third, the reference list of all studies included in the review were screened for additional studies. No year limits or language restrictions were used.

Information sources

The database search included the following: CINAHL (EBSCO), Cochrane Library (academic database for medicine and health science and nursing), Embase (Elsevier), PsycINFO (EBSCO), PubMed (MEDLINE), Scopus (Elsevier) and Web of Science (academic database across all scientific and technical disciplines, ranging from medicine and social sciences to arts and humanities). These databases were selected as they represent typical databases in health care context. Subject headings from each of the databases were included in the search strategies. Boolean operators ‘AND’ and ‘OR’ were used to combine the search terms. An information specialist from the University of Turku Library was consulted in the formation of the search strategies.

Study selection

All identified citations were collated and uploaded into Covidence software (Covidence systematic review software, Veritas Health Innovation, Melbourne, Australia www.covidence.org ), and duplicates were removed by the software. Titles and abstracts were screened and assessed against the inclusion criteria independently by two reviewers out of four, and any discrepancies were resolved by the third reviewer (MV, KH, TL, WC). Studies meeting the inclusion criteria were retrieved in full and archived in Covidence. Access to one full-text article was lacking: the authors for one study were contacted about the missing full text, but no full text was received. All remaining hits of the included studies were retrieved and assessed independently against the inclusion criteria by two independent reviewers of four (MV, KH, TL, WC). Studies that did not meet the inclusion criteria were excluded, and the reasons for exclusion were recorded in Covidence. Any disagreements that arose between the reviewers were resolved through discussions with XL.

Assessment of methodological quality

Eligible studies were critically appraised by two independent reviewers (YT, SH). Standardized critical appraisal instruments based on the study design were used. First, quasi-experimental studies were assessed using the JBI Critical Appraisal Checklist for Quasi-experimental studies [ 44 ]. Second, case series were assessed using the JBI Critical Appraisal Checklist for Case Series [ 45 ]. Third, mixed methods studies were appraised using the Mixed Methods Appraisal Tool [ 46 ].

To increase inter-reviewer reliability, the review agreement was calculated (SH) [ 47 ]. A kappa greater than 0.8 was considered to represent a high level of agreement (0–0.1). In our data, the agreement was 0.75. Discrepancies raised between two reviewers were resolved through discussion and modifications and confirmed by XL. As an outcome, studies that met the inclusion criteria were proceeded to critical appraisal and assessed as suitable for inclusion in the review. The scores for each item and overall critical appraisal scores were presented.

Data extraction

For data extraction, specific tables were created. First, study characteristics (author(s), year, country, design, number of participants, setting) were extracted by two authors independently (JC, MV) and reviewed by TL. Second, descriptions of the interventions were extracted by two reviewers (JV, JC) using the structure of the TIDIeR (Template for Intervention Description and Replication) checklist (brief name, the goal of the intervention, material and procedure, models of delivery and location, dose, modification, adherence and fidelity) [ 48 ]. The extractions were confirmed (MV).

Third, due to a lack of effectiveness data and a wide heterogeneity between study designs and presentation of outcomes, no attempt was made to pool the quantitative data statistically; the findings of the quantitative data were presented in narrative form only [ 44 ]. The separate data extraction tables for each research question were designed specifically for this study. For both qualitative (and a qualitative component of mixed-method studies) and quantitative studies, the data were extracted and tabulated into text format according to preplanned research questions [ 36 ]. To test the quality of the tables and the data extraction process, three authors independently extracted the data from the first five studies (in alphabetical order). After that, the authors came together to share and determine whether their approaches of the data extraction were consistent with each other’s output and whether the content of each table was in line with research question. No reason was found to modify the data extraction tables or planned process. After a consensus of the data extraction process was reached, the data were extracted in pairs by independent reviewers (WC, TY, SH, GL). Any disagreements that arose between the reviewers were resolved through discussion and with a third reviewer (MV).

Data analysis

We were not able to conduct a meta-analysis due to a lack of effectiveness data based on clinical trials. Instead, we used inductive thematic analysis with constant comparison to answer the research question [ 46 , 49 ] using tabulated primary data from qualitative and quantitative studies as reported by the original authors in narrative form only [ 47 ]. In addition, the qualitizing process was used to transform quantitative data to qualitative data; this helped us to convert the whole data into themes and categories. After that we used the thematic analysis for the narrative data as follows. First, the text was carefully read, line by line, to reveal topics answering each specific review question (MV). Second, the data coding was conducted, and the themes in the data were formed by data categorization. The process of deriving the themes was inductive based on constant comparison [ 49 ]. The results of thematic analysis and data categorization was first described in narrative format and then the total number of studies was calculated where the specific category was identified (%).

Stakeholder involvement

The method of reporting stakeholders’ involvement follows the key components by [ 50 ]: (1) people involved, (2) geographical location, (3) how people were recruited, (4) format of involvement, (5) amount of involvement, (6) ethical approval, (7) financial compensation, and (8) methods for reporting involvement.

In our review, stakeholder involvement targeted nurses and nurse leader in China. Nurse Directors of two hospitals recommended potential participants who received a personal invitation letter from researchers to participate in a discussion meeting. Stakeholders’ participation was based on their own free will. Due to COVID-19, one online meeting (1 h) was organized (25 May 2022). Eleven participants joined the meeting. Ethical approval was not applied and no financial compensation was offered. At the end of the meeting, experiences of stakeholders’ involvement were explored.

The meeting started with an introductory presentation with power points. The rationale, methods, and preliminary review results were shared with the participants [ 51 ].The meeting continued with general questions for the participants: (1) Are you aware of the concepts of evidence-based practice or evidence-based leadership?; (2) How important is it to use evidence to support decisions among nurse leaders?; (3) How is the evidence-based approach used in hospital settings?; and (4) What type of evidence is currently used to support nurse leaders’ decision making (e.g. scientific literature, organizational data, stakeholder views)?

Two people took notes on the course and content of the conversation. The notes were later transcripted in verbatim, and the key points of the discussions were summarised. Although answers offered by the stakeholders were very short, the information was useful to validate the preliminary content of the results, add the rigorousness of the review, and obtain additional perspectives. A recommendation of the stakeholders was combined in the Discussion part of this review increasing the applicability of the review in the real world [ 50 ]. At the end of the discussion, the value of stakeholders’ involvement was asked. Participants shared that the experience of participating was unique and the topic of discussion was challenging. Two authors of the review group further represented stakeholders by working together with the research team throughout the review study.

Search results

From seven different electronic databases, 6053 citations were identified as being potentially relevant to the review. Then, 3133 duplicates were removed by an automation tool (Covidence: www.covidence.org ), and one was removed manually. The titles and abstracts of 3040 of citations were reviewed, and a total of 110 full texts were included (one extra citation was found on the reference list but later excluded). Based on the eligibility criteria, 31 studies (32 hits) were critically appraised and deemed suitable for inclusion in the review. The search results and selection process are presented in the PRISMA [ 52 ] flow diagram Fig.  1 . The full list of references for included studies can be find in Additional file 2 . To avoid confusion between articles of the reference list and studies included in the analysis, the studies included in the review are referred inside the article using the reference number of each study (e.g. ref 1, ref 2).

Search results and study selection and inclusion process [ 52 ]

Characteristics of included studies

The studies had multiple purposes, aiming to develop practice, implement a new approach, improve quality, or to develop a model. The 31 studies (across 32 hits) were case series studies ( n  = 27), mixed methods studies ( n  = 3) and a quasi-experimental study ( n  = 1). All studies were published between the years 2004 and 2021. The highest number of papers was published in year 2020.

Table  2 describes the characteristics of included studies and Additional file 3 offers a narrative description of the studies.

Methodological quality assessment

Quasi-experimental studies.

We had one quasi-experimental study (ref 31). All questions in the critical appraisal tool were applicable. The total score of the study was 8 (out of a possible 9). Only one response of the tool was ‘no’ because no control group was used in the study (see Additional file 4 for the critical appraisal of included studies).

Case series studies . A case series study is typically defined as a collection of subjects with common characteristics. The studies do not include a comparison group and are often based on prevalent cases and on a sample of convenience [ 53 ]. Munn et al. [ 45 ] further claim that case series are best described as observational studies, lacking experimental and randomized characteristics, being descriptive studies, without a control or comparator group. Out of 27 case series studies included in our review, the critical appraisal scores varied from 1 to 9. Five references were conference abstracts with empirical study results, which were scored from 1 to 3. Full reports of these studies were searched in electronic databases but not found. Critical appraisal scores for the remaining 22 studies ranged from 1 to 9 out of a possible score of 10. One question (Q3) was not applicable to 13 studies: “Were valid methods used for identification of the condition for all participants included in the case series?” Only two studies had clearly reported the demographic of the participants in the study (Q6). Twenty studies met Criteria 8 (“Were the outcomes or follow-up results of cases clearly reported?”) and 18 studies met Criteria 7 (“Q7: Was there clear reporting of clinical information of the participants?”) (see Additional file 4 for the critical appraisal of included studies).

Mixed-methods studies

Mixed-methods studies involve a combination of qualitative and quantitative methods. This is a common design and includes convergent design, sequential explanatory design, and sequential exploratory design [ 46 ]. There were three mixed-methods studies. The critical appraisal scores for the three studies ranged from 60 to 100% out of a possible 100%. Two studies met all the criteria, while one study fulfilled 60% of the scored criteria due to a lack of information to understand the relevance of the sampling strategy well enough to address the research question (Q4.1) or to determine whether the risk of nonresponse bias was low (Q4.4) (see Additional file 4 for the critical appraisal of included studies).

Intervention or program components

The intervention of program components were categorized and described using the TiDier checklist: name and goal, theory or background, material, procedure, provider, models of delivery, location, dose, modification, and adherence and fidelity [ 48 ]. A description of intervention in each study is described in Additional file 5 and a narrative description in Additional file 6 .

Leadership problems

In line with the inclusion criteria, data for the leadership problems were categorized in all 31 included studies (see Additional file 7 for leadership problems). Three types of leadership problems were identified: implementation of knowledge into practice, the quality of clinical care, and resources in nursing care. A narrative summary of the results is reported below.

Implementing knowledge into practice

Eleven studies (35%) aimed to solve leadership problems related to implementation of knowledge into practice. Studies showed how to support nurses in evidence-based implementation (EBP) (ref 3, ref 5), how to engage nurses in using evidence in practice (ref 4), how to convey the importance of EBP (ref 22) or how to change practice (ref 4). Other problems were how to facilitate nurses to use guideline recommendations (ref 7) and how nurses can make evidence-informed decisions (ref 8). General concerns also included the linkage between theory and practice (ref 1) as well as how to implement the EBP model in practice (ref 6). In addition, studies were motivated by the need for revisions or updates of protocols to improve clinical practice (ref 10) as well as the need to standardize nursing activities (ref 11, ref 14).

The quality of the care

Thirteen (42%) focused on solving problems related to the quality of clinical care. In these studies, a high number of catheter infections led a lack of achievement of organizational goals (ref 2, ref 9). A need to reduce patient symptoms in stem cell transplant patients undergoing high-dose chemotherapy (ref 24) was also one of the problems to be solved. In addition, the projects focused on how to prevent pressure ulcers (ref 26, ref 29), how to enhance the quality of cancer treatment (ref 25) and how to reduce the need for invasive constipation treatment (ref 30). Concerns about patient safety (ref 15), high fall rates (ref 16, ref 19), dissatisfaction of patients (ref 16, ref 18) and nurses (ref 16, ref 30) were also problems that had initiated the projects. Studies addressed concerns about how to promote good contingency care in residential aged care homes (ref 20) and about how to increase recognition of human trafficking problems in healthcare (ref 21).

Resources in nursing care

Nurse leaders identified problems in their resources, especially in staffing problems. These problems were identified in seven studies (23%), which involved concerns about how to prevent nurses from leaving the job (ref 31), how to ensure appropriate recruitment, staffing and retaining of nurses (ref 13) and how to decrease nurses’ burden and time spent on nursing activities (ref 12). Leadership turnover was also reported as a source of dissatisfaction (ref 17); studies addressed a lack of structured transition and training programs, which led to turnover (ref 23), as well as how to improve intershift handoff among nurses (ref 28). Optimal design for new hospitals was also examined (ref 27).

Main features of evidence-based leadership

Out of 31 studies, 17 (55%) included all four domains of an evidence-based leadership approach, and four studies (13%) included evidence of critical appraisal of the results (see Additional file 8 for the main features of evidence-based Leadership) (ref 11, ref 14, ref 23, ref 27).

Organizational evidence

Twenty-seven studies (87%) reported how organizational evidence was collected and used to solve leadership problems (ref 2). Retrospective chart reviews (ref 5), a review of the extent of specific incidents (ref 19), and chart auditing (ref 7, ref 25) were conducted. A gap between guideline recommendations and actual care was identified using organizational data (ref 7) while the percentage of nurses’ working time spent on patient care was analyzed using an electronic charting system (ref 12). Internal data (ref 22), institutional data, and programming metrics were also analyzed to understand the development of the nurse workforce (ref 13).

Surveys (ref 3, ref 25), interviews (ref 3, ref 25) and group reviews (ref 18) were used to better understand the leadership problem to be solved. Employee opinion surveys on leadership (ref 17), a nurse satisfaction survey (ref 30) and a variety of reporting templates were used for the data collection (ref 28) reported. Sometimes, leadership problems were identified by evidence facilitators or a PI’s team who worked with staff members (ref 15, ref 17). Problems in clinical practice were also identified by the Nursing Professional Council (ref 14), managers (ref 26) or nurses themselves (ref 24). Current practices were reviewed (ref 29) and a gap analysis was conducted (ref 4, ref 16, ref 23) together with SWOT analysis (ref 16). In addition, hospital mission and vision statements, research culture established and the proportion of nursing alumni with formal EBP training were analyzed (ref 5). On the other hand, it was stated that no systematic hospital-specific sources of data regarding job satisfaction or organizational commitment were used (ref 31). In addition, statements of organizational analysis were used on a general level only (ref 1).

Scientific evidence identified

Twenty-six studies (84%) reported the use of scientific evidence in their evidence-based leadership processes. A literature search was conducted (ref 21) and questions, PICO, and keywords were identified (ref 4) in collaboration with a librarian. Electronic databases, including PubMed (ref 14, ref 31), Cochrane, and EMBASE (ref 31) were searched. Galiano (ref 6) used Wiley Online Library, Elsevier, CINAHL, Health Source: Nursing/Academic Edition, PubMed, and the Cochrane Library while Hoke (ref 11) conducted an electronic search using CINAHL and PubMed to retrieve articles.

Identified journals were reviewed manually (ref 31). The findings were summarized using ‘elevator speech’ (ref 4). In a study by Gifford et al. (ref 9) evidence facilitators worked with participants to access, appraise, and adapt the research evidence to the organizational context. Ostaszkiewicz (ref 20) conducted a scoping review of literature and identified and reviewed frameworks and policy documents about the topic and the quality standards. Further, a team of nursing administrators, directors, staff nurses, and a patient representative reviewed the literature and made recommendations for practice changes.

Clinical practice guidelines were also used to offer scientific evidence (ref 7, ref 19). Evidence was further retrieved from a combination of nursing policies, guidelines, journal articles, and textbooks (ref 12) as well as from published guidelines and literature (ref 13). Internal evidence, professional practice knowledge, relevant theories and models were synthesized (ref 24) while other study (ref 25) reviewed individual studies, synthesized with systematic reviews or clinical practice guidelines. The team reviewed the research evidence (ref 3, ref 15) or conducted a literature review (ref 22, ref 28, ref 29), a literature search (ref 27), a systematic review (ref 23), a review of the literature (ref 30) or ‘the scholarly literature was reviewed’ (ref 18). In addition, ‘an extensive literature review of evidence-based best practices was carried out’ (ref 10). However, detailed description how the review was conducted was lacking.

Views of stakeholders

A total of 24 studies (77%) reported methods for how the views of stakeholders, i.e., professionals or experts, were considered. Support to run this study was received from nursing leadership and multidisciplinary teams (ref 29). Experts and stakeholders joined the study team in some cases (ref 25, ref 30), and in other studies, their opinions were sought to facilitate project success (ref 3). Sometimes a steering committee was formed by a Chief Nursing Officer and Clinical Practice Specialists (ref 2). More specifically, stakeholders’ views were considered using interviews, workshops and follow-up teleconferences (ref 7). The literature review was discussed with colleagues (ref 11), and feedback and support from physicians as well as the consensus of staff were sought (ref 16).

A summary of the project findings and suggestions for the studies were discussed at 90-minute weekly meetings by 11 charge nurses. Nurse executive directors were consulted over a 10-week period (ref 31). An implementation team (nurse, dietician, physiotherapist, occupational therapist) was formed to support the implementation of evidence-based prevention measures (ref 26). Stakeholders volunteered to join in the pilot implementation (ref 28) or a stakeholder team met to determine the best strategy for change management, shortcomings in evidence-based criteria were discussed, and strategies to address those areas were planned (ref 5). Nursing leaders, staff members (ref 22), ‘process owners (ref 18) and program team members (ref 18, ref 19, ref 24) met regularly to discuss the problems. Critical input was sought from clinical educators, physicians, nutritionists, pharmacists, and nurse managers (ref 24). The unit director and senior nursing staff reviewed the contents of the product, and the final version of clinical pathways were reviewed and approved by the Quality Control Commission of the Nursing Department (ref 12). In addition, two co-design workshops with 18 residential aged care stakeholders were organized to explore their perspectives about factors to include in a model prototype (ref 20). Further, an agreement of stakeholders in implementing continuous quality services within an open relationship was conducted (ref 1).

Critical appraisal

In five studies (16%), a critical appraisal targeting the literature search was carried out. The appraisals were conducted by interns and teams who critiqued the evidence (ref 4). In Hoke’s study, four areas that had emerged in the literature were critically reviewed (ref 11). Other methods were to ‘critically appraise the search results’ (ref 14). Journal club team meetings (ref 23) were organized to grade the level and quality of evidence and the team ‘critically appraised relevant evidence’ (ref 27). On the other hand, the studies lacked details of how the appraisals were done in each study.

The perceived effects of evidence-based leadership

Perceived effects of evidence-based leadership on nurses’ performance.

Eleven studies (35%) described perceived effects of evidence-based leadership on nurses’ performance (see Additional file 9 for perceived effects of evidence-based leadership), which were categorized in four groups: awareness and knowledge, competence, ability to understand patients’ needs, and engagement. First, regarding ‘awareness and knowledge’, different projects provided nurses with new learning opportunities (ref 3). Staff’s knowledge (ref 20, ref 28), skills, and education levels improved (ref 20), as did nurses’ knowledge comprehension (ref 21). Second, interventions and approaches focusing on management and leadership positively influenced participants’ competence level to improve the quality of services. Their confidence level (ref 1) and motivation to change practice increased, self-esteem improved, and they were more positive and enthusiastic in their work (ref 22). Third, some nurses were relieved that they had learned to better handle patients’ needs (ref 25). For example, a systematic work approach increased nurses’ awareness of the patients who were at risk of developing health problems (ref 26). And last, nurse leaders were more engaged with staff, encouraging them to adopt the new practices and recognizing their efforts to change (ref 8).

Perceived effects on organizational outcomes

Nine studies (29%) described the perceived effects of evidence-based leadership on organizational outcomes (see Additional file 9 for perceived effects of evidence-based leadership). These were categorized into three groups: use of resources, staff commitment, and team effort. First, more appropriate use of resources was reported (ref 15, ref 20), and working time was more efficiently used (ref 16). In generally, a structured approach made implementing change more manageable (ref 1). On the other hand, in the beginning of the change process, the feedback from nurses was unfavorable, and they experienced discomfort in the new work style (ref 29). New approaches were also perceived as time consuming (ref 3). Second, nurse leaders believed that fewer nursing staff than expected left the organization over the course of the study (ref 31). Third, the project helped staff in their efforts to make changes, and it validated the importance of working as a team (ref 7). Collaboration and support between the nurses increased (ref 26). On the other hand, new work style caused challenges in teamwork (ref 3).

Perceived effects on clinical outcomes

Five studies (16%) reported the perceived effects of evidence-based leadership on clinical outcomes (see Additional file 9 for perceived effects of evidence-based leadership), which were categorized in two groups: general patient outcomes and specific clinical outcomes. First, in general, the project assisted in connecting the guideline recommendations and patient outcomes (ref 7). The project was good for the patients in general, and especially to improve patient safety (ref 16). On the other hand, some nurses thought that the new working style did not work at all for patients (ref 28). Second, the new approach used assisted in optimizing patients’ clinical problems and person-centered care (ref 20). Bowel management, for example, received very good feedback (ref 30).

The measured effects of evidence-based leadership

The measured effects on nurses’ performance.

Data were obtained from 20 studies (65%) (see Additional file 10 for measured effects of evidence-based leadership) and categorized nurse performance outcomes for three groups: awareness and knowledge, engagement, and satisfaction. First, six studies (19%) measured the awareness and knowledge levels of participants. Internship for staff nurses was beneficial to help participants to understand the process for using evidence-based practice and to grow professionally, to stimulate for innovative thinking, to give knowledge needed to use evidence-based practice to answer clinical questions, and to make possible to complete an evidence-based practice project (ref 3). Regarding implementation program of evidence-based practice, those with formal EBP training showed an improvement in knowledge, attitude, confidence, awareness and application after intervention (ref 3, ref 11, ref 20, ref 23, ref 25). On the contrary, in other study, attitude towards EBP remained stable ( p  = 0.543). and those who applied EBP decreased although no significant differences over the years ( p  = 0.879) (ref 6).

Second, 10 studies (35%) described nurses’ engagement to new practices (ref 5, ref 6, ref 7, ref 10, ref 16, ref 17, ref 18, ref 21, ref 25, ref 27). 9 studies (29%) studies reported that there was an improvement of compliance level of participants (ref 6, ref 7, ref 10, ref 16, ref 17, ref 18, ref 21, ref 25, ref 27). On the contrary, in DeLeskey’s (ref 5) study, although improvement was found in post-operative nausea and vomiting’s (PONV) risk factors documented’ (2.5–63%), and ’risk factors communicated among anaesthesia and surgical staff’ (0–62%), the improvement did not achieve the goal. The reason was a limited improvement was analysed. It was noted that only those patients who had been seen by the pre-admission testing nurse had risk assessments completed. Appropriate treatment/prophylaxis increased from 69 to 77%, and from 30 to 49%; routine assessment for PONV/rescue treatment 97% and 100% was both at 100% following the project. The results were discussed with staff but further reasons for a lack of engagement in nursing care was not reported.

And third, six studies (19%) reported nurses’ satisfaction with project outcomes. The study results showed that using evidence in managerial decisions improved nurses’ satisfaction and attitudes toward their organization ( P  < 0.05) (ref 31). Nurses’ overall job satisfaction improved as well (ref 17). Nurses’ satisfaction with usability of the electronic charting system significantly improved after introduction of the intervention (ref 12). In handoff project in seven hospitals, improvement was reported in all satisfaction indicators used in the study although improvement level varied in different units (ref 28). In addition, positive changes were reported in nurses’ ability to autonomously perform their job (“How satisfied are you with the tools and resources available for you treat and prevent patient constipation?” (54%, n  = 17 vs. 92%, n  = 35, p  < 0.001) (ref 30).

The measured effects on organizational outcomes

Thirteen studies (42%) described the effects of a project on organizational outcomes (see Additional file 10 for measured effects of evidence-based leadership), which were categorized in two groups: staff compliance, and changes in practices. First, studies reported improved organizational outcomes due to staff better compliance in care (ref 4, ref 13, ref 17, ref 23, ref 27, ref 31). Second, changes in organization practices were also described (ref 11) like changes in patient documentation (ref 12, ref 21). Van Orne (ref 30) found a statistically significant reduction in the average rate of invasive medication administration between pre-intervention and post-intervention ( p  = 0.01). Salvador (ref 24) also reported an improvement in a proactive approach to mucositis prevention with an evidence-based oral care guide. On the contrary, concerns were also raised such as not enough time for new bedside report (ref 16) or a lack of improvement of assessment of diabetic ulcer (ref 8).

The measured effects on clinical outcomes

A variety of improvements in clinical outcomes were reported (see Additional file 10 for measured effects of evidence-based leadership): improvement in patient clinical status and satisfaction level. First, a variety of improvement in patient clinical status was reported. improvement in Incidence of CAUTI decreased 27.8% between 2015 and 2019 (ref 2) while a patient-centered quality improvement project reduced CAUTI rates to 0 (ref 10). A significant decrease in transmission rate of MRSA transmission was also reported (ref 27) and in other study incidences of CLABSIs dropped following of CHG bathing (ref 14). Further, it was possible to decrease patient nausea from 18 to 5% and vomiting to 0% (ref 5) while the percentage of patients who left the hospital without being seen was below 2% after the project (ref 17). In addition, a significant reduction in the prevalence of pressure ulcers was found (ref 26, ref 29) and a significant reduction of mucositis severity/distress was achieved (ref 24). Patient falls rate decreased (ref 15, ref 16, ref 19, ref 27).

Second, patient satisfaction level after project implementation improved (ref 28). The scale assessing healthcare providers by consumers showed improvement, but the changes were not statistically significant. Improvement in an emergency department leadership model and in methods of communication with patients improved patient satisfaction scores by 600% (ref 17). In addition, new evidence-based unit improved patient experiences about the unit although not all items improved significantly (ref 18).

Stakeholder involvement in the mixed-method review

To ensure stakeholders’ involvement in the review, the real-world relevance of our research [ 53 ], achieve a higher level of meaning in our review results, and gain new perspectives on our preliminary findings [ 50 ], a meeting with 11 stakeholders was organized. First, we asked if participants were aware of the concepts of evidence-based practice or evidence-based leadership. Responses revealed that participants were familiar with the concept of evidence-based practice, but the topic of evidence-based leadership was totally new. Examples of nurses and nurse leaders’ responses are as follows: “I have heard a concept of evidence-based practice but never a concept of evidence-based leadership.” Another participant described: “I have heard it [evidence-based leadership] but I do not understand what it means.”

Second, as stakeholder involvement is beneficial to the relevance and impact of health research [ 54 ], we asked how important evidence is to them in supporting decisions in health care services. One participant described as follows: “Using evidence in decisions is crucial to the wards and also to the entire hospital.” Third, we asked how the evidence-based approach is used in hospital settings. Participants expressed that literature is commonly used to solve clinical problems in patient care but not to solve leadership problems. “In [patient] medication and care, clinical guidelines are regularly used. However, I am aware only a few cases where evidence has been sought to solve leadership problems.”

And last, we asked what type of evidence is currently used to support nurse leaders’ decision making (e.g. scientific literature, organizational data, stakeholder views)? The participants were aware that different types of information were collected in their organization on a daily basis (e.g. patient satisfaction surveys). However, the information was seldom used to support decision making because nurse leaders did not know how to access this information. Even so, the participants agreed that the use of evidence from different sources was important in approaching any leadership or managerial problems in the organization. Participants also suggested that all nurse leaders should receive systematic training related to the topic; this could support the daily use of the evidence-based approach.

To our knowledge, this article represents the first mixed-methods systematic review to examine leadership problems, how evidence is used to solve these problems and what the perceived and measured effects of evidence-based leadership are on nurse leaders and their performance, organizational, and clinical outcomes. This review has two key findings. First, the available research data suggests that evidence-based leadership has potential in the healthcare context, not only to improve knowledge and skills among nurses, but also to improve organizational outcomes and the quality of patient care. Second, remarkably little published research was found to explore the effects of evidence-based leadership with an efficient trial design. We validated the preliminary results with nurse stakeholders, and confirmed that nursing staff, especially nurse leaders, were not familiar with the concept of evidence-based leadership, nor were they used to implementing evidence into their leadership decisions. Our data was based on many databases, and we screened a large number of studies. We also checked existing registers and databases and found no registered or ongoing similar reviews being conducted. Therefore, our results may not change in the near future.

We found that after identifying the leadership problems, 26 (84%) studies out of 31 used organizational data, 25 (81%) studies used scientific evidence from the literature, and 21 (68%) studies considered the views of stakeholders in attempting to understand specific leadership problems more deeply. However, only four studies critically appraised any of these findings. Considering previous critical statements of nurse leaders’ use of evidence in their decision making [ 14 , 30 , 31 , 34 , 55 ], our results are still quite promising.

Our results support a previous systematic review by Geert et al. [ 32 ], which concluded that it is possible to improve leaders’ individual-level outcomes, such as knowledge, motivation, skills, and behavior change using evidence-based approaches. Collins and Holton [ 23 ] particularly found that leadership training resulted in significant knowledge and skill improvements, although the effects varied widely across studies. In our study, evidence-based leadership was seen to enable changes in clinical practice, especially in patient care. On the other hand, we understand that not all efforts to changes were successful [ 56 , 57 , 58 ]. An evidence-based approach causes negative attitudes and feelings. Negative emotions in participants have also been reported due to changes, such as discomfort with a new working style [ 59 ]. Another study reported inconvenience in using a new intervention and its potential risks for patient confidentiality. Sometimes making changes is more time consuming than continuing with current practice [ 60 ]. These findings may partially explain why new interventions or program do not always fully achieve their goals. On the other hand, Dubose et al. [ 61 ] state that, if prepared with knowledge of resistance, nurse leaders could minimize the potential negative consequences and capitalize on a powerful impact of change adaptation.

We found that only six studies used a specific model or theory to understand the mechanism of change that could guide leadership practices. Participants’ reactions to new approaches may be an important factor in predicting how a new intervention will be implemented into clinical practice. Therefore, stronger effort should be put to better understanding the use of evidence, how participants’ reactions and emotions or practice changes could be predicted or supported using appropriate models or theories, and how using these models are linked with leadership outcomes. In this task, nurse leaders have an important role. At the same time, more responsibilities in developing health services have been put on the shoulders of nurse leaders who may already be suffering under pressure and increased burden at work. Working in a leadership position may also lead to role conflict. A study by Lalleman et al. [ 62 ] found that nurses were used to helping other people, often in ad hoc situations. The helping attitude of nurses combined with structured managerial role may cause dilemmas, which may lead to stress. Many nurse leaders opt to leave their positions less than 5 years [ 63 ].To better fulfill the requirements of health services in the future, the role of nurse leaders in evidence-based leadership needs to be developed further to avoid ethical and practical dilemmas in their leadership practices.

It is worth noting that the perceived and measured effects did not offer strong support to each other but rather opened a new venue to understand the evidence-based leadership. Specifically, the perceived effects did not support to measured effects (competence, ability to understand patients’ needs, use of resources, team effort, and specific clinical outcomes) while the measured effects could not support to perceived effects (nurse’s performance satisfaction, changes in practices, and clinical outcomes satisfaction). These findings may indicate that different outcomes appear if the effects of evidence-based leadership are looked at using different methodological approach. Future study is encouraged using well-designed study method including mixed-method study to examine the consistency between perceived and measured effects of evidence-based leadership in health care.

There is a potential in nursing to support change by demonstrating conceptual and operational commitment to research-based practices [ 64 ]. Nurse leaders are well positioned to influence and lead professional governance, quality improvement, service transformation, change and shared governance [ 65 ]. In this task, evidence-based leadership could be a key in solving deficiencies in the quality, safety of care [ 14 ] and inefficiencies in healthcare delivery [ 12 , 13 ]. As WHO has revealed, there are about 28 million nurses worldwide, and the demand of nurses will put nurse resources into the specific spotlight [ 1 ]. Indeed, evidence could be used to find solutions for how to solve economic deficits or other problems using leadership skills. This is important as, when nurses are able to show leadership and control in their own work, they are less likely to leave their jobs [ 66 ]. On the other hand, based on our discussions with stakeholders, nurse leaders are not used to using evidence in their own work. Further, evidence-based leadership is not possible if nurse leaders do not have access to a relevant, robust body of evidence, adequate funding, resources, and organizational support, and evidence-informed decision making may only offer short-term solutions [ 55 ]. We still believe that implementing evidence-based strategies into the work of nurse leaders may create opportunities to protect this critical workforce from burnout or leaving the field [ 67 ]. However, the role of the evidence-based approach for nurse leaders in solving these problems is still a key question.

Limitations

This study aimed to use a broad search strategy to ensure a comprehensive review but, nevertheless, limitations exist: we may have missed studies not included in the major international databases. To keep search results manageable, we did not use specific databases to systematically search grey literature although it is a rich source of evidence used in systematic reviews and meta-analysis [ 68 ]. We still included published conference abstract/proceedings, which appeared in our scientific databases. It has been stated that conference abstracts and proceedings with empirical study results make up a great part of studies cited in systematic reviews [ 69 ]. At the same time, a limited space reserved for published conference publications can lead to methodological issues reducing the validity of the review results [ 68 ]. We also found that the great number of studies were carried out in western countries, restricting the generalizability of the results outside of English language countries. The study interventions and outcomes were too different across studies to be meaningfully pooled using statistical methods. Thus, our narrative synthesis could hypothetically be biased. To increase transparency of the data and all decisions made, the data, its categorization and conclusions are based on original studies and presented in separate tables and can be found in Additional files. Regarding a methodological approach [ 34 ], we used a mixed methods systematic review, with the core intention of combining quantitative and qualitative data from primary studies. The aim was to create a breadth and depth of understanding that could confirm to or dispute evidence and ultimately answer the review question posed [ 34 , 70 ]. Although the method is gaining traction due to its usefulness and practicality, guidance in combining quantitative and qualitative data in mixed methods systematic reviews is still limited at the theoretical stage [ 40 ]. As an outcome, it could be argued that other methodologies, for example, an integrative review, could have been used in our review to combine diverse methodologies [ 71 ]. We still believe that the results of this mixed method review may have an added value when compared with previous systematic reviews concerning leadership and an evidence-based approach.

Our mixed methods review fills the gap regarding how nurse leaders themselves use evidence to guide their leadership role and what the measured and perceived impact of evidence-based leadership is in nursing. Although the scarcity of controlled studies on this topic is concerning, the available research data suggest that evidence-based leadership intervention can improve nurse performance, organizational outcomes, and patient outcomes. Leadership problems are also well recognized in healthcare settings. More knowledge and a deeper understanding of the role of nurse leaders, and how they can use evidence in their own managerial leadership decisions, is still needed. Despite the limited number of studies, we assume that this narrative synthesis can provide a good foundation for how to develop evidence-based leadership in the future.

Implications

Based on our review results, several implications can be recommended. First, the future of nursing success depends on knowledgeable, capable, and strong leaders. Therefore, nurse leaders worldwide need to be educated about the best ways to manage challenging situations in healthcare contexts using an evidence-based approach in their decisions. This recommendation was also proposed by nurses and nurse leaders during our discussion meeting with stakeholders.

Second, curriculums in educational organizations and on-the-job training for nurse leaders should be updated to support general understanding how to use evidence in leadership decisions. And third, patients and family members should be more involved in the evidence-based approach. It is therefore important that nurse leaders learn how patients’ and family members’ views as stakeholders are better considered as part of the evidence-based leadership approach.

Future studies should be prioritized as follows: establishment of clear parameters for what constitutes and measures evidence-based leadership; use of theories or models in research to inform mechanisms how to effectively change the practice; conducting robust effectiveness studies using trial designs to evaluate the impact of evidence-based leadership; studying the role of patient and family members in improving the quality of clinical care; and investigating the financial impact of the use of evidence-based leadership approach within respective healthcare systems.

Data availability

The authors obtained all data for this review from published manuscripts.

World Health Organization. State of the world’s nursing 2020: investing in education, jobs and leadership. 2020. https://www.who.int/publications/i/item/9789240003279 . Accessed 29 June 2024.

Hersey P, Campbell R. Leadership: a behavioral science approach. The Center for; 2004.

Cline D, Crenshaw JT, Woods S. Nurse leader: a definition for the 21st century. Nurse Lead. 2022;20(4):381–4. https://doi.org/10.1016/j.mnl.2021.12.017 .

Article   Google Scholar  

Chen SS. Leadership styles and organization structural configurations. J Hum Resource Adult Learn. 2006;2(2):39–46.

Google Scholar  

McKibben L. Conflict management: importance and implications. Br J Nurs. 2017;26(2):100–3.

Article   PubMed   Google Scholar  

Haghgoshayie E, Hasanpoor E. Evidence-based nursing management: basing Organizational practices on the best available evidence. Creat Nurs. 2021;27(2):94–7. https://doi.org/10.1891/CRNR-D-19-00080 .

Majers JS, Warshawsky N. Evidence-based decision-making for nurse leaders. Nurse Lead. 2020;18(5):471–5.

Tichy NM, Bennis WG. Making judgment calls. Harvard Business Rev. 2007;85(10):94.

Sousa MJ, Pesqueira AM, Lemos C, Sousa M, Rocha Á. Decision-making based on big data analytics for people management in healthcare organizations. J Med Syst. 2019;43(9):1–10.

Guo R, Berkshire SD, Fulton LV, Hermanson PM. %J L in HS. Use of evidence-based management in healthcare administration decision-making. 2017;30(3): 330–42.

Liang Z, Howard P, Rasa J. Evidence-informed managerial decision-making: what evidence counts?(part one). Asia Pac J Health Manage. 2011;6(1):23–9.

Hasanpoor E, Janati A, Arab-Zozani M, Haghgoshayie E. Using the evidence-based medicine and evidence-based management to minimise overuse and maximise quality in healthcare: a hybrid perspective. BMJ evidence-based Med. 2020;25(1):3–5.

Shingler NA, Gonzalez JZ. Ebm: a pathway to evidence-based nursing management. Nurs 2022. 2017;47(2):43–6.

Farokhzadian J, Nayeri ND, Borhani F, Zare MR. Nurse leaders’ attitudes, self-efficacy and training needs for implementing evidence-based practice: is it time for a change toward safe care? Br J Med Med Res. 2015;7(8):662.

Article   PubMed   PubMed Central   Google Scholar  

American Nurses Association. ANA leadership competency model. Silver Spring, MD; 2018.

Royal College of Nursing. Leadership skills. 2022. https://www.rcn.org.uk/professional-development/your-career/nurse/leadership-skills . Accessed 29 June 2024.

Kakemam E, Liang Z, Janati A, Arab-Zozani M, Mohaghegh B, Gholizadeh M. Leadership and management competencies for hospital managers: a systematic review and best-fit framework synthesis. J Healthc Leadersh. 2020;12:59.

Liang Z, Howard PF, Leggat S, Bartram T. Development and validation of health service management competencies. J Health Organ Manag. 2018;32(2):157–75.

World Health Organization. Global Strategic Directions for Nursing and Midwifery. 2021. https://apps.who.int/iris/bitstream/handle/10665/344562/9789240033863-eng.pdf . Accessed 29 June 2024.

NHS Leadership Academy. The nine leadership dimensions. 2022. https://www.leadershipacademy.nhs.uk/resources/healthcare-leadership-model/nine-leadership-dimensions/ . Accessed 29 June 2024.

Canadian Nurses Association. Evidence-informed decision-making and nursing practice: Position statement. 2018. https://hl-prod-ca-oc-download.s3-ca-central-1.amazonaws.com/CNA/2f975e7e-4a40-45ca-863c-5ebf0a138d5e/UploadedImages/documents/Evidence_informed_Decision_making_and_Nursing_Practice_position_statement_Dec_2018.pdf . Accessed 29 June 2024.

Hasanpoor E, Hajebrahimi S, Janati A, Abedini Z, Haghgoshayie E. Barriers, facilitators, process and sources of evidence for evidence-based management among health care managers: a qualitative systematic review. Ethiop J Health Sci. 2018;28(5):665–80.

PubMed   PubMed Central   Google Scholar  

Collins DB, Holton EF III. The effectiveness of managerial leadership development programs: a meta-analysis of studies from 1982 to 2001. Hum Res Dev Q. 2004;15(2):217–48.

Cummings GG, Lee S, Tate K, Penconek T, Micaroni SP, Paananen T, et al. The essentials of nursing leadership: a systematic review of factors and educational interventions influencing nursing leadership. Int J Nurs Stud. 2021;115:103842.

Clavijo-Chamorro MZ, Romero-Zarallo G, Gómez-Luque A, López-Espuela F, Sanz-Martos S, López-Medina IM. Leadership as a facilitator of evidence implementation by nurse managers: a metasynthesis. West J Nurs Res. 2022;44(6):567–81.

Young SK. Evidence-based management: a literature review. J Nurs Adm Manag. 2002;10(3):145–51.

Williams LL. What goes around comes around: evidence-based management. Nurs Adm Q. 2006;30(3):243–51.

Fraser I. Organizational research with impact: working backwards. Worldviews Evidence-Based Nurs. 2004;1:S52–9.

Roshanghalb A, Lettieri E, Aloini D, Cannavacciuolo L, Gitto S, Visintin F. What evidence on evidence-based management in healthcare? Manag Decis. 2018;56(10):2069–84.

Jaana M, Vartak S, Ward MM. Evidence-based health care management: what is the research evidence available for health care managers? Eval Health Prof. 2014;37(3):314–34.

Tate K, Hewko S, McLane P, Baxter P, Perry K, Armijo-Olivo S, et al. Learning to lead: a review and synthesis of literature examining health care managers’ use of knowledge. J Health Serv Res Policy. 2019;24(1):57–70.

Geerts JM, Goodall AH, Agius S, %J SS. Medicine. Evidence-based leadership development for physicians: a systematic literature review. 2020;246: 112709.

Barends E, Rousseau DM, Briner RB. Evidence-based management: The basic principles. Amsterdam; 2014. https://research.vu.nl/ws/portalfiles/portal/42141986/complete+dissertation.pdf#page=203 . Accessed 29 June 2024.

Stern C, Lizarondo L, Carrier J, Godfrey C, Rieger K, Salmond S, et al. Methodological guidance for the conduct of mixed methods systematic reviews. JBI Evid Synthesis. 2020;18(10):2108–18. https://doi.org/10.11124/JBISRIR-D-19-00169 .

Lancet T. 2020: unleashing the full potential of nursing. Lancet (London, England). 2019. p. 1879.

Välimäki MA, Lantta T, Hipp K, Varpula J, Liu G, Tang Y, et al. Measured and perceived impacts of evidence-based leadership in nursing: a mixed-methods systematic review protocol. BMJ Open. 2021;11(10):e055356. https://doi.org/10.1136/bmjopen-2021-055356 .

The Joanna Briggs Institute. Joanna Briggs Institute reviewers’ manual: 2014 edition. Joanna Briggs Inst. 2014; 88–91.

Pearson A, White H, Bath-Hextall F, Salmond S, Apostolo J, Kirkpatrick P. A mixed-methods approach to systematic reviews. JBI Evid Implement. 2015;13(3):121–31.

Johnson RB, Onwuegbuzie AJ. Mixed methods research: a research paradigm whose time has come. Educational Researcher. 2004;33(7):14–26.

Hong, Pluye P, Bujold M, Wassef M. Convergent and sequential synthesis designs: implications for conducting and reporting systematic reviews of qualitative and quantitative evidence. Syst Reviews. 2017;6(1):61. https://doi.org/10.1186/s13643-017-0454-2 .

Ramis MA, Chang A, Conway A, Lim D, Munday J, Nissen L. Theory-based strategies for teaching evidence-based practice to undergraduate health students: a systematic review. BMC Med Educ. 2019;19(1):1–13.

Sackett DL, Rosenberg WM, Gray JM, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn’t. Bmj. British Medical Journal Publishing Group; 1996. pp. 71–2.

Goodman JS, Gary MS, Wood RE. Bibliographic search training for evidence-based management education: a review of relevant literatures. Acad Manage Learn Educ. 2014;13(3):322–53.

Aromataris E, Munn Z. Chapter 3: Systematic reviews of effectiveness. JBI Manual for Evidence Synthesis. 2020; https://synthesismanual.jbi.global .

Munn Z, Barker TH, Moola S, Tufanaru C, Stern C, McArthur A et al. Methodological quality of case series studies: an introduction to the JBI critical appraisal tool. 2020;18(10): 2127–33.

Hong Q, Pluye P, Fàbregues S, Bartlett G, Boardman F, Cargo M, et al. Mixed methods Appraisal Tool (MMAT) Version 2018: user guide. Montreal: McGill University; 2018.

McKenna J, Jeske D. Ethical leadership and decision authority effects on nurses’ engagement, exhaustion, and turnover intention. J Adv Nurs. 2021;77(1):198–206.

Maxwell M, Hibberd C, Aitchison P, Calveley E, Pratt R, Dougall N, et al. The TIDieR (template for intervention description and replication) checklist. The patient Centred Assessment Method for improving nurse-led biopsychosocial assessment of patients with long-term conditions: a feasibility RCT. NIHR Journals Library; 2018.

Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Res Psychol. 2006;3(2):77–101.

Pollock A, Campbell P, Struthers C, Synnot A, Nunn J, Hill S, et al. Stakeholder involvement in systematic reviews: a scoping review. Syst Reviews. 2018;7:1–26.

Braye S, Preston-Shoot M. Emerging from out of the shadows? Service user and carer involvement in systematic reviews. Evid Policy. 2005;1(2):173–93.

Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. Syst Reviews. 2021;10(1):1–11.

Porta M. Pilot investigation, study. A dictionary of epidemiology. Oxford University Press Oxford; 2014. p. 215.

Kreis J, Puhan MA, Schünemann HJ, Dickersin K. Consumer involvement in systematic reviews of comparative effectiveness research. Health Expect. 2013;16(4):323–37.

Joseph ML, Nelson-Brantley HV, Caramanica L, Lyman B, Frank B, Hand MW, et al. Building the science to guide nursing administration and leadership decision making. JONA: J Nurs Adm. 2022;52(1):19–26.

Gifford W, Davies BL, Graham ID, Tourangeau A, Woodend AK, Lefebre N. Developing Leadership Capacity for Guideline Use: a pilot cluster Randomized Control Trial: Leadership Pilot Study. Worldviews Evidence-Based Nurs. 2013;10(1):51–65. https://doi.org/10.1111/j.1741-6787.2012.00254.x .

Hsieh HY, Henker R, Ren D, Chien WY, Chang JP, Chen L, et al. Improving effectiveness and satisfaction of an electronic charting system in Taiwan. Clin Nurse Specialist. 2016;30(6):E1–6. https://doi.org/10.1097/NUR.0000000000000250 .

McAllen E, Stephens K, Swanson-Biearman B, Kerr K, Whiteman K. Moving Shift Report to the Bedside: an evidence-based Quality Improvement Project. OJIN: Online J Issues Nurs. 2018;23(2). https://doi.org/10.3912/OJIN.Vol23No02PPT22 .

Thomas M, Autencio K, Cesario K. Positive outcomes of an evidence-based pressure injury prevention program. J Wound Ostomy Cont Nurs. 2020;47:S24.

Cullen L, Titler MG. Promoting evidence-based practice: an internship for Staff nurses. Worldviews Evidence-Based Nurs. 2004;1(4):215–23. https://doi.org/10.1111/j.1524-475X.2004.04027.x .

DuBose BM, Mayo AM. Resistance to change: a concept analysis. Nursing forum. Wiley Online Library; 2020. pp. 631–6.

Lalleman PCB, Smid GAC, Lagerwey MD, Shortridge-Baggett LM, Schuurmans MJ. Curbing the urge to care: a bourdieusian analysis of the effect of the caring disposition on nurse middle managers’ clinical leadership in patient safety practices. Int J Nurs Stud. 2016;63:179–88.

Article   CAS   PubMed   Google Scholar  

Martin E, Warshawsky N. Guiding principles for creating value and meaning for the next generation of nurse leaders. JONA: J Nurs Adm. 2017;47(9):418–20.

Griffiths P, Recio-Saucedo A, Dall’Ora C, Briggs J, Maruotti A, Meredith P, et al. The association between nurse staffing and omissions in nursing care: a systematic review. J Adv Nurs. 2018;74(7):1474–87. https://doi.org/10.1111/jan.13564 .

Lúanaigh PÓ, Hughes F. The nurse executive role in quality and high performing health services. J Nurs Adm Manag. 2016;24(1):132–6.

de Kok E, Weggelaar-Jansen AM, Schoonhoven L, Lalleman P. A scoping review of rebel nurse leadership: descriptions, competences and stimulating/hindering factors. J Clin Nurs. 2021;30(17–18):2563–83.

Warshawsky NE. Building nurse manager well-being by reducing healthcare system demands. JONA: J Nurs Adm. 2022;52(4):189–91.

Paez A. Gray literature: an important resource in systematic reviews. J Evidence-Based Med. 2017;10(3):233–40.

McAuley L, Tugwell P, Moher D. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet. 2000;356(9237):1228–31.

Sarah S. Introduction to mixed methods systematic reviews. https://jbi-global-wiki.refined.site/space/MANUAL/4689215/8.1+Introduction+to+mixed+methods+systematic+reviews . Accessed 29 June 2024.

Whittemore R, Knafl K. The integrative review: updated methodology. J Adv Nurs. 2005;52(5):546–53.

Download references

Acknowledgements

We want to thank the funding bodies, the Finnish National Agency of Education, Asia Programme, the Department of Nursing Science at the University of Turku, and Xiangya School of Nursing at the Central South University. We also would like to thank the nurses and nurse leaders for their valuable opinions on the topic.

The work was supported by the Finnish National Agency of Education, Asia Programme (grant number 26/270/2020) and the University of Turku (internal fund 26003424). The funders had no role in the study design and will not have any role during its execution, analysis, interpretation of the data, decision to publish, or preparation of the manuscript.

Author information

Authors and affiliations.

Department of Nursing Science, University of Turku, Turku, FI-20014, Finland

Maritta Välimäki, Tella Lantta, Kirsi Hipp & Jaakko Varpula

School of Public Health, University of Helsinki, Helsinki, FI-00014, Finland

Maritta Välimäki

Xiangya Nursing, School of Central South University, Changsha, 410013, China

Shuang Hu, Jiarui Chen, Yao Tang, Wenjun Chen & Xianhong Li

School of Health and Social Services, Häme University of Applied Sciences, Hämeenlinna, Finland

Hunan Cancer Hospital, Changsha, 410008, China

Gaoming Liu

You can also search for this author in PubMed   Google Scholar

Contributions

Study design: MV, XL. Literature search and study selection: MV, KH, TL, WC, XL. Quality assessment: YT, SH, XL. Data extraction: JC, MV, JV, WC, YT, SH, GL. Analysis and interpretation: MV, SH. Manuscript writing: MV. Critical revisions for important intellectual content: MV, XL. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Xianhong Li .

Ethics declarations

Ethics approval and consent to participate.

No ethical approval was required for this study.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Differences between the original protocol

We modified criteria for the included studies: we included published conference abstracts/proceedings, which form a relatively broad knowledge base in scientific knowledge. We originally planned to conduct a survey with open-ended questions followed by a face-to-face meeting to discuss the preliminary results of the review. However, to avoid extra burden in nurses due to COVID-19, we decided to limit the validation process to the online discussion only.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, supplementary material 4, supplementary material 5, supplementary material 6, supplementary material 7, supplementary material 8, supplementary material 9, supplementary material 10, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Välimäki, M., Hu, S., Lantta, T. et al. The impact of evidence-based nursing leadership in healthcare settings: a mixed methods systematic review. BMC Nurs 23 , 452 (2024). https://doi.org/10.1186/s12912-024-02096-4

Download citation

Received : 28 April 2023

Accepted : 13 June 2024

Published : 03 July 2024

DOI : https://doi.org/10.1186/s12912-024-02096-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Evidence-based leadership
• Health services administration
• Organizational development
• Quality in healthcare

BMC Nursing

ISSN: 1472-6955

• General enquiries: [email protected]

Systematic Review: Autism Spectrum Disorder and the Gut Microbiota

Information & authors, metrics & citations, view options, conclusion:, introduction, search strategy, selection citeria, box 1. summations, limitations, study selection.

Study Characteristics

Alpha and beta diversity.

Specific Phyla and Genus Level Differences

Intervention Outcomes

Functional characteristics, conclusions, information, published in.

• autism spectrum disorders (ASD)
• gut microbiota
• gut-brain-axis
• systematic review

Affiliations

Competing interests, export citations.

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. Simply select your manager software from the list below and click Download. For more information or tips please see 'Downloading to a citation manager' in the Help menu .

To download the citation to this article, select your reference manager software.

There are no citations for this item

View options

Login options.

Already a subscriber? Access your subscription through your login credentials or your institution for full access to this article.

Not a subscriber?

Subscribe Now / Learn More

PsychiatryOnline subscription options offer access to the DSM-5-TR ® library, books, journals, CME, and patient resources. This all-in-one virtual library provides psychiatrists and mental health professionals with key resources for diagnosis, treatment, research, and professional development.

Need more help? PsychiatryOnline Customer Service may be reached by emailing [email protected] or by calling 800-368-5777 (in the U.S.) or 703-907-7322 (outside the U.S.).

Share article link

Copying failed.

PREVIOUS ARTICLE

Next article, request username.

Can't sign in? Forgot your username? Enter your email address below and we will send you your username

If the address matches an existing account you will receive an email with instructions to retrieve your username

Create a new account

Change password, password changed successfully.

Your password has been changed

Reset password

Can't sign in? Forgot your password?

Enter your email address below and we will send you the reset instructions

If the address matches an existing account you will receive an email with instructions to reset your password.

Your Phone has been verified

How is Value Defined in Molecular Testing in Cancer? A Scoping Review

• Scoping Review
• Open access
• Published: 09 July 2024

Cite this article

You have full access to this open access article

• Alice Minhinnick   ORCID: orcid.org/0000-0002-6032-1469 1 ,
• Francisco Santos-Gonzalez 2 ,
• Michelle Wilson 1 &
• Paula Lorgelly 1  

To identify how value is defined in studies that focus on the value of molecular testing in cancer and the extent to which broadening the conceptualisation of value in healthcare has been applied in the molecular testing literature.

A scoping review was undertaken using Joanna Briggs Institute (JBI) guidance. Medline, Embase, EconLit and Cochrane Library were searched in August 2023. Articles were eligible if they reported costs relative to outcomes, novel costs, or novel outcomes of molecular testing in cancer. Results were synthesised and qualitative content analysis was performed with deductive and inductive frameworks.

Ninety-one articles were included in the review. The majority (75/91) were conventional economic analyses (comparative economic evaluations and budget impact assessments) and undertaken from a healthcare system perspective (38/91). Clinical outcomes dominate the assessment of value (61/91), with quality-adjusted life-years (QALYs) the most common outcome measure (45/91). Other definitions of value were diverse (e.g. psychological impact, access to trials), inconsistent, and largely not in keeping with evolving guidance.

Conclusions

Broader concepts of value were not commonly described in the molecular testing literature focusing on cancer. Conventional approaches to measuring the health costs and outcomes of molecular testing in cancer prevail with little focus on non-clinical elements of value. There are emerging reports of non-clinical outcomes of testing information, particularly psychological consequences. Intrinsic attributes of the testing process and preferences of those who receive testing information may determine the realised societal value of molecular testing and highlight challenges to implementing such a value framework.

Avoid common mistakes on your manuscript.

1 Introduction

Molecular testing has a growing role in modern cancer care for diagnosing cancer, predicting responses to genetically targeted drugs, and estimating prognosis. With rapid advancement in the field of precision oncology, the presence of molecular testing in clinical guidelines and demand for molecular testing continues to rise [ 1 , 2 ]. As is the case for all technological advances in health, it is vital to understand how, where, when and to whom molecular tests provide value for money (i.e., in the economic sense, the necessary evidence base for the quantification of the incremental costs relative to the incremental outcomes). Health technology assessment (HTA) across the technology life cycle [ 3 ]—from molecular testing research and development, regulatory approval, pricing and reimbursement, to optimal clinical implementation—requires robust, readily communicated evidence on value to ensure optimal and equitable patient care.

However, existing evidence available to decision-makers on the costs and outcomes of molecular testing in cancer is limited by methodological challenges, including the lack of an agreed definition for value [ 4 ] and inconsistent approaches to quantifying value [ 5 , 6 , 7 , 8 , 9 , 10 , 11 , 12 ]. Terkola's editorial [ 4 ] summarised the key challenge in economic evaluation in precision medicine: A lack of high-quality data on costs and health outcomes underscored by the lack of a consensus definition of value.

Conventional approaches to measuring incremental costs and incremental benefits (e.g., the commonly used cost-utility analysis (CUA) with quality-adjusted life-year (QALY) [ 13 , 14 ]) have been criticised as not fit for purpose. This is especially the case in fields like precision oncology where diverse short- and long-term outcomes from cancer genomics information can benefit diverse stakeholders and resource utilisation subsequent and consequent to testing influence cost-effectiveness [ 10 , 15 ].

Recognition of a healthcare intervention's non-health benefits (beyond what can be presented by a QALY) hallmarks the contemporary broadening conceptualisation of value in healthcare [ 16 ] and the consequent attention to societal perspectives of value. For healthcare decision-making to represent preferences of society, requisite evidence generation must capture views on health and non-health outcomes [ 17 , 18 ] from a broad stakeholder group [ 16 ]. Actively involved in this discussion, the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) published their 2018 “value flower” [ 17 ], which summarises ordinarily underappreciated elements of value. The same narrative has extended to precision medicine and diagnostics, where recent health economics literature has invited a shift towards a more comprehensive definition of value [ 7 ]. For example, the Office of Health Economics (OHE) White Paper 'Improving the HTA of Complementary Diagnostics' called for "a more comprehensive perspective to include the less tangible benefits [of complementary diagnostics] [ 19 ]. Similarly, the Institute for Clinical and Economic Review’s (ICER) 2020–2023 Value Assessment Framework report suggested that "important modifications" to the evaluation of diagnostic tests are required to account for the distinct uses of diagnostic information, including explicit consideration of "other benefits, disadvantages and contextual considerations ” [ 20 ]. For molecular testing in cancer, where assessment has previously focused heavily on analytical validity, clinical validity and clinical utility [ 21 ], this means a shift towards the generation of evidence that includes non-health benefits such as personal utility [ 22 , 23 ] and process outcomes [ 15 ]. These more ‘novel’ elements of value [ 17 , 18 ] are not yet well integrated into healthcare decision-making (e.g., in Australia or the UK [ 24 , 25 ]), likely biasing previous resource allocation decisions. Similarly, investment in innovation may not have been well aligned with future societal value for money [ 26 ] . A recent systematic review on the cost-effectiveness of companion diagnostics for targeted cancer therapies [ 27 ] concluded that studies fail to observe factors that influence value beyond costs and sensitivity/specificity.

To what extent studies that focus on costs and outcomes of molecular testing in cancer reflect this shift toward a broader conceptualisation of value and the representation of value from a range of perspectives is unclear. Issues relating to social and health equity are pertinent, especially in Aotearoa New Zealand where there are stark and unjust existing inequities in health outcomes for indigenous Māori and Pacific peoples (e.g., Māori are 20% more likely to develop cancer than non-Māori and twice as likely to die from cancer [ 28 ]), and germane to molecular testing in cancer where access is notoriously variable. As such, incorporation of equity or viewpoints of indigenous communities in value definitions warrant attention.

This scoping review aims to identify how value is defined in studies that focus on the value of molecular testing in cancer with a key objective to map use of the more ‘novel’ elements of value.

The review sought to answer the following questions:

How is value defined in studies that focus on the value of molecular testing in cancer?

What elements (particularly novel elements) are considered?

What is the range of evidence that focuses on value (particularly novel elements) of molecular testing in cancer?

2.1 Choice of Review Methodology

Since the main purpose of the review was to identify and map definitions from within the available evidence, a scoping review was selected as the most appropriate approach [ 29 , 30 ]. The methods were informed by JBI guidance [ 30 ] and documented in an a priori protocol (review topic registered to the Open Science Framework; Footnote 1 the protocol is available through the corresponding author). Changes to the protocol, in line with the iterative nature of scoping reviews described in JBI’s population, concept and context framework guidelines [ 30 ], are described in Sects. 2.2.2 , 2.3 and 2.4 . The review was reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) checklist [ 31 ].

2.2 Scope and Eligibility Criteria

The Participant, Concept, Context approach for developing eligibility criteria was adopted.

2.2.1 Participant

There were no eligibility criteria relating to participants.

2.2.2 Concept

The concept being explored was the definitions of value used in studies focusing on the value of molecular testing in cancer. Value is a ubiquitous but inconsistently used term across healthcare and takes different meanings within clinical and economic domains. Common to most definitions of value are a comparison of resources used (costs) relative to outcomes (consequences or benefits) with respect to perspective and decision-making context. As such, ‘focusing on value’ was defined as the attention given to estimating costs, outcomes, or contextual considerations (referred to collectively as the determinants or elements of value). This was regardless of whether an explicit definition of value is given or implicitly inferred by reporting specific elements.

Consistent with reference to molecular testing across disciplines researching precision oncology (e.g., clinical, health economic, biosciences), the definition for molecular testing was kept broad to include its categorisation by clinical use (e.g., pharmacogenomics test, companion diagnostic), economic properties (e.g., complementary diagnostic), and technology type (e.g., immunohistochemistry, DNA sequencing, whole genome sequencing (WGS)). It is considered a process (as opposed to an event discrete in time or place), so a range of diagnostic or care pathways (e.g., cancer screening, risk stratification, treatment selection), including a molecular testing component, were considered relevant, provided there was sufficient focus on reporting the value of molecular testing. ‘Cancer’ was defined as solid organ or haematological malignancy in humans and not limited by tumour type, morphology or stage. Animal studies, in silico studies, and studies focused on pre-implantation diagnosis were excluded.

2.2.3 Context

Articles from any country and research discipline were eligible.

2.3 Sources of Evidence

The review sought to identify primary research studies, including experimental, observational and economic modelling studies published in the past 5 years. This aligned with the publication of the ISPOR’s Special Task Force’s recommendations on value frameworks [ 16 ].

Economic evaluation studies (comparative economic evaluations and budget impact assessments) were included, as well as qualitative, quantitative or mixed methods preference research, which a preliminary review [see Online Supplementary Materials (OSM)] had signposted as key evidence reporting information on personal utility.

Given the interest in identifying novel elements, the review sought to include studies estimating costs relative to outcomes, or costs or outcomes provided that at least one of these was non-clinical. Exclusion criteria were developed to avoid the large body of established literature reporting on molecular testing diagnostic accuracy, predictive or prognostic value, usually without comparison to costs. Although within scope, these types of studies were not included because the preliminary review demonstrated that diagnostic accuracy is an already well-recognised determinant of value for molecular testing in cancer, and inclusion would detract from mapping broader references to value. Similarly, we excluded the literature on microcosting of molecular testing in cancer covered in a recent systematic review [ 32 ].

Reports from health technology organisations in grey literature were considered for inclusion. Editorials, perspective pieces, reviews, abstracts, conference proceedings and protocols were excluded. The review was limited to full text articles published in English.

2.4 Search Strategy

The search strategy followed JBI’s three-step approach [ 33 ], starting with a limited search (Appendix I) in Embase. The preliminary review identified search terms to support the exclusion of diagnostic accuracy studies (e.g., ‘analytical utility’ and ‘positive predictive value’) and to increase the chance of including studies with a focus on non-clinical elements. Informed by analysis of relevant index terms and descriptors in the titles and abstracts of studies identified through the initial search (OSM), a detailed search strategy was developed with the assistance of a University of Auckland research librarian (Appendix II). One reviewer (AM) translated the search strategy of keywords, MeSH and Embase subject headings to undertake searches across Medline, Embase, EconLit and Cochrane Library on 15 August 2023. Grey literature was searched on websites from organisations listed by the International Network of Agencies for HTA. The reference lists of the most relevant articles were manually searched for additional potentially eligible studies [ 34 ].

2.5 Study Selection

Articles were imported into Covidence (Veritas Health Innovation, Melbourne, VIC, Australia) via EndNote X9.3.3 (Clarivate Analytics, PA, USA). Duplicates were removed before entering two-phased screening. Two reviewers (AM and FSG) independently screened titles and abstracts of articles identified through database searching for relevance to the inclusion criteria, and then the full-texts of articles deemed relevant were retrieved and examined for evidence pertaining to the review questions and outcomes of interest. During the two-phased screening, iterative eligibility criteria refinement was made through reviewers' discussions. These iterations aimed to increase the identification of novel value elements and respond to the ambiguity in determining the degree of focus on value of molecular testing (as opposed to the value of a molecularly informed intervention), i.e., the initial definition of ‘focus on value’ proved insufficiently detailed for the attention of this review to identify more novel value elements. Key eligibility criteria refinements were:

Include studies with focus on how testing procedures are implemented, acknowledging that these represent value elements influencing the realised value of molecular testing in cancer.

Include studies even if unclear whether the molecular testing or the molecularly informed intervention has the greater focus, provided there is a clear description of a molecular testing component within a studied intervention.

Exclude studies that primarily aim to estimate the impact of changing a testing eligibility criterion (e.g., expanding the eligible population for population screening where the screening intervention includes molecular testing).

Conflicts arose in the title and abstract and full-text screening processes, as the central concept of focus on value of molecular testing in cancer is not well defined or articulated in the literature. Reasons for excluding full-texts were documented and discrepancies resolved through discussion with two other authors (PL and MW) if necessary, with a return to the definition of value outlined in this article’s introduction.

2.6 Data Extraction

One reviewer (AM) extracted study characteristics and information relating to review questions from the included articles using a pre-defined standard data extraction table developed in Microsoft Word (Appendix III) and translated into Covidence. The extracted information included data pertaining to bibliographic data, information relating to the molecular test, analysis methodology, and value definitions. For value definitions, explicit or implicit definitions were extracted in textual form, with implicit definitions considered the mention of a cost, outcome or contextual consideration from outside article discussions. Value outcome measures (e.g., QALYs) were extracted.

Value elements were first mapped to a binary yes/no for the inclusion of non-clinical outcomes. Non-clinical outcomes were mapped against the ‘non-core’ elements of value of health technologies as depicted in ISPOR Special Task Force’s Elements of Value [ 17 ] (the earliest and most well-known value framework), and the Office of Health Economics (OHE) ‘non-traditional’ elements of value for complementary diagnostics [ 19 ] (Fig. 1 ). ISPOR’s Elements of Value is the earliest and most well-known of the value frameworks. The OHE framework was selected because it was developed in a different country (UK) and specifically for diagnostics. Textual information of novel costs, benefits or contextual considerations was also recorded. Two reviewers (AM and FSG) piloted the extraction form and modified iteratively to ensure consistency and accuracy. Alterations included clarifying extracted date as the earliest date of publication and country of publication as country of ‘focus’. Some extraction fields proved too subjective (e.g., decision-making context and clinical application) and not suitable for descriptive frequency counts. Similarly, molecular test technology was not clearly defined within studies and so could not be sub-categorised for analysis. Ambiguity of interpretation of study reference to ISPOR or OHE value elements was resolved through discussion and returning to the Special Task Force paper and related references [ 16 , 35 , 36 ]. For example, in the context of cascade screening of relatives it was agreed that outcomes for modelled relatives did not fulfil the definition of family spillover, which relates to the impact that illness can have on informal/unpaid caregivers/family members [ 35 ]. Value of knowing and value of reduction in uncertainty were considered mutually inclusive [ 19 , 37 , 38 ]. The second reviewer (FSG) cross-checked completeness and accuracy of the extracted data of 10% of the articles where clinical outcomes only were identified and 10% of the articles where novel elements were identified (to weight the cross-checking onto articles of greater focus for analysis).

Selected frameworks for mapping of non-clinical outcomes. Left: ‘Non-core’ elements of value of health technologies as depicted in International Society for Pharmacoeconomics and Outcomes Research (ISPOR) Special Task Force’s Elements of Value [ 17 ]. Note: Green circles: core elements of value; light blue circles: common but inconsistently used elements of value; dark blue circles: potential novel elements of value; blue line: value element included in traditional payer or health plan perspective; red line: value element also included in societal perspective. Right: Office of Health Economics ‘non-traditional’ elements of value for complementary diagnostics [ 19 ]. Note: Light grey circle: traditional elements of value as considered by health technology assessment (HTA); dark grey circle: expanded value framework (elements not traditionally considered/measured); green line: value from health system perspective; red line: value also included in societal perspective

2.7 Data Synthesis

Descriptive statistics using frequency counts were conducted for study characteristics information to address the mapping of available evidence that focuses on value of molecular testing in cancer objective. Value outcome measures were categorised as ‘conventional economic’ (QALYs, life-years (LYs), disability-adjusted life-years (DALYs), other natural units or a monetary measure (e.g., $)), or ‘non-economic’ and presented with descriptive statistics. A descriptive qualitative context analysis approach was used to collate the extracted value definitions textual data [ 33 ]. The degree to which the available literature has addressed previously proposed novel value elements was made deductively using frequency counts of references (excluding cites in the Discussion section) to value elements from the selected OHE and ISPOR model frameworks. Collaborative and iterative development of an inductive framework to further synthesise the evidence pertaining to novel value definitions was required for those outcomes not readily mapped to the OHE and ISPOR frameworks. The reviewer who had performed the main extraction (AM) undertook open-coding using NVivo (v 14.23.2, QSR International). AM shared these thoughts with the other authors and incorporated their reflections into developing initial categories to describe the extracted data [ 33 ]. After re-presentation of these categories and further iterative modification, a coding framework for organising the data was agreed.

Data are summarised according to review questions using tables, figures and narrative synthesis.

3.1 Search Results

The four database searches returned a total of 1629 results and five articles were identified through grey literature. After removing 556 duplicates, titles and abstracts of the remaining 1086 articles were screened for relevance to the inclusion criteria. Screening resulted in the exclusion of 923 irrelevant articles, and the remaining 163 progressed to full-text review. Of these, 72 were excluded for reasons documented in the PRISMA diagram (Fig. 2 ). Eight relevant articles were identified through hand-searching. In total 91 articles met eligibility criteria for this review.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flowchart of included studies

3.2 Characteristics of Included Articles

The main article characteristics are depicted in Table 1 . These comprised 75/91 (82.42%) conventional economic analyses (with the majority cost-effectiveness (CEA) or CUA) and 16/91 (17.58%) other study types (e.g., mixed methods or qualitative studies).

Most molecular testing scenarios considered within articles were based in the USA, followed by Australia, Canada, the UK, Spain, then China (Fig. 3 ). The most examined perspective was the healthcare system, followed by third-party payer. When accounting for studies that considered multiple perspectives, the 91 articles included 102 perspectives, including 12 patient, nine societal, and five public perspectives.

Map showing geographical distribution of article count by country of focus

There were no observed changes over time in study characteristics (not presented).

3.3 How is Value Defined?

3.3.1 inclusion of non-clinical outcomes.

A minority (30/91, 33.0%) of articles considered non-clinical outcomes. Two-thirds of articles mentioned clinical outcomes alone. Of the conventional economic analyses, the proportion of studies considering non-clinical outcomes was even less (61/75, 81.3% vs. 14/75, 18.7%). There has been a recent observed increase in the inclusion of non-clinical outcomes within conventional economic analyses (Fig. 4 ).

Inclusion of non-clinical outcomes within conventional economic analyses over time. Note: Conventional economic analyses included comparative economic evaluations and budget impact assessments

3.3.2 Outcome Measures

Most studies used conventional economic outcome measures to define value; in these economic evaluations cost per QALY was the sole outcome measure in half (45/91) of included articles. Several studies (17/91) used combinations of conventional economic outcome measures to report value, predominantly QALYs or LYs with a natural unit. Five articles presented LYs, and one study presented DALYs alone.

In the 16 articles that were not conventional economic analyses (e.g., qualitative studies, discrete choice experiments (DCEs)), the most common sole outcome was willingness to pay (WTP) (5/16, 31.3%). Other outcomes included attitudes towards, experiences of, interest in, knowledge and understanding of, and preferences for implementation strategies of molecular testing in cancer.

3.3.3 Mapping of Value Outcomes to ISPOR Non-core Value Elements and Office of Health Economics (OHE) Non-traditional Value Elements

Value of knowing (ISPOR) and reduction in uncertainty (OHE) were commonly identified novel elements, identified 22 times within included articles. Other ISPOR non-core value elements were also identified: equity ( n = 10 articles), productivity ( n = 9), value of hope ( n = 6), family spillover ( n = 6), severity of illness ( n = 5), scientific spillover ( n = 3), insurance value ( n = 2) and real option value ( n = 2). The OHE non-traditional value elements were identified less often: value of hope ( n = 6 articles), cost-savings outside of the health system ( n = 3), scientific spillover ( n = 3), insurance value ( n = 2) and real option value ( n = 2). This mapping of value outcomes is shown in Fig. 5 .

Mapping of value outcomes to International Society for Pharmacoeconomics and Outcomes Research (ISPOR) non-core value elements and Office of Health Economics (OHE) non-traditional value elements

3.3.4 Non-clinical Elements of Value

Numerous novel (not conventionally reported clinically related) elements were identified through this review (a proportion of which could be mapped to OHE and ISPOR frameworks). It became evident, through the inductive approach, that these could be grouped into: (1) attributes of tests highly relevant to molecular testing in cancer and not ordinarily considered in conventional economic analyses (herein referred to as ‘novel test attributes’); (2) attributes of the clinical testing indication; (3) intrinsic attributes of people involved with testing (stakeholders) (herein referred to as ‘people attributes’); (4) non-clinical outcomes of testing; and (5) clinical outcomes of testing highly relevant to molecular testing and not ordinarily considered in conventional economic analyses (herein referred to as ‘novel clinical outcomes’). These five categories are presented within an overall framework (Fig. 6 ). While some elements may fit more than one category, each element was placed into one category determined to be the ‘best-fit’ for the purposes of presenting these data.

Overall framework demonstrating grouping of novel value elements identified from included articles developed through inductive analysis. Note: Equity was implicitly considered across other groups, with the majority of examples relating to people attributes (examination of the heterogeneity of preferences for, experiences of, or willingness to pay for molecular testing), novel test attributes, and indication attributes. Elements of value within these five groups were considered at different levels: individual, family, community, health system and society

3.3.4.1 Novel Test Attributes

Many articles sought to examine attributes intrinsic to properties of the molecular testing. These included reference to the clinical purpose for the test (e.g., screening, companion diagnostic) [ 39 ] with suggestion—especially from the patient perspective—that tests informing of treatment choice are of greater importance than others. For example, in one article “ Finding a new treatment was the overriding motivation… [authors comment that]… identification of a germline variant were generally seen as ancillary ” [ 39 ]. While breadth of test result information was considered in another study that examined patient and clinician preference for receiving test results with or without germline information [ 40 ].

Others examined more operational attributes relating to the test process , such as biopsy requirements (e.g., number [ 41 ], ability to use historical tissue [ 42 ] or nature/invasiveness [ 40 ] and the related issue of tissue preservation [ 40 ]), role/extent of healthcare professional (HCP) involvement for pre-test counselling or return of results [ 43 ], format for return of results (e.g., patient-mediated vs. provider-mediated [ 43 ]), and geographic location of tissue acquisition or tissue processing, data interpretation, storage or return of results [ 40 , 42 ]. A frequently recognised element was the turn-around time (linked to the patient-centred wait-time for results) [ 40 , 41 , 44 , 45 , 46 ]. Related to this, one article mentioned that batching tests for processing can directly, but not necessarily predictably, influence turn-around and wait-times [ 45 ].

3.3.4.2 Indication Attributes

Another category evident from the included articles related to attributes of the clinical indication for testing. These included clinical seriousness or urgency for decision-making. For example, [study participant comments] “ Why did I have to wait until treatment options were exhausted before being eligible? ” and [authors comment that] “ A subgroup of participants reported…they were running out of…options…a ‘ticking time-bomb’ with death increasingly close ” [ 39 ]. One paper considered the rarity of a clinical condition [ 47 ]. Footnote 2

3.3.4.3 People Attributes

Many articles brought attention to attributes that are intrinsic to persons, often presented as heterogeneity of preferences for molecular testing. These most related to the person(s) undergoing testing and included socioeconomic (e.g., ethnicity, rurality, social isolation, income) and clinical (e.g., current health status, reproductive characteristics, competing health priorities, family history [ 48 ]) factors. For example, one study reported a participant’s perception of gene testing as “ unnecessary because of current healthiness ” [ 42 ].

Numerous intrinsic factors can be grouped as psychological preferences, perceptions, and behavioural traits . This included sense of desperation [ 39 ], optimism [ 48 ], hopefulness [ 44 , 47 ], degree of trust in oncologists or science[ 39 ], and aversion or tolerance to uncertainty [ 44 , 47 , 49 ]. Regier et al.’s DCE estimated public preferences and demand for molecular testing that informs treatment choice by outcomes in context of uncertainty and found that “ respondents identified uncertainty as among the most important considerations when providing precision medicine that is valued by patients ” [ 49 ]. Five studies looked at individual’s perceived risk for, or severity of, the cancer type of interest [ 39 , 43 , 47 , 50 , 51 ]. High health monitoring behaviour was identified in one study as a determinant of WTP for breast cancer susceptibility testing [ 48 ]. ‘Self-efficacy’, defined as confidence in one's ability to cope (e.g., with risk prevention strategies) with actionable or non-actionable results was examined in a study investigating preferences and WTP for WGS amongst cancer patients and their families [ 47 ]. Another article mentioned the perceived caution or enthusiasm of HCPs towards experimental treatments as an element influencing their experience of genomic and personalised paediatric oncology [ 52 ].

Relational elements were considered in some studies. These included individual’s sense of responsibility to their family to obtain information that could influence their health [ 47 ], likelihood to recommend testing to others [ 50 ], having a locus of control highly attributed to powerful others [ 48 ], nature of relationship with HCPs [ 52 ], and connectedness with the healthcare system [ 43 ].

Specific skills and knowledge were examined, including at individual and system levels. The most identified skillset for individuals undergoing testing was health (or more specifically genomic) literacy, which was considered in seven articles [ 39 , 42 , 43 , 44 , 53 , 54 , 55 ]. Relatedly, two studies examined heterogeneity of WTP for molecular testing by education level [ 47 , 48 ]. Other specific skills examined included quality of an individual’s communication skills to enable them to share results, as well as their ability to process complex information or manage negative emotions [ 43 ].

Some studies gave attention to the skills and knowledge of test providers. These identified genomic literacy [ 53 ], handling of uncertainty [ 49 ], and managing patient expectations [ 52 ] as key skills necessary for clinical implementation of molecular testing.

Healthcare system-level attributes were also identified. Examples relating to system strategy included the degree of future focus, acceptance of uncertainty, or ability of a health system to adapt to evolving evidence [ 49 , 56 ].

3.3.4.4 Non-clinical Outcomes

The majority of novel consequences of testing identified from the included articles are categorised here as non-clinical outcomes of molecular testing and subcategorised into psychological impacts for those undergoing testing and non-clinical common good outcomes.

Psychological impacts for those undergoing testing The identified psychological outcomes of the person(s) undergoing testing relate to the value of knowing. For example, decreased worry/anxiety [ 54 , 57 ], reduced uncertainty [ 48 ], greater ability to plan (e.g., family planning [ 48 ]), or the value in hope of what the results may lead to [ 39 , 44 , 52 ]. One study identified outcomes from knowing as “ empowering ” [authors comment], providing individuals with an explanation for their diagnosis, absolving the responsibility for causing their own illness, or validation for previous medical decisions [ 57 ]. “ I [study participant comments] just feel like a weight’s been lifted off my shoulders …I was worried about ovarian cancer but I’m not now. I feel really happy about that ” [ 57 ].

In CEAs/CUAs, attempts to quantify the utility of the value of knowing were seen. One study assumed that after receiving a negative test result a woman’s utility increases to that of healthy women [ 58 ]. Another approach was considered within scenario analyses [ 59 , 60 ]. For example, Chandler et al. used utility weighting to assess the possible reassurance or worry about cancer recurrence [ 59 ].

Illustrating the potential disutility of knowing, included studies considered increased worry and anxiety [ 54 ], feeling negative about lack of actionable findings or disutility from receiving an uncertain result [ 61 ], and fear of stigmatisation and guilt [ 62 ]. An identified specific concern was privacy related to who has access to testing results [ 39 ] and implications for insurance [ 51 , 54 ]. “ [If] the final conclusive tests are negative and everything’s fine and then you put your people through two weeks of total terror for not a whole lot of benefits ” [study participant comments] [ 51 ] “ As long as we have a ‘healthy’ political system, this information will probably be used for good. Unfortunately, history shows that humans are able to use such information to harm others ” [study participant comments][ 54 ].

One article highlighted the heterogeneity of value of knowing, with opposition to obtaining molecular testing information on risk “ Leave it in God’s hands and if I get cancer, that’s the card I was dealt ” [study participant comments] [ 51 ] . In contrast, another article included the disutility of not knowing after declining a test [ 63 ]. Another described risk of false reassurance [ 64 ].

Non-clinical common good outcomes An extension of the value of knowing beyond the person(s) undergoing testing was identified in some articles. McMullen et al. gave mention to value of knowing to a whole community and future generations [ 51 ]. Relatedly, two articles examined scientific spillover [ 39 , 44 ].

Implications of time spent undergoing testing, molecularly informed interventions, or with illness were frequently described as productivity gain or loss. These were identified in nine studies [ 56 , 59 , 62 , 64 , 65 , 66 , 67 , 68 ]. An additional two studies linked this to time costs to family [ 62 , 69 ].

Another wider outcome was non-clinical costs-savings such as consideration of informal care costs [ 62 ].

3.3.4.5 Novel Clinical Outcomes

While identifying clinical outcomes was not the objective of this scoping review, some elements were identified as particularly relevant to molecular testing in cancer. They are listed here for their potential novelty because they are not conventionally considered in value assessments.

Magnitude and nature of impact of molecularly informed management on quality of life or life expectancy.

Availability, accessibility, and cost of molecularly informed management.

Access to clinical trials.

Strength of evidence in the face of evolving evidence base.

One article examined preferences for testing dependent on the chance of an actionable outcome [ 40 ].

The concept that molecular testing in cancer’s value is directly connected to access to molecularly informed management was evident in several articles. These considerations spanned cost [ 52 ], public/‘reimbursed setting’ availability [ 40 ], systemic racism [ 51 ], and access to clinical trials [ 45 ]. The strength of evidence for intervention for molecularly informed intervention was represented by factors influencing clinician’s decision-making on the basis of scientific evidence [ 52 ] and extent of medical agreement on changing care [ 49 ].

4 Discussion

4.1 summary of findings and attention to broadening conceptualisation of value.

This scoping review sought to identify how value is defined in studies that focus on the value of molecular testing in cancer and to map the literature addressing this topic. Because of the diversity in molecular testing technologies and applications, this review was not limited to any molecular testing type or clinical setting, instead taking a broad definition to molecular testing in cancer. This review pragmatically defined ‘focus on value [of molecular testing in cancer]’ to answer the review questions. The scoping review was designed to strike a balance between identifying the most common approaches to measuring value for money and identifying novel costs or novel benefits at the same time. As such, the review included studies focusing on costs relative to outcomes (to map trends of this core body of research) and studies that focused on describing at least one novel cost or outcome. Molecular testing in cancer diagnostic accuracy was not the focus of this review, given an extensive established body of literature and general acceptance that this is part of value assessment of molecular testing in cancer [ 70 ]. A wide range of value definitions was identified, but the overriding focus was on molecular testing's clinical outcomes, with 30/91 (33.0%) studies considering non-clinical outcomes. Attention to non-clinical outcomes was primarily seen within qualitative research, for example, in preference studies, but a minority of conventional economic evaluations also included non-clinical outcomes. The data from 2022 suggest that non-clinical outcomes may be increasingly reported within conventional economic evaluations.

This review used two published value frameworks, namely ISPOR Special Task Force’s Elements of Value [ 17 ] and the OHE’s ‘non-traditional’ elements of value for complementary diagnostics [ 19 ], to examine the degree to which existing literature has included contemporarily recognised novel value elements. These frameworks have been designed for general pharmaceuticals (ISPOR) and (non-cancer specific) companion diagnostics (OHE), and so were not designed to accommodate the uniqueness of molecular testing in cancer. For the included molecular testing in cancer studies, the value of knowing and the comparable value of reduction in uncertainty were the most featured, followed by equity and productivity. There was little reference to the remaining six ISPOR and four OHE novel elements, suggesting that some of these previously described value elements may not be relevant to molecular testing in cancer, or alternatively, there remain hurdles to operationalising the assessment and measurement of these novel elements [ 35 ].

The review identified other non-clinical outcomes and contextual considerations suggesting that molecular testing in cancer likely requires a bespoke approach. For these value elements not ordinarily considered in conventional economic analyses, and therefore considered novel, an inductive approach to analysis resulted in grouping of the elements into five categories representing intrinsic and extrinsic factors: (1) novel test attributes; (2) indication attributes; (3) people attributes; (4) non-clinical outcomes of testing; and (5) novel clinical outcomes. The final category was included as despite pertaining to clinical outcomes, these were considered highly specific to molecular testing in cancer, or previously underappreciated. This inductive framework identified a triad of intrinsic test properties, intrinsic clinical indication properties, and attributes of people (stakeholders) involved with testing as contextual considerations likely to influence the realised value of molecular testing in cancer. The two remaining categories relate to consequences of testing information with particular emphasis on psychological utility or disutility of knowing and spillover effects beyond the individual undergoing testing.

4.2 How did Equity Feature? and Other Gaps

There were notable gaps identified from the examined literature. Given the important focus on addressing inequities in cancer outcomes, especially for indigenous populations, this review aimed to identify elements relating to equity or viewpoints of indigenous communities within value definitions. Equity was not explicitly described within value definitions of included articles. Even when equity was considered broadly (recognizing that different people with different levels of advantage require different approaches and resources to get equitable health outcomes [ 71 ]), it rarely featured. Where it did feature, equity was implicitly considered, with the majority of examples relating to examination of the heterogeneity of preferences for, experiences of, or WTP for, molecular testing, which in the inductive framework is mostly embedded within the intrinsic attributes of people category.

Another category where equity was implicitly considered was within test attributes with regards to test process. Specifically, articles identified the barriers to testing (e.g., financial, complexity of navigating health system in context of health literacy or racism) as well as the inequitable distribution of these barriers. Finally, equity is implicitly mentioned in the clinical indication for testing category where attention to rarity was mentioned. This interpretation of equity has been previously used in the context of value assessment with the connection between rarity and unmet need aligning with DrugAbacus’ definitions [ 72 ]. There is debate on whether rarity of disease warrants consideration as an independent determinant of value. The National Institute for Health and Care Excellence (NICE) has not yet included rarity of disease as a modifier, commenting on the “complex interactions between the severity of the diseases, current diagnostic and treatment options, clinical knowledge, research and development, and health service design and delivery” [ 73 ]. The related issue of rarity of molecular testing results is so far unexplored.

4.3 What do the Observed Trends Suggest is the Future Direction of Value Assessment of Molecular Testing in Cancer?

Value assessment frameworks for health technologies have received heightened attention in the past decade [ 16 , 74 ]. Aside from a few exceptions designed for diagnostics (e.g. [ 19 , 75 ]), these have mostly been for pharmaceuticals. ISPOR’s value flower was developed for drugs. Unsurprisingly, it was the flower’s value of information petal that featured most prominently in this review. The earliest descriptions of this value element, now more than 30 years ago, related to diagnostics where a test is valued for the prognostic information it provides apart from its effect on patient management [ 38 ]. The attention to this element suggests that further developments toward more holistic molecular testing in cancer value assessments could take direction from the emerging approaches to broaden the value assessment for diagnostics, especially consideration of the value gained from reduction in uncertainty. On the other hand, identified novel elements emerging from this review (consideration of rarity, observed heterogeneity of preferences based on genomic literary, issues relating to variants of uncertain significance, and likelihood of individuals to share information on germline findings with family) are less specific to diagnostics and more linked to precision medicine. As such, value assessment frameworks developed for precision medicines (e.g., value assessment of orphan drugs for rare diseases [ 76 ]) may provide more helpful guidance. Another alternative is that development of value assessment tools for molecular testing in cancer forges its own path, although arguably this is not the most practical or efficient approach for health technology assessors [ 35 ].

4.4 Implications for Implementation Strategies—Observed Novel Elements Signpost Where Realised Value Could be Optimised

The attention to attributes of the entire testing process and preferences for these attributes suggests that how molecular testing in cancer is implemented likely influences its realised value. Importantly, this signals the need for greater attention to where cost-effectiveness of molecular testing in cancer can be improved with changes in access to genetic counselling, genomics education, etc. One of the included articles emphasised this, highlighting the importance of culturally and linguistically appropriate testing services [ 42 ].

4.5 Limitations of the Scoping Review

First, this review may have missed non-English studies that have focused on value of molecular testing in cancer, particularly grey literature, and studies without full text on the net (i.e., FUTON bias [ 77 ]). Second, search strategy could not incorporate all studies exploring potentially relevant determinants of value, especially the heterogenous body of implementation and preferences literature for which keywords are not consistent. This would include a large body of qualitative literature on genomic testing in cancer (e.g., information preferences [ 78 , 79 , 80 ]), which was not identified in this review because the search strategy required the term ‘value’ (or a related word) in an article’s title. Findings from this review should be considered alongside other molecular testing literature including that focusing on analytical validity, clinical validity and clinical utility. Third, the eligibility criteria adopted could not overcome the inherent ambiguity of the ‘focus on value of molecular testing in cancer’ criterion. The extent to which molecular testing comprises a healthcare intervention is not always clear, and the degree to which a study was focused on value proved very subjective. Fourth, the mapping to emerging guidance was limited to ISPOR and OHE, with potential that review conclusions would have differed had alternative deductive frameworks, for example, Augustovski et al.’s Value framework for the Assessment of Diagnostic Technologies [ 81 ], been selected, especially since ISPOR is not designed for diagnostics.

4.6 Next Steps in Understanding Value of Molecular Testing in Cancer—Exploring Elements of Value and Approaches to Their Measurement

The dominance of clinical value elements and prevalence of use of the QALY concluded from this review demonstrates the tendency for studies estimating molecular testing in cancer 's value to focus on quantifiable and commensurable elements. Combined with the fact this review identified no multi-criteria decision analyses, this suggests a gap in the literature with respect to trialling methods that more deliberatively assess value, like the approach increasingly seen in the HTA of pharmaceuticals [ 82 ].

It is likely that the design of studies that focus on the value of molecular testing in cancer are dictated by the market they are trying to receive reimbursement from. For jurisdictions where the decision and reimbursement of molecular testing and targeted medicines or preventative strategies may be independent of each other (e.g., different decision-making bodies), evidence may be interpreted without due attention to complementarity [ 18 ].

A next logistical step is the further exploration of the non-clinical elements of value important to the range of relevant stakeholders, with attention to the complex interplay and joint influences of psychological consequences of knowing, value of reduced uncertainty, and spillover outcomes in the context of precision oncology’s heterogeneity of outcomes. Once these are better understood the field can move onto the quantification and commensuration of the conventional and novel elements of value. It is not known whether the identified novel elements are reflective of indigenous peoples’ values as this research has not been undertaken, highlighting a priority focus for future research. Also, while this is a scoping exercise to map how value is defined for molecular testing in cancer, our review findings suggest that once elements of value are identified, data for these must be collected.

4.7 Conclusions

Molecular testing in cancer has increased clinical attention, necessitating robust approaches to value assessment to support optimal and equitable implementation. Published studies that focus on the value of molecular testing in cancer highlight the lack of an agreed definition of value in this setting. Ongoing research is warranted to clarify how value is defined in a way that accounts for molecular testing as a process, and with a breadth of outcomes through an equity lens, including psychological benefits and benefits beyond the tested individual.

OSF Registries | Definitions and elements of value used in the assessments of value in molecular testing in cancer: A scoping review.

(Perceived) severity of condition has been allocated to people attributes because it was largely documented as a perception rather than an objective clinical descriptor.

The Business Research Company. Oncology molecular diagnostics global market report 2022. 2022. https://www.businesswire.com/news/home/20220830005582/en/Oncology-Molecular-Diagnostics-Global-Market-Research-Report-2022-PCR-ISH-INNAT-Chips-Microarrays-Sequencing-Mass-Spectroscopy-TMA-Analysis-Forecasts-2016-2021-2021-2026F-2031F---ResearchAndMarkets.com . Accessed 19 Nov 2023.

Ettinger DS, Wood DE, Aisner DL, et al. Non-small cell lung cancer, version 3.2022, NCCN clinical practice guidelines in oncology. J Natl Compr Canc Netw. 2022;20:497–530.

Article   PubMed   Google Scholar  

Regier DA, Pollard S, McPhail M, Bubela T, Hanna TP, Ho C, Weymann D. A perspective on life-cycle health technology assessment and real-world evidence for precision oncology in Canada. NPJ Precis Oncol. 2022;6(1):76.

Terkola R, Antoñanzas F, Postma M. Economic evaluation of personalized medicine: a call for real-world data. Eur J Health Econ. 2017;18(9):1065–7.

Akhmetov I, Bubnov RV. Assessing value of innovative molecular diagnostic tests in the concept of predictive, preventive, and personalized medicine. EPMA J. 2015. https://doi.org/10.1186/S13167-015-0041-3 .

Article   PubMed   PubMed Central   Google Scholar  

Burris HA, Saltz LB, Yu PP. Assessing the Value of next-generation sequencing tests in a dynamic environment. Am Soc Clin Oncol Educ Book. 2018:139–146.

Faulkner E, Holtorf AP, Walton S, et al. Being precise about precision medicine: what should value frameworks incorporate to address precision medicine? A report of the personalized precision medicine special interest group. Value Health. 2020;23:529–39.

Marshall DA, Grazziotin LR, Regier DA, Wordsworth S, Buchanan J, Phillips K, Ijzerman M. Addressing challenges of economic evaluation in precision medicine using dynamic simulation modeling. Value Health. 2020;23:566–73.

Patrinos G, CM-CP. Measuring the value of pharmacogenomics evidence. Wiley Online Library. 2017;102:739–741.

Payne K, Gavan SP, Wright SJ, Thompson AJ. Cost-effectiveness analyses of genetic and genomic diagnostic tests. Nat Rev Genet. 2018;19:235–46.

Article   CAS   PubMed   Google Scholar  

Phillips KA, Ann Sakowski J, Trosman J, Douglas MP, Liang S-Y, Neumann P. The economic value of personalized medicine tests: what we know and what we need to know. Genet Med. 2014;16:251–7.

Berm EJJ, De Looff M, Wilffert B, Boersma C, Annemans L, Vegter S, Van Boven JFM, Postma MJ. Economic evaluations of pharmacogenetic and pharmacogenomic screening tests: a systematic review. Second update of the literature. PLoS ONE. 2016. https://doi.org/10.1371/JOURNAL.PONE.0146262 .

Pliskin JS, Weinstein MC, Shepard DS. Utility functions for life years and health status. Oper Res. 1980;28:206–24.

Article   Google Scholar  

Zeckhauser R, Shepard D. Where now for saving lives. Law Contemp Probl. 1976;40:5.

Pollard S, Weymann D, Chan B, et al. Defining a core data set for the economic evaluation of precision oncology. Value Health. 2022;25:1371–80.

Neumann PJ, Willke RJ, Garrison LP. A health economics approach to US value assessment frameworks—introduction: an ISPOR special task force report [1]. Value Health. 2018;21:119–23.

Lakdawalla DN, Doshi JA, Garrison LP, Phelps CE, Basu A, Danzon PM. Defining Elements of value in health care—a health economics approach: an ISPOR special task force report [3]. Value Health. 2018;21:131–9.

Riley R, Lesteven D, Plun-Favreau J, Ferrara J, Kapitein P, Collins P. The value of knowing and knowing the value: improving the health technology assessment of complementary diagnostics. 2016.

Garrison L, Mestre-Ferrandiz J, Zamora B. The value of knowing and knowing the value: improving the health technology assessment of complementary diagnostics. 2016.

ICER 2020–2023 value assessment framework ICER.

Pitini E, De Vito C, Marzuillo C, D’andrea E, Rosso A, Federici A, Maria ED, Villari P. How is genetic testing evaluated? A systematic review of the literature. Eur J Hum Genet. 2018;26:605–15.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Goranitis I, Best S, Christodoulou J, Stark Z, Boughtwood T. The personal utility and uptake of genomic sequencing in pediatric and adult conditions: eliciting societal preferences with three discrete choice experiments. Genet Med. 2020;22:1311–9.

Kohler JN, Turbitt E, Biesecker BB. Personal utility in genomic testing: a systematic literature review. Eur J Hum Genet. 2017;25:662–8.

NICE. Diagnostics assessment programme manual. 2011. https://www.nice.org.uk/media/default/about/what-we-do/nice-guidance/nice-diagnostics-guidance/diagnostics-assessment-programme-manual.pdf . Accessed 19 Nov 2023.

MSAC. Medical Services Advisory Committee process framework. 2016.

Cook JP, Golec J. How excluding some benefits from value assessment of new drugs impacts innovation. 2017. https://doi.org/10.1002/hec.3507 .

Seo MK, Cairns J. How are we evaluating the cost-effectiveness of companion biomarkers for targeted cancer therapies? A systematic review. BMC Cancer. 2021. https://doi.org/10.1186/S12885-021-08725-4 .

Te Aho o Te Kahu. He Pūrongo Mate Pukupuku o Aotearoa 2020: the state of cancer in New Zealand. 2020.

Munn Z, Peters MDJ, Stern C, Tufanaru C, McArthur A, Aromataris E. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodol. 2018;18:1–7.

Peters M, Godfrey C, McInerney P, Munn Z, Trico A, Khalil H. Chapter 11: scoping reviews. JBI Man Evid Synth. 2020. https://doi.org/10.46658/JBIMES-20-12 .

Tricco AC, Lillie E, Zarin W, et al. PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med. 2018;169:467–73.

Santos Gonzalez F, Mordaunt D, Stark Z, Dalziel K, Christodoulou J, Goranitis I. Microcosting diagnostic genomic sequencing: a systematic review. Genet Med. 2023;25: 100829.

Peters MDJ, Marnie C, Tricco AC, Pollock D, Munn Z, Alexander L, McInerney P, Godfrey CM, Khalil H. Updated methodological guidance for the conduct of scoping reviews. JBI Evid Synth. 2020;18:2119–26.

Louw A, Diener I, Butler DS, Puentedura EJ. The effect of neuroscience education on pain, disability, anxiety, and stress in chronic musculoskeletal pain. Arch Phys Med Rehabil. 2011;92:2041–56.

Neumann PJ, Garrison LP, Willke RJ. The history and future of the ISPOR value flower: addressing limitations of conventional cost-effectiveness analysis. 2022. https://doi.org/10.1016/j.jval.2022.01.010 .

Crummer E, Neumann PJ, Cohen JT, Kim DD. Use of novel and social elements of value in cost-effectiveness analysis. 2022. https://cevr.tuftsmedicalcenter.org/news/2022/use-of-novel-elements-of-value-in-cost-effectiveness-analyses . Accessed 19 Nov 2023.

Neumann PJ, Cohen JT, Hammitt JK, Concannon TW, Auerbach HR, Fang C, Kent DM. Willingness-to-pay for predictive tests with no immediate treatment implications: a survey of US residents. Health Econ. 2010;21:238–51.

Asch DA, Patton JP, Hershey JC. Knowing for the sake of knowing: the value of prognostic information. Med Decis Mak. 1990;10:47–57.

Article   CAS   Google Scholar  

Best MC, Bartley N, Jacobs C, Juraskova I, Goldstein D, Newson AJ, Savard J, Meiser B, Ballinger M, Napier C. Patient perspectives on molecular tumor profiling: “Why wouldn’t you?” BMC Cancer. 2019;19:1–9.

Fifer S, Ordman R, Briggs L, Cowley A. Patient and clinician preferences for genetic and genomic testing in non-small cell lung cancer: a discrete choice experiment. J Pers Med. 2022. https://doi.org/10.3390/jpm12060879 .

Weymann D, Veenstra DL, Jarvik GP, Regier DA. Patient preferences for massively parallel sequencing genetic testing of colorectal cancer risk: a discrete choice experiment. Eur J Hum Genet. 2018;26:1257–65.

Wong XY, Groothuis-Oudshoorn CG, Tan CS, van Til JA, Hartman M, Chong KJ, Ijzerman MJ, Wee H-L. Women’s preferences, willingness-to-pay, and predicted uptake for single-nucleotide polymorphism gene testing to guide personalized breast cancer screening strategies: a discrete choice experiment. Patient Prefer Adher. 2018;12:1837–52.

Sarki M, Ming C, Aissaoui S, et al. Intention to inform relatives, rates of cascade testing, and preference for patient-mediated communication in families concerned with hereditary breast and ovarian cancer and lynch syndrome: the Swiss CASCADE cohort. Cancers (Basel). 2022;14:1636.

Butow P, Davies G, Napier CE, et al. Assessment of the value of tumor variation profiling perceived by patients with cancer. JAMA Netw Open. 2020;3: e204721.

de Alava E, Pareja MJ, Carcedo D, Arrabal N, Garcia J-F, Bernabe-Caro R. Cost-effectiveness analysis of molecular diagnosis by next-generation sequencing versus sequential single testing in metastatic non-small cell lung cancer patients from a south Spanish hospital perspective. Expert Rev Pharmacoecon Outcomes Res. 2022;22:1033–42.

Hurry M, Eccleston A, Dyer M, Hoskins P. Canadian cost-effectiveness model of BRCA-driven surgical prevention of breast/ovarian cancers compared to treatment if cancer develops. Int J Technol Assess Health Care. 2020;36:104–12.

Butow P, Davies G, Napier CE, et al. Value of whole-genome sequencing to Australian cancer patients and their first-degree relatives participating in a genomic sequencing study. J Genet Couns. 2022;31:96–108.

Blouin-Bougie J, Amara N, Bouchard K, Simard J, Dorval M. Disentangling the determinants of interest and willingness-to-pay for breast cancer susceptibility testing in the general population: a cross-sectional Web-based survey among women of Quebec (Canada). BMJ Open. 2018;8: e016662.

Regier DA, Veenstra DL, Basu A, Carlson JJ. Demand for precision medicine: a discrete-choice experiment and external validation study. Pharmacoeconomics. 2020;38:57–68.

Mayer M, Selig K, Tuttelmann F, Dinkel A, Gschwend JE, Herkommer K. Interest in, willingness-to-pay for and willingness-to-recommend genetic testing for prostate cancer among affected men after radical prostatectomy. Fam Cancer. 2019;18:221–30.

McMullen C, Holup J, Davis JV, Foley P, Jacob L, Cottrell E, Bui DP, Wilfond B, Goddard KA. Discordant patient and clinician perspectives on the potential value of genetic services in safety-net clinics. J Health Care Poor Underserv. 2020;31:1347–63.

McCarthy MC, De Abreu LR, McMillan LJ, Meshcheriakova E, Cao A, Gillam L. Finding out what matters in decision-making related to genomics and personalized medicine in pediatric oncology: developing attributes to include in a discrete choice experiment. Patient. 2020;13:347–61.

Chow-White P, Ha D, Laskin J. Knowledge, attitudes, and values among physicians working with clinical genomics: a survey of medical oncologists. Hum Resour Health. 2017;15:1–9.

Clasen K, Gani C, Schroeder C, Riess O, Zips D, Schoffski O, Clasen S. Patient views on genetics and functional imaging for precision medicine: a willingness-to-pay analysis. Per Med. 2022;19:103–12.

Guo F, Hirth JM, Fuchs EL, Cofie LE, Brown V, Kuo Y-F, Fernandez ME, Berenson AB. Knowledge, attitudes, willingness to pay, and patient preferences about genetic testing and subsequent risk management for cancer prevention. J Cancer Educ. 2022;37:362–9.

Sun L, Cui B, Wei X, Sadique Z, Yang L, Manchanda R, Legood R. Cost-effectiveness of genetic testing for all women diagnosed with breast cancer in China. Cancers (Basel). 2022. https://doi.org/10.3390/cancers14071839 .

Young M-A, Forrest LE, Rasmussen V-M, James P, Mitchell G, Sawyer SD, Reeve K, Hallowell N. Making sense of SNPs: women’s understanding and experiences of receiving a personalized profile of their breast cancer risks. J Genet Couns. 2018;27:702–8.

Lourencao M, Simoes Correa Galendi J, Galvao HDCR, et al. Cost-Effectiveness of BRCA 1/2 genetic test and preventive strategies: using real-world data from an upper-middle income country. Front Oncol. 2022;12:951310.

Chandler Y, Schechter CB, Jayasekera J, et al. Cost effectiveness of gene expression profile testing in community practice. J Clin Oncol. 2018;36:554–62.

Eccleston A, Bentley A, Dyer M, Strydom A, Vereecken W, George A, Rahman N. A cost-effectiveness evaluation of germline BRCA1 and BRCA2 testing in UK women with ovarian cancer. Value Health. 2017;20:567–76.

Asphaug L, Melberg HO. The cost-effectiveness of multigene panel testing for hereditary breast and ovarian cancer in Norway. MDM Policy Pract. 2019;4:2381468318821103.

PubMed   PubMed Central   Google Scholar  

Majem M, Alvarez R, Ortega AL, Ruiz de Alda L, Gordo R, Garcia JF, Ivanova-Markova Y, Gonzalez-Dominguez A, San Cristobal RS, Rojo F. Cost-benefit analysis of ALK diagnosis vs. non-diagnosis in patients with advanced non-small cell lung cancer in Spain. Glob Region Health Technol Assess. 2022;9:82–90.

Snowsill T, Coelho H, Huxley N, Jones-Hughes T, Briscoe S, Frayling IM, Hyde C. Molecular testing for Lynch syndrome in people with colorectal cancer: systematic reviews and economic evaluation. Health Technol Assess. 2017;21:1–238.

Pereira C, Areia M, Dinis-Ribeiro M. Cost-utility analysis of genetic polymorphism universal screening in colorectal cancer prevention by detection of high-risk individuals. Dig Liver Dis. 2019;51:1731–7.

Simons MJHG, Retel VP, Ramaekers BLT, et al. Early cost effectiveness of whole-genome sequencing as a clinical diagnostic test for patients with inoperable stage IIIB, C/IV non-squamous non-small-cell lung cancer. Pharmacoeconomics. 2021;39:1429–42.

Ibarrondo O, Alvarez-Lopez I, Freundlich F, Arrospide A, Galve-Calvo E, Gutierrez-Toribio M, Plazaola A, Mar J. Probabilistic cost-utility analysis and expected value of perfect information for the oncotype multigenic test: a discrete event simulation model. Gac Sanit. 2020;34:61–8.

Martínez del Prado P, Alvarez-López I, Domínguez-Fernández S, Plazaola A, Ibarrondo O, Galve-Calvo E, Ancizar-Lizarraga N, Gutierrez-Toribio M, Lahuerta-Martínez A, Mar J. Clinical and economic impact of the 21-gene recurrence score assay in adjuvant therapy decision making in patients with early-stage breast cancer: pooled analysis in 4 Basque Country university hospitals. ClinicoEcono Outcomes Res. 2018:189–199.

Wei X, Cai J, Sun H, Li N, Xu C, Zhang G, Sui Y, Zhuang J, Zheng B. Cost-effectiveness analysis of UGT1A1*6/*28 genotyping for preventing FOLFIRI-induced severe neutropenia in Chinese colorectal cancer patients. Pharmacogenomics. 2019;20:241–9.

Kunst N, Stout NK, O’Brien G, Christensen KD, McMahon PM, Wu AC, Diller LR, Yeh JM. Population-based newborn screening for germline TP53 variants: clinical benefits, cost-effectiveness, and value of further research. J Natl Cancer Inst. 2022;114:722–31.

Neumann PJ, Thorat T, Shi J, Saret CJ, Cohen JT. The changing face of the cost-utility literature, 1990–2012. Value Health. 2015;18:271–7.

Achieving equity | Ministry of Health NZ. https://www.health.govt.nz/about-ministry/what-we-do/achieving-equity . Accessed 7 Dec 2023.

Drug Abacus—Drug Pricing Lab. https://www.drugpricinglab.org/tools/drug-abacus/ . Accessed 7 Dec 2023.

Public board meeting agenda and papers: January 2022 | Public board meetings | Board | Who we are | About | NICE.

Zhang M, Bao Y, Lang Y, Fu S, Kimber M, Levine M, Xie F. What is value in health and healthcare? A systematic literature review of value assessment frameworks. Value Health. 2022;25(2):302–17.

Anonychuk A, Beastall G, Shorter S, Kloss-Wolf R, Neumann P. A framework for assessing the value of laboratory diagnostics. Healthc Manage Forum. 2012:25(3):S4–11.

Blonda A, Denier Y, Huys I, Simoens S. How to value orphan drugs? A review of European value assessment frameworks. Front Pharmacol. 2021;12:631527.

Lancet RW-T. Visibility of research: FUTON bias. 2002. thelancet.com.

Bartley N, Best M, Jacobs C, et al. Cancer patients’ views and understanding of genome sequencing: a qualitative study. J Med Genet. 2020;57:671–6.

Fisher ER, Pratt R, Esch R, Kocher M, Wilson K, Lee W, Zierhut HA. The role of race and ethnicity in views toward and participation in genetic studies and precision medicine research in the United States: a systematic review of qualitative and quantitative studies. Mol Genet Genomic Med. 2020. https://doi.org/10.1002/MGG3.1099 .

Lau-Min KS, Varughese LA, Nelson MN, Cambareri C, Reddy NJ, Oyer RA, Teitelbaum UR, Tuteja S. Preemptive pharmacogenetic testing to guide chemotherapy dosing in patients with gastrointestinal malignancies: a qualitative study of barriers to implementation. BMC Cancer. 2022. https://doi.org/10.1186/S12885-022-09171-6 .

Augustovski F, Alfie V, Alcaraz A, García Martí S, Drummond MF, Pichon-Riviere A. A value framework for the assessment of diagnostic technologies: a proposal based on a targeted systematic review and a multistakeholder deliberative process in Latin America. Value Health. 2021;24:486–96.

Gongora-Salazar P, Rocks S, Fahr P, Health OR-A-V. The use of multicriteria decision analysis to support decision making in healthcare: an updated systematic literature review. Amsterdam: Elsevier; 2023.

Google Scholar  

Download references

Open Access funding enabled and organized by CAUL and its Member Institutions.

Author information

Authors and affiliations.

University of Auckland, Auckland, New Zealand

Alice Minhinnick, Michelle Wilson & Paula Lorgelly

University of Melbourne, Melbourne, VIC, Australia

Francisco Santos-Gonzalez

You can also search for this author in PubMed   Google Scholar

Corresponding authors

Correspondence to Alice Minhinnick or Paula Lorgelly .

Ethics declarations

The research was conducted during AM’s tenure of a Health Research Council of New Zealand Clinical Research Training Fellowship and so the research was supported (in part) by funding from the Health Research Council of New Zealand.

Conflict of interest

AM, FGS, MK and PL have no conflicts of interest that are relevant to the content of this article.

Ethics approval

The study is a literature review and hence does not require ethical approval.

Consent to participate

Not applicable.

Consent for publication (from patients/participants)

Data availability.

The datasets used and/or analysed during this scoping review are available from the corresponding author on reasonable request.

Code availability

Author contributions.

AM, MW and PL conceived and designed the analysis. AM and FGS collected the data. AM, FGS, MW and PL performed the analysis. AM, FGS, MW and PL wrote the paper.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 129 KB)

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License, which permits any non-commercial use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc/4.0/ .

Reprints and permissions

About this article

Minhinnick, A., Santos-Gonzalez, F., Wilson, M. et al. How is Value Defined in Molecular Testing in Cancer? A Scoping Review. Appl Health Econ Health Policy (2024). https://doi.org/10.1007/s40258-024-00901-4

Download citation

Accepted : 18 June 2024

Published : 09 July 2024

DOI : https://doi.org/10.1007/s40258-024-00901-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Find a journal
• Publish with us
• Track your research