research literature process

Research Process :: Step by Step

• Introduction
• Select Topic
• Identify Keywords
• Background Information
• Develop Research Questions
• Refine Topic
• Search Strategy
• Popular Databases
• Evaluate Sources
• Types of Periodicals
• Reading Scholarly Articles
• Primary & Secondary Sources
• Organize / Take Notes
• Writing & Grammar Resources
• Annotated Bibliography
• Literature Review
• Citation Styles
• Paraphrasing
• Privacy / Confidentiality
• Research Process
• Selecting Your Topic
• Identifying Keywords
• Gathering Background Info
• Evaluating Sources

Organize the literature review into sections that present themes or identify trends, including relevant theory. You are not trying to list all the material published, but to synthesize and evaluate it according to the guiding concept of your thesis or research question.  

What is a literature review?

A literature review is an account of what has been published on a topic by accredited scholars and researchers. Occasionally you will be asked to write one as a separate assignment, but more often it is part of the introduction to an essay, research report, or thesis. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries

A literature review must do these things:

• be organized around and related directly to the thesis or research question you are developing
• synthesize results into a summary of what is and is not known
• identify areas of controversy in the literature
• formulate questions that need further research

Ask yourself questions like these:

• What is the specific thesis, problem, or research question that my literature review helps to define?
• What type of literature review am I conducting? Am I looking at issues of theory? methodology? policy? quantitative research (e.g. on the effectiveness of a new procedure)? qualitative research (e.g., studies of loneliness among migrant workers)?
• What is the scope of my literature review? What types of publications am I using (e.g., journals, books, government documents, popular media)? What discipline am I working in (e.g., nursing psychology, sociology, medicine)?
• How good was my information seeking? Has my search been wide enough to ensure I've found all the relevant material? Has it been narrow enough to exclude irrelevant material? Is the number of sources I've used appropriate for the length of my paper?
• Have I critically analyzed the literature I use? Do I follow through a set of concepts and questions, comparing items to each other in the ways they deal with them? Instead of just listing and summarizing items, do I assess them, discussing strengths and weaknesses?
• Have I cited and discussed studies contrary to my perspective?
• Will the reader find my literature review relevant, appropriate, and useful?

Ask yourself questions like these about each book or article you include:

• Has the author formulated a problem/issue?
• Is it clearly defined? Is its significance (scope, severity, relevance) clearly established?
• Could the problem have been approached more effectively from another perspective?
• What is the author's research orientation (e.g., interpretive, critical science, combination)?
• What is the author's theoretical framework (e.g., psychological, developmental, feminist)?
• What is the relationship between the theoretical and research perspectives?
• Has the author evaluated the literature relevant to the problem/issue? Does the author include literature taking positions she or he does not agree with?
• In a research study, how good are the basic components of the study design (e.g., population, intervention, outcome)? How accurate and valid are the measurements? Is the analysis of the data accurate and relevant to the research question? Are the conclusions validly based upon the data and analysis?
• In material written for a popular readership, does the author use appeals to emotion, one-sided examples, or rhetorically-charged language and tone? Is there an objective basis to the reasoning, or is the author merely "proving" what he or she already believes?
• How does the author structure the argument? Can you "deconstruct" the flow of the argument to see whether or where it breaks down logically (e.g., in establishing cause-effect relationships)?
• In what ways does this book or article contribute to our understanding of the problem under study, and in what ways is it useful for practice? What are the strengths and limitations?
• How does this book or article relate to the specific thesis or question I am developing?

Text written by Dena Taylor, Health Sciences Writing Centre, University of Toronto

http://www.writing.utoronto.ca/advice/specific-types-of-writing/literature-review

• << Previous: Annotated Bibliography
• Next: Step 5: Cite Sources >>
• Last Updated: May 31, 2024 3:34 PM
• URL: https://libguides.uta.edu/researchprocess

University of Texas Arlington Libraries 702 Planetarium Place · Arlington, TX 76019 · 817-272-3000

• Internet Privacy
• Accessibility
• Problems with a guide? Contact Us.

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Jan 4, 2024 10:52 AM
• URL: https://guides.library.yale.edu/YSNDoctoral

• USC Libraries
• Research Guides

Organizing Your Social Sciences Research Paper

• 5. The Literature Review
• Purpose of Guide
• Design Flaws to Avoid
• Independent and Dependent Variables
• Glossary of Research Terms
• Reading Research Effectively
• Narrowing a Topic Idea
• Broadening a Topic Idea
• Extending the Timeliness of a Topic Idea
• Academic Writing Style
• Applying Critical Thinking
• Choosing a Title
• Making an Outline
• Paragraph Development
• Research Process Video Series
• Executive Summary
• The C.A.R.S. Model
• Background Information
• The Research Problem/Question
• Theoretical Framework
• Citation Tracking
• Content Alert Services
• Evaluating Sources
• Primary Sources
• Secondary Sources
• Tiertiary Sources
• Scholarly vs. Popular Publications
• Qualitative Methods
• Quantitative Methods
• Insiderness
• Using Non-Textual Elements
• Limitations of the Study
• Common Grammar Mistakes
• Writing Concisely
• Avoiding Plagiarism
• Footnotes or Endnotes?
• Further Readings
• Generative AI and Writing
• USC Libraries Tutorials and Other Guides
• Bibliography

A literature review surveys prior research published in books, scholarly articles, and any other sources relevant to a particular issue, area of research, or theory, and by so doing, provides a description, summary, and critical evaluation of these works in relation to the research problem being investigated. Literature reviews are designed to provide an overview of sources you have used in researching a particular topic and to demonstrate to your readers how your research fits within existing scholarship about the topic.

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . Fourth edition. Thousand Oaks, CA: SAGE, 2014.

Importance of a Good Literature Review

A literature review may consist of simply a summary of key sources, but in the social sciences, a literature review usually has an organizational pattern and combines both summary and synthesis, often within specific conceptual categories . A summary is a recap of the important information of the source, but a synthesis is a re-organization, or a reshuffling, of that information in a way that informs how you are planning to investigate a research problem. The analytical features of a literature review might:

• Give a new interpretation of old material or combine new with old interpretations,
• Trace the intellectual progression of the field, including major debates,
• Depending on the situation, evaluate the sources and advise the reader on the most pertinent or relevant research, or
• Usually in the conclusion of a literature review, identify where gaps exist in how a problem has been researched to date.

Given this, the purpose of a literature review is to:

• Place each work in the context of its contribution to understanding the research problem being studied.
• Describe the relationship of each work to the others under consideration.
• Identify new ways to interpret prior research.
• Reveal any gaps that exist in the literature.
• Resolve conflicts amongst seemingly contradictory previous studies.
• Identify areas of prior scholarship to prevent duplication of effort.
• Point the way in fulfilling a need for additional research.
• Locate your own research within the context of existing literature [very important].

Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper. 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . Los Angeles, CA: SAGE, 2011; Knopf, Jeffrey W. "Doing a Literature Review." PS: Political Science and Politics 39 (January 2006): 127-132; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012.

Types of Literature Reviews

It is important to think of knowledge in a given field as consisting of three layers. First, there are the primary studies that researchers conduct and publish. Second are the reviews of those studies that summarize and offer new interpretations built from and often extending beyond the primary studies. Third, there are the perceptions, conclusions, opinion, and interpretations that are shared informally among scholars that become part of the body of epistemological traditions within the field.

In composing a literature review, it is important to note that it is often this third layer of knowledge that is cited as "true" even though it often has only a loose relationship to the primary studies and secondary literature reviews. Given this, while literature reviews are designed to provide an overview and synthesis of pertinent sources you have explored, there are a number of approaches you could adopt depending upon the type of analysis underpinning your study.

Argumentative Review This form examines literature selectively in order to support or refute an argument, deeply embedded assumption, or philosophical problem already established in the literature. The purpose is to develop a body of literature that establishes a contrarian viewpoint. Given the value-laden nature of some social science research [e.g., educational reform; immigration control], argumentative approaches to analyzing the literature can be a legitimate and important form of discourse. However, note that they can also introduce problems of bias when they are used to make summary claims of the sort found in systematic reviews [see below].

Integrative Review Considered a form of research that reviews, critiques, and synthesizes representative literature on a topic in an integrated way such that new frameworks and perspectives on the topic are generated. The body of literature includes all studies that address related or identical hypotheses or research problems. A well-done integrative review meets the same standards as primary research in regard to clarity, rigor, and replication. This is the most common form of review in the social sciences.

Historical Review Few things rest in isolation from historical precedent. Historical literature reviews focus on examining research throughout a period of time, often starting with the first time an issue, concept, theory, phenomena emerged in the literature, then tracing its evolution within the scholarship of a discipline. The purpose is to place research in a historical context to show familiarity with state-of-the-art developments and to identify the likely directions for future research.

Methodological Review A review does not always focus on what someone said [findings], but how they came about saying what they say [method of analysis]. Reviewing methods of analysis provides a framework of understanding at different levels [i.e. those of theory, substantive fields, research approaches, and data collection and analysis techniques], how researchers draw upon a wide variety of knowledge ranging from the conceptual level to practical documents for use in fieldwork in the areas of ontological and epistemological consideration, quantitative and qualitative integration, sampling, interviewing, data collection, and data analysis. This approach helps highlight ethical issues which you should be aware of and consider as you go through your own study.

Systematic Review This form consists of an overview of existing evidence pertinent to a clearly formulated research question, which uses pre-specified and standardized methods to identify and critically appraise relevant research, and to collect, report, and analyze data from the studies that are included in the review. The goal is to deliberately document, critically evaluate, and summarize scientifically all of the research about a clearly defined research problem . Typically it focuses on a very specific empirical question, often posed in a cause-and-effect form, such as "To what extent does A contribute to B?" This type of literature review is primarily applied to examining prior research studies in clinical medicine and allied health fields, but it is increasingly being used in the social sciences.

Theoretical Review The purpose of this form is to examine the corpus of theory that has accumulated in regard to an issue, concept, theory, phenomena. The theoretical literature review helps to establish what theories already exist, the relationships between them, to what degree the existing theories have been investigated, and to develop new hypotheses to be tested. Often this form is used to help establish a lack of appropriate theories or reveal that current theories are inadequate for explaining new or emerging research problems. The unit of analysis can focus on a theoretical concept or a whole theory or framework.

NOTE: Most often the literature review will incorporate some combination of types. For example, a review that examines literature supporting or refuting an argument, assumption, or philosophical problem related to the research problem will also need to include writing supported by sources that establish the history of these arguments in the literature.

Baumeister, Roy F. and Mark R. Leary. "Writing Narrative Literature Reviews."  Review of General Psychology 1 (September 1997): 311-320; Mark R. Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Kennedy, Mary M. "Defining a Literature." Educational Researcher 36 (April 2007): 139-147; Petticrew, Mark and Helen Roberts. Systematic Reviews in the Social Sciences: A Practical Guide . Malden, MA: Blackwell Publishers, 2006; Torracro, Richard. "Writing Integrative Literature Reviews: Guidelines and Examples." Human Resource Development Review 4 (September 2005): 356-367; Rocco, Tonette S. and Maria S. Plakhotnik. "Literature Reviews, Conceptual Frameworks, and Theoretical Frameworks: Terms, Functions, and Distinctions." Human Ressource Development Review 8 (March 2008): 120-130; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

Structure and Writing Style

I.  Thinking About Your Literature Review

The structure of a literature review should include the following in support of understanding the research problem :

• An overview of the subject, issue, or theory under consideration, along with the objectives of the literature review,
• Division of works under review into themes or categories [e.g. works that support a particular position, those against, and those offering alternative approaches entirely],
• An explanation of how each work is similar to and how it varies from the others,
• Conclusions as to which pieces are best considered in their argument, are most convincing of their opinions, and make the greatest contribution to the understanding and development of their area of research.

The critical evaluation of each work should consider :

• Provenance -- what are the author's credentials? Are the author's arguments supported by evidence [e.g. primary historical material, case studies, narratives, statistics, recent scientific findings]?
• Methodology -- were the techniques used to identify, gather, and analyze the data appropriate to addressing the research problem? Was the sample size appropriate? Were the results effectively interpreted and reported?
• Objectivity -- is the author's perspective even-handed or prejudicial? Is contrary data considered or is certain pertinent information ignored to prove the author's point?
• Persuasiveness -- which of the author's theses are most convincing or least convincing?
• Validity -- are the author's arguments and conclusions convincing? Does the work ultimately contribute in any significant way to an understanding of the subject?

II.  Development of the Literature Review

Four Basic Stages of Writing 1.  Problem formulation -- which topic or field is being examined and what are its component issues? 2.  Literature search -- finding materials relevant to the subject being explored. 3.  Data evaluation -- determining which literature makes a significant contribution to the understanding of the topic. 4.  Analysis and interpretation -- discussing the findings and conclusions of pertinent literature.

Consider the following issues before writing the literature review: Clarify If your assignment is not specific about what form your literature review should take, seek clarification from your professor by asking these questions: 1.  Roughly how many sources would be appropriate to include? 2.  What types of sources should I review (books, journal articles, websites; scholarly versus popular sources)? 3.  Should I summarize, synthesize, or critique sources by discussing a common theme or issue? 4.  Should I evaluate the sources in any way beyond evaluating how they relate to understanding the research problem? 5.  Should I provide subheadings and other background information, such as definitions and/or a history? Find Models Use the exercise of reviewing the literature to examine how authors in your discipline or area of interest have composed their literature review sections. Read them to get a sense of the types of themes you might want to look for in your own research or to identify ways to organize your final review. The bibliography or reference section of sources you've already read, such as required readings in the course syllabus, are also excellent entry points into your own research. Narrow the Topic The narrower your topic, the easier it will be to limit the number of sources you need to read in order to obtain a good survey of relevant resources. Your professor will probably not expect you to read everything that's available about the topic, but you'll make the act of reviewing easier if you first limit scope of the research problem. A good strategy is to begin by searching the USC Libraries Catalog for recent books about the topic and review the table of contents for chapters that focuses on specific issues. You can also review the indexes of books to find references to specific issues that can serve as the focus of your research. For example, a book surveying the history of the Israeli-Palestinian conflict may include a chapter on the role Egypt has played in mediating the conflict, or look in the index for the pages where Egypt is mentioned in the text. Consider Whether Your Sources are Current Some disciplines require that you use information that is as current as possible. This is particularly true in disciplines in medicine and the sciences where research conducted becomes obsolete very quickly as new discoveries are made. However, when writing a review in the social sciences, a survey of the history of the literature may be required. In other words, a complete understanding the research problem requires you to deliberately examine how knowledge and perspectives have changed over time. Sort through other current bibliographies or literature reviews in the field to get a sense of what your discipline expects. You can also use this method to explore what is considered by scholars to be a "hot topic" and what is not.

III.  Ways to Organize Your Literature Review

Chronology of Events If your review follows the chronological method, you could write about the materials according to when they were published. This approach should only be followed if a clear path of research building on previous research can be identified and that these trends follow a clear chronological order of development. For example, a literature review that focuses on continuing research about the emergence of German economic power after the fall of the Soviet Union. By Publication Order your sources by publication chronology, then, only if the order demonstrates a more important trend. For instance, you could order a review of literature on environmental studies of brown fields if the progression revealed, for example, a change in the soil collection practices of the researchers who wrote and/or conducted the studies. Thematic [“conceptual categories”] A thematic literature review is the most common approach to summarizing prior research in the social and behavioral sciences. Thematic reviews are organized around a topic or issue, rather than the progression of time, although the progression of time may still be incorporated into a thematic review. For example, a review of the Internet’s impact on American presidential politics could focus on the development of online political satire. While the study focuses on one topic, the Internet’s impact on American presidential politics, it would still be organized chronologically reflecting technological developments in media. The difference in this example between a "chronological" and a "thematic" approach is what is emphasized the most: themes related to the role of the Internet in presidential politics. Note that more authentic thematic reviews tend to break away from chronological order. A review organized in this manner would shift between time periods within each section according to the point being made. Methodological A methodological approach focuses on the methods utilized by the researcher. For the Internet in American presidential politics project, one methodological approach would be to look at cultural differences between the portrayal of American presidents on American, British, and French websites. Or the review might focus on the fundraising impact of the Internet on a particular political party. A methodological scope will influence either the types of documents in the review or the way in which these documents are discussed.

Other Sections of Your Literature Review Once you've decided on the organizational method for your literature review, the sections you need to include in the paper should be easy to figure out because they arise from your organizational strategy. In other words, a chronological review would have subsections for each vital time period; a thematic review would have subtopics based upon factors that relate to the theme or issue. However, sometimes you may need to add additional sections that are necessary for your study, but do not fit in the organizational strategy of the body. What other sections you include in the body is up to you. However, only include what is necessary for the reader to locate your study within the larger scholarship about the research problem.

Here are examples of other sections, usually in the form of a single paragraph, you may need to include depending on the type of review you write:

• Current Situation : Information necessary to understand the current topic or focus of the literature review.
• Sources Used : Describes the methods and resources [e.g., databases] you used to identify the literature you reviewed.
• History : The chronological progression of the field, the research literature, or an idea that is necessary to understand the literature review, if the body of the literature review is not already a chronology.
• Selection Methods : Criteria you used to select (and perhaps exclude) sources in your literature review. For instance, you might explain that your review includes only peer-reviewed [i.e., scholarly] sources.
• Standards : Description of the way in which you present your information.
• Questions for Further Research : What questions about the field has the review sparked? How will you further your research as a result of the review?

IV.  Writing Your Literature Review

Once you've settled on how to organize your literature review, you're ready to write each section. When writing your review, keep in mind these issues.

Use Evidence A literature review section is, in this sense, just like any other academic research paper. Your interpretation of the available sources must be backed up with evidence [citations] that demonstrates that what you are saying is valid. Be Selective Select only the most important points in each source to highlight in the review. The type of information you choose to mention should relate directly to the research problem, whether it is thematic, methodological, or chronological. Related items that provide additional information, but that are not key to understanding the research problem, can be included in a list of further readings . Use Quotes Sparingly Some short quotes are appropriate if you want to emphasize a point, or if what an author stated cannot be easily paraphrased. Sometimes you may need to quote certain terminology that was coined by the author, is not common knowledge, or taken directly from the study. Do not use extensive quotes as a substitute for using your own words in reviewing the literature. Summarize and Synthesize Remember to summarize and synthesize your sources within each thematic paragraph as well as throughout the review. Recapitulate important features of a research study, but then synthesize it by rephrasing the study's significance and relating it to your own work and the work of others. Keep Your Own Voice While the literature review presents others' ideas, your voice [the writer's] should remain front and center. For example, weave references to other sources into what you are writing but maintain your own voice by starting and ending the paragraph with your own ideas and wording. Use Caution When Paraphrasing When paraphrasing a source that is not your own, be sure to represent the author's information or opinions accurately and in your own words. Even when paraphrasing an author’s work, you still must provide a citation to that work.

V.  Common Mistakes to Avoid

These are the most common mistakes made in reviewing social science research literature.

• Sources in your literature review do not clearly relate to the research problem;
• You do not take sufficient time to define and identify the most relevant sources to use in the literature review related to the research problem;
• Relies exclusively on secondary analytical sources rather than including relevant primary research studies or data;
• Uncritically accepts another researcher's findings and interpretations as valid, rather than examining critically all aspects of the research design and analysis;
• Does not describe the search procedures that were used in identifying the literature to review;
• Reports isolated statistical results rather than synthesizing them in chi-squared or meta-analytic methods; and,
• Only includes research that validates assumptions and does not consider contrary findings and alternative interpretations found in the literature.

Cook, Kathleen E. and Elise Murowchick. “Do Literature Review Skills Transfer from One Course to Another?” Psychology Learning and Teaching 13 (March 2014): 3-11; Fink, Arlene. Conducting Research Literature Reviews: From the Internet to Paper . 2nd ed. Thousand Oaks, CA: Sage, 2005; Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1998; Jesson, Jill. Doing Your Literature Review: Traditional and Systematic Techniques . London: SAGE, 2011; Literature Review Handout. Online Writing Center. Liberty University; Literature Reviews. The Writing Center. University of North Carolina; Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: SAGE, 2016; Ridley, Diana. The Literature Review: A Step-by-Step Guide for Students . 2nd ed. Los Angeles, CA: SAGE, 2012; Randolph, Justus J. “A Guide to Writing the Dissertation Literature Review." Practical Assessment, Research, and Evaluation. vol. 14, June 2009; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016; Taylor, Dena. The Literature Review: A Few Tips On Conducting It. University College Writing Centre. University of Toronto; Writing a Literature Review. Academic Skills Centre. University of Canberra.

Writing Tip

Break Out of Your Disciplinary Box!

Thinking interdisciplinarily about a research problem can be a rewarding exercise in applying new ideas, theories, or concepts to an old problem. For example, what might cultural anthropologists say about the continuing conflict in the Middle East? In what ways might geographers view the need for better distribution of social service agencies in large cities than how social workers might study the issue? You don’t want to substitute a thorough review of core research literature in your discipline for studies conducted in other fields of study. However, particularly in the social sciences, thinking about research problems from multiple vectors is a key strategy for finding new solutions to a problem or gaining a new perspective. Consult with a librarian about identifying research databases in other disciplines; almost every field of study has at least one comprehensive database devoted to indexing its research literature.

Frodeman, Robert. The Oxford Handbook of Interdisciplinarity . New York: Oxford University Press, 2010.

Another Writing Tip

Don't Just Review for Content!

While conducting a review of the literature, maximize the time you devote to writing this part of your paper by thinking broadly about what you should be looking for and evaluating. Review not just what scholars are saying, but how are they saying it. Some questions to ask:

• How are they organizing their ideas?
• What methods have they used to study the problem?
• What theories have been used to explain, predict, or understand their research problem?
• What sources have they cited to support their conclusions?
• How have they used non-textual elements [e.g., charts, graphs, figures, etc.] to illustrate key points?

When you begin to write your literature review section, you'll be glad you dug deeper into how the research was designed and constructed because it establishes a means for developing more substantial analysis and interpretation of the research problem.

Hart, Chris. Doing a Literature Review: Releasing the Social Science Research Imagination . Thousand Oaks, CA: Sage Publications, 1 998.

Yet Another Writing Tip

When Do I Know I Can Stop Looking and Move On?

Here are several strategies you can utilize to assess whether you've thoroughly reviewed the literature:

• Look for repeating patterns in the research findings . If the same thing is being said, just by different people, then this likely demonstrates that the research problem has hit a conceptual dead end. At this point consider: Does your study extend current research?  Does it forge a new path? Or, does is merely add more of the same thing being said?
• Look at sources the authors cite to in their work . If you begin to see the same researchers cited again and again, then this is often an indication that no new ideas have been generated to address the research problem.
• Search Google Scholar to identify who has subsequently cited leading scholars already identified in your literature review [see next sub-tab]. This is called citation tracking and there are a number of sources that can help you identify who has cited whom, particularly scholars from outside of your discipline. Here again, if the same authors are being cited again and again, this may indicate no new literature has been written on the topic.

Onwuegbuzie, Anthony J. and Rebecca Frels. Seven Steps to a Comprehensive Literature Review: A Multimodal and Cultural Approach . Los Angeles, CA: Sage, 2016; Sutton, Anthea. Systematic Approaches to a Successful Literature Review . Los Angeles, CA: Sage Publications, 2016.

• << Previous: Theoretical Framework
• Next: Citation Tracking >>
• Last Updated: May 30, 2024 9:38 AM
• URL: https://libguides.usc.edu/writingguide

Have a language expert improve your writing

Run a free plagiarism check in 10 minutes, automatically generate references for free.

• Knowledge Base
• Dissertation
• What is a Literature Review? | Guide, Template, & Examples

What is a Literature Review? | Guide, Template, & Examples

Published on 22 February 2022 by Shona McCombes . Revised on 7 June 2022.

What is a literature review? A literature review is a survey of scholarly sources on a specific topic. It provides an overview of current knowledge, allowing you to identify relevant theories, methods, and gaps in the existing research.

There are five key steps to writing a literature review:

• Search for relevant literature
• Evaluate sources
• Identify themes, debates and gaps
• Outline the structure
• Write your literature review

A good literature review doesn’t just summarise sources – it analyses, synthesises, and critically evaluates to give a clear picture of the state of knowledge on the subject.

Instantly correct all language mistakes in your text

Be assured that you'll submit flawless writing. Upload your document to correct all your mistakes.

Table of contents

Why write a literature review, examples of literature reviews, step 1: search for relevant literature, step 2: evaluate and select sources, step 3: identify themes, debates and gaps, step 4: outline your literature review’s structure, step 5: write your literature review, frequently asked questions about literature reviews, introduction.

• Quick Run-through
• Step 1 & 2

When you write a dissertation or thesis, you will have to conduct a literature review to situate your research within existing knowledge. The literature review gives you a chance to:

• Demonstrate your familiarity with the topic and scholarly context
• Develop a theoretical framework and methodology for your research
• Position yourself in relation to other researchers and theorists
• Show how your dissertation addresses a gap or contributes to a debate

You might also have to write a literature review as a stand-alone assignment. In this case, the purpose is to evaluate the current state of research and demonstrate your knowledge of scholarly debates around a topic.

The content will look slightly different in each case, but the process of conducting a literature review follows the same steps. We’ve written a step-by-step guide that you can follow below.

Prevent plagiarism, run a free check.

Writing literature reviews can be quite challenging! A good starting point could be to look at some examples, depending on what kind of literature review you’d like to write.

• Example literature review #1: “Why Do People Migrate? A Review of the Theoretical Literature” ( Theoretical literature review about the development of economic migration theory from the 1950s to today.)
• Example literature review #2: “Literature review as a research methodology: An overview and guidelines” ( Methodological literature review about interdisciplinary knowledge acquisition and production.)
• Example literature review #3: “The Use of Technology in English Language Learning: A Literature Review” ( Thematic literature review about the effects of technology on language acquisition.)
• Example literature review #4: “Learners’ Listening Comprehension Difficulties in English Language Learning: A Literature Review” ( Chronological literature review about how the concept of listening skills has changed over time.)

You can also check out our templates with literature review examples and sample outlines at the links below.

Download Word doc Download Google doc

Before you begin searching for literature, you need a clearly defined topic .

If you are writing the literature review section of a dissertation or research paper, you will search for literature related to your research objectives and questions .

If you are writing a literature review as a stand-alone assignment, you will have to choose a focus and develop a central question to direct your search. Unlike a dissertation research question, this question has to be answerable without collecting original data. You should be able to answer it based only on a review of existing publications.

Make a list of keywords

Start by creating a list of keywords related to your research topic. Include each of the key concepts or variables you’re interested in, and list any synonyms and related terms. You can add to this list if you discover new keywords in the process of your literature search.

• Social media, Facebook, Instagram, Twitter, Snapchat, TikTok
• Body image, self-perception, self-esteem, mental health
• Generation Z, teenagers, adolescents, youth

Search for relevant sources

Use your keywords to begin searching for sources. Some databases to search for journals and articles include:

• Your university’s library catalogue
• Google Scholar
• Project Muse (humanities and social sciences)
• Medline (life sciences and biomedicine)
• EconLit (economics)
• Inspec (physics, engineering and computer science)

You can use boolean operators to help narrow down your search:

Read the abstract to find out whether an article is relevant to your question. When you find a useful book or article, you can check the bibliography to find other relevant sources.

To identify the most important publications on your topic, take note of recurring citations. If the same authors, books or articles keep appearing in your reading, make sure to seek them out.

You probably won’t be able to read absolutely everything that has been written on the topic – you’ll have to evaluate which sources are most relevant to your questions.

For each publication, ask yourself:

• What question or problem is the author addressing?
• What are the key concepts and how are they defined?
• What are the key theories, models and methods? Does the research use established frameworks or take an innovative approach?
• What are the results and conclusions of the study?
• How does the publication relate to other literature in the field? Does it confirm, add to, or challenge established knowledge?
• How does the publication contribute to your understanding of the topic? What are its key insights and arguments?
• What are the strengths and weaknesses of the research?

Make sure the sources you use are credible, and make sure you read any landmark studies and major theories in your field of research.

You can find out how many times an article has been cited on Google Scholar – a high citation count means the article has been influential in the field, and should certainly be included in your literature review.

The scope of your review will depend on your topic and discipline: in the sciences you usually only review recent literature, but in the humanities you might take a long historical perspective (for example, to trace how a concept has changed in meaning over time).

Remember that you can use our template to summarise and evaluate sources you’re thinking about using!

Take notes and cite your sources

As you read, you should also begin the writing process. Take notes that you can later incorporate into the text of your literature review.

It’s important to keep track of your sources with references to avoid plagiarism . It can be helpful to make an annotated bibliography, where you compile full reference information and write a paragraph of summary and analysis for each source. This helps you remember what you read and saves time later in the process.

You can use our free APA Reference Generator for quick, correct, consistent citations.

To begin organising your literature review’s argument and structure, you need to understand the connections and relationships between the sources you’ve read. Based on your reading and notes, you can look for:

• Trends and patterns (in theory, method or results): do certain approaches become more or less popular over time?
• Themes: what questions or concepts recur across the literature?
• Debates, conflicts and contradictions: where do sources disagree?
• Pivotal publications: are there any influential theories or studies that changed the direction of the field?
• Gaps: what is missing from the literature? Are there weaknesses that need to be addressed?

This step will help you work out the structure of your literature review and (if applicable) show how your own research will contribute to existing knowledge.

• Most research has focused on young women.
• There is an increasing interest in the visual aspects of social media.
• But there is still a lack of robust research on highly-visual platforms like Instagram and Snapchat – this is a gap that you could address in your own research.

There are various approaches to organising the body of a literature review. You should have a rough idea of your strategy before you start writing.

Depending on the length of your literature review, you can combine several of these strategies (for example, your overall structure might be thematic, but each theme is discussed chronologically).

Chronological

The simplest approach is to trace the development of the topic over time. However, if you choose this strategy, be careful to avoid simply listing and summarising sources in order.

Try to analyse patterns, turning points and key debates that have shaped the direction of the field. Give your interpretation of how and why certain developments occurred.

If you have found some recurring central themes, you can organise your literature review into subsections that address different aspects of the topic.

For example, if you are reviewing literature about inequalities in migrant health outcomes, key themes might include healthcare policy, language barriers, cultural attitudes, legal status, and economic access.

Methodological

If you draw your sources from different disciplines or fields that use a variety of research methods , you might want to compare the results and conclusions that emerge from different approaches. For example:

• Look at what results have emerged in qualitative versus quantitative research
• Discuss how the topic has been approached by empirical versus theoretical scholarship
• Divide the literature into sociological, historical, and cultural sources

Theoretical

A literature review is often the foundation for a theoretical framework . You can use it to discuss various theories, models, and definitions of key concepts.

You might argue for the relevance of a specific theoretical approach, or combine various theoretical concepts to create a framework for your research.

Like any other academic text, your literature review should have an introduction , a main body, and a conclusion . What you include in each depends on the objective of your literature review.

The introduction should clearly establish the focus and purpose of the literature review.

If you are writing the literature review as part of your dissertation or thesis, reiterate your central problem or research question and give a brief summary of the scholarly context. You can emphasise the timeliness of the topic (“many recent studies have focused on the problem of x”) or highlight a gap in the literature (“while there has been much research on x, few researchers have taken y into consideration”).

Depending on the length of your literature review, you might want to divide the body into subsections. You can use a subheading for each theme, time period, or methodological approach.

As you write, make sure to follow these tips:

• Summarise and synthesise: give an overview of the main points of each source and combine them into a coherent whole.
• Analyse and interpret: don’t just paraphrase other researchers – add your own interpretations, discussing the significance of findings in relation to the literature as a whole.
• Critically evaluate: mention the strengths and weaknesses of your sources.
• Write in well-structured paragraphs: use transitions and topic sentences to draw connections, comparisons and contrasts.

In the conclusion, you should summarise the key findings you have taken from the literature and emphasise their significance.

If the literature review is part of your dissertation or thesis, reiterate how your research addresses gaps and contributes new knowledge, or discuss how you have drawn on existing theories and methods to build a framework for your research. This can lead directly into your methodology section.

A literature review is a survey of scholarly sources (such as books, journal articles, and theses) related to a specific topic or research question .

It is often written as part of a dissertation , thesis, research paper , or proposal .

There are several reasons to conduct a literature review at the beginning of a research project:

• To familiarise yourself with the current state of knowledge on your topic
• To ensure that you’re not just repeating what others have already done
• To identify gaps in knowledge and unresolved problems that your research can address
• To develop your theoretical framework and methodology
• To provide an overview of the key findings and debates on the topic

Writing the literature review shows your reader how your work relates to existing research and what new insights it will contribute.

The literature review usually comes near the beginning of your  dissertation . After the introduction , it grounds your research in a scholarly field and leads directly to your theoretical framework or methodology .

Cite this Scribbr article

If you want to cite this source, you can copy and paste the citation or click the ‘Cite this Scribbr article’ button to automatically add the citation to our free Reference Generator.

McCombes, S. (2022, June 07). What is a Literature Review? | Guide, Template, & Examples. Scribbr. Retrieved 3 June 2024, from https://www.scribbr.co.uk/thesis-dissertation/literature-review/

Is this article helpful?

Shona McCombes

Other students also liked, how to write a dissertation proposal | a step-by-step guide, what is a theoretical framework | a step-by-step guide, what is a research methodology | steps & tips.

Libraries | Research Guides

Literature reviews, what is a literature review, learning more about how to do a literature review.

• Planning the Review
• The Research Question
• Choosing Where to Search
• Organizing the Review
• Writing the Review

A literature review is a review and synthesis of existing research on a topic or research question. A literature review is meant to analyze the scholarly literature, make connections across writings and identify strengths, weaknesses, trends, and missing conversations. A literature review should address different aspects of a topic as it relates to your research question. A literature review goes beyond a description or summary of the literature you have read. 

• Sage Research Methods Core Collection This link opens in a new window SAGE Research Methods supports research at all levels by providing material to guide users through every step of the research process. SAGE Research Methods is the ultimate methods library with more than 1000 books, reference works, journal articles, and instructional videos by world-leading academics from across the social sciences, including the largest collection of qualitative methods books available online from any scholarly publisher. – Publisher

• Next: Planning the Review >>
• Last Updated: May 2, 2024 10:39 AM
• URL: https://libguides.northwestern.edu/literaturereviews

• University of Oregon Libraries
• Research Guides

How to Write a Literature Review

Who is my subject librarian.

• Literature Reviews: A Recap
• Reading Journal Articles
• Does it Describe a Literature Review?
• 1. Identify the Question
• 2. Review Discipline Styles
• Searching Article Databases
• Finding Full-Text of an Article
• Citation Chaining
• When to Stop Searching
• 4. Manage Your References
• 5. Critically Analyze and Evaluate
• 6. Synthesize
• 7. Write a Literature Review

This guide has a Creative Commons Attribution-NonCommercial-ShareAlike (CC BY-NC-SA) License.

This license lets others remix, tweak, and build upon your work non-commercially, as long as they credit you and license their new creations under the identical terms.

• Attribution-NonCommercial-ShareAlike 4.0 International (CC BY-NC-SA 4.0) License Deed

This guide is organized around the 7 steps in the graphic to the left to help you understand the process of writing a literature review.

This guide will help you to

• Define a literature review
• Recognize that different fields of study have their own way to perform and write literature reviews
• Prepare to search the literature
• Read critically -- analyze and synthesize
• Prepare to write a literature review

At the end of the tutorial, you will find a quiz that you can submit through Canvas for course credit. 

Graphic from Literature Review (2009) by Machi  and McEvoy.

Text description of "Literature Review Process" for web accessibility

If you have questions related to a field or discipline, consider reaching out to a Subject Librarian by email, phone, or by scheduling an appointment for a free consultation:

• Subject Librarians
• Next: What's a Literature Review? >>
• Last Updated: May 3, 2024 5:17 PM
• URL: https://researchguides.uoregon.edu/litreview

Contact Us Library Accessibility UO Libraries Privacy Notices and Procedures

1501 Kincaid Street Eugene, OR 97403 P: 541-346-3053 F: 541-346-3485

• Visit us on Facebook
• Visit us on Twitter
• Visit us on Youtube
• Visit us on Instagram
• Report a Concern
• Nondiscrimination and Title IX
• Accessibility
• Privacy Policy
• Find People

• University of Texas Libraries

Literature Reviews

• What is a literature review?
• Steps in the Literature Review Process
• Define your research question
• Determine inclusion and exclusion criteria
• Choose databases and search
• Review Results
• Synthesize Results
• Analyze Results
• Librarian Support

What is a Literature Review?

A literature or narrative review is a comprehensive review and analysis of the published literature on a specific topic or research question. The literature that is reviewed contains: books, articles, academic articles, conference proceedings, association papers, and dissertations. It contains the most pertinent studies and points to important past and current research and practices. It provides background and context, and shows how your research will contribute to the field. 

A literature review should: 

• Provide a comprehensive and updated review of the literature;
• Explain why this review has taken place;
• Articulate a position or hypothesis;
• Acknowledge and account for conflicting and corroborating points of view

From  S age Research Methods

Purpose of a Literature Review

A literature review can be written as an introduction to a study to:

• Demonstrate how a study fills a gap in research
• Compare a study with other research that's been done

Or it can be a separate work (a research article on its own) which:

• Organizes or describes a topic
• Describes variables within a particular issue/problem

Limitations of a Literature Review

Some of the limitations of a literature review are:

• It's a snapshot in time. Unlike other reviews, this one has beginning, a middle and an end. There may be future developments that could make your work less relevant.
• It may be too focused. Some niche studies may miss the bigger picture.
• It can be difficult to be comprehensive. There is no way to make sure all the literature on a topic was considered.
• It is easy to be biased if you stick to top tier journals. There may be other places where people are publishing exemplary research. Look to open access publications and conferences to reflect a more inclusive collection. Also, make sure to include opposing views (and not just supporting evidence).

Source: Grant, Maria J., and Andrew Booth. “A Typology of Reviews: An Analysis of 14 Review Types and Associated Methodologies.” Health Information & Libraries Journal, vol. 26, no. 2, June 2009, pp. 91–108. Wiley Online Library, doi:10.1111/j.1471-1842.2009.00848.x.

Meryl Brodsky : Communication and Information Studies

Hannah Chapman Tripp : Biology, Neuroscience

Carolyn Cunningham : Human Development & Family Sciences, Psychology, Sociology

Larayne Dallas : Engineering

Janelle Hedstrom : Special Education, Curriculum & Instruction, Ed Leadership & Policy ​

Susan Macicak : Linguistics

Imelda Vetter : Dell Medical School

For help in other subject areas, please see the guide to library specialists by subject .

Periodically, UT Libraries runs a workshop covering the basics and library support for literature reviews. While we try to offer these once per academic year, we find providing the recording to be helpful to community members who have missed the session. Following is the most recent recording of the workshop, Conducting a Literature Review. To view the recording, a UT login is required.

• October 26, 2022 recording
• Last Updated: Oct 26, 2022 2:49 PM
• URL: https://guides.lib.utexas.edu/literaturereviews

Get science-backed answers as you write with Paperpal's Research feature

What is a Literature Review? How to Write It (with Examples)

A literature review is a critical analysis and synthesis of existing research on a particular topic. It provides an overview of the current state of knowledge, identifies gaps, and highlights key findings in the literature. 1 The purpose of a literature review is to situate your own research within the context of existing scholarship, demonstrating your understanding of the topic and showing how your work contributes to the ongoing conversation in the field. Learning how to write a literature review is a critical tool for successful research. Your ability to summarize and synthesize prior research pertaining to a certain topic demonstrates your grasp on the topic of study, and assists in the learning process. 

Table of Contents

• What is the purpose of literature review? 
• a. Habitat Loss and Species Extinction: 
• b. Range Shifts and Phenological Changes: 
• c. Ocean Acidification and Coral Reefs: 
• d. Adaptive Strategies and Conservation Efforts: 

How to write a good literature review 

• Choose a Topic and Define the Research Question: 
• Decide on the Scope of Your Review: 
• Select Databases for Searches: 
• Conduct Searches and Keep Track: 
• Review the Literature: 
• Organize and Write Your Literature Review: 
• How to write a literature review faster with Paperpal? 
• Frequently asked questions 

What is a literature review?

A well-conducted literature review demonstrates the researcher’s familiarity with the existing literature, establishes the context for their own research, and contributes to scholarly conversations on the topic. One of the purposes of a literature review is also to help researchers avoid duplicating previous work and ensure that their research is informed by and builds upon the existing body of knowledge.

What is the purpose of literature review?

A literature review serves several important purposes within academic and research contexts. Here are some key objectives and functions of a literature review: 2  

1. Contextualizing the Research Problem: The literature review provides a background and context for the research problem under investigation. It helps to situate the study within the existing body of knowledge. 

2. Identifying Gaps in Knowledge: By identifying gaps, contradictions, or areas requiring further research, the researcher can shape the research question and justify the significance of the study. This is crucial for ensuring that the new research contributes something novel to the field. 

Find academic papers related to your research topic faster. Try Research on Paperpal  

3. Understanding Theoretical and Conceptual Frameworks: Literature reviews help researchers gain an understanding of the theoretical and conceptual frameworks used in previous studies. This aids in the development of a theoretical framework for the current research. 

4. Providing Methodological Insights: Another purpose of literature reviews is that it allows researchers to learn about the methodologies employed in previous studies. This can help in choosing appropriate research methods for the current study and avoiding pitfalls that others may have encountered. 

5. Establishing Credibility: A well-conducted literature review demonstrates the researcher’s familiarity with existing scholarship, establishing their credibility and expertise in the field. It also helps in building a solid foundation for the new research. 

6. Informing Hypotheses or Research Questions: The literature review guides the formulation of hypotheses or research questions by highlighting relevant findings and areas of uncertainty in existing literature. 

Literature review example

Let’s delve deeper with a literature review example: Let’s say your literature review is about the impact of climate change on biodiversity. You might format your literature review into sections such as the effects of climate change on habitat loss and species extinction, phenological changes, and marine biodiversity. Each section would then summarize and analyze relevant studies in those areas, highlighting key findings and identifying gaps in the research. The review would conclude by emphasizing the need for further research on specific aspects of the relationship between climate change and biodiversity. The following literature review template provides a glimpse into the recommended literature review structure and content, demonstrating how research findings are organized around specific themes within a broader topic. 

Literature Review on Climate Change Impacts on Biodiversity:

Climate change is a global phenomenon with far-reaching consequences, including significant impacts on biodiversity. This literature review synthesizes key findings from various studies: 

a. Habitat Loss and Species Extinction:

Climate change-induced alterations in temperature and precipitation patterns contribute to habitat loss, affecting numerous species (Thomas et al., 2004). The review discusses how these changes increase the risk of extinction, particularly for species with specific habitat requirements. 

b. Range Shifts and Phenological Changes:

Observations of range shifts and changes in the timing of biological events (phenology) are documented in response to changing climatic conditions (Parmesan & Yohe, 2003). These shifts affect ecosystems and may lead to mismatches between species and their resources. 

c. Ocean Acidification and Coral Reefs:

The review explores the impact of climate change on marine biodiversity, emphasizing ocean acidification’s threat to coral reefs (Hoegh-Guldberg et al., 2007). Changes in pH levels negatively affect coral calcification, disrupting the delicate balance of marine ecosystems. 

d. Adaptive Strategies and Conservation Efforts:

Recognizing the urgency of the situation, the literature review discusses various adaptive strategies adopted by species and conservation efforts aimed at mitigating the impacts of climate change on biodiversity (Hannah et al., 2007). It emphasizes the importance of interdisciplinary approaches for effective conservation planning. 

Strengthen your literature review with factual insights. Try Research on Paperpal for free!    

Writing a literature review involves summarizing and synthesizing existing research on a particular topic. A good literature review format should include the following elements. 

Introduction: The introduction sets the stage for your literature review, providing context and introducing the main focus of your review. 

• Opening Statement: Begin with a general statement about the broader topic and its significance in the field. 
• Scope and Purpose: Clearly define the scope of your literature review. Explain the specific research question or objective you aim to address. 
• Organizational Framework: Briefly outline the structure of your literature review, indicating how you will categorize and discuss the existing research. 
• Significance of the Study: Highlight why your literature review is important and how it contributes to the understanding of the chosen topic. 
• Thesis Statement: Conclude the introduction with a concise thesis statement that outlines the main argument or perspective you will develop in the body of the literature review. 

Body: The body of the literature review is where you provide a comprehensive analysis of existing literature, grouping studies based on themes, methodologies, or other relevant criteria. 

• Organize by Theme or Concept: Group studies that share common themes, concepts, or methodologies. Discuss each theme or concept in detail, summarizing key findings and identifying gaps or areas of disagreement. 
• Critical Analysis: Evaluate the strengths and weaknesses of each study. Discuss the methodologies used, the quality of evidence, and the overall contribution of each work to the understanding of the topic. 
• Synthesis of Findings: Synthesize the information from different studies to highlight trends, patterns, or areas of consensus in the literature. 
• Identification of Gaps: Discuss any gaps or limitations in the existing research and explain how your review contributes to filling these gaps. 
• Transition between Sections: Provide smooth transitions between different themes or concepts to maintain the flow of your literature review. 

Write and Cite as you go with Paperpal Research. Start now for free.   

Conclusion: The conclusion of your literature review should summarize the main findings, highlight the contributions of the review, and suggest avenues for future research. 

• Summary of Key Findings: Recap the main findings from the literature and restate how they contribute to your research question or objective. 
• Contributions to the Field: Discuss the overall contribution of your literature review to the existing knowledge in the field. 
• Implications and Applications: Explore the practical implications of the findings and suggest how they might impact future research or practice. 
• Recommendations for Future Research: Identify areas that require further investigation and propose potential directions for future research in the field. 
• Final Thoughts: Conclude with a final reflection on the importance of your literature review and its relevance to the broader academic community. 

Conducting a literature review

Conducting a literature review is an essential step in research that involves reviewing and analyzing existing literature on a specific topic. It’s important to know how to do a literature review effectively, so here are the steps to follow: 1  

Choose a Topic and Define the Research Question:

• Select a topic that is relevant to your field of study. 
• Clearly define your research question or objective. Determine what specific aspect of the topic do you want to explore? 

Decide on the Scope of Your Review:

• Determine the timeframe for your literature review. Are you focusing on recent developments, or do you want a historical overview? 
• Consider the geographical scope. Is your review global, or are you focusing on a specific region? 
• Define the inclusion and exclusion criteria. What types of sources will you include? Are there specific types of studies or publications you will exclude? 

Select Databases for Searches:

• Identify relevant databases for your field. Examples include PubMed, IEEE Xplore, Scopus, Web of Science, and Google Scholar. 
• Consider searching in library catalogs, institutional repositories, and specialized databases related to your topic. 

Conduct Searches and Keep Track:

• Develop a systematic search strategy using keywords, Boolean operators (AND, OR, NOT), and other search techniques. 
• Record and document your search strategy for transparency and replicability. 
• Keep track of the articles, including publication details, abstracts, and links. Use citation management tools like EndNote, Zotero, or Mendeley to organize your references. 

Review the Literature:

• Evaluate the relevance and quality of each source. Consider the methodology, sample size, and results of studies. 
• Organize the literature by themes or key concepts. Identify patterns, trends, and gaps in the existing research. 
• Summarize key findings and arguments from each source. Compare and contrast different perspectives. 
• Identify areas where there is a consensus in the literature and where there are conflicting opinions. 
• Provide critical analysis and synthesis of the literature. What are the strengths and weaknesses of existing research? 

Organize and Write Your Literature Review:

• Literature review outline should be based on themes, chronological order, or methodological approaches. 
• Write a clear and coherent narrative that synthesizes the information gathered. 
• Use proper citations for each source and ensure consistency in your citation style (APA, MLA, Chicago, etc.). 
• Conclude your literature review by summarizing key findings, identifying gaps, and suggesting areas for future research. 

Whether you’re exploring a new research field or finding new angles to develop an existing topic, sifting through hundreds of papers can take more time than you have to spare. But what if you could find science-backed insights with verified citations in seconds? That’s the power of Paperpal’s new Research feature!  

How to write a literature review faster with Paperpal?

Paperpal, an AI writing assistant, integrates powerful academic search capabilities within its writing platform. With the Research feature, you get 100% factual insights, with citations backed by 250M+ verified research articles, directly within your writing interface with the option to save relevant references in your Citation Library. By eliminating the need to switch tabs to find answers to all your research questions, Paperpal saves time and helps you stay focused on your writing.   

Here’s how to use the Research feature:  

• Ask a question: Get started with a new document on paperpal.com. Click on the “Research” feature and type your question in plain English. Paperpal will scour over 250 million research articles, including conference papers and preprints, to provide you with accurate insights and citations. 
• Review and Save: Paperpal summarizes the information, while citing sources and listing relevant reads. You can quickly scan the results to identify relevant references and save these directly to your built-in citations library for later access. 
• Cite with Confidence: Paperpal makes it easy to incorporate relevant citations and references into your writing, ensuring your arguments are well-supported by credible sources. This translates to a polished, well-researched literature review. 

The literature review sample and detailed advice on writing and conducting a review will help you produce a well-structured report. But remember that a good literature review is an ongoing process, and it may be necessary to revisit and update it as your research progresses. By combining effortless research with an easy citation process, Paperpal Research streamlines the literature review process and empowers you to write faster and with more confidence. Try Paperpal Research now and see for yourself.  

Frequently asked questions

A literature review is a critical and comprehensive analysis of existing literature (published and unpublished works) on a specific topic or research question and provides a synthesis of the current state of knowledge in a particular field. A well-conducted literature review is crucial for researchers to build upon existing knowledge, avoid duplication of efforts, and contribute to the advancement of their field. It also helps researchers situate their work within a broader context and facilitates the development of a sound theoretical and conceptual framework for their studies.

Literature review is a crucial component of research writing, providing a solid background for a research paper’s investigation. The aim is to keep professionals up to date by providing an understanding of ongoing developments within a specific field, including research methods, and experimental techniques used in that field, and present that knowledge in the form of a written report. Also, the depth and breadth of the literature review emphasizes the credibility of the scholar in his or her field.  

Before writing a literature review, it’s essential to undertake several preparatory steps to ensure that your review is well-researched, organized, and focused. This includes choosing a topic of general interest to you and doing exploratory research on that topic, writing an annotated bibliography, and noting major points, especially those that relate to the position you have taken on the topic. 

Literature reviews and academic research papers are essential components of scholarly work but serve different purposes within the academic realm. 3 A literature review aims to provide a foundation for understanding the current state of research on a particular topic, identify gaps or controversies, and lay the groundwork for future research. Therefore, it draws heavily from existing academic sources, including books, journal articles, and other scholarly publications. In contrast, an academic research paper aims to present new knowledge, contribute to the academic discourse, and advance the understanding of a specific research question. Therefore, it involves a mix of existing literature (in the introduction and literature review sections) and original data or findings obtained through research methods. 

Literature reviews are essential components of academic and research papers, and various strategies can be employed to conduct them effectively. If you want to know how to write a literature review for a research paper, here are four common approaches that are often used by researchers.  Chronological Review: This strategy involves organizing the literature based on the chronological order of publication. It helps to trace the development of a topic over time, showing how ideas, theories, and research have evolved.  Thematic Review: Thematic reviews focus on identifying and analyzing themes or topics that cut across different studies. Instead of organizing the literature chronologically, it is grouped by key themes or concepts, allowing for a comprehensive exploration of various aspects of the topic.  Methodological Review: This strategy involves organizing the literature based on the research methods employed in different studies. It helps to highlight the strengths and weaknesses of various methodologies and allows the reader to evaluate the reliability and validity of the research findings.  Theoretical Review: A theoretical review examines the literature based on the theoretical frameworks used in different studies. This approach helps to identify the key theories that have been applied to the topic and assess their contributions to the understanding of the subject.  It’s important to note that these strategies are not mutually exclusive, and a literature review may combine elements of more than one approach. The choice of strategy depends on the research question, the nature of the literature available, and the goals of the review. Additionally, other strategies, such as integrative reviews or systematic reviews, may be employed depending on the specific requirements of the research.

The literature review format can vary depending on the specific publication guidelines. However, there are some common elements and structures that are often followed. Here is a general guideline for the format of a literature review:  Introduction:   Provide an overview of the topic.  Define the scope and purpose of the literature review.  State the research question or objective.  Body:   Organize the literature by themes, concepts, or chronology.  Critically analyze and evaluate each source.  Discuss the strengths and weaknesses of the studies.  Highlight any methodological limitations or biases.  Identify patterns, connections, or contradictions in the existing research.  Conclusion:   Summarize the key points discussed in the literature review.  Highlight the research gap.  Address the research question or objective stated in the introduction.  Highlight the contributions of the review and suggest directions for future research.

Both annotated bibliographies and literature reviews involve the examination of scholarly sources. While annotated bibliographies focus on individual sources with brief annotations, literature reviews provide a more in-depth, integrated, and comprehensive analysis of existing literature on a specific topic. The key differences are as follows: 

References 

• Denney, A. S., & Tewksbury, R. (2013). How to write a literature review.  Journal of criminal justice education ,  24 (2), 218-234. 
• Pan, M. L. (2016).  Preparing literature reviews: Qualitative and quantitative approaches . Taylor & Francis. 
• Cantero, C. (2019). How to write a literature review.  San José State University Writing Center . 

Paperpal is an AI writing assistant that help academics write better, faster with real-time suggestions for in-depth language and grammar correction. Trained on millions of research manuscripts enhanced by professional academic editors, Paperpal delivers human precision at machine speed.  

Try it for free or upgrade to  Paperpal Prime , which unlocks unlimited access to premium features like academic translation, paraphrasing, contextual synonyms, consistency checks and more. It’s like always having a professional academic editor by your side! Go beyond limitations and experience the future of academic writing.  Get Paperpal Prime now at just US$19 a month!

Related Reads:

• Empirical Research: A Comprehensive Guide for Academics 
• How to Write a Scientific Paper in 10 Steps 
• How Long Should a Chapter Be?
• How to Use Paperpal to Generate Emails & Cover Letters?

6 Tips for Post-Doc Researchers to Take Their Career to the Next Level

Self-plagiarism in research: what it is and how to avoid it, you may also like, how to write the first draft of a..., mla works cited page: format, template & examples, how to ace grant writing for research funding..., powerful academic phrases to improve your essay writing , how to write a high-quality conference paper, how paperpal’s research feature helps you develop and..., how paperpal is enhancing academic productivity and accelerating..., how to write a successful book chapter for..., academic editing: how to self-edit academic text with..., 4 ways paperpal encourages responsible writing with ai.

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

2.3 Reviewing the Research Literature

Learning objectives.

• Define the research literature in psychology and give examples of sources that are part of the research literature and sources that are not.
• Describe and use several methods for finding previous research on a particular research idea or question.

Reviewing the research literature means finding, reading, and summarizing the published research relevant to your question. An empirical research report written in American Psychological Association (APA) style always includes a written literature review, but it is important to review the literature early in the research process for several reasons.

• It can help you turn a research idea into an interesting research question.
• It can tell you if a research question has already been answered.
• It can help you evaluate the interestingness of a research question.
• It can give you ideas for how to conduct your own study.
• It can tell you how your study fits into the research literature.

What Is the Research Literature?

The research literature in any field is all the published research in that field. The research literature in psychology is enormous—including millions of scholarly articles and books dating to the beginning of the field—and it continues to grow. Although its boundaries are somewhat fuzzy, the research literature definitely does not include self-help and other pop psychology books, dictionary and encyclopedia entries, websites, and similar sources that are intended mainly for the general public. These are considered unreliable because they are not reviewed by other researchers and are often based on little more than common sense or personal experience. Wikipedia contains much valuable information, but the fact that its authors are anonymous and its content continually changes makes it unsuitable as a basis of sound scientific research. For our purposes, it helps to define the research literature as consisting almost entirely of two types of sources: articles in professional journals, and scholarly books in psychology and related fields.

Professional Journals

Professional journals are periodicals that publish original research articles. There are thousands of professional journals that publish research in psychology and related fields. They are usually published monthly or quarterly in individual issues, each of which contains several articles. The issues are organized into volumes, which usually consist of all the issues for a calendar year. Some journals are published in hard copy only, others in both hard copy and electronic form, and still others in electronic form only.

Most articles in professional journals are one of two basic types: empirical research reports and review articles. Empirical research reports describe one or more new empirical studies conducted by the authors. They introduce a research question, explain why it is interesting, review previous research, describe their method and results, and draw their conclusions. Review articles summarize previously published research on a topic and usually present new ways to organize or explain the results. When a review article is devoted primarily to presenting a new theory, it is often referred to as a theoretical article .

Figure 2.6 Small Sample of the Thousands of Professional Journals That Publish Research in Psychology and Related Fields

Most professional journals in psychology undergo a process of peer review . Researchers who want to publish their work in the journal submit a manuscript to the editor—who is generally an established researcher too—who in turn sends it to two or three experts on the topic. Each reviewer reads the manuscript, writes a critical review, and sends the review back to the editor along with his or her recommendations. The editor then decides whether to accept the article for publication, ask the authors to make changes and resubmit it for further consideration, or reject it outright. In any case, the editor forwards the reviewers’ written comments to the researchers so that they can revise their manuscript accordingly. Peer review is important because it ensures that the work meets basic standards of the field before it can enter the research literature.

Scholarly Books

Scholarly books are books written by researchers and practitioners mainly for use by other researchers and practitioners. A monograph is written by a single author or a small group of authors and usually gives a coherent presentation of a topic much like an extended review article. Edited volumes have an editor or a small group of editors who recruit many authors to write separate chapters on different aspects of the same topic. Although edited volumes can also give a coherent presentation of the topic, it is not unusual for each chapter to take a different perspective or even for the authors of different chapters to openly disagree with each other. In general, scholarly books undergo a peer review process similar to that used by professional journals.

Literature Search Strategies

Using psycinfo and other databases.

The primary method used to search the research literature involves using one or more electronic databases. These include Academic Search Premier, JSTOR, and ProQuest for all academic disciplines, ERIC for education, and PubMed for medicine and related fields. The most important for our purposes, however, is PsycINFO , which is produced by the APA. PsycINFO is so comprehensive—covering thousands of professional journals and scholarly books going back more than 100 years—that for most purposes its content is synonymous with the research literature in psychology. Like most such databases, PsycINFO is usually available through your college or university library.

PsycINFO consists of individual records for each article, book chapter, or book in the database. Each record includes basic publication information, an abstract or summary of the work, and a list of other works cited by that work. A computer interface allows entering one or more search terms and returns any records that contain those search terms. (These interfaces are provided by different vendors and therefore can look somewhat different depending on the library you use.) Each record also contains lists of keywords that describe the content of the work and also a list of index terms. The index terms are especially helpful because they are standardized. Research on differences between women and men, for example, is always indexed under “Human Sex Differences.” Research on touching is always indexed under the term “Physical Contact.” If you do not know the appropriate index terms, PsycINFO includes a thesaurus that can help you find them.

Given that there are nearly three million records in PsycINFO, you may have to try a variety of search terms in different combinations and at different levels of specificity before you find what you are looking for. Imagine, for example, that you are interested in the question of whether women and men differ in terms of their ability to recall experiences from when they were very young. If you were to enter “memory for early experiences” as your search term, PsycINFO would return only six records, most of which are not particularly relevant to your question. However, if you were to enter the search term “memory,” it would return 149,777 records—far too many to look through individually. This is where the thesaurus helps. Entering “memory” into the thesaurus provides several more specific index terms—one of which is “early memories.” While searching for “early memories” among the index terms returns 1,446 records—still too many too look through individually—combining it with “human sex differences” as a second search term returns 37 articles, many of which are highly relevant to the topic.

Depending on the vendor that provides the interface to PsycINFO, you may be able to save, print, or e-mail the relevant PsycINFO records. The records might even contain links to full-text copies of the works themselves. (PsycARTICLES is a database that provides full-text access to articles in all journals published by the APA.) If not, and you want a copy of the work, you will have to find out if your library carries the journal or has the book and the hard copy on the library shelves. Be sure to ask a librarian if you need help.

Using Other Search Techniques

In addition to entering search terms into PsycINFO and other databases, there are several other techniques you can use to search the research literature. First, if you have one good article or book chapter on your topic—a recent review article is best—you can look through the reference list of that article for other relevant articles, books, and book chapters. In fact, you should do this with any relevant article or book chapter you find. You can also start with a classic article or book chapter on your topic, find its record in PsycINFO (by entering the author’s name or article’s title as a search term), and link from there to a list of other works in PsycINFO that cite that classic article. This works because other researchers working on your topic are likely to be aware of the classic article and cite it in their own work. You can also do a general Internet search using search terms related to your topic or the name of a researcher who conducts research on your topic. This might lead you directly to works that are part of the research literature (e.g., articles in open-access journals or posted on researchers’ own websites). The search engine Google Scholar is especially useful for this purpose. A general Internet search might also lead you to websites that are not part of the research literature but might provide references to works that are. Finally, you can talk to people (e.g., your instructor or other faculty members in psychology) who know something about your topic and can suggest relevant articles and book chapters.

What to Search For

When you do a literature review, you need to be selective. Not every article, book chapter, and book that relates to your research idea or question will be worth obtaining, reading, and integrating into your review. Instead, you want to focus on sources that help you do four basic things: (a) refine your research question, (b) identify appropriate research methods, (c) place your research in the context of previous research, and (d) write an effective research report. Several basic principles can help you find the most useful sources.

First, it is best to focus on recent research, keeping in mind that what counts as recent depends on the topic. For newer topics that are actively being studied, “recent” might mean published in the past year or two. For older topics that are receiving less attention right now, “recent” might mean within the past 10 years. You will get a feel for what counts as recent for your topic when you start your literature search. A good general rule, however, is to start with sources published in the past five years. The main exception to this rule would be classic articles that turn up in the reference list of nearly every other source. If other researchers think that this work is important, even though it is old, then by all means you should include it in your review.

Second, you should look for review articles on your topic because they will provide a useful overview of it—often discussing important definitions, results, theories, trends, and controversies—giving you a good sense of where your own research fits into the literature. You should also look for empirical research reports addressing your question or similar questions, which can give you ideas about how to operationally define your variables and collect your data. As a general rule, it is good to use methods that others have already used successfully unless you have good reasons not to. Finally, you should look for sources that provide information that can help you argue for the interestingness of your research question. For a study on the effects of cell phone use on driving ability, for example, you might look for information about how widespread cell phone use is, how frequent and costly motor vehicle crashes are, and so on.

How many sources are enough for your literature review? This is a difficult question because it depends on how extensively your topic has been studied and also on your own goals. One study found that across a variety of professional journals in psychology, the average number of sources cited per article was about 50 (Adair & Vohra, 2003). This gives a rough idea of what professional researchers consider to be adequate. As a student, you might be assigned a much lower minimum number of references to use, but the principles for selecting the most useful ones remain the same.

Key Takeaways

• The research literature in psychology is all the published research in psychology, consisting primarily of articles in professional journals and scholarly books.
• Early in the research process, it is important to conduct a review of the research literature on your topic to refine your research question, identify appropriate research methods, place your question in the context of other research, and prepare to write an effective research report.
• There are several strategies for finding previous research on your topic. Among the best is using PsycINFO, a computer database that catalogs millions of articles, books, and book chapters in psychology and related fields.
• Practice: Use the techniques discussed in this section to find 10 journal articles and book chapters on one of the following research ideas: memory for smells, aggressive driving, the causes of narcissistic personality disorder, the functions of the intraparietal sulcus, or prejudice against the physically handicapped.

Adair, J. G., & Vohra, N. (2003). The explosion of knowledge, references, and citations: Psychology’s unique response to a crisis. American Psychologist, 58 , 15–23.

Research Methods in Psychology Copyright © 2016 by University of Minnesota is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License , except where otherwise noted.

• UConn Library
• Literature Review: The What, Why and How-to Guide
• Strategies to Find Sources

Literature Review: The What, Why and How-to Guide — Strategies to Find Sources

• Getting Started
• Introduction
• How to Pick a Topic
• Evaluating Sources & Lit. Reviews
• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings

The Research Process

Planning : Before searching for articles or books, brainstorm to develop keywords that better describe your research question.

Searching : While searching, take note of what other keywords are used to describe your topic, and use them to conduct additional searches

     ♠ Most articles include a keyword section

     ♠ Key concepts may change names throughout time so make sure to check for variations

Organizing : Start organizing your results by categories/key concepts or any organizing principle that make sense for you . This will help you later when you are ready to analyze your findings

Analyzing : While reading, start making notes of key concepts and commonalities and disagreement among the research articles you find.

♠ Create a spreadsheet  to record what articles you are finding useful and why.

♠ Create fields to write summaries of articles or quotes for future citing and paraphrasing .

Writing : Synthesize your findings. Use your own voice to explain to your readers what you learned about the literature on your topic. What are its weaknesses and strengths? What is missing or ignored?

Repeat : At any given time of the process, you can go back to a previous step as necessary.

Advanced Searching

All databases have Help pages that explain the best way to search their product. When doing literature reviews, you will want to take advantage of these features since they can facilitate not only finding the articles that you really need but also controlling the number of results and how relevant they are for your search. The most common features available in the advanced search option of databases and library online catalogs are:

• Boolean Searching (AND, OR, NOT): Allows you to connect search terms in a way that can either limit or expand your search results 
• Proximity Searching (N/# or W/#): Allows you to search for two or more words that occur within a specified number of words (or fewer) of each other in the database
• Limiters/Filters : These are options that let you control what type of document you want to search: article type, date, language, publication, etc.
• Question mark (?) or a pound sign (#) for wildcard: Used for retrieving alternate spellings of a word: colo?r will retrieve both the American spelling "color" as well as the British spelling "colour." 
• Asterisk (*) for truncation: Used for retrieving multiple forms of a word: comput* retrieves computer, computers, computing, etc.

Want to keep track of updates to your searches? Create an account in the database to receive an alert when a new article is published that meets your search parameters!

• EBSCOhost Advanced Search Tutorial Tips for searching a platform that hosts many library databases
• Library's General Search Tips Check the Search tips to better used our library catalog and articles search system
• ProQuest Database Search Tips Tips for searching another platform that hosts library databases

There is no magic number regarding how many sources you are going to need for your literature review; it all depends on the topic and what type of the literature review you are doing:

► Are you working on an emerging topic? You are not likely to find many sources, which is good because you are trying to prove that this is a topic that needs more research. But, it is not enough to say that you found few or no articles on your topic in your field. You need to look broadly to other disciplines (also known as triangulation ) to see if your research topic has been studied from other perspectives as a way to validate the uniqueness of your research question.

► Are you working on something that has been studied extensively? Then you are going to find many sources and you will want to limit how far back you want to look. Use limiters to eliminate research that may be dated and opt to search for resources published within the last 5-10 years.

• << Previous: How to Pick a Topic
• Next: Evaluating Sources & Lit. Reviews >>
• Last Updated: Sep 21, 2022 2:16 PM
• URL: https://guides.lib.uconn.edu/literaturereview

How to undertake a literature search: a step-by-step guide

Affiliation.

• 1 Literature Search Specialist, Library and Archive Service, Royal College of Nursing, London.
• PMID: 32279549
• DOI: 10.12968/bjon.2020.29.7.431

Undertaking a literature search can be a daunting prospect. Breaking the exercise down into smaller steps will make the process more manageable. This article suggests 10 steps that will help readers complete this task, from identifying key concepts to choosing databases for the search and saving the results and search strategy. It discusses each of the steps in a little more detail, with examples and suggestions on where to get help. This structured approach will help readers obtain a more focused set of results and, ultimately, save time and effort.

Keywords: Databases; Literature review; Literature search; Reference management software; Research questions; Search strategy.

• Databases, Bibliographic*
• Information Storage and Retrieval / methods*
• Nursing Research
• Review Literature as Topic*

• Privacy Policy

Home » Research Process – Steps, Examples and Tips

Research Process – Steps, Examples and Tips

Table of Contents

Research Process

Definition:

Research Process is a systematic and structured approach that involves the collection, analysis, and interpretation of data or information to answer a specific research question or solve a particular problem.

Research Process Steps

Research Process Steps are as follows:

Identify the Research Question or Problem

This is the first step in the research process. It involves identifying a problem or question that needs to be addressed. The research question should be specific, relevant, and focused on a particular area of interest.

Conduct a Literature Review

Once the research question has been identified, the next step is to conduct a literature review. This involves reviewing existing research and literature on the topic to identify any gaps in knowledge or areas where further research is needed. A literature review helps to provide a theoretical framework for the research and also ensures that the research is not duplicating previous work.

Formulate a Hypothesis or Research Objectives

Based on the research question and literature review, the researcher can formulate a hypothesis or research objectives. A hypothesis is a statement that can be tested to determine its validity, while research objectives are specific goals that the researcher aims to achieve through the research.

Design a Research Plan and Methodology

This step involves designing a research plan and methodology that will enable the researcher to collect and analyze data to test the hypothesis or achieve the research objectives. The research plan should include details on the sample size, data collection methods, and data analysis techniques that will be used.

Collect and Analyze Data

This step involves collecting and analyzing data according to the research plan and methodology. Data can be collected through various methods, including surveys, interviews, observations, or experiments. The data analysis process involves cleaning and organizing the data, applying statistical and analytical techniques to the data, and interpreting the results.

Interpret the Findings and Draw Conclusions

After analyzing the data, the researcher must interpret the findings and draw conclusions. This involves assessing the validity and reliability of the results and determining whether the hypothesis was supported or not. The researcher must also consider any limitations of the research and discuss the implications of the findings.

Communicate the Results

Finally, the researcher must communicate the results of the research through a research report, presentation, or publication. The research report should provide a detailed account of the research process, including the research question, literature review, research methodology, data analysis, findings, and conclusions. The report should also include recommendations for further research in the area.

Review and Revise

The research process is an iterative one, and it is important to review and revise the research plan and methodology as necessary. Researchers should assess the quality of their data and methods, reflect on their findings, and consider areas for improvement.

Ethical Considerations

Throughout the research process, ethical considerations must be taken into account. This includes ensuring that the research design protects the welfare of research participants, obtaining informed consent, maintaining confidentiality and privacy, and avoiding any potential harm to participants or their communities.

Dissemination and Application

The final step in the research process is to disseminate the findings and apply the research to real-world settings. Researchers can share their findings through academic publications, presentations at conferences, or media coverage. The research can be used to inform policy decisions, develop interventions, or improve practice in the relevant field.

Research Process Example

Following is a Research Process Example:

Research Question : What are the effects of a plant-based diet on athletic performance in high school athletes?

Step 1: Background Research Conduct a literature review to gain a better understanding of the existing research on the topic. Read academic articles and research studies related to plant-based diets, athletic performance, and high school athletes.

Step 2: Develop a Hypothesis Based on the literature review, develop a hypothesis that a plant-based diet positively affects athletic performance in high school athletes.

Step 3: Design the Study Design a study to test the hypothesis. Decide on the study population, sample size, and research methods. For this study, you could use a survey to collect data on dietary habits and athletic performance from a sample of high school athletes who follow a plant-based diet and a sample of high school athletes who do not follow a plant-based diet.

Step 4: Collect Data Distribute the survey to the selected sample and collect data on dietary habits and athletic performance.

Step 5: Analyze Data Use statistical analysis to compare the data from the two samples and determine if there is a significant difference in athletic performance between those who follow a plant-based diet and those who do not.

Step 6 : Interpret Results Interpret the results of the analysis in the context of the research question and hypothesis. Discuss any limitations or potential biases in the study design.

Step 7: Draw Conclusions Based on the results, draw conclusions about whether a plant-based diet has a significant effect on athletic performance in high school athletes. If the hypothesis is supported by the data, discuss potential implications and future research directions.

Step 8: Communicate Findings Communicate the findings of the study in a clear and concise manner. Use appropriate language, visuals, and formats to ensure that the findings are understood and valued.

Applications of Research Process

The research process has numerous applications across a wide range of fields and industries. Some examples of applications of the research process include:

• Scientific research: The research process is widely used in scientific research to investigate phenomena in the natural world and develop new theories or technologies. This includes fields such as biology, chemistry, physics, and environmental science.
• Social sciences : The research process is commonly used in social sciences to study human behavior, social structures, and institutions. This includes fields such as sociology, psychology, anthropology, and economics.
• Education: The research process is used in education to study learning processes, curriculum design, and teaching methodologies. This includes research on student achievement, teacher effectiveness, and educational policy.
• Healthcare: The research process is used in healthcare to investigate medical conditions, develop new treatments, and evaluate healthcare interventions. This includes fields such as medicine, nursing, and public health.
• Business and industry : The research process is used in business and industry to study consumer behavior, market trends, and develop new products or services. This includes market research, product development, and customer satisfaction research.
• Government and policy : The research process is used in government and policy to evaluate the effectiveness of policies and programs, and to inform policy decisions. This includes research on social welfare, crime prevention, and environmental policy.

Purpose of Research Process

The purpose of the research process is to systematically and scientifically investigate a problem or question in order to generate new knowledge or solve a problem. The research process enables researchers to:

• Identify gaps in existing knowledge: By conducting a thorough literature review, researchers can identify gaps in existing knowledge and develop research questions that address these gaps.
• Collect and analyze data : The research process provides a structured approach to collecting and analyzing data. Researchers can use a variety of research methods, including surveys, experiments, and interviews, to collect data that is valid and reliable.
• Test hypotheses : The research process allows researchers to test hypotheses and make evidence-based conclusions. Through the systematic analysis of data, researchers can draw conclusions about the relationships between variables and develop new theories or models.
• Solve problems: The research process can be used to solve practical problems and improve real-world outcomes. For example, researchers can develop interventions to address health or social problems, evaluate the effectiveness of policies or programs, and improve organizational processes.
• Generate new knowledge : The research process is a key way to generate new knowledge and advance understanding in a given field. By conducting rigorous and well-designed research, researchers can make significant contributions to their field and help to shape future research.

Tips for Research Process

Here are some tips for the research process:

• Start with a clear research question : A well-defined research question is the foundation of a successful research project. It should be specific, relevant, and achievable within the given time frame and resources.
• Conduct a thorough literature review: A comprehensive literature review will help you to identify gaps in existing knowledge, build on previous research, and avoid duplication. It will also provide a theoretical framework for your research.
• Choose appropriate research methods: Select research methods that are appropriate for your research question, objectives, and sample size. Ensure that your methods are valid, reliable, and ethical.
• Be organized and systematic: Keep detailed notes throughout the research process, including your research plan, methodology, data collection, and analysis. This will help you to stay organized and ensure that you don’t miss any important details.
• Analyze data rigorously: Use appropriate statistical and analytical techniques to analyze your data. Ensure that your analysis is valid, reliable, and transparent.
• I nterpret results carefully : Interpret your results in the context of your research question and objectives. Consider any limitations or potential biases in your research design, and be cautious in drawing conclusions.
• Communicate effectively: Communicate your research findings clearly and effectively to your target audience. Use appropriate language, visuals, and formats to ensure that your findings are understood and valued.
• Collaborate and seek feedback : Collaborate with other researchers, experts, or stakeholders in your field. Seek feedback on your research design, methods, and findings to ensure that they are relevant, meaningful, and impactful.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Context of the Study – Writing Guide and Examples

Figures in Research Paper – Examples and Guide

Significance of the Study – Examples and Writing...

Literature Review – Types Writing Guide and...

Theoretical Framework – Types, Examples and...

Table of Contents – Types, Formats, Examples

• Process: Scholarly Literature
• Definitions
• Reading a Scholarly Article

Ask a Librarian

Scholarly literature.

Scholars share their research and scholarship through scholarly publications channels. Typically fields of study have specialized journals or publishers. Academic articles may be review articles, original research papers, position papers, case studies among others.

Scholarly articles are typically written for a very specialized audience of fellow scholars and practitioners. Authors may assume that the reader has prior knowledge about the field. Don't be surprised if you find jargon or unfamiliar language! Try to get an overview of what the author's main points, claims, and questions may be. Many scholarly articles have similar structures or headings that may help you find your way. Look for sections like the abstract, introduction, section headings, and conclusion. They may offer cues that help you to infer the article's purpose and main argument.

• "Reading Games: Strategies for Reading Scholarly Sources." By Karen Rosenberg. This essay is a chapter in Writing Spaces: Readings on Writing, Volume 2, a peer-reviewed open textbook series for the writing classroom.

Anatomy of a Scholarly Article

Click on the gold plus boxes to learn more about the parts of a scholarly article.

• Text Version: Anatomy of a Scholarly Article
• << Previous: Definitions
• University of Colorado Boulder Libraries
• Research Guides
• Research Strategies
• Last Updated: May 31, 2024 10:38 AM
• URL: https://libguides.colorado.edu/strategies/scholarly
• © Regents of the University of Colorado

Application of Deep Learning for Code Smell Detection: Challenges and Opportunities

• Review Article
• Published: 03 June 2024
• Volume 5 , article number  614 , ( 2024 )

Cite this article

• Mouna Hadj-Kacem 1 &
• Nadia Bouassida 1  

Code smells are indicators of deeper problems in source code that affect the system maintainability and evolution. Detecting code smells is crucial as a software maintenance task. Recently, there has been a growing interest in utilizing deep learning techniques for code smell detection. However, there is limited research on the current state-of-the-art in this topic. To bridge this gap, this paper conducts a systematic literature review to investigate the application of deep learning in code smell detection. We have followed a well-defined methodology for conducting a systematic literature review in the field of software engineering. Through this process, we have identified a total of 30 primary studies. The reviewed studies have been thoroughly analysed according to different research aspects in the detection process, including the used code representations and learning algorithms. We have also explored the frequently identified code smells in terms of type and number, where the results have revealed a research gap in this context. Our analysis has also focused on determining how detection performance was evaluated and validated. Specifically, we have compiled a list of available datasets and compared between them based on various criteria. It was observed that the oracles within these datasets were developed using different approaches, but supported a limited number of identified code smells. Furthermore, through our investigation, we have highlighted several challenges that need to be addressed. Alongside the challenges, we have identified numerous promising opportunities that serve to improve future research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price includes VAT (Russian Federation)

Instant access to the full article PDF.

Rent this article via DeepDyve

Institutional subscriptions

https://github.com/dspinellis/tokenizer .

April A, Abran A. Software maintenance management: evaluation and continuous improvement. Hoboken: John Wiley & Sons; 2012.

Google Scholar  

Fowler M, Beck K, Brant J, Opdyke W, Roberts D. Refactoring: improving the design of existing code. Chennai: Pearson Education India; 1999.

Kaur A. A systematic literature review on empirical analysis of the relationship between code smells and software quality attributes. Arch Comput Methods Eng. 2020;27(4):1267–96.

Article   MathSciNet   Google Scholar  

Palomba F, Bavota G, Di Penta M, Fasano F, Oliveto R, De Lucia A. On the diffuseness and the impact on maintainability of code smells: a large scale empirical investigation. Empir Softw Eng. 2018;23(3):1188–221.

Article   Google Scholar  

Soh Z, Yamashita A, Khomh F, Guéhéneuc Y-G. Do code smells impact the effort of different maintenance programming activities? In: 23rd international conference on software analysis, evolution, and reengineering, pp. 393–402, vol. 1; 2016. IEEE.

Abbes M, Khomh F, Gueheneuc Y-G, Antoniol G. An empirical study of the impact of two antipatterns, blob and spaghetti code, on program comprehension. In: 15th European conference on software maintenance and reengineering, pp. 181–190; 2011. IEEE.

Politowski C, Khomh F, Romano S, Scanniello G, Petrillo F, Guéhéneuc Y-G, Maiga A. A large scale empirical study of the impact of spaghetti code and blob anti-patterns on program comprehension. Inf Softw Technol. 2020;122: 106278.

Khomh F, Di Penta M, Gueheneuc Y-G. An exploratory study of the impact of code smells on software change-proneness. In: 16th working conference on reverse engineering; 2009. pp. 75–84. IEEE.

Cunningham W. The WyCash portfolio management system. ACM SIGPLAN OOPS Messenger. 1992;4(2):29–30.

Arcelli FF, Mäntylä MV, Zanoni M, Marino A. Comparing and experimenting machine learning techniques for code smell detection. Empir Softw Eng. 2016;21(3):1143–91.

Palomba F, Bavota G, Di Penta M, Oliveto R, De Lucia A, Poshyvanyk D. Detecting bad smells in source code using change history information. In: 28th IEEE/ACM international conference on automated software engineering; 2013. pp. 268–278. IEEE.

Moha N, Guéhéneuc Y-G, Duchien L, Le Meur A-F. Decor: a method for the specification and detection of code and design smells. IEEE Trans Softw Eng. 2010;36(1):20–36. https://doi.org/10.1109/TSE.2009.50 .

LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–44.

Wang S, Liu T, Tan L. Automatically learning semantic features for defect prediction. In: International conference on software engineering; 2016. pp. 297–308. IEEE.

Hellendoorn VJ, Devanbu P. Are deep neural networks the best choice for modeling source code? In: 11th joint meeting on foundations of software engineering; 2017. pp. 763–773. https://doi.org/10.1145/3106237.3106290 .

Tufano M, Palomba F, Bavota G, Oliveto R, Di Penta M, De Lucia A, Poshyvanyk D. When and why your code starts to smell bad (and whether the smells go away). IEEE Trans Softw Eng. 2017;43(11):1063–88.

Fontana FA, Zanoni M. Code smell severity classification using machine learning techniques. Knowl Based Syst. 2017;128:43–58.

Mens T, Tourwe T. A survey of software refactoring. IEEE Trans Softw Eng. 2004;30(2):126–39. https://doi.org/10.1109/TSE.2004.1265817 .

Rasool G, Arshad Z. A review of code smell mining techniques. J Softw Evol Process. 2015;27(11):867–95.

Fernandes E, Oliveira J, Vale G, Paiva T, Figueiredo E. A review-based comparative study of bad smell detection tools. In: 20th international conference on evaluation and assessment in software engineering; 2016. pp. 1–12.

Sabir F, Palma F, Rasool G, Guéhéneuc Y-G, Moha N. A systematic literature review on the detection of smells and their evolution in object-oriented and service-oriented systems. Softw Pract Exp. 2019;49(1):3–39.

Lewowski T, Madeyski L. Code smells detection using artificial intelligence techniques: a business-driven systematic review. Dev Inf Knowl Manag Bus Appl. 2022;3:285–319.

Bafandeh Mayvan B, Rasoolzadegan A, Javan Jafari A. Bad smell detection using quality metrics and refactoring opportunities. J Softw Evol Process. 2020;32(8):2255.

Reis J, Abreu F, Figueiredo Carneiro G, Anslow C. Code smells detection and visualization: a systematic literature review. Arch Comput Methods Eng. 2022;29(1):47–94.

Kaur A, Dhiman G. A review on search-based tools and techniques to identify bad code smells in object-oriented systems. In: Harmony search and nature inspired optimization algorithms. Springer. 2019. pp. 909-–21.

Azeem MI, Palomba F, Shi L, Wang Q. Machine learning techniques for code smell detection: a systematic literature review and meta-analysis. Inf Softw Technol. 2019;108:115–38.

Caram FL, Rodrigues BRDO, Campanelli AS, Parreiras FS. Machine learning techniques for code smells detection: a systematic mapping study. Int J Softw Eng Knowl Eng. 2019;29(02):285–316.

Al-Shaaby A, Aljamaan H, Alshayeb M. Bad smell detection using machine learning techniques: a systematic literature review. Arab J Sci Eng. 2020;45:2341–69.

Lei M, Li H, Li J, Aundhkar N, Kim D-K. Deep learning application on code clone detection: a review of current knowledge. J Syst Softw. 2022;184: 111141.

Rattan D, Bhatia R, Singh M. Software clone detection: a systematic review. Inf Softw Technol. 2013;55(7):1165–99.

Paulo Sobrinho EV, De Lucia A, Almeida Maia M. A systematic literature review on bad smells-5 w’s: which, when, what, who, where. IEEE Trans Softw Eng. 2018;47(1):17–66.

Petersen K, Feldt R, Mujtaba S, Mattsson M. Systematic mapping studies in software engineering. In: 12th international conference on evaluation and assessment in software engineering; 2008. pp. 1–10.

Kitchenham B, Charters S, et al. Guidelines for performing systematic literature reviews in software engineering. UK: EBSE Technical Report; 2007.

ACM Digital Library. https://dl.acm.org . Accessed 31 Jan 2023.

IEEEXplore Digital Library. https://ieeexplore.ieee.org . Accessed 31 Jan 2023.

ScienceDirect. https://www.sciencedirect.com . Accessed 31 Jan 2023.

Springer Link. https://link.springer.com . Accessed 31 Jan 2023.

Scopus. https://www.scopus.com . Accessed 31 Jan 2023.

Mongeon P, Paul-Hus A. The journal coverage of web of science and scopus: a comparative analysis. Scientometrics. 2016;106:213–28.

Guo X, Shi C, Jiang H. Deep semantic-based feature envy identification. In: 11th Asia-Pacific symposium on internetware; 2019. pp. 1–6.

Yu J, Mao C, Ye X. A novel tree-based neural network for android code smells detection. In: 21st international conference on software quality, reliability and security; 2021. pp. 738–748. IEEE. https://doi.org/10.1109/QRS54544.2021.00083 .

Tarwani S, Chug A. Application of deep learning models for code smell prediction. In: 10th international conference on reliability, Infocom technologies and optimization (trends and future directions); 2022. pp. 1–5. IEEE. https://doi.org/10.1109/ICRITO56286.2022.9965048 .

Barbez A, Khomh F, Guéhéneuc Y-G. Deep learning anti-patterns from code metrics history. In: international conference on software maintenance and evolution; 2019. pp. 114–124. IEEE. https://doi.org/10.1109/ICSME.2019.00021 .

Liu H, Jin J, Xu Z, Zou Y, Bu Y, Zhang L. Deep learning based code smell detection. IEEE Trans Softw Eng. 2021;47(9):1811–37. https://doi.org/10.1109/TSE.2019.2936376 .

Bhave A, Sinha R. Deep multimodal architecture for detection of long parameter list and switch statements using distilbert. In: 22nd international working conference on source code analysis and manipulation; 2022. pp. 116–120. IEEE. https://doi.org/10.1109/SCAM55253.2022.00018 .

Hadj-Kacem M, Bouassida N. Deep representation learning for code smells detection using variational auto-encoder. In: International joint conference on neural networks; 2019. pp. 1–8. IEEE. https://doi.org/10.1109/IJCNN.2019.8851854 .

Yu D, Xu Y, Weng L, Chen J, Chen X, Yang Q. Detecting and refactoring feature envy based on graph neural network. In: 33rd international symposium on software reliability engineering; 2022. pp. 458–469. IEEE. https://doi.org/10.1109/ISSRE55969.2022.00051 .

Das AK, Yadav S, Dhal S. Detecting code smells using deep learning. In: TENCON 2019-2019 IEEE region 10 conference (TENCON); 2019. pp. 2081–2086. IEEE. https://doi.org/10.1109/TENCON.2019.8929628 .

Ren S, Shi C, Zhao S. Exploiting multi-aspect interactions for god class detection with dataset fine-tuning. In: 45th annual computers, software, and applications conference; 2021. pp. 864–873. IEEE. https://doi.org/10.1109/COMPSAC51774.2021.00119 .

Wang H, Liu J, Kang J, Yin W, Sun H, Wang H. Feature envy detection based on bi-lstm with self-attention mechanism. In: Intl conf on parallel & distributed processing with applications, big data & cloud computing, sustainable computing & communications, social computing & networking (ISPA/BDCloud/SocialCom/SustainCom); 2020. pp. 448–457. IEEE.

Zhang M, Jia J. Feature envy detection with deep learning and snapshot ensemble. In: 9th international conference on dependable systems and their applications; 2022. pp. 215–223. IEEE. https://doi.org/10.1109/DSA56465.2022.00037 .

Yin X, Shi C, Zhao S. Local and global feature based explainable feature envy detection. In: 45th annual computers, software, and applications conference; 2021. pp. 942–951. IEEE. https://doi.org/10.1109/COMPSAC51774.2021.00127 .

Kovacevic A, Slivka J, Vidakovic D, Grujic K-G, Luburic N, Prokic S, Sladic G. Automatic detection of long method and god class code smells through neural source code embeddings. Expert Syst Appl. 2022;204: 117607. https://doi.org/10.1016/j.eswa.2022.117607 .

Zhang Y, Ge C, Hong S, Tian R, Dong C, Liu J. Delesmell: code smell detection based on deep learning and latent semantic analysis. Knowl Based Syst. 2022;255: 109737. https://doi.org/10.1016/j.knosys.2022.109737 .

Ardimento P, Aversano L, Bernardi ML, Cimitile M, Iammarino M. Temporal convolutional networks for just-in-time design smells prediction using fine-grained software metrics. Neurocomputing. 2021;463:454–71. https://doi.org/10.1016/j.neucom.2021.08.010 .

Imam AT, Al-Srour BR, Alhroob A. The automation of the detection of large class bad smell by using genetic algorithm and deep learning. J King Saud Univ Comput Inf Sci. 2022; 34(6, Part A):2621–2636. https://doi.org/10.1016/j.jksuci.2022.03.028 .

Gupta H, Kulkarni TG, Kumar L, Neti LBM, Krishna A. An empirical study on predictability of software code smell using deep learning models. In: International conference on advanced information networking and applications; 2021. pp. 120–132. Springer.

Hadj-Kacem M, Bouassida N. Improving the identification of code smells by combining structural and semantic information. In: 26th international conference on neural information processing; 2019. pp. 296–304. Springer.

Mhawish MY, Gupta M. Predicting code smells and analysis of predictions: using machine learning techniques and software metrics. J Comput Sci Technol. 2020;35:1428–45.

Hadj-Kacem M, Bouassida N. A hybrid approach to detect code smells using deep learning. In: 13th international conference on evaluation of novel approaches to software engineering; 2018. pp. 137–146. SciTePress.

Sharma T, Efstathiou V, Louridas P, Spinellis D. Code smell detection by deep direct-learning and transfer-learning. J Syst Softw. 2021;176: 110936.

Dewangan S, Rao RS, Mishra A, Gupta M. Code smell detection using ensemble machine learning algorithms. Appl Sci. 2022;12(20):10321.

Zhao S, Shi C, Ren S, Mohsin H. Correlation feature mining model based on dual attention for feature envy detection. In: 34th International conference on software engineering and knowledge engineering. KSI Research Inc.; 2022. pp. 634–9. https://doi.org/10.18293/SEKE2022-009 .

Khleel NAA, Nehéz K. Deep convolutional neural network model for bad code smells detection based on oversampling method. Indones J Electr Eng Comput Sci. 2022;26(3):1725–35.

Hamdy A, Tazy M. Deep hybrid features for code smells detection. J Theor Appl Inf Technol. 2020;98(14):2684–96.

Jeevanantham M, Jones J. Extension of deep learning based feature envy detection for misplaced fields and methods. Int J Intell Eng Syst. 2022;15(1):563–74.

Zhang Y, Dong C. Mars: Detecting brain class/method code smell based on metric–attention mechanism and residual network. J Softw Evol Process. 2021;36:e2403.

Xu W, Zhang X. Multi-granularity code smell detection using deep learning method based on abstract syntax tree. In: 33rd international conference on software engineering and knowledge engineering; 2021, vol. 7, pp. 503–9.

Li Y, Zhang X. Multi-label code smell detection with hybrid model based on deep learning. In: 34th International conference on software engineering and knowledge engineering. KSI Research Inc.; 2022. pp. 42–7. https://doi.org/10.18293/SEKE2022-077 .

Allamanis M, Barr ET, Devanbu P, Sutton C. A survey of machine learning for big code and naturalness. ACM Comput Surv. 2018;51(4):1–37.

Kitchenham B. What’s up with software metrics?—A preliminary mapping study. J Syst Softw. 2010;83(1):37–51.

Lanza M, Marinescu R. Object-oriented metrics in practice: using software metrics to characterize, evaluate, and improve the design of object-oriented systems. Berlin: Springer; 2007.

Marinescu C, Marinescu R, Florin Mihancea P, Ratiu D, Wettel R. iPlasma: an integrated platform for quality assessment of object-oriented design. In: International conference on software maintenance; 2005. pp. 77–80.

Ferme V. Jcodeodor: A software quality advisor through design flaws detection. Master’s thesis. Milano: University of Milano-Bicocca. 2013.

Hindle A, Barr ET, Gabel M, Su Z, Devanbu P. On the naturalness of software. Commun ACM. 2016;59(5):122–31.

Goller C, Kuchler A. Learning task-dependent distributed representations by backpropagation through structure. In: International conference on neural networks, vol. 1; 1996. pp. 347–352. IEEE.

Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997;9(8):1735–80.

Kingma DP, Welling M. Auto-encoding variational bayes. 2013. arXiv:1312.6114 . preprint.

Peng H, Mou L, Li G, Liu Y, Zhang L, Jin Z. Building program vector representations for deep learning. In: 8th international conference on knowledge science, engineering and management; 2015. pp. 547–553. Springer.

Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. 2014. arXiv preprint arXiv:1409.0473 .

Trindade RPF, Silva Bigonha MA, Ferreira K A M. Oracles of bad smells: a systematic literature review. In: Brazilian symposium on software engineering; 2020. pp. 62–71. ACM.

Tempero E, Anslow C, Dietrich ., Han T, Li J, Lumpe M, Melton H, Noble J. The qualitas corpus: A curated collection of java code for empirical studies. In: Asia Pacific software engineering conference; 2010. pp. 336–345. https://doi.org/10.1109/APSEC.2010.46 .

PMD (PMD). http://pmd.sourceforge.net/ .

Nongpong K. Integrating code smells detection with refactoring tool support. PhD thesis, The University of Wisconsin-Milwaukee. 2012.

Wieman R. Anti-pattern scanner: an approach to detect anti-patterns and design violations. LAP Lambert Academic Publishing; 2011.

Marinescu R. Measurement and quality in object-oriented design. In: 21st IEEE international conference on software maintenance; 2005. pp. 701–704.

Di Nucci D, Palomba F, Tamburri DA, Serebrenik A, De Lucia A. Detecting code smells using machine learning techniques: are we there yet? In: 25th international conference on software analysis, evolution and reengineering; 2018. pp. 612–621. IEEE.

Liu H, Xu Z, Zou Y. Deep learning based feature envy detection. In: 33rd ACM/IEEE international conference on automated software engineering; 2018. pp. 385–396. ACM.

Palomba F, Di Nucci D, Tufano M, Bavota G, Oliveto R, Poshyvanyk D, De Lucia A. Landfill: An open dataset of code smells with public evaluation. In: 12th working conference on mining software repositories; 2015. pp. 482–485. IEEE.

Sharma T, Kessentini M. Qscored: A large dataset of code smells and quality metrics. In: 18th international conference on mining software repositories; 2021. pp. 590–594. IEEE.

Madeyski L, Lewowski T. Mlcq: Industry-relevant code smell data set. In: 24th international conference on evaluation and assessment in software engineering; 2020. pp. 342–347. ACM.

Sharma T, Efstathiou V, Louridas P, Spinellis D. On the feasibility of transfer-learning code smells using deep learning. 2019. arXiv preprint arXiv:1904.03031 .

Fokaefs M, Tsantalis N, Chatzigeorgiou A. Jdeodorant: Identification and removal of feature envy bad smells. In: International conference on software maintenance. IEEE; 2007. pp. 519–520.

Terra R, Valente MT, Miranda S, Sales V. JMove: a novel heuristic and tool to detect move method refactoring opportunities. J Syst Softw. 2018;138:19–36.

Palomba F, Panichella A, Lucia AD, Oliveto R, Zaidman A. A textual-based technique for smell detection. In: 24th international conference on program comprehension. IEEE; 2016. pp. 1–10.

Guggulothu T, Moiz SA. Code smell detection using multi-label classification approach. Softw Qual J. 2020;28(3):1063–86.

Alazba A, Aljamaan H. Code smell detection using feature selection and stacking ensemble: An empirical investigation. Inf Softw Technol. 2021;138: 106648.

Dewangan S, Rao RS, Mishra A, Gupta M. A novel approach for code smell detection: an empirical study. IEEE Access. 2021;9:162869–83.

Ampatzoglou A, Bibi S, Avgeriou P, Verbeek M, Chatzigeorgiou A. Identifying, categorizing and mitigating threats to validity in software engineering secondary studies. Inf Softw Technol. 2019;106:201–30.

Download references

This study was not funded.

Author information

Authors and affiliations.

Mir@cl Laboratory, Sfax University, Sfax, Tunisia

Mouna Hadj-Kacem & Nadia Bouassida

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Mouna Hadj-Kacem .

Ethics declarations

Conflict of interest.

The authors declare that they have no Conflict of interest.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Hadj-Kacem, M., Bouassida, N. Application of Deep Learning for Code Smell Detection: Challenges and Opportunities. SN COMPUT. SCI. 5 , 614 (2024). https://doi.org/10.1007/s42979-024-02956-5

Download citation

Received : 10 September 2023

Accepted : 07 May 2024

Published : 03 June 2024

DOI : https://doi.org/10.1007/s42979-024-02956-5

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Systematic literature review
• Code smell detection
• Deep learning techniques
• Code representation
• Find a journal
• Publish with us
• Track your research

• Open access
• Published: 03 June 2024

Continue nursing education: an action research study on the implementation of a nursing training program using the Holton Learning Transfer System Inventory

• MingYan Shen 1 , 2 &
• ZhiXian Feng 1 , 2  

BMC Medical Education volume  24 , Article number:  610 ( 2024 ) Cite this article

Metrics details

To address the gap in effective nursing training for quality management, this study aims to implement and assess a nursing training program based on the Holton Learning Transfer System Inventory, utilizing action research to enhance the practicality and effectiveness of training outcomes.

The study involved the formation of a dedicated training team, with program development informed by an extensive situation analysis and literature review. Key focus areas included motivation to transfer, learning environment, and transfer design. The program was implemented in a structured four-step process: plan, action, observation, reflection.

Over a 11-month period, 22 nurses completed 14 h of theoretical training and 18 h of practical training with a 100% attendance rate and 97.75% satisfaction rate. The nursing team successfully led and completed 22 quality improvement projects, attaining a practical level of application. Quality management implementation difficulties, literature review, current situation analysis, cause analysis, formulation of plans, implementation plans, and report writing showed significant improvement and statistical significance after training.

The study confirms the efficacy of action research guided by Holton’s model in significantly enhancing the capabilities of nursing staff in executing quality improvement projects, thereby improving the overall quality of nursing training. Future research should focus on refining the training program through long-term observation, developing a multidimensional evaluation index system, exploring training experiences qualitatively, and investigating the personality characteristics of nurses to enhance training transfer effects.

Peer Review reports

Introduction

The “Medical Quality Management Measures“ [ 1 ] and “Accreditation Standards for Tertiary Hospitals (2020 Edition)” [ 2 ] both emphasize the importance of using quality management tools in medical institutions to carry out effective quality management [ 3 ]. However, there is a notable gap in translating theoretical training into effective, practical application in clinical settings [ 4 ]. This gap is further highlighted in the context of healthcare quality management, as evidenced in studies [ 5 ] which demonstrate the universality of these challenges across healthcare systems worldwide.

Addressing this issue, contemporary literature calls for innovative and effective training methods that transition from passive knowledge acquisition to active skill application [ 6 ]. The Holton Learning Transfer System Inventory [ 7 ] provides a framework focusing on key factors such as motivation, learning environment, and transfer design [ 7 , 8 , 9 ]. This study aims to implement a nursing training program based on the Holton model, using an action research methodology to bridge the theoretical-practical gap in nursing education.

Quality management training for clinical nurses has predominantly been characterized by short-term theoretical lectures, a format that often fails to foster deep engagement and lasting awareness among nursing personnel [ 10 ]. The Quality Indicator Project in Taiwan’s nursing sector, operational for over a decade, demonstrates the effective use of collective intelligence and scientific methodologies to address these challenges [ 11 ]. The proposed study responds to the need for training programs that not only impart knowledge but also ensure the practical application of skills in real-world nursing settings, thereby contributing to transformative changes within the healthcare system [ 12 ].

In April 2021, the Nursing Education Department of our hospital launched a quality improvement project training program for nurses. The initiation of this study is underpinned by the evident disconnect between theoretical training and the practical challenges nurses face in implementing quality management initiatives, a gap also identified in the work [ 13 ]. By exploring the efficacy of the Holton Learning Transfer System Inventory, this study seeks to enhance the practical application of training and significantly contribute to the field of nursing education and quality management in healthcare.

Developing a nursing training program with the Holton Learning Transfer System Inventory

Establishing a research team and assigning roles.

There are 10 members in the group who serve as both researchers and participants, aiming to investigate training process issues and solutions. The roles within the group are as follows: the deputy dean in charge of nursing is responsible for program review and organizational support, integrating learning transfer principles in different settings [ 14 ]; the deputy director of the Nursing Education Department handles the design and implementation of the training program, utilizing double-loop learning for training transfer [ 15 ]; the deputy director of the Nursing Department oversees quality control and project evaluation, ensuring integration of evidence-based practices and technology [ 16 ] and the deputy director of the Quality Management Office provides methodological guidance. The remaining members consist of 4 faculty members possessing significant university teaching experience and practical expertise in quality control projects, and 2 additional members who are jointly responsible for educational affairs, data collection, and analysis. Additionally, to ensure comprehensive pedagogical guidance in this training, professors specializing in nursing pedagogy have been specifically invited to provide expertise on educational methodology.

Current situation survey

Based on the Holton Learning Transfer System Inventory (refer to Fig.  1 ), the appropriate levels of Motivation to Improve Work Through Learning (MTIWL), learning environment, and transfer design are crucial in facilitating changes in individual performance, thereby influencing organizational outcomes [ 17 , 18 ]. Motivation to Improve Work Through Learning (MTIWL) is closely linked to expectation theory, fairness theory, and goal-setting theory, significantly impacting the positive transfer of training [ 19 ]. Learning environment encompasses environmental factors that either hinder or promote the application of learned knowledge in actual work settings [ 20 ]. Transfer design, as a pivotal component, includes training program design and organizational planning.

To conduct the survey, the research team retrieved 26 quality improvement reports from the nursing quality information management system, which were generated by nursing units in 2020. A checklist was formulated, and a retrospective evaluation was conducted across eight aspects, namely, team participation, topic selection feasibility, method accuracy, indicator scientificity, program implementation rate, effect maintenance, and promotion and application. Methods employed in the evaluation process included report analysis, on-site tracking, personnel interviews, and data review within the quality information management system [ 21 ]. From the perspective of motivation [ 22 ], learning environment [ 23 ], and transfer design, a total of 14 influencing factors were identified. These factors serve as a reference for designing the training plan and encompass the following aspects: lack of awareness regarding importance, low willingness to participate in training, unclear understanding of individual career development, absence of incentive mechanisms, absence of a scientific training organization model, lack of a training quality management model, inadequate literature retrieval skills and support, insufficient availability of practical training materials and resources, incomplete mastery of post-training methods, lack of cultural construction plans, suboptimal communication methods and venues, weak internal organizational atmosphere, inadequate leadership support, and absence of platforms and mechanisms for promoting and applying learned knowledge.

Learning Transfer System Inventory

Development of the training program using the 4W1H approach

Drawing upon Holton’s Learning Transfer System Inventory and the hospital training transfer model diagram, a comprehensive training outline was formulated for the training program [ 24 , 25 ]. The following components were considered:

(1) Training Participants (Who): The training is open for voluntary registration to individuals with an undergraduate degree or above, specifically targeting head nurses, responsible team leaders, and core members of the hospital-level nursing quality control team. Former members who have participated in quality improvement projects such as Plan-Do-Check-Act Circle (PDCA) or Quality control circle (QCC) are also eligible.

(2) Training Objectives (Why): At the individual level, the objectives include enhancing the understanding of quality management concepts, improving the cognitive level and application abilities of project improvement methods, and acquiring the necessary skills for nursing quality improvement project. At the team level, the aim is to enhance effective communication among team members and elevate the overall quality of communication. Moreover, the training seeks to facilitate collaborative efforts in improving the existing nursing quality management system and processes. At the operational level, participants are expected to gain the competence to design, implement, and manage nursing quality improvement project initiatives. Following the training, participants will lead and successfully complete a nursing quality improvement project, which will undergo a rigorous audit.

(3) Training Duration (When): The training program spans a duration of 11 months.

(4) Training Content (What): The program consists of 14 h of theoretical courses and 18 h of practical training sessions, as detailed in Table  1 .

(5) Quality Management Approach (How): To ensure quality throughout the training process, two team members are assigned to monitor the entire training journey. This encompasses evaluating whether quality awareness education, quality management knowledge, and professional skills training are adequately covered. Additionally, attention is given to participants’ learning motivation, the emphasis placed on active participation in training methods, support from hospital management and relevant departments, as well as participants’ satisfaction and assessment results. Please refer to Fig.  2 for a visual representation.

In-house training model from Holton Learning Transfer

Implementation of the nursing project training program using the action research method

The first cycle (april 2021).

In the initial cycle, a total of 22 nurses were included as training participants after a self-registration process and qualification review. The criteria used to select these participants, elaborated in Section Development of the training program using the 4W1H approach, ‘Development of the Training Program,’ were meticulously crafted to capture a broad spectrum of experience, expertise, and functional roles within our hospital’s nursing staff. The primary focus was to investigate their learning motivation. The cycle comprised the following key activities:

(1) Training Objectives: The focus was on understanding the learning motivation of the participating nurses.

(2) Theoretical Training Sessions: A total of 7 theoretical training sessions, spanning 14 class hours, were completed. The contents covered various aspects, including an overview of nursing quality improvement projects, methods for selecting project topics, common tools used in nursing quality improvement projects, effective leadership strategies to promote project practices, literature retrieval and evaluation methods, formulation and promotion of project plans, and writing project reports. Detailed course information, including the title, content, and class hours, is listed in Table  1 . At the end of each training session, a course satisfaction survey was conducted.

(3) Assessment and Reporting: Following the completion of the 7 training sessions, a theoretical assessment on quality management knowledge was conducted. Additionally, nurses were organized to present their plans for special projects to be carried out during the training. Several issues were identified during this cycle:

Incomplete Literature Review Skills: Compared to other quality control tools, nursing quality improvement project places more emphasis on the scientific construction of project plans. The theoretical evaluation and interviews with nurses highlighted the incomplete and challenging nature of their literature review skills.

Insufficient Leadership: Among the participants, 6 individuals were not head nurses, which resulted in a lack of adequate leadership for their respective projects.

Learning environment and Support: The learning environment, as well as the support from hospital management and relevant departments, needed to be strengthened.

Second cycle (may-october 2021)

In response to the issues identified during the first cycle, our approach in the second cycle was both corrective and adaptive, focusing on immediate issues while also setting the stage for addressing any emerging challenges. The team members actively implemented improvements during the second cycle. The key actions taken were as follows:

(1) Establishing an Enabling Organizational Environment: The quality management department took the lead, and multiple departments collaborated in conducting the “Hospital Safety and Quality Red May” activity. This initiative aimed to enhance the overall quality improvement atmosphere within the hospital. Themed articles were also shared through the hospital’s WeChat public account.

(2) Salon-style Training Format: The training sessions were conducted in the form of salons, held in a meeting room specifically prepared for this purpose. The room was arranged with a round table, warm yellow lighting, green plants, and a coffee bar, creating a conducive environment for free, democratic, and equal communication among the participants. The salon topics included revising project topic selection, conducting current situation investigations, facilitating communication and guidance for literature reviews, formulating improvement plans, implementing those plans, and writing project reports. After the projects were presented, quality management experts provided comments and analysis, promoting the transformation of training outcomes from mere memory and understanding to higher-level abilities such as application, analysis, and creativity.

(3) Continuous Support Services: Various support services were provided to ensure ongoing assistance. This included assigning nursing postgraduates to aid in literature retrieval and evaluation. Project team members also provided on-site guidance and support, actively engaging in the project improvement process to facilitate training transfer.

(4) Emphasis on Spiritual Encouragement: The Vice President of Nursing Department actively participated in the salons and provided feedback on each occasion. Moreover, the President of the hospital consistently commended the training efforts during the weekly hospital meetings.

Issues identified in this cycle

(1) Inconsistent Ability to Write Project Documents: The proficiency in writing project documents for project improvement varied among participants, and there was a lack of standardized evaluation criteria. This issue had the potential to impact the quality of project dissemination.

(2) Lack of Clarity Regarding the Platform and Mechanism for Training Result Transfer: The platform and mechanisms for transferring training results were not clearly defined, posing a challenge in effectively sharing and disseminating the outcomes of the training.

The third cycle (November 2021-march 2022)

During the third cycle, the following initiatives were undertaken.

(1) Utilizing the “Reporting Standards for Quality Improvement Research (SQUIRE)”, as issued by the US Health Care Promotion Research, to provide guidance for students in writing nursing project improvement reports.

(2) Organizing a hospital-level nursing quality improvement project report meeting to acknowledge and commend outstanding projects.

(3) Compiling the “Compilation of Nursing Quality Improvement Projects” for dissemination and exchange among nurses both within and outside the hospital.

(4) Addressing the issue of inadequate management of indicator monitoring data, a hospital-level quality index management platform was developed. The main evaluation data from the 22 projects were entered into this platform, allowing for continuous monitoring and timely intervention.

Effect evaluation

To assess the efficacy of the training, a diverse set of evaluation metrics, encompassing both outcome and process measures [ 26 ]. These measures can be structured around the four-level training evaluation framework proposed by Donald Kirkpatrick [ 27 ].

Process evaluation

Evaluation method.

To assess the commitment and support within the organization, the process evaluation involved recording the proportion of nurses’ classroom participation time and the presence of leaders during each training session. Additionally, a satisfaction survey was conducted after the training to assess various aspects such as venue layout, time arrangement, training methods, lecturer professionalism, content practicality, and interaction. On-site recycling statistics were also collected for project evaluation purposes.

Evaluation results the results of the process evaluation are as follows

Nurse training participation rate: 100%.

Training satisfaction rate (average): 97.75%.

Proportion of nurses’ participation time in theoretical training sessions (average): 36.88%.

Proportion of nurses’ participation time in salon training sessions (average): 74.23%.

Attendance rate of school-level leaders: 100%.

Results evaluation

Assessment of theoretical knowledge of quality management.

To evaluate the effectiveness in enhancing the trainees’ theoretical knowledge of quality management, the research team conducted assessments before the training, after the first round of implementation, and after the third round of implementation. Assessments to evaluate the effectiveness of the training program were conducted immediately following the first round of implementation, and after the third round of implementation. This dual-timing approach was designed to evaluate both the immediate impact of the training and its sustained effects over time, addressing potential influences of memory decay on the study results. The assessment consisted of a 60-minute examination with different question types, including 30 multiple-choice questions (2 points each), 2 short-answer questions (10 points each), and 1 comprehensive analysis question (20 points). The maximum score achievable was 100 points.

The assessment results are as follows:

Before training (average): 75.05 points.

After the first round of implementation (average): 82.18 points.

After the third round of implementation (average): 90.82 points.

Assessment of difficulty in quality management project implementation

To assess the difficulty of implementing quality management projects, the trainees completed the “Quality Management Project Implementation Difficulty Assessment Form” before and after the training. They self-evaluated 10 aspects using a 5-point scale, with 5 indicating the most difficult and 1 indicating no difficulty. The evaluation results before and after implementation are presented in Table  2 .

Statistically significant differences were found in the following items: literature review, current situation analysis, cause analysis, plan formulation, implementation plan, and report writing. This indicates that the training significantly enhanced the nurses’ confidence and ability to tackle practical challenges.

Evaluation of transfer effect

To assess how effectively the training translated into practical applications. The implementation of the 22 quality improvement projects was evaluated using the application hierarchy analysis table. The specific results are presented in Table  3 .

In addition, the “Nursing Project Guidance Manual” and “Compilation of Nursing Project Improvement Projects” were compiled and distributed to the hospital’s management staff, nurses, and four collaborating hospitals, receiving positive feedback. The lecture titled “Improving Nurses’ Project Improvement Ability Based on the Training Transfer Theory Model” shared experiences with colleagues both within and outside the province in national and provincial teaching sessions in 2022. Furthermore, four papers were published on the subject.

The effectiveness of the training program based on the Holton Learning transfer System Inventory

The level of refined management in hospitals is closely tied to the quality management awareness and skills of frontline medical staff. Quality management training plays a crucial role in improving patient safety management and fostering a culture of quality and safety. Continuous quality improvement is an integral part of nursing management, ensuring that patients receive high-quality and safe nursing care. Compared to the focus of existing literature on the individual performance improvements following nursing training programs [ 28 , 29 , 30 ], our study expands the evaluation framework to include organizational performance metrics. Our research underscores a significantly higher level of organizational engagement as evidenced by the 100% attendance rate of school-level leaders. The publication of four papers related to this study highlights not only individual performance achievements but also significantly broadens the hospital organization’s impact on quality management, leading to meaningful organizational outcomes.

Moreover, our initiative to incorporate indicators of quality projects into a hospital-level evaluation index system post-training signifies a pivotal move towards integrating quality improvement practices into the very fabric of organizational operations. In training programs, it is essential not only to achieve near-transfer, but also to ensure that nurses continuously apply the acquired management skills to their clinical work, thereby enhancing quality, developing their professional value, and improving organizational performance. The Holton learning Transfer System Inventory provides valuable guidance on how to implement training programs and evaluate their training effect.

This study adopts the training transfer model as a framework to explore the mechanisms of “how training works” rather than simply assessing “whether training works [ 31 ].” By examining factors such as Motivation to Improve Work Through Learning (MTIWL), learning environment, and transfer design, the current situation is analyzed, underlying reasons are identified, and relevant literature is reviewed to develop and implement training programs based on the results of a needs survey. While individual transfer motivation originates from within the individual, it is influenced by the transfer atmosphere and design. By revising the nurse promotion system and performance management system and aligning them with career development, nurses’ motivation to participate and engage in active learning has significantly increased [ 32 ]. At the learning environment level, enhancing the training effect involves improving factors such as stimulation and response that correspond to the actual work environment [ 33 ]. This project has garnered attention and support from hospital-level leaders, particularly the nursing dean who regularly visits the training site to provide guidance, which serves as invaluable recognition. Timely publicity and recognition of exemplary project improvement initiatives have also increased awareness and understanding of project knowledge among doctors and nurses, fostering a stronger quality improvement atmosphere within the team.

Transfer design, the most critical component for systematic learning and mastery of quality management tools, is achieved through theoretical lectures, salon exchanges, and project-based training. These approaches allow nurses to gain hands-on experience in project improvement under the guidance of instructors. Throughout the project, nurses connect project management knowledge and skills with practical application, enabling personal growth and organizational development through problem-solving in real work scenarios. Finally, a comprehensive evaluation of the training program was conducted, including assessments of theoretical knowledge, perception of management challenges, and project quality. The results showed high satisfaction among nurses, with a satisfaction rate of 97.75%. The proportion of nurses’ participation time in theoretical and practical training classes was 36.88% and 74.23%, respectively. The average score for theoretical knowledge of quality management increased from 75.05 to 90.82. There was also a significant improvement in the evaluation of the implementation difficulties of quality management projects. Moreover, 22 nurses successfully led the completion of one project improvement project, with six projects focusing on preventing the COVID-19 pandemic, demonstrating valuable crisis response practices.

Action research helps to ensure the quality of organizational management of training

Well-organized training is the basis for ensuring the scientific and standardized development of nursing project improvement activities. According to the survey results of the current situation, there is a lot of room for improvement in the training quality; since it is the first time to apply the Holton training transfer model to the improvement training process of nurses in the hospital, in order to allow the nurses to have sufficient time to implement and evaluate the improvement project, the total training time Set at 11 months, a strong methodology is required to ensure training management during this period. Action research is a research method that closely combines research with solving practical problems in work. It is a research method aimed at solving practical problems through self-reflective exploration in realistic situations, emphasizing the participation of researchers and researchees. Practice, find problems in practice, and adjust the plan in a timely manner. According to the implementation of the first round, it was found that nurses had insufficient literature review skills, insufficient leadership, and lack of support from hospital management and related departments [ 32 ]. In the second round, the training courses were carried out in the form of salons. The project team members went deep into the project to improve on-site guidance, arranged graduate students to assist in document retrieval and evaluation, and promoted the transfer of training; the “Hospital Safety and Quality Red May” activity was carried out, and the vice president of nursing Regularly participate in the salon and make comments. The problems exposed after this round of implementation are the low quality of the project improvement project document, and the unclear platform and mechanism for the transfer of training results. In the third round, the “Reporting Standards for Quality Improvement Research (SQUIRE)” was used to standardize the writing of the report [ 33 ], and the “Compilation of Nursing Project Improvement Projects” was completed, and the main evaluation data of 22 projects were entered into the hospital-level quality index management platform for continuous monitoring and intervention. As of May 2022, the effect maintenance data of each project has reached the target value. It can not only produce useful improvement projects, but also help to promote the dissemination and penetration of quality awareness.

Future research directions

Drawing on the Holton training evaluation model, this study implemented nurse quality improvement project training using action research methodology, resulting in a successful exploration practice, and achieving positive transfer effects. To further advance this research area, the following future research directions are recommended:

Summarize the experiences gained from this action research training and continue to refine and enhance the training program. Through ongoing practice, reflection, and refinement in subsequent training sessions, long-term observation of the transfer effects can be conducted to establish an effective experiential model that can serve as a reference for future initiatives.

Develop a multidimensional evaluation index system for assessing transfer effects. A comprehensive framework that captures various dimensions of transfer, such as knowledge application, skill utilization, and behavior change, should be established. This will enable a more holistic and accurate assessment of the training program’s impact on the participants and the organization.

Conduct qualitative research to explore the training experiences of nurses. By gathering in-depth insights through interviews or focus group discussions, a deeper understanding of the nurses’ perceptions, challenges, and facilitators of training transfer can be obtained. This qualitative exploration will provide valuable information to further refine and tailor the training program to meet the specific needs and preferences of the nurses.

Investigate the personality characteristics of nurses who actively engage in training transfer and consider developing them as internal trainers. By identifying and cultivating nurses with a proactive attitude and a strong inclination towards knowledge application and skill development, the organization can enhance employee participation and initiative. These internal trainers can play a crucial role in motivating their colleagues and driving the transfer of training outcomes into daily practice.

By pursuing these future research directions, the field of healthcare and nursing care can continue to advance in optimizing training programs, enhancing transfer effects, and ultimately improving the quality of care and patient outcomes.

Limitations

The research was conducted with a cohort of 22 nurses and a 10-member research team from Grade 3, Class A hospitals in China Southeast. This specific composition and the relatively small sample size may affect the generalizability of our findings. The experiences and outcomes observed in this study might not fully encapsulate the diverse challenges and environments encountered by nursing professionals in varying healthcare settings. The significant improvements noted in the capabilities of the participating nursing staff underscore the potential impact of the training program. However, the study’s focus on a specific demographic—nurses from high-grade hospitals in a developed urban center—may limit the external validity of the findings.

Conclusions

This study affirms the efficacy of the Holton Learning Transfer System Inventory-based training program, coupled with action research, in significantly advancing nursing quality management practices. The strategic incorporation of motivation to improve work through learning, an enriched learning environment, and thoughtful transfer design significantly boosted the nurses’ engagement, knowledge acquisition, and practical application of quality management tools in their clinical work.

It highlights the importance of continuous learning, organizational support, and methodological flexibility in achieving sustainable improvements in healthcare quality and safety. Future endeavors should aim to expand the scope of this training model to diverse nursing contexts and evaluate its long-term impact on organizational performance and patient care outcomes.

Data availability

The datasets generated and/or analyzed during the current study are not publicly available due to hospital policy but are available from the corresponding author on reasonable request.

National Health and Family Planning Commission. Medical Quality Management Measures. Published 2016. Accessed September 19, 2018. http://www.nhc.gov.cn/fzs/s3576/201610/ae125f28eef24ca7aac57c8ec530c6d2.shtml .

National Health Commission of the People’s Republic of China. Notice of the National Health Commission on Printing and Distributing the Standards for the Review of Third-level Hospitals (2020 Edition) [EB/OL]. (2021-10-21) [2022-07-17].

Grossu-Leibovica D, Kalkis H. Total quality management tools and techniques for improving service quality and client satisfaction in the healthcare environment: a qualitative systematic review. Manage Sci Lett. 2023;13(2):118–23. https://doi.org/10.1051/shsconf/202213102009 .

Article   Google Scholar  

Shade L, Reeves K, Rees J, Hendrickson L, Halladay J, Dolor RJ, Bray P, Tapp H. Research nurses as practice facilitators to disseminate an asthma shared decision making intervention. BMC Nurs. 2020;19:40. https://doi.org/10.1186/s12912-020-00414-0 .

Ali J, Jusoh A, Abbas A, Nor K. Global trends of Service Quality in Healthcare: a bibliometric analysis of Scopus Database. J Contemp Issues Bus Government. 2021;27:2917.

Google Scholar  

Ahmed FA, Choudhary RA, Khan H, Ayub F, Hassan SSU, Munir T, Asif F, Ajani K, Jaffer M, Tharani Z, Aboumatar HJ, Haider A, Latif A. Incorporating Patient Safety and Quality Course into the nursing curriculum: an Assessment of Student gains. J Patient Saf. 2023;19(6):408–14. https://doi.org/10.1097/PTS.0000000000001146 .

Holton EF. Holton’s evaluation model: new evidence and construct elaborations. Adv Developing Hum Resour. 2005;7(1):37–54. https://doi.org/10.1177/1523422304272080 .

Rigot SK, DiGiovine KM, Boninger ML, Hibbs R, Smith I, Worobey LA, Connerton C, Mason J, Bonhotal S. (2023). Peer-Led Functional Mobility and Transfer Training. Nurse Educator, 48(5), 286.

Wang C, Liu A, Xu J, et al. Work Method and Effect of Hospital Quality Management Circle Stage Management model [J]. Chin Hosp. 2016;20(12):23–5.

Elshama S. Quality Management in Medical Education between Theory and Application: paradigm shift or falsification of. J Clin Case Rep Stud. 2022;3:1–5. https://doi.org/10.31579/2690-8808/109 .

Chang S-J, Huang HH-C, Li-Hua, Chang H. Taiwan quality indicator project and hospital productivity growth. Omega. 2011;39(1):14–22.

Arnold AP, Laurene Finley [email protected], Roberta G, Sands, Joretha Bourjolly & Victoria Stanhope. (2012) Training Mental Health Providers in Cultural Competence: A Transformative Learning Process, American Journal of Psychiatric Rehabilitation, 15:4, 334–356, https://doi.org/10.1080/15487768.2012.733287 .

Chang S-J, Hsiao H-C, Huang L-H, Chang H. 2011. Taiwan quality indicator project and hospital productivity growth, Omega, Elsevier, vol. 39(1), pages 14–22, January. 2010.01.006.

Finn F, Chesser-Smyth P. Promoting learning transfer in Preceptor Preparation. J Nurses Prof Dev. 2013;29:309–15. https://doi.org/10.1097/NND.0000000000000014 .

Guzman G, Fitzgerald J, Fulop L, Hayes K, Poropat A, Avery M, Campbell S, Fisher R, Gapp R, Herington C, McPhail R, Vecchio N. How best practices are copied, transferred, or translated between health care facilities: a conceptual framework. Health Care Manage Rev. 2015;40:193–202. https://doi.org/10.1097/HMR.0000000000000023 .

Billings D, Connors H, Skiba D. Benchmarking Best practices in web-based nursing courses. Adv Nurs Sci. 2001;23:41–52. https://doi.org/10.1097/00012272-200103000-00005 .

Devos C, Dumay X, Bonami M, Bates R, Holton E. The learning transfer system inventory (LTSI) translated into French: Internal structure and predictive validity. Eur Economics: Labor Social Conditions eJournal. 2007. https://doi.org/10.1111/j.1468-2419.2007.00280.x .

Mongkolsirikiet K, Akaraborworn C, Research Model. (2019). A Revisit of Holton’s HRD Evaluation and (2005) for Learning Transfer., 12, 15–34. https://doi.org/10.14456/JCDR-HS.2019.12 .

Yaqub Y, Singh A. Impact of training design on trainees’ motivation: an empirical study. Industrial Commercial Train. 2021. https://doi.org/10.1108/ict-05-2021-0038 .

Dewettinck K, Dijk H. Linking Belgian employee performance management system characteristics with performance management system effectiveness: exploring the mediating role of fairness. Int J Hum Resource Manage. 2013;24:806–25. https://doi.org/10.1080/09585192.2012.700169 .

Duprez V, Vandecasteele T, Verhaeghe S, Beeckman D, Hecke A. The effectiveness of interventions to enhance self-management support competencies in the nursing profession: a systematic review. J Adv Nurs. 2017;73:1807–24. https://doi.org/10.1111/jan.13249 .

Norouzi S, Mogadam F. Experiences of nursing Student\‘s clinical evaluation: a qualitative content analysis. J Med Educ Dev. 2016;11:134–45.

Niskala J, Kanste O, Tomietto M, Miettunen J, Tuomikoski A, Kyngäs H, Mikkonen K. Interventions to improve nurses’ job satisfaction: a systematic review and meta-analysis. J Adv Nurs. 2020. https://doi.org/10.1111/jan.14342 .

Gkioka M, Schneider J, Kruse A, Tsolaki M, Moraitou D, Teichmann B. Evaluation and Effectiveness of Dementia Staff Training Programs in General Hospital settings: a narrative synthesis with Holton’s three-level Model Applied. J Alzheimers Dis. 2020;78:1089–108. https://doi.org/10.3233/JAD-200741 .

Ghazvini A, Shukur Z. Awareness training transfer and Information Security Content Development for Healthcare Industry. Int J Adv Comput Sci Appl. 2016. https://doi.org/10.14569/IJACSA.2016.070549 . 7.

Ragsdale J, Berry A, Gibson J, Herber-Valdez C, Germain L, Engle D. Evaluating the effectiveness of undergraduate clinical education programs. Med Educ Online. 2020;25. https://doi.org/10.1080/10872981.2020.1757883 .

Kirkpatrick DL. Techniques for evaluation training programs. J Am Soc Train Dir. 1959;13:21–6.

Kirkman TR. High Fidelity Simulation Effectiveness in Nursing Students’ Transfer of Learning International Journal of Nursing Education Scholarship, vol. 10, no. 1, 2013, pp. 171–176. https://doi.org/10.1515/ijnes-2012-0009 .

Chen SL, Huang TW, Liao IC, Liu C. Development and validation of the simulation learning effectiveness inventory. J Adv Nurs. 2015;71(10):2444–53. https://doi.org/10.1111/jan.12707 .

Ayed A, Khalaf I, Fashafsheh I, Saleh A, Bawadi H, Abuidhail J, Thultheen I, Joudallah H. Effect of High-Fidelity Simulation on Clinical Judgment among nursing students. Inquiry: J Med Care Organ Provis Financing. 2022;59. https://doi.org/10.1177/00469580221081997 .

Yun J, Kim D, Park Y. The influence of informal learning and learning transfer on nurses’ clinical performance: a descriptive cross-sectional study. Nurse Educ Today. 2019. https://doi.org/10.1016/J.NEDT.2019.05.027 .

Bhatti M, Ali S, Isa M, Battour M. Training transfer and transfer motivation: the influence of individual, environmental, situational, Training Design, and affective reaction factors. Perform Improv Q. 2014;27:51–82. https://doi.org/10.1002/PIQ.21165 .

Kontoghiorghes C. Factors affecting training effectiveness in the context of the introduction of a New Technology–A U.S. Case Study. Int J Train Dev. 2001;5:248–60. https://doi.org/10.1111/1468-2419.00137 .

Download references

This study was funded by Department of Education of Zhejiang Province, Grant Number jg20220475.

Author information

Authors and affiliations.

School of Nursing, Zhejiang Shuren University, 8 Shuren Road, 310015, Hangzhou, ZheJiang, China

MingYan Shen & ZhiXian Feng

Department of Nursing, Shulan (Hangzhou) Hospital, Shulan International Medical College, Zhejiang Shuren University, 310022, Hangzhou, China

You can also search for this author in PubMed   Google Scholar

Contributions

The following statements specify the individual contributions of each author to the manuscript titled “Continue Nursing Education: An Action Research Study on the Implementation of a Nursing Training Program Using the Holton Learning Transfer System Inventory”:ZhiXian Feng conceived and designed the analysis; led the research team and coordinated the project; critically reviewed and revised the manuscript for important intellectual content; oversaw the implementation of the training program; MingYan Shen conducted the research; collected and organized the data; analyzed and interpreted the data; contributed to the statistical analysis; wrote the initial draft of the manuscript; managed logistics and operational aspects of the study.

Corresponding author

Correspondence to ZhiXian Feng .

Ethics declarations

Ethics approval and consent to participate.

Approval of this study was granted by the Research Ethics Committee of Shulan Hospital (Approval no. KY2021042).

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Prior to participation, all subjects (or their legal guardians) were informed about the nature, objectives, potential benefits, and risks of the study. Written informed consent was obtained from all subjects and/or their legal guardians. All data were collected and processed maintaining strict confidentiality and anonymity, safeguarding the privacy and rights of all participants.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Shen, M., Feng, Z. Continue nursing education: an action research study on the implementation of a nursing training program using the Holton Learning Transfer System Inventory. BMC Med Educ 24 , 610 (2024). https://doi.org/10.1186/s12909-024-05552-6

Download citation

Received : 16 December 2023

Accepted : 13 May 2024

Published : 03 June 2024

DOI : https://doi.org/10.1186/s12909-024-05552-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Nursing education
• Quality improvement
• Action research
• Holton Learning Transfer System Inventory

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

We couldn’t find any results matching your search.

Please try using other words for your search or explore other sections of the website for relevant information.

We’re sorry, we are currently experiencing some issues, please try again later.

Our team is working diligently to resolve the issue. Thank you for your patience and understanding.

News & Insights

Rio Tinto To Invest US$143 Mln To Develop Research & Development Facility In Western Australia

June 03, 2024 — 09:12 pm EDT

Written by RTTNews.com for RTTNews  ->

(RTTNews) - Rio Tinto (RTPPF.PK, RIO.L, RIO, RTNTF.PK) said it will invest US$143 million to develop a research and development facility in Western Australia to further assess the effectiveness of its low-carbon ironmaking process, BioIron, to support decarbonising the global steel value chain.

The company noted that BioIron uses raw biomass and microwave energy instead of coal to convert Pilbara iron ore to metallic iron in the steelmaking process.

The BioIron facility will include a pilot plant that will be ten times bigger than its predecessor in Germany.

The plant has been designed in collaboration with University of Nottingham, Metso Corporation and Western Australian engineering company Sedgman Onyx. Fabrication of the equipment will begin this year, with commissioning expected in 2026. These works are expected to support up to 60 construction jobs.

According to the company, the research and development facility will employ around 30 full-time employees and include space for equipment testing to support further scaling up of the BioIron technology, while developing a workforce highly skilled in steel decarbonisation and supporting WA universities and research organisations.

The views and opinions expressed herein are the views and opinions of the author and do not necessarily reflect those of Nasdaq, Inc.

Stocks mentioned

More related articles.

This data feed is not available at this time.

Sign up for the TradeTalks newsletter to receive your weekly dose of trading news, trends and education. Delivered Wednesdays.

To add symbols:

• Type a symbol or company name. When the symbol you want to add appears, add it to My Quotes by selecting it and pressing Enter/Return.
• Copy and paste multiple symbols separated by spaces.

These symbols will be available throughout the site during your session.

Your symbols have been updated

Edit watchlist.

• Type a symbol or company name. When the symbol you want to add appears, add it to Watchlist by selecting it and pressing Enter/Return.

Opt in to Smart Portfolio

Smart Portfolio is supported by our partner TipRanks. By connecting my portfolio to TipRanks Smart Portfolio I agree to their Terms of Use .

• Tools and Resources
• Customer Services
• African Literatures
• Asian Literatures
• British and Irish Literatures
• Latin American and Caribbean Literatures
• North American Literatures
• Oceanic Literatures
• Slavic and Eastern European Literatures
• West Asian Literatures, including Middle East
• Western European Literatures
• Ancient Literatures (before 500)
• Middle Ages and Renaissance (500-1600)
• Enlightenment and Early Modern (1600-1800)
• 19th Century (1800-1900)
• 20th and 21st Century (1900-present)
• Children’s Literature
• Cultural Studies
• Film, TV, and Media
• Literary Theory
• Non-Fiction and Life Writing
• Print Culture and Digital Humanities
• Theater and Drama
• Share This Facebook LinkedIn Twitter
• Back to News

What’s New: May 2024

May 22, 2024.

In May 2024, 99 full new articles and 11 revised articles, spread across 23 subjects, have been published on the Oxford Research Encyclopedias platform.

Discover a selection of featured articles:

• Anthropology: “ Food Sovereignty " by M.P. Pimbert and Priscilla Claeys
• Climate: “ Radical Perspectives on Climate Change: From Critical Theory to Eco-Marxism and Beyond " by Alf Hornborg
• Communication: “ Social Interaction in VR " by Eugy Han and Jeremy N. Bailenson
• Linguistics: “ Intonation in the Romance Languages " by Barbara Gili Fivela
• Natural Hazard Science: “ Gender, International Law, and Disasters ” by Marie Aronsson-Storrier

Browse all recently published articles | All freely available articles

The Oxford Research Encyclopedias (OREs) offer long-form overview articles written, peer-reviewed, and edited by leading scholars. The OREs are available by annual subscription or individual purchase to libraries and institutions, so ask your librarian if you have access. If you don’t, then recommend this essential resource to your librarian today .

Archives Archives

Printed from Oxford Research Encyclopedias, Literature. Under the terms of the licence agreement, an individual user may print out a single article for personal use (for details see Privacy Policy and Legal Notice).

date: 04 June 2024

• Cookie Policy
• Privacy Policy
• Legal Notice
• Accessibility
• [66.249.64.20|81.177.182.136]
• 81.177.182.136

• Open access
• Published: 28 May 2024

Situational analysis of hypertension management at primary health care level in São Paulo, Brazil: population, healthcare professional and health system perspectives

• Marta S. Palmeirim 1 , 2 ,
• Yara C. Baxter 3 ,
• Mariana Silveira 4 ,
• Rafael V. Maggion 4 ,
• Beatriz Aquino 4 ,
• Álvaro Avezum 6 , 7 , 8 , 9 ,
• Jasmina Saric 1 , 2 ,
• Louise Morgan 10 ,
• Luciano F. Drager 5 , 11 ,
• Luiz A. Bortolotto 11 , 12 ,
• Suely Miya S. Rollemberg 13 ,
• Marcia M. C. de Lima 13 ,
• Edmir P. R. Albuquerque 13 ,
• Olivia Jones 10 ,
• Peter Steinmann 1 , 2 ,
• Theresa Reiker 3 &
• Johannes Boch 3  

BMC Health Services Research volume  24 , Article number:  668 ( 2024 ) Cite this article

179 Accesses

Metrics details

Government-led, population-wide initiatives are crucial for advancing the management of hypertension − a leading cause of cardiovascular disease (CVD) morbidity and mortality. An urban population health initiative was conducted against this backdrop, focussing on hypertension in the primary health system in São Paulo, Brazil. Within the frame of the initiative and under the supervision and leadership of the municipal health authorities, a situational analysis was conducted on the needs in hypertension management, marking the first phase of a Design Thinking process. This article describes the situational analysis process and presents the identified elements to be strengthened considering hypertension diagnosis, treatment and control.

First, a mixed-methods approach was used, starting with a literature review of municipal hypertension data followed by meetings ( N  = 20) with the local public health administration to assess health system level components. To investigate activities on hypertension diagnosis, treatment and control, nine primary healthcare units were selected from two districts of São Paulo city– Itaquera and Penha– which received an online form addressed to managers, participated in conversation circles of staff and patients, and underwent shadowing of community health agents.

Data gave rise to two main outputs: (i) a patient care journey map; and (ii) a matrix summarizing the identified needs at patient, healthcare professional and health system level for diagnosis, treatment and control of hypertension. Patient awareness and knowledge of hypertension was found to be insufficient and its management needs to be improved. For health professionals, disease awareness, technical training, more time dedicated to patients, and simplified guidelines and clinical decision-making tools for hypertension management were identified as principal needs. The situational analysis found that the healthcare systems efficiency might be improved by establishing defined treatment and care delivery goals with a focus on outcomes and implemented through action plans.

Conclusions

This situational analysis identified several needs related to hypertension control in São Paulo that are in line with global challenges to improve the control of CVD risk factors. Findings were also confirmed locally in an expansion phase of this situational analysis to additional primary care facilities. As a consequence, solutions were designed, promptly taken up and implemented by the municipal health secretariat.

Peer Review reports

Hypertension is the leading cause of death globally. The condition affects about 1.4 billion adults and is the main risk factor for cardiovascular diseases (CVD) [ 1 , 2 ]. The management of hypertension improves cardiovascular population health, reduces early mortality and avoidable hospitalizations [ 2 , 3 , 4 ]. 70% of people with hypertension live in low- and middle-income countries, where health systems are largely focused on the delivery of services for acute healthcare needs, as opposed to managing chronic conditions amenable to prevention. The latter requires long-term sustained engagement and interaction between patients, healthcare professionals and the health system to efficiently manage conditions [ 5 ], improvement in event-free survival along with better quality of life, and medical cost reduction. To achieve these goals, the identification of specific barriers and needs related to effective hypertension control is required to develop targeted interventions and spark the necessary transformation of health systems and their services delivery models [ 6 ].

Like many other countries across the globe, Brazil has undergone a major shift in its epidemiological profile; burdened historically by infectious diseases, the country’s main public health challenges today stem from non-communicable diseases (NCDs) [ 7 ]. The increase of NCDs observed in Brazil and elsewhere was precipitated by an increasingly sedentary lifestyle and changing eating habits, leading to a high prevalence of obesity and other CVD risk factors [ 2 , 8 ]. In 2016, CVDs were the leading cause of morbidity and mortality [ 9 ]. A recent study exploring hypertension prevalence in different South American countries found a 53% prevalence among 5,557 Brazilians aged 35 to 70 years from both rural and urban settings [ 10 ]. Investing in corrective and preventive behavioural approaches in the face of ever-increasing treatment expenses has become a priority for the government [ 11 , 12 ]. Studies have shown that the Brazilian primary health system should be strengthened in the area of disease management pathways (including for NCDs) with a people centred perspective [ 13 ].

The challenges of NCDs, as in other parts of the world, also impact São Paulo city, Brazil’s most populated city, despite the constant efforts of local health authorities to reduce their prevalence and burden [ 14 , 15 ]. In São Paulo, the prevalence of hypertension in the adult population has been estimated to be 23.2% in a telephone-based survey [ 16 ]. However, the same survey found large variability in prevalence within the adult population; in people aged 50 to 69 years of age, the prevalence of hypertension varied from 34.5 to 51.3%. In São Paulo city, half of the population depends on Brazil’s unified public health system (Sistema Único de Saúde; SUS), which, among other services, provides universal coverage and access to free medication [ 17 ]. Through different initiatives and reforms, the public health program for São Paulo focuses on the reduction of mortality due to NCDs and includes hypertension management. Prominent examples are the NCD Strategic Action Plan to which São Paulo state adhered to, the tobacco control laws, salt reduction policies and public sport facility programs, which all have relevance for municipal planning of priorities [ 18 , 19 ]. A study from 2011, investigating care pathways of already diagnosed hypertensive patients in the public health care sector of São Paulo, documented a treatment adherence level of 20% [ 20 ] and that 35–50% of patients treated for hypertension had their blood pressure controlled [ 20 , 21 ]. These findings reveal opportunities to increase treatment adherence, quality of care, access to medicines, patient follow-up and care coordination [ 14 ].

The initiative Better Hearts Better Cities (entitled “Cuidando de Todos” in Brazil), applying the CARDIO (Care, Access, policy Reform, Data and digital, Intersectoral collaboration and local Ownership) approach [ 22 ], was designed as a multidisciplinary, multisector initiative to support local authorities in addressing urban cardiovascular population health, including hypertension as primary risk factor. The initiative was implemented in Ulaanbaatar, Mongolia; Dakar, Senegal; and São Paulo, Brazil. In Brazil, the initiative was executed in collaboration with the São Paulo Municipal Health Secretariat (Secretaria Municipal de Saúde; SMS-SP). This partnership aimed at co-creating and accelerating the implementation of a model to approach NCDs. The implementation methodology was guided by a Design Thinking approach [ 23 ]. The current article details the diagnosis step of the methodology, which focused on assessing the primary needs in the public health system in São Paulo. The identified needs of patients, healthcare professionals and the health system with respect to hypertension diagnosis, treatment and control will be presented.

São Paulo city is one of the five most populous metropolis in the American continent and the largest financial and corporate centre in Latin America [ 24 ]. São Paulo was recently included in the “Global Cities of the Future 2021/2022”, occupying the fourth place of the ten “Megacities of the Future” with the best strategies for attracting foreign direct investment [ 25 ]. With more than 12 million inhabitants on 1,521 km 2 [ 26 ], São Paulo was selected for the implementation of Better Hearts Better Cities. Reasons include the range of opportunities to improve hypertension and CV risk factor control, São Paulo’s urban characteristics such as the close co-existence of high-, low- and middle-income features, the developed health system (SUS) and the complexity of urban management, especially in health [ 27 ]. After agreeing to jointly execute the initiative, the SMS-SP selected the districts of Itaquera and Penha to host the first implementation.

Conceptual approach to problem identification

In São Paulo, the initiative was implemented between 2018 and 2020 in the stages of groundwork, diagnosis, exploration, co-creation, implementation and scale-up. This initiative followed a Design Thinking process [ 23 ], using the double diamond approach [ 28 ]. Besides factual roadblocks and concrete challenges, the Design Thinking process also considers latent needs of the health system. These needs may be abstract and emotional and, as such, can be difficult to express or to recognize in traditional needs assessments [ 29 ]. To address this, Design Thinking focuses on the analysis of the end-user and beneficiary perspective in a clarification step (here referred to as situational analysis). Information collected in this step through interviews and shadowing is essential to frame key issues correctly, identify opportunities for improvement and ideate adequate solutions [ 30 ], which can be subsequently implemented in partnership with the local health authorities. This situational analysis and its results of the health care ecosystem’s needs for patients, healthcare professionals and the system itself are described in the current manuscript. The process used to conduct this situational analysis is aligned with the recommendations of the World Health Organization (WHO) HEARTS package [ 31 ] and the World Heart Federation [ 6 ].

Primary health care structure in São Paulo

The situational analysis took place in 2018 in the district of Itaquera and in 2020 in the district of Penha. These districts were selected due to their representativeness of low- and middle-income communities. Additionally, primary healthcare units (Unidades Básica de Saúde; UBS) with different health system management models are present in these districts; the primary health care network of São Paulo is composed of 450 UBSs (traditional model) [ 32 ] and 117 outpatient medical care centres (Assistência Médica Ambulatorial; AMA) [ 33 ], of which 87 are integrated with a UBS [ 34 ]. The UBS network is staffed by different types of primary care teams (Equipes de Atenção Básica; EAB), including those organized according to the family health strategy (Estratégia Saúde da Família; ESF), a Brazilian management model of primary care delivery [ 35 ]. In Brazil, the organization, management and execution of services and activities of primary health care is the responsibility of the municipality. In São Paulo, health facilities can be directly managed by the SMS-SP (direct administration) [ 36 , 37 ] or, alternatively, the management can be delegated to social organizations (e.g. cooperatives), private companies, non-governmental or philanthropic organizations [ 38 ]. Table  1 depicts the characteristics related to primary care infrastructure in Itaquera, Penha and São Paulo city as a whole.

Situational analysis procedures

The situational analysis was conducted by a team composed of members of the Novartis Foundation (global initiative coordinator), the Instituto Tellus (local implementation partner), the São Paulo Society of Cardiology (Sociedade de Cardiologia da Cidade de São Paulo; SOCESP), as well as members of the SMS-SP. The municipal authorities approved the initiative and its strategy, and took ownership for the execution, facilitated communication with internal stakeholders, helped establish a first technical working group and identified the first implementation districts. The SMS-SP assigned the initiative to the chronic diseases and NCD care (Doenças e Agravos Não Transmissíveis; DANT) technical department. This department was created in 2018 to elaborate a municipal care protocol for NCDs and harmonize the use of different national and international guidelines by physicians in the public health system of São Paulo. Thus, the initiative fully supported the priorities of the SMS-SP.

With this governance established, the team started the first part of the situational analysis by addressing health system components with a focus on primary care delivery. The investigation was conducted at the SMS-SP, in order to understand health system building blocks, such as governance, importance of and priority given to primary care, currently available resources and gaps, data management priorities and available information management systems, current guidelines, their application, and the policy landscape. Special consideration was on other programs related to hypertension in Brazil, and recent initiatives to address other chronic diseases such as diabetes. To complement this information, the team conducted a review of the available literature and municipal data concerning CVDs for São Paulo.

Secondly, continuous follow-up discussions with the NCD technical department at the SMS-SP supported the contextualisation of the findings from the first part of the analysis. Inter-disciplinary workshops with relevant partners (e.g. SMS-SP, public officials representing primary care and/or NCD management, district health care delivery, medical societies, global initiative coordinator, local implementation partner, international scientific partners) were conducted to further deepen the understanding of the health system concerning diagnosis, treatment and control of hypertension. The discussions clarified the relation of municipal strategy to operational execution in the UBS in terms of technical decisions, perception and execution of best practices.

Finally, a total of nine UBS (six in the district of Itaquera and three in the district of Penha) were selected by the SMS-SP to be included in this situational analysis and undergo a thorough investigation using several data collection tools.

Data collection approaches and tools

To gain deep knowledge and understanding of the entirety of the health system, a mixed-methods approach based on the Design Thinking process [ 23 , 39 ] and the AEIOU framework (Activities, Environments, Interactions, Objects, and Users) was pursued [ 40 ]. In addition to the use of these frameworks, to gain a complete understanding of the conditions and potential for interaction with the population in and beyond the health system, other than primary health care unit workers, the team also included relevant focal points within the district, such as community champions (e.g. samba and football club, spiritual leaders) and reference points (e.g. shopping mall, high traffic venues, green areas). Table  2 presents the type of information collected from the different stakeholder groups to help pinpoint unmet needs at primary care level. This was executed using three tools targeting different respondents based on their role in the health system:

An online form consisting of open- and closed-ended questions was applied to collect data for an initial diagnosis of the needs in each UBS (Supplementary files 1 and 2 ; one for Itaquera and one for Penha). An online version of the online form was sent to the UBS managers who were invited to complete it. The online form covered, among other topics, physical and personnel structure of the UBS, use of digital technology, data collection and recording tools and service time of each professional. Additionally, there were a few questions, targeting the work conducted by the community health agents (CHAs), regarding their knowledge around hypertension, their degree of patient engagement, their knowledge on use of medication, and their practice in encouraging healthy lifestyle behaviour. Between the implementation of the online form in Itaquera and Penha, it underwent some adaptations and fine-tuning based on the experience in Itaquera; hence, there were two versions of the online form (Supplementary files 1 and 2 ).

Conversation circles ( n  = 43) were carried out with UBS staff and patients in the UBS offices. For each respondent category, a specific conversation guide with lead questions was available. The goal was to, in each UBS, include one person from each of the following categories: (i) patient; (ii) doctor; (iii) nurse; (iv) community health agent; (v) pharmacist; (vi) admission officer; (vii) UBS manager, and (viii) school health programme manager. Participants were from the same district and selected by the UBS. Additionally, the situational analysis team included experts in the area of cardiovascular health, population health and health systems, among which were directors of medical societies such as SOCESP, NCDs forum [ 41 ] and representatives of the Brazilian Ministry of Health.

Shadowing was conducted by members of the implementation team who observed two CHAs from one selected UBS throughout their day of work [ 40 ]. The main goal of this approach was to understand the relationship between CHAs and hypertensive patients. Additionally, CHAs were asked about their beliefs, attitudes and perception of hypertension, as well as about their observations and perceptions on the population’s diet and physical activity in the context of the disease cluster. Observations and responses were recorded using a standard data collection form (Supplementary file 3 ).

Data analysis

The information gathered by the implementation team using online forms, conversation circles, shadowing activities and government workshops, was compiled, tabulated and systematized by Instituto Tellus. This provided a first assessment of the existing needs, which resulted in two main outputs. The first was a patient care journey map, investigating the interactions between patients and the primary care team and mapping challenges and opportunities. This map served as an interactive and iterative tool during data collection, allowing for the visualization of details of the system through which patients with hypertension navigate and, therefore, helped clarify needs and bottlenecks present at different stages of the patient’s journey. The second main output was a matrix presenting the identified needs presented along three major components (i.e. patients, healthcare professionals and health system) for the diagnosis, treatment and control stages of hypertensive patients.

The gaps identified during this step were taken up in the co-creation sessions with the city health authorities, members of the American Heart Association (AHA), SOCESP and other primary care practitioners for refinement and to brainstorm for solutions. This process has been described in detail elsewhere [ 23 ]. All results were discussed and validated by the SMS-SP, which confirmed the adequateness and provided further input.

All nine UBS managers who were asked to respond to the online form returned it and all 30 people who were planned to include in conversation circles attended.

The nine pilot UBSs were representative of the different UBS administrative models of the Municipality of São Paulo: two traditional UBS, four AMA integrated with a UBSs, and three UBSs with ESF. Of these, one of the traditional UBS was under direct administration and the remaining eight were managed by a social organization. These UBSs served between 12’000 and 39’964 people. The operational models, management types, number of people and adults covered and years in operation for each of the nine UBSs are presented in Table  3 . The newest UBS had been in operation for 10 years, whereas the oldest had been established 45 years ago. In terms of data collection systems, all UBS used SIGA Saúde (Sistema Integrado de Gestão de Assistência à Saúde de São Paulo) for the registration of patients’ appointments and to record basic data, and Gestão de Sistemas de Saúde (GSS; Health system Management), which is the medicines logistics system. It was also found that all nine UBSs had an integrated pharmacy.

The analysis of the results from the nine pilot UBSs revealed that, to improve the quality of the data collected, the online form needed to be revised. Questions that were found to not bring relevant insights were removed, (e.g. “how many functioning printers does your UBS have?”), while additional questions were introduced to fill data gaps noticed.

Patient care journey map

The results from the online forms, conversation circles and shadowing of CHAs were consolidated and mapped out in the patient care journey map (Supplementary file 4 : Figure 1), which catalogues the structure of the observations of primary needs along a timeline. The starting point of each overlapping journey was a research-based fictional persona that represented a particular group (i.e. patient, community health agent, doctor, pharmacist, nurse) [ 42 ]. Each persona’s journey started with a short description of the official requirements from the Brazilian comprehensive health and nutrition promotion practices in primary healthcare (Práticas Integrais de Promoção da Saúde e Nutrição na Atenção Básica; PINAB). The journey also showed explicit and latent needs. By visually exposing the information collected, the team was able to trigger discussions among health managers, clinical team members and health authorities and to understand whether the results obtained by the implementation team corresponded to the actual needs experienced by healthcare professionals and managers. This map helped identify several opportunities on how to improve the patients’ experience. Those participating in the exercise reported that it was the first time this complex system of interactions was depicted in a visual way and was, thus, insightful to gain oversight of paint points, process overlaps and opportunities to optimize team work.

Main needs and barriers identified

The mapped barriers for the different CVD management stages (i.e. diagnosis, treatment and control) for patients, healthcare professionals and the health system resulting from the pilot phase are presented in Table  4 . Conversation circles found, for instance, that the patients’ knowledge and awareness of the importance of hypertension risks and of regularly measuring blood pressure was poor, particularly in men and young people. According to patients, these issues, coupled with a poor understanding of the treatment and problems in the health system such as insufficient staff, in turn, according to health professionals, led to low adherence to treatment. From the patient perspective, for successful control of their blood pressure, they needed a stronger bond and a good relationship with the health staff to increase their engagement.

From the perspective of healthcare professionals, there was a lack of training, and the quality in medical record entries required attention. Concerning treatment, healthcare professionals felt that the training of CHAs on the topic of NCDs, the motivation provided to patients and the quality of guidelines needs to improve. As an example, hypertension was diagnosed following the 7th Brazilian guideline [ 43 ], but the situation analysis revealed that due to the complexity of the guideline, those best-practices are only partially applied. Concerning the control of hypertension, the presented findings showed that healthcare professionals needed more time with patients to explain treatment in detail, but also highlighted some of the inefficiencies in the health system that would need to be resolved.

Concerning the health system, a perception that 15-minute appointments were too short and that this negatively impacted both the diagnosis and treatment steps was mentioned by both health professionals and patients. Finally, the system also lacked clear goals and action plans when it came to diagnosis, treatment, and control (i.e. follow-up of patients).

For each of the needs and barriers identified for patients, health professionals and the health system (Table  4 ), brainstorming sessions led to the outlining of prototype solutions. These prototype solutions later underwent a selection phase and were developed into the solutions that were implemented; these final solutions are presented in a second publication in this series of publications on the Better Hearts Better Cities initiative in Brazil.

The situational analysis and its findings were the first steppingstone of the Design Thinking process used in the urban population health initiative, Better Hearts Better Cities, in São Paulo city. The data captured on health system components and different stakeholders, such as patients, healthcare professionals, managers and decision-makers, has led to the identification of several needs relating to the diagnosis, treatment and control of hypertension in low- and middle-income settings. Findings were used as a basis for the design of specific interventions targeting the identified needs, which were subsequently taken up and implemented.

The analysis showed that the patient’s perspective encompassed elements connected to disease awareness (specifically regarding men and youth), self-care (including no standard possibility to track disease status, low adherence, missing follow-up medical appointments, and low motivation), difficulties of patients to engage in the care process, build an emotional connection to primary care teams, and lack of consistent early detection and standards of care. For healthcare professionals, needs in optimizing and clarifying guideline-based care, treatment and follow-up, improving information management (e.g. quality of medical record and completeness of documentation), enhancing standard continuous training programs on disease and treatment, and reinforcing team-based care were identified. From the health system perspective, potential was seen to define structured processes to upgrade efficiency, and establish defined treatment and care delivery goals with a focus on outcomes, action plans and applicable guidelines and tools on diagnosis, treatment and control. All the findings of this situational analysis were later confirmed during an expansion phase of the same methodology implemented in another 18 UBS in Itaquera and 18 UBS in Penha. Additionally, the results of this analysis are in line with roadblocks identified by entities such as the World Heart Federation [ 6 ] or the Organisation for Economic Co-operation and Development [ 44 ]. Hence, it is assumed that the identified challenges including primary care management, patient pathway/journey, and workforce up-skilling are transferable to other geographies and health systems.

To our knowledge, this work is the first that displays the health system in São Paulo or even Brazil, its team structures, responsibilities, and needs on a comprehensive patient journey map. This situational analysis found that, although aware of the complexity of the health system, the visualization of the journey map gave health system managers and healthcare professionals a new perspective on the system that they navigate daily. This provided valuable insights for discussion of opportunities and challenges to address latent and explicit needs, process optimization and task-shifting.

In the context of this urban population health initiative, we evaluated the existing gaps in the healthcare system by engaging with health professionals and patients. This situational analysis provided valuable insights that directly fed into the design of specific interventions and further informed the CARDIO4Cities approach. The CARDIO4Cities approach, which stands for Care, Access, Policy Reform, Data and Digital, Intersectoral Collaboration, and Local Ownership, as described by Aerts and Boufford [ 22 ], was developed based on these insights. This comprehensive situational analysis aided in identifying the specific needs and opportunities within the healthcare system, enabling the development of targeted interventions. The implementation of the CARDIO4Cities approach in São Paulo, guided by the São Paulo health authorities, has been proven to be impactful [ 45 , 46 ]. We believe that a precise understanding of the situation and the healthcare system’s needs is crucial for devising tailored strategies to improve cardiovascular health. During the diagnostic phase, the information gathered through this situational analysis was used to co-create a total of 14 solutions in collaboration with various stakeholders. The objective of these solutions was to reform hypertension management, optimize healthcare delivery, and increase cardiovascular population health.

This process described in the paper showed that despite being resource-intensive due to the assessment being embedded in the primary care context and its daily operation, the methodology led to findings that were perceived to be focused on core needs of the beneficiaries, namely the patients, healthcare professionals, managers and decision-makers and as such were meaningful, endorsed and accepted as appropriate. The findings allowed for the development of sustainable interventions, enabling local stakeholders to feel fully involved and empowered in the whole process. This focus on actual needs is a major strength of the approach, which was reinforced by the bottom-up analysis of the situation. Another strength is that this initiative was able to bring together all types of stakeholders in a participatory and inclusive manner making sure all voices were heard. Finally, the careful selection of the pilot UBSs, whereby every administration model was represented, was of great importance.

Limitations of this work include that the first version of the online form was too complex for respondents and, therefore, it needed to be adapted. Consequently, the composition of questions asked to UBS managers slightly differed between the two districts. Moreover, the team realized, at the start of the project, that some stakeholders felt they might be judged by what they mentioned, feeling observed and showing some discomfort when it came to sharing any bottlenecks in the system. This hesitancy was, most likely, a result of previous negative experiences where the same stakeholders participated in projects without ever seeing any outcomes or benefits. Thus, it was fundamental to clarify that the aim was to build a partnership offering a safe space to share any issues, in a constructive manner. Finally, it was not always easy to obtain consistent data on CVDs from the different stakeholders as there were several sources to collect the data from. Lastly, the rapid staff turnover at the SMS-SP was, at times, limiting the efficacious progress of the analysis.

Despite all the efforts the SMS-SP has put into reducing the burden of hypertension in São Paulo city, its prevalence remains high and its control remains low. This situational analysis, conducted in close collaboration with SMS-SP, allowed the Better Hearts Better Cities initiative to apply an innovative methodology to identify opportunities for improvement within the health system and established an operational baseline for boosting the diagnosis, treatment, and control of patients with hypertension living in low- and middle-income settings of São Paulo city. In doing so, it laid the foundation for the design of solutions and successful roll-out and expansion of the project. Based on the experience and results from both the pilot and expansion phases, the team recommends the use of this methodology in future studies that aim at identifying needs within a health system based on a well-defined focal area. Employing bottom-up, collaborative, inclusive and needs-focused approaches drives change management, reinforces sustainability and up-take of solutions in the implementation phase.

Data availability

The datasets used and/or analysed during the situational analysis are available from the corresponding author on reasonable request.

Abbreviations

Activities, Environments, Interactions, Objects, and Users

American Heart Association

Assistência Médica Ambulatorial (outpatient medical care centres)

Blood Pressure

Community Health Agents

Conselho Federal de Farmácia (federal pharmacy council)

Conselho Regional de Farmácia (regional pharmacy council)

Coordenadoria Geral de Saúde (regional health coordinator)

Cardiovascular Disease

Equipes de Atenção Básica (primary care teams)

Estratégia Saúde da Família (family health strategy)

Gestão de Sistemas de Saúde (health system management)

Inquérito de Saúde (health survey)

Núcleo Ampliado de Saúde da Família (expanded family health center)

Non-communicable Diseases

Non-governmental Organization

Organização Social de Sáude (health social organization)

Práticas Integrais de Promoção da Saúde e Nutrição na Atenção Básica (comprehensive health and nutrition promotion practices in primary healthcare)

Plano Municipal de Educação Permanente (municipal permanent education plan)

Sistema Integrado de Gestão de Assistência (integrated assistance management system)

Secretaria Municipal de Saúde (municipal health secretariat)

Sociedade de Cardiologia do Estado de São Paulo

Supervisão Técnica de Saúde (technical health supervision)

Sistema Único de Saúde (public health system)

Secretaria de Vigilância em Saúde/Ministério da Saúde (health surveillance secretariat/ministry of health)

Unidade Básica de Saúde (primary healthcare units)

World Health Organization

GBD 2019 Risk Factors Collaborators. Global burden of 87 risk factors in 204 countries and territories, 1990–2019: a systematic analysis for the Global Burden of Disease study 2019. Lancet. 2020;396.

Mills KT, Bundy JD, Kelly TN, Reed JE, Kearney PM, Reynolds K, et al. Global disparities of hypertension prevalence and control: a systematic analysis of population-based studies from 90 countries. Circulation. 2016;134:441–50.

Article   PubMed   PubMed Central   Google Scholar  

Yusuf S, Rangarajan S, Teo K, Islam S, Li W, Liu L, et al. Cardiovascular risk and events in 17 low-, middle-, and high-income countries. N Engl J Med. 2014;371:818–27.

Article   CAS   PubMed   Google Scholar  

WHO. Hypertension factsheet. https://www.who.int/news-room/fact-sheets/detail/hypertension . Accessed 25 Feb 2022.

Lall D, Engel N, Devadasan N, Horstman K, Criel B. Models of care for chronic conditions in low/middle-income countries: a ‘best fit’ framework synthesis. BMJ Glob Health. 2018;3:e001077.

Jeemon P, Séverin T, Amodeo C, Balabanova D, Campbell NRC, Gaita D, et al. World Heart Federation roadmap for hypertension– a 2021 update. Global Heart. 2021;16:63.

Schramm JMDA, Oliveira AFD, Leite IDC, Valente JG, Gadelha ÂMJ, Portela MC, et al. Epidemiological transition and the study of burden of disease in Brazil. Cien Saude Colet. 2004;9:897–908.

Article   Google Scholar  

Whelton PK, He J, Appel LJ, Cutler JA, Havas S, Kotchen TA, et al. Primary prevention of hypertension: clinical and public health advisory from the National High blood pressure education program. JAMA. 2002;288:1882–8.

Article   PubMed   Google Scholar  

Jardim PC, Souza WK, Lopes RD, Brandão AA, Malachias MV, Gomes MM, et al. I RBH - first Brazilian hypertension registry. Arq Bras Cardiol. 2016;107:93–8.

PubMed   PubMed Central   Google Scholar  

Lamelas P, Diaz R, Orlandini A, Avezum A, Oliveira G, Mattos A, et al. Prevalence, awareness, treatment and control of hypertension in rural and urban communities in latin American countries. J Hypertens. 2019;37:1813–21.

Levine S, Malone E, Lekiachvili A, Briss P. Health care industry insights: why the use of preventive services is still low. Prev Chronic Dis. 2019;16:E30.

Piccini RX, Facchini LA, Tomasi E, Siqueira FV, Silveira DS, Thumé E, et al. Promotion, prevention and arterial hypertension care in Brazil. Rev Saude Publica. 2012;46:543–50.

OECD. Primary health care in Brazil. 2021. https://www.oecd-ilibrary.org/docserver/120e170e-en.pdf?expires=1645783803&id=id&accname=ocid195445&checksum=C3282B7E483DB0A81A9D57484B29B1E7 . Accessed 25 Feb 2022.

São Paulo City Hall. Programa de metas 2017–2020. 2017. https://www.nossasaopaulo.org.br/portal/arquivos/PMSP_Programa-de-Metas-2017-2020.pdf . Accessed 25 Feb 2022.

São Paulo City Hall. Plano municipal de saúde 2018 a 2021. 2018. https://www.prefeitura.sp.gov.br/cidade/secretarias/upload/saude/plano_municipal_de_saude_2018_2021.pdf . Accessed 27 Jun 2022.

Municipal Health Secretariat of São Paulo City. ISA Capital-SP 2015 - Systemic arterial hypertension. Municipal Health Secretariat of São Paulo City. 2015. https://www.prefeitura.sp.gov.br/cidade/secretarias/upload/saude/arquivos/publicacoes/ISA_2015_HAS.pdf . Accessed 1 Mar 2022.

(Municipal Health Secretariat of the São Paulo City Hall. DATASUS, Tabnet.). http://tabnet.saude.sp.gov.br/tabcgi.exe?tabnet/ind47b_matriz.def . Accessed 25 Feb 2022.

Saric J, Aerts A, Anne M, Barboza J, Boch J, Dashdorj N, et al. Assessing the contributions of an urban population health initiative to shift political priority towards cardiovascular health in Brazil, Mongolia and Senegal. BMJ Global Health; submitted.

Ministry of Health [Brazil]. Plano de ações estratégicas para o enfrentamento das doenças crônicas e agravos não transmissíveis no Brasil 2021–2030. Brasília: Ministry of Health [Brazil]. 2021. https://www.gov.br/saude/pt-br/centrais-de-conteudo/publicacoes/publicacoes-svs/doencas-cronicas-nao-transmissiveis-dcnt/09-plano-de-dant-2022_2030.pdf/ .

Pierin AM, Marroni SN, Taveira LA, Benseñor IJ. [Hypertension control and related factors at primary care located in the west side of the city of São Paulo, Brazil]. Cien Saude Colet. 2011;16(Suppl 1):1389–400.

Mion D Jr., Pierin AM, Bensenor IM, Marin JC, Costa KR, Henrique LF, et al. Hypertension in the city of São Paulo: self-reported prevalence assessed by telephone surveys. Arq Bras Cardiol. 2010;95:99–106.

PubMed   Google Scholar  

Aerts A, Boufford JI. A new whole-of-city strategy for addressing cardiovascular population health. Cities Health. 2021:1–7.

Jarrett C, Baxter YC, Boch J, Carrasco C, Cobos D, Dib KM, et al. Deconstructing Design thinking as a tool for the implementation of a population health initiative. Health Res Policy Syst. 2022;20:91.

Z/Yen. Leading financial centers in Latin America and Caribbean as of March 2021. https://www.statista.com/statistics/370365/leading-financial-centers-latin-america/ .

fDi Intelligence. Global cities of the future 2021/2022. fDi Intell. 2021:26–38.

Instituto Brasileiro de Geografia e Estatística. Cidades e Estados - São Paulo. https://www.ibge.gov.br/cidades-e-estados/sp/sao-paulo.html?msclkid=d6d3aeefb99a11ec974b6ac9faff198f . Accessed 11 Apr 2022.

Alves TF, Almeida FA, Brito FA, Tourinho FSV, de Andrade SR. Regulation and use of health information systems in Brazil and abroad: integrative review. Comput Inf Nurs. 2021.

Design Council UK. What is the framework for innovation? Design Council’s evolved Double Diamond. https://www.designcouncil.org.uk/news-opinion/what-framework-innovation-design-councils-evolved-double-diamond . Accessed 25 Feb 2022.

Datar SM, Bowler CN. Design thinking and innovative problem solving. In: Jordi, Canals, editors. Shaping entrepreneurial mindsets: innovation and entrepreneurship in leadership development. London: Palgrave Macmillan; 2015. pp. 119–38.

Chapter   Google Scholar  

Campbell NRC, Dashdorj N, Baatarsuren U, Myanganbayar M, Dashtseren M, Unurjargal T, et al. Assessing healthcare professional knowledge, attitudes, and practices on hypertension management. Announcing a new World Hypertension League resource. J Clin Hypertens. 2017;19:830–2.

WHO. Hearts: technical package for cardiovascular disease management in primary health care. 2020. https://apps.who.int/iris/handle/10665/252661 . Accessed 25 Feb 2022.

Mendes EV. A construção Social Da atenção primária à saúde. Brasília: Conselho Nacional de Secretários de Saúde; 2014.

Google Scholar  

Melo DO, Silva SR, Castro LL. Evaluation of drug prescription quality indicators in a primary health care facility with different models of health care. Epidemiol Serv Saude. 2016;25:259–70.

São Paulo City Hall. Directrizes operacionais - unidade básica de saúde. 2016. https://www.prefeitura.sp.gov.br/cidade/secretarias/upload/diretrizesoperacionaisatencaobasica.pdf . Accessed 25 Feb 2022.

Kessler M, Lima SBS, Weiller TH, Lopes LFD, Ferraz L, Thumé E. Longitudinality in primary health care: a comparison between care models. Rev Bras Enferm. 2018;71:1063–71.

Ireland M, Cavalini L, Girardi S, Araujo EC, Lindelow M. Expanding the primary health care workforce through contracting with nongovernmental entities: the cases of Bahia and Rio De Janeiro. Hum Resour Health. 2016;14:6.

Scheffer MC, Aith FMA. O sistema de saúde brasileiro. Clín Méd. 2016;1:355–65.

Silva VC, Barbosa PR, Hortale VA. Partnerships in health systems: social organization as limits and possibilities in the Family Health Strategy management. Cien Saude Colet. 2016;21:1365–76.

Moggridge B. Design interactions. Cambridge, Massachusetts. London, England: The MIT; 2007.

Martin B, Hanington B. Universal methods of design: 100 ways to research complex problems, develop innovative ideas, and design effective solutions. Rockport; 2012.

Forum DCNTs. https://www.forumdcnts.org/ . Accessed 18 May 2021.

Williams I, Brereton M, Donovan J, McDonald K, Millard T, Tam A, et al. A collaborative rapid persona-building workshop: creating design personas with health researchers. Int J Sociotechnology Knowl Dev. 2014;6:17–35.

Malachias MVB, Souza WKSB, Plavnik FL, Rodrigues CIS, Brandão AA, Neves MFT, et al. 7th Brazilian guideline of arterial hypertension. Arq Bras Cardiol. 2016;107:1–83.

OECD. Cardiovascular disease and diabetes: policies for better health and quality of care. 2015. https://www.oecd-ilibrary.org/content/publication/9789264233010-en .

Boch J, Venkitachalam L, Santana A, Jones O, Reiker T, Rosiers SD, et al. Implementing a multisector public-private partnership to improve urban hypertension management in low-and middle- income countries. BMC Public Health. 2022;22:2379.

Reiker T, Des Rosiers S, Boch J, Partha G, Venkitachalam L, Santana A, et al. Population health impact and economic evaluation of the CARDIO4Cities approach to improve urban hypertension management. PLOS Glob Public Health. 2023;3:e0001480.

Download references

Acknowledgements

We specifically thank the municipal authorities from São Paulo, Brazil, for leading and owning this initiative; all healthcare professionals, managers, nurses, doctors, community agents and other healthcare professionals who participated in the initiatives co-creation and roll out; and all patients who participated in the implementation. We are grateful to the UMANE Foundation for the strategic partnership and co-funding as well as all local partners and stakeholders who contributed to the initiative, with a special thanks to the teams of the Instituto Tellus and IQVIA in Brazil. We would like to acknowledge Ariane Abreu for her support reviewing the manuscript.

The Better Hearts Better Cities initiative and the presented paper are funded by the Novartis Foundation. Novartis Foundation contributed to the design of the situational analysis and the collection and interpretation of data and in writing of the manuscript.

Open access funding provided by University of Basel

Author information

Authors and affiliations.

Swiss Tropical and Public Health Institute, Allschwil, Switzerland

Marta S. Palmeirim, Jasmina Saric & Peter Steinmann

University of Basel, Basel, Switzerland

Novartis Foundation, Basel, Switzerland

Yara C. Baxter, Theresa Reiker & Johannes Boch

Instituto Tellus, São Paulo, Brazil

Mariana Silveira, Rafael V. Maggion & Beatriz Aquino

Sociedade de Cardiologia do Estado de São Paulo (SOCESP), São Paulo, Brazil

Luciano F. Drager

Hospital Alemão Oswaldo Cruz, São Paulo, Brazil

Álvaro Avezum

São Paulo University, São Paulo, Brazil

McMaster University, Hamilton, Canada

World Heart Federation, Geneva, Switzerland

American Heart Association, Dallas, TX, US

Louise Morgan & Olivia Jones

Unidade de Hipertensão, Instituto do Coração (InCor), Hospital das Clínicas, University of São Paulo, São Paulo, Brazil

Luciano F. Drager & Luiz A. Bortolotto

Sociedade Brasileira de Hipertensão, São Paulo, Brazil

Luiz A. Bortolotto

Secretaria Municipal da Saúde, São Paulo, Brazil

Suely Miya S. Rollemberg, Marcia M. C. de Lima & Edmir P. R. Albuquerque

You can also search for this author in PubMed   Google Scholar

Contributions

MSP, JB, MS and YCB conceived and planned the manuscript, collected and analyzed the data. MSP, JS, PS, AA, LFD, LAB, SMSR, MMCL, EPRA, RVM, BA, MS, LM, OJ, TR, JB, YCB contributed to interpreting the results and writing the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Marta S. Palmeirim .

Ethics declarations

Competing interests.

The Novartis Foundation funded the Better Hearts Better Cities initiative and the current manuscript. The American Heart Association is funded by the Novartis Foundation to provide technical assistance and resources to support professional education, quality improvement, monitoring, and evaluation. The views expressed in this manuscript are those of the authors and do not represent the views of the American Heart Association. The Swiss Tropical and Public Health Institute, the Sociedade de Cardiologia do Estado de São Paulo (SOCESP), the Instituto Tellus and IQVIA receive funding from the Novartis Foundation to contribute to the implementation and/or evaluation of the Better Hearts Better Cities initiative. None of the authors has declared any additional conflict of interest.

Ethics approval and consent to participate

Local approval agreements for data collection within the Better Hearts Better Cities initiative has been obtained from the Municipal Health Secretariat of São Paulo (CEP-SMS; 5.311.415). Consent was obtained from all study participants.

Consent for publication

Not applicable.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1: Questionnaire used in the district of Itaquera.

Supplementary material 2: questionnaire used in the district of penha., supplementary material 3: shadowing of community health agent (cha) form., 12913_2024_10978_moesm4_esm.tif.

Supplementary Material 4: Figure 1. Patient care journey map with personas representing patients, community health agents, doctors, pharmacists and nurses. BP, blood pressure; CHA, community health agent; CFF, Conselho Federal de Farmácia (federal pharmacy council); CRF, Conselho Regional de Farmácia (regional pharmacy council); ISA, Inquérito de Saúde (health survey); PINAB, Práticas Integrais de Promoção da Saúde e Nutrição na Atenção Básica (comprehensive health and nutrition promotion practices in primary healthcare); SIGA, Sistema Integrado de Gestão de Assistência (integrated assistance management system); SO, health social organization; SVS/MS, Secretaria de Vigilância em Saúde/Ministério da Saúde (health surveillance secretariat/ministry of health); UBS, unidade básica de saúde (primary healthcare unit).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Palmeirim, M.S., Baxter, Y.C., Silveira, M. et al. Situational analysis of hypertension management at primary health care level in São Paulo, Brazil: population, healthcare professional and health system perspectives. BMC Health Serv Res 24 , 668 (2024). https://doi.org/10.1186/s12913-024-10978-1

Download citation

Received : 25 July 2023

Accepted : 10 April 2024

Published : 28 May 2024

DOI : https://doi.org/10.1186/s12913-024-10978-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Hypertension
• Cardiovascular diseases
• Situational analysis
• Primary health care
• Design thinking

BMC Health Services Research

ISSN: 1472-6963

• General enquiries: [email protected]