documentary research method

• Skip to main content
• Skip to primary sidebar
• Skip to footer
• QuestionPro

• Solutions Industries Gaming Automotive Sports and events Education Government Travel & Hospitality Financial Services Healthcare Cannabis Technology Use Case NPS+ Communities Audience Contactless surveys Mobile LivePolls Member Experience GDPR Positive People Science 360 Feedback Surveys
• Resources Blog eBooks Survey Templates Case Studies Training Help center

Documentary Research: What it is, methodology & free examples

Social scientists often conduct documentary research. Its primary use is to assess various documents in the interest of social or historical value. Researchers also conduct documentary research to study multiple documents surrounding events or individuals.

What is documentary research?

Documentary research is the research conducted through the use of official documents or personal documents as the source of information.

Documents can include anything from the following: 

• Directories
• Government statistical publications
• Gramophone records
• Photographs
• Computer files

The above may not fit the traditional bill of a “document”, but companies can use them towards documentary research since they contain information.

Documentary research is similar to content analysis, which involves studying existing information recorded in media, texts, and physical items. Here, data collection from people is not required to conduct research. Hence, this is a prime example of secondary research.

It is essential to consider the quality of the documents while using it as evidence on social relations and social meanings. Keep in mind that, unlike surveys and research interviews, the records are originally published/generated without keeping the purpose of research in mind. It is good practice to cross-verify documents against other similar documents before reaching a decision.

Documentary research examples

Bellow, we can find a few real-life examples of documentary research applied to companies’ daily events.

1. Social research studies

Although documentary research is not used extensively today, it is the go-to research method to conduct social research studies. For example, Karl Marx and Emile Durkheim used documentary research extensively for their research.

Karl Marx used documents like:

• Her Majesty Inspectors of Factories Reports
• Royal Commission
• Inland Revenue Reports

There’s also a record of his use of reports by the Medical Officer of the Privy Council, reports on children’s employment in factories, the Corn-laws, the Banking Acts, and Census Reports for Wales and England to name a few.

Durkheim, one of the founders of Sociology, wrote a book on suicide, which is recognized as the first modern example of a methodical and consistent use of documents for social research.

2. Archival inquiry

The field of sociology has a popular, longstanding tradition of documentary inquiry. Many historians refer to and rely on primary documents for their research. Historians give historical documents more emphasis while conducting research. Of course, as we evolve, virtual documents like emails will play a significant role in research activities conducted by these researchers.

3. Aesthetic interpretation

Documentary research is not limited to text only. Pictures, paintings, videos, audio files, monuments are also used to conduct research. Documentary researchers use these tools in addition to texts while studying social sciences. The use of these tools adds to the authenticity of the textual research, or may very well point out deviations in the findings.

This deviation suggests that investigators research more to draw accurate conclusions.

Documentary research methodology

Documentary research, if conducted thoroughly, can help develop a hypothesis or prove or disprove an existing theory. This of course depends on the methodology applied and the depth of research conducted. The researcher must conduct his/her own secondary research to analyze the contents before extracting it. They must handle the data scientifically.

Follow this four-step approach to control the quality of the content:

The authenticity of the documents

The credibility of the documents

Representativeness of the documents

The meaning derived from the documents

Let’s take a look at these in detail.

Authenticity implies whether the document’s origin is reliable, is the evidence genuine, are the intentions sincere, and what were the commitments to creating the document. The authenticity of the source is the primary criterion of documentary research.

Credibility means the subjective and objective components that make one believe the source of information and whether the data is free from distortion and error. The information must be trustworthy and must have some level of expertise.

Representativeness refers to whether the document represents a more extensive collection of the data point, and it is an aggregation of the topic being studied. That said, documents get distorted with time due to the inclusion of new factors, and a check has to be made to ensure the documents are representative.

Meaning means whether the findings are understandable and clear to be called evidence. The goal of examining documents is to understand its significance and meaning. Researchers must find out whether the document fits within the historical context or not.

Advantages of documentary study

Here are the advantages of the documentary research method:

• Data readily available: Data is readily available in various sources. You only need to know where to look and how to use it. The data is available in different forms, and harnessing it is the real challenge.
• Inexpensive and economical: The data for research is already collected and published in either print or other forms. The researcher does not need to spend money and time like they do to collect market research insights and gather data. They need to search for and compile the available data from different sources.
• Saves time: Conducting market research is time-consuming. Responses will not come in quickly as expected, and gathering global responses will take a huge amount of time. If you have all the reference documents available (or you know where to find them), research is relatively quick.
• Non-bias: Primary data collection tends to be biased. This bias depends on a lot of factors like the age of the respondents, the time they take the survey, their mentality while taking the survey, their gender, their feelings towards certain ideas, to name a few. The list goes on and on when it comes to surveying bias.
• A researcher is not necessary during data collection: The researcher doesn’t need to be present during data collection. It is practically impossible for the researcher to be present at every point of the data source, especially thinking about the various data sources.
• Useful for hypothesis: Use historical data to draw inferences of the current or future events. Conclusions can be drawn from the experience of past events and data available for them. 

Disadvantages of the method

Here are the disadvantages of the documentary research method:

• Limited data: Data is not always available, especially when you need to cross-verify a theory or strengthen your argument based on different forms of data.
• Inaccuracies: As the data is historical and published, there is almost no way of ascertaining if the data is accurate or not. 
• Incomplete documents: Often, documents can be incomplete, and there is no way of knowing if there are additional documents to refer to on the subject.
• Data out of context: The data that the researcher refers to may be out of context and may not be in line with the concept the researcher is trying to study. Its because the research goal is not thought of when creating the original data. Often, researchers have to make do with the available data at hand.

QuestionPro provides the best market research platform to uncover complex insights that can propel your business to the forefront of your industry.

START A FREE TRIAL

MORE LIKE THIS

Exploring Types of Correlation for Patterns and Relationship 

Jun 10, 2024

Life@QuestionPro: The Journey of Kristie Lawrence

Jun 7, 2024

How Can I Help You? — Tuesday CX Thoughts

Jun 5, 2024

Why Multilingual 360 Feedback Surveys Provide Better Insights

Jun 3, 2024

Other categories

• Academic Research
• Artificial Intelligence
• Assessments
• Brand Awareness
• Case Studies
• Communities
• Consumer Insights
• Customer effort score
• Customer Engagement
• Customer Experience
• Customer Loyalty
• Customer Research
• Customer Satisfaction
• Employee Benefits
• Employee Engagement
• Employee Retention
• Friday Five
• General Data Protection Regulation
• Insights Hub
• Life@QuestionPro
• Market Research
• Mobile diaries
• Mobile Surveys
• New Features
• Online Communities
• Question Types
• Questionnaire
• QuestionPro Products
• Release Notes
• Research Tools and Apps
• Revenue at Risk
• Survey Templates
• Training Tips
• Uncategorized
• Video Learning Series
• What’s Coming Up
• Workforce Intelligence

• Subscriber Services
• For Authors
• Publications
• Archaeology
• Art & Architecture
• Bilingual dictionaries
• Classical studies
• Encyclopedias
• English Dictionaries and Thesauri
• Language reference
• Linguistics
• Media studies
• Medicine and health
• Names studies
• Performing arts
• Science and technology
• Social sciences
• Society and culture
• Overview Pages
• Subject Reference
• English Dictionaries
• Bilingual Dictionaries

Recently viewed (0)

• Save Search
• Share This Facebook LinkedIn Twitter

Related Content

Related overviews.

life history

content analysis

More Like This

Show all results sharing these subjects:

documentary research

Quick reference.

Research that uses personal and official documents as a source material. Documents used by social scientists may include such things as newspapers, diaries, stamps, directories, handbills, maps, government statistical publications, photographs, paintings, gramophone records, tapes, and computer files.

The most important consideration in using documents is their quality as evidence on social meanings and social relations. Unlike survey questionnaires or interview transcripts, documents have generally been compiled for purposes other than research, and their value must be thoroughly assessed before they can be used. It has been suggested that documents must be assessed against four criteria: authenticity, credibility, representativeness, and meaning (see John Scott, A Matter of Record, 1990). The criterion of authenticity involves assessing documents for their soundness and authorship. Soundness refers to whether the document is complete and whether it is an original or a sound copy. Authorship concerns issues of forgery or fraud and matters of collective or institutional authorship. Authorship is assessed through both internal evidence on vocabulary and literary style and external evidence from chemical tests on paper and ink. The criterion of credibility concerns the sincerity and accuracy of a document. All documents are selective, as it is impossible to construct accounts independent of particular standpoints, but they can be more or less credible as accounts, depending on the motives from which a point of view is adopted and whether the account gives an accurate report from that standpoint. To assess the accuracy of a report it is necessary to look at the conditions under which it was compiled and, in particular, how close the author was to the events reported. The criterion of representativeness involves an assessment of the survival and availability of relevant documents. It is important to know whether the documents consulted are representative of all the relevant documents that once existed, and this depends upon what proportion of the relevant documents have been stored or retained and whether they are available for researchers to use. The availability of official documents may often be limited by considerations of confidentiality and official secrecy. The meaning of documents is the most important matter and arises at two levels. The first level is the literal understanding of a document, by which is meant its physical readability, whether it is in a language that can be read, and such issues as dating. Once this practical matter has been resolved the more fundamental interpretative meaning must be addressed. Interpretation is a hermeneutic task through which an appreciation of the social and cultural context and forms of discourse that structure a text is reached. This involves methods of textual analysis and content analysis.

A useful recent overview of documentary research can be found in Lindsay Prior's Using Documents in Social Research (2003). Scott's A Matter of Record covers the issues involved in handling a range of documents, while Ken Plummer's Documents of Life, 2 (2001) gives good coverage of personal documents. See also life-history; personal documents.

From:   documentary research   in  A Dictionary of Sociology »

Subjects: Social sciences — Sociology

Related content in Oxford Reference

Reference entries.

View all related items in Oxford Reference »

Search for: 'documentary research' in Oxford Reference »

• Oxford University Press

PRINTED FROM OXFORD REFERENCE (www.oxfordreference.com). (c) Copyright Oxford University Press, 2023. All Rights Reserved. Under the terms of the licence agreement, an individual user may print out a PDF of a single entry from a reference work in OR for personal use (for details see Privacy Policy and Legal Notice ).

date: 11 June 2024

• Cookie Policy
• Privacy Policy
• Legal Notice
• Accessibility
• [195.158.225.230]
• 195.158.225.230

Character limit 500 /500

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Health Policy Plan
• v.35(10); 2020 Dec

Document analysis in health policy research: the READ approach

Sarah l dalglish.

1 Department of International Health, Johns Hopkins School of Public Health, 615 N. Wolfe St, Baltimore, MD 21205, USA

2 Institute for Global Health, University College London, Institute for Global Health 3rd floor, 30 Guilford Street, London WC1N 1EH, UK

Hina Khalid

3 School of Humanities and Social Sciences, Information Technology University, Arfa Software Technology Park, Ferozepur Road, Lahore 54000, Pakistan

Shannon A McMahon

4 Heidelberg Institute of Global Health, Medical Faculty and University Hospital, University of Heidelberg, Im Neuenheimer Feld 130/3, 69120 Heidelberg, Germany

Associated Data

Document analysis is one of the most commonly used and powerful methods in health policy research. While existing qualitative research manuals offer direction for conducting document analysis, there has been little specific discussion about how to use this method to understand and analyse health policy. Drawing on guidance from other disciplines and our own research experience, we present a systematic approach for document analysis in health policy research called the READ approach: (1) ready your materials, (2) extract data, (3) analyse data and (4) distil your findings. We provide practical advice on each step, with consideration of epistemological and theoretical issues such as the socially constructed nature of documents and their role in modern bureaucracies. We provide examples of document analysis from two case studies from our work in Pakistan and Niger in which documents provided critical insight and advanced empirical and theoretical understanding of a health policy issue. Coding tools for each case study are included as Supplementary Files to inspire and guide future research. These case studies illustrate the value of rigorous document analysis to understand policy content and processes and discourse around policy, in ways that are either not possible using other methods, or greatly enrich other methods such as in-depth interviews and observation. Given the central nature of documents to health policy research and importance of reading them critically, the READ approach provides practical guidance on gaining the most out of documents and ensuring rigour in document analysis.

Key Messages

• Rigour in qualitative research is judged partly by the use of deliberate, systematic procedures; however, little specific guidance is available for analysing documents, a nonetheless common method in health policy research.
• Document analysis is useful for understanding policy content across time and geographies, documenting processes, triangulating with interviews and other sources of data, understanding how information and ideas are presented formally, and understanding issue framing, among other purposes.
• The READ (Ready materials, Extract data, Analyse data, Distil) approach provides a step-by-step guide to conducting document analysis for qualitative policy research.
• The READ approach can be adapted to different purposes and types of research, two examples of which are presented in this article, with sample tools in the Supplementary Materials .

Introduction

Document analysis (also called document review) is one of the most commonly used methods in health policy research; it is nearly impossible to conduct policy research without it. Writing in early 20th century, Weber (2015) identified the importance of formal, written documents as a key characteristic of the bureaucracies by which modern societies function, including in public health. Accordingly, critical social research has a long tradition of documentary review: Marx analysed official reports, laws, statues, census reports and newspapers and periodicals over a nearly 50-year period to come to his world-altering conclusions ( Harvey, 1990 ). Yet in much of social science research, ‘documents are placed at the margins of consideration,’ with privilege given to the spoken word via methods such as interviews, possibly due to the fact that many qualitative methods were developed in the anthropological tradition to study mainly pre-literate societies ( Prior, 2003 ). To date, little specific guidance is available to help health policy researchers make the most of these wells of information.

The term ‘documents’ is defined here broadly, following Prior, as physical or virtual artefacts designed by creators, for users, to function within a particular setting ( Prior, 2003 ). Documents exist not as standalone objects of study but must be understood in the social web of meaning within which they are produced and consumed. For example, some analysts distinguish between public documents (produced in the context of public sector activities), private documents (from business and civil society) and personal documents (created by or for individuals, and generally not meant for public consumption) ( Mogalakwe, 2009 ). Documents can be used in a number of ways throughout the research process ( Bowen, 2009 ). In the planning or study design phase, they can be used to gather background information and help refine the research question. Documents can also be used to spark ideas for disseminating research once it is complete, by observing the ways those who will use the research speak to and communicate ideas with one another.

Documents can also be used during data collection and analysis to help answer research questions. Recent health policy research shows that this can be done in at least four ways. Frequently, policy documents are reviewed to describe the content or categorize the approaches to specific health problems in existing policies, as in reviews of the composition of drowning prevention resources in the United States or policy responses to foetal alcohol spectrum disorder in South Africa ( Katchmarchi et al. , 2018 ; Adebiyi et al. , 2019 ). In other cases, non-policy documents are used to examine the implementation of health policies in real-world settings, as in a review of web sources and newspapers analysing the functioning of community health councils in New Zealand ( Gurung et al. , 2020 ). Perhaps less frequently, document analysis is used to analyse policy processes, as in an assessment of multi-sectoral planning process for nutrition in Burkina Faso ( Ouedraogo et al. , 2020 ). Finally, and most broadly, document analysis can be used to inform new policies, as in one study that assessed cigarette sticks as communication and branding ‘documents,’ to suggest avenues for further regulation and tobacco control activities ( Smith et al. , 2017 ).

This practice paper provides an overarching method for conducting document analysis, which can be adapted to a multitude of research questions and topics. Document analysis is used in most or all policy studies; the aim of this article is to provide a systematized method that will enhance procedural rigour. We provide an overview of document analysis, drawing on guidance from disciplines adjacent to public health, introduce the ‘READ’ approach to document analysis and provide two short case studies demonstrating how document analysis can be applied.

What is document analysis?

Document analysis is a systematic procedure for reviewing or evaluating documents, which can be used to provide context, generate questions, supplement other types of research data, track change over time and corroborate other sources ( Bowen, 2009 ). In one commonly cited approach in social research, Bowen recommends first skimming the documents to get an overview, then reading to identify relevant categories of analysis for the overall set of documents and finally interpreting the body of documents ( Bowen, 2009 ). Document analysis can include both quantitative and qualitative components: the approach presented here can be used with either set of methods, but we emphasize qualitative ones, which are more adapted to the socially constructed meaning-making inherent to collaborative exercises such as policymaking.

The study of documents as a research method is common to a number of social science disciplines—yet in many of these fields, including sociology ( Mogalakwe, 2009 ), anthropology ( Prior, 2003 ) and political science ( Wesley, 2010 ), document-based research is described as ill-considered and underutilized. Unsurprisingly, textual analysis is perhaps most developed in fields such as media studies, cultural studies and literary theory, all disciplines that recognize documents as ‘social facts’ that are created, consumed, shared and utilized in socially organized ways ( Atkinson and Coffey, 1997 ). Documents exist within social ‘fields of action,’ a term used to designate the environments within which individuals and groups interact. Documents are therefore not mere records of social life, but integral parts of it—and indeed can become agents in their own right ( Prior, 2003 ). Powerful entities also manipulate the nature and content of knowledge; therefore, gaps in available information must be understood as reflecting and potentially reinforcing societal power relations ( Bryman and Burgess, 1994 ).

Document analysis, like any research method, can be subject to concerns regarding validity, reliability, authenticity, motivated authorship, lack of representativity and so on. However, these can be mitigated or avoided using standard techniques to enhance qualitative rigour, such as triangulation (within documents and across methods and theoretical perspectives), ensuring adequate sample size or ‘engagement’ with the documents, member checking, peer debriefing and so on ( Maxwell, 2005 ).

Document analysis can be used as a standalone method, e.g. to analyse the contents of specific types of policy as they evolve over time and differ across geographies, but document analysis can also be powerfully combined with other types of methods to cross-validate (i.e. triangulate) and deepen the value of concurrent methods. As one guide to public policy research puts it, ‘almost all likely sources of information, data, and ideas fall into two general types: documents and people’ ( Bardach and Patashnik, 2015 ). Thus, researchers can ask interviewees to address questions that arise from policy documents and point the way to useful new documents. Bardach and Patashnik suggest alternating between documents and interviews as sources as information, as one tends to lead to the other, such as by scanning interviewees’ bookshelves and papers for titles and author names ( Bardach and Patashnik, 2015 ). Depending on your research questions, document analysis can be used in combination with different types of interviews ( Berner-Rodoreda et al. , 2018 ), observation ( Harvey, 2018 ), and quantitative analyses, among other common methods in policy research.

The READ approach

The READ approach to document analysis is a systematic procedure for collecting documents and gaining information from them in the context of health policy studies at any level (global, national, local, etc.). The steps consist of: (1) ready your materials, (2) extract data, (3) analyse data and (4) distil your findings. We describe each of these steps in turn.

Step 1. Ready your materials

At the outset, researchers must set parameters in terms of the nature and number (approximately) of documents they plan to analyse, based on the research question. How much time will you allocate to the document analysis, and what is the scope of your research question? Depending on the answers to these questions, criteria should be established around (1) the topic (a particular policy, programme, or health issue, narrowly defined according to the research question); (2) dates of inclusion (whether taking the long view of several decades, or zooming in on a specific event or period in time); and (3) an indicative list of places to search for documents (possibilities include databases such as Ministry archives; LexisNexis or other databases; online searches; and particularly interview subjects). For difficult-to-obtain working documents or otherwise non-public items, bringing a flash drive to interviews is one of the best ways to gain access to valuable documents.

For research focusing on a single policy or programme, you may review only a handful of documents. However, if you are looking at multiple policies, health issues, or contexts, or reviewing shorter documents (such as newspaper articles), you may look at hundreds, or even thousands of documents. When considering the number of documents you will analyse, you should make notes on the type of information you plan to extract from documents—i.e. what it is you hope to learn, and how this will help answer your research question(s). The initial criteria—and the data you seek to extract from documents—will likely evolve over the course of the research, as it becomes clear whether they will yield too few documents and information (a rare outcome), far too many documents and too much information (a much more common outcome) or documents that fail to address the research question; however, it is important to have a starting point to guide the search. If you find that the documents you need are unavailable, you may need to reassess your research questions or consider other methods of inquiry. If you have too many documents, you can either analyse a subset of these ( Panel 1 ) or adopt more stringent inclusion criteria.

Exploring the framing of diseases in Pakistani media

In Table 1 , we present a non-exhaustive list of the types of documents that can be included in document analyses of health policy issues. In most cases, this will mean written sources (policies, reports, articles). The types of documents to be analysed will vary by study and according to the research question, although in many cases, it will be useful to consult a mix of formal documents (such as official policies, laws or strategies), ‘gray literature’ (organizational materials such as reports, evaluations and white papers produced outside formal publication channels) and, whenever possible, informal or working documents (such as meeting notes, PowerPoint presentations and memoranda). These latter in particular can provide rich veins of insight into how policy actors are thinking through the issues under study, particularly for the lucky researcher who obtains working documents with ‘Track Changes.’ How you prioritize documents will depend on your research question: you may prioritize official policy documents if you are studying policy content, or you may prioritize informal documents if you are studying policy process.

Types of documents that can be consulted in studies of health policy

During this initial preparatory phase, we also recommend devising a file-naming system for your documents (e.g. Author.Date.Topic.Institution.PDF), so that documents can be easily retrieved throughout the research process. After extracting data and processing your documents the first time around, you will likely have additional ‘questions’ to ask your documents and need to consult them again. For this reason, it is important to clearly name source files and link filenames to the data that you are extracting (see sample naming conventions in the Supplementary Materials ).

Step 2. Extract data

Data can be extracted in a number of ways, and the method you select for doing so will depend on your research question and the nature of your documents. One simple way is to use an Excel spreadsheet where each row is a document and each column is a category of information you are seeking to extract, from more basic data such as the document title, author and date, to theoretical or conceptual categories deriving from your research question, operating theory or analytical framework (Panel 2). Documents can also be imported into thematic coding software such as Atlas.ti or NVivo, and data extracted that way. Alternatively, if the research question focuses on process, documents can be used to compile a timeline of events, to trace processes across time. Ask yourself, how can I organize these data in the most coherent manner? What are my priority categories? We have included two different examples of data extraction tools in the Supplementary Materials to this article to spark ideas.

Case study Documents tell part of the story in Niger

Document analyses are first and foremost exercises in close reading: documents should be read thoroughly, from start to finish, including annexes, which may seem tedious but which sometimes produce golden nuggets of information. Read for overall meaning as you extract specific data related to your research question. As you go along, you will begin to have ideas or build working theories about what you are learning and observing in the data. We suggest capturing these emerging theories in extended notes or ‘memos,’ as used in Grounded Theory methodology ( Charmaz, 2006 ); these can be useful analytical units in themselves and can also provide a basis for later report and article writing.

As you read more documents, you may find that your data extraction tool needs to be modified to capture all the relevant information (or to avoid wasting time capturing irrelevant information). This may require you to go back and seek information in documents you have already read and processed, which will be greatly facilitated by a coherent file-naming system. It is also useful to keep notes on other documents that are mentioned that should be tracked down (sometimes you can write the author for help). As a general rule, we suggest being parsimonious when selecting initial categories to extract from data. Simply reading the documents takes significant time in and of itself—make sure you think about how, exactly, the specific data you are extracting will be used and how it goes towards answering your research questions.

Step 3. Analyse data

As in all types of qualitative research, data collection and analysis are iterative and characterized by emergent design, meaning that developing findings continually inform whether and how to obtain and interpret data ( Creswell, 2013 ). In practice, this means that during the data extraction phase, the researcher is already analysing data and forming initial theories—as well as potentially modifying document selection criteria. However, only when data extraction is complete can one see the full picture. For example, are there any documents that you would have expected to find, but did not? Why do you think they might be missing? Are there temporal trends (i.e. similarities, differences or evolutions that stand out when documents are ordered chronologically)? What else do you notice? We provide a list of overarching questions you should think about when viewing your body of document as a whole ( Table 2 ).

Questions to ask your overall body of documents

HIV and viral hepatitis articles by main frames (%). Note: The percentage of articles is calculated by dividing the number of articles appearing in each frame for viral hepatitis and HIV by the respectivenumber of sampled articles for each disease (N = 137 for HIV; N = 117 for hepatitis). Time frame: 1 January 2006 to 30 September 2016

Representations of progress toward Millennium Development Goal 4 in Nigerien policy documents. Sources: clockwise from upper left: ( WHO 2006 ); ( Institut National de la Statistique 2010 ); ( Ministè re de la Santé Publique 2010 ); ( Unicef 2010 )

In addition to the meaning-making processes you are already engaged in during the data extraction process, in most cases, it will be useful to apply specific analysis methodologies to the overall corpus of your documents, such as policy analysis ( Buse et al. , 2005 ). An array of analysis methodologies can be used, both quantitative and qualitative, including case study methodology, thematic content analysis, discourse analysis, framework analysis and process tracing, which may require differing levels of familiarity and skills to apply (we highlight a few of these in the case studies below). Analysis can also be structured according to theoretical approaches. When it comes to analysing policies, process tracing can be particularly useful to combine multiple sources of information, establish a chronicle of events and reveal political and social processes, so as to create a narrative of the policy cycle ( Yin, 1994 ; Shiffman et al. , 2004 ). Practically, you will also want to take a holistic view of the documents’ ‘answers’ to the questions or analysis categories you applied during the data extraction phase. Overall, what did the documents ‘say’ about these thematic categories? What variation did you find within and between documents, and along which axes? Answers to these questions are best recorded by developing notes or memos, which again will come in handy as you write up your results.

As with all qualitative research, you will want to consider your own positionality towards the documents (and their sources and authors); it may be helpful to keep a ‘reflexivity’ memo documenting how your personal characteristics or pre-standing views might influence your analysis ( Watt, 2007 ).

Step 4. Distil your findings

You will know when you have completed your document review when one of the three things happens: (1) completeness (you feel satisfied you have obtained every document fitting your criteria—this is rare), (2) out of time (this means you should have used more specific criteria), and (3) saturation (you fully or sufficiently understand the phenomenon you are studying). In all cases, you should strive to make the third situation the reason for ending your document review, though this will not always mean you will have read and analysed every document fitting your criteria—just enough documents to feel confident you have found good answers to your research questions.

Now it is time to refine your findings. During the extraction phase, you did the equivalent of walking along the beach, noticing the beautiful shells, driftwood and sea glass, and picking them up along the way. During the analysis phase, you started sorting these items into different buckets (your analysis categories) and building increasingly detailed collections. Now you have returned home from the beach, and it is time to clean your objects, rinse them of sand and preserve only the best specimens for presentation. To do this, you can return to your memos, refine them, illustrate them with graphics and quotes and fill in any incomplete areas. It can also be illuminating to look across different strands of work: e.g. how did the content, style, authorship, or tone of arguments evolve over time? Can you illustrate which words, concepts or phrases were used by authors or author groups?

Results will often first be grouped by theoretical or analytic category, or presented as a policy narrative, interweaving strands from other methods you may have used (interviews, observation, etc.). It can also be helpful to create conceptual charts and graphs, especially as this corresponds to your analytical framework (Panels 1 and 2). If you have been keeping a timeline of events, you can seek out any missing information from other sources. Finally, ask yourself how the validity of your findings checks against what you have learned using other methods. The final products of the distillation process will vary by research study, but they will invariably allow you to state your findings relative to your research questions and to draw policy-relevant conclusions.

Document analysis is an essential component of health policy research—it is also relatively convenient and can be low cost. Using an organized system of analysis enhances the document analysis’s procedural rigour, allows for a fuller understanding of policy process and content and enhances the effectiveness of other methods such as interviews and non-participant observation. We propose the READ approach as a systematic method for interrogating documents and extracting study-relevant data that is flexible enough to accommodate many types of research questions. We hope that this article encourages discussion about how to make best use of data from documents when researching health policy questions.

Supplementary Data

Supplementary data are available at Health Policy and Planning online.

Supplementary Material

Czaa064_supplementary_data, acknowledgements.

The data extraction tool in the Supplementary Materials for the iCCM case study (Panel 2) was conceived of by the research team for the multi-country study ‘Policy Analysis of Community Case Management for Childhood and Newborn Illnesses’. The authors thank Sara Bennett and Daniela Rodriguez for granting permission to publish this tool. S.M. was supported by The Olympia-Morata-Programme of Heidelberg University. The funders had no role in the decision to publish, or preparation of the manuscript. The content is the responsibility of the authors and does not necessarily represent the views of any funder.

Conflict of interest statement . None declared.

Ethical approval. No ethical approval was required for this study.

• Abdelmutti N, Hoffman-Goetz L.  2009. Risk messages about HPV, cervical cancer, and the HPV vaccine Gardasil: a content analysis of Canadian and U.S. national newspaper articles . Women & Health   49 : 422–40. [ PubMed ] [ Google Scholar ]
• Adebiyi BO, Mukumbang FC, Beytell A-M.  2019. To what extent is fetal alcohol spectrum disorder considered in policy-related documents in South Africa? A document review . Health Research Policy and Systems   17 : [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Atkinson PA, Coffey A.  1997. Analysing documentary realities In: Silverman D (ed). Qualitative Research: Theory, Method and Practice . London: SAGE. [ Google Scholar ]
• Bardach E, Patashnik EM.  2015. Practical Guide for Policy Analysis: The Eightfold Path to More Effective Problem Solving . Los Angeles: SAGE. [ Google Scholar ]
• Bennett S, Dalglish SL, Juma PA, Rodríguez DC.  2015. Altogether now… understanding the role of international organizations in iCCM policy transfer . Health Policy and Planning   30 : ii26–35. [ PubMed ] [ Google Scholar ]
• Berner-Rodoreda A, Bärnighausen T, Kennedy C  et al.  2018. From doxastic to epistemic: a typology and critique of qualitative interview styles . Qualitative Inquiry   26 : 291–305. 1077800418810724. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Bowen GA.  2009. Document analysis as a qualitative research method . Qualitative Research Journal   9 : 27–40. [ Google Scholar ]
• Bryman A.  1994. Analyzing Qualitative Data .
• Buse K, Mays N, Walt G.  2005. Making Health Policy . New York: Open University Press. [ Google Scholar ]
• Charmaz K.  2006. Constructing Grounded Theory: A Practical Guide through Qualitative Analysis . London: SAGE. [ PubMed ] [ Google Scholar ]
• Claassen L, Smid T, Woudenberg F, Timmermans DRM.  2012. Media coverage on electromagnetic fields and health: content analysis of Dutch newspaper articles and websites . Health, Risk & Society   14 : 681–96. [ Google Scholar ]
• Creswell JW.  2013. Qualitative Inquiry and Research Design . Thousand Oaks, CA: SAGE. [ Google Scholar ]
• Dalglish SL, Rodríguez DC, Harouna A, Surkan PJ.  2017. Knowledge and power in policy-making for child survival in Niger . Social Science & Medicine   177 : 150–7. [ PubMed ] [ Google Scholar ]
• Dalglish SL, Surkan PJ, Diarra A, Harouna A, Bennett S.  2015. Power and pro-poor policies: the case of iCCM in Niger . Health Policy and Planning   30 : ii84–94. [ PubMed ] [ Google Scholar ]
• Entman RM.  1993. Framing: toward clarification of a fractured paradigm . Journal of Communication   43 : 51–8. [ Google Scholar ]
• Fournier G, Djermakoye IA.  1975. Village health teams in Niger (Maradi Department) In: Newell KW (ed). Health by the People . Geneva: WHO. [ Google Scholar ]
• Gurung G, Derrett S, Gauld R.  2020. The role and functions of community health councils in New Zealand’s health system: a document analysis . The New Zealand Medical Journal   133 : 70–82. [ PubMed ] [ Google Scholar ]
• Harvey L.  1990. Critical Social Research . London: Unwin Hyman. [ Google Scholar ]
• Harvey SA.  2018. Observe before you leap: why observation provides critical insights for formative research and intervention design that you’ll never get from focus groups, interviews, or KAP surveys . Global Health: Science and Practice   6 : 299–316. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Institut National de la Statistique. 2010. Rapport National sur les Progrès vers l'atteinte des Objectifs du Millénaire pour le Développement. Niamey, Niger: INS. [ Google Scholar ]
• Kamarulzaman A.  2013. Fighting the HIV epidemic in the Islamic world . Lancet   381 : 2058–60. [ PubMed ] [ Google Scholar ]
• Katchmarchi AB, Taliaferro AR, Kipfer HJ.  2018. A document analysis of drowning prevention education resources in the United States . International Journal of Injury Control and Safety Promotion   25 : 78–84. [ PubMed ] [ Google Scholar ]
• Krippendorff K.  2004. Content Analysis: An Introduction to Its Methodology . SAGE. [ Google Scholar ]
• Marten R.  2019. How states exerted power to create the Millennium Development Goals and how this shaped the global health agenda: lessons for the sustainable development goals and the future of global health . Global Public Health   14 : 584–99. [ PubMed ] [ Google Scholar ]
• Maxwell JA.  2005. Qualitative Research Design: An Interactive Approach , 2 nd edn. Thousand Oaks, CA: Sage Publications. [ Google Scholar ]
• Mayring P.  2004. Qualitative Content Analysis . In: Flick U, von Kardorff E, Steinke I (eds).   A Companion to Qualitative Research SAGE. [ Google Scholar ]
• Ministère de la Santé Publique. 2010. Enquête nationale sur la survie des enfants de 0 à 59 mois et la mortalité au Niger 2010. Niamey, Niger: MSP. [ Google Scholar ]
• Mogalakwe M.  2009. The documentary research method—using documentary sources in social research . Eastern Africa Social Science Research Review   25 : 43–58. [ Google Scholar ]
• Nelkin D.  1991. AIDS and the news media . The Milbank Quarterly   69 : 293–307. [ PubMed ] [ Google Scholar ]
• Ouedraogo O, Doudou MH, Drabo KM  et al.  2020. Policy overview of the multisectoral nutrition planning process: the progress, challenges, and lessons learned from Burkina Faso . The International Journal of Health Planning and Management   35 : 120–39. [ PubMed ] [ Google Scholar ]
• Prior L.  2003. Using Documents in Social Research . London: SAGE. [ Google Scholar ]
• Shiffman J, Stanton C, Salazar AP.  2004. The emergence of political priority for safe motherhood in Honduras . Health Policy and Planning   19 : 380–90. [ PubMed ] [ Google Scholar ]
• Smith KC, Washington C, Welding K  et al.  2017. Cigarette stick as valuable communicative real estate: a content analysis of cigarettes from 14 low-income and middle-income countries . Tobacco Control   26 : 604–7. [ PMC free article ] [ PubMed ] [ Google Scholar ]
• Strömbäck J, Dimitrova DV.  2011. Mediatization and media interventionism: a comparative analysis of Sweden and the United States . The International Journal of Press/Politics   16 : 30–49. [ Google Scholar ]
• UNICEF. 2010. Maternal, Newborn & Child Surival Profile. Niamey, Niger: UNICEF . [ Google Scholar ]
• Watt D.  2007. On becoming a qualitative researcher: the value of reflexivity . Qualitative Report   12 : 82–101. [ Google Scholar ]
• Weber M.  2015. Bureaucracy In: Waters T, Waters D (eds). Rationalism and Modern Society: New Translations on Politics, Bureaucracy, and Social Stratification . London: Palgrave MacMillan. [ Google Scholar ]
• Wesley JJ.  2010. Qualitative Document Analysis in Political Science.
• World Health Organization. 2006. Country Health System Fact Sheet 2006: Niger. Niamey, Niger: WHO.
• Yin R.  1994. Case Study Research: Design and Methods . Thousand Oaks, CA: Sage. [ Google Scholar ]

Published by

Follow this blog

Documentary analysis: the research method that can level the playing field

by Aimee Grant 22nd March 2022

Whenever I say that I ‘do documentary analysis’ or ‘research with documents’ I tend to get a confused look. I think this is because unlike ‘doing interviews’, it’s not clear what documentary analysis means! So, let me start out by telling you a little bit about documentary analysis, and then I’ll move on to my experience of doing documentary analysis and explain why I think it’s a fantastic leveller for students and researchers with a wide variety of needs.

At its most basic, documentary analysis is research which involves documents as the data. Going back to the ‘doing interviews’ analogy, instead of talking to a person to find out about their views or experiences, we would find and then analyse things that people have written. This also allows us to understand something about their views and experiences of the thing they’re writing about. The type of data that are used for documentary analysis aren’t only things that are in hard copy; these days we also look at the things people are creating online. We also don’t restrict ourselves to words but also include images in our analysis, because sometimes images are used to imply things that are so offensive they could not be ‘said out loud’ ( Stuart Hall talks about this ). Examples of documents include images like adverts on the side of buses (and anywhere else), packaging of products we use and buy, passports, meeting notes, letters, diaries and anything else with words or images on it.

Moving on to analysis and there are dozens and dozens of qualitative analysis techniques. What may be surprising to know is that there is not a single analysis technique called ‘documentary analysis’. This is something that is definitely not as clear as it could be. Instead, returning to our comparison with interviews, any analysis technique can be used to consider the data using both methods. This includes things like thematic analysis, discourse analysis and content analysis. Detailed examples of documentary analysis using each of these analysis methods can be found in my first book, Doing Excellent Social Research with Documents .

My experiences of using the method stem from my research using documents as data as part of my undergraduate dissertation (looking at how UK Housing Benefit procedures contribute to homelessness). As part of my masters (examining an example of a policy failure) and PhD (on lived experiences of welfare reform for disabled people and staff ), I used this method again, alongside other qualitative methods. To be honest, at that time I used the documents as ‘second-class’ data, placing more emphasis on the interview and observational data that I undertook. I had greater knowledge of how to handle this data, which I supplemented and triangulated with the available documents. Following my PhD, however, I moved out of academia. As a disabled academic awaiting multiple surgeries, I did not have the luxury of taking a job that required me to move out of my local NHS trust area, which severely limited my academic job opportunities.

That said, I definitely didn’t feel that I was ‘done’ with being an academic, and so I started to look for ways that I could continue my research. One major issue was that without being part of a university, I did not have access to an ethics committee. This led me back to documents – many documentary analysis studies don’t require ethical approval. The exception to this in general seems to be where you would need to log in to a website to access the content you wish to analyse. That said, as researchers, we need to be careful to consider the ethical implications of using data when ethical review isn’t required (see chapter 4 of my new book, Doing your Research Project with Documents ).

Another barrier was that I worked full time 9-5 in a charity, and then for the NHS. Using documents as research data allows you to do the research whenever or wherever you want to, unlike for example, needing to interview somebody. I remember sitting on a Megabus on the way to visit friends in London, for example, while coding newspaper articles relating to treatment targets (like the UK four-hour waiting time) in A&E departments . Likewise, when my day job looked like it wasn’t going to produce any journal articles for a while, I was able to do an analysis of comments about breastfeeding in public on the Mail Online website (the online version of the Daily Mail ) and also compare it to content relating to the same incident on Twitter.

Documentary analysis also allowed me to apply for small pots of funding, and to know that there would be a piece of research which could be published from it; for example this research involved an analysis of Twitter content relating to waterpipe smoking , with Dr Hannah O’Mahoney working for me for two months. Had we needed to interview people, or conduct observations, those two months would likely have been taken up with data collection alone, which was an issue that Dr Dawn Mannay and I found with interview-based projects that used visual methods; the data was brilliant, but it was challenging to do these pieces of work with tiny pots of seed funding.

To conclude, documentary analysis is a flexible method which can be used with and without theory and using a wide range of analysis approaches. It has many advantages for researchers and students alike who are disabled, working alongside their studies or research careers, or who have caring responsibilities. I would be surprised if there were a single topic in the world that could not be at least partially understood through critically reading things that have been written about it. If you are carrying out your first research project using documents, or feel like you are not confident with the methodology, you may wish to read my new book, published by Policy Press, Doing your Research Project with Documents .

Aimee Grant is a qualitative researcher with a long-standing interest in documentary analysis. She is based at Swansea University’s Centre for Lactation, Infant Feeding and Translational Research.

Find out more about impact, influence and engagement at Policy Press  here . Policy Press newsletter subscribers receive a 25% discount – sign up here . The views and opinions expressed on this blog site are solely those of the original blog post authors and other contributors. These views and opinions do not necessarily represent those of the Policy Press and/or any/ all contributors to this site.

Image Credit: Chiara F  on  Unsplash

Related posts

Why we need to think critically about social policy, why does public sexual harassment matter, how the far-right impacted activism in the brazilian amazon, the risks of care: justice, family carers and the pandemic, a time for radical empathy.

T h e   N e w   S c h o o l

Course catalog, documentary research methods, schools of public engagement: media.

Credits : 3

This course is a non-production course in documentary research methodology. With a focus on the American documentary tradition, this course introduces the theoretical and practical foundations necessary for conducting research in the context of designing and developing documentary film content. We will examine award-winning documentary films as models, and will look at their process, their choices, and the emotional impact of these decisions in an attempt to understand the various stages of thought and of research that went in to creating the final film as a piece of art. The readings represent a diversity of approaches to qualitative research, and raise questions about what constitutes "good" research; and as such, what adds to our knowledge of the social world. We will ask such questions as: What is a good documentary subject? Why should we do documentary research, and how does doing it implicate us in a complex web of issues concerning the politics of representation and the social construction of meaning? Methodologies examined include print, photo and archival research; observation and direct cinema; interviews and oral histories; and participant observation; among others. Students will create a literature and film review on their own documentary subject, and will design, research, and write up their own documentary proposal and treatment.

College : Schools of Public Engagement (NS)

Department : Media (MED)

Campus : New York City (GV)

Course Format : Seminar (R)

Modality : Online - Synchronous

Max Enrollment : 18

Add/Drop Deadline : September 9, 2024 (Monday)

Online Withdrawal Deadline : November 17, 2024 (Sunday)

Seats Available : Yes

Status : Open *

* Status information is updated every few minutes. The status of this course may have changed since the last update. Open seats may have restrictions that will prevent some students from registering. Updated: 11:44pm EDT 6/10/2024

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

The Documentary Research Method – Using Documentary Sources in Social Research

2009, Eastern Africa Social Science Research Review

Related Papers

DR. JASHIM UDDIN AHMED

Frank Ezuruike

Wivian Weller

I International Conference on Media Ethics – Proceedings (pp. 223-234)

Willemien Sanders

Documentary scholars have presupposed a certain documentary practice, and have ethically evaluated this practice, with a focus on the need for protection for a vulnerable and ignorant participant. But times have changed. By researching the experience of ethical challenges by both documentary filmmakers and participants, through both a quantitative and a qualitative project, a practice surfaces in which the participant is committed to and involved in the project; conflict between filmmaker and participant is recurrent, but non-decisive, as communication, trust, and commitment act as a safety-net. Hence, questions about documentary ethics need to address this contemporary practice, rather than a past one.

Mass Communication and Society

Matthew C . Nisbet

Recent films such as Fahrenheit 9/11, Supersize Me, An Inconvenient Truth, Food Inc., and Sicko have generated attention to how documentaries can shape debates over social issues and policy questions. Documentaries are no longer conventionally perceived as a passive experience intended solely for informal learning or entertainment. Instead, with ever increasing frequency, these films are considered part of a larger effort to spark debate, mold public opinion, shape policy, and build activist networks. In addition to these new forms ...

Steph M. Anderson , Carolina Muñoz Proto

In recent years, psychologists have begun to use video more frequently in qualitative research, in particular, within research on social justice. The non-confidential nature inherent in video, however, raises new ethical challenges for the field of psychology to address. Building upon a growing literature on video-based research, in this article, we use an illustrative case study to examine how researchers' sense of ethical responsibility can find guidance from, clash against, or fill gaps left by extant federal and disciplinary ethical requirements. We focus specifically on issues of confidentiality and representation, highlighting the challenges and possibilities that video creates in relation to participants' power, dignity, and participation and arguing that psychologists must systematically engage questions about ethical responsibilities throughout the design and implementation phases of a research project. In doing so, psychologists, their community partners, and students will be better able to articulate and problematize their assumptions and intentions regarding video work.

Palgrave Macmillan

Nasir Uddin

The methodology of social research has been transformed along with the metamorphosis of the socioeconomic and political rhetoric and the reality across the globe. Because of its various experience and experiments depending on the subjects, contexts and regions, social research methodology itself has become an issue of research in the twenty-first century. The fieldwork challenges vary from discipline to discipline and from region to region because every discipline in social sciences, despite having some identical broader frameworks, has its distinctive tone of interpretation and every region has its own social, political, historical and ecological settings. Social research involves the researcher's engagement, interpersonal attachment, and impersonal affection in dealing with persons; the object of study (See, Uddin, 2011). Since social research and its methodologies tend to change and evolve from time to time, new and newer fieldwork experiences are needed to reformulate the model and module of social research methods and methodologies. Hence, a new form of methodology could be framed that reflects the researcher's personal, political, social, cultural and emotional holdings being developed and formed throughout the researcher's socialization and socio-cultural upbringing across time and space (See, Uddin, 2022). Consequently, social research is not simply a typical 'academic practice' that follows an innocent 'professional genre' and 'research rituals'; rather it heavily entangles with the researcher's lived experiences.

National Centre for Research Methods

Michael Mair

On the 21st and 22nd November at the University of Liverpool in London, NCRM held an innovation forum to explore the embryonic field of “investigative social research” and the methods that underpin it. A dynamic and frequently high impact contemporary field, investigative social research encompasses work by non-governmental organisation/civil society researchers, data and investigative journalists, open source investigators, lawyers and independent researchers alongside social scientists of all kinds, from anthropologists, criminologists, epidemiologists, geographers, historians and sociologists through to those involved in accounting, economics and financial studies as well as data science. This emerging global field is characterised by the breadth of output it produces: often fast-circulating studies, news stories, reports, trackers and apps which attract global public attention. Researchers in the field make heavy use of: “new data technologies and analytics and other means of intellectual cross pollination, exchanging ideas and sometimes working and writing together, side by side, across borders, and genres each of them with different perspectives, backgrounds, interests, professional expertise, not to mention internationally and culturally diverse geographic and economic circumstances” (Lewis 2018: 23). In this context, investigative researchers are developing approaches which, as Ruppert and Savage observe, “engage with new forms of data and analytic techniques, undertake rich empirical analysis as well as develop new resources for understanding [the world and what happens in it]” (Ruppert and Savage 2009: 17). In so doing, researchers are contributing to the development of distinctive new “ways of knowing” (ibid.). No longer the preserve of universities and academic disciplines, this opening up and reworking of “the methods and practices that researchers and analysts use to make sense of data” and do useful things with it (Arribas-Bel & Reades 2018: 5) is happening across disciplinary, sectoral and geographical boundaries (with collaborations spanning the globe and involving researchers from countries in the Global South as much as the North). Investigative social research, as the forum showed, is often data intensive, digitally enabled, highly collaborative and impactful and gives rise to its own distinctive “politics of method” (Savage & Burrows 2007, Gray 2019). The purpose of the forum was to explore the methods that enable researchers in this field to pursue topics of social, political and economic import. Whether it is the investigations into the Panama and Paradise Papers, the identification of the Salisbury Novichok poisoners, the documenting of NATO airstrikes as well as civilian deaths across the Middle East, the verification of political violence in Africa through crowd-sourced video and photographic imagery, the tracking and tracing of those infected by COVID-19 during the current pandemic or in quickly contextualising and framing emerging news stories by drawing on new datasets and analytical techniques, investigative social research engages with important aspects of our lives.

Michael Wayne

Much of the attraction of and debates and controversies around the documentary genre derives from it being a hybrid form, straddling both conflicting paradigms within the traditional social sciences on the one hand and the aesthetic dimensions of art and entertainment on the other. Mixed in with these cross-currents, the question of the political or ideological nature of documentary as research, its ‘critical’ or ‘uncritical’ nature vis-à-vis dominant institutions, power relations, commonsense frameworks of explanation, interpretation and embedded cultural assumptions, are never far away. This essay is about this trinity of terms as they pertain to the documentary: critical, creative, research. It asks what it means to discuss documentary as a mode of research, i.e. to what extent this audio-visual based genre overlaps with issues around knowledge production associated with the social sciences; what it means to discuss some documentary films as critical practices, i.e. to define what ‘critical’ might mean for the media generally in the present contemporary context of unleashed global capitalism and how it might best be related to documentary; and this essay asks what it means to discuss documentary as a ‘creative’ practice, i.e. one in which aesthetics plays a key role in its production and consumption. Finally this essay explores the relationships between the critical, creative/aesthetic and research aspects of documentary.

University News

Dr. Sandeep Jagdale

There is a shift of attention experienced by young social scholars who are continuously engaged in either studying or doing research. This continuous endeavor of exploring about social research made them to question their past notions of research, their own work it may be in the form of research done by them or their published research papers in academic journals. Moreover, it is a journey of every social scholar who considers research as a ‘dynamic perspective’ and not as an event. Social research is been much condemned by everyone though it is from pure science subject or social science subject or within social sciences based on quantitative or qualitative shift. The author has an opinion that objectivity of any research in any science can always be in doubt mostly because of human error widely known as conflict of interest. This reflexive article highlights the known concerns about do’s and don’ts of social research. It also tries to cover the most expressed constraints about social research and possible ways to overcome them.

Loading Preview

Sorry, preview is currently unavailable. You can download the paper by clicking the button above.

RELATED PAPERS

Sociological Inquiry

Dorothy ßmith

BULLETIN OF THE „TRANSILVANIA” UNIVERSITY OF BRAŞOV – Series VII: Social Sciences and Law

Florentina Scârneci-Domnișoru

Amin Palangi

Andrew W Chesher

RELATED TOPICS

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

Department of Education and Psychology

Service navigation.

• Mitarbeiter/-innen
• Privacy Policy
• Accessibility Statement
• DE: Deutsch
• EN: English
• Qualitative Research on Human Development

Path Navigation

Documentary Method

The development of the documentary method is closely associated with the Division of Qualitative Research on Human Development. This methodology was first put forward in the 1920s by Karl Mannheim and then taken up again in the 1960s through ethnomethodology, especially by Harold Garfinkel. In its current form, the documentary method first became fruitful for empirical research in the social sciences in 1983, and then especially in 1989, through the work of Ralf Bohnsack at the Institute of Sociology at the University of Erlangen-Nuremberg, originally in connection with the group discussion process.

With this as the basis, the documentary method was then further developed, starting in 1991, at the Division of Qualitative Research on Human Development at Freie Universität Berlin, especially in the context of larger research projects on youth and deviance. This method now applies in a broad field throughout the social sciences and education, even extending as far as information technology and medicine. Its applications range from research into childhood, youth, and gender and adult education to medical sociology, research on policy and organizational cultures, and research on rituals and media use analysis.

The methodical or methodological spectrum covers discussion analysis and the group discussion process, analysis of interviews, participatory observation, and evaluation research, and even includes image and video analysis.

The documentary method forms the methodological foundation for the research conducted at the Center for Qualitative Evaluation and Social Research ( CES) .

• Contact with other divisions, institutes, and chairs that are involved in the further development of the documentary method »
• Selected scholarly monographs and compilations on the documentary method »
• A comprehensive list of literature on the documentary method and of publications that are based on the documentary method is available for download here:  www.dokumentarischemethode.de

• Privacy Policy

Home » ANOVA (Analysis of variance) – Formulas, Types, and Examples

ANOVA (Analysis of variance) – Formulas, Types, and Examples

Table of Contents

Analysis of Variance (ANOVA)

Analysis of Variance (ANOVA) is a statistical method used to test differences between two or more means. It is similar to the t-test, but the t-test is generally used for comparing two means, while ANOVA is used when you have more than two means to compare.

ANOVA is based on comparing the variance (or variation) between the data samples to the variation within each particular sample. If the between-group variance is high and the within-group variance is low, this provides evidence that the means of the groups are significantly different.

ANOVA Terminology

When discussing ANOVA, there are several key terms to understand:

• Factor : This is another term for the independent variable in your analysis. In a one-way ANOVA, there is one factor, while in a two-way ANOVA, there are two factors.
• Levels : These are the different groups or categories within a factor. For example, if the factor is ‘diet’ the levels might be ‘low fat’, ‘medium fat’, and ‘high fat’.
• Response Variable : This is the dependent variable or the outcome that you are measuring.
• Within-group Variance : This is the variance or spread of scores within each level of your factor.
• Between-group Variance : This is the variance or spread of scores between the different levels of your factor.
• Grand Mean : This is the overall mean when you consider all the data together, regardless of the factor level.
• Treatment Sums of Squares (SS) : This represents the between-group variability. It is the sum of the squared differences between the group means and the grand mean.
• Error Sums of Squares (SS) : This represents the within-group variability. It’s the sum of the squared differences between each observation and its group mean.
• Total Sums of Squares (SS) : This is the sum of the Treatment SS and the Error SS. It represents the total variability in the data.
• Degrees of Freedom (df) : The degrees of freedom are the number of values that have the freedom to vary when computing a statistic. For example, if you have ‘n’ observations in one group, then the degrees of freedom for that group is ‘n-1’.
• Mean Square (MS) : Mean Square is the average squared deviation and is calculated by dividing the sum of squares by the corresponding degrees of freedom.
• F-Ratio : This is the test statistic for ANOVAs, and it’s the ratio of the between-group variance to the within-group variance. If the between-group variance is significantly larger than the within-group variance, the F-ratio will be large and likely significant.
• Null Hypothesis (H0) : This is the hypothesis that there is no difference between the group means.
• Alternative Hypothesis (H1) : This is the hypothesis that there is a difference between at least two of the group means.
• p-value : This is the probability of obtaining a test statistic as extreme as the one that was actually observed, assuming that the null hypothesis is true. If the p-value is less than the significance level (usually 0.05), then the null hypothesis is rejected in favor of the alternative hypothesis.
• Post-hoc tests : These are follow-up tests conducted after an ANOVA when the null hypothesis is rejected, to determine which specific groups’ means (levels) are different from each other. Examples include Tukey’s HSD, Scheffe, Bonferroni, among others.

Types of ANOVA

Types of ANOVA are as follows:

One-way (or one-factor) ANOVA

This is the simplest type of ANOVA, which involves one independent variable . For example, comparing the effect of different types of diet (vegetarian, pescatarian, omnivore) on cholesterol level.

Two-way (or two-factor) ANOVA

This involves two independent variables. This allows for testing the effect of each independent variable on the dependent variable , as well as testing if there’s an interaction effect between the independent variables on the dependent variable.

Repeated Measures ANOVA

This is used when the same subjects are measured multiple times under different conditions, or at different points in time. This type of ANOVA is often used in longitudinal studies.

Mixed Design ANOVA

This combines features of both between-subjects (independent groups) and within-subjects (repeated measures) designs. In this model, one factor is a between-subjects variable and the other is a within-subjects variable.

Multivariate Analysis of Variance (MANOVA)

This is used when there are two or more dependent variables. It tests whether changes in the independent variable(s) correspond to changes in the dependent variables.

Analysis of Covariance (ANCOVA)

This combines ANOVA and regression. ANCOVA tests whether certain factors have an effect on the outcome variable after removing the variance for which quantitative covariates (interval variables) account. This allows the comparison of one variable outcome between groups, while statistically controlling for the effect of other continuous variables that are not of primary interest.

Nested ANOVA

This model is used when the groups can be clustered into categories. For example, if you were comparing students’ performance from different classrooms and different schools, “classroom” could be nested within “school.”

ANOVA Formulas

ANOVA Formulas are as follows:

Sum of Squares Total (SST)

This represents the total variability in the data. It is the sum of the squared differences between each observation and the overall mean.

• yi represents each individual data point
• y_mean represents the grand mean (mean of all observations)

Sum of Squares Within (SSW)

This represents the variability within each group or factor level. It is the sum of the squared differences between each observation and its group mean.

• yij represents each individual data point within a group
• y_meani represents the mean of the ith group

Sum of Squares Between (SSB)

This represents the variability between the groups. It is the sum of the squared differences between the group means and the grand mean, multiplied by the number of observations in each group.

• ni represents the number of observations in each group
• y_mean represents the grand mean

Degrees of Freedom

The degrees of freedom are the number of values that have the freedom to vary when calculating a statistic.

For within groups (dfW):

For between groups (dfB):

For total (dfT):

• N represents the total number of observations
• k represents the number of groups

Mean Squares

Mean squares are the sum of squares divided by the respective degrees of freedom.

Mean Squares Between (MSB):

Mean Squares Within (MSW):

F-Statistic

The F-statistic is used to test whether the variability between the groups is significantly greater than the variability within the groups.

If the F-statistic is significantly higher than what would be expected by chance, we reject the null hypothesis that all group means are equal.

Examples of ANOVA

Examples 1:

Suppose a psychologist wants to test the effect of three different types of exercise (yoga, aerobic exercise, and weight training) on stress reduction. The dependent variable is the stress level, which can be measured using a stress rating scale.

Here are hypothetical stress ratings for a group of participants after they followed each of the exercise regimes for a period:

• Yoga: [3, 2, 2, 1, 2, 2, 3, 2, 1, 2]
• Aerobic Exercise: [2, 3, 3, 2, 3, 2, 3, 3, 2, 2]
• Weight Training: [4, 4, 5, 5, 4, 5, 4, 5, 4, 5]

The psychologist wants to determine if there is a statistically significant difference in stress levels between these different types of exercise.

To conduct the ANOVA:

1. State the hypotheses:

• Null Hypothesis (H0): There is no difference in mean stress levels between the three types of exercise.
• Alternative Hypothesis (H1): There is a difference in mean stress levels between at least two of the types of exercise.

2. Calculate the ANOVA statistics:

• Compute the Sum of Squares Between (SSB), Sum of Squares Within (SSW), and Sum of Squares Total (SST).
• Calculate the Degrees of Freedom (dfB, dfW, dfT).
• Calculate the Mean Squares Between (MSB) and Mean Squares Within (MSW).
• Compute the F-statistic (F = MSB / MSW).

3. Check the p-value associated with the calculated F-statistic.

• If the p-value is less than the chosen significance level (often 0.05), then we reject the null hypothesis in favor of the alternative hypothesis. This suggests there is a statistically significant difference in mean stress levels between the three exercise types.

4. Post-hoc tests

• If we reject the null hypothesis, we conduct a post-hoc test to determine which specific groups’ means (exercise types) are different from each other.

Examples 2:

Suppose an agricultural scientist wants to compare the yield of three varieties of wheat. The scientist randomly selects four fields for each variety and plants them. After harvest, the yield from each field is measured in bushels. Here are the hypothetical yields:

The scientist wants to know if the differences in yields are due to the different varieties or just random variation.

Here’s how to apply the one-way ANOVA to this situation:

• Null Hypothesis (H0): The means of the three populations are equal.
• Alternative Hypothesis (H1): At least one population mean is different.
• Calculate the Degrees of Freedom (dfB for between groups, dfW for within groups, dfT for total).
• If the p-value is less than the chosen significance level (often 0.05), then we reject the null hypothesis in favor of the alternative hypothesis. This would suggest there is a statistically significant difference in mean yields among the three varieties.
• If we reject the null hypothesis, we conduct a post-hoc test to determine which specific groups’ means (wheat varieties) are different from each other.

How to Conduct ANOVA

Conducting an Analysis of Variance (ANOVA) involves several steps. Here’s a general guideline on how to perform it:

• Null Hypothesis (H0): The means of all groups are equal.
• Alternative Hypothesis (H1): At least one group mean is different from the others.
• The significance level (often denoted as α) is usually set at 0.05. This implies that you are willing to accept a 5% chance that you are wrong in rejecting the null hypothesis.
• Data should be collected for each group under study. Make sure that the data meet the assumptions of an ANOVA: normality, independence, and homogeneity of variances.
• Calculate the Degrees of Freedom (df) for each sum of squares (dfB, dfW, dfT).
• Compute the Mean Squares Between (MSB) and Mean Squares Within (MSW) by dividing the sum of squares by the corresponding degrees of freedom.
• Compute the F-statistic as the ratio of MSB to MSW.
• Determine the critical F-value from the F-distribution table using dfB and dfW.
• If the calculated F-statistic is greater than the critical F-value, reject the null hypothesis.
• If the p-value associated with the calculated F-statistic is smaller than the significance level (0.05 typically), you reject the null hypothesis.
• If you rejected the null hypothesis, you can conduct post-hoc tests (like Tukey’s HSD) to determine which specific groups’ means (if you have more than two groups) are different from each other.
• Regardless of the result, report your findings in a clear, understandable manner. This typically includes reporting the test statistic, p-value, and whether the null hypothesis was rejected.

When to use ANOVA

ANOVA (Analysis of Variance) is used when you have three or more groups and you want to compare their means to see if they are significantly different from each other. It is a statistical method that is used in a variety of research scenarios. Here are some examples of when you might use ANOVA:

• Comparing Groups : If you want to compare the performance of more than two groups, for example, testing the effectiveness of different teaching methods on student performance.
• Evaluating Interactions : In a two-way or factorial ANOVA, you can test for an interaction effect. This means you are not only interested in the effect of each individual factor, but also whether the effect of one factor depends on the level of another factor.
• Repeated Measures : If you have measured the same subjects under different conditions or at different time points, you can use repeated measures ANOVA to compare the means of these repeated measures while accounting for the correlation between measures from the same subject.
• Experimental Designs : ANOVA is often used in experimental research designs when subjects are randomly assigned to different conditions and the goal is to compare the means of the conditions.

Here are the assumptions that must be met to use ANOVA:

• Normality : The data should be approximately normally distributed.
• Homogeneity of Variances : The variances of the groups you are comparing should be roughly equal. This assumption can be tested using Levene’s test or Bartlett’s test.
• Independence : The observations should be independent of each other. This assumption is met if the data is collected appropriately with no related groups (e.g., twins, matched pairs, repeated measures).

Applications of ANOVA

The Analysis of Variance (ANOVA) is a powerful statistical technique that is used widely across various fields and industries. Here are some of its key applications:

Agriculture

ANOVA is commonly used in agricultural research to compare the effectiveness of different types of fertilizers, crop varieties, or farming methods. For example, an agricultural researcher could use ANOVA to determine if there are significant differences in the yields of several varieties of wheat under the same conditions.

Manufacturing and Quality Control

ANOVA is used to determine if different manufacturing processes or machines produce different levels of product quality. For instance, an engineer might use it to test whether there are differences in the strength of a product based on the machine that produced it.

Marketing Research

Marketers often use ANOVA to test the effectiveness of different advertising strategies. For example, a marketer could use ANOVA to determine whether different marketing messages have a significant impact on consumer purchase intentions.

Healthcare and Medicine

In medical research, ANOVA can be used to compare the effectiveness of different treatments or drugs. For example, a medical researcher could use ANOVA to test whether there are significant differences in recovery times for patients who receive different types of therapy.

ANOVA is used in educational research to compare the effectiveness of different teaching methods or educational interventions. For example, an educator could use it to test whether students perform significantly differently when taught with different teaching methods.

Psychology and Social Sciences

Psychologists and social scientists use ANOVA to compare group means on various psychological and social variables. For example, a psychologist could use it to determine if there are significant differences in stress levels among individuals in different occupations.

Biology and Environmental Sciences

Biologists and environmental scientists use ANOVA to compare different biological and environmental conditions. For example, an environmental scientist could use it to determine if there are significant differences in the levels of a pollutant in different bodies of water.

Advantages of ANOVA

Here are some advantages of using ANOVA:

Comparing Multiple Groups: One of the key advantages of ANOVA is the ability to compare the means of three or more groups. This makes it more powerful and flexible than the t-test, which is limited to comparing only two groups.

Control of Type I Error: When comparing multiple groups, the chances of making a Type I error (false positive) increases. One of the strengths of ANOVA is that it controls the Type I error rate across all comparisons. This is in contrast to performing multiple pairwise t-tests which can inflate the Type I error rate.

Testing Interactions: In factorial ANOVA, you can test not only the main effect of each factor, but also the interaction effect between factors. This can provide valuable insights into how different factors or variables interact with each other.

Handling Continuous and Categorical Variables: ANOVA can handle both continuous and categorical variables . The dependent variable is continuous and the independent variables are categorical.

Robustness: ANOVA is considered robust to violations of normality assumption when group sizes are equal. This means that even if your data do not perfectly meet the normality assumption, you might still get valid results.

Provides Detailed Analysis: ANOVA provides a detailed breakdown of variances and interactions between variables which can be useful in understanding the underlying factors affecting the outcome.

Capability to Handle Complex Experimental Designs: Advanced types of ANOVA (like repeated measures ANOVA, MANOVA, etc.) can handle more complex experimental designs, including those where measurements are taken on the same subjects over time, or when you want to analyze multiple dependent variables at once.

Disadvantages of ANOVA

Some limitations or disadvantages that are important to consider:

Assumptions: ANOVA relies on several assumptions including normality (the data follows a normal distribution), independence (the observations are independent of each other), and homogeneity of variances (the variances of the groups are roughly equal). If these assumptions are violated, the results of the ANOVA may not be valid.

Sensitivity to Outliers: ANOVA can be sensitive to outliers. A single extreme value in one group can affect the sum of squares and consequently influence the F-statistic and the overall result of the test.

Dichotomous Variables: ANOVA is not suitable for dichotomous variables (variables that can take only two values, like yes/no or male/female). It is used to compare the means of groups for a continuous dependent variable.

Lack of Specificity: Although ANOVA can tell you that there is a significant difference between groups, it doesn’t tell you which specific groups are significantly different from each other. You need to carry out further post-hoc tests (like Tukey’s HSD or Bonferroni) for these pairwise comparisons.

Complexity with Multiple Factors: When dealing with multiple factors and interactions in factorial ANOVA, interpretation can become complex. The presence of interaction effects can make main effects difficult to interpret.

Requires Larger Sample Sizes: To detect an effect of a certain size, ANOVA generally requires larger sample sizes than a t-test.

Equal Group Sizes: While not always a strict requirement, ANOVA is most powerful and its assumptions are most likely to be met when groups are of equal or similar sizes.

About the author

Muhammad Hassan

Researcher, Academic Writer, Web developer

You may also like

Critical Analysis – Types, Examples and Writing...

Framework Analysis – Method, Types and Examples

Symmetric Histogram – Examples and Making Guide

Documentary Analysis – Methods, Applications and...

Content Analysis – Methods, Types and Examples

Uniform Histogram – Purpose, Examples and Guide

An official website of the United States government.

Here’s how you know

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• American Job Centers
• Apprenticeship
• Demonstration Grants
• Farmworkers
• Federal Bonding Program
• Foreign Labor Certification
• Indians and Native Americans
• Job Seekers
• Layoffs and Rapid Response
• National Dislocated Worker Grants
• Older Workers
• Skills Training Grants
• Trade Adjustment Assistance
• Unemployment Insurance
• Workforce Innovation and Opportunity Act (WIOA)
• WIOA Adult Program
• Advisories and Directives
• Regulations
• Labor Surplus Area
• Performance
• Recovery-Ready Workplace Resource Hub
• Research and Evaluation
• ETA News Releases
• Updates for Workforce Professionals
• Regional Offices
• Freedom of Information Act
• Office of Apprenticeship
• Office of Foreign Labor Certification
• Office of Grants Management
• Office of Job Corps
• Office of Unemployment Insurance (1-877-S-2JOBS)

The Tools of Apprenticeship.gov

Learn How to File for Unemployment Insurance

Employment Services

ETA Workforce Data Hub

Workforce tools, strategies and technical assistance.

June 6, 2024

Unemployment Insurance Weekly Claims Report

May 30, 2024

May 23, 2024