Lecture 7 - Document and Content Analysis PDF

The Study of Documents Department of Government  Today’s Lecture Documents as a source of information Types of Documents Personal documents Official documents from the state Official documents from private sources Mass media Content an...

The Study of Documents Department of Government  Today’s Lecture Documents as a source of information Types of Documents Personal documents Official documents from the state Official documents from private sources Mass media Content analysis Text Analysis Department of Government  Documentary Analysis Document Study: focus on description, identification of trends, frequencies and interrelationships. Summarise factual info, biographical research, secondary statistical analysis. Content analysis: focus of the research is manifest or latent content of document. Allows conclusions to be made beyond the actual words of the documents. What is the meaning of the document? Text Analysis: uses methods such as semiotics, discourse analysis and hermaneutics. Department of Government  Types of Documents: Personal Documents Diaries, Letters, Emails, Autobiographies, Life Histories, Visual Objects, Video Recordings, Social Media Access (physical or virtual (API)) Representativeness is an important concern. Who wrote/produced the document/output and how typical is it of material of this type Department of Government  Types of Documents: Mass Media Newspapers, magazines, periodicals, film and radio (transcribed or live), podcasts, blogs, vlogs Trace the development of stories Media preferences bias Issue coverage Audience Department of Government  Types of Documents: Official Documents Statistical Information Parliamentary output (Acts, Green Papers, White Papers, Reports) Company Reports Political Party Manifestos Administrative Documents (agendas, minutes, etc) Department of Government  Document Study Descriptive Analysis: summarising the information, what are the main trends. Categorical Analysis: Construct categories before the study and examine the documents in a systematic way using the categories. Exploratory analysis: looking for characteristics in the documents. What is the message that is being conveyed. What is the information, how do you weight it? What is the overall image presented in the document. Comparative analysis: comparing issues across time and countries. Documents are read and compared. Issues and attitudes are examined; gay marriage, abortion. Department of Government  Secondary Analysis of Documents Data is already available Is already in an accessible format, report, etc. Studies existing information to produce new interpretations Addresses a different aspect than the initial author Can employ qualitative or quantitative techniques Is analysing material for a second time. Department of Government  Content Analysis Content analysis: seeking out underlying themes, which are illustrated with short quotations. Can be qualitative or quantitative Analysis looks at forms of communication, intentions of the communicator, text style, attitudes or values. – Frequency, intensity, salience Analysis is non-reactive, document was not produced in the knowledge that it would be content analysed. Analysis is transparent and can be replicated Department of Government  Questions Which Could be Answered by Content Analysis 1. When did news items on this topic begin to appear 2. Which newspapers were fastest in generating an interest in the topic 3. Which newspapers have shown the greatest interest in the topic 4. At what point did interest in the topic begin to drop (news media/social media) 5. Did the approach to the topic change over time 6. What was the tone of the coverage, how did it evolve? Department of Government  Types of Content Analysis Words, phrases, subjects, themes Descriptive content analysis: identifying and describing the main content of the data; thematically, chronologically, etc. Contextual analysis: Studies the research object in context. What is the context of the document Comparative content analysis: comparing texts of different authors, identify differences, in ideology or attitudes. How is the material presented? Processual or particularistic content analysis: elements or aspects of the whole process. Department of Government  Quantitative Content Analysis Decide on the sample parameters: what and when Construct document categories Categories are a set of criteria around a theme or value Categories must be: » Clearly defined » Relate to the research topic » Focus on a specific aspect of the research » Must cover the entire topic Decide on the unit of analysis (words, phrases, sentences, symbols) Code the documents (presence, frequency, intensity of unit of analysis) Statistical testing Notable increase in this type of research due to the wider availability of sources, statistical packages and in particular the huge potential of large language models. Department of Government Quantitative Analysis (Sarantakos, 2006; Bryman 2021) Descriptive Analysis: counting the appearance or frequency of the elements of the research and comparing with other elements Categorical Analysis: Using categories to analyse documents Valence and intensity analysis: data are processed by means of multi-step scales based on theoretical criteria Contingency analysis: employed to make an inference from the text about the personality of the author Contextual Analysis: sequence of concepts in a text is analysed. Used to test for thinking patterns. Department of Government  Advantages and Disadvantages Enables a researcher to May be protected obtain the language and information unavailable to words of the participant public or private access Can be accessed at a time Requires the researcher to convenient to the researcher search out the information Represent data that are in hard-to-find places Requires transcribing or thoughtful, in that optical scanning for participants have given computer processing attention to compiling Materials may be As written evidence it can incomplete save the researcher the time The documents may not be and expense of transcribing authentic or accurate Department of Government  Large Language Models Chat GPT 4 (Linegar et al., 2023; Hackenburg and Margetts, 2024) – Uses language encoders and decoders – To replace previously manual processes Transcribe large bodies of text, summarise large bodies of text, analyze large bodies of text Manifestoes, speeches, parliamentary records Open questions in surveys Generate new content, especially during election campaigns (See Zhang et al., 2023) But caution bias is a serious concern Department of Government  Discussion Question Develop three research questions that could use content analysis? Department of Government  Qualitative Analysis (Sarantakos, 2006; Halpern and Heath 2020) Summative Content Analysis: data are reduced so that the text is integrated, important elements are retained so the text becomes clearer Inductive Category Construction: summarising is used to develop categories from the data Explicating Content Analysis: aims to explain unclear parts of the text, context is used to shed light on the unclear aspects Structuring context analysis: develop structure by putting material in order. Objective hermaneutics: aims to discover latent structures; interested in objective and subjective interpretations. Department of Government  Strengths Weaknesses Content Analysis (Sarantakos: 308) Unobtrusive, has no effect Access can be difficult on the respondent Documents may not be Useful when access to the representative (may relate research topic is not only to a small number of possible people) No respondent or Cannot study unrecorded researcher bias in the data events Low cost Documents may not be complete Coding error may emerge Department of Government  Text Analysis Focus on the perception of the text Semiotics (analysis of symbols) – See beyond and beneath the text Hermaneutics (what is the meaning of the text) – Interpretation of the text within a specific social and historical context Discourse analysis: precise application of content analysis in a qualitative context. Department of Government  Audiovisual Materials Photographs Videotapes Art objects Computer software Film Music Department of Government  Advantages and Disadvantages Unobtrusive method of May be difficult to collecting data interpret Participants can May not be accessible directly share their publicly or privately “reality” The presence of an Creative in that it observer (e.g., film captures attention crew) may be visually disruptive and affect responses Department of Government  Additional Reading Alvarez, R. M., Eberhardt, F., and Linegar, M. (2023). Generative AI and the Future of Elections. Caltech Center for Science, Society, and Public Policy (CSSPP) Policy Brief. Bryman, A., (2021). Social Research Methods. Oxford: Oxford University Press. Hackenburg, K. and Margetts, H., 2024. Evaluating the persuasive influence of political microtargeting with large language models. Proceedings of the National Academy of Sciences, 121(24), p.e2403116121. Halperin, S. and Heath, O., 2020. Political research: methods and practical skills. Oxford University Press, USA. Linegar, M., Kocielnik, R. and Alvarez, R.M., 2023. Large language models and political science. Frontiers in Political Science, 5, p.1257092. Neuendorf, K., (2002) The Content Analysis Guide Book, USA: Sage Publications Krippendorf, K., (1980) Content Analysis; An Introduction to its Methodology, USA: Sage Publications Silverman, D., (1993) Interpreting Qualitative Data; Methods for Analysing Talk, Text and Qualitative Data, London: Sage Publications. Zhang, C., Zhang, C., Zhang, M. and Kweon, I.S., 2023. Text-to-image diffusion models in generative ai: A survey. arXiv preprint arXiv:2303.07909. Weber, R.P., (2001) Basic Content Analysis, USA: Sage Publications Department of Government