Data Analysis and Interpretation PDF

chaPtER 20 aNalyziNg aNd iNterPretiNg data 581 lEaRninG outcoMEs 2. Describe the steps involved in analyzing qualitative research data. Aft...

chaPtER 20 aNalyziNg aNd iNterPretiNg data 581 lEaRninG outcoMEs 2. Describe the steps involved in analyzing qualitative research data. After reading Chapter 20, you should be able to 3. Describe data analysis strategies. do the following: 4. Describe data interpretation strategies. 1. Describe the definitions and purposes of data 5. Describe the steps to be followed to ensure analysis and data interpretation before, during, the credibility of your qualitative research and after data collection. study. data analysis and data sources, including field notes from obser- vations and interviews, questionnaires, maps, interPretation: definition pictures, audio-recorded transcripts, and video- and PurPose recorded observations. On the other hand, data interpretation is an attempt by the researcher to Analyzing qualitative data is a formidable task for find meaning in the data and to answer the “So all qualitative researchers, especially those just what?” question in terms of the implications of starting their careers. Novice researchers follow the findings. Put simply, analysis involves sum- the urgings of mentors who emphasize the need marizing what’s in the data, whereas interpreta- to collect rich data that reveal the perspectives tion involves making sense of—finding meaning and understandings of the research participants. in—those data. After weeks (or months or years) of data collection Data analysis and interpretation are critical steps using a variety of qualitative data collection tech- in the research process that require the researcher niques (e.g., observations, interviews), they find both to know and to understand the data. When themselves sitting in their living rooms surrounded analyzing and interpreting qualitative data, chal- by boxes of data in all shapes and forms! This less- lenge yourself to explore every possible angle and than-romantic image of qualitative researchers is a try to find patterns and seek out new understand- common one. Having immersed themselves in the ings from the data. The techniques outlined in this systematic study of a significant problem, qualita- chapter will serve as guideposts and prompts to tive researchers are confronted with the some- move you through analysis and interpretation as what daunting task of engaging in analysis that efficiently as possible. will represent the mountains of descriptive data accurately. There is no easy way to do this work: It is difficult, time-consuming, and challenging, data analysis during and yet it is potentially the most important part of the research process; it is the part when we try data collection to understand what we have learned through our Data analysis in qualitative research is not left until investigations. all data are collected, as is the case with quantita- Data analysis in qualitative research involves tive research. The qualitative researcher begins summarizing data in a dependable and accurate data analysis from the initial interaction with manner and leads to the presentation of study participants and continues that interaction and findings in a manner that has an air of unde- analysis throughout the entire study. To avoid col- niability. Given the narrative, descriptive, and lecting data that are not important or that come in non-numerical nature of the data that are col- a form that cannot be understood, the researcher lected in a qualitative study, it is not possible to must think, “How am I going to make sense of “number crunch” and quickly reduce the data to the data?” before conducting the study. During a manageable form, as can be done in quantita- the study, the researcher should try to narrow the tive studies. Qualitative data analysis requires topic progressively and to focus on the key aspects that the researcher be patient and reflective in of the participants’ perspectives. Thus, the qualita- a process that strives to make sense of multiple tive researcher goes through a series of steps and 582 chaPtER 20 aNalyziNg aNd iNterPretiNg data iterations: gathering data, examining data, compar- data analysis after data collection ing prior data to newer data, writing field notes before returning to the research site, and making After the data have been collected, the romance plans to gather new data. Data collection and anal- of fieldwork is over, and the researcher must ysis continually interact; the researcher’s emerging concentrate solely on the task of data analysis. hunches or thoughts become the focus for the next The researcher must fully examine each piece of data collection period. information and, building on insights and hunches While gathering data, the researcher reviews gained during data collection, attempt to make everything and asks questions: “Why do partici- sense of the data as a whole. Qualitative data pants act as they do?” “What does this focus mean?” analysis is based on induction: The researcher “What else do I want to know about that par- starts with a large set of data representing many ticipant’s attitude?” “What new ideas have emerged things and seeks to narrow the set progressively in this round of data collection?” “Is this a new into small and important groups of key data. No concept, or is it the same as a previous one?” This predefined variables help to focus analysis, as in ongoing process—almost a protracted discussion quantitative research. The qualitative researcher with oneself—leads to the collection of new impor- constructs meaning by identifying patterns and tant data and the elimination of less useful data. themes that emerge during the data analysis. Anderson and colleagues1 suggested that qual- A problem that faces almost all qualitative itative researchers answer two questions to guide researchers is the lack of agreed-on approaches for their work and reflections: analyzing qualitative data. Some guidelines and gen- eral strategies for analysis exist, but there are few 1. Is your research question still answerable and specific rules for their application. In brief, after data worth answering? are collected, the qualitative researcher undertakes a 2. Are your data collection techniques catching multistage process of organizing, categorizing, syn- the kind of data you want and filtering out thesizing, analyzing, and writing about the data. In the data that you don’t want? most cases, the researcher cycles through the stages more than once in a continual effort to narrow and Consciously pausing during the research process make sense of the data. As a result, the time frame allows you to reflect on what you are attending for data analysis is difficult to determine in advance to and what you are leaving out. Such a reflective because it depends on the nature of the study, stance will continue to guide your data collection the amount of data to be analyzed, and the analytic efforts as well as to allow for early hunches about and synthesizing abilities of the researcher. what you are seeing so far. If you conduct a qualitative study, take time Although ongoing analysis and reflection is to immerse yourself fully in your data—bury a natural part of the qualitative research pro- yourself in what you have. Read and reread, listen cess, it is important to avoid premature actions and relisten, watch and rewatch. Get to know inti- based on early analysis and interpretation of data. mately what you have collected. Struggle with the Researchers engaged in their first systematic study nuances and caveats, the subtleties, the persuasive, tend to collect, analyze, and interpret data zeal- the incomplete. Avoid premature judgment. These ously in rapid-fire fashion. Their efforts can go goals are lofty, but they are at the heart of what awry if they become their own best informants we are trying to achieve in qualitative data analysis and jump to hasty conclusions and impulsive and data interpretation. actions. The qualitative research process takes time; researchers must be wary of the lure of quick-fix strategies and patient enough to avoid the pitfalls of stating research outcomes on the stePs in analyzing basis of premature analysis. QualitatiVe research data If data are to be analyzed thoroughly, they must 1 be organized. Ideally, the researcher will have Studying Your Own School: An Educator’s Guide to Qualitative Practitioner Research (p. 155), by G. L. Anderson, K. Herr, and carefully managed notes, records, and artifacts as A. S. Nihlen, 1994, Thousand Oaks, CA: Corwin Press. they were collected. The importance of attention  chaPtER 20 aNalyziNg aNd iNterPretiNg data 583 FiGuRE 20.1 Data-organizing activities Write dates (month, day, year) on all notes. Sequence all notes with labels (e.g., 6th set of notes). Label notes according to type (such as observer’s notes, memo to self, transcript from interview). Make two photocopies of all notes (field notes, transcripts, etc.) and retain original copies. Organize computer files into folders according to data type and stages of analysis. Make backup copies of all files. Read through data and make sure all information is complete and legible before proceeding to analysis and interpretation. Begin to note themes and patterns that emerge. to detail in managing data becomes all too clear the logical sequence of activities is from reading/ when it is time to write the research. Nevertheless, memoing to description, to classifying, and finally some additional organization at the end of the data to interpretation. However, as a researcher begins collection stage is usually necessary. Figure 20.1 to internalize and reflect on the data, the initial lists some ways to “tidy up” your data, ensure their ordered sequence may lose its structure and completeness, and make them easier to study. become more flexible. If you’ve ever been driving After the data are organized, the analysis can begin home pondering some issue or problem and out in earnest. of the blue had a sudden flash of understanding One way to proceed with analysis is to fol- that provides a solution, you have a sense of how low three iterative, or repeating, steps: reading/ qualitative data analysis takes place. Once you are memoing, describing what is going on in the set- into the data, it is not the three steps that lead to ting, and classifying research data. The process understanding; it is your ability to think, imagine, focuses on (1) becoming familiar with the data create, intuit, and analyze that guides the data anal- and identifying potential themes (i.e., reading/ ysis. Knowing the steps is not enough; the thinker, memoing); (2) examining the data in depth to imaginer, and hypothesizer—that is, the qualitative provide detailed descriptions of the setting, par- researcher—is the data analyzer, and the quality of ticipants, and activity (i.e., describing); and (3) cat- the research analysis depends heavily on the intel- egorizing and coding pieces of data and grouping lectual qualities of the researcher. Let us be very them into themes (i.e., classifying). clear about this process: It is a process of digest- ing the contents of qualitative data and finding Write dates (month, day, year) on all notes. related threads in it. You will not accomplish these Sequence all notes with labels (e.g., sixth set of tasks meaningfully with one or two or more read- notes). ings of your data. To make the kinds of connec- Label notes according to type (such as tions needed to analyze and interpret qualitative observer’s notes, memo to self, transcript from data, you must know your data—really know it, in interview). your head, not just on paper. The process can be Make two photocopies of all notes (field notes, tedious, time-consuming, and repetitious; however, transcripts, etc.) and retain original copies. the steps can help you understand, describe, and Organize computer files into folders according classify qualitative data. to data type and stages of analysis. Make backup copies of all computer files. Read through data and make sure all Reading/Memoing information is complete and legible before The first step in analysis is to read and write proceeding to analysis and interpretation. memos about all field notes, transcripts, and Begin to note themes and patterns that emerge. observer comments to get an initial sense of the The interrelations among these steps are not data. To begin, find a quiet place and plan to spend necessarily linear. At the start of data analysis, a few hours at a time reading through the data. 584 chaPtER 20 aNalyziNg aNd iNterPretiNg data Krathwohl2 wisely pointed out that “the first time classifying you sit down to read your data is the only time you come to that particular set fresh.” It is important Qualitative data analysis is a process of breaking that you write notes in the margins or underline down data into smaller units, determining their sections or issues that seem important to you so import, and putting the pertinent units together in that you have a record of your initial thoughts and a more general, analytical form. Qualitative data sense of the data. Later, when you are deeper into are typically broken down through the process of the analysis, you may find that many of these early classifying or coding; the pieces of data are then impressions are not useful; however, you may also categorized. A category is a classification of ideas find that some initial impressions hold up through- or concepts; categorization, then, is grouping the out. At this stage of analysis, you should also begin data into themes. When concepts in the data are the search for recurring themes or common threads. examined and compared to one another, and con- nections are made, categories are formed. As an example, consider a researcher who is describing conducting a qualitative study on characteristics The next step, describing, involves developing of fifth-grade students’ study methods. Suppose thorough and comprehensive descriptions of the researcher had collected 20 sets of field notes the participants, the setting, and the phenom- (i.e., based on observations) or 20 transcripts enon studied to convey the rich complexity of of interviews. The researcher’s task is to read the research. The descriptions are based on your through all the notes or transcripts and catego- collected observations, interview data, field notes, rize the meanings or understandings that emerge and artifacts. The aim of this step is to provide from the data. The categories provide the basis a narrative picture of the setting and events that for structuring the analysis and interpretation. take place in it so you will have an understanding Without data that are classified and grouped, a of the context in which the study is taking place. researcher has no reasonable way to analyze qual- Attention to the research context is a common and itative studies. However, the categories identified important theme in qualitative research because by one researcher would not necessarily be the the context influences participants’ actions and same as those identified by another researcher, understandings. Because meaning is influenced by even if they analyzed the same data. There is context, analysis (and therefore, interpretation) is no single “correct” way to organize and analyze hampered without a thorough description of the the data. Different researchers produce different context, actions, and interactions of participants. categories from the same data for many reasons, An important concern of qualitative research- including researcher biases, personal interests, ers is portraying the views of the research partici- style, and interpretive focus. pants accurately. The descriptions of the research context, meanings, and social relations can be presented in a number of forms. For example, data analysis strategies you can describe events in chronological order, create a composite of a typical day in the life of a In this section we describe strategies used to ana- participant in the setting, focus on key contextual lyze qualitative data: identifying themes; coding episodes, or illuminate different perspectives of surveys, interviews, and questionnaires; asking the participants. Regardless of the form, it is cru- key questions; doing an organizational review; cial that you describe thoroughly how participants developing a concept map; analyzing antecedents define situations and explain their actions. Also, and consequences; displaying findings; and stating your descriptions should note how interactions what is missing. Each is important in identifying and social relations among the participants may research categories and patterns. have changed during the course of the study. identifying Themes Another way to begin ana- 2 lyzing data is to consider the big picture and start Methods of Educational and Social Science Research: An Integrated Approach, by D. R. Krathwohl (2nd ed., p. 309), to list themes that you have seen emerge in your 1998, New York: Longman. literature review and in the data collection. Are  chaPtER 20 aNalyziNg aNd iNterPretiNg data 585 there patterns that emerge, such as events that with 13 collections of four cards that have some keep repeating themselves, key phrases that par- trait in common (e.g., the number or face value of ticipants use to describe their feelings, or survey the card). Again, you reshuffle the cards. This time, responses that seem to match one another? Noting as you start to sort through the cards, you notice a themes can be helpful during the first reading of different theme (e.g., the suit of the card) and end the data, as part of memoing. In subsequent read- up with four piles of 13 cards. Puzzling. Not to be ings of the data, additional themes may emerge. thwarted in your efforts, you again reshuffle the deck and attempt to settle on an organizing theme. Coding surveys, interviews, and Questionnaires You group together cards (i.e., data) that have suf- One of the most frequent data analysis activities ficient common characteristics that you feel confi- undertaken by qualitative researchers is coding, dent that your analysis is undeniably accurate. But the process of categorically marking or referencing there is just one problem: What do you do with the units of text (e.g., words, sentences, paragraphs, joker that found its way into the pack? And what and quotations) with codes and labels as a way to about that wild card? Where did they come from, indicate patterns and meaning. As you analyze and and where do they fit in? Just when you thought code, you reduce your data to a manageable form. you had it all worked out, in crept something that One way to proceed when working with field challenges the themes you have used to organize notes, transcripts of recorded interviews, pictures, and represent the data you have collected. The maps, and charts is to record important data on shuffling and sorting continues. index cards, which are manageable and allow for A few commonsense guidelines may make this sorting. As you read and reread through the data somewhat overwhelming activity of coding moun- (possibly now reduced to your cards), you can tains of data more manageable: compile the data in categories or themes. Although there is nothing magical about the 1. Gather photocopies of your original data. process of coding, it does take time and a willing- 2. Read through all the data and attach working ness to check that the mountains of descriptive labels to blocks of text. These labels should data have been analyzed accurately and reliably. have meaning for you; they should be a kind The way in which you code the data, in fact, will of shorthand that will serve as reference play a large role in determining the nature of the points when you return to the text later in results. For example, if you approach the data the process. with preconceived categories and assumptions, 3. Literally cut and paste the blocks of text onto you will likely begin analyzing the data by coding index cards so that you now have the data text units according to what you expect to find. in a manageable form (i.e., shuffling cards is Conceptually, you are beginning to construct a much easier than sorting through reams of web of relations that may or may not appear as paper). Use some kind of numbering system you thought they would. On the other hand, if you so that you can track the block of text back approach the data with questions you hope your to the original context in which it appeared. research will illuminate but no clear sense about For example, marking the date and time (e.g., what the findings may be, you will likely start to 1/26 10:15) can help you locate the text in build themes as you read. the journal or field notes from which it was To get an idea of the process of coding, imag- excerpted. Remember, context is important. ine that you are organizing a deck of playing cards, Check that you have correctly labeled the but you don’t know the meaning of any of the sym- text you are trying to funnel into a category bols on the cards. Each card in the deck contains with similar text. data, and the order of the cards is random. As you 4. Start to group together cards that have the initially scan the cards, you have an intuitive sense same or similar labels on them. that the data on some of the cards look similar to 5. Revisit each pile of cards and see if the label data on other cards. You finish looking carefully at still fits or whether similar labels warrant all the cards and reshuffle the deck. Again you look their own category. Seek categories that through the deck, but this time you group together encapsulate similar thoughts and ideas. This the cards with data that look alike. You end up process is similar to brainstorming. 586 chaPtER 20 aNalyziNg aNd iNterPretiNg data For example, in my study of school district change,3 and can get on with doing whatever it is that I found myself with a large pile of 3 × 5 cards that they do with all the paperwork? I admit that I included some of the following notations: have written plans and never followed up on them because I’m too busy getting on with the Card 1. Assistant superintendent urges principals real business of school.” not to reinvent the wheel but to share ideas with each other as they attempt to deal with an Following the four commonsense guidelines identified problem. (In this case the problem was presented earlier in this chapter, the first step of low test scores on the California Achievement “attaching working labels” to blocks of text that Test [CAT].) The assistant superintendent states to are then “cut and pasted” onto cards resulted the principals, “I don’t want any of you to think in the following grouping of cards: Cards 1, 3, that you are alone out there.” and 5 were labeled “Statement of school district Card 2. One of the principals at the meeting approach to school change.” Cards 2 and 4 were comments, “Clearly, the CAT does not test what labeled “Principals’ challenges to school district we teach in our schools. The test was designed approach.” Cards 6 and 7 were labeled “Inaction of for California, not Oregon.” school district approach.” These cards are indicative of the comments Card 3. The next meeting of principals that were captured during interviews with indi- following the release of the CAT scores and vidual principals and observations of principals’ the directive from the superintendent that “all meetings and that collectively provided the con- schools will develop action plans to address text and understanding for the analysis, which areas of weakness identified by the test scores” resulted in a statement of a theme titled “inac- does not include any discussion of action plan tion.” In writing about school change as it related development. to the McKenzie School District, my data analysis Card 4. A principal sums up his feelings about included a “Taxonomy of Managing and Coping standardized testing as follows, “The district Strategies for Educational Change” with themes makes us go through a whole lot of garbage for such as “inaction” that emerged to describe the little outcome or benefit to the teachers and the change process; that is, one of the ways that the students.” McKenzie School District personnel managed and Card 5. Principals’ meeting 3 months coped with educational change was to do noth- following the release of test scores and action ing! I have included this example to demonstrate plan mandate. Action plans were due to the how a theme emerges from the data you collect. curriculum director 7 weeks ago. Principals are I chose the term inaction as a theme because it instructed that they can have another 2 weeks was descriptive (to me) of what was occurring in to complete the plans. the district. The same will be true for your own Card 6. The assistant superintendent analysis—as you code your data and reduce them announces that he will be meeting with to a manageable form, a label will emerge that principals on an individual basis to discuss describes a pattern of behavior. You will be well how action plans for school improvement on your way to making sense of your data! will be implemented. It is 4 weeks before the end of the school year, and 16 weeks Example of coding an interview since the initial directive to develop school improvement action plans. What follows is an annotated interview between a Card 7. One principal commented on the researcher and a bilingual education teacher as an development of the action plan/school example of the researcher’s analysis of the themes improvement plan, “Do I write plans of that emerged from the interview. As this example improvement just to let the central office know illustrates, the process of analyzing an interview that it has been done so that they are satisfied transcript involves careful reading of the transcript to identify broad themes that emerge from the data 3 that will help answer the researcher’s research ques- Mills, G.E. (1988). Managing and coping with multiple educa- tional change. Unpublished doctoral dissertation. Eugene, OR: tions. This in-depth, intimate knowledge and exami- University of Oregon. nation of the data allows qualitative researchers to  chaPtER 20 aNalyziNg aNd iNterPretiNg data 587 categorize themes and ideas that will contribute pervasive, recurring theme that contributes to the to their understanding of the phenomenon under researcher’s understanding of the phenomenon and investigation. In this example, fear of change is a possibly provides an answer to a research question. Coding from a sample interview Transcript Codes Q: Why do you think that English-only teachers fear bilingual education? Themes (and Other ideas) Culture A: I think the fear factor is on a real gut level and personal level. Teachers feel it’s Fear kind of a one-way system in that the teachers who are in the all-English program Fear of change are fearful at a real basic visceral level that their jobs and their livelihood are at Job stability risk. Not to mention their culture, their society, and their known world is at risk Fear of new job by this other language coming into the schools and being acknowledged in the schools. And the teacher might say, “Oh well, because I don’t have Spanish that means I am going to be out of a job. Am I going to be replaced by a bilingual teacher? If you have this program in my school that means you’re going to need bilingual teachers. I am not bilingual so my job is at risk.” Q: Do you think that there is resistance toward expecting all children to learn English? Nativistic A: I think that’s an interpretation that comes out of a model like a 90/10. When movements the child needs to come into the first year and has 90% in Spanish and 10% in Patriotic English, it’s easily perceived that we are withholding English from the child. That is a perception. A 50/50 model is a little more amenable to that because it’s obvious that 50% of the time the child isn’t getting English. Q: There is the old adage that teachers who oppose bilingual education say, “My ancestors never received bilingual education services in public schools and they did just fine.” How do you respond to that kind of attitude toward bilingual education? A: I say that’s old thinking. I think that what your parents or your grandparents had to do when they came here from Italy or Norway, or wherever they came from, to learn another language, the language demand was less than it is today. Q: What about the attitude, “Well they are in the United States and we speak English here so they can learn English. That’s all there is to it.” How would you respond to this attitude? A: That’s a big one. That’s huge. I think that’s a whole cultural, you know, it’s Fear based again in fear. Based again in the fact that the United States is a very isolated island in that we are closed in by two oceans and we have never had the habit of stretching out beyond our borders much, or valuing much of what is beyond our borders. We are xenophobic in that sense. So we haven’t traditionally learned other Fear languages, or been interested in other languages. “Why bother, we’re America, the biggest, the toughest, so why would we value anybody else’s culture or language?” And I think that’s an old thinking as well. It’s an old habit. Asking Key Questions Another strategy used in questions can enable qualitative researchers to data analysis involves asking key questions. Acc- “extend their understanding of the problems and ording to Stringer,4 working through a series of contexts” they have investigated. Such questions may include: “Who is centrally involved?” “Who 4 Action Research: A Handbook for Practitioners (p. 87), by E. T. has resources?” “Which ones?” “What major activi- Stringer, 1996, Thousand Oaks, CA: Sage. ties, events, or issues are relevant to the problem?” 588 chaPtER 20 aNalyziNg aNd iNterPretiNg data “How do acts, activities, and events happen?” solid lines) and influences you have a hunch “When does this problem occur?” Although not all about (using dotted lines). these questions will be applicable to any single 3. Review the concept map to determine any situation, they may provide a starting point for consistencies or inconsistencies that exist qualitative researchers who are engaged individu- between the influences. This forces you back ally or collectively in analysis. to your data to see what’s missing. For example, in a study of the effectiveness Doing an Organizational Review Almost any edu- of a school absenteeism policy, the researcher cational problem is influenced in some way by the concluded that respectfulness, safety, conflict spoken and unspoken rules of organizations, management, discipline, school rules, behavior, including state education departments, school dis- getting along, self-esteem, and academics were tricts, individual schools, teacher unions, and other major indicators of success. Further, the researcher similar organizations. Even in a qualitative study believed that some relationships (real and per- where the emphasis is on the personal story of a ceived) existed between or among these factors single individual or the intimate workings of a (see Figure 20.2). small group, it is sometimes helpful to step back and take a look at the larger setting. Researchers Analyzing Antecedents and Consequences The may consider undertaking an organizational re- process of mapping antecedents (i.e., causes) and view that focuses on several features of the organi- consequences (i.e., effects) is another strategy to zation, including the vision and mission, goals and help qualitative researchers identify the major ele- objectives, structure and operation, and problems ments of their analysis.7 Using this framework and concerns. As Stringer noted:5 “As participants provides a visual representation of the causal rela- work through these issues, they will extend their tionships that the researcher believes exist. It is understanding of the organization and aspects of also helpful to revisit the causal relationships un- its operation that are relevant to their problems, is- covered in your review of the literature to deter- sues, and concerns.” With these features in mind, a mine challenges and support for your analysis and review of a school, for example, may provide in- interpretations. The steps for analyzing anteced- sight into the data you have collected. ents and consequences are as follows: 1. List the influences that emerged from the developing a concept Map analysis for which there appear to be a causal relationship. Stringer6 suggested that concept maps are another 2. Revisit the review of literature to determine useful strategy that helps action research partici- whether the analysis of the study supports, pants to visualize the major influences that have or is challenged by, the findings of previous affected the study. For example, what were the studies. perspectives of the students? Parents? Teachers? 3. Revisit your data to determine if anything is Administrators? A concept map gives participants missing and suggest how your findings may an opportunity to display their analysis of the influence future research. problem and to determine consistencies and incon- sistencies that may exist between the disparate In the study of the effectiveness of a school groups. The steps for developing a concept map absenteeism policy, for example, the concept map include the following: in Figure 20.2 could be expanded to include a mapping of antecedents (causes) and conse- 1. List the major influences that have affected quences (effects) as an outcome of the analysis. the study of your area of focus. In this example, the researcher clearly identified 2. Develop a visual representation of the major (based on his analysis) that a causal relationship influences (factors) connecting the influences existed between absenteeism and academics (stu- with relationships you know exist (using dent performance), and absenteeism and discipline 5 Ibid., pp. 90–91. 7 6 Ibid., p. 91. Ibid., p. 96.  chaPtER 20 aNalyziNg aNd iNterPretiNg data 589 FiGuRE 20.2 Concept map of the factors affecting absenteeism Academics Respectfulness Self-Esteem Safety Absenteeism Conflict Getting Along Management Behavior School Rules Discipline Source: Mills, geoffrey, Action Research: A Guide for the Teacher Researcher, 5th edition, © 2014. reprinted by permission of Pearson education, inc., upper saddle river, Nj. (student behavior). This framework provides a answers, to move beyond our data with unwar- visual representation of the causal relationships ranted assertions that may, in some cases, ultimately that the researcher has identified. It is helpful to lead to embarrassing questions about what we revisit the relationships uncovered in the review of actually did. In keeping with the theme of avoid- literature to determine challenges and support for ing premature judgment (i.e., arriving at answers the analysis and interpretations. to problems without systematic inquiry), the data analysis strategy of stating what’s missing allows Displaying findings It is important to summarize you to hint at what may or should be done next in the information you have collected in an appropriate your quest to understand the findings of your study. and meaningful format that you can share with in- terested colleagues. To do this, it is helpful to “think display” as you consider how to convey your find- Qualitative data analysis: an ings to colleagues. Displays can include matrixes, Example charts, concept maps, graphs, and figures—what- ever works as a practical way to encapsulate the The example that follows is intended to provide findings of the study. These visual displays of data a sense of qualitative analysis. A true qualitative serve an important function for researchers who study would entail more data analysis than shown want to share findings and celebrate their insights here, but the basic ideas represent the process that in a public forum, such as a research conference. a qualitative researcher would undertake when Putting the data into a visual format can also help analyzing data throughout a study. you see new aspects in the data. In this example, the topics under study are the concerns of parents regarding their first child’s stating What is Missing Finally, as part of your entrance into kindergarten and the kindergar- full reporting, you should flag for the consumers ten teacher’s interactions with the students and of your research the pieces of the puzzle that are families. The participants are four parents (three still missing and identify the questions for which female and one male, representing four fami- you have not been able to provide answers. Often lies) and the first child in each of the families; we find ourselves wanting and needing to provide the children attend the same school, and their 590 chaPtER 20 aNalyziNg aNd iNterPretiNg data kindergarten teacher is also a research participant. 3. From your initial analysis, you group the Data collection procedures include observations individual items or topics together into and interviews with students, parents, and the kin- categories that show how the items or dergarten teacher. topics are related. For example, as shown in Data analysis would proceed as follows: Figure 20.3, you could group books, videos, and handouts under a category called Teaching 1. From the field notes of your classroom Materials. You could group together the ways observations, you begin to list some common in which the instruction was carried out— items or topics that you noticed. You individual, small-group, and whole-class—and recorded in your notes that during classroom label this category as Classroom Interactions. instruction, the teacher was using books, Using information from the interviews, videos, and handouts. You also noted that you could construct the category Indirect instruction was directed sometimes toward Communication with Families/Guardians to individual students, sometimes toward the include grading, lesson plans, tests, and report whole class, and sometimes toward students cards. A category of Direct Communication who were working together in small groups. with Families/Guardians could include family 2. From your interviews with the teacher, you conferences, report cards, progress reports, realize that she gave you information about and phone calls to parents or guardians. Notice how she communicated with families about that report cards appear in both the indirect the children. You note that she talked about and the direct communication categories. how she indirectly communicates through 4. You organize your four categories into grading and report cards and how her lesson patterns, which are made up of two or more plans and tests are related to her overall categories. For example, the categories assessment of the students’ work. She also of Teaching Materials and Classroom mentioned that she talks about report cards Interactions indicate a pattern of Instructional directly with families during conferences. Activities. The categories of Indirect She also communicates with families about Communication and Direct Communication their children through progress reports and fit together under a pattern of Teacher– phone calls. Family Interactions. FiGuRE 20.3 Diagram of category levels and organization in the qualitative data analysis of first child in school Patterns Instructional Teacher–Family Activities Interactions Categories Teaching Classroom Indirect Communication Direct Communication Materials Interactions with Families/Guardians with Families/Guardians Topics books handouts small- grading tests family report progress phone (data videos group lesson conferences cards reports calls to pieces) individual whole- plans families student class  chaPtER 20 aNalyziNg aNd iNterPretiNg data 591 You then decide whether you need to collect addi- The following Digital Research Tools for the 21st tional data by interviewing students and parents or Century feature discusses three common computer guardians about their experiences interacting with software packages available to assist qualitative the teacher to confirm your categories and patterns. researchers with the analysis of qualitative data. digital research tools for the 21st century QualitativE data analYsis coMPutER soFtWaRE Computer software to assist with the analysis of qual- Do you have the resources to purchase a itative, narrative data has been available to research- program, or do you know someone who has ers for many years. The key word in this sentence is the program? assist. This software will not do the analysis for you! Do you need to be able to capture specific It is important for novice qualitative researchers to quotes from a large database? remember that computers do not analyze or even Three common and popular qualitative analy- code data. They are designed only to help expedite sis software packages are NVivo 10, Ethnograph these operations when researchers are working v6, and HyperRESEARCH 3.0. with large bodies of text and other kinds of data. The process of coding, retrieving, and subsequently mulling over and making sense of data remains a nVivo 10 laborious process completely controlled by research- NVivo 10 is designed for qualitative researchers who ers. Even if a computer is used, researchers still must need to work with complex data, especially multi- go through the process of creating codes and labels media data. NVivo is designed to assist researchers and keying them into the computer as they read with organizing, classifying, and analyzing data through their interviews, field notes, and audio and and allows the researcher to work with documents, video recordings. Computers are merely handy and PDFs, spreadsheets, audio, video, and pictures. extremely fast labeling and retrieval tools. In ad- More information on NVivo can be found on the dition, researchers must program the computer to QSR International website at qsrinternational.com. retrieve and sort data in specific ways; the machines do not do these tasks automatically. Although com- puters can enhance and broaden qualitative research ethnograph v6 analysis, if you are not connected in some way with Ethnograph v6 is a program designed to help a research university, it is unlikely that you will have qualitative researchers work with text files (in access to the software and the expertise of someone any format) and search for and code segments to teach you how to use it. of interest to the researcher. More information To help with the decision about whether or about Ethnograph v6 can be found on the Qualis not to proceed with locating and learning a quali- Research website at qualisresearch.com. tative data analysis software package, ask yourself the following questions:8 hyperresearch 3.0.2 Are you analyzing large amounts of data (e.g., more than 500 pages of field notes and HyperRESEARCH 3.0.2 is an advanced software transcripts)? program that allows the qualitative researcher to Are you or can you be adequately trained work with text, graphics, audio, and video sources in the use of the programs and in using and to code and retrieve data. More informa- computers in general? tion about HyperRESEARCH can be found on the ResearchWare website at researchware.com. 8 Remember, computer software will not do the List items adapted from Educational Research: Planning, data analysis for you, but it will help you retrieve Conducting, and Evaluating Quantitative and Qualitative Research (5th ed., p. 239), by J. W. Creswell, 2015, Upper categories from a large amount of narrative, audio, Saddle River, NJ: Pearson Education, Inc. video, and photo data. 592 chaPtER 20 aNalyziNg aNd iNterPretiNg data data interPretation The techniques for data interpretation that fol- low are adapted from those presented by Wolcott strategies and by Stringer.10 Because the goal of data interpretation is to find Extend the analysis. One technique that is low meaning in the data, it is based heavily on the on the data interpretation risk scale is to extend connections, common aspects, and links among the analysis of the data by raising questions about the data, especially the identified categories and the study, noting implications that may be drawn patterns. One cannot classify data into categories without actually drawing them. As Wolcott sug- without thinking about the meaning of the catego- gested, “This is a strategy for pointing the way ries. To aid interpretation, researchers must make rather than leading the way.”11 the conceptual bases or understandings of the Connect findings with personal experience. categories explicit and identify clearly the charac- Qualitative research is very personal business, teristics that make each category distinct from the so it makes sense to personalize your interpreta- others. Interpretation requires more conceptual tions. For example, you may present your find- and integrative thinking than data analysis because ings with the prelude, “Based on my experiences interpretation involves identifying and abstract- in conducting this study, this is what I make of it ing important understandings from the detail and all.” Remember, you know your study better than complexity of the data. anyone else. You have been there for every twist The implicit issues in data interpretation are and turn along the way, trying to make sense of the answers to these four questions: discrepant events just when you thought you had it right. Share your interpretations based on your 1. What is important in the data? intimate knowledge and understandings of the 2. Why is it important? research setting. 3. What can be learned from it? Seek the advice of critical friends. If you have 4. So what? difficulty focusing an interpretive lens on your work, rely on your trusted colleagues to offer The researcher’s task, then, is to determine how insights that you may have missed because of to identify what is important, why it is impor- your closeness to the work. Offer your accounts tant, and what it indicates about the participants to colleagues with the request that they share with and context studied. The process for answer- you their possible interpretations. Similarly, you ing these four questions is idiosyncratic to a may ask your informants (e.g., students, parents, large extent. Interpretation is personal, with no teachers, and administrators) for their insights. But hard-and-fast rules to follow as you go about beware! The more opinions you seek, the more the task of interpreting the meaning of data. As you will receive, and often these suggestions will in most qualitative studies, success depends on come with the expectation that you accept the the perspective and interpretive abilities of the advice. Over time you will develop reciprocity with researcher. a cadre of trusted, like-minded colleagues who You may wonder why you should bother with will selflessly fulfill the role of critical friend. Take interpretation, especially because it involves taking the time to build these relationships and reap the risks and making educated guesses that may be rewards they offer. off base. Wolcott9 argued for the importance of Contextualize findings in the literature. interpretation, noting that the interpretations made Uncovering external sources as part of the review by qualitative researchers matter to the lives of of related literature is a powerful way for qualita- those we study. In addition, the process of tive researchers to provide support for the findings interpretation is important because it can chal- of the study. Making these connections also pro- lenge the assumptions and beliefs that researchers vides a way to share with colleagues the existing have about the educational processes they have knowledge base about a research problem and investigated. to acknowledge the unique contribution that the 9 10 Transforming Qualitative Data: Description, Analysis, and Ibid., pp. 39–46; Action Research (p. 87–96), Stringer, 1996. 11 Interpretation, by H. F. Wolcott, 1994, Thousand Oaks, CA: Sage. Transforming Qualitative Data (p. 40), Wolcott, 1994.  chaPtER 20 aNalyziNg aNd iNterPretiNg data 593 qualitative researcher has made to the understand- alienated by our colleagues. Avoid being evan- ing of the topic under study. gelical about your interpretations, connect them Turn to theory. Theory serves a number of closely to your data and analysis, and share your important roles for qualitative researchers. First, newfound understandings with colleagues in an theory provides a way for qualitative research- appropriate manner. ers to link their work to broader issues of the day. Second, theory allows researchers to search for increasing levels of abstraction and to move beyond a purely descriptive account. Levels of ensuring credibility abstraction allow us to communicate the essence in your study of our descriptive work to colleagues at research meetings. Third, theory can provide a rationale or Throughout this chapter, we have emphasized sense of meaning to the work we do. the centrality of the researcher as the integrator Know when to say when! If you don’t feel and interpreter of data. You may infer that this comfortable offering an interpretation, don’t do it. Be emphasis means that researchers have carte satisfied with making suggestions for what may be blanche when analyzing and interpreting data, done next, and use the suggestions yourself as a that is, that they can rely strictly on their per- starting point for the next research cycle. Restate the sonal feelings or preferences. This is definitely problem as you now see it, and explain how you not the case. If qualitative research were based think you will fine-tune your efforts as you strive to solely on producing unsubstantiated opinions, increase your understanding of the phenomenon you with researchers ignoring data that did not con- have investigated. As Wolcott12 cautioned, “[D]on’t firm expectations and failing to examine biases detract from what you have accomplished by tacking of research participants, it would be of little on a wimpy interpretation.” value. Although data analysis and interpretations All researchers, and qualitative researchers in are heavily determined by the researcher, qualita- particular, must face the prospect of not being able tive researchers should respect and respond to to report all the data they have collected. Rarely is established criteria when conducting their stud- every piece of data used in the report of a study. ies. For example, Dey13 identified six questions This reality is difficult for any researcher but may intended to help researchers check the quality of be more so for qualitative researchers because of their data: the time and effort it typically takes them to obtain 1. Are the data based on one’s own observation and understand their data. Remember, however, or on hearsay? that the task of interpreting data is to identify the 2. Are observations corroborated by others? important themes or meanings in the data, not 3. In what circumstances was an observation necessarily every theme. made or reported? A final piece of advice regarding data inter- 4. How reliable are those providing the data? pretation is to share your interpretations wisely. 5. What motivations may have influenced a At some time we have all been exposed to what participant’s report? is called a fad, the pendulum swinging in the 6. What biases may have influenced how an opposite direction, the bandwagon effect, and so observation was made or reported? on. As such, many of us may hesitate to embrace anything new or different that comes our way in Qualitative researchers who attend to these guide- schools, calming ourselves with the mantra, “This, lines for conducting credible data analysis and too, shall pass!” If we as researchers attempt to data interpretation are rewarded with trustworthy use our qualitative research findings only as a research reports that withstand the scrutiny of the soapbox from which we present findings that research community. confirm our beliefs and values, then we risk being 13 Qualitative Data Analysis (p. 224), by I. Dey, 1993, New 12 Ibid., p. 41. York: Routledge. 594 chaPtER 20 aNalyziNg aNd iNterPretiNg data summary DATA AnAlYsis AnD inTERPRETATiOn: 9. Reading/memoing is the process of DEfiniTiOn AnD PuRPOsE writing notes in the field note margins and underlining sections or issues that seem 1. Data analysis in qualitative research involves important during the initial reading of summarizing data dependably and accurately. narrative data. The presentation of the findings of the study 10. Describing involves developing thorough thus has an air of undeniability. and comprehensive descriptions of the 2. Data interpretation is an attempt by the participants, the setting, and the phenomenon researcher to find meaning in the data and to studied to convey the rich complexity of the answer the “So what?” question in terms of the research. implications of the study’s findings. 11. Classifying small pieces of data into more 3. A great deal of data analysis occurs before general categories is the qualitative researcher’s data collection is complete. Researchers think way to make sense and find connections about and develop hunches about what they among the data. Field notes and transcripts are see and hear during data collection. broken down into small pieces of data, and 4. An important step in the ongoing analysis these pieces are integrated into categories and of qualitative data is to reflect on two often into more general patterns. questions: a. Is your research question still answerable DATA AnAlYsis sTRATEGiEs and worth answering? b. Are your data collection techniques catch- 12. Identifying themes is a strategy that relies on ing the kind of data you want and filtering the identification of ideas that have emerged out the data that you don’t want? from the review of literature and in the data 5. It is important to avoid premature actions collection. based on early analysis and interpretation of 13. Coding is the process of marking units of data. text with codes or labels as a way to indicate 6. After fieldwork has been completed, the patterns and meaning in data. It involves the researcher must concentrate solely on the reduction of narrative data to a manageable multistage process of organizing, categorizing, form to allow sorting to occur. synthesizing, analyzing, and writing about the 14. Asking key questions is a strategy that involves data. The researcher works to narrow a large the researcher asking questions such as set of issues and data into small and important “Who is centrally involved?” and “What major groups of key data. activities, events, or issues are relevant to the 7. It is difficult to determine, in advance, problem?” and seeking answers in the data. how long data analysis will take. The time 15. An organizational review helps the researcher frame depends on the nature of the study, understand the school or other organization the amount of data to be analyzed, and the as the larger setting. A review should focus on abilities of the researcher. the vision and mission, goals and objectives, structure, operation, and issues and concerns of the organization under study. sTEPs in AnAlYzinG QuAliTATivE 16. Concept mapping allows the qualitative REsEARCh DATA researcher to create a visual display of the 8. Qualitative data analysis is a cyclical, iterative major influences that have affected the study process of reviewing data for common topics to allow for the identification of consistencies or themes. One approach to analysis is to and inconsistencies between disparate groups. follow three iterative steps: reading/memoing, 17. Analyzing antecedents and consequences describing what is going on in the setting, and allows the researcher to map the causes and classifying research data. effects that have emerged throughout the study.  chaPtER 20 aNalyziNg aNd iNterPretiNg data 595 18. Displaying findings involves using matrixes, 26. Contextualizing the findings of the study in charts, concept maps, graphs, and figures to the related literature involves using the review encapsulate the findings of a study. of related literature to provide support for the 19. Stating what’s missing from the study findings of the study. encourages the researcher to reflect and to 27. Turning to theory encourages researchers to identify any questions for which answers have link their findings to broader issues of the not been provided. day and, in so doing, to search for increasing 20. Many computer programs are available to levels of abstraction and to move beyond a aid in analyzing qualitative data, but it is purely descriptive account. important for novice qualitative researchers to 28. Knowing when to say when means that remember that computers do not analyze or the researcher refrains from offering an code data; researchers do. interpretation when he or she can offer only a wimpy interpretation. DATA inTERPRETATiOn sTRATEGiEs 29. As a qualitative researcher, you should share your interpretations wisely and avoid being 21. Data interpretation is based heavily on evangelical about them. Provide a clear the connections, common aspects, and link among data collection, analysis, and linkages among the data pieces, categories, interpretation. and patterns. Interpretation cannot be meaningfully accomplished unless the EnsuRinG CREDiBiliTY in YOuR sTuDY researcher knows the data in great detail. 22. The aim of interpretation is to answer four 30. To check the credibility (and trustworthiness) questions: What is important in the data? Why of their data, qualitative researchers should is it important? What can be learned from it? ask themselves the following six questions: So what? a. Are the data based on one’s own observa- 23. Extending the analysis is a data interpretation tion or on hearsay? strategy in which the researcher raises b. Are observations corroborated by others? questions about the study, noting implications c. In what circumstances was an observation that may be drawn without actually made or reported? drawing them. d. How reliable are those providing the data? 24. Connecting findings with personal experience e. What motivations may have influenced a encourages the researcher to personalize participant’s report? interpretations based on intimate knowledge f. What biases may have influenced how an and understanding of the research setting. observation was made or reported? 25. Seeking the advice of critical friends involves inviting trusted colleagues to offer insights that may have been missed due to the researcher’s closeness to the study.