Qualitative Data Collection and Techniques PDF

chaPtER 19 fieldwork 563 lEaRninG outcoMEs 2. Describe strategies to address the trustworthiness (i.e., vali...

chaPtER 19 fieldwork 563 lEaRninG outcoMEs 2. Describe strategies to address the trustworthiness (i.e., validity) and replicability After reading Chapter 19, you should be able to (i.e., reliability) of qualitative research. do the following: 3. Describe the steps for getting started as a 1. Describe qualitative data collection sources qualitative researcher ready to begin data and techniques. collection or fieldwork. data collection sources the natural environment as lived by participants, without altering or manipulating it. For certain and techniQues research questions, observation is the most appro- After obtaining entry into a setting and selecting par- priate and effective data collection approach. If ticipants, the qualitative researcher is ready to begin you ask teachers how they handle discipline in data collection, also commonly called fieldwork. their classrooms, for example, you run the risk Fieldwork involves spending considerable time in of collecting biased information—they may not the setting under study, immersing oneself in this remember everything, or they may tell you only setting, and collecting as much relevant informa- about their most successful strategies. By observ- tion as possible and as unobtrusively as possible. ing the classes, you will obtain much more objec- Qualitative researchers collect descriptive—narrative tive information that can be compared to the and visual—non-numerical data to gain insights into self-reports of the research participants. The two the phenomenon of interest. Because the data that common types of observation are participant and are collected should contribute to understanding the nonparticipant observation. phenomenon, data collection is largely determined by the nature of the problem. No one recipe explains Participant Observation how to proceed with data collection efforts. Rather, In participant observation, the observer becomes the researcher must collect the appropriate data to part of and a participant in the situation being contribute to the understanding and resolution of a observed. In other words, the researcher partici- given problem. pates in the situation while observing and collect- Observations, interviews, questionnaires, phone ing data on the activities, people, and physical calls, personal and official documents, photographs, aspects of the setting. There are varying degrees recordings, drawings, journals, email messages of participant observation—a researcher can be and responses, and informal conversations are all an active participant observer; a privileged, active sources of qualitative data. Clearly, many sources observer; or a passive observer. of data are acceptable as long as the collection A benefit of participant observation is that approach is ethical, is feasible, and contributes to it allows the researcher to gain insights and an understanding of the phenomenon under study. develop relationships with participants that would The four data collection techniques we discuss in not be possible if the researcher observed but this chapter are observing, interviewing (including did not participate. However, it also has draw- the use of focus groups and email), administer- backs. The researcher may lose objectivity and ing questionnaires, and examining records. These become emotionally involved with participants, for techniques share one aspect: The researcher is the instance, or may have difficulty participating and primary data collection instrument. collecting data at the same time. In cases where the group under study is tight-knit and closely organized, participation may cause tension for observing both the researcher and group members. Before When qualitative researchers obtain data by watch- adopting the role of a participant observer, the ing the participants, they are observing. The researcher must evaluate the likelihood of partici- emphasis during observation is on understanding pating in the situation and gathering the desired 564 chaPtER 19 fieldwork data simultaneously. If it is not feasible for the the likelihood of distortion from the original obser- researcher to be a full participant observer in the vation also increases. group being studied, it is best to be a nonpartici- Field notes are the data that will be analyzed to pant observer. provide the description and understanding of the research setting and participants; they should be as extensive, clear, and detailed as possible. For exam- Nonparticipant Observation ple, a good researcher won’t simply write, “The In nonparticipant observation, the observer is class was happy.” Instead, the researcher should not directly involved in the situation being describe the activities of the students, the looks observed. In other words, the researcher observes on their faces, their interactions with each other, and records behaviors but does not interact the teachers’ activities, and other observations sug- or participate in the life of the setting under gesting the class was happy. It is a good rule of study. Nonparticipant observers are less intru- thumb to avoid words such as good, happy, useful, sive than participant observers and less likely to and other evaluative terms and replace them with become emotionally involved with participants. words describing behaviors that were seen or Nonparticipant observation may also be best if heard. Figure 19.1 illustrates both the descrip- the researcher does not have the background or tive and the reflective aspects of field notes. As needed expertise to act as a true participant or if you read the figure, notice the clarity and level of the group being observed is too closely organized detail in the researcher’s notes as he described the for the researcher to fit in easily. For example, a physical setting, the actions of the students, and the middle-age researcher probably can’t be a true interactions that took place. Each O.C. entry (i.e., participant in a group of fifth graders. However, observer’s comment) represents a reflection that nonparticipant observers may have more difficulty the researcher had while writing the descriptive obtaining reliable information about participants’ field notes. These reflections represent a more per- opinions, attitudes, and emotional states than par- sonal and subjective aspect of the field notes and ticipant observers do. should be distinguished from the descriptive mate- rial in the notes themselves. In the O.C. entries, Recording Observations you can identify times when the researcher noted something unusual, something that had recurred, Whether you are a participant or nonparticipant or something that had to be explored. observer, you will need a method to document your To aid in taking field notes in the setting, observations. Field notes—qualitative research a researcher often brings a protocol, or list of materials gathered, recorded, and compiled (usually issues, to guide observation. Protocols provide the on-site) during the course of a study—are best. researcher with a focus during the observation and Field notes describe, as accurately and as compre- also provide a common framework for field notes, hensively as possible, all relevant aspects of the making it easier to organize and categorize data situation. They contain two basic types of infor- across various sets of notes. For example, a simple mation: (1) descriptive information about what protocol for observation may include the following the observer has directly seen or heard on-site topics: through the course of the study and (2) reflective information that captures the researcher’s personal Who is being observed? How many people are reactions to observations, the researcher’s experi- involved, who are they, and what individual ences, and the researcher’s thoughts during the roles and mannerisms are evident? observation sessions. Because of the need for clar- What is going on? What is the nature of the ity and detail, notes should be made in the field conversation? What are people saying or doing? whenever possible, during the observation. If nec- What is the physical setting like? How are essary, the researcher may record field notes after people seated, and where? How do the leaving the setting, but recording should be done participants interact with each other? as soon as possible—when the interval between What is the status or roles of who leads, who observing and writing field notes becomes longer, follows, who is decisive, who is not? What is  chaPtER 19 fieldwork 565 FiGuRE 19.1 section of field notes showing detailed description and observer’s comments March 24, 1980 Joe McCloud 11:00 a.m. to 12:30 p.m. Westwood High 6th Set of Notes THE FOURTH-PERIOD CLASS IN MARGE’S ROOM I arrived at Westwood High at five minutes to eleven, the time Marge told me her fourth period started. I was dressed as usual: sport shirt, chino pants, and a Woolrich parka. The fourth period is the only time during the day when all the students who are in the “neurologically impaired/learning disability” program, better known as “Marge’s program,” come together. During the other periods, certain students in the program, two or three or four at most, come to her room for help with the work they are getting in other regular high school classes. It was a warm, fortyish, promise of a spring day. There was a police patrol wagon, the kind that has benches in the back that are used for large busts, parked in the back of the big parking lot that is in front of the school. No one was sitting in it and I never heard its reason for being there. In the circular drive in front of the school was parked a United States Army car. It had insignias on the side and was a khaki color. As I walked from my car, a balding fortyish man in an Army uniform came out of the building and went to the car and sat down. Four boys and a girl also walked out of the school. All were white. They had on old dungarees and colored stenciled t-shirts with spring jackets over them. One of the boys, the tallest of the four, called out, “oink, oink, oink.” This was done as he sighted the police vehicle in the back. O.C.: This was strange to me in that I didn’t think that the kids were into “the police as pigs.” Somehow I associated that with another time, the early 1970s. I’m going to have to come to grips with the assumptions I have about high school due to my own experience. Sometimes I feel like Westwood is entirely different from my high school and yet this police car incident reminded me of mine. Classes were changing when I walked down the halls. As usual there was the boy with girl standing here and there by the lockers. There were three couples that I saw. There was the occasional shout. There were no teachers outside the doors. O.C.: The halls generally seem to be relatively unsupervised during class changes. Two black girls I remember walking down the hall together. They were tall and thin and had their hair elaborately braided with beads all through them. I stopped by the office to tell Mr. Talbor’s (the principal) secretary that I was in the building. She gave me a warm smile. O.C.: I feel quite comfortable in the school now. Somehow I feel like I belong. As I walk down the halls some teachers say hello. I have been going out of my way to say hello to kids that I pass. Twice I’ve been in a stare-down with kids pass- ing in the hall. Saying, “How ya’ doin’?” seems to disarm them. I walked into Marge’s class and she was standing in front of the room with more people than I had ever seen in the room save for her homeroom which is right after second period. She looked like she was talking to the class or was just about to start. She was dressed as she had been on my other visits—clean, neat, well-dressed but casual. Today she had on a striped blazer, a white blouse, and dark slacks. She looked up at me, smiled, and said: “Oh, I have a lot more people here now than the last time.” O.C.: This was in reference to my other visits during other periods where there are only a few students. She seems self- conscious about having such a small group of students to be responsible for. Perhaps she compares herself with the reg- ular teachers who have classes of thirty or so. There were two women in their late twenties sitting in the room. There was only one chair left. Marge said to me something like: “We have two visitors from the central office today. One is a vocational counselor and the other is a physical therapist,” but I don’t remember if those were the words. I felt embarrassed coming in late. I sat down in the only chair available next to one of the women from the central office. They had on skirts and carried their pocketbooks, much more dressed up than the teachers I’ve seen. They sat there and observed. Source: from robert c. bogdan & sari knopp biklen, Qualitative Research for Education: An Introduction to Theories and Methods, 5/e. Published by allyn and bacon, boston, Ma. copyright © 2007 by Pearson education. reprinted by permission of the publisher. (continued ) 566 chaPtER 19 fieldwork FiGuRE 19.1 section of field notes showing detailed description and observer’s comments (Continued ) Below is the seating arrangement of the class today: Mark m Pa Marge's Desk ine ax M Jeff Alfred Leroy Physical Vocational Phil Lou Bob Therapist Counselor Me Jason Alfred (Mr. Armstrong, the teacher’s aide) walked around but when he stood in one place it was over by Phil and Jeff. Marge walked about near her desk during her talk which she started by saying to the class: “Now remember, to- morrow is a fieldtrip to the Rollway Company. We all meet in the usual place, by the bus, in front of the main entrance at 8:30. Mrs. Sharp wanted me to tell you that the tour of Rollway is not specifically for you. It’s not like the trip to G.M. They took you to places where you were likely to be able to get jobs. Here, it’s just a general tour that everybody goes on. Many of the jobs that you will see are not for you. Some are just for people with engineering degrees. You’d better wear comfortable shoes because you may be walking for two or three hours.” Maxine and Mark said: “Ooh,” in protest to the walking. She paused and said in a demanding voice: “OK, any questions? You are all going to be there. (Pause) I want you to take a blank card and write down some questions so you have things to ask at the plant.” She began passing out cards and at this point Jason, who was sitting next to me, made a tutting sound of disgust and said: “We got to do this?” Marge said: “I know this is too easy for you, Jason.” This was said in a sarcastic way but not like a strong put- down. O.C.: It was like sarcasm between two people who know each other well. Marge has known many of these kids for a few years. I have to explore the implications of that for her relations with them. Marge continued: “OK, what are some of the questions you are going to ask?” Jason yelled out “Insurance,” and Marge said: “I was asking Maxine not Jason.” This was said matter of factly without anger toward Jason. Maxine said: “Hours— the hours you work, the wages.” Somebody else yelled out: “Benefits.” Marge wrote these things on the board. She got to Phil who was sitting there next to Jeff. I believe she skipped Jeff. Mr. Armstrong was standing right next to Phil. She said: “Have you got one?” Phil said: “I can’t think of one.” She said: “Honestly Phil. Wake up.” Then she went to Joe, the white boy. Joe and Jeff are the only white boys I’ve seen in the program. The two girls are white. He said: “I can’t think of any.” She got to Jason and asked him if he could think of anything else. He said: “Yeah, you could ask ’em how many of the products they made each year.” Marge said: “Yes, you could ask about production. How about Leroy, do you have any ideas, Leroy?” He said: “No.” Mr. Armstrong was standing over in the corner and saying to Phil in a low voice: “Now you know what kinds of questions you ask when you go for a job?” Phil said: “Training, what kind of training do you have to have?” Marge said: “Oh yes, that’s right, training.” Jason said out loud but not yelling: “How much schooling you need to get it.” Marge kept listing them. O.C.: Marge was quite animated. If I hadn’t seen her like this before I would think she was putting on a show for the people from central office.  chaPtER 19 fieldwork 567 the tone of the session? What beliefs, attitudes, Try to enter the field with no preconceptions. values, and so on, seem to emerge? Try to recognize and dismiss your own How did the meeting end? Was the group assumptions and biases and remain open to divided, united, upset, bored, or relieved? what you see; try to see things through the What activities or interactions seemed unusual participants’ perspectives. or significant? Write your field notes as soon as possible. What was the observer doing during the When you’re done, list the main ideas or session? What was the observer’s level of themes you’ve observed and recorded. Don’t participation in the observation (e.g., participant discuss your observation until the field notes observer, nonparticipant observer, etc.)? are written; discussion may alter your initial perspective. Certainly different studies with different par- Include the date, site, time, and topic on ticipants in different settings would have alterna- every set of field notes. Leave wide margins tive protocol questions. The aim here is not to to write your impressions next to sections of be exhaustive but to encourage you to develop the descriptive field notes. The wide margins and refine some form of protocol that will guide also provide space for doing initial coding and you in answering the overarching question, “What analysis. Write on only one side of a page; this is going on here?” Figure 19.2 illustrates a very strategy will save you much photocopying when simple protocol. the time comes to organize the field notes into A protocol is an important tool that provides different categories. Draw diagrams of the site. structure for recording information from observa- List key words related to your observation, then tion sessions. The following guidelines should also outline what you saw and heard. Use the key help you in recording information and organizing words and outline to write your detailed field notes. field notes successfully: Keep the descriptive and reflective sections Start slowly. Do not assume you know what of field notes separate, although collected you’re looking for until you have some experi- together. ence in the setting and spend some time with Write down your hunches, questions, and the participants. insights after each observation. Use memos. FiGuRE 19.2 sample of observation protocol Setting: Individual Observed: Observation #: (first observation, second, etc.) Observer Involvement: Date/Time: Place: Duration of Observation (indicate start/end times): Descriptive Notes Reflective Notes (Detailed, chronological notes about what (Concurrent notes about the observer’s the observer sees, hears; what occurred; thoughts, personal reactions, experiences) the physical setting) 568 chaPtER 19 fieldwork Number the lines or paragraphs of your field interviews is not to get answers to predetermined notes to help you find particular sections when questions but rather to find out the participants’ per- needed. spectives and what they have experienced. Often Enter your field notes into a computer program informal interviews are used further in the study to for future examination and data analysis. obtain more complex or personal information. Agar1 suggested that researchers have a set of questions interviewing ready to ask participants by guiding the conversation around who, what, where, when, why, and how. An interview is a purposeful interaction in which Using these prompts, researchers will never be at a one person obtains information from another. loss for a question to add to their understanding of Interviews permit researchers to obtain important what is happening in the research setting. data they cannot acquire from observation alone, although pairing observations and interviews pro- Structured Interviews vides a valuable way to gather complementary data. Qualitative researchers may also interview research Interviews can provide information that is inacces- participants formally as part of the data collection sible through observation; for example, observation efforts. In a formal, structured interview, the cannot provide information about past events, or researcher has a specified set of questions that the way things used to be before Mr. Hardnosed elicits the same information from all the respon- became principal, or why Ms. Haddit has had it and dents. A major challenge in constructing any inter- is considering transferring to another school. In view, however, is to phrase questions so that they addition, interview questions can derive from obser- elicit the desired information. Although this point vational data—you may see something and want to may seem obvious, qualitative researchers often ask follow-up questions to understand the reasons feel compelled by tradition to ask a lengthy set of behind particular events. Interviewers can explore questions, many of which stray from their focus. and probe participants’ responses to gather in- When planning structured interviews, consider depth data about participants’ experiences and feel- the following options for ensuring interview quality: ings. Researchers can examine attitudes, interests, feelings, concerns, and values more easily than they Include both open-ended (i.e., divergent) can through observation. Interviews may range in and closed (i.e., convergent) questions in a length from a few minutes to a few hours. They may structured interview. A closed question allows consist of a one-time session or multiple sessions for a brief response such as yes or no, whereas with the same participant. In addition, participants an open-ended question allows for a detailed may be interviewed individually or in groups. response and elaboration on questions in ways Interviews are distinguished by their degree you may not have anticipated. The information of formality and structure. Interviews may be gathered through open-ended questions may formal and planned (e.g., “We’ll meet Tuesday be more difficult to make sense of, but this at 1:00 to discuss your perceptions”) or informal type of question allows the researcher to obtain and unplanned (e.g., “I’m glad I caught you in the important information that may otherwise be corridor; I’ve been meaning to ask you...”). Some considered discrepant. interviews are structured, with a specified set of Pilot-test the questions with a group of questions to be asked, whereas others are unstruc- respondents who share similar characteristics tured, with questions prompted by the flow of the with your research participants to see if interview. Semistructured interviews combine both the questions make sense. The participants’ structured and unstructured approaches. feedback will quickly confirm or challenge the assumptions you made while writing your Unstructured Interviews questions (e.g., about appropriate language). The unstructured interview is little more than a Using the feedback from this group, revise the casual conversation that allows the qualitative questions before interviewing your participants. researcher to inquire into something that has pre- sented itself as an opportunity to learn about some- 1 The Professional Stranger: An Informal Introduction to thing at the research setting. The goal of informal Ethnography, by M. H. Agar, 1980, Orlando, FL: Academic Press.  chaPtER 19 fieldwork 569 Guidelines for Interviewing remember the contents of the interview accurately. Recordings are convenient and reliable, and they Although the concept of an interview seems straight- ensure that the original data are available at any forward, it can be a complex and difficult undertak- time. Although a few participants may balk at ing when the gender, culture, and life experiences being recorded, most participants will not, espe- of the interviewer and participant are quite differ- cially if you promise them confidentiality. Make ent. Challenges can include control of the interview sure that the recording machine is in good work- (i.e., who sets the direction or tone), the accuracy ing order with new batteries or has been charged of responses provided, and the extent to which the recently before entering the interview setting. language of the interviewee and the researcher are Following recorded data collection, it is use- similar enough to permit meaningful inferences ful to transcribe the recordings. A transcription is a about the topic under study. For these reasons, a written record of the events that were recorded. This researcher must always take the time to enter the task is time-consuming, especially for long inter- research setting unobtrusively and build support views. Transcribing one 60-minute recording may and trust with participants before initiating an take 4 or 5 hours. If you do the transcribing instead interview. A trusting relationship is essential if par- of hiring someone (a costly alternative), write the ticipants are to answer questions—particularly about date, subject discussed, and participant (using a sensitive issues—with candor. coded name) on the transcript. Number all pages. The following actions can help improve com- Make sure a different indicator is given and used to munication and facilitate the collection of inter- identify the various persons speaking. view data: The transcripts are like field notes for interview Listen more; talk less. Listening is the most data. They should be reviewed against the record- important part of interviewing. ing for accuracy. Interview transcripts are volumi- Don’t interrupt. Learn how to wait. nous and usually have to be reduced to focus on Tolerate silence. It means the participant is the data pertinent to the study, although sometimes thinking. this type of focus is difficult to achieve. During data Avoid leading questions; ask open-ended analysis, the transcript should be read and impor- questions. tant sections labeled to indicate their importance. Keep participants focused and ask for concrete details. Focus Groups Follow up on what participants say, and ask Another valuable interview technique is the use of questions when you don’t understand. focus groups, which can include several individuals Don’t be judgmental about participants’ views who can contribute to your understanding of your or beliefs; keep a neutral demeanor. Your research problem. A focus group is like a group purpose is to learn about others’ perspectives, interview where you are trying to “collect shared whether you agree with them or not. understanding from several individuals as well as to Don’t debate with participants over their get views from specific people.”2 Focus groups are responses. You are a recorder, not a debater. particularly useful when the interaction between Collecting Data from Interviews individuals will lead to a shared understanding of the questions posed by a teacher-researcher. Interviewers have three basic choices for collect- When conducting focus groups, it is important ing their data: taking notes during the interview, to ensure that all participants have their say and writing notes after the interview, and recording to nurture a group agreement to take turns; that the interview. Although all these approaches can is, participants should understand that the focus be used in a study, the data collection method of group is a group-sharing activity and not some- choice is audio or video recording, which pro- thing to be dominated by one or two participants. vides a verbatim account of the session. Taking notes during the interview can be distracting and 2 can alter the flow of the session. Writing notes Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research, (5th Ed.), by J. W. after the interview is better than trying to write Creswell (5th ed., p. 217), 2015, Upper Saddle River, NJ: Pearson during the interview, but it can be difficult to Education, Inc. 570 chaPtER 19 fieldwork Using a structured or semistructured interview (i.e., junk email) we receive that has been for- schedule, a researcher can pose questions to the warded from someone else’s computer. When group and encourage all participants to respond. these concerns are addressed, such interviews can To use sporting metaphors, use a basketball versus be a useful tool to use in your research setting. a table tennis questioning style: Ask the question, elicit a response, and pass it off to another par- Questionnaires ticipant (i.e., basketball) versus ask the question, Although face-to-face interviews provide intimate accept the response, and ask another question opportunities for the researcher to know how each (i.e., table tennis). Get as much information out respondent feels about a particular issue, inter- of each question as you possibly can and, in the viewing is very time-consuming. A compromise is process, ensure that all group participants have an to use a questionnaire. A questionnaire is a written opportunity to respond. collection of self-report questions to be answered Ideally, the qualitative researcher conducts an by a selected group of research participants. The interview to capture the responses from the focus major difference between a questionnaire and an group and later transcribe the discussion. This pro- interview is that, with a questionnaire, the par- cess is also time-consuming—perhaps even more so ticipant writes the responses on the form provided. than it is for individual interviews, so be prepared Questionnaires allow the researcher to collect to allocate time to ferreting out the nuances of the large amounts of data in a relatively short amount focus group interview and the shared understand- of time. Researchers often administer question- ings that emerge. naires and then conduct follow-up interviews with research participants who provided written feed- Email Interviews back that warrants further investigation. Because a valid and reliable data collection Another approach to interviewing that can be used instrument ensures useful responses, you should effectively by qualitative researchers is the email consider the following guidelines for developing interview. Increasingly, educational environments in and presenting questionnaires: which you are likely to be conducting your research provide email and Internet access for teachers and Avoid a sloppy presentation. Make the que- students. Therefore, it is likely that you will be able stionnaire attractive, and consider using large to use email to interview your research partici- print if necessary. pants. For busy teachers, for example, engaging in Carefully proofread questionnaires before send- an ongoing conversation using email may be less ing them out. Nothing will turn respondents off intrusive—busy professionals can respond to email more quickly than a questionnaire littered with either synchronously (i.e., akin to a real-time con- errors. versation with the researcher) or asynchronously Avoid a lengthy questionnaire. Pilot testing the (i.e., at some other time when researcher and par- instrument will provide a realistic sense of ticipant are not both sitting at their computers). how long respondents need to complete the Email interviews have both pros and cons. For task. Remember that, no matter how much res- example, one advantage of the email interview pondents want to help you, a questionnaire is that you don’t have to transcribe a recorded that’s too long will find its way into the circular interview with a colleague or student—the tran- file instead of back in your hands. scription of the interview has already been done Do not ask unnecessary questions. Asking for you by the respondent! However, researchers unnecessary questions is akin to teachers must be attentive to the ethical issues associated developing tests that don’t match the course with assuring respondents that their text responses content—students are stumped, and teachers will be confidential and anonymous. Most of us learn little about the effectiveness of their are not experts when it comes to technology, but lessons. Likewise, as researchers, we often in general we have concerns that email directed to feel compelled to ask a great deal of trivial someone in particular may be sitting on a server information on a questionnaire. Participants somewhere, accessible to other curious folks! This become frustrated, and we collect information worry is further enhanced by the amount of spam that is tangential to our stated purpose.  chaPtER 19 fieldwork 571 Use structured items with a variety of possible on. With permission, the qualitative researcher can responses. Indicate what you mean by words use these sources of data to gain valuable histori- like often and frequently, and be clear as cal insights, identify potential trends, and explain to how they differ from each other (e.g., how things got to be the way they are. Clerical Do you buy chocolate often? versus Do you assistants, school aides, and student teachers are buy chocolate once a week?). Otherwise, often happy to help with uncovering archival respondents may interpret the meaning of the information and organizing it in a way that is most terms in quite different ways than you intended. useful to the classroom teacher if they believe that Whenever possible, add an Additional the data contribute to the collective understanding Comments section. An Additional Comments of a pressing educational issue. section provides respondents with an opportunity to respond openly to your Journals questions and to raise new questions. These Daily journals kept by teachers provide firsthand comments provide an excellent source of accounts of what is happening in the classroom discrepant data (e.g., “I hadn’t expected a and provide a glimpse of the school from another research participant to say that he or she buys perspective. Students’ journals can provide teach- chocolate every day!”) and an opportunity to ers with a window into the students’ world and follow up with informal interviews to elicit their daily classroom experiences, which can affect more information from the respondents as your future teaching practices meaningfully. Regardless time, energy, and inquisitiveness allow. of your specific research questions, you should Decide whether you want respondents to put encourage journaling by research participants as a their names on the questionnaires or whether way to record their perceptions of what is happen- you will use a number to keep track of who has ing in the research setting. responded. You should assure respondents that their confidentiality will be protected throughout Maps the process. However, you can protect respondents while keeping track of who has Qualitative researchers working in schools often responded and deciding whether they have made find class maps and school maps useful for a num- comments that you feel warrant a follow-up ber of reasons. They provide contextual insights for conversation. If you track respondents’ identities, people who have not visited the school, and they you must communicate that they will not suffer provide the qualitative researcher with a reflective any negative consequences for anything they tool—a way of rethinking the way things are in may share with you; this communication is schools and classrooms. For example, why are the necessary to ensure that they feel comfortable computers in the classroom placed in a bank along supplying honest and forthright answers. one wall, and what are the effects of individual student computer time on other seatwork activities? A class map can record traffic flow in a classroom Examining Records as well as teacher movement during instruction; the Qualitative researchers examine various types of school map may also prove useful for teams of qual- records or documents, including archival documents, itative researchers concerned about the movement journals, maps, video and digital recordings, and arti- and interactions of different grade levels of students facts. Many of these data sources are naturally occur- and any problems that emerge from the traffic flow. ring in educational settings and require only that the For qualitative researchers, context is everything! researcher locate them within the research setting. Figure 19.3 shows an example of a classroom map. Archival Documents Video and Digital Recordings Like classrooms, schools are repositories for all Video and digital recordings provide qualitative sorts of records—student records, standardized test researchers with another valuable, although some- scores, retention rates, minutes of meetings (e.g., what obtrusive, data source. Of course, these tech- faculty, PTA, school board), newspaper clippings niques have some downsides. For example, funny about significant events in the community, and so faces and bizarre comments sometimes appear as 572 chaPtER 19 fieldwork what is happening in classrooms and schools. FiGuRE 19.3 Example of a classroom map Artifacts can include almost everything else that we haven’t already discussed. For exam- ple, schools have tended to move toward what Computers administrators refer to as authentic assess- ment techniques, including the use of student Teacher’s portfolios. A portfolio is a presentation of Desk work that captures an individual student’s work samples over time to demonstrate that Overhead student’s growth. Portfolios, although difficult Projector to quantify, provide the teacher with valu- able outcome data that get at the heart of the Students’ qualitatively different samples of work. Such Desks artifacts are valuable data sources that qualita- B tive researchers may use as starting points for e conversation with research participants. n Digital research tools such as wikis, blogs, c h and Skype can contribute to your qualitative data collection strategies in an ever-changing digital environment (see the accompanying Digital Research Tools for the 21st Century Sink feature). These tools allow the qualitative Cupboards researcher to interact with research partici- Coat Rack pants (particularly those who are geographi- Source: Mills, geoffrey, Action Research: A Guide for the Teacher cally remote) in ways that were previously Researcher, 5th edition, © 2014, p. 87. reprinted by permission of Pearson not thought possible. education, inc., upper saddle river, Nj. a result of the presence of such technology in a classroom for the first time. However, technologies Validity and reliability such as digital cameras and smart phone cameras in QualitatiVe research provide excellent, unobtrusive digital options that are compact, convenient, and in high-definition to boot! validity in Qualitative Research These unobtrusive cameras are far less likely to elicit In qualitative research, validity is the degree to outlandish behavior from participants, and they are which qualitative data gauge accurately what we part of the researcher’s toolkit that can always be are trying to measure. For example, teachers may close at hand. Similarly, most cell phones incorpo- ask, “Are the results of this standardized test really rate digital voice recorder technology that provides valid?” Or teachers may comment, “My students did another useful tool for spontaneous interviews or poorly on the history test I gave them, but I’m not recording researcher’s reflections immediately follow- sure it’s an accurate representation of what they ing fieldwork. However, the use of digital recording really know.” Both scenarios raise issues about the raises the serious issue of time allotment. Watching validity of the data, that is, whether or not the data and listening to digital records and then recording reflect what they were intended to measure. observations require an enormous amount of time. Historically, validity has been linked to numeri- This time commitment is perhaps the number-one cally based quantitative research. However, as quali- challenge for researchers using these data sources. tative research became more popular in the late 1970s and early 1980s, qualitative researchers felt Artifacts pressure to justify and defend the accuracy and cred- Schools and classrooms are rich sources of what ibility of their studies. Two terms commonly used to we call artifacts—written or visual sources of describe validity in qualitative research are trustwor- data that contribute to our understanding of thiness and understanding. Qualitative researchers  chaPtER 19 fieldwork 573 digital research tools for the 21st century Wiki, BloG, and skYPE In addition to survey tools (e.g., SurveyMonkey in chronological order. It is possible to embed and Qualtrics), email, and digital recordings, new photos and videos within your blog as part of web-based tools can provide the qualitative re- the creation of a collaborative research space. A searcher with additional data collection strategies. popular, free blog provider is Blogger (blogger. com), and getting started is as simple as creating Wiki an account and following the template designer. A wiki is a website that allows the easy creation Blogs provide an excellent opportunity for engag- of a web page that can provide a forum for col- ing research participants in a collaborative, secure laborative research. For example, you can create conversation. a wiki focused on an aspect of your research and invite participants (your sample) to a secure set- Skype ting where they can participate in an open forum. Skype is a proprietary software application that Similarly, you can provide links to other websites, allows users to make voice calls and to video- as well as use the wiki as a communication tool conference over the Internet. Founded in 2003 by through which you can communicate with the Estonian, Danish, and Swedish developers, Skype study’s participants. One popular, free wiki pro- boasted 663 million registered users by 2010. By vider is Wikispaces (wikispaces.com). Creation of that same year, 13% of all international call min- a wiki is simple and secure and provides a new utes (54 billion minutes) were reportedly Skype data collection possibility for qualitative research- calls. In short, with the proliferation of computers ers who wish to invite research participants into a with cameras and cell phones with forward-facing collaborative forum. cameras (e.g. the iPhone 5 ®), Skype provides a relatively inexpensive (if not free) qualitative data Blog collection tool for researchers who are unable to A blog (a blend of the term web log) is a type of visit a research site personally to observe or who personal website where you can share informa- wish to interview research participants who are tion about your research and solicit responses located at a distance that makes face-to-face con- from participants while capturing the discourse tact prohibitive. can establish the trustworthiness of their research people. Therefore, qualitative researchers should by addressing the credibility, transferability, depend- include as much detail as possible so others can see ability, and confirmability of their studies and find- the setting for themselves. The researcher should ings.3 First, a researcher must take into account all also address the stability of the data collected (i.e., the complexities in the study and address problems dependability) and the neutrality and objectivity of that are not easily explained (i.e., credibility). The the data (i.e., confirmability). researcher should also include descriptive, context- Let’s look at criteria for measuring the quality relevant statements so that someone hearing about of qualitative research based on these two terms, or reading a report of the study can identify with trustworthiness and understanding, and then look the setting (i.e., transferability). Remember, qualita- at strategies for increasing the validity of your tive researchers believe that everything they study qualitative research. is context-bound; they do not seek to draw conclu- sions that can be generalized to larger groups of Guba’s Criteria for Validity of Qualitative Research 3 Guba’s article “Criteria for Assessing the Trust- “Criteria for assessing the trustworthiness of naturalistic inquiries,” by E. G. Guba, 1981, Educational Communication and worthiness of Naturalistic Inquiries” (1981) speaks Technology Journal, 29, pp. 75–91. directly to qualitative researchers. Guba argued 574 chaPtER 19 fieldwork that the trustworthiness of qualitative inquiry descriptive, context-relevant statements, Guba pro- could be established by addressing the following posed that the researcher should: characteristics of a study: credibility, transferabil- ity, dependability, and confirmability. Collect detailed descriptive data that permit comparison of a given context (classroom or Credibility The credibility of the study refers to school) to other possible contexts to which the researcher’s ability to take into account the transfer might be contemplated. complexities that present themselves in a study and Develop detailed descriptions of the context to to deal with patterns that are not easily explained. make judgments about fit with other contexts To do this, Guba suggested that the following possible. methods be used to establish credibility: The transferability of a qualitative research Do prolonged participation at the study account depends largely on whether the consumer site to overcome distortions produced by of the research can identify with the setting. Include the presence of researchers and to provide as much detail as possible to allow the recipients of researchers with the opportunity to test your work to “see” the setting for themselves. biases and perceptions. Do persistent observation to identify pervasive Dependability According to Guba, dependability qualities as well as atypical characteristics. refers to the stability of the data. To address issues Do peer debriefing to provide researchers with related to the dependability of the data we collect, the opportunity to test their growing insights Guba recommended the following steps: through interactions with other professionals. Overlap methods. This step is similar to a trian- For example, most of us will be able to identify gulation process. Use two or more methods so a “critical friend,” a colleague, or a “significant that the weakness of one is compensated by other”—somebody who is willing and able the strength of another. For example, interviews to help us reflect on our own situations by with students may be used to contribute to your listening, prompting, and recording our insights understanding of what you observed during a throughout the process. lesson. Practice triangulation to compare a variety of Establish an audit trail. This process makes data sources and different methods with one it possible for an external “auditor” (maybe a another in order to cross-check data. critical friend, principal, or graduate student) Collect documents, films, digital recordings, to examine the processes of data collection, artifacts, and other “raw” or “slice-of-life” data analysis, and interpretation. This audit trail may items. Do member checks to test the overall report take the form of a written description of each process and perhaps even access to original with the study’s participants before sharing it in field notes, artifacts, digital-recordings, pictures, final form. archival data, and so on. Establish structural corroboration or coherence to ensure that there are no internal conflicts or Confirmability The final characteristic that Guba contradictions. addresses is the confirmability of the data, or Establish referential adequacy; that is, test the neutrality or objectivity of the data that have analyses and interpretations against documents, been collected. Guba argues that the following two recordings, films, and the like, that were steps can be taken to address this issue: collected as part of the study. Practice triangulation, whereby a variety Transferability Guba’s second criteria of trans- of data sources and different methods are ferability refers to qualitative researchers’ be- compared with one another to cross-check data. liefs that everything they study is context-bound Practice reflexivity, that is, intentionally reveal and that the goal of their work is not to develop underlying assumptions or biases that cause you, “truth” statements that can be generalized to larger as the researcher, to formulate a set of questions groups of people. To facilitate the development of in a particular way and to present findings in a  chaPtER 19 fieldwork 575 taBlE 19.1 Guba’s criteria for ensuring the validity of qualitative research Criteria Definition strategies Credibility The researcher’s ability to take into account all of Do prolonged participation at study site. the complexities that present themselves in a study and to deal with patterns that are not easily Do persistent observation. explained. Do peer debriefing. Practice triangulation. Collect “slice-of-life” data items. Do member checks. Establish structural corroboration or coherence. Establish referential adequacy. Transferability The researcher’s belief that everything is Collect detailed descriptive data. context-bound. Develop detailed descriptions of the context. Dependability The stability of the data. Overlap methods. Establish an audit trail. Confirmability The neutrality or objectivity of the data collected. Practice triangulation. Practice reflexivity. source: from “criteria for assessing the trustworthiness of Naturalistic inquiries,” by e. g. guba, 1981, Educational Communication and Technology Journal, 29(1), pp. 75–91. adapted with permission. particular way. One technique for doing this is to questions elicit painfully brief replies, and we are keep a journal in which reflections and musings left wondering what to do next. Many research- are recorded on a regular basis. ers are in the business of talking for a living, so it comes quite naturally to us to jump in with our own answer for the child. The trustworthiness of Wolcott’s Strategies for Ensuring the our inquiries will be enhanced if we can bite our Validity of Qualitative Research tongue, think of some other probing questions, Taken in concert with the previous discussion and wait patiently (try counting one thousand... about validity criteria, the following strategies pro- two thousand... three thousand...). The best advice vide researchers with practical options for making to researchers is to be patient and allow respon- sure their research is the best it can be.4 dents time to respond. Avoid being your own best informant. Talk little; listen a lot This strategy suggests that researchers should carefully monitor the ratio Record Observations Accurately When conduct- of listening to talking when they conduct inter- ing classroom research, recording observations views, ask questions, or engage children, parents, while you are teaching is nearly impossible. and colleagues in discussions about the prob- However, you should record observations as soon lem being studied. For example, interviewing chil- as possible following a teaching episode to capture dren can be difficult work—our best thought-out the essence of what transpired. Although digital recordings can assist with our efforts to record ac- curately, there will still be many occasions when, 4 Wolcott, H.F. (1994). Transforming Qualitative Data: Descri- as participant observers, we have to rely on our ption, Analysis and Interpretation. Thousand Oaks, CA: Sage. field notes, our journals, or our memories. And for 576 chaPtER 19 fieldwork some of us, relying on memory is becoming an Be Candid Researchers should be candid about increasingly scary thing! their work. If they are writing a narrative that they hope to publish or share with a broader audience, Begin Writing Early In a workday that is already they should make explicit any biases that they crunched by the pressures of time, finding time may have about the inquiry they have undertaken. to write in journals is often difficult. However, Researchers should also make explicit the things if we rely solely on our memories of what has about which they have made judgments because it been happening in our classrooms over an ex- is easy to slip into a narrative that seeks to validate tended period of time, we are likely to fall victim one’s position. Being candid may also provide an to writing romanticized versions of classroom opportunity to be explicit about events that oc- and school life. Make time to write down your curred during the study and that may have affected reflections. The act of writing your recollections the outcomes. For example, high student turnover of a teaching episode or observation will make rates may provide an explanation for fluctuating evident to you what blanks need to be filled in, test scores. for example, what questions need to be asked the next day or what should be the focus of your seek feedback It is always a good idea to seek observations. feedback from colleagues (and perhaps even stu- dents, parents, volunteers, and administrators) on let Readers “see” for Themselves Include pri- a written report. Other readers help to raise ques- mary data in any account to let the readers of your tions about what you as the writer have taken for action research accounts (colleagues, principals, granted. They raise questions about the accuracy university professors) see the data for themselves. of the account and help you to go back to your As Wolcott suggests, “In striking the delicate balance classroom in your quest to get the story right (or at between providing too much detail and too little, I least, not all wrong). would rather err on the side of too much; con- versely, between overanalyzing and underanalyzing Write Accurately Once you have written a descri- data, I would rather say too little”5. This is particu- ption of your research, it is a good idea to read the larly true in a schoolwide action research effort in account aloud or to ask a close colleague to read which you are seeking support for possible change the account carefully to look for contradictions in based on data that you must present to colleagues the text. Often we are too close to the investigation who may not have had a central role in the conduct to see the contradictions that may be blatantly ob- of the study. When sharing your research reports vious to an outsider. Nevertheless, the accuracy of with colleagues, let them see the data. This may the account (whether written or “performed”) is mean using charts, graphs, photographs, film— critical to the validity of the study.6 whatever you have collected. In doing so, you will bring the recipient of your work along in the pro- cess and perhaps earn their buy-in. Showing can be Reliability in Qualitative Research more persuasive than telling. Reliability is the degree to which study data con- sistently measure whatever they measure. Although Report fully In our quest to find neat answers and the term reliability is usually used to refer to solutions to our problems, it is often easy to avoid instruments and tests in quantitative research, keeping track of discrepant events and data. Just qualitative researchers can also consider reliabil- when we think we know the answer, some data ity in their studies, in particular the reliability come along to shatter the illusion of having neatly of the techniques they are using to gather data. resolved the problem! We do not need to be fear- For example, as qualitative researchers examine ful of discrepant data. After all, it is all grist for the the results of their inquiry, they should consider research mill. Although we do not need to report whether the data would be collected consistently if everything, it is helpful to keep track of the discrep- the same techniques were utilized over time. ant data and to seek further explanation. 6 For further discussion of these points and a discussion of “When It Really Matters, Does Validity Really Matter?” see 5 Wolcott, 1994, p. 350. Wolcott, pp. 348–370.  chaPtER 19 fieldwork 577 Reliability, however, is not the same as valid- for the first time always means uncertainty in a ity. Remember, a valid test that measures what it new role—uncertainty about how to act and inter- purports to measure will do so consistently over act with others. Qualitative research, by its very time, but a reliable test may consistently measure nature, is a very intimate and open-ended activity, the wrong thing. and it is common to feel nervous as you learn the ropes, try to establish rapport with participants, Generalizability and get a feel for the setting. Historically, research in education has concerned Bogdan and Biklen7 suggested a number of itself with generalizability, a term that refers to cautions to make the initial days of entry into the the applicability of findings to settings and con- setting less painful: texts different from the one in which they were Set up your first visit so that someone can obtained. That is, based on the behavior of a small introduce you to the participants. group (i.e., a sample) of individuals, researchers Don’t try to accomplish too much in the try to describe, explain, or predict the behavior of first few days. Make your initial visit for a larger group (i.e., the population) of people. This observation short. You will have to take view of generalizability, however, is not directly field notes after each data collection applicable to qualitative research. encounter, so start with brief data collection The goal of qualitative research is to understand episodes to ease into the process of writing what is happening and why. Therefore, qualita- field notes. tive researchers are less concerned than quantita- Ease your way into the context; don’t tive researchers about the generalizability of their storm in. Be relatively passive. Ask general, research. Qualitative researchers are not seeking to nonspecific, noncontroversial questions that define ultimate truths or solutions to problems that allow participants to reply without being can be transferred from a unique setting or sample forced to provide answers they may find to a broader population. Qualitative researchers uncomfortable discussing with a relative do not believe that the only credible research is stranger. The intent is for the participants to that which can be generalized to a larger popula- become comfortable with you gradually, and tion. The power of qualitative research is in the you with them. Then you can increase your relevance of the findings to the researcher or the degree of involvement. audience of the research, although the findings Be friendly and polite. Answer questions that may have some applicability or transferability to a participants and others ask, but try not to say similar setting. too much about the specifics of your presence Armed with your research questions and the and purpose so that you do not influence the qualitative data collection techniques that will help participants. you understand what is going on at your research Do not take what happens in the field setting, you are ready to enter the field and start personally. collecting data. This proposition can be scary for new researchers. Following are suggestions to In short, it is critical that you establish your help smooth your entry into your first qualitative “OKness” with the research participants with research setting. whom you will be working. Regardless of how well-thought-out your study is, if your inter- personal skills are lacking, it will be difficult to getting started develop the trust you need to be accepted into the setting. Having obtained entry into a setting and having selected participants, the qualitative researcher is ready to begin data collection, or fieldwork. Regardless of how much you read, think about, 7 Qualitative Research in Education, by R. C. Bogdan and S. K. and discuss fieldwork, you will not know what it Biklen (3rd ed., pp. 79–81), 1998, Needham Heights, MA: Allyn is really like until you live it. Living an experience & Bacon. 578 chaPtER 19 fieldwork researcher to inquire into something that has summary presented itself as an opportunity to learn about what is happening at the research setting. DATA COllECTiOn sOuRCEs 12. In a structured interview, the researcher has a AnD TEChniQuEs specified set of questions that elicits the same 1. Qualitative data collection, or fieldwork, information from all respondents. involves spending considerable time in the 13. For interviews, researchers should include setting under study, immersing oneself in this convergent and divergent questions and pilot- setting, and collecting relevant information test them with a group of respondents similar unobtrusively. Descriptive narrative and visual to the target sample. data are collected to gain insights into the 14. Following basic guidelines for interviewing phenomena of interest. can help improve communication and can 2. The type of data collected is largely facilitate data collection. determined by the nature of the problem. 15. Interviewers should take notes during the 3. Qualitative research includes data interview, write notes after the interview, or collected through observations, interviews, (preferably) record the interview and later questionnaires, phone calls, personal and transcribe it. official documents, photographs, recordings, 16. A focus group is a group interview. drawings, journals, email messages and Researchers conducting focus groups should responses, and informal conversations. ensure that all participants have a chance to 4. In qualitative research, the researcher is the state their points of view. primary data collection instrument. 17. An email interview can be used to elicit responses from busy professionals who can observing respond to an email either synchronously or asynchronously. 5. When qualitative researchers obtain data by watching the participants, they are observing. 6. A researcher who becomes part of and a Questionnaires participant in the situation under observation 18. A questionnaire is a written collection of self- is called a participant observer. report questions to be answered by a selected 7. A researcher can be an active participant group of research participants. observer; a privileged, active observer; or a 19. Developing and presenting questionnaires passive observer. takes care; questions should be relevant, 8. A nonparticipant observer observes the and the presentation should be attractive. situation but does not participate in the Be sure to protect participants’ confidential situation while observing it. information. 9. Field notes are the records of what the observer has seen or heard. Field notes contain literal examining records descriptions as well as personal reactions and 20. Useful educational records include archival comments on what the observer has experienced documents, journals, maps, video- and audio- and thought about during an observation recordings, and artifacts. session. Field notes may be guided by a protocol developed prior to the observation session. vAliDiTY AnD REliABiliTY in QuAliTATivE REsEARCh interviewing 10. An interview is a purposeful interaction in which one person obtains information from validity in Qualitative research another. 21. Validity is the degree to which qualitative 11. The unstructured interview is like a casual data gauge accurately what we are trying to conversation and allows the qualitative measure.  chaPtER 19 fieldwork 579 22. Qualitative researchers can establish are not seeking ultimate truths. The power of the trustworthiness of their research by qualitative research is in the relevance of the addressing the credibility, transferability, findings to the researcher or the audience of dependability, and confirmability of their the research. studies and findings. GETTinG sTARTED reliability in Qualitative research 26. Set up your first visit to the research setting so 23. Reliability is the degree to which study data that someone is there to introduce you to the measure consistently whatever they measure. participants. 24. A valid test that measures what it purports to 27. During the first few days in the setting, don’t measure will do so consistently over time, but try to accomplish too much, be relatively a reliable test may consistently measure the passive, and be friendly and polite. wrong thing. 28. Do not take what happens in the field personally. Generalizability 25. Qualitative researchers do not generally worry about the generalizability of data because they