Rm Unit 2 PDF

58 | Research Approaches and Data Collection Methods FIGURE 2.1 Amount of time for a store employee to approach and acknowledge the confederate. 80...

58 | Research Approaches and Data Collection Methods FIGURE 2.1 Amount of time for a store employee to approach and acknowledge the confederate. 80 Employee 70 approaches a shopper Amount of time in seconds 60 50 40 30 20 10 0 Formal Informal clothing clothing (Adapted from “Customer Service as a Function of Shopper’s Attire” by P. C. Regan & V. Llamas (2002), Psychological Reports, 90, pp. 203–204.) clothing. As soon as an employee approached and spoke to her, she stopped the timer. As Figure 2.1 indicates, females dressed in formal work clothes were ap- proached more quickly by store employees than were females dressed in informal sports clothing. This is an example of a field study because it was conducted in the natural setting of a mall while engaging in daily activities. It is also an experimental research study because variable manipulation was present (type of dress). Field experiments like this one are not subject to the artificiality problem that exists with laboratory experiments, so field experiments are excellent for studying many problems. Their primary disadvantage is that control of extraneous vari- ables cannot be accomplished as well as it can be in laboratory experiments. For example, in field settings, the researcher cannot prevent participants in com- parison groups from communicating, and the researcher cannot prevent par- ticipants from engaging in other activities that might affect their scores on the dependent variable. Tunnell (1977) states that field experimentation should be conducted in a manner that makes all variables operational in real-world terms. The Regan and Llamas (2002) study included the three dimensions of naturalness identified by Tunnell: natural behavior, natural setting, and natural treatment. The natural be- havior investigated was a store employee approaching a shopper. The setting was natural because the study took place in a mall; the natural treatment was type of dress. In reality, the treatment was imposed by a confederate, but it mirrored a behavior that could have occurred naturally. These are characteristics Tunnell says we must strive for when we conduct field experimentation. M02_CHRI7743_12_GE_C02.indd 58 3/31/14 5:43 PM  Experimental Research Settings | 59 Laboratory Experiments Laboratory The laboratory experiment is a research study that is conducted in the labora- experiment tory, and the investigator precisely manipulates one or more independent variables An experimental and controls the influence of all, or nearly all, extraneous variables. For example, research study that Kassin and Kiechel (1996) realized that there were police reports of individuals is conducted in the who confessed to crimes that they had not committed. They realized that there was controlled environ- no scientific evidence of this phenomenon and were interested in determining if ment of a laboratory they could experimentally demonstrate that vulnerable individuals, under the right circumstances, would confess to an act that they did not commit and internalize this confession to the point that they would confabulate details in memory con- sistent with the confession. To investigate this phenomenon, Kassin and Kiechel constructed a situation in which they manipulated the vulnerability of the research participants as well as the presence of a person falsely incriminating them. In ad- dition, they controlled other variables such as the presence of witnesses and other individuals refuting or confirming the false accusation. To precisely manipulate vul- nerability and the presence of a witness and to control for the impact of extraneous variables, Kassin and Kiechel created a situation within the context of a laboratory setting in which the research participants had to perform a task at either moderate or rapid speed. A rapid-speed completion of the task created a vulnerable condition because the more rapidly the participants had to respond, the greater the likelihood of making a mistake. The results of this study revealed that individuals were more likely to confess to making a mistake they had not made in the vulnerable condi- tion when a confederate, or witness, said that the research participant had made the error. More important, these vulnerable individuals were more likely to inter- nalize the false confession and tell others that they had committed the error. In contrast to the field experiment, the laboratory experiment epitomizes the abil- ity to control or eliminate the influence of extraneous variables. This is accomplished by bringing the problem into an environment apart from the participants’ normal routines. In this environment, outside influences (such as the presence of others and of noise) can be eliminated. However, the price of this increase in control is the artifi- ciality of the situation created. Even though precise results can be obtained from the laboratory, the applicability of these results to the real world must always be verified. Internet Experiments Internet experiment An Internet experiment is an experimental study that is conducted over the An experimental study Internet. As with all types of experiments, the investigator precisely manipulates that is conducted over one or more variables and controls for as many extraneous confounding variables the Internet as possible. The precursor to conducting experiments over the Internet was probably the incorporation of computer automation in experimental research in psychology. As early as the 1970s, researchers were making use of computers in psychological ex- periments to perform tasks such as delivering a standardized and controlled pre- sentation of stimuli and making accurate recordings of responses. Currently, most human experimental research in psychology is aided by computer automation. M02_CHRI7743_12_GE_C02.indd 59 3/31/14 5:43 PM  60 | Research Approaches and Data Collection Methods The move to conduct human psychological experimentation on the Internet was made possible by the development, in 1990, of a new protocol, http, or hypertext transfer protocol. This allowed an Internet browser, such as Netscape Navigator or Internet Explorer, to get a document it located on a server. This document, or Web page, is coded in a language known as hypertext markup language, or HTML, and this language permits the display of text, graphics, or other information on a Web page. With the ability to display such a combination of words, pictures, and sounds on Web pages, the Web grew at an astonishing rate. In 1997, Krantz, Ballard, and Scher conducted an Internet experiment investigating the determinants of female attractiveness and published it in a scientific journal (Musch & Reips, 2000). This was one of the first published Internet experiments. Since that time the number of Internet experiments has grown considerably, and this growth rate is expected to continue because the advantages seem to out- weigh the disadvantages compared to other types of experiments (Birnbaum, 2001). Some advantages identified by Reips (2000, p. 89) include the following: “(1) ease of access to demographically and culturally diverse participant populations, including participants from unique and previously inaccessible target populations; (2) bringing the experiment to the participant instead of the opposite; (3) high statistical power by enabling access to large samples; (4) the direct assessment of motivational confounding by noting the differential dropout rate between treatment conditions because participants in Web experiments are not induced to stay due to, for example, course credit (the italics are ours); (5) cost savings of lab space, person-hours, equipment, and admin- istration.” The disadvantages identified by Reips (2000, p. 89) include issues “such as (1) multiple submissions, (2) lack of experimental control, (3) self-selection, and (4) dropout.” Of these disadvantages, the most significant is lack of experimental control. However, as we will emphasize in later chapters, random assignment to ex- perimental conditions is the most important technique to be included in the design of an experimental study. Reips (2000) points out that this technique can be incor- porated into the design of an Internet experiment with the use of “so-called CGIs, small computer programs that cooperate with the Web Server” (p. 107). STUDY QUESTION 2.5 What are the different research settings in which experimental research is conducted, and what are the advantages and disadvantages of each setting? Nonexperimental Quantitative Research The defining characteristic of nonexperimental quantitative research is that Nonexperimental there is no manipulation of an independent variable. Typically, this is a descriptive quantitative research type of research in which the goal is to provide an accurate description or picture Type of quantitative of a particular situation or phenomenon or to describe the size and direction of research in which the relationships among variables. More advanced and sophisticated nonexperimental independent variable approaches attempt to identify causal relationships through attempting to estab- is not manipulated by lish time ordering of the independent and dependent variables and controlling for the researcher extraneous variables identified by the researcher. M02_CHRI7743_12_GE_C02.indd 60 3/31/14 5:43 PM  Nonexperimental Quantitative Research | 61 When initially investigating a new area, scientists frequently use nonexperi- mental quantitative research to identify existing factors and relationships among them. Such knowledge is later used to formulate hypotheses to be used either in more advanced forms of nonexperimental quantitative research such as path analysis (defined below) or in experimental research. Correlational Study Correlational study In its simplest form, a correlational study consists of measuring two vari- Nonexperimental ables and then determining the degree of relationship that exists between them. research study Consequently, a simple correlational study can be incorporated into other quanti- designed to describe tative research approaches. A relatively old, but still interesting, study commonly relationships among cited in introductory and developmental texts is a study by Conrad and Jones variables and make (1940) regarding the relationship between the IQ scores of parents and those of predictions their offspring. To accomplish the goals of this study, Conrad and Jones measured the IQs of the parents and correlated them with those of their children. In this way, a quantitative index was obtained that described the relationship between these two variables. Because a correlational study is, by definition, a nonexperimental re- search approach, it lacks manipulation of the independent variable. The researcher simply measures variables in their natural state and determines if they are related. The correlational approach is quite effective in enabling us to accomplish the research objectives of description and prediction. If a reliable relationship is found between two variables, we not only have described the relationship but also have gained the ability to predict one variable from knowledge of the other variable. Frequently, multiple variables are used in correlational studies to improve the re- searcher’s ability to make predictions. Here are several dependent variables that psy- chologists have used in prediction studies: major affective disorders in adolescents (Aebi, Metzke, & Steinhausen, 2009), recidivism for sexual offenders (Hanson & Morton-Bourgon, 2009), relapse in depression (Lethbridge & Allen, 2008), supervi- sory ratings of employees (Hermelin, Lievens, & Robertson, 2007), social withdrawal in elementary school children (Booth-LaForce & Oxford, 2008), recovery after mild traumatic brain injury (Stulemeijer, van der Werf, Borm, & Vos, 2008), functional outcomes in schizophrenia (Wittorf, Wiedemann, Buchkremer, & Klingberg, 2008), and self-harm and suicide by adolescents (Larsson & Sund, 2008). As you see in the list of studies provided, prediction studies have an important place in psychological research. The primary weakness of the correlational approach is present when someone assumes that simply because two variables are related that one causes the other. As discussed earlier in this chapter, there are three required conditions for claim- ing that two variables are causally related, and relationship is just one of those. In short, you cannot claim causation unless the other two conditions in Table 2.2 are realized, and this is very difficult in correlational research. Mastering the remain- der of this book will be easier if you will take a moment and memorize the three required conditions for causation provided in Table 2.2. It is very important that you understand that you cannot jump to a conclusion of cause and effect from only knowing that two variables are related. Here is an M02_CHRI7743_12_GE_C02.indd 61 3/31/14 5:43 PM  62 | Research Approaches and Data Collection Methods FIGURE 2.2 A: Presumed independent variable Illustration of the (number of fire trucks) third variable problem in correlations. Z: Third variable Is the relationship between A and B (size of fire) causal or is it due to Z (size of fire)? B: Presumed dependent variable (amount of damage) interesting example (see Figure 2.2). Did you know that there is a relationship be- tween the number of fire trucks responding to a fire and the amount of fire damage? There is a correlation between these two variables: As the number of fire trucks increases, so does the amount of fire damage. Should we conclude that increasing the number of fire trucks causes increased fire damage? No! There is a third variable (i.e., an extraneous variable) operating here: It is the size of fire. As the size of fire increases, so does the number of fire trucks. It is the size of the fire that is actually causing the amount of fire damage not the number of trucks. We call this problem Third variable “the third variable problem,” which refers to the fact that two variables (variables problem A and B) might be correlated not because they are causally related (as in A B) Occurs when but, rather, because there is some “third variable” producing the relationship, and observed relation- once you account for the third variable, it becomes clear that variables A and B are ship between two not causally related; they are just correlated (i.e., condition 1 in Table 2.2). variables is actually Here is one more example. Tea drinking is correlated with lung cancer; people due to a confounding who drink more tea are less likely to get lung cancer. Is it the tea that is prevent- extraneous variable ing lung cancer? No. Tea drinkers have a lower risk for lung cancer because they smoke less. It is very important to remember that you cannot conclude that two vari- ables are causally related when all you know is that the variables are correlated. Whenever you are interested in the issue of causation (i.e., “Do changes in variable A cause changes in variable B?”), you must always consider all three of the required conditions listed earlier in Table 2.2. Relationship is not enough evi- dence. In Chapter 15, we explain two statistical techniques for controlling for ex- traneous or third variables, but, ultimately, you can never know for sure whether you have controlled for all of the third variables that might be operating. Another correlational procedure for obtaining some evidence of causation is known as Path analysis path analysis. The idea is to develop a theoretical model describing how a set of Type of research in variables are related and then to empirically test the theoretical model. which a researcher For example, Turner and Johnson (2003) proposed a theoretical model of hypothesizes a theo- children’s motivation, which is shown in Figure 2.3. Moving from left to right retical causal model in the model, you will see that parenting characteristics (i.e., parents’ educa- and then empirically tion, income, and self-efficacy) were hypothesized to impact parenting beliefs tests the model and parent–child relationships. Next, parenting beliefs and parent–child relation- ships were hypothesized to impact children’s mastery motivation (labeled child’s M02_CHRI7743_12_GE_C02.indd 62 3/31/14 5:43 PM  Nonexperimental Quantitative Research | 63 FIGURE 2.3 Theoretical child Parent’s education mastery “path model.” Each single arrow Parenting shows a hypothesized beliefs direct effect. Two or more arrows in a causal line show a Parent’s Child’s Child’s hypothesized indirect income mastery achievement effect, where one variable is hypoth- Parent– esized to affect a later child relationship variable by operating through one or more Parent’s self- mediating variable(s). efficacy mastery). Last, children’s motivation was hypothesized to impact their academic performance. Remember from Table 2.1 that a mediating variable is a variable that comes in between two variables; given this definition, you can see that the vari- ables in the middle of the figure operate as mediating variables because they are placed between the parent’s characteristic variables on the left and child’s mastery and achievement on the right. The theoretical model shows several hypothesized Direct effect direct effects where one variable has an arrow going from it directly to another An e!ect of one variable. The theoretical model also shows several indirect effects, where one variable directly on variable affects another variable by going through a mediating variable. another variable; Turner and Johnson tested the theoretical model with data collected from 169 depicted as a single African American children and their teachers and parents. The final path analy- arrow in a path model sis model, shown in Figure 2.4, only includes the statistically significant paths. The paths that were not supported were removed. You can see in this “trimmed Indirect effect model” which hypothesized paths were supported by the empirical data. The final An e!ect occurring model suggests that parenting characteristics affect children’s mastery motivation through a mediating through the mediating variable of parent–child relationships. In other words, it variable suggests that parenting characteristics affect the type of parent–child relationships present, and these parent–child relationships affect children’s mastery motiva- tion. The results also suggest that mastery motivation mediates the relationship between parent–child relationships and children’s achievement. In other words, the parent–child relationships have an effect on children’s mastery motivation, and this mastery motivation has an effect on the children’s achievement. The strength of path analysis models (when properly conducted) is that the re- searcher carefully develops a theoretical model and then empirically tests the model. The primary weakness is that the models usually are based on nonexperimental data rather than experimental research data. Therefore, you should not place too much faith in these models. They provide more evidence of causation than is present in a mere correlation between two variables; however, if you want the strongest evidence of causation, you should conduct an experimental research study (if possible). M02_CHRI7743_12_GE_C02.indd 63 3/31/14 5:43 PM  64 | Research Approaches and Data Collection Methods FIGURE 2.4 The trimmed child mastery path model. It is the theoretical path model with the nonsignificant paths eliminated (i.e., it is the model that the data support). The numbers on the lines are called path coefficients, and they show the strength and direction of the relationship (i.e., the closer the numbers are to +1.00 or to –1.00, the stronger the rela- tionship; if the number is positive, then as one variable increases, so does the other variable; if the number is negative, then as one variable increases, the other variable decreases). Parent’s education.18 R 2 =.33 Parenting.51 beliefs R 2 =.13 R 2 =.05.22 Parent’s.17 Child’s Child’s.22 income.51 mastery achievement R 2 =.19.3 5.21 Parent–child relationship.32 Parent’s self- efficacy Natural Manipulation Research Natural Natural manipulation research1 is another type of nonexperimental research manipulation study that examines possible causes that are not usually manipulatable by a re- research searcher, but the causal variable is one that “describes a naturally-occurring Type of research contrast between a treatment and a comparison condition” (Shadish et al., in which the 2002, p. 12). In other words, the independent variable maps onto what one independent variable might view as a natural manipulation occurring in the world. For example, the approximates a destruction of the twin trade towers in New York City on September 11, 2001, naturally occurring had a significant psychological impact on many individuals’ lives. You might manipulation, but it hypothesize that the impact will be greater for people who were near the twin is not manipulated by towers when they collapsed than for individuals who were far away. The causal the researcher or independent variable would be closeness to attack, and the levels of this variable are “near the towers” versus “farther away.” You might operationalize the levels of the independent variable as within 2 miles (for the near condi- tion) versus more than 100 miles away (for the faraway condition). Obviously it would not be possible for an experimenter to manipulate such an event, but one might view it as a naturally occurring experiment. 1 This type of research was formally labeled ex post facto research, a term that has become obsolete. The newer term is natural experiment; however, we believe the term natural manipulation more clearly communicates the key idea of this type of research. Some authors will prefer to classify natural manipulation research as a quasi-experimental approach rather than a nonexperimental approach. M02_CHRI7743_12_GE_C02.indd 64 3/31/14 5:43 PM  Nonexperimental Quantitative Research | 65 In practice, the difference between the kinds of predictor or independent variables examined in correlational and natural manipulation research is small (it is a matter of degree). The difference is one of “degree of manipulation.” In correlational research no manipulation is present, but in natural manipulation research an event occurs that one might view as an approximation of a manipulation. On the one hand, one might argue that a study based on trait variables (e.g., intelligence, extraversion, anxiety, submissiveness) and charac- teristic variables (e.g., height, weight, ethnicity, political affiliation) should be called a correlational study, because these kinds of variables cannot be changed; they are characteristics that remain relatively constant throughout someone’s life. On the other hand, a study based on experience variables (e.g., death of a loved one, experience of an earthquake, winning the lottery, divorce, experience of a hurricane) can be treated as naturally manipulated variables if individuals’ status on these variables change. If you believe the independent variable approximates a natural manipulation, then call the study a natural manipulation study, but if the independent vari- able seems like one that is not naturally manipulated, then call it a correlational study. Also, if your research purpose is predictive, then call your study a predic- tive study. In practice, the best advice might be to call both correlational and natural manipulations “nonexperimental” studies to emphasize the point that the researcher did not manipulate the independent variable or control the con- ditions surrounding the manipulation. Remember that whenever you want to draw a conclusion about cause and effect, you must meet the three required conditions provided in Table 2.2. In both correlational and natural manipula- tions, you will find yourself in trouble on condition 3 shown in Table 2.2. To the degree that a researcher has attempted to determine proper time order of the variables (condition 2) and has systematically controlled for extraneous variables (condition 3), you can upgrade your evaluation of the nonexperimental research study. Also, generally speaking, when the nonexperimental study is based on longitudinal data rather than cross-sectional data (discussed in the next section), the researcher is better able to establish time order, and you can upgrade your evaluation of the research study. An example of a natural manipulation research study is demonstrated in Richards, Hardy, and Wadsworth’s (1997) investigation of the relationship be- tween divorce and psychological functioning of adults. One might view divorce as a natural manipulation independent variable because it marks a categori- cal change in individuals’ status from married to divorced. The researchers hypothesized that adults who had been divorced would report higher rates of anxiety and depression than those who had not (after equating the two groups on several extraneous variables for control). The study included data from the participants at ages 13 and 43. Richards et al. found that the adults at age 43 who had been divorced reported more anxiety and depression than those who had not been divorced. This relation between divorce and depres- sion and anxiety was present after controlling for measures of psychological functioning at age 13. M02_CHRI7743_12_GE_C02.indd 65 3/31/14 5:43 PM  66 | Research Approaches and Data Collection Methods STUDY QUESTIONS 2.6 What is nonexperimental quantitative research? What is the difference between correlational research and natural manipu- lation research? In what ways is correlational research and natural manipulation research similar? Cross-Sectional and Longitudinal Studies Cross-sectional study In a cross-sectional study, the data are collected from research participants dur- Study conducted at ing a single, relatively brief period. The “single” time period is just long enough a single time period, to collect data from all of the participants. In a longitudinal study, the data are and data are collected collected at two or more points in time. Cross-sectional and longitudinal studies from multiple groups; are frequently used in developmental psychology and are used to study changes data are collected associated with age. The primary independent variable might or might not be a during a single, brief manipulated independent variable. In other words, these studies are sometimes time period experimental studies, but perhaps more commonly are nonexperimental studies. Age is a nonmanipulatable variable, so if it is the primary independent variable, Longitudinal study then the study will be nonexperimental. Data are collected at In developmental research, a longitudinal study involves choosing a single two or more points group of participants and measuring them repeatedly at selected time intervals to in time note changes that occur over time in the specified characteristics. For example, Gathercole and Willis (1992) measured a group of children’s phonological mem- ory and their vocabulary knowledge at 4, 5, 6, and 8 years of age to determine if the relationship between these two variables changed as the children got older. On the other hand, in developmental research, a cross-sectional study involves identifying representative samples of individuals that differ on some character- istic, such as age, and measuring these different samples of individuals on the same variable or variable(s) at one point in time. Wagner, Torgesen, Laughon, Simmons, and Rashotte (1993) used the cross-sectional approach in their study of the nature and development of young children’s phonological processing abili- ties. They randomly selected a group of 95 kindergarten and 89 second-grade stu- dents from three elementary schools and administered a number of phonological tasks to both groups to determine whether phonological processing abilities differ among these two age groups. Although the longitudinal and cross-sectional research approaches are fre- quently used in developmental research, this type of study is not confined to this specific area. For example, Moskowitz and Wrubel (2005) took a longitudinal approach to gaining a more in-depth understanding of the meaning of having contracted HIV. To accomplish the purpose of this study, Moskowitz and Wrubel identified 57 gay men testing positive for HIV and then conducted bimonthly interviews over the course of 2 years to identify how these individuals appraise their HIV changes over time. Andersen, Franckowiak, Christmas, Walston, and Crespo (2001) took a cross-sectional approach to assess the relationship be- tween not participating in leisure time physical activity and body weight among various ethnic groups of older U.S. adults. To accomplish the goal of the study, M02_CHRI7743_12_GE_C02.indd 66 3/31/14 5:43 PM  Nonexperimental Quantitative Research | 67 these investigators surveyed a national representative cross-section of the U.S. population (e.g., Hispanic Americans, African Americans, Caucasian Americans) of individuals aged 60 and older regarding their weight and participation in leisure time activity. There has been discussion of the relative advantages and disadvantages of lon- gitudinal and cross-sectional approaches to developmental research. One impor- tant point is that these two approaches have not always produced similar results. The classic example of this discrepancy is seen in the development of intelligence during adulthood. As seen in Figure 2.5, cross-sectional studies have suggested that adult intelligence begins to decline around the age of 30, whereas longitudi- nal studies show an increase or no change in intellectual performance until the age of 50 or 60 (Baltes, Reese, & Nesselroade, 1977). Why? The difference has been attributed to what is called an age-cohort effect. Longitudinal studies follow just one group or age cohort of individuals over time, so all individuals within this cohort experience similar environmental events. However, cross-sectional studies investigate a number of different groups of individuals or different age cohorts. Because of changes in environmental events, these cohorts have not been exposed to similar experiences. For example, members of a 50-year-old cohort would not have been exposed to video games or computers when they were 10 years old, but a group of 11-year-olds would have. Such differences are confounded with actual age differences in cross-sectional studies, where you compare individuals who are of different ages at one point in time. Because of the constraints of time, attrition of participants (i.e., participants dropping out of the study), and costs involved in conducting a longitudinal study, the cohort-sequential design has been suggested as an alternative approach. This FIGURE 2.5 Change in intellectual performance as a function of the longitudinal versus the cross-sectional method. Mean intellectual performance Longitudinal Cross-sectional 10 20 30 40 50 60 Age (From Life-Span Developmental Psychology: Introduction to Research Methods by P. B. Baltes, W. H. Reese, & J. R. Nesselroade. Copyright © 1977 by Brooks/Cole Publishing Company, Pacific Grove, CA 93950, a division of International Thomson Publishing Inc. By permission of the publisher.) M02_CHRI7743_12_GE_C02.indd 67 3/31/14 5:43 PM  68 | Research Approaches and Data Collection Methods Cohort-sequential approach is a hybrid of longitudinal and cross-sectional approaches. The cohort- design sequential design is a design in which different age groups are tested longitu- Design that combines dinally. For example, Chouinard and Roy (2008) were interested in the changes cross-sectional and that occur in students’ academic motivation during adolescence. They recruited longitudinal elements a group of seventh graders and a group of ninth graders and followed them until by following two or they completed 9th and 11th grade, respectively. Use of the cohort-sequential de- more age groups over sign allowed these researchers to gather data from 7th to 11th grade in less time time than would be required by a fully longitudinal study. The cohort-sequential design typically results in less cost, time, and attrition than a fully longitudinal study. STUDY QUESTIONS 2.7 What are cross-sectional and longitudinal designs? How are cross-sectional and longitudinal designs different? How does the cohort-sequential design combine characteristics of cross-sectional and longitudinal designs? Qualitative Research Qualitative research Qualitative research is an interpretive research approach that relies on multiple Interpretive research types of subjective data and investigates people in particular situations in their nat- approach relying on ural environment (Denzin & Lincoln, 1994). This definition has three primary com- multiple types of ponents that are essential to understanding the nature of qualitative research. The subjective data and first component is that qualitative research is interpretive. Qualitative data consist of investigation of people words, pictures, clothing, documents, or other nonnumerical information. During in particular situa- and after the data are collected, the researcher continually attempts to understand tions in their natural the data from the participants’ subjective perspectives. The most important task of the environment qualitative researcher is to understand the insiders’ views. Then the researcher also takes the role of “objective outsider” and relates the interpretive–subjective data to the research purpose and research questions. In qualitative research, the research questions are allowed to evolve, or possibly change, during the study be- cause qualitative research is usually focused on exploring phenomena; in contrast, quantitative research typically does not allow changes of this sort because the focus usually is on hypothesis testing. Qualitative research tends to be most useful for understanding and describing local situations and for theory generation; in contrast, quantitative research tends to be most useful for hypothesis testing. Starting with the interpretive component, let’s examine a qualitative research study in which the researchers (i.e., Schouten & McAlexander, 1995) became participant observers of the subculture of consumerism associated with Harley- Davidson motorcycles. In their words: “…with the excitement and trepidation of neophytes, we tiptoed into our fieldwork as naïve nonparticipant observers. At the time of this writing, we have spent the last year deeply immersed in the lifestyle of the HDSC [Harley-Davidson sub culture], ‘passing,’ as bikers…” (p. 44). It was essential that the researchers understand the subculture from the insider’s perspec- tive, rather than from the perspective of an ethnocentric outsider. The researchers noted the general appearance and clothing worn by the Harley-Davidson bikers. Many of the bikers had massive bellies, large biceps, and enjoyed loud, aggressive M02_CHRI7743_12_GE_C02.indd 68 3/31/14 5:43 PM  Qualitative Research | 69 behavior. They adorned their bikes with chrome and leather and wore leather clothing, heavy boots, and gauntlets as well as wallet chains, conches, chrome studs, and other similar hardware. Their motto was “Live to ride, ride to live.” The members were “brothers.” Their core values included total personal freedom, liberation from confinement, patriotism and American Heritage, and machismo. What did this mean? This is where the interpretive component comes in. The concept of real Harley Davidson men seemed to pervade and help explain many aspects of the biker experience from the clothing they wore to their general be- havior and appearance. Members selected this subculture, became socialized, and then continued the tradition by rewarding other members when they displayed the cultural values and behaviors. The most-valued Harley-Davidson bikes were the biggest, heaviest, and loudest ones, which meant that they were the manliest, even though they were not the fastest. All of this was interpreted as conveying a sense “of power, fearsomeness, and invulnerability to the rider” (p. 54). The second component is that qualitative research is multimethod. This means that a variety of methods are used to collect data. These include such diverse data collection methods as an individual’s account of a personal experience, in- trospective analysis, an individual’s life story, interviews with an individual, ob- servation of an individual or individuals, written documents, photographs, and historical information. In many qualitative studies, several of these data collec- tion methods might be used to try to get the best description of an event and the meaning it has for the individual or individuals being studied. This use of sev- Triangulation eral methods is referred to as triangulation, because it is believed that the use Use of multiple data of several methods provides a better understanding of the phenomenon being sources, research investigated. For example, Schouten and McAlexander (1995) collected their methods, investiga- data from formal and informal interviews, observations, and photographs of the tors, and/or theories/ Harley-Davidson bikers. perspectives to cross- The third component of qualitative research is that it is conducted in the field check and corroborate or in the person’s natural setting and surroundings, such as a school classroom, research data and the playground, a board meeting, or a therapy setting. To meet this component of conclusions conducting the research in the natural surroundings of the research participants, Schouten and McAlexander attended rallies of the Harley Owners Group (HOG), as well as biker swap meets and certain club meetings. The final step involved purchasing Harley-Davidson bikes and the appropriate clothing (jeans, black boots, and black leather jackets), followed by wearing the clothing and using the bikes as their primary means of transportation. This heightened personal involve- ment increased the frequency of contact with other “bikers” and allowed the re- searchers to gain an empathic understanding of the bikers’ identity, psyche, and everyday social interactions. From this description of qualitative research, you should be able to see that it is an approach that uses many data collection methods requiring the interpreta- tion of nonnumerical data. The strength of qualitative research is the description and understanding of individuals and groups of individuals with a common iden- tity. Another strength is providing data from which researchers can generate and develop theoretical understandings of phenomena. It is useful for the “logic of discovery” defined in the last chapter. M02_CHRI7743_12_GE_C02.indd 69 3/31/14 5:43 PM  70 | Research Approaches and Data Collection Methods Qualitative research has its limitations just as every other research method does. One weakness of qualitative research is that it is difficult to generalize because the data are based on local, particularistic data. Another weakness is that different quali- tative researchers might provide very different interpretations of the phenomena studied. Another weakness is that objective hypothesis testing procedures are not used. Nonetheless, qualitative data can provide a useful complement to quantitative data and are very useful when the research purpose is exploration and description. The argument that adding qualitative data to psychological studies is benefi- cial has been compelling because during the last decade we have witnessed an in- crease in research that makes use of this type of data. For example, a burgeoning literature has developed in organizational management, social psychology, aging, education, and family studies (Denzin & Lincoln, 1994; Gilgun, Daly, & Handel, 1992; Gubrium & Sankar, 1993; Silverman, 1993) that focuses on the collection and analysis of qualitative data. STUDY QUESTIONS 2.8 What is qualitative research, and explain each of the components included in this definition. What are the strengths and weaknesses of qualitative research? Major Methods of Data Collection In empirical research, researchers collect data, analyze the data, and report and Method of data interpret the results. The term method of data collection refers to how the collection researcher obtains the empirical data to be used to answer his or her research Technique for physi- questions. We contend that there are six major methods of data collection and cally obtaining the that these methods incorporate more specific methods of data collection. We now data to be analyzed in describe these following major methods of data collection: tests, questionnaires, a research study interviews, focus groups, observations, and existing or secondary data. Tests Tests Tests are commonly used data collection instruments or procedures designed to Standardized measure personality, aptitude, achievement, and performance. Many tests are or researcher- standardized and come with information on their reliability, validity, and norms constructed data for comparison. Tests also are frequently constructed by experimental researchers collection instruments for specific variables examined in research studies. Some strengths and weak- designed to measure nesses of tests are provided in Table 2.3. personality, aptitude, As a general rule, you should not construct a new test if one is already avail- achievement, and able. For psychological research purposes, the best source of information about performance the tests you should be using to address your research questions is found in the published psychological research literature. You should always examine the best research in the research area and locate the measures that they use. Another use- ful source of tests and measures is The Directory of Unpublished Experimental Mental Measures (2008), edited by Goldman and Mitchell and published by the American M02_CHRI7743_12_GE_C02.indd 70 3/31/14 5:43 PM  Major Methods of Data Collection | 71 TABLE 2.3 Strengths and Weaknesses of Tests Strengths of tests (especially standardized tests) Can provide measures of many characteristics of people. Often standardized (i.e., the same stimulus is provided to all participants). Allows comparability of common measures across research populations. Strong psychometric properties (high measurement reliability and validity). Availability of reference group data. Many tests can be administered to groups, this saves time. Can provide “hard,” quantitative data. Tests are usually already developed. A wide range of tests is available. Response rate is high for group-administered tests. Ease of data analysis because of quantitative nature of data. Weaknesses of tests (especially standardized tests) Can be expensive if test must be purchased for each research participant. Reactive participant effects such as social desirability can occur. Test might not be appropriate for a local or unique population. Open-ended questions and probing not available. Tests are sometimes biased against certain groups of people. Nonresponse to selected items on the test. Some tests lack psychometric data. Psychological Association. We discuss standardized tests in detail in Chapter 5, and we explain the psychometric properties of reliability and validity and provide some additional sources for locating tests and reviews of tests. In addition to the tests discussed in Chapter 5, however, researchers must sometimes develop a new test to measure the specific knowledge, skills, behavior, or cognitive activity that is being studied. For example, a researcher might need to measure response time to a memory task using a mechanical apparatus or de- velop a test to measure a specific mental or cognitive activity (which obviously cannot be directly observed). Again, the best source for this information is the psychological research literature. Questionnaire Questionnaires Self-report data The second method of data collection is the questionnaire. A questionnaire collection instrument is a self-report data collection instrument that is filled out by research partici- completed by research pants. Questionnaires measure participants’ opinions and perceptions and pro- participants vide self-reported demographic information. They are usually paper-and-pencil M02_CHRI7743_12_GE_C02.indd 71 3/31/14 5:43 PM  72 | Research Approaches and Data Collection Methods instruments (i.e., participants fill them out), but are increasingly being placed on the Web for participants to go to and “fill out.” Questionnaires can include closed- ended items (where respondents must select from the responses given by the researcher) and open-ended items (where respondents provide answers in their own words). We discuss the questionnaire method of data collection extensively Interview in Chapter 12 and explain how to construct a questionnaire. The strengths and Data collection weaknesses of questionnaires are provided in Table 2.4. method in which an interviewer asks the interviewee a series of Interviews questions, often with The third method of data collection is the interview method. An interview is prompting for ad- a situation where the interviewer asks the interviewee a series of questions. ditional information Interviews are conducted in face-to-face situations and over the telephone. It is TABLE 2.4 Strengths and Weaknesses of Questionnaires Strengths of questionnaires Good for measuring attitudes and eliciting other content from research participants. Inexpensive (especially mail questionnaires, Internet, and group-administered questionnaires). Can provide information about participants’ subjective perspectives and ways of thinking. Can administer to probability samples. Quick turnaround for group-administered questionnaires. Perceived anonymity by respondent can be high if situation is carefully controlled. Moderately high measurement validity (i.e., high reliability and validity) for well-constructed and validated questionnaires. Closed-ended items can provide exact information needed by researcher. Open-ended items can provide detailed information in respondents’ own words. Ease of data analysis for closed-ended items. Useful for exploration as well as hypothesis testing research. Weaknesses of questionnaires Usually must be kept short. Reactive effects might occur (e.g., respondents might try to show only what is socially desirable). Nonresponse to selective items. People filling out questionnaires might not recall important information and might lack self-awareness. Response rate may be low for mail and e-mail questionnaires. Open-ended items may reflect differences in verbal ability, obscuring the issues of interest. Data analysis can be time consuming for open-ended items. Measures need validation. M02_CHRI7743_12_GE_C02.indd 72 3/31/14 5:43 PM  Major Methods of Data Collection | 73 also possible to conduct interviews electronically, such as over the Internet. These interviews can be asynchronous (interaction occurs over time) or synchronous (interaction happens in real time). The strengths and weaknesses of interviews as a method of data collection are provided in Table 2.5. In Chapter 12 you will learn how to construct interview protocols, which have much in common with questionnaires. Also in Chapter 12, we provide practical information on how to conduct interviews. Focus Groups Focus group The fourth method of data collection involves the use of focus groups. A focus Collection of data in a group is a situation where a focus group moderator keeps a small and homo- group situation where geneous group (of 6–12 people) focused on the discussion of a research topic a moderator leads or issue. Focus group sessions generally last between 1 and 3 hours and are re- a discussion with a corded using audio and/or videotapes. A focus group should not be viewed as a small group of people group interview because the emphasis is on small-group interaction and in-depth TABLE 2.5 Strengths and Weaknesses of Interviews Strengths of interviews Good for measuring attitudes and most other content of interest. Allows probing and posing of follow-up questions by the interviewer. Can provide in-depth information. Can provide information about participants’ subjective perspectives and ways of thinking. Closed-ended interviews provide exact information needed by researcher. Telephone and e-mail interviews usually provide very quick turnaround. Moderately high measurement validity (i.e., high reliability and validity) for well-constructed and well-tested interview protocols. Can use with probability samples. Relatively high response rates are often attainable. Useful for exploration as well as hypothesis-testing research. Weaknesses of interviews In-person interviews usually are expensive and time consuming. Reactive effects (e.g., interviewees might try to show only what is socially desirable). Investigator effects might occur (e.g., untrained interviewers might distort data because of personal biases and poor interviewing skills). Interviewees might not recall important information and might lack self-awareness. Perceived anonymity by respondents might be low. Data analysis can be time consuming for open-ended items. Measures need validation. M02_CHRI7743_12_GE_C02.indd 73 3/31/14 5:43 PM  74 | Research Approaches and Data Collection Methods discussion among the participants about the issues being studied. Focus groups Observation are especially useful for exploring ideas and obtaining in-depth information about Researcher watches how people think about an issue. The strengths and weaknesses of focus groups and records events or as a method of data collection are provided in Table 2.6. behavioral patterns of people Observation Naturalistic The fifth method of data collection is the observation method, in which the re- observation searcher looks at what people do. Often, it is important to collect observational Observation conducted data (in addition to attitudinal data) because what people say is not always what in real-world situations they do! Researchers can observe participants in natural and/or structured envi- ronments. The former is called naturalistic observation because it is done in Laboratory real-world settings. The latter is called laboratory observation because it is con- observation ducted in a lab or other controlled environment set up by the researcher. Observation conduct- In quantitative research, the researcher standardizes the procedures and col- ed in lab setting set up lects quantitative data. Specifically, the researcher standardizes who is observed, by the researcher what is observed, when and where the observations are to take place, and how TABLE 2.6 Strengths and Weaknesses of Focus Groups Strengths of focus groups Useful for exploring ideas and concepts. Provides window into participants’ internal thinking. Can obtain in-depth information. Can examine how participants react to each other. Allows probing. Most content can be tapped. Allows quick turnaround. Weaknesses of focus groups Sometimes expensive. Might be difficult to find a focus group moderator with good facilitative and rapport-build- ing skills. Reactive and investigator effects might occur if participants feel they are being watched or studied. Might be dominated by one or two participants. Difficult to generalize results if small, unrepresentative samples of participants are used. Might include large amount of extra or unnecessary information. Measurement validity might be low. Usually should not be the only data collection method used in a study. Data analysis can be time consuming because of the open-ended nature of the data. M02_CHRI7743_12_GE_C02.indd 74 3/31/14 5:43 PM  Major Methods of Data Collection | 75 the observations are to take place. Standardized instruments (e.g., checklists) are often used in quantitative observation. Sampling procedures are sometimes used so that the researcher does not have to make continuous observations. For example, a researcher might use time-interval sampling to obtain a representa- Time-interval tive sample of possible observations. Time-interval sampling is conducted by sampling observing during preselected time intervals, such as during the first 5 minutes Observations are of each 30-minute time interval. Conversely, in event sampling the researcher recorded during prese- conducts observations every time that a particular event takes place (e.g., observe lected time intervals every time a participant asks another participant a question). Event sampling is an efficient method of sampling when you want to observe a particular event that Event sampling occurs infrequently. Observations are In qualitative research, observation procedures usually are exploratory and recorded every time a open ended, and the researcher takes extensive field notes. It is helpful to con- particular event occurs sider qualitative observation as falling on a continuum originally developed by social scientist Raymond Gold (1958). Following are the types from least-quali- tative (complete observer) to the most-qualitative observation (complete partici- pant) in nature: Complete observer. Here the researcher observes from the “outside” and, if the setting is a public one, the researcher does not inform the participants that he or she is studying them. Observer-as-participant. Here the researcher spends a limited amount of time “inside” the situation and obtains informed consent to observe the partici- pants for a research study. Participant-as-observer. Here the researcher spends extensive time “inside” the group or situation and always informs the participants that they are being studied and obtains informed consent. Complete participant. Here the researcher becomes a full participating mem- ber of the group. In most cases, the group must be informed and permission granted. If you ever collect observational data, remember the following: (1) Make sure everyone is well trained; (2) be sensitive to your appearance and how people being observed react to you; (3) establish rapport but do not promise anything you cannot deliver; (4) be reflexive, unobtrusive, empathetic, and alert at all times; (5) find an effective way to record what is observed (e.g., note taking, tape recordings); (6) try to validate and corroborate what you think you are seeing; (7) make observations in multiple settings; and (8) spend enough time in the “field” Existing or secondary to obtain sufficient information. The strengths and weaknesses of observational data data are provided in Table 2.7. Collection of data that were left behind or originally used for Existing or Secondary Data something di!erent The sixth and last major method of data collection is the collection of existing or than the current secondary data. This means that the researcher collects or obtains “data” that research study were originally left behind or used for some purpose other than the new research M02_CHRI7743_12_GE_C02.indd 75 3/31/14 5:43 PM  76 | Research Approaches and Data Collection Methods TABLE 2.7 Strengths and Weaknesses of Observational Data Strengths of observational data Allows one to directly see what people do without having to rely on what they say they do. Provides firsthand experience, especially if the observer participates in activities. Can provide relatively objective measurement of behavior (especially for standardized observations). Observer can determine what does not occur. Observer might see things that escape the awareness of people in the setting. Excellent way to discover what is occurring in a setting. Helps in understanding importance of contextual factors. Can be used with participants with weak verbal skills. Might provide information on things people would otherwise be unwilling to talk about. Observer might move beyond selective perceptions of participants in the setting. Good for description. Provides moderate degree of realism (when done outside of the laboratory). Weaknesses of observational data Reasons for observed behavior might be unclear. Reactive effects might occur when respondents know they are being observed (e.g., people being observed might behave in atypical ways). Investigator effects (e.g., personal biases and selective perception of observers). Observer might “go native” (i.e., overidentifying with the group being studied). Sampling of observed people and settings might be limited. Cannot observe large or dispersed populations. Some settings and content of interest cannot be observed. Document Collection of unimportant material might be moderately high. Personal and o"cial documents that were More expensive to conduct than questionnaires and tests. left behind Data analysis can be time consuming. Physical data Any material thing created or left behind study. The most frequently used existing data are documents, physical data, and by humans that might archived research data. Personal documents are documents that were written or provide clues to some recorded for private purposes, such as letters, diaries, and family pictures. Official event or phenomenon documents are documents that were written or recorded for public or private or- ganizations, such as newspapers, annual reports, yearbooks, and meeting minutes. Archived research Physical data are any material thing created or left that might provide informa- data tion about a phenomenon of interest to a researcher, such as the contents of some- Data (usually quanti- one’s trash, wear on the tiles in museums, wear on library books, and soil and tative) originally used DNA on clothes. Archived research data are secondary research data that were for a di!erent research collected by other researchers for other purposes. When data are saved and ar- project chived, others researchers can later use the data. The largest repository of archived M02_CHRI7743_12_GE_C02.indd 76 3/31/14 5:43 PM  Major Methods of Data Collection | 77 quantitative data is the Interuniversity Consortium for Political and Social Research (ICPSR), which is located at the University of Michigan in Ann Arbor, Michigan. The strengths and weaknesses of existing/secondary data are provided in Table 2.8. STUDY QUESTIONS 2.9 What are the six methods of data collection? What are two strengths and weaknesses of each of the six methods of data collection? TABLE 2.8 Strengths and Weaknesses of Existing Data Strengths of documents and physical data Can provide insight into what people think and what they do. Unobtrusive, making reactive and investigator effects very unlikely. Can be collected for time periods occurring in the past (e.g., historical data). Provides useful background and historical data on people, groups, and organizations. Useful for corroboration. Grounded in local setting. Useful for exploration. Strengths of archived research data Archived research data are available on a wide variety of topics. Inexpensive. Often are reliable and valid (high measurement validity). Can study trends. Ease of data analysis. Often based on high quality or large probability samples. Weaknesses of documents and physical data Might be incomplete. Might be representative only of one perspective. Access to some types of content is limited. Might not provide insight into participants’ personal thinking for physical data. Might not apply to general populations. Weaknesses of archived research data Might not be available for the population of interest to you. Might not be available for the research questions of interest to you. Data might be dated. Open-ended or qualitative data usually not available. Many of the most important findings have already been mined from the data. M02_CHRI7743_12_GE_C02.indd 77 3/31/14 5:43 PM  78 | Research Approaches and Data Collection Methods Summary The two major research approaches of quantitative and qualitative research were introduced. Quantitative research (e.g., experimental research and nonex- perimental research) relies on numerical data and qualitative research on non- numerical data. Experimental research is the best type of research for demon- strating cause-and-effect relationships. In experimental research, the researcher actively manipulates the independent variable (IV) and holds all other variables constant so that a difference between the treatment and control groups found on the dependent variable after the manipulation can be attributed to the inde- pendent variable. For example, the researcher might randomly assign partici- pants with a common cold to two groups in order to form two probabilistically equivalent groups at the start of the experiment. The researcher “manipulates the independent variable” by giving a pill with the active ingredient (supposed to cure the cold) to one group and a placebo (pill without the active ingredi- ent) to the other group. The only difference between the two groups is that one received the real pill and the other received the placebo. If the group receiving the pill with the active ingredient improves, but the group receiving the placebo does not improve, then the researcher can conclude that the pill worked (i.e., it caused the treatment group participants to improve). Another way of stating this is to say that the researcher concludes that changes in the IV caused the changes in the dependent variable (DV). The three required conditions for mak- ing a claim of cause and effect are as follows: (1) There must be a relationship between the IV and DV, (2) changes in the IV must occur before the changes in the DV, and (3) the relationship between the IV and DV must not be due to any extraneous or third variable (i.e., there must not be any alternative expla- nation for the relationship observed between the IV and DV). When you want to study cause and effect, the experimental research approach should always be your first choice because it is the strongest type of research for this purpose. Experiments can be conducted in field settings, in the laboratory, or on the Internet. In nonexperimental quantitative research, the researcher is not able to manip- ulate the IV, and the required causal condition 3 (eliminating alternative explana- tions) is always a concern. Two types of nonexperimental quantitative research are correlational research (measuring relationships among variables) and natural manipulation research (when the independent variable approximates a natural manipulation in the world). Correlational research is often used for predictive purposes, but it is also used for testing theoretical models (in a technique called path analysis). Cross-sectional studies (where data are collected during a single time period) and longitudinal studies (where data are collected at two or more time periods) are sometimes used in experimental research, but they are more often used in nonexperimental quantitative research. Longitudinal studies are especially popu- lar in developmental psychology. Longitudinal studies are helpful in establishing causal condition 2 (establishing time ordering of the IV and DV). Qualitative research is an interpretive research approach that relies on multiple types of subjective data and is used to investigate people in particular situations M02_CHRI7743_12_GE_C02.indd 78 3/31/14 5:43 PM  Key Terms and Concepts | 79 in natural environments. It is interpretive (i.e., it attempts to understand the in- siders’ subjective perspectives), multimethod (i.e., it uses multiple data collec- tion methods such as life stories, participant observation, in-depth interviewing, open-ended questionnaires), and conducted in natural real-world settings (i.e., it studies behavior as it naturally occurs rather than manipulating independent variables). Last, the six major methods of data collection (i.e., ways to obtain empirical data) were described. They are as follows: (1) tests (instruments or procedures for measuring personality, achievement, performance, and other more specific experimental outcome variables), (2) questionnaires (i.e., self-report data collec- tion instrument filled out by research participants), (3) interviews (i.e., situation in which an interviewer asks the interviewee a series of questions and probes for clarification and detail when needed), (4) focus groups (i.e., small-group situa- tion, where a group moderator keeps a group of participants focused on discus- sion of research topics of interest), (5) observation (i.e., the researcher looks at what people do rather than asking them), and (6) existing or secondary data (i.e., collection of data left behind for other purposes, such as documents, physical data, and archived data). Key Terms and Archived research data Interviews Concepts Categorical variable Causal description Laboratory experiment Laboratory observation Causal explanation Longitudinal study Causation Manipulation Cause Mediating variable Cause-and-effect relationship Method of data collection Cohort-sequential design Mo