RD Topic 5.pdf

EDWARDS—RESEARCH METHODS LECTURE NOTES PAGE 34 Topic #5 EXPERIMENTAL CONTROL Central issue is, of course, one of research validity. Key questions in research: 1....

EDWARDS—RESEARCH METHODS LECTURE NOTES PAGE 34 Topic #5 EXPERIMENTAL CONTROL Central issue is, of course, one of research validity. Key questions in research: 1. What are the threats to the validity of a contemplated piece of research? 2. What means are available to neutralize these threats? Control—any means used to rule out possible threats to a piece of research. EXPERIMENTAL CONTROL The emphasis is on ability to restrain or guide sources of variability (extraneous variables) in research. With experimental control, we are concerned with our ability to rule out possible threats to our study and also the extent to which we can rule out alternative explanations of the experimental results. Extraneous Variable—variable other than independent or dependent variables; variable that is not the focus of the experiment and may confound the results if not controlled. Strategies to achieve/enhance control 1. Random assignment to groups 2. Subject as own control 3. Instrumentation of response 4. Matching 5. Building nuisance variables into the experiment 6. Statistical control EDWARDS—RESEARCH METHODS LECTURE NOTES PAGE 35 1. Random Assignment To Groups—rules out plausible alternative interpretations due to chance. Controls for both known and unknown effects. 2. Subject As Own Control Within–subjects designs and single–subject designs. One of the most powerful control techniques is to have each participant experience every condition of the experiment. Since each individual is unique and varies in several ways that may affect the experimental outcome, participants as their own control rules out these variations. Potential Problems: A. Practice effects. B. Irreversibility of treatment effects. C. Dependability of treatments Order Effects—changes in participants' performance resulting from the position in which a condition appears in the study e.g., movie preference, warm–up or practice effects in learning studies, Sequence Effects—changes in subjects' performance resulting from interactions among conditions themselves. e.g., taste tests, estimating weights EDWARDS—RESEARCH METHODS LECTURE NOTES PAGE 36 Methods of Control A. Block Randomization—is a control procedure in which the order of conditions is randomized, with each condition being presented once before any condition is repeated; BCAD ADCB This procedure controls for both order and sequencing effects. B. Counterbalancing—a method of control in which the conditions are presented in one order the first time and then another. Reverse—ABC CBA Intrasubject Intragroup Complete Incomplete 3. Instrumentation Of Response—using a measurement or instrumentation process that is objective, standardized, sensitive, and also reliable and valid. Ideally, one should be able to take subjective states (e.g., stress) and use some objective instrumentation to generate measures or scores. e.g., for stress: cortisol levels versus self–report ratings 4. Matching—procedure whereby participants are matched on some variables or characteristics of interest. Matching is typically used when it is not possible to use random assignment. So matching is basically a procedure that attempts to obtain equivalent groups in the absence of random assignment. However, if given the topic domain, it is possible to use random assignment with/after participants have been obtained based on self–selection or natural occurrence, then matching is implemented before participants are randomly assigned to groups. Have to suspect there is an important variable or characteristic on which participants differ that can be measured and participants matched. Also have to suspect that there is a relationship between the matched variable and the DV. e.g., a study to assess the effectiveness of an educational intervention program—what are some variables or dimensions we might want to match people on? Therefore, matching controls for only known effects. EDWARDS—RESEARCH METHODS LECTURE NOTES PAGE 37 5. Moderator Variable—a nuisance variable (another IV) that moderates or influences the relationship between the IV and DV. A control for moderator variables is to build them into the experiment by measuring them, essentially having another IV. 6. Statistical Control A. first determine if the design permits analysis by accepted statistical methods. B. increase the statistical power of a design (e.g., increase sample size). C. use a specialized statistical technique to enhance control (e.g., ANCOVA, partial correlations). D. increase the number of trials. GREATER DEGREES OF CONTROL RESULT IN HIGHER LEVELS OF INTERNAL VALIDITY EDWARDS—RESEARCH METHODS LECTURE NOTES PAGE 38 EXPERIMENTER EFFECTS Rosenthal (1976)—has extensively documented the existence of experimenter effects An experiment is typically a social interaction/situation among people. The issue is that the social nature of an experiment usually results in the taking of roles which are played by the experimenter and participants. Further, that this interaction between the two can influence the outcome of the study. In other words, if the psychological study is a social situation involving interactions among people (researcher and participants), then the social conditions of the study also involve roles played by the researcher and participant. Thus, the interactions between these two can influence the outcome of the study. Basically, experimenter effects tend to be strongest when the experimenter factor is related to the experimental task that is being performed by participants. For example: 1. If the research focuses on judgments of organizational commitment, then if the experimenter is employed by the organization could influence responses. 2. If the study concerns smoking, then whether the experimenter appears to be a smoker or not may be an important variable. A type of experimenter effect is experimenter expectancy The experimenter is seldom a neutral participant in the research project She/he is likely to have some expectations about the outcome of the study with often strongly vested interests in the meeting of these expectations The issue is one of whether the experimenter will somehow behave in ways that bias the research results in the expected direction The concept of experimenter expectancy is different and separate from intentional bias or fraud With expectancies, the interest is in some less than conscious or unconscious nonverbal way of communicating expectations EDWARDS—RESEARCH METHODS LECTURE NOTES PAGE 39 ISSUE: what are the mechanisms that produce these effects? Some plausible hypotheses that have been evaluated and discarded are: A. obvious efforts to influence participants B. cheating C. systematic errors in recording and analyzing data The suggestion is that nonverbal communications of some sort are responsible. This seems to supported by the fact that expectancy effects have also been noticed in animal studies. e.g., Clever Hans, the counting horse (who remembers this story?) EDWARDS—RESEARCH METHODS LECTURE NOTES PAGE 40 Some other forms of experimenter–generated communication to research participants that may also result in experimenter bias may be less subtle and more blatant, although still unintended as illustrated in the participant instructions for a conflict resolution study presented below. Conflict Resolution Presented below are nine stories involving three different kinds of conflict—between individuals in personal relationships, between individuals in organizations, and between countries. You should read the story about each conflict, and read the seven styles of conflict resolution that follow each story. Your task will be to rate the desirability of each style of conflict resolution from 1 (poor) to 10 (excellent). In other words, how desirable is each of the proposed strategies for resolving the conflict presented in the story? In general, there is no one correct solution for a conflict–resolution situation. It is worth remembering, however, that more intelligent people tend to try to defuse rather than exacerbate the conflicts in which they find themselves. There are many ways to defuse the conflict, only some of which are given below. A key listing of the conflict resolution appears after the problems. Reducing experimenter bias 1. More than one experimenter should be included in the design of the study 2. Standardization of the experimenter's behavior in participant–experimenter interactions This can be done by standardizing the instructions and procedures, and in the maintenance of a constant environmental setting The use of end–of–session checklists that are completed by both the experimenter and participants to document their perceptions of the session has also been recommended 3. Double–blind procedures—are those that call for keeping the experimenter in the dark about the expected outcomes or hypothesis being tested as well as the participant. Double–blind procedures serve to minimize bias due to experimenter expectations. Both the participant and experimenter are kept in the dark and are unaware of the research hypotheses and conditions. In contrast, single–blind procedures are those in which (only) the participant is not fully informed of either the nature of the study or the conditions under which she/he is participating. These procedures may sometimes even take the form of deception. Single–blind procedures serve to reduce participant expectations but not experimenter bias or expectancies. EDWARDS—RESEARCH METHODS LECTURE NOTES PAGE 41 4. Automation—automate the procedures, treatments, and all other aspects of the experiment as much as is feasible. The advantage of automation is uniformity The disadvantage is inflexibility