Basics of Experimentation

9. During the course of the experiment, the human subject should be at liberty to bring Week 5 and 6: Basics of Experimentation the experiment to an end, if he has reached the physical or mental state, where con...

9. During the course of the experiment, the human subject should be at liberty to bring Week 5 and 6: Basics of Experimentation the experiment to an end, if he has reached the physical or mental state, where continuation of the experiment seemed to him to be impossible. Controlling Variables 10. During the course of the experiment, the scientist in charge must be prepared to Once you have formulated a terminate the experiment at any stage, if he hypothesis, the next step is to design a has probable cause to believe, in the exercise of the good faith, superior skill and procedure to test it. Much of the research careful judgement required of him, that a currently done in the field of psychology is continuation of the experiment is likely to experimental. Moreover, an understanding of result in injury, disability, or death to the the principles of experimentation strengthens experimental subject. any researcher's potential for success. When it is feasible, psychologists prefer experiments ["Trials of War Criminals before the Nuremberg to other research methods because a properly Military Tribunals under Control Council Law No. 10", Vol. 2, pp. 181-182. Washington, D.C.: U.S. conducted experiment allows us to draw Government Printing Office, 1949.] causal inferences about behavior. When an experiment is well conducted, it is high in internal validity. RESEARCH MISCONDUCT Let’s review the basics of experimental Ethical Principles of Psychologist and Code of psychology –those we discussed in this Conduct course and in the Introduction to Psychology. 5.01 Avoidance of False or Deceptive Statements In a psychology experiment, we first Psychologists do not make false, deceptive, or manipulate the antecedent conditions to fraudulent statements concerning; create at least two different treatment conditions. In other words, we manipulate the Their training, experience, or independent variable to create at least two competence, groups exposed in different levels of Their academic degrees independent variable. At least two treatments Their credentials are required so that we can make statements Their institutional or association about the impact of different sets of afliations, antecedents; if we used only one treatment, Their services, there would be no way to evaluate what The scientific or clinical basis for, or happens to behaviors as the conditions results or degree of succes of, their change. We expose subjects to different treatment conditions so that we can measure services the effects of those conditions on behavior. Their fees Their publications or research findings Why “at least two?” Because in other experiments, we may have more than two groups with different levels of independent DIFFERENCE OF FRAUD AND PLAGIARISM variables. After that, we record the responses or behaviors of subjects under various FRAUD IN SCIENCE IS TYPICALLY conditions and then compare them using THOUGHT OF AS FALSIFYING OR statistics. We can then assess whether our FABRICATING DATA. predictions are confirmed. PLAGIARISM MEANS REPRESENTING When conducted skillfully, an experiment allows us to make cause-and- SOMEONE ELSE’S WORK AS YOUR OWN. effect statements about behavior. If behavior changes as the antecedent conditions change, we can say that the differences in antecedent versus sad, succeeding versus failing- conditions caused the changes in behavior. psychological states can be independent However, a causal inference is justified only variables. Or experimenters may use when a carefully controlled test of our different sets of instructions as independent predictions has been made, and a researcher variables, such as instructing subjects to pay must consider several influences that can attention to the red circles or to the blue threaten an experiment's internal validity. squares. Other times, experimenters will compare the efficacy of different types In this module, we will discuss the of psychological interventions on problem basic methods and problems involved in behaviors; the type of intervention can be an planning a good experiment. independent variable. THE LIST OF INDEPENDENT AND DEPENDENT POTENTIAL INDEPENDENT VARIABLES IS ENDLESS VARIABLES In the simplest experiments, and In the example case, the independent experimental hypothesis states a potential variable is? Answer is found at the lower relationship between two variables: If A right. occurs, then we expect B to follow. In an font color experiment, variables are measurable elements that can vary or take on different To meet the definition of an experiment, at values along some dimension. The least two different treatment conditions are experimental hypothesis expresses a potential required; thus, an IV must be given at least two relationship between two kinds of variables, possible values in every experiment. The the independent and the dependent variable. researcher decides which values of the IV to use. These values are called the levels of the independent variable in the experiment. We will review the previous example we have: The researcher varies the levels of the If we will use colored fonts, then IV by creating different treatment conditions retention of the material improve. within the experiment. Each treatment condition represents one level of the IV. If the IV is to have two levels, there must be two different treatment What is the independent variable? conditions; if three values of the IV are to be used, there must be three different treatment The independent variable (IV) is the conditions, and so on. dimension that an experimenter intentionally manipulates; it is the antecedent the BE CAREFUL NOT TO CONFUSE experimenter chooses to vary. This variable is THE LEVELS OF THE IV WITH THE IV "independent" in the sense that its values are ITSELF. created by the experimenter and are not affected by anything else that happens in the experiment. Suppose you want to study font color. In Independent variables are sometimes your study, you want to investigate whether aspects of the physical environment that can black, red and green color can improve be brought under the experimenter's direct memory. How many IVs do you have? control. Lighting (bright or dim) and noise levels (loud or soft) are some examples There is only 1 IV and that is font color. of environmental variables. Aspects of a In the same study, how many levels of given task, such as difficulty (easy, hard) and independent variables do we have? meaningfulness (nonsense syllables versus real words) may also become independent There are 3 levels of IV – black, red and green color fonts. variables. Sometimes, experimenters manipulate experimental conditions so that subjects experience different psychological How about in ex post facto study? states: anxious versus non-anxious, happy  In an ex post facto study, the by the color of the paper or by personality researcher can explore the way behavior differences. Or, if we studied worker changes as a function of changes in variables productivity by placing incandescent lights in outside the researcher's control. These are one company and fluorescent lights in typically subject variables, characteristics of another, we could not be sure that the the subjects themselves (age, personality differences in productivity were caused by the characteristics, gender, and so on) that cannot lighting and not by pre-existing differences in be manipulated experimentally. It is common the two groups of subjects (like age, for researchers to refer to these quasi- disabilities, etc). This problem is known as experimental variables as independent confounding. variables too. Although the researcher does not manipulate them, quasi-experimental variables are often independent in the sense that the IN A TRUE EXPERIMENT, WE TEST THE researcher selects the particular values that will EFFECTS OF A MANIPULATED be included in the study. INDEPENDENT VAIABLE- NOT THE EFFECTS OF DIFFERENT KINDS OF SUBJECTS. In ex post facto and other quasi- WE HAVE TO MAKE CERTAIN THAT OUR experimental studies, researchers often TREATMENT GROUPS DO NOT CONSIST behave as if they were doing true experiments; OF PEOPLE WHO ARE DIFFERENTON A they have treatments and measured PREEXISTING CHARACTERISTIC. observations. However, in these kinds of studies, the independent variable is not WHAT IS THE DEPENDENT manipulated the way it is when we study VARIABLE? something under our direct control, such as a drug dose or the brightness of a light. In quasi- The dependent variable (DV) is the experiments, the researcher selects rather than particular behavior we expect to change creates the levels of the IV by assigning subjects because of our experimental treatment; it is to treatment groups on the basis of a subject the outcome we are trying to explain. In an variable. In an ex post facto study, individuals experiment, we are testing effects of the IV on in various treatment groups are supposed to the DV. Because we manipulate the IV and differ on a subject variable because the measure its effects on the DV, dependent subject variable is the one we are testing. variables are sometimes called dependent measures. If the hypothesis is correct, different In a true experiment, we test the effects of values of the independent variable should a manipulated independent variable--not the produce changes in the dependent variable. effects of different kinds of subjects. The dependent variable is dependent in the sense that its values are assumed to depend on the values of the independent variable Therefore, in a true experiment, we have to make certain that our treatment groups do not consist of people who are different on As the IV changes value we expect to preexisting characteristics. To guard against see corresponding changes in the value of systematic differences in people in our the DV. treatment groups, we randomly assign subjects to receive different treatments. If we simultaneously varied both a subject variable and a manipulated independent variable, there In the same study, what is our dependent would be no way of knowing whether effects variable? we observed were caused by differences Retention is our dependent variable because its value will depend and is expected to change because of the between subjects or by the IV. font color. For example, if we gave introverts their exams on blue paper and extroverts their exams on yellow paper, we would not know whether differences in their exam grades were caused  Questions: Dependent variables Independent variables Is the material going to be studied in the are also referred to are also referred to as: as: sample experiment considered an independent variable? treatments measures manipulations effects No. The independent variable in the interventions outcomes experiment is one that changes in levels or conditions results values to determine whether it has an effect to the dependent variable. In the experiment, therefore, it is expected that will Selecting appropriate independent and all have the same material to study and then dependent variables is an important part of measure the retention after the treatment designing every experiment. If we expect to conditions. understand the causes of behavior, we need to focus on the relevant antecedent variables. But Sir, you mentioned that any variable can be Remember, if we can specify the antecedents an independent variable. that lead to a particular behavior, we have explained that behavior from a scientific Correct. But not in this case. We have only one viewpoint. independent variable which is font color. If you wish to investigate the effect of difficulty or We also need to assess accurately the nature of material to retention and other related impact of our treatment conditions. In an factors, that may be reserved for another experiment, we cannot rely solely on our experiment. overall impression about whether the independent variable has some effect. We But can we have different materials for the same need more precision: We need an objective study? That is, the material will be different for measure of the effect of the independent the control and the experimental group? variable. We do not want the evaluation of the experiment's outcome to depend on our No. That will create confounding. It will subjective judgment, which might be invalidate your research. The existence of somewhat biased. In addition, our findings will confounding variables in studies make it difficult have more widely understood meaning if they to establish a clear causal link between are presented in terms of an observable treatment and outcome unless appropriate dimension that can be measured again and methods are used to adjust for the effect of the again. By clearly defining the way we are confounders measuring the effect of the independent variable, we make it easier for others to replicate our research. Can we have more than one independent variable in a study? Yes. You can have more than one independent variable. The simplest experiments will have one independent and one dependent variable but you can have two or more. Although the conduct of the experiment will be more difficult (and expensive), it is still feasible.  understand what was done and will be able to Will intelligence be an influential in the sample exactly replicate it. study? Operational definitions are quite Yes! That is why we have to moderate it and different from ordinary dictionary definitions. A make sure that the abilities of our dictionary may define retention as “the fact of participants later on will be average or at keeping something in one's memory”. least at par with each other. This is what we Although the definitions might be adequate for can an extraneous variable. Extraneous everyday use, it is insufficient in the context of variables are all variables, which are not the an experiment because they do not tell us how independent variable, but could affect the to produce different levels or values of the results of the experiment. variables. They do not give us procedures we could follow to make to produce more or less Can the weight be an extraneous variable? learning. Similarly, they contain no information on how to measure or quantify the variables. A variable may be extraneous if may affect the study. If no previous studies links weight to retention then it is not something you OPERATIONAL DEFINITIONS OF will need to control or manage in an experiment. VARIABLE: Operational Definition IT SPECIFIES THE PRECISE MEANING OF A VARIABLE WITHIN When we run an experiment, we AN EXPERIMENT naturally want others will understand what we IT DEFINES A VARIABLE IN TERMS have done. Many concepts have more than OF OBSERVABLE OPERATIONS, one meaning, and those meanings are often PROCEDURES AND vague. If we study variables without defining MEASUREMENTS them exactly, the meaning of our findings will be unclear. As scientists, we also want to be THEY ARE NOT ORDINARY sure that our procedures are stated clearly DICTIONARY DEFINITIONS. enough to enable other researchers to replicate our findings. Defining the Independent Variable: Thus, each IV and each DV has two EXPERIMENTAL OPERATIONAL definitions-ONE CONCEPTUAL DEFINITION DEFINITIONS THAT IS USED IN EVERY LANGUAGE AND ONE OPERATIONAL DEFINITION THAT IS Experimental operational definitions USED IN CARRYING OUT THE explain the precise meaning of the EXPERIMENT. independent variables; these definitions describe exactly what was done to create the An operational definition specifies various treatment conditions of the the precise meaning of a variable within an experiment. An experimental operational experiment: It defines a variable terms of definition includes all the steps that were observable operations, procedures, and followed to set up each value of the measurements. It is an operational definition independent variable. because it clearly describes the operations involved in manipulating or measuring the Suppose we will define font color as: variables in an experiment. RED font. Do you think other researchers will understand the how you conducted your These statements are essential study? Not quite actually. Other researchers because many variables of interest to might ask, how dark or how light is the red psychologists cannot be observed directly, color? What material did you print with the red and operational definitions describe variables color? And many others. in terms of observable reality. We include operational definitions in written reports of In our sample experiment, we can experiments so that other researchers will define font color like this:  Font color is defined as the visual quality of a material. In this research, the piece, The Hill We Climb by Amanda Gorman is printed with Hypothesis: If people will drink coffee before a typeface Arial at 11pt. either with a red font working, then their productivity will improve. with code #ec1346, blue font with code #1741e8 or black font with code #000000 Independent variable: Coffee white short bond paper of 70gsm. Material is read for 15 minutes in a well-lit quiet room. Levels of Independent variable: 3 (no coffee, black coffee, coffee with cream) Is it more understandable now? Dependent variable: Productivity Defining the Dependent Variable: MEASURED OPERATIONAL DEFINITIONS How will you define the independent and dependent variable? Dependent variables are defined by measured operational definitions, which describe what we do to measure the variables. Surely you will need to define coffee clearly such Measured operational definitions of the that other researchers will understand how you conducted your study. What coffee did you use? What brand if dependent variable describe exactly what possible? How much coffee did each of the participant drink? procedures we follow to assess the impact of What is the ratio of coffee to the cream? Does black coffee different treatment conditions. These mean pure barako or does it have sugar? If so, how much? definitions include exact descriptions of the When did they drink it? specific behaviors or responses recorded and explain how those responses are scored. How about productivity? What activity would you ask If we are using scores on a participants to do? What materials and equipment did you standardized test to measure our dependent need which you must specify? How you measure them? variable, we identify the test by name "scores on the Culture Fair Intelligence Test," not simply "scores on an intelligence test." If our dependent measure not standardized, we EXTRANEOUS VARIABLES describe it in enough detail to allow other researchers to repeat our procedures. ANY VARIABLE THAT YOU ARE NOT INTENTIONALLY STUDYING THAT COULD THREATEN Suppose we will define retention as: THE INTERNAL VALIDITY OF YOUR RESULTS. amount of material retained in the minds of students. Will other researchers understand When we experiment we want to create what you mean? Unlikely. Other researchers treatment conditions that will let us clearly see might ask: how did you measure retention? the effects of independent variables. We want What method did you use to measure our experiments to be internally valid, retention? therefore, only the independent variable should change systematically from one In our sample experiment, we can condition to another. In some instances, some retention like this: variables can also threaten an experiment's Retention is defined as the number of external validity by reducing the correct response from a 50-item self-made fill- generalizability of the findings. These are in-the-blank test of the piece, The Hill We called extraneous variables. Climb by Amanda Gorman. Extraneous variables are all variables, which are not the independent variable, but Is it more understandable now? could affect the results of the experiment. Each poses special problems in an experiment. Many can be controlled by the Try to think about this experiment: same procedures, but some require special procedures. Amount of coffee intake and its effect to the productivity of workers. Recall: Confounding occurs when something in increase the chances of an internally valid the experiment changes systematically along with experiment. the independent variable. PHYSICAL VARIABLES ELIMINATION AND CONSTANCY These are aspects of the environment that To make sure that an extraneous might affect the participant’s behavior variable does not affect an experiment, sometimes we just take it out-we eliminate it. If Case: Poor Emily was trying to run a memory noise might confound the results, we test in a experiment that required a lot of concentration. quiet location. If we do not want interruptions, On Thursday, her first day of testing, Emily we hang a sign on the door saying, "Do not recruited subjects in the library and tested disturb. Experiment in progress." them in a quiet reading room. The next day she came back to run the rest of the Ideally, we would like to eliminate all experiment. To her dismay, Emily found that extraneous variables from an experiment, but the reading room closed early on Fridays. The this is easier said than done. Sometimes there only place she could test her subjects was the is no quiet location. Factors such as the building's lobby. It was fairly quiet there, but weather, the lighting, and the paint on the walls people walked by now and then, laughing and are simply there; we cannot eliminate them. talking about plans for the weekend. Instead, we use the second control procedure constancy of conditions. Emily cried, "What a dummy I am! These testing conditions will confound my Constancy of conditions simply means experiment! Why did I run all of my subjects that we keep all aspects of the treatment getting Treatment A yesterday?" conditions as nearly similar as possible. If we cannot eliminate an extraneous variable, we try to make sure that it stays the same in all The day of the week, the testing room, treatment conditions (sana all di nagbabago). the noise, the distractions are all physical We cannot take the paint off the walls, but we variables, aspects of the testing conditions can test all subjects in the same room. That that need to be controlled. Emily's experiment way, we make sure that the pea-green walls was in real trouble because she ran all offend all subjects equally in all conditions. subjects getting Treatment A on a Thursday The same goes for lighting, the comfort of the and all of those getting treatment B on a chairs, the mustiness of the drapes; all stay the Friday. The A group was tested under quiet same for all the subjects. We also try to keep the conditions in one setting; the B group was mechanics of the testing procedures the tested in a different place with more noise and same. For instance, it is helpful to write out distractions. Clearly, there was confounding in instructions to subject before beginning the Emily's experiment because the testing experiment. The written instructions are then conditions changed along with the read to subjects to guarantee that all subjects independent variable. She could have avoided in each condition get exactly the same her problems by using one of the three general instructions. Audio or videotaping the techniques for controlling physical variables: instructions might be even better. Exactly the elimination, constancy of conditions, and same amount of time is allowed for each balancing. subject complete each task-unless time is the independent or dependent variable. Many physical variables such as time of testing, TRIVIA: Many of our researchers try to avoid running any experiments on Fridays. Possibly because they are worried testing location, and mechanical procedure that subjects will be less motivated to participate, that fewer can be kept constant with a little effort. An will actually show up, or whether they themselves are just experimenter may end up controlling some looking forward to the weekend (Myers and Hansen, 2012) variables that would not have affected the We cannot possibly identify all the extraneous results anyway, but it is better to use the variables that influence the outcome of a controls than to have regrets later. If someone study--but we need to try to find as many as can punch holes in the results simply by we can. By using control techniques, we pointing out that the A group had lunch but the B group did not, the experimenter will have a hard time making a strong care for the effects Notice that we have not wiped out the of the independent variable. differences between the two testing rooms; they are just as different as ever. However, the hope is Simply put, everything should be the that the effects of the rooms are the same, or same should the groups be separated! The balanced, for both treatment conditions. For every main difference only is the level of A subject tested in the green room, a B subject is independent variable! also tested in that room; for every A subject tested in the pink room, a B subject is also tested there. Emily could have salvaged her memory BALANCING experiment by using balancing. Instead of testing all treatment A subjects on the first day, she should Sometimes neither elimination nor have randomly assigned each subject to either constancy can be used. Perhaps some variables treatment A or treatment B. Then she would have cannot be eliminated. For example, we would like tested, by chance, roughly half the A subjects and to test in a quiet room, but we do not have access half the B subjects on Thursday. Roughly half the to one. We would like to test all subjects together A subjects would have taken part in the quiet at the same time, but they cannot all come at once. reading room along with roughly half the B What can we do in these situations? subjects. On the second day, Emily would have We know that ideally we should not test continued assigning subjects to the two treatment subjects in two different rooms. But perhaps we conditions at random. She would then have tested have no choice; it is two rooms or nothing. We still about half the A subjects and half the B subjects in want to be sure testing conditions do not change in the noisy lobby. Notice that she still would have a way that is related to the independent variable. had subjects who were tested under two different We can do this through balancing, distributing the testing conditions. But the effects of these effects of an extraneous variable across the different conditions would have been about the same for the treatment conditions of the experiment. two treatments in the experiment, so the testing conditions would not have confounded the results One way that we might do this with room of her experiment. assignment is shown below: We can use balancing for many other variables as well. For example, if we cannot test all subjects at the same time of day, we can arrange Balancing the effects of the testing room things so that we test equal numbers of treatment across two treatment conditions (A and B) A and treatment B subjects before and after lunch. Many physical variables will be balanced across Green Room Pink Room conditions automatically as we assign our subjects to treatment conditions at random. Time of testing, A1 A4 weather conditions, and day of the week are typically controlled in this way. Usually, we do not A2 A5 even think about these sorts of extraneous variables. As long as there is no systematic change in an extraneous variable, things are fine. A3 A6 By using balancing, the study would B1 B4 have internal validity and greater external validity. A cautionary note: When you design your procedures, do not sacrifice internal B2 B5 validity for external validity because internal validity is more important. You cannot have B3 B6 external validity unless a study, first, has internal validity! Note: Half the subjects are assigned randomly to each testing room. Half the subjects in each room are then Remember: assigned randomly to A, half to B. Eliminate extraneous variables whenever you can. We begin by randomly assigning half the subjects to the first testing room. The other Keep treatment conditions as similar as possible. half will be tested in the second room. Next, we randomly assign half the subjects in each room Balance out the effects of other variables, such as the to the treatment condition A; the remaining testing room, by making sure that the effects are subjects will be in treatment condition B. distributed evenly across all treatment conditions. As always, be sure to assign individual subjects to the proper role of a subject. They might not even be treatment conditions at random. consciously aware of the ways in which they alter their behavior when they come into an experiment. For example, subjects might assume a very active role. They might try to guess the hypothesis of the experiment and adjust their responses accordingly. SOCIAL VARIABLES An experimenter generally wants Besides controlling physical variables that participants to be as naive as possible. They might alter the outcome of the experiment, should understand the nature and purpose of the researchers are concerned about social variables, experiment but not the exact hypothesis. The qualities of the relationships between subjects reason for this is simple. If subjects know what we and experimenters that can influence results. expect to find they might try to produce data that will support the hypothesis. WE DO NOT WANT TO INFLUENCE THE OUTCOME OF THE EXPERIMENT BY HAVING On the surface, that might seem like a good SUBJECTS KNOW THE YPOTHESIS. thing. Wouldn't it be wonderful if experiments always confirmed their hypotheses? It would—if the experiments were valid. We want to be able to PARTICIPANTS WILL BE AFFECTED BY: say that the independent variable caused a change in behavior. If behavior changes simply because 1. THEIR SURROUNDINGS subjects think the researcher wants an experiment to turn out in a particular way, the experiment has 2. THE RESEARCHER’S CHARACTERISTICS not measured what it was intended to measure. (uwu) 3. THE RESEARCHER’S BEHAVIOR Does this sort of thing really happen? Do 4. THEIR INTERPRETATION OF WHAT IS subjects' experiences and answers change GOING ON IN THE SITUATION. depending on what they think is supposed to happen in the experiment? Can the demand Two principal social variables, demand characteristics of an experiment lead to changes in characteristics and experimenter bias, can be subjects' behavior? What if subjects simply do controlled through single- and double-blind what they think they are supposed to do? In an experiments. experiment that has now become classic, Orne and Scheibe (1964) devised a clever experiment DEMAND CHARACTERISTICS to evaluate the effects of demand characteristics in Demand characteristics are aspects of the research on sensory deprivation. experimental situation that demand that people behave in a particular way. Traditionally, participants in sensory- Have you ever noticed that people behave in a deprivation experiments were isolated from very peculiar way in elevators! Elevator sensory experiences of all kinds. They wore passengers who are unacquainted rarely look at translucent goggles so that they could not see each other; instead, they stare at some point up on shapes clearly: they were typically restricted to the walls in front of them or move their eyes across lying on a soft bed; they wore coverings over their the ceiling-anywhere but toward the other arms to minimize sensations from movement and passengers. You have probably found yourself touch. White noise (noise of all frequencies that avoiding eye contact with strangers on elevators, sounds very much like static on a radio) was too. This is a good example of what we mean by played to mask ordinary sounds. With these demand characteristics. procedures, prolonged sensory deprivation led to What we do is often shaped by what we emotional upset, disorientation, and even think is expected behavior in a particular hallucinations. Orne and Scheibe followed a very situation. When you enter a classroom, even on different procedure. They placed their subjects the first day of the term, you probably walk in and alone in a room that hardly created any deprivation take a seat. When the professor begins talking, at all. The room was 6 x 7 x 8 feet in size and was you stop what you are doing and listen. You are furnished with a desk and two chairs. The subjects fulfilling a role you learned in your earliest days at could easily hear sounds from the hallway and school. The cue of being in the classroom leads birds chirping outside. They did not wear goggles you to behave in a predictable way. and so had their normal vision. There was water and a sandwich in case they got hungry. They The same applies for research were given a set of arithmetic problems to work on subjects. They want to conform to what they think is if they wanted something to do, and they could move around as much as they wanted to within the Orne and Scheibe's findings implicated room. demand characteristics as a cause of some of the prior sensory-deprivation findings: Subjects led to The difference between the two groups in this expect some strange experience showed study was the way the experiment was presented significantly more signs of disturbance. Compared to each group. In the first treatment condition, with the controls, the experimental subjects gave subjects were interviewed by a man in a white the "impression of almost being tortured" (1964, p. medical coat. A tray of drugs and medical 11). All the subjects in Orne and Scheibe's instruments, labeled "Emergency Tray," was in the experiment experienced the same "deprivation," room. Subjects were told that there was an alarm but only the experimental group showed the usual button in the "deprivation chamber." If they effects of sensory deprivation. For the became upset, they were advised to press the experimental group, the researchers had created the alarm button. They were also told that a physician impression that something unusual would happen to was present. Pulse and blood pressure readings them. The subjects' expectations were confirmed; were taken. Everything possible was done to they experienced a variety of changes that did not create an atmosphere of seriousness and occur for the control subjects, who had a different preparation for a possible crisis. Subjects were set of expectations. The changes were varied and also read some instructions for reporting their at times dramatic: "The buzzing of the fluorescent experiences while in the chamber, including light is growing alternately louder and softer so that possible visual imagery, unusual feelings, and at times it sounds like a jackhammer"; "There are disorientation-the typical sensory-deprivation multicolored spots on the wall"; "The numbers on effects. the number sheets are blurring and assuming Pinilit ko ito gawin. Sorry na. various inkblot forms" (p. 10). Indeed, one subject hit the panic button and listed "disorganization of senses” as one of his reasons for stopping. These Subjects in the other treatment group were findings do not rule out the possibility that some told that they were "control" subjects who were not genuine changes occur when subjects undergo an receiving the experimental treatment. The actual restriction of sensory experience. They do, experimenter wore regular business clothes. however, illustrate the importance of demand There was no emergency tray or alarm button. characteristics in shaping the outcome of such Subjects were told that if they wanted to leave the studies. chamber, they could simply knock on the window. They were given the same general instructions to report their experiences, but everything was done Controlling Demand Characteristics: to create a relaxed impression. When the blood SINGLE-BLIND EXPERIMENTS pressure and pulse readings were made, the control subjects were told that the readings were When we run experiments, we try not to give being taken simply because they had been taken participants clues about what may happen to for the experimental subjects. If we compare the them when they are exposed to the two procedures, we see that Orne and Scheibe independent variable. We do not want to manipulated demand characteristics in their influence the outcome of the experiment by experiment. All subjects were placed in the same having subjects know the hypothesis. A good "deprivation" setting, but the experimental subjects way to control some effects of demand were given every reason to expect that something characteristics is through a single-blind awful, or at least our of the ordinary, might happen experiment, an experiment in which to them. The "control" subjects were treated in a more casual way that probably communicated subjects do not know which treatment they something like, "You'll be in this room for a while." are getting. What would you expect to happen in this When we do a single-blind experiment, we experiment? Would subjects show the usual can disclose some but not all information about the effects of sensory deprivation if we look only at the experiment to subjects. We can disclose what is physical layout of this experiment, we can see that going to happen to them in the experiment we can there is little reason to expect anyone to become also keep them fully informed about the purpose of disoriented or show any other unusual symptoms the study. But we keep them "blind" to one in a short period of time. The subjects had full use thing: We do not tell them what treatment condition of all their senses. Their movements were not they are in. (Sounds family? This is a demonstration of incomplete restricted, they could eat, and they had a task to disclosure too) do if they got bored. This approach is very common in experiments with drugs. If we give a subject some substance, the subject might react based on what We are investigating the effects of room lighting he or she expects the drug to do. For instance, on learning. We are asking people to learn a list suppose we want to test a new drug designed to of 10 made-up words as quickly as possible. reduce anxiety. If we give the drug to several Different people will learn the list under different anxious individuals, some of them will report that lighting conditions, and we will see whether they feel better. But, did the medicine help? We lighting makes a difference. don't know. To answer this question, we need to compare their behavior with that of a group of We have given subjects information about subjects who did not receive the drug. the purpose of the experiment and have told them Researchers know that if you give a person any what would happen to them during the experiment. pill, the person is apt to say that the pill helped. Notice that we have not disclosed our expectations about their performance, and we have not told We call this the placebo effect. To control them which condition they are in. for the possibility of a placebo effect, researchers give one group of subjects a placebo-a pill, an But have we structured the situation so that these injection, or other treatment that contains none of parameters are fairly simple to figure out? the actual medication. Subjects are not told whether Probably we have. Subjects are quite likely to they are receiving the actual drug or the placebo. figure out our hypothesis and behave accordingly Because subjects do not know what effects to expect, If they believe the light is normal or even a bit dim, changes in their behavior are more likely to be they may take longer to learn the list simply caused by the independent variable. because they expect it to take longer. If the light seems bright, they may learn the list more quickly Subjects, however, typically know they are because they expect the added light makes getting some kind of treatment, and so we may be learning easier. This could confound the able to measure the actual effects of a drug by experiment because the expectation vanes itself. Instead, we see the effects of treatment plus systematically with the independent variable it placebo effects that are shaped by the subjects' would take the longer in the 75-watt condition than expectations. We then compare those effects with in the 150-watt condition, just as we predicted, but the effects of the placebo alone. Accounting for the difference might nor be due to the light levels placebo effects has become common in drug at all. studies because the effects can be huge. One meta-analytic study of clinical drug trials with 2,318 depressed patients (Kirsh & What alternatives do we have? We could Saperstein, 1998) showed that about one-fourth of give them less information. We could simply say the reported reductions in depression symptoms that we are investigating the influence of different were probably caused by actual effects of study environments on learning. This might keep antidepressive drugs, another fourth by the natural them from guessing our true hypothesis or it might course of the illness, and over half of the not. Perhaps some experimental subjects guess reductions were probably caused by the placebo that we are testing quiet versus noisy spaces or effect. different kinds of chairs or different colored walls. Whatever they guess, there is always the possibility that their guess will be confirmed by the way they behave. Whatever they guess might alter In other kinds of experiments, we can get the time it would normally take to learn the list. placebo effects too. Suppose we want to conduct Nevertheless, most researchers would agree that a simple learning study; we want to see whether this situation is somewhat better than the first. the brightness of a room influences how casily Chances are that individual subjects would guess people learn. We will test half the subjects in a different things and behave in different ways, but room with normal illumination (a 75-watt bulb), and their behavior would not change systematically we will test the other half under brighter-than- along with the independent variable. This situation normal illumination (a 150-watt bulb). Both groups would not be a confound, but it would certainly will be asked to memorize a list of 10 nonsense make the impact of the independent variable more words-like bragzap and crumdip. The length of difficult to detect. time it takes subjects to memorize all 10 words will be the dependent measure. We want to conduct a single-blind study. Therefore, we might tell subjects: Controlling Demand Characteristics: Cover influenced their self-descriptions. Subjects who Stories were dressed casually rated themselves as having more casual traits than did those who were There is another alternative for controlling dressed more formally. the possibility that subjects may guess the experimental hypothesis-we could use a cover Hannover and Kuhn (2002) could simply story, which is a plausible but false explanation for the have instructed subjects to dress either casualy or procedures used in the study. It is told to disguise the formally for the experiment, told them the actual research hypothesis so that subjects will not experimental hypothesis, and had them guess what it is. themselves on the traits once they arrived for their (Sounds family? This exemplifies deception) experimental session. But if the results had come out the way the researchers predicted, would you Numerous studies have shown that the have been convinced? Probably not. What if the way you dress can impact the kind of impression researchers did not disclose the hypothesis but you make on others. Professors who dress simply instructed subjects to come to the lab casually, for example, are typically judged by dressed a certain way and rate themselves on the students as being more approachable but less traits? Would you have been convinced if the data knowledgeable than if they dress more formally. came out as predicted? No, because there is a Similarly, students who dress casually tend to be strong chance that subjects might figure out such judged by their teachers as less intelligent and a simple hypothesis: "You told me to dress this way lower in academic potential than if they dress more because you want me to rate myself certain way, formally. This effect is usually explained as an so I’ll rate myself that way." example of how individuals apply stereotypes to form impressions. This is the type of situation in which a cover story must be developed. And we have to be clever Hannover and Kuhnan (2002), however, because the cover story has to give the subjects explored an alternative hypothesis in an interesting an explanation for what we want them to do dress way. They believed that the style of dress might formally or casually-without tipping them off to the actually influence how people feel about hypothesis being tested. The cover story provided themselves and, therefore, change the way they subjects with an explanation for dressing in a behave toward others, resulting in people forming certain way that had nothing to do with the different impressions of them. To test this, they had experimental hypothesis. Given this plausible college students rate themselves on a number of explanation, subjects would not look for another traits while dressed in one of two ways: casually explanation and would be unlikely to discover the (jeans and a sweatshirt) or more formally (a blazer real experimental hypothesis. Of course, and nice pants or skirt). But Hannover and Kuhnan Hannover and Kuhnan explained the real purpose were justifiably concerned that their subjects might of the style of dress to subjects at the end of the guess the experimental hypothesis, so they experimental session. created a clever cover story. Remember: Cover stories involve A few days before the experiment, subjects deception. Deception is a departure from fully were informed that they would be participating in a informed consent, and you will need to study of eyewitness identification and that they persuade your IRB that using deception is the would be part of a "police line-up." Thus, they only way to test your hypothesis. Researchers needed to show up at the lab dressed similarly to must increasingly be concerned about the rights of a description of the ostensible criminal-half were the participant as the departure from fully informed told to dress casually; the other half were told to consent becomes more dramatic. For this reason, dress more formally. To maintain the cover story, cover stories should be used sparingly. Remember each participant was told to stand in front of a one- that debriefing is required for all subjects in such way mirror and then to walk slowly back and forth experiments. If you believe your experiment can so that the "eyewitnesses in the other room could have internal validity without a cover story, do not see them. Then, subjects were taken to another use one. room where they participated in a supposedly unrelated trait study to help out graduate student EXPERIMENTER BIAS who was working on a dissertation. Here, they Perhaps without realizing it, an were asked to press computer keys labeled "me" experimenter can give subjects cues that tell them or "not me" to describe themselves on 40 different how the experimenter would like them to respond. traits. The traits had been pretested to reflect traits Subjects will often comply with these subtle generally ascribed to people who were casually or requests and give the data the experimenter is formally dressed (e.g., easygoing, tolerant, seeking. emotional versus cultivated, accurate, restrained). As predicted, subjects' own style of dress strongly  Imagine the experimenter running a Experimenters might also treat subjects perception experiment. She asks, "Does the light differently depending on what they expect from in the circular path move faster, slower, or at the them. They might give more time to subject’s who same speed as the light in the square path?" As have gone through a particular treatment. This she says "faster," she leans forward slightly, raises outcome is called the Rosenthal effect after the her eyebrows, and speaks a little louder. Most of man who first reported it. It also called her subjects say that the light in the circular path the Pygmalion effect, after the legend in which the moved "faster." We call this sort of sculptor Pygmalion fell in love with own statue of influence experimenter bias; the experimenter does the perfect woman. The Rosenthal effect can be something that creates confounding in the another source of confounding in an experiment. experiment. The experimenter could give a cue to respond in a particular way or might behave Experimenter bias occurs in other ways, differently in different treatment conditions. too. Experimenter might also make errors in recording the data from the experiment. They Dr. R. might be warm and friendly in one might "misread" a scale or score an item treatment condition, but he might seem indifferent incorrectly. in the other condition. That is all right if the experimenter's demeanor is the independent variable. If it is not, it can confound the results. The Pygmalion Effect Subjects in the first treatment condition might feel The work of Rosenthal and Jacobsen (1968), among more at ease and so perform better. Sometimes, others, shows that teacher expectations influence as in the case of Dr. R., the nature of the student performance. Positive expectations influence treatments can bring on experimenter bias. Dr. R. performance positively, and negative expectations might have found the first treatment condition influence performance negatively. Rosenthal and much more interesting than the second. If so, then Jacobson originally described the phenomenon as the Dr. R. probably found it much more fun to run Pygmalion Effect. sessions in which subjects received the first treatment, and his preference showed up as “When we expect certain behaviors of others, we are increased warmth toward subjects in this condition. likely to act in ways that make the expected behavior But experimenter bias doesn't always work in this more likely to occur.” (Rosenthal and Babad, 1985) way. In terms of teaching, faculty who gripe about students establish a climate of failure, but faculty who value their students’ abilities create a climate of success. What Imagine that one of Dr. R's.conditions was kind of learning climate are you creating through your particularly noxious for subjects and Dr. R. knew it. expectations? Researchers do not enjoy putting subjects through unpleasant procedures. Every time Dr. R. knew Pygmalion in Tradition that a subject would be undergoing the noxious Pygmalion in Ovid’s Metamorphoses (Book X) was a procedure, he became somewhat anxious, and he sculptor who fell in love with an ivory statue of his own transmitted this anxiety to subjects. As you can making. Enamored by the beauty of his own making, imagine, his behavior would be a particular Pygmalion begs the gods to give him a wife in the problem if the dependent variable was anxiety. If likeness of the statue. The gods grant the request, and Dr. R’s hypothesis was that the noxious procedure the statue comes to life. George Bernard Shaw would make subjects more anxious than the other adopted Pygmalion for the title of his play about procedure, his experiment would be confounded. Professor Henry Higgins whose sense of self-efficacy Subjects in the noxious condition could have is grandiose: “You see this creature with her curbstone picked up the cue from Dr. R. that there was English... in three months I could pass that girl off as something to be anxious about, and so they were. a duchess at an ambassador’s garden party.” Effects on subjects' anxiety might have had every bit as much to do with Dr. R’s demeanor as with Pygmalion Research in the Classroom the independent variable. So far it seems that The original research of Rosenthal and Jacobsen experimenter effects can be a problem only with focused on an experiment at an elementary school human subjects, but experimenter effects can be where students took intelligence pre-tests. Rosenthal just as important in animal studies as they are in and Jacobsen then informed the teachers of the names human ones. After all, a rat will not notice smile of twenty percent of the students in the school who and then learn faster. Probably not, but were showing “unusual potential for intellectual growth” experimenters might handle rats more and handle and would bloom academically within the year. them more gently if they think the rats are special. Unknown to the teachers, these students were selected randomly with no relation to the initial test. When Rosenthal and Jacobson tested the students eight months later, they discovered that the randomly differently depending on which child drew the selected students who teachers thought would bloom picture. Sometimes it is not clear how we should scored significantly higher. Rosenthal insists that the score a particular item. LaTanya drew what Pygmalion effect also applies to higher education: appears to be a green orange. Oranges really are There've been experiments looking at college algebra green before they get ripe, but they are usually classes at the Air Force Academy, a study of orange in pictures. Should we score this picture as undergraduates in engineering; there've been lots of "abstract" or not? Scoring the dependent studies at the college level since the book came out measures is no time to be making subjective confirming the findings... In fact, the original research judgments. We should have worked out these conducted when I was at the University of North kinds of issues before running the experiment. But Dakota was all done with graduate students and under- if these questions do arise in scoring, we might find graduates (Rhem, 1999). Why does the Pygmalion ourselves deciding in favor of the more abstract effect occur? “If you think your students can’t achieve rating if we knew we were scoring a picture drawn very much, are not too bright, you may be inclined to by a child who saw the cartoon. We might distort teach simple stuff, do lots of drills, read from your the scoring a little by being inconsistent; we would notes, give simple assignments calling for simplistic bias the data so that they would support our own answers” (Rhem, 1999). hypothesis. In doing so, we have created an instrumentation threat to the internal validity of the experiment. It is easy to do this without even realizing it. Controlling Experimenter Bias: Double-Blind Experiments One of the best ways of controlling for experimenter bias is to run a double-blind How can we eliminate experimenter effects from experiment, one in which the subjects do not know research? The first step, of course, is to be aware of which treatment they are receiving, and the them. We want to be sure we do not do anything experimenter does not know either. The value of the that will contaminate the data. By following a set of double-blind experiment is this: If the experimenter written directions, timing all phases of the does not know which treatment the subject is getting, experiment, and being as consistent as possible, he or she cannot bias the responses in any systematic we can avoid some mistakes. We make our way. The subject is kept in the dark, to o, so the observations as objective as possible. We try to set effects of demand characteristics are up experiments to minimize the amount of controlled along with experimenter bias. personal contact between an experimenter and a subject so that unintentional bias does not happen. Remember: But sometimes we just cannot anticipate how bias Single-blind procedures can control for can creep into an experiment. demand characteristics in these studies by keeping subjects unaware of their condition, but single-blind procedures do not control for Let us say we are doing a study of cartoons experimenter bias. Whenever the experimenter is and children's art. We want to see whether children aware of the drug condition of the subject, it is who have just watched a cartoon will draw more possible that measurement of the dependent abstract pictures than will children who have just variable can be biased by this knowledge. watched a filmed version of the same story. The Therefore, whenever possible, the best procedure cartoon group sees animated drawings of people; to use in these studies is the double-blind. the film group sees real people acting out the same story. We show the children the cartoon or film, PERSONALITY VARIABLES and then we ask them to draw pictures. We have developed a way of scoring the Experimenters children's drawings for abstractness. Our measure Personal characteristics, or personality includes a number of specific dimensions, such as variables, that an experimenter brings to the whether objects are colored in their true-to-life experimental setting can be important, too. An colors. We will score each child's picture on the experimenter who is warm and friendly can elicit abstractness scale. If our hypothesis is correct,the very different responses from subjects than can drawings of the children who saw the cartoon will one who is cold and aloof: Subjects sometimes be more abstract than the drawings of children who learn better, talk more, get better scores on saw the film. intelligence and adjustment tests, and are typically As we sit down to score the pictures, we more compliant and eager to please when the notice different features in the drawings. We might experimenter acts in a friendly manner (Rosenthal, also notice that we tend to score somewhat 1976).  Rosenthal and Rosnow (1969) conveniently Social desirability as a tendency to present oneself summarize the effects. For example, results of in a better light rather than in a truthful manner. People naturally want others to view them personality tests have shown that volunteers are favorably with respect to socially acceptable likely to be more sociable and score higher in values, behaviors, beliefs, and opinions. social desirability than nonvolunteers. Volunteers tend to hold more liberal social and political attitudes and tend to be less authoritarian than Apparently, experimenters who score high nonvolunteers. Interestingly, people willing to on the social desirability scale are particularly good volunteer for experiments also tend to score higher at being likeable experimenters; they are more on intelligence tests, on the average, than do enthusiastic, more friendly, and smile more at their nonvolunteers. Whether these differences remain subjects. Interestingly, experimenters who come today is an empirical question, but if they do exist, across as likeable can alter the kinds of responses, they would suggest that experiments using as well as the number of responses, obtained from university samples might have somewhat less subjects. For instance, studies on interviewing external validity than researchers would wish. styles have shown that a likeable interviewer will These differences would be expected to limit the collect better data, more usable responses, and generalizability of some kinds of experiments more fewer "I don't know's" from respondents. But keep than others. in mind that an experimenter who improves subjects' performance might attenuate an -RANDOM ASSIGNMENTS experiment's external validity; such high levels of performance might not be obtained in another Context Variables setting. Conversely, an experimenter who appeared hostile, aggressive, or authoritarian -SUBJECTS SELECT THE EXPERIMENT (demanding obedience) would get less than -THOSE THAT COME ABOUT FROM PROCEDURES optimal performance from research subjects (Rosenthal, 1976). The research setting is a novel CREATED BY THE ENVIRONMENT, OR CONTEXT OF one for most subjects, and they are often a bit THE RESEARCH SETTING anxious. We want to establish rapport with subjects so that they will feel at ease, but (unless Context variables are those that come the experimenter's demeanor is the independent about from procedures created by the variable) we do not want to make subjects behave environment, or context, of the research setting. in an atypical way. So be pleasant, but remember Context variables include subject recruitment, that you can affect the outcome your experiment. selection, and assignment procedures, as well as typical problems encountered in research on a It is extremely important to maintain university population. There are two basic kinds of consistency in your interactions with subjects. context variables: those occurring when subjects Remember that when you act more friendly (or select their own experiments and those produced hostile, or anxious, etc.) toward subjects from time when experimenters select their own subjects. to time, you are introducing another source of variability into the experiment. The more you vary your behavior, the more you are likely to produce When the Subjects Select the Experiment variability in the responses of your subjects. The more variability in their responses, the harder it will When you run an experiment using human be to detect the effect of the independent variable. subjects, you may be allowed to recruit your Most important, however you act toward subjects, subjects from a subject pool of psychology do it consistently across all the treatment students. Most departments of psychology in major conditions. Do not allow your behavior to change universities rely on students to serve as research systematically along with the conditions of the subjects from time to time. At research universities, experiment. If your behavior is different in the psychology students are typically required either to experimental and control conditions, the participate in a certain number of experiments or experiment will not have internal validity because to fulfill a substitute requirement. Frequently, a list you are a potential history confound. With many of experiments needing subjects is posted on a personality variables, the influence is website or in a central location on campus. unpredictable and can interact with characteristics Sometimes students can sign up for a session on of particular subjects. the spot, or other arrangements can be made. Students can often look over what is available, Volunteer Subjects select one that seems interesting, and make an During the 1960s, it was popular to conduct appointment to participate. studies looking at the kinds of personality variables Usually, two factors are involved in the that discriminated volunteers from nonvolunteers; decision Am I free at a time when subjects are needed? even if person 5 is the president of the university or has six rings in each nostril. You must plan your And what kind of experiment is it? procedures acquiring subjects as carefully as you Often, the only information that potential plan your experimental procedures, and you must subjects are given is the name of the experiment. keep your demeanor uniform throughout the Sometimes these titles can bias your sample, selection process. reducing the external validity of your experiment. Suppose students have the following two choices: "The Memory Test Experiment" and "The Heavy Metal Music Experiment." Do you think students who volunteer for the two experiments might be somewhat different? Of course. To avoid getting a biased sample, researchers try to keep the names as neutral as possible, without sounding boring: "The Impression Formation Experiment," "The Close Relationships Experiment." Instead of "The Sexual Practices Survey," a questionnaire study could be called "Contemporary Judgments." A biased sample greatly reduces your ability to generalize your findings; the experiment will have less external validity. The time for subjects to decide not to participate in your experiment, if that happens, is after the experiment has been explained to them at the beginning of the session. Always keep a record of subjects who drop out (and what condition of the experiment they were assigned to) for your research report, even though it is unlikely that your experiment will include conditions that are noxious enough to produce a subject mortality threat. Keep in mind that some subjects who volunteer probably will miss their sessions for a variety of reasons so it is always a good idea to plan for about 10% more sessions that you actually need. When the Experimenter Selects the Subjects You may be required to recruit your own subjects from the university population or from outside the university community. Random selection procedures also applies here. If you do not select your subjects randomly, your sample will be biased, resulting in less generalizability. If you use a convenience sample, your results might be generalizable beyond these subjects. There are some common pitfalls to avoid when select subjects. It is always best to use people you do not know because the behavior of your friends in your experiment will differ from more typical subjects. Once you have chosen population (perhaps other university students), you cannot recruit only those people who smile and seem friendly and approachable. Control for this possible threat by constructing a procedure randomly selecting the people you approach ahead of time, and stick to it. For instance, if you decide to ask every fifth person who walks by, you must adhere to this procedure