Experiments 2 - S. PDF

1 Basic Experiments Today’s Plan 2 Controlling for Extraneous Variables 1 Basic Experiments Internal Validity Internal Validity Our confidence in CAUSE & EFFECT Is it that the IV causes changes in the DV? Internal Validity High Internal...

1 Basic Experiments Today’s Plan 2 Controlling for Extraneous Variables 1 Basic Experiments Internal Validity Internal Validity Our confidence in CAUSE & EFFECT Is it that the IV causes changes in the DV? Internal Validity High Internal Validity? Well-designed and conducted experiment = confident it’s ONLY the IV causing the DV Low Internal Validity? Confound is present = NOT confident it’s ONLY the IV causing the DV Internal vs External Validity External validity Deals with generalizability → extent to which we can be sure we can generalize our results to different populations Ways to Establish Internal Validity Choose a good research design Control for extraneous variables Basic Experiments IV has 2 levels → only 2 possible groups or conditions Basic steps: 1) Select participants Probability or Non-probability sampling? 2) Assign participants to levels of the IV Random? Eliminate selection differences? (Equivalent groups?) Basic steps: 3) Introduce the IV Operational definition? 4) Measure the DV Operational definition? Basic Experiments 3 basic designs Independent groups Repeated measures Matched pairs Assigning Participants Independent Groups Different participants are assigned to each level of the independent variable Random assignment (ideal) Aka between-subjects design Independent Groups Posttest-only design Only 1 measurement that takes place after the manipulation Independent Groups Pretest-posttest design When a pretest (or SCREEN) is given to participants prior to the IV to ensures participant equivalence 3 reasons to add a pretest: Small sample; need specific participants; high attrition/mortality Pretest-posttest design Disadvantages: Time-consuming + awkward to administer May reduce external validity Sensitize participant (demand characteristics)… Participant figures out what the study is about and acts differently Pretest-posttest design Solutions to sensitization: Use deception Embed the pretest with other irrelevant measures Solomon four-group design Concerned that Pretest may have an impact on your results? Then consider the… Solomon four-group design: Assess impact of the pretest by treating the pretest as a second independent variable Half of participants (in both groups) receive pre-test Solomon four-group design Repeated Measures Design Repeated Measures Design Same participants experience all levels of the independent variable Aka within-subjects design Repeated Measures Design Advantages: Need fewer participants Sensitive to statistical differences Less (noise/error) variance as participants serve as their own control group Repeated Measures vs Independent Groups Is recall memory affected by listening to story or reading story silently? Independent groups Repeated measures design design Listen Read silently Listen Read silently Difference 54 58 P1 54 58 +4 68 71 P2 68 71 +3 75 82 P3 75 82 +7 Mean = 65.8 Mean = 70.3 Repeated Measures Design Disadvantages: Order of treatment presentation may affect the DV (order effect) Practice effects → repeated practice makes them better Fatigue effects → become tired, bored, or distracted Contrast effects → the 2 conditions are contrasted Solutions to order effects: Complete or partial counterbalancing Spacing time intervals Rest periods may counteract fatigue effects and contrast effects But… more time between trials may result in greater attrition because study takes longer Matched Pairs Design Match participants on a characteristic Usually the DV (or related to it) Ensures participants are equivalent on the matching variable prior to intro of IV Matched Pairs Design Matched Pairs Design Advantages Disadvantages Minimizes noise Time consuming and (differences between difficult to match participants) Need twice the amount of No order effects participants Developmental Research Study the way people change over time as they age Has its own terminology for the basic designs Design Cross-sectional: All Same Time Different groups participants get same questions at same point in time Group 1 Group 2 AKA independent groups design Group 3 Design Longitudinal: Tracks Same Group Different times changes over time (days, months, years) AKA Repeated Group 1 Group 1 Group 1 Measures design Time 1 Time 2 Time 3 Choosing a Design One is not superior to the other Things to consider: Reverse-ability of effects? Generalizability of results?  Controlling 2 Extraneous Variables Controlling Extraneous Variables 1. Randomization 2. Elimination 3. Constancy 4. Balancing 5. Counterbalancing 1. Randomization Random selection/assignment Stratified random sampling Only certain participants With few participants not as effective 2. Elimination Completely remove problem Might not be possible 3. Constancy All participants experience the same ‘level’ of the variable of concern E.g., Same researcher(s) tests all participants E.g., Using placebos creates constancy in expectations 4. Balancing Ensuring groups are ‘balanced’ on the extraneous variable E.g., hungry/not hungry tested in morning AND in afternoon Do results differ? If not, you can attribute this to IV, not external variable(s) If yes, then EV at work… 5. Counterbalancing Control for order effects and carryover effects CHILD 1 CHILD 2 5. Counterbalancing Types of Counterbalancing: Within-subject – each participant receives all ‘orders’ Participant experiences conditions more than once Not great 5. Counterbalancing 5. Counterbalancing Types of counterbalancing: Within-group – each participant gets a different order Complete – ensure equal number of participant with each possible order Depends on number of items 5. Counterbalancing Child 1 Child 2 Child 3 Child 4 Child 5 Child 6 5. Counterbalancing Complete counterbalancing 2 conditions = 2 x 1 = 2 orders 3 conditions = 3 x 2 x 1 = 6 orders 4 conditions = 4 x 3 x 2 x 1 = 24 orders 5 conditions = 5 x 4 x 3 x 2 x 1 = 120 orders 5. Counterbalancing 5. Counterbalancing Types of counterbalancing: Within-group – each participant gets a different order Incomplete - ensure equal number of participant with only some of the possible orders Systematic rotation (computer can do this) 5. Counterbalancing Problem with counterbalancing: differential carryover Effect of carryover depends on previous treatment/condition E.g., Having COKE after PEPSI has a larger impact than having PEPSI after COKE Must be aware of this potential Ways to Establish Internal Validity Choose a good research design Control for extraneous variables Additional Considerations Controlling for participant expectations Demand characteristics: cues embedded in a study that inform the participant how they are expected to behave Use deception Placebo effects (not limited to drugs) Include a placebo group Additional Considerations Controlling for experimenter expectations Experimenter bias: any intentional or unintentional influence that the experimenter exerts on participants to confirm the hypothesis Aka expectancy effects Use a double-blind procedure, training, or computers Potential Exam Questions List and explain the major steps toward planning a basic experiment Describe the pretest-posttest design, including the advantages and disadvantages of using a pretest Describe the matched-pairs design and reasons to use this design Potential Exam Questions Contrast a between-subjects design with a within- subjects design, including advantages and disadvantages of each Describe how extraneous variables affect the internal validity of an experiment and how to control for them Describe ways to control participant expectations and experimenter expectations