Experimental Research Lecture Notes PDF

10/7/24 2 Lecture Overview Ch. 5 – Experimental...

10/7/24 2 Lecture Overview Ch. 5 – Experimental u Experiment basics Research u Experimental design PSYC 2900U – RESEARCH METHODS u Experimentation and validity KARLA EMENO u Practical considerations 1 1 2 What is an experiment? 3 Remember: 4 Correlation is NOT causation. u Carefully controlled scientific procedure that manipulates variables to determine cause and effect Only an u Key features: experiment can determine cause u Independent variable (factor that is and effect. manipulated) vs. dependent variable (factor that is measured) u Experimental group (receives Random treatment) vs. control group (receives assignment is no treatment) VERY important. 3 4 Manipulation of the IV 5 Two Features of an Experiment 6 1. Manipulation of the Independent Variable u To manipulate an IV means to change its level systematically u Can “cause” this IV to occur so that different groups of participants are exposed to different levels of that variable. u This “cause” elicits the response (DV) Quasi Experiment- u (Random Sampling) Can typically u Ex: Whether the kids watched a violent or nonviolent TV manipulate the IV, show in Bobo doll experiment. but cannot 2. Control of Extraneous Variables randomly assign u Single Factor Two-Level Design – Experiments involving a participants to single IV with 2 conditions. u Can hold variable(s) constant conditions u Single Factor Multi-Level Design – Experiments involving a u Limit variable to a specific category single IV with more than 2 conditions. u (Random Assignment) 5 6 1  10/7/24 7 8 Treatment and Control Conditions Types of Control Conditions u A treatment is any intervention meant to change people’s u No-treatment control condition behaviour for the better. u Zero Treatment (they get nothing) u Ex: Psychotherapies, medical treatments, reduce u Problem: placebo effect prejudice, promote conservation u Placebo Control group u Treatment Condition – The group receiving the treatment. u Given Placebo u Control Condition – The group not receiving the treatment. u Checks for placebo effect u Randomized Clinical Trial – An experiment that researches u Wait-list control condition the effectiveness of psychotherapies and medical u Delayed treatment treatments. u Checks for “expectation of improving” 7 8 9 10 Types of Designs Types of Designs Within-Subjects Experiments: Between-Subjects Experiments: u Each participant tested in all conditions (Blocked) u Each participant tested only once u Order Effects u Relies on random assignment uCarryover Effects u Block randomization might also be used u Practice Effects u Matched group design – Participants are matched u Fatigue effect on the DV or extraneous variables prior to u Context effects manipulating the IV u Counterbalancing: Complete; random 9 10 Order Effects 11 Counterbalancing 12 u Carryover Effect – An effect of being tested in one condition on participants’ behavior in later conditions u Counterbalancing – Testing different participants in u Practice Effect – P articipants perform a task better in later different orders conditions because they have had a chance to u Complete Counterbalancing – An equal number of practice it participants complete each possible order of u Fatigue Effect – Participants perform a task worse in later conditions conditions because they become tired or bored u Random Counterbalancing – The order of the u Context Effect – Being tested in one condition changes conditions is randomly determined for each how participants perceive stimuli or interpret their task in participant later conditions 11 12 2  10/7/24 13 14 Which design is better? Internal Validity Between Subject: Within Subject: u An experiment is high in internal validity IF the u Conceptually simple u Controls for extraneous way it was conducted supports the conclusion u Less testing time for an participant variables that the IV caused any observed differences in individual (lowered noise) the DV. u Fewer subjects u Avoids carryover u Manipulation of the IV and controlling for effects and no need u Less testing time overall extraneous variables (such as using random for counterbalancing u *Best option if the participant assignment) provides strong support for causal u *Best when you only has time and don’t think get one shot carryover is an issue conclusions. 13 14 15 16 External Validity Construct Validity u An experiment is high in external validity IF the way it was conducted supports generalizing the results to u Construct Validity – The research question is people and situations beyond those actually studied. clearly operationalized by the study’s methods u Mundane Realism – When the participants and the situation studied are similar to those that the researchers want to generalize to and that participants encounter u Operationalization – The specification of exactly every day how the research question will be studied in the u Psychological Realism – Where the same mental process experiment design is used in both the laboratory and in the real world. 15 16 17 18 Statistical Validity & Prioritizing Validities LAST CLASS – Face Validity Statistical Validity: u How accurate a measure looks on the surface u Concerns the proper statistical treatment of data and the u It can be assessed quantitatively, but it is usually assessed soundness of the researchers’ statistical conclusions informally. u Face validity is weak evidence that a measure is measuring Prioritizing Validities: what it is supposed to because: u Often it is not possible to have high validity in all four areas – u People’s intuitions about human behaviour are often wrong internal, external, construct, and statistical u Many established scales in psychology work quite well u Many psychology studies have high internal and construct despite lacking face validity validity but sometimes sacrifice external validity 17 18 3  10/7/24 LAST CLASS – Criterion Validity 19 LAST CLASS – Content and Discriminant 20 Validity u The extent to which scores on a measure correlate with other Content Validity: variables (known as criteria) that one would expect them to be correlated with u The extent to which your test accurately/actually measures the u Ex: Test anxiety being negatively correlated with exam behavior you are trying to measure (conceptual definition) performance and course grades u Typically assessed informally by carefully comparing the u Concurrent validity – Criterion is measured at the same time as the measurement method to the conceptual definition construct Discriminant Validity: u Predictive validity – Criterion is measured at some point in the future u The extent to which scores on a measure DO NOT correlate with (after the construct has been measured) other UNrelated variables u Convergent validity – When new measures are correlated with u Low correlations suggest that the measure is reflecting a existing established measures of the same construct. conceptually distinct construct 19 20 21 22 Recruiting Participants Standardizing the Procedure u At the start of the project, the researcher must consider how u Experimenter Expectancy Effect – When the they will obtain participants. experimenter’s expectations about how participants u Subject Pool – An established group of people who have “should” behave in the experiment affect how the agreed to be contacted about participating in research participants behave studies u Double-Blind Study – When neither the participant nor u Need to remember that those who volunteer to participate the experimenter knows to which condition the in psychological research have been found to differ from participant is assigned those who don’t u Researchers should standardize the procedure as much u Ex: more educated, more interested in research topic, higher as possible so that it is carried out in the same way for all IQ, greater need for approval participants, regardless of the condition they are in 21 22 23 Assignment #2: Literature Review (20%) 24 Manipulation Checks & Pilot Testing u Due: Submit in Canvas by 11:59 pm on Wednesday, October 23rd u For this assignment, you will write a Literature Review that incorporates a Manipulation Check: minimum of 8 peer-reviewed psychological papers (ideally from u A separate measure of the construct the researcher is trying Assignment 1, but you can select new ones if you prefer). to manipulate to confirm that the IV was successfully u This literature review will summarize past research, make it clear how manipulated. your work contributes/adds/builds upon what has already been done, and mention why this is an important area of study. Pilot Test: u You will end Assignment 2 with a Current Study section that outlines your u A small-scale study conducted to make sure that a new research question and hypothesis (or hypotheses). Your variables and procedure works as planned the general study idea should be briefly introduced here as well. u Can also be used to help decide on study stimuli/materials u Instructions for Assignment 2 (including the grading rubric), as well as an example template, have been added to Canvas. 23 24 4  10/7/24 Upcoming Dates 25 Questions about Chapter 5? 26 u Tuesday, Oct. 15 – Fall Study Week – NO LECTURE OR th TUTORIAL u Tuesday, Oct. 22nd – Ch. 6 Non-Experimental Research u Wednesday, Oct. 23rd – Assignment 2 due in Canvas u Tuesday, Oct. 29th – Ch. 7 Survey Research u Tuesday, Nov. 5th – Ch. 8 Quasi-Experimental Designs u Tuesday, Nov. 12th – Ch. 9 Factorial Designs u Tuesday, Nov. 19th – Ch. 10 Single-Subject Research u Wednesday, Nov. 20th – Assignment 3 due in Canvas u Tuesday, Nov. 26th – Ch. 11 Presenting your Research 25 26 5