PSYCH 306 Research Methods in Psychology Chapter 11: Factorial Designs PDF

PSYCH 306 Research Methods in Psychology Chapter 11: Factorial Designs Page 1  Plans for Remaining Weeks of Psych 306 1. Final Exam will be on December 9, 6:30pm-9:30pm: will be multiple choice questions (about 60) aimed to last 2 hours. Covers materi...

PSYCH 306 Research Methods in Psychology Chapter 11: Factorial Designs Page 1  Plans for Remaining Weeks of Psych 306 1. Final Exam will be on December 9, 6:30pm-9:30pm: will be multiple choice questions (about 60) aimed to last 2 hours. Covers material only since the last exam (lectures and readings). MC questions are similar in format to exam 1 questions. 2. HW3: will be graded over next 2 weeks, due to need to focus on final exam. 3. Reminder about SONA: 2% extra credit toward final grade for SONA participation. Total grades = 37% (exam 1) + 37% (exam 2) + 26% (HW1, 3, 4 = 7 points each; HW2 = 5 points). Page 2 Outline Factorial designs: the basics Main effects and interactions – Interpreting results Types of factorial designs: – Pure (between-groups) factorial, within-subjects factorial, mixed-factorial, combined factorial Higher order designs Advantages and disadvantages Page 3  Factorial Designs So far, we have considered the relationship between one IV and one DV Human behaviour or experience is determined by multiple factors (not just one IV) Happiness is a good example because it is determined by many factors Genetics Environment Internal state of mind Lyubomirsky, Sheldon & and Schkade (2005) Page 4  Factorial Designs What is a Factor? An independent variable (IV) in an experiment Factorial design = research design that includes two or more factors - a two-factor design has two IVs - a three-factor design has three IVs Example: a 2 X 3 X 2 design is a three-factor design with a total of 12 conditions formed from the first IV (2 conditions), second IV (3 conditions), and third IV (2 conditions) = 12 conditions total because all factors are CROSSED (all possible combinations of the 3 factors are included) Page 5 Factorial Designs Factorial designs allow us to examine complex relationships among variables in a single study A factorial design examines: the effects of more than one IV on a DV the interactions between the IVs - combined effects When there is more than one IV in a study, we refer to the IVs as factors When all of the factors are manipulated: experimental factorial design When one factor is not manipulated: quasi-experimental factorial design Page 6  Factorial Designs and Interactions Consider simplest factorial design with 2 factors (IVs) An Interaction among 2 Factors = the effect of one IV at each level of another IV Rate your sandwich preferences: 7 Mustard Preference - With/without Avocado 6 No Mustard - With/without Mustard Mustard 5 A Rated 4 Deliciousness of Sandwich 3 B 2 1 No Avocado Avocado Interaction Avocado Preference = Difference (A) - Difference (B) = Difference in Mustard-related preferences (without avocado) Page 7 minus difference in Mustard-related preferences (with avocado) Structure of a Two-Factor Experiment Presentation Type On Paper On Screen Fixed Time Exam scores for a Exam scores for participant group a participant A 2-Factor Experiment who studied text group who with Presentation Type presented on paper studied text Study Time (Factor A) and Study for a fixed time. presented on Time (Factor B) crossed screen for a fixed time. Self- Exam scores for a Exam scores for Regulated participant group a participant Time who studied text group who presented on paper studied text for a self-regulated presented on time. screen for a self- regulated time. The levels of one factor determine the columns; the levels of the second factor determine the rows. Page 8 Two-Factor Design A two-factor design can be represented by a matrix  Each cell corresponds to a separate treatment condition (a specific combination of the factors). – Data provide three separate and distinct sets of information.  Describes how the two factors independently and jointly affect behavior Page 9  Main Effects A two-factor study with two main effects: Present lists of objects, amount of viewing time (fixed or self-regulated); ask for list from memory later Study time (fixed, self-regulated) and Presentation type (printed pictures, on computer screen). Dependent variable = Score on memory test for material presented A main effect = difference in mean values of the levels of one factor The patterns of data show main effects for both factors with no interaction. On Paper On Screen Fixed time M = 22 M = 18 Overall M = 20 Self-regulated M = 18 M = 14 Overall M time = 16 Overall M = Overall M = 20 16 Page 10  Main Effects A two-factor study with two main effects: Present lists of objects, amount of viewing time (fixed or self-regulated); ask for list from memory later Study time (fixed, self-regulated) and Presentation type (printed pictures, on computer screen). Dependent variable = Score on memory test for material presented A main effect = difference in mean values of the levels of one factor The patterns of data show main effects for both factors with no interaction. 25 Memory Accuracy 20 (out of 30) 15 10 Study Time Fixed 5 Self-regulated 0 On Paper On Screen Presentation Mode Page 11 Interaction Between Factors Same Example with different data: The same main effects, but also an interaction between the 2 variables. On Paper On Screen Fixed time M = 20 M = 20 Overall M = 20 Self-regulated M = 20 M = 12 Overall M = time 16 Overall M = Overall M = 20 16 Page 12 Interaction Between Factors Same Example with different data: the same main effects, but also an interaction between the 2 variables. 25 Memory Accuracy 20 (out of 30) 15 10 Study Time 5 Fixed Self-regulated 0 On Paper On Screen Presentation Mode Page 13  A Two-way Interaction An interaction between two factors: – An interaction exists when the effects of one factor depend on the level of a second factor. – Non-parallel lines between two factors in a 2x2 plot generally indicate an interaction. Study Time Study Time Two factors interact No interaction Presentation Type Presentation Type Page 14  A Two-way Interaction Interactions can take many different forms: Fixed Time Fixed Time 20 Mean Exam Score 20 Mean Exam Score 15 15 Self-regulated 10 Time 10 Self-regulated Time 5 5 Two factors interact Two factors interact 0 0 Paper Screen Paper Screen Mode of Presentation Mode of Presentation An interaction means that the effects of one factor depend on the levels of a second factor. Page 15 Between- and Within-subject Factors Factorial designs can have: 1) All between-subject factors 2) All within-subject factors 3) A mix of between-subject and within-subject factors: called “mixed” design Example of mixed design: Treatment (between-subjects) X Test time (Within-subject) Pretest Posttest Treatment Group Pretest scores for Posttest scores for participants who receive participants who receive the treatment the treatment Control Group Pretest scores for Posttest scores for participants who do not participants who do no receive the treatment receive the treatment Page 16 Between- and Within-subject Factors 2 x 2 Mixed Factorial Design: "Mixed" = some IVs are between-subject, others are within-subject Between-Subject: Driver Age (Younger / older) Within-Subject: Cell phone/ no cell (hands-free conversation) DV: Time to brake (car simulator) in response to obstacle Strayer & Drews (2004) The dashboard, steering wheel, gas and brake pedals taken from Ford Crown Victoria sedan with an automatic transmission. Page 17  Between- and Within-subject Factors 2 main effects: Age + Cell phones both increase braking times No interaction: Cell phones do not create a larger difference for older drivers than for younger drivers Graphing Conventions: Use bar chart for categorical X variables (line graphs display the slopes) 1200 1200 Cell phone Mean Braking Time (ms) 1000 No phone 1000 800 800 600 Cell phone 600 400 No phone 400 200 200 Younger Older Younger Older Page 18 Strayer & Drews (2004) Participant Age Participant Age  Independence of Main Effects and Interactions A two-factor study allows researchers to evaluate three separate sets of mean differences: – The mean differences from the main effect of factor A – The mean differences from the main effect of factor B – The mean differences from the interaction of factors A and B Page 19  Computing a Two-way Interaction Compare the mean differences in rows with the mean differences in columns A) If the size and the direction of the row differences are the same as the corresponding column differences = no interaction. B) If the size and direction of the differences change from rows to columns = evidence of an interaction. Younger Older Means No phone 780 912 846 Cell phone 912 1086 999 Means 846 999 Factorial Designs Used often in psychological research because human behaviour is complex and is influenced by a variety of interacting factors Factorial designs tend to have higher ecological (real world) validity than one-IV designs Example: Factors contributing to a poor exam mark: Lack of sleep Lack of study time Page 21 Factorial Designs We could examine each factor in a separate experiment: 1. Effects of sleep time on exam performance. 2. Effects of study time on exam performance. Or we can consider both questions in one factorial study: 3. What are the combined effects of sleep time and study time on exam performance? Page 22 Factorial Designs In this example, there are 2 factors (sleep time, study time) and each factor has 2 levels (high versus low) – Can be represented by a matrix Each factor is denoted by a letter and each level of the IV is represented by a number – Factor A = rows; Factor B = columns Factor B Factor A sufficient study insufficient study sufficient sleep A1B1 A1B2 insufficient sleep A2B1 A2B2 Page 23 Factorial Designs 2 x 2 factorial design = 4 cells Factor B: Study Time Sleep High Sleep High Factor A: Study High Study Low Sleep Sleep Low Sleep Low Study High Study Low Page 24  Factorial Designs The number of IVs and the number of levels for each IV indicates the total # conditions Example: 2 X 3 design showing only main effects Two IV; first = 2 levels, second=3 levels Factor A Factor B Factor A Factor B Factor C 2 levels 3 levels 2 levels 2 levels 3 levels Example: 2 X 2 X 3 design Three IV; first = 2 levels, second=2 levels, third = 3 levels Page 25 Factorial Designs What is the effect of mood on memory? – Does your mood affect memory more while learning new information or while recalling that information? 2 X 2 Factorial design = 4 cells Factor A Learn learn happy learn sad Factor B recall happy recall happy Recall learn happy learn sad recall sad recall sad Page 26 Factorial Designs What are the effects of anxiety on test performance? – Single studies can yield confusing results when large individual differences – Consider each participant’s original level of anxiety Factor A Factor B test anxiety no test anxiety High trait anxiety A1B1 A2B1 Med. Trait anxiety A1B2 A2B2 Low trait anxiety A1B3 A2B3 Page 27 Factorial Designs 2 x 3 factorial design = 6 cells Factor A: Test anxiety Yes No Factor B High / yes High / no Trait anxiety Med / yes Med / no Low / yes Low / no Page 28 Independence of Main Effects and Interactions The ability to mix factors within a single study allows researchers to blend several different research strategies within one study. Researchers can develop studies that address scientific questions that could not be answered by a single strategy. Example: Independent factors of Test Anxiety (2 levels) and Trait Anxiety (3 levels) = 2 X 3 design 3 tests: Effects of Test Anxiety Effects of Trait Anxiety Interaction of Test Anxiety and Trait Anxiety Page 29 Factorial Designs Now let’s say you wanted to add a third IV, such as gender You would end up with a 2 x 3 x 2 factorial design 2 levels for text anxiety 3 levels for trait anxiety 2 levels for gender Page 30 Factorial Designs Each factor: test anxiety, trait anxiety, and gender is a between-subjects factor Cannot be manipulated within subject This yields a 2 x 3 x 2 factorial design with a need for many subjects 2 levels for text anxiety 3 levels for trait anxiety 2 levels for gender Page 31  Factorial Designs 2 x 3 x 2 factorial design = 12 cells Factor C C1 C2 Factor A Factor A A1 A2 A1 A2 B1 A1B1C1 A2B1C1 B1 A1B1C2 A2B1C2 Factor B B2 A1B2C1 A2B2C1 B2 A1B2C2 A2B2C2 B3 A1B3C1 A2B3C1 B3 A1B3C2 A2B3C2 Page 32  Factorial Designs 2 x 3 x 2 factorial design = 12 cells Gender Male Female Test anxiety Test anxiety Yes No Yes No Trait anxiety high M /Y /Hi M /N / Hi high F /Y /Hi F /N /Hi med M/Y/Med M /N /Med med F /Y /Med F /N /Med low M /Y /Lo M /N /Lo low F /Y /Lo F /N /Lo Page 33 Factorial Designs Main effects: – Treatment differences between levels of a given factor main effect for Factor A main effect for Factor B Main effect for Factor C And so on Page 34  Factorial Designs Interaction effects: – Combined effect of factors – An interaction occurs when the effect of one factor depends on the level of another factor Trait Anxiety A x B interaction effect: Test Anxiety X Trait Anxiety A x B x C interaction effect: Test Anxiety X Trait Anxiety X Gender Page 35  Factorial Designs A factorial design tests at least 3 hypotheses For a 2x2 factorial design, there are 3 null hypotheses: There is no significant difference between the levels of Factor A There is no significant difference between the levels of Factor B There is no significant interaction of Factors A & B The null hypotheses correspond to the statistical tests of: Main effects of Factor A Main effects of Factor B Interaction of Factors A & B Page 36  Factorial Designs A factorial design tests alternative hypotheses as well: The alternative hypotheses for a 2 x 2 factorial design can take the form of (these are only examples): Factor A: Level 1 > Level 2 Factor B: Level 1 < Level 2 Interaction: Level 1 of Factor A > Level 2 of Factor A when Factor B = Level 1 than when Factor B = Level 2 and so on Savvy researchers have in mind some alternative hypotheses, based on previous experiments, when they design factorial experiments Page 37 Factorial Designs Consider 2 x 2 x 3 factorial design = 12 cells Main effects: Does Factor A differ across levels? Does Factor B differ across levels? Does Factor C differ across levels? Possible Interactions in 3-factor design: A xB AxC BxC 3 two-way interactions between all pairs of 3 factors 1 three-way interaction A xBx C between all factors Page 38 Factorial Designs Original 2 x 2 example: What is the effect of mood on memory? – Does your mood matter more during learning new information or at recall of that information? 2 X 2 Factorial design = 4 cells Factor A Learn Happy Learn Sad Recall learn happy learn sad Happy recall happy recall happy Factor B learn happy learn sad Recall Sad recall sad recall sad Page 39  Warning about Means–based Examples: The Variance Matters, Too All examples in this lecture Means X, Y of 2 Factors assume small X Y differ significantly overlap of variances - small amount of overlap in variances Means X, Y of 2 Factors X Do Not differ significantly YY -large amount of overlap in variances Warning: Statistical tests (ANOVA) are needed to determine whether the difference in mean values exceeds the variance in the factors sufficiently to be statistically significant Page 40  Warning about Means–based Examples: The Variance Matters Too 25 All examples in 20 Means X, Y of 2 Factors this lecture 15 differ significantly assume small - small amount of 10 overlap 5 overlap in variances 0 X Y 25 Means X, Y of 2 Factors 20 X Do Not differ significantly 15 YY -large amount of 10 5 overlap in variances 0 X Y Warning: Statistical tests (ANOVA) are needed! Page 41  Interpreting Results Look for main effects first. Compare the column means for Factor A Means are identical; no main effect for Factor A Page 42 Interpreting Results Compare the row means to determine main effect for Factor B Means are not identical; main effect for Factor B Page 43 Interpreting Results To look for interactions, check if difference in the row means (e.g., d =10) is consistent across the columns If differences are identical, there is no interaction Page 44  Interpreting Results To look for main effects, compare the column means for Factor A, and the row means for Factor B Factor A Factor B learn happy learn sad recall happy M = 20 M = 20 M = 20 Compare recall sad M = 10 M = 10 M = 10 Factor B M = 15 M = 15 Compare Factor A To look for interactions, check if difference in row means (e.g., 10) is consistent across columns. If so, parallel lines in graph. If not, = Page 45 interaction Interpreting Results Main effect of A: No Main effect of B: Yes Interaction A x B: No Page 46 Interpreting Results Factor A Factor B learn happy learn sad recall happy M = 20 M = 20 M = 20 recall sad M = 20 M = 20 M = 20 M = 20 M = 20 30 Score Recall (B) 20 Happy Sad No main Effect - A 10 No main effect - B No A x B interaction Happy Sad Learn (Factor A) Page 47 Interpreting Results Factor A Factor B learn happy learn sad recall happy M = 20 M = 20 M = 20 ( d = 0) ( d =0) recall sad M = 20 M = 20 M = 20 M = 20 M = 20 30 Score Recall (B) 20 Happy No main effect - A Sad No main effect - B 10 No A x B interaction Happy Sad Learn (A) Page 48 Interpreting Results Factor A Factor B learn happy learn sad recall happy M = 20 M = 20 M = 20 (d=10) ( d=10) recall sad M = 10 M = 10 M = 10 M = 15 M = 15 30 Score Recall (B) 20 Happy No Main effect -A 10 Main Effect - B Sad No A x B Happy Sad Learn (A) Page 49 Interpreting Results Factor A Factor B learn happy learn sad recall happy M = 30 M = 40 M = 35 ( d = 20) ( d = 20) recall sad M = 10 M = 20 M = 15 M = 20 M = 30 40 Recall (B) Happy 30 Main effect - A Score Main effect - B 20 Sad No A x B 10 Happy Sad Learn (A) Page 50 Interpreting Results Factor A Factor B learn happy learn sad recall happy M = 10 M = 20 M = 15 ( d = 0) ( d = 0) recall sad M = 10 M = 20 M = 15 M = 10 M = 20 40 30 Score Recall (B) 20 Happy Main effect - A Sad No main effect - B 10 No A x B Happy Sad Learn(A) Page 51 Interpreting Results Factor A Factor B learn happy learn sad recall happy M = 10 M = 40 M = 35 ( d = 0) ( d = 20) recall sad M = 10 M = 20 M = 15 M = 10 M = 30 40 Recall Happy Main effect - A 30 Score Main effect - B 20 Sad Interaction A x B 10 Happy Sad Learn(A) Page 52 Interpreting Results Factor A Factor B learn happy learn sad recall happy M = 20 M = 40 M = 30 ( d = 10) ( d = 30) recall sad M = 10 M = 10 M = 10 M = 15 M = 25 40 Recall Happy 30 Main effect - A Score Main effect - B 20 Interaction A x B 10 Sad L/H L/S Factor A Page 53 Interpreting Results Factor A Factor B learn happy learn sad recall happy M = 20 M = 10 M = 15 ( d = 10) ( d = -10) recall sad M = 10 M = 20 M = 15 M = 15 M = 15 40 30 Score Recall 20 Sad No main effect - A No main effect - B 10 Happy Interaction A x B Happy Sad Learn ( A) Page 54 Interpreting Results When discussing results, consider the interaction before or at same time as considering main effects – The presence of an interaction can obscure or distort the main effects of each factor – When reporting an interaction, your statement should include the phrase “depends on” If there is no interaction, discuss the main effects by themselves Page 55  Before Next Lecture: Reserve the final exam date of December 9, 6:30pm-9:30pm Consider SONA participation (ends in November) Page 56 Interpreting Results Example: The effects of TV viewing duration (Factor A) on GPA (DV) depends on the content of the television program (Factor B). 100 Educational 70 GPA 40 10 Noneducational Short Long When the program is educational, GPA increases with longer viewing time. When the program is non-educational, GPA decreases with longer viewing time. Page 57 Interpreting Results Example: The effects of TV viewing duration (Factor A) on GPA (DV) does not depend on the content of the television program (Factor B). 100 Educational 70 Noneducational GPA 40 10 Short Long Longer viewing time results in an increase in GPA regardless of whether the program is educational or not. Page 58 Types There are several types of factorial designs: 1) Pure (between-subjects) factors 2) Within-subjects factors 3) Mixed design (between- + within-subjects factors) Higher order factorial designs = factorial designs with three or more factors Page 59 1. Pure Factorial Designs Design in which all factors are being manipulated Between-groups design: – Different groups of participants are randomly assigned to each cell of the design Factor A Factor B Level 1 Level 2 Level 1 Group 1 Group 3 Level 2 Group 2 Group 4 Page 60 1. Pure Factorial Designs Consider children’s fear when going to bed – You learn that darkness is not the only cause – Children also report having fearful images in their heads when they go to bed Factor A Factor B Light on Light off Neutral A1B1 A2B1 images Scary images A1B2 A2B2 Page 61 1. Pure Factorial Designs 80 children are assigned to one of four possible conditions in this 2 x 2 pure factorial design Factor A Illumination On / neutral Off / neutral Factor B N = 20 N = 20 Images On / scary Off / scary N = 20 N = 20 Page 62 1. Pure Factorial Designs Factor A Factor B Light on Light off Neutral images M = 98 M = 99 M = 98.5 Scary images M = 98 M = 114 M = 106.5 M = 98 M = 107 Is there an interaction? How do you know? Page 63 1. Pure Factorial Designs Factor A Factor B Light on Light off Neutral images M = 98 M = 99 M = 98.5 0 -15 -8 Scary images M = 98 M = 114 M = 106.5 M = 98 M = 107 A x B Interaction Page 64 1. Pure Factorial Designs The two variables interact. Darkness produces anxiety in combination with fearful images Main effect of A: Yes Main effect of B: Yes AxB interaction: Yes The effect of lighting on fear of sleeping alone depends on image content. Children are more fearful sleeping with the lights off, especially when having read stories with scary images. Page 65 1. Pure Factorial Designs Disadvantages: – Can require many participants because all factors are between-subjects – Individual differences can become confounding variables (as in single-factor between-subjects designs) But… Page 66 1. Pure Factorial Designs Advantages: – Avoids problems with order effects Different participants in each group, each person only experiences a condition once – Best when… Many participants available, individual differences are relatively small, and order effects might be a problem Page 67 2. Within-Subjects Factorial Designs Single group of participants in all separate conditions In a 2 x 2 factorial design, participants would experience all 4 conditions Factor A Illumination Factor B On / neutral Off / neutral N = 20 Same 20 Ps Images On / scary Off / scary Same 20 Ps Same 20 Ps In a 2 x 3 factorial design, participants would experience 6 different conditions (6 scores per participant) Page 68 2. Within-Subjects Factorial Designs Disadvantages: – Many factors means that participants experience many different conditions – Very time-consuming, and likelihood of attrition is higher Increases chances of testing effects (practice/fatigue) Makes it difficult to counterbalance orders to control order effects But… Page 69 2. Within-Subjects Factorial Designs Advantages: – Fewer participants needed – Reduces individual differences – Best when… Individual differences are large, and order effects will not be a problem Page 70 3. Mixed Factorial Designs Mixed designs: within- and between-subjects factors A factorial study that combines both within- and between- subject factors  Used when one factor is expected to threaten validity A common example of a mixed design is a factorial study with one between-subjects factor and one within- subjects factor. Page 71 3. Mixed Factorial Designs Both between-group and within-subject designs can be combined in a factorial design Used when experimenter wants the advantages of a between-subjects design for one factor, while within- subjects design is preferable for second factor Full design = 2 (between-group) X 3 (within-group) factors Page 72 3. Mixed Factorial Designs What if we wanted to compare children’s fear of the dark when a parent is present compared to when the child is alone? – This suggests 2 groups We might also want to look at how this interacts with different levels of illumination – Light, dim, dark The most logical way to do this would be repeated measures of our two groups across the three different lighting conditions Page 73 3. Mixed Factorial Designs Here we have 40 children separated into 2 conditions (with or without parents) in this 2 x 3 mixed factorial design Factor A – Between-Subjects W/Parent Wo/parent Factor B – N = 20 N = 20 Within-Subjects Light W / Light A / Light Dim W / Dim A / Dim Dark W / Dark A / Dark Page 74 3. Mixed Factorial Designs Factor A - Between Factor B - Within w/parent wo/parent Light M = 98 M = 115 M = 106.5 Dim M = 98 M = 115 M = 106.5 Dark M = 98 M = 115 M = 106.5 M = 98 M = 115 Page 75 3. Mixed Factorial Designs Children’s anxiety is higher only when they are alone, regardless of illumination Parent present (A) Main effect - A No main effect - B No A x B interaction Illumination (B) Page 76  Pretest–Posttest Control Group Designs Example of a two-factor mixed design. One factor is a between- subjects factor. Pretest–posttest is a within-subjects factor. Group (Between-S) Pretest Posttest Treatment Group Pretest scores for Posttest scores for participants who participants who receive the treatment receive the treatment Control Group Pretest scores for Posttest scores for participants who do participants who do not receive the no receive the treatment treatment Page 77 Higher-Order Factorial Designs More complex designs involving three or more factors In the three-factor design, the researcher evaluates main effects for each of the three factors A, B, C Plus the two-way interactions: A x B, A x C, B x C Plus the three-way interaction: A x B x C Page 78 Higher Order Designs Consider a four-way interaction in a memory experiment in which the experimenter examined the combined effects of: F1: gender: male / female F2: age older adult / middle adult / young adult F3: time of day morning/ afternoon / evening F4: types of words abstract / concrete The DV was the number of correctly recalled items Which are between- and within-group factors? Page 79  Higher Order Designs Consider a 4-factor crossed design (A x B x C x D) There are 4 main effects: A, B, C, D Memory test by: F1: gender (M/F) F2: age (20/40/60) And 6 two-way interactions: F3: time of day (am/pm) Ax B Ax C A xD B x C BxD CxD F4: words (noun/verb) And 4 three-way interactions: AxBxC AxBxD AxCxD BxCxD And 1 four-way interaction: Ax BxCxD Interpretation of higher-order interactions can be very complex Page 80 Higher Order Designs 3-factor design (x1, x2, x3) 4-factor design (x1, x2, x3, x4) x4 x3 x3 x2 x2 x1 x1 Page 81  Higher Order Designs Consider 2 (sex) x 3 (age) x 2 (word type) x 2 (morning/evening) design The significant findings were: – Females in the middle adult and young adult groups remembered abstract words better than concrete words in the evening – Males in the older and middle adult groups remembered both abstract & concrete words well in the morning How can these results be interpreted? How can these results be implemented in practice? Implications: Try to avoid more than 3 factors in factorial designs unless you have clear predictions for interactions Page 82 Advantages to Factorial Designs Highly efficient designs which allow studying: – Effect of many factors simultaneously – Interactions of factors – Can replicate and expand upon existing study all in one study Instead of reducing individual differences by holding constant (e.g., age), can include as another factor in the study Complex nature provides real advantages, but also some challenges (especially interpretation) High external validity Page 83 Disadvantages More chance of having confounds than single IV designs and same problems controlling for them Interpretations are no better than correlational studies if factors are not manipulated Too many factors make interpretation confusing May require a more stringent alpha level due to multiple statistical tests Page 84 Statistical Analysis of Factorial Designs Depends partly on whether the factors are: – Between-subjects – Within-subjects – Some mixture of between- and within-subjects The standard practice includes: – Computing the mean for each treatment condition (cell) and – Using ANOVA to evaluate the statistical significance of the mean differences Page 85 Factorial Designs: Other Uses 1) Replication: repeating a previous study and incorporating a new “replication” factor (first / second replication) Example: Effects of age, gender, concrete/abstract nouns on memory for words: 3 X 2 X 2 design Replication: add emotional salience (high/low) of words to same variables (age, gender, concrete/abstract nouns): 3 X 2 X 2 X 2 (emotional salience) design Advantages: test of replicability of first design extension of main findings to additional factor Page 86  Factorial Designs: Other Uses 2) Expanding the design: add a factor in the form of new participant characteristics to reduce variance Example: age, gender, concrete/abstract nouns: 3 X 2 X 2 Expansion: Add participant characteristic of languages spoken (monolingual / multilingual): 3X2X2X2 Purpose is to reduce the variance within groups by using the participant variable as another factor Advantages: Reduces individual differences within each group Does not sacrifice external validity Page 87  Factorial Designs: Other Uses 2) Adding participant characteristics to reduce the variance: Example from meta-analysis of Emergency Medicine Research identified 2476 publications (from 2006-2009) IVs (many): effects of crowding and time until emergency medical treatment on cardiovascular disease outcomes Participant variable of Gender: 11% of articles treated Gender as a control variable (balanced across groups but not analyzed) 2% included Gender in primary hypotheses (IV) 18% included Gender as additional IV (to reduce variance) Safdar et al (2011) Page 88 Factorial Designs: Other Uses For within-subjects designs: 3) Using the order of treatments as an additional factor Makes it possible to evaluate any order effects that exist in the data There are three possible outcomes that can occur: -No order effects -Symmetrical order effects (same order effects across other factors) -Asymmetrical order effects (order interacts with other factors) Page 89  Factorial Designs: Other Uses 3) Using the order of treatments as an additional factor to reduce variance Why? Because the randomization of participants to conditions does not always remove bias from unknown extraneous variables Because it is not possible to always randomize condition orders when there are many conditions 4-factor design (x1, x2, x3, x4) x4 X3 x3 Lopez et al (2023) Page 90 x2 x2  Factorial Designs: Analysis Uses Example: Taste ratings for 8 different types of chocolate IV = which chocolate (1, 2, 3, 4, 5, 6, 7, 8) where 1,3,5,7 = milk chocolate 2,4,6,8 = dark chocolate Perhaps the sweeter milk chocolates change the taste of the immediately following dark (less-sweet) chocolates Condition orders used (too many possible for 8 levels of IV): A) 1 3 2 4 5 7 6 8 B) 8 6 7 5 4 2 3 1 Page 91  Factorial Designs: Analysis Uses Condition orders (too many possible for 8 levels of IV): A) 1 3 2 4 5 7 6 8 B) 8 6 7 5 4 2 3 1 Solution: run the analysis as 8 (chocolate) X 2 (order) factorial design: IV of chocolate (8) is within-subjects IV of condition order (A or B) is between-subjects Reduces need for multiple orders; reduces variance among chocolate responses due to order Page 92  NEXT TIME: CHAPTER 10 Quasi- & Non-Experimental Designs Page 93