Lecture 3 Repeated Measures ANOVA 2023 2024.pptx

1 REPEATE DMEASURE S ANOVA (Lecture 3) Week 2 Learning objectives 2  Describe repeated-measures ANOVA repeated measures ANOVA = within-subjects ANOVA  Rationale of repeated-measures ANOVA:     benefits Issues to be mindful of, assumptions … Conduct and interpret ANOVA Report ANOVA results. JAS...

1 REPEATE DMEASURE S ANOVA (Lecture 3) Week 2 Learning objectives 2  Describe repeated-measures ANOVA repeated measures ANOVA = within-subjects ANOVA  Rationale of repeated-measures ANOVA:     benefits Issues to be mindful of, assumptions … Conduct and interpret ANOVA Report ANOVA results. JASP calls it repeated measures ANOVA, but both terms are widely used so we should be familiar with them. Reminder slide: The two main experimental designs between-subjects design two or more independent groups of subjects are tested (i.e. males vs. females; depressed vs. controls; young vs. old). One group of muppets receive treatment 1 Different group of muppets receive treatment 2 Repeated-measures design (aka within-subjects design), same subjects are tested under all the experimental conditions. Same group of muppets tested once… (treatment 1) … and then again (treatment 2) … and then again (treatment 3) … 3 Example of between-subjects experiment 4   Let’s test if weather conditions affect our maths problem solving… 27 undergrads assigned to 1 of 3 conditions (groups).       Raining outside Snowing outside Sunny outside Subjects took 30 minutes to solve problems H0: no significant difference in number of problems solved between the 3 conditions. H1: there will be at least two means that differ significantly. Variability between subjects… 5 Group 1: raining  Group 2: snow Group 3: sunny If you randomly sample these three groups, hopefully many different individual characteristics will be cancelled out. Repeated-measures ANOVA: rationale 6  If we use a repeated measure design, individual differences should matter less: same people tested in all conditions.  Testing same group under 3 conditions:  Rain, Snow, Sun Repeated ANOVA: sources of variance 7 1. Variance created by our manipulation 2. Variance created by individual differences is ‘cancelled out’ by repeatedly testing the same participants. Benefit of repeated measures designs 8  Increased sensitivity   Error variance is reduced (by ‘cancelling out’ individual differences). Economy    Fewer participants are needed. What about fatigue or learning? If you use the same people in many different experimental conditions, they tend to be tired, or become expert at your task. We try to counter this by using ‘randomization’ and ‘counterbalancing’ (discussed in a later lecture on Experimental Design). Repeated measure ANOVA 9        Research question: effect of weather on problem solving IV – weather conditions DV – number of problems solved Same participants go through all three conditions Design: one-way repeated measures ANOVA H0: no significant difference in number of problems solved in 3 conditions. H1: at least one pair of conditions will differ. Partitioning the variance It’s all about the cake! 10 Reminder slide: partitioning variability for between-subjects ANOVA. Dependent variable 11 3. Within-group 1. Total variability 2. Between-group variability (error) variability Group1 Group2 Group3 Group1 Group2 Group3 Group1 Group2 Group3 Partitioning variability for repeatedmeasures ANOVA – Example 12 Subje ct Numb er: 1 2 3 4 5 6 7 8 9 Raining Snowing Sunny 12 17 14 13 15 17 15 13 18 15 12 13 14 17 16 13 18 15 20 20 17 16 18 20 18 16 21 Partitioning variability for repeated-measures ANOVA 13  We want to know if the means for the three conditions differ. Subjec Raining Snowin Sunn g y t Numb er 1 12 15 20 2 17 12 20 3 14 13 17 4 13 14 16 5 15 17 18 6 17 16 20 7 15 13 18 8 13 18 16 9 18 15 21 Mean 14.89 14.78 18.4 Partitioning variability for repeatedmeasures ANOVA 14  As in betweensubjects ANOVA, mean difference reflect    Experimental manipulation (weather) Error However, the error here is variability within each person, not between people. Subjec Raining Snowin Sunn g y t Numb er 1 12 15 20 2 17 12 20 3 14 13 17 4 13 14 16 5 15 17 18 6 17 16 20 7 15 13 18 8 13 18 16 9 18 15 21 Mean 14.89 14.78 18.4 Partitioning variability for repeatedmeasures ANOVA 15    For example, subject 1’s performance change between conditions is partly due to our treatment and partly due to ‘natural’ variability in their performance. But, this ‘natural’ variability will not be the same across people. If only we could think of a magic formula to cancel out this ‘natural’ variability! Subjec Raining Snowin Sunn g y t Numb er 1 12 15 20 2 17 12 20 3 14 13 17 4 13 14 16 5 15 17 18 6 17 16 20 7 15 13 18 8 13 18 16 9 18 15 21 Mean 14.89 14.78 18.4 Within-subjects variability (SSwithin-subjects) 16 calculate variability of each subject around their own mean and sum them all up… Subje Raining Snowi Sunn Mean ng y ct Numb er SSwithin-subjects 15.7 1 12 15 20 16.3 2 17 12 20 SStreatment SSresidual (error) 14.6 3 14 13 17 14.3 4 13 14 16 Is there more treatment 16.6 15 17 18 variability or ‘error’ variability in 5 SSwithin-subjects? 17.6 6 17 16 20 15.3 7 15 13 18 15.6 8 13 18 16 18.0 9 18 15 21 Mean 14.89 14.78 18.4 16.04  Treatment (model) variability (SStreatment) 17  Variability of treatment means from the grand mean  SSmodel Subje Raining Snowi Sunn Mean ng y ct Numb er 15.7 1 12 15 20 16.3 2 17 12 20 14.6 3 14 13 17 14.3 4 13 14 16 16.6 5 15 17 18 17.6 6 17 16 20 15.3 7 15 13 18 15.6 8 13 18 16 18.0 9 18 15 21 grand mean Mean 14.89 14.78 18.4 16.04 The repeated-measures ANOVA ratio 18   SSmodel and SSresidual are adjusted by df to produce Mean Squaremodel and Msresidual The df error for repeated measures is (nr. treatments – 1)*(nr. participants – 1) = 16  Then we proceed as usual to calculate the F-ratio Are we doing better than betweensubjects ANOVA? 19     In general, unsystematic (residual or error) variability = individual differences + differences in how someone behaves at different times Individual differences are a factor in betweensubjects ANOVA but not in repeated-measures ANOVA. So residual variability ‘should’ be lower in repeated-measures ANOVA… Let’s check by analysing the same data twice Once as repeated-measures ANOVA  Once as between-subjects ANOVA… (this would be the wrong approach here)  Are we doing better than betweensubjects ANOVA? 20   SSmodel is the same (variation of the three treatment means around grand mean). But, the error (or residual) variance is different  Smaller for the repeated-measures  We managed to reduce the error variance!  This is great news, because reducing the denominator of the F-ratio increases the F-ratio. (the between analysis - wrong) (the repeated-measures analysis (Small print: in some ‘exotic’ scenarios, which we won’t discuss the between analysis may produce a larger ratio.) Assumptions in repeated-measures ANOVA 21 The sphericity assumption 22  “the variance of the difference across pairs of conditions (treatments) should be almost the same”  Assessed using Mauchly’s test  p <.05, sphericity is violated  p >.05, sphericity is not violated. What is sphericity? 23 Raining - Raining - Snowing Raining Snowing- Raining Sunny Snowing - Sunny Snowing - Sunny - Sunny -3 -3 5 5 1 1 -1 -1 -2 -2 1 1 2 2 -5 -5 3 3 9.86 -8 -8 -3 -3 -3 -3 -3 -3 -3 -3 -3 -3 -3 -3 -3 -3 -3 -3 2.78 -5 -5 -8 -8 -4 -4 -2 -2 -1 -1 -4 -4 -5 -5 2 2 -6 -6 8.75 What is sphericity? 24    Is the variance in the middle difference column too different? This might indicate a violation of the sphericity assumption. But how different would be too different?  Mauchly’s test  Footnote: Notice we need at least 3 columns (i.e. conditions) to compare the variance of these differences. Two columns would only produce a single column of differences with a single variance – nothing to compare to! Mauchly’s test 25 Mauchly’s test has a χ2 distribution. “Mauchly's Test of Sphericity indicated that the assumption of sphericity has not been violated, χ2(2) = 3.35, p =.19.” We can tweak the numbers to increase the differences between the variances of differences columns… 26 Rainin Snowin Sunny g g 12 17 14 13 35 35 35 13 18 15 12 13 14 5 5 5 18 15 20 20 17 16 10 10 10 16 21 Raining Snowing -3 5 1 -1 30 30 30 -5 3 233.75 Raining Sunny -8 -3 -3 -3 25 25 25 -3 -3 210.44 Snowing Sunny -5 -8 -4 -2 -5 -5 -5 2 -6 7.94 Corrections for violation of sphericity assumption 27  Methods to correct for this:    Greenhouse-Geisser Huynh-Fieldt Estimate These correction rely on adjusting the degrees of freedom. Repeated measures ANOVA: Main Output table 28 Our original example. Repeated measures ANOVA: Main Output table 29 Our altered example where sphericity was violated. Greenhouse-Geisser is a correction used when assumption is not met, i.e., Mauchley’s test p <.05. How does this correction work? Magic formulas for ‘correcting’ the degrees of freedom (see df column above). Repeated measures ANOVA 30   F-value is significant, great! At least one pair of means should be different, but which pair? (There are some exotic exceptions to this, see e.g., link in notes) Mean plot Individual datapoints Posthocs for repeated-measures ANOVA 31 Some of the familiar post-hoc tests are not valid with repeated-measures ANOVA.  E.g., let’s try a Tukey posthoc in JASP… Posthoc tests in repeated measures ANOVA 32 Compare all pairs of means Some more conservative than others. In other words they may be: Less ‘powerful’ Less ‘sensitive’. More likely to result in a ‘false-negative’ error. The Bonferroni test is more conservative than the Holm test. Posthoc tests in repeated measures ANOVA 33  Many posthocs are based on a t-test but with the pvalue adjusted by some formula to penalize for number of tests carried out. Adjusted p-values vs. raw p-values for different posthocs (based on example dataset with 25 34 Repeated measures design ANOVA, Example with two IVs   Same example as before (three weather conditions) but now we test participants on both easy and difficult problems. We now have two repeated measures independent variables.  IV1: weather  IV2: problem difficulty  DV: number of problems solved. (Remember, participants contribute Repeated measures ANOVA: another example … 35 Sunny day Subject Easy Hard 1 21 17 2 23 15 3 18 18 4 20 13 5 22 16 6 20 17 7 18 14 8 19 13 9 22 12 10 17 15 Rainy day Easy Hard 15 12 12 17 13 14 14 13 17 15 16 17 13 15 18 13 15 18 17 16 Each column represents a combination of weather condition and task’s difficulty. Table: Tests of repeated-measures Effects 36 What is this “Weather * Problem Difficulty ” term and why is it significant? 37 An interaction term…  The effect of weather significantly depends on how difficult the problems were.  We will talk more about interaction terms when discussing factorial ANOVA.  JASP plot: Can you take a guess at what the interaction term means here? 38 Readings 39