13_ANOVA.pdf

BEHL 2005 / BEHL 2019 (UO) Introductory Research Methods COMPARING MORE THAN TWO GROUPS: ANOVA Professor Hannah Keage What are we going to cover? One-way ANOVA Content from this lecture references: ANOVA • ANOVA = analysis of variance. • Investigates differences in means. • In this course, we...

BEHL 2005 / BEHL 2019 (UO) Introductory Research Methods COMPARING MORE THAN TWO GROUPS: ANOVA Professor Hannah Keage What are we going to cover? One-way ANOVA Content from this lecture references: ANOVA • ANOVA = analysis of variance. • Investigates differences in means. • In this course, we will look at oneway between group ANOVAs. • Each participant in a different group. • >2 groups (if 2 groups, use a t-test). • One-way refers to having one IV/factor. Each IV/factor will have >2 levels. ANOVA • The key ANOVA estimate is the F. • F = (model variance) / (error variance). • The model variance and error variance terms are calculated through “sums of squares”; we won’t go into these calculations, but don’t be surprised to see these terms in your jamovi output. • Remember, error variance often referred to as residuals. • The larger the F, the more variance you are explaining in your DV by your IV, as compared to error (although, F is not a direct measure of effect size). • The larger the value of F the more evidence we have against the null hypothesis. Effect sizes for ANOVA • F is not an effect size measure. • Partial eta2 or η2. • η2 = 0.01 indicates a small effect • η2 = 0.06 indicates a medium effect • η2 = 0.14 indicates a large effect My F is statistically significant! • This relates to a main effect of an IV/factor with >2 levels (as you progress in psychology, you’ll also assess interaction factors). • This only means that there is at least 1 statistically significant difference between your groups. • It does not tell you WHERE that significant difference(s) is(are). Post-hoc tests • Tell you where the significant group difference(s) is(are). • You can only run post-doc tests if you F is statically significant (p<.05 if you have a standard alpha of 0.05). 6 comparisons Group 1 v Group 2 Group 1 v Group 3 Group 1 v Group 4 Group 2 v Group 3 Group 2 v Group 4 Group 3 v Group 4 Why don’t I just run lots of t-tests? • Essentially you could achieve a similar outcome by skipping over the ANOVA and running six t-tests. But if you did this, you’d run into issues around multiple comparisons related to Type I error rate. • Remember, if alpha = 0.05, then the Type I error rate = 5%. • Remember, Type I error rate is the probability of rejecting the null hypothesis given its true; false positive. • Central principle behind null hypothesis testing is that we control our Type I error rate. • When we consider families of tests, e.g. the six tests required to test differences between our four groups (e.g. 6*0.05=0.30), then the Type I error rate inflates. Correction for multiple comparisons • Adjusting the alpha to ensure you don’t inflate your Type I error rate. • Bonferroni method, alpha/number of tests you’ll run. • E.g. 0.05/6 = 0.008. Therefore alpha now 0.008 and you’ll only judge statistical significance (p value) when <.008. • There are other methods for methods. 6 comparisons Group 1 v Group 2 Group 1 v Group 3 Group 1 v Group 4 Group 2 v Group 3 Group 2 v Group 4 Group 3 v Group 4 p=.007 p=.046 p=.009 p=.027 p=.128 p=.369 ANOVA Reporting example: F(3,36) = 2.31, p=.002, η2 =.008 There was a statistically significant main effect of TV watching on fatigue, with a medium effect. Post-hoc tests revealed that this effect was driven by those who watched more than 6 hours per day having significantly higher fatigue levels than those who watched less than 1 hour per day (large effect; p=.007). All other post-hoc tests were not statistically significant (p≥.008) and had negligible to weak effects. F(degrees of freedom group, participants) = esimate, p=x, η2 =x BEHL 2005 / BEHL 2019 (UO) Introductory Research Methods COMPARING MORE THAN TWO GROUPS: ANOVA Professor Hannah Keage