IPR Term 2 Week 9 - ANOVA

Introduction to ANOVA

• ANOVA stands for Analysis of Variance and is covered in IPR Term 2 Week 9.
• The lesson will cover the basic concepts of ANOVA, its terminology, and purpose.
• The set of videos introduce t-tests and ANOVA.
• The videos cover the concept of assumptions and different types of ANOVA tests.

Recap of T-Tests

• T-tests compare the means of two groups.
• T-tests are used to determine if a significant difference exists between the means of two groups.
• Matched/same groups (paired t-tests) or different/independent groups (independent t-tests) can be compared using T-tests.
• T-tests can be used in cases when Group A had longer reaction times or more errors than Group B.
• If a study contains more than two groups, ANOVA should be used instead of T-tests.

Multiple Groups

• ANOVA comes into play if there's a need to manipulate more than one independent variable (IV).
• ANOVA is used when comparing three different treatment groups or five different university sports teams etc.
• Running multiple T-tests has a risk of making a Type I error: a false positive.
• The significance level (alpha .05) is the probability of making a Type I error, or a 5% chance.
• Each test that has is done has a 5% chance of producing a false positive.
• Overall probability of making at least one Type I error (false positive) is 14%.

ANOVA

• ANOVA controls the overall Type 1 error rate, and comparisons of each group are combined into one test.
• ANOVA can evaluate if there are any significant differences between groups as a whole.
• If the ANOVA test is significant, at least one of the groups is different from the others.
• ANOVA doesn't specify where the differences lie between the groups.

Post-hoc Analysis

• Post-hoc analysis helps determine where the differences lie between the groups.
• Post-hoc means after the fact is perform an overall analysis(like an ANOVA) that showed its significant result.
• Run post-hoc tests,(e.g., Tukey's test of pairwise comparisons) if the ANOVA test returns a significant result.
• Each test is runs, controlling for the risk of Type 1 errors.
• Note: Pairwise comparisons are a type of post-hoc tests is specific to types of comparison that compare two groups at a time(e.g., Group A vs Group B).
• A priori tests can also be know as 'planned comparisons'.
• A priori means before the fact, where specific tests or comparisons are planned before data is collected and analysed.

Talent Show Example

• ANOVA is compared to a panel of judges assessing overall quality and talent between multiple contestants.
• ANOVA does not identify the best performer, but indicates if enough evidence suggests at least one contestant performed better or worse.
• Post-hoc comparisons are compared to audience members making specific comparisons and votes about who they like best.
• Examples of talent show comparison are Singer A vs Singer B, Singer A vs Singer C, or Singer B vs Singer C.
• Allowing the audience to compare the singers directly, without knowing is a difference between them is like going straight to Tukey without ANOVA.
• Audience members could end up comparing singers who are not actually different in performance.
• ANOVA acts as the judge that gives the global assessment, where making sure there is a meaningful difference.
• Audience members can determine which specific singers are better than the others after the judge has determined if there is a significant difference.

General Linear Model

• ANOVA is part of a larger statistical framework of the General Linear Model (GLM).
• GLM analyses relationships between variables and is used for ANOVA and regression,specifically between the DV and one or more IVs.
• GLMs can handle different types of data (continuous or categorical),different types of models (e.g. regression or ANOVA),group means or linear relationships, and multiple IVs (or predictors).

What is ANOVA

• ANOVA(Analysis of Variance)compares the means of three or more groups.
• Interested in the variability in the data where the IV(s) are categorical.
• ANOVA compares how the mean of the DV differs across different IVs or different levels of the IV.
• ANOVA compares between-group vs within-group variance.
• This provides a test statistic(F): a ratio of between-group to within-group variance.
• Largers F-statistic shows group means differ significantly.
• A result is significant if the associated p-value is less than 0.05.
• Variance can occur in the data because of experimental manipulation or random sample.
• An 'F-ratio' can be calculated to determine if the variance explained is more/less than the variance due to random chance.
• Total Sum of Squares (SS) represents overall variability in the data (DV).

Calculating ANOVA

• Looks at how much each data points differs from a grand mean (the mean of all data points across all groups).
• Variability in the data is attributed to different sources.
• How much total variability exists in the data is looked at.
• Total Sum of Squares (SS) broken down into experimental manipulation(between-group Sum of Squares(SSB)) and Random sample-Within-group Sum of Squares(SSW).
• Experimental manipulation(Between-group Sum of Squares(SSB)) due to the differences between the groups that effect an experimental manipulation.
• Random sample(Within-group Sum of Squares(SSW)) due to the differences within each group causing random error.
• Total Sum of Squares (SS) = SSbetween + SSwithin

ANOVA Output

• The output looks the same in other tests, and shows a 'line of results'.
• An example of the 'line of results' is F(2,12) = 5.23, p = .123.
• Reject the null hypothesis if it is significant.
• Assuming there is at least one group that is significant perform post-hoc Tukey's tests to compare groups.
• If not significant, stop the analysis.

ANOVA Assumptions

• Normality: Both groups/conditions need to be normally distributed and follow a normal (Gaussian) distribution (bell shaped curve). Examples: tests in JASP e.g. Shapiro-Wilk test.
• Independent Observations: Data points need to be independent of each other.
• Homoscedasticity: The variance spread of data is consistent among variables/groups(Homogeneity of variance). Examples: JASP e.g, Levene's test.

ANOVA Types

• One-Way ANOVA: the means of three or more groups based on one independent variable (IV) are compared. This asks if there are significant differences between the group means.
• Two-Way ANOVA: The effect of two independent variables on a dependent variable. This asks if there are significant differences in the 'main effects' of teaching approach and class time on learning, and how do they interact.
• Repeated Measures ANOVA: This occurs when the same participants are measured multiple times under different conditions or at different time points. This asks it the effect of repeated or levels of one or more independent variables can effect learning outcomes over time.