Week 1: Intermediate Statistical Methods PDF

PALS0045: Intermediate Statistical Methods Week 1: The General Linear Model (GLM) and ANOVA Learning outcomes Understanding the GLM ANOVA as a linear model Decomposing the variance in ANOVA Ma...

PALS0045: Intermediate Statistical Methods Week 1: The General Linear Model (GLM) and ANOVA Learning outcomes Understanding the GLM ANOVA as a linear model Decomposing the variance in ANOVA Main effects, F-values and p-values Post-hoc tests Assumptions of the GLM General linear model (GLM) Flexible statistical approach that can be used to test any hypothesis for which the outcome is numeric (rather than categorical) Which of the following statistical tests that we covered last year have a numeric dependent variable?  Regression  Chi-squared  ANOVA  T-test Regression, t-tests and ANOVA Regression, correlation, t-test and ANOVA are all types of general linear model If you had a question such as “Are people’s reaction times faster in the morning or evening?” A t-test, one-way ANOVA, correlation or regression would give you the same p-value General linear model All GLM models can be expressed as: Outcomei= (model) + errori Tests only differ in how the model is specified Types of GLM ANOVA (week 1 & 2): useful for factorial designs with categorical predictors E.g., placebo vs drug, morning vs evening dose (Multiple) Regression (week 3): Independent variable(s) can be numeric, categorical, or both E.g., Does age and/or IQ predict reaction time? ANCOVA (week 4): Independent variables are categorical and numeric E.g., Does handedness and age predict reaction time? ANOVA terminology ANOVA stands for Analysis Of VAriance Types of ANOVA: One-way ANOVA Two-way ANOVA Factorial ANOVA Single categorical IV Two categorical IVs General term for an arbitrary number of categorical IVs Example: Drug dose: Drug dose x Time of day: Conditions Placebo Placebo morning within a factor/ IVs are Low dose Low dose evening IV are known as known as High dose High dose “levels” “factors” ANOVA terminology ANOVA also differs based on how subjects are allocated to conditions: Between-subject ANOVA Within-subject ANOVA Mixed ANOVA Different participants in Same participants included Some factors are between (different each level of the factor(s) in all levels of the factor(s) participants) some are within (same participants) Time of testing: Time of testing: Drug dose x Time of testing Example: Morning Morning Placebo Morning Midday Midday Drug Evening Evening Evening One-way ANOVA One-factor ANOVAs test hypotheses about mean group differences in situations where we have 2 or more groups We will see how it works by analysing a dataset of reading development in children (Adedeji et al., 2023). 30 children from grades 3, 4, and 5 read short sentences. Are there differences in reading speed across the 3 grades?  One-way ANOVA In ANOVA, we test for a difference across all levels of the factor. Null hypothesis: there is no difference in reading speed between the grades: H0 : µ1 = µ2 = µ3 Null model predicts that all group means will be equal to grand mean µ (overall mean of all data points). NB: µ indicates the population mean for each grade level  One-way ANOVA Alternative hypothesis: there is a difference in reading speed between the grades: H1 : µ1≠ µ2 ≠ µ3 Note that H1 predicts a difference between the groups, but doesn’t specify where the difference is. NB: the group means can differ in many ways- these are just a few examples ANOVA as a ratio of variances ∑ = sum up Variance: Variance measures dispersion: 𝑌𝑌 = mean variability in ∑𝑁𝑁 2 𝑌𝑌𝑖𝑖 = individual score differences 𝑖𝑖=1(𝑌𝑌𝑖𝑖 − 𝑌𝑌) between 𝑁𝑁 individual scores and the ANOVA compares the variance between groups (group mean (average) differences) to the amount of variance within groups (error). Dispersion: If between-group variance is large compared to within- extent to which group variance, then we have evidence against the null a distribution is hypothesis of no group differences. stretched or squeezed One-way ANOVA equation GLM provides a method for measuring within and between group variance that generalizes to situations with more than one factor The GLM equation is: Error (difference 𝑌𝑌𝑖𝑖𝑖𝑖 = 𝜇𝜇 + 𝐴𝐴𝑖𝑖 + 𝑆𝑆(𝐴𝐴)𝑖𝑖𝑖𝑖 between group mean and individual score) Individual Group mean Grand score difference from mean (dependent) grand mean This equation is computed for each data point in the dataset Subscripts: Yij = the jth person in the ith group One-way ANOVA equation More simply, we can represent the GLM equation as: 𝑌𝑌𝑖𝑖𝑖𝑖 = 𝜇𝜇 + 𝐴𝐴𝑖𝑖 + 𝑆𝑆(𝐴𝐴)𝑖𝑖𝑖𝑖 Outcome (dependent)= Model + Error Estimating equations To compute the GLM equations we must estimate the values for the components using the following estimation equations: Dots mean average across values. So average score (Y) across groups (i) and individuals (j) Grand mean: 𝜇𝜇 = 𝑌𝑌.. 𝑖𝑖 = 𝑌𝑌𝑖𝑖. − 𝜇𝜇 Group effects: 𝐴𝐴 Hats mean that this is the sample estimate, 𝑖𝑖𝑖𝑖 = 𝑌𝑌𝑖𝑖𝑖𝑖 − 𝜇𝜇 − 𝐴𝐴̂ 𝑖𝑖 Error: 𝑆𝑆(𝐴𝐴) not the actual population value We have 10 children in each of the 3 groups, so we need 30 GLM equations that require a total of 34 terms 1 grand mean, 3 group effects and 30 error terms (1 per subject) Data Let’s visualise the data from all 30 subjects: Estimating grand mean Grand mean is the average reading time across all grades and participants Estimating group effects Group effects are the deviation of each group mean from the grand mean: 𝐴𝐴𝑖𝑖 = 𝑌𝑌𝑖𝑖. − 𝜇𝜇 𝐴𝐴1 = 11.5- 10.64= 0.86 𝐴𝐴2 = 10.86- 10.64= 0.22 𝐴𝐴3 = 9.55- 10.64= -1.09  Estimating errors Error terms (residuals) are: Residuals (error) 𝑖𝑖𝑖𝑖 = 𝑌𝑌𝑖𝑖𝑖𝑖 − 𝜇𝜇 − 𝐴𝐴̂ 𝑖𝑖 𝑆𝑆(𝐴𝐴) (i.e., subtract the respective group mean from each observation) NB: Note that 𝜇𝜇̂ and 𝐴𝐴̂ 𝑖𝑖 sum up to the mean for each group since 𝐴𝐴̂ 𝑖𝑖 shows the deviation from the grand mean Calculations in R We can apply the same calculations to our data in R: Sums of squares We now want a way of summarising the amount of variance associated with each component of the model. We do that by calculating the sum of squares: we square each number in the column and then add them all up. SStotal = SSµ + SSA + SSS(A) NB: Squaring numbers gets rid of negative values! Mean squares We want to compare within and between group variance, but we cannot just compare SSA with SSS(A) as sums depend on ratio of groups to participants. The sum of squares are affected by the number of group levels and participants in the design. Therefore, we divide the sum of squares by the degrees of freedom; the number of parameters that are freely estimated in the model. This gives us the Mean Squares: 𝑆𝑆𝑆𝑆𝐴𝐴 𝑆𝑆𝑆𝑆𝑆𝑆(𝐴𝐴) Between group variance: 𝑀𝑀𝑀𝑀𝐴𝐴 = Within group variance: 𝑀𝑀𝑀𝑀𝑆𝑆(𝐴𝐴) = 𝑑𝑑𝑑𝑑𝐴𝐴 𝑑𝑑𝑑𝑑𝑆𝑆(𝐴𝐴) Degrees of freedom Degrees of freedom in one-way ANOVA are calculated as follows: dfA = number of groups in factor -1 dfS(A)= number of participants - number of groups Mean squares in R Divide sum of squares by degrees of freedom: NB: Degrees of freedom for SSA is 3-1= 2 because we have 3 groups Degrees of freedom for SSS(A) is 30-3= 27 because we have 30 participants and 3 groups F statistic and p-value F statistic or (F ratio) is the ratio of variance due to differences between groups to variance within groups 𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 𝑀𝑀𝑀𝑀𝐴𝐴 𝐹𝐹 = = 𝑀𝑀𝑀𝑀𝑀𝑀𝑀𝑀 𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠𝑠 𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒𝑒 𝑀𝑀𝑀𝑀𝑆𝑆(𝐴𝐴) We can then derive the p-value from the F-value: NB: Critical F-value is the value at which we declare significance (given our degrees of freedom and alpha level) One-way ANOVA in R We saw how we can compute the analysis ourselves, but in practice we will use ready-made functions to run ANOVAs. We get the same results. Nice! Post-hoc tests in R We found a significant main effect of grade- there is a significant difference in reading speed between some of the grades. But which ones?! To find out, we need to perform post-hoc tests. Significant difference! NB: A significant main effect in ANOVA suggests a difference in the factor, but we can’t say where! Writing up the findings in APA style A one-way ANOVA with Reading speed as the dependent variable and Grade (3 levels: 3rd, 4th and 5th) tested the hypothesis that reading speed changes with school grades. The results showed a significant main effect of grade, F(2, 27) = 3.69, p = 0.038, ηg2= 0.21. We can therefore reject the null hypothesis that the group means are equal. Post-hoc t-tests with a Bonferroni correction showed that children in Grade 5 (M= 9.55 s; SD= 1.44 s) had significantly faster reading speed compared to children in Grade 3 (M= 11.5 s; SD= 1.82 s), t(27)= 2.66, p= 0.038. There were no significant differences between Grade 4 and Grade 5 (p= 0.16), or Grade 3 and Grade 4 (p= 0.39). GLM assumptions 1) Observations are i.i.d- independent and identically distributed How to check? - Determined by design! Example: coin tosses - “Independent” means values are independent from each other are both independent (knowing 1 value cannot predict other values) (one toss doesn’t affect the next) and identical - “Identically” means observations are generated by the same (generated by the same process process) GLM assumptions 2) Visual inspection: the violon plots (different groups) should have approx. the same vertical spread. 2) Homogeneity of variances: the variances across all levels of the factor (in between- subject ANOVA) are the same How to check? 1) Levene’s test: non-significant value means we can assume homogeneity of variances. NB: Beware of Levene’s test with large sample sizes! It becomes overly sensitive. GLM assumptions We can see some deviations here! 3) Normality of residuals: the residuals (error) are normally distributed. How to check? Do a QQ plot of residuals: dots should roughly fall along the line. NB: GLMs only predict that the residuals should be normally distributed, not the dependent variable itself. What if assumptions aren’t met? A few options: 1. Carry out an ANOVA anyway (reduces power and increases risk of a Type II error). 2. Transform the dependent or use other more suitable models (covered later in the course). 3. Use a “nonparametric” test which doesn’t require the data to conform to these assumptions. For one-way ANOVA, this is the Kruskal–Wallis test. Reading Miller and Haden (2011) Chapters 1-3 Understanding degrees of freedom: https://medium.com/@dlectus/degrees-of- freedom-simply-explained-a96cafa3b39f