Lecture 4 - Non-parametric statistics 2023 2024.pptx

Non-parametric statistics Lecture 4 1 Overview  Checking distributional assumptions  Kruskal-Wallis  Friedman  A flavour of resampling methods 2 Assumptions we usually worry about in ANOVA (for the p-value to be valid) Between-subjects ANOVA – Normality (related to this, no outliers) – Homogenei...

Non-parametric statistics Lecture 4 1 Overview  Checking distributional assumptions  Kruskal-Wallis  Friedman  A flavour of resampling methods 2 Assumptions we usually worry about in ANOVA (for the p-value to be valid) Between-subjects ANOVA – Normality (related to this, no outliers) – Homogeneity of variance – Independence of scores (e.g. not accidentally do a between-subjects ANOVA on repeated-measures data). 3 (1) Homogeneity of variance Group 2 is more ‘spread out’ 4 (1) Homogeneity of variance Check with Levine’s test – Pooling each score’s variation around mean using a magic formula… ANOVA is robust, but only when sample sizes are equal. Important to collect data with equal n’s per group. The biases can be quite exotic: – Larger group – higher variance  F-ratio more conservative (likelier to produce false negative) – Larger group – smaller variance  F-ratio more liberal (likelier to produce false positive) (2) Is the distribution ‘bell-shaped’ (approx. normal or ‘Gaussian’)? skewness, kurtosis Different interpretations but for the sake of significance testing: – “Sampling distribution of the parameter we are testing (e.g., group means) is normal.” – With big enough samples this will be the case (‘central limit theorem’). – How big is “big enough” – rule of thumb ≥ 30 See this simulation for an illustration: https://onlinestatbook.com/stat_sim/sampling_dist/index.html 6 (2) Is the distribution ‘bell-shaped’ (approx. normal or ‘Gaussian’)? skewness, kurtosis 7 JASP will calculate those for you. Using the Std. Error we can check them for significance (use with caution – see next slide) 8 Direct tests of normality (best to avoid) Kolmogorov–Smirnov Shapiro–Wilk (recommended as more accurate) High false positive rate for larger samples and low power for smaller samples. Best to avoid these tests and rely on visual inspection of the data (e.g. histograms). ANOVA robust to violations of normality when group sizes are equal. 9 One-way ANOVAs and their non-parametric alternatives Parametric ANOVA Non parametric test alternative One-way betweenKruskal–Wallis subjects One-way repeatedFriedman measuresnon-parametric tests also exist for the (Alternative independent- and dependent-samples t-test.) 10 Non-parametric alternatives Can be less powerful than corresponding parametric test. – If assumptions would justify a parametric test (don’t do a non-parametric test ‘just to be safe’) – By ranking data we lose information about magnitude of difference between scores. 11 Kruskal–Wallis (alternative to one-way between-subjects ANOVA) Based on ranking the data 1. Rank all cases, ignoring group membership 2. Sum up ranks for each group 3. Plug those sums of ranks into magic formula to generate H statistic. Sum of Rank ranks Grou ed per p Data data group 1 5 4 6 1 3 2 2 7 6 9 2 4 3 3 1 1 6 3 6 5 4. The H statistic has a known probability distribution df = 1 df = 5 df=10 – ‘Chi square’ – – df = nr. groups – 1 allows us to calculate a p-value (just as we do with our F-ratio in ANOVA using the F distribution). 5. JASP will do the grunt work for you… 12 Follow-up tests Pairwise comparison with a Bonferroni correction (we’ve discussed this as an ANOVA posthoc). – very conservative – high chance of false negative Sequential Methods (less conservative). – Rank p-values – Adjust alpha by each p-value’s rank. – Holm’s test – start with smallest p-value. Stop when first non-significant p-value reach. All p-values after this deemed nonsignificant. – Hochberg’s test – start with largest pvalue. Stop when first significant p-value reached. All p-values after this deemed significant. 13 Friedman test (for repeated measures) Calculate ranks for each subject Calculate mean ranks per treatment Magic formula to produce an If nr. observations > 10 square Fr statistic … Fr follows a Chi- distribution. We can test it for significance As with Kruskal–Wallis, follow up using pair-wise or sequential procedure. As usual, JASP will do the grunt work for you… 14 A major disadvantage of non-parametric tests – Developed for limited range of situations – Not available for more complex designs 15 Bootstrapping Mark the 2.5% and 97.5% percentile in this distribution. This is now our 95% confidence interval for the ‘true’ population mean. 16 compare 95% bootstrapped confidence intervals for the caffeine data to posthoc results (last lecture) Bootstrapping Computationally expensive – relies on advent of computers (no ‘paper and pencil’ calcs) Estimate properties of sampling distribution from data Create parameter confidence intervals A number of procedures implemented in a free statistical program called R ( www.r-project.org) Good visual illustration of the method: https://www.youtube.com/watch?v=zecArldpptY 17 Readings 18