Data Analysis in Psychology - ANOVA Models PDF

Data Analysis in Psychology ANalysis Of VAriance (ANOVA) Statistical Methods Data Analysis in Psychology One-Way ANOVA  Hypothesis Testing Experimental Research Design with more than two groups R X O1 G1...

Data Analysis in Psychology ANalysis Of VAriance (ANOVA) Statistical Methods Data Analysis in Psychology One-Way ANOVA  Hypothesis Testing Experimental Research Design with more than two groups R X O1 G1 R O2 G2 R O3 G3  One Way ANOVA One-Way ANOVA One-Way ANOVA (“ANalysis Of VAriance") compares the means of two or more independent groups in order to determine whether there is statistical evidence that the associated population means are significantly different. One-Way ANOVA is a parametric test. This test is also known as: One-Factor ANOVA One-Way Analysis of Variance Between Subjects ANOVA The variables used in this test are known as: Dependent variable Independent variable (also known as the grouping variable, or factor) This variable divides cases into two or more mutually exclusive levels, or groups  One Way ANOVA The One-Way ANOVA is often used to analyze data from the following types of studies: Field studies Experiments Quasi-experiments Note: Both the One-Way ANOVA and the Independent Samples t Test can compare the means for two groups. However, only the One-Way ANOVA can compare the means across three or more groups. Note: If the grouping variable has only two groups, then the results of a one- way ANOVA and the independent samples t test will be equivalent. In fact, if you run both an independent samples t test and a one-way ANOVA in this situation, you should be able to confirm that t2=F.  One Way ANOVA Your data must meet the following requirements: 1. Dependent variable that is continuous (i.e., interval or ratio level) 2. Independent variable that is categorical (i.e., two or more groups) 3. Cases that have values on both the dependent and independent variables 4. Independent samples/groups (i.e., independence of observations) 1. There is no relationship between the subjects in each sample. This means that: 1. subjects in the first group cannot also be in the second group 2. no subject in either group can influence subjects in the other group 3. no group can influence the other group 5. Random sample of data from the population 6. Normal distribution (approximately) of the dependent variable for each group (i.e., for each level of the factor) 1. Non-normal population distributions, especially those that are thick-tailed or heavily skewed, considerably reduce the power of the test 2. Among moderate or large samples, a violation of normality may yield fairly accurate p values 7. Homogeneity of variances (i.e., variances approximately equal across groups) 1. When this assumption is violated and the sample sizes differ among groups, the p value for the overall F test is not trustworthy. These conditions warrant using alternative statistics that do not assume equal variances among populations, such as the Browne-Forsythe or Welch statistics (available via Options in the One-Way ANOVA dialog box). 2. When this assumption is violated, regardless of whether the group sample sizes are fairly equal, the results may not be trustworthy for post hoc tests. When variances are unequal, post hoc tests that do not assume equal variances should be used (e.g., Dunnett’s C).  One Way ANOVA Researchers often follow a few rules of thumb for one-way ANOVA: Each group should have at least 20 subjects (ideally more; inferences for the population will be more tenuous with too few subjects: → use power analysis to determine the required sample size) Balanced designs (i.e., same number of subjects in each group) are ideal; extremely unbalanced designs increase the possibility that violating any of the requirements/assumptions will threaten the validity of the ANOVA F test However, Anova is fairly robust to this violation (Fisher said)  One Way ANOVA Hypotheses The null and alternative hypotheses of one-way ANOVA can be expressed as: H0: µ1 = µ2 = µ3 =... = µk ("all k groups means are equal") H1: µ1 ≠ µ2 ≠ µ3 ≠... ≠ µk → At least one µi different ("at least one of the k group means is not equal to the others") where µi is the population mean of the ith group (i = 1, 2,..., k)  One Way ANOVA Note: The One-Way ANOVA is considered an omnibus (Latin for “all”) test because the F test indicates whether the model is significant overall—i.e., whether or not there are any significant differences in the means between any of the groups. Stated another way, this says that at least one of the means is different from the others. However, it does not indicate which mean is different. Determining which specific pairs of means are significantly different requires either contrasts or post hoc (Latin for “after this”) tests.  One Way ANOVA Test Statistic The test statistic for a One-Way ANOVA is denoted as F (F-ratio). For an independent variable with k groups, the F statistic evaluates whether the group means are significantly different. Because the computation of the F statistic is slightly more involved than computing the paired or independent samples t test statistics, it's extremely common for all of the F statistic components to be depicted in a table like the following:  One Way ANOVA Decomposition of Source of Variations → Sum of Squares [SS] Total Variability SST = SSB + SSE SST (N-1) Between Groups Within Groups SSB (K-1) (error/residuals) SSE (N-k)  One Way ANOVA What are the degrees of freedom (df) ? The degrees of freedom (df) are the number of independent pieces of information Degrees of freedom refers to the maximum number of logically independent values, which are values that have the freedom to vary in a given data sample. Degrees of freedom are commonly discussed in relation to various forms of hypothesis testing in statistics, such as a chi-square, t-test, ANOVA In ANOVA analysis once the Sum of Squares (e.g., SSB SSE) are calculated, they are divided by corresponding DF to get Mean Squares (e.g. MSB, MSE), which are the variance of the corresponding quantity.  One Way ANOVA ANOVA Table: how it typically looks like  One Way ANOVA SSB = the Between Groups Sum of Squares SSW = the Within Sum of Squares SST = the Total Sum of Squares (SST = SSB + SSW) dfr = the model degrees of freedom (equal to dfr = k - 1) dfe = the within degrees of freedom (equal to dfe = n - k ) k = the total number of groups (levels of the independent variable) n = the total number of valid observations dfT = the total degrees of freedom (equal to dfT = dfr + dfe = n - 1) MSB = SSB/dfr = the between group mean square MSW = SSE/dfe = the mean square error or within group  One Way ANOVA SSB = the Between Groups Sum of Squares SSW = the Within Sum of Squares SST = the Total Sum of Squares (SST = SSB + SSW) dfr = the model degrees of freedom (equal to dfr = k - 1) dfe = the within degrees of freedom (equal to dfe = n - k ) k = the total number of groups (levels of the independent variable) n = the total number of valid observations dfT = the total degrees of freedom (equal to dfT = dfr + dfe = n - 1) MSB = SSB/dfr = the between group mean square MSW = SSE/dfe = the mean square error or within group  One Way ANOVA The terms Treatment (or Model) and Error are the terms most commonly used in natural sciences and in traditional experimental design texts. In the social sciences and PSYCHOLOGY, it is more common to see the terms Between groups instead of "Treatment", and Within groups instead of "Error". The between/within terminology is what SPSS uses in the one-way ANOVA procedure. Treatment Between Error Within/Residuals  One Way ANOVA The F-ratio is the Between Groups variance (or Mean Squares Between → MSB) divided by the Within-groups Variance (or Mean Squares Within → MSW)  One Way ANOVA How to read an ANOVA Table: Summary Overview (jamovi) Source of Variation  One Way ANOVA How to read an ANOVA Table: Summary Overview (jamovi) Sum of Squares (SS) SST = SSB + SSW = 26,2 + 31,1 = 57,3  One Way ANOVA How to read an ANOVA Table: Summary Overview (jamovi) Degrees of Freadom (df) dfT = dfr + dfe = 8 + 1 = 9 (→ 10 ps)  One Way ANOVA How to read an ANOVA Table: Summary Overview (jamovi) Mean of Squares (MS) MSB = SSB/dfb = the between group mean square MSW = SSE/dfw = the mean square error or within group  One Way ANOVA How to read an ANOVA Table: Summary Overview (jamovi) F-ratio  One Way ANOVA How to read an ANOVA Table: Summary Overview (jamovi) p-value of F-ratio  One Way ANOVA How to read an ANOVA Table: Summary Overview One-Way ANOVA … how to run it with jamovi ….  One Way ANOVA Experimental design (example of post-test only with control group design) Mental Coach Training Sport Self-Efficacy scale NO Mental Coach Training  One Way ANOVA Quasi-Experimental design (example of post-test only with control group design no random) Mental Coach Training Sport Self-Efficacy scale NO Mental Coach Training  One Way ANOVA One-Way ANOVA with two Groups  One Way ANOVA One-Way ANOVA with two Groups  Hypothesis Testing Experimental Research Design with R X O1 G1 more than two groups R O2 G2 R O3 G3  Hypothesis Testing Experimental Research Design with R X O1 G1 more than two groups R O2 G2 R O3 G3 joyzepam anxifree Mood Gain If Ss assignment to the different experimental conditions cannot be randomized → Quasi-Experimental Design  One Way ANOVA One-Way ANOVA with three Groups  One Way ANOVA One-Way ANOVA with three Groups  One Way ANOVA One-Way ANOVA with three Groups  One Way ANOVA One-Way ANOVA with three Groups  One Way ANOVA One-Way ANOVA with three Groups Effect size in ANOVA Models  Effect size in ANOVA Effect size → Strength of Association a quantitative measure of the magnitude of an (experimental) effect the magnitude of the difference between (two) groups e.g., Cohen’s d, Hedges’s g … a value measuring the strength of the relationship between two variables e.g., r, r² , O.R., R.R., … (f²) Eta squared, Partial Eta squared and Omega squared represents measures of effect size that are commonly used in ANOVA models.  Effect size in ANOVA Eta squared is a measure of effect size that is commonly used in ANOVA models. It measures the proportion of variance associated with each main effect and interaction effect in an ANOVA model. How to Calculate Eta Squared: The formula to calculate Eta squared is straightforward: Eta squared = SSeffect / SStotal where: SSeffect: The sum of squares of an effect for one variable. SStotal: The total sum of squares in the ANOVA model.  Effect size in ANOVA Eta squared The value for Eta squared ranges from 0 to 1, where values closer to 1 indicate a higher proportion of variance that can be explained by a given variable in the model. The following rules of thumb are used to interpret values for Eta squared:.01: Small effect size.06: Medium effect size.14 or higher: Large effect size WHY? The p-value can only tell us whether or not there is some significant association between two variables (i.e., not due to chance) A measure of effect size (like Eta squared, Cohen’s d, r, OR …) can tell us the strength of association between the variables → answering the question “how much”?  Effect size in ANOVA Partial Eta squared is a measure of effect size that is commonly used in ANOVA models. It measures the proportion of variance explained by a given variable of the total variance remaining after accounting for variance explained by other variables in the model. How to Calculate Partial Eta Squared The formula to calculate Partial eta squared is as follows: Partial eta squared = SSeffect / (SSeffect + SSerror) where: SSeffect: The sum of squares of an effect for one variable. SSerror: The sum of squares error in the ANOVA model. IN ONE-WAY ANOVA → ETA-SQUARED AND PARTIAL ETA-SQUARED ARE EQUAL  Effect size in ANOVA Omega squared (ω2) is a measure of effect size in ANOVA models. It is an estimate of how much variance in the response variables are accounted for by the explanatory variables. Omega squared is widely viewed as a lesser biased alternative to eta-squared, especially when sample sizes are small. ω2 = [SSeffect – (dfeffect)(MSerror)] / MSerror + SStotal Where: SSeffect: The sum of squares of an effect for one variable. df: degrees of freedom of the effect MSerror: The Means Sum of Squares of error/residual (error variance) SStotal: The total sum of squares in the ANOVA model.  Effect size in ANOVA Summary of effect size estimation used in ANOVA Models Effect sizes in ANOVA  Effect size in ANOVA Olejnik and Algina (2003) proposed also generalized eta squared (η2G) which excludes variation from other factors from the effect size calculation to make the effect size comparable with designs in which these factors were not manipulated but includes variance due to individual differences (to make the effect size comparable with between-subjects designs where this individual variance cannot be controlled for) When all factors are manipulated between participants η2G and η2p are identical.  Effect size in ANOVA Olejnik and Algina (2003) proposed generalized eta squared (η2G) Repeated Measures and Mixed Design ANOVAs One-Way ANOVA … how to run it with jamovi …. Effect size in jamovi Data Analysis in Psychology Factorial ANOVA - Between subjects designs or “two gustis is melios che one” - … “two gustis is melios che one” … 1995 https://www.youtube.com/watch?v=fKHbscGAy98  Factorial ANOVA Factorial Analysis of Variance (ANOVA) is a statistical procedure that allows researchers to explore the influence of two or more independent variables (factors) on a single dependent variable. In contrast to a one-way ANOVA, a factorial ANOVA uses two or more independent variables (factors) with two or more categories to predict change in a single dependent variable. If the factorial ANOVA has 2 Between factors, it is also called Two-way ANOVA  Factorial ANOVA Many experimental designs use factorial ANOVA to explore differences between treatment groups while considering individual characteristics. In this regards, Factorial ANOVAs offer at least two advantages. First, factorial ANOVAs allow researchers to explore how multiple independent variables affect change the in the dependent variable. This effect is measured with individual main effects for each factor, along with the interaction effect with all factors. Second, factorial ANOVAs are a more powerful test because they reduce potential error variance.  Factorial ANOVA A Factor is a categorical variable used for analysis with two or more categories. Each category represents a value on the factor and can be used to group participants in the study. Factorial ANOVA designs are defined based on the number of factors and the number of categories of each factor used in the study  Factorial ANOVA Factorial - multiple factors A factorial design is an experiment/study with two (or more) factors (independent variables). 2 x 4 design means two independent variables, one with 2 levels and one with 4 levels "conditions" or "groups" are calculated by multiplying the levels, so a 2x4 design has 8 different conditions/sub-groups Results: Main effects Interaction effects  Factorial ANOVA Factorial - multiple factors examples A study using biological sex (males and females) and three different treatment/experimental conditions (A, B, C) would result in a 2 × 3 factorial ANOVA design The factors are crossed in the study and create six groups that represent the factor categories: males, treatment A; males, treatment B; males, treatment C; females, treatment A; females, treatment B; females, treatment C.  Factorial ANOVA Example of 2-factor between groups ANOVA (2x3 Design) Sex by Treatments Treatments Treatment A Treatment B Treatment C Males Sex Females Factorial ANOVAs can be even more complex and include more than two factors. A study with three factors all with two categories would be a 2 × 2 × 2, whereas A factorial ANOVA with two factors with four categories each would be a 4 × 4.  Factorial ANOVA Example of 2-factor between groups ANOVA (2x2 Design)  Factorial ANOVA Example of 2-factor between groups ANOVA (2x2 Design)  Factorial ANOVA Partitioning the variability of 2-factors between groups ANOVA with degrees of freedom N-1 K-1 N-K p-1 q-1 (p-1)*(q-1)  Factorial ANOVA Factorial ANOVAs have two or more independent variables, meaning multiple F tests and post hoc analyses are used to find differences between categories. Main effects refer to the mean comparisons for each factor. These effects point to differences between categories on each individual factor included in the design. Interaction effects test how the effect of one factor is impacted by the categories on one or more other factors. Interaction effects explore how the combined categories created by the two main effects differ from each other They show how combinations of categories can influence the dependent variable.  Factorial ANOVA One-way ANOVA vs. Two-way ANOVA  Factorial ANOVA Examples of Possible Factorial Anova Factorial Anova  Factorial ANOVA Factorial ANOVA with two factors and multiple F tests (Anova table)  Factorial ANOVA Factorial ANOVA with two factors and multiple F tests (Anova table)  Factorial ANOVA Factorial ANOVA with two factors and multiple F tests (jamovi output)  Factorial ANOVA Factorial ANOVA with two factors and multiple F tests (jamovi output)  Factorial ANOVA Possible effects  Factorial ANOVA Possible effects Factorial ANOVA … how to run it with jamovi ….  Factorial ANOVA 2 x 2 Factorial Anova (Two-way Anova) IV Group (treatment vs. control) Sex (M vs. F) DV Memory Factorial ANOVA  Factorial ANOVA 2 x 2 Factorial Anova (Two-way Anova) Group (treatment vs. control) Sex (M vs. F) DV: Memory  Factorial ANOVA 2 x 2 Factorial Anova (Two-way Anova) Group (treatment vs. control) Sex (M vs. F) DV: Memory  Factorial ANOVA 2 x 2 Factorial Anova (Two-way Anova) Group (treatment vs. control) Sex (M vs. F) DV: Memory Means of Main Effects of Group and Sex  Factorial ANOVA Means of Interaction Group by Sex  Factorial ANOVA Post hoc comparison → Group by Sex  Factorial ANOVA Means of Interaction Group by Sex  Factorial ANOVA Means Plots of Interaction Group by Sex Factorial ANOVA … how to run it with jamovi ….  Data Analysis in Psychology Anova -Repeated Measures and Mixed Anova Designs-  Factorial ANOVA Factorial ANOVAs can be designed in three ways based on participant data: Between subjects We did One-Way Anova, Factorial Anova Within subjects (→ repeated measures) and Mixed design (Between factors and within factors)  Factorial ANOVA Factorial ANOVAs Between-subjects design has participants that are contributing to only one of the groups. A between-subjects design leads researchers to conclude changes and differences in means can be attributed to specific characteristics of the group(s). The earlier a 2 × 2 factorial ANOVA examples represents a between-subjects design since each participant provides information to only one group (e.g., Sex × Treatment group) and researchers can determine influence based on the characteristics associated with specific groups.  Factorial ANOVA Factorial ANOVAs can be designed in three ways based on participant data: Between subjects One-Way Anova, Factorial Anova Now Within subjects (→ repeated measures) and Mixed design (Between factors and within factors)  Factorial ANOVA Factorial ANOVAs designs Within-subjects design refers to an experiment in which participants contribute information to each group in the experiment. Within-subject designs are used when data are collected from the same participants at multiple time points → repeated measures  Factorial ANOVA Different ANOVAs designs: One-way, Factorial, Repeated Measures (Within Ss)  Factorial ANOVA Different ANOVAs designs Repeated Measures RM (Within Subjects Ss) Repeated Measures O1 O2 O3 O4 O5 G1 without treatment time Repeated Measures O1 O2 X O3 O4 G1 with treatment Interrupted time series time  Factorial ANOVA Factorial ANOVAs designs A mixed design includes at least one independent variable measured within subjects and at least one independent variable measured between subjects. Within (→ Ss measured at different time points) Between (→ effect of treatment or other grouping variable, e.g. sex …) R O1 Xa O2 G1 R O3 Xb O4 G2  Factorial ANOVA Mixed ANOVAs designs One (or more) Between Groups Factor and One (or more) Within Subjects Factor (Repeated Measure) R O1 X O2 G1 R O3 O4 G2 time  Factorial ANOVA Mixed ANOVAs designs: One (or more) Between Groups Factor and One (or more) Within Subjects Factor (Repeated Measure) Repeated Measures: The DV is measured more than one time over the same Ss  Solomon Design Solomon Design … remember???? R O1 X O2 G1 R O3 O4 G2 R X O5 G3 R O6 G4 time  Solomon Design How do you analyze data gathered with such designs???? R O1 X O2 G1 ????????? R O3 O4 G2 R X O5 G3 ????????? R O6 G4  Solomon Design and Anova How do you analyze data gathered with such designs???? R O1 X O2 G1 Mixed design Anova R O3 O4 G2 R X O5 G3 One-way Anova R O6 G4 Solomon Design and Anova Data Analysis in Psychology ANOVA – Within subjects designs -  Repeated Measures ANOVA RM Design - Example  Repeated Measures ANOVA Partitioning the Source of Variations Repeated Measures ANOVA … how to run it with jamovi ….  Repeated Measures ANOVA RM – Example A sample of 25 patients measured 4 times 2 times before the therapy, 2 times after the therapy Repeated Measures O1 O2 X O3 O4 G1 with treatment Interrupted time series time Repeated Measures ANOVA  Repeated Measures ANOVA RM – Example A sample of 25 patients measured 4 times 2 times before the therapy, 2 times after the therapy No Between Subjects Factor!!!  Repeated Measures ANOVA RM – Example A sample of 25 patients measured 4 times 2 times before the therapy, 2 times after the therapy  Repeated Measures ANOVA RM – Table of the means  Repeated Measures ANOVA RM – Means Plot (with CI) Repeated Measures ANOVA Data Analysis in Psychology ANOVA – Mixed designs -  Mixed ANOVA ANOVA mixed designs: Between Ss Factor + Within Ss Factor (RM) R O1 X O2 G1 R O3 O4 G2 time  Mixed ANOVA Partitioning the Source of Variations Total Variability Between Subjects Within Subjects Between Within Groups RM factor Interaction Residual Groups (residual) (Within Ss) (BG*RM) Mixed ANOVA … how to run it with jamovi ….  Mixed ANOVA Mixed ANOVA – Example A sample of 53 patients measured 2 times Experimental Group: 1 time before the therapy, 1 time after the therapy Control Group: 2 times with no therapy DV: Language R O1 X O2 G1 R O3 O4 G2 time Mixed ANOVA  Mixed ANOVA Mixed ANOVA – Example Experimental Group: 1 time before the therapy, 1 times after the therapy Control Group: 2 times no therapy  Mixed ANOVA Partitioning the Source of Variations – jamovi output Total Variability Between Subjects Between Within Groups Groups (residual)  Mixed ANOVA Partitioning the Source of Variations – jamovi output Total Variability Within Subjects RM factor Interaction Residual (Within Ss) (BG*RM)  Mixed ANOVA Mixed Anova – Table of the means  Mixed ANOVA Mixed Anova – Plot of the means  Mixed ANOVA Mixed Anova – Post-hoc comparison  Mixed ANOVA Mixed Anova – Means Plot (with CI) Mixed ANOVA … how to run it with jamovi …. one more example  Mixed ANOVA Example Exploring the Impact of Physical Activity and Exercise on Cognitive Abilities in Obese Children. The research design employs a 2x2 structure, featuring a between-group factor (experimental group with a physical activity program vs. control group with no program) and a repeated measure factor (pretest administered before the program and a post-test conducted after the program). The variables under examination include attention ability and effortful control. What are the Ivs??? What are the DVs??? What type of analysis would you apply?  Mixed ANOVA Mixed ANOVA – Example A sample of 10 patients measured 2 times Experimental Group: 1 time before the program, 1 time after the program Control Group: 2 times with no therapy DV: Attention + Control R O1 X O2 G1 R O3 O4 G2 time Mixed ANOVA … how to run it with jamovi …. What we dealt with: where are we now? What we dealt with: where are we now? What we dealt with: where are we now? Where are we going next? Where are we going next? Relationships between variables Where are we going next? Relationships between variables Where are we going next? Relationships between variables The End