ANOVA: Analysis of Variance

Chapter 9: ANOVA - Analysis of Variance

• ANOVA is a statistical method used to compare means of two or more populations.
• This method is applicable in medicine, engineering, and business.
• Key concepts, assumptions, hypotheses, and test statistics are important elements within ANOVA.
• One-way, two-way, and repeated measures ANOVA are different types of ANOVA.

Basic Concepts of ANOVA

• ANOVA works by dividing the total variation in a dataset into different sources of variation.
• The total variation is measured by the total sum of squares (SST).
• SST represents the sum of squared differences between each observation and the overall mean.
• SST = Sum of squares due to treatment (SST) + Sum of squares due to error (SSE).
• SST indicates variation between the means of different treatment groups.
• SSE indicates variation within each treatment group.

Assumptions of ANOVA

• Data should be normally distributed.
• Populations should have equal variances.
• Observations should be independent.
• Not meeting these assumptions may invalidate ANOVA results.

Hypotheses of ANOVA

• Null Hypothesis ($H_0$) states that the means of all populations are equal: $H_0: \mu_1 = \mu_2 =... = \mu_k$.
• Alternative Hypothesis ($H_1$) states that at least one population mean is different from the others.
• $H_1$ is written as: At least one $\mu_i$ is different from the others
• $\mu_i$ represents the mean of the $i$th population, and $k$ is the number of populations.

Test Statistic of ANOVA

• The F-statistic is the test statistic, which is the ratio of the mean square due to treatment (MST) to the mean square due to error (MSE).
• MST is calculated by dividing SST by the degrees of freedom due to treatment (dfT).
• dfT is equal to the number of treatment groups minus 1.
• MSE is calculated by dividing SSE by the degrees of freedom due to error (dfE).
• dfE is equal to the total number of observations minus the number of treatment groups.
• The F-statistic follows an F-distribution with dfT and dfE degrees of freedom.

One-Way ANOVA

• Used to compare means of two or more populations with one independent variable factor.
• For instance, comparing the average test scores of students using three different teaching methods.
• Steps include:
  • State the null and alternative hypotheses.
  • Calculate the SST, SST, and SSE.
  • Calculate the MST and MSE.
  • Calculate the F-statistic.
  • Determine the p-value.
  • Make a decision based on the p-value.

Two-Way ANOVA

• Used to compare means of two or more populations with two or more independent variable factors.
• As an example, comparing average test scores of students taught using three different methods and coming from two different schools.
• The steps are similar to one-way ANOVA.

Repeated Measures ANOVA

• Used to compare means of two or more populations when observations are correlated.
• As an example, comparing the blood pressure of patients before and after they take a new drug.
• Repeated measures ANOVA considers the correlation between observations to increase the test's power.

Conclusion

• ANOVA is effective for comparing means of two or more populations.
• This method is used across various fields.
• Understanding its assumptions and using the correct type is important.