ANOVA: Sum of squares, F-ratio & Hypothesis

Overview of Analysis of Variance (ANOVA)

• ANOVA is an inferential hypothesis-testing procedure.
• ANOVA is used to evaluate mean differences between two or more treatments or populations.
• Both ANOVA and t-tests use sample data to test hypotheses about population means.
• T-tests are limited to comparing only two treatments.
• ANOVA can compare two or more treatments or populations simultaneously.
• ANOVA provides more flexibility in designing experiments and interpreting results.
• The goal of ANOVA is to determine if the mean differences observed among samples provide enough evidence to conclude that there are mean differences among the populations.
• ANOVA determines whether to reject the null hypothesis (H₀).
• Figure 12.1 describes of a typical situation where ANOVA would be used to determine if the means of three samples are significantly different:
  • Population 1 (Treatment 1) has Sample 1, where n = 15, M = 23.1, SS = 114.
  • Population 2 (Treatment 2) has Sample 2, where n = 15, M = 28.5, SS = 130.
  • Population 3 (Treatment 3) has Sample 3, where n = 15, M = 20.8, SS = 101.

Terminology in ANOVA

• Factor: A variable that designates the groups being compared.
• Independent variable: A manipulated variable to create treatment conditions in an experiment.
• Quasi-independent variable: A non-manipulated variable used to designate groups.
• Levels: The individual groups or treatment conditions that make up a factor.
• Example:
  • Factor 1: Therapy technique.
  • Factor 2: Time.
  • Levels: Each group is tested at three different times (repeated measures).
• Single-factor design: Design uses one independent (or quasi-independent) variable.
• Independent-measures design: Design uses separate groups of participants for each treatment condition.
• Two-factor design (Factorial design): Combines two different factors.

Statistical Hypotheses for ANOVA

• Null hypothesis (H₀): States no treatment effect exists; all population means are the same, such µ₁ = µ₂ = µ₃.
• Alternative hypothesis (H₁): States a treatment effect exists; a real, significant difference exists between population means.
  • Any difference between any two of the populations under study can be an alternative.
  • Can be expressed as μ₁ ≠ μ₂ ≠ μ₃ OR μ₁ = μ₃, but μ₂ differs.

Type I Errors and Multiple-Hypothesis Tests

• ANOVA is advantageous for comparing multiple mean differences at once.
• ANOVA avoids the risk of committing a Type I error, which increases with the number of tests used to compare two of the multiple treatments being studied.
• Testwise alpha level: The alpha level you select for each individual hypothesis test.
• Experimentwise alpha level: The total probability of a Type I accumulated from all of the separate tests in the experiment.
• As the number of tests increases, so does the experimentwise alpha level.
• Consider an experiment with 3 treatments:
  • Three separate t-tests would be needed to compare all the mean differences.
  • If all tests use α = .05, each test has a 5% risk of Type I error.
  • These risks accumulate to produce an inflated experimentwise alpha level.
  • ANOVA can compare them all at once, avoiding this inflation.

The Logic of Analysis of Variance

• The first step in ANOVA is to determine the total variability for the entire dataset.
• This is done by combining all the scores from separate samples to obtain a general measure of variability.
• The next step is to break down or analyze the components of the total variability.
• ANOVA breaks down the total variability into two basic components: between-treatments variance and within-treatments variance.

Between- and Within-Treatments Variance

• Between-treatments variance: Measures the overall difference between treatment conditions.
  • These differences caused by sampling or treatment effects.
• Within-treatment variance: Measures variations of scores within each treatment condition.
  • These differences are random and unsystematic.
  • Within-treatment variance occurs when no treatment effects cause the scores to differ.
• Example: the scores in the no-phone condition are M = 4, and in the hand-held condition M = 1.
  • This indicates there is variance between treatments.
  • There is also variance within treatments, since not all the scores in the no-phone condition are equal.

The F-Ratio: The Test Statistic for ANOVA

• The test statistic for ANOVA is the F-ratio.

• The test statistic for ANOVA is similar to the t-statistic.

• ANOVA uses variance to accurately define and measure differences among two or more sample means simultaneously.

• F-ratio compares the between-treatments and within-treatments variances.

• For independent-measures ANOVA:

  F = variance (differences) between sample means / variance (differences) expected with no treatment effect = variance between treatments / variance within treatments

• A large F-ratio value indicates that the sample mean differences are larger than would be expected if no treatment effects exist.

  F = systematic treatment effects + random, unsystematic differences / random, unsystematic differences

• If no treatment exists, the differences between treatments are caused by random factors.

• Both the numerator and denominator measure random differences and s should be rougly the same size.

• If no treatment, the F-ratio should be around 1.00.

  F = 0 + random, unsystematic differences / random, unsystematic differences

• Variance is caused by random and unsystematic variability, and the F-ratio is called the error term:

  • Indicates that the numerator should be larger than the denominator.
  • The F-ratio should be larger than 1.00.

• H₀ for the F-ratio computation would state no evidence of a treatment effect.

• Therefore, the F is near 1.00.

• H₁ would state that a treatment effect exists, resulting in a large F-ratio value.

• Because the denominator of the F-ratio measures only random and unsystematic variability, it is called the error term.

• The error term provides a measure of the variance caused by random, unsystematic differences.

ANOVA Notation and Formulas

• Variables in ANOVA are denoted different from the notations: so far.
• k = Number of treatment conditions/levels of a factor.
• n = Number of scores in each treatment
• N = Total number of scores in the entire study (N = kn).
• T = Treatment total; the sum of total scores for each treatment condition.
  • T = ΣΧ.
• G = Grand total; the sum of all scores in the study.
• SS = sum of squares.
• M = mean.
• ANOVA makes use of many calculations and formulas.
• F-ratio: F = variance between treatments/ variance within treatments.
• Sample variance (s²): sample variance = s² = SS/df
• It is important to be able to compute the between- and within-treatments variances.
• The SS and 𝑑𝑓 for the variances, and for the total study, is crucial.
• The entire process of ANOVA can require nine calculations. The final goal is an F ratio.

Analysis of Sum of Squares (SS)

• ANOVA requires us to first take the total SS and partition the value into between-treatments and within-treatments.
• The Total sum of squares (SS Total) is the sum of squares for the entire set.
• The Within-treatment Sum-of-squares (SS within) is the sum of squares inside each treatment..
• The between-treatments sum of squares can also use the treatment totals.

Analysis of Degrees of Freedom

• The analysis of degrees of freedom (df) follows the same pattern as the analysis of SS and it also involves finding the df for the total set of N scores.
• It will then partition this value into df between treatments and df within-treatments.

More Key information regarding SS and df.

• Important information to use as analysis are done:
  • Total: is the entire set of scores or all scores across all treatment conditions
  • Within: deals with the differences in each separate condition
  • Between: deals with the differences between the conditions

Calculation of Variances (MS) and the F-Ratio

• The term mean square (aka mean of squared deviations, or MS) is used instead of variance.
• The formulas from earlier are the same, but you are just using MS between and MS within
• F-ratio simply compares MS between and MS Within
• ANOVA summary table organizes the results of the analysis in one place and can concisely present the ANOVA results.

Hypothesis Testing and Effect Size with ANOVA

• If the null hypothesis is false, the F-ratio should be greater than 1.00
• The first step of this is to exam the distribution of F-Ratios
• The distribution of F-ratio has the cut off starting at 0 but it is tappered to the right
• F- Ratio's are always positive
• How the shape of distribution depends on Df of MS Value
• The F-distribution table, much shows the critical values for F:
  • To use the table you must know the DF values for F-ratio (Numerator & Denominator) for hypothesis test
  • Top of the Table indicates - The DF values for Numerator
  • Denominator indicates - The DF values of the leftmost column

Steps:

1. Find the SS to obtain SS between and SS within
2. Use the SS Values and df values to calculate the two variances MS/Between and MS/Within
3. Use the two Mass values to compute the F-Ratio

Measuring Effect Size for ANOVA

• Compute the percentage of variance accounted for

Example of Reporting the Results of ANOVA (APA).

• APA has specific instructions for the best ways to to report the Data

Assumptions for the Independent-Measures ANOVA

• That the independent ANOVA requires:
  1. Observations within each sample must be independent.
  2. The populations from which the sample are selected must be normal. ,
  3. There must be Homogeneity of variance.

Post Hoc Tests

• (Posttests) are additional hypothesis that take place after ANOVA:
  • that have already determined which mean differences are NOT already Significant -if you Reject the H, the hypothesis than it is 3 or more the Treatments

Tukey's Honestly Significant Difference ( HSD) Test

• used in psychological, researches computes the honestly significant difference, or the minimum difference between treatment means that is necessary for the amount of Significance:
• If the difference exceeds that the HSD conclude that there is significant difference

Scheffe Test

• It has a safety factor when considering:
  • When Comparing only two treatments. the value of K comes from the original experiment to computed for (k - 1).
  • The Critical value for the "Scheffe F-ratio" is the same way and used to evaluated the F-Ratio used to compare all overall ANOVA's.

Understanding A Conceptual View of ANOVA

• A conceptual idea that always measure the difference between treatments.
• The T Value will show an Extreme side that the scores with value show differences there in mean.
• A extreme side that values show difference in means
• Numerator of the F- ratio always measure difference btw treatments
• Bigger mean differences bigger-ratio
• Denominator of F measures the sample of the scores in sample treatment
• Sample with Smaller F ratio
• Numbers of scores in sample that also impact the result in anova if other factors are held constant
• Greater samples if any case increases likely the hood of rejecting the the hypothesis

Test your knowledge of ANOVA (Analysis of Variance) with this quiz! Questions cover sum of squares, degrees of freedom, mean squares, null hypothesis, F-ratio interpretation and the assumptions.