ANOVA: Analysis of Variance

• Analysis of Variance, or ANOVA, is covered in Unit Three of SSBA II – Major 08.

Variance

• Variance is the average squared deviation of a random variable from its mean.
• Variance measures the variability to determine how spread out data points are from the average value.
• Different distributions can have the same mean but different amounts of variability.
• The idea of variance can be used in descriptive statistics, hypothesis testing, and many other statistical topics.

Analysis of Variance (ANOVA)

• ANOVA studies the effects of one or more independent variables on a single dependent variable.
• For example, the effect of a practice schedule (independent variable) on psychological test performance (dependent variable).
• ANOVA determines if the exposure of a sample to an independent variable significantly enhances the variance of the dependent variable above its variance due to random factors.

Experimental Design

• This involves scientifically planning an experiment to explore the effect of independent variables on a specific dependent variable.

Opting for Uncontrolled or Controlled Experiments is the First Key Step

• An uncontrolled experiment example would be studying the effects of adolescent tobacco smoking habits on adult pulmonary carcinoma.
• Controlled investigations involve "fixed" treatment variables where levels, methods, modes, and application timings are controlled to eliminate random errors.
• Controlled application can help contain or eliminate relevant variables affecting the dependent variable.

Choosing Retrospective or Prospective Methods Is the Second Key Step

• The retrospective method is used for uncontrolled experiments with investigator-independent classification variables to explore a possible past cause for existing events or changes.
• The prospective method is used mainly in controlled experiments, where "fixed" experimental treatments are applied under the control of the investigator as independent variables.

Conducting a Pilot Experiment is the Third Key Step

• A pilot experiment using small groups is performed before planning a full-scale investigation.

Levels and Replications

• Levels consist of amounts, intensities, amplitudes, categories or doses of the independent variable (factor) applied to the sample.
• The number of groups (k) used in an experiment depends on the number of levels of the factor applied.
• To study the effect of a hypoglycemic factor (independent variable) on blood sugar (dependent variable), individuals from a sample would be allocated to three groups (k=3), and three levels, such as 0, 5, and 10 micrograms would be administered.
• Minimizing experimental errors requires each level of the factor (independent variable) to be applied to more than one individual, constituting a group, which is known as the replication of each level of the factor.
• If an experiment intends ten replications of each of three levels of a factor, each of the three groups should consist of 10 individuals (n=10) chosen at random from the sample.
• A sample of 310 or 30 cases (nk) is to be drawn initially from the population.

Single-Factor and Factorial Experiments

• A single-factor or single-classification experiment explores changes in the dependent variable due to the exposure of groups from a sample to the levels of a single independent variable.
• A factorial experiment studies the effect of combinations of different chosen levels of more than one independent variable (factor) on a given dependent variable.
• These can be two-way, three-way, or four-way classification experiments based on the number of factors applied.

Sample Size

• The sample size should be determined statistically before drawing a sample for the experiment.
• Samples should be sufficiently large to ensure that they are representative of the population and includes individuals or categories in the same proportions that they occur in the population.

Randomization

• Randomization should be ensured both in sampling and in treatment with the independent variable.

Minimizing Effects of Relevant Variables

• Many relevant variables not intended to be applied in an experiment may influence or affect the dependent variable or sway the experimental result.
• The effects of such variables need to be countered or minimized by proper experiment design and procedures.

Statistical Treatments

• Data from any experiment or investigation is subjected to statistical tests for analysis, interpretation, inference, and prediction.

Analysis of Variance (ANOVA) as an Extension of the t-test

• ANOVA is an extension and generalization of Student's t-test
• ANOVA is preferable and far more powerful, and can be applied to two or more groups simultaneously.
• ANOVA can estimate the strength of association between the dependent variable and the independent variable, and helps in minimizing experimental errors through rigorous experiment design.

ANOVA and Variance Components

• ANOVA tests the difference between the variances of two or more groups.
• For example, using a single independent variable, the one-way ANOVA analyzes different components of the total variance (s²t) of the sample.
• ANOVA estimates the relative magnitudes of the within-groups variance (s²w) due to uncontrolled random factors, and the between-groups variance (s²b) which may have been influenced by the applied independent variable.
• The aim is to determine if s²b can be explained away by the null hypothesis that it does not differ significantly from the s²w.
• The basic ANOVA statistic is the variance ratio or F ratio.

Classification of ANOVA

• ANOVA methods vary depending on the number of independent variables used.
• A single-classification or one-way ANOVA investigates the effects of a single independent variable on the dependent variable.
• The number of applied levels of the independent variable determines the number of groups in the experiment, where the size of each group equals the number of replications of the given level of the independent variable.
• For example, a one-way ANOVA may be applied to tracheal ventilation values (dependent variable) measured in three groups of 20 insects each, after their exposure to three doses of a pesticide (independent variable), to find if the pesticide changes the tracheal ventilation significantly.
• Higher orders of ANOVA, such as two-way and three-way ANOVAs, are applied in a factorial experiment where the simultaneous effects of more than one independent variable are being investigated.
• The number of groups used corresponds to the chosen number of combinations of different levels of the independent variables, each such combination being applied to one of the groups. The size of each group corresponds to the desired number of replications of each combination of the independent variables.

Models of ANOVA

• Different models of ANOVA have to be used according to the nature(s) of the independent variable(s) in the experiment.

Fixed Model or Model I ANOVA

• This is used to explore "fixed" or controlled treatment effects and analyzes the variances of a dependent variable in experiments using "fixed" experimental treatment(s) as independent variable(s).
• A model I ANOVA studies the effects of chosen and controlled levels of drugs, hormones, ions, radiations, experimental lesions or ablations of the brain, temperature, etc.
• Also, it can be used on physical properties, chemical constituents, structural components, activities, functional aspects and behaviors of organisms or the effects of practice, learning methods, etc., on performance.

Random Model or Model II ANOVA

• This explores the effects of chosen random factors on the dependent variable.
• The model analyzes the variances of a dependent variable in experiments where the groups have been exposed to independent variables such as sex, race, age, genotypes, habitats, and home environments.
• The independent variables are randomly changing classification variables, largely beyond the investigator's control.

Mixed Model or Model III ANOVA

• Unlike models I and II which may be either one-way or of higher orders, model III is always a two-way or a still higher order of ANOVA. Some independent variable(s) must be "fixed" experimental treatment(s), while the other(s) must be uncontrolled classification variable(s).

Assumptions of ANOVA

• It has four assumptions, those being Random Assignment, a Normal Distribution, Independence of Errors and Homoscedasticity.

Random Assignment

• The experimental design should provide for random sampling so that each individual of the population has an equal probability of being chosen for a group, and the choice of each individual is independent of the choice of others.
• Randomization of treatment should also be ensured for different levels of the independent variable(s), wherever possible.

Normal Distribution

• The dependent variable should have a normal distribution within the studied population
• It should be reasonable to assume that the error terms, i.e., the deviations of individual scores from the respective group means, are distributed normally.

Independence of Errors

• The error terms, i.e., the deviations of individual scores from the group mean, should be independent of each other.
• This is an alternative form of the assumption that the individual scores occur at random and independent of each other.

Homoscedasticity

• The assumption of homoscedasticity implies that the groups drawn for an experiment possess homogeneous variances initially.
• They should have been drawn from the same population (or closely similar populations) so that their initial variances may be considered as different estimates of the same population variance, differing only due to their sampling errors.
• It should thus be reasonable to assume that the error terms of individuals of different groups have homogeneous dispersions.

Additivity

• Different factors, including the independent variables used, produce separate bits of variations of the dependent variable, and these variations add up to give the total variation of the latter.
• This additive property of variations, due to different factors, enables the analysis of the total variance (s) of the dependent variable into its various components

One-Way Anova

• It investigates the effects of a single independent variable on the dependent variable and is undertaken to find whether or not the exposure of different groups of subjects or cases to different levels of a single independent variable has produced significant differences in the variance between the groups.
• One-way ANOVA may be either model I or model II according to the independent variable is a "fixed" experimental treatment or an uncontrolled classification variable.
• It cannot be a model III ANOVA, which is always a higher order of ANOVA with more than one independent variable.

Computation of The Variance Ratio

• One-way ANOVA consists of the computation and interpretation of the ANOVA statistic F, which is the variance ratio between the between-groups variance and the within-groups variance in this class of ANOVA.

One-Way ANOVA vs Repeated Measures One-Way ANOVA

• One-Way ANOVAs and Repeated-measures One-Way ANOVAs both rely on and use a singular categorical predictor variable as well as a singluar continuous response variable.
• One-Way ANOVAs feature subjects that will generally only appear in one group.
• Repeated Measures One Way ANOVAs feature subjects that may appear in each group.

Correlations in Statistics

• Correlation is a statistical measure that describes the strength and direction of the relationship between two or more variables.
• Correlations help to understand how changes in one variable might be associated with changes in another.
• For example, a psychologist might use correlations to find the connection between levels of stress and sleep quality to determine if higher stress leads to poorer sleep.

The Use of Correlations

• Correlations are widely used in psychological research:
• Identifying relationships between variables (e.g., intelligence and academic performance).
• Making predictions (e.g., predicting depression based on social support levels).
• Establishing associations in non-experimental studies where manipulation of variables is unethical or impractical.

Positive Correlations

• This is when both variables increase or decrease together
• One example of this is a correlation between self-esteem and life-satisfaction.

Negative Correlations

• This occurs when one variable increases, the other decreases.
• A good example of this is when stress levels correlated with lower memory retention.

Zero Correlation

• This is when no relationship exists between the two variables.
• For example, shoe size and intelligence have no discernible relation.

Measuring Correlation

• Correlation can be weak, moderate, or strong based on the correlation coefficient value.
• The strength and direction of a correlation is represented by the Pearson correlation coefficient (r), which ranges from -1 to +1.
• +1 = Perfect positive correlation
• -1 = Perfect negative correlation
• 0 = No correlation

Correlation vs Causations

• Correlation should not be confused with causality.
• Correlation does not mean Causation

Plotting Data Points

• Each pair of values (X, Y) is plotted as a point on a graph
• The x-axis is one variable and the y-axis represents the other.

Observing Patterns

• The overall pattern of the plotted points helps determine whether a correlation exists
• Visible trends (upward or downward) suggest a correlation while random scatter suggests little to no correlation.
• Points that slope upward from left to right will yield Positive Correlations (e.g., height vs. weight)
• Points that slope downward from left to right yield Negative Correlations (e.g., stress vs. productivity)
• Points that are widely scattered yield Zero Correlation.

Assessing the Strengh of Correlations

• Points that closely follow a straight line yield strong correlations
• Points that are widely scattered yield very weak correlations

Identifying Outliers

• Scatter plots detect outliers (points that can significantly deviate from the trend), which may ultimately affect the correlation calculations.

Advantages of Correlations

• Correlation will quantify the strenth and direction of the relationship between two variables.
• Correlation is simple and cheap, making it accessible for various studies.
• Correlation can provide insight into relationships before regression

Disadvantages of Correlations

• Correlation can be a misleading predictor of causation
• Correlation only picks up on linear relationships
• Correlation is disrupted greatly by outliers

What to Conclude from Correlation

• Just because there's a correlation, that doesn't mean there's a causation! The observed correlation is likely due to another relationship that's not being observed or considered
• Zero correlations do not necessarily mean that variables are not related at atll

Understanding Asymmetrical

• The regression will examine the nature of the relationship
• Asymmetrical = identifies dependent/independent variables.
• The Slope indicates change in one unit.

Understanding Regression

• Regression assesses the relationship between two or more correlated variables.
• The variable being predicted is the dependent variable.
• The variable used is for prediction is known as the independent variable/predictor.
• Regression is only valid for specific populations.
• Simple regression or simple linear regression models the relationship between 2 variables.
• Independent variables influence dependent variables

Difference Between Correlation and Regression

• Correlations measure the relationship between variables.
• Regressions are estimates based on what's being correlated
• Bivariates involve two variables with multivariate stats falling under several variables
• A correlation coefficient is relative. Regression coefficients are measuring change.