Regression Module 1: Variance and Coefficients

10 Questions

What is the assumption of regression that states that the variance of the residuals should be equal across all levels of the predictor variable?

Homogenous

What measure of influence is used to detect unusual values on the independent and dependent variables?

Cooks Distance

What is the purpose of using the t-distribution in regression analysis?

To account for increased error at the top and bottom of the regression line

What does the standardized regression coefficient for each predictor variable in multiple regression analysis provide?

The change in the dependent variable for a one-unit change in the independent variable while controlling for other predictors

Why might there be a significant association in a Pearson's correlation, but not in multiple regression analysis?

Because multiple regression analysis controls for the variance shared with other predictors, while Pearson's correlation does not

What is the purpose of maximizing SS Regression in a regression analysis?

To explain significant variation in the outcome variable.

What does a R2 value of 0 indicate in a regression model?

The independent variables do not explain any variation in the dependent variable.

What is the difference between unstandardized coefficients and standardized coefficients (Beta) in a regression analysis?

Unstandardized coefficients explain what each independent variable uniquely adds, while standardized coefficients (Beta) allow for comparison of different predictors by standardizing to a standard deviation.

What is the purpose of semi-partial correlation (Part) in a regression analysis?

To measure the unique contribution of a specific independent variable to the dependent variable after accounting for the effects of other variables in the model.

What is the purpose of visual inspection of residual plots in regression diagnostics?

To identify unusual scores or outliers that may have a large influence on the solution.

Study Notes

Variance

Maximise Sum of Squares (SS) Regression: goal of regression analysis
Minimise SS Error: residual variation not explained by the model

Regression Coefficient R2

Represents the proportion of variance in the dependent variable explained by independent variables
Ranges from 0 (no explanation) to 1 (full explanation)

Statistical Control

Shared variance: not included in the coefficients table
Unstandardised Coefficients: explain unique contribution of each independent variable
Standardized coefficients (Beta): allow comparison of different predictors
Semi-partial correlation (Part): measures unique contribution of a specific independent variable

Identifying Unusual Scores

Regression diagnostics: identify unusual scores influencing the solution
Outlier score: unusual on IV, studentised residual, and visual identification
Cut-offs for identifying outliers: 2.58 (0.01) and 3.29 (0.001)

Measures of Influence

Cook's Distance: measures influence of a data point on the regression slope
Leverage: uses distance to measure influence
Mahalanobis Distance: uses chi-square to measure influence

Assumptions of Regression

Linearity: relationship between predictors and outcome
Normally Distributed: residuals should be normally distributed
Homogenous: equal variances across all levels of predictors
Independence of observations: cannot be statistically tested

Additional Notes

Partial correlation: measures relationship between two variables controlling for other variables
Regression equation: predicts outcome variable based on predictor variables