Pitfalls in Regression Analysis

Study Notes

Occurs when two or more independent variables are highly correlated, making it difficult to separate their individual effects on the dependent variable.
Identification methods:
- Calculate Variance Inflation Factor (VIF) for each independent variable; VIF above 10 suggests significant multicollinearity.
- Check pairwise correlation coefficients between independent variables; correlations above 0.8 indicate potential multicollinearity.
Remedies:
- Remove or combine highly correlated variables.
- Apply Principal Component Analysis (PCA) to reduce dimensionality while retaining variance.

Occurs when the variance of residuals is not constant across all levels of the independent variables.
Identification methods:
- Plot residuals against predicted values or independent variables; funnel shape or systematic pattern suggests heteroscedasticity.
- Perform Breusch-Pagan test; significant p-value indicates heteroscedasticity.
Remedies:
- Use heteroscedasticity-robust standard errors to correct for non-constant variance.
- Apply transformations like logarithms to stabilize variance of residuals.
- Apply Weighted Least Squares (WLS) to give different weights to observations, stabilizing variance.

Occurs when residuals are correlated across observations, typically in time series data.
Identification methods:
- Calculate Durbin-Watson statistic; values significantly different from 2 suggest autocorrelation.
- Plot residuals over time; patterns indicate autocorrelation.
Remedies:
- Adjust standard errors using Newey-West standard errors to account for autocorrelation.
- Include lagged dependent or independent variables in the model.

Occurs when a relevant variable is left out of the model, leading to biased and inconsistent coefficient estimates.
Identification methods:
- Use theory and domain knowledge to identify potentially omitted variables.
- Perform specification tests like the Ramsey RESET test to detect omitted variable bias.
Remedies:
- Add the omitted variables to the model if data is available.

Occurs when an independent variable is correlated with the error term, often due to omitted variable bias, measurement error, or simultaneity.
Identification method:
- Compare OLS and IV estimates using the Hausman test; significant differences indicate endogeneity.
Remedies:
- Use Instrumental Variables (IV) regression with instruments that are correlated with the endogenous variable but uncorrelated with the error term.

Occurs when the functional form of the model is incorrect (e.g., omitting quadratic terms when the relationship is non-linear).
Identification methods:
- Check for systematic patterns in residuals.
- Perform goodness-of-fit tests like the Ramsey RESET test to identify misspecification.