Statistics: Normality and Transformations

Classical Linear Regression Model Assumptions and Diagnostics

• Classical linear regression models (CLRM) have several key assumptions about the error terms
• These assumptions are crucial for valid statistical inferences
• Violation of these assumptions can lead to biased and inconsistent estimates

CLRM Disturbance Term Assumptions

• Expected value of the error term is zero: E(ε_t) = 0
• Variance of the error term is constant (homoscedasticity): Var(ε_t) = σ²
• Covariance between any two error terms is zero: cov(ε_i, ε_j) = 0 for i ≠ j
• Error terms are independent of the explanatory variables (X)
• Error terms follow a normal distribution: ε_t ~ N(0, σ²)

Detecting Violations of CLRM Assumptions

• Methods to test for violations of CLRM assumptions are needed
• Graphs and formal tests are employed for diagnostics, such as the Goldfeld-Quandt and White's tests

Assumption 1: E(ε_t) = 0

• This assumption means the average value of the error term is zero
• Checking the mean of the residuals will give you a measure
• A constant term in your regression equation is necessary for this assumption to hold

Assumption 2: Var(ε_t) = σ²

• This assumption means the variance of the errors is constant (homoscedasticity)
• Heteroscedasticity means the variance of the errors changes over time
• The Goldfeld-Quandt test or White's test can check this assumption of heteroscedasticity

Detection of Heteroscedasticity: The GQ Test

• The GQ test splits the entire sample into two sub-samples to assess equality of variance in errors
• Calculate the residual variances for each sub-sample
• The ratio of the larger to smaller residual variance is the GQ test statistic

Detection of Heteroscedasticity: The White's Test

• White's test is a general approach to test for heteroscedasticity
• An auxiliary regression is required, incorporating terms for the explanatory variables, their squares, and cross products
• A high R² in this auxiliary regression suggests significant heteroscedasticity

Consequences of Heteroscedasticity

• OLS estimation still provides unbiased coefficient estimates but isn't the Best Linear Unbiased Estimator (BLUE) in the presence of heteroscedasticity
• Standard errors calculated using the usual formula are likely to be inappropriate, leading to incorrect inferences
• R-squared might be inflated due to existence of positively correlated residuals

Dealing with Heteroscedasticity

• If the cause of the heteroscedasticity is known, employ a generalized least squares (GLS) method
• Divide the regression equation by a variable related to variance to reduce heteroscedasticity

Autocorrelation

• CLRM assumes uncorrelated error terms
• Residuals in a model show patterns suggestive of autocorrelation if present

Detecting Autocorrelation: The Durbin-Watson Test

• The Durbin-Watson test examines first-order autocorrelation
• The test statistic, denoted as DW, measures autocorrelation
• It compares the DW statistic with critical values to determine if you reject the null hypothesis that the errors are uncorrelated (DW≈2)

Detecting Autocorrelation: The Breusch Godfrey Test

• A generalized test for autocorrelation that checks for the possibility that the error terms in a given regression equation are correlated over time (nth-order)
• This test determines whether the null hypothesis of no autocorrelation can be rejected

Consequences of Ignoring Autocorrelation

• Coefficient estimates remain unbiased under autocorrelation but become less efficient
• Standard errors are inappropriate, leading to incorrect inferences
• R² values are often inflated under autocorrelation

Remedial Measures for Autocorrelation

• Employ Generalized Least Squares (GLS) method
• Transform variables where data suggests a theoretical reason
• Redevelop or modify the regression model

Multicollinearity

• Multicollinearity occurs when explanatory variables are highly correlated with each other
• Perfect multicollinearity makes estimating all coefficients impossible

Measuring Multicollinearity

• Method 1: Examine the correlation matrix to understand the correlation between explanatory variables
• Method 2: Variance Inflation Factor (VIF) measures how much variance is inflated for each regressor

Solutions to Multicollinearity

• Traditional techniques like ridge regression or principal components aren't very effective in solving multicollinearity
• Consider dropping one or more collinear variables
• Transforming variables or collecting more data (better frequency)

Functional Form Misspecification

• Linear functional form is often assumed but may be incorrect
• Employ Ramsey's RESET test to detect misspecification of the functional form by adding higher-order terms of fitted values as regressors

Adopting the Wrong Functional Form

• If the RESET test indicates mis-specification, consider how the model could be improved
• Transformation of the data (e.g., using logarithms) can often resolve the non-linearity issues

Assessing Normality of Error Terms

• Testing for normality of errors is important to have reliable hypothesis tests
• Employ the Bera-Jarque test statistic

Handling Non-Normality

• Non-normality often stems from extreme residuals (outliers)
• Using dummy variables to address the influential extreme residuals can be effective

Omission of an Important Variable / Inclusion of an Irrelevant Variable

• Omitting key variables can bias the coefficients of variables that remain in the model
• Including unrelated variables doesn't impact bias but reduces efficiency.

Parameter Stability Tests

• Assumes the regression coefficients are constant throughout the sample duration
• Chow test used to check if these are equivalent across different sub-samples

This quiz covers essential concepts in statistics related to normality testing and transformations applicable to linearize multiplicative models. It includes questions on the Bera Jarque test, characteristics of statistical distributions, and remedies for non-normality in residuals. Enhance your understanding of skewness, kurtosis, and their implications in statistical analysis.