5

Classical Linear Regression Model Assumptions and Diagnostics

• The Classical Linear Regression Model (CLRM) assumes specific characteristics of the disturbance terms.
• These assumptions include:
  • E(u) = 0: The expected value of the disturbance term is zero.
  • Var(u₁) = σ² < ∞: The variance of the disturbance term is constant and finite.
  • Cov (u₁, u₁) = 0, i ≠ j: Disturbance terms are uncorrelated with each other.
  • The X matrix is non-stochastic or fixed in repeated samples: The independent variables are not random.
  • u₁ ~ N(0, σ²): The disturbance terms follow a normal distribution with mean zero and constant variance.

Investigating Violations of CLRM Assumptions

• Diagnostic tests are used to check if the CLRM assumptions are violated.
• Potential violations can result in:
  • Inaccurate coefficient estimates.
  • Inaccurate standard errors.
  • Incorrect distribution of test statistics.

Statistical Distributions for Diagnostic Tests

• F-tests and χ²-tests (sometimes called LM tests) are employed for diagnostics.
• F-tests compare restricted and unrestricted regression models.
• χ²-tests, often called LM tests, have a single degree of freedom parameter.

Assumption 1: E(u) = 0

• The mean of the disturbance terms is zero.
• The mean of the residuals will always be zero if the model has an intercept (constant term).

Assumption 2: Var(u₁) = σ² < ∞

• The variance of the errors (disturbances) is constant, a condition known as homoscedasticity.
• If errors do not have a constant variance, they are heteroscedastic.
• Heteroscedasticity implies that the variance of residuals varies across observations.

Detection of Heteroscedasticity: The GQ Test

• The Goldfeld-Quandt (GQ) test detects heteroscedasticity by splitting the data into two sub-samples.
• The null hypothesis is that the variances of the disturbances are equal across sub-samples.
• The test statistic, GQ, is the ratio of the larger residual variance to the smaller.

Detection of Heteroscedasticity: The White Test

• White's test is a general test for heteroscedasticity.
• This test runs auxiliary regressions on the squared and cross-products of the explanatory variables.
• The test statistic is related to the R² of the auxiliary regression.

Consequences of Using OLS in the Presence of Heteroscedasticity

• OLS estimation still provides unbiased coefficient estimates but they are not efficient (BLUE property).
• Using OLS in the presence of heteroscedasticity might yield inaccurate standard errors which could lead to incorrect inferences.

How do we Deal with Heteroscedasticity?

• If the cause of heteroscedasticity is known—e.g. the variance of the errors is related to another variable—then Generalized Least Squares (GLS) can be used.
• Other solutions include transforming variables (e.g. logs) or adopting White's corrected standard errors

Autocorrelation

• Autocorrelation occurs when disturbance terms in a regression model are correlated across observations.
• Autocorrelation can create bias in model estimates and make standard errors inaccurate.

Detection of Autocorrelation: Durbin-Watson Test

• The Durbin-Watson (DW) test assesses first-order autocorrelation.
• The null hypothesis of the test is no autocorrelation(p = 0).
• The DW statistic is calculated from the residuals.
  • Values close to 2 suggest little autocorrelation.
  • Values away from 2 may indicate autocorrelation.

Another Test for Autocorrelation: Breusch-Godfrey Test

• The Breusch-Godfrey test can detect multiple orders of autocorrelation.
• The model includes lagged residuals as regressors to capture autocorrelation patterns.
• The test statistic follows a χ² distribution.

Consequences of Ignoring Autocorrelation

• Coefficient estimates derived using Ordinary Least Squares (OLS) can be inefficient (not Best Linear Unbiased Estimates) even with a large number of sample sizes.
• R² values may be inflated if there is positive autocorrelation in the residuals.

"Remedies" for Autocorrelation

• If the form of autocorrelation is known, Generalized Least Squares (GLS) procedures such as Cochrane-Orcutt can be used.

Dynamic Models

• Static models assume the dependent variable depends only on current independent variables, while dynamic models consider the influence of previous values of both the dependent and independent variables.

Why Include Lags

• Inertia, over-reactions, or overlapping moving averages are common reasons for including lags in macro-economic variables.
• Lagged variables may indicate a significant relationship or omitted variables.

Models in First Difference

• Transitioning to a first-difference model involves subtracting the previous period's values from the current values of variables.

The Long Run Static Equilibrium Solution

• The long-run equilibrium value(s) of dependent variable is derived by simplifying the relationship.

Problems with Adding Lagged Regressors

• Introducing lagged dependent variables might violate the assumption of exogeneity (non-stochasticity) of the regressors.

Multicollinearity

• Multicollinearity occurs when there is a high linear correlation among the predictor variables in a multiple regression model.

Measuring Multicollinearity

• Correlation matrix examination to determine how strongly the independent variables are correlated.

Solutions to the Problem of Multicollinearity

• Traditional approaches (ridge regression, principal components) may create additional problems in the solution.
• Strategies such as dropping one of the correlated variables, combining variables into a ratio, or increasing the sample size may solve multicollinearity problems.

Adopting the Wrong Functional Form

• The RESET test diagnoses misspecification problems in the functional form of a regression model.
• For example, adding polynomial terms of the fitted values to an auxiliary regression can help detect a mis-specified model.
• Transforming variables into logs is one way to linearise non-linear structures.

Testing the Normality Assumption

• The normality assumption is crucial for hypothesis testing's accuracy.
• Bera and Jarque normality tests for errors (disturbances) look at skewness and kurtosis.
• These tests look at whether the coefficients of skewness and kurtosis of the disturbance terms are jointly zero.
• If the model violations in normality exist, dummy variables (e.g., for periods of particular economic factors) can be used to correct the misspecification.

What to do if Non-Normality?

• Alternatives exist if normality testing for skewness and kurtosis produces non-zero coefficients.
• Dummy variables may address particular outliers to correct the non-normal issue.

Omission of an Important Variable or Inclusion of an Irrelevant Variable

• Omitting a relevant, correlated variable biases the estimates of other coefficients and the intercept.
• Including irrelevant variables increases estimation complexity without improving accuracy.

Parameter Stability Tests

• Parameter stability tests are used to identify whether the regression parameters (coefficients) remain constant throughout the sample period.

The Chow Test

• The Chow test (analysis of variance test) divides the data into two sub-periods to compare the restricted and unrestricted models.
• The test statistic follows an F-distribution and uses the ratio of the restricted (unrestricted) RSS from the models.
• Test to see if the parameter values are stable between two periods using the estimated F-distribution statistical.

The Predictive Failure Test

• The predictive failure test evaluates predictive performance.
• The test statistic is obtained from RSS (restricted) and RSS1 (from a sub-sample).
• The statistic follows an F-distribution to determine predictive validity.

Backwards vs. Forwards Predictive Failure

• These models are used to determine the robustness of model coefficients depending if the model is estimated on a forward sub-sample or in backward direction.

Measurement Errors

• Measurement error in the independent variables, but not in the dependent variable, will cause biased, inconsistent coefficient estimates towards zero.
• Measurement error in the dependent variable does not cause the coefficient estimates to be inconsistent but will result in larger standard errors than expected in the model.

A Strategy for Building Econometric Models

• An econometric model tries to provide a reasonably accurate estimation about variables’ relationship using the given data in the sample period.
• The model should satisfy assumptions of the Classical Linear Regression Model (CLRM), be as parsimonious (simple) as possible, and be theoretically sound.

2 Approaches to Building Econometric Models

• The specific-to-general approach starts with a simple model and progressively adds complexity, often overlooking detailed diagnostic checks.
• The general-to-specific approach begins with a larger model to test the statistical and theoretical consistency of model coefficients.