3

Chapter 3: A Brief Overview of the Classical Linear Regression Model

• Regression is a crucial econometric tool.
• Regression analysis describes and evaluates the relationship between a dependent variable and one or more independent variables.
• The dependent variable (y) is usually affected by a set of independent variables denoted by x.
• Alternative names for the variables (y and x) include dependent variable, regressand, effect variable, explained variable. Independent variables, regressors, causal variables, and explanatory variables.
• The number of independent variables can vary, in introductory cases there is only one independent variable.
• Regression differs from correlation as it treats the dependent and independent variables differently, The dependent variable (y) is assumed to be random (stochastic), while the independent variables (x's) are fixed in repeated samples.

Simple Regression

• Simple regression involves only one independent variable.
• Examples include the relationship between asset returns and market risk, the long-term relationship between stock prices and dividends and constructing optimal hedge ratios.

Simple Regression: Example

• This section provides an example of data on excess returns of a fund manager's portfolio (fund XXX) and a market index.
• The data is used to illustrate the relationship between excess returns on the fund and excess returns on the market index.
• Data includes excess returns for each year, and provides scatter plots of the fund's monthly return versus the market's excess monthly return.

Finding a Line of Best Fit

• The general equation for a straight line (y = a + bx) is used to fit the data.
• The equation is, however, deterministic.
• Adding a random disturbance term (u) to the equation creates a stochastic model.
• Random errors in measurement are important, and represent random outside influences on the dependent variable.

Ordinary Least Squares (OLS)

• OLS is a common method for fitting a line to data.
• The method involves minimizing the total sum of squared deviations (residuals) between the actual data points and the fitted line.
• Using the notation:     - y_t : actual data point at time t     - ŷ_t : fitted value from the regression line at time t     - û_t : residual, y_t - ŷ_t

Determining the Regression Coefficients

• Coefficients (a and β) are chosen to minimize the vertical distances from data points to the fitted line.
• This is achieved by minimizing the residual sum of squares.

Deriving the OLS Estimator

• The OLS derivation involves differentiating the sum of squared errors with respect to a and β to find the values that minimize the sum of squared errors.
• The derivation process results in formulas for calculating the coefficients a and β.

Estimator or Estimate?

• Estimators are formulas used to calculate coefficients, while estimates are the numerical values derived from these formulas.

The Assumptions Underlying The Classical Linear Regression Model (CLRM)

• The CLRM assumptions specify how the unobservable error terms (u) are generated.     - E(u_t) = 0 : Errors have zero mean     - Var(u_t) = σ² : Constant variance of errors     - Cov(u_i, u_j) = 0 : Errors are uncorrelated     - Cov(u_t, x_t)=0 : No relationship between the error and the corresponding independent variable.
• Additional assumption: u_t is normally distributed.

Properties of the OLS Estimator

• Under the specified assumptions, OLS estimators (â and β) are best linear unbiased estimators (BLUE).     - "Best" : minimizing variance among the class of linear unbiased estimators.     - "Linear" : a linear function of the dependent variable.     - "Unbiased" : on average, the expected value of the estimator is equal to the true value of the parameter being estimated.

Consistency/Unbiasedness/Efficiency

• OLS estimators are consistent, meaning they converge to true values as the sample size increases.
• OLS estimators are unbiased, meaning that their expected values are equal to the true values.
• An estimator is efficient if it is unbiased and has the smallest variance compared to other unbiased estimators.

Precision and Standard Errors

• Regression estimates are sample-specific.
• Standard errors (SE) measure the reliability of the estimate.
• Standard errors are useful for assessing uncertainty around parameter estimates.

Estimating the Variance of the Disturbance Term

• The variance of the error term (u) is estimated using the average squared errors (residuals).
• An unbiased estimator of σ² is calculated as the sum of squared errors divided by the degrees of freedom.

Some Comments on the Standard Error Estimators

• The greater the variance of the errors, the larger the standard errors.
• A larger sum of squared x values (Σ(x_t - x)²) results in smaller standard errors of the coefficients.

Linearity

• OLS requires linearity in parameters (not necessarily in variables).
• This means that coefficients are not multiplied, divided, squared, or cubed.
• Some models(e.g., exponential models) can be converted to linear models using substitutions/manipulations.

Linear and Non-linear Models

Estimator or Estimate?

• Estimators are formulas to determine coefficients, and estimates are the numerical results.

###Hypothesis Testing: Some Concepts

• Inferences about population parameters are made using hypotheses.
• This involves a null hypothesis (the statement being tested) and an alternative hypothesis.

Hypothesis Testing: The Test of Significance Approach

• The test statistic (e.g. t-ratio) compares the estimate with the null hypothesis value and calculates the significance of the test.
• A significance level (e.g., 5%) determines the rejection region.
• The critical value from t-tables establishes the cutoff for the decision to reject or fail to reject the null hypothesis.

The Confidence Interval Approach to Hypothesis Testing

• Confidence intervals provide a range of plausible values for a parameter.
• Given a confidence level, the calculated interval will likely contain the true value of the parameter.

The Probability Distribution of the Least Squares Estimators

• Under CLRM, the parameter estimates (â, β) follow a normal distribution.
• Standard errors reflect the uncertainty associated with the parameter estimates.

Testing Hypotheses: The Test of Significance Approach (continued)

A Note on the t and the Normal Distribution

• The t-distribution is similar to the standard normal distribution but has additional degrees of freedom.
• The distribution has the same shape and center as normal.

Comparing the t and the Normal Distribution

• The t distribution approaches the standard normal distribution as the number of degrees of freedom grows larger (approaches infinity).

The Confidence Interval Approach to Hypothesis Testing (continued)

How to Carry out a Hypothesis Test Using Confidence Intervals

How to Carry out a Hypothesis Test Using Confidence Intervals (continued)

Confidence Intervals Versus Tests of Significance

Constructing Tests of Significance and Confidence Intervals: An Example

Determining the Rejection Region

Performing the Test

Testing Other Hypotheses

Changing the Size of the Test

Changing the Size of the Test: The Conclusion

Some More Terminology

The Errors That We Can Make Using Hypothesis Tests

The Trade-off Between Type I and Type II Errors

A Special Type of Hypothesis Test: The t-ratio

The t-ratio: An Example

What Does the t-ratio Tell Us?

An Example of the Use of a Simple t-test to Test a Theory in Finance

Frequency Distribution of t-ratios of Mutual Fund Alphas

Can UK Unit Trust Managers "Beat the Market"?

The Overreaction Hypothesis and the UK Stock Market

The Overreaction Hypothesis and the UK Stock Market (continued)

Methodology

Portfolio Formation

Portfolio Formation and Portfolio Tracking Periods

Constructing Winner and Loser Returns

Allowing for Differences in the Riskiness of the Winner and Loser Portfolios

Is there an Overreaction Effect in the UK Stock Market? Results

Testing for Seasonal Effects in Overreactions

Conclusions