Week 4 - Simple Regression Model (Econometrics)

Week 4: Simple Regression BT22203: Econometrics Model: PART I The classical linear regression model Assumption 1: Regression model is linear in parameter Assumption 2: The explanatory variable is uncorrelated with the disturbance term and non-stochastic Assumption 3: The mean value of the distu...

Week 4: Simple Regression BT22203: Econometrics Model: PART I The classical linear regression model Assumption 1: Regression model is linear in parameter Assumption 2: The explanatory variable is uncorrelated with the disturbance term and non-stochastic Assumption 3: The mean value of the disturbance term is zero The other factors or forces are not related to 𝑋𝑖 and therefore, given the value of 𝑋𝑖 , their mean value is zero. Assumption 4: The variance of each 𝑢𝑖 is constant or homoscedastic - The conditional distribution of each Y population corresponding to the given value of X has the same variance. - The individual Y values are spread around their mean values with the same variance Assumption 5: There is no correlation between two error terms - There is no systematic relationship between two error terms - Positive correlation: if one error term (𝑢) is above the mean value, another 𝑢 will also be above the mean value - Negative correlation: if one 𝑢 is below the mean value, another 𝑢 has to be above the mean value, or vice versa Assumption 6: The regression model is correctly specified. Alternatively, there is no specification bias or specification error in the model used in empirical analysis. VARIANCES AND STANDARD ERRORS OF OLS ESTIMATORS The sampling variability of these estimators show how they vary from sample to sample. These sampling variabilities are measured by the variances or standard errors (se) of the estimators The variances and standard errors of the OLS estimators given in Eqs. (2.16) and (2.17) are as follows:  The homoscedastic is estimated from the following formula: The standard error of the Y values or standard error of the regression (SER): SER is often used as a summary measure of the goodness of fit of the estimated regression line BT22203: Econometrics, Lecturer: Saizal Pinjaman WHY OLS? THE PROPERTIES OF OLS ESTIMATORS Gauss – Markov Theorem: Given the assumptions of the classical linear regression model, the OLS estimators have minimum variance in the class of linear estimators; that is, they are BLUE (best linear unbiased estimators) Property 1: 𝑏1 and 𝑏2 are linear estimators Property 2: They are unbiased; that is, 𝐸(𝑏1 ) = 𝐵1 and 𝐸(𝑏2 ) = 𝐵2. Property 3: that is, the OLS estimator of the error variance is unbiased. Property 4: 𝑏1 and 𝑏2 are efficient estimators; that is, var (𝑏1 ) and var (𝑏2 ) are less than the variance of any other linear unbiased estimator of 𝐵1 and 𝐵2.  The sampling or probability distributions of OLS estimators Assumption 7: The error terms 𝑢𝑖 follows the normal distribution with mean zero and variance 𝜎 2 that is: Central limit theorem: If there is a large number of independent and identically distributed random variables, then, with a few exceptions, the distribution of their sum tends to be a normal distribution as the number of such variables increases indefinitely. 𝑏1 and 𝑏2 that are linear functions of the normally distributed variable 𝑢𝑖 prove that 𝑏1 and 𝑏2 themselves are normally distributed. Simple Regression Model: PART II Hypothesis testing Suppose someone suggests that annual family income has no relationship to a student’s math S.A.T. score: H 0 : B2 = 0 It is deliberately chosen to find out whether Y is related to X at all Reject the zero-null hypothesis (H0) in favor of the alternative hypothesis (H1), which says, for example, that 𝐵2 ≠ 0 Our numerical results show that b2 = 0.0013. But we cannot look at the numerical results alone, for we know that because of sampling fluctuations, the numerical value will change from sample to sample. We can use either: The confidence interval approach or The test of significance approach (focus of this lecture) Test of significance approach to hypothesis testing The decision to accept or reject H0 is made on the basis of the value of the test statistic obtained from the sample data.  To use the t test in any concrete application, we need to know three things: 1. The d.f., which are always (n - 2) for the two-variable model 2. The level of significance (α) 1, 5, or 10 percent levels are usually used in empirical analysis. Instead of arbitrarily choosing the α value, you can find the p-value and reject the null hypothesis if the computed p-value is sufficiently low. 3. Whether we use a one-tailed or two-tailed test  A Two-Tailed Test: Assume that 𝐻0 : 𝐵2 = 0 and 𝐻0 : 𝐵2 ≠ 0. Using Eq. (3.29), we find that: Now from the t table, find that for 8 d.f., critical t values are as shown: The absolute value of t, exceeds the critical t value at the chosen level of significance, we can reject the null hypothesis. Based on p value, the t statistic of 5.4354 is less than 0.01 (specifically 0.0006). Thus, if we were to reject the null hypothesis that the true slope coefficient is zero at this p value, we would be wrong in six out of ten thousand occasions.  A One-Tailed Test: Assume that 𝐻0 : 𝐵2 ≤ 0 and 𝐻0 : 𝐵2 > 0 ; here the alternative hypothesis is one-sided. The t-testing procedure remains exactly the same as before. For 8 d.f., the critical t value (right-tailed) is: For the math S.A.T. example, we first compute the t value as if the null hypothesis were that 𝐵2 = 0. We have already seen that this t value is: The t value exceeds any of the critical values and reject the hypothesis that annual family income has no relationship to math S.A.T. scores. How good is the regression line: the coefficient of determination (𝑟 2 ) Coefficient of determination (𝑟 2 ) measures the goodness of fit of the model. It tells us how well the estimated regression line fits the actual Y values.  is the total variation of the actual Y values about their sample ത It is called Total Sum of Squares (RSS). mean 𝑌. is the the total variation of the estimated Y values about their mean value 𝑌ത෠ − 𝑌ത. It is called the explained sum of squares (ESS). is the residual sum of squares (RSS) or residual or unexplained variation of the Y values about the regression line. If the chosen SRF fits the data quite well, ESS should be much larger than RSS. By dividing Equation (3.36) by TSS on both sides  Now let us define: The r2 measures the proportion or percentage of the total variation in Y explained by the regression model. Two properties of r2 may be noted: It is a non-negative quantity Its limits are 0 ≤r2 ≤1  Formula to compute 𝑟 2 𝑟 2 for the Math S.A.T example: In our math S.A.T. ex- ample X, the income variable, explains about 79 percent of the variation in math S.A.T. scores. The Coefficient of Correlation, r the sample coefficient of correlation, r, as a measure of the strength of the linear relationship between two variables Y and X. The sign of r (+ or -) can be determined from the nature of the problem. Thus, for the math S.A.T. example: Normality tests Test 1: Histograms of residuals Simple graphical device of the probability density function (PDF) of a random variable The horizontal axis: the values of the variable of interest are divided into suitable intervals The vertical axis: rectangles equal in height to the number of observations Test 2: Normal probability plot (NPP) Horizontal axis (X-axis): values of the variable of interest (say, OLS residuals ei), Vertical axis (Y-axis): the expected values of this variable if its distribution were normal. If the variable is in fact from the normal population, the NPP will approximate a straight line. Test 3: Jarque-Bera test This test at first computes Coefficient of skewness (s) which measure the asymmetry of a PDF Kurtosis (K) which measure how tall or flat a PDF in relation to the normal distribution of a random variable A normal distribution has skewness of zero and kurtosis is 3 Jarque and Bera have developed the following statistic: Where n is the sample size, S represents skewness, and K represents kurtosis. Under the normality assumption, the JB statistic follows the chi-square distribution with 2 d.f. asymptotically or : If the computed chi-square value from Eq. (3.47) exceeds the critical chi-square value for 2 d.f. at the chosen level of significance, we reject the null hypothesis of normal distribution