Statistics "Fill in the blank spaces" quiz

Linear Regression Overview

• Linear regression is used to explain the relationship between a dependent variable and one or more independent variables.
• The dependent variable is a continuous variable, while the independent variables can be continuous, discrete, or indicator variables.
• The simple linear regression model has one independent variable, while the multiple linear regression model has two or more independent variables.
• The linear regression model describes how the dependent variable is related to the independent variables and the error term.
• The estimated regression equation shows how to calculate predicted values of the dependent variable using the values of the independent variables.
• The least squares method is used to calculate the coefficients so that the errors are as small as possible.
• The coefficient of determination (R-squared) provides a measure of the goodness of fit for the estimated regression equation.
• Adjusted R-squared corrects for the number of independent variables and is preferred to R-squared.
• The t-test is used to determine whether the relationship between the dependent variable and one independent variable is significant.
• The F-test is used to test whether the relationship between the dependent variable and all independent variables is significant.
• The ANOVA table shows the total variation, explained variation due to regression, and unexplained variation due to error.
• The goal is to find a regression model with coefficients that are jointly significant.

Linear Regression Overview

• Linear regression is used to explain the relationship between a dependent variable and one or more independent variables.
• The dependent variable is a continuous variable, while the independent variables can be continuous, discrete, or indicator variables.
• The simple linear regression model has one independent variable, while the multiple linear regression model has two or more independent variables.
• The linear regression model describes how the dependent variable is related to the independent variables and the error term.
• The estimated regression equation shows how to calculate predicted values of the dependent variable using the values of the independent variables.
• The least squares method is used to calculate the coefficients so that the errors are as small as possible.
• The coefficient of determination (R-squared) provides a measure of the goodness of fit for the estimated regression equation.
• Adjusted R-squared corrects for the number of independent variables and is preferred to R-squared.
• The t-test is used to determine whether the relationship between the dependent variable and one independent variable is significant.
• The F-test is used to test whether the relationship between the dependent variable and all independent variables is significant.
• The ANOVA table shows the total variation, explained variation due to regression, and unexplained variation due to error.
• The goal is to find a regression model with coefficients that are jointly significant.

OLS Asymptotics

• An estimator is consistent if it converges in probability to the population parameter as the sample size increases.
• The probability that the estimator obtained from a sample size will be arbitrarily close to the population parameter goes to 1 as the sample size increases.
• The OLS estimator is unbiased under assumptions 1-4 (with the zero conditional mean assumption).
• Under assumptions 1-4’ (with the assumption that the regressors are uncorrelated with the error term), the OLS estimator is consistent.
• Unbiasedness is ideal but if it cannot be achieved in a small sample, then consistency can be achieved with a large sample.
• Omitted variable bias occurs when a relevant variable is omitted from the model and the coefficient will be biased if the omitted variable and the included variable are correlated.
• The OLS estimator is consistent if the omitted variable is irrelevant or uncorrelated.
• Under assumptions 1-5 (Gauss Markov assumptions), the coefficients have asymptotically normal sampling distribution.
• In large samples, the normality assumption is not always needed for the OLS estimators to be normal and the t-tests and F-tests to be valid.
• OLS properties hold for any sample, including expected values and unbiasedness under assumptions 1-4 and variance formulas under assumptions 1-5.
• Gauss-Markov theorem (BLUE) holds under assumptions 1-5.
• As the sample size increases, standard errors change at a rate of 1/sample size and with larger sample size, standard errors are lower, leading to more significance of the coefficients.

Introduction to Linear Regression

• Linear regression models the relationship between a dependent variable and one or more independent variables.
• The regression model can be in linear or log-linear form, and taking logs of variables changes the interpretation of coefficients.
• Gauss Markov assumptions are standard assumptions for the linear regression model, including linearity in parameters, random sampling, no perfect collinearity, exogeneity, and homoscedasticity.
• Homoscedasticity means the variance of the error term is constant for each independent variable while heteroscedasticity means the variance differs.
• The unbiasedness of the OLS estimators is derived from Gauss Markov assumptions.
• The standard errors measure how precisely the regression coefficients are calculated, and lower variance in error term and higher variance in independent variable is desirable.
• The variance of the error term can be estimated, and the variances of the OLS estimators depend on it.
• The sample variability in OLS coefficients depends on the variances of the error term and independent variable.
• The coefficients are random as the sample is random, and the expected values of the sample coefficients are the population parameters.
• The relationship between y and x is linear in the population, but the regression model can have logged, squared, or interaction variables.
• Exogeneity or zero conditional mean implies the expected value of the error term given independent variable x is zero.
• Homoscedasticity is when the variance of the error term is constant for each independent variable.

Introduction to Linear Regression

• Linear regression models the relationship between a dependent variable and one or more independent variables.
• The regression model can be in linear or log-linear form, and taking logs of variables changes the interpretation of coefficients.
• Gauss Markov assumptions are standard assumptions for the linear regression model, including linearity in parameters, random sampling, no perfect collinearity, exogeneity, and homoscedasticity.
• Homoscedasticity means the variance of the error term is constant for each independent variable while heteroscedasticity means the variance differs.
• The unbiasedness of the OLS estimators is derived from Gauss Markov assumptions.
• The standard errors measure how precisely the regression coefficients are calculated, and lower variance in error term and higher variance in independent variable is desirable.
• The variance of the error term can be estimated, and the variances of the OLS estimators depend on it.
• The sample variability in OLS coefficients depends on the variances of the error term and independent variable.
• The coefficients are random as the sample is random, and the expected values of the sample coefficients are the population parameters.
• The relationship between y and x is linear in the population, but the regression model can have logged, squared, or interaction variables.
• Exogeneity or zero conditional mean implies the expected value of the error term given independent variable x is zero.
• Homoscedasticity is when the variance of the error term is constant for each independent variable.

Heteroscedasticity and its Consequences

• Heteroscedasticity refers to a scenario where the variance of the error term differs with the independent variables.

• Under heteroscedasticity, OLS estimators are still unbiased and consistent, but the variance formulas for the OLS estimators are not valid.

• The t-tests and F-tests are not valid under heteroscedasticity, and the OLS estimator is not the best linear unbiased estimator (BLUE).

• Hypothesis testing for heteroscedasticity involves testing whether the expected value of the error term varies with the independent variables.

• The Breusch-Pagan test, White test, and Alternative White test are commonly used tests for heteroscedasticity.

• Robust standard errors should be used when heteroscedasticity is found.

• Weighted Least Squares (WLS) can be used to estimate the model if the heteroskedasticity form is known.

• Feasible Generalized Least Squares (FGLS) transforms the variables to get homoscedasticity if the heteroscedasticity form is not known.

• The R-squared for the regressions of squared residuals on independent variables is used to calculate the test statistics for heteroscedasticity tests.

• The F-test and LM-test are commonly used tests for overall significance of heteroscedasticity.

• The regression model for price needs correction for heteroscedasticity.

• The R-squared for the regressions of squared residuals on independent variables is used to calculate the test statistics for heteroscedasticity tests for log price.Heteroscedasticity and Regression Models

• Heteroscedasticity is a condition where the variance of the errors is not constant across the range of values of the independent variable.

• Heteroscedasticity can lead to biased coefficients, incorrect variance for the coefficients, and invalid t-tests and F-tests.

• Three tests are used to identify heteroscedasticity: Breusch-Pagan test, White test, and Alternative White test.

• The Breusch-Pagan test and White test are based on regressing the squared residuals on the independent variables.

• The Alternative White test is based on regressing the squared residuals on the fitted values and squared fitted values.

• If the heteroscedasticity form is known, weighted least squares (WLS) can be used to correct for it.

• WLS assigns weights to each observation based on the inverse of the variance of the error term.

• If the heteroscedasticity form is unknown, feasible generalized least squares (FGLS) can be used.

• FGLS assigns weights to each observation based on the inverse of the estimated variance of the error term.

• The coefficients are the same for ordinary least squares (OLS) and OLS with robust standard errors, but the standard errors and significance can differ.

• The coefficients are different for OLS as compared to WLS and FGLS because of the use of weights.

• After correcting for heteroscedasticity, the results of the various regression models (OLS, OLS with robust standard errors, WLS, and FGLS) are similar, except for the loss of significance of one coefficient in some models.