Simple Linear Regression

• In simple linear regression, we study the relationship between two variables: the dependent variable (y) and the independent variable (x).

• The goal of regression analysis is to develop a model that helps us understand if two variables are related and to make predictions.

• In simple linear regression, we define a dependent variable (y) and an independent variable (x), where y is the variable we're trying to predict, and x is the variable we're using to predict y.

• The simple linear regression model is defined as y = Beta0 + Beta1*x + Epsilon, where Beta0 is the y-intercept, Beta1 is the slope, and Epsilon is the error term.

• The slope (Beta1) tells us whether the line is increasing or decreasing and how steep it is.

• A positive slope indicates a positive relationship between x and y, while a negative slope indicates a negative relationship.

• A flat line indicates no relationship between x and y.

• Beta0 (y-intercept) is the value of y when x is zero.

• In a sample, we use B0 and B1 to estimate the true population parameters of Beta0 and Beta1.

• The estimated simple linear regression equation is y-hat = B0 + B1*x.

• The goal is to find the line that fits the data the best, which is the line that minimizes the distance of each data point from the line.

• To find the best-fitting line, we use the least squares method, which minimizes the sum of the squared differences between each observed y value and its predicted value.

• The coefficient of determination (R-squared) measures how well the regression line fits the data.

• R-squared is calculated as SSR/SST, where SSR is the sum of the squares due to regression and SST is the total sum of squares.

• A good fit is indicated by a high R-squared value, which means the regression line explains a large proportion of the variation in y.

• In the example given, the data is a scatter plot of the number of hours studied (x) and the grade on an exam (y).

• The estimated regression equation is y-hat = 55.4 + 4.74*x.

• Using this equation, we can predict the grade on an exam for a given number of hours studied.

• For example, if a student studies for 3 hours, we can predict their grade to be approximately 69.268.- In simple linear regression, the total sum of squares (SST) is equal to the sum of the squared errors (SSE) and the sum of the squared regression (SSR).

• SST is the sum of the squared differences between each actual observation (Yi) and the average (Y Bar).

• SSE is the sum of the squared differences between each actual observation (Yi) and the predicted value (y hat).

• SSR is the sum of the squared differences between the predicted value (y hat) and the average (Y Bar).

• The coefficient of determination (R²) is calculated by dividing SSR by SST.

• R² measures the percentage of variability in the dependent variable (Y) that can be explained by the independent variable (X).

• In the example, R² is 0.9505, which means that 95.05% of the variability in grades can be explained by the number of hours studied.

• The correlation coefficient (R) is calculated by taking the square root of R² and using the sign of the slope.

• R measures the strength of association between X and Y, with values ranging from -1 to 1.

• In the example, R is 0.9749, which indicates a strong positive linear relationship between the number of hours studied and grades.

• The regression line is calculated using the least squares method, with a slope of 4.74 and a y-intercept of 55.0.

• The line of regression is a good fit to the data, with some data points exactly on the line and others above or below it.

• The average grade (Y Bar) is 77.8, which is calculated by summing up all 10 grades and dividing by 10.

• The Y Bar line is plotted on the scatter diagram, with the 10 data points closer to the y hat line than the Y Bar line.

• SST measures how well the observations cluster around the Y Bar line.

• SSR measures the explained variation, which is the deviation between the predicted value of Y and the average value of Y.

• SSE measures the unexplained variation, which is the deviation between the actual value of Y and the predicted value of Y.

• The significance of the relationship can be tested using a hypothesis test, with a null hypothesis that the slope is equal to zero and an alternative hypothesis that the slope is not equal to zero.

• The test statistic is calculated as B1 over sb1, where sb1 is the standard error for the slope.

• The standard error for the slope (sb1) is calculated using the formula sb1 = s / sqrt(Σ(x - xbar)^2).

• The test statistic is 12.39, which falls in the rejection region, indicating that there is a significant relationship between grades and number of hours studied.

• The P-value approach can also be used to test the significance of the relationship, with a P-value less than 0.01 indicating that there is a significant relationship between grades and number of hours studied.