Regression Overview and Excel Techniques

Regression Overview

• Regression is a statistical technique to predict the relationship between several independent variables and one dependent variable.
• It's a supervised learning technique.
• The best-fit curve can be linear (straight line) or non-linear.
• Fit quality is measured by the correlation coefficient (r).
• R² represents the variance explained by the curve, and r is the square root of the explained variance.

Learning Objectives

• Understand the concept of regression.
• Learn how to perform regression in Excel.
• Understand how to improve regression model prediction.
• Understand logistic regression.
• Note the advantages and disadvantages of regression.
• Complete a hands-on Excel regression exercise.

What is Regression?

• A well-known statistical method for predicting relationships between multiple independent variables and one dependent variable.
• A supervised learning technique used to find the best-fit curve for a dependent variable in a multi-dimensional space.

How to Perform Regression (Steps)

• List all available variables for the model.
• Identify the dependent variable (DV) of interest.
• Visually examine relationships between variables of interest.
• Determine how to predict the DV using other variables.

Case Study: Data-Driven Prediction

• Nate Silver is a political forecaster leveraging big data and analytics.
• He successfully predicted the 2012 presidential election outcome in all 50 states, including swing states.
• He also correctly predicted the outcome of 31 of 33 Senate races.
• Political elections forecasting is now considered a scientific discipline.
• This involves developing hypotheses, gathering data, analyzing it, and using sophisticated models/algorithms.

Correlations and Relationships

• Categorize variables based on relationships and independence.
• Correlation measures the strength of a relationship.
• Correlation ranges from 0 to 1, with 1 indicating a perfect relationship.
• A correlation of 0 implies no relationship.
• Relationships can be positive, negative (inverse).
• The correlation coefficient (r) ranges from -1 to +1, with 0 representing no relationship.

Visual Look at Relationships (Scatter Plots)

• A scatter plot visually displays the relationship between two variables.
• It plots all data points on a two-dimensional graph.

Regression Exercise (Regression Equation)

• A regression model is generally a linear equation.
• The equation represents y = β0 + β1x + ε
• y is the dependent variable to predict.
• x is the independent/predictor variable.
• There could be multiple predictor variables (x1, x2, etc.) in a model.
• A model can only have one dependent variable (y).

House Data (Example)

• Example of using regression to predict house price based on size.
• Plotted data demonstrates a positive correlation between price and size (sqft).
• The relationship might not be perfect.
• Further details need to analyze the data.

Correlation and Regression (House Data Example)

• Coefficient of correlation is 0.891.
• R² = 0.794; variance in house prices explained by the size.
• Regression equation: House Price ($) = 139.48 * Size(sqft) – 54191

House Data (Correlation and Regression) (More Variables)

• House price strongly correlates with both size and number of rooms (#Rooms).
• Including rooms in the model strengthens it.
• The correlation coefficient for three variables is 0.984, explaining 97% of the total variance.

Predict the House Price (Example)

• For a house of 2000 sq ft and 3 rooms, predicted price is $214,963.

Non-linear Regression Exercise

• Relationships may be curvilinear; not all relationships are linear.
• Example: Electricity consumption (kWh) varies with temperature (temp).
• Visual inspection may reveal a curvilinear relationship.
• Non-linear regression model considers polynomial terms (e.g. Temp², etc.).
• R² value of the model will change after accounting for higher terms.

Predict Energy Consumption (Example)

• Example of a non-linear regression model: Energy Consumption = 15.87 * Temp² - 1911 * Temp + 67245
• Predict energy consumption for a specific temperature.

Logistic Regression

• Regression models often predict continuous values.
• Logistic regression can predict binary outcomes (yes/no).
• Logistic regression models measure relationships between categorical dependent variables and one or more independent variables.
• Example: Predicting if a patient has a disease based on characteristics like age, gender, etc.

Logistic Regression (Details)

• Logistic regression uses probability scores as predictions.
• It transforms the dependent variable (odds of being a 'case') into a continuous value (logit).

Advantages of Regression Models

• Easy to understand, built on basic statistical principles.
• Simple algebraic equations for easy comprehension and use.
• Goodness of fit measured by correlation coefficients and related statistics.
• Competitive predictive power compared to other methods.
• Includes all relevant variables for better model accuracy.

Disadvantages of Regression Models

• Prone to poor data quality (missing values, non-normal distributions).
• Collinearity issues (strong correlations among independent variables).
• Can be unreliable with many variables.
• Does not automatically handle non-linear relationships.
• Works only with numeric data; categorical data may need transformations.

Which Technique to Use?

• Choose regression for continuous target variables.
• Use classification for discrete/categorical target variables (options).

In Class Exercise (Example)

• Create a regression model to predict Test 2 score based on Test 1 scores.
• Predict the Test 2 score for someone who scored 46 in Test 1.
• Identify the dependent (Test 2) and independent (Test 1) variables.

This quiz explores the fundamentals of regression, a powerful statistical method for predicting relationships between variables. Participants will learn about both linear and non-linear regression, as well as how to implement regression techniques using Excel. Additionally, the quiz covers logistic regression and critically examines its advantages and disadvantages.