Correlation and Regression Overview

Correlation and Regression

• Inferential statistics allow predictions based on sample data.
• Key goals of correlation and regression are to determine relationships between variables and make predictions from data trends.
• Real-world examples include the relationship between study hours and exam scores, and predicting sales based on advertising spend.

Correlation

• Correlation measures the strength and direction of a relationship between two variables.
• Correlation is used in bivariate analysis.
• Types of Correlation:
  • Positive Correlation: One variable increases, the other increases (e.g., height and weight).
  • Negative Correlation: One variable increases, the other decreases (e.g., exercise and weight loss).
  • No Correlation: No consistent relationship exists (e.g., shoe size and test scores).
• Correlation does not imply causation.

How to Measure Correlation

• Pearson Correlation Coefficient (r): measures the strength and direction of linear relationships between continuous variables.
• Sign of r denotes the association's nature.
• Value of r denotes the strength of the association.
• Range: -1 to +1
  • +1: Perfect positive correlation
  • -1: Perfect negative correlation
  • 0: No correlation

Regression

• Correlation shows a relationship, but regression quantifies it and allows predictions.
• Regression tells us how to draw the straight line described by the correlation.
• The regression technique predicts some variables by knowing others.
• It derives a mathematical equation to predict one parameter knowing the value of the other parameter.
• Example: Knowing the relationship between study hours and grades, regression predicts grades based on study hours.

Regression Analysis

• The process of predicting a dependent variable (Y) using an independent variable (X).
• To understand regression analysis, understand:
  • Dependent Variable: The main factor to understand or predict.
  • Independent Variables: Factors hypothesized to impact the dependent variable.

Regression Analysis: Drawing a line

• Draw a line through the middle of all data points on a chart
• This is known as the regression line
• The regression line represents the relationship between the independent variable and the dependent variable

Best-Fit Line

• Linear regression aims to fit a straight line, ŷ = ax+b, to data that gives the best prediction of y for any value of x.
• The line minimizes the distance (residuals) between data and the fitted line.

Types of Regression: Simple Linear Regression

• One independent variable predicts a dependent variable (continuous).
• Equation: y=b0+b1x+e
  • y: Dependent variable
  • x: Independent variable
  • b0: Intercept
  • b1: Slope (change in y for a one-unit change in x)
  • e: Error term

Types of Regression: Multiple Linear Regression

• Multiple regression determines the effect of multiple independent variables on a single dependent variable.
• The different independent variables are combined linearly, each with its own regression coefficient.

Interpreting Regression Output: R-squared (R²)

• R² tells us how well the independent variable(s) explain variation in the dependent variable.
• Range: 0 to 1
  • Closer to 1: The model explains most of the variation.
  • Closer to 0: The model explains very little variation.

Interpreting Regression Output: Coefficients

• Coefficients of independent variables indicate how much the dependent variable changes for each one-unit increase in the independent variable.
• Positive coefficient: Y increases as X increases
• Negative coefficient: Y decreases as X increases

This quiz covers the fundamental concepts of correlation and regression in inferential statistics. It explores the relationships between variables, how to measure correlation, and provides real-world examples for better understanding. Test your knowledge of the key principles that aid in making predictions from data trends.