Business Analytics: Regression Models Overview

The Story So Far

• Supervised learning and unsupervised learning are the two main types of machine learning
• Classification and clustering are both discrete types of supervised learning
• Regression is a continuous type of supervised learning
• Point models and linear regression are two examples of regression models
• Dimensionality reduction and regression are both components of unsupervised learning

Today’s Learning Objectives

• The two fundamental analytics models covered: Multiple Linear Regression (MLR) and Multiple Logistic Regression.
• Understanding the equations behind both models.
• Comprehending the differences between regression and classification techniques and what a hyperplane is.
• Understanding the concept of “Parametric Modelling” and its importance in business analytics.
• Recognizing why “optimization” is crucial to successful business analytics.

Experian

• A leading user of modern business analytics
• They use analytics for internal processes and externally by selling analytics services and products in 80 countries.
• They have 18,000 employees in 39 countries with a revenue of approximately £4.7 billion.
• Experian provides examples of specific projects in industries such as credit risk, market intelligence, consumer insight, and vehicle insight.

Experian Projects

• Opticians Store Locations: This project aims to predict the likely number of customers to a specific store location.
• Credit Rating Prediction: This project aims to predict total losses to the business if customers default.
• Northern Power Grid: This aims to predict the vulnerability of customers in certain locations to power outages.

Customer “Numbers” Modelling:

• Experian uses more than one input variable to make predictions.
• When your linear regression problem has more than one input variable and one output, you're doing multiple linear regression.

Customer “Conversion” Modelling

• Experian uses Logistic Regression to predict whether potential customers will convert based on direct mail campaigns.

Multiple Linear Regression (MLR)

• MLR is used when predicting a continuous dependent variable (output) from more than one independent variable (input).
• In MLR, the relationship between variables is visualized with the help of planes instead of simple straight lines.
• MLR models are trained by tuning parameters so the model's predictions minimize error when tested on historical data.

“Parametric” Models

• Parametric models use equations with parameters that must be trained.
• Optimization uses an "optimizer" or "solver" to find the best parameters for the model.

Optimizing Parameters

• To find the best parameters, an objective function (generally an equation that represents error) is defined.
• The optimizer/solver receives instructions on which parameters to adjust.
• The optimization process continues until it finds a parameterization with minimal identified error or until time or patience runs out.

Optimizing Parameters in “MLR”

• The objective function for MLR is the total error the model produces when tested on historical data.
• This objective function traditionally minimizes the sum of squared errors (residuals) or the difference between the model's prediction and the actual value for each data point.

Testing an Optimized Model

• To determine the trustworthiness of the model, formal testing procedures are used.
• Various parametric tests assign a p-value to each coefficient in a regression, which indicates significance and reliability.
• The model that makes the best predictions is usually chosen.

Classification Problems

• Classification problems involve predicting a class or category when a continuous variable isn't being predicted.
• Examples of classification problems include predicting purchase/no-purchase, defaulter/non-defaulter, churner/non-churner, and converter/non-converter.
• Linear regression isn't suitable for classification problems as it's designed for predicting continuous variables.

Classification Business Problems

• Classifiers are more common than regressors in business analytics.
• Examples of classification problems in various fields include: malignant/benign in public analytics, breach/non-breach in consumer analytics, upturn/downturn in financial analytics, win/loss/draw in sports analytics, outbreak/no outbreak in public health analytics, lapser/non-lapser in customer retention, give credit/reject in credit scoring, and buy/sell in stock trading.

Linear Regression and Classification Problems

• Linear regression isn't suitable for classification problems because it cannot give accurate classifications when predicting categorical variables.

The Solution: Logistic Regression

• Logistic Regression solves classification problems by using a different line for the model and a probabilistic space.