Python Logistic Regression Examples

Introduction

Logistic Regression is a popular machine learning algorithm in Python used for binary classification problems. This means it's used when we have two classes, such as healthy versus sick, pass or fail, etc. It can also be used for multi-class classification problems where there are more than two classes but those classes must be mutually exclusive. The logistic regression model is trained with labeled data - this means we already know which class each datapoint belongs to, so the model learns from these examples.

To understand how Logistic Regression works and why it's useful, let's dive into some Python code examples using popular machine learning libraries like scikit-learn.

Understanding Logistic Regression with Examples

Let's start by importing the necessary libraries:

import numpy as np
from sklearn.linear_model import LogisticRegression

Training a Logistic Regression Model

We can train a Logistic Regression model using the fit() method of the LogisticRegression class:

X = np.array([[1, 2, 3],
              [4, 5, 6],
              [7, 8, 9]])
y = np.array([0, 1, 0])

model = LogisticRegression()
model.fit(X, y)

In this example, X is a numpy array containing the features of our data. Each row represents an example, and each column represents a feature. y is the label for each example.

Predicting with a Trained Logistic Regression Model

Once we have a trained model, we can predict the class of new data using the predict() method:

new_data = np.array([[10, 11, 12]])

prediction = model.predict(new_data)

This will return the predicted class for the new data.

Feature Importance in Logistic Regression

Logistic Regression also provides a measure of feature importance through the coef_ attribute:

print(model.coef_)

This attribute returns the weight of each feature in the model.

Probabilities with Logistic Regression

Logistic Regression can also output the probability that the new data belongs to each class. This can be useful when we want to make a decision based on the probability rather than just the predicted class:

probability = model.predict_proba(new_data)

Logistic Regression vs. Decision Trees

Logistic Regression and Decision Trees are both used for classification problems, but they approach these problems differently. Logistic Regression is a linear model, which means it tries to find a line that best separates the classes. Decision Trees, on the other hand, create a tree-like model of decisions and their possible consequences. They are often used when the data is more complex and cannot be easily separated by a line.

Conclusion

Through these Python examples, we've explored the basics of logistic regression, including how to train a model, make predictions, understand feature importance, and even calculate probabilities. Logistic Regression is a versatile tool for classification problems, and understanding its mechanics can help you make informed decisions about your data.