Data Preprocessing for Machine Learning

Data Preprocessing

• Data preprocessing prepares raw data, rendering it suitable for machine learning.
• This is a crucial initial step in creating a machine learning model.
• Real-world data contains noises, missing values, and may be in an unusable format.

Steps for Data Preprocessing

• Acquire the dataset
• Import crucial libraries
• Import the dataset
• Identify and handle missing values
• Encode categorical data
• Split the dataset
• Feature scaling

Acquiring the Dataset

• This is the initial step in data preprocessing for machine learning.
• A dataset comprises data gathered from multiple and disparate sources, then combined into a proper format.
• Dataset formats differ according to use cases; business datasets differ from medical ones, containing different data.
• Data should be put in CSV, HTML, or XLSX file formats.

Importing Libraries

• Numpy is a Python package for scientific calculation
• It is used for mathematical operations.
• Numpy adds multidimensional arrays and matrices.
• Pandas is a Python library for data manipulation and analysis.
• Pandas is used for importing and managing data sets.
• Matplotlib is a Python 2D plotting library.
• Matplotlib is used to plot various types of charts.

Code Example

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

Sample Dataset Characteristics

• Dataset stored in "Data.csv" file.
• It includes 10 instances/examples.
• Three independent variables: 'Country', 'Age', and 'Salary'.
• One dependent variable: 'Purchased'.
• Two missing values, one each in 'Age' and 'Salary' independent variables.
• 'Country' is a categorical variable.

Importing the Dataset

• Save your Python file in the directory containing the data set.
• read_csv() function of the pandas library reads a CSV file.
• Separating independent and dependent variables are required for every Machine Learning model.
• Employ the iloc[] function from the Pandas library to extract independent variables.

Deleting Rows with NaN Values code

X = pd.DataFrame(X)
X = X.dropna()
X = np.array(X)

Identifying and Handling Missing Values

• Handle missing values correctly during data preprocessing.
• Failure to handle missing values leads to incorrect conclusions and inferences.
• Methods to handle missing data include:
  • Deleting a particular row
  • Imputing the data by:
    • Replacing with the mean
    • Replacing with the median
    • Replacing with the most frequently occurring value
    • Replacing with a constant value

Deleting a Particular Row

• Remove rows with null values or with >75% missing values in a particular column.
• Only recommended when the dataset has adequate samples.
• Ensure removal doesn't introduce bias.

Impute Data

• Imputation adds variance, but can negate losses efficiently.
• Imputation can yield better results.

Code - Replacing NAN (Most Frequent)

from sklearn.impute import SimpleImputer
imputer = SimpleImputer(missing_values=np.nan, strategy='most_frequent')
imputer.fit(X[:, 0:3])
X[:, 0:3] = imputer.transform(X[:, 0:3])

Code - Replacing NAN (Median/Mean)

from sklearn.impute import SimpleImputer
imputer = SimpleImputer(missing_values=np.nan, strategy-'median')
imputer.fit(X[:, 1:3])
X[:, 1:3] = imputer.transform(X[:, 1:3])

Encoding The Data

• Data types include continues, ordinal and nominal data.
• Associated encoding methods include binning, mapping and one hot encoding.

Encoding the Data

• Categorical data has specific categories in a dataset.
• Machine Learning models rely on mathematical equations.
• Use numbers in the equations.
• How to encode
  • Categorical data
    • Ordinal data mapping
    • One hot encoding (nominal data)
  • Continuous data
    • Binning
    • Normalization

Mapping (Ordinal data)

• Categorical columns include eye_color (Nominal), Satisfaction (Ordinal), and Upsell (Nominal).
• Satisfaction is ordinal.
• Order matters in the satisfaction column.

Mapping (Ordinal data) Code:

from sklearn.preprocessing import LabelEncoder
labelencoder = LabelEncoder()
dataset['satisfaction'] = labelencoder.fit_transform(dataset['satisfaction'])

One-Hot Encoding (Nominal Data)

• Nominal data lacks order.
• ML models may assume correlations when mapping nominal data as ordinal data, producing faulty output.
• Use Dummy Encoding to eliminate this issue.
• Dummy variables use 0 or 1 to indicate absence or presence of a specific categorical effect.
• 1 indicates presence; other variables become 0, in dummy encoding.
• Number of columns equals the number of categories.

Splitting the Dataset

• Split data into separate training and test sets
• It enhances the performance of your machine learning Model
• Training involves feeding data to the machine, test runs it after training.
• Training data and test data enable the model to identify correlations between models.
• Training Set denotes the subset of a data that will be run to train the ML model
• Test set enables the test of the machine in evaluation for outcomes relative to the trained ML model
• Usually, the dataset is split into 70:30 ratio or 80:20 ratio.

Splitting Data Code

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=1)

Code includes the variables:

• X_train is used for features in the training data
• X_text is used for features in the test data
• y_train is used for dependant variables in training data
• y_test is used for the independent variable in testing data

train_test_split() Function Parameters:

• Arrays of data.
• Test size to specify the the dividing ratio between the training and test sets.
• "random_state" sets seed for a random generator so that the output is always is its set to zero.

Encoding the Continuous Data

• Encoding can be either binning, or normalization.

Feature Scaling

• Marks the end of the data preprocessing in Machine Learning.
• Method to standardize independent variables of a dataset within a specific range.
• Feature Scaling Limits the range of variables so you can compare them on common ground.
• It prevents algorithms from being influenced by higher values

Feature Scaling

• Feature scaling can be standardization, or normalization.

Code - Standardization

from sklearn.preprocessing import StandardScaler
sc = StandardScaler()
X_train[:, 3:] = sc.fit_transform(X_train[:, 3:])
X_test[:, 3:] = sc.transform(X_test[:, 3:])

• To standardize data of the test set, mean and standard deviation values of the training set are used (no data leaking).
• transform() function is used for the test set to avoid to the fit_transform() function.

Code - Min Max normalization

from sklearn.preprocessing import MinMaxScaler
sc = MinMaxScaler()
X_train[:, 3:] = sc.fit_transform(X_train[:, 3:])
X_test[:, 3:] = sc.transform(X_test[:, 3:])

• To standardize the data of the test set, max- and min values of the training set are used. So, there is no data leaking.
• Transform() uses a fit_transform() function.

Data Binning

• Data binning data in bins/buckets, in the sense that it replaces values contained into a small interval with a single representative value for that interval.
• Binning can be applied to
  • convert numeric values to categorical values
    • binning by distance
    • binning by frequency
  • Reduce numeric values: Quantization (or sampling)

Binning

• Binning is a technique for data smoothing.
• Data smoothing is employed to remove noise from data. Three techniques are used for data smoothing:
  • binning
  • regression
  • outlier analysis

Binning by Distance Procedures

• Import the dataset
• Compute the range of values and find the edges of intervals/bins
• Define labels
• Convert numeric values into categorical labels
• Plot the histogram to see the distribution

Binning by Frequency Procedures

   - Import the dataset
   - Define the labels
   - Use qcut of the pandas library for data binning
   - Plot the histogram to see the distribution

Binning and Sampling Techniques:

-binning by mean: Each value in a bin is replaced by the mean value of the bin.

-Binning by median: Each bin value is replaced by its bin median value.

• Binning by boundary: Each bin value is replaced by the closest boundary value, i.e. maximum or minimum value of the bin.