Feature Engineering Techniques

Feature Engineering Techniques

• Techniques for enhancing data quality and improving machine learning model performance include data imputation, data normalization, one-hot encoding, feature engineering in time series and NLP, and data dimensionality reduction.

Data Imputation

• Methods for handling missing values include using the next or previous value, K-Nearest Neighbors, maximum or minimum values, missing value prediction, most frequent values, average or linear interpolation, rounded mean or moving average, or median value, and fixed values.

Data Normalization

• Min-max normalization: Scales data to a specific range (typically 0 to 1).

  • Formula: y = (x - xmin) / (xmax - xmin) where 'x' is the original value, 'xmin' is the minimum value, and 'xmax' is the maximum value.

• Z-score normalization: Centers the data around a mean of zero and scales it by its standard deviation.

  • Formula: y = (x - mean(x)) / stddev(x) where 'x' is the original value, 'mean(x)' is the mean, and 'stddev(x)' is the standard deviation.

• Normalization by decimal scaling: Scales data to have a maximum absolute value less than 1.

  • Formula: y = x / 10^j, where 'j' is the smallest integer.

One-Hot Encoding

• Converts categorical variables into numerical representations. Replaces categories with binary vectors (e.g., Red = [1, 0, 0], Green = [0, 1, 0], Blue = [0, 0, 1]).

Log Transform

• Applied to data for various reasons and is useful for feature scaling
• Can improve normality, reduce skewness and helps handle outliers. Particularly useful on skewed datasets

Handling Outliers

• Outlier detection: Methods to identify unusual data points in a data set.
• Remove Outliers: Eliminating outliers from a data set
• Transform outliers: Methods such as log transformations, to reduce or normalize the effect of outliers.
• Imputing outliers: Replacing outliers with more typical values like means, medians, modes, or nearest neighbors.

Feature Engineering in Time Series Analysis

• Second-order differences: Finding differences between successive data points to determine if data is stationary.
  • Second order difference: y'(t) - y'(t-1)
  • Formula: y = x(t) – x(t-1) & y'(t) = y(t) - y(t-1)
• Logarithm: Calculating the logarithm of a value to smooth variations in data and help achieve seasonality. Formula examples include log(y(t)) & log(y'(t)).
• Seasonal-trend decomposition: a method that decomposes a time series into its constituent components: trend, seasonality, and remainder. This facilitates identifying patterns/seasonality.

Feature Engineering in Natural Language Processing (NLP)

• Bag of words: Represents text by counting the occurrences of each word.
• Term Frequency-Inverse Document Frequency (TF-IDF): Weights words by their frequency in a document and inverse frequency across the entire corpus (collection of documents). Formula: TF-IDF = TF * IDF
• Word2Vec: Converts words into numerical vectors, capturing semantic relationships between words.

Feature Selection

• Techniques to select the most relevant features for a machine learning task.
  • Unsupervised: Drop incomplete features/ features with high multicollinearity
  • Supervised: Forward selection, backward selection, recursive feature elimination, Chi-squared tests, Mutual Information tests and Pearson's R, Kendall's Tau, Spearman's Rho and F-score features

Data Dimensionality Reduction

• Techniques to reduce the number of variables in data while preserving important information.
  • Principal Component Analysis (PCA): Creates new uncorrelated variables (principal components) from existing ones.
  • Linear Discriminant Analysis (LDA): Finds directions in a dataset that best separate between classes.
  • Autoencoders: Neural networks that learn to compress and reconstruct data, resulting in a reduced representation.

Explore various feature engineering techniques essential for enhancing data quality and improving machine learning model performance. This quiz covers methods like data imputation, normalization techniques, and approaches for handling missing values, aimed at data science and analytics enthusiasts.