Introduction to Machine Learning Concepts

Machine Learning Intro

• Machine learning utilizes various disciplines like biology, mathematics, statistics, and computer science.
• Computational science and data science are closely related fields that utilize machine learning.

Defining the Problem

• Classification: Categorizing data based on shared characteristics.
• Prediction: Determining the outcome of a future event.
• Feature Selection: Deciding which features are important based on meaningfulness, correlations, and other factors.
• Filter: Tests for correlations to select features. Both univariate and multivariate methods are used.
• Wrapper: Selects and tests groups of features.
• Embedded: Feature selection is part of the algorithm itself.

Decision Tree

• Works with continuous, discrete, and categorical data.
• General Algorithm:
  • Determines the best split for the data.
  • Moves samples along the tree based on the split criteria.
  • Repeats steps one and two.

Random Forest

• General Algorithm
  • Subsets features randomly with replacement.
  • Constructs decision trees based on the subsets.
  • Utilizes an ensemble method for final prediction.

Random Forest Variable Importance

• MeanDecreaseGini: Measures the importance of features by calculating the decrease in Gini impurity when a feature is used for splitting nodes in the decision trees.
• Higher MeanDecreaseGini values indicate more important features.
• Random Forests can be used to rank features by importance.
• Random Forest models can be evaluated based on the number of features used.
• The number of features used can affect the cross-validation error of the model.

K Nearest Neighbors

• Groups a sample with the closest K matches.
• Supervised Classification: Uses K nearest neighbors to determine the class of an unknown sample.

K Nearest Neighbors Data Requirements

• Data must be binary or continuous.
• Data can be transformed to meet these requirements.

Machine Learning Introduction

• Machine learning is a vast field that combines elements from computer science, biology, mathematics, statistics, and computational science.
• It involves the development of computer systems that can learn from data and make predictions or decisions without explicit programming.

Defining the Problem

• It's crucial to clearly define the problem for machine learning applications.
• Consider whether you're focusing on classification (categorizing data based on shared characteristics) or prediction (determining future events).
• Key considerations include accuracy, sensitivity, specificity, throughput, required features, sample selection, and sample size.

Feature Selection

• Feature selection focuses on identifying the most important features or variables that contribute to the machine learning model's success.
• The goal is to minimize redundancy and improve efficiency.
• There are three main approaches:
  • Filter Methods (Univariate & Multivariate): Test for correlations among features.
  • Wrapper Methods: Selects and tests groups of features.
  • Embedded Methods: Integrated into the learning algorithm.

Decision Tree

• A decision tree is a flowchart-like structure used to visualize and predict outcomes based on sequential decisions.
• It works with various data types, including continuous, discrete, and categorical.
• The algorithm involves:
  • Determining the optimal split based on the data.
  • Moving samples along the tree branches based on decision criteria.
  • Repeating these steps until a final prediction is reached.

Random Forest

• A random forest is a powerful ensemble learning method combining multiple decision trees.
• The algorithm involves:
  • Subsetting features randomly with replacement.
  • Building individual decision trees using these subsets.
  • Aggregating the predictions from all trees to make a final prediction.

Random Forest Variable Importance

• Random forests offer a mechanism to assess the importance of different features in the model.
• MeanDecreaseGini: A metric that measures the average decrease in impurity (Gini index) when a particular feature is excluded from the model.
• Features with high MeanDecreaseGini values are considered more important.

K Nearest Neighbors (KNN)

• KNN is a supervised classification algorithm that classifies a sample based on its similarity to its k nearest neighbors.
• Requires binary or continuous data (transformation is possible for other types).
• The algorithm assigns the sample to the class that is most prevalent among its k nearest neighbors.

Machine Learning Intro

• Biologists, mathematicians, statisticians and computer scientists contribute to the development of machine learning models.
• Machine learning is a subset of artificial intelligence that enables computers to learn from data without explicit programming.

Defining the Problem

• Choosing between classification and prediction depends on the objective of your analysis.
• Factors to consider when defining a machine learning problem:
  • Classification vs. Prediction:
    • Classification aims to categorize data based on shared characteristics.
    • Prediction attempts to forecast future events.
  • Accuracy, Sensitivity, and Specificity: Choosing which aspect is more important depends on your application.
  • Throughput and Use: Consider if the model is for high-throughput analysis or limited application.
  • Feature Selection: Identifying the essential features for your analysis.
  • Sample Selection: Choosing appropriate samples and the number of samples to use.

Feature Selection

• Feature Selection aims to identify the most relevant features for a machine learning model.
• There are three main approaches:
  • Filter Methods: Univariate and multivariate tests are used to assess correlations between features and the target variable.
  • Wrapper Methods: Groups of features are selected and evaluated iteratively.
  • Embedded Methods: Feature selection is incorporated into the algorithm itself.

Decision Tree

• Decision trees can handle various data types: continuous, discrete, and categorical.
• The algorithm works by:
  • Identifying the best split based on specific metrics.
  • Categorizing samples along the tree based on the split criteria.
  • Repeating the process until a termination condition is met.

Random Forest

• Random forests are an ensemble method that combines multiple decision trees for a final prediction.
• The algorithm involves:
  • Subsetting features randomly with replacement for each tree.
  • Building individual decision trees using the subsetted features.
  • Aggregating the predictions from all trees for a final output.

Random Forest Variable Importance

• Random forests can measure the importance of each feature using the MeanDecreaseGini metric.
• Features with higher MeanDecreaseGini values are generally considered more important for the model's performance.
• The MeanDecreaseGini metric indicates the reduction in impurity (Gini index) achieved by a feature when it is used to make a split in the decision tree.
• Features with higher MeanDecreaseGini values tend to be more predictive of the target variable and therefore considered more important.

K Nearest Neighbors (KNN)

• KNN is a supervised classification algorithm that uses distance to the k nearest neighbors to classify a data point.
• KNN requires binary or continuous data.
• The data can be transformed to accommodate the algorithm.