Evaluation Metrics for Binary Classification

Evaluation Metrics

• Classification in binary (2-class) problems:
  • True Positives (TP): correctly identified positives (class 1) instances
  • True Negatives (TN): correctly identified negatives (class 0) instances
  • False Positives (FP): incorrectly classified as positives (class 1) that are really negatives (class 0)
  • False Negatives (FN): incorrectly classified as negatives (class 0) that are really positives (class 1)
• Confusion Matrix: a table used to evaluate the performance of a classification model
• Evaluation metrics formulas:
  • Precision = TP / (TP + FP)
  • Recall = TP / (TP + FN)
  • Accuracy = (TP + TN + FP + FN)
  • F1 score = 2 * (Precision * Recall) / (Precision + Recall)

Features

• Types of features:
  • Raw pixels
  • Histograms
  • GIST descriptors
  • SIFT descriptor
  • ...

Artificial Intelligence and Computer Vision

• Application domains:
  • Automotive: self-driving cars, driver assistance
  • Manufacturing: industrial inspection, quality assurance
  • Security and Defense: surveillance
  • Agriculture: crop monitoring, precision agriculture, pest control
  • Retail: customer tracking, theft detection, automatic checkout
  • Healthcare: medical image analysis, computer-aided diagnosis
  • Entertainment: cinema and digital games
• Tasks:
  • Image classification
  • Object detection
  • Semantic segmentation
  • Instance segmentation
  • Tracking
  • ...

Machine Learning and Deep Learning

• Machine learning:
  • Subset of artificial intelligence
  • Type of supervised learning
  • Uses labeled data to train models
• Deep learning:
  • Subset of machine learning
  • Uses neural networks to train models
  • Can be used for computer vision tasks

Neural Networks

• Types of neural networks:
  • Feedforward neural networks
  • Convolutional neural networks (CNNs)
  • Recurrent neural networks (RNNs)
• Neural networks for classification in computer vision:
  • Use of convolutional neural networks (CNNs)
  • Classification of images into different classes
• Evaluation and metrics for neural networks:
  • Use of precision, recall, accuracy, and F1 score
• Training neural networks:
  • Use of labeled data to train models
  • Optimization of model parameters
• Implementation challenges:
  • Computational resources
  • Handling large datasets
  • Overfitting and underfitting

Traditional Approach vs Deep Learning Approach

• Traditional approach:
  • Hand-engineered features
  • Classification using machine learning algorithms
• Deep learning approach:
  • Automatic feature learning
  • Use of neural networks for classification### Classifiers
• Variety of classifiers: Support Vector Machines (SVM), Naïve Bayes, Bayesian networks, Logistic regression, Randomized Forests, Boosted Decision Trees, K-nearest neighbor (KNN), Neural networks, and Deep Learning

Learning a Classifier

• Goal: learn a function to predict labels from features given a set of features with corresponding labels
• Example: classify points x1 and x2 into categories (e.g., o or x)

SVM Classifier

• Calculate Max Margin Hyperplane to separate classes

KNN Classifier

• Determine class by looking at K nearest neighbors and their classes

Evaluation Metrics

• Classification in binary (2-class) problems:
  • True Positives (TP): correctly identified positives (class 1) instances
  • True Negatives (TN): correctly identified negatives (class 0) instances
  • False Positives (FP): incorrectly classified as positives (class 1) that are really negatives (class 0)
  • False Negatives (FN): incorrectly classified as negatives (class 0) that are really positives (class 1)
• Confusion Matrix: a table used to evaluate the performance of a classification model
• Evaluation metrics formulas:
  • Precision = TP / (TP + FP)
  • Recall = TP / (TP + FN)
  • Accuracy = (TP + TN + FP + FN)
  • F1 score = 2 * (Precision * Recall) / (Precision + Recall)

Features

• Types of features:
  • Raw pixels
  • Histograms
  • GIST descriptors
  • SIFT descriptor
  • ...

Artificial Intelligence and Computer Vision

• Application domains:
  • Automotive: self-driving cars, driver assistance
  • Manufacturing: industrial inspection, quality assurance
  • Security and Defense: surveillance
  • Agriculture: crop monitoring, precision agriculture, pest control
  • Retail: customer tracking, theft detection, automatic checkout
  • Healthcare: medical image analysis, computer-aided diagnosis
  • Entertainment: cinema and digital games
• Tasks:
  • Image classification
  • Object detection
  • Semantic segmentation
  • Instance segmentation
  • Tracking
  • ...

Machine Learning and Deep Learning

• Machine learning:
  • Subset of artificial intelligence
  • Type of supervised learning
  • Uses labeled data to train models
• Deep learning:
  • Subset of machine learning
  • Uses neural networks to train models
  • Can be used for computer vision tasks

Neural Networks

• Types of neural networks:
  • Feedforward neural networks
  • Convolutional neural networks (CNNs)
  • Recurrent neural networks (RNNs)
• Neural networks for classification in computer vision:
  • Use of convolutional neural networks (CNNs)
  • Classification of images into different classes
• Evaluation and metrics for neural networks:
  • Use of precision, recall, accuracy, and F1 score
• Training neural networks:
  • Use of labeled data to train models
  • Optimization of model parameters
• Implementation challenges:
  • Computational resources
  • Handling large datasets
  • Overfitting and underfitting

Traditional Approach vs Deep Learning Approach

• Traditional approach:

  • Hand-engineered features
  • Classification using machine learning algorithms

• Deep learning approach:

  • Automatic feature learning
  • Use of neural networks for classification### Classifiers

• Classifiers include Support Vector Machines (SVM), Naïve Bayes, Bayesian networks, Logistic regression, Randomized Forests, Boosted Decision Trees, K-nearest neighbor (KNN), Neural networks, and Deep Learning.

Learning a Classifier

• Given a set of features with corresponding labels, learn a function to predict the labels from the features.
• SVM calculates the Max Margin Hyperplane.
• KNN looks at K neighbors and uses their class.

Machine Learning Algorithms

• A list of ML algorithms can be found at <a href="https://noeliagorod.com/2020/03/16/a-tour-of-machine-learning-algorithms/">https://noeliagorod.com/2020/03/16/a-tour-of-machine-learning-algorithms/</a>.

Deep Learning Vocabulary

• Deep Learning is also known as Deep Neural Networks (DNN), Deep Structural Learning, or Deep Belief Networks.
• Recurrent Neural Networks (RNN) are related to Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU).
• Convolutional Neural Networks (CNN) are also known as ConvNets.
• Transformers and Stable Diffusion are other Deep Learning concepts.

Neural Networks

• A neuron is a linear function with an optional non-linear activation.
• A Neural Network is a linear function in the form yi = Σ xj*wij + bi.
• Neural Networks can be used for image classification and other computer vision tasks.

Deep Neural Networks

• A Deep Neural Network is a neural network with multiple layers.
• Deeper Neural Networks have more layers.

Activations

• Activations are used for intermediate neurons in Neural Networks.

Learn about evaluation metrics for binary classification problems, including True Positives, True Negatives, False Positives, and more.