AT Lecture 7

CNN Applications

• CNNs are utilized for image classification, object localization, and object detection.

Object Localization

• Entails identifying rectangular regions where fixed objects occur in an image.
• Four numbers are used to identify a bounding box; these specify the coordinates of the top-left corner, the length, and the width.
• Object localization is often integrated with object classification.

Object Detection

• Used when there are a variable number of objects of different classes in an image.
• Aims to identify all objects and their corresponding classes within the image.
• Object localization architecture can't be used for object detection, as the required number of classification and regression heads is unknown.

R-CNN (Region Based Convolutional Neural Network)

• The R-CNN approach initially generates category-independent region proposals, like candidate bounding boxes.
• It then extracts features from each candidate region using a deep convolutional neural network.
• Finally, classifies these features as one of the known classes, linear SVM classifier models are commonly used.
• It has three main modules: Region Proposal, Feature Extractor, and Classifier.

Fast R-CNN

• R-CNN is slow.
• This slowness is due to passing CNN-based feature extraction for each candidate region. There is is no sharing of computation.
• Approximately 2,000 proposed regions exist per image at test-time.

Faster R-CNN

• Includes a region proposal network.

Transfer Learning

• Building strong machine learning models typically requires a large amount of training data.
• Transfer learning is useful because small training jobs are common, and labeled data is often scarce.
• Transfer learning reuses the knowledge gained from "source" tasks is reused for a "target" task that lacks abundant labeled data.
• A common intuition is that networks can use representations form one task for another.
• Instance-based approaches leverage appropriate data from a source task to supplement target task training.
• Feature representation-based approaches leverage source task weight matrices.
• Using trained weights in a source network the input can be transferred by fine-tuning or retraining the final dense layer.

Improving Transferability in Transfer Learning

• This involves the selection of a source model.
• Similarity measures between source and target tasks help improve transferability improvements.
• Improper base dataset/model choices may result in degraded performance compared to no transfer learning due to negative transfer.
• Degree of fine-tuning has to be tuned when using transfer learning.
• This tuning includes selecting layers to fine-tune/freeze and learning rate. Higher rates can wash away what as originally in the network.

Good and bad sources

• Selecting a base model that is similar to a training task performs better.

Selection of Source Models

• Training requires time and resources, the goal is to avoid a lengthy optimization process.
• The P2L (Predict to Learn) approach estimates the appropriateness of a previously trained model for use with a new learning task, therefore improving the speed of training.
• P2L requires one forward pass of the target data set through a single reference model.
• Two attributes account for the efficacy of P2L: Similarity between source and target datasets, and size of source dataset.

Transfer Learning: Basic Finetuning

• A deep network pre-trained on a large dataset is typically used.
• One replaces the last (classification) layer with a randomly initialized layer.
• Train only the new layer's weights, using the "frozen" embedding.

Transferability

• Some networks are more transferable than others, based upon their architecture.

Learn to Transfer

• One must chose the right source model.
• One must chose the right method to use, such as shallow learning driving SVMs, fine tuning, or single source or ensemble.
• Which layers to freeze/fine tune should be chosen (ResNet101, GoogLeNet, VGG16 etc)
• One must chose the right curriculum.

IBM Visual Recognition Service

• Cloud based service that supports both image classification and object detection.
• Provides the use of pretrained models or developing custom models.

Exploiting Web Data for Training

• Web and social media are important sources of vision research data.
• Datasets like ImageNet, PASCAL VOC, and MS COCO use human intelligence to filter out noisy labels.
• Automated labeling can be used to do this without human intervention.

Automated Labeling of Images

• This may include pseudo-labeling.
• Exploit data from the wild to capture source representations.
• This includes semi-supervised and weakly supervised methods to train and classify images.

Semi-supervised Learning

• Involves training a teacher model and then running it on unlabeled data.

Labeling unlabeled images

• One needs to run the teacher model on each unlabeled examples to calculate the soft max predictions to use as new labels.

Parameters P and K

• Concerns why to predict one class over the other.
• Allows the image to be replicated for each class.

Performance of teacher-student learning

• Important parameters is fine tuning the model on the correct data.

Learning from weak labels

• Facebook contains tons of data like this.

Weakly Supervised Learning Architecture

• Trains CNNs to predict the words from the images.
• A multi-label learning problem with very noisy labels.
• Standard convnet architecture is used.
• There is a 100K class multi-class logistic loss function.

Experimental evaluation

• Uses CNNs for transferring other vision tasks.

Analyzing the word embeddings

• The output layer is a word embedding.

Feature Vector Embeddings

• Average over the activations from layer n.
• For n=N-1, the vector is 4096 dimensions.

Divergence in Feature Space

• Measure dataset similarity through feature space.

Divergence based Pseudo Labeling to Improve Transfer Learning

• For each unlabeled sample, calculate its distance (in feature space) from a set of named anchor points representing known and labeled categories, like animal, plant, tool
• Construct pseudo-labels for unlabeled samples based on these distances
  • Pseudo-labels are a sequence of semantically descriptive names: e.g., <tool, plant>
• Train a source model using these automatically generate pseudo labels
• Finetune the source model on target dataset

Pseudo Label Datasets

• The closer your training data is to your target the better the results.

Siamese Networks

• A 2 path neural network, with loss being calculated between 2 streams rather than a single set.

Contrastive Loss

• A neural network loss algorithm used in siamese networks to perform contrastive learning.
• A similarity between 2 transformed data points.

Explaining Contrastive Loss Function

• Aims to minimise intra-class distance and maximise inter-class distance

Why we need margin m in contrastive loss?

• Introduces a margin into the objective when points are dissimilar that can push the dissimilar points apart.