Convolutional Neural Networks Overview

Convolutional Neural Networks (CNNs)

• CNNs are specialized deep neural networks, processing structured data
• Unlike fully connected networks, CNNs have layers to extract spatial or temporal information hierarchically
• CNNs are powerful for visual or sequential data tasks
• Convolutional Layer: applies filters to input data, sliding over the input to compute a dot product between filter weights and input values producing a feature map that highlights specific features (edges, textures, patterns)
• ReLU Activation: applied element-wise after the convolution operation to introduce non-linearity, enabling the model to learn complex patterns
• CNN applications include image recognition, face recognition, object detection, and x-ray analysis, often achieving human-level performance in some tasks

Types of Learning

• Supervised Learning: model learns from labeled data, mapping inputs to outputs accurately (minimizing prediction error to actual labels); used in image classification, object detection
• Unsupervised Learning: deals with unlabeled data, uncovering hidden patterns, structures, or relationships in data without guidance; e.g., clustering

Deep vs Shallow Architectures

• Deep Architectures: have multiple layers (dozens or hundreds) capable of learning hierarchical and abstract representations of data; particularly powerful for complex tasks
• Shallow Architectures: typically have one or two layers, capable of learning intricate patterns but computationally more efficient for simpler problems

Supervised vs Unsupervised + Deep vs Shallow

• Supervised + Deep: Models learn from labeled data with many layers, handling complex tasks (e.g., Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs))
• Supervised + Shallow: Models learn from labeled data with fewer layers, performing adequately on small datasets or requiring high interpretability (e.g., Perceptrons, Support Vector Machines (SVMs))
• Unsupervised + Deep: Architectures uncovering patterns in unlabeled data through multiple layers (e.g., Deep Belief Networks (DBNs), Sparse/Denoising Autoencoders)
• Unsupervised + Shallow: Computationally efficient and used in exploratory data analysis with relatively simple datasets (e.g., Restricted Boltzmann Machines (RBMs), Autoencoders)

Basic Components of CNNs

• CNNs consist of interconnected layers working to transform input data into meaningful patterns for uses such as classification, detection, or segmentation
• Layers extract increasingly complex and abstract features from input data

Types of Layers in a CNN

• Convolutional Layers: The core building blocks, responsible for extracting features from input data using filters that slide across the image; detect patterns (e.g., edges, textures)
• Pooling Layers: Reduce spatial dimensions and computational complexity of feature maps while preserving important features; downsample feature maps
• Fully Connected Layers: Act as decision-making components, transforming extracted features into a final output (e.g., classifications, predictions)

Pooling Layer Operations

• Max Pooling: Select the maximum value from each pooling window; captures the most significant feature
• Average Pooling: Compute the average value of all elements in the pooling window; useful for capturing the average response of an area

Training Process

• Loss Function: A critical component guiding the training process; quantifies the error between the network's prediction and the true labels.
• Training Phase: Critical phase where the network learns patterns and relationships in the input data for tasks like classification, regression, and object detection
• Gradient Descent: Used to iteratively update the network's parameters to minimize the difference between its predictions and actual target values