합성곱 신경망 (CNN) 아키텍처

• Convolutional Neural Networks (CNNs) are a class of deep neural networks, most commonly applied to analyzing visual imagery.
• CNNs are designed to automatically and adaptively learn spatial hierarchies of features from input images.
• CNNs are used in various applications, including image and video recognition, image classification, object detection, and medical image analysis.

CNN Architectures

• LeNet-5: One of the earliest CNN architectures, designed for handwritten digit recognition.
  • Featured convolutional layers, subsampling layers, and fully connected layers.
• AlexNet: Significantly deeper than LeNet-5, it achieved breakthrough performance in the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) in 2012.
  • Used ReLU activation functions and dropout for regularization.
• VGGNet: Explored the impact of network depth, using very small (3x3) convolutional filters uniformly throughout the entire architecture.
  • VGGNet comes in variants such as VGG16 and VGG19, where the number indicates the weight layers in the network.
• GoogLeNet (Inception): Introduced the Inception module, designed to allow for more efficient use of computing resources and enable deeper networks.
  • Used parallel convolutional pathways with varying filter sizes.
  • Did not employ fully connected layers at the end, relying instead on average pooling.
• ResNet: Introduced residual connections (skip connections) to address the vanishing gradient problem in very deep networks.
  • Allows training of networks with hundreds or even thousands of layers.
• DenseNet: Further extends the idea of ResNet by connecting each layer to every other layer in a feed-forward fashion.
  • Enhances feature reuse and reduces the number of parameters.
• MobileNet: Designed for mobile and embedded vision applications, focusing on efficiency.
  • Uses depthwise separable convolutions to reduce the computational cost.
• EfficientNet: Employs a compound scaling method to uniformly scale all dimensions of depth/width/resolution with a set of scaling coefficients.
  • Achieves better accuracy and efficiency than previous models.

CNN Layers

• Convolutional Layer: The core building block of a CNN.
  • Performs a convolution operation on the input, using a set of learnable filters.
  • Filters detect specific features in the input, such as edges, corners, or textures.
  • Multiple filters are applied to each location in the input, creating a feature map.
• Activation Function: Applies a non-linear transformation to the output of each convolutional layer.
  • Examples include ReLU (Rectified Linear Unit), sigmoid, and tanh.
  • ReLU is commonly used due to its simplicity and effectiveness in overcoming the vanishing gradient problem.
• Pooling Layer: Reduces the spatial size of the feature maps, reducing the number of parameters and computational complexity.
  • Max pooling selects the maximum value from each patch of the feature map.
  • Average pooling computes the average value from each patch.
• Batch Normalization Layer: Normalizes the activations of the previous layer for each mini-batch.
  • Helps to accelerate training and reduce sensitivity to network initialization.
• Fully Connected Layer: Connects every neuron in one layer to every neuron in the next layer.
  • Typically used in the final layers of a CNN for classification tasks.
  • Maps the learned features into the final output classes.
• Dropout Layer: Randomly sets a fraction of input units to 0 at each update during training time.
  • Prevents overfitting by reducing the interdependence of neurons.
• Padding: Adding layers of zeros to the border of the images
  • Used for controlling the spatial size of the output features.
  • Types of padding: Valid, Same, Full
• Strides: Determines how many pixels the filter will move over at a time.
  • Strides of greater than 1 will downsize the image

CNN Training

• Data Collection and Preparation: Gathering a large and diverse dataset of images.
  • Preprocessing the images, including resizing, normalization, and data augmentation.
  • Splitting the data into training, validation, and test sets.
• Model Definition: Selecting an appropriate CNN architecture, such as AlexNet, VGGNet, ResNet, or a custom design.
  • Defining the layers, activation functions, and other hyperparameters of the network.
• Weight Initialization: Setting the initial values of the network's weights.
  • Methods include random initialization, Xavier initialization, and He initialization.
  • Proper initialization is crucial for effective training.
• Forward Propagation: Passing the input images through the network to compute the output predictions.
  • Each layer performs its respective operations, transforming the input until the final output is obtained.
• Loss Function: Quantifying the difference between the predicted outputs and the ground truth labels.
  • Common loss functions include cross-entropy loss for classification and mean squared error for regression.
• Backpropagation: Computing the gradients of the loss function with respect to the network's weights.
  • Uses the chain rule to propagate the gradients backwards through the network.
• Optimization Algorithm: Updating the network's weights based on the computed gradients.
  • Common optimization algorithms include stochastic gradient descent (SGD), Adam, and RMSprop.
  • The goal is to minimize the loss function and improve the network's accuracy.
• Hyperparameter Tuning: Optimizing the hyperparameters of the network, such as learning rate, batch size, and number of epochs.
  • Techniques include grid search, random search, and Bayesian optimization.
• Regularization: Applying techniques to prevent overfitting, such as dropout, weight decay, and batch normalization.
  • Regularization helps to improve the generalization performance of the network.
• Evaluation: Assessing the performance of the trained network on the validation and test sets.
  • Metrics include accuracy, precision, recall, F1-score, and area under the ROC curve (AUC).
• Deployment: Deploying the trained network for real-world applications, such as image classification, object detection, or image segmentation.
• Data Augmentation: Artificially increasing the size of the training set by applying transformations to the original images.
  • Common transformations include rotations, flips, zooms, and translations.
  • Helps to improve the network's robustness and generalization ability.
• Transfer Learning: Utilizing pre-trained CNN models on large benchmark datasets, such as ImageNet.
  • Fine-tuning the pre-trained models on a specific task with a smaller dataset.
  • Can significantly reduce training time and improve performance.