CNN Techniques Overview

Output passed directly to the next layer without modification during initialization.

Preprocessing technique to standardize features so each has a mean of zero and variance of one.

Technique that normalizes outputs of each neural network layer during training.

A condition where the average value of a dataset is zero after standardization.

A condition where the dataset variance is standardized to one after processing.

Technique to increase dataset diversity and size through transformations.

Process of adjusting layer activations to have certain statistical properties during training.

Mean and variance calculated from a small data subset during batch normalization.

A technique to prevent overfitting by temporarily disabling neurons during training.

When a model learns training data too well, failing to generalize to new data.

The process of moving inputs through the network to get predictions.

The ability of a model to perform well on unseen data, not just training data.

The process of updating model weights based on prediction errors.

Mathematical variables used in dropout to randomly choose which neurons to disable.

Data used to train a model, where learning occurs and patterns are recognized.

Adjusting activations during inference to maintain output consistency.

The expected output of a neuron over numerous samples or training rounds.

The probability that a neuron will be dropped out during training, usually set between 0 and 1.

The stage where the trained model is used to make predictions on new data without dropout.

A model that performs well across different datasets and avoids overfitting.

Training evaluates skill development, while validation assesses performance on unseen data.

Irrelevant or random information that can mislead the model during training.

A technique to improve gradient flow in deep learning models.

The fundamental computational unit in a neural network that processes information.

Numerical values received by a neuron that are weighted and summed.

Adjusting the importance of input signals in a neuron.

A constant added to the weighted sum in a neuron, influencing its output.

A function that introduces non-linearity into the output of a neuron.

Random alterations applied to data examples to generate diverse outputs for training.

A small, randomly selected subset of examples from the dataset used for training.

A model's ability to maintain performance despite variations in input data.

Characteristics of data that remain unchanged under certain transformations.

The steps undertaken to improve a model's performance using data.

The integration of random elements in data augmentation to enhance feature learning.

A difficulty in training deep networks where gradient signals become very small, hindering weight updates.

Gradients are low or nearly zero in deeper layers, impacting training effectiveness.

Deep networks struggle to learn robust features when earlier layers get weak gradient signals.

Inadequately trained early layers lead to poor extraction of low-level features.

A form of residual connection that enhances flow and prevents vanishing gradients during backpropagation.

A structured comprehension of data built from layered learning in networks.

Gradients bypass intermediate layers, preventing excessive diminishing during backpropagation.