Neural Networks Overview and Objectives

A basic unit of computation in a neural network that receives input, processes it, and produces an output.

A numerical value associated with an input to a neuron, representing its relative importance.

A function applied to the weighted sum of inputs to a neuron, determining its output.

A connection between neurons, allowing information to flow between them.

The process of adjusting weights in a neural network to improve its performance.

A collection of interconnected neurons organized into layers.

The rate at which a neuron fires, represented by the activation function.

A type of activation function that outputs values between 0 and 1.

An artificial neural network where connections between units do not form a cycle. Information flows in one direction, from input to output, without loops.

A rule or algorithm that modifies the parameters of a neural network to improve its performance. Weights and thresholds are typically adjusted.

A type of neural network with a single layer of neurons. It can be used for simple classification tasks, like separating linearly separable data.

A type of neural network composed of multiple layers of neurons. It can learn complex patterns and is widely used for various tasks.

A specialized type of feedforward neural network that excels in image recognition tasks. It uses convolutional filters to extract features from input images.

A type of neural network where connections between neurons form cycles. It's well-suited for processing sequential data, like natural language or time series.

A neural network with a hidden layer that learns complex patterns from data. It's a powerful tool for various applications, including classification and regression.

A type of activation function that introduces non-linearity by outputting the input if it is positive, or zero otherwise.

An activation function that takes the input and outputs a value between 0 and 1. It's commonly used for binary classification problems.

Similar to Sigmoid, this activation function outputs values between -1 and 1. It's often used for bipolar classification tasks.

The process of training a neural network using labeled data to adjust its parameters and improve its performance. The goal is to minimize the error between predicted outputs and actual outputs.

The weights and biases of the perceptron are initialized with specific initial values. These values often start at zero (wi = b = 0).

The input values are directly assigned to the activation units (xi = si). This process happens for each input element.

The response of the output unit is computed using a specific formula that involves summing the weighted inputs and applying an activation function (sigma).

The weights are adjusted based on the error between the output and target values. The change in weight depends on the learning rate and the input value.

The error is calculated for the output unit and updated based on the target value. The direction and magnitude of the update depend on the learning rate α. and the error difference.

The network is trained iteratively by adjusting the weights based on the error until the error becomes minimal and the changes in weights are negligible.

A type of neural network that uses loops to process sequences of data, enabling it to learn from temporal dependencies.

The units (neurons) in a CNN are organized in a way similar to the visual cortex, with each unit responding to a localized region of the input (called the receptive field).

CNNs typically require large amounts of data to train effectively, as they need to learn complex patterns from images or videos.

The process of combining multiple features learned by the CNN filters to identify the presence of specific objects within the image.

The process of adjusting the weights and biases within the RNN to minimize errors and improve its ability to predict future outputs.

RNNs are well-suited for tasks that involve processing sequential data, such as speech recognition, handwriting recognition, and natural language processing.

The input values in a training dataset. Each row represents a separate training sample.

The desired output for each input sample in a training dataset. It's what the network should learn to produce.

A single iteration of the training process where the neural network processes all training samples once. It's like one full round of learning.

The weighted sum of all inputs in a neuron, which determines the neuron's activation value.

The output value of a neuron after applying an activation function. It represents the neuron's response.

The learning rate for updating weights and bias during training. It controls the step size for each update.

The bias term added to the net input before applying the activation function. It adjusts the neuron's activation threshold.