شبكات هوبفيلد

Associative Memories

• An associative memory is a content-addressable structure, mapping input patterns to output patterns.
• Two types exist: autoassociative and heteroassociative.
• Auto-association retrieves a previously stored pattern that closely resembles the current pattern.
• Hetero-association retrieves a pattern different from the input, potentially differing in type and format.

Hopfield Networks

• Hopfield Networks are a type of recurrent neural network, designed with the brain in mind.

• They enable the recognition of familiar faces in unfamiliar environments within a brief timeframe (100-200 milliseconds).

• They permit the recollection of complete sensory experiences (sounds, scenes) from partial sensory input (like a few bars of music).

• They form associations between different things.

• They use a different type of network compared to multilayer neural networks trained using backpropagation.

• Hopfield Networks use recurrent neural networks to achieve associative memory characteristics.

• Recurrent neural networks include feedback loops from outputs to inputs.

• These feedback loops significantly affect the network's learning capabilities.

• The stability of recurrent network structures was a concern in the 1960s and 70s, with limited ability to predict network stability.

• John Hopfield's 1982 formulation addressed this, providing a physical principle for storing information in dynamically stable networks.

• Hopfield Networks are a special type of dynamic network (recurrent).

• It forms a single layer feedback network first introduced by John Hopfield (1982).

• Hopfield Networks offer several key capabilities:

  • Forming patterns or classifications.
  • Solving optimization problems.
  • Restoring patterns.
  • Handling noisy patterns.

Hopfield Network Architecture

• A Hopfield network's architecture is a single-layer network with a set number of neurons (in one example N=4).
• Each neuron has connections (weights) to every other neuron in the network.

Hopfield Network Equation

• A typical Hopfield network equation (in matrix form) shows the relationship between synaptic weights and memorized states.

Hopfield Network Activation Function

• The Hopfield Network uses the Sign activation function, a characteristic of McCulloch and Pitts neurons.

State Vector

• The current state of a Hopfield network is determined by the outputs of all its neurons.
• A state vector defines this current state (y₁, y₂, ...,yₙ).

Storage Phase: Weights

• Synaptic weights in a Hopfield network are often represented in matrix form using a specific calculation involving memory states, identity matrix, and matrix transposition.

Possible states and Stability

• Stable states in a Hopfield network are pivotal for pattern recognition.

Hopfield Network Test

• A Hopfield Network is triggered with input data (vector)
• The network responds using its calculation with input weights
• The outcome is compared with the initial input pattern.

Fundamental Memories

• While most states within the network are unstable, fundamental memories attract similar states.
• These memories are pivotal for handling unstable states (e.g., correcting errors in partial input).
• Fundamental memories are those stable states of the Hopfield network.

Example Network Analysis

• Example networks with varying numbers of neurons and states (e.g., three-neuron network) demonstrate different stable states arising from various input patterns.

Stability Analysis

• Stability analysis in a Hopfield network investigates the dynamic system's stability using the concept of an energy function.

• The energy function's definition encompasses the network's state variables and synaptic weights, providing insights into the network's stability characteristics.

• Stability is assessed through evaluating the energy function's transformation, with decreasing energy (over time) reflecting stability within the system.

• Stability is often analyzed using bipolar (or binary) notation.

Updating Mechanism

• The updating procedure in a Hopfield network is often asynchronous (random selection of neurons for update).
• This procedure proceeds until no further changes occur (stable state is reached).

Asynchronous Update

• The updating process occurs asynchronously, with only one neuron changing state at each point.

Memorized Patterns

• The Hopfield network is characterized as content addressable, directly referencing stored patterns.
• Fixed point attractors in the Hopfield network correspond to the stored patterns, and are minima of the energy function.