Artificial Intelligence: Encoder-Decoder, Auto-Encoder, PCA Quiz

An Auto-Encoder consists of an NN Encoder, an NN Decoder, and a reconstruction ability.

PCA stands for Principle Component Application.

The objective in a Deep Auto-encoder is to maximize the bottleneck layer.

In an Encoder-Decoder for NLP, the Decoder generates the contextualized representation.

Attention Mechanism plays a crucial role in Encoder-Decoder networks by providing flexibility but not context vector importance.

Cross-Attention is used in neural machine translation to align and translate target sentences.

Auto-encoder is a supervised learning method that creates a compact representation of input objects.

The encoder and decoder in an auto-encoder neural network work independently to reconstruct the original input object.

Deep auto-encoders always require a symmetric layer structure for more complex representations.

Denoising auto-encoders are used to add noise to input data for better reconstruction.

Auto-encoders can be applied in Convolutional Neural Networks (CNNs) for tasks like pooling and activation.

Encoder and Decoder in basic RNN-based Encoder-Decoder Networks have different internal structures.

The context vector 'c' from the Encoder carries essential input information to the Decoder.

In Encoder-Decoder Attention models, the Attention Mechanism enables the Decoder to focus only on the final state of the Encoder.

Attention scores determine the relevance of each Encoder hidden state to the current Decoder state.

Transformers are a type of neural network architecture that require sequential processing of input data.

In Transformers, the Multi-head Attention mechanism helps capture relationships between different parts of the input sequence.

Self-Attention in Transformers is used to find relevant vectors within the output sequence.