5 Questions
Which of the following best describes the purpose of attention in a sequential decoder?
To focus on the most relevant parts of the input sequence for each output
What is the formula for computing the attention score αᵢⱼ in the context of attention?
αᵢⱼ = softmax(eⱼ)
What does the alignment model f in the context of attention represent?
The scores of how well the inputs around position j and the output at position i match
How can the alignment model f be approximated?
By using a small neural network
What is the purpose of the context vector c in the context of attention?
To compute the attention scores αᵢⱼ
This quiz tests your understanding of attention mechanisms in sequential decoders and how they alleviate the vanishing gradient problem. Explore the concept of computing attention and learn about context vectors and weighted sums.
Make Your Own Quizzes and Flashcards
Convert your notes into interactive study material.
Get started for free
