Mastering Attention Mechanisms in Sequential Decoders

To alleviate the vanishing gradient problem

To compute the alignment model f

To focus on the most relevant parts of the input sequence for each output (correct)

To compute the context vector c

αᵢⱼ = softmax(eⱼ) (correct)

αᵢⱼ = softmax(f(i, j))

αᵢⱼ = softmax(e)

αᵢⱼ = softmax(hⱼ)

The amount of attention the ith output should pay to the jth input

The scores of how well the inputs around position j and the output at position i match (correct)

The hidden state from the previous timestep

The encoder state for the jth input

<p>By using a small neural network</p> Signup and view all the answers

<p>To compute the attention scores αᵢⱼ</p> Signup and view all the answers