Questions and Answers
What is the value of Emb in the equation Emb = [S0S] 𝑋?
(4096, 1)
What is the output shape of the first attention block?
(1, 4096)
What is the formula for calculating attention in the self-attention mechanism?
softmax(𝑄𝐾𝑇 / 𝑑𝑘)𝑉
What is the purpose of the cache in the self-attention mechanism?
Signup and view all the answers
What is the shape of the input sequence to the second attention block?
Signup and view all the answers
What is the output shape of the second attention block?
Signup and view all the answers
What is the type of attention used in the decoder?
Signup and view all the answers
What is the purpose of the softmax function in the attention mechanism?
Signup and view all the answers
What is the shape of the input sequence to the third attention block?
Signup and view all the answers
What is the output shape of the fourth attention block?
Signup and view all the answers
