Recent Lessons

Show all results for ""

Feature Overview

Ace your exams with our all-in-one platform for creating and sharing quizzes and tests.

Quizzes

Create quizzes and tests automatically from your content using AI.

Flashcards

Automatically turn your notes into digital flashcards.

Share, Export & Embed

Share with classmates or export to Excel and your learning management system.

Stats & Reporting

Auto-grading quizzes and tests with detailed stats and reports.

Mobile Apps

The smarter way to study – wherever you are.

Pricing

Search...

10 Questions

0 Views

Transformers and Sequence Embeddings

Choose a study mode

Play Quiz

Study Flashcards

Spaced Repetition

Chat to lesson

Podcast

Play an AI-generated podcast conversation about this lesson

Questions and Answers

What is the value of Emb in the equation Emb = [S0S] 𝑋?

(4096, 4096)

(1, 1)

(4096, 1) (correct)

(1, 4096)

What is the output shape of the first attention block?

(4096, 4096)

(4096, 1)

(1, 1)

(1, 4096) (correct)

What is the formula for calculating attention in the self-attention mechanism?

softmax(𝑄𝐾𝑇 / 𝑑𝑘)𝑉 (correct)

softmax(𝑄𝐾)𝑇 / 𝑑𝑘 𝑉

softmax(𝑄𝐾𝑇) / 𝑑𝑘 𝑉

softmax(𝑄𝐾𝑇) 𝑉

What is the purpose of the cache in the self-attention mechanism?

To reduce the computational cost of the attention mechanism Signup and view all the answers

What is the shape of the input sequence to the second attention block?

(2, 4096) Signup and view all the answers

What is the output shape of the second attention block?

(2, 4096) Signup and view all the answers

What is the type of attention used in the decoder?

Scaled dot-product attention Signup and view all the answers

What is the purpose of the softmax function in the attention mechanism?

To normalize the output of the attention mechanism Signup and view all the answers

What is the shape of the input sequence to the third attention block?

(3, 4096) Signup and view all the answers

What is the output shape of the fourth attention block?

(4, 4096) Signup and view all the answers