Podcast
Play an AI-generated podcast conversation about this lesson
Questions and Answers
What is the main training objective of a Masked Language Model (MLM)?
What is the main training objective of a Masked Language Model (MLM)?
- To identify the most common tokens in a sentence
- To predict the tokens based on the previous words only
- To generate new tokens from the input text
- To predict the original tokens that were masked out (correct)
How does a Masked Language Model (MLM) differ from traditional language models in terms of contextual understanding?
How does a Masked Language Model (MLM) differ from traditional language models in terms of contextual understanding?
- MLMs predict masked words based only on their previous tokens
- MLMs use both preceding and following tokens to predict masked words, unlike traditional models (correct)
- Traditional models predict words solely based on their position in a sequence
- Traditional models predict words in a bidirectional manner
What special token is commonly used to replace masked words in a Masked Language Model (MLM)?
What special token is commonly used to replace masked words in a Masked Language Model (MLM)?
- `[FILL]`
- `[MASK]` (correct)
- `(HIDE)`
- ``
How does the self-supervised nature of Masked Language Models (MLMs) relate to label generation?
How does the self-supervised nature of Masked Language Models (MLMs) relate to label generation?
Signup and view all the answers
What type of context does a Masked Language Model (MLM) leverage to predict masked tokens?
What type of context does a Masked Language Model (MLM) leverage to predict masked tokens?
Signup and view all the answers
What type of loss function is typically used in Masked Language Models (MLMs) during training?
What type of loss function is typically used in Masked Language Models (MLMs) during training?
Signup and view all the answers
