Masked Language Model (MLM) Overview
6 Questions
0 Views

Choose a study mode

Play Quiz
Study Flashcards
Spaced Repetition
Chat to lesson

Podcast

Play an AI-generated podcast conversation about this lesson

Questions and Answers

What is the main training objective of a Masked Language Model (MLM)?

  • To identify the most common tokens in a sentence
  • To predict the tokens based on the previous words only
  • To generate new tokens from the input text
  • To predict the original tokens that were masked out (correct)

    • How does a Masked Language Model (MLM) differ from traditional language models in terms of contextual understanding?

  • MLMs predict masked words based only on their previous tokens
  • MLMs use both preceding and following tokens to predict masked words, unlike traditional models (correct)
  • Traditional models predict words solely based on their position in a sequence
  • Traditional models predict words in a bidirectional manner

    • What special token is commonly used to replace masked words in a Masked Language Model (MLM)?

  • `[FILL]`
  • `[MASK]` (correct)
  • `(HIDE)`
  • ``

    • How does the self-supervised nature of Masked Language Models (MLMs) relate to label generation?

    <p>MLMs generate their own labels from the input text</p> Signup and view all the answers

    What type of context does a Masked Language Model (MLM) leverage to predict masked tokens?

    <p>Bidirectional or non-directional context</p> Signup and view all the answers

    What type of loss function is typically used in Masked Language Models (MLMs) during training?

    <p>Cross-entropy loss function</p> Signup and view all the answers

    More Like This

    Use Quizgecko on...
    Browser
    Open
    Browser
    Browser