Language Modelling Lecture 1 Quiz

Language Modelling

• Language modelling has applications in machine translation, speech recognition, spell checking, natural language generation, and more.

Introduction to Lecture

• The lecturer mentions having a warm conversation with the class, which includes "amazing ladies and gentlemen".
• The lecturer had a delicious tea and mentions the weather (large winds tonight) and a popular book (The tail of two cities).

Real-World Examples of Language Modelling

• Google autocompletion is a real-world example of language modelling.
• Google Translate is another example of language modelling.

Natural Language Generation

• An example of natural language generation can be found at https://pdos.csail.mit.edu/archive/scigen/#generate.

Modelling Text with Multinomial Distribution

• Multinomial models the outcome of a single event out of K possibilities.
• The parameter of a multinomial model is the probability of each possibility.
• In the context of text data, K represents the vocabulary size.
• At each position in the text, one of the words in the vocabulary appears.

Maximum Likelihood Solution

• Assume word i appeared mi times, and let the total number of words in a document be N.
• The maximum likelihood solution is calculated based on the probability of each word.

Probabilistic Modelling

• Probabilistic modelling is a key concept in language modelling.
• Conditional probabilities are used in language modelling.

Language Modelling with N-Grams

• N-Grams are used in language modelling.
• Conditionals are an important aspect of language modelling with N-Grams.