Perplexity in NLP Quiz and Flashcards: Understanding N-gram

What is the main purpose of Name Entity Recognition (NER)?

How does Masked Language Modelling help learners?

What does pragmatic analysis in NLP primarily focus on?

In NLP, what does perplexity refer to?

What is the primary function of entity chunking in Name Entity Recognition?

How does Pragmatic Analysis contribute to understanding language?

In the context of NLP, what role does Perplexity play in language models?

What aspect of language does Pragmatic Analysis focus on?

"Chunking" entities in NER involves:

Perplexity in NLP

Perplexity measures the uncertainty in predicting text in NLP
It is a way to evaluate language models
Low perplexity is desirable, indicating less difficulty in handling complicated problems
High perplexity is undesirable, indicating a high failure rate in handling complicated problems

N-gram in NLP

An n-gram is a sequence of n words
It helps identify sentences that appear more frequently
Assigning probability to n-gram occurrences can aid in next-word predictions and spelling error corrections

Differences between AI, Machine Learning, and NLP

Not provided in the given text (question without answer)

Self-Attention

Self-attention is not a supervised learning technique
It is a powerful tool in NLP

LDA (Latent Dirichlet Allocation)

LDA is an unsupervised learning model
It is used for topic modeling
The selection of the number of topics in LDA depends on the size of the data
The number of topic terms is not directly proportional to the size of the data

Hyperparameters in LDA

Alpha (α) represents the density of topics generated within documents
Beta (β) represents the density of terms generated within topics

Issues with ReLu

Exploding gradient: solved by gradient clipping
Dying ReLu: solved by parametric ReLu
Mean and variance of activations are not 0 and 1: partially solved by subtracting around 0.5 from activation

This quiz covers the concept of perplexity in NLP, which is a measure of uncertainty in predicting text, and explains the significance of high and low perplexity. It also explores the definition of n-gram in NLP as a sequence of n words commonly used in language modeling.