12 Questions
What are some typical features used in natural language processing?
Word at the current position, previous words, next words, prefix or suffix of the current word, position in the sentence, lowercase, Capitalized, ALLCAPS, CamelCase, Numbers, M1xed, contains a hyphen, number, word shape
What is the feature function in Natural Language Processing?
Feature functions are binary functions
What is the role of Maximum Entropy Markov Models (MEMM) in NLP?
MEMM is a log-linear model with feature weights that help in optimization by constraining the expectation of each feature for consistency
What is the Label Bias Problem in NLP?
It refers to the bias in labeling due to the average outgoing weight of smaller labels
What is the solution to the Label Bias Problem?
Global normalization and CRF are solutions to the Label Bias Problem
In the Hidden Markov Model, which probability do we optimize?
joint probability
How is the Maximum Entropy Model optimized to satisfy constraints?
It is optimized by choosing the model that satisfies constraints with Generalized Iterative Scaling (GIS) and smoothing to reduce overfitting
What do we optimize directly with Maximum Entropy Taggers?
conditional probability
Why is fitting the states to the observed sentence in Unsupervised Hidden Markov Models insufficient for most applications?
states could be predefined, transitions need to be learned, test data may have unknown words, words are not independent
How do we handle rare and unknown words in the context of state transitions?
treat rare words like unknown words
What is one idea mentioned for handling rare words in state transitions?
use the total average transitions
How can we infer the most likely type of an unknown word based on word similarity?
uppercase likely a noun
Explore the concepts of conditional probability and joint probability in Hidden Markov Models. Understand the differences in maximizing conditional probability and joint probability given observations. Learn how Maximum Entropy Taggers optimize conditional probability directly in Hidden Markov Models.
Make Your Own Quizzes and Flashcards
Convert your notes into interactive study material.
Get started for free
