18 Questions
What is the main motivation for using Variational Inference?
Many interesting distributions are too difficult to compute
Explain the concept of Evidence Lower Bound (ELBO).
Using Jensen’s inequality, ELBO is a lower bound used in Variational Inference to approximate the intractable Kullback-Leibler divergence.
How is the Kullback-Leibler divergence related to Variational Inference?
The Kullback-Leibler divergence measures how diverged the approximate distribution is from the true distribution in Variational Inference.
Why is finding the tightest lower bound important in Variational Inference?
Finding the tightest lower bound helps in getting close to the true optimum in approximating the posterior distribution.
What role does the Mean Field Variational Inference play in Variational Inference?
Mean Field Variational Inference posits a factorized approximation and optimizes the variables to approximate the true posterior distribution.
How does Variational Inference differ from Monte Carlo methods?
Variational Inference involves optimizing an approximation to the true distribution, while Monte Carlo methods involve sampling from the true distribution.
What technique can be used to solve the optimization problem in Latent Dirichlet Allocation (LDA)?
Gradient descent
In the context of LDA, why is it challenging to choose priors for the model?
Variable dependencies onto the priors make the problem difficult.
What approach is taken to simplify the problem of choosing priors in LDA?
Make all variables independent and give each its own prior.
What is the key benefit of the variational approximation in LDA?
The variational posterior decomposes nicely due to independence assumptions.
What is the objective function that needs to be optimized in Variational LDA?
Evidence Lower Bound (ELBO)
What function is commonly used as a fast approximation to the digamma function due to its computational efficiency?
Digamma function
What is the main purpose of Gibbs sampling in Markov-Chain-Monte-Carlo (MCMC) methods?
To update variables incrementally by choosing new values more likely given the current state of other variables.
In the context of Markov processes, what does the transition function depend on?
The transition function depends on the probabilities of moving to a new state based only on the previous state.
Why is it common to use only every th sample in estimation processes?
To reduce autocorrelation in the samples.
What is the role of the transition function in Markov processes?
To determine the probabilities of moving to different states based on the current state.
How does Gibbs sampling help estimate hidden variables?
By iteratively updating variables based on conditional distributions and sampling new values.
What is the importance of ergodicity in Markov-Chain-Monte-Carlo (MCMC) methods?
Ergodicity ensures that the chain explores the entire state space and converges to the correct distribution.
Explore the concept of Mean Field Variational Inference for Latent Dirichlet Allocation (LDA) and the challenges in choosing priors due to variable dependencies. Learn about making variables independent, assigning separate priors to each variable, and optimizing the search space for LDA.
Make Your Own Quizzes and Flashcards
Convert your notes into interactive study material.
Get started for free
