Podcast Beta
Questions and Answers
What are some algorithm variants developed to address the slowness of dense matrix multiplications in NMF?
Alternating Least Squares, Projected-gradient ALS, Quasi-Newton optimization, Hierarchical Alternating Least Squares, and more
What is the probabilistic version of LSI/LSA called and what kind of model does it use?
Probabilistic Latent Semantic Indexing (PLSI); graphical mixture model
According to GaGo05, what relationship exists between PLSA and NMF?
Any (local) maximum likelihood solution of PLSA is a solution of NMF with KL divergence.
What is the main focus of the theoretical analysis by Donoho and Stodden regarding NMF?
Signup and view all the answers
What are the two variants of Non-negative Matrix Factorization (NMF) and what do they aim to minimize?
Signup and view all the answers
What is the initialization step in the Gradient descent method for squared error in NMF?
Signup and view all the answers
What is the purpose of maximizing in the Gradient descent method for squared error in NMF?
Signup and view all the answers
What should be repeated until the improvement of is below a threshold in the Gradient descent method for squared error?
Signup and view all the answers
What is a common issue with the Gradient descent method for squared error in NMF?
Signup and view all the answers
Why is it common to add a small constant to all values in NMF?
Signup and view all the answers
Study Notes
Non-Negative Matrix Factorization (NMF)
- NMF aims to find two non-negative matrix factors, W and H, from a given data matrix X, such that X ≈ WH.
- There are two variants of NMF: Squared Error and Divergence (KL-Divergence).
Squared Error NMF
- Minimizes the squared error between X and WH.
- Uses a gradient descent method.
- The Multiplicative Update method is used for minimizing the squared error.
- The algorithm iterates between maximizing W and maximizing H until the improvement of the objective function is below a threshold.
KL-Divergence NMF
- Minimizes the KL-Divergence between X and WH.
- Uses a gradient descent method.
- The Multiplicative Update method is used for minimizing the KL-Divergence.
- The algorithm iterates between maximizing W and maximizing H until the improvement of the objective function is below a threshold.
Limitations of NMF
- NMF may get stuck in a local fixpoint or saddlepoint, and does not guarantee to find the optimum.
- Both W and H must be non-negative, so a small constant is often added to all values.
Other NMF Algorithms
- Alternating Least Squares (ALS) [PaTa94]
- Projected-gradient ALS [Lin07]
- Quasi-Newton optimization [ZdCi06]
- Hierarchical Alternating Least Squares [CiPh09]
- Initialization methods: spherical k-means, SVD [BBLP07], and others [LMAC14]
- Beta divergence [FéId11]
- Non-negative tensor factorization [CiPh09]
Relation to pLSI
- pLSI (Probabilistic Latent Semantic Indexing) is a probabilistic version of LSI/LSA.
- pLSI uses a graphical mixture model with word probability, topic probability, and non-negative probabilities.
- Any (local) maximum likelihood solution of PLSA is a solution of NMF with KL divergence.
Theoretical Analysis
- David L. Donoho and Victoria Stodden have analyzed NMF theoretically.
Studying That Suits You
Use AI to generate personalized quizzes and flashcards to suit your learning preferences.
Related Documents
Description
Test your knowledge on Non-Negative Matrix Factorization (NMF) and NNMF. Understand the parameter optimization process to find two matrices that minimize squared error or divergence. Learn about the Multiplicative Update and Gradient Descent methods for squared error.
