18-Archetypal-Analysis-and-MF-Conclusions.pdf

Archetypal Analysis [CuBr94] Inspired by -means, represent points not by means, but by extreme examples (“archetypes”) where with the archetypes in. The additional constraints mean: ▶ every archetype is a convex combination of existing data points ▶ every data point is a convex combination of archet...

Archetypal Analysis [CuBr94] Inspired by -means, represent points not by means, but by extreme examples (“archetypes”) where with the archetypes in. The additional constraints mean: ▶ every archetype is a convex combination of existing data points ▶ every data point is a convex combination of archetypes Optimal archetypes would be the convex hull of the data points. I.e., archetypes approximate the convex hull of a data set with points. Can also be optimized using the Frank-Wolfe algorithm [BKHT15]. 24 Conclusions: Matrix Factorization Matrix factorization is a very commonplace data analysis technique. ▶ implicitly or explicitly used inside many algorithms e.g., PCA, SVD, LSA, pLSI, -means, spectral clustering, word embeddings, NMF, … ▶ with squared error, SVD is optimal (and hence better than NMF) but SVD may use negative values, NMF only non negative values ▶ non-negativity constraints can make the result more interpretable (in particular for text) ▶ do not use as a black box – choose constraints to suit your needs ▶ not convex: optimization does not necessarily find the global optimum may not have a unique answer – regularization may be helpful ▶ theoretically guaranteed convergence exists, but is much slower 32