Introduction to Matrix Factorization and SVD PDF
Document Details
Uploaded by ThrillingTuba
Tags
Summary
This document introduces matrix factorization techniques, with a focus on Singular Value Decomposition (SVD). It explains how SVD is used in various data analysis tasks, including PCA, multidimensional scaling, and spectral clustering. The document also touches on the optimization problems related to SVD and its limitations.
Full Transcript
Motivation We see matrix factorizations repeatedly in data analysis: ▶ PCA and SVD in linear algebra (other courses) ▶ Multidimensional scaling (MDS, statistics) ▶ Spectral clustering (other courses) ▶ Latent Semantic Indexing ▶ Word embeddings But the factors we get usually also include negative te...
Motivation We see matrix factorizations repeatedly in data analysis: ▶ PCA and SVD in linear algebra (other courses) ▶ Multidimensional scaling (MDS, statistics) ▶ Spectral clustering (other courses) ▶ Latent Semantic Indexing ▶ Word embeddings But the factors we get usually also include negative terms, that are hard to interpret. Furthermore scalability is a problem – we need this on huge matrices. 3 Matrix Factorization Objective Given a data matrix the parameter with find two matrixes , and , such that Allows different variants: ▶ precise definition of approximating the matrix ( )? ▶ additional requirements for and ? 4 SVD is a Matrix Factorization SVD allows us to decompose any matrix where is an orthogonal is a diagonal is an orthogonal Truncated SVD: , matrix, matrix, matrix. , into additional constraints by computing the first Can be interpreted as two factors by simply using, e.g., , rows only. or , 6 Singular Value Decomposition: Optimization Problem With eigenvalues where , , in descending order, we have for any : are the matrixes , , truncated to columns. With the approximations and the residuals ➜ SVD gives us least-squares best approximations of rank. 7
