singular value decomposition
(1.8 hours to learn)
Summary
The singular value decomposition is a factorization of a matrix A into three matrices UDV^T, where D is diagonal and U and V have orthonormal columns. It's closely related to the eigenvalues and eigenvectors of A^T A and A A^T. It gives a way of analyzing general matrices (not necessarily square) in terms of things somewhat analogous to eigenvalues. Common applications include latent semantic analysis (LSA) and principal component analysis (PCA), a dimensionality reduction algorithm.
Context
This concept has the prerequisites:
- orthonormal bases (The SVD includes orthonormal bases for the row and column spaces.)
- matrix multiplication (The SVD is a matrix factorization.)
- matrix transpose (The SVD includes a transpose.)
- spectral decomposition (The SVD is equivalent to the spectral decomposition of a particular PSD matrix.)
- positive definite matrices (The SVD is equivalent to the spectral decomposition of a particular PSD matrix.)
- four fundamental subspaces (The SVD gives orthonormal bases for the four fundamental subspaces.)
Core resources (read/watch one of the following)
-Free-
→ MIT Open Courseware: Linear Algebra (2011)
Videos for an introductory linear algebra course focusing on numerical methods.
Location:
Lecture "Singular value decomposition"
-Paid-
→ Introduction to Linear Algebra
An introductory linear algebra textbook with an emphasis on numerical methods.
Location:
Section 6.7, "Singular value decomposition (SVD)," pages 363-369
See also
- Principal component analysis (PCA) is a data analysis method based on the SVD.