singular value decomposition
(1.8 hours to learn)
The singular value decomposition is a factorization of a matrix A into three matrices UDV^T, where D is diagonal and U and V have orthonormal columns. It's closely related to the eigenvalues and eigenvectors of A^T A and A A^T. It gives a way of analyzing general matrices (not necessarily square) in terms of things somewhat analogous to eigenvalues. Common applications include latent semantic analysis (LSA) and principal component analysis (PCA), a dimensionality reduction algorithm.
This concept has the prerequisites:
- orthonormal bases (The SVD includes orthonormal bases for the row and column spaces.)
- matrix multiplication (The SVD is a matrix factorization.)
- matrix transpose (The SVD includes a transpose.)
- spectral decomposition (The SVD is equivalent to the spectral decomposition of a particular PSD matrix.)
- positive definite matrices (The SVD is equivalent to the spectral decomposition of a particular PSD matrix.)
- four fundamental subspaces (The SVD gives orthonormal bases for the four fundamental subspaces.)
Core resources (read/watch one of the following)
→ MIT Open Courseware: Linear Algebra (2011)
Videos for an introductory linear algebra course focusing on numerical methods.
Location: Lecture "Singular value decomposition"
→ Introduction to Linear Algebra
An introductory linear algebra textbook with an emphasis on numerical methods.
Location: Section 6.7, "Singular value decomposition (SVD)," pages 363-369
- Principal component analysis (PCA) is a data analysis method based on the SVD.
- create concept: shift + click on graph
- change concept title: shift + click on existing concept
- link together concepts: shift + click drag from one concept to another
- remove concept from graph: click on concept then press delete/backspace
- add associated content to concept: click the small circle that appears on the node when hovering over it
- other actions: use the icons in the upper right corner to optimize the graph placement, preview the graph, or download a json representation