# singular value decomposition

(1.8 hours to learn)

## Summary

The singular value decomposition is a factorization of a matrix A into three matrices UDV^T, where D is diagonal and U and V have orthonormal columns. It's closely related to the eigenvalues and eigenvectors of A^T A and A A^T. It gives a way of analyzing general matrices (not necessarily square) in terms of things somewhat analogous to eigenvalues. Common applications include latent semantic analysis (LSA) and principal component analysis (PCA), a dimensionality reduction algorithm.

## Context

This concept has the prerequisites:

- orthonormal bases (The SVD includes orthonormal bases for the row and column spaces.)
- matrix multiplication (The SVD is a matrix factorization.)
- matrix transpose (The SVD includes a transpose.)
- spectral decomposition (The SVD is equivalent to the spectral decomposition of a particular PSD matrix.)
- positive definite matrices (The SVD is equivalent to the spectral decomposition of a particular PSD matrix.)
- four fundamental subspaces (The SVD gives orthonormal bases for the four fundamental subspaces.)

## Core resources (read/watch one of the following)

## -Free-

→ MIT Open Courseware: Linear Algebra (2011)

Videos for an introductory linear algebra course focusing on numerical methods.

Location:
Lecture "Singular value decomposition"

## -Paid-

→ Introduction to Linear Algebra

An introductory linear algebra textbook with an emphasis on numerical methods.

Location:
Section 6.7, "Singular value decomposition (SVD)," pages 363-369

## See also

- Principal component analysis (PCA) is a data analysis method based on the SVD.