singular value decomposition
1. complex
An \(m \times n\) matrix \(A\) can be written as \(U\Sigma V*\) where
- \(U\) is \(m \times m\)
- \(\Sigma\) is \(\m times n\)
- \(V\) is \(n \times n\)
- \(U\) and \(V\) are unitary matrices
- \(\Sigma\) only has non-zero values on the diagonal. These values are all non-negative.
2. real
If \(A\) is real then
- \(U\) and \(V\) are orthogonal matrices
3. uniqueness
- The SVD is not unique. At the very least, the singular values (and corresponding vectors) can be permuted. (TODO: Why?)
- Moreover, think about eigendecompositions with repeated eigenvalues. You can choose any basis you want for the corresponding eigenspaces. Somthing similar happens for repeated singular values (see stack overflow).
4. relation to eigendecomposition
- see also diagonalizable matrix
- Note that for an eigendecomposition \(A = PDP^{-1}\), the eigenvectors of \(P\) do not need to be orthogonal, unless it is a symmetric matrix (see diagonalizable matrix)
- Also, the eigenvalues do not have to be non-zero unless \(A\) is positive semidefinite (see positive semidefinite)
- So, for the case of a positive semidefinite (which is also symmetric by definition) matrix, the eigendecomposition works as an SVD