Singular Value Decomposition
Definition
U,V orthogonal; Σ diagonal with singular values σ₁≥σ₂≥…≥0. Generalizes eigendecomposition to rectangular matrices.
Examples
Example 1. What do singular values represent?
Solution. The 'stretching factors' of the linear transformation A.
In Depth
SVD is the most important matrix factorization in applied mathematics. Every \(m\times n\) matrix \(A\) has an SVD \(A=U\Sigma V^T\) where \(U\) (\(m\times m\)) and \(V\) (\(n\times n\)) are orthogonal and \(\Sigma\) (\(m\times n\)) is diagonal with non-negative entries \(\sigma_1\geq\sigma_2\geq\ldots\geq0\).
The rank-\(k\) truncated SVD \(A_k=U_k\Sigma_k V_k^T\) is the best rank-\(k\) approximation to \(A\) in both the 2-norm and Frobenius norm (Eckart–Young theorem). This is the mathematical basis of image compression, latent semantic analysis, and collaborative filtering (Netflix-style recommendation).
The condition number \(\kappa(A)=\sigma_1/\sigma_n\) measures sensitivity of the linear system \(Ax=b\) to perturbations. A large condition number means the system is ill-conditioned — small errors in \(b\) can cause large errors in \(x\). SVD-based solvers (pseudoinverse) handle rank-deficient and ill-conditioned systems robustly.
Further Reading & Context
The study of singular value decomposition connects to many areas of mathematics and its applications. Understanding the foundational definitions and theorems provides the basis for advanced work in analysis, algebra, and applied mathematics.
Historical development: most mathematical concepts evolved over centuries, with contributions from mathematicians across many cultures. The modern axiomatic treatment provides rigor, while computational tools enable practical application.
In modern mathematics, this topic appears in graduate courses and research across pure and applied mathematics. Connections to computer science, physics, and engineering make it a versatile and important area of study. Mastery of the core results and techniques opens doors to research in number theory, analysis, geometry, and beyond.
Recommended next steps: work through the standard theorems with full proofs, explore the connections to related topics listed above, and practice with a variety of problems ranging from computational exercises to theoretical proofs. The interplay between different areas of mathematics is one of the subject's greatest rewards.