QR Decomposition
Definition
A=QR where Q has orthonormal columns and R is upper triangular. Used in least squares and eigenvalue algorithms.
Examples
Example 1. Why use QR for least squares?
Solution. Q^TQ=I simplifies normal equations to Rx=Q^Tb.
In Depth
The QR decomposition factors \(A\) (\(m\times n\), \(m\geq n\)) as \(A=QR\) where \(Q\) is \(m\times m\) orthogonal and \(R\) is \(m\times n\) upper triangular. The 'thin' QR has \(Q\) as \(m\times n\) with orthonormal columns and \(R\) as \(n\times n\) upper triangular.
Gram–Schmidt orthogonalization computes QR by successively orthogonalizing columns of \(A\). Modified Gram–Schmidt is numerically more stable. Householder reflections and Givens rotations are the preferred numerical methods, achieving backward stability.
Least squares via QR: \(\min\|A\mathbf{x}-\mathbf{b}\|\) reduces to \(\min\|R\mathbf{x}-Q^T\mathbf{b}\|\), solved by back substitution on the upper triangular system \(R\mathbf{x}=Q^T\mathbf{b}\). This avoids forming \(A^TA\) (which squares the condition number) and is the standard approach in practice.
The QR algorithm for eigenvalues iterates \(A_k=Q_kR_k\), \(A_{k+1}=R_kQ_k\). Each step is an orthogonal similarity transformation, preserving eigenvalues. With shifts, it converges cubically and is the standard algorithm for dense eigenvalue problems (used in LAPACK's \texttt{dgeev}).
QR decomposition is used in signal processing (QR-based adaptive filters), statistics (orthogonal regression), and numerical linear algebra (computing pseudoinverses, solving constrained least-squares problems). Its stability makes it preferable to normal equations in almost all practical applications.
Key Properties & Applications
Householder reflections \(H=I-2\mathbf{v}\mathbf{v}^T/\|\mathbf{v}\|^2\) are orthogonal matrices that reflect across the hyperplane perpendicular to \(\mathbf{v}\). Applying \(n-1\) Householder reflections to \(A\) produces the QR decomposition. This is backward stable and is the standard method in LAPACK.
Givens rotations \(G(i,j,\theta)\) zero out a single element by rotating in the \((i,j)\) plane. They are used for sparse QR (where Householder would destroy sparsity) and for updating QR when a row is added or deleted.
The QR algorithm with shifts converges to Schur form \(A=QTQ^T\) (\(T\) upper triangular, \(Q\) orthogonal). The eigenvalues are the diagonal entries of \(T\). With Francis double-shift, convergence is typically cubic. This is the algorithm used in MATLAB's \texttt{eig} and NumPy's \texttt{linalg.eig}.
Further Reading & Context
The study of qr decomposition connects to many areas of mathematics and its applications. Understanding the foundational definitions and theorems provides the basis for advanced work in analysis, algebra, and applied mathematics.
Historical development: most mathematical concepts evolved over centuries, with contributions from mathematicians across many cultures. The modern axiomatic treatment provides rigor, while computational tools enable practical application.
In modern mathematics, this topic appears in graduate courses and research across pure and applied mathematics. Connections to computer science, physics, and engineering make it a versatile and important area of study. Mastery of the core results and techniques opens doors to research in number theory, analysis, geometry, and beyond.
Recommended next steps: work through the standard theorems with full proofs, explore the connections to related topics listed above, and practice with a variety of problems ranging from computational exercises to theoretical proofs. The interplay between different areas of mathematics is one of the subject's greatest rewards.