LU Decomposition
Definition
A=LU where L is lower triangular (1s on diagonal) and U is upper triangular. Used for efficient solving of Ax=b.
Examples
Example 1. LU of [[2,1],[4,3]].
Solution. L=[[1,0],[2,1]], U=[[2,1],[0,1]].
In Depth
LU decomposition factors a matrix as \(A=LU\) where \(L\) is lower triangular (with 1s on the diagonal) and \(U\) is upper triangular. It is essentially Gaussian elimination recorded in matrix form. Solving \(A\mathbf{x}=\mathbf{b}\) reduces to two triangular solves: \(L\mathbf{y}=\mathbf{b}\) then \(U\mathbf{x}=\mathbf{y}\).
Partial pivoting (PA=LU) permutes rows to place the largest element in the pivot position, improving numerical stability. Complete pivoting (PAQ=LU) also permutes columns. In practice, partial pivoting is almost always sufficient and is the default in LAPACK and NumPy.
LU decomposition costs \(O(n^3/3)\) flops — the same as Gaussian elimination. Once computed, each solve costs only \(O(n^2)\). This makes LU ideal when solving \(A\mathbf{x}=\mathbf{b}\) for many right-hand sides \(\mathbf{b}\) with the same \(A\).
The Cholesky decomposition \(A=LL^T\) applies when \(A\) is symmetric positive definite. It is twice as fast as LU and numerically more stable. It is used in statistics (sampling from multivariate normals), optimization (Newton's method), and finite element analysis.
Block LU decomposition handles structured matrices efficiently. The Schur complement \(S=D-CA^{-1}B\) appears in the block factorization of \([[A,B],[C,D]]\) and is fundamental in control theory, statistics (conditional distributions), and the analysis of bordered systems.
Key Properties & Applications
The LU decomposition is used to compute determinants efficiently: \(\det(A)=\det(L)\det(U)=\prod u_{ii}\) (since \(\det(L)=1\) for unit lower triangular \(L\)). With partial pivoting, \(\det(A)=\pm\prod u_{ii}\) (sign depends on the permutation).
Sparse LU decomposition is used for large sparse systems arising in finite element analysis, circuit simulation, and computational fluid dynamics. Fill-reducing orderings (AMD, METIS) minimize the number of nonzeros introduced during factorization.
The LDL\(^T\) decomposition factors a symmetric matrix as \(A=LDL^T\) with unit lower triangular \(L\) and diagonal \(D\). It is more efficient than Cholesky for indefinite matrices and is used in interior-point optimization methods.
Further Reading & Context
The study of lu decomposition connects to many areas of mathematics and its applications. Understanding the foundational definitions and theorems provides the basis for advanced work in analysis, algebra, and applied mathematics.
Historical development: most mathematical concepts evolved over centuries, with contributions from mathematicians across many cultures. The modern axiomatic treatment provides rigor, while computational tools enable practical application.
In modern mathematics, this topic appears in graduate courses and research across pure and applied mathematics. Connections to computer science, physics, and engineering make it a versatile and important area of study. Mastery of the core results and techniques opens doors to research in number theory, analysis, geometry, and beyond.
Recommended next steps: work through the standard theorems with full proofs, explore the connections to related topics listed above, and practice with a variety of problems ranging from computational exercises to theoretical proofs. The interplay between different areas of mathematics is one of the subject's greatest rewards.