Orthogonality
Key Concepts
| Property | Statement |
|---|---|
| Orthogonal | ⟨u,v⟩=0 |
| Projection | \(\text{proj}_v u=\frac{\langle u,v\rangle}{\langle v,v\rangle}v\) |
| Complement | W⊥ = all vectors orthogonal to W |
Examples
Example 1. Project (3,4) onto (1,0).
Solution. proj=(3,0).
In Depth
Two vectors are orthogonal if their inner product is zero. A set of vectors is orthogonal if every pair is orthogonal; orthonormal if additionally each vector has unit norm. Orthonormal bases simplify computations enormously: coordinates are inner products, and the matrix of an orthonormal basis is orthogonal (\(Q^TQ=I\)).
The orthogonal complement of a subspace \(W\) is \(W^\perp=\{v:\langle v,w\rangle=0\text{ for all }w\in W\}\). Every vector decomposes uniquely as \(v=w+w^\perp\) with \(w\in W\) and \(w^\perp\in W^\perp\). This orthogonal decomposition is the basis of projection and least squares.
Orthogonal matrices (\(Q^TQ=QQ^T=I\)) preserve lengths and angles: \(\|Q\mathbf{v}\|=\|\mathbf{v}\|\) and \(\langle Q\mathbf{u},Q\mathbf{v}\rangle=\langle\mathbf{u},\mathbf{v}\rangle\). They represent rotations and reflections. Their eigenvalues lie on the unit circle in \(\mathbb{C}\).
The QR decomposition \(A=QR\) (\(Q\) orthogonal, \(R\) upper triangular) is computed by Gram–Schmidt or Householder reflections. It is numerically stable and used for solving least-squares problems, computing eigenvalues (QR algorithm), and solving linear systems.
Fourier series express a function as an infinite sum of orthogonal sinusoids: \(f(x)=\sum_{n=-\infty}^\infty c_n e^{inx}\) where \(c_n=\frac{1}{2\pi}\int_{-\pi}^\pi f(x)e^{-inx}dx\). The exponentials \(e^{inx}\) form an orthonormal basis of \(L^2[-\pi,\pi]\), making Fourier analysis a special case of orthogonal decomposition.
Key Properties & Applications
The projection matrix \(P=A(A^TA)^{-1}A^T\) projects onto the column space of \(A\). It satisfies \(P^2=P\) (idempotent) and \(P^T=P\) (symmetric). The complementary projection \(I-P\) projects onto the orthogonal complement (the left null space of \(A\)).
Orthogonal polynomials (Legendre, Chebyshev, Hermite, Laguerre) are orthogonal with respect to specific inner products on function spaces. They are used in numerical integration (Gaussian quadrature), approximation theory, and quantum mechanics (Hermite polynomials for the harmonic oscillator).
The discrete Fourier transform (DFT) matrix \(F_{jk}=\omega^{jk}/\sqrt{n}\) (\(\omega=e^{2\pi i/n}\)) is unitary: \(F^*F=I\). Its columns are orthonormal vectors. The FFT computes the DFT in \(O(n\log n)\) operations by exploiting the recursive structure of the DFT matrix.
Further Reading & Context
The study of orthogonality connects to many areas of mathematics and its applications. Understanding the foundational definitions and theorems provides the basis for advanced work in analysis, algebra, and applied mathematics.
Historical development: most mathematical concepts evolved over centuries, with contributions from mathematicians across many cultures. The modern axiomatic treatment provides rigor, while computational tools enable practical application.
In modern mathematics, this topic appears in graduate courses and research across pure and applied mathematics. Connections to computer science, physics, and engineering make it a versatile and important area of study. Mastery of the core results and techniques opens doors to research in number theory, analysis, geometry, and beyond.
Recommended next steps: work through the standard theorems with full proofs, explore the connections to related topics listed above, and practice with a variety of problems ranging from computational exercises to theoretical proofs. The interplay between different areas of mathematics is one of the subject's greatest rewards.