Regression Analysis

By SkytrailGroup Mathematics Editorial Team

Simple Linear Regression

\[y=\beta_0+\beta_1 x+\varepsilon\]

OLS Estimates

PropertyStatement
Slope\(\hat{{\beta}}_1=\frac{{\sum(x_i-\bar{{x}})(y_i-\bar{{y}})}}{{\sum(x_i-\bar{{x}})^2}}\)
Intercept\(\hat{{\beta}}_0=\bar{{y}}-\hat{{\beta}}_1\bar{{x}}\)

Goodness of Fit

R² = SSR/SST measures the proportion of variance in y explained by x. Ranges from 0 to 1.

Examples

Example 1. x=[1,2,3], y=[2,4,5]. Find the regression line.

Solution. x̄=2, ȳ=11/3. β̂₁=(1·(−1/3)+0·(1/3)+1·(2/3))/2 = 1.5. β̂₀ = 11/3−1.5·2 = 0.67. ŷ = 0.67+1.5x.

Deep Dive: Regression Analysis

This section builds durable understanding of regression analysis in statistics through definition-first reasoning, theorem mapping, and error-checking workflows.

Use a two-pass method: first derive the structure symbolically, then validate with a concrete numerical or geometric test case.

Visual Intuition

Convert algebra into a diagram, graph, or dependency map before solving. Visual-first analysis reduces sign errors and makes assumptions explicit.

Checklist: domain constraints - symmetry - limiting behavior - sanity check at special values.

Practice Set

Practice A. Re-derive one key formula on this page from first principles and annotate each transformation.

Target. Your final line should include assumptions, derivation path, and a quick verification.

Practice B. Build an application scenario using regression analysis and solve it with both symbolic and numeric methods.

Target. Compare outputs and explain any approximation gap.

References & Editorial Notes

  • Stewart, Calculus.
  • Strang, Introduction to Linear Algebra.
  • Apostol, Mathematical Analysis.

Editorial update: Reviewed on 2026-04-14 for notation consistency, conceptual clarity, and exercise quality.