Maths 3.4Lesson outlineLesson 4 of 7 in Module 3
Linear Systems, Rank and Least Squares
Solve `Ax = b` exactly when it can be solved, and in the least-squares sense when it cannot. Derive the normal equations.
Video in production.
Learn
In this lesson11 parts
- 01More equations than unknowns
- 02A system as a matrix equation
- 03A third equation
- 04Span, independence and rank
- 05Least squares as a projection
- 06Solving it by hand
- 07Checkpoint: least squares so far
- 08The hero cars in closed form
- 09Solve, and why libraries skip XᵀX
- 10Least squares in NumPy
- 11Recap, challenge, and next: eigenvectors
Key terms
The words this lesson introduces, each in one line. The module’s glossary collects them all.
- linear system
- equations in which each unknown is only multiplied by a number and added |
x₁ + x₂ = 3 - span
- every combination of a set of vectors | a plane through the origin
- column space
- the span of a matrix's columns, every possible
Ax| - linear independence
- no column is a combination of the others |
(1, 1, 1)and(0, 1, 2) - rank
- the number of independent columns, the dimension of the column space |
2 - residual
b − Ax, what the fit misses |(−0.5, 1, −0.5)- least squares
- the
xthat makes the squared length of the residual smallest |x̂ = (1.5, 0.5) - normal equations
AᵀAx̂ = Aᵀb, from the residual being perpendicular to every column |[[3, 3], [3, 5]] x̂ = (6, 7)
Quiz
Your first pick on each question is the one that counts. Getting one wrong here is how the lesson sticks.