ML 4.8Lesson outlineLesson 8 of 9 in Module 4
Ridge Regression
Penalise the size of the weights to trade a little bias for less variance, and choose the penalty's strength by cross-validation.
Video in production.
Lesson outline: Ridge Regression
In this lesson12 parts
- 01Weights that blow up
- 02Eight features, forty four columns
- 03Least squares on all forty four
- 04The same linear model
- 05Weights, and a strength
- 06Squared error plus a penalty
- 07The closed form, and why it always exists
- 08Predicting
- 09Choosing alpha
- 10Where ridge misleads
- 11Code
- 12Recap, your challenge, and what comes next
Builds on
This lesson stands on its own. These go deeper into what it uses.
Key terms: Ridge Regression
The words this lesson introduces, each in one line. The module’s glossary collects them all, and the AI glossary has every term across the courses.
- bias, variance (reminder, ML 4.3)
- The two parts of the error that the penalty trades.
- ridge regression
- Linear regression with a penalty on the squared size of the weights.
- penalty strength
- The hyperparameter that sets how much the penalty counts:
alphain scikit-learn, λ in papers. - regularization
- Adding a penalty on the weights to the objective.
- L2 penalty
- The sum of squared weights.
- L2 norm (reminder, Maths 2.2)
- The square root of the sum of squares.
- shrinkage
- Pulling the weights toward zero.
- coefficient path
- Every weight plotted against the penalty strength.
- collinearity (reminder, ML 3.7)
- Features that move together.
Quiz: Ridge Regression
Your first pick on each question is the one that counts. Getting one wrong here is how the lesson sticks.