ML 3.5Lesson outlineLesson 5 of 7 in Module 3
Why Gradient Descent Is Slow: Conditioning and Feature Scaling
Predict from the data's shape how many updates descent will need, and see why scaling the features fixes it.
Video in production.
Lesson outline: Why Gradient Descent Is Slow: Conditioning and Feature Scaling
In this lesson8 parts
- 01Descent on weight in pounds and model year
- 02Curvature along each direction
- 03The condition number predicts the count
- 04Testing the prediction
- 05Centring, then scaling
- 06Back to the original units
- 07Code
- 08Recap, your challenge, and what comes next
Builds on
This lesson stands on its own. These go deeper into what it uses.
Key terms: Why Gradient Descent Is Slow: Conditioning and Feature Scaling
The words this lesson introduces, each in one line. The module’s glossary collects them all, and the AI glossary has every term across the courses.
- Hessian (reminder, Maths 4.5)
- The matrix of second derivatives; for the mean squared error,
2XᵀX / n. - curvature along a direction
- How fast the slope of the loss changes as one moves that way.
- condition number
- The largest eigenvalue of the Hessian over the smallest, κ.
- ill-conditioned
- Having a large condition number.
- contraction factor
- The number the remaining error is multiplied by at each update.
- centring
- Subtracting each feature's mean.
- standardising
- Centring each feature, then dividing it by its standard deviation.
Quiz: Why Gradient Descent Is Slow: Conditioning and Feature Scaling
Your first pick on each question is the one that counts. Getting one wrong here is how the lesson sticks.