ML 3.2Free videoLesson 2 of 7 in Module 3
Gradient Descent
Find the minimum of a loss without solving equations, by stepping against the gradient, and tell from the loss curve whether a run converges, crawls or diverges.
Watch the lesson: Gradient Descent
In this lesson9 parts
- 01A loss with no formula
- 02One parameter, then two
- 03Searching a grid
- 04Stepping against the derivative
- 05The learning rate decides
- 06Two parameters on the cars
- 07Just above the limit
- 08The descent loop in NumPy
- 09Recap, your challenge, and what comes next
Builds on
This lesson stands on its own. These go deeper into what it uses.
Key terms: Gradient Descent
The words this lesson introduces, each in one line. The module’s glossary collects them all, and the AI glossary has every term across the courses.
- gradient descent
- Repeatedly stepping the parameters against the gradient of the loss.
- update rule
w ← w − η · L′(w).- learning rate
- The hyperparameter that scales each step, η.
- iteration
- One application of the update rule; also called an update.
- convergence
- The parameters settling at the minimum.
- divergence
- Each step landing further from the minimum.
- tolerance
- How small the gradient must be before descent stops.
- loss curve
- The loss plotted after every update.
- local minimum (narration)
- A low point that is not the lowest, where descent can end on a loss with several.
- learning-rate schedule
- A rule that changes the learning rate during training.
Quiz: Gradient Descent
Your first pick on each question is the one that counts. Getting one wrong here is how the lesson sticks.