Gradient Descent: how AI models and neural networks learn

In this lesson

1. 0:00Why gradient descent is an iterative algorithmExplain why machine learning often approaches an answer through repeated updates instead of solving for it directly.
2. 2:33Importance of signRead the derivative’s sign to choose a downhill direction without first identifying which side of the minimum you are on.
3. 5:28The update ruleWrite and apply the update rule x = x - r * f′(x) to move toward a lower function value.
4. 8:14The iteration loop, stopping criterion and toleranceTurn repeated updates into a loop and choose a stopping criterion and tolerance suitable for a real computer.
5. 11:06Why the step size shrinks automaticallyUnderstand why update steps shrink as the derivative becomes smaller and examine the factor controlling that shrinkage.
6. 12:18The learning rate hyperparameterExplain the learning rate as a hyperparameter and recognise the effects of choosing it too small or too large.
7. 14:52Oscillation, divergence and learning rate schedulesRecognise oscillation and divergence, and explore how a learning rate schedule can improve an oscillating run.
8. 19:37Gradient vectors, and a loss only gradient descent can solveExtend the update rule to gradient vectors and apply it to a loss without a closed-form answer.