Gradient Descent for Linear Regression step by step

In this lesson

1. 0:00The loss function L(w), and why w is the variableWrite the linear regression loss as a function of the model parameters, hold the data fixed, and differentiate the loss to obtain the updates.
2. 6:52The cost of one update: a full pass over the dataUnderstand why a full gradient update visits every data row and how that determines the cost of training.
3. 8:39Stochastic gradient descent (SGD) and the batch sizeIntroduce stochastic gradient descent and batch size, explain why batches are redrawn, and examine why disagreeing per-row gradients can still guide learning.
4. 14:41Implementing SGD, and scikit-learn's SGDRegressorImplement SGD in Python, compare it with scikit-learn’s SGDRegressor, and interpret small differences between their results.
5. 18:42Summary, local minima, and solving directly insteadReview what gradient descent can infer from a local derivative, what multiple minima mean for training, and when solving directly is an alternative.