Maths 4.8Lesson outlineLesson 8 of 9 in Module 4
The Chain Rule Through a Computation
Compute the gradient of a loss that is built from many steps, by passing derivatives backwards through a graph of operations.
Video in production.
Learn
In this lesson10 parts
- 01A gradient through many steps
- 02A computation graph
- 03The backward pass
- 04Reuse
- 05Vector gradients and the shape check
- 06Checkpoint: the backward pass so far
- 07Logistic regression's log loss, one patient
- 08All four hundred patients
- 09The backward pass in Python
- 10Recap, challenge, and next: constraints
Key terms
The words this lesson introduces, each in one line. The module’s glossary collects them all.
- computation graph
- a calculation drawn as nodes, one operation each, joined by the values they pass |
u = 3a,v = u + b,f = v² - forward pass
- compute each node's value from its inputs, in order |
3,5,25 - local derivative
- the derivative of one node with respect to its own input |
∂f/∂v = 2v = 10 - backward pass
- start from the output and multiply local derivatives back along each edge |
∂f/∂a = 10 · 1 · 3 = 30 - backpropagation
- the backward pass on a computation graph, each local derivative used once | how deep learning libraries compute gradients
Quiz
Your first pick on each question is the one that counts, and a right one earns a coin. Getting one wrong here is how the lesson sticks.