Day 21: Computational graphs and the backpropagation connection (review)

Forward: n1 = 6, n2 = 7, L = 49. Backward: dL/dn2 = 14; dL/dn1 = 14, dL/dc = 14; dL/da = 14*3 = 42; dL/db = 14*2 = 28. Finite differences ((L(2+h,3,1) - L(2-h,3,1)) / 2h with h = 1e-5, and likewise for b, c) return about 42.0000, 28.0000, 14.0000 — the same numbers a micrograd-style engine would report for .grad on each input.

(a) Two paths through the shared u: each contributes u = 3, total dL/da = 2 * 3 = 6, matching the direct derivative of (a + 1)^2 = 2(a + 1). A single-path count yields 3 — off by exactly one path, the reusable-variable counterexample. (b) dw/dx = 2(u + v)(du/dx + dv/dx) = 2 * 3 * (2 + 2) = 24 at x = 1; expanding (x^2 + 2x)^2 = x^4 + 4x^3 + 4x^2 and differentiating termwise gives 4x^3 + 12x^2 + 8x = 24 at x = 1 - the same, and w = 9 at x = 1. Finite differences confirm about 6.0000 and 24.0000.

The harness prints approximately 24.0000, 135.0000, 40.0000, 3.0000 and 7.0000, a gradient (6.0000, 8.0000) with norm 10.0000, and 42.0000, 28.0000, 14.0000 — every hand result from the block confirmed numerically. Retrieval answers: derivative = limit of (f(x+h) - f(x))/h as h -> 0; chain rule dy/dx = (dy/du)(du/dx); partial = ordinary derivative with other variables frozen; the gradient is the steepest-ascent direction whose norm equals that maximum slope; a shared variable's total derivative sums one product per dependency path.

Part 1: n1 = 2, n2 = 4, L = 7; backward dL/dn2 = 1, dL/dn1 = 4, so dL/da = 8, dL/db = 4, dL/dc = 1; finite differences return about 8.0000, 4.0000, 1.0000. Part 2: the derivative does not exist at 0 (left 0 vs right 1); engines pick a convention, often 0, which is an engineering choice rather than a mathematical derivative — treated inputs at exactly 0 deserve attention. Part 3: a passing plan names each missed item and the day to revisit, e.g., kink cases -> Day 19-20 edge activities, path sums -> Day 20 act-03, graph passes -> Day 21 act-01.