第 36 日：訓練循環、學習率與工程故障陳列室

操作指引

先預測再運行。每個 epoch 在樣本循環前清空參數梯度，每個樣本新建前向計算圖、計算平方 loss 並 backward 累加梯度；全部樣本之後 step 一次。zero_grad 只要在本批首次反傳前即可，下面將其放在 epoch 開頭。四樣本使用活動36-02的同一初始參數和數據，初始平均損失為2.106。

for epoch in range(EPOCHS):
    for p in model.parameters():
        p.grad = 0.0                 # 1. zero stale gradients
    total = 0.0
    for (x, y) in dataset:
        out = model(x)               # 2. forward
        loss = (out - y) ** 2        # 3. loss
        loss.backward()              # 4. backward (accumulates batch grads)
        total += loss.data
    for p in model.parameters():
        p.data -= lr * p.grad        # 5. step along -gradient
    log(epoch, total / len(dataset))

操作指引

在倉庫根目錄執行以下命令。兩次運行恢復相同九個初始權重與四個樣本。每個epoch重新建立參數葉子，每個樣本重建內部圖；四個梯度累加後更新。日誌記錄更新前的均值損失；更新用梯度之和，不是均值，改變約定會改變有效學習率。

python3 - <<'EOF'
from projects.autograd.engine import Value
initial = [0.5, -0.25, 0.1, -0.5, 1.0, -0.2, 1.0, -2.0, 0.3]
data = [((2.0,-1.0),1.0), ((0.5,0.5),-1.0), ((-1.0,1.0),-1.0), ((1.0,1.0),1.0)]
for lr in (0.05, 5.0):
    params = initial[:]
    for epoch in range(40):
        W = [Value(v) for v in params]  # Fresh leaves start with zero gradients.
        total = 0.0
        for (x1, x2), y in data:
            a1 = (W[0]*x1 + W[1]*x2 + W[2]).tanh()
            a2 = (W[3]*x1 + W[4]*x2 + W[5]).tanh()
            loss = (a1*W[6] + a2*W[7] + W[8] - y)**2
            loss.backward()  # Fresh interior graph, shared batch parameter leaves.
            total += loss.data
        if epoch in (0, 10, 20, 30, 39):
            print(lr, epoch, total / len(data))
        params = [p.data - lr*p.grad for p in W]  # Sum-gradient update, not mean.
EOF

操作指引

故障A：執行下方同一計算圖反傳兩次的命令，測量w11.grad。故障B：僅將b1設為8.0使z1=9.25，記錄tanh導數及隱藏權重梯度。故障C：計算 sum(w*x for w,x in zip([0.5,-0.25], [2.0,-1.0,999.0]))，指出額外輸入如何被丟棄。

python3 - <<'EOF'
import sys; sys.path.insert(0, '.')
from projects.autograd.engine import Value
x1, x2 = Value(2.0), Value(-1.0)
W = [Value(v) for v in (0.5,-0.25,0.1, -0.5,1.0,-0.2, 1.0,-2.0,0.3)]
a1 = (x1*W[0]+x2*W[1]+W[2]).tanh(); a2 = (x1*W[3]+x2*W[4]+W[5]).tanh()
loss = (a1*W[6]+a2*W[7]+W[8]-1.0)**2
loss.backward(); print('after 1st:', W[0].grad)
loss.backward(); print('after 2nd:', W[0].grad)
EOF

操作指引

不使用AI：默寫 zero_grad→forward→loss→backward→step。某次運行每輪梯度不斷累加且損失發散，寫出症狀、機制和修復；獨立完成測驗，為錯點選擇補救鏈接。