第 36 日：训练循环、学习率与工程故障陈列室

操作指引

先预测再运行。每个 epoch 在样本循环前清空参数梯度，每个样本新建前向计算图、计算平方 loss 并 backward 累加梯度；全部样本之后 step 一次。zero_grad 只要在本批首次反传前即可，下面将其放在 epoch 开头。四样本使用活动36-02的同一初始参数和数据，初始平均损失为2.106。

for epoch in range(EPOCHS):
    for p in model.parameters():
        p.grad = 0.0                 # 1. zero stale gradients
    total = 0.0
    for (x, y) in dataset:
        out = model(x)               # 2. forward
        loss = (out - y) ** 2        # 3. loss
        loss.backward()              # 4. backward (accumulates batch grads)
        total += loss.data
    for p in model.parameters():
        p.data -= lr * p.grad        # 5. step along -gradient
    log(epoch, total / len(dataset))

操作指引

在仓库根目录执行以下命令。两次运行恢复相同九个初始权重与四个样本。每个epoch重新建立参数叶子，每个样本重建内部图；四个梯度累加后更新。日志记录更新前的均值损失；更新用梯度之和，不是均值，改变约定会改变有效学习率。

python3 - <<'EOF'
from projects.autograd.engine import Value
initial = [0.5, -0.25, 0.1, -0.5, 1.0, -0.2, 1.0, -2.0, 0.3]
data = [((2.0,-1.0),1.0), ((0.5,0.5),-1.0), ((-1.0,1.0),-1.0), ((1.0,1.0),1.0)]
for lr in (0.05, 5.0):
    params = initial[:]
    for epoch in range(40):
        W = [Value(v) for v in params]  # Fresh leaves start with zero gradients.
        total = 0.0
        for (x1, x2), y in data:
            a1 = (W[0]*x1 + W[1]*x2 + W[2]).tanh()
            a2 = (W[3]*x1 + W[4]*x2 + W[5]).tanh()
            loss = (a1*W[6] + a2*W[7] + W[8] - y)**2
            loss.backward()  # Fresh interior graph, shared batch parameter leaves.
            total += loss.data
        if epoch in (0, 10, 20, 30, 39):
            print(lr, epoch, total / len(data))
        params = [p.data - lr*p.grad for p in W]  # Sum-gradient update, not mean.
EOF

操作指引

故障A：执行下方同一计算图反传两次的命令，测量w11.grad。故障B：仅将b1设为8.0使z1=9.25，记录tanh导数及隐藏权重梯度。故障C：计算 sum(w*x for w,x in zip([0.5,-0.25], [2.0,-1.0,999.0]))，指出额外输入如何被丢弃。

python3 - <<'EOF'
import sys; sys.path.insert(0, '.')
from projects.autograd.engine import Value
x1, x2 = Value(2.0), Value(-1.0)
W = [Value(v) for v in (0.5,-0.25,0.1, -0.5,1.0,-0.2, 1.0,-2.0,0.3)]
a1 = (x1*W[0]+x2*W[1]+W[2]).tanh(); a2 = (x1*W[3]+x2*W[4]+W[5]).tanh()
loss = (a1*W[6]+a2*W[7]+W[8]-1.0)**2
loss.backward(); print('after 1st:', W[0].grad)
loss.backward(); print('after 2nd:', W[0].grad)
EOF

操作指引

不使用AI：默写 zero_grad→forward→loss→backward→step。某次运行每轮梯度不断累加且损失发散，写出症状、机制和修复；独立完成测验，为错点选择补救链接。