神经语言模型流水线：从索引到预激活
Agent 辅助工程检查
前置要求
第 40 天的嵌入查表与上下文拼接；第 34-36 天的 MLP 机制。
学习目标
走完 索引 → C 的行 → 拼接 x → 隐藏层预激活 的完整流水线，并手算输入向量与预激活向量。
操作指引
必做任务：沿用第 40 天的表 C 与上下文 ('a','b')，故 x = [0.2, -0.1, -0.4, 0.5]。今天权重固定：W1 = [[0.5,-0.2,0.1],[0.3,0.4,-0.3],[-0.1,0.2,0.6],[0.2,-0.5,0.4]]，b1 = [0.1,-0.1,0.0]。计算各隐藏层预激活 pre_j = sum_i x_i * W1[i][j] + b1[j]。先记录你的预测值再查看推演。选做任务：预测哪个输入坐标对 pre_1 影响最大。
自主演练与记录
1. 用上面的 x 与 W1 计算 pre_0、pre_1、pre_2，写出每一个乘积。
2. 预测是否有某个预激活恰好为 0。
3. 按顺序说出从原始符号到 pre 的流水线各阶段。
pre_0 = 0.2*0.5 + (-0.1)*0.3 + (-0.4)*(-0.1) + 0.5*0.2 + 0.1。求和前保持每个乘积的符号。
pre_0 = 0.10 - 0.03 + 0.04 + 0.10 + 0.10 = 0.31；pre_1 = -0.04 - 0.04 - 0.08 - 0.25 - 0.10 = -0.51；pre_2 = 0.02 + 0.03 - 0.24 + 0.20 + 0.00 = 0.01。没有一个恰好为 0，但 pre_2 足够小，tanh 在那里近似线性。各阶段：符号 → 索引 → C 的行 → 拼接 → x*W1 + b1 → tanh → *W2 + b2 → softmax。
手算预激活写出每个乘积并与 [0.31, -0.51, 0.01] 一致，学习者能不看笔记按顺序叙述流水线各阶段。
权威源资源与权利证明
makemore: Character-level language models on names.txt
对应章节 / 时间戳: Segment wiring the MLP forward pass: C[X] -> hidden tanh layer -> output logits
验证版本与日期: Repository state verified 2026-10-30; lecture published 2022
Standard beginner walkthrough of the Bengio-style MLP language model; the MIT-licensed reference implementation pairs with a stepwise derivation that matches the Days 40-42 worked numbers exactly.
Python basics, matrix multiplication, and the Day 37-39 tokenization and n-gram context
All forward-pass arithmetic, shapes, parameter counts, and loss values are reproduced as original hand-worked examples in the lesson text; no video, subtitle, or transcript is required.
对比候选资源与选型分析 (3)
The original architecture paper; selected jointly as the theoretical citation while Makemore supplies the stepwise implementation walkthrough.
Strong PyTorch rigor but assumes fluent tensor programming; deferred to the Days 43-60 pretraining blocks rather than the first embedding contact.
Strong visual intuition for network mechanics but covers image classification rather than embedding tables and next-token language modeling; retained as optional supplementary viewing.
A Neural Probabilistic Language Model
对应章节 / 时间戳: Section 2 (the mapping g: context feature vectors to next-word probabilities)
验证版本与日期: JMLR Volume 3 (2003); verified 2026-10-30
The original paper introducing jointly learned word feature vectors and the neural probabilistic language model; grounds the curriculum's embedding and MLP derivations in the primary literature.
Probability basics and linear algebra
Every worked derivation in the lessons is self-contained and offline-complete; the paper is further reading only.
对比候选资源与选型分析 (3)
Selected jointly: Makemore reproduces this architecture at character scale with MIT-licensed code; the paper supplies the original formalism and the n-gram comparison framing.
Strong textbook coverage of n-grams through neural LMs; kept as secondary reading because the primary derivation follows the original paper's notation.
Foundational for embedding geometry but trains on context-prediction objectives rather than the full next-token probabilistic model studied this week; revisited in later representation blocks.