第 52 日：自回归文本生成：温度调节、Top-k 与核采样 (Top-p)

01 120 分钟

自回归采样机制：贪心解码与温度缩放 (Temperature)

Agent 辅助工程分析

前置要求

第 48 日 Decoder-Only 前向传播与第 41 日 Softmax 基础。

学习目标

建立下一词预测分布公式 p_i = softmax(z_i / T)，阐明温度 T 如何精确调控输出分布的信息熵。

操作指引

给定 logits z = [2.0, 1.0, 0.0]：(1) 贪心解码取 argmax 为索引 0；(2) T=1.0 时概率为 [0.665, 0.245, 0.090]；(3) 计算 T=0.5 下的概率 p = [0.867, 0.117, 0.016]；(4) 证明 T->0 退化为贪心 argmax，T->inf 趋于均匀分布。

自主演练与记录

1. 计算 z = [2.0, 1.0, 0.0] 在 T=2.0 时的 Softmax 概率分布。
2. 为什么过低的温度（如 T=0.2）在减少幻觉的同时更容易导致复读机式死循环？
3. 若在代码中直接传入 T=0.0 进行 z / T 计算，会抛出什么运行时异常？

标记为已完成

独立掌握标准

能准确手算温度缩放分布，深入解释创造性与死循环之间的博弈平衡。

权威源资源与权利证明

linking arXiv non-exclusive license / Open Access

The Curious Case of Neural Text Degeneration

作者 / 机构: Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, Yejin Choi | 原始出处链接 ↗

对应章节 / 时间戳: Section 2.1 Text Generation Decoding Strategies and Section 3.3 Sampling with Temperature

验证版本与日期: ICLR 2020 / arXiv:1904.09751v2 (Verified: 2026-10-30)

入选理由:

Primary source introducing Nucleus (Top-p) sampling and analyzing the failure of greedy decoding (loops, repetition).

来源前置条件:

Softmax probabilities, cumulative probability distributions from Days 25-27 and 41-42.

安全回退方案（离线弹性）:

Top-p truncation algorithms and worked probability cutoffs are completely explained in the lesson.

对比候选资源与选型分析 (2)

Fan et al. Hierarchical Neural Story Generation (Top-k) ↗ compared_alternative [Open Access]

Introduces top-k sampling; Holtzman et al. dynamic nucleus threshold is compared directly against top-k.

Hugging Face Generation Strategies Guide ↗ compared_alternative [Apache 2.0]

Practical library guide; Holtzman provides the foundational mathematical reasoning.

linking MIT

nanoGPT: sample.py autoregressive generation script

作者 / 机构: Andrej Karpathy | 原始出处链接 ↗

对应章节 / 时间戳: sample.py: temperature and top_k settings passed to model.generate (model.py: GPT.generate)

验证版本与日期: Git master (Verified: 2026-10-30)

入选理由:

Compact historical code reference for the named implementation functions; CS336 spring2025 provides the lead component exercises. Do not infer top-p, atomic persistence or cloud budget enforcement from nanoGPT.

来源前置条件:

PyTorch inference, torch.multinomial, and Day 48 decoder-only forward pass.

安全回退方案（离线弹性）:

The generation loop is implemented in runnable local scripts.

对比候选资源与选型分析 (2)

llama.cpp sampling implementation ↗ compared_alternative [MIT]

C++ implementation with high performance; nanoGPT is readable Python matched to our curriculum.

PyTorch text generation tutorial ↗ compared_alternative [Modified BSD]

Encoder-decoder model; nanoGPT decoder-only autoregressive loop is the direct fit for modern LLMs.

02 120 分钟

词表截断策略：Top-k 与核采样 (Top-p / Nucleus)

Agent 辅助工程分析

前置要求

活动 01。

学习目标

实现 Top-k 与 Top-p 截断算法，阐明为何 Top-p 能够动态适应概率分布的平坦与陡峭程度。

操作指引

给定排序后的词元概率 p = [0.40, 0.30, 0.15, 0.10, 0.05]：(1) k=2 时截断并重新归一化为 [0.571, 0.429]；(2) p=0.80 核采样阈值下，累积和分别为 0.40、0.70、0.85（超过 0.80），保留前 3 个词元并重新归一化为 [0.471, 0.353, 0.176]。

自主演练与记录

1. 在 p_threshold=0.60 下，Top-p 截断会保留哪些词元？
2. 当模型高度确信（如 p=[0.98, 0.01, 0.01]）时，固定 Top-k 存在什么明显缺陷？
3. 用 3 行伪代码概括核采样的计算步骤。

标记为已完成

独立掌握标准

准确手算核采样截断过程，清晰阐明动态词表自适应的优越性。

权威源资源与权利证明

linking arXiv non-exclusive license / Open Access

The Curious Case of Neural Text Degeneration

作者 / 机构: Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, Yejin Choi | 原始出处链接 ↗

对应章节 / 时间戳: Section 3.1 Nucleus Sampling (with Section 2.1 Text Generation Decoding Strategies)

验证版本与日期: ICLR 2020 / arXiv:1904.09751v2 (Verified: 2026-10-30)

入选理由:

Primary source introducing Nucleus (Top-p) sampling and analyzing the failure of greedy decoding (loops, repetition).

来源前置条件:

Softmax probabilities, cumulative probability distributions from Days 25-27 and 41-42.

安全回退方案（离线弹性）:

Top-p truncation algorithms and worked probability cutoffs are completely explained in the lesson.

对比候选资源与选型分析 (2)

Fan et al. Hierarchical Neural Story Generation (Top-k) ↗ compared_alternative [Open Access]

Introduces top-k sampling; Holtzman et al. dynamic nucleus threshold is compared directly against top-k.

Hugging Face Generation Strategies Guide ↗ compared_alternative [Apache 2.0]

Practical library guide; Holtzman provides the foundational mathematical reasoning.

linking MIT

nanoGPT: sample.py autoregressive generation script

作者 / 机构: Andrej Karpathy | 原始出处链接 ↗

对应章节 / 时间戳: sample.py: temperature and top_k settings passed to model.generate (model.py: GPT.generate)

验证版本与日期: Git master (Verified: 2026-10-30)

入选理由:

Compact historical code reference for the named implementation functions; CS336 spring2025 provides the lead component exercises. Do not infer top-p, atomic persistence or cloud budget enforcement from nanoGPT.

来源前置条件:

PyTorch inference, torch.multinomial, and Day 48 decoder-only forward pass.

安全回退方案（离线弹性）:

The generation loop is implemented in runnable local scripts.

对比候选资源与选型分析 (2)

llama.cpp sampling implementation ↗ compared_alternative [MIT]

C++ implementation with high performance; nanoGPT is readable Python matched to our curriculum.

PyTorch text generation tutorial ↗ compared_alternative [Modified BSD]

Encoder-decoder model; nanoGPT decoder-only autoregressive loop is the direct fit for modern LLMs.

03 120 分钟

重复惩罚机制 (Repetition Penalty) 与随机种子复现性

Agent 辅助工程分析

前置要求

活动 01 与 02。

学习目标

对已有上下文词元应用惩罚系数 theta，验证固定随机种子保证多轮生成的确定性轨迹。

操作指引

若词元 i 已在上下文中出现：若 z_i > 0 则 z_i = z_i / theta；若 z_i < 0 则 z_i = z_i * theta (theta > 1.0)。验证 torch.manual_seed(42) 使连续两次生成结果逐词元完全一致。

自主演练与记录

1. 对 logits = [3.0, -2.0, 1.0] 应用 theta=1.2，已知词元 0 和 1 曾出现过，计算调整后的 logits。
2. 为何对正数 logit 采用除法，而对负数 logit 采用乘法？
3. 编写两行断言测试证明 seed=42 下生成的词元序列完全一致。

标记为已完成

独立掌握标准

掌握惩罚计算规则，证明固定随机种子下的生成确定性。

权威源资源与权利证明

linking arXiv non-exclusive license / Open Access

The Curious Case of Neural Text Degeneration

作者 / 机构: Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, Yejin Choi | 原始出处链接 ↗

对应章节 / 时间戳: Section 5.3 Repetition and Appendix B Example Generations

验证版本与日期: ICLR 2020 / arXiv:1904.09751v2 (Verified: 2026-10-30)

入选理由:

Primary source introducing Nucleus (Top-p) sampling and analyzing the failure of greedy decoding (loops, repetition).

来源前置条件:

Softmax probabilities, cumulative probability distributions from Days 25-27 and 41-42.

安全回退方案（离线弹性）:

Top-p truncation algorithms and worked probability cutoffs are completely explained in the lesson.

对比候选资源与选型分析 (2)

Fan et al. Hierarchical Neural Story Generation (Top-k) ↗ compared_alternative [Open Access]

Introduces top-k sampling; Holtzman et al. dynamic nucleus threshold is compared directly against top-k.

Hugging Face Generation Strategies Guide ↗ compared_alternative [Apache 2.0]

Practical library guide; Holtzman provides the foundational mathematical reasoning.

linking MIT

nanoGPT: sample.py autoregressive generation script

作者 / 机构: Andrej Karpathy | 原始出处链接 ↗

对应章节 / 时间戳: sample.py: temperature and top_k settings passed to model.generate (model.py: GPT.generate)

验证版本与日期: Git master (Verified: 2026-10-30)

入选理由:

Compact historical code reference for the named implementation functions; CS336 spring2025 provides the lead component exercises. Do not infer top-p, atomic persistence or cloud budget enforcement from nanoGPT.

来源前置条件:

PyTorch inference, torch.multinomial, and Day 48 decoder-only forward pass.

安全回退方案（离线弹性）:

The generation loop is implemented in runnable local scripts.

对比候选资源与选型分析 (2)

llama.cpp sampling implementation ↗ compared_alternative [MIT]

C++ implementation with high performance; nanoGPT is readable Python matched to our curriculum.

PyTorch text generation tutorial ↗ compared_alternative [Modified BSD]

Encoder-decoder model; nanoGPT decoder-only autoregressive loop is the direct fit for modern LLMs.

04 120 分钟

独立采样生成函数实现与完整性验证

完全独立闭卷考核

前置要求

完成活动 01 至 03。

学习目标

在 PyTorch 中独立编写完整的 generate() 函数，集成温度、Top-k、Top-p 与自回归循环。

操作指引

编写 def generate(model, idx, max_new_tokens, temperature=1.0, top_k=None, top_p=None)。断言输入 (B, T) 经循环生成后输出形状为 (B, T + max_new_tokens)。

自主演练与记录

1. 给出完整的 generate 函数实现代码。
2. 验证当设置 top_k=1 时输出与贪心 argmax 结果完全等价。
3. 当生成长度超出模型的上下文窗口 block_size 时，必须添加什么保护措施？

标记为已完成

独立掌握标准

独立实现功能完整且通过贪心等价性测试的 generate 函数。

权威源资源与权利证明

linking MIT

nanoGPT: sample.py autoregressive generation script

作者 / 机构: Andrej Karpathy | 原始出处链接 ↗

对应章节 / 时间戳: sample.py: temperature and top_k settings passed to model.generate (model.py: GPT.generate)

验证版本与日期: Git master (Verified: 2026-10-30)

入选理由:

Compact historical code reference for the named implementation functions; CS336 spring2025 provides the lead component exercises. Do not infer top-p, atomic persistence or cloud budget enforcement from nanoGPT.

来源前置条件:

PyTorch inference, torch.multinomial, and Day 48 decoder-only forward pass.

安全回退方案（离线弹性）:

The generation loop is implemented in runnable local scripts.

对比候选资源与选型分析 (2)

llama.cpp sampling implementation ↗ compared_alternative [MIT]

C++ implementation with high performance; nanoGPT is readable Python matched to our curriculum.

PyTorch text generation tutorial ↗ compared_alternative [Modified BSD]

Encoder-decoder model; nanoGPT decoder-only autoregressive loop is the direct fit for modern LLMs.

linking arXiv non-exclusive license / Open Access

The Curious Case of Neural Text Degeneration

作者 / 机构: Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, Yejin Choi | 原始出处链接 ↗

对应章节 / 时间戳: Section 3.1 Nucleus Sampling (with Section 2.1 Text Generation Decoding Strategies)

验证版本与日期: ICLR 2020 / arXiv:1904.09751v2 (Verified: 2026-10-30)

入选理由:

Primary source introducing Nucleus (Top-p) sampling and analyzing the failure of greedy decoding (loops, repetition).

来源前置条件:

Softmax probabilities, cumulative probability distributions from Days 25-27 and 41-42.

安全回退方案（离线弹性）:

Top-p truncation algorithms and worked probability cutoffs are completely explained in the lesson.

对比候选资源与选型分析 (2)

Fan et al. Hierarchical Neural Story Generation (Top-k) ↗ compared_alternative [Open Access]

Introduces top-k sampling; Holtzman et al. dynamic nucleus threshold is compared directly against top-k.

Hugging Face Generation Strategies Guide ↗ compared_alternative [Apache 2.0]

Practical library guide; Holtzman provides the foundational mathematical reasoning.