第 52 日：自回歸文本生成：溫度調節、Top-k 與核採樣 (Top-p)

01 120 分鐘

自回歸採樣機制：貪心解碼與溫度縮放 (Temperature)

Agent 輔助工程分析

前置要求

第 48 日 Decoder-Only 前向傳播與第 41 日 Softmax 基礎。

學習目標

建立下一詞預測分佈公式 p_i = softmax(z_i / T)，闡明溫度 T 如何精確調控輸出分佈的信息熵。

操作指引

給定 logits z = [2.0, 1.0, 0.0]：(1) 貪心解碼取 argmax 為索引 0；(2) T=1.0 時概率為 [0.665, 0.245, 0.090]；(3) 計算 T=0.5 下的概率 p = [0.867, 0.117, 0.016]；(4) 證明 T->0 退化為貪心 argmax，T->inf 趨於均勻分佈。

自主實作與記錄

1. 計算 z = [2.0, 1.0, 0.0] 在 T=2.0 時的 Softmax 概率分佈。
2. 為什麼過低的溫度（如 T=0.2）在減少幻覺的同時更容易導致複讀機式死循環？
3. 若在代碼中直接傳入 T=0.0 進行 z / T 計算，會拋出什麼運行時異常？

標記為已完成

獨立掌握標準

能準確手算溫度縮放分佈，深入解釋創造性與死循環之間的博弈平衡。

權威來源資源與權利證明

linking arXiv non-exclusive license / Open Access

The Curious Case of Neural Text Degeneration

作者 / 機構: Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, Yejin Choi | 原始出處連結 ↗

對應章節 / 時間戳: Section 2.1 Text Generation Decoding Strategies and Section 3.3 Sampling with Temperature

驗證版本與日期: ICLR 2020 / arXiv:1904.09751v2 (Verified: 2026-10-30)

入選理由:

Primary source introducing Nucleus (Top-p) sampling and analyzing the failure of greedy decoding (loops, repetition).

來源前置條件:

Softmax probabilities, cumulative probability distributions from Days 25-27 and 41-42.

安全回退方案（離線彈性）:

Top-p truncation algorithms and worked probability cutoffs are completely explained in the lesson.

對比候選資源與選型分析 (2)

Fan et al. Hierarchical Neural Story Generation (Top-k) ↗ compared_alternative [Open Access]

Introduces top-k sampling; Holtzman et al. dynamic nucleus threshold is compared directly against top-k.

Hugging Face Generation Strategies Guide ↗ compared_alternative [Apache 2.0]

Practical library guide; Holtzman provides the foundational mathematical reasoning.

linking MIT

nanoGPT: sample.py autoregressive generation script

作者 / 機構: Andrej Karpathy | 原始出處連結 ↗

對應章節 / 時間戳: sample.py: temperature and top_k settings passed to model.generate (model.py: GPT.generate)

驗證版本與日期: Git master (Verified: 2026-10-30)

入選理由:

Compact historical code reference for the named implementation functions; CS336 spring2025 provides the lead component exercises. Do not infer top-p, atomic persistence or cloud budget enforcement from nanoGPT.

來源前置條件:

PyTorch inference, torch.multinomial, and Day 48 decoder-only forward pass.

安全回退方案（離線彈性）:

The generation loop is implemented in runnable local scripts.

對比候選資源與選型分析 (2)

llama.cpp sampling implementation ↗ compared_alternative [MIT]

C++ implementation with high performance; nanoGPT is readable Python matched to our curriculum.

PyTorch text generation tutorial ↗ compared_alternative [Modified BSD]

Encoder-decoder model; nanoGPT decoder-only autoregressive loop is the direct fit for modern LLMs.

02 120 分鐘

詞表截斷策略：Top-k 與核採樣 (Top-p / Nucleus)

Agent 輔助工程分析

前置要求

活動 01。

學習目標

實現 Top-k 與 Top-p 截斷算法，闡明為何 Top-p 能夠動態適應概率分佈的平坦與陡峭程度。

操作指引

給定排序後的詞元概率 p = [0.40, 0.30, 0.15, 0.10, 0.05]：(1) k=2 時截斷並重新歸一化為 [0.571, 0.429]；(2) p=0.80 核採樣閾值下，累積和分別為 0.40、0.70、0.85（超過 0.80），保留前 3 個詞元並重新歸一化為 [0.471, 0.353, 0.176]。

自主實作與記錄

1. 在 p_threshold=0.60 下，Top-p 截斷會保留哪些詞元？
2. 當模型高度確信（如 p=[0.98, 0.01, 0.01]）時，固定 Top-k 存在什麼明顯缺陷？
3. 用 3 行偽代碼概括核採樣的計算步驟。

標記為已完成

獨立掌握標準

準確手算核採樣截斷過程，清晰闡明動態詞表自適應的優越性。

權威來源資源與權利證明

linking arXiv non-exclusive license / Open Access

The Curious Case of Neural Text Degeneration

作者 / 機構: Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, Yejin Choi | 原始出處連結 ↗

對應章節 / 時間戳: Section 3.1 Nucleus Sampling (with Section 2.1 Text Generation Decoding Strategies)

驗證版本與日期: ICLR 2020 / arXiv:1904.09751v2 (Verified: 2026-10-30)

入選理由:

Primary source introducing Nucleus (Top-p) sampling and analyzing the failure of greedy decoding (loops, repetition).

來源前置條件:

Softmax probabilities, cumulative probability distributions from Days 25-27 and 41-42.

安全回退方案（離線彈性）:

Top-p truncation algorithms and worked probability cutoffs are completely explained in the lesson.

對比候選資源與選型分析 (2)

Fan et al. Hierarchical Neural Story Generation (Top-k) ↗ compared_alternative [Open Access]

Introduces top-k sampling; Holtzman et al. dynamic nucleus threshold is compared directly against top-k.

Hugging Face Generation Strategies Guide ↗ compared_alternative [Apache 2.0]

Practical library guide; Holtzman provides the foundational mathematical reasoning.

linking MIT

nanoGPT: sample.py autoregressive generation script

作者 / 機構: Andrej Karpathy | 原始出處連結 ↗

對應章節 / 時間戳: sample.py: temperature and top_k settings passed to model.generate (model.py: GPT.generate)

驗證版本與日期: Git master (Verified: 2026-10-30)

入選理由:

Compact historical code reference for the named implementation functions; CS336 spring2025 provides the lead component exercises. Do not infer top-p, atomic persistence or cloud budget enforcement from nanoGPT.

來源前置條件:

PyTorch inference, torch.multinomial, and Day 48 decoder-only forward pass.

安全回退方案（離線彈性）:

The generation loop is implemented in runnable local scripts.

對比候選資源與選型分析 (2)

llama.cpp sampling implementation ↗ compared_alternative [MIT]

C++ implementation with high performance; nanoGPT is readable Python matched to our curriculum.

PyTorch text generation tutorial ↗ compared_alternative [Modified BSD]

Encoder-decoder model; nanoGPT decoder-only autoregressive loop is the direct fit for modern LLMs.

03 120 分鐘

重複懲罰機制 (Repetition Penalty) 與隨機種子復現性

Agent 輔助工程分析

前置要求

活動 01 與 02。

學習目標

對已有上下文詞元應用懲罰係數 theta，驗證固定隨機種子保證多輪生成的確定性軌跡。

操作指引

若詞元 i 已在上下文中出現：若 z_i > 0 則 z_i = z_i / theta；若 z_i < 0 則 z_i = z_i * theta (theta > 1.0)。驗證 torch.manual_seed(42) 使連續兩次生成結果逐詞元完全一致。

自主實作與記錄

1. 對 logits = [3.0, -2.0, 1.0] 應用 theta=1.2，已知詞元 0 和 1 曾出現過，計算調整後的 logits。
2. 為何對正數 logit 採用除法，而對負數 logit 採用乘法？
3. 編寫兩行斷言測試證明 seed=42 下生成的詞元序列完全一致。

標記為已完成

獨立掌握標準

掌握懲罰計算規則，證明固定隨機種子下的生成確定性。

權威來源資源與權利證明

linking arXiv non-exclusive license / Open Access

The Curious Case of Neural Text Degeneration

作者 / 機構: Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, Yejin Choi | 原始出處連結 ↗

對應章節 / 時間戳: Section 5.3 Repetition and Appendix B Example Generations

驗證版本與日期: ICLR 2020 / arXiv:1904.09751v2 (Verified: 2026-10-30)

入選理由:

Primary source introducing Nucleus (Top-p) sampling and analyzing the failure of greedy decoding (loops, repetition).

來源前置條件:

Softmax probabilities, cumulative probability distributions from Days 25-27 and 41-42.

安全回退方案（離線彈性）:

Top-p truncation algorithms and worked probability cutoffs are completely explained in the lesson.

對比候選資源與選型分析 (2)

Fan et al. Hierarchical Neural Story Generation (Top-k) ↗ compared_alternative [Open Access]

Introduces top-k sampling; Holtzman et al. dynamic nucleus threshold is compared directly against top-k.

Hugging Face Generation Strategies Guide ↗ compared_alternative [Apache 2.0]

Practical library guide; Holtzman provides the foundational mathematical reasoning.

linking MIT

nanoGPT: sample.py autoregressive generation script

作者 / 機構: Andrej Karpathy | 原始出處連結 ↗

對應章節 / 時間戳: sample.py: temperature and top_k settings passed to model.generate (model.py: GPT.generate)

驗證版本與日期: Git master (Verified: 2026-10-30)

入選理由:

Compact historical code reference for the named implementation functions; CS336 spring2025 provides the lead component exercises. Do not infer top-p, atomic persistence or cloud budget enforcement from nanoGPT.

來源前置條件:

PyTorch inference, torch.multinomial, and Day 48 decoder-only forward pass.

安全回退方案（離線彈性）:

The generation loop is implemented in runnable local scripts.

對比候選資源與選型分析 (2)

llama.cpp sampling implementation ↗ compared_alternative [MIT]

C++ implementation with high performance; nanoGPT is readable Python matched to our curriculum.

PyTorch text generation tutorial ↗ compared_alternative [Modified BSD]

Encoder-decoder model; nanoGPT decoder-only autoregressive loop is the direct fit for modern LLMs.

04 120 分鐘

獨立採樣生成函數實現與完整性驗證

完全獨立閉卷考核

前置要求

完成活動 01 至 03。

學習目標

在 PyTorch 中獨立編寫完整的 generate() 函數，集成溫度、Top-k、Top-p 與自回歸循環。

操作指引

編寫 def generate(model, idx, max_new_tokens, temperature=1.0, top_k=None, top_p=None)。斷言輸入 (B, T) 經循環生成後輸出形狀為 (B, T + max_new_tokens)。

自主實作與記錄

1. 給出完整的 generate 函數實現代碼。
2. 驗證當設置 top_k=1 時輸出與貪心 argmax 結果完全等價。
3. 當生成長度超出模型的上下文窗口 block_size 時，必須添加什麼保護措施？

標記為已完成

獨立掌握標準

獨立實現功能完整且通過貪心等價性測試的 generate 函數。

權威來源資源與權利證明

linking MIT

nanoGPT: sample.py autoregressive generation script

作者 / 機構: Andrej Karpathy | 原始出處連結 ↗

對應章節 / 時間戳: sample.py: temperature and top_k settings passed to model.generate (model.py: GPT.generate)

驗證版本與日期: Git master (Verified: 2026-10-30)

入選理由:

Compact historical code reference for the named implementation functions; CS336 spring2025 provides the lead component exercises. Do not infer top-p, atomic persistence or cloud budget enforcement from nanoGPT.

來源前置條件:

PyTorch inference, torch.multinomial, and Day 48 decoder-only forward pass.

安全回退方案（離線彈性）:

The generation loop is implemented in runnable local scripts.

對比候選資源與選型分析 (2)

llama.cpp sampling implementation ↗ compared_alternative [MIT]

C++ implementation with high performance; nanoGPT is readable Python matched to our curriculum.

PyTorch text generation tutorial ↗ compared_alternative [Modified BSD]

Encoder-decoder model; nanoGPT decoder-only autoregressive loop is the direct fit for modern LLMs.

linking arXiv non-exclusive license / Open Access

The Curious Case of Neural Text Degeneration

作者 / 機構: Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, Yejin Choi | 原始出處連結 ↗

對應章節 / 時間戳: Section 3.1 Nucleus Sampling (with Section 2.1 Text Generation Decoding Strategies)

驗證版本與日期: ICLR 2020 / arXiv:1904.09751v2 (Verified: 2026-10-30)

入選理由:

Primary source introducing Nucleus (Top-p) sampling and analyzing the failure of greedy decoding (loops, repetition).

來源前置條件:

Softmax probabilities, cumulative probability distributions from Days 25-27 and 41-42.

安全回退方案（離線彈性）:

Top-p truncation algorithms and worked probability cutoffs are completely explained in the lesson.

對比候選資源與選型分析 (2)

Fan et al. Hierarchical Neural Story Generation (Top-k) ↗ compared_alternative [Open Access]

Introduces top-k sampling; Holtzman et al. dynamic nucleus threshold is compared directly against top-k.

Hugging Face Generation Strategies Guide ↗ compared_alternative [Apache 2.0]

Practical library guide; Holtzman provides the foundational mathematical reasoning.