凸碗形函數上的梯度下降機制
僅在記錄預測與觀察證據之後允許求助。
前置要求
完成第 29 天評估；掌握數學模組中的單變數微分（冪法則）。
學習目標
推導梯度下降更新 x ← x − η·2x = x(1 − 2η)，並由因子 (1 − 2η) 預測整條優化軌跡。
操作指引
這裡的優化指迭代地降低損失。在 f(x) = x² 上梯度為 2x，每步把 x 乘以 (1 − 2η)。從 x₀ = 4、η = 0.1 出發，因子為 0.8：x₁ = 3.2、x₂ = 2.56、x₃ = 2.048，一般地 x_k = 4·0.8^k。先手工推演，再用短 Python 迴圈確認。注意 η = 0.5 時因子為 0——在這個純二次函數上一步收斂。
自主實作與記錄
對 f(x) = x²、x₀ = 4：(a) 寫出 x_k 的閉式；(b) 計算 η = 0.1 時的 x₁、x₂、x₃ 與 x₁₀；(c) 對 f(x) = (a/2)x²，因子為 (1 − ηa)——推導穩定條件 |1 − ηa| < 1 及其蘊含的學習率上界。
x_k = 4·(1−2η)^k；0.8^10 ≈ 0.1074，故 x₁₀ ≈ 0.4295。上界：−1 < 1 − ηa < 1 → 0 < η < 2/a；對 x²，a = 2，故 η < 1。
(a) x_k = 4·(1−2η)^k。(b) η = 0.1：x₁ = 3.2、x₂ = 2.56、x₃ = 2.048、x₁₀ ≈ 0.4295。(c) |1 − ηa| < 1 ⇔ 0 < η < 2/a；對 f = x²（a = 2）上界為 η < 1——而 η = 0.5 恰好一步落到極小值。
閉式推導正確，四個迭代值全部正確，並陳述 η < 2/a 條件及 a = 2 的特例。
權威來源資源與權利證明
Dive into Deep Learning (d2l.ai)
對應章節 / 時間戳: Chapter Optimization Algorithms: Gradient Descent
驗證版本與日期: Current online edition (Verified: 2026-10-30)
Primary open reference tying evaluation discipline (model selection, underfitting, overfitting) and optimization algorithms (gradient descent, minibatch) to the rest of the curriculum.
Linear algebra, calculus, and probability at the Days 19-28 level.
All required formulas and worked numbers are reproduced as original explanations; the book is further reading.
對比候選資源與選型分析 (2)
Detailed treatment of optimization difficulties (chapter 8) including saddle points and plateaus; all-rights-reserved text kept as a linking-only alternative.
Strong practitioner framing, but organized around the fastai library and video lessons rather than self-contained mathematics.
Gradient Descent, How Neural Networks Learn
對應章節 / 時間戳: Neural networks chapter 2: the gradient and the descent update
驗證版本與日期: Neural networks series chapter 2, 2017 (Verified: 2026-10-30)
Visual intuition connecting the gradient vector to steepest descent; complements d2l.ai's worked equations for the optimization day.
High-school functions and slope intuition.
The lesson contains complete derivations and numerical traces; the video is optional visual intuition.
對比候選資源與選型分析 (2)
A detailed optimization text, kept as an advanced alternative; its scope exceeds a single study day.
Rigorous discussion of saddle points and plateaus in high-dimensional optimization; kept as a linking-only alternative.