第 24 天：張量、廣播、reshape 與記憶體台帳

01 120 分鐘

張量：軸、秩與模型權重區塊

僅在記錄預測與觀察證據後才可使用協助。

前置要求

完成第 23 天；若矩陣 shape 不熟練，使用列出的復原連結。

學習目標

把張量描述為秩等於軸數的 n 維陣列；說出模型張量的 batch、time、channel 軸；並追蹤從 shape 為 (V x d) 的表 E 做嵌入查找得到 shape (B, T, d) 輸出的全過程。

操作指引

純量是 0 秩，向量是 1 秩，矩陣是 2 秩，一批嵌入序列是 shape (B, T, d) 的 3 秩張量：batch 軸、time 軸、channel 軸。嵌入查找就是按行 gather：shape (B, T) 的 ids 查表 E (V, d)，產出 (B, T, d)。權重矩陣是 2 秩張量；帶 batch 與 time 的隱藏狀態是 3 秩。

自主實作與記錄

某小模型詞表 V = 50000，嵌入寬度 d = 512。預測：(a) 表 E 中的純量權重數；(b) shape (B=2, T=3) 的 ids 查表後的輸出 shape；(c) 該輸出裝了多少個 float32 值以及佔多少位元組。

Mark as Completed

獨立掌握標準

不借助提示從 (V, d) 與 (B, T) 推出 (B, T, d)，精確算出兩個元素計數，並能說出每個軸的物理意義（哪個樣本、哪個位置、哪個特徵）。

權威來源資源與權利證明

linking BSD 3-Clause License

NumPy Reference: The N-dimensional array (ndarray)

作者 / 機構: NumPy Developers | 原始出處連結 ↗

對應章節 / 時間戳: ndarray.ndim / ndarray.shape and axis ordering

驗證版本與日期: NumPy 2.x documentation (Verified: 2026-10-30)

入選理由:

Primary semantics for ndarray shape, strides, reshape orders, and view-versus-copy behavior used in tensor memory accounting.

來源前置條件:

Basic Python

安全回退方案（離線彈性）:

Shape, stride, and memory-ledger exercises are fully explained with local runnable examples.

對比候選資源與選型分析 (2)

PyTorch Documentation: torch.Tensor ↗ compared_alternative [Modified BSD]

Equivalent tensor semantics in the framework used later in the curriculum; NumPy reference selected to keep Days 22-24 dependency-free.

NumPy User Guide: Broadcasting ↗ compared_selected_for_days_22_24 [BSD 3-Clause License]

Selected as co-primary for Day 24: the broadcasting rules page pairs with the ndarray attribute reference.

Essence of Linear Algebra

作者 / 機構: Grant Sanderson (3Blue1Brown) | 原始出處連結 ↗

對應章節 / 時間戳: Chapter 3 review: a matrix is a rank-2 tensor

驗證版本與日期: 2016-2023 (Verified: 2026-10-30)

入選理由:

Unequaled geometric and visual intuition for vectors, linear transformations, matrix multiplication, and inverses before tensor operations.

來源前置條件:

High-school coordinate geometry

安全回退方案（離線彈性）:

Geometric explanations, worked formulas, and coordinate exercises are fully provided as original lesson text.

對比候選資源與選型分析 (2)

MIT OCW 18.06 Linear Algebra (Prof. Gilbert Strang) ↗ compared_alternative [CC BY-NC-SA 4.0]

Established textbook/lecture approach, strong mathematical rigor alongside 3Blue1Brown visual intuition.

Linear Algebra Done Right (Sheldon Axler) ↗ compared_rejected_for_curriculum [CC BY-NC (4th edition, Open Access)]

Theoretical determinant-free proof structure is mathematically elegant but less aligned with computational tensor engineering.

02 120 分鐘

廣播規則與失敗診斷

獨立練習：先記錄預測再核對結果；記錄證據後才可使用協助。

前置要求

完成上一活動；如受阻，使用列出的復原連結。

學習目標

運用廣播規則——從尾部對齊各軸，每對維度必須相等或有一方為 1——在執行程式前預測輸出並診斷失敗。

操作指引

廣播適用於任意秩的張量：從右向左比較兩個 shape。缺失的軸按大小 1 計，大小為 1 的軸被拉伸而不複製記憶體。MLP 的偏置加法 (B, T, d_ff) + (d_ff,) 把偏置沿 batch 和 time 廣播。(2, 3) 與 (4,) 這一對會失敗，因為末軸 3 != 4。

自主實作與記錄

預測每個結果或失敗：(a) [[1,2,3],[4,5,6]] + [10,20,30]；(b) [[1],[2]] + [[10,20,30]]；(c) shape (2,3) + shape (4,)；(d) 隱藏狀態 (32,128,768) + 偏置 (768,)。

Mark as Completed

獨立掌握標準

無協助預測全部四項（三個 shape 和一個失敗），精確重現兩組數值，並解釋 (2,3) + (4,) 為何按尾軸規則必然失敗。

權威來源資源與權利證明

linking BSD 3-Clause License

NumPy User Guide: Broadcasting

作者 / 機構: NumPy Developers | 原始出處連結 ↗

對應章節 / 時間戳: General broadcasting rules and incompatible-shape errors

驗證版本與日期: NumPy 2.x documentation (Verified: 2026-10-30)

入選理由:

Normative definition of broadcasting rules, shape mismatch errors, and implicit axis expansion shared by NumPy, PyTorch, and other tensor frameworks.

來源前置條件:

Basic Python and array indexing

安全回退方案（離線彈性）:

Broadcasting rules, failure cases, and memory-layout caveats are fully explained with local runnable examples.

對比候選資源與選型分析 (2)

PyTorch Documentation: Broadcasting semantics ↗ compared_alternative [Modified BSD]

Equivalent rules for PyTorch tensors; NumPy chosen because Day 24 exercises run without a PyTorch dependency, while the rule wording is identical.

NumPy Reference: The N-dimensional array (ndarray) ↗ compared_selected_for_days_22_24 [BSD 3-Clause License]

Selected as co-primary for Day 24: normative shape/strides/reshape semantics complementing the broadcasting rules page.

linking BSD 3-Clause License

NumPy Reference: The N-dimensional array (ndarray)

作者 / 機構: NumPy Developers | 原始出處連結 ↗

對應章節 / 時間戳: Implicit dimension expansion for element-wise ops

驗證版本與日期: NumPy 2.x documentation (Verified: 2026-10-30)

入選理由:

Primary semantics for ndarray shape, strides, reshape orders, and view-versus-copy behavior used in tensor memory accounting.

來源前置條件:

Basic Python

安全回退方案（離線彈性）:

Shape, stride, and memory-ledger exercises are fully explained with local runnable examples.

對比候選資源與選型分析 (2)

PyTorch Documentation: torch.Tensor ↗ compared_alternative [Modified BSD]

Equivalent tensor semantics in the framework used later in the curriculum; NumPy reference selected to keep Days 22-24 dependency-free.

NumPy User Guide: Broadcasting ↗ compared_selected_for_days_22_24 [BSD 3-Clause License]

Selected as co-primary for Day 24: the broadcasting rules page pairs with the ndarray attribute reference.

03 120 分鐘

reshape、檢視 vs 複製與記憶體台帳

僅在記錄預測與觀察證據後才可使用協助。

前置要求

完成上一活動；如受阻，使用列出的復原連結。

學習目標

在知道元素數不變、順序為行主序（row-major）的前提下預測 reshape 結果；區分檢視與複製（轉置是非連續檢視，轉置後的 reshape 可能複製）；為真實模型張量記『元素數 × 元素大小』的記憶體台帳。

操作指引

reshape 以 C（行主序）順序重讀同一塊平坦緩衝區，所以元素數必須守恆。許多 reshape 和所有普通轉置回傳檢視——同一記憶體上的新軸元資料。非連續張量（例如 .T 之後）可能迫使 reshape 複製。位元組數 = 元素數 * 元素大小；float32 元素大小為 4。

自主實作與記錄

設 shape (2,3) 的 a = [[1,2,3],[4,5,6]]。預測：(a) a reshape 成 (3,2)；(b) a 展平成 (6,)；(c) a.T.reshape(6) 能否共享 a 的記憶體；(d) shape (32, 128, 768) 的 float32 隱藏狀態的位元組數。

Mark as Completed

獨立掌握標準

預測全部四項結果，說出 reshape 的元素數守恆，並能用轉置例子加 12 MiB 台帳項解釋檢視 vs 複製的注意事項。

權威來源資源與權利證明

linking BSD 3-Clause License

NumPy Reference: The N-dimensional array (ndarray)

作者 / 機構: NumPy Developers | 原始出處連結 ↗

對應章節 / 時間戳: ndarray.reshape, ndarray.T, strides, and non-contiguous copies

驗證版本與日期: NumPy 2.x documentation (Verified: 2026-10-30)

入選理由:

Primary semantics for ndarray shape, strides, reshape orders, and view-versus-copy behavior used in tensor memory accounting.

來源前置條件:

Basic Python

安全回退方案（離線彈性）:

Shape, stride, and memory-ledger exercises are fully explained with local runnable examples.

對比候選資源與選型分析 (2)

PyTorch Documentation: torch.Tensor ↗ compared_alternative [Modified BSD]

Equivalent tensor semantics in the framework used later in the curriculum; NumPy reference selected to keep Days 22-24 dependency-free.

NumPy User Guide: Broadcasting ↗ compared_selected_for_days_22_24 [BSD 3-Clause License]

Selected as co-primary for Day 24: the broadcasting rules page pairs with the ndarray attribute reference.

linking BSD 3-Clause License

NumPy User Guide: Broadcasting

作者 / 機構: NumPy Developers | 原始出處連結 ↗

對應章節 / 時間戳: Broadcasting never materializes the expanded axes

驗證版本與日期: NumPy 2.x documentation (Verified: 2026-10-30)

入選理由:

Normative definition of broadcasting rules, shape mismatch errors, and implicit axis expansion shared by NumPy, PyTorch, and other tensor frameworks.

來源前置條件:

Basic Python and array indexing

安全回退方案（離線彈性）:

Broadcasting rules, failure cases, and memory-layout caveats are fully explained with local runnable examples.

對比候選資源與選型分析 (2)

PyTorch Documentation: Broadcasting semantics ↗ compared_alternative [Modified BSD]

Equivalent rules for PyTorch tensors; NumPy chosen because Day 24 exercises run without a PyTorch dependency, while the rule wording is identical.

NumPy Reference: The N-dimensional array (ndarray) ↗ compared_selected_for_days_22_24 [BSD 3-Clause License]

Selected as co-primary for Day 24: normative shape/strides/reshape semantics complementing the broadcasting rules page.

04 120 分鐘

獨立掌握：張量 shape 與廣播稽核

獨立掌握檢查（無 AI 協助）。

前置要求

完成上一活動；如受阻，使用列出的復原連結。

學習目標

在無協助下端對端稽核一個層正規化式計算：預測帶 keepdims 的歸約 shape、廣播減法、廣播失敗和記憶體台帳項；任何錯誤都要寫出復原筆記。

操作指引

閉卷進行：先預測，再計算，後比較。每個張量的 shape 寫在其值旁邊。若結果出乎意料，記錄誤用了哪條規則並記下復原連結。

自主實作與記錄

任務：(a) 對 shape (2,3) 的 x = [[1,2,3],[4,5,6]]，計算最後一軸上帶 keepdims 的均值及其 shape；(b) x - mean 的廣播結果；(c) (2,3)+(3,)、(2,3)+(4,)、(2,1)+(1,3) 中哪個失敗、為什麼；(d) shape (2, 3, 50000) 的 float32 logits 張量位元組數。

Mark as Completed

獨立掌握標準

四個答案全部經無協助驗證且值旁標註了 shape；說出廣播失敗規則，並用一句話說明歸約中 keepdims 存在的意義。

權威來源資源與權利證明

linking BSD 3-Clause License

NumPy Reference: The N-dimensional array (ndarray)

作者 / 機構: NumPy Developers | 原始出處連結 ↗

對應章節 / 時間戳: Full ndarray attribute reference for self-check

驗證版本與日期: NumPy 2.x documentation (Verified: 2026-10-30)

入選理由:

Primary semantics for ndarray shape, strides, reshape orders, and view-versus-copy behavior used in tensor memory accounting.

來源前置條件:

Basic Python

安全回退方案（離線彈性）:

Shape, stride, and memory-ledger exercises are fully explained with local runnable examples.

對比候選資源與選型分析 (2)

PyTorch Documentation: torch.Tensor ↗ compared_alternative [Modified BSD]

Equivalent tensor semantics in the framework used later in the curriculum; NumPy reference selected to keep Days 22-24 dependency-free.

NumPy User Guide: Broadcasting ↗ compared_selected_for_days_22_24 [BSD 3-Clause License]

Selected as co-primary for Day 24: the broadcasting rules page pairs with the ndarray attribute reference.

linking BSD 3-Clause License

NumPy User Guide: Broadcasting

作者 / 機構: NumPy Developers | 原始出處連結 ↗

對應章節 / 時間戳: Broadcasting rules reference for self-check

驗證版本與日期: NumPy 2.x documentation (Verified: 2026-10-30)

入選理由:

Normative definition of broadcasting rules, shape mismatch errors, and implicit axis expansion shared by NumPy, PyTorch, and other tensor frameworks.

來源前置條件:

Basic Python and array indexing

安全回退方案（離線彈性）:

Broadcasting rules, failure cases, and memory-layout caveats are fully explained with local runnable examples.

對比候選資源與選型分析 (2)

PyTorch Documentation: Broadcasting semantics ↗ compared_alternative [Modified BSD]

Equivalent rules for PyTorch tensors; NumPy chosen because Day 24 exercises run without a PyTorch dependency, while the rule wording is identical.

NumPy Reference: The N-dimensional array (ndarray) ↗ compared_selected_for_days_22_24 [BSD 3-Clause License]

Selected as co-primary for Day 24: normative shape/strides/reshape semantics complementing the broadcasting rules page.