第 24 天：张量、广播、reshape 与内存台账

01 120 分钟

张量：轴、秩与模型权重块

仅在记录预测与观察证据后才可使用协助。

前置要求

完成第 23 天；若矩阵 shape 不熟练，使用列出的恢复链接。

学习目标

把张量描述为秩等于轴数的 n 维数组；说出模型张量的 batch、time、channel 轴；并追踪从 shape 为 (V x d) 的表 E 做嵌入查找得到 shape (B, T, d) 输出的全过程。

操作指引

标量是 0 秩，向量是 1 秩，矩阵是 2 秩，一批嵌入序列是 shape (B, T, d) 的 3 秩张量：batch 轴、time 轴、channel 轴。嵌入查找就是按行 gather：shape (B, T) 的 ids 查表 E (V, d)，产出 (B, T, d)。权重矩阵是 2 秩张量；带 batch 与 time 的隐藏状态是 3 秩。

自主演练与记录

某小模型词表 V = 50000，嵌入宽度 d = 512。预测：(a) 表 E 中的标量权重数；(b) shape (B=2, T=3) 的 ids 查表后的输出 shape；(c) 该输出装了多少个 float32 值以及占多少字节。

Mark as Completed

独立掌握标准

不借助提示从 (V, d) 与 (B, T) 推出 (B, T, d)，精确算出两个元素计数，并能说出每个轴的物理含义（哪个样本、哪个位置、哪个特征）。

权威源资源与权利证明

linking BSD 3-Clause License

NumPy Reference: The N-dimensional array (ndarray)

作者 / 机构: NumPy Developers | 原始出处链接 ↗

对应章节 / 时间戳: ndarray.ndim / ndarray.shape and axis ordering

验证版本与日期: NumPy 2.x documentation (Verified: 2026-10-30)

入选理由:

Primary semantics for ndarray shape, strides, reshape orders, and view-versus-copy behavior used in tensor memory accounting.

来源前置条件:

Basic Python

安全回退方案（离线弹性）:

Shape, stride, and memory-ledger exercises are fully explained with local runnable examples.

对比候选资源与选型分析 (2)

PyTorch Documentation: torch.Tensor ↗ compared_alternative [Modified BSD]

Equivalent tensor semantics in the framework used later in the curriculum; NumPy reference selected to keep Days 22-24 dependency-free.

NumPy User Guide: Broadcasting ↗ compared_selected_for_days_22_24 [BSD 3-Clause License]

Selected as co-primary for Day 24: the broadcasting rules page pairs with the ndarray attribute reference.

Essence of Linear Algebra

作者 / 机构: Grant Sanderson (3Blue1Brown) | 原始出处链接 ↗

对应章节 / 时间戳: Chapter 3 review: a matrix is a rank-2 tensor

验证版本与日期: 2016-2023 (Verified: 2026-10-30)

入选理由:

Unequaled geometric and visual intuition for vectors, linear transformations, matrix multiplication, and inverses before tensor operations.

来源前置条件:

High-school coordinate geometry

安全回退方案（离线弹性）:

Geometric explanations, worked formulas, and coordinate exercises are fully provided as original lesson text.

对比候选资源与选型分析 (2)

MIT OCW 18.06 Linear Algebra (Prof. Gilbert Strang) ↗ compared_alternative [CC BY-NC-SA 4.0]

Established textbook/lecture approach, strong mathematical rigor alongside 3Blue1Brown visual intuition.

Linear Algebra Done Right (Sheldon Axler) ↗ compared_rejected_for_curriculum [CC BY-NC (4th edition, Open Access)]

Theoretical determinant-free proof structure is mathematically elegant but less aligned with computational tensor engineering.

02 120 分钟

广播规则与失败诊断

独立练习：先记录预测再核对结果；记录证据后才可使用协助。

前置要求

完成上一活动；如受阻，使用列出的恢复链接。

学习目标

运用广播规则——从尾部对齐各轴，每对维度必须相等或有一方为 1——在运行代码前预测输出并诊断失败。

操作指引

广播适用于任意秩的张量：从右向左比较两个 shape。缺失的轴按大小 1 计，大小为 1 的轴被拉伸而不复制内存。MLP 的偏置加法 (B, T, d_ff) + (d_ff,) 把偏置沿 batch 和 time 广播。(2, 3) 与 (4,) 这一对会失败，因为末轴 3 != 4。

自主演练与记录

预测每个结果或失败：(a) [[1,2,3],[4,5,6]] + [10,20,30]；(b) [[1],[2]] + [[10,20,30]]；(c) shape (2,3) + shape (4,)；(d) 隐藏状态 (32,128,768) + 偏置 (768,)。

Mark as Completed

独立掌握标准

无协助预测全部四项（三个 shape 和一个失败），精确复现两组数值，并解释 (2,3) + (4,) 为何按尾轴规则必然失败。

权威源资源与权利证明

linking BSD 3-Clause License

NumPy User Guide: Broadcasting

作者 / 机构: NumPy Developers | 原始出处链接 ↗

对应章节 / 时间戳: General broadcasting rules and incompatible-shape errors

验证版本与日期: NumPy 2.x documentation (Verified: 2026-10-30)

入选理由:

Normative definition of broadcasting rules, shape mismatch errors, and implicit axis expansion shared by NumPy, PyTorch, and other tensor frameworks.

来源前置条件:

Basic Python and array indexing

安全回退方案（离线弹性）:

Broadcasting rules, failure cases, and memory-layout caveats are fully explained with local runnable examples.

对比候选资源与选型分析 (2)

PyTorch Documentation: Broadcasting semantics ↗ compared_alternative [Modified BSD]

Equivalent rules for PyTorch tensors; NumPy chosen because Day 24 exercises run without a PyTorch dependency, while the rule wording is identical.

NumPy Reference: The N-dimensional array (ndarray) ↗ compared_selected_for_days_22_24 [BSD 3-Clause License]

Selected as co-primary for Day 24: normative shape/strides/reshape semantics complementing the broadcasting rules page.

linking BSD 3-Clause License

NumPy Reference: The N-dimensional array (ndarray)

作者 / 机构: NumPy Developers | 原始出处链接 ↗

对应章节 / 时间戳: Implicit dimension expansion for element-wise ops

验证版本与日期: NumPy 2.x documentation (Verified: 2026-10-30)

入选理由:

Primary semantics for ndarray shape, strides, reshape orders, and view-versus-copy behavior used in tensor memory accounting.

来源前置条件:

Basic Python

安全回退方案（离线弹性）:

Shape, stride, and memory-ledger exercises are fully explained with local runnable examples.

对比候选资源与选型分析 (2)

PyTorch Documentation: torch.Tensor ↗ compared_alternative [Modified BSD]

Equivalent tensor semantics in the framework used later in the curriculum; NumPy reference selected to keep Days 22-24 dependency-free.

NumPy User Guide: Broadcasting ↗ compared_selected_for_days_22_24 [BSD 3-Clause License]

Selected as co-primary for Day 24: the broadcasting rules page pairs with the ndarray attribute reference.

03 120 分钟

reshape、视图 vs 拷贝与内存台账

仅在记录预测与观察证据后才可使用协助。

前置要求

完成上一活动；如受阻，使用列出的恢复链接。

学习目标

在知道元素数不变、顺序为行主序的前提下预测 reshape 结果；区分视图与拷贝（转置是非连续视图，转置后的 reshape 可能拷贝）；为真实模型张量记『元素数 × 元素大小』的内存台账。

操作指引

reshape 以 C（行主序）顺序重读同一块平坦缓冲区，所以元素数必须守恒。许多 reshape 和所有普通转置返回视图——同一内存上的新轴元数据。非连续张量（例如 .T 之后）可能迫使 reshape 拷贝。字节数 = 元素数 * 元素大小；float32 元素大小为 4。

自主演练与记录

设 shape (2,3) 的 a = [[1,2,3],[4,5,6]]。预测：(a) a reshape 成 (3,2)；(b) a 展平成 (6,)；(c) a.T.reshape(6) 能否共享 a 的内存；(d) shape (32, 128, 768) 的 float32 隐藏状态的字节数。

Mark as Completed

独立掌握标准

预测全部四项结果，说出 reshape 的元素数守恒，并能用转置例子加 12 MiB 台账项解释视图 vs 拷贝的注意事项。

权威源资源与权利证明

linking BSD 3-Clause License

NumPy Reference: The N-dimensional array (ndarray)

作者 / 机构: NumPy Developers | 原始出处链接 ↗

对应章节 / 时间戳: ndarray.reshape, ndarray.T, strides, and non-contiguous copies

验证版本与日期: NumPy 2.x documentation (Verified: 2026-10-30)

入选理由:

Primary semantics for ndarray shape, strides, reshape orders, and view-versus-copy behavior used in tensor memory accounting.

来源前置条件:

Basic Python

安全回退方案（离线弹性）:

Shape, stride, and memory-ledger exercises are fully explained with local runnable examples.

对比候选资源与选型分析 (2)

PyTorch Documentation: torch.Tensor ↗ compared_alternative [Modified BSD]

Equivalent tensor semantics in the framework used later in the curriculum; NumPy reference selected to keep Days 22-24 dependency-free.

NumPy User Guide: Broadcasting ↗ compared_selected_for_days_22_24 [BSD 3-Clause License]

Selected as co-primary for Day 24: the broadcasting rules page pairs with the ndarray attribute reference.

linking BSD 3-Clause License

NumPy User Guide: Broadcasting

作者 / 机构: NumPy Developers | 原始出处链接 ↗

对应章节 / 时间戳: Broadcasting never materializes the expanded axes

验证版本与日期: NumPy 2.x documentation (Verified: 2026-10-30)

入选理由:

Normative definition of broadcasting rules, shape mismatch errors, and implicit axis expansion shared by NumPy, PyTorch, and other tensor frameworks.

来源前置条件:

Basic Python and array indexing

安全回退方案（离线弹性）:

Broadcasting rules, failure cases, and memory-layout caveats are fully explained with local runnable examples.

对比候选资源与选型分析 (2)

PyTorch Documentation: Broadcasting semantics ↗ compared_alternative [Modified BSD]

Equivalent rules for PyTorch tensors; NumPy chosen because Day 24 exercises run without a PyTorch dependency, while the rule wording is identical.

NumPy Reference: The N-dimensional array (ndarray) ↗ compared_selected_for_days_22_24 [BSD 3-Clause License]

Selected as co-primary for Day 24: normative shape/strides/reshape semantics complementing the broadcasting rules page.

04 120 分钟

独立掌握：张量 shape 与广播审计

独立掌握检查（无 AI 协助）。

前置要求

完成上一活动；如受阻，使用列出的恢复链接。

学习目标

在无助协助下端到端审计一个层归一化式计算：预测带 keepdims 的归约 shape、广播减法、广播失败和内存台账项；任何错误都要写出恢复笔记。

操作指引

闭卷进行：先预测，再计算，后比较。每个张量的 shape 写在其值旁边。若结果出乎意料，记录误用了哪条规则并记下恢复链接。

自主演练与记录

任务：(a) 对 shape (2,3) 的 x = [[1,2,3],[4,5,6]]，计算最后一轴上带 keepdims 的均值及其 shape；(b) x - mean 的广播结果；(c) (2,3)+(3,)、(2,3)+(4,)、(2,1)+(1,3) 中哪个失败、为什么；(d) shape (2, 3, 50000) 的 float32 logits 张量字节数。

Mark as Completed

独立掌握标准

四个答案全部经无协助验证且值旁标注了 shape；说出广播失败规则，并用一句话说明归约中 keepdims 存在的意义。

权威源资源与权利证明

linking BSD 3-Clause License

NumPy Reference: The N-dimensional array (ndarray)

作者 / 机构: NumPy Developers | 原始出处链接 ↗

对应章节 / 时间戳: Full ndarray attribute reference for self-check

验证版本与日期: NumPy 2.x documentation (Verified: 2026-10-30)

入选理由:

Primary semantics for ndarray shape, strides, reshape orders, and view-versus-copy behavior used in tensor memory accounting.

来源前置条件:

Basic Python

安全回退方案（离线弹性）:

Shape, stride, and memory-ledger exercises are fully explained with local runnable examples.

对比候选资源与选型分析 (2)

PyTorch Documentation: torch.Tensor ↗ compared_alternative [Modified BSD]

Equivalent tensor semantics in the framework used later in the curriculum; NumPy reference selected to keep Days 22-24 dependency-free.

NumPy User Guide: Broadcasting ↗ compared_selected_for_days_22_24 [BSD 3-Clause License]

Selected as co-primary for Day 24: the broadcasting rules page pairs with the ndarray attribute reference.

linking BSD 3-Clause License

NumPy User Guide: Broadcasting

作者 / 机构: NumPy Developers | 原始出处链接 ↗

对应章节 / 时间戳: Broadcasting rules reference for self-check

验证版本与日期: NumPy 2.x documentation (Verified: 2026-10-30)

入选理由:

Normative definition of broadcasting rules, shape mismatch errors, and implicit axis expansion shared by NumPy, PyTorch, and other tensor frameworks.

来源前置条件:

Basic Python and array indexing

安全回退方案（离线弹性）:

Broadcasting rules, failure cases, and memory-layout caveats are fully explained with local runnable examples.

对比候选资源与选型分析 (2)

PyTorch Documentation: Broadcasting semantics ↗ compared_alternative [Modified BSD]

Equivalent rules for PyTorch tensors; NumPy chosen because Day 24 exercises run without a PyTorch dependency, while the rule wording is identical.

NumPy Reference: The N-dimensional array (ndarray) ↗ compared_selected_for_days_22_24 [BSD 3-Clause License]

Selected as co-primary for Day 24: normative shape/strides/reshape semantics complementing the broadcasting rules page.