23 / 90

Day 23: Matrix Multiplication, Shape Ledgers, Transpose & Inverse

Multiply matrices as composed linear maps, keep an explicit (m x k)(k x n) -> m x n shape ledger for every product, and build intuition for transpose, identity, and inverse. Connect every product to the weight matrices inside a model. Every activity records prediction, observation, review, and recovery inside the 480-minute ledger.

Mathematics & Algorithmic Foundations 480 active minutes (8 hours focused study)
01 120 min

Matrix multiplication as composed linear maps

Assistance allowed only after prediction and observed evidence are recorded.

Prerequisites

Complete Day 22; use the listed recovery link if matrix-vector products are shaky.

Learning Objective

Compute matrix products entry-by-entry as row-times-column dot products, interpret A*B as the composed map 'apply B then apply A', and demonstrate that matrix multiplication is not commutative.

Instructions

Entry (A*B)_ij is the dot product of row i of A with column j of B. Geometrically (A*B)*x = A*(B*x): the product matrix applies the inner map first, so composition reads right-to-left. Because composition depends on order, A*B and B*A generally differ.

Practice & Prediction Log

Let A = [[1,2],[3,4]] and B = [[0,1],[1,0]] (B swaps a vector's coordinates). Predict both products before computing: (a) A*B; (b) B*A. Are they equal?

Independent Mastery Criterion

You compute both products correctly, explain the right-to-left composition reading, and give one geometric reason the two orders differ (swap-then-map vs map-then-swap).

Authoritative Source Resources & Provenance

linking 3Blue1Brown copyright (all rights reserved); site FAQ permits only short clips under stated conditions No video, audio, transcript or frame redistribution permitted

Essence of Linear Algebra

Author: Grant Sanderson (3Blue1Brown) | Original Source Link ↗

Relevant Section / Range: Chapter 4: Matrix multiplication as composition

Verified Version & Date: 2016-2023 (Verified: 2026-10-30)

Selection Rationale:

Unequaled geometric and visual intuition for vectors, linear transformations, matrix multiplication, and inverses before tensor operations.

Source Prerequisites:

High-school coordinate geometry

Safe Fallback (Offline Resilience):

Geometric explanations, worked formulas, and coordinate exercises are fully provided as original lesson text.

Compared Candidate Sources & Selection Analysis (2)
MIT OCW 18.06 Linear Algebra (Prof. Gilbert Strang) ↗ compared_alternative [CC BY-NC-SA 4.0]

Established textbook/lecture approach, strong mathematical rigor alongside 3Blue1Brown visual intuition.

Linear Algebra Done Right (Sheldon Axler) ↗ compared_rejected_for_curriculum [CC BY-NC (4th edition, Open Access)]

Theoretical determinant-free proof structure is mathematically elegant but less aligned with computational tensor engineering.

linking CC BY-NC-SA 4.0 noncommercial share-alike

18.06 Linear Algebra (Prof. Gilbert Strang)

Author: MIT OpenCourseWare (Prof. Gilbert Strang) | Original Source Link ↗

Relevant Section / Range: Lecture 3: matrix multiplication and its four viewpoints

Verified Version & Date: Spring 2010 (Verified: 2026-10-30)

Selection Rationale:

Established row-picture and column-picture treatment of linear systems, matrix products, and invertibility; rigorous complement to the 3Blue1Brown visual series.

Source Prerequisites:

High-school algebra and coordinate geometry

Safe Fallback (Offline Resilience):

All row/column-picture derivations and worked examples are written as original lesson text and run offline.

Compared Candidate Sources & Selection Analysis (3)
3Blue1Brown: Essence of Linear Algebra ↗ compared_selected_for_days_22_24 [3Blue1Brown copyright (all rights reserved); site FAQ permits only short clips under stated conditions]

Selected as co-primary: geometric visualizations paired with Strang's algebraic rigor.

Linear Algebra Done Right (Sheldon Axler) ↗ compared_rejected_for_curriculum [CC BY-NC (4th edition, Open Access)]

Determinant-free theoretical treatment is elegant but further from the computational shape accounting required here.

Immersive Linear Algebra (online interactive book) ↗ compared_rejected_for_days_22_24 [Free online (site terms; no OSI/CC license declared)]

Interactive figures are attractive, but the absence of a declared open license makes rights disposition unclear; kept out of the resource set.

02 120 min

Shape ledgers: predicting product dimensions

Independent practice: record predictions before checking results; assistance only after evidence is logged.

Prerequisites

Complete the previous activity; use the listed recovery link if blocked.

Learning Objective

Keep a written shape ledger for every product: (m x k)(k x n) -> (m x n), the inner dimensions must match, and a mismatch is an error to predict, not to memorize.

Instructions

Write both shapes, check that the inner dimensions agree, then predict the output shape before computing a single entry. Apply the same ledger to model blocks: an MLP takes X of shape (batch x d_model) through W1 (d_model x d_ff) then W2 (d_ff x d_model). Each weight matrix is a rank-2 tensor inside the model.

Practice & Prediction Log

Predict shapes and values. (a) C (2x3) = [[1,0,2],[-1,3,1]] times D (3x2) = [[3,1],[2,1],[1,0]]: output shape? Compute all four entries. (b) Ledger for a transformer MLP: X (32 x 768) * W1 (768 x 3072) then * W2 (3072 x 768). (c) Is (2x3)(2x3) valid?

Independent Mastery Criterion

You predict every output shape before computing, reproduce C*D = [[5,1],[4,2]] exactly, and state the rule 'inner dimensions must match' with one failing example.

Authoritative Source Resources & Provenance

linking CC BY-NC-SA 4.0 noncommercial share-alike

18.06 Linear Algebra (Prof. Gilbert Strang)

Author: MIT OpenCourseWare (Prof. Gilbert Strang) | Original Source Link ↗

Relevant Section / Range: Lecture 3: (m x k)(k x n) -> m x n dimension rule

Verified Version & Date: Spring 2010 (Verified: 2026-10-30)

Selection Rationale:

Established row-picture and column-picture treatment of linear systems, matrix products, and invertibility; rigorous complement to the 3Blue1Brown visual series.

Source Prerequisites:

High-school algebra and coordinate geometry

Safe Fallback (Offline Resilience):

All row/column-picture derivations and worked examples are written as original lesson text and run offline.

Compared Candidate Sources & Selection Analysis (3)
3Blue1Brown: Essence of Linear Algebra ↗ compared_selected_for_days_22_24 [3Blue1Brown copyright (all rights reserved); site FAQ permits only short clips under stated conditions]

Selected as co-primary: geometric visualizations paired with Strang's algebraic rigor.

Linear Algebra Done Right (Sheldon Axler) ↗ compared_rejected_for_curriculum [CC BY-NC (4th edition, Open Access)]

Determinant-free theoretical treatment is elegant but further from the computational shape accounting required here.

Immersive Linear Algebra (online interactive book) ↗ compared_rejected_for_days_22_24 [Free online (site terms; no OSI/CC license declared)]

Interactive figures are attractive, but the absence of a declared open license makes rights disposition unclear; kept out of the resource set.

linking BSD 3-Clause License

NumPy Reference: The N-dimensional array (ndarray)

Author: NumPy Developers | Original Source Link ↗

Relevant Section / Range: ndarray.shape semantics and matrix product dimensions

Verified Version & Date: NumPy 2.x documentation (Verified: 2026-10-30)

Selection Rationale:

Primary semantics for ndarray shape, strides, reshape orders, and view-versus-copy behavior used in tensor memory accounting.

Source Prerequisites:

Basic Python

Safe Fallback (Offline Resilience):

Shape, stride, and memory-ledger exercises are fully explained with local runnable examples.

Compared Candidate Sources & Selection Analysis (2)
PyTorch Documentation: torch.Tensor ↗ compared_alternative [Modified BSD]

Equivalent tensor semantics in the framework used later in the curriculum; NumPy reference selected to keep Days 22-24 dependency-free.

NumPy User Guide: Broadcasting ↗ compared_selected_for_days_22_24 [BSD 3-Clause License]

Selected as co-primary for Day 24: the broadcasting rules page pairs with the ndarray attribute reference.

03 120 min

Transpose, identity, and inverse intuition

Assistance allowed only after prediction and observed evidence are recorded.

Prerequisites

Complete the previous activity; use the listed recovery link if blocked.

Learning Objective

Use the transpose as a mirror across the diagonal with (A*B)^T = B^T * A^T, treat the identity I as the 'do nothing' map, and read A^-1 as the map that undoes A - which exists only when A collapses nothing.

Instructions

Transpose swaps rows and columns: (A^T)_ij = A_ji, and products reverse order under transpose. The identity matrix I satisfies I*x = x. An inverse A^-1 satisfies A*A^-1 = I. A matrix with a nonzero null space collapses a direction to zero, so no inverse can recover it; for a 2x2 matrix, det = ad - bc = 0 signals this collapse.

Practice & Prediction Log

(a) With C*D = [[5,1],[4,2]] from the last activity, predict (C*D)^T and verify it equals D^T * C^T. (b) For A = [[2,1],[1,1]], verify A^-1 = [[1,-1],[-1,2]] by computing A * A^-1. (c) S = [[1,2],[2,4]]: find det(S) and a nonzero x with S*x = 0; can S be inverted?

Independent Mastery Criterion

Your trace verifies A * A^-1 = I entry-by-entry, you demonstrate the transpose product rule numerically, and you explain 'singular means some direction is destroyed' using S*(2,-1) = 0.

Authoritative Source Resources & Provenance

linking 3Blue1Brown copyright (all rights reserved); site FAQ permits only short clips under stated conditions No video, audio, transcript or frame redistribution permitted

Essence of Linear Algebra

Author: Grant Sanderson (3Blue1Brown) | Original Source Link ↗

Relevant Section / Range: Chapters 5-6: determinants and inverse matrices

Verified Version & Date: 2016-2023 (Verified: 2026-10-30)

Selection Rationale:

Unequaled geometric and visual intuition for vectors, linear transformations, matrix multiplication, and inverses before tensor operations.

Source Prerequisites:

High-school coordinate geometry

Safe Fallback (Offline Resilience):

Geometric explanations, worked formulas, and coordinate exercises are fully provided as original lesson text.

Compared Candidate Sources & Selection Analysis (2)
MIT OCW 18.06 Linear Algebra (Prof. Gilbert Strang) ↗ compared_alternative [CC BY-NC-SA 4.0]

Established textbook/lecture approach, strong mathematical rigor alongside 3Blue1Brown visual intuition.

Linear Algebra Done Right (Sheldon Axler) ↗ compared_rejected_for_curriculum [CC BY-NC (4th edition, Open Access)]

Theoretical determinant-free proof structure is mathematically elegant but less aligned with computational tensor engineering.

linking CC BY-NC-SA 4.0 noncommercial share-alike

18.06 Linear Algebra (Prof. Gilbert Strang)

Author: MIT OpenCourseWare (Prof. Gilbert Strang) | Original Source Link ↗

Relevant Section / Range: Lecture 3: inverses and transposes; Lecture 4: factorization

Verified Version & Date: Spring 2010 (Verified: 2026-10-30)

Selection Rationale:

Established row-picture and column-picture treatment of linear systems, matrix products, and invertibility; rigorous complement to the 3Blue1Brown visual series.

Source Prerequisites:

High-school algebra and coordinate geometry

Safe Fallback (Offline Resilience):

All row/column-picture derivations and worked examples are written as original lesson text and run offline.

Compared Candidate Sources & Selection Analysis (3)
3Blue1Brown: Essence of Linear Algebra ↗ compared_selected_for_days_22_24 [3Blue1Brown copyright (all rights reserved); site FAQ permits only short clips under stated conditions]

Selected as co-primary: geometric visualizations paired with Strang's algebraic rigor.

Linear Algebra Done Right (Sheldon Axler) ↗ compared_rejected_for_curriculum [CC BY-NC (4th edition, Open Access)]

Determinant-free theoretical treatment is elegant but further from the computational shape accounting required here.

Immersive Linear Algebra (online interactive book) ↗ compared_rejected_for_days_22_24 [Free online (site terms; no OSI/CC license declared)]

Interactive figures are attractive, but the absence of a declared open license makes rights disposition unclear; kept out of the resource set.

04 120 min

Independent mastery: products, shapes, and invertibility

Independent Mastery Check (no AI assistance).

Prerequisites

Complete the previous activity; use the listed recovery link if blocked.

Learning Objective

Without assistance, compute a 2x2 product, predict an attention shape ledger, and decide whether a given matrix is invertible; write a recovery note for any miss.

Instructions

Closed book: predict first, compute second, compare. Log every shape before multiplying. If a result surprises you, record which rule was misapplied and note the recovery link.

Practice & Prediction Log

Tasks: (a) P*M for P = [[1,1],[0,1]], M = [[2,0],[1,1]]; (b) attention ledger: Q (8 x 64) times K^T where K is (8 x 64), then softmax, then times V (8 x 64): predict each output shape; (c) is [[2,4],[1,2]] invertible?

Independent Mastery Criterion

All predictions verified unaided; the (8x8) attention-score shape and the singularity argument are written in your own words with the full shape ledger.

Authoritative Source Resources & Provenance

linking CC BY-NC-SA 4.0 noncommercial share-alike

18.06 Linear Algebra (Prof. Gilbert Strang)

Author: MIT OpenCourseWare (Prof. Gilbert Strang) | Original Source Link ↗

Relevant Section / Range: Lecture 1-3 problem sets on products and inverses

Verified Version & Date: Spring 2010 (Verified: 2026-10-30)

Selection Rationale:

Established row-picture and column-picture treatment of linear systems, matrix products, and invertibility; rigorous complement to the 3Blue1Brown visual series.

Source Prerequisites:

High-school algebra and coordinate geometry

Safe Fallback (Offline Resilience):

All row/column-picture derivations and worked examples are written as original lesson text and run offline.

Compared Candidate Sources & Selection Analysis (3)
3Blue1Brown: Essence of Linear Algebra ↗ compared_selected_for_days_22_24 [3Blue1Brown copyright (all rights reserved); site FAQ permits only short clips under stated conditions]

Selected as co-primary: geometric visualizations paired with Strang's algebraic rigor.

Linear Algebra Done Right (Sheldon Axler) ↗ compared_rejected_for_curriculum [CC BY-NC (4th edition, Open Access)]

Determinant-free theoretical treatment is elegant but further from the computational shape accounting required here.

Immersive Linear Algebra (online interactive book) ↗ compared_rejected_for_days_22_24 [Free online (site terms; no OSI/CC license declared)]

Interactive figures are attractive, but the absence of a declared open license makes rights disposition unclear; kept out of the resource set.

linking 3Blue1Brown copyright (all rights reserved); site FAQ permits only short clips under stated conditions No video, audio, transcript or frame redistribution permitted

Essence of Linear Algebra

Author: Grant Sanderson (3Blue1Brown) | Original Source Link ↗

Relevant Section / Range: Chapters 4-6 review

Verified Version & Date: 2016-2023 (Verified: 2026-10-30)

Selection Rationale:

Unequaled geometric and visual intuition for vectors, linear transformations, matrix multiplication, and inverses before tensor operations.

Source Prerequisites:

High-school coordinate geometry

Safe Fallback (Offline Resilience):

Geometric explanations, worked formulas, and coordinate exercises are fully provided as original lesson text.

Compared Candidate Sources & Selection Analysis (2)
MIT OCW 18.06 Linear Algebra (Prof. Gilbert Strang) ↗ compared_alternative [CC BY-NC-SA 4.0]

Established textbook/lecture approach, strong mathematical rigor alongside 3Blue1Brown visual intuition.

Linear Algebra Done Right (Sheldon Axler) ↗ compared_rejected_for_curriculum [CC BY-NC (4th edition, Open Access)]

Theoretical determinant-free proof structure is mathematically elegant but less aligned with computational tensor engineering.