90 / 90

Day 90: Portfolio Assembly, Provenance Audit & the Next-Learning Plan

Assemble a reproducible evidence portfolio over the accepted Learning Projects, re-run the public verification commands, and close the course with an honest, evidence-based plan.

Self-Directed Learning and Python 480 active minutes (8 hours)
01 120 min

The Portfolio Checklist: Provenance, Reproducibility, Claims, and Limitations

Agent-Assisted Engineering Check

Prerequisites

Day 89 mastery ledger; the completed Learning Project sequence.

Learning Objective

Define the checklist every portfolio entry must satisfy — claim, evidence path, reproduction command, observed result, cost/authorization status, and limitations — grounded in the accepted project evidence and industry provenance formats.

Instructions

1. Required fields per portfolio entry: {project, claim, evidence_path, verify_command, observed_result, status, limitations}. Status vocabulary is fixed: verified-local (command-verified on this machine), verified-disposable (verified inside disposable fixtures), blocked-awaiting-authorization (requires approval/paid/live access not yet granted).
2. Map the accepted evidence to entries:
- Autograd project (ticket 17): projects/autograd engine + gradcheck; verified-local via python3 projects/autograd/cli.py forward-backward and gradcheck.
- Resumable pretraining (tickets 18/19): projects/pretrain/artifacts/run_artifact.json — 10-step CPU smoke, train_loss 4.0316, val_loss 4.0372, 7440 params; verified-local. Cloud-GPU pretraining evidence: blocked-awaiting-authorization.
- LoRA mechanics (tickets 20/21): projects/lora/artifacts — loss 4.0976→4.0945, perplexity 60.20→60.01, merge_verified true (mechanics check only); cloud before/after comparison: blocked-awaiting-authorization per projects/lora/evidence/comparison_contract.json.
- Evaluable agent (tickets 22/23): projects/agent_learning deterministic-v1 scorer + fixture; verified-local. Live-model agent evaluation: not-run-awaiting-authorization per evidence/live_eval_contract.json.
- CI delivery practice (ticket 24): scripts/ci_practice_runner.py + tests/engineering/test_ci_practice.py; verified-disposable.
3. Honesty rules, verbatim: never upgrade a blocked item to verified; cite the commit hash and artifact path for every claim; a smoke-scale result is never presented as convergence.
4. Model-card and datasheet conventions define which limitation fields industry expects — copy the fields, not the text.

Practice & Prediction Log

Draft checklist.md listing the six required fields per entry, the three status values, and the two honesty rules; annotate which accepted project maps to each status.

Independent Mastery Criterion

Checklist covers all six fields, the three statuses, and correctly labels each accepted project's verified vs blocked-awaiting-authorization evidence.

Authoritative Source Resources & Provenance

linking CC BY 4.0

The Turing Way: A Handbook for Reproducible, Ethical and Collaborative Data Science

Author: The Turing Way Community | Original Source Link ↗

Relevant Section / Range: Reproducible Research chapter: version control, testing, environment capture

Verified Version & Date: Zenodo DOI 10.5281/zenodo.3233853, continuously updated (Verified: 2026-10-30)

Selection Rationale:

Community-maintained CC-BY handbook codifying reproducibility checklists (versioned code, environment capture, rerunnable commands); directly defines the Day 90 reproduction-verification and provenance-audit workflow.

Source Prerequisites:

Basic version control and command-line familiarity.

Safe Fallback (Offline Resilience):

The portfolio checklist and verification log schema are authored locally; reproduction commands run without the handbook.

Compared Candidate Sources & Selection Analysis (2)
Wilson et al. Good enough practices in scientific computing (PLOS CB 2017) ↗ compared_alternative [CC BY 4.0]

Concise practices paper; The Turing Way's maintained, checklist-driven coverage of environments and reproducibility is broader and kept current.

Sandve et al. Ten Simple Rules for Reproducible Computational Research (PLOS CB 2013) ↗ compared_alternative [CC BY 4.0]

Foundational rules list; The Turing Way expands the same rules into actionable checklists used verbatim for the Day 90 verification run.

linking arXiv non-exclusive license / Open Access

Model Cards for Model Reporting

Author: Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru | Original Source Link ↗

Relevant Section / Range: Section 4 Model Card Sections (4.1-4.9)

Verified Version & Date: FAT* 2019 / arXiv:1810.03993v2 (Verified: 2026-10-30)

Selection Rationale:

Primary standard for structured, transparent model reporting, mandating explicit disclosure of intended uses, evaluation slices, out-of-scope tasks, ethical considerations, and negative results.

Source Prerequisites:

Evaluation reporting, demographic and performance slicing, model limitations and caveats.

Safe Fallback (Offline Resilience):

Complete Model Card Markdown templates, section schemas, and evaluation disclosure checklists are provided in the curriculum.

Compared Candidate Sources & Selection Analysis (2)
Bommasani et al. The Foundation Model Transparency Index ↗ compared_alternative [Open Access]

Corporate transparency audit index; Mitchell et al. provides the concrete technical artifact specification for individual model releases.

Pushkarna et al. Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI ↗ compared_alternative [Open Access]

Dataset documentation focus; Mitchell et al. provides the required specification for trained model artifact reporting.

02 120 min

Assemble the Portfolio Manifest

Independent Mastery Check (no AI assistance).

Prerequisites

Day 90 Activity 1 checklist; assistance condition: independent.

Learning Objective

Write portfolio_manifest.json mapping every learning claim to a repository artifact path, a reproduction command, an honest status, and a limitations note — unaided, with every path resolving inside the repo.

Instructions

1. Manifest schema per entry: {project, claim, evidence_path, verify_command, observed_result, status, limitations}.
2. Write one entry per accepted project (autograd, pretrain, lora, agent_learning, ci-practice) plus one for the curriculum itself.
3. Every evidence_path must resolve in the repository; verify each with ls or test -f before recording.
4. Every claim cites the observed value: e.g., autograd gradcheck max-abs-diff within tolerance; pretrain run_artifact.json total_parameters=7440, val_loss=4.0372; lora eval_results.json loss_delta=0.0031, merge_verified=true; agent_learning scorer output for the summarize-notes fixture; ci_practice_runner stages evidence.
5. Limitations field is mandatory and specific: e.g., '10-step CPU smoke verifies plumbing, not convergence'; 'deterministic adapter is not a live model'; 'synthetic instruction data is not a benchmark'; 'random-weight smoke mechanics are not quality evidence'; 'contract validation is not an authorized cloud run'.

Practice & Prediction Log

Emit portfolio_manifest.json with at least 5 project entries + 1 curriculum entry; assert each evidence_path exists and each status uses the fixed vocabulary.

Independent Mastery Criterion

Manifest resolves real artifact paths, honest statuses, and explicit limitations for every claim — produced unaided.

Authoritative Source Resources & Provenance

linking arXiv non-exclusive license / Open Access

Datasheets for Datasets

Author: Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, Kate Crawford | Original Source Link ↗

Relevant Section / Range: Section 3 Questions and Workflow (3.1 Motivation through 3.7 Maintenance)

Verified Version & Date: Communications of the ACM 2021 / arXiv:1803.09010v8 (Verified: 2026-10-30)

Selection Rationale:

Foundational literature introducing standardized documentation for machine learning datasets, specifying questions on dataset composition, collection mechanisms, preprocessing, and distribution rights.

Source Prerequisites:

Dataset provenance, curation motivation, preprocessing transparency, maintenance lifecycle.

Safe Fallback (Offline Resilience):

Datasheet questionnaires, provenance schemas, and dataset auditing checklists are completely integrated into lesson materials.

Compared Candidate Sources & Selection Analysis (2)
Bender & Friedman. Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science ↗ compared_alternative [CC BY 4.0 (ACL Anthology, 2016 or later)]

Linguistic demographic documentation focus; Gebru et al. provides the universal engineering datasheet framework adopted across AI.

Holland et al. The Dataset Nutrition Label: A Framework to Drive Higher Data Quality Standards ↗ compared_alternative [Open Access]

Nutrition label metaphor; Gebru et al. is a widely cited framework for dataset documentation in both academic and industrial practice.

linking arXiv non-exclusive license / Open Access

Model Cards for Model Reporting

Author: Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru | Original Source Link ↗

Relevant Section / Range: Section 4 Model Card Sections (4.1-4.9)

Verified Version & Date: FAT* 2019 / arXiv:1810.03993v2 (Verified: 2026-10-30)

Selection Rationale:

Primary standard for structured, transparent model reporting, mandating explicit disclosure of intended uses, evaluation slices, out-of-scope tasks, ethical considerations, and negative results.

Source Prerequisites:

Evaluation reporting, demographic and performance slicing, model limitations and caveats.

Safe Fallback (Offline Resilience):

Complete Model Card Markdown templates, section schemas, and evaluation disclosure checklists are provided in the curriculum.

Compared Candidate Sources & Selection Analysis (2)
Bommasani et al. The Foundation Model Transparency Index ↗ compared_alternative [Open Access]

Corporate transparency audit index; Mitchell et al. provides the concrete technical artifact specification for individual model releases.

Pushkarna et al. Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI ↗ compared_alternative [Open Access]

Dataset documentation focus; Mitchell et al. provides the required specification for trained model artifact reporting.

03 120 min

Reproducibility Verification: Re-Run the Public Project Commands

Agent-Assisted Engineering Check

Prerequisites

Day 90 Activity 2 manifest; assistance condition: agent_assisted (the agent may help capture logs, but the verification evidence is the command output itself).

Learning Objective

Re-run each project's public verification command on a clean shell, capture outputs, and confirm the portfolio's claims reproduce — recording environment details and any drift honestly.

Instructions

1. Record the environment first: python3 --version, uname -a (or platform equivalent), git rev-parse --short HEAD.
2. Run each public command and capture stdout + exit code:
- python3 projects/autograd/cli.py forward-backward
- python3 projects/autograd/cli.py gradcheck
- python3 -m unittest tests.unit.test_autograd_engine
- python3 -m projects.agent_learning.cli evaluate --fixture summarize-notes
- python3 projects/lora/evidence/validate_contract.py (note --require-authorized must FAIL while approval is absent — record the refusal as correct behavior)
- python3 scripts/audit_calendar_ledger.py
3. Compare each output against the recorded artifact field (e.g., gradcheck tolerances, ledger totals, contract status blocked-awaiting-authorization). Run each command twice — once cold, once after reading the artifact — and diff the outputs.
4. Log verification_log.json: {command, exit_code, key_output, artifact_field_match: true|false, notes}. Any drift (path changes, version differences, deleted artifacts) is recorded, not hidden.

Practice & Prediction Log

Emit verification_log.json covering all six command groups; each row carries the actual observed output summary and the match verdict.

Independent Mastery Criterion

All reproduction commands executed with captured outputs; expected refusals marked correctly; mismatches recorded, not hidden.

Authoritative Source Resources & Provenance

linking CC BY 4.0

The Turing Way: A Handbook for Reproducible, Ethical and Collaborative Data Science

Author: The Turing Way Community | Original Source Link ↗

Relevant Section / Range: Reproducible Research chapter: version control, testing, environment capture

Verified Version & Date: Zenodo DOI 10.5281/zenodo.3233853, continuously updated (Verified: 2026-10-30)

Selection Rationale:

Community-maintained CC-BY handbook codifying reproducibility checklists (versioned code, environment capture, rerunnable commands); directly defines the Day 90 reproduction-verification and provenance-audit workflow.

Source Prerequisites:

Basic version control and command-line familiarity.

Safe Fallback (Offline Resilience):

The portfolio checklist and verification log schema are authored locally; reproduction commands run without the handbook.

Compared Candidate Sources & Selection Analysis (2)
Wilson et al. Good enough practices in scientific computing (PLOS CB 2017) ↗ compared_alternative [CC BY 4.0]

Concise practices paper; The Turing Way's maintained, checklist-driven coverage of environments and reproducibility is broader and kept current.

Sandve et al. Ten Simple Rules for Reproducible Computational Research (PLOS CB 2013) ↗ compared_alternative [CC BY 4.0]

Foundational rules list; The Turing Way expands the same rules into actionable checklists used verbatim for the Day 90 verification run.

linking BSD-3-Clause

PyTorch Tutorial: Saving and Loading Models

Author: PyTorch Documentation Team | Original Source Link ↗

Relevant Section / Range: Saving & Loading a General Checkpoint for Inference and/or Resuming Training

Verified Version & Date: PyTorch 2.x Documentation (Verified: 2026-10-30)

Selection Rationale:

Official documentation detailing state_dict serialization, optimizer state saving, and safe loading patterns.

Source Prerequisites:

Python, PyTorch tensors and nn.Module basics.

Safe Fallback (Offline Resilience):

All checkpoint serialization semantics, dictionary schema keys, and safe recovery steps are self-contained in lesson code.

Compared Candidate Sources & Selection Analysis (2)
TensorFlow tf.train.Checkpoint Guide ↗ compared_alternative [Apache 2.0]

TensorFlow trackable checkpoint architecture; PyTorch's explicit dictionary is our target implementation standard.

Hugging Face Safetensors Documentation ↗ compared_alternative [Apache 2.0]

Safetensors provides safe zero-copy tensor storage without pickle, compared directly when discussing pickle execution risks.

04 120 min

Final Review and the Evidence-Based Next-Learning Plan

Independent Mastery Check (no AI assistance).

Prerequisites

Day 90 Activities 1-3 complete; Day 89 mastery ledger.

Learning Objective

Produce the final curriculum review and next-learning plan: a spaced-review schedule anchored to the ledger's remaining gaps, continuation into the retained advanced annual route, and an honest completion statement.

Instructions

1. Summarize the mastery ledger: per-domain before/after totals, remaining gaps with recovery routes.
2. Schedule spaced retrieval for every still-unmet item at +1, +3, +7, and +30 days — each with a named resource and a new variant task, not the same item.
3. Map continued study to the retained advanced annual route (nav link: 历史年度路线 / legacy roadmap at /lessons/index.html): extended probability and statistics, larger-scale pretraining, deployment and serving — topics outside the 90-day scope.
4. Write the completion statement verbatim in spirit: finishing the 90-day course produces Learning Evidence for the demonstrated tasks — it is not an official certification, does not guarantee universal three-month mastery, and assisted results never stand in for independent ability. Unmet criteria remain visibly unmet.
5. Emit next_learning_plan.json: {remaining_gaps[], scheduled_reviews[], annual_route_links[], honest_scope_statement}.

Practice & Prediction Log

Produce next_learning_plan.json with scheduled reviews for every remaining gap and the honest scope statement recorded verbatim.

Independent Mastery Criterion

Final review document, spaced schedule, annual-route links, and honest scope statement completed — unmet criteria still visibly unmet.

Authoritative Source Resources & Provenance

linking Journal publisher copyright (all rights reserved) - lawful external linking only

Improving Students' Learning With Effective Learning Techniques: Promising Directions From Cognitive and Educational Psychology

Author: John Dunlosky, Katherine A. Rawson, Elizabeth J. Marsh, Mitchell J. Nathan, Daniel T. Willingham | Original Source Link ↗

Relevant Section / Range: Psychological Science in the Public Interest 14(1): 4-58; practice testing and distributed practice sections

Verified Version & Date: Psychological Science in the Public Interest, 2013 (Verified: 2026-10-30)

Selection Rationale:

Detailed comparative review rating practice testing and distributed practice as high-utility techniques while rating highlighting and rereading low-utility; anchors the retrieval, correction, and spaced-review design of Days 88-90.

Source Prerequisites:

None beyond general academic reading.

Safe Fallback (Offline Resilience):

Curriculum restates the utility ratings and application conditions in original prose; every task remains runnable offline.

Compared Candidate Sources & Selection Analysis (2)
Pashler et al. Learning Styles: Concepts and Evidence ↗ compared_alternative [Publisher copyright (originally Wiley; now SAGE Publications)]

Important negative result on learning-styles matching but narrower scope; Dunlosky et al. rates a broader set of techniques directly usable for this curriculum.

Brown, Roediger & McDaniel: Make It Stick: The Science of Successful Learning ↗ compared_alternative [Publisher copyright (Harvard University Press)]

Strong trade-book synthesis of the same evidence base; the peer-reviewed review is preferred for citation precision and technique-by-technique utility ratings.

linking CC BY-SA 4.0 (book text) / modified MIT (sample code) share-alike

Dive into Deep Learning (d2l.ai)

Author: Aston Zhang, Zachary C. Lipton, Mu Li, Alexander J. Smola | Original Source Link ↗

Relevant Section / Range: Section: Model Selection, Underfitting, and Overfitting; Chapter: Optimization Algorithms

Verified Version & Date: Current online edition (Verified: 2026-10-30)

Selection Rationale:

Primary open reference tying evaluation discipline (model selection, underfitting, overfitting) and optimization algorithms (gradient descent, minibatch) to the rest of the curriculum.

Source Prerequisites:

Linear algebra, calculus, and probability at the Days 19-28 level.

Safe Fallback (Offline Resilience):

All required formulas and worked numbers are reproduced as original explanations; the book is further reading.

Compared Candidate Sources & Selection Analysis (2)
Deep Learning (Goodfellow, Bengio, Courville) ↗ compared_alternative [MIT Press; free HTML for reading, all rights reserved]

Detailed treatment of optimization difficulties (chapter 8) including saddle points and plateaus; all-rights-reserved text kept as a linking-only alternative.

fast.ai: Practical Deep Learning for Coders ↗ compared_alternative [Course site copyright / Apache-2.0 (fastai code)]

Strong practitioner framing, but organized around the fastai library and video lessons rather than self-contained mathematics.