15 / 90

Day 15: Reproduce, evaluate, and hand off the text project

Run the complete actual project, use held-out behavior checks, and document reusable deterministic data preparation.

Self-Directed Learning & Python 480 active minutes (breaks and unattended runtime excluded)
01 120 min

Reproduce the complete project

Agent-assisted execution is allowed only with command, permission, and output review.

Prerequisites

Days 13–14 and a clean temporary environment.

Learning Objective

Execute the actual text_tools CLI from an unrelated working directory and regenerate stats, vocabulary, splits, and manifest artifacts.

Instructions

Required work: run stats and split commands against names.txt or a documented local corpus, capture versions/seeds, and compare outputs. Optional work: run a second corpus.

Practice and prediction

Predict the artifact names, split counts, and manifest fields before execution. Explain which timestamp field prevents manifests from being byte-identical while split files remain reproducible.

Learning Project Submission & Evidence

0 / 4096 bytes
Declared Assistance Condition

Current Submitted Evidence Details

No submission recorded yet.
Independent mastery criterion

Learner reproduces the public command and explains observed artifacts from a process-isolated run.

Source resources and provenance

original_explanation Apache-2.0 AND CC-BY-SA-4.0 share-alike

Text and Data Tools Learning Project

Author: Curriculum team (delivered in Ticket 16) | Original Source Link ↗

Relevant Section / Range: cli.py --stats/--split output artifacts: train.txt, val.txt, test.txt, stats.json, vocab.json, manifest.json

Verified Version & Date: Verified against the checked-in implementation on 2026-10-30

Selection Rationale:

Reproduction must target the actual project and its real artifact names so later tokenizer and model experiments can consume the same outputs.

Source Prerequisites:

Days 13–14 evidence: CLI usage and disposable Git workflows

Safe Fallback (Offline Resilience):

A clean temporary corpus and output directory reproduce all artifacts offline.

Compared Candidate Sources & Selection Analysis (2)
Names corpus (Karpathy makemore names.txt) ↗ compared_alternative [MIT]

A natural example corpus for the reproduction run; any documented local corpus also works, so it is not a required download.

tinyshakespeare corpus (karpathy/char-rnn) ↗ compared_alternative [MIT]

Another natural local corpus for reproduction; any documented corpus satisfies the exercise.

02 120 min

Independent held-out evaluation

Independent Mastery Check: no AI or reference implementation during the first run.

Prerequisites

Fresh held-out corpus and no reference answer visible.

Learning Objective

Use independent examples to test Unicode, empty input, malformed encoding, repeated records, ratio validation, and reproducibility.

Instructions

Required work: predict outcomes, run the CLI, record exit codes/stderr/artifacts, and classify pass, failure, or inconclusive honestly. Optional work: add a test only after the behavior is understood.

Practice and prediction

Explain why an empty file can succeed with zero statistics, why strict decoding should fail, and why equal lines make content-only leakage checks insufficient.

Independent mastery criterion

Unaided evaluation includes expected versus observed results and separates deterministic checks from substantive review.

Source resources and provenance

original_explanation Apache-2.0 AND CC-BY-SA-4.0 share-alike

Text and Data Tools Learning Project

Author: Curriculum team (delivered in Ticket 16) | Original Source Link ↗

Relevant Section / Range: CLI edge behavior: empty input, strict/replace decoding, ratio validation, seed determinism

Verified Version & Date: Verified against the checked-in implementation on 2026-10-30

Selection Rationale:

Held-out cases are chosen against the real CLI seams (encoding, ratios, seeds) rather than imagined behavior.

Source Prerequisites:

Completion of day-15-act-01

Safe Fallback (Offline Resilience):

All held-out inputs are small local files created by the learner in a temporary directory.

Compared Candidate Sources & Selection Analysis (2)
Python unittest/pytest documentation ↗ compared_alternative [PSF License Version 2]

Useful for formalizing cases later, but the evaluation is intentionally run at the CLI boundary first.

Hypothesis property-based testing ↗ compared_alternative [MPL-2.0]

Property-based generation could extend held-out coverage later; manual edge cases keep the evaluation transparent first.

03 120 min

Data provenance and future model reuse

Agent-assisted engineering check: agent may challenge the report, but evidence and claims are learner-verified.

Prerequisites

Completed held-out evaluation and manifest artifacts.

Learning Objective

Explain how deterministic data preparation becomes a contract for later tokenizer, language-model, and evaluation experiments.

Instructions

Required work: trace source hash → ordered records → seed → split hashes → later consumer, and state where leakage or distribution shift can still occur. Optional work: sketch a tokenizer input contract.

Practice and prediction

Why should a later model experiment consume the recorded train split and untouched validation/test splits rather than reshuffle after seeing metrics?

Learning Project Submission & Evidence

0 / 4096 bytes
Declared Assistance Condition

Current Submitted Evidence Details

No submission recorded yet.
Independent mastery criterion

Learner gives a traceable reuse explanation with at least three assumptions and three limitations.

Source resources and provenance

original_explanation Apache-2.0 AND CC-BY-SA-4.0 share-alike

Text and Data Tools Learning Project

Author: Curriculum team (delivered in Ticket 16) | Original Source Link ↗

Relevant Section / Range: Generated manifest.json provenance fields: source_sha256 → decoded_text_sha256 → total_lines → seed → split_counts → split_sha256 (plus leakage_check); consumer/reuse metadata is not in manifest.json but in project.json reusable_by_projects (makemore, minbpe, nanogpt)

Verified Version & Date: Verified against the checked-in implementation on 2026-10-30

Selection Rationale:

The reuse contract is declared by the project's own manifest and project.json, preserving attribution and provenance for later study days.

Source Prerequisites:

Completion of day-15-act-02

Safe Fallback (Offline Resilience):

The manifest generated in earlier activities carries all required provenance fields.

Compared Candidate Sources & Selection Analysis (2)
Datasheets for Datasets (Gebru et al.) ↗ compared_alternative [arXiv non-exclusive distribution]

The academic framing of dataset documentation; the project manifest already implements the subset needed for deterministic reuse.

Croissant dataset metadata format (MLCommons) ↗ compared_alternative [Apache-2.0]

Richer dataset-metadata schema; the project manifest already records the needed provenance subset.

04 120 min

Independent portfolio handoff

Independent final handoff: the sealed unaided original and its evidence are preserved first; later editorial assistance is disclosed, unassessed, and cannot create or overwrite mastery evidence.

Prerequisites

All project, Git, and evaluation evidence is retained.

Learning Objective

Produce a PR-style handoff containing requirements, commands, results, review findings, assumptions, limits, and unresolved blockers.

Instructions

Required work: record your predicted/expected evidence first, run your own checks to collect the real observed evidence (your own checks are part of the unaided attempt), then write, lock (submit, date) and PRESERVE the completed first unaided report with its real evidence BEFORE any editorial, reference, or AI review; link only local/disposable evidence and state that external publication and paid compute are not authorized. Optional work: a separately labeled UNASSESSED polish may follow but cannot overwrite the sealed original; a substantive gap revealed later requires a fresh unaided remediation check.

Practice and prediction

A complete handoff must let a later learner rerun the project without trusting your narrative. List the exact command, seed, corpus identity, expected outputs, and known limits.

Independent mastery criterion

Portfolio is complete and honest: the sealed unaided original plus its real evidence is preserved before any editorial/AI comparison, and a reviewer can rerun the project and distinguish tested behavior, human review, and unavailable evidence.

Source resources and provenance

original_explanation Apache-2.0 AND CC-BY-SA-4.0 share-alike

Text and Data Tools Learning Project

Author: Curriculum team (delivered in Ticket 16) | Original Source Link ↗

Relevant Section / Range: project.json license and supported_languages attribution carried into the handoff

Verified Version & Date: Verified against the checked-in implementation on 2026-10-30

Selection Rationale:

The handoff must cite the real project, its license, and its reproducible commands so a reviewer can rerun without trusting the narrative.

Source Prerequisites:

Completion of day-15-act-03

Safe Fallback (Offline Resilience):

All linked evidence is local or disposable by design; external publication is not required or authorized.

Compared Candidate Sources & Selection Analysis (2)
Choose a License guidance ↗ compared_alternative [CC BY 3.0 (site content) / MIT (site code)]

Helpful for understanding the Apache-2.0 AND CC-BY-SA-4.0 attribution carried in project.json; not required to complete the handoff.

SPDX License List ↗ compared_alternative [spdx.org site copyright (All Rights Reserved footer; list license not verified) / linked reference]

Standard license identifiers behind the project's Apache-2.0 AND CC-BY-SA-4.0 declaration.