Day 81: Controlled Delivery, Merge Recovery & Verified Rollback

01 120 min

Controlled Delivery Pipeline and Release Evidence

Agent-Assisted Engineering Check

Prerequisites

Day 80 approved review packet; Ticket 24 CI practice runner.

Learning Objective

Run the clean-install and controlled-deployment pipeline stages for the approved change and capture machine-verifiable release evidence instead of screenshots of commands.

Instructions

1. Package and verify: python3 scripts/ci_practice_runner.py --stage clean_install --json — the runner copies projects/text_tools into a pristine sandbox, creates input data, and executes the CLI end-to-end.
2. Inspect the evidence payload: stage clean_install success=true, returncode==0, valid JSON stats output (total_chars, total_tokens, vocab_size).
3. Understand the delivery model: versioned release directories behind a symlink flip (blue/green-style), so rollback is a pointer change, not a rebuild.
4. Note the authorization boundary: this pipeline proves the artifact locally; pushing to a shared remote, real deployment, and any paid infrastructure still require explicit separate authorization.

Practice & Prediction Log

Save the stage output to release_evidence.json and annotate: artifact path, exit code, the health-check command used, and why symlink-based versioned releases make rollback cheap.

Mark as Completed

Independent Mastery Criterion

Produce a repeatable, machine-verifiable release record for an approved change.

Authoritative Source Resources & Provenance

linking CC BY 4.0

GitHub Actions Documentation

Author: GitHub, Inc. | Original Source Link ↗

Relevant Section / Range: N/A

Verified Version & Date: GitHub Docs continuous publication (Verified: 2026-10-30)

Selection Rationale:

Primary reference for pipeline stages, required status checks, and branch protection mechanics that the local CI practice runner mirrors in a disposable environment.

Source Prerequisites:

YAML familiarity, shell commands, and Git hosting vocabulary.

Safe Fallback (Offline Resilience):

Pipeline stage semantics (lint, test, package, deploy gate) are exercised offline by scripts/ci_practice_runner.py; no GitHub account or push authorization is required for the practice.

Compared Candidate Sources & Selection Analysis (2)

GitLab CI/CD Documentation ↗ compared_alternative [CC BY-SA 4.0]

Equivalent pipeline concepts under a different provider; GitHub Actions matches the hosted-workflow conventions used in this project's delivery story.

Jenkins User Documentation ↗ compared_alternative [CC BY-SA 4.0]

Self-hosted CI server documentation; GitHub Actions better matches the hosted pipeline model the learner will meet first.

linking CC BY-NC-ND 4.0 noncommercial no-derivatives

Site Reliability Engineering, Chapter 8: Release Engineering

Author: Dinah McNutt (ed. Betsy Beyer, Tim Harvey), in Site Reliability Engineering (eds. Betsy Beyer, Chris Jones, Jennifer Petoff, Niall Richard Murphy; Google) | Original Source Link ↗

Relevant Section / Range: Continuous Build and Deployment (Building, Branching, Testing, Packaging, Deployment)

Verified Version & Date: O'Reilly 2016 free web edition (Verified: 2026-10-30)

Selection Rationale:

Widely used industry reference for hermetic builds, release approval gates, and rollback-ready deployment doctrine that anchor Day 81's controlled delivery and verified rollback exercises.

Source Prerequisites:

Build and deployment vocabulary; version control basics.

Safe Fallback (Offline Resilience):

The controlled delivery and rollback runbook is fully executable offline via the scripts/ci_practice_runner.py deploy stage, which emits recorded evidence payloads.

Compared Candidate Sources & Selection Analysis (2)

Martin Fowler: BlueGreenDeployment ↗ compared_alternative [Copyright Martin Fowler; free public web article]

Focused single-pattern bliki on blue/green cutover; the SRE chapter covers the wider release-engineering lifecycle including build hermeticity and rollback policy.

Foundational paid text on deployment pipelines; the SRE web edition provides equivalent operational doctrine at zero cost with a stable public URL.

02 120 min

Merge Conflict Resolution and Safe Recovery in a Disposable Repository

Independent Mastery Check (no AI assistance).

Prerequisites

Day 81 Activity 1.

Learning Objective

Resolve a real merge conflict unaided inside a disposable repository, preserving unrelated collaborator changes, and practice non-destructive recovery when a merge goes wrong.

Instructions

1. Execute the conflict stage: python3 scripts/ci_practice_runner.py --stage conflict --json. It builds a disposable bare 'upstream' repo (labeled simulation — no network, no pushes) with a genuinely diverged branch.
2. When conflict markers appear, read both sides: HEAD vs incoming. Keep the feature change AND the unrelated collaborator edit — do not delete either blindly.
3. Verify semantically, not textually: run the project tests inside the resolved tree before completing the merge.
4. Practice the escape hatch: git merge --abort returns to pre-merge state; git stash / a backup branch protects uncommitted work. Never run reset --hard or clean -fdx against work you cannot reproduce.
5. Confirm the runner reports conflict_resolved=true and unrelated_changes_preserved=true.

Practice & Prediction Log

Write recovery_log.md: the conflicted file, the two competing hunks, your resolution rationale, the post-merge test output, and the abort path you verified.

Mark as Completed

Independent Mastery Criterion

Resolve a genuine merge conflict independently, verified by a passing test run, with a practiced non-destructive recovery path.

Authoritative Source Resources & Provenance

linking CC BY-NC-SA 3.0 noncommercial share-alike

Pro Git, Second Edition

Author: Scott Chacon and Ben Straub | Original Source Link ↗

Relevant Section / Range: Chapter 3 (Git Branching), Section 7.7 (Reset Demystified), Sections 5.2/6.2 (Contributing to a Project)

Verified Version & Date: Second edition (Apress, 2014) (Verified: 2026-10-30)

Selection Rationale:

Freely licensed reference for branching, merging, conflict resolution, and safe recovery vocabulary used through the delivery workflow.

Source Prerequisites:

Basic git status/log/diff vocabulary from earlier study days

Safe Fallback (Offline Resilience):

git help <command> and the Git manual pages ship with Git and cover the same commands offline; every required workflow is stated locally in the lesson.

Compared Candidate Sources & Selection Analysis (2)

GitHub Skills interactive courses ↗ compared_alternative [MIT (varies by course)]

Interactive but requires a GitHub account and an external service; disposable local repositories keep the exercise self-contained.

Vendor-owned prose with no permissive license; Pro Git covers the same ground under CC BY-NC-SA.

linking CC BY-NC-ND 4.0 noncommercial no-derivatives

Site Reliability Engineering, Chapter 8: Release Engineering

Author: Dinah McNutt (ed. Betsy Beyer, Tim Harvey), in Site Reliability Engineering (eds. Betsy Beyer, Chris Jones, Jennifer Petoff, Niall Richard Murphy; Google) | Original Source Link ↗

Relevant Section / Range: Continuous Build and Deployment: Branching

Verified Version & Date: O'Reilly 2016 free web edition (Verified: 2026-10-30)

Selection Rationale:

Widely used industry reference for hermetic builds, release approval gates, and rollback-ready deployment doctrine that anchor Day 81's controlled delivery and verified rollback exercises.

Source Prerequisites:

Build and deployment vocabulary; version control basics.

Safe Fallback (Offline Resilience):

The controlled delivery and rollback runbook is fully executable offline via the scripts/ci_practice_runner.py deploy stage, which emits recorded evidence payloads.

Compared Candidate Sources & Selection Analysis (2)

Martin Fowler: BlueGreenDeployment ↗ compared_alternative [Copyright Martin Fowler; free public web article]

Focused single-pattern bliki on blue/green cutover; the SRE chapter covers the wider release-engineering lifecycle including build hermeticity and rollback policy.

Foundational paid text on deployment pipelines; the SRE web edition provides equivalent operational doctrine at zero cost with a stable public URL.

03 120 min

Health Failure Detection and the Verified Rollback Runbook

Agent-Assisted Engineering Check

Prerequisites

Day 81 Activity 2.

Learning Objective

Deploy the reviewed release to a disposable staging target, detect a failed deployment through observable health checks, and execute a documented rollback that verifiably restores the known-good version.

Instructions

1. Execute the deployment stage: python3 scripts/ci_practice_runner.py --stage deploy --json. It deploys release 1.0.0 (HEALTHY), then release 1.1.0 carrying a simulated schema fault.
2. Detect failure through the health seam: the v1.1.0 CLI exits code 2 with 'FATAL ERROR: Incompatible database schema version 2' — v2_health=UNHEALTHY. Detection is the nonzero exit plus invalid output, not a human opinion.
3. Run the rollback runbook: re-point the 'current' symlink back to releases/1.0.0, re-run the health command, confirm restored_version=1.0.0 and restored_health=HEALTHY.
4. Read release_evidence.json: the history array must show deploy 1.0.0 HEALTHY → deploy 1.1.0 UNHEALTHY → rollback_restored 1.0.0 HEALTHY.

Practice & Prediction Log

Write rollback_log.md containing: the failing health-check output verbatim, each rollback step, the post-rollback health output, and the full history array from release_evidence.json.

Mark as Completed

Independent Mastery Criterion

Execute a rollback whose success is proven by observable post-restore health evidence — verified restoration, not assumed recovery.

Authoritative Source Resources & Provenance

linking CC BY-NC-ND 4.0 noncommercial no-derivatives

Site Reliability Engineering, Chapter 8: Release Engineering

Author: Dinah McNutt (ed. Betsy Beyer, Tim Harvey), in Site Reliability Engineering (eds. Betsy Beyer, Chris Jones, Jennifer Petoff, Niall Richard Murphy; Google) | Original Source Link ↗

Relevant Section / Range: Continuous Build and Deployment: Deployment

Verified Version & Date: O'Reilly 2016 free web edition (Verified: 2026-10-30)

Selection Rationale:

Widely used industry reference for hermetic builds, release approval gates, and rollback-ready deployment doctrine that anchor Day 81's controlled delivery and verified rollback exercises.

Source Prerequisites:

Build and deployment vocabulary; version control basics.

Safe Fallback (Offline Resilience):

The controlled delivery and rollback runbook is fully executable offline via the scripts/ci_practice_runner.py deploy stage, which emits recorded evidence payloads.

Compared Candidate Sources & Selection Analysis (2)

Martin Fowler: BlueGreenDeployment ↗ compared_alternative [Copyright Martin Fowler; free public web article]

Focused single-pattern bliki on blue/green cutover; the SRE chapter covers the wider release-engineering lifecycle including build hermeticity and rollback policy.

Foundational paid text on deployment pipelines; the SRE web edition provides equivalent operational doctrine at zero cost with a stable public URL.

Continuous Integration

Author: Martin Fowler | Original Source Link ↗

Relevant Section / Range: Practices of Continuous Integration: Make the Build Self-Testing; Fix Broken Builds Immediately

Verified Version & Date: Second edition web article (May 2006) (Verified: 2026-10-30)

Selection Rationale:

Standard practitioner reference defining the CI contract this block rehearses: every change merges to a mainline verified by an automated build and test run, and failures are fixed before further work proceeds.

Source Prerequisites:

Version control basics, automated test suites, and build vocabulary.

Safe Fallback (Offline Resilience):

The CI gate contract (lint, test, package, verdict) is executed locally by scripts/ci_practice_runner.py with no external dependency.

Compared Candidate Sources & Selection Analysis (2)

GitHub Actions Documentation ↗ compared_alternative [CC BY 4.0]

Provider-specific workflow syntax; Fowler's article states the vendor-neutral CI discipline the learner must internalize first, while the docs are kept as the pipeline-mechanics source.

Vendor-oriented tutorial; Fowler's original definition keeps the explanation tool-agnostic and historically important.

04 120 min

Assessment: Delivery Report, Result Explanation and Limitations

Independent Mastery Check (no AI assistance).

Prerequisites

Day 81 Activities 1-3.

Learning Objective

Write the complete engineering delivery report tying requirement → review → CI → deployment → rollback into one auditable narrative, with explicit limitations and the authorization boundary stated.

Instructions

1. Assemble the packet: requirement.md, verdict.md, review_notes.md, review_packet.md, release_evidence.json, recovery_log.md, rollback_log.md.
2. Write the result explanation: for each acceptance criterion, cite the artifact and command that prove it; for each failure encountered, cite the observed output and the verified recovery.
3. Separate evidence classes honestly: independent mastery (your tests, your verdicts, your conflict resolution) vs agent-assisted performance (the produced diff, hunk summaries).
4. State limitations: deterministic-v1 adapter is a fixture, not live-model capability evidence; deployment targets are disposable; no shared remote was touched.
5. Restate the authorization boundary: pushes to shared remotes, production deployment, paid execution, and destructive operations were not performed and require specific authorization.

Practice & Prediction Log

Produce delivery_report.md with sections: Requirement & Acceptance, Verification Table, CI & Delivery Evidence, Rollback Evidence, Independent vs Agent-Assisted Evidence, Limitations & Authorization Boundary.

Mark as Completed

Independent Mastery Criterion

Deliver an auditable engineering report whose every claim re-executes, with honest separation of independent skill from agent-assisted output.

Authoritative Source Resources & Provenance

linking CC BY-NC-ND 4.0 noncommercial no-derivatives

Site Reliability Engineering, Chapter 8: Release Engineering

Author: Dinah McNutt (ed. Betsy Beyer, Tim Harvey), in Site Reliability Engineering (eds. Betsy Beyer, Chris Jones, Jennifer Petoff, Niall Richard Murphy; Google) | Original Source Link ↗

Relevant Section / Range: Philosophy: Hermetic Builds; Enforcement of Policies and Procedures

Verified Version & Date: O'Reilly 2016 free web edition (Verified: 2026-10-30)

Selection Rationale:

Widely used industry reference for hermetic builds, release approval gates, and rollback-ready deployment doctrine that anchor Day 81's controlled delivery and verified rollback exercises.

Source Prerequisites:

Build and deployment vocabulary; version control basics.

Safe Fallback (Offline Resilience):

The controlled delivery and rollback runbook is fully executable offline via the scripts/ci_practice_runner.py deploy stage, which emits recorded evidence payloads.

Compared Candidate Sources & Selection Analysis (2)

Martin Fowler: BlueGreenDeployment ↗ compared_alternative [Copyright Martin Fowler; free public web article]

Focused single-pattern bliki on blue/green cutover; the SRE chapter covers the wider release-engineering lifecycle including build hermeticity and rollback policy.

Foundational paid text on deployment pipelines; the SRE web edition provides equivalent operational doctrine at zero cost with a stable public URL.

linking CC BY 4.0

GitHub Actions Documentation

Author: GitHub, Inc. | Original Source Link ↗

Relevant Section / Range: N/A

Verified Version & Date: GitHub Docs continuous publication (Verified: 2026-10-30)

Selection Rationale:

Primary reference for pipeline stages, required status checks, and branch protection mechanics that the local CI practice runner mirrors in a disposable environment.

Source Prerequisites:

YAML familiarity, shell commands, and Git hosting vocabulary.

Safe Fallback (Offline Resilience):

Pipeline stage semantics (lint, test, package, deploy gate) are exercised offline by scripts/ci_practice_runner.py; no GitHub account or push authorization is required for the practice.

Compared Candidate Sources & Selection Analysis (2)

GitLab CI/CD Documentation ↗ compared_alternative [CC BY-SA 4.0]

Equivalent pipeline concepts under a different provider; GitHub Actions matches the hosted-workflow conventions used in this project's delivery story.

Jenkins User Documentation ↗ compared_alternative [CC BY-SA 4.0]

Self-hosted CI server documentation; GitHub Actions better matches the hosted pipeline model the learner will meet first.