Day 80: Diff Review, Independent Tests & CI Failure Recovery

01 120 min

Agent Diff Inspection and the PR-Style Review Checklist

Agent-Assisted Engineering Check

Prerequisites

Day 79; Day 14-15 Git collaboration basics; the Ticket 24 review checklist (curriculum/engineering/ci_delivery_rollback).

Learning Objective

Inspect the agent-produced diff line by line like a pull-request reviewer, applying the six-point checklist before any change is allowed toward CI.

Instructions

1. Stage nothing yet: run git diff and git status, then read every hunk the agent produced.
2. Apply the Ticket 24 checklist to each hunk: correctness and spec alignment; test adequacy; no assertion tampering; backward compatibility of APIs and file formats; zero secrets or unsafe shell calls; clean non-destructive history.
3. Classify every hunk accept / question / reject and write a one-line reason.
4. Let the agent summarize hunks and locate suspicious lines, but the verdict is yours — agent assistance ends at the decision.
5. Reject immediately on: weakened test assertions, out-of-scope files, hardcoded credentials, or unexplained reformatting.

Practice & Prediction Log

Produce review_notes.md: one row per diff hunk (file, hunk range, verdict, reason) plus the three most important checklist findings, each tied to a quoted line.

Mark as Completed

Independent Mastery Criterion

Complete a structured review in which every diff hunk carries an explicit, evidence-backed verdict.

Authoritative Source Resources & Provenance

linking CC BY-NC-SA 3.0 noncommercial share-alike

Pro Git, Second Edition

Author: Scott Chacon and Ben Straub | Original Source Link ↗

Relevant Section / Range: Chapter 3 (Git Branching), Section 7.7 (Reset Demystified), Sections 5.2/6.2 (Contributing to a Project)

Verified Version & Date: Second edition (Apress, 2014) (Verified: 2026-10-30)

Selection Rationale:

Freely licensed reference for branching, merging, conflict resolution, and safe recovery vocabulary used through the delivery workflow.

Source Prerequisites:

Basic git status/log/diff vocabulary from earlier study days

Safe Fallback (Offline Resilience):

git help <command> and the Git manual pages ship with Git and cover the same commands offline; every required workflow is stated locally in the lesson.

Compared Candidate Sources & Selection Analysis (2)

GitHub Skills interactive courses ↗ compared_alternative [MIT (varies by course)]

Interactive but requires a GitHub account and an external service; disposable local repositories keep the exercise self-contained.

Vendor-owned prose with no permissive license; Pro Git covers the same ground under CC BY-NC-SA.

Continuous Integration

Author: Martin Fowler | Original Source Link ↗

Relevant Section / Range: Practices of Continuous Integration: Everyone can see what's happening

Verified Version & Date: Second edition web article (May 2006) (Verified: 2026-10-30)

Selection Rationale:

Standard practitioner reference defining the CI contract this block rehearses: every change merges to a mainline verified by an automated build and test run, and failures are fixed before further work proceeds.

Source Prerequisites:

Version control basics, automated test suites, and build vocabulary.

Safe Fallback (Offline Resilience):

The CI gate contract (lint, test, package, verdict) is executed locally by scripts/ci_practice_runner.py with no external dependency.

Compared Candidate Sources & Selection Analysis (2)

GitHub Actions Documentation ↗ compared_alternative [CC BY 4.0]

Provider-specific workflow syntax; Fowler's article states the vendor-neutral CI discipline the learner must internalize first, while the docs are kept as the pipeline-mechanics source.

Vendor-oriented tutorial; Fowler's original definition keeps the explanation tool-agnostic and historically important.

02 120 min

Independent Verification Tests at the Public Seam

Independent Mastery Check (no AI assistance).

Prerequisites

Day 80 Activity 1.

Learning Objective

Write your own verification tests for the acceptance criteria — without agent help — at the public interface seam, so the change is proven by independent evidence rather than the agent's claim.

Instructions

1. Derive one unittest case per acceptance criterion from Activity 79-1.
2. Test at the public seam: call cli.evaluate / run_task / the public tool functions, not private internals.
3. Follow red-green: predict the expected value before running, watch a genuinely failing test fail, then verify it passes against the real implementation.
4. Cover one fault path (invalid arguments) and one budget invariant (steps <= 6) in addition to the happy path.
5. Never let the agent write or weaken the tests that judge its own work.

Practice & Prediction Log

Create tests/projects/test_day79_change.py with three unittest cases mapped to your acceptance criteria; run python -m unittest tests.projects.test_day79_change -v and record the output.

Mark as Completed

Independent Mastery Criterion

Author independent tests at the public seam that would catch a wrong implementation even if the agent claims success.

Authoritative Source Resources & Provenance

Continuous Integration

Author: Martin Fowler | Original Source Link ↗

Relevant Section / Range: Practices of Continuous Integration: Make the Build Self-Testing; Fix Broken Builds Immediately

Verified Version & Date: Second edition web article (May 2006) (Verified: 2026-10-30)

Selection Rationale:

Standard practitioner reference defining the CI contract this block rehearses: every change merges to a mainline verified by an automated build and test run, and failures are fixed before further work proceeds.

Source Prerequisites:

Version control basics, automated test suites, and build vocabulary.

Safe Fallback (Offline Resilience):

The CI gate contract (lint, test, package, verdict) is executed locally by scripts/ci_practice_runner.py with no external dependency.

Compared Candidate Sources & Selection Analysis (2)

GitHub Actions Documentation ↗ compared_alternative [CC BY 4.0]

Provider-specific workflow syntax; Fowler's article states the vendor-neutral CI discipline the learner must internalize first, while the docs are kept as the pipeline-mechanics source.

Vendor-oriented tutorial; Fowler's original definition keeps the explanation tool-agnostic and historically important.

linking Open Access / CC BY 4.0

SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

Author: Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, Karthik R. Narasimhan | Original Source Link ↗

Relevant Section / Range: Appendix A.3 Execution-Based Validation

Verified Version & Date: ICLR 2024 / arXiv:2310.06770v3 (Verified: 2026-10-30)

Selection Rationale:

Defines the standard pattern this block teaches: a bounded issue statement paired with executable fail-to-pass tests as verifiable acceptance criteria for agent-produced changes.

Source Prerequisites:

Unit testing, issue descriptions, and repository evaluation vocabulary.

Safe Fallback (Offline Resilience):

The acceptance-criteria pattern is exercised locally through the Ticket 22 fixture scorer and the Ticket 24 regression exercise without external datasets.

Compared Candidate Sources & Selection Analysis (2)

HumanEval: Evaluating Large Language Models Trained on Code ↗ compared_alternative [arXiv non-exclusive license / Open Access]

Function-level synthesis benchmark; SWE-bench models the repository-scoped issue-to-diff workflow that this delivery block rehearses.

Multi-SWE-bench: A Multilingual Benchmark for Issue Resolving ↗ compared_alternative [Open Access / CC BY 4.0]

Extends issue resolution to multiple programming languages; the original Python-centric SWE-bench provides a cleaner pedagogical anchor.

03 120 min

The Failing Agent Change: CI Diagnosis and Corrected Resubmission

Agent-Assisted Engineering Check

Prerequisites

Day 80 Activity 2.

Learning Objective

Carry a deliberately failing agent change through the full CI loop: observe the red failure, diagnose the root cause from the traceback, direct a corrected diff, and verify green — without ever touching the test assertions.

Instructions

1. Execute the regression stage: python3 scripts/ci_practice_runner.py --stage regression --json. It injects a controlled defect (unicode_multibyte_stripping) and runs the suite.
2. Read the actual failure: test_failed_red=true with AssertionError: 11 != 17 in test_multilingual_unicode_handling — the defect strips multi-byte characters, not the test.
3. Diagnose like a reviewer: locate the root-cause line in the implementation, not in the assertions; predict the corrected value before rerunning.
4. Apply the reviewed fix and confirm test_restored_green=true with the original assertions intact.
5. Record the RED traceback and the GREEN output verbatim as evidence.

Practice & Prediction Log

Log diagnosis.md: the exact failing assertion line, the root-cause line in implementation code, the applied fix description, and the final 'Ran N tests ... OK' output.

Mark as Completed

Independent Mastery Criterion

Diagnose and correct a real fault proven by unmodified assertions turning green — a failing agent change recovered through evidence, not assertion tampering.

Authoritative Source Resources & Provenance

linking CC BY 4.0

GitHub Actions Documentation

Author: GitHub, Inc. | Original Source Link ↗

Relevant Section / Range: N/A

Verified Version & Date: GitHub Docs continuous publication (Verified: 2026-10-30)

Selection Rationale:

Primary reference for pipeline stages, required status checks, and branch protection mechanics that the local CI practice runner mirrors in a disposable environment.

Source Prerequisites:

YAML familiarity, shell commands, and Git hosting vocabulary.

Safe Fallback (Offline Resilience):

Pipeline stage semantics (lint, test, package, deploy gate) are exercised offline by scripts/ci_practice_runner.py; no GitHub account or push authorization is required for the practice.

Compared Candidate Sources & Selection Analysis (2)

GitLab CI/CD Documentation ↗ compared_alternative [CC BY-SA 4.0]

Equivalent pipeline concepts under a different provider; GitHub Actions matches the hosted-workflow conventions used in this project's delivery story.

Jenkins User Documentation ↗ compared_alternative [CC BY-SA 4.0]

Self-hosted CI server documentation; GitHub Actions better matches the hosted pipeline model the learner will meet first.

Continuous Integration

Author: Martin Fowler | Original Source Link ↗

Relevant Section / Range: Practices of Continuous Integration: Make the Build Self-Testing; Fix Broken Builds Immediately

Verified Version & Date: Second edition web article (May 2006) (Verified: 2026-10-30)

Selection Rationale:

Standard practitioner reference defining the CI contract this block rehearses: every change merges to a mainline verified by an automated build and test run, and failures are fixed before further work proceeds.

Source Prerequisites:

Version control basics, automated test suites, and build vocabulary.

Safe Fallback (Offline Resilience):

The CI gate contract (lint, test, package, verdict) is executed locally by scripts/ci_practice_runner.py with no external dependency.

Compared Candidate Sources & Selection Analysis (2)

GitHub Actions Documentation ↗ compared_alternative [CC BY 4.0]

Provider-specific workflow syntax; Fowler's article states the vendor-neutral CI discipline the learner must internalize first, while the docs are kept as the pipeline-mechanics source.

Vendor-oriented tutorial; Fowler's original definition keeps the explanation tool-agnostic and historically important.

04 120 min

Assessment: Review Verdict and Merge Decision with Evidence

Independent Mastery Check (no AI assistance).

Prerequisites

Day 80 Activities 1-3.

Learning Objective

Assemble the complete pull-request review packet — diff verdicts, independent test results, CI stage output — and issue an approve/reject merge decision backed entirely by re-executable evidence.

Instructions

1. Collect review_notes.md, your unittest output, and the regression stage JSON into one packet.
2. Re-run one piece of evidence yourself (unittest or CI stage) to confirm the claims still hold.
3. Decide APPROVE or REJECT: approve only when every checklist item passes on re-executed evidence; otherwise reject with the specific blocking criterion.
4. Write merge readiness notes: risk level, rollback hint (which version restores service), and remaining limitations.
5. Keep the independent/agent-assisted split explicit: your tests and verdict are independent evidence; the diff is agent-assisted work.

Practice & Prediction Log

Write review_packet.md containing: decision (APPROVE/REJECT), the six checklist results with one-line evidence each, the re-run command and its output, and the rollback note.

Mark as Completed

Independent Mastery Criterion

Issue a defensible merge decision where every approval criterion is backed by evidence you can re-execute.

Authoritative Source Resources & Provenance

linking CC BY-NC-SA 3.0 noncommercial share-alike

Pro Git, Second Edition

Author: Scott Chacon and Ben Straub | Original Source Link ↗

Relevant Section / Range: Chapter 3 (Git Branching), Section 7.7 (Reset Demystified), Sections 5.2/6.2 (Contributing to a Project)

Verified Version & Date: Second edition (Apress, 2014) (Verified: 2026-10-30)

Selection Rationale:

Freely licensed reference for branching, merging, conflict resolution, and safe recovery vocabulary used through the delivery workflow.

Source Prerequisites:

Basic git status/log/diff vocabulary from earlier study days

Safe Fallback (Offline Resilience):

git help <command> and the Git manual pages ship with Git and cover the same commands offline; every required workflow is stated locally in the lesson.

Compared Candidate Sources & Selection Analysis (2)

GitHub Skills interactive courses ↗ compared_alternative [MIT (varies by course)]

Interactive but requires a GitHub account and an external service; disposable local repositories keep the exercise self-contained.

Vendor-owned prose with no permissive license; Pro Git covers the same ground under CC BY-NC-SA.

linking Open Access / CC BY 4.0

SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering

Author: John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, Ofir Press | Original Source Link ↗

Relevant Section / Range: Appendix B.4 Failure Modes

Verified Version & Date: NeurIPS 2024 / arXiv:2405.15793v2 (Verified: 2026-10-30)

Selection Rationale:

Primary literature on agent-computer interfaces showing that bounded tool commands, trajectory inspection, and environment constraints determine whether an agent can carry an engineering task to verifiable completion.

Source Prerequisites:

Agent loops, tool protocols, and basic software engineering workflow vocabulary.

Safe Fallback (Offline Resilience):

The permission/budget control profile, tool contract schema, and trace audit procedure are fully implemented in the local bounded agent fixture (projects/agent_learning).

Compared Candidate Sources & Selection Analysis (2)

OpenHands: An Open Platform for AI Software Developers as Generalist Agents ↗ compared_alternative [Open Access / CC BY 4.0]

Broader generalist agent platform; SWE-agent isolates the agent-computer interface design principles most relevant to bounded task delivery.

AutoCodeRover: Autonomous Program Improvement ↗ compared_alternative [Open Access / CC BY 4.0]

Focuses on autonomous program repair via code search; SWE-agent better matches the curriculum's bounded-tool and human-verification emphasis.