綜合複習：證據種類、預註冊與基準紀律
Agent 輔助工程檢查
前置要求
Day 83 的解讀備忘。
學習目標
在翻看筆記之前，憑記憶提取本單元核心區分——確定性治具證據與受阻上線證據、預註冊與事後編故事、結果指標與工具行為指標。
操作指引
1. 檢索練習：憑記憶寫出四個指標族、六個契約任務 ID，以及 evidence_kind 為 deterministic 與 live 的區別。
2. 對照 live_eval_contract.json 自查回憶結果，把每一處遺漏記入追趕計畫。
3. 用一段話概括 blocked-awaiting-authorization 狀態：存在什麼（確定性執行、契約、校驗器），不存在什麼（任何上線測量值）。
4. 解釋為什麼基準紀律——先凍結任務集、基準與假設——是本單元的核心習慣。
自主實作與記錄
撰寫 recall_log.md：憑記憶列出四個指標族、全部六個任務 ID、確定性對上線的證據規則，然後對照契約自評。
想不起某個任務 ID 或指標族，就說明它該進間隔複習清單。
recall = {'metrics':['outcome_passed','tool_behavior_passed','budget_passed','failure_handling_passed'],'tasks':6,'rule':'deterministic fixture evidence != live model evidence; live_value stays null until authorized'}
憑記憶準確重現單元核心概念，並誠實記錄缺口。
權威來源資源與權利證明
AgentBench: Evaluating LLMs as Agents
對應章節 / 時間戳: Section 4.1 Evaluation Setup and Section 4.3 Analysis
驗證版本與日期: ICLR 2024 / arXiv:2308.03688v2 (Verified: 2026-10-30)
First systematic multi-environment benchmark measuring LLM-as-agent ability across eight interactive settings, establishing fixed task suites, environment-specific scoring, and comparable evaluation conventions.
LLM prompting, tool-use environments, and multi-step task decomposition.
The curriculum fixes its own six-task evaluation suite with deterministic-v1 fixture analogs, so all protocol-design practice runs offline without reproducing AgentBench environments.
對比候選資源與選型分析 (2)
Real computer-control benchmark for open-ended desktop tasks; AgentBench covers a wider environment spectrum for foundational evaluation design.
Generalist web-agent benchmark with 2,350 tasks; AgentBench provides broader multi-domain coverage for studying evaluation protocol design.
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
對應章節 / 時間戳: N/A
驗證版本與日期: ICLR 2024 / arXiv:2310.06770v3 (Verified: 2026-10-30)
Defines the standard pattern this block teaches: a bounded issue statement paired with executable fail-to-pass tests as verifiable acceptance criteria for agent-produced changes.
Unit testing, issue descriptions, and repository evaluation vocabulary.
The acceptance-criteria pattern is exercised locally through the Ticket 22 fixture scorer and the Ticket 24 regression exercise without external datasets.
對比候選資源與選型分析 (2)
Function-level synthesis benchmark; SWE-bench models the repository-scoped issue-to-diff workflow that this delivery block rehearses.
Extends issue resolution to multiple programming languages; the original Python-centric SWE-bench provides a cleaner pedagogical anchor.