综合复习：证据种类、预注册与基线纪律
Agent 辅助工程检查
前置要求
Day 83 的解读备忘。
学习目标
在翻看笔记之前，凭记忆提取本单元核心区分——确定性夹具证据与受阻在线证据、预注册与事后编故事、结果指标与工具行为指标。
操作指引
1. 检索练习：凭记忆写出四个指标族、六个契约任务 ID，以及 evidence_kind 为 deterministic 与 live 的区别。
2. 对照 live_eval_contract.json 自查回忆结果，把每一处遗漏记入追赶计划。
3. 用一段话概括 blocked-awaiting-authorization 状态：存在什么（确定性运行、契约、校验器），不存在什么（任何在线测量值）。
4. 解释为什么基线纪律——先冻结任务集、基线与假设——是本单元的核心习惯。
自主演练与记录
撰写 recall_log.md：凭记忆列出四个指标族、全部六个任务 ID、确定性对在线的证据规则，然后对照契约自评。
想不起某个任务 ID 或指标族，就说明它该进间隔复习清单。
recall = {'metrics':['outcome_passed','tool_behavior_passed','budget_passed','failure_handling_passed'],'tasks':6,'rule':'deterministic fixture evidence != live model evidence; live_value stays null until authorized'}
凭记忆准确复现单元核心概念，并诚实记录缺口。
权威源资源与权利证明
AgentBench: Evaluating LLMs as Agents
对应章节 / 时间戳: Section 4.1 Evaluation Setup and Section 4.3 Analysis
验证版本与日期: ICLR 2024 / arXiv:2308.03688v2 (Verified: 2026-10-30)
First systematic multi-environment benchmark measuring LLM-as-agent ability across eight interactive settings, establishing fixed task suites, environment-specific scoring, and comparable evaluation conventions.
LLM prompting, tool-use environments, and multi-step task decomposition.
The curriculum fixes its own six-task evaluation suite with deterministic-v1 fixture analogs, so all protocol-design practice runs offline without reproducing AgentBench environments.
对比候选资源与选型分析 (2)
Real computer-control benchmark for open-ended desktop tasks; AgentBench covers a wider environment spectrum for foundational evaluation design.
Generalist web-agent benchmark with 2,350 tasks; AgentBench provides broader multi-domain coverage for studying evaluation protocol design.
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
对应章节 / 时间戳: N/A
验证版本与日期: ICLR 2024 / arXiv:2310.06770v3 (Verified: 2026-10-30)
Defines the standard pattern this block teaches: a bounded issue statement paired with executable fail-to-pass tests as verifiable acceptance criteria for agent-produced changes.
Unit testing, issue descriptions, and repository evaluation vocabulary.
The acceptance-criteria pattern is exercised locally through the Ticket 22 fixture scorer and the Ticket 24 regression exercise without external datasets.
对比候选资源与选型分析 (2)
Function-level synthesis benchmark; SWE-bench models the repository-scoped issue-to-diff workflow that this delivery block rehearses.
Extends issue resolution to multiple programming languages; the original Python-centric SWE-bench provides a cleaner pedagogical anchor.