85 / 90

第 85 天:综合项目章程 —— 问题、范围与模型选型

将先前已验证的项目整合为一个有边界的综合项目:定义问题与用户,冻结验收契约,并基于真实证据论证模型选择。

AI Agent 与工程实践 480 分钟有效学习时间(8 小时)
01 120 分钟

综合项目章程:问题、用户与有界范围

Agent 辅助工程检查

前置要求

第 82-84 天的科学化 Agent 评测;projects/agent_learning 中的有界 Agent 学习项目及其四语言指南。

学习目标

撰写一页综合项目章程,把具体问题、具名用户、明确的范围边界和非目标绑定在一起,复用此前已验证的有界 Agent,而不是在综合项目内部偷藏一个新的无界系统。

操作指引

1. 重读 projects/agent_learning/README.md 与 agent.py,列出已注册的有界工具(list_notes、read_note、write_report)和夹具任务 summarize-notes。
2. 撰写问题陈述:综合项目在已验证的“笔记到报告”Agent 基础上,恰好新增一个有界工具(例如 count_words 或 list_deadlines)和一个额外的固定任务,仍由现有的独立评分器评分。
3. 具名用户(把学习笔记整理成报告的学习者)与使用场景(本地、离线优先的一次性夹具工作区)。
4. 划出范围界线。范围内:确定性适配器、已注册的有界工具、固定夹具、步数/重试预算。范围外:开放网页浏览、任意 shell 执行、无界重试、生产环境自主运行。
5. 声明非目标:在工单 23 授权门获批之前不作任何真机模型声明;不新建 Agent 框架;不做隐藏的网络调用。
6. 将草案保存为 capstone_charter.md。

自主演练与记录

起草包含 Problem、User、In-Scope、Out-of-Scope、Non-Goals 各节的 capstone_charter.md,然后把三个功能请求分类为范围内或范围外:(a)“读取夹具中的第三个笔记”;(b)“允许 Agent 随意 pip-install 任何包”;(c)“新增一个有界的字数统计工具”。

独立掌握标准

产出的章程中每个范围内事项都能映射到已声明的约束,且非目标明确排除至少三项能力(如网页浏览、任意代码执行、无界重试)。

权威源资源与权利证明

linking Open Access / CC BY 4.0

ReAct: Synergizing Reasoning and Acting in Language Models

作者 / 机构: Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, Yuan Cao | 原始出处链接 ↗

对应章节 / 时间戳: Sections 2-4: interleaved reasoning/action protocol; Figure 1 trace format

验证版本与日期: ICLR 2023 / arXiv:2210.03629v3 (Verified: 2026-10-30)

入选理由:

Primary literature for interleaving explicit reasoning steps with bounded tool actions; anchors the capstone's requirement that every step produces an inspectable observation inside a budgeted loop.

来源前置条件:

Language-model prompting, tool-call loops, and basic action/observation traces.

安全回退方案(离线弹性):

The curriculum restates the action/observation loop and reimplements a deterministic three-tool agent locally; no paper text is imported.

对比候选资源与选型分析 (2)

Pure reasoning traces without tool actions; ReAct adds the observable action/observation protocol that the capstone harness enforces.

Embodied closed-loop planning for robotics; ReAct's text-domain tool protocol maps directly onto the capstone's bounded tools.

linking arXiv non-exclusive license / Open Access

Model Cards for Model Reporting

作者 / 机构: Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru | 原始出处链接 ↗

对应章节 / 时间戳: Section 4 Model Card Sections (4.1-4.9)

验证版本与日期: FAT* 2019 / arXiv:1810.03993v3 (Verified: 2026-10-30)

入选理由:

Standard framework for disclosing a model's intended use, evaluation conditions, and limitations; supplies the capstone's model-choice and provenance documentation pattern.

来源前置条件:

Basic model evaluation vocabulary: dataset, metric, intended use.

安全回退方案(离线弹性):

The curriculum provides a local model-decision table template covering intended use, evaluation data, and caveats.

对比候选资源与选型分析 (2)
Gebru et al.: Datasheets for Datasets ↗ compared_alternative [Open Access]

Dataset-side documentation rather than model-side; used alongside Model Cards in the provenance table rather than as a replacement.

OpenAI GPT-4 System Card ↗ compared_alternative [All Rights Reserved (web publication)]

Vendor-specific safety disclosure; Model Cards provide the general reusable schema.

02 120 分钟

验收契约:固定任务、基线与上限

独立掌握检查(无 AI 辅助)

前置要求

第 85 天活动 1 的章程草案;projects/agent_learning/evidence/live_eval_contract.json 中的工单 23 契约模式。

学习目标

把章程转化为可度量的验收契约:固定任务集、确定性基线及其报告哈希、指标定义、资源上限、失败用例,以及评审者将检查的确切提交证据。

操作指引

1. 仿照 projects/agent_learning/evidence/live_eval_contract.json 的模式创建 capstone_acceptance.json。
2. 绑定任务集:保留六个 live-eval 任务类别(成功、两个畸形工具请求、输入缺失、循环、预算耗尽),并加入章程中的一个新固定任务;在任何调优之前冻结。
3. 记录基线:适配器 deterministic-v1,夹具 summarize-notes,expected_report_sha256 = 9e8315034af750009a4b2cfb261a077428afe441bc45c602cb7ead32ac0fd787,以及基线命令 python -m projects.agent_learning.cli evaluate --fixture summarize-notes --output 。
4. 定义每个任务的指标:status、steps、retries、终止原因、报告哈希一致性,对齐契约评分器的四个维度(结果、工具行为、预算、失败处理)。
5. 设定资源上限:每任务最多 6 步、每故障最多 2 次重试、30 分钟运行时上限、任何已授权真机运行 5.00 美元费用上限,以及从契约复制的停止条件。
6. 列出提交证据:章程、本契约、基线台账、逐任务结果、失败日志、出处表。

自主演练与记录

编写包含全部九个必需元素(problem、user、scope、model_choice、baseline、evaluation_set、failure_cases、resource_limits、submission_evidence)的 capstone_acceptance.json,真机运行字段显式置为 null 并标注 not-run-awaiting-authorization。

独立掌握标准

产出一份契约,使评审者在任何运行之前即可预判每个任务的通过/失败;真机运行字段保持 null 并带 not-run-awaiting-authorization 标注。

权威源资源与权利证明

linking Open Access / CC BY 4.0

AgentBench: Evaluating LLMs as Agents

作者 / 机构: Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, et al. | 原始出处链接 ↗

对应章节 / 时间戳: Section 4.1 Evaluation Setup and Section 4.3 Analysis

验证版本与日期: ICLR 2024 / arXiv:2308.03688v4 (Verified: 2026-10-30)

入选理由:

Systematic multi-environment agent benchmark; supplies the capstone's fixed-task-set discipline and per-task scoring decomposition used by the acceptance contract.

来源前置条件:

Agent loops, tool use, and benchmark design fundamentals.

安全回退方案(离线弹性):

The capstone task set is fixed locally in the acceptance contract and does not require downloading the benchmark.

对比候选资源与选型分析 (2)

Deep single-domain web tasks with hosted infrastructure; AgentBench's multi-environment view better fits a general agent evaluation.

Assistant-level real-world questions; also cited by the capstone for outcome-level task design.

linking arXiv non-exclusive license / Open Access

GAIA: a benchmark for General AI Assistants

作者 / 机构: Gregoire Mialon, Clementine Fourrier, Craig Swift, Thomas Wolf, Yann LeCun, Thomas Scialom | 原始出处链接 ↗

对应章节 / 时间戳: Sections 3-4: level design, tooling requirements, human vs agent gap

验证版本与日期: arXiv:2311.12983v3 (Verified: 2026-10-30)

入选理由:

Defines graded assistant tasks with unambiguous factual scoring; motivates the capstone's distinction between success tasks, malformed-request handling, and failure cases.

来源前置条件:

Agent capabilities, tool use, and evaluation metrics.

安全回退方案(离线弹性):

The capstone evaluation tasks are defined in the local acceptance contract and need no external download.

对比候选资源与选型分析 (2)
Liu et al.: AgentBench: Evaluating LLMs as Agents ↗ compared_alternative [Open Access / CC BY 4.0]

Multi-environment execution benchmark; GAIA adds graded real-world assistance tasks, and both inform the fixed task taxonomy.

Zhou et al.: WebArena ↗ compared_alternative [Open Access]

Web-only task environment; GAIA covers general assistant capability beyond browsing.

03 120 分钟

模型选型:自训适配器 vs 现有模型

Agent 辅助工程检查

前置要求

第 85 天活动 1-2;工单 19 的证据包与工单 21 的 LoRA 对比契约。

学习目标

通过检视云端预训练证据包与 LoRA 对比契约,决定驱动综合项目 Agent 的模型;当合适的现有模型对最终 Agent 表现更好时,允许选用现有模型。

操作指引

1. 打开 curriculum/projects/docs/pretrain-cloud-evidence/evidence-package.json:注意 4 层/4 头/64 维嵌入的 GPT 配方、tinyshakespeare 数据出处与 SHA-256,以及 run_status=blocked——预训练工作教会的是数据到检查点的完整回路,并不产出可用模型。
2. 打开 projects/lora/evidence/comparison_contract.json:注意 base_model openai-community/gpt2,LoRA r=8 alpha=16 作用于 q_proj/v_proj,seed 42,划分 0.8/0.1/0.1,status=blocked-awaiting-authorization,以及只预先声明方向的成功标准。
3. 阅读 projects/agent_learning/evidence/model_selection_plan.md:一个推荐的本地开放权重候选,加三个提供方 API 备选,各自记录版本、接入、数据用途与费用条件。
4. 构建决策表。候选:(a) 预训练配方自训 GPT;(b) LoRA 适配的 gpt2;(c) 推荐的现有开放模型。按已观测证据(而非愿望)给每个候选评分:确定性基线一致性、许可证/再分发权、固定修订、费用上限。
5. 如实记录决策:deterministic-v1 适配器仍是离线评测基线;对于任何已授权的真机运行,推荐的开放权重模型是可接受的,因为自训产物尚未被证明——预训练与 LoRA 的结果指导设计,但不强制采用。
6. 将分析保存为 model_decision.md,每个候选一行许可证记录。

自主演练与记录

产出 model_decision.md:一张三行候选表(证据文件、许可证、结论),加一段说明,指名所选的真机运行候选,并解释为何自训方案指导了设计却不驱动最终 Agent。

独立掌握标准

以对全部三个证据产物的引用、每个候选的许可证核查、以及零虚构测量,论证模型选择。

权威源资源与权利证明

linking arXiv non-exclusive license / Open Access

Datasheets for Datasets

作者 / 机构: Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna Wallach, Hal Daumé III, Kate Crawford | 原始出处链接 ↗

对应章节 / 时间戳: Section 3 Questions and Workflow (3.1 Motivation through 3.7 Maintenance)

验证版本与日期: Communications of the ACM 64(12) / arXiv:1803.09010v8 (Verified: 2026-10-30)

入选理由:

Standard questionnaire for documenting dataset origin, composition, rights, and maintenance; structures the capstone evidence package's data-provenance table.

来源前置条件:

Dataset provenance, splits, and licensing vocabulary.

安全回退方案(离线弹性):

The provenance table template in the capstone instructions restates the required disclosure fields locally.

对比候选资源与选型分析 (2)
Mitchell et al.: Model Cards for Model Reporting ↗ compared_alternative [Open Access]

Model-side reporting rather than dataset-side; the capstone provenance table cites both.

Bender & Friedman: Data Statements for Natural Language Processing ↗ compared_alternative [CC BY 4.0 (ACL Anthology, 2016 or later)]

NLP-specific demographic disclosure; Datasheets cover the general lifecycle needed for arbitrary capstone data.

linking arXiv non-exclusive license / Open Access

Model Cards for Model Reporting

作者 / 机构: Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru | 原始出处链接 ↗

对应章节 / 时间戳: Section 4 Model Card Sections (4.1-4.9)

验证版本与日期: FAT* 2019 / arXiv:1810.03993v3 (Verified: 2026-10-30)

入选理由:

Standard framework for disclosing a model's intended use, evaluation conditions, and limitations; supplies the capstone's model-choice and provenance documentation pattern.

来源前置条件:

Basic model evaluation vocabulary: dataset, metric, intended use.

安全回退方案(离线弹性):

The curriculum provides a local model-decision table template covering intended use, evaluation data, and caveats.

对比候选资源与选型分析 (2)
Gebru et al.: Datasheets for Datasets ↗ compared_alternative [Open Access]

Dataset-side documentation rather than model-side; used alongside Model Cards in the provenance table rather than as a replacement.

OpenAI GPT-4 System Card ↗ compared_alternative [All Rights Reserved (web publication)]

Vendor-specific safety disclosure; Model Cards provide the general reusable schema.

04 120 分钟

考核:章程评审门与范围冻结

独立掌握检查(无 AI 辅助)

前置要求

第 85 天活动 1-3。

学习目标

对照固定的九元素清单,对章程与验收契约执行结构化 go/no-go 评审,冻结范围,并为任何未达标元素撰写恢复计划。

操作指引

1. 准备九元素清单:具体问题、具名用户、范围边界、模型选择、基线、评测集、失败用例、资源上限、提交证据。
2. 逐元素给章程与契约评分;每个元素标为 PRESENT 或 MISSING,并附一行指向其定义位置的指针。
3. 若全部元素齐备,冻结范围:记录 scope_frozen=true 与冻结日期;此后的功能请求必须走复审。
4. 若有元素缺失,写出 NO-GO 判定,指明缺失元素及修复它的具体活动(act-01、act-02 或 act-03)。
5. 对一个范围蔓延请求(“加聊天界面”)做干跑,展示它无法通过范围界线。
6. 将签署的评审保存为 charter_review.json。

自主演练与记录

产出 charter_review.json:九条带证据指针的清单判定、最终 GO 或 NO-GO,以及——无论判定如何——对“加聊天界面”的范围蔓延裁定。

独立掌握标准

完成一次评审门:每条判定都引用已存证据,范围冻结或 NO-GO 决定明确且可复现。

权威源资源与权利证明

linking Open Access / CC BY 4.0

ReAct: Synergizing Reasoning and Acting in Language Models

作者 / 机构: Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, Yuan Cao | 原始出处链接 ↗

对应章节 / 时间戳: Sections 2-4: interleaved reasoning/action protocol; Figure 1 trace format

验证版本与日期: ICLR 2023 / arXiv:2210.03629v3 (Verified: 2026-10-30)

入选理由:

Primary literature for interleaving explicit reasoning steps with bounded tool actions; anchors the capstone's requirement that every step produces an inspectable observation inside a budgeted loop.

来源前置条件:

Language-model prompting, tool-call loops, and basic action/observation traces.

安全回退方案(离线弹性):

The curriculum restates the action/observation loop and reimplements a deterministic three-tool agent locally; no paper text is imported.

对比候选资源与选型分析 (2)

Pure reasoning traces without tool actions; ReAct adds the observable action/observation protocol that the capstone harness enforces.

Embodied closed-loop planning for robotics; ReAct's text-domain tool protocol maps directly onto the capstone's bounded tools.

linking Open Access / CC BY 4.0

AgentBench: Evaluating LLMs as Agents

作者 / 机构: Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, et al. | 原始出处链接 ↗

对应章节 / 时间戳: Section 4.1 Evaluation Setup and Section 4.3 Analysis

验证版本与日期: ICLR 2024 / arXiv:2308.03688v4 (Verified: 2026-10-30)

入选理由:

Systematic multi-environment agent benchmark; supplies the capstone's fixed-task-set discipline and per-task scoring decomposition used by the acceptance contract.

来源前置条件:

Agent loops, tool use, and benchmark design fundamentals.

安全回退方案(离线弹性):

The capstone task set is fixed locally in the acceptance contract and does not require downloading the benchmark.

对比候选资源与选型分析 (2)

Deep single-domain web tasks with hosted infrastructure; AgentBench's multi-environment view better fits a general agent evaluation.

Assistant-level real-world questions; also cited by the capstone for outcome-level task design.