模型请求与“动作—观察”控制循环
Agent 辅助工程检查
前置要求
Day 70-72 的模型评估概念，以及随 Agent 工具路线引入的有界 Agent Learning Project（projects/agent_learning）。
学习目标
解释使用工具的 Agent 控制循环的五个阶段——模型请求、动作提案、工具分发、结构化观察、继续循环或终止——并把每个阶段映射到确定性的 projects.agent_learning 实验装置上。
操作指引
学习来源：在本活动现有的阅读与练习时段内，参考 Hugging Face Agents Course: Introduction to Agents 的 Unit 1: What is an Agent?; What are Tools?; agent workflow; Summary; Quiz，用自己的话解释一处与本地任务的联系。保留现有练习与验收检查；若外部资料无法访问，使用课程自写的解释。1. 控制循环：每一步运行时把任务、对话历史与可用工具 Schema 发给模型；模型返回一个动作提案；运行时验证后把它分发到有界工具；工具的结构化观察被追加到历史中；循环重复直至某个终止条件触发。
2. ReAct（Yao 等，2022）在真实环境中把推理轨迹与动作交错。本 Learning Project 刻意只保留 action 与 observation 事件——从不记录 thought 或 chain_of_thought 字段，因此你收集的证据是可观察行为，而不是隐藏推理。
3. 装置的状态图：
IDLE -> MODEL_REQUEST -> ACTION_PROPOSED -> TOOL_DISPATCH -> OBSERVATION ->（回到 MODEL_REQUEST）... -> TERMINATE
run_task 的终止原因：goal_reached（write_report 观察为 ok）、tool_failure（失败观察数超过 max_retries）、step_budget（步数先达到 max_steps）。
4. 在有界项目中：DeterministicModelAdapter.next_action(history) 扮演模型角色；_tool_call(action, workspace) 负责验证与执行；run_task 掌管循环与预算；evaluate() 事后独立地重新评分工作区产物。
5. 诚实边界：该装置演示循环结构与错误词汇，并未实现 JSON-RPC 帧、能力协商或流式传输——它是教学模型，不是协议合规客户端。
自主演练与记录
在运行任何东西之前，写出你对 fixture summarize-notes 预测的有序动作列表（工具名加参数）、预测的最终 status、steps、retries 与终止原因。
阅读 projects/agent_learning/agent.py 中 DeterministicModelAdapter.next_action：策略先列出笔记，再读取 alpha.txt 与 beta.txt，两者都被观察到后才写报告。
# 预测轨迹 / predicted trace
predicted = [
{"name": "list_notes", "arguments": {}},
{"name": "read_note", "arguments": {"filename": "alpha.txt"}},
{"name": "read_note", "arguments": {"filename": "beta.txt"}},
{"name": "write_report", "arguments": {"content": "Alpha is the first item.\nBeta is the second item.\n"}}
]
# 预测 RunResult: status="completed", steps=4, retries=0, termination="goal_reached"
能独立说出全部五个循环阶段、三个终止原因及触发条件，并说明为什么动作/观察证据不包含隐藏的思维链。
权威源资源与权利证明
Hugging Face Agents Course: Introduction to Agents
对应章节 / 时间戳: Unit 1: What is an Agent?; What are Tools?; agent workflow; Summary; Quiz
验证版本与日期: Public edition checked 2026-11-01; moving pages may change
A sequenced introduction, examples and quizzes before formal ReAct/tool API references. Adopt the control-loop concepts, not framework-specific execution or private-reasoning logs.
Python, language-model messages and JSON; local agent exercises use only action/observation traces.
Use the original localized explanations, exercises and checks already present in this activity; no external account or purchase is required for that fallback.
对比候选资源与选型分析 (2)
Retained as the method-definition paper; not a self-contained beginner exercise course.
Retained for one protocol example; vendor message formats are not interchangeable.
ReAct: Synergizing Reasoning and Acting in Language Models
对应章节 / 时间戳: Section 3 & 4 (ReAct Formulation and Synergy)
验证版本与日期: ICLR 2023 / arXiv:2210.03629v2 (Verified: 2026-10-30)
Foundational academic paper introducing the interleaved Thought-Action-Observation control loop in LLM agents, establishing formal interleaving of task reasoning and external tool execution.
Prompt engineering, token generation basics, and conversational interaction loops.
Complete algorithmic step traces, state loop diagrams, and control loop pseudocode are self-contained in curriculum materials.
对比候选资源与选型分析 (2)
Pure reasoning without environmental actions or external tool grounding; ReAct extends reasoning into an interactive environment action-observation loop.
Early neuro-symbolic modular router concept; ReAct offers simpler, prompting-native action-observation loop dynamics suitable for standard API models.
Toolformer: Language Models Can Teach Themselves to Use Tools
对应章节 / 时间戳: Section 2 (Approach: Tool Use and Filtering)
验证版本与日期: NeurIPS 2023 / arXiv:2302.04761v1 (Verified: 2026-10-30)
Original paper demonstrating self-supervised discovery and execution of tool API calls (calculator, QA, search, calendar) via inline tokenized tool invocations.
Transformer autoregressive generation, API calling syntax, and self-supervised filtering.
Inline tool invocation formats and self-contained mock execution traces are provided in curriculum notes.
对比候选资源与选型分析 (2)
Focuses on massive API retrieval and documentation formatting; Toolformer focuses on the fundamental token-level seam between model generation and tool execution.
Iterative search-and-execute augmentation; Toolformer provides clearer self-supervised API call tokenization syntax.