模型請求與「動作—觀察」控制循環
Agent 輔助工程檢查
前置要求
Day 70-72 的模型評估概念，以及隨 Agent 工具路線導入的有界 Agent Learning Project（projects/agent_learning）。
學習目標
解釋使用工具的 Agent 控制循環的五個階段——模型請求、動作提案、工具分派、結構化觀察、繼續循環或終止——並把每個階段對映到確定性的 projects.agent_learning 實驗裝置上。
操作指引
學習來源：在本活動現有的閱讀與練習時段內，參考 Hugging Face Agents Course: Introduction to Agents 的 Unit 1: What is an Agent?; What are Tools?; agent workflow; Summary; Quiz，用自己的話解釋一處與本地任務的聯繫。保留現有練習與驗收檢查；若外部資料無法存取，使用課程自寫的解釋。1. 控制循環：每一步執行時系統把任務、對話歷史與可用工具 Schema 發給模型；模型回傳一個動作提案；執行時驗證後把它分派到有界工具；工具的結構化觀察被附加到歷史中；循環重複直到某個終止條件觸發。
2. ReAct（Yao 等，2022）在真實環境中把推理軌跡與動作交錯。本 Learning Project 刻意只保留 action 與 observation 事件——從不記錄 thought 或 chain_of_thought 欄位，因此你收集的證據是可觀察行為，而不是隱藏推理。
3. 裝置的狀態圖：
IDLE -> MODEL_REQUEST -> ACTION_PROPOSED -> TOOL_DISPATCH -> OBSERVATION ->（回到 MODEL_REQUEST）... -> TERMINATE
run_task 的終止原因：goal_reached（write_report 觀察為 ok）、tool_failure（失敗觀察數超過 max_retries）、step_budget（步數先達到 max_steps）。
4. 在有界專案中：DeterministicModelAdapter.next_action(history) 扮演模型角色；_tool_call(action, workspace) 負責驗證與執行；run_task 掌管循環與預算；evaluate() 事後獨立地重新評分工作區產物。
5. 誠實邊界：此裝置示範循環結構與錯誤詞彙，並未實作 JSON-RPC 訊框、能力協商或串流——它是教學模型，不是協議合規客戶端。
自主實作與記錄
在執行任何東西之前，寫出你對 fixture summarize-notes 預測的有序動作列表（工具名加參數）、預測的最終 status、steps、retries 與終止原因。
閱讀 projects/agent_learning/agent.py 中 DeterministicModelAdapter.next_action：策略先列出筆記，再讀取 alpha.txt 與 beta.txt，兩者都被觀察到後才寫報告。
# 預測軌跡 / predicted trace
predicted = [
{"name": "list_notes", "arguments": {}},
{"name": "read_note", "arguments": {"filename": "alpha.txt"}},
{"name": "read_note", "arguments": {"filename": "beta.txt"}},
{"name": "write_report", "arguments": {"content": "Alpha is the first item.\nBeta is the second item.\n"}}
]
# 預測 RunResult: status="completed", steps=4, retries=0, termination="goal_reached"
能獨立說出全部五個循環階段、三個終止原因及觸發條件，並說明為什麼動作/觀察證據不包含隱藏的思維鏈。
權威來源資源與權利證明
Hugging Face Agents Course: Introduction to Agents
對應章節 / 時間戳: Unit 1: What is an Agent?; What are Tools?; agent workflow; Summary; Quiz
驗證版本與日期: Public edition checked 2026-11-01; moving pages may change
A sequenced introduction, examples and quizzes before formal ReAct/tool API references. Adopt the control-loop concepts, not framework-specific execution or private-reasoning logs.
Python, language-model messages and JSON; local agent exercises use only action/observation traces.
Use the original localized explanations, exercises and checks already present in this activity; no external account or purchase is required for that fallback.
對比候選資源與選型分析 (2)
Retained as the method-definition paper; not a self-contained beginner exercise course.
Retained for one protocol example; vendor message formats are not interchangeable.
ReAct: Synergizing Reasoning and Acting in Language Models
對應章節 / 時間戳: Section 3 & 4 (ReAct Formulation and Synergy)
驗證版本與日期: ICLR 2023 / arXiv:2210.03629v2 (Verified: 2026-10-30)
Foundational academic paper introducing the interleaved Thought-Action-Observation control loop in LLM agents, establishing formal interleaving of task reasoning and external tool execution.
Prompt engineering, token generation basics, and conversational interaction loops.
Complete algorithmic step traces, state loop diagrams, and control loop pseudocode are self-contained in curriculum materials.
對比候選資源與選型分析 (2)
Pure reasoning without environmental actions or external tool grounding; ReAct extends reasoning into an interactive environment action-observation loop.
Early neuro-symbolic modular router concept; ReAct offers simpler, prompting-native action-observation loop dynamics suitable for standard API models.
Toolformer: Language Models Can Teach Themselves to Use Tools
對應章節 / 時間戳: Section 2 (Approach: Tool Use and Filtering)
驗證版本與日期: NeurIPS 2023 / arXiv:2302.04761v1 (Verified: 2026-10-30)
Original paper demonstrating self-supervised discovery and execution of tool API calls (calculator, QA, search, calendar) via inline tokenized tool invocations.
Transformer autoregressive generation, API calling syntax, and self-supervised filtering.
Inline tool invocation formats and self-contained mock execution traces are provided in curriculum notes.
對比候選資源與選型分析 (2)
Focuses on massive API retrieval and documentation formatting; Toolformer focuses on the fundamental token-level seam between model generation and tool execution.
Iterative search-and-execute augmentation; Toolformer provides clearer self-supervised API call tokenization syntax.