Model Requests and the Action-Observation Control Loop
Agent-Assisted Engineering Check
Prerequisites
Days 70-72 model evaluation concepts and the bounded agent Learning Project (projects/agent_learning) introduced with your Agent Tool Track.
Learning Objective
Explain the five stages of a tool-using agent's control loop - model request, action proposal, tool dispatch, structured observation, and loop-or-terminate - and map each stage onto the deterministic projects.agent_learning harness.
Instructions
Study source: within this activity's existing reading/practice time, use Hugging Face Agents Course: Introduction to Agents, Unit 1: What is an Agent?; What are Tools?; agent workflow; Summary; Quiz, then explain one connection to the local task in your own words. Keep the local exercise and acceptance checks; use its original explanation if the external source is unavailable. 1. The control loop: on every step the runtime sends the task plus conversation history plus available tool schemas to the model; the model returns one proposed action; the runtime validates and dispatches it to a bounded tool; the tool's structured observation is appended to history; the loop repeats until a termination condition fires.
2. ReAct (Yao et al. 2022) interleaves reasoning traces with actions in a real environment. Our Learning Project deliberately keeps only action and observation events - it never records thought or chain_of_thought fields, so the evidence you collect is observable behavior, not hidden reasoning.
3. State diagram for the harness:
IDLE -> MODEL_REQUEST -> ACTION_PROPOSED -> TOOL_DISPATCH -> OBSERVATION -> (back to MODEL_REQUEST) ... -> TERMINATE
Termination reasons in run_task: goal_reached (a write_report observation is ok), tool_failure (failed observations exceed max_retries), step_budget (the step counter reaches max_steps first).
4. In the bounded project: DeterministicModelAdapter.next_action(history) plays the model role; _tool_call(action, workspace) validates and executes; run_task owns the loop and budgets; evaluate() independently re-scores the workspace artifacts afterward.
5. Honest scope: the fixture demonstrates loop shape and error vocabulary. It does not implement JSON-RPC framing, capability negotiation, or streaming - it is a teaching model, not a protocol-compliant client.
Practice & Prediction Log
Before running anything, write your predicted ordered action list for fixture summarize-notes (tool names plus arguments), the predicted final status, steps, retries, and termination reason.
Read DeterministicModelAdapter.next_action in projects/agent_learning/agent.py: the policy first lists notes, then reads alpha.txt and beta.txt, then writes the report once both files were observed.
predicted = [
{"name": "list_notes", "arguments": {}},
{"name": "read_note", "arguments": {"filename": "alpha.txt"}},
{"name": "read_note", "arguments": {"filename": "beta.txt"}},
{"name": "write_report", "arguments": {"content": "Alpha is the first item.\nBeta is the second item.\n"}}
]
# predicted RunResult: status="completed", steps=4, retries=0, termination="goal_reached"
Independently name all five loop stages, all three termination reasons with triggers, and state why action/observation evidence excludes hidden chain-of-thought.
Authoritative Source Resources & Provenance
Hugging Face Agents Course: Introduction to Agents
Relevant Section / Range: Unit 1: What is an Agent?; What are Tools?; agent workflow; Summary; Quiz
Verified Version & Date: Public edition checked 2026-11-01; moving pages may change
A sequenced introduction, examples and quizzes before formal ReAct/tool API references. Adopt the control-loop concepts, not framework-specific execution or private-reasoning logs.
Python, language-model messages and JSON; local agent exercises use only action/observation traces.
Use the original localized explanations, exercises and checks already present in this activity; no external account or purchase is required for that fallback.
Compared Candidate Sources & Selection Analysis (2)
Retained as the method-definition paper; not a self-contained beginner exercise course.
Retained for one protocol example; vendor message formats are not interchangeable.
ReAct: Synergizing Reasoning and Acting in Language Models
Relevant Section / Range: Section 3 & 4 (ReAct Formulation and Synergy)
Verified Version & Date: ICLR 2023 / arXiv:2210.03629v2 (Verified: 2026-10-30)
Foundational academic paper introducing the interleaved Thought-Action-Observation control loop in LLM agents, establishing formal interleaving of task reasoning and external tool execution.
Prompt engineering, token generation basics, and conversational interaction loops.
Complete algorithmic step traces, state loop diagrams, and control loop pseudocode are self-contained in curriculum materials.
Compared Candidate Sources & Selection Analysis (2)
Pure reasoning without environmental actions or external tool grounding; ReAct extends reasoning into an interactive environment action-observation loop.
Early neuro-symbolic modular router concept; ReAct offers simpler, prompting-native action-observation loop dynamics suitable for standard API models.
Toolformer: Language Models Can Teach Themselves to Use Tools
Relevant Section / Range: Section 2 (Approach: Tool Use and Filtering)
Verified Version & Date: NeurIPS 2023 / arXiv:2302.04761v1 (Verified: 2026-10-30)
Original paper demonstrating self-supervised discovery and execution of tool API calls (calculator, QA, search, calendar) via inline tokenized tool invocations.
Transformer autoregressive generation, API calling syntax, and self-supervised filtering.
Inline tool invocation formats and self-contained mock execution traces are provided in curriculum notes.
Compared Candidate Sources & Selection Analysis (2)
Focuses on massive API retrieval and documentation formatting; Toolformer focuses on the fundamental token-level seam between model generation and tool execution.
Iterative search-and-execute augmentation; Toolformer provides clearer self-supervised API call tokenization syntax.