org.llm4s.agent.graph.toolloop.ToolLoop
See theToolLoop companion object
The model/tool loop of an agent family as one graph - the Stage 0 proof of runtime-owned tool results and resumable approval ((#1269)), running org.llm4s.agent.graph.tool.AgentTools ((#1278)), for every LoopAgent of a family ((#1328)). Each agent's nodes are prefixed with its id; input is the one shared entry:
input -> <id>/model --fan-out, one task per call--> <id>/call-tool --(join "<id>/tool-batch")--> <id>/collect -> <id>/model
\ \--suspend--> <id>/approval --/
\--final answer--> <id>/finish \--suspend--> <id>/ask/<tool> --/
- A turn takes an AgentInput. On a new thread
inputimports itshistory- no system message, and a valid conversation - and makes the root the active agent; on an existing thread ahistoryis refused. It then runs the root'sbeforeAgentstack on the query and, when another agent is active, that agent's stack on the result, appends the user message, resets LoopKeys.turn and routes to the active agent's model. ALeftfrom either stack, or a blank query, blocks the run without the query or a model call; a new thread still keeps its imported history and the root as its active agent. The turn's TurnOutput is its outcome and the active agent. <id>/modelcounts the turn's steps: at the agent'smaxStepsit ends the turn with TurnOutcome.StepLimitReached without calling the model. It counts the run's tokens too - the thread's usage since the run was admitted, plus the task's own - and at the run'sRunBudgets.maxTokensends the turn with TurnOutcome.TokenLimitReached, also without calling the model; the call in flight when the budget is crossed is never interrupted, so a run can overshoot it by one call. Otherwise it sends the agent's system prompt, then the history; the prompt is never stored.- The middleware stack (#1279) runs at the run's boundaries and around each call:
inputruns everybeforeAgenton the user's text,modelrunsModelStep.nextinside everywrapModelCall, andfinishruns everyafterAgent(in reverse) on the final answer, replacing the stored answer's content when it changes. The root's boundary hooks guard the whole family:inputruns the root'sbeforeAgentstack, then the active agent's, and<id>/finishof a handoff target runs its ownafterAgentstack, then the root's. ALeftfrom either hook, a blank query frombeforeAgentand a blank answer fromafterAgentblock the run (design 4.13): it ends as a finished failure,RunResult.Failed, carrying that error, and the thread stays usable; a cancellation is not a Block. An input Block commits only a new thread's seed - its imported history and the root as active agent - and an output Block retracts the blocked turn: every key the turn wrote, a middleware's or a tool's, returns to its value before the turn (recorded in TurnState.priorValues), the turn is removed from the history, and the agent and transfer the turn started with are restored; only usage is kept (design ยง4.22).wrapModelCallandwrapToolCallstay per agent. A middleware'stoolsjoin the loop's org.llm4s.agent.graph.tool.ToolSet, and itswritesare the keys every one of its hooks may update: the boundary hooks' updates commit with the input and the finish, a model wrapper's with the model step, a tool wrapper's with aSuccess. Before each model step every middleware'spromptSectionsjoin the agent's system prompt, never stored. Every provider call is counted in the thread's usage, whichever response the wrappers keep. - Each call is its own task. Its tool is looked up, its raw arguments validated against the tool's
argumentSchema, decoded and checked by the tool'svalidateDecoded- any failure is an error result the model sees, before any middleware or the tool runs. Then the middleware stack'swrapToolCallchain, the first middleware outermost, with the tool at its centre. - A
NeedsApproval, from a wrapper or the tool, suspends the call with an ApprovalRequest naming its ApprovalSource, and resumes at the agent'sapprovalnode.ApproveandEditcheck the arguments again (an edit's are new) and run the whole chain again withToolContext.approved, so a deny rule still refuses an edit; a secondNeedsApprovalfrom an approved call is an error result. A tool'sAsksuspends with a ToolQuestionRequest at that tool'sask/<name>node, which runs the tool'sresumewith the decoded answer inside the same chain, with the approval the call asked with. - A middleware extending
AgentMiddleware.Askingasks a typed question (#1704): frombeforeAgent(theinputnode),wrapModelCall(<id>/model),afterAgent(<id>/finish) orwrapToolCall(each call). The asking task suspends, storing nothing but the usage of model calls a wrapper made before asking, at<id>/asked/<middleware>/<hook>with a MiddlewareQuestionRequest; once answered, it runs again with every answer given so far, so the hook's stack runs again from the outermost middleware and the asking one continues (GivenAnswer). - The loop, not the tool, records exactly one ToolResult per call - for success, failure, denial, rejection and unknown tools alike, however often a wrapper ran the tool; the results key refuses a second one. A tool's thrown exception is that call's error result.
Fatal, an update to a key neither the tool nor any middleware declares, and a question it does not declare fail the run withGraphError.ToolFailed; a throwing wrapper fails it withGraphError.MiddlewareFailed. Either leaves the checkpointRunning, sorecoverre-runs only that call. - A call whose locks (
AgentToolSpec.withLocks) conflict with an earlier call's waits in ToolLoop.queue: the model step fans out only the calls no earlier call conflicts with, andcollect, when the barrier releases, fans out the next round from the queue, so conflicting calls run in successive supersteps in call order, each seeing what the earlier committed (two supersteps per further round). collectruns only when the barrier releases; once the queue is empty, it checks every call of the batch has exactly one result, and appends theToolMessages in call order. The model never sees a partial batch. A call cut short by cancellation, aFatalor a crash records no result, so the batch stays open and the thread waits forrecover, which re-runs only that call (design 4.4, 5.3).- Every call carries an org.llm4s.agent.graph.IdempotencyKey, derived by the model step from where it ran on the thread (checkpoint, superstep, task) and the call's id, and recorded with the call: each run of the call - a retrying wrapper, approval, an answered question,
recover- sees the same key inToolContext.idempotencyKey, and a call of another model request a new one, even when the provider reuses its id and the run itsRunId. - The model node re-checks the history with
Message.validateConversationbefore every call, and refuses to send a request - as everywrapModelCallleft it - in which a tool call lacks exactly one result straight after its message (ToolResultRule); a model message whose call ids repeat or are blank is refused before it is stored, as a blank answer is, and importedhistorymust keep the same rule. - Each LoopHandoff is offered to the agent's model as a stand-in tool
handoff_to_<target>(HandoffTools), which call-tool never runs. When a handoff is an assistant message's only call,<id>/modelstores the message andToolMessage("Transferred to <target>"), makes the target the active agent and routes to<target>/model; the next turn starts there too. A handoff with other calls, or two handoffs (to one target or to several), is refused: every call gets an error result and the same agent is asked again. Both cost the step the model call took. The turn's steps are shared by every agent: each model node checks its own agent'smaxStepsagainst the turn's count so far, so a target with a lower limit than its source can end the turn with TurnOutcome.StepLimitReached before its first call. The handoff call'sreasonargument is offered to the model but neither validated nor used for routing; the tool name alone picks the target. Each transfer is recorded in LoopKeys.transfer. WithpreserveContext = falsethe target is sent the last user message before the transfer, then the transfer's assistant message onwards; the history itself is untouched. A non-root agent with no recorded transfer to it fails the call rather than being sent the full history.
Attributes
- Companion
- object
- Source
- ToolLoop.scala
- Graph
-
- Supertypes
Members list
In this article
