ToolLoop

org.llm4s.agent.graph.toolloop.ToolLoop
See theToolLoop companion object
final class ToolLoop

The model/tool loop of an agent family as one graph - the Stage 0 proof of runtime-owned tool results and resumable approval ((#1269)), running org.llm4s.agent.graph.tool.AgentTools ((#1278)), for every LoopAgent of a family ((#1328)). Each agent's nodes are prefixed with its id; input is the one shared entry:

input -> <id>/model --fan-out, one task per call--> <id>/call-tool --(join "<id>/tool-batch")--> <id>/collect -> <id>/model
            \                                          \--suspend--> <id>/approval  --/
             \--final answer--> <id>/finish             \--suspend--> <id>/ask/<tool> --/
  • A turn takes an AgentInput. On a new thread input imports its history - no system message, and a valid conversation - and makes the root the active agent; on an existing thread a history is refused. It then runs the root's beforeAgent stack on the query and, when another agent is active, that agent's stack on the result, appends the user message, resets LoopKeys.turn and routes to the active agent's model. A Left from either stack, or a blank query, blocks the run without the query or a model call; a new thread still keeps its imported history and the root as its active agent. The turn's TurnOutput is its outcome and the active agent.
  • <id>/model counts the turn's steps: at the agent's maxSteps it ends the turn with TurnOutcome.StepLimitReached without calling the model. It counts the run's tokens too - the thread's usage since the run was admitted, plus the task's own - and at the run's RunBudgets.maxTokens ends the turn with TurnOutcome.TokenLimitReached, also without calling the model; the call in flight when the budget is crossed is never interrupted, so a run can overshoot it by one call. Otherwise it sends the agent's system prompt, then the history; the prompt is never stored.
  • The middleware stack (#1279) runs at the run's boundaries and around each call: input runs every beforeAgent on the user's text, model runs ModelStep.next inside every wrapModelCall, and finish runs every afterAgent (in reverse) on the final answer, replacing the stored answer's content when it changes. The root's boundary hooks guard the whole family: input runs the root's beforeAgent stack, then the active agent's, and <id>/finish of a handoff target runs its own afterAgent stack, then the root's. A Left from either hook, a blank query from beforeAgent and a blank answer from afterAgent block the run (design 4.13): it ends as a finished failure, RunResult.Failed, carrying that error, and the thread stays usable; a cancellation is not a Block. An input Block commits only a new thread's seed - its imported history and the root as active agent - and an output Block retracts the blocked turn: every key the turn wrote, a middleware's or a tool's, returns to its value before the turn (recorded in TurnState.priorValues), the turn is removed from the history, and the agent and transfer the turn started with are restored; only usage is kept (design ยง4.22). wrapModelCall and wrapToolCall stay per agent. A middleware's tools join the loop's org.llm4s.agent.graph.tool.ToolSet, and its writes are the keys every one of its hooks may update: the boundary hooks' updates commit with the input and the finish, a model wrapper's with the model step, a tool wrapper's with a Success. Before each model step every middleware's promptSections join the agent's system prompt, never stored. Every provider call is counted in the thread's usage, whichever response the wrappers keep.
  • Each call is its own task. Its tool is looked up, its raw arguments validated against the tool's argumentSchema, decoded and checked by the tool's validateDecoded - any failure is an error result the model sees, before any middleware or the tool runs. Then the middleware stack's wrapToolCall chain, the first middleware outermost, with the tool at its centre.
  • A NeedsApproval, from a wrapper or the tool, suspends the call with an ApprovalRequest naming its ApprovalSource, and resumes at the agent's approval node. Approve and Edit check the arguments again (an edit's are new) and run the whole chain again with ToolContext.approved, so a deny rule still refuses an edit; a second NeedsApproval from an approved call is an error result. A tool's Ask suspends with a ToolQuestionRequest at that tool's ask/<name> node, which runs the tool's resume with the decoded answer inside the same chain, with the approval the call asked with.
  • A middleware extending AgentMiddleware.Asking asks a typed question (#1704): from beforeAgent (the input node), wrapModelCall (<id>/model), afterAgent (<id>/finish) or wrapToolCall (each call). The asking task suspends, storing nothing but the usage of model calls a wrapper made before asking, at <id>/asked/<middleware>/<hook> with a MiddlewareQuestionRequest; once answered, it runs again with every answer given so far, so the hook's stack runs again from the outermost middleware and the asking one continues (GivenAnswer).
  • The loop, not the tool, records exactly one ToolResult per call - for success, failure, denial, rejection and unknown tools alike, however often a wrapper ran the tool; the results key refuses a second one. A tool's thrown exception is that call's error result. Fatal, an update to a key neither the tool nor any middleware declares, and a question it does not declare fail the run with GraphError.ToolFailed; a throwing wrapper fails it with GraphError.MiddlewareFailed. Either leaves the checkpoint Running, so recover re-runs only that call.
  • A call whose locks (AgentToolSpec.withLocks) conflict with an earlier call's waits in ToolLoop.queue: the model step fans out only the calls no earlier call conflicts with, and collect, when the barrier releases, fans out the next round from the queue, so conflicting calls run in successive supersteps in call order, each seeing what the earlier committed (two supersteps per further round).
  • collect runs only when the barrier releases; once the queue is empty, it checks every call of the batch has exactly one result, and appends the ToolMessages in call order. The model never sees a partial batch. A call cut short by cancellation, a Fatal or a crash records no result, so the batch stays open and the thread waits for recover, which re-runs only that call (design 4.4, 5.3).
  • Every call carries an org.llm4s.agent.graph.IdempotencyKey, derived by the model step from where it ran on the thread (checkpoint, superstep, task) and the call's id, and recorded with the call: each run of the call - a retrying wrapper, approval, an answered question, recover - sees the same key in ToolContext.idempotencyKey, and a call of another model request a new one, even when the provider reuses its id and the run its RunId.
  • The model node re-checks the history with Message.validateConversation before every call, and refuses to send a request - as every wrapModelCall left it - in which a tool call lacks exactly one result straight after its message (ToolResultRule); a model message whose call ids repeat or are blank is refused before it is stored, as a blank answer is, and imported history must keep the same rule.
  • Each LoopHandoff is offered to the agent's model as a stand-in tool handoff_to_<target> (HandoffTools), which call-tool never runs. When a handoff is an assistant message's only call, <id>/model stores the message and ToolMessage("Transferred to <target>"), makes the target the active agent and routes to <target>/model; the next turn starts there too. A handoff with other calls, or two handoffs (to one target or to several), is refused: every call gets an error result and the same agent is asked again. Both cost the step the model call took. The turn's steps are shared by every agent: each model node checks its own agent's maxSteps against the turn's count so far, so a target with a lower limit than its source can end the turn with TurnOutcome.StepLimitReached before its first call. The handoff call's reason argument is offered to the model but neither validated nor used for routing; the tool name alone picks the target. Each transfer is recorded in LoopKeys.transfer. With preserveContext = false the target is sent the last user message before the transfer, then the transfer's assistant message onwards; the history itself is untouched. A non-root agent with no recorded transfer to it fails the call rather than being sent the full history.

Attributes

Companion
object
Source
ToolLoop.scala
Graph
Supertypes
class Object
trait Matchable
class Any

Members list

Value members

Concrete methods

def answer[Ans : ReadWriter](id: InterruptId, answer: Ans): (InterruptId, Value)

One question's answer, of the asking tool's answer type, to merge with answers into one resume map. An answer that does not decode as the tool's type becomes that call's error result.

One question's answer, of the asking tool's answer type, to merge with answers into one resume map. An answer that does not decode as the tool's type becomes that call's error result.

Attributes

Source
ToolLoop.scala

Answers for GraphRuntime.resume or CompiledGraph.resume. Every agent's approval node takes the same answer codec, so a decision encodes the same whichever agent asked.

Answers for GraphRuntime.resume or CompiledGraph.resume. Every agent's approval node takes the same answer codec, so a decision encodes the same whichever agent asked.

Attributes

Source
ToolLoop.scala

The tasks static breakpoints hold, in the order they were held; a tool call held before it runs names its call.

The tasks static breakpoints hold, in the order they were held; a tool call held before it runs names its call.

Attributes

Source
ToolLoop.scala

The task calls a suspended run is waiting on because their children suspended (design 4.46): each one's question is an org.llm4s.agent.delegation.DelegatedQuestion.

The task calls a suspended run is waiting on because their children suspended (design 4.46): each one's question is an org.llm4s.agent.delegation.DelegatedQuestion.

Attributes

Source
ToolLoop.scala

The middleware questions a suspended run is waiting on; read each one's question with ToolLoop.middlewareQuestion.

The middleware questions a suspended run is waiting on; read each one's question with ToolLoop.middlewareQuestion.

Attributes

Source
ToolLoop.scala

The tool questions a suspended run is waiting on; read each one's question with ToolLoop.question. A task call waiting on its suspended child is not one of them: see delegatedQuestions.

The tool questions a suspended run is waiting on; read each one's question with ToolLoop.question. A task call waiting on its suspended child is not one of them: see delegatedQuestions.

Attributes

Source
ToolLoop.scala

The approval requests a suspended run is waiting on, at any agent's approval node.

The approval requests a suspended run is waiting on, at any agent's approval node.

Attributes

Source
ToolLoop.scala

Concrete fields