org.llm4s.agent.graph.toolloop

Members list

Type members

Classlikes

final case class AgentInput(query: String, history: Vector[Message])

One turn's input: the user's query and, on a new thread only, the conversation to import before it.

One turn's input: the user's query and, on a new thread only, the conversation to import before it.

Attributes

Source
LoopAgent.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all

A reviewer's answer. Edit runs the call with new arguments, recording them in the assistant message first.

A reviewer's answer. Edit runs the call with new arguments, recording them in the assistant message first.

Attributes

Source
ToolLoop.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
final case class ApprovalRequest

The question an approval interrupt asks: one tool call, from one assistant message, with the call's org.llm4s.agent.graph.IdempotencyKey, which the call keeps when it is approved or edited. answered carries the answers middleware questions of the call were given before it asked, so its chain runs again with them once it is approved.

The question an approval interrupt asks: one tool call, from one assistant message, with the call's org.llm4s.agent.graph.IdempotencyKey, which the call keeps when it is approved or edited. answered carries the answers middleware questions of the call were given before it asked, so its chain runs again with them once it is approved.

Attributes

Companion
object
Source
ToolLoop.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all

Attributes

Companion
class
Source
ToolLoop.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type

Who asked for an approval: the tool itself, or a middleware's tool wrapper. All resume at the same approval node.

Who asked for an approval: the tool itself, or a middleware's tool wrapper. All resume at the same approval node.

Attributes

Source
ToolLoop.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
final case class BreakpointRequest

A task a static breakpoint holds, as an agent reports it: its node (<agent>/model, <agent>/call-tool, <agent>/finish; after it ran, also the approval, question and middleware-question nodes that continue those steps; or any other a run's RunConfig names), the phase, and - for a tool call held before it runs - the call. Answer it with org.llm4s.agent.graph.Breakpoint.proceed.

A task a static breakpoint holds, as an agent reports it: its node (<agent>/model, <agent>/call-tool, <agent>/finish; after it ran, also the approval, question and middleware-question nodes that continue those steps; or any other a run's RunConfig names), the phase, and - for a tool call held before it runs - the call. Answer it with org.llm4s.agent.graph.Breakpoint.proceed.

Attributes

Companion
object
Source
ToolLoop.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all

Attributes

Companion
class
Source
ToolLoop.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
final case class GivenAnswer

The answer a middleware question was given: kept while the hook's stack runs again, so that the middleware - middleware of agent agent's stack - finds it with AgentMiddleware.Asking.answered.

The answer a middleware question was given: kept while the hook's stack runs again, so that the middleware - middleware of agent agent's stack - finds it with AgentMiddleware.Asking.answered.

Attributes

Companion
object
Source
ToolLoop.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object GivenAnswer

Attributes

Companion
class
Source
ToolLoop.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
final case class LockSchedule(locks: Map[String, Vector[String]], waiting: Vector[ToolTask], refused: Set[String])

One batch's lock schedule (AgentToolSpec.withLocks), stored in ToolLoop.queue: the locks each call was scheduled with, by call id and rendered (LockKey.render, which LockKey(_) reads back), and the calls still to run, in call order. The batch keeps the order its schedule gives, whatever the lock functions say by the time a later round, a resume or a recover runs.

One batch's lock schedule (AgentToolSpec.withLocks), stored in ToolLoop.queue: the locks each call was scheduled with, by call id and rendered (LockKey.render, which LockKey(_) reads back), and the calls still to run, in call order. The batch keeps the order its schedule gives, whatever the lock functions say by the time a later round, a resume or a recover runs.

Attributes

Companion
object
Source
ToolLoop.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object LockSchedule

Attributes

Companion
class
Source
ToolLoop.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
final case class LoopAgent

One agent of a ToolLoop family: its model, tools, middleware, system prompt, step limit and handoffs. Its nodes are prefixed with its id.

One agent of a ToolLoop family: its model, tools, middleware, system prompt, step limit and handoffs. Its nodes are prefixed with its id.

Value parameters

maxPromptSectionTokens

the most every middleware's prompt sections may add to one request together, estimated at four characters a token: a request over it fails the run (design §4.22)

maxSteps

the limit this agent's model node applies to the turn's step count, which every agent of the turn shares: at it the turn ends with TurnOutcome.StepLimitReached instead of a further call, so an agent handed a turn that has already used its limit makes no call

output

the response format this agent's answers must match (design 4.26), or none

systemPrompt

sent first on every model call of this agent, never stored in the history

Attributes

Companion
object
Source
LoopAgent.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object LoopAgent

Attributes

Companion
class
Source
LoopAgent.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
LoopAgent.type
final case class LoopHandoff(target: AgentId, reason: Option[String], preserveContext: Boolean)

A transfer the agent may make to target; with preserveContext = false the target sees only the transfer onwards.

A transfer the agent may make to target; with preserveContext = false the target sees only the transfer onwards.

Attributes

Source
LoopAgent.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object LoopKeys

State keys the tool loop owns: no tool or middleware may declare them.

State keys the tool loop owns: no tool or middleware may declare them.

Attributes

Source
LoopKeys.scala
Supertypes
class Object
trait Matchable
class Any
Self type
LoopKeys.type

An operation on the message history, applied to the committed value in commit order.

An operation on the message history, applied to the committed value in commit order.

Attributes

Source
Messages.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object Messages

Attributes

Source
Messages.scala
Supertypes
class Object
trait Matchable
class Any
Self type
Messages.type

The hook a middleware asked its question from.

The hook a middleware asked its question from.

Attributes

Source
ToolLoop.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
final case class MiddlewareQuestionRequest

The question a middleware interrupt asks (org.llm4s.agent.graph.middleware.AgentMiddleware.Asking): middleware of agent agent's stack asked question, encoded with its declared question codec, from hook. Read it as the middleware's type with ToolLoop.middlewareQuestion; answer it with the middleware's answer type.

The question a middleware interrupt asks (org.llm4s.agent.graph.middleware.AgentMiddleware.Asking): middleware of agent agent's stack asked question, encoded with its declared question codec, from hook. Read it as the middleware's type with ToolLoop.middlewareQuestion; answer it with the middleware's answer type.

The rest is what the hook runs again with once answered: the turn's input (BeforeAgent); the tool call and whether it was approved (WrapToolCall); and the answers other questions of the same hook run were answered with.

Attributes

Companion
object
Source
ToolLoop.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all

Attributes

Companion
class
Source
ToolLoop.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
final class ModelCall

One invocation of an agent's model: the task's RunContext, the agent, and this attempt's number (from 1).

One invocation of an agent's model: the task's RunContext, the agent, and this attempt's number (from 1).

Attributes

Source
ToolLoop.scala
Supertypes
class Object
trait Matchable
class Any
trait ModelStep

The model call: the conversation so far, and the tools on offer, to the model's completion.

The model call: the conversation so far, and the tools on offer, to the model's completion.

Attributes

Companion
object
Source
ToolLoop.scala
Supertypes
class Object
trait Matchable
class Any
object ModelStep

Attributes

Companion
trait
Source
ToolLoop.scala
Supertypes
class Object
trait Matchable
class Any
Self type
ModelStep.type
@Experimental
final class OutputFormat

The response format of an agent's answer (design 4.26): the JSON schema its final answer must match, and the type it must read as. A model call of the agent that offers no tools carries the schema as its responseFormat (see ModelStep.fromClient), the system prompt always states it, and an answer is checked inside the model step: an invalid one is retried once, and a second ends the turn with org.llm4s.agent.graph.GraphError.OutputInvalid. Build one with OutputFormat.of; AgentBuilder.withOutput does.

The response format of an agent's answer (design 4.26): the JSON schema its final answer must match, and the type it must read as. A model call of the agent that offers no tools carries the schema as its responseFormat (see ModelStep.fromClient), the system prompt always states it, and an answer is checked inside the model step: an invalid one is retried once, and a second ends the turn with org.llm4s.agent.graph.GraphError.OutputInvalid. Build one with OutputFormat.of; AgentBuilder.withOutput does.

Attributes

Companion
object
Source
OutputFormat.scala
Supertypes
class Object
trait Matchable
class Any
object OutputFormat

Attributes

Companion
class
Source
OutputFormat.scala
Supertypes
class Object
trait Matchable
class Any
Self type
final case class PriorValue(value: Option[VersionedJson], written: Option[String])

What an output Block needs to undo a turn's writes to one key: its value before the turn first wrote it, encoded with the key's codec (None: no stored entry), and the fingerprint of what the turn's own writes last left there (written; None: no stored entry). A Block restores the key only while it still holds written, so a later write by anyone else - updateState while the turn waits for review - is kept (design §4.22).

What an output Block needs to undo a turn's writes to one key: its value before the turn first wrote it, encoded with the key's codec (None: no stored entry), and the fingerprint of what the turn's own writes last left there (written; None: no stored entry). A Block restores the key only while it still holds written, so a later write by anyone else - updateState while the turn waits for review - is kept (design §4.22).

Attributes

Source
LoopAgent.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
final case class StoredMessage(id: String, message: Message)

A conversation message with a stable, graph-owned id. Core org.llm4s.llmconnect.model.Message stays id-free; ids exist so operations can name the message they change, and are derived from the task that wrote the message so a re-run produces the same ids.

A conversation message with a stable, graph-owned id. Core org.llm4s.llmconnect.model.Message stays id-free; ids exist so operations can name the message they change, and are derived from the task that wrote the message so a re-run produces the same ids.

Attributes

Source
Messages.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
final class ToolLoop

The model/tool loop of an agent family as one graph - the Stage 0 proof of runtime-owned tool results and resumable approval ((#1269)), running org.llm4s.agent.graph.tool.AgentTools ((#1278)), for every LoopAgent of a family ((#1328)). Each agent's nodes are prefixed with its id; input is the one shared entry:

The model/tool loop of an agent family as one graph - the Stage 0 proof of runtime-owned tool results and resumable approval ((#1269)), running org.llm4s.agent.graph.tool.AgentTools ((#1278)), for every LoopAgent of a family ((#1328)). Each agent's nodes are prefixed with its id; input is the one shared entry:

input -> <id>/model --fan-out, one task per call--> <id>/call-tool --(join "<id>/tool-batch")--> <id>/collect -> <id>/model
            \                                          \--suspend--> <id>/approval  --/
             \--final answer--> <id>/finish             \--suspend--> <id>/ask/<tool> --/
  • A turn takes an AgentInput. On a new thread input imports its history - no system message, and a valid conversation - and makes the root the active agent; on an existing thread a history is refused. It then runs the root's beforeAgent stack on the query and, when another agent is active, that agent's stack on the result, appends the user message, resets LoopKeys.turn and routes to the active agent's model. A Left from either stack, or a blank query, blocks the run without the query or a model call; a new thread still keeps its imported history and the root as its active agent. The turn's TurnOutput is its outcome and the active agent.
  • <id>/model counts the turn's steps: at the agent's maxSteps it ends the turn with TurnOutcome.StepLimitReached without calling the model. It counts the run's tokens too - the thread's usage since the run was admitted, plus the task's own - and at the run's RunBudgets.maxTokens ends the turn with TurnOutcome.TokenLimitReached, also without calling the model; the call in flight when the budget is crossed is never interrupted, so a run can overshoot it by one call. Otherwise it sends the agent's system prompt, then the history; the prompt is never stored.
  • The middleware stack (#1279) runs at the run's boundaries and around each call: input runs every beforeAgent on the user's text, model runs ModelStep.next inside every wrapModelCall, and finish runs every afterAgent (in reverse) on the final answer, replacing the stored answer's content when it changes. The root's boundary hooks guard the whole family: input runs the root's beforeAgent stack, then the active agent's, and <id>/finish of a handoff target runs its own afterAgent stack, then the root's. A Left from either hook, a blank query from beforeAgent and a blank answer from afterAgent block the run (design 4.13): it ends as a finished failure, RunResult.Failed, carrying that error, and the thread stays usable; a cancellation is not a Block. An input Block commits only a new thread's seed - its imported history and the root as active agent - and an output Block retracts the blocked turn: every key the turn wrote, a middleware's or a tool's, returns to its value before the turn (recorded in TurnState.priorValues), the turn is removed from the history, and the agent and transfer the turn started with are restored; only usage is kept (design §4.22). wrapModelCall and wrapToolCall stay per agent. A middleware's tools join the loop's org.llm4s.agent.graph.tool.ToolSet, and its writes are the keys every one of its hooks may update: the boundary hooks' updates commit with the input and the finish, a model wrapper's with the model step, a tool wrapper's with a Success. Before each model step every middleware's promptSections join the agent's system prompt, never stored. Every provider call is counted in the thread's usage, whichever response the wrappers keep.
  • Each call is its own task. Its tool is looked up, its raw arguments validated against the tool's argumentSchema, decoded and checked by the tool's validateDecoded - any failure is an error result the model sees, before any middleware or the tool runs. Then the middleware stack's wrapToolCall chain, the first middleware outermost, with the tool at its centre.
  • A NeedsApproval, from a wrapper or the tool, suspends the call with an ApprovalRequest naming its ApprovalSource, and resumes at the agent's approval node. Approve and Edit check the arguments again (an edit's are new) and run the whole chain again with ToolContext.approved, so a deny rule still refuses an edit; a second NeedsApproval from an approved call is an error result. A tool's Ask suspends with a ToolQuestionRequest at that tool's ask/<name> node, which runs the tool's resume with the decoded answer inside the same chain, with the approval the call asked with.
  • A middleware extending AgentMiddleware.Asking asks a typed question (#1704): from beforeAgent (the input node), wrapModelCall (<id>/model), afterAgent (<id>/finish) or wrapToolCall (each call). The asking task suspends, storing nothing but the usage of model calls a wrapper made before asking, at <id>/asked/<middleware>/<hook> with a MiddlewareQuestionRequest; once answered, it runs again with every answer given so far, so the hook's stack runs again from the outermost middleware and the asking one continues (GivenAnswer).
  • The loop, not the tool, records exactly one ToolResult per call - for success, failure, denial, rejection and unknown tools alike, however often a wrapper ran the tool; the results key refuses a second one. A tool's thrown exception is that call's error result. Fatal, an update to a key neither the tool nor any middleware declares, and a question it does not declare fail the run with GraphError.ToolFailed; a throwing wrapper fails it with GraphError.MiddlewareFailed. Either leaves the checkpoint Running, so recover re-runs only that call.
  • A call whose locks (AgentToolSpec.withLocks) conflict with an earlier call's waits in ToolLoop.queue: the model step fans out only the calls no earlier call conflicts with, and collect, when the barrier releases, fans out the next round from the queue, so conflicting calls run in successive supersteps in call order, each seeing what the earlier committed (two supersteps per further round).
  • collect runs only when the barrier releases; once the queue is empty, it checks every call of the batch has exactly one result, and appends the ToolMessages in call order. The model never sees a partial batch. A call cut short by cancellation, a Fatal or a crash records no result, so the batch stays open and the thread waits for recover, which re-runs only that call (design 4.4, 5.3).
  • Every call carries an org.llm4s.agent.graph.IdempotencyKey, derived by the model step from where it ran on the thread (checkpoint, superstep, task) and the call's id, and recorded with the call: each run of the call - a retrying wrapper, approval, an answered question, recover - sees the same key in ToolContext.idempotencyKey, and a call of another model request a new one, even when the provider reuses its id and the run its RunId.
  • The model node re-checks the history with Message.validateConversation before every call, and refuses to send a request - as every wrapModelCall left it - in which a tool call lacks exactly one result straight after its message (ToolResultRule); a model message whose call ids repeat or are blank is refused before it is stored, as a blank answer is, and imported history must keep the same rule.
  • Each LoopHandoff is offered to the agent's model as a stand-in tool handoff_to_<target> (HandoffTools), which call-tool never runs. When a handoff is an assistant message's only call, <id>/model stores the message and ToolMessage("Transferred to <target>"), makes the target the active agent and routes to <target>/model; the next turn starts there too. A handoff with other calls, or two handoffs (to one target or to several), is refused: every call gets an error result and the same agent is asked again. Both cost the step the model call took. The turn's steps are shared by every agent: each model node checks its own agent's maxSteps against the turn's count so far, so a target with a lower limit than its source can end the turn with TurnOutcome.StepLimitReached before its first call. The handoff call's reason argument is offered to the model but neither validated nor used for routing; the tool name alone picks the target. Each transfer is recorded in LoopKeys.transfer. With preserveContext = false the target is sent the last user message before the transfer, then the transfer's assistant message onwards; the history itself is untouched. A non-root agent with no recorded transfer to it fails the call rather than being sent the full history.

Attributes

Companion
object
Source
ToolLoop.scala
Supertypes
class Object
trait Matchable
class Any
object ToolLoop

Attributes

Companion
class
Source
ToolLoop.scala
Supertypes
class Object
trait Matchable
class Any
Self type
ToolLoop.type
final case class ToolQuestionRequest

The question a tool's Ask interrupt carries: the call that asked, from one assistant message, the question encoded with the tool's declared question codec, the call's org.llm4s.agent.graph.IdempotencyKey, and whether the call was approved before it asked - resume sees the same ToolContext.approved and ToolContext.idempotencyKey. Find it with ToolLoop.questions, read its question as the tool's type with ToolLoop.question, and answer it with ToolLoop.answer.

The question a tool's Ask interrupt carries: the call that asked, from one assistant message, the question encoded with the tool's declared question codec, the call's org.llm4s.agent.graph.IdempotencyKey, and whether the call was approved before it asked - resume sees the same ToolContext.approved and ToolContext.idempotencyKey. Find it with ToolLoop.questions, read its question as the tool's type with ToolLoop.question, and answer it with ToolLoop.answer.

Attributes

Companion
object
Source
ToolLoop.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all

Attributes

Companion
class
Source
ToolLoop.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
final case class ToolResult(assistantMessageId: String, toolCallId: String, parts: Vector[ContentPart], isError: Boolean)

The result the loop wrote for one call, waiting for the batch barrier: the parts the call's ToolMessage will carry, or, for an error, its message as one text part.

The result the loop wrote for one call, waiting for the batch barrier: the parts the call's ToolMessage will carry, or, for an error, its message as one text part.

Attributes

Source
ToolLoop.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
final case class ToolTask(assistantMessageId: String, call: ToolCall, idempotencyKey: IdempotencyKey)

One model-issued tool call, scheduled as its own task, with the org.llm4s.agent.graph.IdempotencyKey the model step derived for it. The key is recorded with the call rather than derived again where it runs, because recover runs the call at a new checkpoint: every run of the call reads the same key from here.

One model-issued tool call, scheduled as its own task, with the org.llm4s.agent.graph.IdempotencyKey the model step derived for it. The key is recorded with the call rather than derived again where it runs, because recover runs the call at a new checkpoint: every run of the call reads the same key from here.

Attributes

Source
ToolLoop.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
final case class Transfer(source: AgentId, target: AgentId, messageId: String)

The last handoff the thread made: from source to target, by the stored assistant message messageId. Recorded by the model node when it routes a transfer; the target's model reads it to build the view a transfer without preserved context sends.

The last handoff the thread made: from source to target, by the stored assistant message messageId. Recorded by the model node when it routes a transfer; the target's model reads it to build the view a transfer without preserved context sends.

Attributes

Source
LoopAgent.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all

How a turn ended.

How a turn ended.

Attributes

Source
LoopAgent.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
final case class TurnOutput(outcome: TurnOutcome, activeAgent: AgentId)

A turn's result: how it ended, and the agent active at its end - the one the next turn starts with.

A turn's result: how it ended, and the agent active at its end - the one the next turn starts with.

Attributes

Source
LoopAgent.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
final case class TurnState(steps: Int, outcome: Option[TurnOutcome], startAgent: Option[AgentId], startTransfer: Option[Transfer], priorValues: Map[String, PriorValue])

The current turn: the model calls it has made, its outcome once it has one, and what an output Block restores when it retracts the turn - the agent and transfer it started with, and priorValues: by key id, each key the turn may write, as it was before the turn and as the turn left it (PriorValue), one entry per key. Reset by each turn's input; a turn that ends clears priorValues (design §4.22).

The current turn: the model calls it has made, its outcome once it has one, and what an output Block restores when it retracts the turn - the agent and transfer it started with, and priorValues: by key id, each key the turn may write, as it was before the turn and as the turn left it (PriorValue), one entry per key. Reset by each turn's input; a turn that ends clears priorValues (design §4.22).

Attributes

Source
LoopAgent.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all