Generated with

llm4s-core/org.llm4s/org.llm4s.agent/Agent

Agent

org.llm4s.agent.Agent

See theAgent companion object

class Agent(client: LLMClient)

Core agent implementation for orchestrating LLM interactions with tool calling.

The Agent class coordinates five concerns that are each handled by a dedicated module:

'''Guardrail validation''' — GuardrailApplicator
'''Trace formatting and file I/O''' — AgentTraceFormatter
'''Handoff delegation''' — HandoffExecutor
'''Tool execution''' — ToolProcessor
'''Streaming / strategy execution''' — AgentStreamingExecutor

This class is the primary orchestration entry point. It initialises agent state, drives the InProgress → WaitingForTools → Complete state machine via runStep and run, and delegates each concern to the appropriate module.

== Key Features ==

'''Tool Calling''': Automatically executes tools requested by the LLM
'''Multi-turn Conversations''': Maintains conversation state across interactions
'''Handoffs''': Delegates to specialist agents when appropriate
'''Guardrails''': Input/output validation with composable guardrail chains
'''Streaming Events''': Real-time event callbacks during execution

== Security == By default, agents have a maximum step limit of 50 to prevent infinite loops. This can be overridden by setting maxSteps explicitly.

== Basic Usage ==

for {
 providerConfig <- Llm4sConfig.provider()
 client <- LLMConnect.getClient(providerConfig)
 agent = new Agent(client)
 tools = new ToolRegistry(Seq(myTool))
 state <- agent.run("What is 2+2?", tools)
} yield state.conversation.messages.last.content

== With Guardrails ==

agent.run(
 query = "Generate JSON",
 tools = tools,
 inputGuardrails = Seq(new LengthCheck(1, 10000)),
 outputGuardrails = Seq(new JSONValidator())
)

== With Streaming Events ==

agent.runWithEvents("Query", tools) { event =>
 event match {
   case AgentEvent.TextDelta(text, _) => print(text)
   case AgentEvent.ToolCallCompleted(name, result, _, _, _, _) =>
     println(s"Tool $$name returned: $$result")
   case _ => ()
 }
}

Value parameters

client: The LLM client for making completion requests

Attributes

See also: AgentState for the state management during execution

Handoff for agent-to-agent delegation

org.llm4s.agent.guardrails.InputGuardrail for input validation

org.llm4s.agent.guardrails.OutputGuardrail for output validation
Companion: object
Graph
Supertypes: class Object

trait Matchable

class Any

Members list

Value members

Concrete methods

Appends a new user message to an existing conversation and runs the agent to completion.

Preserves the full conversation history from previousState (tools, system message, prior messages) so the LLM has context from earlier turns. When contextWindowConfig is supplied, the history is pruned before the LLM call to avoid exceeding the model's token limit.

previousState must be in Complete or Failed status. Calling with InProgress, WaitingForTools, or HandoffRequested is a programming error and returns a ValidationError immediately without running the agent.

Value parameters

context: Tracing, debug logging, and trace file path.
contextWindowConfig: When set, prunes the oldest messages to keep the conversation within the model's token budget.
inputGuardrails: Applied to newUserMessage; default none.
maxSteps: Step cap for this turn; None for unlimited.
newUserMessage: The follow-up message to process.
outputGuardrails: Applied to the final assistant message; default none.
previousState: State returned by a prior run or continueConversation call; must be Complete or Failed.

Attributes

Returns

Right(state) on success; Left(ValidationError) when previousState is not terminal, or Left on guardrail or LLM failure.

Example

val result = for {
 providerCfg <- /* load provider config */
 client      <- org.llm4s.llmconnect.LLMConnect.getClient(providerCfg)
 tool        <- WeatherTool.toolSafe
 tools       = new ToolRegistry(Seq(tool))
 agent       = new Agent(client)
 state1     <- agent.run("What's the weather in Paris?", tools)
 state2     <- agent.continueConversation(state1, "And in London?")
 state3     <- agent.continueConversation(state2, "Which is warmer?")
} yield state3

Continue a conversation with streaming events.

Value parameters

context: Cross-cutting concerns
contextWindowConfig: Optional configuration for context pruning
inputGuardrails: Validate new message before processing
maxSteps: Optional limit on reasoning steps
newUserMessage: The new user message to process
onEvent: Callback for streaming events
outputGuardrails: Validate response before returning
previousState: The previous agent state (must be Complete or Failed)

Attributes

Returns: Result containing the new agent state

Continue a conversation with a configurable tool execution strategy.

Value parameters

context: Cross-cutting concerns
contextWindowConfig: Optional configuration for automatic context pruning
ec: ExecutionContext for async operations
inputGuardrails: Validate new message before processing
maxSteps: Optional limit on reasoning steps for this turn
newUserMessage: The new user message to process
outputGuardrails: Validate response before returning
previousState: The previous agent state (must be Complete or Failed)
toolExecutionStrategy: Strategy for executing multiple tool calls

Attributes

Returns: Result containing the new agent state after processing the message

Renders the agent state as a human-readable markdown document.

Delegates to AgentTraceFormatter.formatStateAsMarkdown.

Intended for debugging and post-run inspection. The output format is not stable across library versions; do not parse the result programmatically.

Value parameters

state: Agent state to render.

Attributes

Returns: markdown string covering the conversation transcript, tool arguments, tool results, and execution log entries.

Initializes a new AgentState ready to be driven by runStep or run.

Synthesizes a built-in system prompt (step-by-step tool-use instructions) and appends systemPromptAddition when provided. Each Handoff in handoffs is converted into a synthetic tool registered alongside the caller-supplied tools, so the LLM can trigger a handoff just like any other tool call.

The system prompt is stored in AgentState.systemMessage rather than as the first message in AgentState.conversation. This separation allows the system prompt to be injected at every LLM API call without polluting the mutable conversation history — important for context-window pruning, where we must never drop the system instructions.

Value parameters

completionOptions: LLM parameters (temperature, maxTokens, reasoning effort, etc.) forwarded on every call in this run.
handoffs: Agents to delegate to; each becomes a callable tool.
query: The user message that opens the conversation.
systemPromptAddition: Text appended to the default system prompt; use this to inject domain-specific instructions without replacing the built-in tool-use guidance.
tools: Tools available for the agent to invoke during this run.

Attributes

Returns: the initialized state, or Left when synthetic handoff-tool creation fails (e.g. invalid tool name or schema).

Drives an already-initialized state to completion, failure, or the step limit.

One ''logical step'' = LLM call + subsequent tool execution. The InProgress→WaitingForTools transition and the WaitingForTools→InProgress transition together consume one step from the budget. A final LLM call with no tool calls (→ Complete) does not consume an extra step.

The loop is implemented as a tail-recursive local function. This avoids stack overflow on long-running agents that perform many reasoning turns; a chain of 50+ steps would otherwise accumulate 50+ stack frames for a non-tail-recursive implementation.

When a AgentStatus.HandoffRequested status is detected, control is transferred to the target agent with the same maxSteps budget.

Value parameters

context: Cross-cutting concerns for this run.
initialState: State produced by initializeSafe or a previous run.
maxSteps: Maximum number of LLM+tool round-trips before the run is aborted with AgentStatus.Failed("Maximum step limit reached"). None removes the limit — this is an explicit opt-out intended for bounded workflows such as unit tests where mock clients never loop. Omit None in production.

Attributes

Returns: Right(state) when the run reaches Complete or Failed; Left only when an LLM call returns an error before any terminal state is reached.

Runs a new query to completion, failure, or the step limit.

Combines initializeSafe and run into a single call with input and output guardrail support. The full pipeline is:

Input guardrails are evaluated; the first failure short-circuits to Left.
State is initialised via initializeSafe; Left on handoff-tool creation failure.
The agent loop runs until a terminal status or maxSteps is exhausted.
Output guardrails are evaluated on the final assistant message; the first failure short-circuits to Left.

Value parameters

completionOptions: LLM parameters forwarded on every call.
context: Tracing, debug logging, and trace file path.
handoffs: Agents to delegate to; each becomes a callable tool.
inputGuardrails: Applied to query before any LLM call; default none.
maxSteps: Maximum LLM+tool round-trips; defaults to Agent.DefaultMaxSteps. Pass None to remove the cap (use with caution in production).
outputGuardrails: Applied to the final assistant message; default none.
query: The user message to process.
systemPromptAddition: Text appended to the built-in system prompt.
tools: Tools the LLM may invoke during this run.

Attributes

Returns: Right(state) when the pipeline completes; Left on guardrail failure, handoff-tool creation failure, or a non-recoverable LLM error.

Collect all events during execution into a sequence.

Convenience method that runs the agent and returns both the final state and all events that were emitted during execution.

Value parameters

completionOptions: Completion options
context: Cross-cutting concerns
maxSteps: Limit on the number of steps (default: 50 for safety). Set to None for unlimited steps (use with caution).
query: The user query to process
systemPromptAddition: Optional system prompt addition
tools: The registry of available tools

Attributes

Returns: Tuple of (final state, all events)

Run multiple conversation turns sequentially. Each turn waits for the previous to complete before starting. This is a convenience method for running a complete multi-turn conversation.

Value parameters

completionOptions: Completion options
context: Cross-cutting concerns
contextWindowConfig: Optional configuration for automatic context pruning
followUpQueries: Additional user messages to process in sequence
initialQuery: The first user message
maxStepsPerTurn: Step limit per turn (default: Agent.DefaultMaxSteps for safety). Set to None for unlimited steps (use with caution).
systemPromptAddition: Optional system prompt addition
tools: Tool registry for the conversation

Attributes

Returns

Result containing the final agent state after all turns

Example

val result = agent.runMultiTurn(
 initialQuery = "What's the weather in Paris?",
 followUpQueries = Seq(
   "And in London?",
   "Which is warmer?"
 ),
 tools = tools
)

Advances the agent by exactly one state-machine transition.

One ''step'' is either:

An LLM call (in InProgress state), which transitions the agent to WaitingForTools when tools were requested or Complete when no tool calls were made. One LLM call = one billing unit.
A tool-execution batch (in WaitingForTools state), which processes all pending tool calls and transitions back to InProgress (or to HandoffRequested).

Counting LLM call + tool execution together as ''one logical step'' ensures consistent billing semantics and prevents maxSteps from being exhausted by tool executions rather than LLM reasoning turns.

States that are already terminal (Complete, Failed, HandoffRequested) are returned unchanged — callers do not need to guard against double-stepping.

Value parameters

context: Cross-cutting concerns (tracing, debug logging, trace file path).
state: Current agent state; its .status field determines the transition.

Attributes

Returns: the state after the transition, or Left when the LLM call fails. Tool execution failures are captured as AgentStatus.Failed inside a Right, not as a Left, so they are visible in the final state.

Runs the agent with streaming events for real-time progress tracking.

This method provides fine-grained visibility into agent execution through a callback that receives org.llm4s.agent.streaming.AgentEvent instances as they occur. Events include:

Token-level streaming during LLM generation
Tool call start/complete notifications
Agent lifecycle events (start, step, complete, fail)

Value parameters

completionOptions: Optional completion options for LLM calls
context: Cross-cutting concerns
handoffs: Available handoffs (default: none)
inputGuardrails: Validate query before processing (default: none)
maxSteps: Limit on the number of steps to execute (default: Agent.DefaultMaxSteps for safety). Set to None for unlimited steps (use with caution).
onEvent: Callback invoked for each event during execution
outputGuardrails: Validate response before returning (default: none)
query: The user query to process
systemPromptAddition: Optional additional text to append to the default system prompt
tools: The registry of available tools

Attributes

Returns

Either an error or the final agent state

Example

import org.llm4s.agent.streaming.AgentEvent._
agent.runWithEvents(
 query = "What's the weather?",
 tools = weatherTools,
 onEvent = {
   case TextDelta(delta, _) => print(delta)
   case ToolCallStarted(_, name, _, _) => println(s"[Calling $$name]")
   case AgentCompleted(_, steps, ms, _) => println(s"Done in $$steps steps")
   case _ =>
 }
)

Runs the agent with a configurable tool execution strategy.

This method enables parallel or rate-limited execution of multiple tool calls, which can significantly improve performance when the LLM requests multiple independent tool calls (e.g., fetching weather for multiple cities).

Value parameters

completionOptions: Optional completion options for LLM calls
context: Cross-cutting concerns
ec: ExecutionContext for async operations
handoffs: Available handoffs (default: none)
inputGuardrails: Validate query before processing (default: none)
maxSteps: Limit on the number of steps to execute (default: Agent.DefaultMaxSteps for safety). Set to None for unlimited steps (use with caution).
outputGuardrails: Validate response before returning (default: none)
query: The user query to process
systemPromptAddition: Optional additional text to append to the default system prompt
toolExecutionStrategy: Strategy for executing multiple tool calls: - Sequential: One at a time (default, safest) - Parallel: All tools simultaneously - ParallelWithLimit(n): Max n tools concurrently
tools: The registry of available tools

Attributes

Returns

Either an error or the final agent state

Example

import scala.concurrent.ExecutionContext.Implicits.global
// Execute weather lookups in parallel
val result = agent.runWithStrategy(
 query = "Get weather in London, Paris, and Tokyo",
 tools = weatherTools,
 toolExecutionStrategy = ToolExecutionStrategy.Parallel
)
// Limit concurrency to avoid rate limits
val result = agent.runWithStrategy(
 query = "Search for 10 topics",
 tools = searchTools,
 toolExecutionStrategy = ToolExecutionStrategy.ParallelWithLimit(3)
)

Overwrites traceLogPath with the markdown-formatted agent state.

Delegates to AgentTraceFormatter.writeTraceLog.

File-write failures are swallowed: the error is logged at ERROR level via SLF4J but is not surfaced to the caller. The method always returns Unit so that tracing never affects agent control flow.

Value parameters

state: Agent state to render and persist.
traceLogPath: Absolute or relative path to the output file; the file is created or truncated on each call.

Attributes

Deprecated methods

Attributes

Deprecated: [Since version 0.3.0] Use continueConversation(..., context = AgentContext(...))

Attributes

Deprecated: [Since version 0.3.0] Use continueConversationWithEvents(..., context = AgentContext(...))

Attributes

Deprecated: [Since version 0.3.0] Use continueConversationWithStrategy(..., context = AgentContext(...))

Attributes

Deprecated: [Since version 0.3.0] Use run(..., context = AgentContext(...))

Attributes

Deprecated: [Since version 0.3.0] Use runCollectingEvents(..., context = AgentContext(...))

Attributes

Deprecated: [Since version 0.3.0] Use runMultiTurn(..., context = AgentContext(...))

Attributes

Deprecated: [Since version 0.3.0] Use runStep(state, context)

Attributes

Deprecated: [Since version 0.3.0] Use runStep(state, context)

Attributes

Deprecated: [Since version 0.3.0] Use runWithEvents(..., context = AgentContext(...))

Attributes

Deprecated: [Since version 0.3.0] Use runWithStrategy(..., context = AgentContext(...))

In this article

Generated with