Generated with

org.llm4s.agent

Members list

Packages

Memory system for agent context and knowledge retention.

The memory package provides a comprehensive system for agents to remember information across conversations, learn about entities, and retrieve relevant context for responses.

== Core Concepts ==

'''Memory Types:'''

Conversation - Messages from conversation history
Entity - Facts about people, organizations, concepts
Knowledge - Information from external documents
UserFact - Learned user preferences and background
Task - Records of completed tasks and outcomes

'''Components:'''

memory.Memory - Individual memory entry
memory.MemoryStore - Storage backend trait
memory.MemoryManager - High-level memory operations
memory.MemoryFilter - Composable filtering predicates

== Quick Start ==

import org.llm4s.agent.memory._

// Create a memory manager
val manager = SimpleMemoryManager.empty

// Record a user fact
val updated = manager.recordUserFact(
 "User prefers Scala over Java",
 userId = Some("user123")
)

// Record an entity fact
val entityId = EntityId.fromName("Scala")
val withEntity = updated.flatMap(_.recordEntityFact(
 entityId = entityId,
 entityName = "Scala",
 fact = "Scala is a programming language",
 entityType = "technology"
))

// Retrieve relevant context for a query
val context = withEntity.flatMap(_.getRelevantContext("Tell me about Scala"))

== Memory Filtering ==

Filters can be composed for complex queries:

import org.llm4s.agent.memory.MemoryFilter._

// Simple filters
val conversationsOnly = conversations
val importantOnly = important(0.7)

// Combined filters
val recentImportant = after(weekAgo) && important(0.5)
val entityOrKnowledge = entities || knowledge

// Recall with filter
store.recall(recentImportant, limit = 10)

== Storage Backends ==

Multiple backends are available:

memory.InMemoryStore - Fast, ephemeral storage for testing
(Future) SQLiteStore - Persistent local storage
(Future) VectorStore - Semantic search with embeddings

Attributes

See also: memory.SimpleMemoryManager for basic memory management

memory.MemoryFilter for filtering options

Streaming event support for agent execution.

This package provides fine-grained visibility into agent execution through an event-based streaming API. Events are emitted in real-time during agent execution, enabling:

'''Real-time UIs''': Stream LLM tokens to users as they're generated
'''Progress tracking''': Monitor multi-step agent execution
'''Tool visibility''': See which tools are being invoked and their results
'''Debugging''': Trace agent behavior with timestamped events

==Event Categories==

'''Text Events''': Token-level streaming during LLM generation

TextDelta: A chunk of generated text
TextComplete: Generation complete for current step

'''Tool Events''': Tool invocation lifecycle

ToolCallStarted: Tool invocation requested
ToolCallCompleted: Tool execution finished
ToolCallFailed: Tool execution failed

'''Agent Lifecycle''': Start, step, complete, fail

AgentStarted: Execution began
StepStarted: New step started
StepCompleted: Step finished
AgentCompleted: Execution succeeded
AgentFailed: Execution failed

'''Handoff Events''': Agent-to-agent delegation

HandoffStarted: Delegating to another agent
HandoffCompleted: Handoff finished

==Usage==

import org.llm4s.agent.Agent
import org.llm4s.agent.streaming.AgentEvent._

agent.runWithEvents(
 query = "What's the weather in London?",
 tools = weatherTools,
 onEvent = {
   case TextDelta(delta, _) =>
     print(delta)  // Stream to console

   case ToolCallStarted(_, name, args, _) =>
     println(s"\n[Calling $$name with $$args]")

   case ToolCallCompleted(_, name, result, _, _, _) =>
     println(s"[Tool $$name returned: $$result]")

   case AgentCompleted(state, steps, duration, _) =>
     println(s"\n[Done in $$steps steps, $${duration}ms]")

   case AgentFailed(error, _, _) =>
     println(s"\n[Error: $$error]")

   case _ => // Ignore other events
 }
)

==Collecting Events==

To collect all events for later processing:

val events = scala.collection.mutable.ArrayBuffer[AgentEvent]()

agent.runWithEvents(
 query = "...",
 tools = tools,
 onEvent = events += _
)

// Analyze events after execution
val toolCalls = events.collect { case tc: ToolCallStarted => tc }
val totalTextLength = events.collect { case TextDelta(d, _) => d.length }.sum

Attributes

See also: Agent.runWithEvents for the streaming API

AgentEvent for the event type hierarchy

Type members

Classlikes

Core agent implementation for orchestrating LLM interactions with tool calling.

The Agent class provides a flexible framework for running LLM-powered workflows with support for tools, guardrails, handoffs, and streaming events.

== Key Features ==

'''Tool Calling''': Automatically executes tools requested by the LLM
'''Multi-turn Conversations''': Maintains conversation state across interactions
'''Handoffs''': Delegates to specialist agents when appropriate
'''Guardrails''': Input/output validation with composable guardrail chains
'''Streaming Events''': Real-time event callbacks during execution

== Basic Usage ==

for {
 client <- LLMConnect.fromEnv()
 agent = new Agent(client)
 tools = new ToolRegistry(Seq(myTool))
 state <- agent.run("What is 2+2?", tools)
} yield state.conversation.messages.last.content

== With Guardrails ==

agent.run(
 query = "Generate JSON",
 tools = tools,
 inputGuardrails = Seq(new LengthCheck(1, 10000)),
 outputGuardrails = Seq(new JSONValidator())
)

== With Streaming Events ==

agent.runWithEvents("Query", tools) { event =>
 event match {
   case AgentEvent.TextDelta(text, _) => print(text)
   case AgentEvent.ToolCallCompleted(name, result, _, _, _, _) =>
     println(s"Tool $name returned: $result")
   case _ => ()
 }
}

Value parameters

client: The LLM client for making completion requests

Attributes

See also: AgentState for the state management during execution

Handoff for agent-to-agent delegation

org.llm4s.agent.guardrails.InputGuardrail for input validation

org.llm4s.agent.guardrails.OutputGuardrail for output validation
Supertypes: class Object

trait Matchable

class Any

Represents the current state of an agent run.

Value parameters

availableHandoffs: Available handoffs for this agent (used for detecting handoff tool calls)
completionOptions: LLM completion options (temperature, maxTokens, etc.)
conversation: The conversation history (without system message - that's in systemMessage field)
initialQuery: The initial user query that started this conversation (optional for multi-turn)
logs: Execution logs for this turn
status: Current agent status (InProgress, WaitingForTools, Complete, or Failed)
systemMessage: The system message (injected at API call time, not stored in conversation)
tools: The available tool registry

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: AgentState.type

Status of the agent run

Attributes

Companion: object
Supertypes: class Object

trait Matchable

class Any
Known subtypes: object Complete

class Failed

class HandoffRequested

object InProgress

object WaitingForTools

Attributes

Companion: trait
Supertypes: trait Sum

trait Mirror

class Object

trait Matchable

class Any
Self type: AgentStatus.type

Configuration for automatic context window management.

Provides flexible strategies for pruning conversation history to stay within token or message count limits.

Value parameters

maxMessages: Maximum number of messages to keep (simpler alternative to token-based)
maxTokens: Maximum number of tokens to keep (if specified, requires tokenCounter)
minRecentTurns: Minimum number of recent turns to preserve (even if limit exceeded)
preserveSystemMessage: Always keep the system message (recommended)
pruningStrategy: Strategy for pruning messages when limits are exceeded

Attributes

Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Represents a handoff to another agent.

Handoffs provide a simpler alternative to DAG-based orchestration for common delegation patterns. The LLM decides when to invoke a handoff by calling a generated handoff tool.

Example:

val generalAgent = new Agent(client)
val specialistAgent = new Agent(client)

generalAgent.run(
 "Explain quantum entanglement",
 tools,
 handoffs = Seq(
   Handoff(
     targetAgent = specialistAgent,
     transferReason = Some("Requires physics expertise"),
     preserveContext = true
   )
 )
)

Value parameters

preserveContext: Whether to transfer conversation history (default: true)
targetAgent: The agent to hand off to
transferReason: Optional reason for the handoff (shown to LLM in tool description)
transferSystemMessage: Whether to transfer system message (default: false)

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: Handoff.type

Strategies for pruning conversation history when context limits are exceeded.

Attributes

Companion: object
Supertypes: class Object

trait Matchable

class Any
Known subtypes: class Custom

object MiddleOut

object OldestFirst

class RecentTurnsOnly

Attributes

Companion: trait
Supertypes: trait Sum

trait Mirror

class Object

trait Matchable

class Any
Self type: PruningStrategy.type

In this article

Generated with