ModelContext

org.llm4s.agent.graph.middleware.ModelContext
See theModelContext companion object
@Experimental
final class ModelContext

What a middleware's beforeAgent, wrapModelCall, promptSections and afterAgent hooks see: the task's org.llm4s.agent.graph.RunContext and the committed org.llm4s.agent.graph.ThreadState the task read. A hook never sees an update made in the same superstep, its own included: an update is visible from the next one.

The loop counts every model call it makes for the agent. A hook that calls a model itself - or hands a client to a library that does, such as a summariser, a memory manager or an LLM judge - wraps the client with accounted, so those calls are counted too, also when the task later fails (org.llm4s.agent.graph.RunEvent.RunFailed carries them). Usage that arrives without a client call is recorded with recordUsage.

Attributes

Companion
object
Graph
Supertypes
class Object
trait Matchable
class Any

Members list

Value members

Concrete methods

def accounted(client: LLMClient): LLMClient

client, with the usage of every complete and streamComplete through it - and of completeStructured, which completes through complete - recorded in this task's usage and announced with a durable ModelCallCompleted (attempts 0). close does not close client: the hook that owns it does.

client, with the usage of every complete and streamComplete through it - and of completeStructured, which completes through complete - recorded in this task's usage and announced with a durable ModelCallCompleted (attempts 0). close does not close client: the hook that owns it does.

Attributes

def recordUsage(model: String, usage: TokenUsage, estimatedCost: Option[Double]): Unit

Records usage that arrived without a client call, as one request to model.

Records usage that arrived without a client call, as one request to model.

Attributes

Concrete fields