What a middleware's beforeAgent, wrapModelCall, promptSections and afterAgent hooks see: the task's org.llm4s.agent.graph.RunContext and the committed org.llm4s.agent.graph.ThreadState the task read. A hook never sees an update made in the same superstep, its own included: an update is visible from the next one.
The loop counts every model call it makes for the agent. A hook that calls a model itself - or hands a client to a library that does, such as a summariser, a memory manager or an LLM judge - wraps the client with accounted, so those calls are counted too, also when the task later fails (org.llm4s.agent.graph.RunEvent.RunFailed carries them). Usage that arrives without a client call is recorded with recordUsage.
Under a token budget (RunBudgets.maxTokens) every accounted call is checked before it is made, as the loop checks its own: at the budget it is refused with a Left. In wrapModelCall and promptSections the model step then ends the turn TokenLimitReached, whatever the wrapper does with the refusal; in beforeAgent and afterAgent the hook handles it as any failed call (an LLM guardrail blocks). A completion that reports no usage is charged an estimate (the request's messages in, the completion out), never zero, so the budget still advances.
Attributes
- Companion
- object
- Graph
-
- Supertypes
-
class Objecttrait Matchableclass Any