Long-term semantic memory for an agent (design ยง4.31): before each turn it recalls what earlier threads of the same tenant recorded and shows it to the model; after the turn it records the exchange, and, when consolidate is set, replaces groups of old memories with a summary.
Tenant scope.stores is asked once per tenant (RunContext.config.tenantId) for that tenant's store; the store is held in one cell per tenant, whose writes are serialised. Every thread of a tenant shares it; no two tenants may share a store instance (refused). A failed resolution is asked again on the next turn.
The manager is built for each hook call by manager(view, context.accounted(client)), over a view of the tenant's store whose writes are the middleware's effects, so every model call it makes is counted in the thread's usage.
Recall (beforeAgent) runs getRelevantContext on the turn's input, cuts it to maxRecallTokens * 4 characters and stores it in SemanticMemoryMiddleware.RecalledKey; every model step of the turn shows it as the prompt section semantic-memory. A resolution or configuration failure refuses the input before any model call; a failed search is skipped, announced by MemoryEvents.Skipped.
Recording (afterAgent) records the turn's query and answer through the manager as one replayable effect (memory.record, C4): on a org.llm4s.agent.memory.TransactionalMemoryStore a replay after a crash returns the recorded outcome and never writes twice; on any other store a replay of a write that may have run is refused. Each memory is cut to maxRecordChars.
Consolidation (afterAgent, after recording) runs the manager's consolidateMemories on memories older than olderThan; each group it replaces is one effect (memory.consolidate, keyed by the group's ids). It needs a transactional store: on any other, the input is refused.
Bounds. Every write evicts, in its own transaction, the memories of lowest importance, oldest first, beyond bounds, so a tenant's store stays bounded whether or not consolidation is on.
Failures after the answer are skipped and announced by MemoryEvents.Skipped: the answer is never retracted because memory could not be written. A crash is replayed by recover from C4's records.
Register it outside guardrail middleware - before it, or with runsBefore naming it - so that a blocked answer is never recorded: afterAgent runs in reverse stack order, and a guardrail's block stops the others. An effect that applied stays applied: a Block restores thread state, not the store.
The key is registered only by an agent family with a SemanticMemoryMiddleware: adding one to an existing agent changes its graph's structure.
Value parameters
bounds
the most each tenant's store keeps
client
the client the manager calls a model with; every call is accounted to the run
clock
where consolidation reads "now" from
consolidate
when to consolidate old memories after a turn; None never
id
this middleware's id
manager
builds the manager over a store and a client, e.g. LLMMemoryManager(config, _, _)
recall
what to recall before a turn; None recalls nothing
record
how to record a turn; None records nothing
runsBefore
middleware this one must run outside of, such as MiddlewareId("guardrails")
stores
the store of each tenant (None for runs without one): a store scoped to that tenant
Sees the run's input before anything else runs: returns it, possibly changed, with an update to this middleware's writes that the turn's first model call sees, or Left to block the run.
Sees the run's input before anything else runs: returns it, possibly changed, with an update to this middleware's writes that the turn's first model call sees, or Left to block the run.
Content for the system prompt of each model request: collected before every model step from the committed state, in stack order, and never stored. Each middleware bounds its own; the agent bounds them all together (maxPromptSectionTokens). Left fails the run.
Content for the system prompt of each model request: collected before every model step from the committed state, in stack order, and never stored. Each middleware bounds its own; the agent bounds them all together (maxPromptSectionTokens). Left fails the run.
f on tenant's store as the middleware's last write left it, resolving it if no run of the tenant has yet. f runs under the tenant's lock, so it never overlaps a turn's read or write of the store: read through this, not through a reference kept from the resolver, while runs may be active.
f on tenant's store as the middleware's last write left it, resolving it if no run of the tenant has yet. f runs under the tenant's lock, so it never overlaps a turn's read or write of the store: read through this, not through a reference kept from the resolver, while runs may be active.
The state keys this middleware's hooks may write: an update from beforeAgent, wrapModelCall or afterAgent, or one wrapToolCall adds to a Success. A hook's write to any other key fails the run.
The state keys this middleware's hooks may write: an update from beforeAgent, wrapModelCall or afterAgent, or one wrapToolCall adds to a Success. A hook's write to any other key fails the run.
Wraps one model call. A wrapper may rewrite the request (inject a note, filter tools), call next more than once (retry, fallback), transform its response, add an update to its writes (ModelResponse.addUpdate), or return Left, which fails the run. Only an AgentMiddleware.Asking wrapper suspends, by asking its question. Every call next makes is counted in the thread's usage, whichever response the wrapper returns.
Wraps one model call. A wrapper may rewrite the request (inject a note, filter tools), call next more than once (retry, fallback), transform its response, add an update to its writes (ModelResponse.addUpdate), or return Left, which fails the run. Only an AgentMiddleware.Asking wrapper suspends, by asking its question. Every call next makes is counted in the thread's usage, whichever response the wrapper returns.
Filtering ModelRequest.tools shapes what the model is offered and is not a permission control: a tool the model calls anyway still runs through wrapToolCall, where denial belongs.
Wraps one tool call, after its arguments were validated. A wrapper denies with ToolOutcome.Error("Denied: ..."), asks for approval with NeedsApproval(reason) (unless context.approved), fails the run with Fatal, and short-circuits by not calling next. It may call next more than once (retry) and transform the outcome next returns.
Wraps one tool call, after its arguments were validated. A wrapper denies with ToolOutcome.Error("Denied: ..."), asks for approval with NeedsApproval(reason) (unless context.approved), fails the run with Fatal, and short-circuits by not calling next. It may call next more than once (retry) and transform the outcome next returns.