When a model request grows past trigger, replaces its older messages with a summary, kept in the thread and extended as the thread grows (design ยง4.34). The stored history is never edited.
The trigger is measured on the outgoing request as it would be sent with the stored summary applied, system prompt included: its estimated tokens (tokenCounter) or its message count.
When it fires, every message after the summary but the last keepRecent goes to compressor with the summary so far; the cut moves earlier so a tool call is never separated from its results. The new summary goes straight into this request, and is written to SummarizationMiddleware.key with the stored id of the last message it covers, committed with the model step.
Later requests send the summary in place of the messages it covers - one UserMessage straight after the system prompt - and the messages after it unchanged.
The bound: a summary is at most maxSummaryTokens estimated tokens at four characters a token, cut after the compressor returns and again when a stored one is sent.
An output Block restores the key, so a blocked turn's summary is never kept. A model compressor's calls go through ModelContext.accounted, so they are counted in the thread's usage. Use this or ContextWindowMiddleware, which prunes with no model call, not both.
Value parameters
compressor
writes the summary
keepRecent
how many of the latest messages are always sent as they are; at least 1
maxSummaryTokens
the bound on the summary, in estimated tokens at four characters a token; at least SummaryCompressor.MinSummaryTokens (32)
tokenCounter
the estimate a SummaryTrigger.Tokens trigger is measured with
Wraps one model call. A wrapper may rewrite the request (inject a note, filter tools), call next more than once (retry, fallback), transform its response, add an update to its writes (ModelResponse.addUpdate), or return Left, which fails the run. Only an AgentMiddleware.Asking wrapper suspends, by asking its question. Every call next makes is counted in the thread's usage, whichever response the wrapper returns.
Wraps one model call. A wrapper may rewrite the request (inject a note, filter tools), call next more than once (retry, fallback), transform its response, add an update to its writes (ModelResponse.addUpdate), or return Left, which fails the run. Only an AgentMiddleware.Asking wrapper suspends, by asking its question. Every call next makes is counted in the thread's usage, whichever response the wrapper returns.
Filtering ModelRequest.tools shapes what the model is offered and is not a permission control: a tool the model calls anyway still runs through wrapToolCall, where denial belongs.
The state keys this middleware's hooks may write: an update from beforeAgent, wrapModelCall or afterAgent, or one wrapToolCall adds to a Success. A hook's write to any other key fails the run.
The state keys this middleware's hooks may write: an update from beforeAgent, wrapModelCall or afterAgent, or one wrapToolCall adds to a Success. A hook's write to any other key fails the run.
Sees the run's input before anything else runs: returns it, possibly changed, with an update to this middleware's writes that the turn's first model call sees, or Left to block the run.
Sees the run's input before anything else runs: returns it, possibly changed, with an update to this middleware's writes that the turn's first model call sees, or Left to block the run.
Content for the system prompt of each model request: collected before every model step from the committed state, in stack order, and never stored. Each middleware bounds its own; the agent bounds them all together (maxPromptSectionTokens). Left fails the run.
Content for the system prompt of each model request: collected before every model step from the committed state, in stack order, and never stored. Each middleware bounds its own; the agent bounds them all together (maxPromptSectionTokens). Left fails the run.
Wraps one tool call, after its arguments were validated. A wrapper denies with ToolOutcome.Error("Denied: ..."), asks for approval with NeedsApproval(reason) (unless context.approved), fails the run with Fatal, and short-circuits by not calling next. It may call next more than once (retry) and transform the outcome next returns.
Wraps one tool call, after its arguments were validated. A wrapper denies with ToolOutcome.Error("Denied: ..."), asks for approval with NeedsApproval(reason) (unless context.approved), fails the run with Fatal, and short-circuits by not calling next. It may call next more than once (retry) and transform the outcome next returns.