Keeps a large tool result out of the conversation (design ยง4.34). A Success whose text is over thresholdTokens estimated tokens (four characters a token) is stored in ToolOutputOffload.key, cut to maxResultBytes, and the model is shown instead a header naming the path /large_tool_results/<idempotencyKey> and the first lines of the text. OffloadBackend, mounted at ToolOutputOffload.Mount in a CompositeBackend, serves the stored results to file tools, read-only.
Identity. The path is the call's IdempotencyKey, the runtime's identity of the call: a provider that reuses a tool call id cannot make an older result's path name a newer call's content, and a rerun of the same call (recover) replaces its own entry.
The bound. The stored results are at most maxTotalBytes, as a checkpoint stores them: each update evicts the oldest until they fit, when it is applied, so the bound holds under a parallel batch and whether or not a SummarizationMiddleware runs. Reading an evicted path is a tool error that says so.
Content parts. Only text is offloaded. Every non-text part of a result is passed on as it is, after the preview.
Cleanup. With a SummarizationMiddleware, results whose path appears in a tool result the summary covers are dropped at the next model step. The bound does not depend on it.
An output Block restores the key, so a blocked turn's results are never kept.
Value parameters
maxResultBytes
the most bytes of one stored result; at least 1 and at most maxTotalBytes
maxTotalBytes
the most bytes all stored results take together
thresholdTokens
a result over this many estimated tokens is offloaded; at least 1000
Sends every tool result of the request within the threshold, whoever produced it: an outcome given instead of running the tool - the loop's argument, approval and timeout errors, a denial from a middleware outside this one - never reaches wrapToolCall, so its text is cut here, as it is sent. A Success over the threshold was already offloaded. Media parts are kept.
Sends every tool result of the request within the threshold, whoever produced it: an outcome given instead of running the tool - the loop's argument, approval and timeout errors, a denial from a middleware outside this one - never reaches wrapToolCall, so its text is cut here, as it is sent. A Success over the threshold was already offloaded. Media parts are kept.
Wraps one tool call, after its arguments were validated. A wrapper denies with ToolOutcome.Error("Denied: ..."), asks for approval with NeedsApproval(reason) (unless context.approved), fails the run with Fatal, and short-circuits by not calling next. It may call next more than once (retry) and transform the outcome next returns.
Wraps one tool call, after its arguments were validated. A wrapper denies with ToolOutcome.Error("Denied: ..."), asks for approval with NeedsApproval(reason) (unless context.approved), fails the run with Fatal, and short-circuits by not calling next. It may call next more than once (retry) and transform the outcome next returns.
The state keys this middleware's hooks may write: an update from beforeAgent, wrapModelCall or afterAgent, or one wrapToolCall adds to a Success. A hook's write to any other key fails the run.
The state keys this middleware's hooks may write: an update from beforeAgent, wrapModelCall or afterAgent, or one wrapToolCall adds to a Success. A hook's write to any other key fails the run.
Sees the run's input before anything else runs: returns it, possibly changed, with an update to this middleware's writes that the turn's first model call sees, or Left to block the run.
Sees the run's input before anything else runs: returns it, possibly changed, with an update to this middleware's writes that the turn's first model call sees, or Left to block the run.
Content for the system prompt of each model request: collected before every model step from the committed state, in stack order, and never stored. Each middleware bounds its own; the agent bounds them all together (maxPromptSectionTokens). Left fails the run.
Content for the system prompt of each model request: collected before every model step from the committed state, in stack order, and never stored. Each middleware bounds its own; the agent bounds them all together (maxPromptSectionTokens). Left fails the run.