StreamingAccumulator

org.llm4s.llmconnect.streaming.StreamingAccumulator
See theStreamingAccumulator companion object
@Stable
final class StreamingAccumulator

Accumulates streaming chunks into a complete response. Handles content accumulation, tool call accumulation, thinking content, and token tracking.

Mutable and not thread-safe: use one accumulator per stream, fed from the thread that reads it. Tool calls are keyed by id, so every chunk of a streamed tool call - continuations included - must carry its call's id; a tool-call chunk with an empty id is ignored. Clients whose wire format identifies continuations only by index must map them back to the id first.

Attributes

Companion
object
Graph
Supertypes
class Object
trait Matchable
class Any

Members list

Value members

Concrete methods

def addChunk(chunk: StreamedChunk): Unit

Add a streaming chunk to the accumulator

Add a streaming chunk to the accumulator

Attributes

def addThinkingDelta(delta: String): Unit

Add thinking content delta directly

Add thinking content delta directly

Attributes

def clear(): Unit

Clear the accumulator state

Clear the accumulator state

Attributes

def currentContent: String

Get the current accumulated content

Get the current accumulated content

Attributes

def currentThinking: Option[String]

Get the current accumulated thinking content

Get the current accumulated thinking content

Attributes

Get the current tool calls, in the order each was first seen in the stream

Get the current tool calls, in the order each was first seen in the stream

Attributes

def hasThinking: Boolean

Check if there is any thinking content

Check if there is any thinking content

Attributes

def isComplete: Boolean

Check if accumulation is complete

Check if accumulation is complete

Attributes

Convert accumulated data to a Completion, stamped with the current time

Convert accumulated data to a Completion, stamped with the current time

Attributes

def toCompletion(created: Instant): Result[Completion]

Convert accumulated data to a Completion, stamped with the given creation time. Pure given the accumulator's current state - split out from the no-arg overload so the conversion itself is testable without a real clock.

Convert accumulated data to a Completion, stamped with the given creation time. Pure given the accumulator's current state - split out from the no-arg overload so the conversion itself is testable without a real clock.

Attributes

def updateTokens(prompt: Int, completion: Int): Unit

Update the prompt and completion token counts.

Update the prompt and completion token counts.

The completion's total becomes their sum plus any thinking tokens; cache counts set by updateUsage are kept. Use updateUsage when the provider reports more than these two.

Attributes

def updateTokensWithThinking(prompt: Int, completion: Int, thinking: Int): Unit

Update token counts including thinking tokens.

Update token counts including thinking tokens.

The completion's total becomes their sum; cache counts set by updateUsage are kept.

Attributes

def updateUsage(usage: TokenUsage): Unit

Merge a provider's usage report into the accumulated usage, every org.llm4s.llmconnect.model.TokenUsage field included, so a streamed org.llm4s.llmconnect.model.Completion reports what a non-streaming one does.

Merge a provider's usage report into the accumulated usage, every org.llm4s.llmconnect.model.TokenUsage field included, so a streamed org.llm4s.llmconnect.model.Completion reports what a non-streaming one does.

Call it once per usage report the stream carries. Each field follows the latest report that states it: promptTokens and completionTokens when positive, the Option fields (thinkingTokens, cachedTokens, cacheCreationTokens) when defined, and totalTokens when positive. A field a report leaves at zero or None keeps its earlier value, so a provider that splits its usage across events (input and cache counts first, output last) can pass each part as it arrives. When no report states a total, the completion's total is prompt plus completion plus thinking tokens.

Value parameters

usage

the usage as the provider reported it, in llm4s's terms: thinkingTokens the reasoning tokens, cachedTokens the prompt tokens read from the provider's prompt cache, cacheCreationTokens those written to it

Attributes