org.llm4s.llmconnect.streaming

Members list

Type members

Classlikes

case class AccumulatorSnapshot(content: String, thinking: Option[String], toolCalls: Seq[ToolCall], messageId: Option[String], finishReason: Option[String], promptTokens: Int, completionTokens: Int, thinkingTokens: Int)

Snapshot of accumulator state

Snapshot of accumulator state

Attributes

Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all

Anthropic-specific streaming response handler

Anthropic-specific streaming response handler

Attributes

Supertypes
class Object
trait Matchable
class Any

Base implementation of StreamingResponseHandler with common functionality

Base implementation of StreamingResponseHandler with common functionality

Attributes

Supertypes
class Object
trait Matchable
class Any
Known subtypes

OpenAI-specific streaming response handler

OpenAI-specific streaming response handler

Attributes

Supertypes
class Object
trait Matchable
class Any
object SSEParser

Server-Sent Events (SSE) parser for streaming LLM responses.

Server-Sent Events (SSE) parser for streaming LLM responses.

Handles the SSE format used by both OpenAI and Anthropic streaming endpoints. SSE format consists of fields like:

  • data:
  • event:
  • id:
  • retry:
  • :

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
SSEParser.type

Accumulates streaming chunks into a complete response. Handles content accumulation, tool call accumulation, thinking content, and token tracking.

Accumulates streaming chunks into a complete response. Handles content accumulation, tool call accumulation, thinking content, and token tracking.

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any

Factory for creating accumulators

Factory for creating accumulators

Attributes

Companion
class
Supertypes
class Object
trait Matchable
class Any
Self type
case class StreamingOptions(onStart: () => Unit, onChunk: StreamedChunk => Unit, onError: LLMError => Unit, onComplete: Completion => Unit, bufferSize: Int, connectionTimeout: Long, readTimeout: Long)

Options for streaming completions with callbacks for different events. Provides a more flexible alternative to the simple onChunk callback.

Options for streaming completions with callbacks for different events. Provides a more flexible alternative to the simple onChunk callback.

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all

Companion object with convenience constructors

Companion object with convenience constructors

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type

Handles streaming responses from LLM providers. Manages the lifecycle of streaming, chunk accumulation, and error handling.

Handles streaming responses from LLM providers. Manages the lifecycle of streaming, chunk accumulation, and error handling.

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
Known subtypes

Factory for creating provider-specific handlers

Factory for creating provider-specific handlers

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type