LLMClientZ

org.llm4s.zio.LLMClientZ
See theLLMClientZ companion object
trait LLMClientZ

ZIO wrapper for LLMClient.

Blocking LLM calls are shifted to ZIO's blocking thread pool via ZIO.attemptBlockingInterrupt, keeping the fiber executor free. Interrupting the fiber (or a timeout) interrupts the provider thread, which is how llm4s providers are cancelled: they keep the interrupt and return Left(CancelledError). An exception thrown by a provider is a defect. LLMError is used directly as the error channel type — no wrapping needed.

Attributes

Companion
object
Source
LLMClientZ.scala
Graph
Supertypes
class Object
trait Matchable
class Any

Members list

Value members

Abstract methods

def agent(id: String)(configure: AgentBuilder => AgentBuilder): IO[LLMError, AgentZ]

Creates an AgentZ backed by this client: Agent.builder(id, client) with configure applied. A builder that does not build fails with its LLMError.

Creates an AgentZ backed by this client: Agent.builder(id, client) with configure applied. A builder that does not build fails with its LLMError.

Attributes

Source
LLMClientZ.scala
def complete(conversation: Conversation, options: CompletionOptions): ZIO[Any, LLMError, Completion]

Attributes

Source
LLMClientZ.scala

Streams chunks incrementally as a ZStream.

Streams chunks incrementally as a ZStream.

The underlying blocking call runs on an interruptible blocking thread and pushes each chunk into a bounded queue (backpressure: the provider thread blocks while the consumer lags). Chunks are delivered as they arrive, not after the call completes. If the call fails mid-stream, chunks already received are emitted first and the stream then fails with the LLMError. Stopping consumption early (e.g. take) or interrupting the fiber interrupts the blocking call.

Attributes

Source
LLMClientZ.scala