org.llm4s.effect.cats

Members list

Type members

Classlikes

trait AgentIO[F[_]]

cats-effect wrapper for Agent.

cats-effect wrapper for Agent.

Lifts every Result[AgentResult] into F[AgentResult], raising an LLMError as LLMException in the error channel. run and continueConversation start the turn on a blocking thread and then await it, so cancelling the fiber cancels the turn itself (AgentRun.cancel) and returns once it has ended: its model call and tool calls are interrupted, and the thread is left for recover. recover and resume do the same through Agent.startRecover and Agent.startResume.

stream, streamResume and streamRecover run a turn as an fs2 stream of its events (AgentStreamItem.Event), then its result (AgentStreamItem.Done); interrupting the stream cancels the turn as cancelling run does. Stopping early (take(n)) also cancels the turn. A consumer too slow for the stream's buffer never holds the run up: it loses live events (text deltas, tool progress) and receives one StreamEvent.LiveGap with their count where they were dropped; durable events are kept, and the run carries on. A consumer that stops draining altogether fails the stream (and so cancels the turn) once more than Agent.StreamCapacity durable events wait, as the runtime disconnects a listener that far behind.

A model call that throws instead of returning Left ends the turn with a GraphError.NodeFailed whose cause is the LLMError the runtime made of the throwable; it is raised as an LLMException, not as the raw throwable.

Intentionally a thin wrapper: build the Agent with its tools, middleware (guardrails), handoffs and options through Agent.builder, or LLMClientIO.agent.

Attributes

Companion
object
Source
AgentIO.scala
Supertypes
class Object
trait Matchable
class Any
object AgentIO

Attributes

Companion
trait
Source
AgentIO.scala
Supertypes
class Object
trait Matchable
class Any
Self type
AgentIO.type

An item of an agent's event stream: each of the run's events, then its result.

An item of an agent's event stream: each of the run's events, then its result.

Attributes

Source
AgentStreamItem.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
trait LLMClientIO[F[_]]

cats-effect wrapper for LLMClient.

cats-effect wrapper for LLMClient.

All blocking LLM calls are shifted to the blocking thread pool via Async[F].interruptible, keeping the compute pool free for fibers. Cancelling the fiber (or a timeout) interrupts the provider thread, which is how llm4s providers are cancelled: they keep the interrupt and return Left(CancelledError). LLMError values are surfaced as LLMException in the F error channel.

Attributes

Companion
object
Source
LLMClientIO.scala
Supertypes
class Object
trait Matchable
class Any
object LLMClientIO

Attributes

Companion
trait
Source
LLMClientIO.scala
Supertypes
class Object
trait Matchable
class Any
Self type
final class LLMException(val error: LLMError) extends RuntimeException

Bridges LLMError into the cats-effect error channel as a Throwable.

Bridges LLMError into the cats-effect error channel as a Throwable.

Attributes

Source
LLMException.scala
Supertypes
class Exception
class Throwable
trait Serializable
class Object
trait Matchable
class Any
Show all