org.llm4s.ox

Direct-style operations over an LLMClient, built on SoftwareMill's Ox: a streamed completion as an Ox Flow, bounded fan-out and a race between providers. import org.llm4s.ox.* brings them into scope.

Core's calls already block the calling thread and return a Result; interrupting that thread cancels the call, which then returns Left(CancelledError). These operations keep that model and add what plain direct style leaves to the caller: a pull-based stream, a parallelism bound and the interruption of work nobody waits for. Every fork runs on a JDK 21 virtual thread.

Carries an LLMError in a failed Ox Flow, which can only fail with a Throwable. The error is the LLMError the provider reported. LLMClientOx.accumulate turns it back into a Left.

Ox retry presets for Result-returning llm4s calls.

What is retried is the library's one rule for automatic retry, the one ReliableClient and the agent graph use (org.llm4s.reliability.RetryPolicy's isRetryable): a recoverable error, except a ServiceError or an APIError with a client-error status (any 4xx but 408 and 429) and an OptimisticLockFailure. Every other error, a CancelledError included, is returned at once.

val answer: Result[Completion] = RetrySchedule.retry()(client.complete(conversation))
val custom = retryEither(RetrySchedule.config[Completion](Schedule.fixedInterval(2.seconds).maxRetries(5)))(
 client.complete(conversation)
)