llm4s-core
API
Generated with
llm4s-core/org.llm4s/org.llm4s.reliability

org.llm4s.reliability

package org.llm4s.reliability

Members list

Type members

Classlikes

final case class CircuitBreakerConfig(failureThreshold: Int, recoveryTimeout: Duration, successThreshold: Int)

Circuit breaker configuration for service resilience.

Circuit breaker configuration for service resilience.

Value parameters

failureThreshold

Number of consecutive failures before opening circuit

recoveryTimeout

Time to wait before attempting recovery (half-open state)

successThreshold

Number of successes in half-open state to close circuit

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object CircuitBreakerConfig

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
sealed trait CircuitState

Circuit breaker state.

Circuit breaker state.

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
Known subtypes
object Closed
object HalfOpen
object Open
object CircuitState

Attributes

Companion
trait
Supertypes
trait Sum
trait Mirror
class Object
trait Matchable
class Any
Self type
final case class ReliabilityConfig(retryPolicy: RetryPolicy, circuitBreaker: CircuitBreakerConfig, deadline: Option[Duration], enabled: Boolean)

Configuration for reliable LLM provider calls.

Configuration for reliable LLM provider calls.

Provides retry logic, circuit breakers, and deadline enforcement to make LLM API calls resilient to transient failures.

Value parameters

circuitBreaker

Circuit breaker configuration

deadline

Maximum time to wait for operation completion

enabled

Whether reliability features are enabled (for opt-out)

retryPolicy

Retry policy for transient failures

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object ReliabilityConfig

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
object ReliabilitySyntax

Implicit syntax extensions for adding reliability to existing clients.

Implicit syntax extensions for adding reliability to existing clients.

Import this to add .withReliability() method to any LLMClient.

Example:

import org.llm4s.reliability.ReliabilitySyntax._

val client = OpenAIClient(config, metrics).map(_.withReliability("openai"))

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
final class ReliableClient(underlying: LLMClient, providerName: String, config: ReliabilityConfig, collector: Option[MetricsCollector]) extends LLMClient

Wrapper that adds reliability features to any LLMClient.

Wrapper that adds reliability features to any LLMClient.

Provides:

  • Retry with configurable policies (exponential backoff, linear, fixed)
  • Circuit breaker to fail fast when service is down
  • Deadline enforcement to prevent hanging operations
  • Metrics tracking for retry attempts and circuit breaker state

Thread-safety: Uses AtomicInteger/AtomicReference for circuit breaker state management to ensure correct behavior under concurrent access.

Value parameters

collector

Optional metrics collector for observability

config

Reliability configuration

providerName

Explicit provider name for stable metrics labels

underlying

The client to wrap

Attributes

Companion
object
Supertypes
trait LLMClient
trait AutoCloseable
class Object
trait Matchable
class Any
object ReliableClient

Attributes

Companion
class
Supertypes
class Object
trait Matchable
class Any
Self type
object ReliableProviders

Convenience methods for wrapping provider clients with reliability features.

Convenience methods for wrapping provider clients with reliability features.

Provides easy-to-use factory methods for creating reliable versions of each LLM provider client (OpenAI, Anthropic, Gemini, Ollama, OpenRouter, Zai).

Example usage:

// Create a reliable OpenAI client with default settings
val client = ReliableProviders.openai(
 OpenAIConfig(
   apiKey = "sk-...",
   model = "gpt-4o"
 )
)

// Or with custom reliability config
val aggressiveClient = ReliableProviders.openai(
 OpenAIConfig(apiKey = "sk-...", model = "gpt-4o"),
 ReliabilityConfig.aggressive
)

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
sealed trait RetryPolicy

Retry policy for transient failures.

Retry policy for transient failures.

Defines how many times to retry and how long to wait between attempts.

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
object RetryPolicy

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type
Generated with