llm4s-docs
API
Generated with
llm4s-docs/org.llm4s/org.llm4s.reliability

org.llm4s.reliability

package org.llm4s.reliability

Members list

Type members

Classlikes

@Stable
final case class CircuitBreakerConfig

Circuit breaker configuration for service resilience.

Circuit breaker configuration for service resilience.

Value parameters

failureThreshold

Number of consecutive failures before opening circuit

recoveryTimeout

Time to wait before attempting recovery (half-open state)

successThreshold

Number of successes in half-open state to close circuit

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object CircuitBreakerConfig

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
@Stable
sealed trait CircuitState

Circuit breaker state.

Circuit breaker state.

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
Known subtypes
object Closed
object HalfOpen
object Open
object CircuitState

Attributes

Companion
trait
Supertypes
trait Sum
trait Mirror
class Object
trait Matchable
class Any
Self type
@Stable
final case class RateLimitConfig

Local token-bucket rate limiting configuration.

Local token-bucket rate limiting configuration.

Value parameters

burstCapacity

Maximum tokens the bucket can hold, i.e. the largest burst allowed

enabled

Whether local rate limiting is applied

requestsPerMinute

Sustained request rate once the bucket is empty

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object RateLimitConfig

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
@Stable
final case class ReliabilityConfig

Configuration for reliable LLM provider calls.

Configuration for reliable LLM provider calls.

Provides retry logic, circuit breakers, and deadline enforcement to make LLM API calls resilient to transient failures.

Value parameters

circuitBreaker

Circuit breaker configuration

deadline

Maximum time to wait for operation completion

enabled

Whether reliability features are enabled (for opt-out)

rateLimit

Local rate limiting configuration

retryPolicy

Retry policy for transient failures

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object ReliabilityConfig

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
@Stable
final class ReliableClient(underlying: LLMClient, providerName: String, config: ReliabilityConfig, collector: Option[MetricsCollector], clock: () => Instant, sleep: FiniteDuration => Unit) extends LLMClient

Wrapper that adds reliability features to any LLMClient.

Wrapper that adds reliability features to any LLMClient.

Provides:

  • Retry with configurable policies (exponential backoff, linear, fixed)
  • Circuit breaker to fail fast when service is down
  • Deadline enforcement to prevent hanging operations
  • Local token-bucket rate limiting (RateLimitConfig), checked before every attempt
  • Metrics tracking for retry attempts and circuit breaker state

Thread-safety: Uses AtomicInteger/AtomicReference for circuit breaker state management to ensure correct behavior under concurrent access.

Value parameters

clock

the current time, read for deadlines and circuit-breaker recovery; injectable for tests

collector

Optional metrics collector for observability

config

Reliability configuration

providerName

Explicit provider name for stable metrics labels

sleep

how the client waits between attempts; injectable so a test can record the delays chosen, or throw InterruptedException to simulate an interrupted wait

underlying

The client to wrap

Attributes

Supertypes
trait LLMClient
trait AutoCloseable
class Object
trait Matchable
class Any
@Stable
sealed trait RetryPolicy

Retry policy for transient failures.

Retry policy for transient failures.

Defines how many times to retry and how long to wait between attempts.

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
object RetryPolicy

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type
Generated with