org.llm4s.llmconnect

Mixin trait providing standard lifecycle management and metrics wrapping for LLMClient implementations.

Tracks whether the client has been closed via an AtomicBoolean and provides:

• validateNotClosed — a pre-check returning Left(ConfigurationError) once closed.
• close() — an idempotent close that delegates to releaseResources() exactly once.
• completeWithMetrics — combines lifecycle validation with metrics recording for Completion results, eliminating boilerplate in complete / streamComplete.

Concrete clients mix in this trait, supply providerName and modelName, and optionally override releaseResources() to free provider-specific resources (HTTP clients, SDK connections, etc.).

Appends provider exchanges as JSON Lines to a local file.

This is intended as the first practical debugging sink for development. It creates parent directories on demand and writes one redacted JSON object per line.

Core interface for interacting with Large Language Model providers.

Abstracts communication with various LLM APIs (OpenAI, Azure OpenAI, Anthropic, etc.), providing a unified interface for completion requests, streaming responses, and token management. Implementations handle provider-specific authentication, message formatting, and tool calling.

Stateless helper functions for retrying LLM completion and streaming calls.

Retries only on recoverable errors (e.g. rate limit, timeout). Fails immediately on non-recoverable errors.

'''Retry delay precedence''' (honors upstream backpressure):

• If the error provides a provider retry-delay hint (e.g. retryDelay on org.llm4s.error.RateLimitError, org.llm4s.error.ServiceError) and it is present and positive, that value is used so we do not retry before the server is ready.
• Otherwise we fall back to local exponential backoff (baseDelay * 2^attempt) to avoid tight retry loops.
• The chosen delay is always capped at 30 seconds so waits remain bounded.

Constructs an LLMClient from provider configuration.

Provider selection is determined entirely by the runtime type of the ProviderConfig supplied: an AnthropicConfig produces an Anthropic client, an OpenAIConfig produces an OpenAI or OpenRouter client (the latter when baseUrl contains "openrouter.ai"), and so on. Azure uses OpenAIClient internally — AzureConfig carries the deployment endpoint and API-version fields that OpenAI does not require.

Client-construction options for LLMConnect.

This lets library users opt into runtime behaviors such as metrics and provider exchange logging without folding those concerns into org.llm4s.llmconnect.config.ProviderConfig.

Minimal first-pass representation of a captured provider exchange.

This intentionally starts small, but now carries enough timing and correlation metadata to serve as a low-level debugging record that higher level tracing or tooling can reference later.

Runtime configuration for capturing raw LLM provider exchanges.

Exchange logging is disabled by default. Applications can opt in by supplying a ProviderExchangeSink through ProviderExchangeLogging.Enabled when constructing a client.

Sink for completed provider exchanges.

Library code emits completed exchanges here when exchange logging is enabled. Sinks are deliberately simple and synchronous in the first version; callers can always wrap them later if they want batching or async behavior.