org.llm4s.llmconnect

Core interface for interacting with Large Language Model providers.

Abstracts communication with various LLM APIs (OpenAI, Azure OpenAI, Anthropic, etc.), providing a unified interface for completion requests, streaming responses, and token management. Implementations handle provider-specific authentication, message formatting, and tool calling.

Stateless helper functions for retrying LLM completion and streaming calls.

Retries only on recoverable errors (e.g. rate limit, timeout). Fails immediately on non-recoverable errors.

'''Retry delay precedence''' (honors upstream backpressure):

• If the error provides a provider retry-delay hint (e.g. RateLimitError.retryDelay, ServiceError.retryDelay) and it is present and positive, that value is used so we do not retry before the server is ready.
• Otherwise we fall back to local exponential backoff (baseDelay * 2^attempt) to avoid tight retry loops.
• The chosen delay is always capped at 30 seconds so waits remain bounded.