llm4s-docs
API
Generated with
llm4s-docs/org.llm4s/org.llm4s.llmconnect

org.llm4s.llmconnect

package org.llm4s.llmconnect

Members list

Packages

package org.llm4s.llmconnect.auth
package org.llm4s.llmconnect.caching
package org.llm4s.llmconnect.config
package org.llm4s.llmconnect.model
package org.llm4s.llmconnect.provider
package org.llm4s.llmconnect.serialization
package org.llm4s.llmconnect.spi
package org.llm4s.llmconnect.streaming
package org.llm4s.llmconnect.utils

Type members

Classlikes

@Stable
trait BaseLifecycleLLMClient extends LLMClient, MetricsRecording

Mixin trait providing standard lifecycle management and metrics wrapping for LLMClient implementations.

Mixin trait providing standard lifecycle management and metrics wrapping for LLMClient implementations.

Tracks whether the client has been closed via an AtomicBoolean and provides:

  • validateNotClosed — a pre-check returning Left(ConfigurationError) once closed.
  • close() — an idempotent close that delegates to releaseResources() exactly once.
  • completeWithMetrics — combines lifecycle validation with metrics recording for Completion results, eliminating boilerplate in complete / streamComplete.

Concrete clients mix in this trait, supply providerName and modelName, and optionally override releaseResources() to free provider-specific resources (HTTP clients, SDK connections, etc.).

Attributes

Supertypes
trait LLMClient
trait AutoCloseable
class Object
trait Matchable
class Any
Show all
Known subtypes
class GeminiClient
class OllamaClient
class OpenAIClient
class CohereClient
class XaiClient
class ZaiClient
Show all
@Stable
class EmbeddingClient(provider: EmbeddingProvider, tracer: Option[Tracing], operation: String, val providerId: Option[ProviderId])(using service: ModelRegistryService)

Embeds text through an org.llm4s.llmconnect.provider.EmbeddingProvider. With a tracer, each call is traced: a completed call as one org.llm4s.trace.TraceEvent.EmbeddingUsageRecorded (with the call's provider and duration) and, when its model's pricing is known, a cost; a failed call as one org.llm4s.trace.TraceEvent.ErrorOccurred naming the embeddings operation (#2275).

Embeds text through an org.llm4s.llmconnect.provider.EmbeddingProvider. With a tracer, each call is traced: a completed call as one org.llm4s.trace.TraceEvent.EmbeddingUsageRecorded (with the call's provider and duration) and, when its model's pricing is known, a cost; a failed call as one org.llm4s.trace.TraceEvent.ErrorOccurred naming the embeddings operation (#2275).

Value parameters

providerId

the canonical id of the provider, which tracing reports; EmbeddingClient.from sets it

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
object EmbeddingClient

Attributes

Companion
class
Supertypes
class Object
trait Matchable
class Any
Self type
@Stable
final case class JsonlProviderExchangeSink extends ProviderExchangeSink

Appends provider exchanges as JSON Lines to a local file.

Appends provider exchanges as JSON Lines to a local file.

This is intended as the first practical debugging sink for development. It creates parent directories on demand and writes one redacted JSON object per line.

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object JsonlProviderExchangeSink

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
@Stable
trait LLMClient extends AutoCloseable

Core interface for interacting with Large Language Model providers.

Core interface for interacting with Large Language Model providers.

Abstracts communication with various LLM APIs (OpenAI, Azure OpenAI, Anthropic, etc.), providing a unified interface for completion requests, streaming responses, and token management. Implementations handle provider-specific authentication, message formatting, and tool calling.

Attributes

Companion
object
Supertypes
trait AutoCloseable
class Object
trait Matchable
class Any
Known subtypes
class GeminiClient
class OllamaClient
class OpenAIClient
class CohereClient
class XaiClient
class ZaiClient
Show all
object LLMClient

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type
LLMClient.type
@Stable
object LLMClientRetry

Stateless helper functions for retrying LLM completion and streaming calls.

Stateless helper functions for retrying LLM completion and streaming calls.

Retries the errors RetryPolicy.isRetryable retries (e.g. rate limit, timeout, a 5xx), by the same rule. Fails immediately on any other error, including a client-error response.

'''Retry delay precedence''' (honors upstream backpressure):

  • If the error provides a provider retry-delay hint (e.g. retryDelay on org.llm4s.error.RateLimitError, org.llm4s.error.ServiceError) and it is present and positive, that value is used so we do not retry before the server is ready.
  • Otherwise we fall back to local exponential backoff (baseDelay * 2^attempt) to avoid tight retry loops.
  • The chosen delay is always capped at 30 seconds so waits remain bounded.

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
@Stable
object LLMConnect

Constructs an LLMClient from provider configuration.

Constructs an LLMClient from provider configuration.

Which client is built is decided by the org.llm4s.llmconnect.spi.ProviderRegistry: the config names a provider through org.llm4s.llmconnect.config.ProviderConfig.providerId, and that provider's org.llm4s.llmconnect.spi.ProviderDescriptor builds its own client. LLMConnect itself knows no provider names, and llm4s-core ships no provider: every one - the project's own provider modules and a third party's alike - is reached the same way.

Pass a different registry with using to resolve against a custom set of providers; with none in scope, org.llm4s.llmconnect.spi.ProviderRegistry.default is used.

Attributes

See also

org.llm4s.config.Llm4sConfig.defaultProvider to load the configured default named provider

LLMClient for the conversation and completion API

Example 
for {
 registry <- Llm4sConfig.modelRegistryService()
 cfg      <- Llm4sConfig.defaultProvider()
 client   <- LLMConnect.getClient(cfg)(using registry)
} yield client
Supertypes
class Object
trait Matchable
class Any
Self type
LLMConnect.type
@Stable
final case class LlmClientOptions

Client-construction options for LLMConnect.

Client-construction options for LLMConnect.

This lets library users opt into runtime behaviors such as metrics, provider exchange logging and their own HTTP transport without folding those concerns into org.llm4s.llmconnect.config.ProviderConfig.

Build one with LlmClientOptions.apply (or start from LlmClientOptions.default) and change it with the with* setters; the constructor and copy are private, so a new option never breaks callers.

Value parameters

exchangeLogging

whether, and where, provider request/response exchanges are logged; disabled by default

httpClient

the HTTP client an SDK-free provider sends its requests through instead of building its own; None (the default) builds one from the section's proxy and TLS settings. The caller owns a supplied client: closing the LLM client does not close it, and the section's network settings do not apply to it. A provider whose ProviderFeatures.customHttpClient is false (the default; the SDK-based ones keep it) ignores it.

metrics

collector the client reports request metrics to; no-op by default

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object LlmClientOptions

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
@Stable
final case class ProviderExchange

Minimal first-pass representation of a captured provider exchange.

Minimal first-pass representation of a captured provider exchange.

This intentionally starts small, but now carries enough timing and correlation metadata to serve as a low-level debugging record that higher level tracing or tooling can reference later.

Build one with ProviderExchange.apply and change it with the with* setters (an Option field's setter takes the value or an Option); the constructor and copy are private, so a new field never breaks callers.

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object ProviderExchange

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
@Stable
enum ProviderExchangeLogging

Runtime configuration for capturing raw LLM provider exchanges.

Runtime configuration for capturing raw LLM provider exchanges.

Exchange logging is disabled by default. Applications can opt in by supplying a ProviderExchangeSink through ProviderExchangeLogging.Enabled when constructing a client.

Attributes

Companion
object
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object ProviderExchangeLogging

Attributes

Companion
enum
Supertypes
trait Sum
trait Mirror
class Object
trait Matchable
class Any
Self type
@Stable
enum ProviderExchangeOutcome

Attributes

Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
@Stable
trait ProviderExchangeSink

Sink for completed provider exchanges.

Sink for completed provider exchanges.

Library code emits completed exchanges here when exchange logging is enabled. Sinks are deliberately simple and synchronous in the first version; callers can always wrap them later if they want batching or async behavior.

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
Known subtypes
object ProviderExchangeSink

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type
Generated with