org.llm4s.llmconnect.config

@Stable

Configuration for the Anthropic Claude API.

Prefer AnthropicConfig.fromValues, which validates the values and resolves contextWindow and reserveCompletion from the bundled model catalogue. The constructor is private: build one with the companion apply and adjust it with the with* setters. Java and Kotlin, which cannot see Scala default arguments, use AnthropicConfig.apply(apiKey, model) and the setters, so adding a field never breaks them.

Value parameters

apiKey: Anthropic API key; redacted in toString.
apiKeyProvider: asked for the key on every request instead of sending apiKey, so a rotated key is used without rebuilding the client (#2313): a section's apiKeyFile sets one. Never set together with workloadIdentity.
baseUrl: API base URL; defaults to AnthropicConfig.DEFAULT_BASE_URL.
contextWindow: Model's total token capacity (prompt + completion combined).
model: Model identifier, e.g. "claude-sonnet-4-5-latest".
reserveCompletion: Tokens held back from prompt history for the completion.
timeouts: how long a request and a stream may take: the section's timeouts block. An absent value keeps the client's own default (ProviderTimeouts)
webSearch: whether the model may search the web: the section's webSearch key. When set, every request carries Anthropic's server-side web_search tool, and the sources the answer cites arrive on Completion.citations. Off by default; each search is billed by Anthropic.
workloadIdentity: workload identity federation instead of apiKey, which is then empty. The SDK posts the identity token to <baseUrl>/v1/oauth/token and the access token it gets back to every request, so baseUrl must be https://api.anthropic.com (plain http only to a loopback host, for tests). AnthropicConfig.fromValues checks this, and AnthropicClient refuses a config built any other way that breaks it.

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

trait ProviderConfig

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: AnthropicConfig.type

@Stable

Anthropic workload identity federation: the Anthropic SDK reads the identity token (e.g. a SPIFFE JWT-SVID kept fresh by spiffe-helper) from identityTokenFile on every exchange and presents it to <baseUrl>/v1/oauth/token under federationRuleId. The ids are redacted in toString.

The SDK reads the file itself, so llm4s's own checks on an identity token file (a regular file of at most 64 KiB, read with a bounded read) do not guard its reads. AnthropicClient applies them when the client is built (AnthropicWorkloadIdentity.checkTokenFile): a path that is a directory, a FIFO, a device or a file larger than a token is refused then. A path that does not exist yet is accepted, since an identity agent such as spiffe-helper may write it after the application starts; and the SDK's later reads are unchecked, so point it at a file the agent maintains.

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: AnthropicWorkloadIdentity.type

@Stable

Configuration for Azure OpenAI deployments.

Although Azure exposes an OpenAI-compatible API, it uses a different URL structure (per-deployment endpoint) and requires an apiVersion query parameter. org.llm4s.llmconnect.LLMConnect constructs an org.llm4s.llmconnect.provider.OpenAIClient internally; this config carries the Azure-specific fields that OpenAIConfig does not have.

Prefer AzureConfig.fromValues, which validates the values and resolves contextWindow and reserveCompletion from the bundled model catalogue. The constructor is private: build one with the companion apply and adjust it with the with* setters. Java and Kotlin, which cannot see Scala default arguments, use AzureConfig.apply(endpoint, apiKey, model) and the setters, so adding a field never breaks them.

Value parameters

apiKey: Azure API key; redacted in toString.
apiKeyProvider: asked for the key on every request instead of sending apiKey, so a rotated key is used without rebuilding the client (#2477): a section's apiKeyFile sets one. Never set together with tokenProvider.
apiVersion: Azure OpenAI API version string, e.g. "2025-01-01-preview".
contextWindow: Model's total token capacity (prompt + completion combined).
endpoint: Azure OpenAI deployment endpoint URL, e.g. "https://my-resource.openai.azure.com/openai/deployments/my-deploy".
model: Deployment name used as the model identifier.
reserveCompletion: Tokens held back from prompt history for the completion.
timeouts: how long a request and a stream may take: the section's timeouts block. An absent value keeps the client's own default (ProviderTimeouts)
tokenProvider: keyless Microsoft Entra ID authentication (#2274): when set, each request carries Authorization: Bearer <token> from it instead of the api-key header, and apiKey is unused (empty). A section with authentication = "entra-id" gets one from llm4s-azure-identity; in code, build one with that module's AzureEntraId, or implement org.llm4s.llmconnect.auth.AccessTokenProvider directly. Use AzureConfig.keyless or withTokenProvider.

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

trait ProviderConfig

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: AzureConfig.type

Configuration for the AWS Bedrock Converse API.

Authentication, in order of precedence: explicit credentials; the shared-config profile; otherwise the AWS default credential chain (environment variables, ~/.aws/credentials, EC2 instance profile, ECS task role, ...).

Build one with BedrockConfig.fromValues, which validates it and resolves contextWindow and reserveCompletion from the model registry.

Value parameters

contextWindow: Total token capacity of the model (prompt plus completion).
credentials: Explicit credentials; the default chain is used when absent.
endpointUrl: Endpoint override, e.g. a VPC endpoint or a test server.
model: Bedrock model id or inference-profile id, e.g. "anthropic.claude-3-5-sonnet-20241022-v2:0" or "us.anthropic.claude-3-5-sonnet-20241022-v2:0".
profile: AWS shared-config profile name.
region: AWS region that hosts the model, e.g. "us-east-1".
reserveCompletion: Tokens held back from prompt history for the completion.
timeouts: how long a Converse call and a ConverseStream call may take: the section's timeouts block. An absent value keeps the client's default: the AWS SDK's own for a request, and no limit for a stream (ProviderTimeouts).

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

trait ProviderConfig

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: BedrockConfig.type

Explicit AWS credentials for BedrockConfig.

Temporary credentials (STS, SSO, an assumed role) need the sessionToken as well; without it the signature is rejected.

Attributes

Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

@Experimental

Configuration for the Cohere API.

Prefer CohereConfig.fromValues over the companion apply; it resolves contextWindow and reserveCompletion automatically from the model name, and normalises the base URL to Cohere's OpenAI-compatibility endpoint.

This was in llm4s-core until Cohere moved onto llm4s-openai-compatible's shared client (#1132); its package is unchanged. The client now speaks Cohere's Compatibility API (/compatibility/v1/chat/completions) rather than the native v2 /v2/chat, which is what gives it streaming, tool calling and structured output (#925).

The constructor is private: build one with the companion apply, whose defaults cover every field after model, and adjust it with the with* setters. Java and Kotlin, which cannot see Scala default arguments, use CohereConfig.apply(apiKey, model) and the setters, so adding a field never breaks them (#2264).

Value parameters

apiKey: Cohere API key; redacted in toString.
apiKeyProvider: asked for the key on every request instead of sending apiKey, so a rotated key is used without rebuilding the client (#2477): a section's apiKeyFile sets one
baseUrl: The Compatibility API base; defaults to CohereConfig.DEFAULT_BASE_URL. Requests go to <baseUrl>/chat/completions. A native API root such as https://api.cohere.com is accepted and mapped - see CohereConfig.compatibilityBaseUrl.
contextWindow: Model's total token capacity (prompt + completion combined).
model: Model identifier, e.g. "command-a-03-2025".
reserveCompletion: Tokens held back from prompt history for the completion.
timeouts: how long a request and a stream may take: the section's timeouts block. An absent value keeps the client's own default (ProviderTimeouts)

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

trait ProviderConfig

class Object

trait Matchable

class Any
Show all

@Experimental

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: CohereConfig.type

@Stable

Resolves context window and output-reserve token counts for a given model using the model registry.

Attributes

Companion: object
Supertypes: class Object

trait Matchable

class Any

Attributes

Companion: class
Supertypes: class Object

trait Matchable

class Any
Self type: ContextWindowResolver.type

@Stable

Configuration for the DeepSeek API.

Prefer DeepSeekConfig.fromValues over the companion apply; it resolves contextWindow and reserveCompletion automatically, and logs a warning for unknown or legacy model names.

The constructor is private: build one with the companion apply, whose defaults cover every field after model, and adjust it with the with* setters. Java and Kotlin, which cannot see Scala default arguments, use DeepSeekConfig.apply(apiKey, model) and the setters, so adding a field never breaks them (#2264).

Value parameters

apiKey: DeepSeek API key; redacted in toString.
apiKeyProvider: asked for the key on every request instead of sending apiKey, so a rotated key is used without rebuilding the client (#2477): a section's apiKeyFile sets one
baseUrl: API base URL; defaults to DeepSeekConfig.DEFAULT_BASE_URL.
contextWindow: Model's total token capacity (prompt + completion combined).
model: Model identifier, e.g. "deepseek-chat" or "deepseek-reasoner".
reserveCompletion: Tokens held back from prompt history for the completion.
timeouts: how long a request and a stream may take: the section's timeouts block. An absent value keeps the client's own default (ProviderTimeouts)

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

trait ProviderConfig

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: DeepSeekConfig.type

@Stable

Configuration for a text embedding model, pairing a model identifier with its output vector size.

Used by embedding providers and the model dimension registry to resolve the expected dimensionality of embeddings produced by a given model.

Value parameters

dimensions: Number of dimensions in the embedding vectors produced by this model. Build one with EmbeddingModelConfig.apply and change it with the with* setters; the constructor and copy are private, so a new field never breaks callers.
name: Model identifier (e.g. "text-embedding-3-small", "voyage-3-large").

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: EmbeddingModelConfig.type

@Stable

Configuration required to connect to an embedding provider endpoint.

Value parameters

apiKey: authentication key for the provider; redacted in toString
apiKeyProvider: supplies the key per request in place of apiKey, so a rotated key is used without rebuilding the provider (#2478): a block's apiKeyFile (or the vendor's shared llm4s.credentials.<id>.apiKeyFile) sets org.llm4s.llmconnect.auth.ApiKeyProvider.file, and code sets any implementation with withApiKeyProvider. apiKey then holds the key read at load. An embedding provider reads the key with currentApiKey. Build one with EmbeddingProviderConfig.apply and change it with the with* setters; the constructor and copy are private, so a new field never breaks callers.
baseUrl: base URL of the embedding service (e.g. https://api.openai.com/v1)
model: name of the embedding model to use
network: the proxy and TLS settings its HTTP calls use: llm4s.http. The default, org.llm4s.http.HttpNetworkSettings.default, is no proxy and the JDK's trust.
timeouts: how long an embedding request may take: the section's timeouts.request. Absent means the provider's own default (two minutes). timeouts.stream has no meaning for an embedding call.

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: EmbeddingProviderConfig.type

@Stable

Configuration for the Google Gemini API.

Prefer GeminiConfig.fromValues, which validates the values, normalises baseUrl and resolves contextWindow and reserveCompletion from the bundled model catalogue. The constructor is private: build one with the companion apply and adjust it with the with* setters. Java and Kotlin, which cannot see Scala default arguments, use GeminiConfig.apply(apiKey, model) and the setters, so adding a field never breaks them.

Value parameters

apiKey: Google API key; redacted in toString.
apiKeyProvider: asked for the key on every request instead of sending apiKey, so a rotated key is used without rebuilding the client (#2477): a section's apiKeyFile sets one
baseUrl: API base URL; defaults to GeminiConfig.DEFAULT_BASE_URL.
contextWindow: Model's total token capacity (prompt + completion combined).
googleSearch: whether to ground answers in Google Search: the section's googleSearch key. When set, every request carries Gemini's google_search tool, the model may search before answering, and the sources it used arrive on Completion.citations. Off by default; billed by Google.
model: Model identifier, e.g. "gemini-2.0-flash".
reserveCompletion: Tokens held back from prompt history for the completion.
timeouts: how long a request and a stream may take: the section's timeouts block. An absent value keeps the client's own default (ProviderTimeouts)

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

trait ProviderConfig

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: GeminiConfig.type

Connection and metadata settings for the Langfuse tracing backend.

Read from the llm4s.tracing.langfuse block, which llm4s-observability's reference.conf binds to LANGFUSE_URL, LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_ENV, LANGFUSE_RELEASE and LANGFUSE_VERSION. With TRACING_MODE=langfuse that block reaches the backend as TracingSettings.extras (LangfuseConfig.fromExtras); to read it whatever the mode, use org.llm4s.config.LangfuseConfigLoader.

This lived in llm4s-core until slice 6 (#1133), as TracingSettings.langfuse. The package is unchanged.

toString redacts both keys so that the config can be safely logged.

Value parameters

env: deployment environment tag attached to every trace; defaults to "production"
publicKey: Langfuse public API key; the backend refuses to start without it
release: application release identifier forwarded to Langfuse; defaults to "1.0.0"
secretKey: Langfuse secret API key; redacted in toString
url: Langfuse ingestion endpoint; defaults to LangfuseConfig.DEFAULT_URL
version: SDK/integration version forwarded to Langfuse; defaults to "1.0.0"

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: LangfuseConfig.type

@Stable

Configuration specifying local model names for non-text modality embedding.

Holds the model identifiers used by local encoders to produce embeddings for image, audio, and video content. These models run locally (e.g. via ONNX or stub implementations) rather than calling a remote API.

Value parameters

audioModel: Local model name for audio embeddings (e.g. "wav2vec2-base").
imageModel: Local model name for image embeddings (e.g. "openclip-vit-b32").
videoModel: Local model name for video embeddings (e.g. "timesformer-base"). Build one with LocalEmbeddingModels.apply and change it with the with* setters; the constructor and copy are private, so a new modality never breaks callers.

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: LocalEmbeddingModels.type

@Experimental

Configuration for the Mistral AI API.

Prefer MistralConfig.fromValues over the companion apply; it resolves contextWindow and reserveCompletion automatically from the model name.

This was in llm4s-core until Mistral moved onto llm4s-openai-compatible's shared client (#1132); its package is unchanged.

The constructor is private: build one with the companion apply, whose defaults cover every field after model, and adjust it with the with* setters. Java and Kotlin, which cannot see Scala default arguments, use MistralConfig.apply(apiKey, model) and the setters, so adding a field never breaks them (#2264).

Value parameters

apiKey: Mistral API key; redacted in toString.
apiKeyProvider: asked for the key on every request instead of sending apiKey, so a rotated key is used without rebuilding the client (#2477): a section's apiKeyFile sets one
baseUrl: API root; defaults to MistralConfig.DEFAULT_BASE_URL. Requests go to <baseUrl>/v1/chat/completions - see MistralConfig.apiBaseUrl.
contextWindow: Model's total token capacity (prompt + completion combined).
model: Model identifier, e.g. "mistral-large-latest".
reserveCompletion: Tokens held back from prompt history for the completion.
timeouts: how long a request and a stream may take: the section's timeouts block. An absent value keeps the client's own default (ProviderTimeouts)

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

trait ProviderConfig

class Object

trait Matchable

class Any
Show all

@Experimental

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: MistralConfig.type

@Stable

Lookup service for embedding model vector dimensions.

Answers from the embedding providers in the caller's org.llm4s.llmconnect.spi.ProviderRegistry: each org.llm4s.llmconnect.spi.EmbeddingProviderDescriptor declares the dimensions of the models it knows, so a provider module brings its own and nothing here needs editing when one is added.

The one provider answered locally is local, the non-text encoders used by ModelSelector (llm4s-rag). It is not an embedding provider - nothing can be configured with it - so it has no descriptor to declare them. That table is only a fallback: an application that registers a real embedding provider under the id local gets that provider's dimensions, exactly as EmbeddingClient.from gets that provider's client.

Attributes

Supertypes: class Object

trait Matchable

class Any
Self type: ModelDimensionRegistry.type

@Stable

Configuration for a locally-running Ollama instance.

Ollama requires no API key — authentication is handled at the network level by controlling access to the Ollama endpoint. Prefer OllamaConfig.fromValues, which validates the values and resolves contextWindow and reserveCompletion from the bundled model catalogue. The constructor is private: build one with the companion apply and adjust it with the with* setters. Java and Kotlin, which cannot see Scala default arguments, use OllamaConfig.apply(model, baseUrl) and the setters, so adding a field never breaks them.

Value parameters

baseUrl: Ollama server URL, e.g. "http://localhost:11434".
contextWindow: Model's total token capacity (prompt + completion combined).
model: Model identifier as registered in Ollama, e.g. "llama3".
reserveCompletion: Tokens held back from prompt history for the completion.
timeouts: how long a request and a stream may take: the section's timeouts block. An absent value keeps the client's own default (ProviderTimeouts)

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

trait ProviderConfig

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: OllamaConfig.type

@Stable

Configuration for the generic openai-compatible provider: any endpoint that speaks the OpenAI /chat/completions API, such as Groq, Together, Fireworks, a vLLM, LM Studio or llama.cpp server, or an internal gateway.

Nothing is known about the model up front, so the context window and completion reserve come from config, defaulting to the conservative OpenAICompatibleConfig.DEFAULT_CONTEXT_WINDOW and OpenAICompatibleConfig.DEFAULT_RESERVE_COMPLETION. Set them to the model's real limits so context compression neither truncates early nor overflows.

Prefer OpenAICompatibleConfig.fromValues, which validates the values. The constructor is private: build one with the companion apply, whose defaults cover every field after baseUrl, and adjust it with the with* setters. Java and Kotlin, which cannot see Scala default arguments, use OpenAICompatibleConfig.apply(model, baseUrl) and the setters, so adding a field never breaks them.

Value parameters

apiKey: sent as Authorization: Bearer <key>; None sends no Authorization header, for servers that need none. Redacted in toString.
apiKeyProvider: asked for the bearer key on every request instead of sending apiKey, so a rotated key is used without rebuilding the client (#2313): a section's apiKeyFile sets one. Never set together with tokenExchange or an Authorization entry in headers, which would replace it.
baseUrl: API base URL; requests go to <baseUrl>/chat/completions.
contextWindow: the model's total token capacity (prompt + completion).
headers: extra headers sent on every request, e.g. a gateway's own auth header. Values are redacted in toString.
model: model identifier sent in every request.
reserveCompletion: tokens held back from prompt history for the completion.
streamUsage: whether a streaming request asks for token usage with "stream_options": {"include_usage": true}. Servers that follow OpenAI (vLLM, Ollama's /v1) stream no usage without it; turn it off for an endpoint that rejects the field. A named section sets it with the streamUsage key.
timeouts: how long a request and a stream may take: the section's timeouts block. An absent value keeps the client's own default (ProviderTimeouts)
tokenExchange: workload-identity auth: the identity token is exchanged here for the bearer token, which replaces apiKey; never set together with apiKey or an Authorization entry in headers, and its tokenUrl and baseUrl must be https (plain http only to a loopback host). The exchanged token goes to baseUrl, a host the user chooses, so no host is allow-listed here. OpenAICompatibleConfig.fromValues checks these, and OpenAICompatibleClient refuses a config built any other way that breaks them.

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

trait ProviderConfig

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: OpenAICompatibleConfig.type

@Stable

Configuration for the OpenAI API and providers that implement the OpenAI-compatible REST interface.

baseUrl governs which backend is contacted: "https://api.openai.com/v1" reaches OpenAI directly, while a URL containing "openrouter.ai" causes org.llm4s.llmconnect.LLMConnect to route to OpenRouter. Azure OpenAI uses AzureConfig, not this class.

This config lives in llm4s-openai-compatible, with OpenRouter, and llm4s-openai - whose OpenAI and Requesty providers build it - depends on that module for it (#1132). Its package is unchanged from when it was in llm4s-core.

Prefer OpenAIConfig.fromValues, which validates the values and resolves contextWindow and reserveCompletion from the bundled model catalogue. The constructor is private: build one with the companion apply and adjust it with the with* setters. Java and Kotlin, which cannot see Scala default arguments, use OpenAIConfig.apply(apiKey, model) and the setters, so adding a field never breaks them.

Value parameters

apiKey: OpenAI API key; redacted in toString.
apiKeyProvider: asked for the key on every request instead of sending apiKey, so a rotated key is used without rebuilding the client (#2313): a section's apiKeyFile sets one. Never set together with workloadIdentity.
baseUrl: API base URL; determines provider routing in org.llm4s.llmconnect.LLMConnect.
contextWindow: Model's total token capacity (prompt + completion combined).
explicitProviderId: the provider this config belongs to, when a descriptor says so: requesty for Requesty and openrouter for OpenRouter. None - what a config built by hand gets - infers it from baseUrl; see providerId.
model: Model identifier, e.g. "gpt-4o".
organization: Optional OpenAI organisation ID.
reserveCompletion: Tokens held back from prompt history for the completion.
timeouts: how long a request and a stream may take: the section's timeouts block. An absent value keeps the client's own default (ProviderTimeouts)

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

trait ProviderConfig

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: OpenAIConfig.type

@Stable

OpenAI workload identity federation: the identity token (e.g. a SPIFFE JWT-SVID) the OpenAI SDK exchanges at OpenAI's token endpoint for the service account's access token. The ids are redacted in toString.

Value parameters

clientId: sent with the exchange when the identity provider requires one
identityProviderId: the OpenAI workload identity provider (OPENAI_IDENTITY_PROVIDER_ID)
identityToken: where the identity token comes from; read afresh on each exchange
serviceAccountId: the OpenAI service account to act as (OPENAI_SERVICE_ACCOUNT_ID)

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: OpenAIWorkloadIdentity.type

Connection settings for an OpenTelemetry collector.

Spans are exported via OTLP/gRPC to endpoint. Read from the llm4s.tracing.opentelemetry block, which this module's reference.conf binds to OTEL_SERVICE_NAME and OTEL_EXPORTER_OTLP_ENDPOINT; with TRACING_MODE=opentelemetry (or otel) that block reaches the backend as TracingSettings.extras (OpenTelemetryConfig.fromExtras).

This lived in llm4s-core until slice 6 (#1133), as TracingSettings.openTelemetry. The package is unchanged.

Value parameters

captureContent: record message content - prompts, completions, an agent's query, tool input and output - on spans, redacted, as the GenAI conventions' gen_ai.input.messages / gen_ai.output.messages; false by default, as the conventions recommend, since content often holds personal or confidential data (#2275)
endpoint: OTLP/gRPC collector address; defaults to "http://localhost:4317"
headers: additional HTTP headers sent with each OTLP export request (e.g. authentication tokens for hosted collectors)
serviceName: logical service name attached to every span as service.name; defaults to "llm4s-agent"

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: OpenTelemetryConfig.type

@Experimental

Configuration for Perplexity's Agent API (POST /v1/agent).

Build one with PerplexityConfig.fromValues; a provider = "perplexity" section builds one for you.

Value parameters

apiKey: the Perplexity API key, sent as a bearer token; redacted in toString.
apiKeyProvider: asked for the key on every request instead of apiKey: a section's apiKeyFile sets one.
baseUrl: the API host, https://api.perplexity.ai by default; requests go to <baseUrl>/v1/agent (or <baseUrl>/agent when the base already ends in /v1).
contextWindow: the model's context window in tokens.
headers: extra HTTP headers sent with every request: the section's headers.
model: the Agent API model, in its provider/model form, e.g. "perplexity/sonar" or "openai/gpt-5.6-sol".
network: proxy and TLS settings.
reserveCompletion: tokens held back from the prompt for the reply; also the max_output_tokens sent to an anthropic/ model when the request sets none, since the Agent API requires one for them.
timeouts: how long a request and a stream may take: the section's timeouts block.
webSearch: whether each request offers the model Perplexity's web_search tool. Each search the model runs is billed per invocation.

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

trait ProviderConfig

class Object

trait Matchable

class Any
Show all

@Experimental

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: PerplexityConfig.type

@Stable

Identifies a specific LLM provider, model, and connection details.

Each subtype carries the credentials, endpoint URL, and context-window metadata needed to construct an org.llm4s.llmconnect.LLMClient via org.llm4s.llmconnect.LLMConnect. Instances are normally obtained from org.llm4s.config.Llm4sConfig.defaultProvider or org.llm4s.config.Llm4sConfig.provider, which resolve configured named providers under llm4s.providers.

Prefer each subtype's fromValues factory over its primary constructor: fromValues resolves contextWindow and reserveCompletion automatically from the model name, so you only need to supply credentials and endpoint. It returns a Result: a blank credential or endpoint is a org.llm4s.error.ConfigurationError naming the provider and the field, never a thrown exception.

This trait is deliberately '''not''' sealed. In Scala 3 sealed confines subtypes to the same source file, which would keep every provider's config in this one file forever and make a provider supplied by another module impossible. Implementations are therefore expected from outside llm4s-core, and consumers must not assume the set of subtypes is closed: describe a config through providerId, endpointUrl and withModel rather than by pattern-matching on its runtime type.

Attributes

Companion: object
Supertypes: class Object

trait Matchable

class Any
Known subtypes: class AnthropicConfig

class AzureConfig

class BedrockConfig

class CohereConfig

class DeepSeekConfig

class GeminiConfig

class MistralConfig

class OllamaConfig

class OpenAICompatibleConfig

class OpenAIConfig

class PerplexityConfig

class VertexAIConfig

class WatsonXConfig

class XaiConfig

class ZaiConfig
Show all

Attributes

Companion: trait
Supertypes: class Object

trait Matchable

class Any
Self type: ProviderConfig.type

@Stable

How long a provider's HTTP calls may take, as configured in the timeouts block of a named provider section:

llm4s.providers.my-openai {
 provider = "openai"
 model    = "gpt-4o"
 timeouts { request = 3m, stream = 15m }
}

Each value is optional. An absent value means ''this provider's own default'', which differs by client (for example 2 minutes for a Gemini or Ollama request and 10 minutes for its stream), so a client reads a timeout with requestOr or streamOr and supplies its own default. A section without a timeouts block behaves exactly as it did before the block existed.

request bounds a call that returns one response: a completion, an embedding.
stream bounds a streamed completion, from the request to the end of the stream. It is separate because a long generation legitimately outlives a request that returns at once.

A value must be positive and finite: java.net.http.HttpRequest.timeout rejects anything else, and Duration.Inf is not a FiniteDuration. Loading a section checks it (validated); a value set with withRequest or withStream on a hand-built config is not checked until a client uses it.

The constructor is private: build one with the companion apply or ProviderTimeouts.default and adjust it with the with* setters, so adding a field later never breaks a caller.

Value parameters

request: the timeout of a call that returns one response, or None for the client's default
stream: the timeout of a streamed completion, or None for the client's default

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: ProviderTimeouts.type

@Stable

Tracing configuration used by org.llm4s.trace.Tracing.

mode selects the backend. Core builds NoOp and Console itself; Console reads captureContent from extras (the llm4s.tracing.console block, false by default, so the console prints no prompt or completion text unless enabled). Every other mode is served by a org.llm4s.trace.spi.TracingBackend on the classpath, which reads its own keys from extras: the block llm4s.tracing.<mode> for the selected mode, flattened to strings. The backend's module ships that block's defaults and ${?VAR} bindings in its own reference.conf - the arrangement provider descriptors have with NamedProviderConfig.extras. So llm4s-observability reads llm4s.tracing.langfuse (LangfuseConfig.fromExtras) and llm4s-observability-otel reads llm4s.tracing.opentelemetry.

Until slice 6 (#1133) this class also carried typed langfuse and openTelemetry fields; they left core with their backends.

toString redacts the extras values, which may be credentials.

Value parameters

extras: every value under llm4s.tracing.<mode> for the selected mode, as a string, keyed by its path within that block (publicKey, headers.Authorization); a list or object value is rendered as HOCON. Empty when the block is absent. Build one with TracingSettings.apply and change it with the with* setters; the constructor and copy are private, so a new field never breaks callers.
mode: selects the tracing backend: Console, NoOp, or Named for a backend outside core (Named("langfuse"), Named("opentelemetry"), ...)

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: TracingSettings.type

@Stable

Configuration for Google Cloud Vertex AI.

Uses the Gemini-compatible REST format (aiplatform.googleapis.com) with OAuth2 bearer-token authentication via Application Default Credentials (ADC).

Prefer VertexAIConfig.fromValues, which validates the values and resolves contextWindow and reserveCompletion from the bundled model catalogue. The constructor is private: build one with the companion apply and adjust it with the with* setters. Java and Kotlin, which cannot see Scala default arguments, use VertexAIConfig.apply(projectId, model) and the setters, so adding a field never breaks them.

Value parameters

contextWindow: Maximum token capacity for prompt + completion combined.
credentialFilePath: Optional path to a Google JSON credential file. Falls back to GOOGLE_APPLICATION_CREDENTIALS env, then gcloud's application_default_credentials.json ($CLOUDSDK_CONFIG if set; else %APPDATA%\gcloud on Windows; else ~/.config/gcloud), then the GCE metadata server.
location: GCP region, e.g. "us-central1".
model: Model identifier, e.g. "gemini-2.0-flash".
projectId: GCP project ID that owns the Vertex AI resources.
reserveCompletion: Tokens reserved for the completion response.
timeouts: how long a request and a stream may take: the section's timeouts block. An absent value keeps the client's own default (ProviderTimeouts)
wellKnownCredentialPath: where gcloud's well-known application_default_credentials.json would be, resolved at the configuration edge by org.llm4s.config.VertexAIAdcConfigLoader (the descriptor sets it when it builds the client). None skips that lookup.

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

trait ProviderConfig

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: VertexAIConfig.type

Configuration for IBM watsonx.ai.

watsonx.ai hosts IBM Granite, Llama, Mistral and other models behind a governed API. Auth is IBM Cloud IAM: the apiKey is exchanged for a short-lived bearer token. Inference runs in a project (projectId) or a deployment space (spaceId); setting both is rejected. Prefer WatsonXConfig.fromValues over the primary constructor.

Value parameters

apiKey: IBM Cloud API key exchanged for IAM bearer tokens; redacted in toString.
apiKeyProvider: asked for the API key on every request instead of exchanging apiKey, so a rotated key is exchanged for a new IAM token without rebuilding the client (#2477): a section's apiKeyFile sets one.
apiVersion: watsonx API version date sent as the version query parameter, e.g. "2026-09-25".
baseUrl: watsonx.ai ML API base URL, e.g. "https://us-south.ml.cloud.ibm.com".
contextWindow: Model's total token capacity (prompt + completion combined).
iamUrl: IBM Cloud IAM token endpoint.
model: Model identifier, e.g. "ibm/granite-4-h-small".
projectId: watsonx project ID; may be empty only when spaceId is set.
reserveCompletion: Tokens held back from prompt history for the completion.
spaceId: Optional watsonx deployment space ID, used in place of the project (never together with it).
timeouts: how long a request and a stream may take: the section's timeouts block. An absent value keeps the client's own default (ProviderTimeouts).

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

trait ProviderConfig

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: WatsonXConfig.type

@Experimental

Configuration for the xAI (Grok) API.

Prefer XaiConfig.fromValues over the companion apply; it resolves contextWindow and reserveCompletion from the model registry (provider xai), falling back to xAI's published windows for models the registry does not list.

The constructor is private: build one with the companion apply, whose defaults cover every field after model, and adjust it with the with* setters. Java and Kotlin, which cannot see Scala default arguments, use XaiConfig.apply(apiKey, model) and the setters, so adding a field never breaks them (#2264).

Value parameters

apiKey: xAI API key; redacted in toString.
apiKeyProvider: asked for the key on every request instead of sending apiKey, so a rotated key is used without rebuilding the client (#2477): a section's apiKeyFile sets one
baseUrl: API base URL; defaults to XaiConfig.DEFAULT_BASE_URL. Requests go to <baseUrl>/chat/completions.
contextWindow: Model's total token capacity (prompt + completion combined).
model: Model identifier, e.g. "grok-4.7".
reserveCompletion: Tokens held back from prompt history for the completion.
timeouts: how long a request and a stream may take: the section's timeouts block. An absent value keeps the client's own default (ProviderTimeouts)

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

trait ProviderConfig

class Object

trait Matchable

class Any
Show all

@Experimental

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: XaiConfig.type

@Stable

Configuration for the Z.ai GLM API.

Prefer ZaiConfig.fromValues over the companion apply; it resolves contextWindow and reserveCompletion automatically from the model name.

The constructor is private: build one with the companion apply, whose defaults cover every field after model, and adjust it with the with* setters. Java and Kotlin, which cannot see Scala default arguments, use ZaiConfig.apply(apiKey, model) and the setters, so adding a field never breaks them (#2264).

Value parameters

apiKey: Z.ai API key; redacted in toString.
apiKeyProvider: asked for the key on every request instead of sending apiKey, so a rotated key is used without rebuilding the client (#2477): a section's apiKeyFile sets one
baseUrl: API base URL; defaults to ZaiConfig.DEFAULT_BASE_URL.
contextWindow: Model's total token capacity (prompt + completion combined).
model: Model identifier, e.g. "GLM-4.7".
reserveCompletion: Tokens held back from prompt history for the completion.
timeouts: how long a request and a stream may take: the section's timeouts block. An absent value keeps the client's own default (ProviderTimeouts)

Attributes

Companion: object
Supertypes: trait Serializable

trait Product

trait Equals

trait ProviderConfig

class Object

trait Matchable

class Any
Show all

Attributes

Companion: class
Supertypes: trait Product

trait Mirror

class Object

trait Matchable

class Any
Self type: ZaiConfig.type

org.llm4s.llmconnect.config

Members list

Type members

Classlikes

Value parameters

Attributes

Attributes

Attributes

Attributes

Value parameters

Attributes

Attributes

Value parameters

Attributes

Attributes

Attributes

Value parameters

Attributes

Attributes

Attributes

Attributes

Value parameters

Attributes

Attributes

Value parameters

Attributes

Attributes

Value parameters

Attributes

Attributes

Value parameters

Attributes

Attributes

Value parameters

Attributes

Attributes

Value parameters

Attributes

Attributes

Value parameters

Attributes

Attributes

Attributes

Value parameters

Attributes

Attributes

Value parameters

Attributes

Attributes

Value parameters

Attributes

Attributes

Value parameters

Attributes

Attributes

Value parameters

Attributes

Attributes

Value parameters

Attributes

Attributes

Attributes

Attributes

Value parameters

Attributes

Attributes

Value parameters

Attributes

Attributes

Value parameters

Attributes

Attributes

Value parameters

Attributes

Attributes

Value parameters

Attributes

Attributes

Value parameters

Attributes