llm4s-docs
API
Generated with
llm4s-docs/org.llm4s/org.llm4s.llmconnect/org.llm4s.llmconnect.spi

org.llm4s.llmconnect.spi

package org.llm4s.llmconnect.spi

Members list

Type members

Classlikes

trait Llm4sProviderModule

A unit of provider registration: everything one module contributes.

A unit of provider registration: everything one module contributes.

The discovered unit is a module rather than a single ProviderDescriptor so that one artifact can supply several related providers — llm4s-openai contributes OpenAI, Azure, OpenRouter and Requesty from a single entry.

Implementations must be a plain class with a public no-arg constructor, '''not''' a Scala object: java.util.ServiceLoader instantiates the named class, and an object exposes its instance as a MODULE$ field instead. Delegate to an object from the class if you want one:

class MyProviderModule extends Llm4sProviderModule:
 def chatProviders: Seq[ProviderDescriptor] = Seq(MyProvider)

Classpath discovery via META-INF/services arrives with PR 3 of #1131; until then a module is registered explicitly through ProviderRegistry.of or ProviderRegistry.withModule.

Attributes

Supertypes
class Object
trait Matchable
class Any
Known subtypes
final case class ProviderConfigSpec(requiresApiKey: Boolean, requiresBaseUrl: Boolean, requiresEndpoint: Boolean, defaultBaseUrl: Option[String], baseUrlExample: String, endpointDescription: String)

The shape of a provider's llm4s.providers.<name> section: which fields it requires, and what to tell the user when one is missing.

The shape of a provider's llm4s.providers.<name> section: which fields it requires, and what to tell the user when one is missing.

This replaces the per-provider NamedProviderValidator implementations that llm4s-core used to hold, one object per provider in a shared file. A provider now declares its own requirements, and a single generic validator turns that declaration into the same error messages as before.

== Why defaults live here rather than in HOCON == Chat-provider config is keyed by the user's ''instance'' name (llm4s.providers.my-openai.baseUrl), so a reference.conf fragment shipped by a provider module cannot express "the default baseUrl for anything whose provider = "openai"" — it does not know the instance name. defaultBaseUrl is therefore code, and is the single source for a provider's default endpoint.

Value parameters

baseUrlExample

example shown when a required baseUrl is missing.

defaultBaseUrl

base URL used when the section omits one.

endpointDescription

what this provider means by endpoint, shown when a required one is missing. Providers repurpose the field (Azure: deployment name; Vertex AI: GCP project id).

requiresApiKey

the section must carry a non-empty apiKey.

requiresBaseUrl

the section must carry a non-empty baseUrl; set this only when there is no defaultBaseUrl.

requiresEndpoint

the section must carry a non-empty endpoint.

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object ProviderConfigSpec

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
trait ProviderDescriptor

Everything llm4s needs to know about one LLM provider, supplied by the provider itself.

Everything llm4s needs to know about one LLM provider, supplied by the provider itself.

A descriptor is the whole extension point: implement it, register it (by listing it in an Llm4sProviderModule, or by handing it to ProviderRegistry.of), and the provider becomes reachable from configuration, validation, client construction and model discovery without editing anything in llm4s-core. Before this existed, adding a provider meant edits to roughly eight shared files — see #1131.

Attributes

Example 
object BedrockProvider extends ProviderDescriptor:
 val id         = ProviderId("bedrock")
 val configSpec = ProviderConfigSpec(requiresApiKey = true, requiresEndpoint = true)
 def buildConfig(providerName: String, section: NamedProviderConfig)(using
   ContextWindowResolver
 ): Result[ProviderConfig] = ...
 def buildClient(config: ProviderConfig, options: LlmClientOptions)(using
   ModelRegistryService
 ): Result[LLMClient] =
   ProviderDescriptor.expectConfig[BedrockConfig](id, config).flatMap(BedrockClient(_, options.metrics))
Companion
object
Supertypes
class Object
trait Matchable
class Any
Known subtypes
object AzureProvider
object ZaiProvider
Show all
object ProviderDescriptor

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type
final case class ProviderFeatures(streaming: Boolean, toolCalling: Boolean)

What a provider implementation can actually do, declared by the provider itself.

What a provider implementation can actually do, declared by the provider itself.

This is a '''static''' declaration made by a ProviderDescriptor, not a runtime discovery: it describes the client this build ships, so a caller can ask "will streaming work here?" without making a call and reading the error. Contrast org.llm4s.llmconnect.utils.ProviderCapabilities, which is a health-check snapshot attached to a live connection.

The flags default to true because a provider that omits them is claiming the full interface; a provider that cannot honour part of it must say so, and saying so is then visible to users rather than buried in a Left at call time. Cohere and Mistral currently declare streaming = false — see #925.

Value parameters

streaming

whether LLMClient.streamComplete is implemented.

toolCalling

whether the provider accepts tool/function definitions.

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object ProviderFeatures

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
final class ProviderRegistry

The set of providers this build can resolve.

The set of providers this build can resolve.

A registry is an immutable value, not a mutable global: withProvider and withModule return a new registry, and an application that wants a different set constructs one with ProviderRegistry.of and passes it in. Core code resolves a registry through a using clause, so the default is ProviderRegistry.default unless the caller supplies another.

Lookup is by ProviderId and never throws: an id nothing handles produces a Left naming the ids that are registered, which is the difference between a usable error and "provider openai not registered".

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
object ProviderRegistry

Attributes

Companion
class
Supertypes
class Object
trait Matchable
class Any
Self type
Generated with