OpenAICompatibleProvider

org.llm4s.llmconnect.provider.OpenAICompatibleProvider

Registration for the generic openai-compatible provider: any endpoint speaking the OpenAI /chat/completions API, with no module and no code.

A named provider section with provider = "openai-compatible" needs a baseUrl and a model; apiKey is optional (local servers need none), and contextWindow, reserveCompletion and headers may be set. Several sections can use it side by side:

llm4s.providers {
 groq-main {
   provider = "openai-compatible"
   baseUrl  = "https://api.groq.com/openai/v1"
   model    = "llama-3.3-70b-versatile"
   apiKey   = ${?GROQ_API_KEY}
   contextWindow = 131072
 }
 local-vllm {
   provider = "openai-compatible"
   baseUrl  = "http://localhost:8000/v1"
   model    = "Qwen/Qwen2.5-7B-Instruct"
 }
}

The client is OpenAICompatibleClient with the standard dialect: no reasoning parameters and no provider-specific decoding. A provider that needs those gets its own OpenAICompatibleDialect and descriptor in this module.

Attributes

Graph
Supertypes
class Object
trait Matchable
class Any
Self type

Members list

Value members

Concrete methods

Constructs the client for a ProviderConfig this provider produced.

Constructs the client for a ProviderConfig this provider produced.

config is whatever the caller passed, so implementations must check the type rather than cast — ProviderDescriptor.expectConfig does that and produces the standard mismatch error.

Attributes

def buildConfig(providerName: String, section: NamedProviderConfig)(using ContextWindowResolver): Result[ProviderConfig]

Turns a validated config section into the runtime ProviderConfig for this provider.

Turns a validated config section into the runtime ProviderConfig for this provider.

The section has already been checked against configSpec, so required fields are present; a Left here means something configSpec cannot express (a malformed value, say).

Value parameters

providerName

the user's instance name (llm4s.providers.<name>), for error messages.

Attributes

Inherited methods

def aliases: Set[String]

Alternative spellings accepted in provider = "...", folded onto id.

Alternative spellings accepted in provider = "...", folded onto id.

For example Gemini accepts "google". Aliases are matched after the same trim/lowercase canonicalisation as id itself.

Attributes

Inherited from:
ProviderDescriptor

What this provider's client implements. Defaults to the full interface.

What this provider's client implements. Defaults to the full interface.

Attributes

Inherited from:
ProviderDescriptor

Concrete fields

Which config fields this provider requires, and its default endpoint.

Which config fields this provider requires, and its default endpoint.

Attributes

val id: ProviderId

Canonical id, e.g. ProviderId("openai"). Must be unique within a registry.

Canonical id, e.g. ProviderId("openai"). Must be unique within a registry.

Attributes

override val modelLister: Option[ProviderModelLister]

Live model discovery, when the provider's API offers it.

Live model discovery, when the provider's API offers it.

Attributes