Call OpenAI, Anthropic, Gemini or Ollama from Scala

With LLM4S, your Scala code calls every provider through one type, LLMClient: OpenAI, Anthropic, Google Gemini, Ollama and the others listed below. Which provider and model a client uses is configuration, so switching from one to another changes a line of application.conf and a dependency, not your code.

  1. Choose the provider in configuration
  2. Supported providers
  3. Several providers at once

Choose the provider in configuration

Each provider you use is a named section of your application.conf, and provider names the default:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

llm4s {
  providers {
    provider = "openai-main"

    openai-main {                   # key from OPENAI_API_KEY
      provider = "openai"
      model    = "gpt-4o-mini"
    }

    claude {                        # key from ANTHROPIC_API_KEY
      provider = "anthropic"
      model    = "claude-sonnet-4-20250514"
    }

    ollama-local {                  # no key: a model on your machine
      provider = "ollama"
      model    = "llama3.2"
      baseUrl  = "http://localhost:11434"
    }
  }
}

Each provider module reads its vendor’s usual environment variable for the API key, so a section needs only provider and model. Then one function builds a client for any section:

1
2
3
4
5
6
7
8
9
10
11

import org.llm4s.config.Llm4sConfig
import org.llm4s.llmconnect.{ LLMClient, LLMConnect }
import org.llm4s.model.ModelRegistryService
import org.llm4s.types.Result

def clientFor(section: String): Result[LLMClient] = for {
  providerConfig  <- Llm4sConfig.provider(section)
  registryService <- Llm4sConfig.modelRegistryService()
  given ModelRegistryService = registryService
  client <- LLMConnect.getClient(providerConfig)
} yield client

clientFor("claude") and clientFor("ollama-local") return the same type, and the rest of your program (client.complete(...), client.streamComplete(...), tools, agents, RAG) does not change. Llm4sConfig.defaultProvider() loads the section provider names. The configuration guide lists every key, and providers has each provider’s setup.

Each provider is its own module: add the ones you call, at the same version as llm4s-core.

1
2
3
4
5
6

libraryDependencies ++= Seq(
  "org.llm4s" %% "llm4s-core"      % "0.5.1",
  "org.llm4s" %% "llm4s-openai"    % "0.5.1",
  "org.llm4s" %% "llm4s-anthropic" % "0.5.1",
  "org.llm4s" %% "llm4s-ollama"    % "0.5.1"
)

Supported providers

This list is generated from the code: sbt generateCompatibilityMatrix reads the provider modules of the repository, and CI fails when the committed list is out of date, so a new provider module appears here with no edit to this page. The compatibility matrix has the full table (API keys, workload identity, the live test suites).

provider id Aliases Chat module Streaming Tool calling Embeddings module
anthropic   llm4s-anthropic yes yes -
azure   llm4s-openai yes yes -
bedrock   llm4s-bedrock yes yes -
cohere   llm4s-openai-compatible yes yes llm4s-cohere
deepseek   llm4s-openai-compatible yes yes -
gemini google llm4s-gemini yes yes -
jina   - - - llm4s-jina
mistral   llm4s-openai-compatible yes yes -
ollama   llm4s-ollama yes yes llm4s-ollama
openai   llm4s-openai yes yes llm4s-openai
openai-compatible   llm4s-openai-compatible yes yes -
openrouter   llm4s-openai-compatible yes yes -
perplexity   llm4s-perplexity yes yes -
requesty   llm4s-openai yes yes -
vertexai vertex llm4s-gemini yes yes -
voyage voyageai - - - llm4s-voyage
watsonx   llm4s-watsonx yes yes -
xai   llm4s-openai-compatible yes yes -
zai   llm4s-openai-compatible yes yes -

Any server that speaks OpenAI’s chat-completions API, such as vLLM, LM Studio or a gateway, also works through the openai-compatible id with configuration alone (recipes). A vendor that is missing can be added as your own module: see writing a provider.

Several providers at once

Because every client has the same type, using two providers is building two clients. The fallback recipe retries a provider with ReliableClient and asks a second provider when the first still fails.