Installation

Get LLM4S up and running in minutes.

Table of contents

  1. Prerequisites
    1. Verify Prerequisites
  2. Add LLM4S to Your Project
    1. SBT
    2. Maven
    3. Gradle (Kotlin DSL)
    4. Gradle (Groovy DSL)
    5. Scala CLI
    6. Mill
    7. Multi-Module Project
    8. Align versions with the BOM
    9. Unreleased changes (main)
  3. Quick Start with the Starter Kit
  4. Optional Dependencies
    1. For RAG, vector stores, chunking, reranking and document extraction
    2. For agent memory
    3. For MCP (Model Context Protocol)
    4. For speech (STT / TTS)
    5. For Ollama (local models)
    6. For Gemini and Vertex AI
    7. For Anthropic
    8. For OpenAI, Azure OpenAI and Requesty
    9. For Azure OpenAI with Microsoft Entra ID
    10. For DeepSeek, Z.ai, OpenRouter, Mistral, Cohere and any OpenAI-compatible endpoint
    11. For AWS Bedrock
    12. For Voyage AI embeddings
    13. For Jina AI embeddings
    14. For Cohere embeddings
    15. For the Jev decision model
    16. For IBM watsonx.ai
    17. For Perplexity
    18. For Java, Spring Boot, cats-effect and ZIO
    19. For Kotlin (coroutines and Flow)
    20. For image generation and vision
    21. For media types (MediaType, MediaCategory)
    22. For agents (Agent, guardrails, handoffs, graphs)
    23. For durable agent threads (SQLite checkpointer)
    24. For agent threads shared across machines (PostgreSQL checkpointer)
    25. For semantic memory in agents
    26. For built-in tools (web search, HTTP, filesystem, shell)
    27. For tapir endpoints as tools
    28. For tracing and metrics (Langfuse, OpenTelemetry, in-process trace store, Prometheus)
    29. For Neo4j (knowledge graph store)
    30. Structured output from a case class
    31. Typed prompts
    32. Capability-typed clients
    33. Testing your own code (the testkit)
    34. Checking your setup (the doctor)
    35. Trying llm4s in the REPL
    36. For provider authors (testing a provider module)
    37. For checkpoint store authors (testing a Checkpointer)
    38. For Workspace (Containerized Execution)
  5. Provider Setup
    1. Get API Keys
      1. OpenAI
      2. Anthropic
      3. Azure OpenAI
      4. Ollama (Local)
  6. Verify Installation
  7. Troubleshooting
    1. “missing required fields: apiKey”
    2. “Configured provider ‘…’ was not found” or an unknown provider id
    3. Compilation Errors
    4. Dependency Resolution Issues
  8. Next Steps
  9. Additional Resources

Prerequisites

Before installing LLM4S, ensure you have:

  • Java Development Kit (JDK) 21
  • Scala 3.7.x (llm4s 0.5.1 is built with 3.7.1)
  • sbt 1.10.6 or higher
  • An API key from at least one LLM provider (OpenAI, Anthropic, Azure OpenAI, …), or a local Ollama server, which needs none

Verify Prerequisites 

1
2
3
4
5
6
7
8

# Check Java version
java -version  # Should show 21

# Check Scala version
scala -version  # 3.7.1

# Check SBT version
sbt version  # 1.10.6 or higher

Add LLM4S to Your Project

Artifact coordinates changed in 0.4.0. Every published module now carries an llm4s- prefix (core → llm4s-core, workspaceClient → llm4s-workspace-client, and so on). Releases up to and including 0.3.4 remain available under the old names. See the migration guide for the full old → new table.

SBT

Add LLM4S to your build.sbt:

1
2
3
4
5
6

// Scala 3
ThisBuild / scalaVersion := "3.7.1"
libraryDependencies ++= Seq(
  "org.llm4s" %% "llm4s-core"   % "0.5.1",
  "org.llm4s" %% "llm4s-openai" % "0.5.1"  // or the provider module you use
)

llm4s-core holds no provider client: add the module of each provider you call (llm4s-openai, llm4s-anthropic, llm4s-gemini, llm4s-ollama, …; see Optional Dependencies). Without one, loading a provider section fails because no module registers that provider. The Maven and Gradle snippets below show llm4s-core only; add the provider module the same way, with the _3 suffix (llm4s-openai_3).

Maven 

1
2
3
4
5
6
7

<!-- For Scala 3 -->
<dependency>
    <groupId>org.llm4s</groupId>
    <artifactId>llm4s-core_3</artifactId>
    <version>0.5.1</version>
</dependency>

For Java, the Maven quick start has a complete pom.xml for llm4s-java-api_3.

Gradle (Kotlin DSL)

Gradle does not resolve Scala cross-version suffixes automatically — you must append the _3 suffix explicitly (LLM4S is Scala 3 only):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

// build.gradle.kts
repositories { mavenCentral() }

dependencies {
    implementation("org.llm4s:llm4s-core_3:0.5.1")
}

// Pin the Scala 3 library (scala3-library_3 only; scala-library stays at 2.13.x)
configurations.all {
    resolutionStrategy.eachDependency {
        if (requested.group == "org.scala-lang" && requested.name == "scala3-library_3") {
            useVersion("3.7.1")
        }
    }
}

Gradle (Groovy DSL) 

1
2
3
4
5
6
7
8
9
10
11
12
13
14

// build.gradle
repositories { mavenCentral() }

dependencies {
    implementation 'org.llm4s:llm4s-core_3:0.5.1'
}

configurations.all {
    resolutionStrategy.eachDependency { details ->
        if (details.requested.group == 'org.scala-lang' && details.requested.name == 'scala3-library_3') {
            details.useVersion '3.7.1'
        }
    }
}

Using Spring Boot, Ktor, or encountering dependency conflicts? See the full Gradle integration guide for dependency exclusion recipes and the dependency conflicts reference.

Scala CLI

No build at all: declare the dependency in the file.

1
2
3
4

//> using scala 3.7.1
//> using jvm 21
//> using dep org.llm4s::llm4s-core:0.5.1
//> using dep org.llm4s::llm4s-openai:0.5.1

Add the provider module you use next to llm4s-core (llm4s-openai above). The Scala CLI quick start has a complete file that makes a first call.

Mill

In build.mill (Mill 1.x), :: adds the Scala suffix as %% does in sbt:

1
2
3
4
5
6
7
8
9

val llm4sVersion = "0.5.1"

object app extends ScalaModule {
  def scalaVersion = "3.7.1"
  def mvnDeps = Seq(
    mvn"org.llm4s::llm4s-core:$llm4sVersion",
    mvn"org.llm4s::llm4s-openai:$llm4sVersion"
  )
}

The Mill quick start has a complete project: a first call, its configuration, and a test that needs no API key.

Multi-Module Project

If you have a multi-module project:

1
2
3
4
5
6
7
8
9

lazy val myProject = (project in file("."))
  .settings(
    name := "my-llm-project",
    scalaVersion := "3.7.1",
    libraryDependencies ++= Seq(
      "org.llm4s" %% "llm4s-core"   % "0.5.1",
      "org.llm4s" %% "llm4s-openai" % "0.5.1"
    )
  )

Align versions with the BOM

A project that uses several llm4s modules (a provider, llm4s-rag, llm4s-observability) can state the version once, in the bill of materials llm4s-bom, and leave it off every module. The BOM contains no code; it supplies the versions released together. Explicit overrides and dependency conflict resolution can still select different versions, so check the resolved dependency graph.

Maven and Gradle have no notion of Scala cross-versioning, so the modules are written with the Scala binary suffix (llm4s-core_3), and the BOM itself has none (llm4s-bom).

Maven (import scope):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.llm4s</groupId>
      <artifactId>llm4s-bom</artifactId>
      <version>0.5.1</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependencies>
  <dependency>
    <groupId>org.llm4s</groupId>
    <artifactId>llm4s-core_3</artifactId>
  </dependency>
  <dependency>
    <groupId>org.llm4s</groupId>
    <artifactId>llm4s-openai_3</artifactId>
  </dependency>
</dependencies>

Gradle (Kotlin DSL; in Groovy write implementation platform("org.llm4s:llm4s-bom:...")):

1
2
3
4
5

dependencies {
    implementation(platform("org.llm4s:llm4s-bom:0.5.1"))
    implementation("org.llm4s:llm4s-core_3")
    implementation("org.llm4s:llm4s-openai_3")
}

Gradle platform(...) treats the BOM versions as recommendations that other dependencies can upgrade. For strict version pinning in an application, use enforcedPlatform(...) instead. Publishing libraries should generally keep platform(...) to avoid forcing their versions on consumers. See the Gradle platform documentation.

Mill imports it with bomMvnDeps; the Mill quick start shows how.

sbt has no BOM import, so the BOM is optional there: keep one val for the version, as the sections below do (%% adds the _3 suffix itself):

1
2
3
4
5
6

val llm4sVersion = "0.5.1"

libraryDependencies ++= Seq(
  "org.llm4s" %% "llm4s-core"   % llm4sVersion,
  "org.llm4s" %% "llm4s-openai" % llm4sVersion
)

Unreleased changes (main)

No snapshots are published: releases are cut from tags only, and nothing publishes between them. To try what is on main, build it and publish it to your local repository:

1
2
3

git clone https://github.com/llm4s/llm4s && cd llm4s
sbt publishLocal   # for sbt; use `sbt publishM2` for Maven or Gradle (mavenLocal())
sbt version        # prints the version it published, for example 0.5.1+12-abc12345-SNAPSHOT

Then depend on that version, for example libraryDependencies += "org.llm4s" %% "llm4s-core" % "0.5.1+12-abc12345-SNAPSHOT" (use the one sbt version printed). This guide describes the latest release; main may have modules and configuration that it does not have yet.

Quick Start with the Starter Kit

The fastest way to a working sbt project is the llm4s.g8 template:

1
2
3
4
5

sbt new llm4s/llm4s.g8 --llm4s_version=0.5.1

# Answer the remaining prompts (name, package, scala_version, ...) or accept the defaults
cd <your project name>
sbt run

Pass --llm4s_version (or type the version at its prompt) so the project starts on 0.5.1, then check the generated build.sbt: it needs llm4s-core and at least one provider module at that version, because since 0.5.0 llm4s-core ships no provider. The template is being moved to 0.5.x (llm4s.g8#6), which adds llm4s-ollama and llm4s-openai itself; a template from before that defaults to llm4s 0.4.0 and lists llm4s-core only, so add the provider you use, for example "org.llm4s" %% "llm4s-openai" % "0.5.1".

The generated project has an sbt build, a logging backend, an application.conf with named provider sections, a small sample program and its tests.

View the starter kit →

Optional Dependencies

llm4s-core is the spine: types, configuration, the LLMClient interface and the tool API. Providers, the agent runtime and everything below are separate modules, released at the same version. Check Maven Central for available artifacts.

For RAG, vector stores, chunking, reranking and document extraction 

1
2

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-rag" % llm4sVersion

Brings llm4s-knowledgegraph with it, along with Tika, POI, PDFBox, jsoup and the AWS S3 client — the document-extraction and loader dependencies that llm4s-core no longer carries. Package names are unchanged, so existing org.llm4s.rag.* imports keep working; see the migration note.

To keep the vectors in an Elasticsearch or OpenSearch cluster you already run, add the Elasticsearch store module (Experimental), which holds both ElasticsearchVectorStore and OpenSearchVectorStore. It talks to the REST API through llm4s-core’s HTTP client, so it brings no client library; see Elasticsearch and OpenSearch.

1
2

// same version as llm4s-rag
libraryDependencies += "org.llm4s" %% "llm4s-vectorstore-elasticsearch" % llm4sVersion

To keep them in Redis (Redis Stack, or Redis 8 and later) or Valkey (with valkey-search), add the Redis store (Experimental). It brings the Jedis client; see Redis and Valkey.

1
2

// same version as llm4s-rag
libraryDependencies += "org.llm4s" %% "llm4s-vectorstore-redis" % llm4sVersion

For agent memory 

1
2
3
4
5

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-memory" % llm4sVersion

// only if you store memories in Postgres/pgvector
libraryDependencies += "org.llm4s" %% "llm4s-memory-postgres" % llm4sVersion

llm4s-memory carries MemoryStore, the memory managers, and the in-memory and SQLite-backed stores; it adds sqlite-jdbc to your classpath and nothing else. PostgresMemoryStore lives in llm4s-memory-postgres, which brings HikariCP and the Postgres JDBC driver — the two dependencies llm4s-core no longer carries. Package names are unchanged, so existing org.llm4s.agent.memory.* imports keep working; see the migration note.

For MCP (Model Context Protocol) 

1
2

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-mcp" % llm4sVersion

Carries the MCP client, server, transports (stdio, HTTP, SSE) and MCPToolRegistry. It adds no third-party dependency of its own. Package names are unchanged, so existing org.llm4s.mcp.* imports keep working; see the migration note.

For speech (STT / TTS) 

1
2

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-speech" % llm4sVersion

Carries speech-to-text (Vosk for offline recognition, Whisper), text-to-speech (Tacotron 2), and the audio IO, conversion and validation helpers. This is the module that brings Vosk, a 25 MB dependency that used to sit on every llm4s-core user’s classpath — which is much of the point of the split. Package names are unchanged; see the migration note.

For Ollama (local models) 

1
2

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-ollama" % llm4sVersion

Carries the Ollama chat client, the Ollama embedding provider and Ollama model listing. It adds no third-party dependency of its own. Adding it is all the registration there is: the module declares itself to the provider registry, so provider = "ollama" and EMBEDDING_MODEL=ollama/<model> resolve with no code change. Without it, both fail with an error saying ollama is not registered and naming the providers that are. Package names are unchanged; see the migration note.

For Gemini and Vertex AI 

1
2

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-gemini" % llm4sVersion

Carries both of Google’s chat providers: the Gemini API client and model listing, and the Vertex AI client with its OAuth authentication. It adds no third-party dependency of its own - Vertex AI authentication is implemented without a Google SDK. Adding it is all the registration there is: the module declares itself to the provider registry, so provider = "gemini" (or "google") and provider = "vertexai" (or "vertex") resolve with no code change. Without it, they fail with an error saying the provider is not registered and naming the providers that are. Package names are unchanged; see the migration note.

For Anthropic 

1
2

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-anthropic" % llm4sVersion

Carries the Anthropic Claude chat client and Anthropic model listing. This is the module that brings the Anthropic Java SDK (com.anthropic:anthropic-java), which used to sit on every llm4s-core user’s classpath. Adding it is all the registration there is: the module declares itself to the provider registry, so provider = "anthropic" resolves with no code change. Without it, it fails with an error saying the provider is not registered and naming the providers that are. Package names are unchanged; see the migration note.

For OpenAI, Azure OpenAI and Requesty 

1
2

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-openai" % llm4sVersion

Carries the three providers that share OpenAIClient - OpenAI, Azure OpenAI and Requesty - the OpenAI embedding provider, and OpenAI and Requesty model listing. This is the module that brings OpenAI’s Java SDK (com.openai:openai-java, with OkHttp, Jackson and the Kotlin standard library), which serves Azure OpenAI as well. It replaced Microsoft’s deprecated com.azure:azure-ai-openai, which used to sit on every llm4s-core user’s classpath; llm4s-core depends on no vendor SDK. Adding it is all the registration there is: the module declares itself to the provider registry, so provider = "openai", "azure" and "requesty", and EMBEDDING_MODEL=openai/<model>, resolve with no code change. Without it, they fail with an error saying the provider is not registered and naming the providers that are.

OpenRouter, DeepSeek and Z.ai speak the OpenAI wire format with no SDK, and are in llm4s-openai-compatible (below), which this module depends on for OpenAIConfig. llm4s-rag’s RAGConfig.default embeds with openai, so a RAG pipeline built from the default needs this module too, or .withEmbeddings(...) naming a provider you do ship. Package names are unchanged; see the migration note.

For Azure OpenAI with Microsoft Entra ID 

1
2

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-azure-identity" % llm4sVersion

Keyless authentication for Azure OpenAI (#2274): an azure section with authentication = "entra-id" takes Microsoft Entra ID tokens from Azure’s DefaultAzureCredential (a managed identity, workload identity or az login) instead of an API key, and AzureEntraId turns any TokenCredential into the tokens a config or embedding provider takes. It brings Azure’s azure-identity (MIT, with azure-core, MSAL4J, Netty and Reactor), which is why it is not part of llm4s-openai. See Enterprise authentication.

For DeepSeek, Z.ai, OpenRouter, Mistral, Cohere and any OpenAI-compatible endpoint 

1
2

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-openai-compatible" % llm4sVersion

One SDK-free client for the OpenAI /chat/completions API, and the providers that run on it: DeepSeek (provider = "deepseek"), Z.ai ("zai"), OpenRouter ("openrouter"), Mistral ("mistral"), Cohere ("cohere", through Cohere’s OpenAI-compatibility API), xAI ("xai"), and a generic "openai-compatible" provider for any other endpoint that speaks the API - Groq, Together, Fireworks, NVIDIA NIM, OrcaRouter, a vLLM, LM Studio or llama.cpp server, an internal gateway - configured with a baseUrl and a model, and no code. See OpenAI-compatible endpoints for a configuration recipe per provider. The module brings no dependency beyond llm4s-core, and adding it registers the providers. Package names are unchanged; see the migration note, and for Mistral and Cohere this one.

For AWS Bedrock 

1
2

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-bedrock" % llm4sVersion

The AWS Bedrock chat provider (provider = "bedrock"), over the Converse and ConverseStream APIs, so one client reaches Claude, Llama, Mistral, Nova and Titan models. It brings the AWS SDK for Java v2 bedrockruntime artifact (Apache-2.0, the same SDK release train llm4s-rag uses for S3); llm4s-core gains no dependency. A section needs a region and a model:

1
2
3
4
5

llm4s.providers.claude-bedrock {
  provider = "bedrock"
  model    = "anthropic.claude-3-5-sonnet-20241022-v2:0"
  region   = ${?AWS_REGION}
}

Credentials come from the AWS default credential chain (environment variables, ~/.aws, an instance or task role), or from a profile, or from explicit accessKeyId / secretAccessKey (plus sessionToken for temporary credentials).

For Voyage AI embeddings 

1
2

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-voyage" % llm4sVersion

The Voyage AI embedding provider (EMBEDDING_MODEL=voyage/<model>), with its llm4s.embeddings.voyage config block. It is the first community provider module under modules/providers/, and brings no dependency beyond llm4s-core.

For Jina AI embeddings 

1
2

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-jina" % llm4sVersion

The Jina AI embedding provider (EMBEDDING_MODEL=jina/<model>), with its llm4s.embeddings.jina config block and a typed JinaTask (retrieval.query, retrieval.passage, …). A community provider module under modules/providers/; it brings no dependency beyond llm4s-core.

For Cohere embeddings 

1
2

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-cohere" % llm4sVersion

The Cohere embedding provider (EMBEDDING_MODEL=cohere/<model>), on Cohere’s native /v2/embed, with its llm4s.embeddings.cohere config block and a typed CohereInputType (search_document, search_query, …). Cohere chat is a different module: llm4s-openai-compatible. A community provider module under modules/providers/; it brings no dependency beyond llm4s-core.

For the Jev decision model 

1
2

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-jev" % llm4sVersion

A typed client for TypeSafe’s Jev decision model: send a state and typed questions (Noul, Choice, Score), get typed answers. It is not a chat provider and returns no Completion. Configured under llm4s.jev, with the key in TYPESAFE_API_KEY. A community module under modules/providers/; it brings no dependency beyond llm4s-core. See Jev decision model.

For IBM watsonx.ai 

1
2

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-watsonx" % llm4sVersion

The IBM watsonx.ai chat provider (provider = "watsonx"), brings no dependency beyond llm4s-core. See IBM watsonx.ai.

For Perplexity 

1

libraryDependencies += "org.llm4s" %% "llm4s-perplexity" % llm4sVersion

The Perplexity Agent API chat provider (provider = "perplexity"): answers grounded in web search, with citations. It brings no dependency beyond llm4s-core. See Perplexity.

For Java, Spring Boot, cats-effect and ZIO 

1
2
3
4
5
6
7
8
9

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-java-api"             % llm4sVersion  // Java facade
libraryDependencies += "org.llm4s" %% "llm4s-java-rag"             % llm4sVersion  // Java facade for RAG (Experimental)
libraryDependencies += "org.llm4s" %% "llm4s-reactor"              % llm4sVersion  // Reactor Flux over the Java facade (Experimental)
libraryDependencies += "org.llm4s" %% "llm4s-spring-boot-starter"  % llm4sVersion  // Spring Boot auto-configuration
libraryDependencies += "org.llm4s" %% "llm4s-effect"               % llm4sVersion  // cats-effect 3 / fs2
libraryDependencies += "org.llm4s" %% "llm4s-zio"                  % llm4sVersion  // ZIO 2 / ZIO Streams
libraryDependencies += "org.llm4s" %% "llm4s-pekko"                % llm4sVersion  // Apache Pekko Streams
libraryDependencies += "org.llm4s" %% "llm4s-ox"                   % llm4sVersion  // Ox direct style (Experimental)

llm4s-java-api is the Java-friendly facade (a runnable Gradle sample is in modules/samples/gradle-java, and a Quarkus application with a CDI producer in modules/samples/quarkus-java, see Quarkus) and the base of the Spring starter, which adds the llm4s.* properties, an LLM4STemplate and an Actuator health indicator. llm4s-java-rag puts llm4s-rag’s pipeline behind one Java class, JRag (Java RAG); it is a separate artifact so that a Java facade user who only calls a model does not receive document parsing and the database drivers. llm4s-reactor turns a streamed reply into a Reactor Flux<String> (Java Reactor), apart from the facade so that it does not bring reactor-core to an application without Reactor. llm4s-effect, llm4s-zio and llm4s-pekko wrap LLMClient and Agent for their effect and streaming systems. llm4s-ox adds an Ox Flow of a streamed reply, bounded fan-out, races and retries to direct-style code (Direct style).

For Kotlin (coroutines and Flow)

llm4s-kotlin-api is the Kotlin coroutine API over the Java facade: LLMClientKt and AgentKt with suspend functions, and Flows of an agent turn’s events. It is a Kotlin artifact, so it has no _3 suffix, and it is released at the same version as every other llm4s module, so the BOM manages it too:

llm4s-kotlin-api 0.5.1 is not on Maven Central yet. A fault in the release workflow left it out of the 0.5.0 and 0.5.1 releases; the fix is pending (#1976). Until a release includes it, the coordinates below do not resolve (the BOM lists it, but every other entry of the BOM resolves). Build it locally instead, as described below the snippet; it then depends on the released llm4s-java-api_3.

1
2
3
4
5
6

// build.gradle.kts
dependencies {
    implementation(platform("org.llm4s:llm4s-bom:0.5.1"))
    implementation("org.llm4s:llm4s-kotlin-api")   // brings llm4s-java-api_3 and kotlinx-coroutines-core
    implementation("org.llm4s:llm4s-ollama_3")     // the provider module you call
}

Without the BOM, write the version on it: implementation("org.llm4s:llm4s-kotlin-api:0.5.1"). It runs on JDK 21, like the Java facade it wraps. The module is built by Gradle (modules/kotlin-api), not sbt, so sbt publishLocal does not build it. To build it, check out the release tag (git checkout v0.5.1) and run gradle publishToMavenLocal -Pllm4sVersion=0.5.1 in modules/kotlin-api, then add mavenLocal() to your repositories; it depends on llm4s-java-api_3 at that version, which is on Maven Central. Without -Pllm4sVersion it builds at 0.1.0-SNAPSHOT; to try it from main, run sbt publishM2 first and pass the version sbt published.

Its entry point is org.llm4s.kotlin.Llm4s: createDefaultClient() builds an LLMClientKt from application.conf, and wrapClient(JLlmClient) wraps one configured in code with the Java facade’s builders (see the Java guide’s Configure in code); createAgent and wrapAgent give an AgentKt. A Kotlin DSL declares tools (tools { tool(...) { ... } }), completion options (completionOptions { ... }) and clients (Llm4s.openAi { ... }); see the Java guide’s Kotlin DSL.

For image generation and vision 

1
2

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-image" % llm4sVersion

Carries image generation (org.llm4s.imagegeneration — Stable Diffusion, Stability AI, Hugging Face, OpenAI) and image processing (org.llm4s.imageprocessing — the OpenAI and Anthropic vision clients plus a local javax.imageio processor). It adds no third-party dependency of its own, and brings llm4s-media with it.

Package names are unchanged, so existing org.llm4s.imagegeneration.* and org.llm4s.imageprocessing.* imports keep working — but note that image formats did change, in llm4s-media. See the migration note.

For media types (MediaType, MediaCategory) 

1
2

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-media" % llm4sVersion

The shared vocabulary the multimodal modules speak: MediaType (MIME string, canonical extension, category, and lookups by extension, path or MIME type) and MediaCategory. It has no dependencies at all and does no I/O — deciding what a file is from its bytes needs Tika and lives in llm4s-rag.

You will usually get it transitively, from llm4s-core or llm4s-rag; declare it directly only if you name these types in your own signatures. It replaces three overlapping image-format types that used to ship in llm4s-core, which is a source break — see the migration note.

For agents (Agent, guardrails, handoffs, graphs) 

1

libraryDependencies += "org.llm4s" %% "llm4s-agent" % llm4sVersion // same version as llm4s-core

The agent runtime - org.llm4s.agent (Agent, AgentResult, guardrails, handoffs, the graph runtime) and the console assistant, org.llm4s.assistant - lives here. Agent memory is the separate llm4s-memory; the ready-made tools are llm4s-agent-tools. llm4s-core keeps what the agent is built on: LLMClient, the tool API and the tracing contract. Package names are unchanged. See the migration note.

For durable agent threads (SQLite checkpointer) 

1

libraryDependencies += "org.llm4s" %% "llm4s-agent-checkpoint-sqlite" % llm4sVersion // same version as llm4s-agent

SqliteCheckpointer (org.llm4s.agent.graph.sqlite) keeps graph and agent threads - checkpoints, pending writes, event logs and run claims - in one SQLite file, so a run survives a restart and recover continues it in a new process. It brings the SQLite JDBC driver (org.xerial:sqlite-jdbc), which llm4s-agent itself does not depend on. See Durable Checkpointers.

For agent threads shared across machines (PostgreSQL checkpointer) 

1

libraryDependencies += "org.llm4s" %% "llm4s-agent-checkpoint-postgres" % llm4sVersion // same version as llm4s-agent

PostgresCheckpointer (org.llm4s.agent.graph.postgres) keeps the same threads in a PostgreSQL schema that runtimes on several machines share, with claims and retention judged by the database’s clock. It brings the PostgreSQL JDBC driver and HikariCP. See Durable Checkpointers.

For semantic memory in agents 

1

libraryDependencies += "org.llm4s" %% "llm4s-agent-memory" % llm4sVersion // same version as llm4s-agent

SemanticMemoryMiddleware (org.llm4s.agent.harness.memory) recalls a tenant’s memories before each turn and records the turn after it, on an llm4s-memory store. It depends on llm4s-agent and llm4s-memory. See Semantic Memory Middleware.

For built-in tools (web search, HTTP, filesystem, shell) 

1

libraryDependencies += "org.llm4s" %% "llm4s-agent-tools" % llm4sVersion // same version as llm4s-core

BuiltinTools and everything under org.llm4s.toolapi.builtin - DateTime, Calculator, UUID, JSON, file access, HTTP, shell, and the Brave, DuckDuckGo and Exa search tools - live here, with ToolsConfigLoader and the llm4s.tools settings the search tools read. llm4s-core keeps the tool API they implement (ToolFunction, ToolRegistry). Package names are unchanged. See the migration note.

For tapir endpoints as tools 

1

libraryDependencies += "org.llm4s" %% "llm4s-tapir" % llm4sVersion // same version as llm4s-core

EndpointTools turns tapir endpoints into tools, each schema derived from its endpoint’s inputs, called through sttp client 3. It depends on tapir-core 1.11, not on the agent runtime (Experimental). See Tapir Endpoints as Tools.

For tracing and metrics (Langfuse, OpenTelemetry, in-process trace store, Prometheus) 

1
2
3
4
5
6
7

// same version as llm4s-core
libraryDependencies ++= Seq(
  "org.llm4s" %% "llm4s-observability"            % llm4sVersion, // TRACING_MODE=langfuse
  "org.llm4s" %% "llm4s-observability-otel"       % llm4sVersion, // TRACING_MODE=opentelemetry
  "org.llm4s" %% "llm4s-observability-prometheus" % llm4sVersion, // llm4s.metrics.enabled = true
  "org.llm4s" %% "llm4s-observability-micrometer" % llm4sVersion   // MicrometerMetrics over your MeterRegistry
)

llm4s-core builds only the console (default) and noop tracers. llm4s-observability registers Langfuse and carries the in-process trace collector and store (TraceCollectorTracing, org.llm4s.trace.store, org.llm4s.trace.model) and CostTracker, with no third-party dependency; llm4s-observability-otel registers OpenTelemetry and brings the OpenTelemetry SDK. Adding the dependency is all the registration there is, and the LANGFUSE_* / OTEL_* variables keep their names. llm4s-observability-prometheus carries PrometheusMetrics, the /metrics endpoint and MetricsConfigLoader, with the Prometheus client and HTTP server; core keeps only the MetricsCollector interface every client accepts. Package names are unchanged. See the migration notes for Langfuse and the collector and for Prometheus.

llm4s-observability-micrometer is the Micrometer counterpart: a MetricsCollector that records into a MeterRegistry you already have, with the same series names as the Prometheus module. It depends on llm4s-core and micrometer-core. See Micrometer metrics.

For Neo4j (knowledge graph store) 

1
2

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-knowledgegraph-neo4j" % llm4sVersion

Neo4jGraphStore, a graph store for llm4s-knowledgegraph backed by Neo4j. It depends on llm4s-core and llm4s-knowledgegraph and brings the Neo4j driver.

Structured output from a case class 

1
2
3

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-schema-derivation" % llm4sVersion
libraryDependencies += "org.llm4s" %% "llm4s-schema-iron"       % llm4sVersion  // bounds from Iron refinement types

SchemaOf and client.completeStructuredOf[A]: the JSON schema of completeStructured, derived from a case class with derives. It is experimental, and depends on llm4s-core only. See Derived Schemas. llm4s-schema-iron (experimental) adds the bounds of Iron refinement types, such as Int :| Interval.Closed[1, 5], to the derived schema and refuses a reply that breaks them; it brings Iron. See Iron refinements.

Typed prompts 

1
2

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-prompt" % llm4sVersion

The prompt"..." and system"..." interpolators: each value is rendered by a Promptable checked at compile time, and Untrusted input is fenced. It is experimental, and depends on llm4s-core only. See Typed Prompts.

Capability-typed clients 

1
2

// same version as llm4s-core
libraryDependencies += "org.llm4s" %% "llm4s-capabilities" % llm4sVersion

client.requireCapabilities[Vision & Tools](provider, model) and Capable: check a model’s capabilities once, and passing a client not checked for one fails to compile. It is experimental, and depends on llm4s-core only. See Capability-Typed Clients.

Testing your own code (the testkit) 

1
2

// same version as llm4s-core; test scope only
libraryDependencies += "org.llm4s" %% "llm4s-testkit" % llm4sVersion % Test

A scriptable fake LLMClient for the tests of code that calls llm4s: canned replies, scripted tool calls, injected errors and recorded requests, with no network and no API key. It depends on llm4s-core only. See Testing with the Testkit.

Checking your setup (the doctor) 

1
2

// same version as llm4s-core; test scope only
libraryDependencies += "org.llm4s" %% "llm4s-doctor" % llm4sVersion % Test

DoctorCli checks your JDK, application.conf, default provider, provider module, API key and, with --local or --live, the model server, and says what to fix: sbt "Test/runMain org.llm4s.doctor.DoctorCli". It depends on llm4s-core only. See Doctor for Maven, Gradle and Coursier.

Trying llm4s in the REPL 

1
2

// same version as llm4s-core; for the REPL, worksheets and scripts, not for production code
libraryDependencies += "org.llm4s" %% "llm4s-repl" % llm4sVersion

ask, extract, chat(), history and lastUsage behind one import org.llm4s.repl.*, with an offline session that needs no API key. It is experimental, and depends on llm4s-core, llm4s-schema-derivation and llm4s-testkit. See Try llm4s in the REPL.

For provider authors (testing a provider module) 

1
2

// same version as llm4s-core; test scope only
libraryDependencies += "org.llm4s" %% "llm4s-provider-testkit" % llm4sVersion % Test

The checks a provider module’s own Llm4s<Name>ModuleSpec makes: discovery, sole ownership of its ids, explicit registration and the config-to-client round trip. It requires JDK 21: its interruption checks and its local test server run on virtual threads. See Writing a provider.

For checkpoint store authors (testing a Checkpointer) 

1
2

// same version as llm4s-agent; test scope only
libraryDependencies += "org.llm4s" %% "llm4s-agent-testkit" % llm4sVersion % Test

CheckpointerContract, the suite every Checkpointer must pass: commits, event numbering, compaction, run-claim leases and fencing, and two runtimes contending over one store. It requires JDK 21, as llm4s-agent does. See Durable Checkpointers.

For Workspace (Containerized Execution) 

1

libraryDependencies += "org.llm4s" %% "llm4s-workspace-client" % "0.5.1"

And install Docker:

1
2
3
4
5
6
7
8

# macOS
brew install docker

# Ubuntu/Debian
sudo apt-get install docker.io

# Verify
docker --version

llm4s-workspace-client brings llm4s-workspace-shared, the wire protocol the client and the runner image speak, so you depend on it directly only if you implement the runner side.

Provider Setup

llm4s reads provider configuration from named sections in your application.conf, not from environment variables of its own. Keep secrets in environment variables and bind them into the section with ${?VAR}:

1
2
3
4
5
6
7
8
9
10
11

# src/main/resources/application.conf
llm4s {
  providers {
    provider = "openai-main"          # the default: the name of a section below

    openai-main {
      provider = "openai"
      model    = "gpt-4o-mini"
    }
  }
}

1

export OPENAI_API_KEY=sk-...

The section needs no apiKey line: llm4s-openai binds OPENAI_API_KEY to OpenAI’s shared key, which a section without a key of its own uses (see API keys for a second account). Nothing reads LLM_MODEL. The Configuration guide shows sections for every provider, how to switch between them, and which environment variables llm4s reads without any binding.

If you keep your variables in a .env file (add it to .gitignore!), load it before running:

1
2

source .env
sbt run

Get API Keys

OpenAI

  1. Go to platform.openai.com
  2. Sign up or log in
  3. Navigate to API Keys
  4. Click Create new secret key
  5. Copy the key (starts with sk-)

Anthropic

  1. Go to console.anthropic.com
  2. Sign up or log in
  3. Navigate to API Keys
  4. Click Create Key
  5. Copy the key (starts with sk-ant-)

Azure OpenAI

  1. Create an Azure account
  2. Navigate to Azure OpenAI Service
  3. Create a resource
  4. Deploy a model (e.g., gpt-4o)
  5. Copy the API Key and Endpoint

Ollama (Local)

  1. Install Ollama: ollama.com
  2. Pull a model: ollama pull llama2
  3. Start server: ollama serve
  4. No API key needed!

Verify Installation

Create a simple test file VerifyInstall.scala:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

import org.llm4s.config.Llm4sConfig
import org.llm4s.llmconnect.LLMConnect
import org.llm4s.llmconnect.model.{ Conversation, UserMessage }
import org.llm4s.model.ModelRegistryService

object VerifyInstall extends App {
  println("Testing LLM4S installation...")

  val result = for {
    providerConfig <- Llm4sConfig.defaultProvider()
    registry       <- Llm4sConfig.modelRegistryService()
    given ModelRegistryService = registry
    client   <- LLMConnect.getClient(providerConfig)
    response <- client.complete(Conversation(Seq(UserMessage("Say 'LLM4S is working!'"))))
  } yield response

  result match {
    case Right(completion) =>
      println("✅ Success!")
      println(s"Response: ${completion.content}")
    case Left(error) =>
      println("❌ Error:")
      println(error)
  }
}

Run it:

1

sbt run

Expected output:

1
2
3

Testing LLM4S installation...
✅ Success!
Response: LLM4S is working!

Troubleshooting

“missing required fields: apiKey”

Problem: the provider section has no API key.

Solution:

  1. Check the vendor’s variable is set in the shell that starts the JVM: echo $OPENAI_API_KEY (if you use a .env file, source .env first), and that the provider’s module (llm4s-openai) is a dependency - it is what binds the variable
  2. If the section sets its own apiKey = ${?SOME_VAR}, check that variable too
  3. Check the error names the section you meant to load - only that section is validated, so if it names another, llm4s.providers.provider points somewhere you did not expect

“Configured provider ‘…’ was not found” or an unknown provider id

Problem: llm4s.providers.provider names a section that does not exist, or a section’s provider = "..." id is not registered.

Solution: make llm4s.providers.provider match a section name exactly, and add the module for the provider id (for example llm4s-openai for provider = "openai"; see Optional Dependencies). Model names go in the section’s model without a provider prefix: model = "gpt-4o" for OpenAI, not openai/gpt-4o.

Compilation Errors

Problem: Scala version mismatch.

Solution:

1
2
3

# Clean and recompile
sbt clean
sbt compile

Dependency Resolution Issues

Problem: Can’t resolve LLM4S dependency.

Solution:

  • Releases are on Maven Central, so no additional resolver is needed
  • With Maven or Gradle, write the Scala suffix yourself: llm4s-core_3, not llm4s-core (the BOM, llm4s-bom, has none)
  • No snapshots are published: to use main, build it locally (see Unreleased changes)
  • llm4s-kotlin-api is not on Maven Central for 0.5.1 yet (see For Kotlin)

Next Steps

Now that LLM4S is installed:

  1. Write your first program → - Create a simple LLM application
  2. Configure providers → - Set up multiple LLM providers
  3. Explore examples → - Browse 69 working examples

Additional Resources

Installation complete! Ready to write your first program →