Configuration Guide

Configure LLM4S for multiple providers, environments, and use cases.

Table of contents

  1. Configuration Hierarchy
  2. Environment Variables
    1. Core Settings
    2. Complete Environment Setup
    3. Load Environment Variables
  3. Application Configuration (HOCON)
    1. Create application.conf
    2. Access Configuration in Code
  4. Provider-Specific Configuration
    1. OpenAI
    2. Anthropic
    3. Azure OpenAI
    4. Ollama (Local Models)
  5. Embeddings Configuration
    1. OpenAI Embeddings
    2. Voyage AI Embeddings
    3. Ollama Embeddings (Local)
    4. Using Embeddings in Code
    5. System Properties (Alternative)
  6. Tracing Configuration
    1. Console Tracing (Development)
    2. Langfuse (Production)
    3. Disable Tracing
  7. Multi-Provider Setup
  8. System Properties
  9. Environment-Specific Configurations
    1. Development
    2. Staging
    3. Production
  10. Configuration Best Practices
    1. ✅ DO
    2. ❌ DON’T
  11. Validation Example
  12. Troubleshooting
    1. Problem: “API key not found”
    2. Problem: “Invalid model format”
    3. Problem: “Configuration not loading”
    4. Problem: “Provider mismatch”
  13. Next Steps
  14. Additional Resources

Configuration Hierarchy

LLM4S uses a hierarchical configuration system with the following precedence (highest to lowest):

  1. System properties (-Dkey=value JVM flags)
  2. Environment variables (.env file or shell exports)
  3. application.conf (HOCON format in your project)
  4. reference.conf (Library defaults)

Environment Variables

The simplest way to configure LLM4S is through environment variables.

Core Settings 

1
2
3
4
5

# Required: Model selection (determines provider)
LLM_MODEL=openai/gpt-4o

# Required: API key for your chosen provider
OPENAI_API_KEY=sk-proj-...

Complete Environment Setup

Create a .env file in your project root:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64

# ===================
# Model Selection
# ===================
# Format: <provider>/<model-name>
# Supported providers: openai, anthropic, azure, ollama
LLM_MODEL=openai/gpt-4o

# ===================
# OpenAI Configuration
# ===================
OPENAI_API_KEY=sk-proj-...
OPENAI_BASE_URL=https://api.openai.com/v1  # Optional: Override endpoint
OPENAI_ORGANIZATION=org-...                # Optional: Organization ID

# ===================
# Anthropic Configuration
# ===================
ANTHROPIC_API_KEY=sk-ant-...
ANTHROPIC_BASE_URL=https://api.anthropic.com  # Optional
ANTHROPIC_VERSION=2023-06-01                  # Optional: API version

# ===================
# Azure OpenAI Configuration
# ===================
AZURE_API_KEY=your-azure-key
AZURE_API_BASE=https://your-resource.openai.azure.com
AZURE_DEPLOYMENT_NAME=gpt-4o
AZURE_API_VERSION=2024-02-15-preview  # Optional

# ===================
# Ollama Configuration
# ===================
OLLAMA_BASE_URL=http://localhost:11434
# No API key needed for Ollama

# ===================
# Tracing Configuration
# ===================
TRACING_MODE=langfuse  # Options: langfuse, console, none

# Langfuse settings (if using TRACING_MODE=langfuse)
LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_SECRET_KEY=sk-lf-...
LANGFUSE_URL=https://cloud.langfuse.com  # or self-hosted

# ===================
# Embeddings Configuration (RAG)
# ===================
# Provider: openai, voyage, or ollama
EMBEDDING_PROVIDER=openai
OPENAI_EMBEDDING_BASE_URL=https://api.openai.com/v1
OPENAI_EMBEDDING_MODEL=text-embedding-3-small

# Or for local embeddings (no API key needed):
# EMBEDDING_PROVIDER=ollama
# OLLAMA_EMBEDDING_MODEL=nomic-embed-text

# ===================
# Workspace Configuration (Optional)
# ===================
WORKSPACE_DIR=/tmp/llm4s-workspace
WORKSPACE_IMAGE=llm4s/workspace-runner:latest
WORKSPACE_PORT=8080
WORKSPACE_CONTAINER_NAME=llm4s-workspace

Load Environment Variables

Option 1: Source the file

1
2

source .env
sbt run

Option 2: Use sbt-dotenv plugin

Add to project/plugins.sbt:

1

addSbtPlugin("au.com.onegeek" %% "sbt-dotenv" % "2.1.233")

Variables automatically load when SBT starts!

Option 3: IntelliJ IDEA

  1. Run → Edit Configurations
  2. Environment variables → Add from file
  3. Select your .env file

Application Configuration (HOCON)

For more complex configurations, use application.conf:

Create application.conf

Create src/main/resources/application.conf:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

llm {
  # Model configuration
  model = ${?LLM_MODEL}  # Fallback to env var
  model = "openai/gpt-4o"  # Default if not set

  # Generation parameters
  temperature = 0.7
  max-tokens = 2000
  top-p = 1.0

  # Provider configurations
  providers {
    openai {
      api-key = ${?OPENAI_API_KEY}
      base-url = ${?OPENAI_BASE_URL}
      base-url = "https://api.openai.com/v1"  # Default
      organization = ${?OPENAI_ORGANIZATION}
    }

    anthropic {
      api-key = ${?ANTHROPIC_API_KEY}
      base-url = ${?ANTHROPIC_BASE_URL}
      base-url = "https://api.anthropic.com"
      version = ${?ANTHROPIC_VERSION}
      version = "2023-06-01"
    }

    azure {
      api-key = ${?AZURE_API_KEY}
      api-base = ${?AZURE_API_BASE}
      deployment-name = ${?AZURE_DEPLOYMENT_NAME}
      api-version = ${?AZURE_API_VERSION}
      api-version = "2024-02-15-preview"
    }

    ollama {
      base-url = ${?OLLAMA_BASE_URL}
      base-url = "http://localhost:11434"
    }
  }
}

# Tracing configuration
tracing {
  mode = ${?TRACING_MODE}
  mode = "none"  # Default: disabled

  langfuse {
    public-key = ${?LANGFUSE_PUBLIC_KEY}
    secret-key = ${?LANGFUSE_SECRET_KEY}
    url = ${?LANGFUSE_URL}
    url = "https://cloud.langfuse.com"
  }
}

# Context window management
context {
  max-messages = 50
  preserve-system-message = true
  pruning-strategy = "oldest-first"  # oldest-first, middle-out, recent-turns
}

Access Configuration in Code 

1
2
3
4
5
6
7
8
9
10

import org.llm4s.config.ConfigReader

// Load LLM configuration
val llmConfig = ConfigReader.LLMConfig()

// Load provider-specific config
val providerConfig = ConfigReader.Provider()

// Load tracing config
val tracingConfig = ConfigReader.TracingConf()

Provider-Specific Configuration

OpenAI 

1
2
3
4
5
6
7
8
9
10
11

# Model selection
LLM_MODEL=openai/gpt-4o          # Latest GPT-4o
LLM_MODEL=openai/gpt-4-turbo     # GPT-4 Turbo
LLM_MODEL=openai/gpt-3.5-turbo   # GPT-3.5

# Required
OPENAI_API_KEY=sk-proj-...

# Optional
OPENAI_BASE_URL=https://api.openai.com/v1
OPENAI_ORGANIZATION=org-...

Available Models:

  • gpt-4o - Latest GPT-4 Optimized
  • gpt-4-turbo - Fast GPT-4
  • gpt-4 - Standard GPT-4
  • gpt-3.5-turbo - Fast and cheap

Anthropic 

1
2
3
4
5
6
7
8
9
10

# Model selection
LLM_MODEL=anthropic/claude-sonnet-4-5-latest
LLM_MODEL=anthropic/claude-opus-4-5-latest

# Required
ANTHROPIC_API_KEY=sk-ant-...

# Optional
ANTHROPIC_BASE_URL=https://api.anthropic.com
ANTHROPIC_VERSION=2023-06-01

Available Models:

  • claude-sonnet-4-5-latest - Latest Claude 4.5 Sonnet
  • claude-opus-4-5-latest - Most capable Claude 4.5 Opus
  • claude-3-haiku-latest - Fastest

Azure OpenAI 

1
2
3
4
5
6
7
8
9
10

# Model selection
LLM_MODEL=azure/gpt-4o

# Required
AZURE_API_KEY=your-azure-key
AZURE_API_BASE=https://your-resource.openai.azure.com
AZURE_DEPLOYMENT_NAME=gpt-4o  # Your deployment name

# Optional
AZURE_API_VERSION=2024-02-15-preview

Ollama (Local Models) 

1
2
3
4
5
6
7
8
9

# Model selection
LLM_MODEL=ollama/llama2
LLM_MODEL=ollama/mistral
LLM_MODEL=ollama/codellama

# Required
OLLAMA_BASE_URL=http://localhost:11434

# No API key needed!

Install Ollama:

1
2
3
4
5
6
7
8

# Install
curl -fsSL https://ollama.com/install.sh | sh

# Pull a model
ollama pull llama2

# Start server
ollama serve

Embeddings Configuration

LLM4S supports multiple embedding providers for RAG (Retrieval-Augmented Generation) workflows.

OpenAI Embeddings 

1
2
3
4

EMBEDDING_PROVIDER=openai
OPENAI_EMBEDDING_BASE_URL=https://api.openai.com/v1
OPENAI_EMBEDDING_MODEL=text-embedding-3-small
OPENAI_API_KEY=sk-...  # Same key as LLM

Available Models:

  • text-embedding-3-small - Fast, cost-effective (1536 dimensions)
  • text-embedding-3-large - Higher quality (3072 dimensions)
  • text-embedding-ada-002 - Legacy model (1536 dimensions)

Voyage AI Embeddings 

1
2
3
4

EMBEDDING_PROVIDER=voyage
VOYAGE_EMBEDDING_BASE_URL=https://api.voyageai.com/v1
VOYAGE_EMBEDDING_MODEL=voyage-3
VOYAGE_API_KEY=pa-...

Available Models:

  • voyage-3 - General purpose
  • voyage-3-large - Higher quality
  • voyage-code-2 - Code-optimized

Ollama Embeddings (Local) 

1
2
3
4

EMBEDDING_PROVIDER=ollama
OLLAMA_EMBEDDING_BASE_URL=http://localhost:11434  # Optional, this is default
OLLAMA_EMBEDDING_MODEL=nomic-embed-text
# No API key needed!

Available Models:

  • nomic-embed-text - General purpose (768 dimensions)
  • mxbai-embed-large - Higher quality (1024 dimensions)
  • all-minilm - Lightweight (384 dimensions)

Install embedding model:

1

ollama pull nomic-embed-text

Using Embeddings in Code 

1
2
3
4
5
6
7
8

import org.llm4s.llmconnect.EmbeddingClient
import org.llm4s.agent.memory.LLMEmbeddingService

// Create embedding client from environment
val client = EmbeddingClient.fromEnv()

// Or use the higher-level service
val embeddingService = LLMEmbeddingService.fromEnv()

System Properties (Alternative)

When running via sbt, use system properties for reliable configuration:

1
2
3
4

sbt -Dllm4s.embeddings.provider=ollama \
    -Dllm4s.embeddings.ollama.model=nomic-embed-text \
    -Dllm4s.ollama.baseUrl=http://localhost:11434 \
    "run"

Tracing Configuration

Console Tracing (Development) 

1

TRACING_MODE=console

Output appears in stdout:

1
2
3

[TRACE] Completion request: UserMessage(What is Scala?)
[TRACE] Token usage: 150 tokens
[TRACE] Response time: 1.2s

Langfuse (Production) 

1
2
3
4

TRACING_MODE=langfuse
LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_SECRET_KEY=sk-lf-...
LANGFUSE_URL=https://cloud.langfuse.com

Get keys from Langfuse:

  1. Sign up at langfuse.com
  2. Create a project
  3. Navigate to Settings → API Keys
  4. Copy public and secret keys

Disable Tracing 

1

TRACING_MODE=none

Multi-Provider Setup

Switch between providers without code changes:

1
2
3
4
5
6
7
8
9
10
11

# Development: Use Ollama (free, local)
LLM_MODEL=ollama/llama2
OLLAMA_BASE_URL=http://localhost:11434

# Staging: Use OpenAI GPT-3.5 (cheap)
LLM_MODEL=openai/gpt-3.5-turbo
OPENAI_API_KEY=sk-...

# Production: Use Claude Sonnet (best quality)
LLM_MODEL=anthropic/claude-sonnet-4-5-latest
ANTHROPIC_API_KEY=sk-ant-...

Your code remains the same:

1

val client = LLMConnect.create()  // Auto-selects based on LLM_MODEL

System Properties

Override any setting via JVM flags:

1
2
3

sbt -DLLM_MODEL=openai/gpt-4o \
    -DOPENAI_API_KEY=sk-... \
    run

Or in build.sbt:

1
2
3
4

javaOptions ++= Seq(
  s"-DLLM_MODEL=openai/gpt-4o",
  s"-DOPENAI_API_KEY=${sys.env.getOrElse("OPENAI_API_KEY", "")}"
)

Environment-Specific Configurations

Development

Create .env.dev:

1
2
3

LLM_MODEL=ollama/llama2
OLLAMA_BASE_URL=http://localhost:11434
TRACING_MODE=console

Staging

Create .env.staging:

1
2
3
4
5

LLM_MODEL=openai/gpt-3.5-turbo
OPENAI_API_KEY=${STAGING_OPENAI_KEY}
TRACING_MODE=langfuse
LANGFUSE_PUBLIC_KEY=${STAGING_LANGFUSE_PUBLIC}
LANGFUSE_SECRET_KEY=${STAGING_LANGFUSE_SECRET}

Production

Create .env.prod:

1
2
3
4
5

LLM_MODEL=anthropic/claude-sonnet-4-5-latest
ANTHROPIC_API_KEY=${PROD_ANTHROPIC_KEY}
TRACING_MODE=langfuse
LANGFUSE_PUBLIC_KEY=${PROD_LANGFUSE_PUBLIC}
LANGFUSE_SECRET_KEY=${PROD_LANGFUSE_SECRET}

Load the appropriate file:

1
2

source .env.prod
sbt run

Configuration Best Practices

✅ DO

  1. Use environment variables for secrets (API keys)
  2. Use application.conf for non-sensitive defaults
  3. Add .env to .gitignore to prevent committing secrets
  4. Use different configs for dev/staging/prod
  5. Validate configuration at startup
1
2
3
4
5
6
7
8

val config = ConfigReader.Provider()
config.fold(
  error => {
    println(s"Configuration error: $error")
    System.exit(1)
  },
  _ => println("Configuration loaded successfully")
)

❌ DON’T

  1. Don’t hardcode API keys in source code
  2. Don’t commit .env files to version control
  3. Don’t use System.getenv() directly (use ConfigReader)
  4. Don’t mix production keys in development
  5. Don’t skip validation - fail fast on bad config

Validation Example 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

import org.llm4s.config.ConfigReader
import org.llm4s.llmconnect.LLMConnect

object ValidateConfig extends App {
  println("Validating LLM4S configuration...")

  val validation = for {
    llmConfig <- ConfigReader.LLMConfig()
    providerConfig <- ConfigReader.Provider()
    client <- LLMConnect.create()
  } yield {
    println(s"✅ Model: ${providerConfig.model}")
    println(s"✅ Provider: ${providerConfig.getClass.getSimpleName}")
    println(s"✅ Client ready: ${client.getClass.getSimpleName}")
  }

  validation match {
    case Right(_) =>
      println("✅ Configuration valid!")
    case Left(error) =>
      println(s"❌ Configuration error: $error")
      System.exit(1)
  }
}

Troubleshooting

Problem: “API key not found”

Check:

  1. .env file exists and is in project root
  2. Environment variable is spelled correctly
  3. .env file is sourced: source .env
  4. No quotes around values in .env (unless needed)

Problem: “Invalid model format”

Fix: Use correct format:

1
2
3
4
5

# ✅ Correct
LLM_MODEL=openai/gpt-4o

# ❌ Wrong
LLM_MODEL=gpt-4o

Problem: “Configuration not loading”

Debug:

1
2
3
4

import com.typesafe.config.ConfigFactory

val config = ConfigFactory.load()
println(config.getString("llm.model"))

Problem: “Provider mismatch”

Ensure your LLM_MODEL matches the API key provider:

1
2
3
4
5
6
7

# ✅ Correct
LLM_MODEL=openai/gpt-4o
OPENAI_API_KEY=sk-...

# ❌ Wrong
LLM_MODEL=openai/gpt-4o
ANTHROPIC_API_KEY=sk-ant-...  # Wrong provider!

Next Steps

Configuration complete! Now you can:

  1. Explore features → - Dive into agents, tools, and more
  2. Browse examples → - See configuration in action
  3. User guide → - Learn core concepts
  4. Observability → - Set up tracing

Additional Resources

Ready to build? Explore what’s next →