Configuration Guide

Configure LLM4S for multiple providers, environments, and use cases.

Table of contents

  1. Configuration Hierarchy
  2. Environment Variables
    1. Core Settings
    2. Complete Environment Setup
    3. Load Environment Variables
  3. Application Configuration (HOCON)
    1. Preferred: Named Providers
    2. Access Configuration in Code
  4. Provider-Specific Configuration
    1. OpenAI
    2. Anthropic
    3. Azure OpenAI
    4. Ollama (Local Models)
    5. Google Gemini
  5. Embeddings Configuration
    1. Unified Format (Recommended)
    2. Available Models
    3. Legacy Format (Still Supported)
    4. Using Embeddings in Code
    5. System Properties (Alternative)
  6. Tracing Configuration
    1. Console Tracing (Development)
    2. Langfuse (Production)
    3. OpenTelemetry
    4. Disable Tracing
  7. Multi-Provider Setup
  8. System Properties
  9. Environment-Specific Configurations
    1. Development
    2. Staging
    3. Production
  10. Configuration Best Practices
    1. ✅ DO
    2. ❌ DON’T
  11. Validation Example
  12. Troubleshooting
    1. Problem: “API key not found”
    2. Problem: “Invalid model format”
    3. Problem: “Configuration not loading from application.conf”
    4. Problem: “Provider mismatch”
    5. Problem: “Rate limiting / 429 errors”
    6. Problem: “Slow responses or timeouts”
    7. Problem: “OutOfMemoryError with embeddings”
  13. Next Steps
  14. Additional Resources

Configuration Hierarchy

LLM4S uses a hierarchical configuration system with the following precedence (highest to lowest):

  1. System properties (-Dkey=value JVM flags)
  2. Environment variables (.env file or shell exports)
  3. application.conf (HOCON format in your project)
  4. reference.conf (Library defaults)

Environment Variables

Use environment variables for secrets and for selecting the default named provider you want Llm4sConfig.defaultProvider() to resolve.

Core Settings 

1
2
3
4
5

# Required: default named provider
LLM4S_PROVIDER=openai-main

# Required: API key for the configured provider
OPENAI_API_KEY=sk-proj-...

Complete Environment Setup

Create a .env file in your project root:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86

# ===================
# Named Provider Selection
# ===================
LLM4S_PROVIDER=openai-main

# ===================
# OpenAI Configuration
# ===================
OPENAI_API_KEY=sk-proj-...
OPENAI_BASE_URL=https://api.openai.com/v1  # Optional: Override endpoint
OPENAI_ORGANIZATION=org-...                # Optional: Organization ID

# ===================
# Anthropic Configuration
# ===================
ANTHROPIC_API_KEY=sk-ant-...
ANTHROPIC_BASE_URL=https://api.anthropic.com  # Optional
ANTHROPIC_VERSION=2023-06-01                  # Optional: API version

# ===================
# Azure OpenAI Configuration
# ===================
AZURE_API_KEY=your-azure-key
AZURE_API_BASE=https://your-resource.openai.azure.com
AZURE_DEPLOYMENT_NAME=gpt-4o
AZURE_API_VERSION=2024-02-15-preview  # Optional

# ===================
# Ollama Configuration
# ===================
OLLAMA_BASE_URL=http://localhost:11434
# No API key needed for Ollama

# ===================
# Google Gemini Configuration
# ===================
GOOGLE_API_KEY=your-google-api-key
GEMINI_BASE_URL=https://generativelanguage.googleapis.com/v1beta  # Optional

# ===================
# Cohere Configuration
# ===================
COHERE_API_KEY=your-cohere-api-key
COHERE_BASE_URL=https://api.cohere.com  # Optional

# ===================
# Tracing Configuration
# ===================
TRACING_MODE=langfuse  # Options: langfuse, opentelemetry, console, none

# Langfuse settings (if using TRACING_MODE=langfuse)
LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_SECRET_KEY=sk-lf-...
LANGFUSE_URL=https://cloud.langfuse.com  # or self-hosted

# OpenTelemetry settings (if using TRACING_MODE=opentelemetry)
OTEL_SERVICE_NAME=llm4s-agent
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
OTEL_EXPORTER_OTLP_HEADERS=authorization=Bearer <token>  # Optional

# ===================
# Embeddings Configuration (RAG)
# ===================
# Unified format: provider/model (recommended)
EMBEDDING_MODEL=openai/text-embedding-3-small  # Uses default base URL
# OPENAI_API_KEY is shared with LLM config above

# Or for Voyage embeddings:
# EMBEDDING_MODEL=voyage/voyage-3
# VOYAGE_API_KEY=pa-...

# Or for local embeddings with Ollama (no API key needed):
# EMBEDDING_MODEL=ollama/nomic-embed-text

# Optional: Override default base URLs
# OPENAI_EMBEDDING_BASE_URL=https://api.openai.com/v1
# VOYAGE_EMBEDDING_BASE_URL=https://api.voyageai.com/v1
# OLLAMA_EMBEDDING_BASE_URL=http://localhost:11434

# ===================
# Workspace Configuration (Optional)
# ===================
WORKSPACE_DIR=/tmp/llm4s-workspace
WORKSPACE_IMAGE=llm4s/workspace-runner:latest
WORKSPACE_PORT=8080
WORKSPACE_CONTAINER_NAME=llm4s-workspace

Load Environment Variables

Option 1: Source the file

1
2

source .env
sbt run

Option 2: Use sbt-dotenv plugin

Java 16+ Compatibility Issue: The sbt-dotenv plugin can cause a java.lang.reflect.InaccessibleObjectException at startup on Java 16+ due to module system restrictions. Use Option 1 or Option 3 as alternatives, or apply the fix below.

Add to project/plugins.sbt:

1

addSbtPlugin("au.com.onegeek" %% "sbt-dotenv" % "2.1.233")

Variables automatically load when SBT starts!

Java 16+ Fix: If you see InaccessibleObjectException, create (or add to) .jvmopts in your project root:

1
2

--add-opens=java.base/java.lang=ALL-UNNAMED
--add-opens=java.base/java.lang.reflect=ALL-UNNAMED

Restart sbt after adding this file. This grants the plugin the reflective access it needs and works on all platforms.

Option 3: IntelliJ IDEA

  1. Run → Edit Configurations
  2. Environment variables → Add from file
  3. Select your .env file

Application Configuration (HOCON)

For more complex configurations, use application.conf:

Preferred: Named Providers

For production systems and applications that need multiple configured providers, prefer the llm4s.providers section.

This keeps:

  • provider names and structure in HOCON
  • secrets in environment variables
  • the environment variable names under your control

For example, ${?FOO} is perfectly valid if that is the name you want to use.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

llm4s {
  providers {
    provider = "openai-main"

    openai-main {
      provider = "openai"
      model = "gpt-4o-mini"
      apiKey = ${?OPENAI_MAIN_API_KEY}
      baseUrl = ${?OPENAI_MAIN_BASE_URL}
      organization = ${?OPENAI_MAIN_ORGANIZATION}
    }

    gemini-main {
      provider = "gemini"
      model = "gemini-2.0-flash"
      apiKey = ${?GEMINI_MAIN_API_KEY}
      baseUrl = ${?GEMINI_MAIN_BASE_URL}
    }

    ollama-local {
      provider = "ollama"
      model = "llama3.2"
      baseUrl = ${?OLLAMA_LOCAL_BASE_URL}
    }
  }
}

You can then access the new configuration path with:

1
2
3
4
5
6

import org.llm4s.config.Llm4sConfig

val providersResult = Llm4sConfig.providers()
val defaultName     = Llm4sConfig.defaultProviderName()
val defaultConfig   = Llm4sConfig.defaultProvider()
val namedConfig     = Llm4sConfig.provider("openai-main")

Named providers can also be used for model discovery:

1
2
3
4

import org.llm4s.config.Llm4sConfig

val defaultModels = Llm4sConfig.listModels()
val namedModels   = Llm4sConfig.listModels("openai-main")

See the runnable samples for end-to-end examples:

1
2
3

sbt "samples/runMain samples.basic.NamedProviderModelListingExample"
sbt "samples/runMain samples.basic.SerialNamedProviderModelListingExample"
sbt "samples/runMain samples.basic.ParallelNamedProviderModelListingExample"

If llm4s.providers.provider is set, it must match one of the configured provider names. If it is absent, the providers config still loads successfully, but requesting the default provider will fail clearly at runtime.

Access Configuration in Code 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

import org.llm4s.config.Llm4sConfig

object ConfigurationLoader {
  def load(): Unit = {
    // 1. Load configuration at the application boundary
    val configResult = for {
      providerConfig <- Llm4sConfig.defaultProvider()
      tracingConfig <- Llm4sConfig.tracing()
      embeddingsConfig <- Llm4sConfig.embeddings()
    } yield (providerConfig, tracingConfig, embeddingsConfig)

    // 2. Pass configuration to your application components
    configResult match {
      case Right((provider, tracing, embeddings)) =>
        // Initialize your application with loaded config
        // val app = new MyApplication(provider, tracing, embeddings)
        // app.start()
        println("Configuration loaded successfully")
      
      case Left(error) =>
        Console.err.println(s"Failed to load configuration: $error")
        System.exit(1)
    }
  }
}

Provider-Specific Configuration

OpenAI 

1
2
3
4
5
6
7
8
9

# Example named provider selection
LLM4S_PROVIDER=openai-main

# Required
OPENAI_API_KEY=sk-proj-...

# Optional
OPENAI_BASE_URL=https://api.openai.com/v1
OPENAI_ORGANIZATION=org-...

Available Models:

  • gpt-4o - Latest GPT-4 Optimized
  • gpt-4-turbo - Fast GPT-4
  • gpt-4 - Standard GPT-4
  • gpt-3.5-turbo - Fast and cheap

Anthropic 

1
2
3
4
5
6
7
8
9

# Example named provider selection
LLM4S_PROVIDER=anthropic-main

# Required
ANTHROPIC_API_KEY=sk-ant-...

# Optional
ANTHROPIC_BASE_URL=https://api.anthropic.com
ANTHROPIC_VERSION=2023-06-01

Available Models:

  • claude-sonnet-4-5-latest - Latest Claude 4.5 Sonnet
  • claude-opus-4-5-latest - Most capable Claude 4.5 Opus
  • claude-3-haiku-latest - Fastest

Azure OpenAI 

1
2
3
4
5
6
7
8
9
10

# Example named provider selection
LLM4S_PROVIDER=azure-main

# Required
AZURE_API_KEY=your-azure-key
AZURE_API_BASE=https://your-resource.openai.azure.com
AZURE_DEPLOYMENT_NAME=gpt-4o  # Your deployment name

# Optional
AZURE_API_VERSION=2024-02-15-preview

Ollama (Local Models) 

1
2
3
4
5
6
7

# Example named provider selection
LLM4S_PROVIDER=ollama-local

# Required
OLLAMA_BASE_URL=http://localhost:11434

# No API key needed!

Install Ollama:

1
2
3
4
5
6
7
8

# Install
curl -fsSL https://ollama.com/install.sh | sh

# Pull a model
ollama pull llama2

# Start server
ollama serve

Google Gemini 

1
2
3
4
5
6
7
8
9
10

# Model selection
LLM_MODEL=gemini/gemini-2.0-flash
LLM_MODEL=gemini/gemini-1.5-pro
LLM_MODEL=gemini/gemini-1.5-flash

# Required
GOOGLE_API_KEY=your-google-api-key

# Optional
GEMINI_BASE_URL=https://generativelanguage.googleapis.com/v1beta

Available Models:

  • gemini-2.0-flash - Latest Gemini 2.0 (fast, multimodal)
  • gemini-2.5-flash - Gemini 2.5 Flash (thinking model)
  • gemini-1.5-pro - High capability (1M token context)
  • gemini-1.5-flash - Fast and efficient (1M token context)

Get API Key:

  1. Go to Google AI Studio
  2. Click “Get API Key”
  3. Create or select a project
  4. Copy the generated API key

Embeddings Configuration

LLM4S supports multiple embedding providers for RAG (Retrieval-Augmented Generation) workflows.

Use EMBEDDING_MODEL with the format provider/model-name, similar to LLM_MODEL:

1
2
3
4
5
6
7
8
9
10

# OpenAI embeddings
EMBEDDING_MODEL=openai/text-embedding-3-small
OPENAI_API_KEY=sk-...  # Same key as LLM

# Voyage AI embeddings
EMBEDDING_MODEL=voyage/voyage-3
VOYAGE_API_KEY=pa-...

# Ollama embeddings (local, no API key needed)
EMBEDDING_MODEL=ollama/nomic-embed-text

Default base URLs are used automatically:

  • OpenAI: https://api.openai.com/v1
  • Voyage: https://api.voyageai.com/v1
  • Ollama: http://localhost:11434

Override base URLs if needed:

1
2

EMBEDDING_MODEL=openai/text-embedding-3-small
OPENAI_EMBEDDING_BASE_URL=https://custom.openai.com/v1

Available Models

OpenAI:

  • text-embedding-3-small - Fast, cost-effective (1536 dimensions)
  • text-embedding-3-large - Higher quality (3072 dimensions)
  • text-embedding-ada-002 - Legacy model (1536 dimensions)

Voyage AI:

  • voyage-3 - General purpose
  • voyage-3-large - Higher quality
  • voyage-code-2 - Code-optimized

Ollama (local):

  • nomic-embed-text - General purpose (768 dimensions)
  • mxbai-embed-large - Higher quality (1024 dimensions)
  • all-minilm - Lightweight (384 dimensions)

Install Ollama embedding model:

1

ollama pull nomic-embed-text

Legacy Format (Still Supported)

The legacy format using EMBEDDING_PROVIDER is still supported for backward compatibility:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

# OpenAI
EMBEDDING_PROVIDER=openai
OPENAI_EMBEDDING_BASE_URL=https://api.openai.com/v1
OPENAI_EMBEDDING_MODEL=text-embedding-3-small
OPENAI_API_KEY=sk-...

# Voyage AI
EMBEDDING_PROVIDER=voyage
VOYAGE_EMBEDDING_BASE_URL=https://api.voyageai.com/v1
VOYAGE_EMBEDDING_MODEL=voyage-3
VOYAGE_API_KEY=pa-...

# Ollama
EMBEDDING_PROVIDER=ollama
OLLAMA_EMBEDDING_BASE_URL=http://localhost:11434
OLLAMA_EMBEDDING_MODEL=nomic-embed-text

Using Embeddings in Code 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34

import org.llm4s.config.Llm4sConfig
import org.llm4s.llmconnect.EmbeddingClient
import org.llm4s.llmconnect.config.EmbeddingModelConfig
import org.llm4s.agent.memory.LLMEmbeddingService

// Core logic depends on injected service
class RAGService(embeddingService: LLMEmbeddingService) {
  def processDocuments(docs: Seq[String]): Unit = {
    // Use embeddingService...
  }
}

// Configuration boundary
object RAGApplication extends App {
  val startup = for {
    // 1. Load config
    (provider, cfg) <- Llm4sConfig.embeddings()
    model <- Llm4sConfig.textEmbeddingModel()
    
    // 2. Build dependencies
    client <- EmbeddingClient.from(provider, cfg)
    
    // 3. Create service
    service = LLMEmbeddingService(
      client, 
      EmbeddingModelConfig(model.modelName, model.dimensions)
    )
  } yield new RAGService(service)

  startup.fold(
    err => println(s"Startup failed: $err"),
    service => println("RAG Service started successfully")
  )
}

System Properties (Alternative)

When running via sbt, use system properties for reliable configuration:

1
2
3
4
5
6
7
8

# Using unified format (recommended)
sbt -Dllm4s.embeddings.model=ollama/nomic-embed-text \
    "run"

# Or with legacy format
sbt -Dllm4s.embeddings.provider=ollama \
    -Dllm4s.embeddings.ollama.model=nomic-embed-text \
    "run"

Tracing Configuration

Console Tracing (Development) 

1

TRACING_MODE=console

Output appears in stdout:

1
2
3

[TRACE] Completion request: UserMessage(What is Scala?)
[TRACE] Token usage: 150 tokens
[TRACE] Response time: 1.2s

Langfuse (Production) 

1
2
3
4

TRACING_MODE=langfuse
LANGFUSE_PUBLIC_KEY=pk-lf-...
LANGFUSE_SECRET_KEY=sk-lf-...
LANGFUSE_URL=https://cloud.langfuse.com

Get keys from Langfuse:

  1. Sign up at langfuse.com
  2. Create a project
  3. Navigate to Settings → API Keys
  4. Copy public and secret keys

OpenTelemetry

OpenTelemetry provides distributed tracing for observability across your infrastructure.

Setup:

1
2
3
4

TRACING_MODE=opentelemetry
OTEL_SERVICE_NAME=llm4s-agent
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
OTEL_EXPORTER_OTLP_HEADERS=authorization=Bearer <token>  # Optional

Environment Variables:

Variable Default Description
TRACING_MODE none Must be set to opentelemetry
OTEL_SERVICE_NAME llm4s Service name for trace identification
OTEL_EXPORTER_OTLP_ENDPOINT http://localhost:4317 OpenTelemetry Collector endpoint
OTEL_EXPORTER_OTLP_HEADERS (empty) Additional headers (comma-separated: key1=value1,key2=value2)

Example Configuration:

1
2
3
4
5
6
7
8
9
10
11
12
13

# Local Jaeger (via Docker)
TRACING_MODE=opentelemetry
OTEL_SERVICE_NAME=llm4s-agent
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317

# Cloud Provider with Authentication
TRACING_MODE=opentelemetry
OTEL_SERVICE_NAME=llm4s-runner
OTEL_EXPORTER_OTLP_ENDPOINT=https://otel-collector.example.com:4317
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer sk-otel-token

# Custom Headers (multiple values)
OTEL_EXPORTER_OTLP_HEADERS=API-Key=secret123,Environment=production

Spans Generated:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

Span: LLM Completion
  ├─ gen_ai.request.model: gpt-4o
  ├─ gen_ai.usage.input_tokens: 150
  └─ gen_ai.usage.output_tokens: 50

Span: Tool Execution: web_search
  ├─ tool.name: web_search
  ├─ tool.input: "Scala 3 features"
  └─ duration_ms: 1234

Span: Token Usage - completion
  ├─ operation: completion
  ├─ gen_ai.usage.total_tokens: 200
  └─ cost.usd: 0.0042

Span: Agent State Updated
  ├─ status: RUNNING
  ├─ message_count: 5
  └─ log_count: 12

Setting Up Jaeger (Local Development):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

# Start Jaeger all-in-one
docker run -d \
  -p 6831:6831/udp \
  -p 6832:6832/udp \
  -p 5778:5778 \
  -p 16686:16686 \
  -p 14268:14268 \
  -p 14250:14250 \
  -p 9411:9411 \
  jaegertracing/all-in-one:latest

# Configure LLM4S
export TRACING_MODE=opentelemetry
export OTEL_SERVICE_NAME=llm4s-dev
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317

# View traces at http://localhost:16686

Code Usage:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

import org.llm4s.config.Llm4sConfig
import org.llm4s.trace.Tracing

// Automatic initialization from environment variables
val tracing = Tracing.create(Llm4sConfig.tracing())

// Or create directly
import org.llm4s.trace.OpenTelemetryTracing
val tracer = OpenTelemetryTracing.fromEnvironment()

// Use tracer
val result = for {
  _ <- tracer.traceEvent(TraceEvent.AgentInitialized("query", List("tool1")))
  _ <- tracer.traceTokenUsage(usage, "gpt-4o", "completion")
} yield ()

// Always shutdown
tracer.shutdown()

Disable Tracing 

1

TRACING_MODE=none

Multi-Provider Setup

Switch between providers without code changes:

1
2
3
4
5
6
7
8
9
10
11

# Development: Use Ollama (free, local)
LLM_MODEL=ollama/llama2
OLLAMA_BASE_URL=http://localhost:11434

# Staging: Use OpenAI GPT-3.5 (cheap)
LLM_MODEL=openai/gpt-3.5-turbo
OPENAI_API_KEY=sk-...

# Production: Use Claude Sonnet (best quality)
LLM_MODEL=anthropic/claude-sonnet-4-5-latest
ANTHROPIC_API_KEY=sk-ant-...

Your code remains the same:

1

val client = Llm4sConfig.provider().flatMap(LLMConnect.getClient)

System Properties

Override any setting via JVM flags:

1
2
3

sbt -DLLM_MODEL=openai/gpt-4o \
    -DOPENAI_API_KEY=sk-... \
    run

Or in build.sbt:

1
2
3
4

javaOptions ++= Seq(
  s"-DLLM_MODEL=openai/gpt-4o",
  s"-DOPENAI_API_KEY=${sys.env.getOrElse("OPENAI_API_KEY", "")}"
)

Environment-Specific Configurations

Development

Create .env.dev:

1
2
3

LLM_MODEL=ollama/llama2
OLLAMA_BASE_URL=http://localhost:11434
TRACING_MODE=console

Staging

Create .env.staging:

1
2
3
4
5

LLM_MODEL=openai/gpt-3.5-turbo
OPENAI_API_KEY=${STAGING_OPENAI_KEY}
TRACING_MODE=langfuse
LANGFUSE_PUBLIC_KEY=${STAGING_LANGFUSE_PUBLIC}
LANGFUSE_SECRET_KEY=${STAGING_LANGFUSE_SECRET}

Production

Create .env.prod:

1
2
3
4
5
6
7
8
9
10
11
12
13

LLM_MODEL=anthropic/claude-sonnet-4-5-latest
ANTHROPIC_API_KEY=${PROD_ANTHROPIC_KEY}

# Tracing with OpenTelemetry
TRACING_MODE=opentelemetry
OTEL_SERVICE_NAME=llm4s-prod
OTEL_EXPORTER_OTLP_ENDPOINT=https://otel-collector.prod.example.com:4317
OTEL_EXPORTER_OTLP_HEADERS=Authorization=Bearer ${OTEL_AUTH_TOKEN}

# Or use Langfuse
# TRACING_MODE=langfuse
# LANGFUSE_PUBLIC_KEY=${PROD_LANGFUSE_PUBLIC}
# LANGFUSE_SECRET_KEY=${PROD_LANGFUSE_SECRET}

Load the appropriate file:

1
2

source .env.prod
sbt run

Configuration Best Practices

✅ DO

  1. Use environment variables for secrets (API keys)
  2. Use application.conf for non-sensitive defaults
  3. Add .env to .gitignore to prevent committing secrets
  4. Use different configs for dev/staging/prod
  5. Validate configuration at startup
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

import org.llm4s.config.Llm4sConfig

object ApplicationBoundary {
  def validateAndStart(): Unit = {
    // Fail fast at startup if config is invalid
    Llm4sConfig.provider().fold(
      error => {
        Console.err.println(s"FATAL: Configuration error: $error")
        System.exit(1)
      },
      config => {
        println(s"Configuration loaded for model: ${config.model}")
        // startApplication(config)
      }
    )
  }
}

❌ DON’T

  1. Don’t hardcode API keys in source code
  2. Don’t commit .env files to version control
  3. Don’t use System.getenv() directly (use Llm4sConfig)
  4. Don’t mix production keys in development
  5. Don’t skip validation - fail fast on bad config

Validation Example 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

import org.llm4s.config.Llm4sConfig
import org.llm4s.llmconnect.LLMConnect

object ValidateConfig extends App {
  println("Validating LLM4S configuration...")

  // Perform validation at the application edge
  val validationResult = for {
    providerConfig <- Llm4sConfig.provider()
    client <- LLMConnect.getClient(providerConfig)
  } yield (providerConfig, client)

  validationResult match {
    case Right((config, client)) =>
      println(s"✅ Model: ${config.model}")
      println(s"✅ Provider: ${config.getClass.getSimpleName}")
      println(s"✅ Client ready: ${client.getClass.getSimpleName}")
      println("✅ Configuration valid!")
      
    case Left(error) =>
      Console.err.println(s"❌ Configuration error: $error")
      System.exit(1)
  }
}

Troubleshooting

Problem: “API key not found”

Symptoms:

1

Left(ConfigurationError("Missing API key for provider: openai"))

Root causes:

  1. Environment variable not set
  2. Typo in variable name (case-sensitive)
  3. .env file not loaded in your shell session
  4. Using wrong shell (check with echo $SHELL)

Debug steps:

1
2
3
4
5
6
7
8
9
10
11
12

# Check if variable is set
echo $OPENAI_API_KEY

# List all LLM4S-related variables
env | grep -E '(LLM_MODEL|OPENAI|ANTHROPIC|AZURE|OLLAMA)'

# Source .env explicitly
source .env
echo $OPENAI_API_KEY  # Should show your key

# Run SBT with explicit env loading
sbt -Dconfig.override_with_env_vars=true run

Problem: “Invalid model format”

Symptoms:

1

Left(ConfigurationError("Invalid model format: gpt-4o"))

Fix: Models must include the provider prefix:

1
2
3
4
5
6
7
8

# ✅ Correct format: provider/model-name
LLM_MODEL=openai/gpt-4o
LLM_MODEL=anthropic/claude-3-5-sonnet-20241022
LLM_MODEL=ollama/llama3.2

# ❌ Wrong - missing provider
LLM_MODEL=gpt-4o
LLM_MODEL=claude-3-5-sonnet

Problem: “Configuration not loading from application.conf”

Symptoms: Environment variables work but application.conf is ignored.

Debug:

1
2
3
4
5

import com.typesafe.config.ConfigFactory

val config = ConfigFactory.load()
println(config.hasPath("llm4s.provider"))  // Should be true
println(config.getString("llm4s.provider.model"))  // Should show your model

Common causes:

  • application.conf not in src/main/resources/
  • HOCON syntax error (missing quotes, wrong nesting)
  • Environment variable overriding config (env vars have higher precedence)

Fix: Validate HOCON syntax:

1
2

# Test config loading
sbt "runMain com.typesafe.config.impl.ConfigImpl"

Problem: “Provider mismatch”

Symptoms:

1
2
3

import org.llm4s.error.AuthenticationError

Left(AuthenticationError("Invalid API key"))

Root cause: Model provider doesn’t match API key provider.

1
2
3
4
5
6
7
8

# ✅ Check your configuration
echo $LLM_MODEL        # Should be openai/...
echo $OPENAI_API_KEY   # Should start with sk-proj-...

# ❌ Common mismatch:
LLM_MODEL=openai/gpt-4o
ANTHROPIC_API_KEY=sk-ant-...  # ❌ Wrong! Need OPENAI_API_KEY
OPENAI_API_KEY=              # ❌ Wrong - key is empty

Problem: “Rate limiting / 429 errors”

Symptoms:

1
2
3

import org.llm4s.error.RateLimitError

Left(RateLimitError("Rate limit exceeded"))

Solutions:

  1. Add retry logic: ```scala import scala.concurrent.duration._ import org.llm4s.error.RateLimitError import org.llm4s.types.Result

// Note: This example uses Thread.sleep for simplicity. // In production, prefer scala.concurrent or cats-effect for non-blocking delays. def retryWithBackoffA: Result[A] = { (1 to maxRetries).foldLeft(op) { (result, attempt) => result match { case Left(_: RateLimitError) if attempt < maxRetries => Thread.sleep(Math.pow(2, attempt).toLong * 1000) // Exponential backoff op case other => other } } }

1
2
3
4
5
6
7
8

2. **Use a cheaper model for testing:**
```bash
# Development
LLM_MODEL=openai/gpt-4o-mini  # 60x cheaper

# Production
LLM_MODEL=openai/gpt-4o
  1. Switch to Ollama for development: 
    1

    LLM_MODEL=ollama/llama3.2  # Free, no rate limits

Problem: “Slow responses or timeouts”

Symptoms: Requests take 30+ seconds or timeout.

Solutions:

  1. Increase timeout in application.conf: 
    1
2
3

    llm4s {
  request-timeout = 60 seconds  # Default is 30s
}
  2. Use streaming for long responses: ```scala // Non-streaming: waits for full response client.complete(Conversation(Seq(UserMessage(“Your query”))))

// Streaming: get tokens as they arrive client.streamComplete( Conversation(Seq(UserMessage(“Your query”))) ) { chunk => chunk.content.foreach(print) }

1
2
3
4
5
6
7

3. **Reduce response length:**
```scala
client.complete(
  Conversation(Seq(UserMessage("Your query"))),
  CompletionOptions(maxTokens = Some(500))  // Limit response length
)

Problem: “OutOfMemoryError with embeddings”

Symptoms: JVM crashes when processing large document collections.

Solutions:

  1. Batch embeddings instead of loading all at once: ```scala val documents: List[String] = loadDocuments() val batchSize = 100

val embeddings = documents.grouped(batchSize).flatMap { batch => embedder.embed(batch) match { case Right(emb) => emb case Left(err) => println(s”Batch failed: $err”) List.empty } }.toList

1
2
3
4
5

2. **Increase JVM heap:**
```bash
export SBT_OPTS="-Xmx4G -Xss4M"
sbt run
  1. Use a local embedding model (smaller memory footprint): 
    1

    LLM_EMBEDDING_MODEL=ollama/nomic-embed-text

Next Steps

Configuration complete! Now you can:

  1. Explore features → - Dive into agents, tools, and more
  2. Browse examples → - See configuration in action
  3. User guide → - Learn core concepts
  4. Observability → - Set up tracing

Additional Resources

Ready to build? Explore what’s next →