Example Gallery

Explore 70 working examples covering all LLM4S features.

Table of contents

  1. Start with the cookbook
  2. Quick Navigation
  3. Java Offline Example
  4. Java AI Service Example
  5. Playground
    1. CliPlaygroundDemo
  6. Basic Examples
    1. BasicLLMCallingExample
    2. StreamingExample
    3. AdvancedStreamingExample
    4. BasicLLMCallingWithTrace
    5. EnhancedTracingExample
    6. LangfuseSampleTraceRunner
    7. OllamaExample
    8. OllamaStreamingExample
    9. AgentLLMCallingExample
    10. ProviderFallbackExample
    11. ProviderDiscoveryExample
  7. Agent Examples
    1. SingleStepAgentExample
    2. MultiStepAgentExample
    3. MultiTurnConversationExample
    4. LongConversationExample
    5. ConversationPersistenceExample
    6. MCPAgentExample
    7. BuiltinToolsAgentExample
    8. DeepResearchExample
    9. CodingWorkspaceExample
    10. ParallelDeepResearchExample
  8. Tool Examples
    1. WeatherToolExample
    2. MultiToolExample
    3. ErrorMessageDemonstration
    4. ImprovedErrorMessageDemo
    5. BuiltinToolsExample
    6. ParallelToolExecutionExample
  9. Guardrails Examples
    1. BasicInputValidationExample
    2. CustomGuardrailExample
    3. CompositeGuardrailExample
    4. JSONOutputValidationExample
    5. MultiTurnToneValidationExample
    6. FactualityGuardrailExample
    7. LLMJudgeGuardrailExample
  10. Handoff Examples
    1. SimpleTriageHandoffExample
    2. MathSpecialistHandoffExample
    3. ContextPreservationExample
  11. Memory Examples
    1. BasicMemoryExample
    2. ConversationMemoryExample
    3. MemoryWithAgentExample
    4. SQLiteMemoryExample
    5. VectorMemoryExample
    6. DocumentQAExample (RAG)
  12. Context Management
    1. ContextPipelineExample
  13. Embeddings
    1. EmbeddingExample
    2. EvalDatasetExample
    3. HybridSearchExample
    4. S3LoaderExample
    5. DocumentExtractionInspector
  14. RAG in a Box
  15. MCP Examples
    1. MCPToolExample
  16. Streaming Examples
    1. BasicStreamingExample
    2. StreamingWithProgressExample
    3. Agent event streams
  17. Reasoning Examples
    1. ReasoningModesExample
  18. Model Examples
    1. ModelMetadataExample
    2. CostTrackingExample
    3. MultiProviderComparisonExample
  19. Other Examples
    1. Interactive Assistant
    2. Speech
    3. Actions
  20. Running Examples
    1. Basic Run Command
    2. Choosing a Provider
    3. Browse Source
  21. Learning Paths
    1. Beginner Path
    2. Intermediate Path
    3. Advanced Path
  22. Next Steps

Start with the cookbook

The cookbook has complete programs for the things you would actually build, one page each, with the problem, the program, how to run it, what to change for a real provider and the pitfalls. Each runs with no API key and is run in CI:

Quick Navigation

Category Count Description
Playground 1 Offline demo — no API key needed
Java Offline Example 1 Test application code with a scripted model
Basic Examples 9 Getting started, streaming, tracing
Agent Examples 10 Multi-turn agents, persistence, built-in tools, the harness exit examples
Tool Examples 7 Tool calling, built-in tools, parallel execution
Guardrails Examples 7 Input/output validation, LLM-as-Judge
Handoff Examples 3 Agent-to-agent delegation
Memory Examples 6 Short/long-term memory, vector search, RAG
Streaming Examples 9 Real-time responses, agent event streams
Reasoning Examples 1 Extended thinking modes
Context Management 8 Token windows, compression
Embeddings 7 Vector search, RAG, document extraction
RAG in a Box - Production RAG server (external project)
MCP Examples 3 Model Context Protocol
Model Examples 1 Model metadata and capabilities
Other Examples 8 Speech, actions, utilities

Java Offline Example

File: JavaOfflineExample.java

A support-ticket classifier accepts a JLlmClient and runs against FakeLlmClient, without an API key or network access. It demonstrates a BILLING reply, an unknown reply falling back to OTHER, and a RATE_LIMIT failure handled through LlmResult. The program also prints the prompts recorded by the fake.

1
2

sbt "samples/runMain org.llm4s.samples.java.JavaOfflineExample"
sbt "samples/testOnly org.llm4s.samples.java.JavaOfflineExampleSpec"

See Testing your Java code for scripting replies, failures and checking calls.

Java AI Service Example

File: JavaAiServiceExample.java

An annotated SupportAssistant interface that Llm4s.aiService implements: a templated system message, an enum and a record as return types, an LlmResult for explicit failure handling and a @MemoryId conversation, all answered by FakeLlmClient, without an API key or network access.

1
2

sbt "samples/runMain org.llm4s.samples.java.JavaAiServiceExample"
sbt "samples/testOnly org.llm4s.samples.java.JavaAiServiceExampleSpec"

See Java AI services for the annotations, return types, memory and tools.

Playground

Location: modules/samples/src/main/scala/org/llm4s/samples/playground/

CliPlaygroundDemo

File: CliPlaygroundDemo.scala

Fully offline playground demo — no API key or network access required. Replays 6 pre-recorded scenarios with ANSI-coloured output, then drops into an interactive prompt loop so you can type your own queries and see simulated traced responses in real time.

1

sbt "samples/runMain org.llm4s.samples.playground.CliPlaygroundDemo"

What it demonstrates:

  • Simple text completion
  • Multi-turn conversation
  • Tool calling
  • Agent pipeline (multi-step orchestration)
  • Error handling and recovery
  • Token usage awareness
  • Interactive REPL-style prompt loop (Scenario 7)

Perfect for: First-time users and workshop attendees exploring LLM4S before wiring up a real provider

View source →

Basic Examples

Location: modules/samples/src/main/scala/org/llm4s/samples/basic/

BasicLLMCallingExample

File: BasicLLMCallingExample.scala

Simple multi-turn conversations demonstrating system, user, and assistant messages.

1

sbt "samples/runMain org.llm4s.samples.basic.BasicLLMCallingExample"

What it demonstrates:

  • Creating a conversation with multiple message types
  • System message for setting assistant behavior
  • Multi-turn context with AssistantMessage
  • Token usage tracking
  • Error handling with Result types

View source →

StreamingExample

File: StreamingExample.scala

Compare streaming vs non-streaming responses with performance metrics.

1

sbt "samples/runMain org.llm4s.samples.basic.StreamingExample"

What it demonstrates:

  • Real-time token-by-token output
  • Performance comparison (streaming vs batch)
  • Chunk processing
  • Measuring response times

View source →

AdvancedStreamingExample

File: AdvancedStreamingExample.scala

More complex streaming patterns with error handling and state management.

1

sbt "samples/runMain org.llm4s.samples.basic.AdvancedStreamingExample"

What it demonstrates:

  • Advanced error handling during streaming
  • State management across chunks
  • Progress tracking
  • Stream termination handling

BasicLLMCallingWithTrace

File: BasicLLMCallingWithTrace.scala

Basic LLM calls with integrated tracing for observability.

1
2
3

# Configure tracing
export TRACING_MODE=console
sbt "samples/runMain org.llm4s.samples.basic.BasicLLMCallingWithTrace"

What it demonstrates:

  • Console tracing integration
  • Token usage tracking
  • Request/response logging
  • Performance metrics

EnhancedTracingExample

File: EnhancedTracingExample.scala

Advanced tracing with detailed token usage and agent state tracking.

1
2

export TRACING_MODE=console
sbt "samples/runMain org.llm4s.samples.basic.EnhancedTracingExample"

What it demonstrates:

  • Detailed trace information
  • Agent state tracking
  • Token usage analysis
  • Multi-level tracing

LangfuseSampleTraceRunner

File: LangfuseSampleTraceRunner.scala

Production-grade tracing with Langfuse backend.

1
2
3
4

export TRACING_MODE=langfuse
export LANGFUSE_PUBLIC_KEY=pk-lf-...
export LANGFUSE_SECRET_KEY=sk-lf-...
sbt "samples/runMain org.llm4s.samples.basic.LangfuseSampleTraceRunner"

What it demonstrates:

  • Langfuse integration
  • Production observability
  • Trace persistence
  • Analytics dashboard

Learn more about Langfuse →

OllamaExample

File: OllamaExample.scala

Using local Ollama models instead of cloud providers.

1
2
3
4
5
6
7
8

# Start Ollama
ollama serve &
ollama pull llama2

# Run example - the samples' default section is ollama-local;
# OLLAMA_MODEL and OLLAMA_BASE_URL are bound by the samples' application.conf
export OLLAMA_MODEL=llama2
sbt "samples/runMain org.llm4s.samples.basic.OllamaExample"

What it demonstrates:

  • Local model execution
  • No API key required
  • Provider flexibility
  • Cost-free development

OllamaStreamingExample

File: OllamaStreamingExample.scala

Streaming responses with local Ollama models.

1
2

export OLLAMA_MODEL=llama2   # read by the samples' application.conf
sbt "samples/runMain org.llm4s.samples.basic.OllamaStreamingExample"

To enable raw provider exchange logging and write timestamped JSONL files to a specific directory:

1
2

export OLLAMA_MODEL=llama2
sbt "samples/runMain org.llm4s.samples.basic.OllamaStreamingExample /tmp/my-provider-exchanges"

What it demonstrates:

  • Local streaming
  • Ollama-specific features
  • Performance characteristics
  • Raw provider exchange logging to JSONL

AgentLLMCallingExample

File: AgentLLMCallingExample.scala

Making LLM calls from within an agent context.

1

sbt "samples/runMain org.llm4s.samples.basic.AgentLLMCallingExample"

What it demonstrates:

  • Agent-based LLM calls
  • Context management
  • Agent state tracking

ProviderFallbackExample

File: ProviderFallbackExample.scala

LLM calls with automatic provider fallback for reliability.

1

sbt "samples/runMain org.llm4s.samples.basic.ProviderFallbackExample"

What it demonstrates:

  • Multiple provider configurations
  • Automatic fallback on failure
  • Enhanced reliability

ProviderDiscoveryExample

File: ProviderDiscoveryExample.scala

Inspect providers and embedding providers discovered from the classpath, and compare with explicit registration.

1

sbt "samples/runMain org.llm4s.samples.basic.ProviderDiscoveryExample"

What it demonstrates:

  • Dynamic provider discovery via ProviderRegistry.discover()
  • Inspecting ProviderRegistryReport (report.describe, ids, embeddingIds)
  • Contrasting classpath discovery with explicit module registration (ProviderRegistry.ofModules)
  • Diagnosing missing providers, collisions, and load failures without API keys or network access

Agent Examples

Location: modules/samples/src/main/scala/org/llm4s/samples/agent/

SingleStepAgentExample

File: SingleStepAgentExample.scala

A plain agent run with a step limit, printing the messages the run produced. (For step-level events, see the streaming examples.)

1

sbt "samples/runMain org.llm4s.samples.agent.SingleStepAgentExample"

What it demonstrates:

  • Agent.builder(...).withMaxSteps(n)
  • Tool calling in a run
  • Reading AgentResult.messages

Perfect for: Understanding what a run produces

View source →

MultiStepAgentExample

File: MultiStepAgentExample.scala

Complete agent execution from start to finish with automatic tool calling.

1

sbt "samples/runMain org.llm4s.samples.agent.MultiStepAgentExample"

What it demonstrates:

  • Automatic agent execution
  • Tool calling loop
  • Conversation completion
  • Final response generation

Perfect for: Production-ready agent patterns

MultiTurnConversationExample

File: MultiTurnConversationExample.scala

Functional, immutable multi-turn conversation API (Phase 1.1).

1

sbt "samples/runMain org.llm4s.samples.agent.MultiTurnConversationExample"

What it demonstrates:

  • continueConversation() pattern
  • Immutable results, a conversation carried by its thread
  • No var or mutation
  • Clean functional style

Key code:

1

val result2 = agent.continueConversation(result1, "Follow-up question")

View source →

LongConversationExample

File: LongConversationExample.scala

Long conversations with automatic context window pruning.

1

sbt "samples/runMain org.llm4s.samples.agent.LongConversationExample"

What it demonstrates:

  • runMultiTurn() helper method
  • Automatic token management
  • Context window pruning strategies
  • Memory-efficient conversations

Key code:

1
2
3
4

val config = ContextWindowConfig(
  maxMessages = Some(20),
  pruningStrategy = PruningStrategy.OldestFirst
)

View source →

ConversationPersistenceExample

File: ConversationPersistenceExample.scala

Save a run’s messages and load them as the history of a new thread.

1

sbt "samples/runMain org.llm4s.samples.agent.ConversationPersistenceExample"

What it demonstrates:

  • Saving a completed run’s messages to disk
  • Loading them as history on a new thread
  • JSON serialization
  • Session management

Key code:

1
2

Files.writeString(path, write(result.messages))
agent.run(newThreadId, "Next question", RunConfig(), history = loaded)

View source →

MCPAgentExample

File: MCPAgentExample.scala

Agents with Model Context Protocol (MCP) tool integration.

1

sbt "samples/runMain org.llm4s.samples.agent.MCPAgentExample"

What it demonstrates:

  • MCP tool integration in agents
  • External tool servers
  • Protocol fallback handling

BuiltinToolsAgentExample

File: BuiltinToolsAgentExample.scala

Agent using built-in tools (DateTime, Calculator, web search, etc.).

1

sbt "samples/runMain org.llm4s.samples.agent.BuiltinToolsAgentExample"

View source →

DeepResearchExample

File: DeepResearchExample.scala

The first harness exit example (design §4.38): HarnessProfile.deepAgents on state files, with a skill and an AGENTS.md the application ships (StaticFileBackend), permission rules, offload of a long fetched page, summarisation, semantic memory, a typed answer and the SQLite checkpointer. Its first “process” stops once the first tool result is committed; the second recovers the run from the same file and prints the report the agent wrote.

1

sbt "samples/runMain org.llm4s.samples.harness.DeepResearchExample"

What it demonstrates:

  • Tool policy: reads anywhere, writes under /research/ and to /report.md, fetches from one site
  • Context management: offload to /large_tool_results, read back by path; summarisation
  • Streaming: the plan (TodosUpdated) and the answer (TextDelta) through EventHandlers
  • Checkpoint and recover in a fresh runtime; persisted artifacts read back from SQLite

DeepResearchAcceptanceSpec runs the same agent on a scripted model, with a crash at each point that matters.

CodingWorkspaceExample

File: CodingWorkspaceExample.scala

The second harness exit example: a coding agent in a workspace container (WorkspaceFileBackend, execute), under rules that allow writes under /src/ and ask before execute. The run suspends when the agent asks to run its check; a fresh runtime over the same SQLite file shows you the command, asks you to approve it, and resumes the run with your answer: an approved command runs once, a rejected one not at all. Needs Docker and the workspace-runner image (sbt workspaceRunner/Docker/publishLocal).

1

sbt "samples/runMain org.llm4s.samples.harness.CodingWorkspaceExample llm4s/workspace-runner:<tag>"

CodingWorkspaceAcceptanceSpec runs the same agent on a scripted model over an in-memory workspace; CodingWorkspaceIntegrationSpec (sbt testWorkspace) runs the scenario against the container.

ParallelDeepResearchExample

File: ParallelDeepResearchExample.scala

The subagents exit example (design §4.49): deep research ported to parallel specialists. A lead on HarnessProfile.deepAgents gives each question to a typed researcher subagent (granted the lead’s fetch_page), all in one message, at most two at once, and has a critic subagent review the draft before it answers with a typed brief. Its first “process” stops once the first child’s report is committed; the second recovers the lead from the same SQLite file without running the finished child again, and asks you about any fetch outside the allowed site.

1

sbt "samples/runMain org.llm4s.samples.harness.ParallelDeepResearchExample"

What it demonstrates:

  • TypedAgent[I, O] specialists through task_<id> tools: typed input and typed reports, nothing else crosses
  • DelegationConfig(maxConcurrentChildren = 2) bounding a batch of three
  • A child’s approval suspending the lead, answered through Agent.resume; a child’s failure as a tool error
  • The merged stream (nested = true): TaskStarted, each child’s text, TaskEnded
  • recover in a fresh runtime that reuses finished children, and forget deleting every child thread

ParallelDeepResearchAcceptanceSpec runs the same agent on a scripted model: parallel, crash, failure, approval, cancel, streams and cleanup.

Tool Examples

Location: modules/samples/src/main/scala/org/llm4s/samples/toolapi/

WeatherToolExample

File: WeatherToolExample.scala

Simple tool definition and execution.

1

sbt "samples/runMain org.llm4s.samples.toolapi.WeatherToolExample"

What it demonstrates:

  • Basic tool creation with ToolFunction
  • Parameter schema definition
  • Tool execution
  • Return value handling

Key code:

1
2
3
4
5

val weatherTool = ToolFunction(
  name = "get_weather",
  description = "Get current weather for a location",
  function = getWeather _
)

View source →

MultiToolExample

File: MultiToolExample.scala

Multiple tools with different parameter types.

1

sbt "samples/runMain org.llm4s.samples.toolapi.MultiToolExample"

What it demonstrates:

  • Calculator tool
  • Search tool
  • Multiple tools in one registry
  • Tool precedence and selection

Key code:

1

val tools = new ToolRegistry(Seq(calculatorTool, searchTool))

View source →

ErrorMessageDemonstration

File: ErrorMessageDemonstration.scala

Error handling in tool execution with helpful messages.

1

sbt "samples/runMain org.llm4s.samples.toolapi.ErrorMessageDemonstration"

What it demonstrates:

  • Tool validation errors
  • Helpful error messages
  • Error recovery patterns

ImprovedErrorMessageDemo

File: ImprovedErrorMessageDemo.scala

Enhanced error reporting for better debugging.

1

sbt "samples/runMain org.llm4s.samples.toolapi.ImprovedErrorMessageDemo"

What it demonstrates:

  • Detailed error context
  • Stack traces
  • Debugging information

BuiltinToolsExample

File: BuiltinToolsExample.scala

Using the built-in tools library (DateTime, Calculator, UUID, JSON, HTTP, etc.).

1

sbt "samples/runMain org.llm4s.samples.toolapi.BuiltinToolsExample"

View source →

ParallelToolExecutionExample

File: ParallelToolExecutionExample.scala

Executing multiple tool calls in parallel with different strategies.

1

sbt "samples/runMain org.llm4s.samples.toolapi.ParallelToolExecutionExample"

View source →

Guardrails Examples

Location: modules/samples/src/main/scala/org/llm4s/samples/guardrails/

BasicInputValidationExample

File: BasicInputValidationExample.scala

Basic input validation with built-in guardrails.

1

sbt "samples/runMain org.llm4s.samples.guardrails.BasicInputValidationExample"

What it demonstrates:

  • LengthCheck guardrail for input size validation
  • ProfanityFilter for content filtering
  • Declarative validation before agent processing
  • Clear error messages for validation failures

View source →

CustomGuardrailExample

File: CustomGuardrailExample.scala

Build custom guardrails for application-specific validation.

1

sbt "samples/runMain org.llm4s.samples.guardrails.CustomGuardrailExample"

What it demonstrates:

  • Implementing custom InputGuardrail trait
  • Keyword requirement validation
  • Reusable validation logic
  • Testing validation success and failure cases

Key code:

1
2
3
4
5

class KeywordRequirementGuardrail(requiredKeywords: Set[String]) extends InputGuardrail {
  def validate(value: String): Result[String] = {
    // Custom validation logic
  }
}

View source →

CompositeGuardrailExample

File: CompositeGuardrailExample.scala

Combine multiple guardrails with different composition strategies.

1

sbt "samples/runMain org.llm4s.samples.guardrails.CompositeGuardrailExample"

What it demonstrates:

  • Sequential composition (all must pass in order)
  • All composition (all must pass, run in parallel)
  • Any composition (at least one must pass)
  • Error accumulation and reporting

Key code:

1
2
3
4
5

val allGuardrails = CompositeGuardrail.all(Seq(
  LengthCheck(min = 10, max = 1000),
  ProfanityFilter(),
  customGuardrail
))

View source →

JSONOutputValidationExample

File: JSONOutputValidationExample.scala

Validate LLM outputs are valid JSON.

1

sbt "samples/runMain org.llm4s.samples.guardrails.JSONOutputValidationExample"

What it demonstrates:

  • Output guardrails (run after LLM response)
  • JSON format validation
  • Structured output enforcement
  • Integration with agent workflows

View source →

MultiTurnToneValidationExample

File: MultiTurnToneValidationExample.scala

Validate conversational tone across multiple turns.

1

sbt "samples/runMain org.llm4s.samples.guardrails.MultiTurnToneValidationExample"

What it demonstrates:

  • ToneValidator for output validation
  • Maintaining consistent tone
  • Multi-turn conversation with guardrails
  • Professional/friendly tone enforcement

View source →

FactualityGuardrailExample

File: FactualityGuardrailExample.scala

LLM-as-Judge guardrail for validating factual accuracy of responses.

1

sbt "samples/runMain org.llm4s.samples.guardrails.FactualityGuardrailExample"

View source →

LLMJudgeGuardrailExample

File: LLMJudgeGuardrailExample.scala

Using LLM-as-Judge for content safety, quality, and tone validation.

1

sbt "samples/runMain org.llm4s.samples.guardrails.LLMJudgeGuardrailExample"

View source →

Handoff Examples

Location: modules/samples/src/main/scala/org/llm4s/samples/handoff/

SimpleTriageHandoffExample

File: SimpleTriageHandoffExample.scala

Basic agent-to-agent handoff for routing queries to specialists.

1

sbt "samples/runMain org.llm4s.samples.handoff.SimpleTriageHandoffExample"

View source →

MathSpecialistHandoffExample

File: MathSpecialistHandoffExample.scala

Handoff to a math specialist agent for complex calculations.

1

sbt "samples/runMain org.llm4s.samples.handoff.MathSpecialistHandoffExample"

View source →

ContextPreservationExample

File: ContextPreservationExample.scala

Preserving conversation context when handing off between agents.

1

sbt "samples/runMain org.llm4s.samples.handoff.ContextPreservationExample"

View source →

Memory Examples

Location: modules/samples/src/main/scala/org/llm4s/samples/memory/

BasicMemoryExample

File: BasicMemoryExample.scala

Getting started with the memory system for recording facts and retrieving context.

1

sbt "samples/runMain org.llm4s.samples.memory.BasicMemoryExample"

View source →

ConversationMemoryExample

File: ConversationMemoryExample.scala

Using memory to maintain context across conversation turns.

1

sbt "samples/runMain org.llm4s.samples.memory.ConversationMemoryExample"

View source →

MemoryWithAgentExample

File: MemoryWithAgentExample.scala

Integrating memory with agent workflows for personalized responses.

1

sbt "samples/runMain org.llm4s.samples.memory.MemoryWithAgentExample"

View source →

SQLiteMemoryExample

File: SQLiteMemoryExample.scala

Persistent memory storage using SQLite backend.

1

sbt "samples/runMain org.llm4s.samples.memory.SQLiteMemoryExample"

View source →

VectorMemoryExample

File: VectorMemoryExample.scala

Semantic memory search using embeddings and vector store.

1

sbt "samples/runMain org.llm4s.samples.memory.VectorMemoryExample"

View source →

DocumentQAExample (RAG)

File: DocumentQAExample.scala

Complete RAG (Retrieval-Augmented Generation) pipeline demonstrating document Q&A with semantic search.

1
2
3
4
5
6
7
8
9

# With mock embeddings (no API key needed for embeddings), answering with
# the default ollama-local section - or select another, see "Running Examples"
sbt "samples/runMain org.llm4s.samples.rag.DocumentQAExample"

# With real OpenAI embeddings: EMBEDDING_MODEL is bound by llm4s-core, and
# OPENAI_API_KEY by llm4s-openai (the key OpenAI chat sections use too)
export EMBEDDING_MODEL=openai/text-embedding-3-small
export OPENAI_API_KEY=sk-...
sbt "samples/runMain org.llm4s.samples.rag.DocumentQAExample"

What it demonstrates:

  • Document loading and text extraction
  • Text chunking with configurable size and overlap
  • Embedding generation (mock or real via OpenAI/VoyageAI)
  • Vector storage with SQLite backend
  • Semantic similarity search
  • RAG prompt construction with context
  • Answer generation with source citations

Key code:

1
2
3
4
5
6
7

// Ingest documents
val chunks = ChunkingUtils.chunkText(text, chunkSize = 800, overlap = 150)
store.store(Memory.fromKnowledge(chunk, source = fileName))

// Query with semantic search
val results = store.search(query, topK = 4)
val answer = client.complete(buildRAGPrompt(query, results))

View source →

Context Management

Location: modules/samples/src/main/scala/org/llm4s/samples/context/

All 8 context management examples demonstrate advanced token window management, compression, and optimization strategies.

ContextPipelineExample

End-to-end context management pipeline with compaction and squeezing.

1

sbt "samples/runMain org.llm4s.samples.context.ContextPipelineExample"

View all context examples →

Embeddings

Location: modules/samples/src/main/scala/org/llm4s/samples/embeddingsupport/

EmbeddingExample

Complete embedding pipeline with similarity search and visualization.

1

sbt "samples/runMain org.llm4s.samples.embeddingsupport.EmbeddingExample"

What it demonstrates:

  • Creating embeddings from text
  • Similarity scoring
  • Vector search
  • Result visualization
  • Chunking and preprocessing

View all embedding examples →

EvalDatasetExample

EvalDatasetExample.scala builds a golden set of evaluation questions without an API key. It validates inputs before storing them, selects questions by tag and ID, and shows how a snapshot stays fixed as the dataset grows. A JSONL round trip ends with five imported examples and one skipped line.

1

sbt "samples/runMain org.llm4s.samples.rag.EvalDatasetExample"

File: HybridSearchExample.scala

Runs one query through HybridSearcher with each fusion strategy (VectorOnly, KeywordOnly, RRF, WeightedScore) and prints the rank, id and scores of each result. Fully offline: the embeddings are hand-written 3-dimensional vectors, so no API key or network is needed.

1

sbt "samples/runMain org.llm4s.samples.rag.HybridSearchExample"

What it demonstrates:

  • HybridSearcher.inMemory() and indexing a document into both the vector store and the keyword index
  • A query where vector and keyword search disagree about the best document
  • How RRF and weighted-score fusion reconcile the two lists, and how the weights move the winner

View source →

S3LoaderExample

File: S3LoaderExample.scala

Load and ingest documents from AWS S3 buckets with full PDF/DOCX support.

1
2
3
4
5
6
7

# First, set AWS credentials
export AWS_ACCESS_KEY_ID=your-key
export AWS_SECRET_ACCESS_KEY=your-secret
# and an embedding model, as provider/model
export EMBEDDING_MODEL=ollama/nomic-embed-text

sbt "samples/runMain org.llm4s.samples.rag.S3LoaderExample"

What it demonstrates:

  • Loading documents from S3 buckets
  • PDF and DOCX extraction from cloud storage
  • Incremental sync with change detection via ETags
  • Using S3Loader with RAG.sync()
  • LocalStack testing setup

Key Code:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

import org.llm4s.rag.loader.s3.S3Loader

// Load all supported documents from S3
val loader = S3Loader(
  bucket = "my-documents",
  prefix = "docs/",
  region = "us-east-1"
)

// Sync with change detection
rag.sync(loader) match {
  case Right(stats) =>
    println(s"Added: ${stats.added}, Updated: ${stats.updated}")
  case Left(err) =>
    // The bucket could not be listed: no credentials, missing bucket, access denied.
    // Nothing was deleted from the index.
    println(s"Error: ${err.message}")
}

A listing that fails - on the first page or a later one - is a Left, never a successful sync of 0 documents; an empty bucket is Right with 0. An object that is listed but cannot be read (a transient GetObject error) keeps its indexed version rather than being deleted as gone. With no AWS credentials the example ends with:

1
2
3
4
5

Sync failed: Failed to list S3 objects from s3://my-documents/docs/: Unable to load credentials from any of the providers in the chain ...
Common issues:
  - Check AWS credentials are configured
  ...
Skipping the example query: the index was not synced.

View source →

DocumentExtractionInspector

File: DocumentExtractionInspector.scala

Shows what llm4s-rag’s TikaDocumentExtractor makes of a file or a directory of files, so you can check extraction quality before indexing. For each file it prints the MIME type Tika detects, the MediaType it resolves to, the extractor used (PDFBox, POI, UTF-8 text or the Tika fallback), the character and line counts, the extractor’s metadata and a 200-character preview. A file that cannot be extracted is reported with its error and the run carries on. Fully offline: no API key or network.

1
2
3
4
5
6

# No argument: writes sample .txt, .md, .html, .docx and .pdf files (and one corrupt PDF) to a temp dir
sbt "samples/runMain org.llm4s.samples.rag.DocumentExtractionInspector"

# A single file, or every file under a directory
sbt "samples/runMain org.llm4s.samples.rag.DocumentExtractionInspector /path/to/report.pdf"
sbt "samples/runMain org.llm4s.samples.rag.DocumentExtractionInspector /path/to/docs"

What it demonstrates:

  • TikaDocumentExtractor.detectMimeType and extract, and MediaType.fromMimeType from llm4s-media
  • That detection reads content before the file name: a .pdf holding plain text is detected as text/plain
  • Extraction failures surfacing as Result values rather than exceptions

Sample output for the generated DOCX:

1
2
3
4
5
6
7
8
9

== /tmp/llm4s-extraction-.../report.docx
   MIME type : application/vnd.openxmlformats-officedocument.wordprocessingml.document
   Media type: unrecognised
   Extractor : Apache POI (XWPF)
   Format    : application/vnd.openxmlformats-officedocument.wordprocessingml.document
   Characters: 46
   Lines     : 2
   Metadata  : byteLength=2294, filename=report.docx, mimeType=application/vnd.openxmlformats-officedocument.wordprocessingml.document
   Preview   : Quarterly report Revenue grew in every region.

View source →

RAG in a Box

Repository: github.com/llm4s/rag_in_a_box

RAG in a Box is a production-ready RAG server built on the LLM4S framework. It provides a complete solution for document ingestion, semantic search, and AI-powered question answering.

Key Features:

  • REST API for document management and querying
  • Multi-format document support (text, markdown, PDF, URLs)
  • Configurable chunking strategies (simple, sentence, markdown, semantic)
  • Hybrid search with RRF fusion (vector + keyword)
  • Vue.js admin dashboard with document browser and analytics
  • Docker Compose and Kubernetes deployment options
  • JWT authentication, Prometheus metrics, health checks

Current Status:

  • 194 backend tests, 8 frontend E2E specs
  • Security scanning (OWASP, Anchore)
  • Comprehensive documentation

View RAG in a Box →

MCP Examples

Location: modules/samples/src/main/scala/org/llm4s/samples/mcp/

MCPToolExample

Basic MCP tool usage with automatic protocol fallback.

1

sbt "samples/runMain org.llm4s.samples.mcp.MCPToolExample"

What it demonstrates:

  • MCP server connection
  • Tool discovery
  • Protocol handling (stdio, HTTP, SSE)
  • Tool execution

View all MCP examples →

Streaming Examples

Location: modules/samples/src/main/scala/org/llm4s/samples/streaming/

BasicStreamingExample

Fundamental streaming with chunk processing.

1

sbt "samples/runMain org.llm4s.samples.streaming.BasicStreamingExample"

StreamingWithProgressExample

Streaming with real-time progress feedback.

1

sbt "samples/runMain org.llm4s.samples.streaming.StreamingWithProgressExample"

Agent event streams

agent.stream with withStreaming(): text deltas live, model and tool events as they commit. See the streaming guide.

Example Shows
StreamingAgentExample withStreaming(), printing TextDelta, attempt resets
StreamingWithToolsExample ToolCallStarted/ToolCallResult live, ToolExecuted durable with duration
EventCollectionExample collecting one run’s events, replaying the durable ones
AgentStreamIOExample (samples/catseffect) fs2 stream of AgentStreamItem
AgentStreamZIOExample (samples/zio) ZStream of AgentStreamItem 
1

sbt "samples/runMain org.llm4s.samples.streaming.StreamingAgentExample"

Reasoning Examples

Location: modules/samples/src/main/scala/org/llm4s/samples/reasoning/

ReasoningModesExample

File: ReasoningModesExample.scala

Using extended thinking/reasoning modes with OpenAI o1/o3 and Anthropic Claude.

1

sbt "samples/runMain org.llm4s.samples.reasoning.ReasoningModesExample"

View source →

Model Examples

Location: modules/samples/src/main/scala/org/llm4s/samples/model/

ModelMetadataExample

File: ModelMetadataExample.scala

Querying model capabilities, pricing, and context limits.

1

sbt "samples/runMain org.llm4s.samples.model.ModelMetadataExample"

View source →

CostTrackingExample

File: CostTrackingExample.scala

What a call costs, at three levels: per request (Completion.estimatedCost), per agent run (AgentResult.usage after a real Agent.run that calls a tool), and per session (a CostTracker the client reports to). It also shows how to price a model the registry does not know, with a ModelRegistryService built from your own ModelMetadata (an immutable snapshot, so nothing global changes), and how MetricsCollector.compose feeds several collectors from one client. A model with no price shows as unknown, never as a guessed number.

1

sbt "samples/runMain org.llm4s.samples.metrics.CostTrackingExample"

View source →

MultiProviderComparisonExample

File: MultiProviderComparisonExample.scala

The same prompt against several providers, side by side: each one’s answer, the tokens it reported and the latency of the call. Providers are named sections under llm4s.providers in your configuration (openai-main, anthropic-main, gemini-main by default; pass other names as arguments), not provider/model strings. A provider that is not configured, or has no key, is reported with the reason and the others still run.

1
2

sbt "samples/runMain org.llm4s.samples.basic.MultiProviderComparisonExample"
sbt "samples/runMain org.llm4s.samples.basic.MultiProviderComparisonExample openai-main ollama-local"

View source →

Other Examples

Interactive Assistant

Location: modules/samples/src/main/scala/org/llm4s/samples/assistant/

AssistantAgentExample - Interactive terminal assistant with session management.

1

sbt "samples/runMain org.llm4s.samples.assistant.AssistantAgentExample"

Commands: /help, /new, /save, /sessions, /quit

Speech

Location: modules/samples/src/main/scala/org/llm4s/samples/

SpeechSamples - Speech-to-text (Vosk, Whisper) and text-to-speech integration.

1

sbt "samples/runMain org.llm4s.samples.SpeechSamples"

Actions

Location: modules/samples/src/main/scala/org/llm4s/samples/actions/

SummarizationExample - Text summarization workflow.

1

sbt "samples/runMain org.llm4s.samples.actions.SummarizationExample"

Running Examples

Basic Run Command 

1

sbt "samples/runMain <fully-qualified-class-name>"

Choosing a Provider

The samples load their provider with Llm4sConfig.defaultProvider(), from modules/samples/src/main/resources/application.conf. Nothing reads LLM_MODEL. Out of the box the default is the ollama-local section, using the model llama3:latest; OLLAMA_MODEL and OLLAMA_BASE_URL override it. Pull the default model once, or name one you already have (ollama list):

1
2
3

ollama pull llama3                 # the samples' default model
# or: export OLLAMA_MODEL=llama3.2 # a model already installed
sbt "samples/runMain org.llm4s.samples.basic.BasicLLMCallingExample"

To use a cloud provider, add a section to modules/samples/src/main/resources/application.local.conf (git-ignored, and included by the samples’ application.conf):

1
2
3
4
5
6

llm4s.providers {
  openai-main {
    provider = "openai"
    model    = "gpt-4o"
  }
}

and select it with LLM4S_PROVIDER, which the samples’ application.conf binds to llm4s.providers.provider. The section needs no apiKey: llm4s-openai binds OPENAI_API_KEY to OpenAI’s shared key.

1
2
3

export OPENAI_API_KEY=sk-...
export LLM4S_PROVIDER=openai-main
sbt "samples/runMain org.llm4s.samples.basic.BasicLLMCallingExample"

Only the section being loaded is validated, so a section whose key you have not set does no harm until you select it. See Configuration for more.

The chat-tui sample is the exception: ChatTuiConfig reads LLM_MODEL=<provider>/<model> and the matching API-key variable itself, and falls back to Llm4sConfig.defaultProvider() when LLM_MODEL is unset.

Browse Source

All examples are in the samples directory on GitHub.

Learning Paths

Beginner Path

  1. BasicLLMCallingExample
  2. StreamingExample
  3. WeatherToolExample
  4. SingleStepAgentExample

Intermediate Path

  1. MultiTurnConversationExample
  2. MultiToolExample
  3. LongConversationExample
  4. ConversationPersistenceExample

Advanced Path

  1. ContextPipelineExample
  2. EmbeddingExample
  3. MCPToolExample
  4. Interactive Assistant

Next Steps

Found a useful example pattern? Share it in our Discord!

Table of contents