Example Gallery

Explore 70 working examples covering all LLM4S features.

Table of contents

  1. Quick Navigation
  2. Basic Examples
    1. BasicLLMCallingExample
    2. StreamingExample
    3. AdvancedStreamingExample
    4. BasicLLMCallingWithTrace
    5. EnhancedTracingExample
    6. LangfuseSampleTraceRunner
    7. OllamaExample
    8. OllamaStreamingExample
    9. AgentLLMCallingExample
  3. Agent Examples
    1. SingleStepAgentExample
    2. MultiStepAgentExample
    3. MultiTurnConversationExample
    4. LongConversationExample
    5. ConversationPersistenceExample
    6. MCPAgentExample
    7. AsyncToolAgentExample
    8. BuiltinToolsAgentExample
  4. Tool Examples
    1. WeatherToolExample
    2. LLMWeatherExample
    3. MultiToolExample
    4. ErrorMessageDemonstration
    5. ImprovedErrorMessageDemo
    6. BuiltinToolsExample
    7. ParallelToolExecutionExample
  5. Guardrails Examples
    1. BasicInputValidationExample
    2. CustomGuardrailExample
    3. CompositeGuardrailExample
    4. JSONOutputValidationExample
    5. MultiTurnToneValidationExample
    6. FactualityGuardrailExample
    7. LLMJudgeGuardrailExample
  6. Handoff Examples
    1. SimpleTriageHandoffExample
    2. MathSpecialistHandoffExample
    3. ContextPreservationExample
  7. Memory Examples
    1. BasicMemoryExample
    2. ConversationMemoryExample
    3. MemoryWithAgentExample
    4. SQLiteMemoryExample
    5. VectorMemoryExample
    6. DocumentQAExample (RAG)
  8. Context Management
    1. ContextPipelineExample
  9. Embeddings
    1. EmbeddingExample
  10. MCP Examples
    1. MCPToolExample
  11. Streaming Examples
    1. BasicStreamingExample
    2. StreamingWithProgressExample
    3. StreamingAgentExample
    4. EventCollectionExample
  12. Reasoning Examples
    1. ReasoningModesExample
  13. Model Examples
    1. ModelMetadataExample
  14. Other Examples
    1. Interactive Assistant
    2. Speech
    3. Actions
  15. Running Examples
    1. Basic Run Command
    2. With Environment Variables
    3. Browse Source
  16. Learning Paths
    1. Beginner Path
    2. Intermediate Path
    3. Advanced Path
  17. Next Steps

Quick Navigation

Category Count Description
Basic Examples 9 Getting started, streaming, tracing
Agent Examples 8 Multi-turn agents, persistence, async tools
Tool Examples 7 Tool calling, built-in tools, parallel execution
Guardrails Examples 7 Input/output validation, LLM-as-Judge
Handoff Examples 3 Agent-to-agent delegation
Memory Examples 6 Short/long-term memory, vector search, RAG
Streaming Examples 4 Real-time responses, agent events
Reasoning Examples 1 Extended thinking modes
Context Management 8 Token windows, compression
Embeddings 5 Vector search, RAG
MCP Examples 3 Model Context Protocol
Model Examples 1 Model metadata and capabilities
Other Examples 8 Speech, actions, utilities

Basic Examples

Location: modules/samples/src/main/scala/org/llm4s/samples/basic/

BasicLLMCallingExample

File: BasicLLMCallingExample.scala

Simple multi-turn conversations demonstrating system, user, and assistant messages.

1

sbt "samples/runMain org.llm4s.samples.basic.BasicLLMCallingExample"

What it demonstrates:

  • Creating a conversation with multiple message types
  • System message for setting assistant behavior
  • Multi-turn context with AssistantMessage
  • Token usage tracking
  • Error handling with Result types

View source →

StreamingExample

File: StreamingExample.scala

Compare streaming vs non-streaming responses with performance metrics.

1

sbt "samples/runMain org.llm4s.samples.basic.StreamingExample"

What it demonstrates:

  • Real-time token-by-token output
  • Performance comparison (streaming vs batch)
  • Chunk processing
  • Measuring response times

View source →

AdvancedStreamingExample

File: AdvancedStreamingExample.scala

More complex streaming patterns with error handling and state management.

1

sbt "samples/runMain org.llm4s.samples.basic.AdvancedStreamingExample"

What it demonstrates:

  • Advanced error handling during streaming
  • State management across chunks
  • Progress tracking
  • Stream termination handling

BasicLLMCallingWithTrace

File: BasicLLMCallingWithTrace.scala

Basic LLM calls with integrated tracing for observability.

1
2
3

# Configure tracing
export TRACING_MODE=console
sbt "samples/runMain org.llm4s.samples.basic.BasicLLMCallingWithTrace"

What it demonstrates:

  • Console tracing integration
  • Token usage tracking
  • Request/response logging
  • Performance metrics

EnhancedTracingExample

File: EnhancedTracingExample.scala

Advanced tracing with detailed token usage and agent state tracking.

1
2

export TRACING_MODE=console
sbt "samples/runMain org.llm4s.samples.basic.EnhancedTracingExample"

What it demonstrates:

  • Detailed trace information
  • Agent state tracking
  • Token usage analysis
  • Multi-level tracing

LangfuseSampleTraceRunner

File: LangfuseSampleTraceRunner.scala

Production-grade tracing with Langfuse backend.

1
2
3
4

export TRACING_MODE=langfuse
export LANGFUSE_PUBLIC_KEY=pk-lf-...
export LANGFUSE_SECRET_KEY=sk-lf-...
sbt "samples/runMain org.llm4s.samples.basic.LangfuseSampleTraceRunner"

What it demonstrates:

  • Langfuse integration
  • Production observability
  • Trace persistence
  • Analytics dashboard

Learn more about Langfuse →

OllamaExample

File: OllamaExample.scala

Using local Ollama models instead of cloud providers.

1
2
3
4
5
6
7
8

# Start Ollama
ollama serve &
ollama pull llama2

# Run example
export LLM_MODEL=ollama/llama2
export OLLAMA_BASE_URL=http://localhost:11434
sbt "samples/runMain org.llm4s.samples.basic.OllamaExample"

What it demonstrates:

  • Local model execution
  • No API key required
  • Provider flexibility
  • Cost-free development

OllamaStreamingExample

File: OllamaStreamingExample.scala

Streaming responses with local Ollama models.

1
2

export LLM_MODEL=ollama/llama2
sbt "samples/runMain org.llm4s.samples.basic.OllamaStreamingExample"

What it demonstrates:

  • Local streaming
  • Ollama-specific features
  • Performance characteristics

AgentLLMCallingExample

File: AgentLLMCallingExample.scala

Making LLM calls from within an agent context.

1

sbt "samples/runMain org.llm4s.samples.basic.AgentLLMCallingExample"

What it demonstrates:

  • Agent-based LLM calls
  • Context management
  • Agent state tracking

Agent Examples

Location: modules/samples/src/main/scala/org/llm4s/samples/agent/

SingleStepAgentExample

File: SingleStepAgentExample.scala

Step-by-step agent execution with detailed debugging output.

1

sbt "samples/runMain org.llm4s.samples.agent.SingleStepAgentExample"

What it demonstrates:

  • Manual control over agent execution
  • Debugging agent behavior
  • Step-by-step tool calling
  • State inspection

Perfect for: Understanding how agents work internally

View source →

MultiStepAgentExample

File: MultiStepAgentExample.scala

Complete agent execution from start to finish with automatic tool calling.

1

sbt "samples/runMain org.llm4s.samples.agent.MultiStepAgentExample"

What it demonstrates:

  • Automatic agent execution
  • Tool calling loop
  • Conversation completion
  • Final response generation

Perfect for: Production-ready agent patterns

MultiTurnConversationExample

File: MultiTurnConversationExample.scala

Functional, immutable multi-turn conversation API (Phase 1.1).

1

sbt "samples/runMain org.llm4s.samples.agent.MultiTurnConversationExample"

What it demonstrates:

  • continueConversation() pattern
  • Immutable state management
  • No var or mutation
  • Clean functional style

Key code:

1

val state2 = agent.continueConversation(state1, "Follow-up question")

View source →

LongConversationExample

File: LongConversationExample.scala

Long conversations with automatic context window pruning.

1

sbt "samples/runMain org.llm4s.samples.agent.LongConversationExample"

What it demonstrates:

  • runMultiTurn() helper method
  • Automatic token management
  • Context window pruning strategies
  • Memory-efficient conversations

Key code:

1
2
3
4

val config = ContextWindowConfig(
  maxMessages = Some(20),
  pruningStrategy = PruningStrategy.OldestFirst
)

View source →

ConversationPersistenceExample

File: ConversationPersistenceExample.scala

Save and load agent state for resumable conversations.

1

sbt "samples/runMain org.llm4s.samples.agent.ConversationPersistenceExample"

What it demonstrates:

  • Saving conversation state to disk
  • Loading and resuming conversations
  • JSON serialization
  • Session management

Key code:

1
2

AgentState.saveToFile(state, "/tmp/conversation.json")
val loadedState = AgentState.loadFromFile("/tmp/conversation.json", tools)

View source →

MCPAgentExample

File: MCPAgentExample.scala

Agents with Model Context Protocol (MCP) tool integration.

1

sbt "samples/runMain org.llm4s.samples.agent.MCPAgentExample"

What it demonstrates:

  • MCP tool integration in agents
  • External tool servers
  • Protocol fallback handling

AsyncToolAgentExample

File: AsyncToolAgentExample.scala

Agent with parallel tool execution using different strategies.

1

sbt "samples/runMain org.llm4s.samples.agent.AsyncToolAgentExample"

View source →

BuiltinToolsAgentExample

File: BuiltinToolsAgentExample.scala

Agent using built-in tools (DateTime, Calculator, web search, etc.).

1

sbt "samples/runMain org.llm4s.samples.agent.BuiltinToolsAgentExample"

View source →

Tool Examples

Location: modules/samples/src/main/scala/org/llm4s/samples/toolapi/

WeatherToolExample

File: WeatherToolExample.scala

Simple tool definition and execution.

1

sbt "samples/runMain org.llm4s.samples.toolapi.WeatherToolExample"

What it demonstrates:

  • Basic tool creation with ToolFunction
  • Parameter schema definition
  • Tool execution
  • Return value handling

Key code:

1
2
3
4
5

val weatherTool = ToolFunction(
  name = "get_weather",
  description = "Get current weather for a location",
  function = getWeather _
)

View source →

LLMWeatherExample

File: LLMWeatherExample.scala

Using tools with LLM calls - agent automatically calls tools.

1

sbt "samples/runMain org.llm4s.samples.toolapi.LLMWeatherExample"

What it demonstrates:

  • LLM tool calling
  • Automatic tool selection
  • Tool result integration
  • Natural language to tool execution

MultiToolExample

File: MultiToolExample.scala

Multiple tools with different parameter types.

1

sbt "samples/runMain org.llm4s.samples.toolapi.MultiToolExample"

What it demonstrates:

  • Calculator tool
  • Search tool
  • Multiple tools in one registry
  • Tool precedence and selection

Key code:

1

val tools = new ToolRegistry(Seq(calculatorTool, searchTool))

View source →

ErrorMessageDemonstration

File: ErrorMessageDemonstration.scala

Error handling in tool execution with helpful messages.

1

sbt "samples/runMain org.llm4s.samples.toolapi.ErrorMessageDemonstration"

What it demonstrates:

  • Tool validation errors
  • Helpful error messages
  • Error recovery patterns

ImprovedErrorMessageDemo

File: ImprovedErrorMessageDemo.scala

Enhanced error reporting for better debugging.

1

sbt "samples/runMain org.llm4s.samples.toolapi.ImprovedErrorMessageDemo"

What it demonstrates:

  • Detailed error context
  • Stack traces
  • Debugging information

BuiltinToolsExample

File: BuiltinToolsExample.scala

Using the built-in tools library (DateTime, Calculator, UUID, JSON, HTTP, etc.).

1

sbt "samples/runMain org.llm4s.samples.toolapi.BuiltinToolsExample"

View source →

ParallelToolExecutionExample

File: ParallelToolExecutionExample.scala

Executing multiple tool calls in parallel with different strategies.

1

sbt "samples/runMain org.llm4s.samples.toolapi.ParallelToolExecutionExample"

View source →

Guardrails Examples

Location: modules/samples/src/main/scala/org/llm4s/samples/guardrails/

BasicInputValidationExample

File: BasicInputValidationExample.scala

Basic input validation with built-in guardrails.

1

sbt "samples/runMain org.llm4s.samples.guardrails.BasicInputValidationExample"

What it demonstrates:

  • LengthCheck guardrail for input size validation
  • ProfanityFilter for content filtering
  • Declarative validation before agent processing
  • Clear error messages for validation failures

View source →

CustomGuardrailExample

File: CustomGuardrailExample.scala

Build custom guardrails for application-specific validation.

1

sbt "samples/runMain org.llm4s.samples.guardrails.CustomGuardrailExample"

What it demonstrates:

  • Implementing custom InputGuardrail trait
  • Keyword requirement validation
  • Reusable validation logic
  • Testing validation success and failure cases

Key code:

1
2
3
4
5

class KeywordRequirementGuardrail(requiredKeywords: Set[String]) extends InputGuardrail {
  def validate(value: String): Result[String] = {
    // Custom validation logic
  }
}

View source →

CompositeGuardrailExample

File: CompositeGuardrailExample.scala

Combine multiple guardrails with different composition strategies.

1

sbt "samples/runMain org.llm4s.samples.guardrails.CompositeGuardrailExample"

What it demonstrates:

  • Sequential composition (all must pass in order)
  • All composition (all must pass, run in parallel)
  • Any composition (at least one must pass)
  • Error accumulation and reporting

Key code:

1
2
3
4
5

val allGuardrails = CompositeGuardrail.all(Seq(
  LengthCheck(min = 10, max = 1000),
  ProfanityFilter(),
  customGuardrail
))

View source →

JSONOutputValidationExample

File: JSONOutputValidationExample.scala

Validate LLM outputs are valid JSON.

1

sbt "samples/runMain org.llm4s.samples.guardrails.JSONOutputValidationExample"

What it demonstrates:

  • Output guardrails (run after LLM response)
  • JSON format validation
  • Structured output enforcement
  • Integration with agent workflows

View source →

MultiTurnToneValidationExample

File: MultiTurnToneValidationExample.scala

Validate conversational tone across multiple turns.

1

sbt "samples/runMain org.llm4s.samples.guardrails.MultiTurnToneValidationExample"

What it demonstrates:

  • ToneValidator for output validation
  • Maintaining consistent tone
  • Multi-turn conversation with guardrails
  • Professional/friendly tone enforcement

View source →

FactualityGuardrailExample

File: FactualityGuardrailExample.scala

LLM-as-Judge guardrail for validating factual accuracy of responses.

1

sbt "samples/runMain org.llm4s.samples.guardrails.FactualityGuardrailExample"

View source →

LLMJudgeGuardrailExample

File: LLMJudgeGuardrailExample.scala

Using LLM-as-Judge for content safety, quality, and tone validation.

1

sbt "samples/runMain org.llm4s.samples.guardrails.LLMJudgeGuardrailExample"

View source →

Handoff Examples

Location: modules/samples/src/main/scala/org/llm4s/samples/handoff/

SimpleTriageHandoffExample

File: SimpleTriageHandoffExample.scala

Basic agent-to-agent handoff for routing queries to specialists.

1

sbt "samples/runMain org.llm4s.samples.handoff.SimpleTriageHandoffExample"

View source →

MathSpecialistHandoffExample

File: MathSpecialistHandoffExample.scala

Handoff to a math specialist agent for complex calculations.

1

sbt "samples/runMain org.llm4s.samples.handoff.MathSpecialistHandoffExample"

View source →

ContextPreservationExample

File: ContextPreservationExample.scala

Preserving conversation context when handing off between agents.

1

sbt "samples/runMain org.llm4s.samples.handoff.ContextPreservationExample"

View source →

Memory Examples

Location: modules/samples/src/main/scala/org/llm4s/samples/memory/

BasicMemoryExample

File: BasicMemoryExample.scala

Getting started with the memory system for recording facts and retrieving context.

1

sbt "samples/runMain org.llm4s.samples.memory.BasicMemoryExample"

View source →

ConversationMemoryExample

File: ConversationMemoryExample.scala

Using memory to maintain context across conversation turns.

1

sbt "samples/runMain org.llm4s.samples.memory.ConversationMemoryExample"

View source →

MemoryWithAgentExample

File: MemoryWithAgentExample.scala

Integrating memory with agent workflows for personalized responses.

1

sbt "samples/runMain org.llm4s.samples.memory.MemoryWithAgentExample"

View source →

SQLiteMemoryExample

File: SQLiteMemoryExample.scala

Persistent memory storage using SQLite backend.

1

sbt "samples/runMain org.llm4s.samples.memory.SQLiteMemoryExample"

View source →

VectorMemoryExample

File: VectorMemoryExample.scala

Semantic memory search using embeddings and vector store.

1

sbt "samples/runMain org.llm4s.samples.memory.VectorMemoryExample"

View source →

DocumentQAExample (RAG)

File: DocumentQAExample.scala

Complete RAG (Retrieval-Augmented Generation) pipeline demonstrating document Q&A with semantic search.

1
2
3
4
5
6
7
8
9
10
11

# With mock embeddings (no API key needed for embeddings)
export LLM_MODEL=openai/gpt-4o
export OPENAI_API_KEY=sk-...
sbt "samples/runMain org.llm4s.samples.rag.DocumentQAExample"

# With real OpenAI embeddings
export OPENAI_EMBEDDING_BASE_URL=https://api.openai.com
export OPENAI_EMBEDDING_MODEL=text-embedding-3-small
export LLM_MODEL=openai/gpt-4o
export OPENAI_API_KEY=sk-...
sbt "samples/runMain org.llm4s.samples.rag.DocumentQAExample"

What it demonstrates:

  • Document loading and text extraction
  • Text chunking with configurable size and overlap
  • Embedding generation (mock or real via OpenAI/VoyageAI)
  • Vector storage with SQLite backend
  • Semantic similarity search
  • RAG prompt construction with context
  • Answer generation with source citations

Key code:

1
2
3
4
5
6
7

// Ingest documents
val chunks = ChunkingUtils.chunkText(text, chunkSize = 800, overlap = 150)
store.store(Memory.fromKnowledge(chunk, source = fileName))

// Query with semantic search
val results = store.search(query, topK = 4)
val answer = client.complete(buildRAGPrompt(query, results))

View source →

Context Management

Location: modules/samples/src/main/scala/org/llm4s/samples/context/

All 8 context management examples demonstrate advanced token window management, compression, and optimization strategies.

ContextPipelineExample

End-to-end context management pipeline with compaction and squeezing.

1

sbt "samples/runMain org.llm4s.samples.context.ContextPipelineExample"

View all context examples →

Embeddings

Location: modules/samples/src/main/scala/org/llm4s/samples/embeddingsupport/

EmbeddingExample

Complete embedding pipeline with similarity search and visualization.

1

sbt "samples/runMain org.llm4s.samples.embeddingsupport.EmbeddingExample"

What it demonstrates:

  • Creating embeddings from text
  • Similarity scoring
  • Vector search
  • Result visualization
  • Chunking and preprocessing

View all embedding examples →

MCP Examples

Location: modules/samples/src/main/scala/org/llm4s/samples/mcp/

MCPToolExample

Basic MCP tool usage with automatic protocol fallback.

1

sbt "samples/runMain org.llm4s.samples.mcp.MCPToolExample"

What it demonstrates:

  • MCP server connection
  • Tool discovery
  • Protocol handling (stdio, HTTP, SSE)
  • Tool execution

View all MCP examples →

Streaming Examples

Location: modules/samples/src/main/scala/org/llm4s/samples/streaming/

BasicStreamingExample

Fundamental streaming with chunk processing.

1

sbt "samples/runMain org.llm4s.samples.streaming.BasicStreamingExample"

StreamingWithProgressExample

Streaming with real-time progress feedback.

1

sbt "samples/runMain org.llm4s.samples.streaming.StreamingWithProgressExample"

StreamingAgentExample

File: StreamingAgentExample.scala

Agent with real-time event streaming using runWithEvents().

1

sbt "samples/runMain org.llm4s.samples.streaming.StreamingAgentExample"

View source →

EventCollectionExample

File: EventCollectionExample.scala

Collecting and processing agent execution events.

1

sbt "samples/runMain org.llm4s.samples.streaming.EventCollectionExample"

View source →

Reasoning Examples

Location: modules/samples/src/main/scala/org/llm4s/samples/reasoning/

ReasoningModesExample

File: ReasoningModesExample.scala

Using extended thinking/reasoning modes with OpenAI o1/o3 and Anthropic Claude.

1

sbt "samples/runMain org.llm4s.samples.reasoning.ReasoningModesExample"

View source →

Model Examples

Location: modules/samples/src/main/scala/org/llm4s/samples/model/

ModelMetadataExample

File: ModelMetadataExample.scala

Querying model capabilities, pricing, and context limits.

1

sbt "samples/runMain org.llm4s.samples.model.ModelMetadataExample"

View source →

Other Examples

Interactive Assistant

Location: modules/samples/src/main/scala/org/llm4s/samples/assistant/

AssistantAgentExample - Interactive terminal assistant with session management.

1

sbt "samples/runMain org.llm4s.samples.assistant.AssistantAgentExample"

Commands: /help, /new, /save, /sessions, /quit

Speech

Location: modules/samples/src/main/scala/org/llm4s/samples/

SpeechSamples - Speech-to-text (Vosk, Whisper) and text-to-speech integration.

1

sbt "samples/runMain org.llm4s.samples.SpeechSamples"

Actions

Location: modules/samples/src/main/scala/org/llm4s/samples/actions/

SummarizationExample - Text summarization workflow.

1

sbt "samples/runMain org.llm4s.samples.actions.SummarizationExample"

Running Examples

Basic Run Command 

1

sbt "samples/runMain <fully-qualified-class-name>"

With Environment Variables 

1
2
3

export LLM_MODEL=openai/gpt-4o
export OPENAI_API_KEY=sk-...
sbt "samples/runMain org.llm4s.samples.basic.BasicLLMCallingExample"

Browse Source

All examples are in the samples directory on GitHub.

Learning Paths

Beginner Path

  1. BasicLLMCallingExample
  2. StreamingExample
  3. WeatherToolExample
  4. SingleStepAgentExample

Intermediate Path

  1. MultiTurnConversationExample
  2. MultiToolExample
  3. LongConversationExample
  4. ConversationPersistenceExample

Advanced Path

  1. ContextPipelineExample
  2. EmbeddingExample
  3. MCPToolExample
  4. Interactive Assistant

Next Steps

Found a useful example pattern? Share it in our Discord!