Your First LLM4S Program

Build your first LLM-powered application in minutes.

Table of contents

  1. Hello, LLM!
    1. Create the File
    2. Run It
    3. Expected Output
  2. Understanding the Code
    1. 1. The Configuration Boundary
    2. 2. Complete with Messages
    3. 3. Result Handling
  3. Adding Conversation Context
    1. Message Types
  4. Adding Tool Calling
  5. Streaming Responses
  6. Error Handling Patterns
    1. Basic Error Handling
    2. Using fold
    3. Mapping Results
  7. Configuration Options
    1. Using Different Models
    2. Provider-Specific Settings
  8. Complete Working Example
  9. Common Patterns
    1. Pattern 1: Simple Query
    2. Pattern 2: With System Prompt
    3. Pattern 3: Agent with Tools
    4. Pattern 4: Streaming
  10. Performance Optimization
    1. 1. Reuse Client Instances
    2. 2. Use Streaming for Long Responses
    3. 3. Parallelize Independent Requests
    4. 4. Set Appropriate Timeouts
    5. 5. Use Cheaper Models for Development
    6. 6. Batch Embeddings
    7. 7. Monitor Token Usage
  11. Running the Examples
  12. Next Steps
  13. Learning Resources

Hello, LLM!

Let’s start with the simplest possible LLM4S program - a “Hello World” that asks the LLM a question.

Create the File

The examples use llm4s 0.5.1 on Scala 3.7.x and JDK 21. Add llm4s-core and the provider module you call, here llm4s-openai (see Installation), to build.sbt:

1
2
3
4
5
6
7

val llm4sVersion = "0.5.1"

libraryDependencies ++= Seq(
  "org.llm4s" %% "llm4s-core"   % llm4sVersion,
  "org.llm4s" %% "llm4s-openai" % llm4sVersion,
  "org.llm4s" %% "llm4s-agent"  % llm4sVersion  // only for the agent examples further down
)

Then create HelloLLM.scala.

It reads the provider from a named section in src/main/resources/application.conf:

1
2
3
4
5
6
7
8
9
10

llm4s {
  providers {
    provider = "openai-main"          # the default: the name of a section below

    openai-main {
      provider = "openai"
      model    = "gpt-4o-mini"
    }
  }
}

The Configuration guide has sections for Anthropic, Gemini, Azure and Ollama; the program below does not change when you switch.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

import org.llm4s.config.Llm4sConfig
import org.llm4s.llmconnect.{LLMClient, LLMConnect}
import org.llm4s.llmconnect.model.{ Conversation, UserMessage }
import org.llm4s.model.ModelRegistryService

// 1. Core Logic: Depends only on the injected client, not configuration
class HelloLLM(client: LLMClient) {
  def sayHello(): Unit = {
    val result = client.complete(Conversation(Seq(UserMessage("What is Scala?"))))

    result match {
      case Right(completion) =>
        println(s"Response: ${completion.content}")
      case Left(error) =>
        println(s"Error: $error")
    }
  }
}

// 2. Configuration Boundary: The application entry point
object Main extends App {
  val startup = for {
    providerConfig <- Llm4sConfig.defaultProvider()
    registry       <- Llm4sConfig.modelRegistryService()
    given ModelRegistryService = registry
    client <- LLMConnect.getClient(providerConfig)
  } yield new HelloLLM(client).sayHello()

  startup.left.foreach(err => println(s"Startup Error: $err"))
}

Run It 

1
2
3
4

# llm4s-openai binds OPENAI_API_KEY to OpenAI's shared key, which the section uses
export OPENAI_API_KEY=sk-...

sbt run

Expected Output 

1
2
3

Response: Scala is a high-level programming language that combines
object-oriented and functional programming paradigms. It runs on the
JVM and is known for its strong type system and concurrency support.

Understanding the Code

Let’s break down what’s happening:

1. The Configuration Boundary 

1
2
3
4

providerConfig <- Llm4sConfig.defaultProvider()
registry       <- Llm4sConfig.modelRegistryService()
given ModelRegistryService = registry
client <- LLMConnect.getClient(providerConfig)

In LLM4S, we follow a strict configuration boundary. The entry point (Main) builds the client:

  • Loads the typed config of the section llm4s.providers.provider names in application.conf

  • Selects the appropriate provider (OpenAI, Anthropic, etc.)

  • Injects the LLMClient into your core logic (HelloLLM).

2. Complete with Messages 

1
2
3
4

val response = client.complete(
  Conversation(List(UserMessage("What is Scala?"))),
  CompletionOptions()
)
  • conversation: The conversation messages (User, Assistant, System), wrapped in Conversation
  • options: Per-request completion settings; the model is selected when the client is configured
  • Returns Result[Completion]

3. Result Handling 

1
2
3
4

result match {
  case Right(completion) => // Success
  case Left(error) => // Error
}

LLM4S uses Result types instead of exceptions:

  • Right(value) = success
  • Left(error) = failure

This makes error handling explicit and type-safe.

Adding Conversation Context

Let’s make our program more interesting with a multi-turn conversation:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

import org.llm4s.llmconnect.LLMClient
import org.llm4s.llmconnect.model._

class ConversationExample(client: LLMClient) {
  def run(): Unit = {
    val result = client.complete(
      Conversation(Seq(
        SystemMessage("You are a helpful programming tutor."),
        UserMessage("What is Scala?"),
        AssistantMessage("Scala is a high-level programming language..."),
        UserMessage("How does it compare to Java?")
      ))
    )

    result.fold(
      error => println(s"Error: $error"),
      completion => println(s"Response: ${completion.content}")
    )
  }
}

Message Types

LLM4S provides three message types:

Type Purpose Example
SystemMessage Set LLM behavior “You are a helpful assistant”
UserMessage User’s input “What is Scala?”
AssistantMessage LLM’s previous responses “Scala is a programming language…”

Adding Tool Calling

Now let’s give the LLM access to a tool:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

import org.llm4s.llmconnect.LLMClient
import org.llm4s.toolapi.{ Schema, ToolBuilder, ToolRegistry }
import org.llm4s.agent.Agent

class ToolExample(client: LLMClient) {
  // Define a simple tool
  def getWeather(location: String): String = {
    s"The weather in $location is sunny and 72°F"
  }

  private val weatherSchema = Schema
    .`object`[Map[String, Any]]("Weather query parameters")
    .withProperty(Schema.property("location", Schema.string("The city to look up")))

  def run(): Unit = {
    val result = for {
      weatherTool <- ToolBuilder[Map[String, Any], String](
                       "get_weather",
                       "Get current weather for a location",
                       weatherSchema
                     ).withHandler(extractor => extractor.getString("location").map(getWeather)).buildSafe()
      agent <- Agent.builder("weather-agent", client)
        .withTools(new ToolRegistry(Seq(weatherTool)))
        .build()
      result <- agent.run("What's the weather in Paris?")
    } yield result

    result.fold(
      error => println(s"Error: $error"),
      r => println(s"Final response: ${r.answer.getOrElse(r.status.toString)}")
    )
  }
}

The agent will:

  1. Understand you’re asking about weather
  2. Call the get_weather tool with “Paris”
  3. Use the tool result to formulate a response

Streaming Responses

For real-time output, use streaming:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

import org.llm4s.llmconnect.LLMClient
import org.llm4s.llmconnect.model._

class StreamingExample(client: LLMClient) {
  def run(): Unit = {
    val result = client.streamComplete(
      Conversation(Seq(UserMessage("Write a short poem about Scala"))),
      onChunk = chunk => chunk.content.foreach(print)  // Print each token as it arrives
    )

    result.fold(
      error => println(s"Error: $error"),
      _ => println("\nDone!")
    )
  }
}

Output appears token-by-token in real-time, like ChatGPT!

Error Handling Patterns

Basic Error Handling 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

import org.llm4s.error.{ AuthenticationError, NetworkError, ValidationError }
import org.llm4s.llmconnect.model._

// Assuming client is injected
val result = client.complete(Conversation(Seq(UserMessage("Hello"))))
result match {
  case Right(completion) =>
    // Success
    println(completion.content)
  case Left(error) =>
    // Handle different error types
    error match {
      case e: NetworkError        => println(s"Network issue: ${e.message}")
      case e: AuthenticationError => println(s"Auth failed: ${e.message}")
      case e: ValidationError     => println(s"Invalid input: ${e.message}")
      case _                      => println(s"Error: $error")
    }
}

Using fold 

1
2
3
4

result.fold(
  error => println(s"Failed: $error"),
  completion => println(s"Success: ${completion.content}")
)

Mapping Results 

1
2

val content: Result[String] = result.map(_.content)
val uppercased: Result[String] = content.map(_.toUpperCase)

Configuration Options

Using Different Models 

1
2
3
4
5

import org.llm4s.config.Llm4sConfig

// The model belongs to the provider section, so to use another model load the section that names it,
// for example one with provider = "openai" and model = "gpt-4-turbo"
val gpt4Turbo = Llm4sConfig.provider("openai-gpt4-turbo")

Provider-Specific Settings

The model and provider come from the section in application.conf; per-request settings such as temperature and token limits go in CompletionOptions:

1
2
3
4

openai-main {
  provider = "openai"
  model    = "gpt-4o"
}

1

client.complete(conversation, CompletionOptions(temperature = 0.7, maxTokens = Some(1000)))

Complete Working Example

Here’s a complete example combining everything we’ve learned:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

import org.llm4s.config.Llm4sConfig
import org.llm4s.llmconnect.{LLMClient, LLMConnect}
import org.llm4s.llmconnect.model._
import org.llm4s.toolapi.{ Schema, ToolBuilder, ToolRegistry }
import org.llm4s.agent.Agent

class ComprehensiveAgent(client: LLMClient) {
  def calculate(expression: String): String = {
    // Simple calculator (use proper eval in production!)
    s"Result: ${expression} = 42"
  }

  private val calcSchema = Schema
    .`object`[Map[String, Any]]("Calculator parameters")
    .withProperty(Schema.property("expression", Schema.string("The expression to evaluate")))

  def run(): Unit = {
    println("🚀 Starting LLM4S Example...")

    // Tools belong to the agent, so give them to the builder
    val result = for {
      calcTool <- ToolBuilder[Map[String, Any], String](
                    "calculate",
                    "Evaluate a mathematical expression",
                    calcSchema
                  ).withHandler(extractor => extractor.getString("expression").map(calculate)).buildSafe()
      agent <- Agent.builder("calc-agent", client)
        .withTools(new ToolRegistry(Seq(calcTool)))
        .build()
      result <- agent.run("What is 6 times 7? Please use the calculator.")
    } yield result

    result match {
      case Right(r) =>
        println(s"✅ Success!")
        println(s"Response: ${r.answer.getOrElse(r.status.toString)}")
        println(s"Messages exchanged: ${r.messages.length}")

      case Left(error) =>
        println(s"❌ Error: $error")
        System.exit(1)
    }
  }
}

// Application Entry Point
object ComprehensiveMain extends App {
  val startup = for {
    providerConfig <- Llm4sConfig.defaultProvider()
    registry       <- Llm4sConfig.modelRegistryService()
    given org.llm4s.model.ModelRegistryService = registry
    client <- LLMConnect.getClient(providerConfig)
  } yield new ComprehensiveAgent(client).run()

  startup.left.foreach(err => println(s"Startup Error: $err"))
}

Common Patterns

Pattern 1: Simple Query 

1
2
3
4

// Assuming client is injected
val response = client.complete(
  Conversation(Seq(UserMessage("Your question here")))
)

Pattern 2: With System Prompt 

1
2
3
4
5
6
7

// Assuming client is injected
val response = client.complete(
  Conversation(Seq(
    SystemMessage("You are an expert in..."),
    UserMessage("Question")
  ))
)

Pattern 3: Agent with Tools 

1
2
3
4
5

// Assuming client is injected
val result = for {
  agent  <- Agent.builder("assistant", client).withTools(new ToolRegistry(myTools)).build()
  result <- agent.run("User query")
} yield result

Pattern 4: Streaming 

1
2
3
4
5

// Assuming client is injected
val completion = client.streamComplete(
  Conversation(Seq(UserMessage("Your question here"))),
  onChunk = chunk => chunk.content.foreach(print)
)

Performance Optimization

1. Reuse Client Instances

Don’t create a new client for every request:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

// Assuming client: LLMClient is injected into your class

// ✅ Good: Reuse existing injected client
(1 to 10).foreach { i =>
  client.complete(Conversation(Seq(UserMessage(s"Question $i"))))
}

// ❌ Bad: Creating new client inside the loop (wasteful and violates boundaries)
(1 to 10).foreach { i =>
  for {
    providerConfig <- Llm4sConfig.defaultProvider()
    registry       <- Llm4sConfig.modelRegistryService()
    given ModelRegistryService = registry
    badClient <- LLMConnect.getClient(providerConfig)  // Don't do this!
    response <- badClient.complete(
      Conversation(Seq(UserMessage(s"Q$i")))
    )
  } yield response
}

2. Use Streaming for Long Responses

Streaming gets you the first token faster and improves perceived latency:

1
2
3
4
5
6
7
8
9
10
11
12

import org.llm4s.llmconnect.model._

// Assuming client is injected
val streamResult = client.streamComplete(
  Conversation(Seq(UserMessage("Write a long essay about Scala"))),
  onChunk = chunk => chunk.content.foreach(print)  // Prints as tokens arrive
)

streamResult match {
  case Right(completion) => println(s"\nCompleted with ${completion.usage.map(_.totalTokens).getOrElse(0)} tokens")
  case Left(error) => println(s"Error: $error")
}

When to stream:

  • User-facing applications (chat interfaces)
  • Long-form content generation
  • When you need to show progress

When not to stream:

  • Short queries (< 100 tokens)
  • When you need the full response before processing
  • Background batch jobs

3. Parallelize Independent Requests

Process multiple unrelated queries concurrently:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

import scala.concurrent.{Future, ExecutionContext, Await}
import scala.concurrent.duration._
import ExecutionContext.Implicits.global

// Assuming client is injected
val queries = List(
  "What is Scala?",
  "What is functional programming?",
  "What is the JVM?"
)

// Run all queries in parallel
val futures = queries.map { query =>
  Future {
    client.complete(Conversation(Seq(UserMessage(query))))
  }
}

val results = Await.result(Future.sequence(futures), 30.seconds)
results.foreach {
  case Right(response) => println(response.content)
  case Left(error) => println(s"Error: $error")
}

4. Set Appropriate Timeouts

Each client has a default request and stream timeout (two minutes for a completion, five or ten for a stream), and a provider section can change them:

1
2
3
4
5

llm4s.providers.my-openai {
  provider = "openai"
  model    = "gpt-4o"
  timeouts { request = 3m, stream = 15m }
}

See Timeouts for the keys, the defaults per provider and what each timeout bounds. For long-form generation, prefer streaming (above).

5. Use Cheaper Models for Development

Change the section’s model, or point llm4s.providers.provider at another section (see Switching providers):

1
2
3
4
5
6
7
8
9
10
11
12
13
14

llm4s.providers {
  provider = "ollama-local"            # development: free, local

  ollama-local {
    provider = "ollama"
    model    = "llama3.2"
    baseUrl  = "http://localhost:11434"
  }

  openai-main {                        # production: select with provider = "openai-main"
    provider = "openai"
    model    = "gpt-4o-mini"           # much cheaper than gpt-4o
  }
}

Only the section being loaded is validated, so OPENAI_API_KEY need not be set while ollama-local is the default - only where openai-main is selected.

6. Batch Embeddings

When generating embeddings for RAG:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

import org.llm4s.llmconnect.model.EmbeddingRequest

// ✅ Good: Batch processing
val documents: List[String] = loadDocuments()  // say, a thousand documents
val batchSize = 100

val allEmbeddings = documents.grouped(batchSize).flatMap { batch =>
  embedder.embed(EmbeddingRequest(batch, embeddingModel)) match {
    case Right(response) => response.embeddings
    case Left(error) =>
      println(s"Batch failed: $error")
      List.empty
  }
}.toList

// ❌ Bad: One at a time (slow, expensive)
val embeddings = documents.map { doc =>
  embedder.embed(EmbeddingRequest(List(doc), embeddingModel))
}

7. Monitor Token Usage

Track costs in production:

1
2
3
4
5
6
7
8
9
10

val response = client.complete(Conversation(Seq(UserMessage("Hello"))))

response match {
  case Right(completion) =>
    // Note: usage returns Option[TokenUsage], so these are Option[Int]
    println(s"Prompt tokens: ${completion.usage.map(_.promptTokens)}")
    println(s"Completion tokens: ${completion.usage.map(_.completionTokens)}")
    println(s"Total tokens: ${completion.usage.map(_.totalTokens)}")
  case Left(error) => println(s"Error: $error")
}

Running the Examples

Note: Use the same Main pattern shown above to run the ConversationExample, ToolExample, and StreamingExample classes.

Next Steps

Great job! You’ve written your first LLM4S programs. Now explore:

  1. Configuration → - Learn about advanced configuration
  2. Agent Framework → - Build sophisticated agents
  3. Tool Calling → - Create custom tools
  4. Examples → - Browse 69 working examples

Learning Resources

Ready to dive deeper? Configure your environment →