Build AI agents in Scala

To build an AI agent in Scala, add llm4s-agent and give Agent.builder a model client and the tools the model may call: the agent loops between the model and your tools until it has an answer. The example below adds an input guardrail and streams the agent’s events, in about 25 lines.

  1. Add the modules
  2. An agent with a tool, a guardrail and streaming
  3. Going further

Add the modules

The agent runtime is its own module, next to the core and the module of your model provider:

1
2
3
4

libraryDependencies ++= Seq(
  "org.llm4s" %% "llm4s-agent"  % "0.5.1",
  "org.llm4s" %% "llm4s-openai" % "0.5.1"
)

llm4s-agent brings llm4s-core. The client is built from your configuration as on the first-call page, and you can test the agent with no model at all using llm4s-testkit.

An agent with a tool, a guardrail and streaming 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

import org.llm4s.agent.{ Agent, AgentResult }
import org.llm4s.agent.events.AgentEvents
import org.llm4s.agent.graph.ThreadId
import org.llm4s.agent.graph.middleware.GuardrailMiddleware
import org.llm4s.agent.guardrails.builtin.{ LengthCheck, ProfanityFilter }
import org.llm4s.llmconnect.LLMClient
import org.llm4s.toolapi.{ Schema, ToolBuilder, ToolRegistry }
import org.llm4s.types.Result

def weatherAgent(client: LLMClient, question: String): Result[AgentResult] = for {
  forecast <- ToolBuilder[Map[String, Any], String](
    name = "get_forecast",
    description = "Returns today's forecast for a city",
    schema = Schema
      .`object`[Map[String, Any]]("Forecast parameters")
      .withProperty(Schema.property("city", Schema.string("The city to look up")))
  ).withHandler(args => args.getString("city").map(city => s"Sunny and 18 C in $city")).buildSafe()
  agent <- Agent
    .builder("weather", client)
    .withSystemPrompt("You answer questions about the weather.")
    .withTools(new ToolRegistry(Seq(forecast)))
    .withMiddleware(new GuardrailMiddleware(input = Seq(new LengthCheck(1, 2000), new ProfanityFilter()), Nil))
    .withStreaming()
    .build()
  run <- agent.stream(ThreadId("weather-1"), question) {
    case AgentEvents.TextDelta(delta)    => print(delta.text)
    case AgentEvents.ToolCallStarted(tc) => println(s"[calling ${tc.tool}]")
    case _                               => ()
  }
  result <- run.await()
} yield result

What each part does:

  • The tool. ToolBuilder declares a function the model may call, with a JSON schema for its arguments; the handler reads its arguments as Either values, so a missing argument is a value, not an exception. The tools guide goes further, and typed tools derive the schema from a case class.
  • The guardrails. GuardrailMiddleware checks the input before the model sees it. A question outside the length limit ends the run as AgentStatus.Blocked, an outcome you match on rather than an error, and the model is never called. Output guardrails, LLM-as-judge checks and your own are in the guardrails guide.
  • Streaming. withStreaming() streams the model’s answer, and agent.stream hands every event of the turn to your function: text as it arrives, tool calls as they start, and more (streaming events).
  • The result. AgentResult carries the status, the thread’s messages and the token usage; result.answer is the final answer when the run completed. A provider or tool failure is a Left.

The run belongs to a thread (ThreadId): call agent.run(threadId, ...) again with the same id to continue the conversation.

Going further