Home | LLM4S

Why LLM4S

Production AI engineering, the way JVM teams already build software

Most LLM tooling is Python-first. LLM4S brings agents, tools and retrieval to Scala and the JVM, so they run in the services, data platforms and pipelines you already operate, with no Python sidecar.

Result[A]

Errors are values

Every call returns a Result with typed error classes instead of throwing, so failures from rate limits to bad tool arguments are handled where the compiler can see them.
Error handling
derives

Typed tools and structured output

Describe a tool's arguments or a model's answer as a case class and derive its JSON schema with derives SchemaOf, so the schema, the parser and your code cannot drift apart (Experimental).
Derived schemas
SPI

One API, many providers

Sixteen chat providers, from OpenAI, Anthropic, Gemini and Bedrock to local Ollama, behind one small provider SPI. Adding a provider is adding a dependency; switching is a config change.
Providers
Agent

Agents built for production

Guardrails, including LLM-as-judge, handoffs between specialist agents, streaming events, human review and durable checkpoints in SQLite or PostgreSQL so a run survives a restart.
Agents
RAG

Retrieval and memory

Chunking, embeddings, pgvector, Qdrant, SQLite and Elasticsearch stores, hybrid keyword and vector search, reranking, RAG evaluation and agent memory.
RAG and vector stores
OTel

Observable and governed

OpenTelemetry, Langfuse, Prometheus and Micrometer integrations, cost tracking, API-key redaction, a sandboxed workspace for tools, MCP with auth, and a published threat model.
Observability

Your types are the contract

From a case class to a checked model answer

Write the type once. LLM4S derives the JSON schema the provider enforces, sends it, and reads the answer back into your type, all inside one Result.

Provider and model come from application.conf, keys from the environment
No exceptions to catch: a missing key, a refused request or an unreadable answer is a Left
The same type works as an agent's typed answer or a tool's arguments

Write your first program Browse examples

import org.llm4s.config.Llm4sConfig
import org.llm4s.llmconnect.LLMConnect
import org.llm4s.llmconnect.model.{ Conversation, UserMessage }
import org.llm4s.model.ModelRegistryService
import org.llm4s.schema.*
import org.llm4s.types.Result
import upickle.default.ReadWriter

case class Ticket(
  @description("One-line summary of the problem") title: String,
  @description("low, medium or high") priority: String
) derives SchemaOf, ReadWriter

val ticket: Result[Ticket] = for {
  providerConfig <- Llm4sConfig.defaultProvider()
  registry       <- Llm4sConfig.modelRegistryService()
  given ModelRegistryService = registry
  client <- LLMConnect.getClient(providerConfig)
  ticket <- client.completeStructuredOf[Ticket](
    Conversation(Seq(UserMessage("Checkout fails with a 500")))
  )
} yield ticket

Fits your stack

Works with the JVM ecosystem you already use

Modular artifacts: depend on the core and add only the providers and integrations you need, pinned together by one bill of materials.

Model providers

OpenAI
Anthropic
Azure OpenAI
Google Gemini
Vertex AI
Amazon Bedrock
IBM watsonx.ai
Mistral
Cohere
DeepSeek
xAI
Z.ai
OpenRouter
Requesty
Ollama
Any OpenAI-compatible endpoint

Languages, frameworks and runtimes

Protocols, stores and operations

Model Context Protocol
OpenTelemetry
Langfuse
Prometheus
Micrometer
PostgreSQL + pgvector
Qdrant
SQLite
Elasticsearch
Neo4j
Docker sandbox

Use cases

What teams build with it

Assistants inside JVM services
Support, operations and internal assistants that run in your Spring Boot, Pekko or plain JVM service, with guardrails on what goes in and comes out.
Question answering over your documents
RAG over manuals, tickets and wikis with hybrid search, reranking, permission-aware retrieval and evaluation you can run in CI.
Extraction into typed records
Invoices, emails and forms turned into case classes, checked against a schema, inside your existing data pipelines.
Agentic workflows
Multi-step agents with tools, handoffs and human review, checkpointed so long runs resume after a crash or a deploy.
Developer tooling
Issue triage, coding agents and MCP servers that expose your systems to any MCP-capable client.
Voice and multimodal apps
Speech to text, text to speech, image generation and vision, behind the same typed API.

Showcase

Open-source projects built with LLM4S

Public projects from the LLM4S organisation and the community. Listing is not an endorsement.

See all projects Add your project

llm4s/szork

Szork

A voice-controlled, Zork-style text adventure that uses LLM4S for speech, image generation, tools and agentic gameplay.

agentsspeechimagestools
llm4s/rag_in_a_box

RAG in a Box

A turnkey RAG deployment powered by LLM4S, with a REST API for ingestion and querying, PostgreSQL with pgvector, and hybrid search.

ragservice
com-vitthalmirji/toon4s

toon4s

Token-Oriented Object Notation (TOON) for the JVM, a compact encoding for sending structured data to LLMs.

librarytokens
gruggiero/Adk4s

Adk4s

An Agentic Development Kit in Scala, built on top of LLM4S.

agentsframework
gruggiero/llm4s-tripper

llm4s-tripper

A travel-planning agent built with LLM4S and workflows4s, with cats-effect and MCP tools.

agentsmcpcats-effect
pavanvamsi3/triage4s

triage4s

Evidence-backed AI issue triage for Scala projects, powered by LLM4S, with a read-only GitHub dashboard.

agentsdeveloper tools

Students and early-career engineers

Become an AI engineer in the open, for free

Skip the expensive bootcamp. Learn LLM engineering on a real, production-minded codebase: build agents and RAG pipelines, then contribute code that maintainers review in public. Every merged pull request is part of your public record.

Follow the learning path Good first issues

1
Get startedInstall LLM4S and make your first call
2
Build an agentTools, guardrails and multi-turn conversations
3
Add retrievalEmbeddings, vector stores and RAG evaluation
4
Connect tools and MCPTyped tools and Model Context Protocol servers
5
ContributeGood first issues, weekly Dev Hour, mentorship

Programs and mentorship

Mentored paths into open source

2025

Google Summer of Code

LLM4S took part in GSoC 2025 under the Scala Center umbrella: four funded interns from the United States, Switzerland and India worked on the project.
Meet the 2025 cohort
2026 tracks

LFX, ESoC and GSoC

The roadmap lists LFX Mentorship 2026, European Summer of Code 2026 and GSoC 2026 as mentorship tracks, each with a public list of project ideas.
See the project ideas
Every Sunday

LLM4S Dev Hour

A weekly mob-programming session on Discord at 09:00 London time, open to everyone, from first contribution to hundredth.
Join the next session
Partners

Host an intern

Universities and companies can partner with the foundation to fund or co-mentor internships on LLM4S.
Partner with us

Events

Upcoming events

Meet the LLM4S community at a talk near you, or join the weekly Dev Hour online.

Explore events Follow the Luma calendar

23 Oct 2026 · 10:00-15:00 IST

E-Cell Orientation & Pitch Please | Vivekananda Institute of Professional Studies-Technical Campus (VIPS-TC)

LLM4S Foundation: A Retrospective

Vivekananda Institute of Professional Studies-Technical Campus (VIPS-TC), Pitampura, Delhi, India 🇮🇳
Kannupriya Kalra
Event page Slides For queries: Contact E-Cell Instagram post 1 Instagram post 2 LinkedIn post 1 LinkedIn post 2 LinkedIn post 3 LinkedIn post 4 Bluesky Reddit X/Twitter Mastodon
31 Oct 2026 · 12:00-13:00 IST

DataverseHub

LLM4S Foundation: A Retrospective

Apple India, Gachibowli, Hyderabad, India 🇮🇳
Kannupriya Kalra
Event page
17 Nov 2026

Open Source Singapore

LLM4S 1.0: From Idea to Production AI in Scala/JVM

Singapore 🇸🇬
Kannupriya Kalra
Event page
27 Nov 2026

LLMday + Prompt Engineering Conference London Q4 2026

LLM4S 1.0: From Idea to Production AI in Scala/JVM

Everyman Canary Wharf, London, UK 🇬🇧
Kannupriya Kalra, Rory Graves
Event page

LLM4S Dev Hour: every Sunday at 09:00 London time on Discord. Next: Sunday, 09:00 London time.

Talks

LLM4S Around the World

Snapshots from LLM4S talks at Scala meetups, conferences, developer events and big-tech hackathons. Select a picture to open it full size.

See all talks and events

DataverseHub 2026, Hyderabad – LLM4S Foundation: A Retrospective
E-Cell Orientation & Pitch Please, VIPS-TC 2026 – poster
E-Cell Orientation & Pitch Please, VIPS-TC – LLM4S Foundation: A Retrospective
Microsoft Global Hackathon 2026 – Build with LLMs: Open Source AI for the JVM
AI Compute & Hardware Conference 2025, San Jose – Functional Intelligence
Oaisys AI Practitioners Conference 2025, Pune – Shubham Vishwakarma at the venue
Oaisys AI Practitioners Conference 2025, Pune – Show and Tell, Shubham Vishwakarma
Oaisys AI Practitioners Conference 2025, Pune – Show and Tell, Kannupriya Kalra
Oaisys AI Practitioners Conference 2025, Pune – LLM4S: Building Reliable AI Systems in the JVM Ecosystem
GEN AI London 2025 – Building Reliable AI Systems
GSoC Mentor Summit 2025, Munich – LLM4S banner
ICFP/SPLASH 2025, Singapore – Mentoring in the Scala Ecosystem
GSoC 2025 Lightning Talks, hosted by the Scala Center, Lausanne
Zürich Scala Enthusiasts 2025 – banner
Scala Days 2025, Lausanne – Szork inner-world demo slide
Scala Days 2025, Lausanne – the LLM4S team
Scala Days 2025, Lausanne – official talk banner
London Scala Users Group 2025 – flyer
London Scala Users Group 2025 – banner
London Scala Users Group 2025 – Kannupriya Kalra's multimodal slide
London Scala Users Group 2025 – Rory Graves's Szork demo
London Scala Users Group 2025 – LLM4S session banner
Dallas Scala Enthusiasts 2025 – talk banner
Functional World 2025 by Scalac, Gdańsk, Poland
Scala India 2025 – talk banner
Bay Area Scala, San Francisco – generative AI slide
Bay Area Scala, San Francisco – introductory slide
Bay Area Scala, Tubi office, San Francisco: where LLM4S was introduced

Roadmap

The road to 1.0

1.0 freezes a stable spine with a binary-compatibility promise. The date is intentionally not fixed: the frozen surface ships once it has survived two releases.

0.4.xArtifacts renamed to llm4s-*
0.5.0Modular artifacts, provider SPI, Java, Spring Boot, cats-effect and ZIO modules
0.5.1Multimodal messages, agent harness SPIs, Agent Skills
Pre-1.0Designs final, MiMa compatibility baseline set
1.0Frozen, binary-compatible surface across 1.x

Read the roadmap What 1.0 freezes

Support the foundation

LLM4S is community-led and maintained by a small core team, mostly in volunteer time. Your support pays for maintainer time, contributor onboarding, documentation, release engineering, CI, security hardening, the weekly Dev Hour and internships. Every contribution is handled in public through Open Collective.

Support on Open Collective How funding is used

CompaniesSponsor the project your JVM services depend on, or talk to us about partnering.
Grants and institutionsWe welcome grants and institutional support for open infrastructure.
UniversitiesPartner with us on internships and mentored student projects.

Build reliable LLM applications in Scala, on the JVM you already run

LLM4S in numbers

Errors are values

Typed tools and structured output

One API, many providers

Agents built for production

Retrieval and memory

Observable and governed

From a case class to a checked model answer

Model providers

Languages, frameworks and runtimes

Protocols, stores and operations

Assistants inside JVM services

Question answering over your documents

Extraction into typed records

Agentic workflows

Developer tooling

Voice and multimodal apps

Become an AI engineer in the open, for free

Google Summer of Code

LFX, ESoC and GSoC

LLM4S Dev Hour

Host an intern

Keep type-safe AI on the JVM open, maintained and free