LLM4S 0.5.1: multimodal messages and the agent harness

The agent harness grows on the 0.5.0 runtime, and messages carry images and documents.

What is new

Multimodal content parts: user messages and tool results carry text, images and documents. Each provider sends the parts its API accepts and refuses the others before sending (#1913).
Backend SPIs: FileBackend (state, composite and store), SandboxBackend and LongTermStore, with FileBackendContract (#1904).
Filesystem tools: FileTools(backend) provides ls, read_file, write_file, edit_file, delete, glob and grep over any FileBackend (#1905).
Context management: SummarizationMiddleware and ToolOutputOffload (#1908).
Agent Skills, instruction memory and the harness profile: SkillsMiddleware (SKILL.md), InstructionMemoryMiddleware (AGENTS.md), and HarnessProfile with AgentBuilder.withHarness (#1909, #1910, #1915). The agents guide shows each of them.

Upgrading

llm4s is still stabilising before 1.0. Until the binary-compatibility baseline is set, any release may break, patch releases included (compatibility policy). 0.5.1 has two breaking changes:

UserMessage and ToolMessage carry parts and have no extractor, and ToolOutcome.Success takes a ToolContent. See the migration note.
build() refuses a SemanticMemoryMiddleware placed inside guardrail middleware. See the migration note.

Install

libraryDependencies ++= Seq(
  "org.llm4s" %% "llm4s-core"   % "0.5.1",
  "org.llm4s" %% "llm4s-openai" % "0.5.1" // or the provider module you use
)

Every artifact, with its tier: installation guide.

Known issue: org.llm4s:llm4s-kotlin-api:0.5.1 was not published either (the release’s version check missed sbt’s tab-indented version). Every Scala and Java artifact is on Maven Central. llm4s-kotlin-api is not on Maven Central yet: build it locally, as the installation guide describes.

Full list of changes: CHANGELOG · GitHub release