Migration Guide

Upgrading from 0.4.1? Start with Upgrading from 0.4.1 to 0.5.0, one page grouped by what you use. The notes below are the per-change detail it links to. From 0.5.0 to 0.5.1 the source breaks are Multimodal content parts and Harness profile and middleware placement. After 0.5.1, MemoryManagerConfig is growth-safe, and so are ToolRetryPolicy, GeminiConfig, VertexAIConfig and AzureConfig, core’s tool, client, embedding, registry and tracing configs, core’s HTTP response, embedding, pricing, usage and exchange types the TraceEvent cases, the llmconnect.model types, the provider SPI, caching, streaming and exchange-sink types, the context types, the tool schemas and tool-call types, two ToolCallError and ToolParameterError cases, the AssistantError cases, RerankProviderConfig and the remaining config, model and util types; OpenAIToolHelper is internal; ServiceError’s extractor gains retryAfter; console tracing prints no content by default; and llm4s-java-api and the Spring Boot starter no longer bring the provider modules.

A supplied HTTP client is the caller’s to close (#2341)

LlmClientOptions.withHttpClient hands the SDK-free provider clients (ollama, gemini, vertexai, watsonx, openai-compatible and its dialects) an Llm4sHttpClient to send their requests through. A client supplied this way, or to the httpClient constructor parameter of OllamaClient, GeminiClient, VertexAIClient, WatsonXClient or an OpenAICompatibleClient, is no longer closed when the LLM client is closed: the caller owns it. Close it yourself when your application stops. A client the provider builds itself is still closed with it.

LlmClientOptions gains httpClient: Option[Llm4sHttpClient] and ProviderFeatures gains customHttpClient, false by default (and for openai, azure, requesty, anthropic and bedrock, whose SDKs keep their own transport). A provider of your own that sends its requests through options.httpClient opts in with ProviderFeatures.default.withCustomHttpClient(true). Their apply overloads without the new field remain; a pattern match on either type’s extractor binds one more field.

Credentials are Secret (#1932)

ApiKey is now an opaque type over org.llm4s.security.Secret, and every credential field in a config is a Secret (or Option[Secret]): provider and embedding configs, HTTP proxy and TLS-store passwords, the MCP server key, Langfuse keys, search-tool keys, image, speech, watsonx, jev and Bedrock keys, the Cohere reranker key, and store credentials (Postgres, Neo4j, Qdrant, Elasticsearch/OpenSearch, Redis, the S3 loader, the URL loader token). Every Secret is registered, and removed by exact match from the error messages, exchange logs and diagnostic text llm4s builds (trace content in a follow-up).

Before:

OpenAIConfig(apiKey = "sk-...", model = "gpt-4o", baseUrl = "https://api.openai.com/v1")

After:

import org.llm4s.security.Secret

OpenAIConfig(apiKey = Secret("sk-..."), model = "gpt-4o", baseUrl = "https://api.openai.com/v1")

• Read the value with .reveal. toString prints ***.
• Convenience factories that took a String still do.
• VectorStoreFactory.Config: pass credentials with withSecret("password" | "apiKey", Secret(...)). withOption("password" | "apiKey", ...) is refused with a ConfigurationError.
• Values under 8 characters are not registered, so they are not masked by exact match.
• Java: the builders still take String. Constructing a config directly needs Secret.apply(key) (import org.llm4s.security.Secret;).

Multimodal content parts (#1913)

In 0.5.1. Stage 3 slice F8b (#1913) gives user messages and tool results content parts: text, images and documents. Code that builds text messages and reads content needs no change. Source breaks, with no shims:

• UserMessage(parts: Vector[ContentPart]) and ToolMessage(parts: Vector[ContentPart], toolCallId). UserMessage(text) and ToolMessage(text, toolCallId) still build a one-text-part message, and content is the text view: the text parts, joined by newlines, "" for a message that is all media. A pattern binds the parts now, so match on the type:

  // before
  case UserMessage(content)          => send(content)
  case ToolMessage(content, callId)  => sendResult(callId, content)
  // after
  case u: UserMessage => send(u.content)          // or map u.parts
  case t: ToolMessage => sendResult(t.toolCallId, t.content)
  

  The two are no longer case classes and have no extractor, so every positional pattern is a compile error: an extractor of the parts would have kept s"$content" compiling and printing the parts, and one of the text would have kept a client compiling that drops media. msg.copy(content = ...) becomes msg.withParts(...).

• ToolOutcome.Success(content: ToolContent, update), with ToolContent.Json(value) for the JSON result a tool returned before and ToolContent.Parts(parts) for images and documents:

  ToolOutcome.Success(ujson.Str("done"))                    // before
  ToolOutcome.Success(ToolContent.Json(ujson.Str("done")))  // after
  ToolOutcome.Success(ToolContent.Parts(Vector(ContentPart.Text("The chart:"), chart)))
  

  Parts with no content, or with more than ContentPart.MaxInlineBytes of inline bytes in total, is the call’s error result.

• A custom LLMClient that matched UserMessage(content) must decide what to do with media parts: send them, or return Left(UnsupportedContentPartError(provider, part.kind, role)) before sending. org.llm4s.llmconnect.provider.PartEncoding is private[llm4s]; a provider outside llm4s uses ContentPart.kind and the error directly, and runs ProviderModuleChecks.assertContentParts in its module spec.
• Stored data needs nothing. The JSON of a one-text message is unchanged, and the old shape reads, so memory stores, saved conversations and thinking bindings are unaffected. An agent thread’s messages and tool-results keys move to schema version 2 and migrate on restore; a thread written by this version cannot be restored by an earlier one.
• A client refuses what it cannot send, before sending: a part kind, source or format its wire format cannot carry is UnsupportedContentPartError. A provider’s own limits (how many images, how large a request, which formats a model reads) are the provider’s: its 400 or 413 comes back as a typed ValidationError or ServiceError with its message. The guide’s table lists what each takes.
• An agent thread holds at most 20 MiB of inline media (Messages.MaxInlineBytes): media added past it is replaced by a text part naming what was removed. Context budgets (ConversationTokenCounter, ContextPruning) count media by estimate.
• Inline media is bounded at 5 MiB per part by ContentPart.image / ContentPart.document, which return a Result. Every agent checkpoint copies the thread’s messages, so larger media goes at a MediaSource.Url or in a MediaSource.ProviderFile. See Multimodal content.

  Subagents and isolation (Phase 4)

Not in a release yet. Stage 4 lets an agent delegate to child agents on threads of their own (#2044). Each slice adds its breaks here.

Child threads (#2045)

A thread can now be a child thread: its checkpoints record a RunOrigin naming its parent (design 4.41).

• Checkpoint format 8 and SQLite schema 6, with no way back. Format 8 adds Checkpoint.origin, Checkpoint.outcome (how a settled run ended, written with every terminal checkpoint) and Checkpoint.deleting (a cascade delete that has begun; admission refuses such a thread with GraphError.ThreadDeleting until deleteThread finishes it); schema 6 adds parent_thread_id and child_order to llm4s_checkpoint_threads. Existing threads read and continue unchanged: format 7 reads as a top-level thread with no outcome, and the schema step backfills nothing. An older build refuses a format-8 checkpoint (UnsupportedCheckpointFormat) and a schema-6 file, so upgrade every process sharing a store together.
• A Checkpointer implements children. children(parent: ThreadId, after: Option[ThreadId], limit: Int): Result[Vector[ChildThread]] is a new abstract method: list the threads whose latest checkpoint’s origin.parent is parent, in the order each first named it, paged after after, never a deleted one, and refuse an unknown after with a ValidationError. Index the parent as you commit a checkpoint (checkpoint.origin.map(_.parent)). A store that only forwards to another forwards this too. CheckpointerContract checks it.
• deleteThread and Agent.forget cascade. Deleting a thread first deletes every child thread below it; a child with a live run anywhere refuses the whole delete with ThreadBusy, deleting nothing.
• Checkpointer.deleteThread(threadId, token) is deleteThread(threadId, token, fence: Option[(ThreadId, FencingToken)]) (#2063). With Some((other, otherToken)) a store deletes only if token is the thread’s current claim and otherToken is other’s, both checked in the same lock or transaction as the delete; otherwise it returns GraphError.StaleClaim - for the thread when its own claim is stale (checked first), else for other - and deletes nothing. None is the previous behaviour. The runtime fences a cascade’s deletes by the root’s claim and retention’s by the parent’s. A wrapper forwards it: def deleteThread(threadId: ThreadId, token: FencingToken, fence: Option[(ThreadId, FencingToken)]) = underlying.deleteThread(threadId, token, fence). CheckpointerContract checks it.
• A store checks Commit.fence (#2071). Commit gains fence: Option[(ThreadId, FencingToken)] (withFence). With Some((other, otherToken)) a store applies the commit only if its token is the thread’s current claim and otherToken is other’s, both checked in the same lock or transaction as the write; otherwise it returns GraphError.StaleClaim - for the thread when its own claim is stale (checked first), else for other - and writes nothing. The runtime fences every admission and settlement a parent drives on a child (its first commit, recoverChild, resumeChild, settleChild, the abandon at start) by the parent’s claim, so a stale parent task never registers or runs a child after a successor took the parent over. A wrapper that forwards the Commit whole needs no change. CheckpointerContract checks it.
• Child threads refuse direct calls. start, recover, resume, fork, updateState and deleteThread (and the Agent methods over them) on a child thread are refused with the new GraphError.ChildThread; delete its parent to delete it. An exhaustive match on GraphError needs the new case.
• Settled children are pruned. A thread keeps its newest 64 settled children (RunConfig.withChildRetention); older ones are deleted as the parent’s run completes and when its next run starts, where a child an earlier run left Running or Suspended is first settled Failed("abandoned", ...).
• Checkpoint.apply and RunConfig.apply gain parameters with defaults (origin, outcome; childRetention); calls that pass every previous argument resolve to the kept overloads unchanged.

Token budgets and the run’s deadline (#2047)

• AgentStatus.TokenLimitReached is a new case, so an exhaustive match on AgentStatus needs it. A turn ends with it when the run’s token budget is used; like StepLimitReached it is a completed turn with no answer. In Java and Kotlin, AgentStatusKind.TOKEN_LIMIT_REACHED is a new constant, so a switch or when without a default needs it too. TurnOutcome.TokenLimitReached likewise, for code reading a TurnOutput.
• RunBudgets.maxTokens is a new field (default None, no budget). RunBudgets(maxSupersteps, timeout, maxConcurrency) still compiles; a pattern case RunBudgets(s, t, c) needs a fourth binder. Set a budget with RunBudgets.default.withMaxTokens(n); a run can overshoot it by the one model call in flight (design 4.43).

Child-run supervision (#2046)

The runner that delegated calls use to run a child is internal (design 4.42); two things change for everyone:

• A tool call’s own model usage is counted. The usage a tool call’s task records (a tool that calls a model through its context, or a delegated child’s roll-up) now commits to the thread’s usage with the call’s result, so AgentResult.usage can be higher than before for agents with such tools.
• The state key ids delegation-prior-usage, delegation-input, delegation-counted and delegation-grant are reserved. Every agent graph registers them; a tool or middleware declaring a key with one of these ids is refused at build().

Child suspension through the parent (#2050)

A child that stops for an approval or a question now suspends its parent (design 4.46):

• AgentStatus.Suspended has a fifth field, delegated (default empty): a subagent’s interrupts, under composite ids <parent>~<child>. A positional pattern case AgentStatus.Suspended(a, q, m, b) needs a fifth binder, case AgentStatus.Suspended(a, q, m, b, _); one matching by type (case s: AgentStatus.Suspended) is unchanged. Answer a delegated interrupt with the usual helpers by its composite id. In Java and Kotlin, pending lists each one as its kind in the child, under the composite id.
• A suspended child is no longer a tool error. Code that waited for the parent model to see subagent '<id>' suspended now gets the parent Suspended instead, and must answer or abandon it.
• ToolLoop.questions no longer lists a task call waiting on its child; ToolLoop.delegatedQuestions does.

Explicit inheritance (#2049)

• RunEvent.RunFailed, RunCancelled and RunTimedOut gain steps: Long (default 0): how many of the run’s failed attempts were an agent’s model step, which a recovered child counts against its step grant (design 4.45). A positional pattern needs another binder: case RunEvent.RunCancelled(_) becomes RunCancelled(_, _), and RunFailed(message, usage) becomes RunFailed(message, usage, _). Stored events read unchanged (a missing steps is 0).

Nested streams and tracing (#2051)

A parent’s subscribers can follow its children, and a child’s trace names its parent (design 4.47):

• StreamEvent.Nested(child, event) is a new case, so an exhaustive match on StreamEvent needs it. Only a subscription made with nested = true receives one; add case _: StreamEvent.Nested => () where you match exhaustively.
• TraceEvent.AgentRunEnded gains parentThreadId and parentRunId (Option[String], default None), after timestamp. Construction is unchanged; a positional pattern case AgentRunEnded(t, r, a, s, m, u, ts) needs two more binders, or match on the type and read the fields.
• OpenTelemetry span names. The ended-run span Agent Run is now invoke_agent <agent>, and the tool span Tool Execution: <tool> is now execute_tool <tool>, each with gen_ai.operation.name; a dashboard or alert keyed on the old names needs the new ones. A child run’s span carries llm4s.parent.thread_id and llm4s.parent.run_id.
• RunHandle.subscribe takes nested (subscribe(capacity, nested = false)), so a RunHandle of your own implements the two-argument form. A call site is unchanged.
• A subagent traces to its parent’s Tracing. An agent built withTracing now also traces its subagents’ runs, each ending with its own AgentRunEnded.

Subagents in the harness profile (#2052)

HarnessProfile gains subagents: SubagentConfig (design 4.48). HarnessProfile() has SubagentConfig.none, so the minimal profile is unchanged; HarnessProfile.deepAgents and HarnessConfigLoader.load() now turn on the general-purpose subagent.

• A deepAgents agent offers a task tool. An application tool named task now clashes at build(): rename it, or turn the subagent off with profile.withSubagents(SubagentConfig.none) (or llm4s.agent.harness.subagents.generalPurpose.enabled = false).
• Permission rules decide task calls. A deepAgents profile with permission rules denies task unless a rule allows it, as for write_todos: add PermissionRule.allow(Operation.Invoke, "task:general-purpose") (or "task:*").
• withDelegation beside a harness is refused. The profile’s SubagentConfig.delegation sets the bounds: profile.withSubagents(profile.subagents.withDelegation(config)).
• A harness agent’s delegation middleware runs inside its permissions and outside its other middleware, not innermost, so its prompt section comes before the harness’s. Subagents added with AgentBuilder.withSubagents beside a harness follow the profile’s own.

Production adapters (Stage 5)

Not in a release yet. Stage 5 runs one agent service on several machines (#2129). Each slice adds its breaks here.

The PostgreSQL checkpointer (#2130)

llm4s-agent-checkpoint-postgres adds PostgresCheckpointer (design 4.50). Adopting it needs nothing of existing code; these change for every store:

• RetentionPolicy.maxAge is judged by commit time. prune compares the store’s clock with the time the store committed each checkpoint and event, by that same clock, no longer with the producer’s Checkpoint.createdAt and event timestamp, so a writer with a skewed clock makes nothing too old or too young. With one clock for the runtime and the store, as before, nothing changes. A custom Checkpointer records a commit time with each checkpoint and event and judges maxAge by it; CheckpointerContract checks it, with a skewed producer.
• SQLite schema 7, with no way back. It adds the commit times to llm4s_checkpoint_history and llm4s_checkpoint_events. Rows written before it have none and are judged by their producer’s timestamps, as before. A schema-6 build refuses a schema-7 file, so upgrade every process sharing a file together.
• GraphError.UnsupportedStoreSchema(store, found, supported) is a new case, returned when a store’s storage was migrated by a newer build (the PostgreSQL store’s refusal of a newer schema). An exhaustive match on GraphError needs it.
• Upgrading a PostgreSQL deployment. Instances of releases that change neither the store’s schema nor the checkpoint format can be replaced one at a time. A release that changes either needs every instance stopped before the first new one starts: an old instance that meets the newer schema refuses to open with UnsupportedStoreSchema rather than writing anything.

The thread registry (#2132)

Threads can be listed and searched (design 4.52). Code that only runs agents changes nothing; these change:

• Two new abstract Checkpointer methods. threads(query: ThreadQuery): Result[ThreadPage] and incompleteThreads(tenantId, after, limit): Result[ThreadPage]. A custom store implements both (a forwarding wrapper forwards both), tenant-scoped, newest and oldest first by the time the store last committed a checkpoint or metadata to the thread, and passes CheckpointerContract’s new cases.
• Commit.metadata: Option[ThreadMetadata]. A commit carrying it replaces the thread’s metadata, fenced like the rest of the commit; a store keeps it until another replaces it or the thread is deleted. Commit.apply keeps its earlier forms.
• RunConfig.metadata is now used. A top-level start with non-empty metadata sets it on the thread, and refuses metadata past ThreadMetadata’s limits (32 entries, keys of A-Z a-z 0-9 _ . : / - up to 64 characters, values up to 256) with a ValidationError before it claims anything. Change it later with GraphRuntime.updateMetadata.
• Schemas move on, with no way back. SQLite schema 8 and PostgreSQL schema 2 add the registry’s columns and indexes, backfilled from each thread’s latest checkpoint when the store opens. An older build refuses the new schema, so upgrade every process sharing a file or database together.

One trace per run (#2137)

• Positional patterns on TraceEvent.CustomEvent, TokenUsageRecorded, ErrorOccurred and AgentRunEnded need one more binder, for the new last field span: Option[TraceSpan] (case TraceEvent.CustomEvent(name, data, at, _)). Constructor calls are unchanged: span defaults to None.
• OpenTelemetry dashboards and queries. An agent run is now one trace: invoke_agent <agent> is the parent of the run’s chat <model> (new: one per model call, SpanKind.CLIENT), execute_tool <tool> (now SpanKind.INTERNAL) and its other events, and a delegated child’s invoke_agent is under the execute_tool task that started it. A query that counted one trace per event, or keyed on the old parentless spans, needs rewriting; filter by gen_ai.operation.name instead.
• Langfuse trace ids. A traced agent run’s Langfuse trace id is now its 32-hex trace id (the OpenTelemetry one), not the run id, which moves to the trace’s run_id metadata. A child run has no trace of its own: find it inside its parent’s trace, under the execute_tool task observation. A run traced without a placement (a hand-built AgentRunEnded) is unchanged.
• Continuing a trace. To put a run in an upstream request’s trace, pass its traceparent: RunConfig().withTraceContext(TraceContext.parse(header).toOption). A resume or recover with no context continues the trace its work started in.
• Stored data needs nothing. RunOrigin.trace and Checkpoint.trace are optional; a thread stored without them reads as None. A resume or recover given no traceContext now continues the trace its work started in (Checkpoint.trace) rather than starting one.

Store-wide retention and tombstone reclamation (#2059)

A store can be swept, a bounded page at a time, and a deleted thread’s tombstone is no longer kept for good (design 4.53). Code that only runs agents changes nothing; these change:

• A new abstract Checkpointer method. sweep(policy: SweepPolicy, after: Option[ThreadId], guard: SweepGuard): Result[SweepPage] prunes every thread the store holds by policy.retention, as prune does, and reclaims tombstones older than policy.tombstoneAge, at most policy.batchSize threads per call. A custom store implements it (a forwarding wrapper forwards it) and passes CheckpointerContract’s new cases; a store whose cases share one database also overrides the contract’s new isolatedCheckpointer hook with storage of its own.
• A reclaimed id numbers from the store’s reclamation mark. Once a sweep has reclaimed a tombstone, a new thread of an id with no tombstone - any id, not only a reclaimed one - numbers its events from the highest first number of every tombstone reclaimed, not from 1, so a reader of the reclaimed log still gets GraphError.ThreadDeleted. Code that assumed a new thread’s first event is 1 reads from afterSeq = 0 instead, as a reader of a deleted id already had to.
• Paging children after a reclaimed child refuses that cursor with a ValidationError, as for any id the store does not know as a child.
• Schemas move on, with no way back. SQLite schema 9 and PostgreSQL schema 3 record when each tombstone was deleted and hold the reclamation mark; the tombstones a store already holds are dated from the upgrade. An older build refuses the new schema, so upgrade every process sharing a file or database together.

The run queue (#2134)

Runs can be submitted to a durable queue that workers on any machine take (design 4.55). Code that only runs agents changes nothing; these change:

• A new abstract Checkpointer method. latestOfRuns(threadId, runIds): Result[Option[Checkpoint]] answers the newest checkpoint any of a set of runs committed on a thread, by an index on the run. A custom store implements it (a forwarding wrapper forwards it) and passes CheckpointerContract’s new cases; a store that also offers a RunQueue keeps, through prune, the newest checkpoint of every run an open entry recorded, and passes RunQueueContract.
• Schemas move on, with no way back. SQLite schema 10 and PostgreSQL schema 4, the step after store-wide retention’s, add run_id to every history row (backfilled from its JSON when the store opens, and indexed by thread, run and position) and the queue’s two tables. An older build refuses the new schema, so upgrade every process sharing a file or database together.

Cancel and the busy-thread policy (#2136)

A queued run can be cancelled from any machine, and a submission’s BusyThreadPolicy is acted on in full (design 4.57). A direct start, resume or recover on a busy thread is still refused with ThreadBusy, and AgentRun.cancel of a direct run still leaves its thread Running for recover. These change:

• Reject also refuses a thread a direct run holds. A Reject submission (the default) is refused with GraphError.ThreadBusy while a live claim holds the thread, not only while it has an open queue entry. Submit with Enqueue to wait behind the direct run instead.
• A new abstract RunQueue method. cancel(tenantId, target, reason) closes an unstarted entry at once and asks a started one to stop; a custom queue implements it, answers the request with every lease it grants or renews (Lease.stop) and on QueueEntry.stopRequest, and passes RunQueueContract’s new cases.
• Schemas move on again. SQLite schema 11 and PostgreSQL schema 5 add each queue entry’s stop request. An older build refuses the new schema, so upgrade every process sharing a file or database together.

Workspace command confinement (#1983)

Not in a release yet. On Linux, the workspace runner now starts every execute command under Landlock and, unless the network is allowed, a seccomp filter (#1983, design 4.39). Behaviour breaks, with no opt-out; off Linux (macOS, Windows) nothing changes:

• Commands lose the network by default. networkAllowed (default false) now binds commands: socket fails for every family but AF_UNIX, so sbt, curl or a package manager that fetches fails. Set networkAllowed = true where a command needs it; in a container, pass the whole config:

  new ContainerisedWorkspace(dir, image, 8080,
    sandboxConfig = Some(WorkspaceSandboxConfig.Permissive.copy(networkAllowed = true)))
  

• readOnlyPaths and allowedPaths bind commands too, not only the runner’s own file operations: a command cannot write under a read-only path (nor create one that does not exist yet), cannot create or delete entries directly in a directory above one, and cannot read outside allowedPaths.
• A kernel without Landlock ABI v3 (Linux 6.2), or an image without the llm4s-confine helper, refuses every command SANDBOX_UNAVAILABLE; it never runs one unconfined. An amd64 image under emulation (Docker Desktop’s Rosetta on Apple silicon) is such a case: build the image for the host architecture.
• A runner started outside the image grants commands /tmp only with -XX:+DisableAttachMechanism and -XX:+PerfDisableSharedMem, and then refuses git commands (EXECUTION_FAILED) while its java.io.tmpdir lies in /tmp or another path commands may write: set it to an owner-only directory elsewhere, as the image does (/var/tmp/llm4s-runner).

Workspace backends (#1906)

Not in a release yet. Stage 3 slice F2 (#1906) puts the workspace behind the harness’s FileBackend and SandboxBackend (design 4.33). Source and behaviour breaks, with no shims:

• org.llm4s.toolapi.WorkspaceTools and createDefaultWorkspaceTools are deleted. Their ToolFunctions called the workspace directly, with no locks, no replay guard and no bounds of their own. Build the harness tools over the workspace instead and give the agent a ToolSet; codegen.CodeWorker does this itself:

  // before
  WorkspaceTools.createDefaultWorkspaceTools(workspace).map(tools => builder.withTools(new ToolRegistry(tools)))
  // after
  for
    files   <- WorkspaceFileBackend("ws1", workspace)
    sandbox <- WorkspaceSandboxBackend("ws1", workspace)
    tools   <- ToolSet.of((FileTools(files) :+ ExecuteTool(sandbox))*)
  yield builder.withTools(tools)
  

  Old tool	Replacement	Notes
  explore_files	ls, glob	paths are absolute (/src), the workspace root is /
  read_file	read_file	numbered lines, paged by offset / limit
  write_file	write_file	creates parent directories; no append mode
  modify_file	edit_file	replaces exact text (old_string, new_string), not line ranges
  search_files	grep	a regex; include filters files
  execute_command	execute	workdir (default /), timeout_seconds; the result is text, exit code: N then stdout: and stderr:

  An execute call is the resource (Execute, <command>), so it needs a PermissionMiddleware rule if you use one.

• WorkspaceAgentInterface has two new methods and two new parameters. fileSnapshot(path, includeContent, followLinks) and deleteFile(path, expectedHash); writeFile(..., expectedHash) and executeCommand(..., maxOutputBytes) (each stream’s bound in UTF-8, JSON-escaped bytes). A class implementing the interface implements them; a caller that passes arguments by position is unaffected. The wire messages’ new fields have defaults, so an older client and a newer runner (or the reverse) still talk, without the new calls.
• ContainerisedWorkspace keeps the runner’s error code. A refused call used to throw WorkspaceAgentException(..., "EXECUTION_FAILED") with the code in the message; it now throws the runner’s own code (FILE_NOT_FOUND, PATH_NOT_ALLOWED, …), as WorkspaceAgentInterfaceRemote does. EXECUTION_FAILED now means the transport failed.
• The runner enforces the sandbox’s paths and limits (#870): a file operation or a command’s working directory outside allowedPaths is PATH_NOT_ALLOWED, a mutation under readOnlyPaths is PATH_READ_ONLY, and a write, append or line edit that would leave a file over maxFileSize is SIZE_LIMIT_EXCEEDED. Each path check is made on what the path names through symbolic links as well as on its text, so a file operation through a link that leads outside the workspace is now PATH_ESCAPE_ATTEMPT. A config that set these and relied on them being ignored now has its calls refused.
• Command output is kept as it was written, up to maxOutputSize bytes per stream (UTF-8, JSON-escaped, as it is carried). The direct executeCommand path used to read it line by line, ending every line with \n (a final line without one gained it, \r\n became \n) and keeping whole lines past the bound.

// a FileBackend and a SandboxBackend over a running ContainerisedWorkspace
for
  files   <- WorkspaceFileBackend("ws1", workspace)
  sandbox <- WorkspaceSandboxBackend("ws1", workspace)
yield (files, ExecuteTool(sandbox))

Direct style with Ox: llm4s-ox (#2332)

Nothing changes for existing code: llm4s-ox is a new, Experimental module over llm4s-core and Ox. To stream a reply as an Ox Flow, fan out, race providers or retry with Ox, add "org.llm4s" %% "llm4s-ox" and import org.llm4s.ox.*; see Direct style and Ox.

The doctor is llm4s-doctor (#2283)

The doctor moved from the unpublished llm4s-config-policy to the published llm4s-doctor, package org.llm4s.doctor (Doctor, DoctorCli, DoctorReport, DoctorCheck, CheckStatus, DoctorOptions). Run it from your own project (Doctor): sbt "Test/runMain org.llm4s.doctor.DoctorCli" with llm4s-doctor as a test dependency. In a checkout of this repository sbt "configPolicy/runMain org.llm4s.configpolicy.DoctorCli" still works.

• Doctor.run(options) checks the application’s own configuration; Doctor.run(source, options, ports) and DoctorPorts are internal.
• Without --classpath, the “Provider module” step reports the module as found “on this classpath”: the doctor’s classpath is now the one it is launched on, not one with every built-in provider module.

llm4s-repl, a new module (#2342)

Nothing to change: llm4s-repl is a new, experimental module, and no other module depends on it. To try llm4s from the Scala CLI REPL, a Metals worksheet or a script, add it (with a provider module) and import org.llm4s.repl.*; see Try llm4s in the REPL. Keep it out of production code: it prints to the console and keeps the conversation in memory.

llm4s-java-api and the Spring Boot starter no longer bring the provider modules (#1496)

llm4s-java-api depended on llm4s-openai, llm4s-anthropic, llm4s-ollama, llm4s-gemini and llm4s-openai-compatible, so every Java user received the OpenAI and Anthropic SDKs, OkHttp, Jackson and the Kotlin standard library (dependency footprint). They are now optional dependencies: the facade’s POM brings llm4s-core and llm4s-agent, and you declare the provider modules you call, at the same version. llm4s-spring-boot-starter does the same for llm4s-openai, llm4s-anthropic and llm4s-ollama, the providers its flat llm4s.provider keys build.

You use	Add
Llm4s.openAi(), Llm4s.azureOpenAi(), a section with provider = "openai", "azure" or "requesty", or openai/ embeddings	org.llm4s:llm4s-openai_3
Llm4s.anthropic(), or provider = "anthropic"	org.llm4s:llm4s-anthropic_3
Llm4s.gemini(), or provider = "gemini" or "vertexai"	org.llm4s:llm4s-gemini_3
Llm4s.ollama(), provider = "ollama", or ollama/ embeddings	org.llm4s:llm4s-ollama_3
Llm4s.openAiCompatible(), or DeepSeek, Mistral, Cohere chat, OpenRouter, Z.ai, xAI or openai-compatible	org.llm4s:llm4s-openai-compatible_3

dependencies {
    implementation("org.llm4s:llm4s-java-api_3:$llm4sVersion")
    implementation("org.llm4s:llm4s-openai_3:$llm4sVersion")   // was brought by llm4s-java-api
}

Without the module, nothing fails at startup: a typed builder’s build(), or the starter’s llm4s.provider, returns a CONFIGURATION failure that names the artifact to add, and a configured section whose provider is absent fails as it always did for modules outside the facade. With llm4s-bom imported, the provider module needs no version. The Kotlin API (llm4s-kotlin-api) depends on llm4s-java-api, so the same applies to it.

Smaller breaks since 0.4.1

Changes too small for a section of their own. Each is listed under Breaking changes in the CHANGELOG.

One proxy and TLS setting for every provider client (#2273)

Every provider client now honours llm4s.http (Corporate networks):

• Core’s reference.conf binds llm4s.http.proxy.url to HTTPS_PROXY and nonProxyHosts to NO_PROXY. An application whose environment sets HTTPS_PROXY now sends provider traffic through that proxy (loopback hosts stay direct). To keep the old direct route, set llm4s.http.proxy.url = "", or list the hosts in NO_PROXY. A value that is not http://host:port fails the load, naming llm4s.http.proxy.url.
• http is a built-in key of a provider section, like timeouts: a provider declaring an extra named http must rename it.
• BedrockConfig and WatsonXConfig (public case classes) gain a last field, network, with a default: a pattern match on all their fields gains one _. EmbeddingProviderConfig and OpenAICompatibleClient.Settings gain network too; their previous apply signatures remain.
• Llm4sHttpClient.create() is unchanged and ignores llm4s.http; Llm4sHttpClient.create(network) takes the settings Llm4sConfig.httpNetwork() loads.

The embeddings sample’s settings left Llm4sConfig (#2265)

Llm4sConfig exposed settings only the embeddings sample (EmbeddingExample in modules/samples) read, and llm4s-core’s reference.conf bound their variables. Under the single-consumer rule they moved to the samples with their keys (#2265), and the duplicate entry points went:

Removed from llm4s-core	Now
Llm4sConfig.embeddingsChunking(), loadEmbeddingsChunking(), EmbeddingsChunkingSettings	EmbeddingSampleConfig.chunking(), EmbeddingChunkingSettings (samples)
Llm4sConfig.embeddingsInputs(), loadEmbeddingsInputs(), EmbeddingsInputSettings	EmbeddingSampleConfig.inputs(), EmbeddingInputSettings (samples)
Llm4sConfig.embeddingsUi(), loadEmbeddingsUiSettings(), EmbeddingsUiSettings	EmbeddingSampleConfig.ui(), EmbeddingUiSettings (samples)
Llm4sConfig.experimentalStubsEnabled	EmbeddingSampleConfig.experimentalStubs() (samples)
Llm4sConfig.localEmbeddingModels()	EmbeddingSampleConfig.localModels() (samples); or build a LocalEmbeddingModels
Llm4sConfig.loadTextEmbeddingModel()	Llm4sConfig.textEmbeddingModel(), unchanged
ConfigKeys.EMBEDDING_INPUT_PATH, EMBEDDING_QUERY, CHUNK_SIZE, CHUNK_OVERLAP, CHUNKING_ENABLED	the string literals
llm4s.embeddings.{inputPath,inputPaths,query,chunking,ui,experimentalStubs,localModels}	llm4s.samples.embeddings.* in the samples’ application.conf

EmbeddingSampleConfig and its types are in org.llm4s.samples.embeddingsupport, which is not published: an application that used these settings defines its own, or copies the sample’s. LocalEmbeddingModels stays in llm4s-core, as FileEmbeddingConfig (llm4s-rag) and ModelSelector take it, so build one directly:

import org.llm4s.llmconnect.config.LocalEmbeddingModels

val localModels = LocalEmbeddingModels(imageModel = "openclip-vit-b32", audioModel = "wav2vec2-base", videoModel = "timesformer-base")

The environment variables (EMBEDDING_INPUT_PATH, EMBEDDING_INPUT_PATHS, EMBEDDING_QUERY, CHUNK_SIZE, CHUNK_OVERLAP, CHUNKING_ENABLED, MAX_ROWS_PER_FILE, TOP_DIMS_PER_ROW, GLOBAL_TOPK, SHOW_GLOBAL_TOP, COLOR, TABLE_WIDTH, ENABLE_EXPERIMENTAL_STUBS, LLM4S_EMBEDDINGS_LOCAL_{IMAGE,AUDIO,VIDEO}_MODEL) still drive the sample, which binds them itself; no llm4s library module reads them any more. Their defaults are unchanged. One difference: a value that is set but cannot be read (CHUNK_SIZE=abc) is now a ConfigurationError instead of silently falling back to the default.

The Spring Boot starter’s refused completeAsync fails with an LlmException (#2276)

LLM4STemplate.completeAsync now delegates to JLlmClient.completeAsync (#2276), whose futures fail only with an LlmException. When the llm4sTaskExecutor refuses a call (its queue full, or shut down), the future’s cause is an LlmException of kind OTHER whose own cause is the RejectedExecutionException; it was the RejectedExecutionException itself. Code that tests e.getCause() instanceof RejectedExecutionException tests e.getCause().getCause() instead.

Postgres stores take a javax.sql.DataSource (#2300)

The Postgres stores accept a connection pool the application already has, of any kind (#2300), so the parameters that took HikariCP’s concrete pool are widened to javax.sql.DataSource:

Before	Now
PgVectorStore.create(dataSource: HikariDataSource, tableName, ownsDataSource)	PgVectorStore.create(dataSource: DataSource, tableName, ownsDataSource)
PgVectorStore(dataSource: HikariDataSource, tableName)	PgVectorStore(dataSource: DataSource, tableName)
PgKeywordIndex(dataSource: HikariDataSource, tableName[, language])	PgKeywordIndex(dataSource: DataSource, tableName[, language])
-	HybridSearcher.pgvectorShared(dataSource, vectorTableName, keywordTableName[, defaultStrategy])
-	PgSearchIndex.fromDataSource(dataSource, vectorTableName)
-	PostgresMemoryStore.fromDataSource(dataSource, tableName, embeddingService)

A HikariDataSource is a DataSource, so code that passes one compiles unchanged; code compiled against 0.5.x must be recompiled. A store built over a DataSource never closes it unless PgVectorStore.create is given ownsDataSource = true, and then only if the DataSource is AutoCloseable. PgSearchIndex.close() used to close its pool unconditionally; it now closes only a pool it created from a PgConfig. A store over PGSimpleDataSource or another non-pooling DataSource opens a connection per call. See Sharing your application’s connection pool.

Completion.created is an Instant (#2262)

A point in time in the public API is a java.time.Instant (#2262), and Completion.created was the last raw timestamp: it was a Long of epoch seconds. Completion(created = ...), withCreated and StreamingAccumulator.toCompletion(created) now take an Instant, and completion.created returns one. Providers that report a creation time (the OpenAI-format created field) still send epoch seconds on the wire; the client converts it, and a reply without one gives Instant.EPOCH, as 0L did before. Providers that report none stamp the time the reply arrived, as before.

import java.time.Instant

val completion = Completion(id = "1", created = Instant.ofEpochSecond(1700000000L), content = "hi", model = "m",
  message = AssistantMessage("hi"))                     // was created = 1700000000L
val seconds: Long = completion.created.getEpochSecond   // where code needs the old number

Test fixtures that passed 0L pass Instant.EPOCH.

Unused speculative core types removed (#2345)

Three groups of public types in llm4s-core had no user in any llm4s module and are deleted before the 1.0 baseline (#2345), rather than frozen into it:

• org.llm4s.types.typeclass.Encoder and its implicit EncoderOps: encode to JSON with upickle directly (upickle.default.writeJs(value) given a ReadWriter), or keep a one-method trait of your own.
• org.llm4s.types.typeclass.LLMCapable (with LLMCapableOps and its String / Conversation instances): build a Conversation with Conversation.create(UserMessage(text)) and read a reply with completion.content.
• org.llm4s.llmconnect.model.CompletionChunk and ChunkDelta: no client ever produced them. Streaming delivers StreamedChunk (content, toolCall, finishReason, thinkingDelta) to streamComplete’s onChunk.

Tier hygiene before the baseline (#2358)

Public types of frozen modules that would otherwise have been frozen by mistake at the MiMa baseline (#2358) are now marked or placed to match their tier:

• org.llm4s.trace.AnsiColors is private[llm4s]: only ConsoleTracing and a sample used it. Define the escape codes you need yourself ("\u001b[32m" is green, "\u001b[0m" resets), or use a library such as fansi.
• org.llm4s.core.safety.UsingOps moved from llm4s-core to llm4s-agent-tools, its only consumer, and org.llm4s.llmconnect.utils.ModelSelector from llm4s-core to llm4s-rag. Packages are unchanged, so imports stay as they are; add the dependency if you used one without it. scala.util.Using.resource does what UsingOps.using does without either.
• The org.llm4s.assistant types (AssistantAgent, Command, SessionManager, SessionState, SessionInfo, SessionSummary) are @Experimental, matching their Beta tier in 1.0 Scope: MiMa does not freeze them, and they can change in a minor release with a migration note. Nothing about them changed in this release.

The file tools’ default blockedPaths covers each platform (#1408)

FileConfig()’s blockedPaths defaults to the new FileConfig.DefaultBlockedPaths, the system and credential locations of the platform the JVM runs on (#1408, finding F6). It was /etc, /var, /sys, /proc and /dev everywhere, which on Windows blocked nothing. Now:

• Linux and other POSIX systems add /boot, /root and /run; macOS adds /private/etc, /private/var and /Library/Keychains; Windows has C:\Windows, C:\ProgramData\Microsoft\Crypto and C:\ProgramData\ssh instead of the POSIX entries.
• Every platform blocks the credential files under the user’s home (.ssh, .gnupg, .aws, .azure, .config/gcloud, .kube, .docker, .config/gh, .netrc, .git-credentials, .pgpass, .password-store, .npmrc, .pypirc, .m2/settings.xml, .m2/settings-security.xml, .ivy2/.credentials, .sbt/.credentials), plus the GNOME keyring on Linux, ~/Library/Keychains on macOS, and on Windows _netrc, NTUSER.DAT and the Credential Manager, DPAPI, gcloud, GnuPG, GitHub CLI and PostgreSQL stores under AppData.
• A blocked entry is matched without regard to case, so /ETC/hosts is blocked by /etc.

An agent that read one of these through the default config is now refused. To keep reading it, pass your own list, best built from the default: FileConfig(blockedPaths = FileConfig.DefaultBlockedPaths.filterNot(_.endsWith(".npmrc"))). The tool descriptions write the user’s home directory as ~. A config that sets blockedPaths itself is unchanged, apart from the case-insensitive match.

Console tracing prints no content by default (#2311)

The default tracing mode is still console, but ConsoleTracing now prints metadata only (#2311): event types, timestamps, models, ids, tool names, success flags, durations, token counts and costs. The query of AgentInitialized, the content of CompletionReceived, the input and output of ToolExecuted, the message and context of an error and an image generation’s error message print as [withheld: N chars], and a CustomEvent shows its field names instead of its data. An application that built tracing from Llm4sConfig.tracing() and never set TRACING_MODE no longer sends prompts and answers into whatever collects its stdout.

No code has to change. To see content again while developing, opt in:

llm4s.tracing.console.captureContent = true   # or LLM4S_TRACING_CONSOLE_CAPTURE_CONTENT=true

or construct the tracer with new ConsoleTracing(captureContent = true); new ConsoleTracing() prints metadata only. A captureContent value other than true or false is a ConfigurationError from Tracing.fromSettings (Tracing.create logs it and falls back to NoOpTracing). A hand-built TracingSettings(TracingMode.Console) has no extras, so it gets the safe default.

OpenTelemetry spans follow the GenAI conventions; content is opt-in (#2275)

llm4s-observability-otel names and annotates model calls by the OpenTelemetry GenAI semantic conventions, version 1.37.0 (#2275; tables in Monitoring). Dashboards, alerts and queries that select the old names need updating:

Before	Now
span LLM Completion	chat <model> (CLIENT)
span Embedding Usage - <operation>	embeddings <model> (CLIENT); the label stays in operation
span Error of a TracingLLMClient or EmbeddingClient call	chat <model> / embeddings <model> with error.type and status ERROR
attribute completion_id	gen_ai.response.id
attribute tool_calls	llm4s.model_call.tool_calls
attribute content (first 1000 characters of the answer)	gen_ai.output.messages, only with captureContent
attribute input of LLM4S Agent Run, tool.input / tool.output of execute_tool	only with captureContent

Agent spans keep their names (invoke_agent, chat, execute_tool) and gain gen_ai.provider.name.

No span carries message content by default. To record it again, redacted, set:

llm4s.tracing.opentelemetry.captureContent = true   # or OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=true

or pass OpenTelemetryConfig(captureContent = true). Plain and streamed completions are traced by wrapping the client, TracingLLMClient(client, tracing, providerConfig); an EmbeddingClient built by EmbeddingClient.from reports its provider. TraceEvent.CompletionReceived, ErrorOccurred, EmbeddingUsageRecorded and AgentRunEnded gained fields (provider, duration and the call’s details), so a positional pattern on one of them no longer compiles: match on the type and read the fields (case e: TraceEvent.CompletionReceived => e.model). Their provider is an Option[ProviderId], and withProvider takes a ProviderId (.withProvider(ProviderId("openai"))); toJson still writes the plain id. LLMClient and EmbeddingClient gained providerId: Option[ProviderId] (default None), which a client that wraps another should forward, as did the agent’s ModelStep; AgentEvents.ModelCallCompleted carries it as provider: Option[ProviderId], stored as the plain id, so events stored before it still read. EmbeddingClient takes providerId as its fourth parameter, before the ModelRegistryService: Java and Kotlin callers of the constructor pass Option.empty() there.

Only remote calls are CLIENT spans (#2531)

llm4s-observability-otel gives an event the CLIENT span kind only when it records a request to a remote service: a completion, an embedding or image generation call, an agent’s model call, or the failure of one of them (an ErrorOccurred with an operation). Every other event, and any event added later, is INTERNAL, OpenTelemetry’s default (#2531). A dashboard or query that selects CLIENT spans no longer sees:

Span	Was	Now
Error (an ErrorOccurred with no operation)	CLIENT	INTERNAL
ToolExecuted’s execute_tool <tool>	CLIENT	INTERNAL, as an agent’s execute_tool span already was
CostRecorded, CacheHit, CacheMiss, RAGOperationCompleted	CLIENT	INTERNAL
a CustomEvent with no span	CLIENT	INTERNAL, as a placed one already was

No API changes.

ServiceError extractor and isRecoverableStatus (#2266)

ServiceError’s extractor binds all five fields, retryAfter last (#2266); the hand-written four-field unapply, which left out retryAfter, is gone. A positional pattern needs a fifth binder:

case ServiceError(message, status, provider, requestId)    => ... // before
case ServiceError(message, status, provider, requestId, _) => ... // after
case e: ServiceError                                       => ... // or match on the type

isRecoverableStatus is now a method of ServiceError rather than of the public implicit class ServiceError.ServiceErrorOps, which an import ServiceError.* would have pulled in. error.isRecoverableStatus compiles unchanged; delete any import ServiceError.ServiceErrorOps.

Not a source change, but visible: HttpResponse#ensureSuccess now puts the response’s request id (x-request-id, request-id, x-amzn-requestid or apim-request-id) and its Retry-After delay on the ServiceError, so its message gains (Request ID ...) when the provider sent one, and Result#mapServiceError keeps both fields. A retry now waits the delay the provider asked for rather than the default backoff.

LLMClientPekko.streamComplete materializes Future[Done] (#1848)

streamComplete is now a Source[StreamedChunk, Future[Done]], like every AgentPekko stream (#1848). The future completes once the provider call has returned, after a cancellation too, so a caller can wait until nothing is left running. Code that only runs the stream is unchanged; code that names the type changes it, and code that wanted the old value drops it:

val chunks: Source[StreamedChunk, NotUsed] = client.streamComplete(conversation).mapMaterializedValue(_ => NotUsed)

The OpenAI-compatible provider configs are growth-safe (#2264)

DeepSeekConfig, ZaiConfig, XaiConfig, MistralConfig and CohereConfig (llm4s-openai-compatible) follow the growth-prone pattern, as OpenAIConfig and OpenAICompatibleConfig beside them already did (#2264). The classes are final, new and .copy are private, and:

• the companion apply keeps its argument order - apiKey, model, baseUrl, contextWindow, reserveCompletion, timeouts - and every field after model now has a default: DEFAULT_BASE_URL, the new DEFAULT_CONTEXT_WINDOW and DEFAULT_RESERVE_COMPLETION, and ProviderTimeouts.default. Positional and named calls compile unchanged;
• apply(apiKey, model) is the entry point for Java and Kotlin, which do not see Scala default arguments;
• withApiKey, withBaseUrl, withContextWindow and withReserveCompletion join withModel and withTimeouts.

fromValues, which validates and resolves the context window from the model, is still the preferred way in.

import org.llm4s.llmconnect.config.DeepSeekConfig

val config = DeepSeekConfig("sk-...", "deepseek-chat").withContextWindow(64000) // was .copy(contextWindow = 64000)

MemoryManagerConfig is growth-safe (#1847)

MemoryManagerConfig follows the growth-prone pattern (#1847): build it with MemoryManagerConfig(...) (every option keeps its default) or a preset, and change it with withAutoExtractEntities, withDefaultImportance, withContextTokenBudget and withConsolidationConfig. new and .copy are private:

val config = MemoryManagerConfig.default.withContextTokenBudget(500) // was .copy(contextTokenBudget = 500)

toString is now the case-class form, MemoryManagerConfig(false,0.5,2000,ConsolidationConfig(50,false)).

Core config types are growth-safe (#2346)

ToolExecutionConfig, LlmClientOptions, EmbeddingProviderConfig, EmbeddingModelConfig, LocalEmbeddingModels, ModelRegistryConfig and TracingSettings (llm4s-core) follow the growth-prone pattern (#2346), so a new field - ToolExecutionConfig has an obvious next one - no longer breaks Java and Kotlin callers or compiled code. Each is a final case class whose constructor and copy are private:

• Construction is unchanged: X(...) with the same arguments, positional or named, and the same defaults, still compiles. new X(...) does not; drop the new. ToolExecutionConfig.default is new, beside LlmClientOptions.default and ModelRegistryConfig.default.

• Replace .copy with a setter, one per field:

  Type	Setters
  ToolExecutionConfig	withTimeout, withRetryPolicy (the value or an Option)
  LlmClientOptions	withMetrics, withExchangeLogging
  EmbeddingProviderConfig	withBaseUrl, withModel, withApiKey, withTimeouts
  EmbeddingModelConfig	withName, withDimensions
  LocalEmbeddingModels	withImageModel, withAudioModel, withVideoModel
  ModelRegistryConfig	withResourcePath, withFilePath, withUrl (the value or an Option)
  TracingSettings	withMode, withExtras

  val cfg  = embeddingConfig.withModel("text-embedding-3-large")   // was .copy(model = ...)
  val exec = ToolExecutionConfig.default.withTimeout(5.seconds)    // was .copy(timeout = Some(5.seconds))
  val tr   = settings.withMode(TracingMode.NoOp)                    // was settings.copy(mode = TracingMode.NoOp)
  

• Java and Kotlin: call the companion’s apply (EmbeddingModelConfig.apply("m", 1536)) instead of the constructor, and chain the setters.
• ToolExecutionConfig, EmbeddingModelConfig and TracingSettings are now final.

Pattern matching (case EmbeddingModelConfig(name, dims)) and the fields are unchanged.

Core data types are growth-safe (#2356)

HttpResponse, HttpRawResponse, StreamingHttpResponse, JsonHttpResponse (org.llm4s.http), EmbeddingResponse, EmbeddingUsage, UsageSummary, ModelUsage (org.llm4s.llmconnect.model), ModelPricing (org.llm4s.model), ProviderExchange (org.llm4s.llmconnect) and DiscoveredModel (org.llm4s.config) follow the growth-prone pattern (#2356). Each is a final case class whose constructor and copy are private:

• Construction is unchanged: X(...) with the same arguments, positional or named, and the same defaults, still compiles - HttpResponse(200, body) included. new X(...) does not; drop the new.

• Replace .copy with a setter, one per field:

  Type	Setters
  HttpResponse, HttpRawResponse, StreamingHttpResponse, JsonHttpResponse	withStatusCode, withBody, withHeaders
  EmbeddingResponse	withEmbeddings, withMetadata, withModality, withDim, withUsage (the last three take the value or an Option)
  EmbeddingUsage	withPromptTokens, withTotalTokens
  ModelPricing	with<Field> for each price, e.g. withInputCostPerToken (the value or an Option)
  UsageSummary	withRequestCount, withInputTokens, withOutputTokens, withThinkingTokens, withTotalCost, withByModel
  ModelUsage	withRequestCount, withInputTokens, withOutputTokens, withThinkingTokens, withTotalCost
  ProviderExchange	with<Field> for each field, e.g. withOutcome; an Option field’s setter takes the value or an Option
  DiscoveredModel	withName, withProvider, withMetadata

  val throttled = response.withStatusCode(429)                    // was response.copy(statusCode = 429)
  val sized     = embeddings.withDim(1536)                        // was .copy(dim = Some(1536))
  val priced    = pricing.withInputCostPerToken(2.5e-6)           // was .copy(inputCostPerToken = Some(2.5e-6))
  

• JSON is unchanged. ModelPricing, UsageSummary and ModelUsage keep their ReadWriters and their constructor defaults, so they write the same bytes and a document that omits a field still reads.
• UsageSummary.bigDecimalRw is private. It was a public implicit ReadWriter[BigDecimal] that import UsageSummary._ (or .*) pulled into scope, changing how every BigDecimal in that file serialised. UsageSummary.rw still uses it, so UsageSummary JSON is the same; code that relied on the import for its own BigDecimal fields gets upickle’s default codec, which also writes a string. Define your own given if you need UsageSummary’s lenient reading (a number, a string, anything else as zero).
• Java and Kotlin: call the companion’s apply (HttpResponse.apply(200, "ok", headers)) instead of the constructor, and chain the setters.
• HttpResponse, HttpRawResponse, StreamingHttpResponse, EmbeddingUsage, ModelPricing, UsageSummary and ModelUsage are now final.

Pattern matching (case HttpResponse(status, body, headers)) and the fields are unchanged.

llmconnect.model types are growth-safe (#2378)

Conversation, SystemMessage, ToolCall, ResponseFormat.JsonSchema, EmbeddingVector, MultimediaEmbeddingRequest, ImageInput, AudioInput, VideoInput and RawMediaInput (org.llm4s.llmconnect.model) follow the growth-prone pattern (#2378). Each is a final case class whose constructor and copy are private:

• Construction is unchanged: X(...) with the same arguments, positional or named, and the same defaults, still compiles. new X(...) does not; drop the new.

• Replace .copy with a setter, one per field:

  Type	Setters
  Conversation	withMessages (beside addMessage and addMessages)
  SystemMessage	withContent
  ToolCall	withId, withName, withArguments
  ResponseFormat.JsonSchema	withSchema, withName, withStrict
  EmbeddingVector	withId, withModality, withModel, withDim, withValues, withMeta
  MultimediaEmbeddingRequest	withInputs, withModel, withModality, withMeta
  ImageInput	withWidth, withHeight, withChannels, withData
  AudioInput	withSamples, withSampleRate
  VideoInput	withFrameWidth, withFrameHeight, withChannels, withFps, withFrames
  RawMediaInput	withData, withMime

  val pruned = conversation.withMessages(kept)                   // was conversation.copy(messages = kept)
  val edited = call.withArguments(ujson.Obj("city" -> "Lyon"))   // was call.copy(arguments = ...)
  val loose  = format.withStrict(false)                          // was format.copy(strict = false)
  

• JSON is unchanged. Conversation, SystemMessage and ToolCall keep their ReadWriters and write the same bytes.
• Java and Kotlin: call the companion’s apply (ToolCall.apply("id", "name", args)) instead of the constructor, and chain the setters.
• Conversation, ToolCall and ResponseFormat.JsonSchema are now final.

Pattern matching (case ToolCall(id, name, args)) and the fields are unchanged.

TraceEvent cases are growth-safe (#2377)

The thirteen TraceEvent cases - AgentInitialized, CompletionReceived, ToolExecuted, ErrorOccurred, TokenUsageRecorded, AgentRunEnded, CustomEvent, EmbeddingUsageRecorded, CostRecorded, RAGOperationCompleted, ImageGenerationCompleted, CacheHit and CacheMiss (org.llm4s.trace) - follow the growth-prone pattern (#2377). Each is a final case class whose constructor and copy are private:

• Construction is unchanged: TraceEvent.X(...) with the same arguments, positional or named, and the same defaults (timestamp = Instant.now(), span = None, and each optional metric None) still compiles. new X(...) does not; drop the new.

• Replace .copy with a setter, one per field, named with<Field> - withTimestamp on every case, withSpan on the four that carry one, withName on CustomEvent, withStatus/withMessages/withUsage on AgentRunEnded, and so on. An Option field’s setter takes the value or an Option (withSpan(span), withSpan(None)).

  val renamed = event.withName("prefix_" + event.name)   // was event.copy(name = "prefix_" + event.name)
  val placed  = event.withSpan(span)                     // was event.copy(span = Some(span))
  

• Java and Kotlin: call the companion’s apply instead of the constructor, and chain the setters.
• The thirteen cases are now final. Pattern matching (case TraceEvent.CacheHit(similarity, threshold, ts)), the fields and toJson are unchanged.

AssistantError cases are growth-safe (#2383)

The seven AssistantError cases - IOError, EOFError, DisplayError, SessionError, FileError, SerializationError and CommandParseError (org.llm4s.error) - follow the growth-prone pattern (#2383), as the LLMError cases already did. Each is a final case class whose constructor and copy are private:

• Construction is unchanged: AssistantError.X(...) with the same arguments, positional or named, and the same defaults (cause = None, llmCause = None, EOFError’s operation = "read") still compiles, as do the smart constructors (AssistantError.fileReadFailed, sessionNotFound, …). new X(...) does not; drop the new.

• Replace .copy with a setter, one per field: withMessage and withCause on every case that has the field, withOperation, withSessionId, withLlmCause, withPath, withDataType, withDisplayType, withCommand and withParseOperation. An Option field’s setter takes the value or an Option (withCause(ex), withCause(None)).

  val withLlm = err.withLlmCause(llmError)   // was err.copy(llmCause = Some(llmError))
  

• Java and Kotlin: call the companion’s apply instead of the constructor, and chain the setters.
• Pattern matching (case AssistantError.FileError(message, path, operation, cause)), the fields, context and formatted are unchanged.

RerankProviderConfig is growth-safe (#2478)

RerankProviderConfig (org.llm4s.reranker, llm4s-rag) gains apiKeyProvider, the key a reranker asks for per request (a rotated apiKeyFile), and follows the growth-prone pattern (#2478): its constructor and copy are private.

• Construction is unchanged: RerankProviderConfig(baseUrl, apiKey, model), positional or named, still compiles. new RerankProviderConfig(...) does not; drop the new.

• Replace .copy with a setter: withBaseUrl, withApiKey, withModel and withApiKeyProvider (the provider or an Option).

  val other = config.withModel("rerank-v3.5")   // was config.copy(model = "rerank-v3.5")
  

• Pattern matching and the fields are unchanged; pattern matching now binds four fields.

Provider SPI, caching, streaming and exchange-sink types are growth-safe (#2379)

JsonlProviderExchangeSink (org.llm4s.llmconnect), CacheEntry, CacheStats (org.llm4s.llmconnect.caching), OpenAICompatibleClient.Settings (llm4s-openai-compatible), SSEParser.SSEEvent (org.llm4s.llmconnect.streaming) and EmbeddingProviderSection, ProviderConfigKey, ProviderModuleReport, ProviderDiscoveryFailure, ProviderIdCollision and ProviderRegistryReport (org.llm4s.llmconnect.spi) follow the growth-prone pattern (#2379). Each is a final case class whose constructor and copy are private:

• Construction is unchanged: X(...) with the same arguments, positional or named, and the same defaults, still compiles. new X(...) does not; drop the new.

• Replace .copy with a setter, one per field (an Option field’s setter takes the value or an Option):

  Type	Setters
  JsonlProviderExchangeSink	withPath
  CacheEntry	withEmbedding, withResponse, withTimestamp, withOptions
  CacheStats	withSize, withHits, withMisses, withTotalRequests, withHitRatePercent
  OpenAICompatibleClient.Settings	withProviderName, withDisplayName, withModel, withBaseUrl, withCredential, withContextWindow, withReserveCompletion, withTimeouts
  SSEEvent	withData, withEvent, withId, withRetry
  EmbeddingProviderSection	withApiKey, withBaseUrl, withModel, withTimeouts
  ProviderConfigKey	withName, withDescription, withRequired, withDefault, withEnv, withDeprecatedAliases
  ProviderModuleReport	withModuleClass, withProviderIds, withSource, withEmbeddingProviderIds
  ProviderDiscoveryFailure	withDetail, withFailure
  ProviderIdCollision	withKind, withId, withKeptModule, withDroppedModule
  ProviderRegistryReport	withDiscovered, withModules, withFailures, withCollisions

  // was ProviderConfigKey.optional(...).copy(env = Some("AWS_PROFILE"))
  val key = ProviderConfigKey.optional("profile", "the AWS profile").withEnv("AWS_PROFILE")
  // was section.copy(apiKey = resolvedKey)
  val keyed = section.withApiKey(resolvedKey)
  // was settings.copy(baseUrl = "https://eu.acme.example/v1")
  val moved = settings.withBaseUrl("https://eu.acme.example/v1")
  

• Java and Kotlin: call the companion’s apply (CacheStats.apply(size, hits, misses, total, rate)) instead of the constructor, and chain the setters.
• CacheEntry, CacheStats and SSEEvent are now final.

Pattern matching (case ProviderConfigKey(name, description, required, default, env, aliases)) and the fields are unchanged.

Tool schemas and tool-call types are growth-safe (#2381)

StringSchema, NumberSchema, IntegerSchema, BooleanSchema, ArraySchema, ObjectSchema, PropertyDefinition, SafeParameterExtractor, ToolCallRequest and ToolFunction (org.llm4s.toolapi) follow the growth-prone pattern (#2381). Each is a final case class whose constructor and copy are private:

• Construction is unchanged: Schema.string(...), Schema.property(...), ToolBuilder and X(...) with the same arguments and defaults still compile. new X(...) does not; drop the new (ToolFunction[T, R](...)).

• Replace .copy with a setter:

  Type	Setters
  StringSchema	withDescription, withEnum, withMinLength, withMaxLength (the last three also take an Option), withLengthConstraints
  NumberSchema, IntegerSchema	withDescription, withMinimum, withMaximum, withExclusiveMinimum, withExclusiveMaximum, withMultipleOf (each also takes an Option), withRange, withExclusiveRange; NumberSchema also withIsInteger
  BooleanSchema	withDescription
  ArraySchema	withDescription, withItemSchema, withMinItems, withMaxItems (the value or an Option), withSizeConstraints, withUniqueItems
  ObjectSchema	withDescription, withProperties, withProperty, withAdditionalProperties
  PropertyDefinition	withName, withSchema, withRequired
  SafeParameterExtractor	withParams, withCall (the call or an Option)
  ToolCallRequest	withFunctionName, withArguments
  ToolFunction	withName, withDescription, withSchema, withHandler

  val renamed = tool.withName("search_v2")                       // was tool.copy(name = "search_v2")
  val open    = schema.withAdditionalProperties(true)            // was schema.copy(additionalProperties = true)
  

• Java and Kotlin: new SafeParameterExtractor(params) becomes SafeParameterExtractor.apply(params); the other types are built through their companion’s apply or Schema.

Pattern matching (case ToolCallRequest(name, args)) and the fields are unchanged.

Config, model and util types are growth-safe (#2384)

Llm4sConfig.TextEmbeddingModelSettings, ProvidersConfigModel.ProvidersConfig (org.llm4s.config), TransformationResult (org.llm4s.model) and SecretPatterns.SecretMatch (org.llm4s.util) follow the growth-prone pattern (#2384). Each is a final case class whose constructor and copy are private:

• Construction is unchanged: X(...) with the same arguments, positional or named, still compiles; TransformationResult(options, messages) keeps requiresFakeStreaming = false as its default. new X(...) does not; drop the new.

• Replace .copy with a setter, one per field, named with<Field>:

  Type	Setters
  TextEmbeddingModelSettings	withProvider, withModelName, withDimensions
  ProvidersConfig	withSelectedProvider (a ProviderName or an Option), withNamedProviders
  TransformationResult	withOptions, withMessages, withRequiresFakeStreaming
  SecretMatch	withSecretType, withValue, withStartIndex, withEndIndex

  val fake = result.withRequiresFakeStreaming(true) // was result.copy(requiresFakeStreaming = true)
  

• Java and Kotlin: call the companion’s apply (TransformationResult.apply(options, messages, false)) instead of the constructor, and chain the setters.

Pattern matching (case SecretMatch(secretType, value, start, end)) and the fields are unchanged.

OpenAIToolHelper is internal (#2360)

org.llm4s.toolapi.OpenAIToolHelper (llm4s-openai) is now private[llm4s] (#2360). Its methods took and returned openai-java’s ChatCompletionCreateParams.Builder and ChatCompletionTool, which would have tied llm4s-openai’s 1.x compatibility to that SDK’s. OpenAIClient still uses it. If you built openai-java requests yourself, take the tool definitions as JSON and hand them to the SDK:

val tools: ujson.Arr = new ToolRegistry(myTools).getOpenAITools()

ToolCallError and ToolParameterError cases (#2382)

ToolCallError.UnknownFunction and ToolParameterError.MissingParameter (org.llm4s.toolapi) follow the growth-prone pattern (#2382): each carries a self-correction hint that may gain company (suggested names, say), so each is a final case class whose constructor and copy are private.

• Construction is unchanged: ToolCallError.UnknownFunction("x"), UnknownFunction("x", Seq("a", "b")) and ToolParameterError.MissingParameter("p", "string") still compile with the same defaults. new X(...) does not; drop the new.

• Replace .copy with a setter:

  Type	Setters
  ToolCallError.UnknownFunction	withToolName, withAvailableTools
  ToolParameterError.MissingParameter	withParameterName, withExpectedType, withAvailableParameters

  val hinted = err.withAvailableTools(registry.tools.map(_.name)) // was err.copy(availableTools = ...)
  

Pattern matching (case UnknownFunction(name, available)) and the ToolCallErrorJson payloads are unchanged. The other cases (ExecutionError, HandlerError, InvalidArguments, Timeout, NullArguments, Cancelled, NullParameter, TypeMismatch, InvalidNesting, MultipleErrors) are closed records whose fields name exactly what went wrong; they are unchanged.

Context types are growth-safe (#2380)

CompressionRule, ContextStep, PipelineStep, ManagedConversation, ConversationWindow, TokenUsageInfo, StructuredInfo, HistoryDigest, TokenBreakdown, MessageTokenInfo and SemanticBlock (org.llm4s.context) follow the growth-prone pattern (#2380). Each is a final case class whose constructor and copy are private:

• Construction is unchanged: X(...) with the same arguments, positional or named, still compiles; SemanticBlock(...) keeps expectingAssistantResponse = false as its default. new X(...) does not; drop the new.

• Replace .copy with a setter, one per field, named with<Field>:

  Type	Setters
  CompressionRule	withName, withApply
  ContextStep	withName, withConversation, withTokensBefore, withTokensAfter, withApplied
  PipelineStep	withName, withMessages, withTokensBefore, withTokensAfter, withApplied
  ManagedConversation	withConversation, withOriginalTokens, withFinalTokens, withSteps
  ConversationWindow	withConversation, withUsage, withWasTrimmed, withRemovedMessageCount
  TokenUsageInfo	withCurrentTokens, withBudgetLimit, withWithinBudget, withUtilizationPercentage
  StructuredInfo	withIdentifiers, withUrls, withConstraints, withStatusCodes, withErrors, withDecisions, withToolUsage, withOutcomes
  HistoryDigest	withBlockId, withBlockType, withContent, withOriginalTokens
  TokenBreakdown	withTotalTokens, withMessages, withOverhead
  MessageTokenInfo	withRole, withTokens, withPreview
  SemanticBlock	withId, withMessages, withBlockType, withExpectingAssistantResponse

  val over = usage.withWithinBudget(false) // was usage.copy(withinBudget = false)
  

• Java and Kotlin: call the companion’s apply (TokenUsageInfo.apply(850, 1000, true, 85)) instead of the constructor, and chain the setters.

Pattern matching (case ContextStep(name, conversation, before, after, applied)) and the fields are unchanged.

GeminiConfig, VertexAIConfig and AzureConfig are growth-safe (#2263)

GeminiConfig, VertexAIConfig (llm4s-gemini) and AzureConfig (llm4s-openai) follow the growth-prone pattern, as OpenAIConfig, AnthropicConfig and OllamaConfig already did (#2263), so a new field - as googleSearch was - no longer breaks Java and Kotlin callers or compiled code. The constructor and copy are private:

• Scala: GeminiConfig(...), VertexAIConfig(...) and AzureConfig(...) with the same arguments, positional or named (timeouts, googleSearch and wellKnownCredentialPath keep their defaults), still compile. Replace config.copy(contextWindow = n) with config.withContextWindow(n); each field has a setter - withApiKey, withBaseUrl, withReserveCompletion and withGoogleSearch on Gemini; withProjectId, withLocation, withCredentialFilePath and withWellKnownCredentialPath (the value or an Option) on Vertex AI; withEndpoint, withApiKey and withApiVersion on Azure - besides withModel and withTimeouts.
• Java and Kotlin: call the companion’s short apply and chain setters instead of the constructor - GeminiConfig.apply(apiKey, "gemini-2.0-flash").withGoogleSearch(true), VertexAIConfig.apply(projectId, model), AzureConfig.apply(endpoint, apiKey, deployment).withApiVersion("2024-10-21"). They take the default base URL, location or API version and a context window from the model name; fromValues still consults the bundled model catalogue.

Pattern matching (case GeminiConfig(...)) is unchanged.

ToolRetryPolicy is growth-safe (#1708)

ToolRetryPolicy follows the growth-prone pattern, so that it could gain jitter (#1708): build it with ToolRetryPolicy(...), whose arguments are unchanged apart from the new jitter (default 0.0, no jitter), and change it with withMaxAttempts, withBaseDelay, withBackoffFactor and withJitter. new and .copy are private:

import org.llm4s.toolapi.ToolRetryPolicy
import scala.concurrent.duration._

val policy = ToolRetryPolicy(maxAttempts = 3, baseDelay = 200.millis).withMaxAttempts(5) // was .copy(maxAttempts = 5)

AssistantMessage and Completion.thinking (#1381)

AssistantMessage is a growth-prone type (#1381): build it with AssistantMessage(...) and change it with withContent, withToolCalls and withThinking; .copy is private. A positional pattern takes four fields: case AssistantMessage(content, toolCalls, thinking, thinkingBinding). Completion loses its thinking parameter and withThinking: Completion.thinking is message.thinkingText, so set it with completion.withMessage(completion.message.withThinking(...)).

JSONValidator checks property types (#922)

A schema’s properties.<field>.type is now enforced on the top-level fields the output contains (#922). Output that passed because the type was ignored may now be rejected; correct the schema or the output. Nested properties, items and enum are still not checked.

Result.traverse and recoverWithBackoff (#960)

Result.traverse stops calling its function at the first Left, so side effects no longer run for the elements after a failure (#960). ErrorRecovery.recoverWithBackoff returns an error it does not retry unchanged instead of wrapping it in an ExecutionError, and retries a ServiceError only for a 5xx, 429 or 408 status.

Vertex AI Application Default Credentials (#1939)

VertexAIAuthProvider’s wellKnownCredentialPath is an Option[String], and None (the default) looks up no gcloud credentials file (#1939). A client built from a named section finds the file as Google’s libraries do. A hand-built VertexAIClient keeps the fallback by setting the path:

VertexAIAdcConfigLoader.load().map(config.withWellKnownCredentialPath)

GOOGLE_APPLICATION_CREDENTIALS still wins.

AudioFormat.Mp3 (#1307)

AudioFormat has a new case, Mp3 (#1307), so an exhaustive match needs one more branch. PCM stays the default; the WAV writers and AudioPreprocessing.standardizeForSTT refuse MP3 with a ValidationError.

RegexValidator’s constructor

The primary constructor of RegexValidator takes a compiled Pattern and a description. The Regex-based constructors remain, so source compiles unchanged, but code compiled against 0.4.1 must be recompiled.

Stage 3 migration: the Deep Agents harness

In 0.5.0, except the slices for #1904, #1905, #1908, #1909, #1910, #1913 and #1915, which are in 0.5.1. Stage 3 (#1901) builds the harness as middleware and tools on AgentBuilder. Each slice adds its part here as it lands:

• #1902, kernel hooks for the harness - state-writing hooks, prompt sections, accounted model calls, and a Block that restores the whole turn: below.
• #1934, restoring a checkpoint after the graph gains a key - which shape changes keep a thread restorable: below.
• #1903, tool permissions, timeouts and batch locks: below.
• #1907, todos: below.
• #1908, context management - summarisation and large tool-output offload: below.
• #1911, semantic memory - the new llm4s-agent-memory module and TransactionalMemoryStore: below.
• #1914, typed custom streams: below.
• #1915, the harness profile and middleware placement checks: below.
• #1921, replay of external effects - ReplayGuard, Checkpointer.effect and Commit.effects: below.
• #1926, the tenant on a run claim - ClaimRequest and RunClaim carry it: below.

Design: docs/design/typed-agent-runtime/4.22-stage-3-slice-c1-kernel-hooks.md (§4.22) onwards.

Kernel hooks for the harness (#1902)

Code that only builds agents with the built-in middleware needs no change. A middleware of your own that overrides beforeAgent, afterAgent or wrapModelCall does. Source breaks, with no shims:

• The boundary hooks take a ModelContext and return a BoundaryResult. context.run is the RunContext they took before; context.state the committed state the task read. Wrap the text, and add an update to keys the middleware declares in writes if it has one:

  // before
  override def beforeAgent(input: String, context: RunContext): Result[String] = Right(input.trim)
  // after
  override def beforeAgent(input: String, context: ModelContext): Result[BoundaryResult] =
    Right(BoundaryResult(input.trim))
  

  AgentMiddleware.Asking.answered takes the ModelContext too.

• wrapModelCall takes a ModelContext and its next returns a ModelResponse. A wrapper that passed next through needs only the new signature. One that read the completion reads response.completion; one that changed it uses response.withCompletion(c), which keeps the inner wrappers’ updates; one that answered without calling next returns ModelResponse(completion). A wrapper adds writes with response.addUpdate(update).
• writes covers every hook. An update from any hook may touch only that middleware’s own writes; any other key fails the run with a ValidationError.
• An output Block restores every key the turn wrote. Before, only the turn’s messages, the active agent and the transfer were rolled back, and a key a tool or middleware wrote during a blocked turn kept its new value. Now every such key returns to its value before the turn; usage is kept. A file, sandbox or memory store outside the thread state is not restored.
• Usage counts provider calls. A retrying wrapper’s discarded calls are now in AgentResult.usage, and a completion a wrapper returned without calling the model (a cache, a reviewed answer) no longer counts as a request. ModelCallCompleted is one event per provider call, attempts that call’s attempt number (0 for a hook’s own call through ModelContext.accounted). A hook that calls a model itself - or hands a client to a library that does - wraps the client with context.accounted(client), so its calls are counted.
• RunEvent.RunFailed gains usage; RunCancelled and RunTimedOut become case classes with usage: the calls of the attempts that committed nothing - a task that failed, or one the cancel or deadline cut short. Match them as RunFailed(message, _), RunCancelled(_) and RunTimedOut(_); build them as RunCancelled(). Events stored before decode with None.
• Guardrails take the hook’s context. Guardrail gains validate(value, context: ModelContext), defaulting to validate(value); GuardrailMiddleware calls it. A guardrail of your own that calls a model overrides it to call through context.accounted(client); one that combines guardrails passes context on. A subclass of LLMGuardrail that called evaluateWithLLM(content) passes the client too: evaluateWithLLM(content, llmClient).
• An output Block keeps edits made by others. A key updateState changed while the turn waited for review keeps the edit; only keys still holding the turn’s writes are restored.
• New hook promptSections adds system-prompt content per request, never stored; together the sections are capped by AgentBuilder.withMaxPromptSectionTokens (default 8192, four characters a token, measured on the rendered prompt, separators included) and to 64 per request, over which the run fails. LoopAgent gains maxPromptSectionTokens and TurnState gains priorValues; a stored TurnState without it decodes.

Restoring after the graph changes (#1934)

Each checkpoint now records its graph’s shape (GraphSnapshot.shape, CompiledGraph.shape), and a restore accepts the changes the restored run cannot observe (design §4.29). No code change is needed; what is refused changes:

• Now restored: a thread at rest (Completed, or Failed after a Block) into a graph of another version, with other edges or joins, or more nodes or keys - for an agent, a family with another prompt, or with a tool, middleware or handoff added. Any thread into a graph that only gained keys (each reads its initial value) or gained nodes no stored node has a static edge or join to.
• Still refused: a node removed; a key removed while the thread holds a value for it (finish with the old graph, Remove the entry, or forget the thread); and, for a thread in flight (suspended, or Running after a crash), another version, a stored node’s changed out-edges, or a changed join over stored nodes. Resume or recover such a thread with the graph it was written by before changing it.
• Snapshots written before this change carry no shape. They restore into their own version when the graph only gained keys, and are refused for anything else until they run once more.
• RestoreRejected problems name the change; “snapshot’s graph structure differs from this graph’s” is gone.
• Effect records (#1921) stay valid. A thread restored in flight keeps its task ids, so a task recover runs again replays its recorded effects; a refused restore commits nothing and leaves them in place.
• A harness key that should stay removable Removes its entry when it is empty again: removal is judged by the stored entry, which cannot be decoded once its key is gone.

  Tool permissions, timeouts and batch locks (#1903)

Design §4.23. Additive, bar one reserved key: an agent without PermissionMiddleware, withTimeout or withLocks behaves as before.

• tool-queue is a key the loop owns. A tool or middleware that declares a StateKey with id tool-queue is refused at build, as tool-results is. Rename it.
• ToolCallRequest carries resources. ToolCallRequest(spec, call) still compiles and gives the tool’s defaultResource, (Invoke, <name>); the loop builds every request with the call’s own resources. A wrapper that builds one for a tool with withResources passes them: ToolCallRequest(spec, call, resources).
• PermissionMiddleware denies what no rule allows, a tool without withResources included: add PermissionRule.allow(Operation.Invoke, "<name>") (or "*") for each tool the agent may call. See Tool Permissions, Timeouts and Locks.
• Adopting locks adds the tool-queue key. The loop registers tool-queue only for an agent with a tool that declares withLocks. Since #1934 an added key keeps old checkpoints restorable, so threads checkpointed before the first tool adopted locks continue, at rest or suspended. Dropping every lock while a suspended thread still has calls queued in tool-queue is refused with RestoreRejected rather than stranding them: resume such a thread with its locks first. Once an agent has the key, changing lock functions is safe for threads in flight: a batch keeps the lock schedule it was made with. Agents without locks, timeouts and resources included, restore their existing threads unchanged.
• Locked calls cost supersteps. Each further round of a batch whose calls hold conflicting locks costs two supersteps; raise RunBudgets.maxSupersteps if a long batch of conflicting calls reaches it.

Todos (#1907)

Design §4.30. Additive: TodoMiddleware (org.llm4s.agent.harness) gives an agent a write_todos tool and a plan prompt section.

• Adding TodoMiddleware to an existing agent registers the todos key, and its tool’s lock registers tool-queue. Since #1934 a thread at rest continues, its plan starting empty. A thread in flight (suspended, or Running after a crash) is refused with RestoreRejected: resume or recover it with the old agent first. So is a thread last checkpointed before #1934, until it has run once under the agent it was written by. Removing TodoMiddleware refuses a thread whose plan holds items. Agents without it are unchanged.
• todos is a key id the middleware owns. A StateKey of your own with id todos in a family that also uses TodoMiddleware is refused at build, as two keys with one id are. Rename it.
• With PermissionMiddleware, allow the tool: PermissionRule.allow(Operation.Invoke, "write_todos").

Context management (#1908)

Design §4.34. Additive: SummarizationMiddleware and ToolOutputOffload (org.llm4s.agent.harness).

• Adding either to an existing agent registers its key (summary, offloads), which changes the agent’s version as any middleware key does (§4.29): a thread at rest continues, starting with no summary or offloads; a thread in flight is refused until it is resumed or recovered with the old agent. Agents without them are unchanged.
• summary and offloads are key ids the middleware own. A StateKey of your own with either id in a family that uses the middleware is refused at build. Rename it.
• Use SummarizationMiddleware or ContextWindowMiddleware, not both. ContextWindowMiddleware is unchanged, the pruning-only option.

Semantic memory (#1911)

Design §4.31. SemanticMemoryMiddleware is in a new module, llm4s-agent-memory (package org.llm4s.agent.harness.memory); add the dependency to use it. In llm4s-memory:

• InMemoryStore, SQLiteMemoryStore and PostgresMemoryStore are TransactionalMemoryStores. Nothing changes for code that only stores and recalls. InMemoryStore carries its applied writes and fencing tokens in its value, and clear() keeps them.
• MemoryStore.replace(add, remove) is new, with a default that stores, then deletes; the SQL stores override it as one transaction. A store of your own that writes in place should override it too.
• LLMMemoryManager.consolidateMemories replaces each group with one store.replace(Seq(summary), groupIds) instead of a store followed by one delete per memory. A store of your own that overrides store and delete but not replace gets the same writes as before, in the same order.
• PostgresMemoryStore.Config.tableName is at most 55 characters (was 63), so the names derived from it (<table>_applied, <table>_fences, the idx_<table>_... indexes) stay distinct within PostgreSQL’s 63-byte identifier limit. A longer name is refused when the config is built: give the table a shorter name.
• SQLite memory files move to schema version 2. Opening a file creates its two new tables, memory_applied_writes and memory_fences, and records version 2 beside version 1 in schema_version. A Postgres store creates <table>_applied and <table>_fences beside its table.
• Adding SemanticMemoryMiddleware to an existing agent registers the semantic-memory-recalled key and changes the agent’s version (§4.29). A thread at rest restores, with nothing recalled until its next turn, and restores again if the middleware is removed (the key is cleared at the end of each turn). A thread in flight - suspended, or Running after a crash - is refused with RestoreRejected: resume or recover it with the old agent first. Agents without it are unchanged.

Harness profile and middleware placement (#1915)

Design §4.37. HarnessProfile (org.llm4s.agent.harness), AgentBuilder.withHarness and org.llm4s.config.HarnessConfigLoader are additions; llm4s-agent now ships a reference.conf with the llm4s.agent.harness block. One behaviour changes:

• build() refuses a middleware order that a middleware declares wrong (AgentMiddleware.placement), with a ValidationError naming both middleware and the fix. A SemanticMemoryMiddleware registered after (inside) a GuardrailMiddleware or GuardrailReviewMiddleware - which recorded answers the guardrail then blocked - is refused: register it first, or name the guardrail’s id in its runsBefore. So is one on a handoff target when the root agent has guardrail middleware, since the root’s afterAgent runs after every agent’s own: put both on the same agent. A SummarizationMiddleware registered after a ToolOutputOffload is refused too: register it first.

Typed custom streams (#1914)

EventHandlers and the effect modules’ events(run, ...) are additions. One behaviour changes:

• An fs2, ZIO, Pekko or Java agent stream whose consumer stops draining now fails once more than Agent.StreamCapacity (1024) durable events wait for it, with an error naming the last seq it took, as the runtime already disconnects a listener that far behind (Disconnected(lastSeq, Lagging)). Before, the stream’s buffer kept every durable event without limit. A stream* that fails this way cancels its turn, as any failure of those streams does; resume from the thread’s log with GraphRuntime.subscribe(threadId, afterSeq = lastSeq). A consumer that keeps up, even slowly, is unaffected: a run’s own events stay far below the bound.

Replay of external effects (#1921)

Effects outside thread state replay through ReplayGuard: an intent and an outcome per EffectKey, kept in the thread’s own store. Code that only runs agents or graphs compiles and works unchanged. Code that implements Checkpointer, or wraps one, changes:

• Checkpointer gains effect(threadId, key): Result[Option[EffectRecord]]. A wrapper forwards it:

  def effect(threadId: ThreadId, key: EffectKey) = underlying.effect(threadId, key)
  

• Commit gains effects: Vector[EffectRecord], defaulted in apply, so existing calls compile. A store must do three things with it:
  • Upsert a commit’s records by key.
  • On every commit that carries a checkpoint, first remove each record for which commit.releasesEffect(record.taskId, record.lineage) holds.
  • Remove a thread’s records in deleteThread. Never remove them in prune.

  Persist records with EffectRecord.toJson and read them with EffectRecord.fromJson. CheckpointerContract gains the cases that check all of this.

• GraphError gains EffectReplayRefused(operation, reason). A match on GraphError that must be exhaustive gains a case.
• The SQLite schema moves to version 3 (llm4s_checkpoint_effects). A version-2 file is migrated when it is opened. A build from before this change refuses a version-3 file, as it refuses any newer schema.
• A Transactional target fences itself by the claim token. AppliedKeys.forget(key, token) and the effect given to ReplayGuard.transactional (token => ...) refuse a token older than the newest seen for key.scope. A Verifiable target’s effect is a compare-and-set against the plan’s before.
• A store keys release by the record’s task and lineage. Commit.releasesEffect(taskId, lineage) replaces releasesEffect(taskId, writer); persist EffectRecord.lineage with the record (SQLite: schema 4, task_lineage), and PendingWrite.resultDigest with the write. A store that persists the write’s and the snapshot’s JSON whole needs no change; the snapshot gains PendingTask.resumeDigest and ParkedContinuation.questionDigest / resumeDigest. GraphSnapshot.liveTasks is a set of (id, lineage) pairs. The checkpoint format moves to 7 (PendingTask.lineage, originLineage); format 6 never shipped (v0.4.1 has no graph checkpoints), so a format-6 checkpoint restores with no legacy keys: each task gets a fresh root lineage at admission, and any effect record it wrote is ignored.
• A tool call’s idempotency key now names the issuing model task and the call itself (IdempotencyKey.ofCall), not the model call’s checkpoint: a request asked again that answers with the same call gets the same key. IdempotencyKey.derive is removed: the loop no longer uses it, and a test builds a key with IdempotencyKey("...").
• ToolQuestionRequest gains phase: Option[IdempotencyKey] (defaulted; a stored request without it decodes as None). An asking tool’s resume gets effect keys of its own, distinct from execute’s.
• New, opt-in API: RunContext.taskKey, RunContext.effectKey and ToolContext.effectKey; ReplayGuard and its Plan; ReplaySupport, StateCheck, AppliedKeys and EffectRecord; and, in llm4s-agent-testkit, ReplayContract, which every external target of the harness passes. See Tool side effects.

Design: docs/design/typed-agent-runtime/4.24-stage-3-slice-c4-replay-of-external-effects.md (§4.24).

Harness backend SPIs (#1904)

Design §4.25. New, opt-in API in org.llm4s.agent.harness; nothing existing changes, except:

• llm4s-agent-testkit depends on llm4s-testkit, whose ScriptedLLMClient drives FileBackendContract’s agent runs. A build that excludes llm4s-testkit from its test classpath adds it back.
• File tools over a StateFileBackend add the files key. Every mutation holds the lock files/state/<keyId> (needed: it keeps two edits of one file in a batch from losing one, and the quota exact), so adding file tools over a StateFileBackend adds its files key and C2’s tool-queue, and changes the agent’s version. Under §4.29 (C5) a thread at rest continues, its files starting empty; a thread in flight (suspended, or Running after a crash) is refused with RestoreRejected until it is finished or forgotten. Taking the tools out again refuses only a thread whose files hold content: the last file’s delete removes the key’s entry, so a thread with no files restores.

Event records carry their claim’s token (#1918)

A run is told from an earlier run with the same RunId by its claim’s fencing token. Code that only runs agents or graphs is unaffected.

• EventRecord gains token: FencingToken, after runId. A Checkpointer builds each record with EventRecord.committed(threadId, seq, commit.token, draft) (the token is a new parameter) and returns and replays it as stored. CheckpointerContract checks that every record carries the token of the commit that stored it.
• Persisted logs stay readable. A store that keeps the record’s JSON, as both shipped stores do, needs no schema change: a record written before this change has no token key and reads as FencingToken(0), which no store issues. An older build reading a newer record ignores the key.
• Positional constructions and patterns gain a field. EventRecord(threadId, seq, runId, checkpointId, ...) becomes EventRecord(threadId, seq, runId, token, checkpointId, ...), and case EventRecord(_, _, _, _, _, _, _, event) gains a _.
• StreamEvent.Live gains token: FencingToken, after runId: the claim token of the run that sent the progress. StreamEvent.Live(threadId, runId, taskId, nodeId, name, version, payload) becomes StreamEvent.Live(threadId, runId, token, taskId, nodeId, name, version, payload), and a positional pattern gains a _. Matching with EventType or EventHandlers, as AgentEvents.TextDelta(d) does, needs no change. A run’s observer and its run-scoped streams no longer deliver progress of another claim under their RunId, and a run that has lost its claim sends none.

The tenant on a run claim (#1926)

A claim records the tenant of the run that holds it, so that neither ThreadBusy nor RunRecovered.takenOverFrom ever names another tenant’s run. Code that only runs agents or graphs needs no change. Code that calls Checkpointer.claim itself, builds a RunClaim, or implements Checkpointer changes:

• ClaimRequest(holder, ttl) is ClaimRequest(holder, tenantId, ttl), tenantId: Option[TenantId], also in ClaimRequest.of; withTenantId sets it. Pass the run’s RunConfig.tenantId, or None for a run of no tenant. claim’s signature is unchanged, so a wrapper that forwards it needs no change.
• RunClaim(threadId, holder, token, expiresAt, takenOverFrom) is RunClaim(threadId, holder, tenantId, token, expiresAt, takenOverFrom): the claim’s own tenant, with withTenantId.
• A store records the tenant with each claim, and names a claim’s holder only to a claim of the same tenant: a refusal is ThreadBusy(..., BusyHolder.Run(holder)) only then, and BusyHolder.Unnamed otherwise, and a takeover’s takenOverFrom is None when the replaced claim was another tenant’s. A claim whose tenant the store cannot tell - one stored before it recorded tenants - is of no tenant. CheckpointerContract checks all of this.
• ThreadBusy from a runtime now names a holder of the caller’s own tenant on a thread with no checkpoint yet: before, a holder in another runtime was Unnamed until the thread’s first checkpoint. A holder of another tenant is Unnamed, also when the thread’s latest checkpoint is the caller’s.
• RunEvent.RunRecovered.takenOverFrom is None for a takeover of another tenant’s claim.
• The SQLite schema moves to version 5 (claim_tenant, claim_tenant_token on llm4s_checkpoint_threads). A version-4 file is migrated when it is opened; a claim it holds then is of no tenant, so its holder is named to nobody until it is released or lapses, at most ttl later. A build from before this change refuses a version-5 file.

Design: docs/design/typed-agent-runtime/4.16-stage-2-slice-1-the-durable-checkpointer-contract.md (§4.16).

Bounded store calls (#1919)

Code that only runs agents needs no change. A graph run may now end with one more error, and a store has one more duty.

• New GraphError.CommitTimedOut(threadId, after), from RunHandle.await() of a graph run with Durability.Async or OnExit whose closing commit stayed in the store for twice ClaimPolicy.ttl. An Agent runs Sync, so never sees it. An exhaustive match on GraphError gains a case. After it, read the thread before recover: the commit may or may not have landed.
• A Checkpointer must return from every call within a bound it documents, failing a call that reaches it with nothing changed. A store whose calls could wait on a lock, another connection or a network with no limit must add a timeout. To have CheckpointerContract check it, override boundedStore with CheckpointerContract.BoundedStore(store, bound, busy).
• SqliteCheckpointer calls fail sooner under contention inside one store: a call waits for the store’s own earlier calls at most busyTimeout (with its turn at the file), then fails with a ProcessingError, where it used to wait with no limit. Raise SqliteCheckpointerConfig.busyTimeout if many threads share one store and its calls are slow.

Design: docs/design/typed-agent-runtime/4.16-stage-2-slice-1-the-durable-checkpointer-contract.md (§4.16).

Stage 2 migration: durable execution

In 0.5.0. Stage 2 (#1699) makes agent and graph threads durable and adds human review. One note covers the whole stage; each slice adds its part here as it lands:

• #1700, the durable Checkpointer contract - run-claim leases, fencing tokens and a store contract suite: below.
• #1703, tool side-effect safety - idempotency keys and one result per tool call: below.
• #1704, human-review interrupts - static breakpoints and typed middleware questions: below.
• #1702, checkpoint history, fork, updateState and retention: below.
• #1701, the SQLite checkpointer - a new module, nothing to change: below.
• #1705, live subscriptions across runtimes - Checkpointer.awaitEventsAfter and WatchPolicy: below.

Design: docs/design/typed-agent-runtime/4.16-stage-2-slice-1-the-durable-checkpointer-contract.md (§4.16) onwards. Guide: Durable Checkpointers.

Durable Checkpointer contract (#1700)

A run now holds a lease on its thread - a RunClaim with an expiry and a FencingToken - which it renews while it runs and releases when it ends, and every commit carries the token. Code that only runs agents or graphs on GraphRuntime or Agent needs no change. Code that implements Checkpointer, builds a Commit, Checkpoint or RunPosition, or matches ThreadBusy does. Source breaks, with no shims:

• Checkpointer gains claim, renew and release. A store grants at most one live claim per thread, judged by its own clock, with a token greater than every token it issued for that thread before, and refuses every commit whose token is not the current claim’s with GraphError.StaleClaim. A wrapper that delegates to another store forwards the three methods:

  def claim(threadId: ThreadId, request: ClaimRequest)                      = underlying.claim(threadId, request)
  def renew(threadId: ThreadId, token: FencingToken, ttl: FiniteDuration)   = underlying.renew(threadId, token, ttl)
  def release(threadId: ThreadId, token: FencingToken)                      = underlying.release(threadId, token)
  

  Test a store with CheckpointerContract from the new llm4s-agent-testkit (see the guide).

• Commit(checkpoint, pendingWrites, events) is Commit(token, checkpoint, pendingWrites, events), with a private constructor: build it with Commit(token, ...) (all but the token default to empty) and change it with withCheckpoint, withPendingWrites, withEvents and withToken. A commit to InMemoryCheckpointer - or any store - now needs a claim first: store.claim(threadId, ClaimRequest(runId, tenantId, ttl)) (the tenant since #1926) returns the RunClaim whose token it carries.
• Checkpoint and RunPosition have private constructors and gain fencingToken: Option[FencingToken]. Build them with Checkpoint(...) and RunPosition(...) as before (the new field defaults to None) and change them with with* setters; checkpoint.copy(parent = p) is checkpoint.withParent(p). A run’s position carries its claim’s token; None outside a runtime.
• Checkpoint.CurrentFormat is 5. Format 4 and earlier checkpoints still read (as written by no claim, fencingToken = None); a build before this change refuses format 5.
• GraphError.ThreadBusy(threadId, latestCheckpoint) gains holder: GraphError.BusyHolder (default BusyHolder.Unnamed): BusyHolder.Run(runId) names the holding run when the caller may see it; Unnamed is another run that cannot be named; Lapsed is a fork, updateState or deleteThread whose own claim lapsed before its write was known to land, with no other run known to hold the thread - a write it had already sent may still land, so read the thread before retrying. A pattern case GraphError.ThreadBusy(t, c) becomes case GraphError.ThreadBusy(t, c, _), and one on a named holder case GraphError.ThreadBusy(_, _, GraphError.BusyHolder.Run(runId)). ThreadBusy is now also returned when another runtime or process holds a live claim on the thread, and by updateState and a fork onto a held target.
• GraphError.ThreadBusy is a RecoverableError (it was a NonRecoverableError): a retry may get past it once the thread is free, so generic retry code - LLMError.isRecoverable, RetryPolicy, ErrorRecovery, and a node given an explicit RetryPolicy (the default node policy retries nothing) - now retries it. Code that treated every GraphError but DeadlineExceeded as final should let ThreadBusy through to its retry. A call wrapped in ErrorRecovery.recoverWithBackoff that is still ThreadBusy after its last attempt now ends with an ExecutionError naming the attempts and the last error, not with the ThreadBusy itself; match on that, and beware that ExecutionError is itself recoverable, so nested retry layers multiply attempts.
• GraphError.StaleClaim(threadId, token: FencingToken, current: Option[FencingToken]) is new: a commit, renewal or release made with a token that is not the thread’s current claim. Build it, or compare with it, with FencingToken(n), not a bare Long. It is the store’s refusal. No runtime call returns it, wrapped or not: a run that loses its claim, or a start, recover or resume whose claim’s commit is refused as stale, ends with ClaimLost (below), and a fork, updateState or deleteThread whose claim lapsed or was taken over mid-operation returns ThreadBusy - nothing was applied, so retry.
• A run that lost its claim ends with GraphError.ClaimLost(threadId, token, current), not CheckpointWriteFailed(StaleClaim). It is new: the run’s lease passed with no renewal landing (the run judges it by its own monotonic clock), or a renewal or commit was refused as stale - the claim’s own commit at admission included, so start, recover and resume return it too. The run stops at once, its tasks interrupted, and commits nothing more. A pattern case GraphError.CheckpointWriteFailed(_, _: GraphError.StaleClaim, _) becomes case _: GraphError.ClaimLost. Unlike CheckpointWriteFailed, it is not a store outage. recover may be attempted once the thread is free: it is refused with ThreadBusy while a successor holds it, a successor that finished leaves the thread’s state showing it, and a lease that lapsed with no successor leaves the run for recover to continue.
• RunEvent.RunRecovered gains takenOverFrom: Option[String], the run id whose expired claim the recovery took over, and RunClaim gains takenOverFrom: Option[RunId], which a store sets when a claim replaces an expired, unreleased one. Both default to None, so a log written before still reads; a pattern case RunEvent.RunRecovered(c, t, p) becomes case RunEvent.RunRecovered(c, t, p, _). A store reports the replaced holder from claim (CheckpointerContract checks it).
• InMemoryCheckpointer takes an optional Clock (default the system clock), by which its claims expire, and GraphRuntime an optional ClaimPolicy (ttl 30 seconds, renewEvery 10 seconds). Java and Kotlin keep new InMemoryCheckpointer(), new GraphRuntime(store) and new GraphRuntime(store, clock). GraphRuntime.inMemory(clock, claims) gives its store the same clock - before, the store used the system clock whatever clock was - so a test that moves clock now moves claim expiry and prune’s ages too.
• GraphRuntime.deleteThread claims the thread before deleting it, so it is refused with ThreadBusy while a run holds the thread in any runtime, not only in its own.

• Checkpointer.deleteThread(threadId) is deleteThread(threadId, token: FencingToken), fenced like a commit: a store deletes only while token is the thread’s current claim, checked atomically with the delete, and otherwise returns GraphError.StaleClaim and deletes nothing. A thread with no claim - an unknown one included - is therefore refused too; claim it first (GraphRuntime.deleteThread does). A wrapper that delegates to another store forwards the token:

  def deleteThread(threadId: ThreadId, token: FencingToken) = underlying.deleteThread(threadId, token)
  

  A store implementation checks the token as its commit does, in the same lock or transaction as the delete. GraphRuntime.deleteThread passes its own claim’s token, and reports a delete whose claim another runtime took over as ThreadBusy (or TenantMismatch, for another tenant’s run), with nothing deleted.

• A new thread under a deleted thread’s id numbers its events on past the deleted thread’s, not from 1 again: one past the deleted thread’s next number, which is never used. A store keeps a tombstone of that number (no data) for each deleted thread that numbered an event, refuses a read of the log after one of the deleted thread’s numbers with the new GraphError.ThreadDeleted(threadId, afterSeq, firstSeq) (Checkpointer.deleted(afterSeq, firstSeq) is the rule), and reads from afterSeq = 0 from the current thread’s first event. Code that assumed a thread’s first event is 1 should read from 0.
• DisconnectReason gains ThreadDeleted. A subscription to a deleted thread - in the deleting runtime, or in another runtime sharing the store, at its next read of the store - ends with Disconnected(lastSeq, ThreadDeleted) and never delivers a new thread of the same id as a continuation of the old one; subscribe again from 0 to follow that one. (A subscription in another runtime that has delivered no event yet cannot be told from a new one, and follows the new thread.) A match on DisconnectReason needs the new case. Before, a subscription in another runtime went on delivering the new thread’s events as if they continued the old log, so the old advice to cancel subscriptions before deleting a thread is no longer needed.
• GraphRuntime.subscribe takes the caller’s RunConfig, and refuses another tenant’s thread. subscribe(threadId, afterSeq, capacity)(listener) is subscribe(threadId, afterSeq, capacity, config)(listener), and TracingSubscriber.attach(runtime, threadId, tracing, afterSeq) gains config too; both default to RunConfig(). A thread whose latest checkpoint is another tenant’s - or, with no checkpoint yet, whose run admitting or executing in this runtime is - is refused with GraphError.TenantMismatch before the subscription joins, as history and checkpoint refuse it; a failed read of the thread is returned. Before, any caller could replay and follow any tenant’s thread. Pass the config your runs use: a multi-tenant caller that subscribed with no config is now refused. RunHandle.subscribe and AgentRun.subscribe use their run’s tenant and are unchanged.
• DisconnectReason gains TenantMismatch. A subscription screens each event against its tenant, so that a thread created under its id by another tenant after it subscribed - or deleted and created again by another tenant through another runtime - is never delivered: it ends with Disconnected(lastSeq, TenantMismatch) before that thread’s first event. A match on DisconnectReason needs the new case.
• A ThreadBusy whose re-read fails names nothing. When a claim is refused, or a run’s claim commit conflicts, the runtime re-reads the thread to check its tenant before naming its checkpoint. If that re-read failed, the error named the store’s own, unchecked checkpoint id - another tenant’s, possibly, with that tenant’s run id in it. It is now ThreadBusy(threadId, None, BusyHolder.Unnamed).
• CheckpointerContract no longer requires a store to delete regardless of claims. It checks a delete with the current token, a stale token and another holder’s live token, a runtime delete whose claim was taken over, the numbering of a recreated thread, ThreadDeleted from eventsAfter and awaitEventsAfter, and a subscription in another runtime ending with ThreadDeleted.
• The SQLite schema is version 2 (first_seq on llm4s_checkpoint_threads). Opening a version-1 file upgrades it in one transaction; a build that knows only version 1 refuses a version-2 file. Stop every process on the older build first: one that already has the file open is not refused, and its deletes are unfenced and drop tombstones.
• There is no unfenced deleteThread. To delete a thread outside a runtime, claim it and delete with the claim’s token. A deleted id that numbered events keeps a small tombstone (its next event number) for good.

Tool side-effect safety (#1703)

Design §4.17. Tools see an idempotency key, and the agent holds every model request to one result per tool call.

// before
val context = ToolContext(run, ToolCallId("call-1"), state)
ApprovalRequest(messageId, call, reason, ApprovalSource.Tool)
ToolQuestionRequest(messageId, call, question)

// after
val context = ToolContext(run, ToolCallId("call-1"), IdempotencyKey("key-1"), state)
ApprovalRequest(messageId, call, reason, ApprovalSource.Tool, IdempotencyKey("key-1"))
ToolQuestionRequest(messageId, call, question, IdempotencyKey("key-1"))

• ToolContext takes the call’s key, after toolCallId: ToolContext(run, toolCallId, idempotencyKey, state, approved), with withIdempotencyKey. You build one only to unit-test a tool; any key will do there (IdempotencyKey("test")).
• ToolTask, ApprovalRequest and ToolQuestionRequest gain idempotencyKey - last in ApprovalRequest and ToolTask, before the defaulted approved in ToolQuestionRequest. Code that only reads them (AgentResult’s Suspended, ToolLoop.requests/questions, the Java and Kotlin PendingInterrupt) is unchanged.
• Pass the key to the systems your tools change. The runtime runs a tool at least once: a retrying wrapToolCall, approval, an answered question and recover after a cancellation, Fatal or crash run a call again, with the same context.idempotencyKey. Send it as the external API’s idempotency key (or store it in a unique column), and keep a tool’s first run, before approval or an answer, free of side effects. See Tool Side Effects.
• A model request with a dangling tool call fails instead of being sent. A wrapModelCall middleware that removes a ToolMessage but keeps its call (or the reverse) now fails the model call with a ValidationError naming the call; prune whole turns, as ContextWindowMiddleware does. The check is positional: each call’s result must be in the run of tool messages straight after its assistant message.
• A model message with repeated or blank tool-call ids is refused, before it is stored, as a blank answer is: the run fails with NodeFailed(ValidationError), and recover asks the model again. Before, a repeated id left the thread unable to advance.
• Imported history must pair positionally too. History that Message.validateConversation accepted only because an earlier turn’s result shared a reused id is refused with a ValidationError.
• No synthetic result for a call cut short. Unchanged, and now stated: a cancelled, Fatal or crashed call records no result, and the thread waits for recover (start returns IncompleteRun), which runs only the unfinished calls with their keys.
• Provider authors: a chat provider that takes tools can run ProviderModuleChecks.assertOneToolResultPerCall(format, response, path)(baseUrl => client) from llm4s-provider-testkit; see Writing a provider.

Human-review interrupts (#1704)

A turn can now wait for a person in two new ways: at a static breakpoint, before or after a node runs, and on a typed question from middleware. Both are interrupts like approvals and tool questions, answered through the same resume, any subset at a time. See Human Review.

• AgentStatus.Suspended has four lists. Suspended(approvals, questions) is Suspended(approvals, questions, middlewareQuestions, breakpoints). The new fields default to empty, so constructing one still compiles, but a pattern must name all four: case AgentStatus.Suspended(approvals, _) becomes case AgentStatus.Suspended(approvals, _, _, _), or match the type, case _: AgentStatus.Suspended.
• PendingInterrupt (graph kernel) has breakpoint: Option[BreakpointPhase], None for a task that suspended itself. For a held task resumeNode is the held node and question its input (before) or null (after).
• RunConfig has interruptBefore and interruptAfter (Set[NodeId], empty by default), with withInterruptBefore / withInterruptAfter. Admission refuses a node the graph does not have with a ValidationError, before the thread is read. AgentBuilder.withInterruptBefore(AgentNode*) and withInterruptAfter(AgentNode*) add breakpoints to every run of an agent; they are not part of its version.
• CompiledGraph.resume refuses a breakpoint answer that is not JSON null (Breakpoint.proceed, AgentResult.proceed(id), Java Answer.proceed(id)) with InvalidResume.
• Middleware can ask. AgentMiddleware.Asking[Q, Ans] declares a question; a hook asks with ask(q) (from beforeAgent, afterAgent, wrapModelCall) or askAbout(q) (from wrapToolCall). A middleware that is not Asking and returns ToolOutcome.Ask from wrapToolCall now fails the run with ToolFailed (“asked a question it does not declare”); before, the question was attributed to the tool. A wrapper that retries on Left should pass a Left(MiddlewareAsked) through, as it passes a cancellation.
• ApprovalRequest and ToolQuestionRequest gain answered: Vector[GivenAnswer] (default empty), the middleware answers a call carries through an approval or a tool question. Both now have a private constructor, as the new MiddlewareQuestionRequest, BreakpointRequest, GivenAnswer and GuardrailReview do: apply still builds them and a pattern still matches them, but copy is private - request.copy(approved = true) becomes request.withApproved(true).
• Java and Kotlin: PendingInterrupt.toolName() and argumentsJson() return Optional<String>. They were Strings; a breakpoint on a model call, or a middleware question about an answer, has no tool call. They are present for every APPROVAL and QUESTION, so p.toolName() becomes p.toolName().get() (or orElse) there. InterruptKind gains MIDDLEWARE_QUESTION and BREAKPOINT: a Kotlin when over it that is used as an expression must cover them, and a Java switch expression too. PendingInterrupt gains middleware(), node() and phase() (the Java enum BreakpointPhase); Answer gains proceed(id).
• Checkpoint.CurrentFormat is 6, with an identity migration from 5: a build that predates breakpoints refuses a format-6 checkpoint rather than run a held task’s continuation as an ordinary task.

Checkpoint history, fork, updateState and retention (#1702)

A store now keeps every checkpoint of a thread, and GraphRuntime gains history, checkpoint, fork, updateState and prune. Code that only runs agents or graphs needs no change. Code that implements Checkpointer, matches GraphError or RunEvent exhaustively, or relies on checkpoint ids does. Source breaks, with no shims:

• Checkpointer gains history, checkpoint and prune, and its contract grows: keep every checkpoint a commit supersedes (pending writes still for the latest only), refuse a checkpoint id already in the history with InvalidCommit, and honour Commit.retractTurn by removing every checkpoint after the newest Completed or Failed one before adding the commit’s. CheckpointerContract checks all of it. A delegating store forwards them:

  def history(threadId: ThreadId, before: Option[String], limit: Int) = underlying.history(threadId, before, limit)
  def checkpoint(threadId: ThreadId, checkpointId: String)            = underlying.checkpoint(threadId, checkpointId)
  def prune(threadId: ThreadId, policy: RetentionPolicy)              = underlying.prune(threadId, policy)
  

• Commit gains retractTurn: Boolean, defaulted to false in Commit(...), with withRetractTurn. The runtime sets it on the Failed checkpoint that closes a run a node blocked.
• GraphError gains CheckpointNotFound(threadId, checkpointId) and ThreadExists(threadId); RunEvent gains StateUpdated and ThreadForked. An exhaustive match needs the new cases.
• Checkpoint ids count along the thread. They are still <runId>/<n>, but n is one more than the parent’s, so the first run on a thread writes <runId>/1, <runId>/2, … as before, and a later run continues the count rather than starting again at 1. Code that built an id such as s"$runId/1" for a later run reads it from history (or latest) instead.
• A Block retracts its turn from the history. The thread’s latest checkpoint after a Block is its Failed checkpoint, as before; the earlier checkpoints of the blocked turn are gone from history, checkpoint and fork.

Checkpoint.CurrentFormat stays 6: nothing changes in a checkpoint’s JSON.

SQLite checkpointer (#1701)

Nothing changes for existing code: the durable store is a new module, llm4s-agent-checkpoint-sqlite, and llm4s-agent is unchanged by it. To keep threads across restarts, add the dependency and replace the in-memory store:

// before: threads live as long as the process
val runtime = GraphRuntime.inMemory()

// after: threads live in a file, and `recover` continues a stopped run in the next process
val runtime = SqliteCheckpointer.open(Path.of("runs.db")).map(store => GraphRuntime(store))

SqliteCheckpointer is in org.llm4s.agent.graph.sqlite. It brings the SQLite JDBC driver, at the version llm4s-memory already uses. Close it with close() when the process is done with it. See Durable Checkpointers.

Live subscriptions across runtimes (#1705)

A subscription now delivers the durable events of commits made through any runtime or process sharing its store, not only through its own runtime. Code that runs agents or graphs, or that implements Checkpointer, compiles and works unchanged. What changes:

• Checkpointer gains awaitEventsAfter(threadId, afterSeq, limit, timeout), with a default that polls eventsAfter, so existing stores need no change. A store that can be told of commits may override it to return as soon as one lands; InMemoryCheckpointer does. A wrapper that delegates to another store should forward it, so that the wrapped store’s notification is used:

  override def awaitEventsAfter(threadId: ThreadId, afterSeq: Long, limit: Int, timeout: FiniteDuration) =
    underlying.awaitEventsAfter(threadId, afterSeq, limit, timeout)
  

• GraphRuntime takes an optional WatchPolicy after ClaimPolicy (default: enabled, pollInterval 250 ms). Java and Kotlin get new GraphRuntime(store, clock, claims). GraphRuntime(store, watch = WatchPolicy.disabled) gives the old behaviour, where other runtimes’ commits are seen only by subscribing again.
• A live subscription holds a second virtual thread, llm4s-watch-<threadId>, which polls a store that has no change notification (SQLite) once per pollInterval. cancel() stops it, as before for the dispatcher.
• A subscription can now end with Disconnected(lastSeq, ReplayFailed(error)) after its replay, when a read of the store fails, for instance because the store was closed under it. Before, only a failed replay ended this way. Resubscribe with afterSeq = lastSeq.
• Other runtimes’ durable events arrive; their live events do not. A listener that assumed every durable event came from its own runtime’s runs should check record.runId.
• CheckpointerContract gains change-notification cases and a sibling(store, clock) hook. A durable store’s spec overrides sibling to open another store over the same storage, so that the cases run between connections.

Stage 1 migration: agent runtime

In 0.5.0 (#1328, with #1329’s events and tracing, which restore the agent event stream #1328 removed). Agent now runs on GraphRuntime: the graph is the only agent loop, AgentState and the legacy loop are deleted, and nothing runs the old loop beside the new one. Tools, guardrails, handoffs and context pruning belong to the agent, set when you build it, and a conversation is carried by its ThreadId instead of by a value you pass back in. Design: docs/design/typed-agent-runtime/4.13-stage-1-slice-2-the-agent-loop-on-the-graph-runtime.md (§4.13). This note covers the whole of Stage 1 (#1326), five slices:

• #1327, kernel completion: below.
• #1328, the agent loop on GraphRuntime: this section.
• #1329, events and tracing: the bullets this section marks #1329.
• #1330, orchestration removed: below.
• #1331, cancellation for non-chat clients: below.

// before
val agent = new Agent(client)
val state = agent.run("query", tools, inputGuardrails = in, outputGuardrails = out, maxSteps = Some(10))

// after
val result = for {
  agent  <- Agent.builder("assistant", client)
              .withTools(tools)
              .withMiddleware(new GuardrailMiddleware(in, out))
              .withMaxSteps(10)
              .build()
  result <- agent.run("query")
} yield result

• new Agent(client).run(q, tools, ...) is Agent.builder(id, client)...build() then run(q). tools, systemMessage, completionOptions, maxSteps and handoffs move to withTools (a ToolRegistry, MCPToolRegistry included, or a ToolSet), withSystemPrompt, withCompletionOptions, withMaxSteps and withHandoffs. build() returns Result[Agent] and refuses clashing tool names, an invalid or duplicate handoff id, and maxSteps < 1. run has the overloads run(query, config), run(threadId, query), run(threadId, query, config) and run(threadId, query, config, history); config is a RunConfig (budgets, deadline).
• Per-run guardrails are middleware. inputGuardrails/outputGuardrails arguments become .withMiddleware(new GuardrailMiddleware(input, output)) on the builder. A block is no longer an error: the run returns Right with AgentStatus.Blocked(guardrail, reason) and the thread stays usable (its checkpoint is Failed, which recover has nothing to continue and the next run accepts). An input block stores nothing of that turn’s query (a new thread still keeps its imported history); an output block removes the whole turn - query, tool calls and results, answer, and any handoff made in it - so result.messages is the history from before the turn; usage keeps the turn’s model calls. Another middleware’s beforeAgent/afterAgent Left ends the run the same way but is returned as that Left, and a blank query - given, or produced by a beforeAgent - is a ValidationError with nothing stored. A guardrail that transforms (PIIMasker) now applies its transformation. Guardrails - and any run-boundary middleware (beforeAgent, afterAgent) - on the root agent guard the whole handoff family: they apply to every turn’s query and final answer, whichever agent is active, outside the active agent’s own. Give them to the root only; model and tool wrappers stay per agent.
• continueConversation(state, ...) is continueConversation(result, ...). It reads only result.threadId; run(threadId, query) is the same call by thread. A turn on a Suspended result is refused, not layered on parked work.
• Threads are kept until forgotten. An AgentState was garbage once dropped; a thread lives in the agent’s runtime, and one-shot run threads stay in the runtime until forget. Call agent.forget(threadId) (or GraphRuntime.deleteThread(threadId, config)) for a conversation you will not continue. Checkpointer implementations gain deleteThread(threadId, token) (fenced by the claim’s token since the #1700 follow-up).
• runMultiTurn with contextWindowConfig is runMultiTurn(first, followUps, config) on an agent built with new ContextWindowMiddleware(config). Pruning trims what is sent to the model and never what the thread stores; the current turn (latest user message onward) is never pruned - the strategy, Custom included, runs only on the history before it, with the budget the turn leaves - the request always starts with a user message (so it may exceed the budget), and the system prompt is outside the budget. runMultiTurn stops at the first status that is not Completed.

• AgentState fields move to AgentResult.

  AgentState	Now
  conversation	result.messages (the thread’s full history, never a system prompt)
  status	result.status (below)
  logs	removed; use withTracing and the graph.* events
  usageSummary	result.usage (accumulated over the thread)
  tools, systemMessage, completionOptions, availableHandoffs, initialQuery	the builder; activeAgent is on the result

• AgentStatus cases are replaced. Complete is Completed(answer); Failed(error) is Left(GraphError...) (provider errors as GraphError.NodeFailed(cause); the thread stays recoverable with recover); InProgress and WaitingForTools are not exposed; HandoffRequested is gone, because a handoff runs inside the same run. New: Blocked(guardrail, reason), StepLimitReached and Suspended(approvals, questions), continued with resume(threadId, answers) and result.approve/reject/edit/reply.
• AgentContext is removed. tracing becomes Agent.builder(...).withTracing(tracing), which emits the runtime’s graph.* custom events for each run; debug and traceLogPath are gone. TraceEvent.AgentStateUpdated is no longer emitted by the agent; #1329 then replaced it with TraceEvent.AgentRunEnded.
• Handoff(agent) is Handoff(id, builder), and transferSystemMessage is removed. The target is an AgentBuilder, not a built agent, and its id must equal the handoff id: Handoff.to("physics", physicsBuilder, "reason"). A target that must hand back to an agent already in the graph is named by id, Handoff.toId("triage", "reason"), so cycles compile. Each agent sends its own system prompt, which is never stored. A self-handoff is refused at build, and a handoff mixed with other tool calls in one message is refused at run time with an error result for every call. preserveContext = false (a parameter of Handoff.to, Handoff.of and Handoff.toId) sends the target the last user message before the transfer, then the transfer onwards.
• runStep, initializeSafe, runWithStrategy and ToolExecutionStrategy (on the agent) are removed. agent.start(threadId, query) returns an AgentRun (threadId, runId, status, await(), cancel()) at once; run is start then await; startRecover and startResume are the same for recover and resume. The parallel tool calls of one message are bounded by RunBudgets.maxConcurrency. ToolExecutionStrategy stays in core as a ToolRegistry feature; only the agent’s use of it is gone. Step-by-step observation is Agent.stream or AgentRun.subscribe (#1329, below).

• runWithEvents, continueConversationWithEvents, runCollectingEvents, AgentEvent and AgentStreamingExecutor are removed (#1329 replaces them with Agent.stream, below). TraceEvent.AgentStateUpdated is replaced by TraceEvent.AgentRunEnded, and AgentState#toTraceEvent is gone.

  Before	After
  agent.runWithEvents(query)(onEvent)	agent.stream(threadId, query)(listener).flatMap(_.await())
  runCollectingEvents	collect in the listener, or replay with GraphRuntime.subscribe(threadId, afterSeq = 0)
  AgentEvent.TextDelta(delta)	AgentEvents.TextDelta(d) (d.text, d.attempt) with withStreaming()
  ToolCallStarted/ToolCallCompleted/ToolCallFailed	AgentEvents.ToolCallStarted, ToolCallResult (live), ToolExecuted (durable, with outcome)
  HandoffStarted/HandoffCompleted	AgentEvents.HandedOff
  InputGuardrail*/OutputGuardrail*	AgentEvents.GuardrailBlocked (on a block); the outcome on AgentStatus.Blocked
  AgentStarted/AgentCompleted/AgentFailed, StepStarted/StepCompleted	kernel events: RunStarted, RunCompleted, RunFailed; ModelCallStarted/ModelCallCompleted
  TraceEvent.AgentStateUpdated	TraceEvent.AgentRunEnded
  context.progress(payload)	context.progress(name, version, payload), or an EventType
  ModelStep.next(messages, tools)	next(messages, tools, call)

  Also: AgentBuilder.withStreaming(), AgentRun.subscribe, AgentIO.stream* and AgentZ.stream* are new. Langfuse traces now use the run id as the trace id and the thread id as the session id. await() returns once the listener has returned from the run’s last event, so whatever it collected is complete. AgentRunEnded.usage is the run’s own usage, not the thread’s: a dashboard that summed the old cumulative figure per run double counted. A slow AgentIO.stream/AgentZ.stream consumer loses live events and gets a StreamEvent.LiveGap with their count; it no longer cancels the run. See the streaming guide.

• Session files: AgentState.saveToFile/loadFromFile become saved messages plus history. Save result.messages (a Vector[Message] has a ReadWriter) from a completed run, and later agent.run(newThreadId, query, RunConfig(), history = saved) on a thread that does not exist. history is refused on an existing thread, with a system message in it (prompts belong to the agents), or if it ends mid tool call. ConversationPersistenceExample shows it. Old session JSON files are not read.
• AgentIO and AgentZ wrap the new Agent. LLMClientIO.agent(id)(configure) and LLMClientZ.agent(id)(configure) take a function over the AgentBuilder (tools now belong to the agent); run, continueConversation, recover and resume take a RunConfig and return the AgentResult. Cancelling the fiber cancels the run. A thrown exception arrives as GraphError.NodeFailed carrying the original.
• CodeWorker. executeTask loses traceLogPath and returns Result[AgentResult]; WorkspaceSettings.traceLogPath and WORKSPACE_TRACE_LOG are removed. AssistantAgent builds its agent once and keeps a thread id; SessionState holds the thread id and the last result, and its consoleConfig parameter is gone.
• Graph loop API. ToolLoop.build(id, version, root, agents: Vector[LoopAgent]) builds a family of agents (the earlier ToolLoop.build(id, version, model, tools, middleware) is gone; LoopAgent(id, model, tools) with withSystemPrompt, withMaxSteps, withMiddleware and withHandoffs replaces its arguments), and ModelStep.next returns a Completion, so a wrapModelCall middleware’s next returns Result[Completion].
• Errors. Provider, tool and middleware failures are Left(GraphError...); the error content a tool failure gives the model is {"error": ...}. A guardrail block, the step limit and a suspension are Right.
• Samples. AsyncToolAgentExample is deleted; StreamingAgentExample, StreamingWithToolsExample and EventCollectionExample are rewritten on Agent.stream (#1329); the other agent samples use the builder.

Kernel completion (#1327)

GraphBuilder.implement, node and resumeNode take retry: RetryPolicy and cache: Option[CachePolicy]. Both default to off, so a graph that sets neither runs as before. CompiledGraph.toMermaid is new. These are source breaks, with no shims:

• ToolContext, GraphError.ToolFailed, ModelRequest and ToolCallRequest have a private constructor and no public copy. Build them with X(...) and change them with withY(...).
• Tool call ids and names are typed. ToolContext.toolCallId, GraphError.ToolFailed.tool and .toolCallId are the opaque ToolCallId and ToolName types: read the string with .value, and make an id with ToolCallId(call.id). ToolCallRequest gains toolCallId and toolName.
• recover re-runs a failed task under the node’s full retry policy. It used to give the task one more try.

Cancellation for non-chat clients and MCP tool hints (#1331)

Embedding, reranker, MCP, image clients and the Whisper and Tacotron2 speech engines now return Left(CancelledError) when interrupted, as the cloud speech clients already did, with the thread’s interrupt flag still set, and never retry it. Before, they reported an error of their own, and some reported a cancelled call as a success. Match CancelledError where you matched those errors: an embedding provider’s embed can now return Left(CancelledError) where it returned only an EmbeddingError, so a match that narrows its Left to EmbeddingError needs a case for it. The CHANGELOG entry for #1331 lists every client’s change. Source breaks, with no shims:

• Image generation errors are LLMErrors, with renamed cases: see Image generation errors are LLMErrors.
• MCPTransportImpl.sendRequest, sendNotification, MCPClient.initialize and getTools return Result instead of Either[String, _]: read the old string as error.message.
• ToolHints moves to llm4s-core: import org.llm4s.toolapi.ToolHints, not org.llm4s.agent.graph.tool.ToolHints.

llm4s-mcp also reads ToolHints from a server’s tool annotations (MCPClient.getToolHints, MCPToolRegistry.toolHints(name)), but only for a server configured with trustAnnotations = true on its MCPServerConfig (default false). Any other server reports no hints, so ToolHints.default (approval required) applies.

Orchestration removed (#1330)

org.llm4s.agent.orchestration is deleted: PlanRunner, Plan, Node, Edge, TypedAgent, Policies, OrchestrationError and CancellationToken, with org.llm4s.types.PlanId and org.llm4s.types.AgentId from llm4s-core (the agent’s id is org.llm4s.agent.AgentId). PlanRunner passed Map[String, Any] between nodes and cast each node to TypedAgent[Any, Any]. A typed graph does the same job with checked handles, checkpoints and recovery. The multi-agent graph recipe is a worked replacement.

Removed	Use instead
TypedAgent[I, O], TypedAgent.fromFunction and the other factories	a GraphNode[I] given to GraphBuilder.node; call an Agent inside the node for an LLM step
Node, Edge, Plan, Plan.builder	GraphBuilder.node / edge / staticJoin / dynamicJoin, then compile(entry)(output)
PlanRunner.execute(plan, inputs, token)	GraphRuntime.start(threadId, graph, input).flatMap(_.await())
PlanRunner(maxConcurrentNodes)	RunConfig with RunBudgets(maxConcurrency = n)
Policies.withRetry	retry = RetryPolicy(...) on GraphBuilder.node / implement
Policies.withTimeout	RunBudgets.withTimeout (the whole run); a node bounds its own calls
Policies.withFallback	ordinary Result code in the node (primary.orElse(fallback))
OrchestrationError	GraphError
CancellationToken	RunHandle.cancel() / AgentRun.cancel(), or interrupting the calling thread
org.llm4s.types.PlanId	RunId
org.llm4s.types.AgentId	org.llm4s.agent.AgentId

// before
val plan   = Plan.builder.addNode(research).addNode(summary).addEdge(Edge("e", research, summary)).build
val result = PlanRunner().execute(plan, Map("research" -> question), token)   // Future[Result[Map[String, Any]]]

// after
val b        = GraphBuilder("research", "v1")
val findings = StateKey.replace[String]("findings", "")
val digest   = StateKey.replace[String]("digest", "")
val summary = b.node[Unit]("summary", writes = Set(digest)) { (_, state, _) =>
  NodeResult.fromResult(for {
    f    <- state.get(findings)
    turn <- summariser.run(s"Summarise: $f")
    text <- turn.answer.toRight(ValidationError("summary", "no answer"))
  } yield Command.empty.update(digest, text))
}
val research = b.node[String]("research", writes = Set(findings)) { (q, _, _) =>
  NodeResult.fromResult(
    researcher.run(q)
      .flatMap(_.answer.toRight(ValidationError("research", "no answer")))
      .map(f => Command.empty.update(findings, f).goto(summary))
  )
}
val handle = b.compile(research)(_.get(digest)).flatMap(GraphRuntime.inMemory().start(ThreadId("t-1"), _, question))
handle.foreach(_.cancel())   // instead of token.cancel()

Agent.run, continueConversation, runMultiTurn, recover and resume now cancel their turn when the calling thread is interrupted, and return once it has ended (waiting up to 5 seconds for the turn to end), so recover can follow at once; a caller already interrupted starts no turn. Cancelling a graph run therefore also cancels the agent turns its nodes are waiting on. Before, the turn kept running after run returned Left(CancelledError). A caller that wants the turn to outlive an interrupt uses start, startRecover or startResume, and awaits the AgentRun itself. With tracing, the cancelled turn’s trace is complete when the call returns. A turn that had already begun committing its outcome when the interrupt came cannot be cancelled: the call returns that outcome (Right, Completed or Suspended), with the interrupt flag still set - test the flag, not only the result, if an interrupt must stop your own code. run(query), whose random thread id a Left does not carry, forgets the thread of a turn that failed or was cancelled once it has ended; name the thread (run(threadId, query)) to recover such a turn. The Java facade’s JAgent.run, continueConversation, resume and recover go through these calls, so an interrupted Java caller cancels its turn too; they used to stop only the wait.

Agent middleware

In 0.5.0 (#1279). The graph tool loop takes a stack of AgentMiddleware in place of a ToolCallPolicy: one ordered extension point with beforeAgent, afterAgent, wrapModelCall and wrapToolCall hooks, for approval, guardrails, logging, retry and rate limits. The graph runtime is Experimental, so these are source breaks with no shims. Design: docs/design/typed-agent-runtime/4.08-stage-0-prototype-agent-middleware.md (§4.8). (The ToolLoop.build signature shown below was generalised by #1328: see Stage 1 migration.)

• ToolCallPolicy and PolicyDecision are removed, and ToolLoop.build loses policy. It takes middleware: Seq[AgentMiddleware] = Nil instead. A policy becomes an AgentMiddleware overriding wrapToolCall: Allow is next(), Deny(reason) is ToolOutcome.Error(s"Denied: $reason"), and RequireApproval(reason) is if context.approved then next() else ToolOutcome.NeedsApproval(reason) - or use ApprovalMiddleware:

  // before
  val policy: ToolCallPolicy = call =>
    call.name match
      case "drop_table" => PolicyDecision.Deny("never allowed")
      case "deploy"     => PolicyDecision.RequireApproval("deploys are irreversible")
      case _            => PolicyDecision.Allow
  ToolLoop.build("assistant", "v1", model, tools, policy)
  
  // after
  val guard = new AgentMiddleware:
    val id = MiddlewareId("guard")
    override def wrapToolCall(request: ToolCallRequest, context: ToolContext)(next: () => ToolOutcome): ToolOutcome =
      request.call.name match
        case "drop_table"                  => ToolOutcome.Error("Denied: never allowed")
        case "deploy" if !context.approved => ToolOutcome.NeedsApproval("deploys are irreversible")
        case _                             => next()
  ToolLoop.build("assistant", "v1", model, tools, Seq(guard))
  
  // or, for approval alone
  ToolLoop.build("assistant", "v1", model, tools, Seq(ApprovalMiddleware.unlessReadOnly))
  

• ApprovalSource.Policy becomes ApprovalSource.Middleware(id), naming the middleware that asked; a tool’s own request is still ApprovalSource.Tool.
• Approve now re-runs the chain. It used to skip the policy and run the tool; it now runs the whole middleware chain again, from the outermost wrapper, with ToolContext.approved = true, as Edit does with the new arguments. A deny rule that depends only on the call refuses the same calls as before; a wrapper that asks for approval must pass when context.approved is set, or the call becomes an error result (asked for approval again).
• Guardrails in the graph loop apply their transformations. GuardrailMiddleware(input, output) runs input guardrails in beforeAgent and output guardrails in afterAgent, each on the value the previous one returned, so a guardrail that rewrites its input (PIIMasker, say) changes what the model sees and what the run answers. The legacy Agent validates every guardrail against the original value and keeps it, so it never applied a transformation; it is unchanged. A Block fails the run with the error CompositeGuardrail.all reports.

Agent tool contract and handoff ids

In 0.5.0 (#1278). The graph tool loop runs AgentTools, whose arguments are validated against their schema (rendered non-strict, so optional fields may be omitted) before any middleware or the tool runs, and legacy handoffs take an explicit id. The graph runtime is Experimental, so these are source breaks with no shims. Design: docs/design/typed-agent-runtime/4.07-stage-0-prototype-agent-tool-contract.md (§4.7).

(Since #1328, Handoff targets are builders and ToolLoop.build takes a family of LoopAgents: see Stage 1 migration.)

• LoopTool is AgentTool[A]. A tool now has an AgentToolSpec[A] - name, description and a core SchemaDefinition[A], with a ReadWriter[A] for the arguments - and receives them decoded, with a ToolContext (run, call id, thread state, approved):

  // before
  val search = LoopTool("search")((call, approved) => ToolOutcome.Completed(run(call.arguments)))
  
  // after
  final case class SearchArgs(query: String) derives ReadWriter
  val schema = Schema.`object`[SearchArgs]("Search arguments").withRequiredField("query", Schema.string("Query"))
  val spec   = AgentToolSpec[SearchArgs]("search", "Search the index", schema)
  val search = AgentTool(spec)((args, context) => ToolOutcome.Success(ToolContent.Json(ujson.Str(run(args.query)))))
  

  LoopTool.fromToolFunction(f) becomes AgentTool.fromToolFunction(f); its arguments are now validated against the function’s schema too.

• toolloop.ToolOutcome is tool.ToolOutcome. Completed(content: String) becomes Success(content: ujson.Value, update) (a ujson.Str is recorded as the string itself); Failed(message) becomes Error(message); NeedsApproval is unchanged. Ask and Fatal are new: Fatal(error) fails the run with GraphError.ToolFailed, reported inside GraphError.NodeFailed, and the run is recoverable. A thrown exception is still an error result.
• Approval now arrives in the context. LoopTool’s approved parameter is context.approved.
• State updates are declared. A tool’s Success update may touch only the keys in its writes; anything else fails the run. ToolLoop.build refuses a tool that declares ToolLoop.results or Messages.key.
• ToolLoop.build takes a ToolSet. Build it with ToolSet.of(tools*), which returns Left(ValidationError) for an invalid name ([a-zA-Z0-9_-]{1,64}), a duplicate name, an argument schema that is not an object, or a schema keyword the validator cannot check. AgentToolSpec.apply throws IllegalArgumentException for an invalid name.
• ModelStep.next takes the tool set. next(messages) becomes next(messages, tools), and ModelStep.fromClient(client, options) replaces options.tools with tools.toolFunctions, so pass the tools to ToolLoop.build, not in CompletionOptions.
• Invalid arguments never reach the tool. Arguments that break the schema, fail to decode or fail withValidation become the error result Invalid arguments for '<tool>': .... Edited approval arguments are checked again, and a middleware’s tool wrapper can still deny them.

• Handoffs take an id. Handoff(agent, ...) becomes Handoff(id, agent, ...), and Handoff.to(agent) / Handoff.to(agent, reason) become Handoff.to(id, agent) / Handoff.to(id, agent, reason), which throw IllegalArgumentException for an invalid id; Handoff.of(id, agent, reason) returns a Result. An id matches [a-zA-Z0-9_-]{1,52} and is unique within one run’s handoffs; an agent run given an invalid or duplicate id fails with ValidationError before any model call. The handoff tool is handoff_to_<id> rather than handoff_to_agent_<hash>, so a conversation stored with the old name does not match a handoff.

  // before
  agent.run(query, tools, handoffs = Seq(Handoff.to(physicsAgent, "Physics expertise required")))
  
  // after
  agent.run(query, tools, handoffs = Seq(Handoff.to("physics", physicsAgent, "Physics expertise required")))
  

Run API for graph runs

In 0.5.0 (#1277). GraphRuntime.start, recover and resume return a RunHandle once the thread is claimed, and the run executes on a thread the runtime owns. The graph runtime is Experimental, so these are source breaks with no shims. Design: docs/design/typed-agent-runtime/4.06-stage-0-prototype-run-api-and-event-dispatch.md (§4.6).

• NodeContext is RunContext. context.taskId, context.nodeId and context.superstep become context.position.taskId, .nodeId and .superstep; emit and progress are unchanged, and isCancelled is new. Clients and tools stay captured by node closures; resolve per-tenant or per-thread resources from context.config.tenantId or context.position.
• Limits are per run. compile(entry, maxSupersteps) becomes compile(entry), and ToolLoop.build drops maxSupersteps. Pass RunConfig(budgets = RunBudgets(maxSupersteps = ..., timeout = ..., maxConcurrency = ...)) to each run; RunBudgets.of validates untrusted values as a Result, while apply and the with* setters throw IllegalArgumentException for a non-positive value.
• CompiledGraph.run(input) is removed. Use GraphRuntime.inMemory().start(threadId, graph, input).flatMap(_.await()).
• step(execution) is step(threadId, execution, config). It runs at most config.budgets.maxConcurrency tasks at a time and checks no superstep limit.
• start/recover/resume(..., runId, durability) become (..., config, durability) and return Result[RunHandle[O]]; the run id is config.runId. Call handle.await() for the RunResult. Cancel with handle.cancel(): interrupting the caller no longer cancels the run, and interrupting a thread blocked in await returns Left(CancelledError) while the run continues. A cancel that arrives once the run has begun committing its outcome - completed, suspended or failed - is ignored, and the run ends with that outcome.
• recover and resume take the thread first, as start does. recover(graph, threadId, ...) becomes recover(threadId, graph, ...), and resume(graph, threadId, answers, ...) becomes resume(threadId, graph, answers, ...). Both arguments have different types, so the compiler flags every call to swap.
• Subscribers have their own thread. subscribe gains capacity (default 1024, at least 2). Listeners run on the subscription’s dispatcher thread, not a task or committing thread. A listener that falls behind by more than capacity durable events is disconnected; a listener that throws is disconnected rather than ignored. StreamEvent gains LiveGap(dropped) and Disconnected(lastSeq, reason); resubscribe with afterSeq = lastSeq to continue without a gap (TracingSubscriber.attach included: a lagging tracer is not re-attached for you). A subscription belongs to the thread, not one run, and keeps a parked dispatcher thread until cancel(); cancel subscriptions you no longer need.
• Tenants are checked. A run whose RunConfig.tenantId differs from the one on the thread’s latest checkpoint is refused with GraphError.TenantMismatch(threadId, requested), which names only the caller’s tenant, never the owner’s; None and Some differ. The check comes first: a wrong-tenant call gets TenantMismatch rather than IncompleteRun, PendingInterrupts, NothingToRecover, NotSuspended or ThreadBusy. Checkpoints move to format 3, and earlier checkpoints read as having no tenant.
• GraphError is classified per case. It extends LLMError rather than NonRecoverableError; every case is still a NonRecoverableError except the new DeadlineExceeded, which is a RecoverableError - recover it with a new budget. Code that relied on GraphError <: NonRecoverableError should match the case.
• New cases break exhaustive matches. GraphError.DeadlineExceeded, TenantMismatch and RunCrashed, RunEvent.RunTimedOut, and StreamEvent.LiveGap and Disconnected are new. RunStarted is now a case class, and it, RunRecovered and RunResumed gain tenantId and principal; events in the old encoding still read.

Provider configs: build with apply, change with with*

In 0.5.0 (#1388). OpenAIConfig, AnthropicConfig and OllamaConfig follow the growth-prone pattern (pass 5 below), so a new field no longer breaks Java and Kotlin callers. The constructor and copy are private:

• Scala: OpenAIConfig(...) with the full field list, positional or named, still compiles. Replace config.copy(baseUrl = url) with config.withBaseUrl(url); each field has a setter (withApiKey, withModel, withOrganization, withContextWindow, …), and OpenAIConfig’s Option fields take the value or an Option.
• Java and Kotlin: call the companion’s short apply and chain setters instead of the constructor - OpenAIConfig.apply(apiKey, "gpt-4o").withOrganization("org-1"), AnthropicConfig.apply(apiKey, model), OllamaConfig.apply(model, baseUrl). It takes the default base URL and a context window from the model name; fromValues still consults the bundled model catalogue.

Image generation errors are LLMErrors

In 0.5.0 (#1331). llm4s-image’s generation clients return Either[LLMError, _], so an interrupted call can return CancelledError, and ImageGenerationError extends LLMError. Source breaks, with no shims:

• Five cases are renamed so they no longer share a name with an org.llm4s.error type. Both are LLMErrors, so a match that imported the wrong one would compile and never fire:

  Before (org.llm4s.imagegeneration)	After
  AuthenticationError	ImageAuthenticationError
  RateLimitError	ImageRateLimitError
  ServiceError	ImageServiceError
  ValidationError	ImageValidationError
  UnknownError	ImageUnknownError

• ImageServiceError(message, statusCode): the second field was code: Int, which clashed with LLMError.code: Option[String]. It is a sealed type now (TransientImageServiceError for 0, 408, 429 and 5xx, RejectedImageServiceError otherwise); build and match it through ImageServiceError(message, status) as before.
• A match on a client’s result needs a case for other LLMErrors, CancelledError among them.

Cancellation by interrupt

In 0.5.0 (#1270). An interrupted call now returns Left(CancelledError) with the thread’s interrupt flag still set. Code that matched ExecutionError, TimeoutError or SimpleError to detect an interrupted call should match CancelledError instead, and ToolCallError.Cancelled for a tool call. CancelledError is non-recoverable and is never retried. Chat clients no longer throw InterruptedException or report UnknownError for an interrupt.

• New cases break exhaustive matches. ErrorKind.Cancelled (metric label cancelled), RunEvent.RunCancelled and ToolCallError.Cancelled are new; a match over ErrorKind, RunEvent or ToolCallError needs a case for each.
• Mid-stream, NetworkError becomes CancelledError. A stream read cut off by an interrupt used to be a recoverable NetworkError; code that retried it should stop on CancelledError.
• DefaultErrorMapper and Try(...).toResult classify cancellations. An exception mapped while the thread is interrupted, or caused by InterruptedException or ClosedByInterruptException, becomes CancelledError rather than UnknownError or NetworkError. A bare InterruptedIOException - OkHttp’s call timeout is one - stays a timeout. Mapping never sets the interrupt flag; code that catches an InterruptedException itself must restore it.
• ReliableClient returns CancelledError, not the local RateLimitError, when interrupted while waiting for a local rate-limit token.
• Graph supersteps run concurrently. The default executor ran a superstep’s tasks one after another; it now runs them concurrently on virtual threads, at most 16 at a time. Node code must be thread-safe, a task does not inherit the caller’s ThreadLocal or MDC context, and in Sync durability, durable events and live progress are delivered on the task threads, not the caller’s. Since #1277 they are delivered on each subscription’s dispatcher thread instead (see above).

Pre-baseline API cleanup, pass 8

In 0.5.0; continues pass 7 below. It closes the last gaps in the frozen modules’ public API found by checking the re-audit against main.

• llm4s-agent’s console UI is internal. ConsoleInterface, ConsoleConfig (and its StyleConfig) and MessageType are private[assistant]: ConsoleConfig’s colours were fansi Attrs and MessageType carried a public cats.Show, which would have frozen both libraries into llm4s-agent’s API. AssistantAgent loses its consoleConfig parameter and the four-argument compatibility constructor; construct it with named arguments:

  new AssistantAgent(client, tools, sessionDir = "./sessions", agentContext = AgentContext.Default)
  

• SessionState.localDateTimeRW, a public implicit upickle codec for java.time.LocalDateTime that any import SessionState._ picked up, is private[assistant].
• org.llm4s.llmconnect.utils.SimilarityUtils is private[llm4s]: only core’s caching client and llm4s-rag use it. Applications needing cosine similarity can compute it directly.

Pre-baseline API cleanup, pass 7

In 0.5.0; continues pass 6 below, which typed the times a caller supplies. This pass types the times the library reports: an elapsed time is a FiniteDuration, a point in time an Instant, and the unit leaves the name. Every JSON, trace and wire format keeps its keys and millisecond values (duration_ms, durationMs, executionTimeMs, …).

// before
case AgentCompleted(state, steps, durationMs, _) => println(s"Done in ${durationMs}ms")
// after
case AgentCompleted(state, steps, duration, _) => println(s"Done in ${duration.toMillis}ms")

Check string interpolation in particular: s"${duration}ms" still compiles, but prints 150 millisecondsms.

Module	Before	After
llm4s-core	TraceEvent.ToolExecuted.duration: Long, RAGOperationCompleted.durationMs, ImageGenerationCompleted.durationMs, Tracing.traceRAGOperation(durationMs)	duration: FiniteDuration
	ProviderExchange.durationMs: Long	duration: FiniteDuration
	RateLimitError.requestsRemaining, RateLimitError.resetTime	removed: no constructor could set them, so they were always None
llm4s-agent	AgentEvent.ToolCallCompleted.durationMs, AgentCompleted.durationMs, AgentEvent.toolCompleted/agentCompleted’s durationMs	duration: FiniteDuration
llm4s-agent-tools	ShellResult.executionTimeMs, HTTPResult.responseTimeMs	executionTime, responseTime (the tool’s JSON keeps the ...Ms keys)
llm4s-rag	TimingInfo.durationMs, durationSeconds, avgPerItemMs	duration, avgPerItem: Option[FiniteDuration]
	ExperimentResult.totalTimeMs/totalTimeSeconds	totalTime: FiniteDuration
	BenchmarkResults.startTime/endTime: Long (epoch ms), totalDurationMs/totalDurationSeconds	Instants, totalDuration: FiniteDuration
llm4s-image	ServiceStatus.averageGenerationTime: Option[Long] (ms)	Option[FiniteDuration]
llm4s-speech	Transcription.processingTimeMs: Option[Long]	processingTime: Option[FiniteDuration]
workspace	ExecuteCommandResponse.durationMs, CommandCompletedMessage.durationMs	duration: FiniteDuration (the protocol keeps durationMs)

Pre-baseline API cleanup, pass 6

In 0.5.0; continues pass 5 below. A time the caller supplies is typed: a duration is a scala.concurrent.duration.FiniteDuration and a point in time a java.time.Instant, never a raw Int/Long whose unit lives in its name or its Scaladoc. Names lose their unit suffix (timeoutMs becomes timeout). Defaults are unchanged, and so are the wire formats: HOCON keys, JSON sent to Exa and the workspace runner, and HTTP headers.

import scala.concurrent.duration.*

// before
Left(RateLimitError("openai", 1000L))              // milliseconds, documented as seconds
CrawlerConfig(delayMs = 500, timeoutMs = 30000)
// after
Left(RateLimitError("openai", 1.second))
CrawlerConfig(delay = 500.millis, timeout = 30.seconds)

llm4s-core errors, retry and reliability

• RecoverableError.retryDelay, RateLimitError.retryAfter/retryDelay and ServiceError.retryDelay are Option[FiniteDuration]; RateLimitError(provider, retryAfter) takes a FiniteDuration and the RateLimitError(message, retryAfter, provider) extractor yields one. RateLimitError.resetTime is an Option[Instant]. The fallback delay is named RateLimitError.DefaultRetryDelay (30 seconds, as before).
• TimeoutError.timeoutDuration, ReliabilityConfig.deadline/withDeadline, CircuitBreakerConfig.recoveryTimeout/withRecoveryTimeout, every RetryPolicy delay (exponentialBackoff, linearBackoff, fixedDelay, custom’s function, delayFor’s result) and ErrorRecovery’s recoveryTimeout were Duration and are FiniteDuration: an infinite delay or deadline could not be slept or added to a clock. Code passing 5.seconds is unchanged.
• ReliableClient and ErrorRecovery.CircuitBreaker take clock: () => Instant (was milliseconds since the epoch); ReliableClient’s sleep, ErrorRecovery’s and LLMClientRetry’s sleepFn take a FiniteDuration (was a Long of milliseconds). ErrorRecovery.recoverWithBackoff’s baseDelay is a FiniteDuration.

llm4s-agent

OrchestrationError.AgentTimeoutError carries timeout: FiniteDuration (was timeoutMs: Long).

Beta modules

Module	Before	After
llm4s-agent-tools	ExaSearchToolConfig/BraveSearchToolConfig/DuckDuckGoSearchToolConfig timeoutMs: Int, ShellConfig.timeoutMs: Long, HttpConfig.timeoutMs: Int	timeout
	ExaSearchToolConfig.livecrawlTimeout: Option[Int] (ms)	Option[FiniteDuration], still sent to Exa in ms
llm4s-rag	CrawlerConfig.delayMs/timeoutMs, withDelay(ms)/withTimeout(ms) (also on WebCrawlerLoader), UrlLoader.timeoutMs/withTimeout(ms)	delay/timeout, withDelay(FiniteDuration)/withTimeout(FiniteDuration)
	RobotsTxtParser.isAllowed/getRules timeoutMs, RobotsTxt.crawlDelay: Option[Int] (s)	timeout, Option[FiniteDuration] (fractional Crawl-delay values are now kept)
	EvaluatorOptions.timeoutMs, ChunkingUtils windowSeconds/clipSeconds, RateLimitedLogger throttleSeconds	timeout, window/clip, throttle
	HikariDefaults.CONNECTION_TIMEOUT_MS/IDLE_TIMEOUT_MS/MAX_LIFETIME_MS	ConnectionTimeout/IdleTimeout/MaxLifetime
llm4s-mcp	MCPServerConfig/transport timeout: Duration, MCPToolRegistry cacheTTL: Duration, MCPServer.stop(delay: Int)	FiniteDuration
	StdioTransportImpl startupTimeoutMs: Int	startupTimeout
llm4s-image	ImageGenerationConfig.timeout: Int (ms), the imagegeneration.provider.HttpClient methods’ timeout: Int	FiniteDuration
	OpenAI/Anthropic vision configs’ connectTimeoutSeconds/requestTimeoutSeconds	connectTimeout/requestTimeout
workspace	executeCommand timeout: Option[Int] (s), WorkspaceSandboxConfig.defaultCommandTimeoutSeconds	Option[FiniteDuration], defaultCommandTimeout; the JSON still carries whole seconds under the old keys

Test doubles of imagegeneration.provider.HttpClient type the timeout as FiniteDuration; a ScalaMock onCall lambda typed _: Int compiles but fails at runtime.

Pre-baseline API cleanup, pass 5

In 0.5.0; continues pass 4 below. It settles the provider-author SPI’s API quality before the binary-compatibility baseline.

Growth-prone data types: construct with named arguments, change with with*

A case class cannot gain a field without breaking binary compatibility - its constructor, apply and copy all change - so the types the library will plausibly extend now have a private constructor, a public companion apply carrying the defaults, and with* setters: CompletionOptions, Completion, StreamedChunk, TokenUsage, ModelCapabilities, ModelMetadata, ProviderConfigSpec, EmbeddingConfigSpec, ProviderFeatures, NamedProviderConfig, ReliabilityConfig, CircuitBreakerConfig, RateLimitConfig, ContextConfig.

Construction is unchanged. .copy(...) is no longer available outside the type:

// before
val opts = CompletionOptions(temperature = 0.2).copy(maxTokens = Some(500))

// after
val opts = CompletionOptions(temperature = 0.2).withMaxTokens(500)

Every field has a withX. An Option field’s setter takes either the value or an Option (withMaxTokens(500), withMaxTokens(None)). Match these types by name (c.usage), not by position: a positional pattern breaks when a field is added.

Llm4sHttpClient returns Result and takes FiniteDuration

Every request method (get, post, postBytes, postMultipart, put, delete, postRaw, postStream) returns Result[...] and never throws for a transport failure: a timeout is a TimeoutError, a connection or I/O failure a NetworkError, an invalid URL, header or timeout a ValidationError, an interruption an ExecutionError (with the interrupt flag restored). A non-2xx status is still a Right. getResult is removed - get is now a Result itself.

// before
val response = Try(http.post(url, headers, body, timeout = 120000)).toResult
// after
import scala.concurrent.duration.*
val response: Result[HttpResponse] = http.post(url, headers, body, timeout = 120.seconds)

HttpRawResponse and StreamingHttpResponse carry headers, and every response type has a case-insensitive header(name). Pass the headers to HttpErrorMapper.mapHttpError(status, body, provider, headers) so a 429’s Retry-After (seconds or an HTTP date) becomes the RateLimitError’s delay; every built-in provider does.

Test doubles that implement the trait return Result[...] and take FiniteDuration; return a Left to simulate a failure instead of throwing. With ScalaMock, .returns(resp) becomes .returns(Right(resp)) and .throws(e) becomes .returns(Left(error)); type the timeout in onCall/where lambdas as FiniteDuration - a lambda typed _: Int still compiles but fails at runtime.

StreamingAccumulator

Before	After
new StreamingAccumulator()	StreamingAccumulator.create() (the class is final)
getCurrentContent, getCurrentThinking, getCurrentToolCalls	currentContent, currentThinking, currentToolCalls
snapshot(), AccumulatorSnapshot	toCompletion(created)
StreamingAccumulator.withInitialState(...)	create(), then addChunk / updateTokens

RequestTransformer and TransformationResult

• TransformationResult.warnings is removed: nothing ever filled it.
• TransformationResult.transform(modelId, options, messages, transformer, dropUnsupported = true)
  • the required transformer now comes before the defaulted flag.
• RequestTransformer#getDisallowedParams is now disallowedParams.

llm4s-provider-testkit

New, Beta. A provider module’s spec mixes in org.llm4s.testkit.ProviderModuleChecks; see Writing a provider. In this repository, CredentialsRoundTrip and LocalProviderTestServer moved from core’s test sources to org.llm4s.testkit.

Internal now

Llm4sConfig.providerFrom(source) and apiKeySourcesFrom(source) took a pureconfig ConfigSource, which is not part of llm4s’s API; they are private[llm4s]. Load configuration with Llm4sConfig.provider(name) / defaultProvider() / apiKeySources().

Pre-baseline API cleanup, pass 4

In 0.5.0; continues pass 3 below. Vendor fields leave NamedProviderConfig.

Config files: no change. organization, endpoint, apiVersion, contextWindow and reserveCompletion keep their names in llm4s.providers.<name> sections. What changed is who owns them: each is now a provider-specific key that only its provider declares.

Key	Declared by
organization	openai, requesty, openrouter
endpoint (required), apiVersion (default V2025_01_01_PREVIEW)	azure
contextWindow, reserveCompletion	openai-compatible

In a section for any other provider these keys used to be read and silently ignored; they are now reported once as unknown keys, with a warning, and dropped - delete them. Vertex AI’s deprecated endpoint/organization aliases for project/location work exactly as before. A non-numeric contextWindow/reserveCompletion is now reported when the section is resolved, naming the key (llm4s.providers.<name>.contextWindow must be a positive whole number, got '...'), rather than as a type error while reading the block.

Scala callers:

1. NamedProviderConfig no longer has organization, endpoint, apiVersion, contextWindow or reserveCompletion. Read the validated value from the section’s extras: config.organization → config.extra("organization") (or OpenAIConfig.OrganizationKey), config.endpoint → config.extra(AzureProvider.EndpointKey), config.apiVersion → config.extra(AzureProvider.ApiVersionKey), config.contextWindow → config.extra(OpenAICompatibleProvider.ContextWindowKey).map(_.toInt) (likewise ReserveCompletionKey). Values are strings.
2. Code constructing NamedProviderConfig(...) drops those arguments; pass them in extras = Map("endpoint" -> ..., ...) if the descriptor needs them. Positional calls NamedProviderConfig(id, model, baseUrl, apiKey, org, endpoint, apiVersion) become NamedProviderConfig(id, model, baseUrl, apiKey).
3. ProviderConfigSpec(requiresEndpoint = true, endpointDescription = "...") → ProviderConfigSpec(extras = Seq(ProviderConfigKey.required("endpoint", "..."))). ProviderConfigSpec.BuiltinKeys is now provider, model, baseUrl, apiKey, headers, and BuiltinAliasKeys is baseUrl, apiKey - a deprecatedAliases entry naming one of the moved keys still works, resolved from the section’s extras.
4. ProviderModelListers.openAICompatible is now in llm4s-openai-compatible (package org.llm4s.config unchanged): a provider module calling it adds that dependency. It no longer sends OpenAI-Organization from the section; pass sectionHeaders = ProviderModelListers.openAIOrganizationHeader (and declare OpenAIConfig.OrganizationConfigKey) to keep it, or any NamedProviderConfig => Map[String, String] to derive other headers. ProviderModelLister and DiscoveredModel stay in llm4s-core.

Pre-baseline API cleanup, pass 3

In 0.5.0; continues pass 2 below.

llmconnect.middleware, ReliableProviders and ReliabilitySyntax are removed

The middleware package had no users outside its own tests and duplicated caching and the metrics every provider client already records. LLMClient is a trait, so a decorator is a class that implements it and delegates. ReliableProviders.wrap and .withReliability(...) were shorthand for one constructor:

// before
val reliable = ReliableProviders.wrap(client, "openai", config)
val other    = client.withReliability("openai")

// after
val reliable = new ReliableClient(client, "openai", config)
val other    = new ReliableClient(client, "openai", ReliabilityConfig.default)

ReliableClient’s companion factories (ReliableClient(client), ReliableClient(client, config), ReliableClient(client, config, metrics), ReliableClient.withProviderName) go too: three of them guessed the provider name from the client’s class name. Use the constructor. For a client built from config, the name is providerConfig.providerId.asString.

Behaviour fix: ReliableClient now applies ReliabilityConfig.rateLimit itself, before every attempt, retries included. Before, only ReliableProviders.wrap honoured it, so a new ReliableClient(...) with rate limiting enabled was not rate limited.

OpenAI’s model rules leave RequestTransformer

RequestTransformer.default now applies only the registry’s capabilities. The o-series constraints (no system message, no native streaming, temperature 1, no sampling penalties) and the max_completion_tokens rule for o-series and gpt-5 moved into llm4s-openai, which is the only client that needs them; the Anthropic and Gemini clients no longer apply them to models named like OpenAI’s.

Removed	Use instead
RequestTransformer#requiresMaxCompletionTokens, TransformationResult.requiresMaxCompletionTokens	none in core; it is an OpenAI wire parameter
DefaultRequestTransformer (now package-private)	RequestTransformer.default(service) or withOverrides(...)
(new)	RequestTransformer.adjusted(service)((modelId, capabilities) => ...), for a provider module’s own rules on top of the registry

Provider-author SPI

The plumbing provider modules build on is a public, frozen SPI; see Writing a provider. Two OpenAI-format helpers moved to llm4s-openai-compatible, with unchanged packages: org.llm4s.llmconnect.model.ResponseFormatMapper and org.llm4s.llmconnect.serialization.{ToolCallDeserializer, StandardToolCallDeserializer}. ProviderResultOps (tapRight / tapLeft) is now private[llm4s].

Pre-baseline API cleanup, pass 2

In 0.5.0; continues pass 1 below.

Removed	Use instead
ToolRegistry#getToolDefinitionsSafe(provider)	getOpenAITools(). It returned the same JSON for every provider it accepted and failed for the rest; every client takes this format
CancellationToken#cancellationFuture, cachedCancellationFuture	whenCancelled, a shared Future[Unit] that succeeds on cancel: token.whenCancelled.map(_ => Left(myError))
CancellationToken#throwIfCancelled(), orchestration.CancellationException	if (token.isCancelled) Left(...)
ManagedResource.fileInputStream, dataOutputStream, byteArrayInputStream	ManagedResource.fromTry(() => Try(new ...), s => Try(s.close())), or scala.util.Using
ManagedResource map / flatMap (ManagedResourceOps)	none: they never released the underlying resource. Nest use calls instead

Behaviour fix: a PlanRunner node cancelled while running now fails the plan with OrchestrationError.PlanExecutionError("Node <id> cancelled", ...). It used to surface as a NodeExecutionError wrapping CancellationException, because the cancellation future only ever failed and the mapping to the cancelled error never ran.

These utilities moved to the one module that uses them. Package names are unchanged, so code that already depends on that module needs nothing; code that used them through llm4s-core alone adds the module:

Utility	Now in
org.llm4s.util.SqlIdentifier, org.llm4s.llmconnect.utils.ChunkingUtils	llm4s-rag
org.llm4s.resource.ManagedResource	llm4s-speech
org.llm4s.util.LiftToResult	llm4s-observability

Pre-baseline API cleanup, pass 1

In 0.5.0; from the spine re-audit. A later pre-1.0 release sets the binary-compatibility baseline, so public API that nothing used, or that was already deprecated, is removed now rather than frozen.

Removed	Use instead
Result.fromTry(t)	t.toResult (import org.llm4s.types.TryOps) or Safety.fromTry(t)
error.isRecoverable	LLMError.isRecoverable(error), or match on RecoverableError / NonRecoverableError
LLMError.fromThrowable(t)	t.toLLMError (org.llm4s.error.ThrowableOps.RichThrowable)
ToolBuilder#build()	buildSafe(), which returns Result[ToolFunction]
ToolRegistry#getToolDefinitions(p)	getToolDefinitionsSafe(p)
LLMCompressor.compress(...), LLMCompressedConversation	ContextManager.withDefaults(tokenCounter, Some(llmClient)) then manageContext(conversation, budget); no direct equivalent (see below)
Agent overloads taking debug, tracing or traceLogPath (run, runStep, continueConversation, runMultiTurn, runWithEvents, continueConversationWithEvents, runCollectingEvents, runWithStrategy, continueConversationWithStrategy)	the same method with context = AgentContext(debug = ..., tracing = ..., traceLogPath = ...)
ContextConfig(..., enableRollingSummary = ..., ...), ContextConfig.legacy(...)	drop the argument (nothing read it); ContextConfig(...) or ContextConfig.default.copy(...)
Safety.sequenceV(xs)	xs.traverse(_.toValidatedNec) with import cats.syntax.all.*, which keeps every error; or, given items and a validator, Result.validateAll(items)(validate). Not Result.sequence, which stops at the first error
LLMError.llmErrorShow, error.show, error.display	error.formatted
org.llm4s.types aliases and wrappers the library never used (CompletionId, ToolName, ToolCallId, Url, MessageId, WorkspaceId, Json, Timeout, TokenCount, the MCP/image/audio/video/plugin/workflow/metrics types, …)	the underlying type, e.g. String, ujson.Value, Long, Int
ConnectionStatus, ProviderCapabilities, ClientHealth, StreamingOptions, RuntimeId, ModelId	none; nothing used them

LLMCompressor.compress summarised a whole conversation with one LLM call, optionally with a custom prompt. Nothing replaces it one-for-one: LLMCompressor.squeezeDigest only shrinks messages already marked [HISTORY_SUMMARY] and returns any other conversation unchanged, so do not swap one call for the other. ContextManager.manageContext is the supported path - it builds the digests, compresses them and trims to the budget (ContextConfig.enableLLMCompression gates the LLM step); the custom prompt has no counterpart.

RateLimitedLogger, ProvidersConfigModel.RawNamedProviderSection / RawProvidersConfig, agent.orchestration.MDCContext and assistant.ShowInstances are now package-private.

org.llm4s.types keeps Result, AsyncResult, TryOps / OptionOps / FutureOps, and the newtypes the library’s APIs take: SessionId, TraceId, FilePath, DirectoryPath, AgentId, PlanId (both removed later with orchestration, #1330), SemanticBlockId, ArtifactKey, ExternalizedContent, ContentSize, HeadroomPercent and the TokenBudget, ContextWindowSize, ByteCount and ExternalizationThreshold aliases.

Slice 7: llm4s-agent - the agent runtime leaves core

The second slice 7 carve (#1242, decision D4) moves the agent runtime into a new module, llm4s-agent. Package names are unchanged, so no import changes; code that uses the agent adds one dependency:

libraryDependencies += "org.llm4s" %% "llm4s-agent" % "<version>"

Moved to llm4s-agent	Package
Agent, AgentState, AgentStatus, AgentContext, Handoff, ContextWindowConfig, AgentTraceFormatter	org.llm4s.agent
Guardrails - the traits, CompositeGuardrail, the built-in validators, the LLM-as-judge and RAG guardrails, PII patterns	org.llm4s.agent.guardrails.*
DAG orchestration	org.llm4s.agent.orchestration
Streaming events (AgentEvent and its cases)	org.llm4s.agent.streaming
The console assistant: AssistantAgent, SessionManager, ConsoleInterface (Beta)	org.llm4s.assistant

Not moved: agent memory (org.llm4s.agent.memory) is already llm4s-memory, which does not depend on llm4s-agent. UsageSummary and ModelUsage moved to org.llm4s.llmconnect.model in llm4s-core just before this (see below). The ready-made tools are llm4s-agent-tools.

What stays in llm4s-core is what the agent is built on: LLMClient and LLMConnect, the tool API (ToolFunction, ToolRegistry), and the tracing contract. An agent run is traced through TraceEvent.AgentStateUpdated, built by AgentState#toTraceEvent, so tracing backends need nothing from the agent module.

Other modules: llm4s-workspace-client now depends on llm4s-agent, for its codegen package. fansi leaves llm4s-core with the assistant, its only user.

Slice 7: llm4s-agent-tools - the built-in tools leave core

The first slice 7 carve (#1242, decisions D2 and D3) moves the ready-made tools into a new module, llm4s-agent-tools. Package names are unchanged, so no import changes; code that uses any of them adds one dependency:

libraryDependencies += "org.llm4s" %% "llm4s-agent-tools" % "<version>"

Moved to llm4s-agent-tools	Package
BuiltinTools and every tool under it: core (DateTime, Calculator, UUID, JSON), filesystem, http, shell, search (Brave, DuckDuckGo, Exa)	org.llm4s.toolapi.builtin.*
WeatherTool	org.llm4s.toolapi.tools
ToolsConfigLoader (was private[config], now public), BraveSearchToolConfig, DuckDuckGoSearchToolConfig, ExaSearchToolConfig	org.llm4s.config
ToolsConfigKeys (new)	org.llm4s.config

What stays in llm4s-core is the tool API: ToolFunction, ToolRegistry, ToolBuilder, Schema, the execution strategies and SafeParameterExtractor. Your own tools need nothing new. llm4s-agent-tools depends on llm4s-core only - not on the agent runtime - so the tools work with plain tool calling through ToolRegistry as well as with Agent.

Configuration is unchanged

The llm4s.tools.brave, .duckduckgo and .exa keys, their defaults, and the BRAVE_SEARCH_API_KEY, BRAVE_SEARCH_COUNT, BRAVE_SEARCH_API_URL, BRAVE_SAFE_SEARCH, DUCK_DUCK_GO_SEARCH_API_URL and EXA_* variables are the same. The block moved from core’s reference.conf to the module’s, beside the code that reads it.

Source breaks

1. The tools need llm4s-agent-tools (table above).

2. Llm4sConfig.loadBraveSearchTool(), loadDuckDuckGoSearchTool() and loadExaSearchTool() are removed. They returned config types that left core. The same methods are on ToolsConfigLoader:

   import org.llm4s.config.ToolsConfigLoader
   
   // before
   val braveConfig = Llm4sConfig.loadBraveSearchTool()
   
   // after
   val braveConfig = ToolsConfigLoader.loadBraveSearchTool()
   

   Each also takes a ConfigSource, to read from one of your own.

3. ConfigKeys.BRAVE_SEARCH_API_KEY is ToolsConfigKeys.BRAVE_SEARCH_API_KEY, with ToolsConfigKeys.EXA_API_KEY beside it.

Slice 7: UsageSummary and ModelUsage move to org.llm4s.llmconnect.model

In 0.5.0; the preparation step of slice 7 (#1242, decision D1).

Slice 7 carves the agent runtime out of llm4s-core into llm4s-agent. UsageSummary and ModelUsage were in org.llm4s.agent, but they depend only on TokenUsage, and llm4s-observability’s CostTracker builds them too. Leaving them in the agent package would make llm4s-observability depend on the agent runtime; moving the file to core while keeping its package would split org.llm4s.agent across two jars. So they move to org.llm4s.llmconnect.model, beside TokenUsage, and stay in llm4s-core.

This is the one change to an import in the slice; every other move keeps its package.

// before
import org.llm4s.agent.{ ModelUsage, UsageSummary }

// after
import org.llm4s.llmconnect.model.{ ModelUsage, UsageSummary }

Code that only reads AgentState.usageSummary or CostTracker.snapshot needs no change. The JSON form is unchanged, so an AgentState saved before the move still loads.

llm4s no longer brings a logging backend

In 0.5.0; found by the slice 6 spine audit (#1133).

Every llm4s module used to declare logback-classic and log4j-to-slf4j as compile dependencies, so they reached your classpath with llm4s. Libraries should leave that choice to the application, and the published modules now declare only slf4j-api.

• If you already configure logging (your own logback, log4j2 via log4j-slf4j2-impl, or another SLF4J 2 backend), nothing changes - except that llm4s no longer adds a second backend or a bridge that clashes with log4j-core.

• If you relied on the logback llm4s brought, declare it yourself; without a backend, SLF4J logs nothing and prints one “no SLF4J providers were found” warning:

  libraryDependencies += "ch.qos.logback" % "logback-classic" % "1.5.34"
  

• If you used Monocle, commons-io or fansi through llm4s, declare them yourself: Monocle and commons-io are no longer dependencies of any llm4s module, and fansi comes only with llm4s-core.

A failed read no longer deletes or clears indexed documents

Follow-up to #1236; in 0.5.0.

LoadResult.Failure gains a fourth field, documentId: Option[String] = None: the id the failed document is (or would be) indexed under. source stays the path, key or URL. RAG.sync uses it to tell a document it could not read from one that has gone from the source.

• A pattern match on LoadResult.Failure needs a fourth argument: case LoadResult.Failure(source, error, recoverable) becomes case LoadResult.Failure(source, error, recoverable, documentId) (or _). Constructor calls, LoadResult.failure(source, error), .source, .error and .recoverable are unchanged; LoadResult.failure(source, error, documentId = id) is new.
• sync keeps a document whose read fails. A Failure with a documentId keeps that document’s indexed version: it is not deleted, and not counted in SyncStats. Previously sync deleted it, because it had not been “seen”. SourceBackedLoader (S3 and every other DocumentSource) and UrlLoader name the document on every per-document failure, FileLoader when extraction fails. Read failures are logged at WARN.
• A Failure with no documentId makes sync skip its deletion pass for that run, as it cannot tell which unlisted document failed. Adds and updates still apply. WebCrawlerLoader leaves it out on purpose (the pages below a failed page went uncrawled), as does FileLoader for a missing path. A custom DocumentLoader should set documentId on per-document failures when it knows the id, or its failures will now hold back deletions.
• refresh reads the whole loader before it clears anything. A ListingFailure - at any point, including a later S3 page - or, under failFast, any Failure returns the error and leaves the index and registry untouched. Previously the index was cleared first and left empty or half-rebuilt. The cost is memory: refresh and refreshAsync now hold every loaded document’s extracted text in memory before clearing; use sync, which streams, for a source too large for that. Without failFast, a document that fails to read is still left out of the rebuilt index.
• DocumentLoaders.successesOnly still drops per-document failures, so a sync through it still deletes a document whose read failed.

Slice 6: llm4s-observability-prometheus - Prometheus leaves core

The second slice 6 carve (#1133, decisions D3 and D4) moves the Prometheus metrics backend into a new module, llm4s-observability-prometheus, which carries the Prometheus client and HTTP server. With it gone, llm4s-core declares no observability dependency. Package names are unchanged, so no import changes:

libraryDependencies += "org.llm4s" %% "llm4s-observability-prometheus" % "<version>"

Moved to llm4s-observability-prometheus	Package
PrometheusMetrics, PrometheusEndpoint	org.llm4s.metrics
MetricsConfigLoader (was private[config], now public)	org.llm4s.config

What stays in llm4s-core is the metrics contract: MetricsCollector (with noop and compose), Outcome and ErrorKind. Every provider client, ReliableClient and the metrics and rate-limiting middleware take a MetricsCollector, so none of them needs the new module; only the code that builds a PrometheusMetrics does. It is kept apart from llm4s-observability so that Prometheus does not reach every llm4s-rag user through that module.

Configuration is unchanged

The keys and their defaults are the same - llm4s.metrics.enabled (false), llm4s.metrics.prometheus.enabled (true) and llm4s.metrics.prometheus.port (9090). They used to be hard-coded in MetricsConfigLoader, because core’s reference.conf had no llm4s.metrics block; that block now ships in the module’s reference.conf, beside the code that reads it.

Source break: Llm4sConfig.metrics() is removed

It returned a PrometheusEndpoint, so it could not stay in a core without Prometheus (D3). MetricsConfigLoader replaces it, with the same result type:

import org.llm4s.config.MetricsConfigLoader

// before
val (metrics, endpoint) = Llm4sConfig.metrics().toOption.get

// after
val (metrics, endpoint) = MetricsConfigLoader.default().toOption.get

MetricsConfigLoader.load(source) reads from a ConfigSource of your own. Its source parameter no longer defaults to ConfigSource.default; call default() for that.

A failed listing fails the sync

#1231; in 0.5.0.

A document loader that cannot enumerate its documents - an S3 listing with no credentials, a missing bucket or access denied, a DirectoryLoader path that does not exist - now yields the new LoadResult.ListingFailure(source, error) instead of a LoadResult.Failure. RAG.sync, ingest and refresh, and their async forms, return its error as the Left. Previously sync returned Right with 0 documents and then deleted every document it had indexed from that source, because it had seen none of them.

• Code that matches on LoadResult needs a ListingFailure case; an exhaustive match without one warns, and fails the build under -Werror.
• RAG.ingest no longer counts a listing error as one failed document in LoadStats: it returns it as the Left, whether or not failFast is set. Code that read stats.errors for "list-error" should handle the Left instead.
• SourceBackedLoader names the source, not "list-error": the ListingFailure’s source is the DocumentSource’s description, e.g. S3(s3://bucket/prefix).
• A sync that fails part-way through a listing (a later S3 page) keeps the documents it synced before the failure and deletes nothing; re-running it once the source is reachable completes the sync. refresh cleared the index before reading the loader, so a listing failure left it empty - but said so; it now leaves the index untouched (see above).

A DocumentSource reports a listing error by yielding a Left from listDocuments(), as before; SourceBackedLoader does the rest. A custom DocumentLoader whose enumeration fails should return LoadResult.listingFailure(source, error).

Vendor credentials: a shared API key per provider

#1132, part of the modularisation programme (#1126); in 0.5.0. 0.4.1 and earlier behave as before.

Credentials belong to a vendor, keyed by provider id; clients belong to a use. Each provider module now binds its vendor’s conventional API-key variable to a shared key, llm4s.credentials.<provider>.apiKey, in its own reference.conf. A client - a chat section, an embeddings block, the Cohere reranker - that sets no apiKey of its own uses it. So with OPENAI_API_KEY set, a chat section needs only provider and model, and OpenAI embeddings need no extra line.

This is additive: configs that set apiKey keep working unchanged, because a client’s own key always wins. Configs that failed for want of an apiKey line now load.

Resolution order

For a client of provider P:

1. its own apiKey: llm4s.providers.<name>.apiKey, llm4s.embeddings.<P>.apiKey, or llm4s.rerank.cohere.apiKey;
2. otherwise llm4s.credentials.<P>.apiKey, where P is the canonical id - a provider = "google" section uses llm4s.credentials.gemini;
3. otherwise a ConfigurationError naming both places: apiKey: set OPENAI_API_KEY, or set apiKey under llm4s.providers.openai-main in application.conf.

The credentials block holds apiKey only: baseUrl, endpoint, model and apiVersion are never defaulted from it. Where each key came from is logged at INFO - for example llm4s.providers.openai-main: API key from llm4s.credentials.openai.apiKey - and the value never is.

# before
openai-main {
  provider = "openai"
  model    = "gpt-4o-mini"
  apiKey   = ${?OPENAI_API_KEY}
}
llm4s.embeddings.openai.apiKey = ${?OPENAI_API_KEY}

# after - with OPENAI_API_KEY exported
openai-main {
  provider = "openai"
  model    = "gpt-4o-mini"
}

The old lines still work and may stay. A section for a second account keeps its own key: apiKey = ${?OPENAI_BATCH_API_KEY}. If that variable is unset, the section now falls back to the shared key instead of failing - see Explicit keys in production.

Variables bound

Provider id	Variable	Module
openai	OPENAI_API_KEY	llm4s-openai
azure	AZURE_OPENAI_API_KEY	llm4s-openai
requesty	REQUESTY_API_KEY	llm4s-openai
anthropic	ANTHROPIC_API_KEY	llm4s-anthropic
gemini	GOOGLE_API_KEY, else GEMINI_API_KEY (Google’s SDK precedence)	llm4s-gemini
deepseek, zai, openrouter, mistral	DEEPSEEK_API_KEY, ZAI_API_KEY, OPENROUTER_API_KEY, MISTRAL_API_KEY	llm4s-openai-compatible
cohere	COHERE_API_KEY	llm4s-openai-compatible and llm4s-rag
voyage	VOYAGE_API_KEY	llm4s-voyage

None for openai-compatible (it has no vendor), ollama (no key) or vertexai (OAuth2: Application Default Credentials, or a service-account file named by apiKey).

Moved bindings

• Voyage: llm4s.embeddings.voyage.apiKey = ${?VOYAGE_API_KEY} became llm4s.credentials.voyage.apiKey = ${?VOYAGE_API_KEY}. VOYAGE_API_KEY works as before, and an explicit llm4s.embeddings.voyage.apiKey still wins.
• Cohere reranker: llm4s.rerank.cohere.apiKey = ${?COHERE_API_KEY} became llm4s.credentials.cohere.apiKey = ${?COHERE_API_KEY}, shared with the Cohere chat provider, so one COHERE_API_KEY serves both. An explicit llm4s.rerank.cohere.apiKey still wins.
• Azure: if you exported AZURE_API_KEY for a section without its own apiKey, rename it to AZURE_OPENAI_API_KEY - the openai SDK’s name, and the one now bound - or keep an apiKey = ${?AZURE_API_KEY} line in the section.

New: RerankerConfigLoader

llm4s-rag’s reference.conf has bound llm4s.rerank since #337, but no code read it. org.llm4s.config.RerankerConfigLoader.load(source) / .default() now does, returning Result[Option[RerankProviderConfig]] for RAG.build(..., resolveRerankerConfig = ...) or RerankerFactory.fromConfig.

Explicit keys in production

llm4s-config-policy’s prod preset (ConfigPolicy.prodSafeDefaults) gains the rule ownApiKey (ConfigPolicy.withOwnApiKeyRequired, checked by ConfigPolicyEngine.checkApiKeySources and run by CheckPolicies): every chat section whose provider requires a key must set its own apiKey, so a section meant for a second account cannot silently bill the default one. A prod check of a config that relies on the shared key now fails; add apiKey = ${?VAR} to each section. Llm4sConfig.apiKeySources() / apiKeySourcesFrom(source) report each section’s ApiKeySource (Section(path) or Credentials(path)) for checks of your own.

Source breaks

These are deliberate removals ahead of the MiMa baseline:

1. EmbeddingConfigSpec.apiKeyPath is removed, with the loader code that resolved it. The shared llm4s.credentials.<id>.apiKey replaces it for every provider: bind your vendor’s variable there in your module’s reference.conf instead of declaring a path.
2. EmbeddingConfigSpec.apiKeyEnv is a Seq[String], not an Option[String]: apiKeyEnv = Some("X") becomes apiKeyEnv = Seq("X"). It lists the variables your module binds to llm4s.credentials.<id>.apiKey, highest precedence first, and is used only to word the missing-key error.
3. ProviderConfigSpec gained apiKeyEnv: Seq[String] (last, with a default), and ProviderConfigSpec.apiKeyAndDefaultBaseUrl an optional second parameter for it. Positional construction still compiles; a pattern match on ProviderConfigSpec needs one more binder.
4. OpenAIConfigKeys.AZURE_API_KEY is now AZURE_OPENAI_API_KEY, with the value "AZURE_OPENAI_API_KEY".
5. The missing-apiKey messages changed. Chat: apiKey: set <VAR>, or set apiKey under llm4s.providers.<name> in application.conf. Embeddings: Missing <id> embeddings apiKey: set <VAR>, or set apiKey under llm4s.embeddings.<id> in application.conf. Code matching the old text needs updating.

For provider authors: bind llm4s.credentials.<id>.apiKey = ${?<VENDOR>_API_KEY} in your module’s reference.conf, using the name the vendor’s own SDK or docs use, and declare the same variable as apiKeyEnv on your ProviderConfigSpec or EmbeddingConfigSpec. Your module’s round-trip spec should prove the two agree.

Slice 6: llm4s-observability - Langfuse, the trace collector and CostTracker leave core

The slice 6 carve (#1133, #1126) moves the tracing integrations that need nothing beyond core into a new module, llm4s-observability. Package names are unchanged, so no import changes; code that uses any of the types below adds one dependency:

libraryDependencies += "org.llm4s" %% "llm4s-observability" % "<version>"

Moved to llm4s-observability	Package
LangfuseTracing, LangfuseBatchSender, DefaultLangfuseBatchSender, LangfuseHttpApiCaller, LangfuseTracingBackend	org.llm4s.trace
TraceCollectorTracing and the rest of TraceCollector.scala	org.llm4s.trace
Trace, Span, SpanId, SpanKind, SpanStatus, SpanEvent, SpanValue, TraceModelJson	org.llm4s.trace.model
TraceStore, InMemoryTraceStore, TraceQuery	org.llm4s.trace.store
CostTracker	org.llm4s.metrics
LangfuseConfig (was in core)	org.llm4s.llmconnect.config
LangfuseConfigKeys, LangfuseConfigLoader (new)	org.llm4s.config

OpenTelemetryConfig moves, in the same package, to llm4s-observability-otel, beside the backend that reads it.

What stays in llm4s-core is the tracing contract (decisions D1 to D5 in #1133): Tracing, TraceEvent, TracingComposer, TracingMode, the TracingBackend SPI, NoOpTracing, ConsoleTracing (the default mode, so a core-only application still traces), TracingSettings, and MetricsCollector. llmconnect, the agent runtime and every provider module use only these, so none of them needs the new module. llm4s-rag depends on it, for RAGASLangfuseObserver; it carries no third-party dependency, so this adds nothing else to a RAG user’s classpath.

Configuration is unchanged

The keys and variables are the same: TRACING_MODE=langfuse, LANGFUSE_PUBLIC_KEY, LANGFUSE_SECRET_KEY, LANGFUSE_URL, LANGFUSE_ENV, LANGFUSE_RELEASE, LANGFUSE_VERSION under llm4s.tracing.langfuse.*, and TRACING_MODE=opentelemetry (or otel), OTEL_SERVICE_NAME, OTEL_EXPORTER_OTLP_ENDPOINT and llm4s.tracing.opentelemetry.headers. Only the module that binds them changed: the llm4s.tracing.langfuse block moved from core’s reference.conf to llm4s-observability’s, and llm4s.tracing.opentelemetry to llm4s-observability-otel’s, each with the code that reads it. Core now reads only llm4s.tracing.mode, and hands the selected mode’s block to its backend as TracingSettings.extras.

So an application that sets TRACING_MODE=langfuse adds llm4s-observability and changes no config. Without it, Tracing.create logs an error naming the artifact and traces nothing - exactly what TRACING_MODE=opentelemetry without llm4s-observability-otel already did:

Tracing mode 'langfuse' is configured but no TracingBackend for it is on the classpath.
Add the 'org.llm4s' %% 'llm4s-observability' dependency. Available modes: console, noop.

Behaviour change: Langfuse without keys does not start

TRACING_MODE=langfuse without LANGFUSE_PUBLIC_KEY or LANGFUSE_SECRET_KEY used to build a LangfuseTracing that logged a warning and dropped every batch. The backend now refuses to start: Tracing.fromSettings returns

Langfuse tracing is selected but llm4s.tracing.langfuse.publicKey (LANGFUSE_PUBLIC_KEY) and
llm4s.tracing.langfuse.secretKey (LANGFUSE_SECRET_KEY) are not set.

and Tracing.create logs that once and returns NoOpTracing. Either way nothing reaches Langfuse, as before; the difference is one clear error instead of a warning per batch. LangfuseTracing.from(config) built directly is unchanged.

Behaviour change: a selected block that is not an object is an error

TracingSettings.extras is the selected mode’s block, llm4s.tracing.<mode>. When that key was present but was not an object - llm4s.tracing { mode = opentelemetry, opentelemetry = "http://collector:4317" } - it was treated as absent, so the backend started on its defaults (here, a collector on localhost) and the operator’s setting was silently ignored. Read failures were swallowed the same way. Llm4sConfig.tracing() now returns a ConfigurationError naming the path instead:

llm4s.tracing.opentelemetry must be an object, but is a string. It holds the settings for
tracing mode 'opentelemetry': write it as llm4s.tracing.opentelemetry { ... }.

A block that is absent (or null) is still an empty extras, and the backend applies its defaults. Only the selected mode’s block is checked: a malformed block for a mode that is not selected is still ignored.

Reading Langfuse settings yourself

TracingSettings no longer has a langfuse field, and extras holds only the selected mode’s block. To read llm4s.tracing.langfuse whatever TRACING_MODE is - to combine Langfuse with Console, or for RAGASLangfuseObserver - use the new loader:

import org.llm4s.config.LangfuseConfigLoader

// before
Llm4sConfig.tracing().map(settings => LangfuseTracing.from(settings.langfuse))
RAGASLangfuseObserver.fromTracingSettings(settings)

// after
LangfuseConfigLoader.default().map(LangfuseTracing.from)
LangfuseConfigLoader.default().map(RAGASLangfuseObserver.from)

Source breaks

These are taken before the MiMa baseline is set, and none of them changes a package name.

1. Langfuse, the collector, its model and store, and CostTracker need llm4s-observability (table above).
2. TracingMode.Langfuse and TracingMode.OpenTelemetry are removed. Core keeps a case only for what it builds itself, Console and NoOp; the others are TracingMode.Named("langfuse") and TracingMode.Named("opentelemetry"), which TracingMode.fromString returns for "langfuse" and for "opentelemetry" / "otel". LangfuseConfig.Mode and OpenTelemetryConfig.Mode name them. A match on the old case objects matches on TracingMode.Named("langfuse") instead.
3. TracingSettings is TracingSettings(mode, extras). The langfuse: LangfuseConfig and openTelemetry: OpenTelemetryConfig fields are removed. A backend reads its block from extras - LangfuseConfig.fromExtras(settings.extras), OpenTelemetryConfig.fromExtras(settings.extras) - and code that built settings by hand passes extras = Map("publicKey" -> ..., "secretKey" -> ...) for Langfuse, or Map("serviceName" -> ..., "endpoint" -> ..., "headers.Authorization" -> ...) for OpenTelemetry.
4. LangfuseConfig moves to llm4s-observability, and OpenTelemetryConfig to llm4s-observability-otel, in the same package.
5. DefaultConfig is removed. Its last four constants moved to LangfuseConfig: DEFAULT_LANGFUSE_URL, _ENV, _RELEASE and _VERSION are LangfuseConfig.DEFAULT_URL, DEFAULT_ENV, DEFAULT_RELEASE and DEFAULT_VERSION.
6. ConfigKeys.LANGFUSE_* are LangfuseConfigKeys.LANGFUSE_* in llm4s-observability.
7. RAGASLangfuseObserver.fromTracingSettings(TracingSettings) is removed: it read the removed TracingSettings.langfuse. Use RAGASLangfuseObserver.from(config) with LangfuseConfigLoader.default(), as above.
8. TraceEvent.createTraceEvent and org.llm4s.llmconnect.model.TraceHelper are removed. Both built Langfuse ingestion JSON - a "trace-create" batch envelope, and event-create / generation-create / span-create envelopes per conversation message - and nothing in llm4s called either; LangfuseTracing builds its own batches. They were Langfuse’s wire format sitting in the core contract, so they are deleted rather than moved. Code that called them builds the JSON itself, or traces through LangfuseTracing (llm4s-observability), which sends the same event types.

Slice 6: tracing backends are discovered, and agent state is a TraceEvent

The first slice 6 change (#1133) lands the extension point for tracing before Langfuse and Prometheus are carved out of llm4s-core, as slice 4 did for providers. Nothing moves module yet, and nothing in a release has it; 0.4.1 and earlier behave as before.

Tracing.traceAgentState(AgentState) is removed

Tracing no longer mentions AgentState, so the tracing contract does not depend on the agent runtime. Agent state is traced as an ordinary event, TraceEvent.AgentStateUpdated, which AgentState#toTraceEvent builds:

// before
tracing.traceAgentState(state)

// after
tracing.traceEvent(state.toTraceEvent)

AgentStateUpdated gained a messages: Seq[Message] field (default empty) before timestamp, carrying the conversation, so that Langfuse still records a trace with one span per message. It is not part of toJson. The agent emits the event exactly where it called traceAgentState.

A custom Tracing implementation drops its traceAgentState override; anything it did there belongs in traceEvent’s AgentStateUpdated case.

Tracing.create finds backends on the classpath

Tracing.create(settings) keeps its signature. It builds NoOp and Console itself and hands every other mode to an org.llm4s.trace.spi.TracingBackend registered in META-INF/services/org.llm4s.trace.spi.TracingBackend:

• OpenTelemetry: llm4s-observability-otel registers OpenTelemetryTracingBackend. The Class.forName reflection core used to find it is gone. Adding the dependency is all it takes, as before.
• Langfuse: registered by core’s own services entry until it is carved into llm4s-observability - which has since happened; see the carve’s note.
• Anything else: implement TracingBackend (a class with a public no-arg constructor, not an object) with mode = TracingMode.Named("yourmode"), declare it in the services file, and TRACING_MODE=yourmode selects it.

Tracing.create still never fails: a missing or broken backend logs an error and gives NoOpTracing. The new Tracing.fromSettings(settings): Result[Tracing] returns that as a ConfigurationError (or the backend’s own error) instead, and Tracing.fromSettings(settings, TracingBackends.of(...)) registers a backend explicitly, for a shaded jar whose services files did not survive.

TracingMode is open

TracingMode gained a Named(name) case and a name member. TracingMode.fromString returns Named for a value it does not recognise, lower-cased, where it used to return NoOp with a warning; a blank value is still NoOp. The end result of an unknown TRACING_MODE is unchanged - Tracing.create gives NoOpTracing - but the log line is now an error that lists the modes that are available.

Behaviour changes

1. An AgentStateUpdated with no messages is exported to Langfuse as a summary trace. The old traceAgentState sent nothing for an empty conversation.
2. Langfuse reports a failed export of the conversation trace as a Left; traceAgentState always returned Right(()). The agent swallows tracing errors either way.

3. The agent state span takes the event’s name. Agent runs used to reach a tracer through traceAgentState, whose span name differed from the one traceEvent gave the same event; they now go through traceEvent, so the attributes are unchanged but the name is not. Update any dashboard or query keyed on the old name:

   Backend	Old name (agent runs)	New name
   OpenTelemetry	Agent State Snapshot	Agent State Updated
   TraceCollectorTracing	agent-state-update	agent_state_updated

4. An OpenTelemetry SDK that fails to start is reported by Tracing.fromSettings and gives NoOpTracing from Tracing.create, rather than a tracer whose every call failed.

Source breaks

1. Tracing.traceAgentState is removed - use traceEvent(state.toTraceEvent).
2. TraceEvent.AgentStateUpdated has a fifth field, messages, before timestamp. A positional AgentStateUpdated(status, messages, logs, timestamp) must name the timestamp (timestamp = ...), and a pattern AgentStateUpdated(a, b, c, ts) needs one more binder.
3. TracingMode has a new case, Named, so an exhaustive match over it needs one more branch, and a new abstract member, name, for anything extending the sealed trait (nothing outside core can).
4. TracingMode.fromString returns Named(...), not NoOp, for an unrecognised value.

From LLM_MODEL to named provider sections

API keys: superseded by Vendor credentials. Provider modules now bind each vendor’s API-key variable (OPENAI_API_KEY, …), so the apiKey = ${?OPENAI_API_KEY} lines below, and the embeddings binding, are optional. What this note says about LLM_MODEL and base-URL variables still holds.

Since #903 (in 0.3.2) removed legacy single-provider loading, nothing in llm4s reads LLM_MODEL, nor a provider’s API-key or base-URL variable (OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_API_KEY, AZURE_API_BASE, OLLAMA_BASE_URL, …). No reference.conf binds them. Setting them has no effect, and an application that relied on them fails at startup with a configuration error. The documentation kept teaching them until #1132’s follow-up.

Configure providers as named sections in your own application.conf, binding secrets from the environment with ${?VAR} - you choose the variable names:

# src/main/resources/application.conf
llm4s {
  providers {
    provider = "openai-main"          # the default: the name of a section below

    openai-main {
      provider = "openai"
      model    = "gpt-4o-mini"
      apiKey   = ${?OPENAI_API_KEY}
    }
  }
}

Before	After
LLM_MODEL=openai/gpt-4o-mini	a section with provider = "openai", model = "gpt-4o-mini", selected by llm4s.providers.provider
OPENAI_API_KEY=... read automatically	apiKey = ${?OPENAI_API_KEY} in that section, and the same variable exported
OPENAI_BASE_URL, OLLAMA_BASE_URL, …	baseUrl = "..." (or baseUrl = ${?OLLAMA_BASE_URL}) in the section; an Ollama section defaults to http://localhost:11434
switching model by changing LLM_MODEL	a second section, selected by changing provider, by -Dllm4s.providers.provider=<name>, by your own provider = ${?LLM4S_PROVIDER} binding, or loaded directly with Llm4sConfig.provider("<name>")
Llm4sConfig.provider()	Llm4sConfig.defaultProvider()

Two things to know:

• Only the section you load is validated (since #1132’s follow-up; in 0.5.0). A section whose required apiKey is unset, or whose provider module is not on the classpath, fails provider("<that section>") - and defaultProvider() when it is the default - but not a load of any other section. Up to 0.4.1 every section was validated on every load, so each environment had to fill in every section or ship a config without it. Llm4sConfig.providers(), which returns every section, still validates them all.
• OpenAI embeddings do not see OPENAI_API_KEY either. With EMBEDDING_MODEL=openai/<model> (which is bound, by llm4s-core’s reference.conf), add llm4s.embeddings.openai.apiKey = ${?OPENAI_API_KEY}. Since #1132’s follow-up the key no longer falls back to llm4s.openai.apiKey, the pre-#903 chat key that nothing else read: if you set that, set llm4s.embeddings.openai.apiKey instead.

Variables that reference.conf files do bind - TRACING_MODE, LANGFUSE_*, OTEL_SERVICE_NAME, OTEL_EXPORTER_OTLP_ENDPOINT, EMBEDDING_MODEL, VOYAGE_API_KEY and others - still work; see the variables llm4s reads. Two tools read LLM_MODEL themselves: the chat-tui sample (ChatTuiConfig) and the config-policy env check (EnvCheckPolicies).

Named providers: provider-specific keys; Vertex AI project and location

A named provider section can now carry keys of the provider’s own, declared by the provider rather than squeezed into the shared fields (#1215). In 0.5.0.

Vertex AI: endpoint becomes project, organization becomes location

Vertex AI used to read its GCP project from endpoint and its region from organization. It now has keys named for what they are:

# before
vertex-main {
  provider     = "vertexai"
  model        = "gemini-2.0-flash"
  endpoint     = "my-gcp-project"
  organization = "europe-west4"
}

# after
vertex-main {
  provider = "vertexai"
  model    = "gemini-2.0-flash"
  project  = "my-gcp-project"
  location = "europe-west4"      # optional, default us-central1
}

The old spellings still work for now, as deprecated aliases: each one logs a warning naming the section and the key to rename it to, and will stop working in a later release. Setting both project and a different endpoint (or location and a different organization) is an error rather than a guess. A missing project is now reported as - project: the GCP project ID that owns your Vertex AI resources (set it in application.conf under llm4s.providers.<name>.project).

Unknown keys are reported

A key in a section that is neither a built-in field nor one the provider declares used to be ignored silently. It is still ignored - existing configs keep loading - but now with a warning that names it and lists the keys the provider accepts, which is how a typo like regoin shows up. An extra key whose value is an object or list is an error, since none takes one.

The missing-baseUrl hint no longer suggests a variable nothing reads

A section missing a required baseUrl used to say set <PROVIDER>_BASE_URL, for example set OLLAMA_BASE_URL, though nothing read such a variable for a named section. It now says where to set the key - baseUrl: set it in application.conf under llm4s.providers.<name>.baseUrl (e.g. ...). Where the provider has a conventional variable (ProviderConfigSpec.baseUrlEnv), the message shows the binding that makes it read, since a named section reads none by itself: openai-compatible adds to read it from OPENAI_COMPATIBLE_BASE_URL, add baseUrl = ${?OPENAI_COMPATIBLE_BASE_URL} to the section. Code matching on the old message text needs updating.

Source breaks

• RawNamedProviderSection and NamedProviderConfig gained a trailing extras parameter with a default, so construction by name or position still compiles; pattern matches that list every field (case NamedProviderConfig(a, b, ...)) need one more.
• ProviderConfigSpec gained baseUrlEnv and extras, both defaulted.
• VertexAIProvider.configSpec no longer sets requiresEndpoint. Code that builds a NamedProviderConfig by hand and passes it straight to VertexAIProvider.buildConfig, skipping validation, must set extras = Map("project" -> ..., "location" -> ...): the deprecated aliases are resolved by validation, not by buildConfig.

For provider authors

Declare provider-specific keys in ProviderConfigSpec.extras and read them from NamedProviderConfig.extras - see CONTRIBUTING.

Slice 5 follow-ups: llm4s-openai-compatible

Fixes to the shared OpenAI-compatible client after its carve (#1132). Nothing here needs a code change unless you depend on the old behaviour.

complete times out after two minutes

OpenAICompatibleClient.complete - DeepSeek, Z.ai, OpenRouter, Mistral, Cohere and the generic openai-compatible provider - sent its request with no timeout, so an endpoint that never answered hung the caller. It now fails after two minutes (OpenAICompatibleClient.RequestTimeout), the value the old Mistral and Cohere clients used; streaming keeps five minutes. Neither is configurable yet (#712): a slow local model with a long prompt should stream.

Streaming requests send stream_options

A streaming request from the generic provider and DeepSeek now carries "stream_options": {"include_usage": true}, so servers that follow OpenAI - vLLM, Ollama’s /v1, Perplexity’s Router - report token usage on streams. Z.ai, OpenRouter, Mistral and Cohere do not send it. An endpoint that rejects unknown fields fails the stream with a 400 or 422; build its config with OpenAICompatibleConfig.fromValues(..., streamUsage = false). A dialect you wrote yourself inherits streamUsageOption = true; override it to false if the provider rejects the field.

Requesty configs report requesty

A config loaded from a provider = "requesty" section reported providerId = openai, because OpenAIConfig inferred its id from the base URL. OpenAIConfig now carries the id its descriptor sets, in a new trailing field explicitProviderId: Option[ProviderId] = None (OpenAIConfig.fromValues has a matching defaulted providerId parameter), so it reports requesty; provider = "openrouter" sets openrouter the same way, so an OpenRouter section with a proxy baseUrl still reaches OpenRouter. With None - a config built by hand, or provider = "openai" - the id is inferred from the base URL as before.

What you may notice:

• llm4s-config-policy sees requesty: an allowedProviders list must name requesty rather than rely on openai, and model patterns match requesty/<model>.
• LLMConnect.getClient(config) on a Requesty config now dispatches to the requesty descriptor, which needs llm4s-openai - as the openai one did.
• OpenAIConfig.toString shows providerId; a pattern match destructuring all six fields must add a seventh.

OPENAI_COMPATIBLE_BASE_URL and OPENAI_COMPATIBLE_API_KEY

These are now the conventional variables for a generic endpoint, named by OpenAICompatibleConfigKeys. Llm4sConfig does not read them - the generic provider has no vendor and so no shared credentials key, and nothing reads LLM_MODEL - so bind them in a section (baseUrl = ${?OPENAI_COMPATIBLE_BASE_URL}). The chat-tui sample accepts LLM_MODEL=openai-compatible/<model> with them, and the config-policy env check reads OPENAI_COMPATIBLE_BASE_URL as that provider’s endpoint instead of OPENAI_BASE_URL.

Slice 5: Mistral, Cohere and Voyage leave core; core ships no provider

The last provider clients leave llm4s-core (#1132). This is in 0.5.0; 0.4.1 and earlier ship Mistral, Cohere and Voyage inside llm4s-core.

Provider	Now in	Add
Mistral (provider = "mistral")	llm4s-openai-compatible, as a dialect	"org.llm4s" %% "llm4s-openai-compatible"
Cohere (provider = "cohere")	llm4s-openai-compatible, as a dialect	"org.llm4s" %% "llm4s-openai-compatible"
Voyage AI embeddings (EMBEDDING_MODEL=voyage/...)	llm4s-voyage (modules/providers/voyage)	"org.llm4s" %% "llm4s-voyage"

Package names are unchanged, and so are the names and constructors of MistralClient, MistralProvider, MistralConfig, CohereClient, CohereProvider, CohereConfig and VoyageAIEmbeddingProvider, so imports compile as before once the dependency is added. No environment variable or config key changed its name; llm4s.embeddings.voyage moved to llm4s-voyage’s reference.conf, so it exists exactly when the module is on the classpath.

Mistral and Cohere stream now

Both are OpenAI-compatible, so they are small dialects on the shared OpenAICompatibleClient rather than clients of their own. Both gain streaming (streamComplete returned “not supported” - #925), streamed tool calls and token usage, tool calling and structured output; their descriptors no longer declare streaming = false.

• Mistral posts to <baseUrl>/v1/chat/completions as before (baseUrl is the API root, https://api.mistral.ai); a baseUrl already ending in /v1 is no longer doubled. Tool-call ids are sent in the nine-character form Mistral insists on; a reasoning model’s thinking is returned as Completion.thinking.
• Cohere now calls Cohere’s OpenAI-compatibility API instead of the native /v2/chat. CohereConfig.DEFAULT_BASE_URL is now https://api.cohere.ai/compatibility/v1 (it was https://api.cohere.com). A configured baseUrl keeps working: one that does not end in /compatibility/v1 is taken to be a Cohere API root, as it was, and gets /compatibility/v1 appended (after a trailing /v1 or /v2 is dropped); CohereConfig.fromValues stores and logs the mapped URL, so endpointUrl and policy checks see where requests really go. A proxy that forwarded only /v2/chat must now forward /compatibility/v1/chat/completions. System messages go under the developer role and a JSON schema as {"type": "json_object", "schema": ...}, as Cohere documents.

Behaviour changes for both, now that they share the client DeepSeek, Z.ai and OpenRouter use:

1. Reply text is not trimmed (the old clients trimmed it).
2. A reply with no text is an empty completion, not a ValidationError - with tool calling, a reply may carry only tool calls.
3. A missing id or created is left "" / 0 rather than a random UUID / the current time (Mistral), and Cohere’s created now comes from the reply.
4. A ToolMessage is sent rather than refused (Mistral) or silently dropped (Cohere).
5. CompletionOptions.reasoning is still not sent to either: Mistral’s reasoning_effort and Cohere’s accept only some models or values.

The shared client itself changed for every provider on it: streamed completions now report token usage and a cost estimate (they always came back with usage = None), and an empty conversation fails with a ValidationError before any request is sent.

Core ships no provider

With those three gone, llm4s-core holds no provider client, and the list that existed only because it did is removed:

• BuiltinProviders and BuiltinProviderModule (org.llm4s.llmconnect.provider) are deleted, with core’s META-INF/services/org.llm4s.llmconnect.spi.Llm4sProviderModule entry.

• ProviderRegistry.builtin is deleted. It would now be empty. Where discovery cannot run - a fat jar whose services files were overwritten - name the provider modules you ship:

  given ProviderRegistry =
    ProviderRegistry.ofModules(new Llm4sOpenAIModule, new Llm4sOpenAICompatibleModule, new Llm4sVoyageModule)
  

ProviderRegistry.default (discovery) is unchanged, and so is every call site that relies on it. Core’s llmconnect/provider package now holds only provider-neutral helpers: CostEstimator, EmbeddingProvider, HttpErrorMapper, MetricsRecording, ProviderExchangeRecorder and ProviderResultOps.

Source breaks

1. BuiltinProviders, BuiltinProviderModule and ProviderRegistry.builtin are removed - see above.
2. ProviderModelListers.Mistral is now MistralModelLister (org.llm4s.config, in llm4s-openai-compatible); MistralProvider.modelLister returns it.
3. ConfigKeys.MISTRAL_API_KEY and MISTRAL_BASE_URL are now on OpenAICompatibleConfigKeys, and ConfigKeys.VOYAGE_API_KEY, VOYAGE_EMBEDDING_BASE_URL and VOYAGE_EMBEDDING_MODEL on VoyageConfigKeys (llm4s-voyage), both in org.llm4s.config. The strings are unchanged.
4. CohereConfig.DEFAULT_BASE_URL changed value - see above.
5. OpenAICompatibleDialect gained four members - sendEmptyAssistantTurns, encodeToolCallId, systemRole and encodeResponseFormat - each defaulting to the standard format, so an existing dialect compiles and behaves as before.

Slice 5: llm4s-openai-compatible

The fifth provider module of slice 5 (#1132), carrying DeepSeek (provider = "deepseek"), Z.ai ("zai"), OpenRouter ("openrouter") and a new generic provider, "openai-compatible", for any other endpoint that speaks the OpenAI /chat/completions API. It is in the build but not yet in a release; 0.4.1 and earlier still ship DeepSeek, Z.ai and OpenRouter inside llm4s-core, and have no generic provider.

Unlike the earlier carves this is a consolidation, not a pure move. DeepSeek, Z.ai and OpenRouter each had their own ~400-line copy of the same SDK-free client; they are now thin subclasses of one OpenAICompatibleClient, each with a small OpenAICompatibleDialect for what genuinely differs (headers, content encoding, reasoning parameters, thinking extraction, tool-call parsing). The module depends on nothing but llm4s-core.

libraryDependencies += "org.llm4s" %% "llm4s-openai-compatible" % version

What moved

Code	Now in
DeepSeekClient, DeepSeekProvider, ZaiClient, ZaiProvider, OpenRouterClient, OpenRouterProvider (org.llm4s.llmconnect.provider)	llm4s-openai-compatible
DeepSeekConfig, ZaiConfig, OpenAIConfig (org.llm4s.llmconnect.config)	llm4s-openai-compatible
ProviderModelListers.DeepSeek / .OpenRouter → DeepSeekModelLister / OpenRouterModelLister (org.llm4s.config)	llm4s-openai-compatible
DefaultConfig.DEFAULT_DEEPSEEK_BASE_URL → DeepSeekConfig.DEFAULT_BASE_URL	llm4s-openai-compatible
DefaultConfig.DEFAULT_OPENROUTER_BASE_URL → OpenRouterProvider.DEFAULT_BASE_URL	llm4s-openai-compatible
ConfigKeys.DEEPSEEK_API_KEY, DEEPSEEK_BASE_URL, OPENROUTER_BASE_URL → OpenAICompatibleConfigKeys (org.llm4s.config)	llm4s-openai-compatible
the commented deepseek-main, zai-main and openrouter-main examples in reference.conf	llm4s-openai-compatible’s reference.conf

New: OpenAICompatibleClient, OpenAICompatibleDialect, OpenAICompatibleConfig, OpenAICompatibleProvider, OpenAICompatibleModelLister, Llm4sOpenAICompatibleModule.

Package names are unchanged, and so are the public shapes of the three clients - their constructors and companion apply overloads - their descriptors and their configs, so new DeepSeekClient(config) or OpenRouterClient(config, metrics) compile as before once the dependency is added.

llm4s-openai users: OpenAIConfig moved here, and llm4s-openai now depends on llm4s-openai-compatible to get it. That module brings no SDK, and nothing changes in your build.

The generic openai-compatible provider

llm4s.providers {
  local-vllm {
    provider = "openai-compatible"
    baseUrl = "http://localhost:8000/v1"   # required
    model = "Qwen/Qwen2.5-7B-Instruct"     # required
    # apiKey = ...                        # optional; no Authorization header without one
    # contextWindow = 32768               # optional; default 8192
    # reserveCompletion = 4096            # optional; default 2048
    # headers { X-Team = "search" }       # optional
  }
}

To support it, a named provider section may now carry contextWindow, reserveCompletion and a headers object, which NamedProviderConfig exposes. Providers other than openai-compatible ignore them. See OpenAI-compatible endpoints.

Registration is the dependency

llm4s-openai-compatible declares Llm4sOpenAICompatibleModule in its META-INF/services, so ProviderRegistry.default finds it. Without the dependency, provider = "deepseek", "zai" and "openrouter" fail with the registry’s error, which names the providers that are registered. ProviderRegistry.builtin no longer includes them; where discovery cannot run:

given ProviderRegistry = ProviderRegistry.ofModules(new Llm4sOpenAICompatibleModule) // `builtin` was removed later in slice 5

Behaviour changes

The three copies had drifted apart; the shared client does each thing one way:

1. DeepSeek returns thinking. deepseek-reasoner’s reasoning_content is now Completion.thinking and is streamed as thinking deltas; the old client dropped it. Its completion_tokens_details.reasoning_tokens is TokenUsage.thinkingTokens.
2. The stream body is closed on every failure. Z.ai and OpenRouter left it open on an error status.
3. Every call records exactly one provider exchange, including a request that cannot be sent (DeepSeek and Z.ai recorded none for a non-streaming one; OpenRouter recorded failures twice).
4. OpenRouter sends assistant content as a string. The old client passed an Option through ujson’s implicit conversion, so "hi" went out as ["hi"] and no content as []; it is now "hi", "" or null.
5. Z.ai reads usage given as an array, which its client meant to support but never matched.
6. Streamed tool calls keep all their arguments. A tool call streamed across several deltas lost every fragment after the first in all three clients: continuations carry only an index, the missing id was defaulted to "", and StreamingAccumulator skips a chunk with no id. The shared client now maps each index to its call’s id for the life of the stream, and a streamed Completion reports its tool calls in toolCalls, as a non-streaming one does.
7. A reply’s message.contentOpt is None when the reply has no text for all three (it was Some("") for DeepSeek and Z.ai); Completion.content is "" either way.
8. Replies are read leniently where the copies threw: a missing id, created or model defaults, a streamed event with no choices is skipped, and a malformed non-streaming reply is a Left for all three (Z.ai could throw). OpenRouter keeps its strict tool-call parsing; DeepSeek and Z.ai keep their lenient one.

Source breaks

1. ProviderModelListers.DeepSeek and .OpenRouter are now DeepSeekModelLister and OpenRouterModelLister, in the same package. The descriptors’ modelLister returns them.
2. DefaultConfig.DEFAULT_DEEPSEEK_BASE_URL and DEFAULT_OPENROUTER_BASE_URL are now DeepSeekConfig.DEFAULT_BASE_URL and OpenRouterProvider.DEFAULT_BASE_URL. The values are unchanged.
3. ConfigKeys.DEEPSEEK_API_KEY, DEEPSEEK_BASE_URL and OPENROUTER_BASE_URL are now on OpenAICompatibleConfigKeys, in the same package. The strings are unchanged.
4. ProviderRegistry.builtin no longer includes deepseek, zai or openrouter - see above.
5. NamedProviderConfig and RawNamedProviderSection gained three trailing fields with defaults (contextWindow, reserveCompletion, headers), so construction by name or by position is unaffected; a pattern match that destructures all seven fields must add three. NamedProviderConfig.toString now redacts the API key and header values.

6. ProviderModelListers.openAICompatible gained two defaulted parameters, extraHeaders and apiKeyRequired; existing calls compile unchanged. It no longer special-cases OpenRouter, whose lister passes its headers explicitly.

7. OpenRouterToolCallDeserializer is removed from org.llm4s.llmconnect.serialization. No client used it after the consolidation. The “double-nested” array it parsed was an artefact of the old OpenRouterClient, not OpenRouter’s format; use StandardToolCallDeserializer (which stays), as OpenRouterClient now does.
8. StreamingResponseHandler is removed, with BaseStreamingResponseHandler, OpenAIStreamingHandler, AnthropicStreamingHandler and StreamingResponseHandler.forProvider (org.llm4s.llmconnect.streaming). No client streamed through them - each parses its own stream and accumulates with StreamingAccumulator, which stays - and forProvider was called only from tests. To assemble streamed chunks yourself, feed them to a StreamingAccumulator.

What did not change

Every configuration key and environment variable for DeepSeek, Z.ai and OpenRouter: their provider ids, apiKey, baseUrl and organization, and their default base URLs.

Slice 5: llm4s-openai moves to openai-java

llm4s-openai’s OpenAIClient - behind provider = "openai", "azure" and "requesty" - now runs on OpenAI’s official Java SDK, com.openai:openai-java, instead of Microsoft’s com.azure:azure-ai-openai. Microsoft has deprecated that SDK (its last release, 1.0.0-beta.16, was on 2025-03-26) and points to openai-java, which also covers Azure OpenAI (#1132).

Most users change nothing. OpenAIClient’s constructors and apply overloads, OpenAIProvider, AzureProvider, RequestyProvider, OpenAIConfig and AzureConfig, every configuration key and every environment variable are unchanged. Your dependency tree changes: llm4s-openai now brings OkHttp, Jackson (2.x) and the Kotlin standard library rather than the Azure core libraries. openai-java checks its Jackson version when a client is built and fails on a Jackson it cannot use (a different major version, anything before 2.13.4, or 2.18.1); if your application pins Jackson, keep it on a compatible 2.x.

How Azure is configured

As before: endpoint is the resource endpoint (https://<resource>.openai.azure.com), model is the deployment name, apiKey is sent as the api-key header, and apiVersion as the api-version query parameter, so a request goes to <endpoint>/openai/deployments/<deployment>/chat/completions?api-version=<version>. The client tells the SDK this is Azure rather than letting it guess from the host name, so an endpoint on your own domain (API Management, a private endpoint) keeps working. Two things are new:

• apiVersion takes either form. The wire form (2024-10-21, 2025-01-01-preview), which the docs have always shown, now works; before, only the Azure SDK’s constant names (V2024_10_21, V2025_01_01_PREVIEW - the form of AzureConfig.DEFAULT_API_VERSION) did. Both still work.
• An endpoint ending in /openai/v1 uses Azure’s unified v1 API: the deployment goes in the request body and api-version is sent only if you set one other than the default.

Source break: AzureToolHelper is now OpenAIToolHelper

AzureToolHelper took and returned Azure SDK types, so it could not survive the SDK. It is replaced, in the same package (org.llm4s.toolapi) and module, by OpenAIToolHelper over openai-java’s types (com.openai.models.chat.completions):

Before (AzureToolHelper)	Now (OpenAIToolHelper)
addToolsToOptions(registry, options: ChatCompletionsOptions): ChatCompletionsOptions	addToolsToParams(registry, builder: ChatCompletionCreateParams.Builder): ChatCompletionCreateParams.Builder
convertToolRegistryToAzureTools(registry): java.util.List[ChatCompletionsToolDefinition]	convertToolRegistryToOpenAITools(registry): java.util.List[ChatCompletionTool]

import org.llm4s.toolapi.{ OpenAIToolHelper, ToolRegistry }
import com.openai.models.chat.completions.ChatCompletionCreateParams

val params = OpenAIToolHelper
  .addToolsToParams(new ToolRegistry(tools), ChatCompletionCreateParams.builder().model("gpt-4o"))

If you called the Azure SDK yourself alongside llm4s, add com.azure:azure-ai-openai to your own build: llm4s-openai no longer brings it.

After 0.5.1 OpenAIToolHelper is internal too: see OpenAIToolHelper is internal.

Behaviour changes

1. Streamed tool calls keep their arguments. A streamed tool call arrives split across deltas, and only the first carries the call’s id; the Azure SDK path keyed calls by id, so every later fragment - usually all of the arguments - was lost. Continuations are now matched by index, fragments are concatenated verbatim, and a streamed Completion reports its tool calls in toolCalls as a non-streaming one does.
2. OpenAIConfig.organization is sent as the OpenAI-Organization header. The Azure SDK ignored it.
3. HTTP errors map by status code: 401 and 403 to AuthenticationError, 429 to RateLimitError, 400 to ValidationError, anything else to ServiceError, each naming the provider. Before, they were classified by searching the exception message, and most became UnknownError.
4. Streamed token usage is read from whichever chunk carries it, including a usage-only final chunk, not only from the chunk with the finish reason.
5. close() releases the SDK’s HTTP client (connections and threads); before it released nothing.
6. Azure and Requesty are labelled as themselves. Their errors (AuthenticationError.provider and the error context), metrics and provider-exchange log now say azure and requesty; every one said openai before. Requesty takes its label from its descriptor. A Requesty config loaded from a provider = "requesty" section also reports providerId = requesty since a later fix (see above); only an OpenAIConfig you build by hand with Requesty’s base URL still infers openai from it - pass providerId = Some(ProviderId("requesty")) to OpenAIConfig.fromValues for that.
7. Several streamed tool calls come back in the order the stream named them, in Completion.toolCalls and on the message. llm4s-core’s StreamingAccumulator kept them in an unordered map, so they could come back in hash order; this applies to every client that streams through it.

Slice 5: llm4s-openai

The fourth provider module of slice 5 (#1132), carrying the three providers that share OpenAIClient - OpenAI (provider = "openai"), Azure OpenAI ("azure") and Requesty ("requesty") - and the OpenAI embedding provider (EMBEDDING_MODEL=openai/<model>). It is in the build but not yet in a release; 0.4.1 and earlier still ship them inside llm4s-core.

With it goes the Azure OpenAI SDK (com.azure:azure-ai-openai), which OpenAIClient is built on: llm4s-core no longer depends on it, and with the Anthropic SDK already gone, core now depends on no vendor SDK at all.

OpenRouter, DeepSeek and Z.ai are not in this module. They speak the OpenAI wire format but each has its own client with no SDK, so bundling them here would make their users download the Azure SDK for nothing. They went on to llm4s-openai-compatible - see above.

What moved

Code	Now in
OpenAIClient, OpenAIProvider, AzureProvider, RequestyProvider, OpenAIEmbeddingProvider (org.llm4s.llmconnect.provider)	llm4s-openai
AzureConfig (org.llm4s.llmconnect.config)	llm4s-openai
AzureToolHelper (org.llm4s.toolapi)	llm4s-openai
ProviderModelListers.OpenAI / .Requesty → OpenAIModelLister / RequestyModelLister (org.llm4s.config)	llm4s-openai
DefaultConfig.DEFAULT_OPENAI_BASE_URL → OpenAIProvider.DEFAULT_BASE_URL	llm4s-openai
DefaultConfig.DEFAULT_REQUESTY_BASE_URL → RequestyProvider.DEFAULT_BASE_URL	llm4s-openai
DefaultConfig.DEFAULT_AZURE_V2025_01_01_PREVIEW → AzureConfig.DEFAULT_API_VERSION	llm4s-openai
ConfigKeys.OPENAI_*, REQUESTY_BASE_URL, AZURE_*, OPENAI_EMBEDDING_* → OpenAIConfigKeys (org.llm4s.config)	llm4s-openai
the commented openai-main, requesty-main and azure-main examples, and the llm4s.embeddings.openai block, in reference.conf	llm4s-openai’s reference.conf

Package names are unchanged, so import org.llm4s.llmconnect.provider.OpenAIClient keeps working once the dependency is added:

libraryDependencies += "org.llm4s" %% "llm4s-openai" % version

What stayed in core

• OpenAIConfig, because OpenRouter builds one too: OpenRouterProvider and OpenRouterClient take an OpenAIConfig, and its providerId answers openrouter for an OpenRouter base URL. It moved when OpenRouter did, to llm4s-openai-compatible, which llm4s-openai now depends on.
• OpenAIStreamingHandler (since removed with StreamingResponseHandler; see llm4s-openai-compatible), the SSE parser behind StreamingResponseHandler.forProvider("openai" | "azure" | "openrouter"), which OpenRouter’s path shares. OpenAIClient streams through the Azure SDK.
• ConfigKeys.OPENROUTER_BASE_URL, still naming OPENAI_BASE_URL (since moved to OpenAICompatibleConfigKeys).
• Strings that do not reach a client: ToolRegistry.getOpenAITools and getToolDefinitionsSafe("openai"), the openai/... model-registry data, the sk- secret pattern, config-policy allow-lists.

Registration is the dependency

llm4s-openai declares Llm4sOpenAIModule in its META-INF/services, so ProviderRegistry.default finds it and provider = "openai", "azure" and "requesty", and EMBEDDING_MODEL=openai/<model>, resolve as before. Without the dependency they fail with the registry’s error, which says the provider is not registered and names the providers that are.

ProviderRegistry.builtin no longer includes them. If you used builtin to avoid classpath discovery (a shaded fat jar, typically), add the module explicitly:

given ProviderRegistry = ProviderRegistry.ofModules(new Llm4sOpenAIModule) // `builtin` was removed later in slice 5

llm4s-rag users: RAGConfig.default embeds with openai/text-embedding-3-small. A pipeline built from the default therefore needs llm4s-openai too; otherwise name the provider you ship with .withEmbeddings("voyage", ...) (or "ollama" with llm4s-ollama). llm4s-rag does not depend on llm4s-openai itself, so it does not bring the Azure SDK.

Source breaks

1. ToolRegistry.addToAzureOptions(options) is removed. Its signature exposed the Azure SDK type ChatCompletionsOptions from core’s ToolRegistry, so core could not drop the SDK while it existed. Call AzureToolHelper.addToolsToOptions(registry, options) instead - same package (org.llm4s.toolapi), now in llm4s-openai, with the same result.
2. ProviderModelListers.OpenAI and .Requesty are now OpenAIModelLister and RequestyModelLister, in the same package (org.llm4s.config). The descriptors’ modelLister returns them, so code that reached a lister through the descriptor is unaffected.
3. Three defaults moved off DefaultConfig: DEFAULT_OPENAI_BASE_URL is now OpenAIProvider.DEFAULT_BASE_URL, DEFAULT_REQUESTY_BASE_URL is now RequestyProvider.DEFAULT_BASE_URL, and DEFAULT_AZURE_V2025_01_01_PREVIEW is now AzureConfig.DEFAULT_API_VERSION. The values are unchanged. The base URLs live on the descriptors rather than on OpenAIConfig because OpenAIConfig stays in core.
4. ConfigKeys.OPENAI_API_KEY, OPENAI_BASE_URL, OPENAI_ORG, REQUESTY_BASE_URL, AZURE_API_BASE, AZURE_API_KEY, AZURE_API_VERSION, OPENAI_EMBEDDING_BASE_URL and OPENAI_EMBEDDING_MODEL are now on OpenAIConfigKeys, in the same package, as ConfigKeys.ANTHROPIC_* became AnthropicConfigKeys. The strings are unchanged.
5. ProviderRegistry.builtin no longer includes openai, azure or requesty, nor the openai embedding provider - see above.

What did not change

Every configuration key and environment variable: llm4s.providers.<name> with provider = "openai", "azure" or "requesty" and their apiKey, baseUrl, organization, endpoint and apiVersion; llm4s.embeddings.openai.*, OPENAI_EMBEDDING_BASE_URL and OPENAI_EMBEDDING_MODEL; and llm4s.openai.apiKey as the key OpenAI embeddings share with chat. (A later change dropped that llm4s.openai.apiKey fallback: the key is llm4s.embeddings.openai.apiKey alone - see From LLM_MODEL to named provider sections.)

Slice 5: llm4s-anthropic

The third provider module of slice 5 (#1132), carrying the Anthropic Claude chat provider (provider = "anthropic"). It is in the build but not yet in a release; 0.4.1 and earlier still ship it inside llm4s-core.

With it goes the Anthropic Java SDK (com.anthropic:anthropic-java): llm4s-core no longer depends on it, so an application that does not use Anthropic no longer carries it. It was also declared, unused, by llm4s-workspace-client, and has been removed from there too.

What moved

Code	Now in
AnthropicClient, AnthropicProvider (org.llm4s.llmconnect.provider)	llm4s-anthropic
AnthropicConfig (org.llm4s.llmconnect.config)	llm4s-anthropic
ProviderModelListers.Anthropic → AnthropicModelLister (org.llm4s.config)	llm4s-anthropic
DefaultConfig.DEFAULT_ANTHROPIC_BASE_URL → AnthropicConfig.DEFAULT_BASE_URL	llm4s-anthropic
ConfigKeys.ANTHROPIC_API_KEY, ConfigKeys.ANTHROPIC_BASE_URL → AnthropicConfigKeys (org.llm4s.config)	llm4s-anthropic
the commented anthropic-main example in reference.conf	llm4s-anthropic’s reference.conf

Package names are unchanged, so import org.llm4s.llmconnect.config.AnthropicConfig keeps working once the dependency is added:

libraryDependencies += "org.llm4s" %% "llm4s-anthropic" % version

Registration is the dependency

llm4s-anthropic declares Llm4sAnthropicModule in its META-INF/services, so ProviderRegistry.default finds it and provider = "anthropic" resolves as before. Without the dependency it fails with the registry’s error, which says the provider is not registered and names the providers that are.

ProviderRegistry.builtin no longer includes Anthropic. If you used builtin to avoid classpath discovery (a shaded fat jar, typically), add the module explicitly:

given ProviderRegistry = ProviderRegistry.ofModules(new Llm4sAnthropicModule) // `builtin` was removed later in slice 5

Source breaks

Three names could not keep their fully-qualified path, because they were members of objects that stay in core:

1. ProviderModelListers.Anthropic is now AnthropicModelLister, in the same package (org.llm4s.config). AnthropicProvider.modelLister returns it, so code that reached the lister through the descriptor is unaffected.
2. DefaultConfig.DEFAULT_ANTHROPIC_BASE_URL is now AnthropicConfig.DEFAULT_BASE_URL, the same place GeminiConfig, DeepSeekConfig and MistralConfig keep theirs. The value is unchanged.
3. ConfigKeys.ANTHROPIC_API_KEY and ConfigKeys.ANTHROPIC_BASE_URL are now AnthropicConfigKeys.ANTHROPIC_API_KEY and AnthropicConfigKeys.ANTHROPIC_BASE_URL, in the same package, as ConfigKeys.OLLAMA_* became OllamaConfigKeys. The strings are unchanged.

What did not change

Every configuration key and environment variable: llm4s.providers.<name> with provider = "anthropic", apiKey and the optional baseUrl.

What names Anthropic without depending on its client stays in core and answers the same with or without llm4s-anthropic: ToolRegistry.getToolDefinitionsSafe("anthropic"), the anthropic/... entries in the embedded model registry data, the sk-ant- secret pattern, config-policy allow-lists, and AnthropicStreamingHandler - the SDK-free SSE parser behind StreamingResponseHandler.forProvider("anthropic"), which AnthropicClient does not use.

Slice 5: llm4s-gemini

The second provider module of slice 5 (#1132), carrying both of Google’s chat providers: the Gemini API (provider = "gemini", alias "google") and Vertex AI (provider = "vertexai", alias "vertex"). It is in the build but not yet in a release; 0.4.1 and earlier still ship both inside llm4s-core.

Why Vertex AI is in the same module

VertexAIClient only calls Google’s publishers/google models - Gemini - using the same JSON request and response format as GeminiClient. The two differ in endpoint (Vertex is scoped to a GCP project and region on aiplatform.googleapis.com) and in authentication (Vertex uses OAuth2, implemented by VertexAIAuthProvider without a Google SDK), not in dependencies. So bundling them costs a Gemini-API user nothing, while splitting Vertex AI out later would be a breaking move for its users; bundling now is the direction that stays safe.

What moved

Code	Now in
GeminiClient, GeminiProvider (org.llm4s.llmconnect.provider)	llm4s-gemini
VertexAIClient, VertexAIProvider, VertexAIAuthProvider (org.llm4s.llmconnect.provider)	llm4s-gemini
GeminiConfig, VertexAIConfig (org.llm4s.llmconnect.config)	llm4s-gemini
ProviderModelListers.Gemini → GeminiModelLister (org.llm4s.config)	llm4s-gemini
DefaultConfig.DEFAULT_GEMINI_BASE_URL → GeminiConfig.DEFAULT_BASE_URL	llm4s-gemini
DefaultConfig.DEFAULT_VERTEXAI_LOCATION → VertexAIConfig.DEFAULT_LOCATION (already existed)	llm4s-gemini
the commented gemini-main example in reference.conf	llm4s-gemini’s reference.conf, with a vertexai-main example beside it

Package names are unchanged, so import org.llm4s.llmconnect.config.GeminiConfig keeps working once the dependency is added:

libraryDependencies += "org.llm4s" %% "llm4s-gemini" % version

Registration is the dependency

llm4s-gemini declares Llm4sGeminiModule in its META-INF/services, so ProviderRegistry.default finds it and provider = "gemini", "google", "vertexai" and "vertex" resolve as before. Without the dependency they fail with the registry’s error, which says the provider is not registered and names the providers that are.

ProviderRegistry.builtin no longer includes Gemini or Vertex AI. If you used builtin to avoid classpath discovery (a shaded fat jar, typically), add the module explicitly:

given ProviderRegistry = ProviderRegistry.ofModules(new Llm4sGeminiModule) // `builtin` was removed later in slice 5

Source breaks

Three names could not keep their fully-qualified path, because they were members of objects that stay in core:

1. ProviderModelListers.Gemini is now GeminiModelLister, in the same package (org.llm4s.config). GeminiProvider.modelLister returns it, so code that reached the lister through the descriptor is unaffected.
2. DefaultConfig.DEFAULT_GEMINI_BASE_URL is now GeminiConfig.DEFAULT_BASE_URL, the same place DeepSeekConfig, CohereConfig and MistralConfig keep theirs. The value is unchanged.
3. DefaultConfig.DEFAULT_VERTEXAI_LOCATION is removed; use VertexAIConfig.DEFAULT_LOCATION, which already held the same "us-central1".

What did not change

Every configuration key and environment variable: llm4s.providers.<name> with provider = "gemini" or "vertexai", the Vertex AI reading of endpoint (GCP project id), organization (region) and apiKey (credential file path), and GOOGLE_APPLICATION_CREDENTIALS for Vertex AI authentication. (endpoint and organization have since become Vertex AI’s project and location; see the note above.)

Strings that name Gemini without depending on its client stay in core and answer the same with or without llm4s-gemini: ToolRegistry.getToolDefinitionsSafe("gemini"), the gemini/... entries in the embedded model registry data, and config-policy allow-lists.

Slice 4 (close-out): the last closed provider list, and fromValues stops throwing

The last items deferred from slice 4 (#1131). Both are source breaks, taken now because the API is not yet frozen; neither has a deprecated shim, following the precedent of ProviderKind in PR 1. It is in the build but not yet in a release, so nothing here affects 0.4.1 or earlier.

org.llm4s.rag.EmbeddingProvider is gone

llm4s-rag kept its own closed list of embedding providers - EmbeddingProvider.OpenAI, Voyage and Ollama - duplicating what the ProviderRegistry has known since PR 4. It was wrong in both directions: an embedding provider from its own module could not be named through it, and it named ollama whether or not llm4s-ollama was on the classpath. It also shared its simple name with org.llm4s.llmconnect.provider.EmbeddingProvider, the embedding client trait.

RAGConfig now names the provider by id, the same id as in EMBEDDING_MODEL=<id>/<model>, and RAG.build resolves it through the registry:

// Before
import org.llm4s.rag.{ EmbeddingProvider, RAG }

RAG.builder()
  .withEmbeddings(EmbeddingProvider.OpenAI, "text-embedding-3-large")

EmbeddingProvider.fromString(name).toRight(...)   // to turn a configured name into one

// After
import org.llm4s.rag.RAG

RAG.builder()
  .withEmbeddings("openai", "text-embedding-3-large")

RAG.builder().withEmbeddings(name)                 // any registered id or alias; no conversion

build on a RAGConfig is an extension from RAG.RAGConfigOps, so the call site needs import org.llm4s.rag.RAG.RAGConfigOps and a ModelRegistryService given (for example from Llm4sConfig.modelRegistryService()); it takes only a using clause, so it is .build, not .build().

RAGConfig.embeddingProvider is a ProviderId, so config.embeddingProvider.name becomes config.embeddingProvider.asString. RAG.build and RAGConfig#build take an implicit ProviderRegistry, resolved to ProviderRegistry.default when none is in scope, so existing call sites compile unchanged and an application with its own registry reaches the providers in it. EmbeddingProvider.values has no replacement in llm4s-rag: the providers are summon[ProviderRegistry].embeddingIds.

Behaviour changes

• An id that is not registered fails RAG.build with the registry’s own “Embedding provider ‘…’ is not registered” error, naming the ids that are - where fromString returned None and every caller wrote its own message. The resolver is asked for the provider’s canonical id, so an alias such as voyageai arrives as voyage.
• The model default moved from llm4s-rag into the provider. RAG used to pick a model per provider from its own table, and ignore the model in the EmbeddingProviderConfig the resolver returned. RAGConfig() still defaults to openai / text-embedding-3-small, but withEmbeddings(provider) without a model now clears any model set earlier and uses, in order: the resolved config’s model, then the provider’s configSpec.defaultModel. With Llm4sConfig.embeddings() as the resolver, that is the model you configured. A provider with neither fails the build naming the provider, rather than guessing.
• Dimensions come from the provider’s dimensionsOf(model) when not set explicitly, rather than from a second table in llm4s-rag. A model its provider does not declare keeps the old fallback of 1536.

fromValues returns Result

Every ProviderConfig subtype’s fromValues factory - OpenAIConfig, AzureConfig, AnthropicConfig, ZaiConfig, GeminiConfig, DeepSeekConfig, CohereConfig, MistralConfig, VertexAIConfig and OllamaConfig - validated its arguments with require(...), so a blank API key threw IllegalArgumentException out of a library whose rule is that errors are values. They now return Result[XConfig], and a blank credential or endpoint is a ConfigurationError carrying the same message as before ("OpenAI apiKey must be non-empty") with the field in missingKeys.

// Before
val config: OpenAIConfig = OpenAIConfig.fromValues("gpt-4o", apiKey, None, baseUrl)
val client = LLMConnect.getClient(config)

// After
val client: Result[LLMClient] =
  OpenAIConfig.fromValues("gpt-4o", apiKey, None, baseUrl).flatMap(LLMConnect.getClient(_))

A ProviderDescriptor.buildConfig that returned fromValues from a for’s yield, or through .map, binds it as a generator or uses .flatMap instead:

// Before
for
  apiKey  <- ProviderDescriptor.requireApiKey(providerName, section)
  baseUrl <- ProviderDescriptor.resolveBaseUrl(providerName, section, configSpec)
yield AcmeConfig.fromValues(section.model.asString, apiKey, baseUrl)

// After
for
  apiKey  <- ProviderDescriptor.requireApiKey(providerName, section)
  baseUrl <- ProviderDescriptor.resolveBaseUrl(providerName, section, configSpec)
  config  <- AcmeConfig.fromValues(section.model.asString, apiKey, baseUrl)
yield config

Code that caught the exception - Try(OpenAIConfig.fromValues(...)).toEither, or a test’s an[IllegalArgumentException] should be thrownBy - matches on the Left instead. Nothing reachable from configuration changes: Llm4sConfig and the provider descriptors already rejected a missing key before calling fromValues, and now propagate its Left rather than letting a blank one throw.

Slice 5: llm4s-ollama

The first provider module of slice 5 (#1132). Ollama goes first because it has the smallest client, no vendor SDK, and a live @Ollama integration tier - so the carve is checked against a real server rather than mocks. It is in the build but not yet in a release; 0.4.1 and earlier still ship Ollama inside llm4s-core.

What moved

Code	Now in
OllamaClient, OllamaProvider, OllamaEmbeddingProvider (org.llm4s.llmconnect.provider)	llm4s-ollama
OllamaConfig (org.llm4s.llmconnect.config)	llm4s-ollama
ProviderModelListers.Ollama → OllamaModelLister (org.llm4s.config)	llm4s-ollama
ConfigKeys.OLLAMA_* → OllamaConfigKeys.OLLAMA_* (org.llm4s.config)	llm4s-ollama
the llm4s.embeddings.ollama reference.conf block	llm4s-ollama’s reference.conf

Package names are unchanged, so import org.llm4s.llmconnect.config.OllamaConfig keeps working once the dependency is added:

libraryDependencies += "org.llm4s" %% "llm4s-ollama" % version

Registration is the dependency

llm4s-ollama declares Llm4sOllamaModule in its META-INF/services, so ProviderRegistry.default finds it and provider = "ollama" and EMBEDDING_MODEL=ollama/<model> resolve as before. Without the dependency both fail with the registry’s error, which says ollama is not registered and names the providers that are.

ProviderRegistry.builtin no longer includes Ollama, because core no longer ships it. If you used builtin to avoid classpath discovery (a shaded fat jar, typically), add the module explicitly:

given ProviderRegistry = ProviderRegistry.ofModules(new Llm4sOllamaModule) // `builtin` was removed later in slice 5

Source breaks

Two names could not keep their fully-qualified path, because they were members of objects that stay in core:

1. ProviderModelListers.Ollama is now OllamaModelLister, in the same package (org.llm4s.config). OllamaProvider.modelLister returns it, so code that reached the lister through the descriptor is unaffected.
2. ConfigKeys.OLLAMA_BASE_URL, OLLAMA_EMBEDDING_BASE_URL and OLLAMA_EMBEDDING_MODEL are now on OllamaConfigKeys, also in org.llm4s.config. The variable names themselves are unchanged.

What did not change

Every configuration key and environment variable: llm4s.providers.<name> with provider = "ollama", llm4s.embeddings.ollama.*, OLLAMA_EMBEDDING_BASE_URL and OLLAMA_EMBEDDING_MODEL. The reference.conf block moved rather than changed; HOCON merges reference files across jars, so the keys exist exactly when the provider does.

TokenizerMapping still recognises the ollama/ model-name prefix. It is a naming convention on model strings rather than a reference to the provider, and it gives the same answer whether or not llm4s-ollama is on the classpath.

Slice 4 follow-up: embedding dimensions move into the provider

One of the two items deferred from slice 4 (#1131), and the precursor to carving llm4s-ollama (#1132). ModelDimensionRegistry was the last central provider list on the embedding side: a map in llm4s-core covering openai, voyage and local. It had no ollama entry, so the documented

EMBEDDING_MODEL=ollama/nomic-embed-text

failed at Llm4sConfig.textEmbeddingModel() with Unknown model 'nomic-embed-text' for provider 'ollama', and RAGASFactory.fromConfigs papered over the same gap with .getOrElse(1536) for a model that is 768-dimensional.

Declaring dimensions

An embedding provider now declares the dimensions of the models it knows, alongside its configSpec:

object JinaEmbeddings extends EmbeddingProviderDescriptor:
  val id = ProviderId("jina")

  override val modelDimensions = Map(
    "jina-embeddings-v3" -> 1024
  )

A provider whose model names have variants overrides dimensionsOf(model) instead - Ollama folds a :latest tag onto the untagged name, and no other tag, because other tags of one model can differ in size. A model a provider does not declare still embeds; only a caller that needs its dimensionality up front is told it is unknown.

Source-compatible signature changes

ModelDimensionRegistry.getDimension, RAGASFactory.fromConfigs and RAGASFactory.basicFromConfigs take an implicit ProviderRegistry, resolved to ProviderRegistry.default when none is in scope. Existing call sites compile unchanged; a caller with its own registry now reaches the providers in it.

ModelDimensionRegistry.localDimension(model) answers the local non-text encoders (openclip-vit-b32, wav2vec2-base, timesformer-base), which have no descriptor because nothing can be configured with them. getDimension("local", ...) still works.

Behaviour changes

• RAGASFactory.fromConfigs and basicFromConfigs return the lookup’s Left for an embedding model its provider does not declare, instead of assuming 1536 dimensions. Build the EmbeddingModelConfig yourself and call RAGASFactory.create or basic for such a model.
• getDimension for an unregistered provider returns the registry’s “not registered” error, naming the embedding providers that are, rather than “Unknown model”.
• voyage-3-large resolves to 1024, its default output size, not 1536.

Slice 4 (PR 5): embedding config moves into the provider

The fifth slice 4 change (#1131), and the follow-up PR 4 named. PR 4 made an embedding provider resolvable from its own module; its configuration was still core’s business:

final private case class EmbeddingsOllamaSection(apiKey: …, baseUrl: …, model: …)
implicit private val embeddingsOllamaSectionReader = …
private val DefaultOllamaEmbeddingBaseUrl = "http://localhost:11434"
private def buildOllamaEmbeddings(…) = …
// plus two `match` arms

— a typed case class, a PureConfig reader, a default, a builder and two dispatch arms per provider. A third-party embedding provider could be registered and then had nothing to be configured with.

Nothing changes for users. EMBEDDING_MODEL, EMBEDDING_PROVIDER, OPENAI_API_KEY, VOYAGE_API_KEY, OLLAMA_EMBEDDING_BASE_URL and every llm4s.embeddings.<id> key behave exactly as before.

The section shape is core’s; what it means is the provider’s

llm4s-core now parses one uniform shape - apiKey, baseUrl, model - for whichever provider was selected, and hands it to the descriptor:

def buildConfig(section: EmbeddingProviderSection, modelOverride: Option[String]): Result[EmbeddingProviderConfig]

Everything it needs arrives in section, already typed. A descriptor reads no configuration itself: raw config access stays in org.llm4s.config, which is the boundary AGENTS.md sets.

Most providers never implement it. Declaring an EmbeddingConfigSpec is enough, and the default implementation resolves the three fields against it:

object JinaEmbeddings extends EmbeddingProviderDescriptor:
  val id = ProviderId("jina")

  override val configSpec = EmbeddingConfigSpec(
    requiresApiKey = true,
    defaultBaseUrl = Some("https://api.jina.ai/v1"),
    apiKeyEnv      = Some("JINA_API_KEY")   // named in the error when it is missing
  )

Defaults are code, environment bindings are HOCON

They used to be both. reference.conf said baseUrl = "http://localhost:11434" and EmbeddingsConfigLoader said DefaultOllamaEmbeddingBaseUrl, with nothing keeping them in step. The default now lives only in the descriptor’s EmbeddingConfigSpec, and each provider’s reference.conf block is reduced to the environment variables it binds:

ollama {
  baseUrl = ${?OLLAMA_EMBEDDING_BASE_URL}
  model   = ${?OLLAMA_EMBEDDING_MODEL}
}

That block is keyed by provider id, so it travels with the provider when the provider moves to its own module - HOCON merges these across jars. ProviderId canonicalises (trim, lowercase) but does not restrict, so an id containing a dot is legal and must be quoted as a single HOCON key; EmbeddingConfigSpec.sectionPath / fieldPath build the path that way, and the paths named in errors are the paths to write:

llm4s.embeddings."acme.embeddings" { apiKey = ${?ACME_API_KEY} }

(Chat config cannot do this: it is keyed by the user’s instance name, which is why ProviderConfigSpec.defaultBaseUrl is code and says so.)

A key that lives somewhere else

Superseded: apiKeyPath was removed in favour of the shared llm4s.credentials.<id>.apiKey - see Vendor credentials. apiKeyEnv is now a Seq[String].

A provider whose key is kept outside its own llm4s.embeddings.<id> section declares where to look instead of the loader special-casing it, and that declaration is also what makes the error name the place the key is really set:

override val configSpec = EmbeddingConfigSpec(
  requiresApiKey = true,
  apiKeyPath     = Some("llm4s.acme.apiKey"),
  apiKeyEnv      = Some("ACME_API_KEY"),
  …
)

Missing acme embeddings apiKey (llm4s.acme.apiKey / ACME_API_KEY)

OpenAI was the case this was written for, reading llm4s.openai.apiKey; it no longer uses it (its key is its own llm4s.embeddings.openai.apiKey), and no provider in this repository does.

apiKeyPath is a declaration, not a read: EmbeddingsConfigLoader resolves it and hands the value back in the section before calling buildConfig. The provider owns the knowledge of where its key lives; org.llm4s.config keeps sole ownership of reading it.

The embedding config entry points take the registry

def embeddings()(using ProviderRegistry): Result[(String, EmbeddingProviderConfig)]
def loadTextEmbeddingModel()(using ProviderRegistry): Result[TextEmbeddingModelSettings]
def textEmbeddingModel()(using ProviderRegistry): Result[TextEmbeddingModelSettings]

Binary-incompatible, source-compatible, as in PRs 3 and 4. Without it an application’s own registry could not reach the loader, and a provider it registered explicitly would resolve for EmbeddingClient.from but not for its configuration.

All three resolve through the same registry, so a provider configurable by one is configurable by all of them: textEmbeddingModel now goes through embeddings rather than calling the loader a second time.

Error messages

Unknown providers now produce the registry’s message, which names what is registered and the scan that found it. Missing-field errors name the config path and the environment variable the descriptor declared. The provider is named by its canonical id (openai, not OpenAI) - the spelling that appears in config.

Slice 4 (PR 4): embedding providers join the SPI

The fourth slice 4 change (#1131), and the last unchecked item on that issue’s list. PRs 2 and 3 made a chat provider self-describing and discoverable. EmbeddingClient.from was still the shape the slice exists to delete:

provider.toLowerCase match
  case "openai" => Right(new EmbeddingClient(OpenAIEmbeddingProvider.fromConfig(cfg)))
  case "voyage" => ...
  case "ollama" => ...
  case other    => Left(EmbeddingError(...))

so an embedding provider was an edit to llm4s-core no matter where its code lived. It is now resolved through the same ProviderRegistry.

Declaring an embedding provider

EmbeddingProviderDescriptor is the embedding counterpart of ProviderDescriptor, and Llm4sProviderModule gained a second list:

object JinaEmbeddings extends EmbeddingProviderDescriptor:
  val id = ProviderId("jina")

  def build(config: EmbeddingProviderConfig): Result[EmbeddingProvider] =
    Right(JinaEmbeddingProvider.fromConfig(config))

final class JinaProviderModule extends Llm4sProviderModule:
  override def embeddingProviders: Seq[EmbeddingProviderDescriptor] = Seq(JinaEmbeddings)

Registration is otherwise identical to PR 3 - the same META-INF/services file, the same ServiceLoader scan, the same escape hatches. A module declares whichever halves it has; both default to empty.

If your module delegates, forward both lists. Every Llm4sProviderModule member defaults to Nil, so a module that forwards only chatProviders contributes no embedding providers and fails silently rather than at compile time.

Why a separate trait, not a method on ProviderDescriptor

The two provider sets overlap without either containing the other: OpenAI and Ollama supply a chat client and an embedding provider, Voyage supplies only embeddings, Anthropic only chat. Folding embeddings into ProviderDescriptor would force an embedding-only provider to implement buildConfig and buildClient only to fail them.

The ids therefore live in two namespaces, and the same id can appear in both — ollama names a chat client and an embedding provider that share nothing but a base URL. ids and embeddingIds list them separately, and a provider that supplies no embeddings fails as such:

Embedding provider ‘anthropic’ (from llm4s.embeddings.model) is not registered. Registered embedding providers: ollama, openai, voyage. If you expected ‘anthropic’, add the dependency that supplies it, or register it explicitly with ProviderRegistry.ofEmbeddings(…).

Each half names the registration call that accepts its own descriptor type - of for chat, ofEmbeddings for embeddings - because following the other one is a compile error.

EmbeddingClient.from takes the registry

def from(provider: String, cfg: EmbeddingProviderConfig)(using
  ModelRegistryService,
  ProviderRegistry
): Result[EmbeddingClient]

Binary-incompatible, source-compatible: existing call sites resolve ProviderRegistry.default through the companion’s given, exactly as the Llm4sConfig methods did in PR 3. An application that registers its own passes it:

given ProviderRegistry = ProviderRegistry.default.withEmbeddingProvider(JinaEmbeddings)
EmbeddingClient.from("jina", cfg)

ProviderRegistry gained findEmbedding, resolveEmbedding, embeddingIds, canonicalEmbeddingId, withEmbeddingProvider and ofEmbeddings; ProviderModuleReport gained embeddingProviderIds. The unknown-provider failure is still an EmbeddingError with code 400, now carrying the registry’s diagnostics as its message.

What did not change

EmbeddingProvider itself, EmbeddingProviderConfig, and each provider’s fromConfig are untouched - OpenAIEmbeddingProvider.fromConfig(cfg) still works and is still the direct route. The three built-in objects simply are their own descriptors now.

Embedding configuration is not part of this change: llm4s.embeddings still has typed openai / voyage / ollama sections in EmbeddingsConfigLoader, so a third-party embedding provider is reachable through EmbeddingClient.from but still needs its config built by the application. Moving config binding into the descriptor, as PR 2 did for chat, is the follow-up.

Slice 4 (PR 3): providers are discovered on the classpath

The third slice 4 change (#1131). PR 2 made a provider a ProviderDescriptor that registers itself; this removes the last manual step. A provider module on the classpath is now found without any registration code at the call site - adding a provider is adding a dependency.

Declaring a provider module

Ship a META-INF/services/org.llm4s.llmconnect.spi.Llm4sProviderModule naming an implementation:

com.example.llm4s.BedrockProviderModule

// Must be a `class` with a public no-arg constructor, not an `object`:
// ServiceLoader instantiates the named class, and a Scala `object` exposes its
// instance as a MODULE$ field instead. (This is also what GraalVM native-image
// needs, via its ServiceLoaderFeature.)
final class BedrockProviderModule extends Llm4sProviderModule:
  override def chatProviders: Seq[ProviderDescriptor] = Seq(BedrockProvider)

That is the whole registration. ProviderRegistry.default - what every Llm4sConfig and LLMConnect call uses when the caller supplies no registry - is now ProviderRegistry.discover(), computed once on first use.

llm4s-core declared its own providers the same way, through org.llm4s.llmconnect.provider.BuiltinProviderModule, with no special case: they were discovered exactly as a third-party module is. (Since slice 5 core ships no provider, and that module is gone.)

One broken jar cannot take out the others

java.util.ServiceLoader’s iterator throws ServiceConfigurationError for an entry it cannot load, and the for-comprehension you would naturally write over it propagates the first such error and abandons every remaining provider. discover drives the iterator by hand and guards each step, so an unusable entry becomes a recorded failure and the scan continues:

val registry = ProviderRegistry.discover()
registry.report.failures.foreach(f => println(f.detail))
println(registry.report.describe)
// Discovery scanned 2 modules; 1 failed: loading a provider module failed: ...
//   - org.llm4s.llmconnect.provider.BuiltinProviderModule [file:/.../llm4s-core.jar]: openai, openrouter, ...
//   ! loading a provider module failed: ... Provider com.example.Missing not found

Failures are logged at WARN as they happen, and the scan summary is appended to the “provider is not registered” error, because the two failure modes that are otherwise invisible are a dependency that was never added and a fat jar whose services files were dropped:

Provider ‘bedrock’ (from llm4s.providers.my-bedrock.provider) is not registered. Registered providers: anthropic, azure, … If you expected ‘bedrock’, add the dependency that supplies it, or register it explicitly with ProviderRegistry.of(…). Discovery scanned 1 module; 0 failed.

Fat jars

Shading tools default to overwriting same-named resources, which silently discards every services file but one. Configure them to concatenate:

// sbt-assembly
assembly / assemblyMergeStrategy := {
  case PathList("META-INF", "services", _*) => MergeStrategy.filterDistinctLines
  case other                                => (assembly / assemblyMergeStrategy).value(other)
}

<!-- maven-shade -->
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>

If you cannot, register explicitly - this is what the escape hatch is for:

val registry = ProviderRegistry.ofModules(new Llm4sOpenAIModule).withProvider(BedrockProvider)
LLMConnect.getClient(config)(using registry)

ProviderRegistry.ofModules builds a registry from the modules you name, with no classpath scan at all. (This section first suggested ProviderRegistry.builtin, the providers compiled into llm4s-core; it was removed when the last of them left core.)

Llm4sConfig takes the registry

Every Llm4sConfig method that reads llm4s.providers now takes an implicit ProviderRegistry: provider, providerConfigs (both), providers, defaultProviderName, defaultProvider, listModels (both), and providerFrom. Existing call sites are unchanged - the companion supplies ProviderRegistry.default - and a caller who wants a different set of providers passes one:

given ProviderRegistry = ProviderRegistry.default.withProvider(MyProvider)
val config = Llm4sConfig.provider("my-provider")   // now resolvable

This is a binary-incompatible change to those signatures, and source-compatible.

What did not change

ProviderDescriptor, ProviderConfigSpec, ProviderFeatures and Llm4sProviderModule are as PR 2 shipped them. ProviderRegistry.of, ofModules, withProvider and withModule behave as before; registries built that way report discovered = false and carry no scan summary.

Slice 4 (PR 2): the provider registration SPI

The second slice 4 change (#1131). PR 1 removed the two structures that made an out-of-module provider impossible - a closed enum and a sealed trait. This one builds the extension point on top: a provider is now a value that describes itself, and everything that used to enumerate providers looks them up instead.

Every provider still ships inside llm4s-core; what changed is that none of them is wired in by hand any more. Splitting them into their own artifacts is slice 5 (#1132).

Adding a provider

Before, adding one chat provider meant editing roughly eight shared files - the closed enum, the sealed config file, two match expressions in LLMConnect, the loader’s dispatch, a validator object, a capabilities object and the capabilities registry. That surface is why 13 open provider PRs all conflict with each other.

Now it is one file:

object BedrockProvider extends ProviderDescriptor:
  val id: ProviderId = ProviderId("bedrock")

  val configSpec: ProviderConfigSpec =
    ProviderConfigSpec(requiresApiKey = true, requiresEndpoint = true)

  def buildConfig(providerName: String, section: NamedProviderConfig)(using
    ContextWindowResolver
  ): Result[ProviderConfig] =
    for
      apiKey   <- ProviderDescriptor.requireApiKey(providerName, section)
      endpoint <- ProviderDescriptor.requireField(
                    providerName, "endpoint", section.endpoint, "llm4s.providers.<name>.endpoint")
    yield BedrockConfig.fromValues(section.model.asString, apiKey, endpoint)

  def buildClient(config: ProviderConfig, options: LlmClientOptions)(using
    ModelRegistryService
  ): Result[LLMClient] =
    ProviderDescriptor
      .expectConfig[BedrockConfig](id, config)
      .flatMap(BedrockClient(_, options.metrics, options.exchangeLogging))

plus a ProviderConfig implementation, a client, and one registration:

val registry = ProviderRegistry.default.withProvider(BedrockProvider)
LLMConnect.getClient(config)(using registry)

Classpath discovery - registering by adding a dependency, with no code at all - is PR 3. Llm4sProviderModule is the service type it will discover; it is already here so that a module can group its providers today.

What is new

All in org.llm4s.llmconnect.spi:

Type	Purpose
ProviderDescriptor	one provider: id, aliases, config shape, features, model lister, and the two builders
ProviderConfigSpec	which section fields the provider requires, its default base URL, and the text shown when a field is missing
ProviderFeatures	what the client actually implements, declared statically (Cohere and Mistral declared streaming = false until slice 5 - #925)
ProviderRegistry	an immutable set of descriptors; of, withProvider, withModule, and lookup that returns Result
Llm4sProviderModule	the unit of registration - one module, several providers

ProviderRegistry is resolved through a using clause with a default given in its companion, so existing call sites are unchanged and a caller who wants a different set passes one:

LLMConnect.getClient(config)                    // ProviderRegistry.default
LLMConnect.getClient(config)(using myRegistry)  // only the providers you registered

What was deleted

All of these were private[llm4s], so this costs users nothing:

Deleted	Replaced by
config.ProviderCapabilities (trait + 12 objects)	ProviderDescriptor
config.ProviderCapabilitiesRegistry	ProviderRegistry
config.NamedProviderValidator (trait) and NamedProviderValidators (12 objects)	ProviderConfigSpec + one generic NamedProviderSectionValidator
the twelve-branch match in NamedProviderLoader	descriptor.buildConfig
the two match expressions in LLMConnect	descriptor.buildClient
the hard-coded "google"/"vertex" alias fold in NamedProviderConfigNormalizer	ProviderDescriptor.aliases

Error messages for missing fields are unchanged; they are now generated from the spec rather than written out per provider.

Source breaks

1. ReliableProviders: seven per-provider factories collapse to wrap.

   // Before - covered 7 of 12 providers, and no provider from another module
   ReliableProviders.openai(config, ReliabilityConfig.aggressive)
   ReliableProviders.anthropic(config)
   
   // After - covers every registered provider
   ReliableProviders.wrap(config, ReliabilityConfig.aggressive)
   ReliableProviders.wrap(config)
   

   The missing five were DeepSeek, Cohere, Mistral, Requesty and Vertex AI. wrap(client, providerName, ...), for a client you already have, is unchanged.

2. OpenAIConfig.providerId is derived from baseUrl. It answers openrouter for a URL containing openrouter.ai and openai otherwise - which is exactly the routing LLMConnect already did with a hard-coded check, now stated by the config itself. If you build an OpenAIConfig for OpenRouter and inspect providerId, the answer changed from openai to openrouter; client routing is unchanged.

3. ProviderModelLister is now public, along with ProviderModelListers and its new openAICompatible(provider, defaultBaseUrl, modelsPath) factory - a provider module needs to supply a model lister, and most providers serve the OpenAI /models shape. The five near-identical per-provider lister objects became calls to that factory.

4. ProviderResultOps and ProviderExchangeRecorder are now public (they were private[provider]). A provider client outside llm4s-core needs both.

What did not change

Llm4sConfig’s signatures, NamedProviderLoader’s results, DiscoveredModel, every ProviderConfig subtype’s fields, and the llm4s.providers.* config format. A configuration that worked before works now, including provider = "google" and provider = "vertex".

ProviderConfig.fromValues still uses require(...), which throws rather than returning a Left. Converting it is a behaviour change (throw → Left) that deserves its own note, and embeddings (EmbeddingClient.from, EmbeddingsConfigLoader’s fixed-arity reader, ModelDimensionRegistry, and the duplicate org.llm4s.rag.EmbeddingProvider name ADT) are still on the old dispatch. Both are tracked under #1131.

Since resolved. The embedding entry points moved onto the registry in PR 4, PR 5 and the dimensions follow-up above; the org.llm4s.rag.EmbeddingProvider ADT was removed, and fromValues converted to Result, in the slice 4 close-out.

Slice 4 (PR 1): ProviderKind becomes ProviderId, ProviderConfig opens up

The first of the slice 4 changes (#1131). No SPI yet - this only removes the two things that make a provider impossible to supply from outside llm4s-core: a closed enum and a sealed trait. It is in the build but not yet in a release, so nothing here affects 0.4.1 or earlier.

This is a clean break: there is no deprecated ProviderKind shim. A shim would have kept a closed list of twelve providers inside the very module whose purpose is to remove it, and it would only have half-worked - case ProviderKind.OpenAI => and def f(k: ProviderKind) would still compile, while .values, .ordinal, .fromOrdinal and exhaustivity would not. A clear compile error beats a partly-working deprecated type.

ProviderKind → ProviderId

// Before
enum ProviderKind:
  case OpenAI; case Anthropic; /* ... ten more */

// After
opaque type ProviderId = String
object ProviderId:
  def apply(raw: String): ProviderId = raw.trim.toLowerCase(Locale.ROOT)  // canonicalises
  extension (id: ProviderId) def asString: String = id

ProviderId is an open vocabulary, not an enumeration. Any string names a provider; whether that provider can be resolved is answered at resolution time, by whatever is on the classpath - which is the whole point. It stays opaque over String, so Option[ProviderId] and Map[ProviderName, ProviderId] still do not box.

Before	After
ProviderKind.OpenAI	ProviderId("openai")
ProviderKind.fromString(s) / fromName(s), returning Option	ProviderId(s), total
kind.name	id.asString
kind.toString → "OpenAI"	id.asString → "openai"
ProviderKind.all	no replacement - ask the thing that resolves providers, not the type
ProviderKind.values / .ordinal / .fromOrdinal / .productPrefix	no replacement
exhaustive match on ProviderKind	match on id.asString, with a default branch

asString lives in object ProviderId rather than beside ModelName.asString and friends, because every newtype in ProviderModelTypes erases to String and a second asString at that level would be a double definition after erasure. Companion-scoped extensions resolve through the opaque type’s implicit scope, so id.asString still needs no extra import.

toString changed value. ProviderKind.OpenAI.toString was "OpenAI"; a ProviderId is its canonical lowercase spelling, so it prints "openai". If you interpolated a provider into log or error text, expect the case to change. Uppercase derivations still work: providerId.asString.toUpperCase is "OPENAI", as providerKind.toString.toUpperCase was.

ProviderConfig is no longer sealed, and describes itself

In Scala 3 sealed restricts extension to the same file, so all ten provider configs were stuck in one 756-line file - not merely in the same jar. ProviderConfig is now a plain trait with three new members:

trait ProviderConfig:
  def providerId: ProviderId              // replaces `val provider: ProviderKind`
  def endpointUrl: Option[String]         // the endpoint this config will contact
  def withModel(model: String): ProviderConfig
  // model, contextWindow, reserveCompletion unchanged

// Before
config.provider == ProviderKind.OpenAI
// After
config.providerId == ProviderId("openai")

The three additions exist so that code describing a config does not have to know the set of subtypes. Four exhaustive matches were deleted rather than moved by using them: ConfigPolicyEngine.providerName, ConfigPolicyEngine.baseUrlOrEndpoint, PrometheusMetricsExample’s provider-name match, and ProviderSetupRuntime.overrideModel. If you have a match on ProviderConfig, that is the migration: reach for providerId, endpointUrl or withModel first, and only keep the match if you genuinely need provider-specific fields.

Losing sealed also means an exhaustive match on ProviderConfig now compiles with a warning - and fails for anyone building with -Werror, as this repo does. Add a default branch, or annotate the scrutinee (config: @unchecked) if you have deliberately accepted the risk.

One behaviour change falls out of this: ConfigPolicyEngine.baseUrlOrEndpoint used to return None for VertexAIConfig, because the old match had no case for it. It now returns Some(computedBaseUrl). A requiredBaseUrlPattern policy that silently reported “no endpoint/baseUrl found” for Vertex AI will now actually check the URL.

Unknown provider ids are no longer rejected while parsing

NamedProviderConfigNormalizer used to fail on an unrecognised provider string with "Configured provider 'x' has unknown provider 'moonbeam'". It now produces a ProviderId unconditionally; only resolution fails, with an error naming what is registered:

No provider capabilities registered for provider 'moonbeam'.
Registered providers: anthropic, azure, cohere, deepseek, gemini, mistral, ollama,
openai, openrouter, requesty, vertexai, zai

This is what lets a provider live in a module llm4s-core has never heard of. The accepted aliases are unchanged - provider = "google" still resolves to gemini, and provider = "vertex" to vertexai - though that table moves into each provider’s descriptor when the SPI lands.

Validation error text is now provider-agnostic

// Before
Azure OpenAI provider 'my-azure' is missing required fields:
// After
Provider 'my-azure' (provider = azure) is missing required fields:

The per-field guidance underneath is unchanged, including the ${?AZURE_API_KEY} substitution hint. (Since superseded: the apiKey line now names the variable the provider module binds - AZURE_OPENAI_API_KEY for Azure - see Vendor credentials.) Only the leading sentence differs, because it used to be generated from a hard-coded display name per provider.

Bug fix: provider = "vertexai" now works at all

ProviderKind.VertexAI existed, NamedProviderLoader built a VertexAIConfig from it, and LLMConnect built a VertexAIClient from that - but Vertex AI was missing from ProviderCapabilitiesRegistry and had no validator object. Since validation routes through that registry, every provider = "vertexai" config failed validation outright, so none of the supporting code was reachable from configuration. Both are now present, and the config path is covered by tests.

What did not change

ProviderConfig.fromValues’s require(...) calls still throw rather than returning Result; converting them is a throw-to-Left behaviour change and is deferred to PR 2. ReliableProviders’ seven per-provider factories are also unchanged here - they collapse to a single registry-routed wrap in PR 2. NamedProviderLoader, NamedProviderValidator, ProviderCapabilities, ProviderCapabilitiesRegistry and ProviderModelLister are all private[llm4s] or private[config], so their reshaping costs users nothing.

Slice 3: llm4s-speech

The last artifact of slice 3 (#1130) and the last package out of llm4s-core in this slice. It is in the build but not yet in a release, so nothing here affects 0.4.1 or earlier.

What moved

Packages	New module
org.llm4s.speech (and speech/io, processing, stt, tts, util)	llm4s-speech

Package names did not change, and there are no source breaks. 16 main and 21 test files moved whole; the only code outside the package that referenced it was a sample.

// Before
libraryDependencies += "org.llm4s" %% "llm4s-core" % version

// After - only if you use speech-to-text or text-to-speech
libraryDependencies ++= Seq(
  "org.llm4s" %% "llm4s-core"   % version,
  "org.llm4s" %% "llm4s-speech" % version
)

What llm4s-core sheds

Vosk (25 MB) and JNA. com.alphacephei:vosk is imported by exactly one file, speech/stt/VoskSpeechToText.scala, and until now sat on the classpath of every llm4s-core user, whether or not they had any use for offline speech recognition. It is the single largest dependency the carve programme has moved.

Deps.jna moves with it, but it is worth being precise about why, because it is not a second dependency: Vosk’s own POM already depends on net.java.dev.jna:jna:5.7.0. The explicit declaration exists to win that version conflict and pull 5.19.1 instead. Dropping it would not remove JNA - it would silently downgrade it to a release that predates Apple Silicon support.

llm4s-workspace-client also declared both, and used neither; those declarations are removed here too, so Vosk genuinely leaves the build for everyone who is not doing speech.

The Vosk resolver is deleted, not moved

build.sbt carried the project’s only third-party resolver:

resolvers += "Vosk Repository" at "https://alphacephei.com/maven/"

It resolved nothing. Vosk 0.3.45 publishes to Maven Central, which is where every build has actually been getting it - the local Coursier cache holds vosk-0.3.45.jar under repo1.maven.org and not a single artifact under alphacephei.com. What it did do was add a third-party host to the lookup path for every artifact in the build, including llm4s’s own inter-module jars, which produced a steady trickle of failed requests to alphacephei.com on every resolve.

So it is removed rather than carried into llm4s-speech, and the build now has no third-party resolvers at all.

Slice 3 is complete

With this, llm4s-core no longer contains rag, knowledgegraph, agent/memory, mcp, imagegeneration, imageprocessing or speech. What remains is the agent runtime, llmconnect, toolapi, config, trace and the provider clients, which slices 4 to 6 address.

Slice 3: llm4s-image

Part of slice 3 of the module carves tracked in #1126; the slice is #1130. It is in the build but not yet in a release, so nothing here affects 0.4.1 or earlier.

What moved

Packages	New module
org.llm4s.imagegeneration	llm4s-image
org.llm4s.imageprocessing	llm4s-image

Package names did not change, and this carve adds no source breaks of its own. The image API’s one source break - image formats becoming org.llm4s.media.MediaType - landed earlier, in llm4s-media, precisely so that this step is a pure file move.

The two packages move together because they are two halves of one subsystem: generate an image, then analyse or convert it. Both are built on the same media vocabulary, and splitting them would leave two artifacts nobody uses apart.

// Before
libraryDependencies += "org.llm4s" %% "llm4s-core" % version

// After - only if you generate images, or analyse them with a vision model
libraryDependencies ++= Seq(
  "org.llm4s" %% "llm4s-core"  % version,
  "org.llm4s" %% "llm4s-image" % version
)

llm4s-image brings llm4s-media with it, so MediaType is on your classpath either way.

What llm4s-core sheds

No third-party dependency: the image clients are built on Llm4sHttpClient, ujson/upickle and javax.imageio from the JDK, all of which core keeps for other reasons. What core sheds is 19 source files and about 3,600 lines of a subsystem most users never touch, along with its edge to llm4s-media - that edge existed only because the image packages were still inside core, and it leaves with them.

Core’s measured statement coverage rises from 74.05% to 74.89% as a result, since the image code was below core’s average.

One test moved with the code

org.llm4s.async.AsyncErrorHandlingSpec lived in core’s test tree under a package name that suggests it is about asynchrony in general. Every one of its assertions exercises an image client - it checks that ImageProcessingClient.analyzeImageAsync and the image generation clients’ Future { blocking { ... } }.recover { ... } pattern surface thrown exceptions as Left rather than as a failed Future. It moves to llm4s-image with the code it tests, keeping its package name.

Had it been left behind it would simply have stopped compiling - but the more useful point is that leaving it would have removed the only coverage of that behaviour from the module that owns it.

Slice 3: llm4s-media

A new module rather than a carve, landed as part of slice 3 (#1130) and ahead of llm4s-image and llm4s-speech, so those two carves can be pure file moves. It is in the build but not yet in a release, so nothing here affects 0.4.1 or earlier.

Why

A media type - a MIME string, a canonical file extension, and whether the thing is an image, audio, video or text - is the one piece of vocabulary every multimodal subsystem needs to name. Because there was nowhere shared to put it, each grew its own. llm4s-core shipped three overlapping enumerations of the same handful of image formats:

Type	Cases	Members
org.llm4s.imagegeneration.ImageFormat	PNG, JPEG, WEBP	extension, mimeType
org.llm4s.imageprocessing.ImageFormat	PNG, JPEG, WEBP, GIF	extension, mimeType
org.llm4s.imageprocessing.MediaType	Jpeg, Png, Gif, WebP, Bmp, Tiff	value

The first two are structurally identical and differ only in package, so a format produced by image generation could not be handed to image processing without a hand-written conversion. The third models the same six formats a third way, in the same package as the second. Meanwhile MediaExtractor in llm4s-rag discriminated on raw MIME prefixes (mimeType.startsWith ("image/")), with no type to name the answer at all.

Carving image and speech out of core without fixing this would have frozen three copies into three artifacts, where consolidating them later costs a cross-module source break rather than an in-module one.

What llm4s-media is

Vocabulary only - no I/O, no content sniffing, no third-party dependencies at all:

• org.llm4s.media.MediaType - mimeType, extension, category, plus fromExtension, fromPath and fromMimeType lookups. Sealed; refines into ImageMediaType and AudioMediaType so an image API can require an image without re-enumerating the cases.
• org.llm4s.media.MediaCategory - Image, Audio, Video, Text, Application, with fromMimeType.

Deciding what a file actually is from its bytes needs Tika and stays in llm4s-rag; that code produces a MIME string and resolves it here. That separation is what lets every consumer depend on llm4s-media without inheriting anything.

Source breaks

This is a source break, taken deliberately ahead of the 1.0 API freeze. All three types above are replaced by org.llm4s.media.MediaType.

Before	After
org.llm4s.imagegeneration.ImageFormat	org.llm4s.media.ImageMediaType
org.llm4s.imageprocessing.ImageFormat	org.llm4s.media.ImageMediaType
org.llm4s.imageprocessing.MediaType	org.llm4s.media.MediaType
ImageFormat.PNG	MediaType.Png
ImageFormat.JPEG	MediaType.Jpeg
ImageFormat.WEBP	MediaType.WebP
ImageFormat.GIF	MediaType.Gif
MediaType.Jpeg.value	MediaType.Jpeg.mimeType

// Before
import org.llm4s.imagegeneration.ImageFormat
val opts = ImageGenerationOptions(format = ImageFormat.PNG)

// After
import org.llm4s.media.MediaType
val opts = ImageGenerationOptions(format = MediaType.Png)

Two lookups changed shape as well. org.llm4s.imageprocessing.MediaType.fromExtension and .fromPath were total, silently returning JPEG for anything they did not recognise - so a .txt file reported as an image and the caller could not tell. The replacements return Option, and callers that genuinely want the old fallback ask for it:

// Before
val mt = MediaType.fromPath(path)                                   // JPEG if unrecognised

// After
val mt = MediaType.imageFromPath(path).getOrElse(MediaType.Jpeg)    // fallback is now visible

AnthropicVisionClient.detectMediaType keeps the old behaviour and its old signature shape - it still answers JPEG for an unrecognised extension, because that is what the Anthropic API assumes for an unlabelled image - but now returns an ImageMediaType.

Adding the dependency

Nothing to add today: llm4s-media arrives as a transitive dependency of llm4s-core (via the image packages, which are still in core) and of llm4s-rag. Declare it directly only if you name MediaType or MediaCategory in your own signatures.

libraryDependencies += "org.llm4s" %% "llm4s-media" % version

Slice 3: llm4s-mcp

Third of the module carves tracked in #1126; slice 3 is #1130, which carves three independent subsystems - mcp, image and speech - one artifact at a time. This note covers mcp; the other two follow. It is in the build but not yet in a release, so nothing here affects 0.4.1 or earlier.

What moved

Packages	New module
org.llm4s.mcp	llm4s-mcp

Package names did not change, and there are no source breaks. Nothing outside org.llm4s.mcp referenced it, so the whole package moved with no facade left behind.

// Before
libraryDependencies += "org.llm4s" %% "llm4s-core" % version

// After - only if you use the Model Context Protocol client, server or tool registry
libraryDependencies ++= Seq(
  "org.llm4s" %% "llm4s-core" % version,
  "org.llm4s" %% "llm4s-mcp"  % version
)

What llm4s-core sheds

Java-WebSocket. Worth being precise about why, because it is not what #1130 predicted: MCP does not use WebSockets at all. Its transports are stdio, HTTP and SSE, built on Llm4sHttpClient and com.sun.net.httpserver. Deps.websocket was declared on llm4s-core and imported by nothing in it - the only WebSocket code in the repo is ContainerisedWorkspace in llm4s-workspace-client, which declares the dependency itself. So core sheds it by dropping a declaration that was never used, and the dependency does not follow mcp anywhere.

If you depend on llm4s-core and were picking up org.java-websocket transitively, declare it yourself.

Configuration keys

None. org.llm4s.mcp reads no reference.conf keys and no Llm4sConfig method names a type that moved.

---

Slice 2: llm4s-memory and llm4s-memory-postgres

Second of the module carves tracked in #1126; slice 2 is #1129. It is in the build but not yet in a release, so nothing here affects 0.4.1 or earlier.

What moved

Packages	New module
org.llm4s.agent.memory, except PostgresMemoryStore	llm4s-memory
org.llm4s.agent.memory.PostgresMemoryStore	llm4s-memory-postgres

Package names did not change, and there are no source breaks in this slice. Nothing outside org.llm4s.agent.memory referenced it, so the whole package moved with no facade left behind. Add the dependency; your imports stay as they are.

// Before
libraryDependencies += "org.llm4s" %% "llm4s-core" % version

// After — only if you use agent memory
libraryDependencies ++= Seq(
  "org.llm4s" %% "llm4s-core"   % version,
  "org.llm4s" %% "llm4s-memory" % version
)

// ...and only if you store memories in Postgres/pgvector
libraryDependencies += "org.llm4s" %% "llm4s-memory-postgres" % version

Why two artifacts

PostgresMemoryStore was the only file in the package that needed a connection pool and a server-side driver. Shipping it alongside InMemoryStore would mean every user of agent memory inherits HikariCP and the Postgres JDBC driver whether or not they ever open a connection. llm4s-memory carries sqlite-jdbc — for the file-backed SQLiteMemoryStore and VectorMemoryStore — and nothing else; llm4s-memory-postgres depends on llm4s-memory and adds the two heavy dependencies.

What llm4s-core sheds

HikariCP and the Postgres JDBC driver leave the core classpath. sqlite-jdbc leaves too: all three used to be declared in the build’s shared settings, which put them on every module’s classpath, so core could not shed them by itself. They are now declared only by the modules that open a connection. If you depend on llm4s-core and use any of the three directly, declare them yourself rather than relying on the transitive edge.

org.llm4s.vectorstore.PostgresVectorHelpers

Unchanged for callers — same package, same object, same methods — but worth knowing where it ships. It is the pgvector text codec ([0.1,0.2,0.3] ⇄ Array[Float]), it names no JDBC type, and it now has consumers in two modules that do not and should not depend on each other: PgVectorStore in llm4s-rag and PostgresMemoryStore in llm4s-memory-postgres. Rather than have either reach for the other, the single copy lives in llm4s-core, which both already depend on. Slice 1 had briefly moved it into llm4s-rag and left a private duplicate in core for PostgresMemoryStore; that duplicate is now gone.

So org.llm4s.vectorstore is split across two jars: this one object in llm4s-core, the rest in llm4s-rag. It resolves the same way on any ordinary classpath.

Configuration keys

None. agent/memory reads no reference.conf keys and no Llm4sConfig method returns a type that moved, so there is nothing to migrate.

---

Slice 1: llm4s-rag and llm4s-knowledgegraph

First of the module carves tracked in #1126; slice 1 is #1128. It is in the build but not yet in a release, so nothing here affects 0.4.1 or earlier.

What moved

Packages	New module
org.llm4s.rag, org.llm4s.vectorstore, org.llm4s.chunking, org.llm4s.reranker, org.llm4s.eval, org.llm4s.extract, org.llm4s.knowledgegraph.graphrag	llm4s-rag
org.llm4s.knowledgegraph (everything except graphrag)	llm4s-knowledgegraph

Package names did not change, with the one deliberate exception described under Source breaks below. Add the dependency; your imports stay as they are.

// Before
libraryDependencies += "org.llm4s" %% "llm4s-core" % version

// After — only if you use RAG, vector stores, chunking, reranking or the knowledge graph
libraryDependencies ++= Seq(
  "org.llm4s" %% "llm4s-core" % version,
  "org.llm4s" %% "llm4s-rag"  % version   // depends on llm4s-knowledgegraph transitively
)

llm4s-rag depends on llm4s-knowledgegraph, so depending on the graph alone is only worth doing if you want the graph without RAG.

llm4s-knowledgegraph-neo4j — a separate, already-published artifact — now depends on llm4s-knowledgegraph instead of llm4s-core, which resolves for you.

What llm4s-core sheds

Six dependencies leave the core classpath: Tika, POI, PDFBox, jsoup, AWS S3 and AWS STS. If you depend on llm4s-core and use any of those directly, declare them yourself rather than relying on the transitive edge.

Source breaks

Three, all of them in this slice on purpose — pre-1.0 is when a duplicate is cheapest to remove.

1. The two document extractors are now one. org.llm4s.rag.extract.DocumentExtractor and org.llm4s.llmconnect.extractors.UniversalExtractor were independent implementations of one job: two Tika instances, two sets of MIME constants, two PDFBox paths, two POI paths. They are now org.llm4s.extract.

Before	After
org.llm4s.rag.extract.DocumentExtractor	org.llm4s.extract.DocumentExtractor
org.llm4s.rag.extract.DefaultDocumentExtractor	org.llm4s.extract.TikaDocumentExtractor
UniversalExtractor.extract(path) → Either[ExtractorError, String]	TikaDocumentExtractor.extractFromPath(path) → Result[ExtractedDocument] (text in .text)
UniversalExtractor.extractFromBytes(bytes, name, mime)	TikaDocumentExtractor.extract(bytes, name, mime)
UniversalExtractor.extractFromStream(in, name, mime)	TikaDocumentExtractor.extractFromStream(in, name, mime) (returns ExtractedDocument)
UniversalExtractor.isTextLike(mime)	TikaDocumentExtractor.canExtract(mime) — also true for legacy .doc
UniversalExtractor.detectMimeType(bytes, name)	unchanged
UniversalExtractor.extractAny(path) and its Extracted / TextContent / ImageContent / AudioContent / VideoContent ADT	org.llm4s.extract.MediaExtractor
org.llm4s.llmconnect.model.ExtractorError	org.llm4s.error.ProcessingError

The package is org.llm4s.extract, not org.llm4s.rag.extract: extraction has two real consumers — RAG document loading and multimodal embedding — and it quarantines the three heaviest dependencies in the build. Naming it outside the rag namespace makes any later decision to give it its own artifact a build-file change rather than a code change.

2. EmbeddingClient.encodePath is now FileEmbedder.encodeFromPath. EmbeddingClient keeps the pure vector API; file reading, MIME sniffing and chunking live in org.llm4s.rag.embed. The six-parameter signature — which included an experimentalStubsEnabled: Boolean, a deployment decision arriving at a call site — became a config object.

// Before
client.encodePath(path, textModel, chunkingCfg, stubsEnabled, localModels)

// After
import org.llm4s.rag.embed.{ FileEmbedder, FileEmbeddingConfig, TextChunkingConfig }

FileEmbedder.encodeFromPath(
  path,
  client,
  FileEmbeddingConfig(
    textModel = textModel,
    localModels = localModels,
    chunking = TextChunkingConfig(enabled = true, size = 1000, overlap = 100),
    experimentalStubs = stubsEnabled
  )
)

UniversalEncoder.TextChunkingConfig is now the top-level org.llm4s.rag.embed.TextChunkingConfig.

3. Llm4sConfig.pgSearchIndex() is now PgSearchIndexConfigLoader.default(). It returned a SearchIndex.PgConfig, which is RAG’s type; Llm4sConfig stays in llm4s-core and cannot name it. The loader itself keeps its package (org.llm4s.config) and its load(source) method, and moves to llm4s-rag.

// Before
val pg = Llm4sConfig.pgSearchIndex()

// After
import org.llm4s.config.PgSearchIndexConfigLoader
val pg = PgSearchIndexConfigLoader.default()

Configuration keys

llm4s.rag.permissions.pg.* and llm4s.rerank.* now ship in llm4s-rag’s reference.conf rather than core’s. HOCON merges reference.conf across jars, so the key paths are unchanged and nothing in your application.conf needs editing — but a build that reads those keys without depending on llm4s-rag no longer gets the defaults.

llm4s.embeddings.* — including chunking and experimentalStubs — stays in core, because Llm4sConfig still reads it there.

---

Artifact coordinate rename (v0.4.0)

Breaking change

Every published artifact under the org.llm4s group was renamed to carry an llm4s- prefix and a consistent kebab-case suffix. This is a coordinate-only change: there are no API changes, no package moves and no source changes in this release. Update your build.sbt, recompile, and you are done.

Old coordinate	New coordinate
"org.llm4s" %% "core"	"org.llm4s" %% "llm4s-core"
"org.llm4s" %% "workspaceShared" (published as workspaceshared)	"org.llm4s" %% "llm4s-workspace-shared"
"org.llm4s" %% "workspaceClient" (published as workspaceclient)	"org.llm4s" %% "llm4s-workspace-client"
"org.llm4s" %% "trace-opentelemetry"	"org.llm4s" %% "llm4s-observability-otel"
"org.llm4s" %% "knowledgegraph-neo4j"	"org.llm4s" %% "llm4s-knowledgegraph-neo4j"

Maven users: the artifactId gains the same prefix, so core_3 becomes llm4s-core_3 (and core_2.13 becomes llm4s-core_2.13).

Why

Two reasons:

1. Consistency. org.llm4s:core is a poor coordinate to read in somebody else’s build file, and the module names were an inconsistent mix of camelCase (silently lowercased by the publish into workspaceclient) and kebab-case.
2. Escaping a bad publish. The core_3 / core_2.13 artifacts carry an accidental mis-published version 2.1.593 (a typo). Maven Central publishes are immutable, so that version cannot be retracted, and some resolvers sort it as the “latest” release. A fresh artifact name is the only way out; documentation is not.

Nobody is stranded

Releases up to and including 0.3.4 remain published, unchanged and resolvable under the old coordinates. Pinning "org.llm4s" %% "core" % "0.3.4" keeps working indefinitely — you only need to change coordinates when you move to 0.4.0 or later.

If you are pinning core with a floating or range version, pin an explicit 0.3.4 before upgrading, so the phantom 2.1.593 is never selected.

Migration steps

1. Replace the old coordinate with the new one in your build.sbt (see the table above).
2. Set the version to 0.4.0 or later.
3. Recompile. No imports, types or method signatures changed.

// Before
libraryDependencies += "org.llm4s" %% "core" % "0.3.4"

// After
libraryDependencies += "org.llm4s" %% "llm4s-core" % "0.4.0"

Note on "org.llm4s" %% "llm4s"

The aggregate llm4s artifact (llm4s_3 / llm4s_2.13) is not published and has not been since 0.2.9 — the root project sets publish / skip := true. Any build file or documentation that depends on "org.llm4s" %% "llm4s" is wrong independently of this rename and should be changed to "org.llm4s" %% "llm4s-core".

Note on llm4s-observability-otel

The OpenTelemetry integration is published as llm4s-observability-otel rather than llm4s-trace-opentelemetry. The name anticipates the llm4s-observability module that the modularisation work will carve out of trace + metrics, so the integration is named once rather than twice.

Unpublished modules

samples, workspaceRunner, workspaceSamples, config-policy, it and benchmarks set publish / skip := true and never reached Maven Central. Their name values were made consistent in the same change, but this has no effect on any downstream build.

---

MessageRole Enum Changes (v0.2.0)

Breaking Change

The MessageRole has been converted from string-based constants to a proper enum type for better type safety.

Before (v0.1.x)

import org.llm4s.llmconnect.model.Message

val message = Message(role = "assistant", content = "Hello")
message.role match {
  case "assistant" => // handle assistant
  case "user" => // handle user
  case _ => // handle other
}

After (v0.2.0)

import org.llm4s.llmconnect.model.{Message, MessageRole}

val message = AssistantMessage(content = "Hello")
// or
val message = Message(role = MessageRole.Assistant, content = "Hello")

message.role match {
  case MessageRole.Assistant => // handle assistant
  case MessageRole.User => // handle user
  case MessageRole.System => // handle system
  case MessageRole.Tool => // handle tool
}

Migration Steps

1. Update imports: Add MessageRole to your imports

   import org.llm4s.llmconnect.model.MessageRole
   

2. Replace string comparisons: Update pattern matches and comparisons

   // Before
   if (message.role == "assistant") { ... }
      
   // After
   if (message.role == MessageRole.Assistant) { ... }
   

3. Update message creation: Use the typed constructors

   // Before
   Message(role = "user", content = "Hello")
      
   // After
   UserMessage(content = "Hello")
   // or
   Message(role = MessageRole.User, content = "Hello")
   

Error Hierarchy Changes (v0.2.0)

New Error Categorization

Errors are now categorized using traits for better type safety and recovery strategies.

Before (v0.1.x)

error match {
  case e: LLMError if e.isRecoverable => // retry logic
  case e: LLMError => // handle non-recoverable
}

After (v0.2.0)

error match {
  case e: RecoverableError => // retry logic
  case e: NonRecoverableError => // handle non-recoverable
}

Error Recovery Pattern

import org.llm4s.error._

def handleError(error: LLMError): Unit = error match {
  case _: RateLimitError => // wait and retry
  case _: TimeoutError => // retry with backoff
  case _: ServiceError with RecoverableError => // retry
  case _: AuthenticationError => // refresh token or fail
  case _: ValidationError => // fix input and retry
  case _ => // non-recoverable, fail
}

Migration Steps

1. Replace isRecoverable checks: Use pattern matching on traits

   // Before
   if (error.isRecoverable) { ... }
      
   // After
   error match {
     case _: RecoverableError => { ... }
     case _ => { ... }
   }
   

2. Update error handling: Use the new trait-based categorization

   // Before
   case e: ServiceError if e.isRecoverable =>
      
   // After
   case e: ServiceError with RecoverableError =>
   

3. Use smart constructors: Create errors using the companion object methods

   // Before
   new RateLimitError(429, "Rate limit exceeded", Some(60.seconds))
      
   // After
   RateLimitError(429, "Rate limit exceeded", Some(60.seconds))
   

Configuration Changes (v0.2.0+)

Llm4sConfig.provider() with no argument, used below, has since become Llm4sConfig.defaultProvider(), and providers are configured as named sections - see From LLM_MODEL to named provider sections.

EnvLoader and legacy ConfigReader → Llm4sConfig

Older versions used EnvLoader and a custom ConfigReader abstraction. These have been superseded by Llm4sConfig (PureConfig‑based) and typed helpers.

Before (v0.1.x)

import org.llm4s.config.EnvLoader

val apiKey = EnvLoader.get("OPENAI_API_KEY")
val model  = EnvLoader.getOrElse("LLM_MODEL", "gpt-4")

or:

import org.llm4s.config.ConfigReader
import org.llm4s.llmconnect.LLMConnect

val client: org.llm4s.types.Result[org.llm4s.llmconnect.LLMClient] =
  ConfigReader.Provider().flatMap(LLMConnect.getClient)

After (post‑0.2.0)

import org.llm4s.config.Llm4sConfig
import org.llm4s.llmconnect.LLMConnect

val client: org.llm4s.types.Result[org.llm4s.llmconnect.LLMClient] =
  for {
    cfg    <- Llm4sConfig.provider()
    client <- LLMConnect.getClient(cfg)
  } yield client

Typed Config: recommended patterns

• Tracing (typed):

  import org.llm4s.config.Llm4sConfig
  import org.llm4s.trace.{ Tracing, EnhancedTracing, TracingMode }
  
  val tracerResult: org.llm4s.types.Result[Tracing] =
    Llm4sConfig.tracing().map(Tracing.create)
  

• Provider model for display (typed):

  val modelNameResult = Llm4sConfig.provider().map(_.model)
  // Prefer completion.model after the API call when available
  

• Workspace (samples):

  import org.llm4s.codegen.WorkspaceConfigSupport
  
  val ws = WorkspaceConfigSupport.load().getOrElse(
    throw new IllegalArgumentException("Failed to load workspace settings")
  )
  

• Embeddings (samples):

  val ui      = org.llm4s.samples.embeddingsupport.EmbeddingUiSettings.loadFromEnv()
    .getOrElse(throw new IllegalArgumentException("Failed to load UI settings"))
  val targets = org.llm4s.samples.embeddingsupport.EmbeddingTargets.loadFromEnv()
    .fold(err => throw new IllegalArgumentException(err.toString), _.targets)
  val query   = org.llm4s.samples.embeddingsupport.EmbeddingQuery.loadFromEnv()
    .fold(_ => None, _.value)
  

Configuration: legacy reader → Llm4sConfig / typed helpers (post‑0.2.0)

Earlier versions used a custom ConfigReader-style abstraction as a catch‑all for configuration. With PureConfig in place and typed helpers available, the preferred path is now:

• Use org.llm4s.config.Llm4sConfig in core code.
• Use explicit typed loaders plus LLMConnect.getClient in application/sample code.

Provider configuration and client creation

Before (legacy reader-based API)

import org.llm4s.config.ConfigReader
import org.llm4s.llmconnect.LLMConnect

val client: org.llm4s.types.Result[org.llm4s.llmconnect.LLMClient] =
  ConfigReader.Provider().flatMap(LLMConnect.getClient)

After

import org.llm4s.config.Llm4sConfig
import org.llm4s.llmconnect.LLMConnect

// Typed path using Llm4sConfig
val client: org.llm4s.types.Result[org.llm4s.llmconnect.LLMClient] =
  for {
    cfg    <- Llm4sConfig.provider()
    client <- LLMConnect.getClient(cfg)
  } yield client

Tracing configuration

Before (legacy reader-based API)

import org.llm4s.config.ConfigReader
import org.llm4s.trace.Tracing

val tracer: Tracing =
  ConfigReader.TracingConf().map(Tracing.create).getOrElse(Tracing.noop)

After

import org.llm4s.config.Llm4sConfig
import org.llm4s.trace.Tracing

val tracer: org.llm4s.types.Result[Tracing] =
  Llm4sConfig.tracing().map(Tracing.create)

Embeddings: provider and client

Before (legacy reader-based API)

import org.llm4s.config.ConfigReader
import org.llm4s.llmconnect.EmbeddingClient

val client: org.llm4s.types.Result[EmbeddingClient] =
  ConfigReader.Embeddings().flatMap { case (provider, cfg) =>
    EmbeddingClient.from(provider, cfg)
  }

After

import org.llm4s.config.Llm4sConfig
import org.llm4s.llmconnect.EmbeddingClient

val client: org.llm4s.types.Result[EmbeddingClient] =
  Llm4sConfig.embeddings().flatMap { case (provider, cfg) =>
    EmbeddingClient.from(provider, cfg)
  }

Workspace settings

Before

import org.llm4s.codegen.WorkspaceSettings

val ws = WorkspaceSettings.load().getOrElse(
  throw new IllegalArgumentException("Failed to load workspace settings")
)

After

import org.llm4s.codegen.WorkspaceConfigSupport

val ws = WorkspaceConfigSupport.load().getOrElse(
  throw new IllegalArgumentException("Failed to load workspace settings")
)

API keys and types

Before (legacy reader-based API)

// Legacy pattern: API key resolved from a generic config reader
def loadApiKey(reader: /* legacy ConfigReader */ Any): Result[ApiKey] =
  ApiKey.unsafe("sk-legacy-key") // placeholder for old behavior

After

import org.llm4s.config.Llm4sConfig

val cfgResult = Llm4sConfig.provider() // Result[ProviderConfig]

• For new code, do not introduce new parameters of reader/ConfigReader types. Prefer:
  • Llm4sConfig in core libraries.
  • Typed helpers plus LLMConnect.getClient (and Llm4sConfig.tracing().map(Tracing.create) / .map(EnhancedTracing.create) for tracing) in applications and samples.
• For existing code that currently depends on a ConfigReader-style abstraction:
  • Start by swapping call sites to use typed helpers (e.g., Llm4sConfig.provider()).
  • Where you need fine-grained control, switch to Llm4sConfig functions instead of calling the legacy reader directly.