org.llm4s.javaapi

Members list

Type members

Classlikes

final class AgentStream

A streaming agent turn from JAgent.stream, JAgent.streamResume or JAgent.streamRecover, whose events are being delivered to its AgentStreamListener: cancel it, or await its result.

A streaming agent turn from JAgent.stream, JAgent.streamResume or JAgent.streamRecover, whose events are being delivered to its AgentStreamListener: cancel it, or await its result.

AgentStream stream = agent.stream(threadId, "Explain monads", listener).get();
LlmResult<JAgentResult> result = stream.await();   // or stream.cancel()

Attributes

Source
AgentStream.scala
Supertypes
class Object
trait Matchable
class Any

Receives an agent turn's events from JAgent.stream, JAgent.streamResume or JAgent.streamRecover: each event in order through onEvent, then exactly one of onComplete (the turn's result) or onError (the error it failed with). Every call is made on the stream's own thread - never the caller's, and never the runtime's - one at a time.

Receives an agent turn's events from JAgent.stream, JAgent.streamResume or JAgent.streamRecover: each event in order through onEvent, then exactly one of onComplete (the turn's result) or onError (the error it failed with). Every call is made on the stream's own thread - never the caller's, and never the runtime's - one at a time.

Only onEvent is abstract, so a Java lambda is a listener:

LlmResult<AgentStream> started = agent.stream(threadId, "Explain monads", event ->
   StreamEvents.decode(AgentEvents.TextDelta(), event).ifPresent(d -> System.out.print(d.text())));
LlmResult<JAgentResult> result = started.get().await();

A listener slower than the stream never holds the turn up: live events (text deltas, tool progress) that do not fit in the stream's buffer are dropped, and the listener receives one StreamEvent.LiveGap with their count where they were dropped. Durable events are kept, up to Agent.StreamCapacity (1024) waiting for the listener; one more ends the stream through onError. A listener that throws from onEvent cancels the turn, and onError receives what it threw; one that sets its own thread's interrupt flag cancels it too, at once, and receives a CancelledError.

Attributes

Source
AgentStreamListener.scala
Supertypes
class Object
trait Matchable
class Any
final class Answer

An answer to one pending approval, question or breakpoint of a suspended turn, for JAgent.resume or JAgent.streamResume: built with Answer.approve, Answer.reject, Answer.edit, Answer.reply or Answer.proceed, with JSON as a String. The id is a PendingInterrupt's, from JAgent.pending.

An answer to one pending approval, question or breakpoint of a suspended turn, for JAgent.resume or JAgent.streamResume: built with Answer.approve, Answer.reject, Answer.edit, Answer.reply or Answer.proceed, with JSON as a String. The id is a PendingInterrupt's, from JAgent.pending.

agent.resume(threadId, List.of(Answer.approve(pending.id())));

A null argument or malformed JSON is reported when the answer is used: resume and streamResume return a failed result.

Attributes

Companion
object
Source
Answer.scala
Supertypes
class Object
trait Matchable
class Any
object Answer

Attributes

Companion
class
Source
Answer.scala
Supertypes
class Object
trait Matchable
class Any
Self type
Answer.type
final class CompletionStream extends AutoCloseable

A completion being streamed from JLlmClient.stream to its CompletionStreamListener: cancel it, or await its reply. It is AutoCloseable, and close() cancels, so a try-with-resources block stops a call its body leaves.

A completion being streamed from JLlmClient.stream to its CompletionStreamListener: cancel it, or await its reply. It is AutoCloseable, and close() cancels, so a try-with-resources block stops a call its body leaves.

try (CompletionStream stream = client.stream(conversation, piece -> ui.append(piece)).get()) {
   LlmResult<JCompletion> reply = stream.await();
}

Attributes

Source
CompletionStream.scala
Supertypes
class Object
trait Matchable
class Any

Receives a streamed completion from JLlmClient.stream: each piece of the reply's text, in order, through onPartial, then exactly one of onComplete (the whole reply) or onError (the error the call failed with, a cancellation included). Calls are made one at a time, never on the caller's thread.

Receives a streamed completion from JLlmClient.stream: each piece of the reply's text, in order, through onPartial, then exactly one of onComplete (the whole reply) or onError (the error the call failed with, a cancellation included). Calls are made one at a time, never on the caller's thread.

Only onPartial is abstract, so a Java lambda is a listener:

CompletionStream stream = client.stream("Tell me a story", System.out::print).get();
LlmResult<JCompletion> reply = stream.await();

A listener that throws from onPartial stops the call (its thread is interrupted, which is how llm4s providers are cancelled), and onError receives what it threw.

Attributes

Source
CompletionStreamListener.scala
Supertypes
class Object
trait Matchable
class Any
final class ConversationBuilder

Builder for constructing a Conversation without using Scala case-class syntax or sequence literals.

Builder for constructing a Conversation without using Scala case-class syntax or sequence literals.

Conversation conv = ConversationBuilder.create()
   .system("You are a helpful assistant.")
   .user("What is the capital of France?")
   .build();

Instances are immutable: every method returns a new builder, so a builder may be shared and forked across threads. A null content argument throws NullPointerException immediately.

Attributes

Companion
object
Source
ConversationBuilder.scala
Supertypes
class Object
trait Matchable
class Any

Attributes

Companion
class
Source
ConversationBuilder.scala
Supertypes
class Object
trait Matchable
class Any
Self type
final class JAgent

Java-friendly wrapper around Agent.

Java-friendly wrapper around Agent.

Exposes run(query) and continueConversation(previous, query), returning LlmResult[JAgentResult] so Java callers do not need to deal with Scala's Either or Result types directly, nor read a Scala value: a JAgentResult's accessors return Strings, JDK types and this package's own types. A conversation is a thread of the agent's in-memory runtime, kept until forget removes it.

run, continueConversation, resume and recover block the calling thread and never throw. Interrupting that thread cancels the turn: the call returns a failed result whose error is a CancelledError, with the interrupt flag still set, once the turn has ended (it waits up to 5 seconds for the turn to end, for a provider that ignores its interrupt), and the conversation's thread is left for recover. A thread already interrupted starts no turn. InterruptedException is never thrown, so the methods declare no checked exception and javac rejects a catch (InterruptedException e) around them - test the result for a CancelledError instead. To cancel a turn without interrupting a thread, stream it and call AgentStream.cancel.

A turn whose tools need approval, whose tools or middleware ask a question, or whose task a static breakpoint holds, ends SUSPENDED: its status's pending() - or JAgent.pending - lists what it waits for, and resume(threadId, answers) answers some or all of it and continues. A turn that failed or was cancelled continues with recover(threadId). Both block, as run does.

LlmResult<JAgentResult> turn = agent.run("Deploy the release");
List<Answer> answers = new ArrayList<>();
for (PendingInterrupt p : turn.get().status().pending()) answers.add(Answer.approve(p.id()));
if (!answers.isEmpty()) turn = agent.resume(turn.get().threadId(), answers);

stream, streamResume and streamRecover run a turn on a thread you name and hand its events to an AgentStreamListener as they happen, returning an AgentStream at once - to await the turn's result, or to cancel it. From Java a thread id is a String.

runAsync, continueConversationAsync, resumeAsync and recoverAsync return a CompletableFuture at once and run the turn on another thread - a new virtual thread per turn, or the executor withAsyncExecutor names. The future completes with the JAgentResult, or exceptionally with the LlmException the blocking call's result would hold. cancel(true) interrupts the turn, which then ends as the blocking call interrupted does: cancelled, the thread of a continued, resumed or recovered conversation left for recover.

Obtain instances via Llm4s.createAgent.

JAgent agent = Llm4s.createAgent(client);
LlmResult<JAgentResult> result = agent.run("Summarise the news today");
result.ifSuccess(r -> System.out.println(r.answer().orElse("(" + r.status().kind() + ")")))
     .ifFailure(e -> System.err.println(e.getMessage()));

Attributes

Companion
object
Source
JAgent.scala
Supertypes
class Object
trait Matchable
class Any
object JAgent

Attributes

Companion
class
Source
JAgent.scala
Supertypes
class Object
trait Matchable
class Any
Self type
JAgent.type
final class JAgentBuilder

An agent under construction, from Llm4s.agentBuilder (#2476): its system prompt, tools, guardrails, context window and streaming, set from Java with Java types only.

An agent under construction, from Llm4s.agentBuilder (#2476): its system prompt, tools, guardrails, context window and streaming, set from Java with Java types only.

LlmResult<JAgent> agent = Llm4s.agentBuilder("support", client)
   .systemPrompt("You are a support assistant.")
   .tools(tools)
   .inputGuardrails(List.of(JGuardrails.length(1, 10_000)))
   .outputGuardrails(List.of(JGuardrails.json()))
   .contextWindow(JContextWindow.lastMessages(20))
   .streaming(true)
   .build();

Immutable: every method returns a new builder, and each setting replaces the one set before. No method throws: a null argument, and anything build cannot make an agent of, is a failed result of build().

Attributes

Source
JAgentBuilder.scala
Supertypes
class Object
trait Matchable
class Any
final class JCompletionOptions

The options of one completion request, built for Java and Kotlin callers and passed to JLlmClient.complete(Conversation, JCompletionOptions).

The options of one completion request, built for Java and Kotlin callers and passed to JLlmClient.complete(Conversation, JCompletionOptions).

JCompletionOptions options = JCompletionOptions.builder()
   .temperature(0.2)
   .maxTokens(512)
   .reasoning(JReasoningEffort.HIGH)
   .build();
LlmResult<String> answer = client.complete(conversation, options);

A setting left unset keeps core's default: temperature 0.7, top-p 1.0, no presence or frequency penalty, and no token limit, reasoning level or thinking budget. A value that may be absent is read as an Optional or OptionalInt, and set either directly or with one, so an empty Optional clears it. No Scala type appears in a signature.

A value: two are equal when every setting is. Tools and response formats are not options here yet.

Attributes

Companion
object
Source
JCompletionOptions.scala
Supertypes
class Object
trait Matchable
class Any

Attributes

Companion
class
Source
JCompletionOptions.scala
Supertypes
class Object
trait Matchable
class Any
Self type
final class JContextWindow

How much of a conversation an agent sends the model on each call, for JAgentBuilder.contextWindow (#2476): the Java form of ContextWindowMiddleware. It prunes only what is sent; the conversation keeps every message, so JAgentResult.messages() is whole.

How much of a conversation an agent sends the model on each call, for JAgentBuilder.contextWindow (#2476): the Java form of ContextWindowMiddleware. It prunes only what is sent; the conversation keeps every message, so JAgentResult.messages() is whole.

Llm4s.agentBuilder("support", client).contextWindow(JContextWindow.lastMessages(20))

Attributes

Companion
object
Source
JContextWindow.scala
Supertypes
class Object
trait Matchable
class Any

Attributes

Companion
class
Source
JContextWindow.scala
Supertypes
class Object
trait Matchable
class Any
Self type
final class JEmbeddingClient

Java-friendly wrapper around core's EmbeddingClient: turns texts into vectors, for semantic search and similarity, with no Scala type in sight.

Java-friendly wrapper around core's EmbeddingClient: turns texts into vectors, for semantic search and similarity, with no Scala type in sight.

JEmbeddingClient embedder = Llm4s.createDefaultEmbeddingClient().get();
JEmbeddings embeddings = embedder.embed(List.of("The cat sat on the mat.", "A kitten lay on the rug.")).get();
List<float[]> vectors = embeddings.vectors();
System.out.println(JEmbeddings.cosineSimilarity(vectors.get(0), vectors.get(1)));

Obtain one with Llm4s.createDefaultEmbeddingClient, which reads the model from llm4s.embeddings.model (EMBEDDING_MODEL), with Llm4s.createEmbeddingClient for a model you name, or wrap a Scala EmbeddingClient you built yourself with Llm4s.wrapEmbeddingClient.

Every embed sends all its texts to the provider in one request, blocks the calling thread and never throws: a failure - of the provider, the network or an argument - comes back inside the LlmResult, and its getKind() says which. An embedding provider's error response reads by its HTTP status, as a chat provider's does: 401 and 403 as AUTHENTICATION, 429 as RATE_LIMIT, 400 as VALIDATION, any other as SERVICE; a provider that cannot be reached reads as NETWORK, and one that does not answer in time as TIMEOUT. Interrupting the blocked thread returns a failed result whose kind is CANCELLED, with the thread's interrupt flag still set; InterruptedException is never thrown. embedAsync makes the same request on another thread and returns a CompletableFuture at once.

Attributes

Companion
object
Source
JEmbeddingClient.scala
Supertypes
class Object
trait Matchable
class Any

Attributes

Companion
class
Source
JEmbeddingClient.scala
Supertypes
class Object
trait Matchable
class Any
Self type
final class JGuardrail extends JOutputGuardrail

A built-in guardrail that checks an agent's query, for JAgentBuilder.inputGuardrails, or its final answer, for JAgentBuilder.outputGuardrails (#2476). Obtain one from JGuardrails.

A built-in guardrail that checks an agent's query, for JAgentBuilder.inputGuardrails, or its final answer, for JAgentBuilder.outputGuardrails (#2476). Obtain one from JGuardrails.

Attributes

Source
JGuardrails.scala
Supertypes
class Object
trait Matchable
class Any
object JGuardrails

The built-in guardrails, for an agent built with Llm4s.agentBuilder (#2476).

The built-in guardrails, for an agent built with Llm4s.agentBuilder (#2476).

Llm4s.agentBuilder("support", client)
   .inputGuardrails(List.of(JGuardrails.length(1, 10_000), JGuardrails.profanity()))
   .outputGuardrails(List.of(JGuardrails.json()))

A guardrail that fails blocks the turn: it ends with status BLOCKED, naming the guardrail and its reason, and the conversation stays usable. No factory throws: an invalid argument (a negative length, a null pattern) gives a guardrail that the agent's build() refuses with a failed result naming the argument.

Attributes

Source
JGuardrails.scala
Supertypes
class Object
trait Matchable
class Any
Self type
final class JLlmClient extends AutoCloseable

Java-friendly wrapper around LLMClient.

Java-friendly wrapper around LLMClient.

The underlying Scala client returns Result[Completion]; this wrapper surfaces it as LlmResult so Java callers never import any Scala types. complete returns the reply's text alone; completion returns the whole reply as a JCompletion - the text, the model that answered, its tool calls, its token usage and estimated cost.

Every complete and completion blocks the calling thread and never throws: a failure comes back inside the LlmResult. Interrupting the thread blocked in a call returns a failed result whose error is a CancelledError, with the thread's interrupt flag still set; InterruptedException is never thrown. The methods therefore declare no checked exception, and javac rejects a catch (InterruptedException e) around them as "never thrown in body of corresponding try statement" - test the result instead:

LlmResult<String> result = client.complete("Long question");
if (result.isFailure() && result.getError().getKind() == LlmErrorKind.CANCELLED) {
   // interrupted while blocked; Thread.currentThread().isInterrupted() is still true
}

Each blocking call has an *Async variant - completeAsync, completionAsync - that returns a CompletableFuture at once and makes the call on another thread: by default a virtual thread of this client's own virtual-thread-per-task executor, which close() shuts down; withAsyncExecutor names another executor. The future completes with the value, or exceptionally with the LlmException the blocking call's result would hold. cancel(true) - and any other completion from outside, such as orTimeout - interrupts the call:

CompletableFuture<String> a = client.completeAsync("Summarise the first report");
CompletableFuture<String> b = client.completeAsync("Summarise the second report");
CompletableFuture.allOf(a, b).join();

Obtain instances via Llm4s.createDefaultClient or Llm4s.createClient, or wrap a Scala LLMClient you built yourself with Llm4s.wrapClient; the constructor is the package's own (#1666).

LlmResult<String> r = client.complete("What is 2+2?");
r.ifSuccess(System.out::println).ifFailure(e -> System.err.println(e.getMessage()));

Attributes

Source
JLlmClient.scala
Supertypes
class Object
trait Matchable
class Any
sealed class JOutputGuardrail

A built-in guardrail that checks an agent's final answer, for JAgentBuilder.outputGuardrails (#2476). Obtain one from JGuardrails. A JGuardrail checks the answer too, and the query as well.

A built-in guardrail that checks an agent's final answer, for JAgentBuilder.outputGuardrails (#2476). Obtain one from JGuardrails. A JGuardrail checks the answer too, and the query as well.

Attributes

Source
JGuardrails.scala
Supertypes
class Object
trait Matchable
class Any
Known subtypes
class JGuardrail
final class JTool

An agent tool defined from Java (#1484), for Llm4s.createAgentWithTools. Build one from a lambda with JTool.builder, or from the Tool-annotated methods of an object with JTool.fromAnnotated:

An agent tool defined from Java (#1484), for Llm4s.createAgentWithTools. Build one from a lambda with JTool.builder, or from the Tool-annotated methods of an object with JTool.fromAnnotated:

LlmResult<JTool> weather = JTool.builder("weather", "The current weather in a city")
   .param("city", String.class, "City name")
   .handler(args -> "Sunny in " + args.getString("city"))
   .build();
LlmResult<List<JTool>> annotated = JTool.fromAnnotated(new WeatherTools());

A parameter is a String, an int, long, short or byte (boxed or not), a double or float, a boolean, or an enum (the model passes a constant's name); its JSON Schema is derived from the type. The model's arguments are read into those Java values before the handler runs, without a JSON library: a missing required argument, or one of the wrong type, is a tool error the model sees, and the handler is not called. So is an exception the handler throws - the agent keeps running.

When an agent runs the tool, JToolArgs.idempotencyKey and JToolArgs.toolCallId describe the call (#1740): a lambda reads them from its args, and an annotated method from a parameter of type JToolArgs, which the model does not see. They are empty outside an agent run.

Attributes

Companion
object
Source
JTool.scala
Supertypes
class Object
trait Matchable
class Any
object JTool

Attributes

Companion
class
Source
JTool.scala
Supertypes
class Object
trait Matchable
class Any
Self type
JTool.type
final class JToolArgs

The arguments of one tool call, read into Java values before the handler runs (#1484): each declared parameter the model passed, as the type it was declared with - a String, an Integer/Long/Short/Byte, a Double/Float, a Boolean or an enum constant. A required parameter is always present; an optional one the model left out is absent (has is false, and its getters return null, or throw for a primitive one).

The arguments of one tool call, read into Java values before the handler runs (#1484): each declared parameter the model passed, as the type it was declared with - a String, an Integer/Long/Short/Byte, a Double/Float, a Boolean or an enum constant. A required parameter is always present; an optional one the model left out is absent (has is false, and its getters return null, or throw for a primitive one).

When an agent runs the tool, they also carry the call being run (#1740): toolCallId and idempotencyKey. The key is the same in every run of one call - a retry, an approval, an answered question, a recover - so a tool with an external side effect passes it to the system it calls, which then does the work once. Both are empty when the tool runs outside an agent, through plain ToolRegistry tool calling, which has no call to describe.

Attributes

Source
JToolArgs.scala
Supertypes
class Object
trait Matchable
class Any
object Llm4s

Entry-point factory for Java callers.

Entry-point factory for Java callers.

All methods return LlmResult so Java code never imports Scala Either or deals with implicit/given parameters directly. The Scala-level ModelRegistryService given is resolved internally and passed explicitly.

=== Quick-start (Java) ===

LlmResult<JLlmClient> r = Llm4s.createDefaultClient();
if (r.isSuccess()) {
   JLlmClient client = r.get();
   client.complete("Hello").ifSuccess(System.out::println);
}

=== A client configured in code === The providers this module brings each have a builder, so a client needs no application.conf and no Scala type: openAi, anthropic, gemini, ollama, openAiCompatible (vLLM, LM Studio, LiteLLM, a gateway) and azureOpenAi. Every other provider - one from its own module, such as llm4s-bedrock - takes its config object through createClient, or a section of application.conf through createDefaultClient.

LlmResult<JLlmClient> r = Llm4s.openAi()
   .apiKey(apiKey)   // read however your application keeps its secrets
   .model("gpt-4o-mini")
   .build();

Attributes

Source
Llm4s.scala
Supertypes
class Object
trait Matchable
class Any
Self type
Llm4s.type
final class LlmException(val error: LLMError) extends RuntimeException

A checked-free runtime exception wrapping an LLMError, thrown only when Java callers dereference a failed LlmResult via LlmResult#get.

A checked-free runtime exception wrapping an LLMError, thrown only when Java callers dereference a failed LlmResult via LlmResult#get.

If the wrapped error carries an underlying Throwable (for example a cause: Option[Throwable] field), it is exposed through getCause.

The failure can be read without Scala types: getKind is a Java enum to switch on, isRecoverable says whether trying again may help, and getRetryAfter and getStatusCode report what the provider said.

LlmException e = result.getError();
switch (e.getKind()) {
   case RATE_LIMIT -> wait(e.getRetryAfter().orElse(Duration.ofSeconds(30)));
   case AUTHENTICATION, CONFIGURATION -> alertOperator(e.getMessage());
   default -> { if (e.isRecoverable()) retry(); else report(e); }
}

Attributes

Companion
object
Source
LlmException.scala
Supertypes
class Exception
class Throwable
trait Serializable
class Object
trait Matchable
class Any
Show all
object LlmException

Attributes

Companion
class
Source
LlmException.scala
Supertypes
class Object
trait Matchable
class Any
Self type
final class LlmResult[A]

Java-friendly wrapper for org.llm4s.types.Result, which is an Either[LLMError, A].

Java-friendly wrapper for org.llm4s.types.Result, which is an Either[LLMError, A].

Rather than requiring Java callers to deal with Scala's Either, Option, or Left/Right, this class surfaces a familiar API modelled after java.util.Optional and CompletableFuture.

LlmResult<String> result = client.complete("What is 2+2?");
result.ifSuccess(System.out::println)
     .ifFailure(e -> System.err.println(e.getMessage()));

Attributes

Companion
object
Source
LlmResult.scala
Supertypes
class Object
trait Matchable
class Any
object LlmResult

Attributes

Companion
class
Source
LlmResult.scala
Supertypes
class Object
trait Matchable
class Any
Self type
LlmResult.type
object StreamEvents

Reads an AgentStreamListener's events from Java without scala.Option.

Reads an AgentStreamListener's events from Java without scala.Option.

agent.stream(threadId, "Explain monads", event ->
   StreamEvents.decode(AgentEvents.TextDelta(), event).ifPresent(d -> System.out.print(d.text())));

A StreamEvent.LiveGap - live events dropped because the listener fell behind - is matched with instanceof StreamEvent.LiveGap, and its count read with dropped().

Attributes

Source
StreamEvents.scala
Supertypes
class Object
trait Matchable
class Any
Self type