JAgent

org.llm4s.javaapi.JAgent
See theJAgent companion object
final class JAgent

Java-friendly wrapper around Agent.

Exposes run(query) and continueConversation(previous, query), returning LlmResult[JAgentResult] so Java callers do not need to deal with Scala's Either or Result types directly, nor read a Scala value: a JAgentResult's accessors return Strings, JDK types and this package's own types. A conversation is a thread of the agent's in-memory runtime, kept until forget removes it.

run, continueConversation, resume and recover block the calling thread and never throw. Interrupting that thread cancels the turn: the call returns a failed result whose error is a CancelledError, with the interrupt flag still set, once the turn has ended (it waits up to 5 seconds for the turn to end, for a provider that ignores its interrupt), and the conversation's thread is left for recover. A thread already interrupted starts no turn. InterruptedException is never thrown, so the methods declare no checked exception and javac rejects a catch (InterruptedException e) around them - test the result for a CancelledError instead. To cancel a turn without interrupting a thread, stream it and call AgentStream.cancel.

A turn whose tools need approval, whose tools or middleware ask a question, or whose task a static breakpoint holds, ends SUSPENDED: its status's pending() - or JAgent.pending - lists what it waits for, and resume(threadId, answers) answers some or all of it and continues. A turn that failed or was cancelled continues with recover(threadId). Both block, as run does.

LlmResult<JAgentResult> turn = agent.run("Deploy the release");
List<Answer> answers = new ArrayList<>();
for (PendingInterrupt p : turn.get().status().pending()) answers.add(Answer.approve(p.id()));
if (!answers.isEmpty()) turn = agent.resume(turn.get().threadId(), answers);

stream, streamResume and streamRecover run a turn on a thread you name and hand its events to an AgentStreamListener as they happen, returning an AgentStream at once - to await the turn's result, or to cancel it. From Java a thread id is a String.

runAsync, continueConversationAsync, resumeAsync and recoverAsync return a CompletableFuture at once and run the turn on another thread - a new virtual thread per turn, or the executor withAsyncExecutor names. The future completes with the JAgentResult, or exceptionally with the LlmException the blocking call's result would hold. cancel(true) interrupts the turn, which then ends as the blocking call interrupted does: cancelled, the thread of a continued, resumed or recovered conversation left for recover.

Obtain instances via Llm4s.createAgent.

JAgent agent = Llm4s.createAgent(client);
LlmResult<JAgentResult> result = agent.run("Summarise the news today");
result.ifSuccess(r -> System.out.println(r.answer().orElse("(" + r.status().kind() + ")")))
     .ifFailure(e -> System.err.println(e.getMessage()));

Attributes

Companion
object
Source
JAgent.scala
Graph
Supertypes
class Object
trait Matchable
class Any

Members list

Value members

Concrete methods

def continueConversation(previous: JAgentResult, query: String): LlmResult[JAgentResult]

Runs query as the next turn of previous's conversation - only its threadId() is read. A null argument yields a failed result. Blocks and handles an interrupt as run does: the turn is cancelled and the result is a CancelledError, the interrupt flag left set, never an InterruptedException; the conversation's thread is left for recover.

Runs query as the next turn of previous's conversation - only its threadId() is read. A null argument yields a failed result. Blocks and handles an interrupt as run does: the turn is cancelled and the result is a CancelledError, the interrupt flag left set, never an InterruptedException; the conversation's thread is left for recover.

Attributes

Source
JAgent.scala
def continueConversationAsync(previous: JAgentResult, query: String): CompletableFuture[JAgentResult]

continueConversation(previous, query) on another thread, as runAsync.

continueConversation(previous, query) on another thread, as runAsync.

Attributes

Source
JAgent.scala
def forget(previous: JAgentResult): LlmResult[Void]

Removes previous's conversation from the agent's runtime. A null argument yields a failed result.

Removes previous's conversation from the agent's runtime. A null argument yields a failed result.

Attributes

Source
JAgent.scala
def forget(threadId: String): LlmResult[Void]

Removes the conversation on threadId from the agent's runtime - one you named for stream, say, whose turn failed, so no result carries its id. Forgetting an unknown thread succeeds; a thread whose turn is still running is refused (GraphError.ThreadBusy), and nothing is removed. A null argument yields a failed result.

Removes the conversation on threadId from the agent's runtime - one you named for stream, say, whose turn failed, so no result carries its id. Forgetting an unknown thread succeeds; a thread whose turn is still running is refused (GraphError.ThreadBusy), and nothing is removed. A null argument yields a failed result.

Attributes

Source
JAgent.scala
def recover(threadId: String): LlmResult[JAgentResult]

Continues threadId's failed or cancelled turn - a stream that was cancelled, a run that failed on a provider error - re-running only the work that did not finish, and returns its result. A thread with nothing to recover is a failed result. Blocks and handles an interrupt as run does: the recovered turn is cancelled and the thread is left for another recover.

Continues threadId's failed or cancelled turn - a stream that was cancelled, a run that failed on a provider error - re-running only the work that did not finish, and returns its result. A thread with nothing to recover is a failed result. Blocks and handles an interrupt as run does: the recovered turn is cancelled and the thread is left for another recover.

Attributes

Source
JAgent.scala
def recoverAsync(threadId: String): CompletableFuture[JAgentResult]

recover(threadId) on another thread, as runAsync.

recover(threadId) on another thread, as runAsync.

Attributes

Source
JAgent.scala
def resume(threadId: String, answers: List[Answer]): LlmResult[JAgentResult]

Answers some of threadId's pending approvals, questions and breakpoints - read them with result.status().pending() - and continues the turn, returning its result; unanswered ones stay pending, so the result can be Suspended again. Build each answer with Answer.approve, Answer.reject, Answer.edit, Answer.reply or Answer.proceed; for an id answered twice, the last answer counts. A null or malformed answer, an empty answers list (GraphError.InvalidResume), an answer to an id the thread does not wait for, or a thread that is not suspended is a failed result. Blocks and handles an interrupt as run does: the turn is cancelled, the result is a CancelledError with the interrupt flag left set, and the thread is left for recover.

Answers some of threadId's pending approvals, questions and breakpoints - read them with result.status().pending() - and continues the turn, returning its result; unanswered ones stay pending, so the result can be Suspended again. Build each answer with Answer.approve, Answer.reject, Answer.edit, Answer.reply or Answer.proceed; for an id answered twice, the last answer counts. A null or malformed answer, an empty answers list (GraphError.InvalidResume), an answer to an id the thread does not wait for, or a thread that is not suspended is a failed result. Blocks and handles an interrupt as run does: the turn is cancelled, the result is a CancelledError with the interrupt flag left set, and the thread is left for recover.

Attributes

Source
JAgent.scala
def resumeAsync(threadId: String, answers: List[Answer]): CompletableFuture[JAgentResult]

resume(threadId, answers) on another thread, as runAsync.

resume(threadId, answers) on another thread, as runAsync.

Attributes

Source
JAgent.scala
def run(query: String): LlmResult[JAgentResult]

Runs query as the first turn of a new conversation. A null query yields a failed result.

Runs query as the first turn of a new conversation. A null query yields a failed result.

Blocks the calling thread until the turn ends. If that thread is interrupted while it waits, the turn is cancelled: the result is a failure whose error is a CancelledError, returned once the turn has ended (it waits up to 5 seconds for the turn to end), and the thread's interrupt flag is left set. A turn that had already begun committing its outcome is not cancelled, and that outcome is returned, the flag still set. A thread already interrupted starts no turn. The conversation's id is not in a failed result, so a failed or cancelled turn's conversation is forgotten. InterruptedException is never thrown, and the method declares none.

Attributes

Source
JAgent.scala
def runAsync(query: String): CompletableFuture[JAgentResult]

run(query) on another thread: returns at once with a future of the turn's result. The future completes exceptionally with the LlmException the blocking call's failed result would hold - of the same LlmErrorKind - and never with another exception, except the CancellationException of a future cancelled yourself. cancel(true), or completing the future yourself (orTimeout), interrupts the turn: it is cancelled as an interrupted run is. cancel(false) lets the turn finish and discards its result.

run(query) on another thread: returns at once with a future of the turn's result. The future completes exceptionally with the LlmException the blocking call's failed result would hold - of the same LlmErrorKind - and never with another exception, except the CancellationException of a future cancelled yourself. cancel(true), or completing the future yourself (orTimeout), interrupts the turn: it is cancelled as an interrupted run is. cancel(false) lets the turn finish and discards its result.

Attributes

Source
JAgent.scala
def stream(threadId: String, query: String, listener: AgentStreamListener): LlmResult[AgentStream]

Runs query as one turn on threadId - a new conversation, or the next turn of one - handing every event of the turn to listener, then its result or error; see AgentStreamListener. Returns at once with the running AgentStream. A refused start (a blank query, a busy thread, a null argument) is a failed result, and listener hears nothing.

Runs query as one turn on threadId - a new conversation, or the next turn of one - handing every event of the turn to listener, then its result or error; see AgentStreamListener. Returns at once with the running AgentStream. A refused start (a blank query, a busy thread, a null argument) is a failed result, and listener hears nothing.

Attributes

Source
JAgent.scala

Continues threadId's failed or cancelled turn, as a stream; see stream.

Continues threadId's failed or cancelled turn, as a stream; see stream.

Attributes

Source
JAgent.scala
def streamResume(threadId: String, answers: List[Answer], listener: AgentStreamListener): LlmResult[AgentStream]

Answers some of threadId's pending approvals, questions and breakpoints and continues, as a stream; see stream. Build each answer with Answer.approve, Answer.reject, Answer.edit, Answer.reply or Answer.proceed, e.g. List.of(Answer.approve(id)); for an id answered twice, the last answer counts. A null or malformed answer is a failed result, and listener hears nothing.

Answers some of threadId's pending approvals, questions and breakpoints and continues, as a stream; see stream. Build each answer with Answer.approve, Answer.reject, Answer.edit, Answer.reply or Answer.proceed, e.g. List.of(Answer.approve(id)); for an id answered twice, the last answer counts. A null or malformed answer is a failed result, and listener hears nothing.

Attributes

Source
JAgent.scala

This agent with its *Async turns run on executor rather than a new virtual thread each. It is the same agent - its conversations are shared - and never shuts executor down. Cancelling a future interrupts the thread running its turn, whatever the executor; an executor that refuses a turn fails its future at once with kind OTHER.

This agent with its *Async turns run on executor rather than a new virtual thread each. It is the same agent - its conversations are shared - and never shuts executor down. Cancelling a future interrupts the thread running its turn, whatever the executor; an executor that refuses a turn fails its future at once with kind OTHER.

Attributes

Throws
NullPointerException

when executor is null

Source
JAgent.scala