JRagAnswerStream

org.llm4s.javaapi.rag.JRagAnswerStream
final class JRagAnswerStream extends AutoCloseable

An answer being streamed from JRag.askStream to its JRagStreamListener (#2727): cancel it, or await the answer. It is AutoCloseable, and close() cancels, so a try-with-resources block stops a call its body leaves. It works as JLlmClient.stream's CompletionStream does: the search and the model call run on a virtual thread of their own, and cancelling interrupts that thread.

try (JRagAnswerStream stream = rag.askStream(question, listener).get()) {
   LlmResult<JRagAnswer> answer = stream.await();
}

Attributes

Source
JRagAnswerStream.scala
Graph
Supertypes
class Object
trait Matchable
class Any

Members list

Value members

Concrete methods

def await(): LlmResult[JRagAnswer]

Blocks until the listener has returned from its last call - JRagStreamListener.onComplete or JRagStreamListener.onError - then returns the same outcome. A call whose thread is interrupted first returns a failed result (CANCELLED) with the interrupt flag still set, and the stream carries on. A call from the listener itself, which would wait on itself, returns a failed result at once.

Blocks until the listener has returned from its last call - JRagStreamListener.onComplete or JRagStreamListener.onError - then returns the same outcome. A call whose thread is interrupted first returns a failed result (CANCELLED) with the interrupt flag still set, and the stream carries on. A call from the listener itself, which would wait on itself, returns a failed result at once.

Attributes

Source
JRagAnswerStream.scala
def cancel(): Unit

Cancels the call: its thread is interrupted, which is how llm4s providers are cancelled, and no further source or piece is delivered. The listener then receives JRagStreamListener.onError with the cancellation - or onComplete, for a call that had already ended. Returns once the listener has heard, or after 5 seconds when a provider ignores the interrupt (logged at WARN). Safe to call more than once, from any thread, the listener's included.

Cancels the call: its thread is interrupted, which is how llm4s providers are cancelled, and no further source or piece is delivered. The listener then receives JRagStreamListener.onError with the cancellation - or onComplete, for a call that had already ended. Returns once the listener has heard, or after 5 seconds when a provider ignores the interrupt (logged at WARN). Safe to call more than once, from any thread, the listener's included.

Attributes

Source
JRagAnswerStream.scala
override def close(): Unit

Cancels the call; see cancel.

Cancels the call; see cancel.

Attributes

Definition Classes
Source
JRagAnswerStream.scala