RagStreams
org.llm4s.javaapi.reactor.RagStreams
object RagStreams
A RAG answer, with its sources, as a Reactor Flux of RagAnswerEvents (#2727): for a Spring WebFlux endpoint that sends the citations over SSE first, then the answer as it is generated. It needs llm4s-java-rag on the classpath, which an application holding a JRag has.
Each method runs JRag.askStream once per subscription, so the Flux:
- is cold: each subscription makes its own search and model call, on a thread of its own;
- emits one
RagAnswerEvent.Sources, then aRagAnswerEvent.Chunkfor each piece of the answer's text, then oneRagAnswerEvent.Completewith the whole answer, then completes; - ends with
onErrorfor a failure - of the search, the model, or a pipeline that refuses the plain query - carrying an LlmException whosegetKind()says what failed, after the elements already delivered; - honours backpressure: a subscriber that requests less than the model produces holds the call's thread rather than losing text;
- cancelling the subscription (
dispose(),take(n), a client that disconnects) interrupts the call, asJRagAnswerStream.cancel()does, without waiting for it.
The sources and the answer travel as elements of one Flux, not as a Mono beside it, so they cannot be subscribed to apart - two subscriptions would make two searches - and their order is the stream's own.
A null argument, a blank question or a pipeline without a chat client is never thrown: the Flux fails on subscription with an LlmException of kind VALIDATION or CONFIGURATION.
Flux<RagAnswerEvent> events = RagStreams.toFlux(rag, "How long do refunds take?");
events.subscribe(event -> {
switch (event) {
case RagAnswerEvent.Sources s -> showSources(s.sources());
case RagAnswerEvent.Chunk c -> System.out.print(c.text());
case RagAnswerEvent.Complete c -> System.out.println();
}
});
Attributes
- Source
- RagStreams.scala
- Graph
-
- Supertypes
- Self type
-
RagStreams.type
Members list
In this article
