ReactiveLLM4STemplate

org.llm4s.spring.ReactiveLLM4STemplate
final class ReactiveLLM4STemplate(client: JLlmClient)

A model's reply as a Reactor Flux<String> of its text (#2279), for Spring WebFlux: return it from a controller with produces = text/event-stream and each piece reaches the browser as the model writes it.

The llm4sReactiveTemplate bean, registered when Reactor and llm4s-reactor are on the classpath. A class of its own, not methods of LLM4STemplate, so that an application without Reactor can still load that one. Each Flux is Streams.toFlux's: cold (each subscription makes its own provider call), failing with an LlmException, and a cancelled subscription - a client that disconnects - interrupts the provider call.

Attributes

Source
ReactiveLLM4STemplate.scala
Graph
Supertypes
class Object
trait Matchable
class Any

Members list

Value members

Concrete methods

def stream(query: String): Flux[String]

The reply to a single user query, as a Flux of its text.

The reply to a single user query, as a Flux of its text.

Attributes

Source
ReactiveLLM4STemplate.scala
def stream(conversation: Conversation): Flux[String]

A conversation's reply, as a Flux of its text.

A conversation's reply, as a Flux of its text.

Attributes

Source
ReactiveLLM4STemplate.scala