ReactiveLLM4STemplate
org.llm4s.spring.ReactiveLLM4STemplate
A model's reply as a Reactor Flux<String> of its text (#2279), for Spring WebFlux: return it from a controller with produces = text/event-stream and each piece reaches the browser as the model writes it.
The llm4sReactiveTemplate bean, registered when Reactor and llm4s-reactor are on the classpath. A class of its own, not methods of LLM4STemplate, so that an application without Reactor can still load that one. Each Flux is Streams.toFlux's: cold (each subscription makes its own provider call), failing with an LlmException, and a cancelled subscription - a client that disconnects - interrupts the provider call.
Attributes
- Source
- ReactiveLLM4STemplate.scala
- Graph
-
- Supertypes
Members list
In this article
