SseLLM4STemplate
org.llm4s.spring.SseLLM4STemplate
A model's reply as a Spring MVC SseEmitter (#2279): return it from a controller and each piece of the reply's text reaches the browser as one server-sent event, as the model writes it.
The llm4sSseTemplate bean, registered when Spring MVC is on the classpath. The provider call runs on a thread of its own (JLlmClient.streamAsPublisher), so the request thread is free at once. The emitter completes after the last piece, and completes with the LlmException when the call fails. When the client disconnects or the request times out, the provider call is interrupted.
Attributes
- Source
- SseLLM4STemplate.scala
- Graph
-
- Supertypes
Members list
In this article
