SseLLM4STemplate

org.llm4s.spring.SseLLM4STemplate
final class SseLLM4STemplate(client: JLlmClient)

A model's reply as a Spring MVC SseEmitter (#2279): return it from a controller and each piece of the reply's text reaches the browser as one server-sent event, as the model writes it.

The llm4sSseTemplate bean, registered when Spring MVC is on the classpath. The provider call runs on a thread of its own (JLlmClient.streamAsPublisher), so the request thread is free at once. The emitter completes after the last piece, and completes with the LlmException when the call fails. When the client disconnects or the request times out, the provider call is interrupted.

Attributes

Source
SseLLM4STemplate.scala
Graph
Supertypes
class Object
trait Matchable
class Any

Members list

Value members

Concrete methods

def stream(query: String): SseEmitter

The reply to a single user query, one event per piece of its text.

The reply to a single user query, one event per piece of its text.

Attributes

Source
SseLLM4STemplate.scala
def stream(conversation: Conversation): SseEmitter

A conversation's reply, one event per piece of its text.

A conversation's reply, one event per piece of its text.

Attributes

Source
SseLLM4STemplate.scala