Llm4sChatModel

org.llm4s.springai.Llm4sChatModel
See theLlm4sChatModel companion object
@Experimental
final class Llm4sChatModel(client: LLMClient) extends ChatModel, StreamingChatModel

An llm4s LLMClient as a Spring AI ChatModel and StreamingChatModel (#2278), so that a Spring AI application's ChatClient, advisors and evaluators run over any llm4s provider, a ReliableClient or a traced client, with no change to the code that calls them.

ChatModel model = Llm4sChatModel.of(Llm4s.createDefaultClient().get());
String answer = ChatClient.create(model).prompt().user("What is 2+2?").call().content();

A Prompt's system, user, assistant (with its tool calls) and tool-response messages become the llm4s conversation; its options' temperature, top-p, max tokens, presence and frequency penalties and tool definitions are sent. The model is the llm4s client's own, so a model named in the options is not sent; the response's metadata names the model that answered. Stop sequences and top-k, which no llm4s client sends, are refused rather than dropped. The reply's tool calls come back as the AssistantMessage's, with finish reason TOOL_CALLS, for Spring AI's tool-calling advisor to run; this model never runs a tool itself.

call throws an LlmException carrying the llm4s error; stream signals it as the Flux's error. stream emits a response per streamed piece of text, then one with no text that carries the tool calls, finish reason, model and usage. Cancelling the subscription interrupts the call.

Attributes

Companion
object
Source
Llm4sChatModel.scala
Graph
Supertypes
trait ChatModel
trait StreamingChatModel
trait StreamingModel[Prompt, ChatResponse]
trait Model[Prompt, ChatResponse]
class Object
trait Matchable
class Any
Show all

Members list

Value members

Concrete methods

override def call(prompt: Prompt): ChatResponse

Attributes

Definition Classes
ChatModel -> Model
Source
Llm4sChatModel.scala
override def getOptions: ChatOptions

No defaults of its own: the llm4s client's configuration applies to whatever a prompt leaves unset. Tool-calling options, so that a ChatClient sends the tools it is given and its tool-calling advisor runs them.

No defaults of its own: the llm4s client's configuration applies to whatever a prompt leaves unset. Tool-calling options, so that a ChatClient sends the tools it is given and its tool-calling advisor runs them.

Attributes

Definition Classes
ChatModel
Source
Llm4sChatModel.scala
override def stream(prompt: Prompt): Flux[ChatResponse]

Attributes

Definition Classes
ChatModel -> StreamingChatModel -> StreamingModel
Source
Llm4sChatModel.scala

Inherited methods

def call(x$0: Message*): String

Attributes

Inherited from:
ChatModel
def call(x$0: String): String

Attributes

Inherited from:
ChatModel
def stream(x$0: Message*): Flux[String]

Attributes

Inherited from:
StreamingChatModel
def stream(x$0: String): Flux[String]

Attributes

Inherited from:
StreamingChatModel

Deprecated and Inherited methods

@Deprecated(since = _, forRemoval = true)
def getDefaultOptions(): ChatOptions

Attributes

Deprecated
[Since version ] see corresponding Javadoc for more information.
Inherited from:
ChatModel