An llm4s LLMClient as a Spring AI ChatModel and StreamingChatModel (#2278), so that a Spring AI application's ChatClient, advisors and evaluators run over any llm4s provider, a ReliableClient or a traced client, with no change to the code that calls them.
ChatModel model = Llm4sChatModel.of(Llm4s.createDefaultClient().get());
String answer = ChatClient.create(model).prompt().user("What is 2+2?").call().content();
A Prompt's system, user, assistant (with its tool calls) and tool-response messages become the llm4s conversation; its options' temperature, top-p, max tokens, presence and frequency penalties and tool definitions are sent. The model is the llm4s client's own, so a model named in the options is not sent; the response's metadata names the model that answered. Stop sequences and top-k, which no llm4s client sends, are refused rather than dropped. The reply's tool calls come back as the AssistantMessage's, with finish reason TOOL_CALLS, for Spring AI's tool-calling advisor to run; this model never runs a tool itself.
call throws an LlmException carrying the llm4s error; stream signals it as the Flux's error. stream emits a response per streamed piece of text, then one with no text that carries the tool calls, finish reason, model and usage. Cancelling the subscription interrupts the call.
Attributes
- Companion
- object
- Source
- Llm4sChatModel.scala
- Graph
-
- Supertypes
-
trait ChatModeltrait StreamingChatModeltrait StreamingModel[Prompt, ChatResponse]trait Model[Prompt, ChatResponse]class Objecttrait Matchableclass Any
