llm4s-docs
0.5.1+352aa8f743c-SNAPSHOT
API
Generated with
llm4s-docs/org.llm4s/org.llm4s.springai

org.llm4s.springai

package org.llm4s.springai

Members list

Type members

Classlikes

@Experimental
final class Llm4sChatModel(client: LLMClient) extends ChatModel, StreamingChatModel

An llm4s LLMClient as a Spring AI ChatModel and StreamingChatModel (#2278), so that a Spring AI application's ChatClient, advisors and evaluators run over any llm4s provider, a ReliableClient or a traced client, with no change to the code that calls them.

An llm4s LLMClient as a Spring AI ChatModel and StreamingChatModel (#2278), so that a Spring AI application's ChatClient, advisors and evaluators run over any llm4s provider, a ReliableClient or a traced client, with no change to the code that calls them.

ChatModel model = Llm4sChatModel.of(Llm4s.createDefaultClient().get());
String answer = ChatClient.create(model).prompt().user("What is 2+2?").call().content();

A Prompt's system, user, assistant (with its tool calls) and tool-response messages become the llm4s conversation; its options' temperature, top-p, max tokens, presence and frequency penalties and tool definitions are sent. The model is the llm4s client's own, so a model named in the options is not sent; the response's metadata names the model that answered. Stop sequences and top-k, which no llm4s client sends, are refused rather than dropped. The reply's tool calls come back as the AssistantMessage's, with finish reason TOOL_CALLS, for Spring AI's tool-calling advisor to run; this model never runs a tool itself.

call throws an LlmException carrying the llm4s error; stream signals it as the Flux's error. stream emits a response per streamed piece of text, then one with no text that carries the tool calls, finish reason, model and usage. Cancelling the subscription interrupts the call.

Attributes

Companion
object
Source
Llm4sChatModel.scala
Supertypes
trait ChatModel
trait StreamingChatModel
trait StreamingModel[Prompt, ChatResponse]
trait Model[Prompt, ChatResponse]
class Object
trait Matchable
class Any
Show all
@Experimental
object Llm4sChatModel

Attributes

Companion
class
Source
Llm4sChatModel.scala
Supertypes
class Object
trait Matchable
class Any
Self type
@Experimental
final class Llm4sEmbeddingModel(client: JEmbeddingClient) extends EmbeddingModel

An llm4s embedding client as a Spring AI EmbeddingModel (#2278), so a Spring AI vector store or evaluator embeds through any llm4s embedding provider.

An llm4s embedding client as a Spring AI EmbeddingModel (#2278), so a Spring AI vector store or evaluator embeds through any llm4s embedding provider.

EmbeddingModel model = new Llm4sEmbeddingModel(Llm4s.createEmbeddingClient(config).get());
float[] vector = model.embed("hello");

The request's texts are sent in one llm4s request, as documents; its options are not read (the model is the client's own). The response's metadata names the model that answered. A failure throws the LlmException the client's result holds. dimensions() is the model's declared length when its provider module lists it, and otherwise Spring AI's default, which embeds a test string.

Attributes

Companion
object
Source
Llm4sEmbeddingModel.scala
Supertypes
trait EmbeddingModel
trait Model[EmbeddingRequest, EmbeddingResponse]
class Object
trait Matchable
class Any
@Experimental
object Llm4sEmbeddingModel

Attributes

Companion
class
Source
Llm4sEmbeddingModel.scala
Supertypes
class Object
trait Matchable
class Any
Self type
@Experimental @AutoConfiguration(value = _, before = _, beforeName = _, after = _, afterName = scala.Array.apply[java.lang.String]("org.llm4s.spring.Llm4sAutoConfiguration")(scala.reflect.ClassTag.apply[java.lang.String](classOf[java.lang.String]))) @ConditionalOnClass(value = _, name = scala.Array.apply[java.lang.String]("org.springframework.ai.chat.model.ChatModel")(scala.reflect.ClassTag.apply[java.lang.String](classOf[java.lang.String])))
class Llm4sSpringAiAutoConfiguration

Exposes the llm4s client as a Spring AI ChatModel bean (#2278): when Spring AI is on the classpath and the context has a JLlmClient - the one the llm4s Spring Boot starter configures from llm4s.*, or the application's own - and no ChatModel bean yet. A ChatClient built from that model then answers through llm4s. An application that defines its own ChatModel keeps it.

Exposes the llm4s client as a Spring AI ChatModel bean (#2278): when Spring AI is on the classpath and the context has a JLlmClient - the one the llm4s Spring Boot starter configures from llm4s.*, or the application's own - and no ChatModel bean yet. A ChatClient built from that model then answers through llm4s. An application that defines its own ChatModel keeps it.

Attributes

Source
Llm4sSpringAiAutoConfiguration.scala
Supertypes
class Object
trait Matchable
class Any
@Experimental
final class SpringAiLlmClient(chatModel: ChatModel, contextWindow: Int, reserveCompletion: Int) extends LLMClient

A Spring AI ChatModel as an llm4s LLMClient (#2278), so that llm4s agents, handoffs and RAG run over the model beans a Spring AI application already configured. From Java, Llm4s.wrapClient(new SpringAiLlmClient(chatModel)) gives the facade's JLlmClient.

A Spring AI ChatModel as an llm4s LLMClient (#2278), so that llm4s agents, handoffs and RAG run over the model beans a Spring AI application already configured. From Java, Llm4s.wrapClient(new SpringAiLlmClient(chatModel)) gives the facade's JLlmClient.

The conversation's system, user, assistant (with its tool calls) and tool messages become the Prompt; the options' temperature, top-p, max tokens, presence and frequency penalties and tools are sent as ToolCallingChatOptions. The Spring AI model returns its tool calls, which the llm4s agent runs. The response's id, model, text, tool calls and usage become the Completion. An exception the model throws, or a Flux error while streaming, is a Left; an LlmException gives back the error it carries.

Value parameters

contextWindow

the model's context window in tokens, which llm4s's context management reads

reserveCompletion

the tokens kept free for the reply

Attributes

Companion
object
Source
SpringAiLlmClient.scala
Supertypes
trait LLMClient
class Object
trait Matchable
class Any
@Experimental
object SpringAiLlmClient

Attributes

Companion
class
Source
SpringAiLlmClient.scala
Supertypes
class Object
trait Matchable
class Any
Self type
Generated with