A Spring AI ChatModel as an llm4s LLMClient (#2278), so that llm4s agents, handoffs and RAG run over the model beans a Spring AI application already configured. From Java, Llm4s.wrapClient(new SpringAiLlmClient(chatModel)) gives the facade's JLlmClient.
The conversation's system, user, assistant (with its tool calls) and tool messages become the Prompt; the options' temperature, top-p, max tokens, presence and frequency penalties and tools are sent as ToolCallingChatOptions. The Spring AI model returns its tool calls, which the llm4s agent runs. The response's id, model, text, tool calls and usage become the Completion. An exception the model throws, or a Flux error while streaming, is a Left; an LlmException gives back the error it carries.
Value parameters
- contextWindow
-
the model's context window in tokens, which llm4s's context management reads
- reserveCompletion
-
the tokens kept free for the reply
Attributes
- Companion
- object
- Source
- SpringAiLlmClient.scala
- Graph
-
- Supertypes
Members list
Value members
Constructors
A client that assumes a 128k-token context window and keeps 4096 tokens for the reply.
A client that assumes a 128k-token context window and keeps 4096 tokens for the reply.
Attributes
- Source
- SpringAiLlmClient.scala
Concrete methods
Executes a blocking completion request and returns the full response.
Executes a blocking completion request and returns the full response.
Sends the conversation to the LLM and waits for the complete response. Use when you need the entire response at once or when streaming is not required.
Value parameters
- conversation
-
conversation history including system, user, assistant, and tool messages
- options
-
configuration including temperature, max tokens, tools, etc. (default: CompletionOptions())
Attributes
- Returns
-
Right(Completion) with the model's response, or Left(LLMError) on failure
- Definition Classes
- Source
- SpringAiLlmClient.scala
Returns the maximum context window size supported by this model in tokens.
Returns the maximum context window size supported by this model in tokens.
The context window is the total tokens (prompt + completion) the model can process in a single request, including all conversation messages and the generated response.
Attributes
- Returns
-
total context window size in tokens (e.g., 4096, 8192, 128000)
- Definition Classes
- Source
- SpringAiLlmClient.scala
Returns the number of tokens reserved for the model's completion response.
Returns the number of tokens reserved for the model's completion response.
This value is subtracted from the context window when calculating available tokens for prompts. Corresponds to the max_tokens or completion token limit configured for the model.
Attributes
- Returns
-
number of tokens reserved for completion
- Definition Classes
- Source
- SpringAiLlmClient.scala
Executes a streaming completion request, invoking a callback for each chunk as it arrives.
Executes a streaming completion request, invoking a callback for each chunk as it arrives.
Streams the response incrementally, calling onChunk for each token/chunk received. Enables real-time display of responses. Returns the final accumulated completion on success.
Value parameters
- conversation
-
conversation history including system, user, assistant, and tool messages
- onChunk
-
callback invoked for each chunk; called synchronously, avoid blocking operations
- options
-
configuration including temperature, max tokens, tools, etc. (default: CompletionOptions())
Attributes
- Returns
-
Right(Completion) with the complete accumulated response, or Left(LLMError) on failure
- Definition Classes
- Source
- SpringAiLlmClient.scala
Inherited methods
Releases resources and closes connections to the LLM provider.
Releases resources and closes connections to the LLM provider.
Call when the client is no longer needed. After calling close(), the client should not be used. Default implementation is a no-op; override if managing resources like connections or thread pools.
Attributes
- Inherited from:
- LLMClient
- Source
- LLMClient.scala
Sends the conversation and parses the response into a typed value using the provided schema.
Sends the conversation and parses the response into a typed value using the provided schema.
Sets ResponseFormat.JsonSchema on the options. OpenAI, Azure OpenAI and Gemini enforce the schema at generation time, as does Ollama 0.5 or later through its format field. Requesty and the OpenAI-compatible providers (including Cohere) send it as response_format, but whether it is enforced is up to the server - for Requesty, a router, the backend model it routes to: one that ignores the field returns unconstrained text. Anthropic falls back to a best-effort system-prompt instruction, which is not schema-enforced. Clients that do not read responseFormat (watsonx, Bedrock) send no schema at all. Because models may wrap JSON in markdown code fences or surround it with prose, the response is normalised (fence stripped, first balanced {...} or [...] extracted) before being deserialised with uPickle into the expected type A. A is read from that text, so a Long field receives an integer above 2^53 exactly; a reader that goes through ujson.Value (such as a bridge to another JSON library) sees it rounded to the nearest double.
The reply is '''not''' validated against the schema: it is only deserialised. A constraint the reader does not check - an enum, a numeric or string bound, additionalProperties = false (extra keys are ignored) - can be violated by a reply that still returns Right(A). Check such constraints on the result yourself.
The schema is derived with strict = true, which lists '''every''' property as required, including a property declared optional with required = false. Only responseFormat is overridden: every other option you pass is forwarded unchanged to complete, where the provider client may adjust or drop options the model does not support, as for any other complete call. name and strict on the format are left at their defaults ("response" and true); call complete with your own ResponseFormat.JsonSchema to set them.
Type parameters
- A
-
target type; must have a corresponding
upickle.default.Reader[A]
Value parameters
- conversation
-
conversation history
- options
-
additional completion options (default: CompletionOptions())
- reader
-
implicit uPickle reader for deserialising the JSON into
A - schema
-
JSON-Schema description of the expected response object
Attributes
- Returns
-
Right(A) on success. Left(ValidationError) with field
structured_outputwhen the reply is not JSON, is JSONnull, or cannot be deserialised asA; any other Left is the provider call's own error, returned unchanged - Inherited from:
- LLMClient
- Source
- LLMClient.scala
Calculates available token budget for prompts after accounting for completion reserve and headroom.
Calculates available token budget for prompts after accounting for completion reserve and headroom.
Formula: (contextWindow - reserveCompletion) * (1 - headroom)
Headroom provides a safety margin for tokenization variations and message formatting overhead.
Value parameters
- headroom
-
safety margin as percentage of prompt budget (default: HeadroomPercent.Standard ~10%)
Attributes
- Returns
-
maximum tokens available for prompt content
- Inherited from:
- LLMClient
- Source
- LLMClient.scala
The canonical id of the provider this client calls (e.g. openai), when it is known. Tracing reports it as the OpenTelemetry GenAI conventions' gen_ai.provider.name (#2275). None by default; a provider client built on org.llm4s.llmconnect.BaseLifecycleLLMClient reports its own, and a client that wraps another forwards the wrapped client's.
The canonical id of the provider this client calls (e.g. openai), when it is known. Tracing reports it as the OpenTelemetry GenAI conventions' gen_ai.provider.name (#2275). None by default; a provider client built on org.llm4s.llmconnect.BaseLifecycleLLMClient reports its own, and a client that wraps another forwards the wrapped client's.
Attributes
- Inherited from:
- LLMClient
- Source
- LLMClient.scala
Validates client configuration and connectivity to the LLM provider.
Validates client configuration and connectivity to the LLM provider.
May perform checks such as verifying API credentials, testing connectivity, and validating configuration. Default implementation returns success; override for provider-specific validation.
Attributes
- Returns
-
Right(()) if validation succeeds, Left(LLMError) with details on failure
- Inherited from:
- LLMClient
- Source
- LLMClient.scala