Llm4s

org.llm4s.javaapi.Llm4s
object Llm4s

Entry-point factory for Java callers.

All methods return LlmResult so Java code never imports Scala Either or deals with implicit/given parameters directly. The Scala-level ModelRegistryService given is resolved internally and passed explicitly.

=== Quick-start (Java) ===

LlmResult<JLlmClient> r = Llm4s.createDefaultClient();
if (r.isSuccess()) {
   JLlmClient client = r.get();
   client.complete("Hello").ifSuccess(System.out::println);
}

=== A client configured in code === The providers this module brings each have a builder, so a client needs no application.conf and no Scala type: openAi, anthropic, gemini, ollama, openAiCompatible (vLLM, LM Studio, LiteLLM, a gateway) and azureOpenAi. Every other provider - one from its own module, such as llm4s-bedrock - takes its config object through createClient, or a section of application.conf through createDefaultClient.

LlmResult<JLlmClient> r = Llm4s.openAi()
   .apiKey(apiKey)   // read however your application keeps its secrets
   .model("gpt-4o-mini")
   .build();

Attributes

Source
Llm4s.scala
Graph
Supertypes
class Object
trait Matchable
class Any
Self type
Llm4s.type

Members list

Value members

Concrete methods

A builder for an agent called id over client, with a system prompt, tools, input and output guardrails (JGuardrails), a context window (JContextWindow) and streaming, set from Java (#2476). createAgent and createAgentWithTools are its zero-option cases. Neither argument is checked here: a null one is a failed result of build().

A builder for an agent called id over client, with a system prompt, tools, input and output guardrails (JGuardrails), a context window (JContextWindow) and streaming, set from Java (#2476). createAgent and createAgentWithTools are its zero-option cases. Neither argument is checked here: a null one is a failed result of build().

LlmResult<JAgent> agent = Llm4s.agentBuilder("support", client)
   .systemPrompt("You are a support assistant.")
   .inputGuardrails(List.of(JGuardrails.length(1, 10_000)))
   .contextWindow(JContextWindow.lastMessages(20))
   .build();

Attributes

Source
Llm4s.scala
def aiService[T](serviceInterface: Class[T]): AiServiceBuilder[T]

A builder for the implementation of serviceInterface, an AI service (#2277): an interface whose methods are annotated with @SystemMessage, @UserMessage, @V and @MemoryId from org.llm4s.javaapi.services. Each call of a method renders its prompt from its arguments, asks the model - in one agent turn when the service has tools - and returns the answer as the method's return type. Experimental.

A builder for the implementation of serviceInterface, an AI service (#2277): an interface whose methods are annotated with @SystemMessage, @UserMessage, @V and @MemoryId from org.llm4s.javaapi.services. Each call of a method renders its prompt from its arguments, asks the model - in one agent turn when the service has tools - and returns the answer as the method's return type. Experimental.

interface Assistant {
   @SystemMessage("You are a support agent for {{product}}. Answer briefly.")
   String answer(@UserMessage String question, @V("product") String product);
}
Assistant assistant = Llm4s.aiService(Assistant.class).client(client).build();

Attributes

Throws
NullPointerException

when serviceInterface is null

Source
Llm4s.scala
def anthropic(): AnthropicClientBuilder

A builder for an Anthropic client: apiKey and model are required (#1489).

A builder for an Anthropic client: apiKey and model are required (#1489).

Attributes

Source
Llm4s.scala
def azureOpenAi(): AzureOpenAiClientBuilder

A builder for an Azure OpenAI client: endpoint, apiKey and model (the deployment) are required (#1489).

A builder for an Azure OpenAI client: endpoint, apiKey and model (the deployment) are required (#1489).

Attributes

Source
Llm4s.scala
def createAgent(client: JLlmClient): JAgent

Wraps a JLlmClient in a JAgent ready to accept natural-language queries, with no tools.

Wraps a JLlmClient in a JAgent ready to accept natural-language queries, with no tools.

Attributes

Source
Llm4s.scala
def createAgent(client: JLlmClient, tools: ToolRegistry): JAgent

Wraps a JLlmClient in a JAgent that can call tools. Tools belong to the agent: an agent whose tools cannot be offered together (a name clash, an invalid schema) fails every run with the reason.

Wraps a JLlmClient in a JAgent that can call tools. Tools belong to the agent: an agent whose tools cannot be offered together (a name clash, an invalid schema) fails every run with the reason.

Attributes

Source
Llm4s.scala
def createAgent(client: JLlmClient, tools: ToolRegistry, streaming: Boolean): JAgent

createAgent with tools, whose model calls stream when streaming is set: then a JAgent.stream of it carries the answer's text deltas (AgentEvents.TextDelta()). Without it, the agent calls the model's complete, and a stream carries no deltas.

createAgent with tools, whose model calls stream when streaming is set: then a JAgent.stream of it carries the answer's text deltas (AgentEvents.TextDelta()). Without it, the agent calls the model's complete, and a stream carries no deltas.

Attributes

Source
Llm4s.scala

Wraps a JLlmClient in a JAgent that can call tools, defined from Java with JTool.builder or JTool.fromAnnotated (#1484). A null argument, a tool listed twice, two tools with one name, or tools the agent cannot offer together is a failed result, not an agent that fails every run.

Wraps a JLlmClient in a JAgent that can call tools, defined from Java with JTool.builder or JTool.fromAnnotated (#1484). A null argument, a tool listed twice, two tools with one name, or tools the agent cannot offer together is a failed result, not an agent that fails every run.

JAgent agent = Llm4s.createAgentWithTools(client, List.of(weather)).get();

Attributes

Source
Llm4s.scala

Creates a JLlmClient from an explicit ProviderConfig. Useful when the caller constructs the config programmatically rather than relying on environment variables.

Creates a JLlmClient from an explicit ProviderConfig. Useful when the caller constructs the config programmatically rather than relying on environment variables.

Attributes

Source
Llm4s.scala

createClient whose client reports every call (request outcome, duration, tokens and cost) to metrics, for example a MicrometerMetrics from llm4s-observability-micrometer over the application's MeterRegistry, or a PrometheusMetrics from llm4s-observability-prometheus. The Spring Boot starter uses it to attach the application's MetricsCollector bean. A null metrics records nothing; a null config is a failed result.

createClient whose client reports every call (request outcome, duration, tokens and cost) to metrics, for example a MicrometerMetrics from llm4s-observability-micrometer over the application's MeterRegistry, or a PrometheusMetrics from llm4s-observability-prometheus. The Spring Boot starter uses it to attach the application's MetricsCollector bean. A null metrics records nothing; a null config is a failed result.

Attributes

Source
Llm4s.scala

Creates a JLlmClient from the default provider: the section of application.conf that llm4s.providers.provider names. A provider's API key comes from its section's apiKey, else from llm4s.credentials (bound to the vendor's usual environment variable, e.g. OPENAI_API_KEY).

Creates a JLlmClient from the default provider: the section of application.conf that llm4s.providers.provider names. A provider's API key comes from its section's apiKey, else from llm4s.credentials (bound to the vendor's usual environment variable, e.g. OPENAI_API_KEY).

Attributes

Source
Llm4s.scala

Creates a JEmbeddingClient for the embedding model configured as chat providers are, in the application's application.conf: llm4s.embeddings.model, provider/model (for example openai/text-embedding-3-small), which EMBEDDING_MODEL sets; the provider's key comes from its llm4s.embeddings.<provider> block or its vendor variable, such as OPENAI_API_KEY.

Creates a JEmbeddingClient for the embedding model configured as chat providers are, in the application's application.conf: llm4s.embeddings.model, provider/model (for example openai/text-embedding-3-small), which EMBEDDING_MODEL sets; the provider's key comes from its llm4s.embeddings.<provider> block or its vendor variable, such as OPENAI_API_KEY.

Does not throw and sends no request: no model configured, an embedding provider that is not on the classpath or a missing key is a failed LlmResult of kind CONFIGURATION, whose message says what to set. A model its provider module does not list, such as an Ollama model you pulled yourself, is created: its dimensions() is empty, and its vectors are as long as the provider makes them.

Attributes

Source
Llm4s.scala

Creates a JEmbeddingClient for model, provider/model (for example openai/text-embedding-3-large or ollama/bge-m3), rather than the model llm4s.embeddings.model names - which need not be set. Everything else comes from configuration as for createDefaultEmbeddingClient: the provider's key from its llm4s.embeddings.<provider> block or its vendor variable, such as OPENAI_API_KEY, and its base URL and timeouts from that block.

Creates a JEmbeddingClient for model, provider/model (for example openai/text-embedding-3-large or ollama/bge-m3), rather than the model llm4s.embeddings.model names - which need not be set. Everything else comes from configuration as for createDefaultEmbeddingClient: the provider's key from its llm4s.embeddings.<provider> block or its vendor variable, such as OPENAI_API_KEY, and its base URL and timeouts from that block.

Does not throw and sends no request: a null model is a failed result of kind VALIDATION; a model not in the form provider/model, an embedding provider that is not on the classpath, or a missing key is one of kind CONFIGURATION, whose message says what to set. A model its provider module does not list is created, as for createDefaultEmbeddingClient.

Attributes

Source
Llm4s.scala
def gemini(): GeminiClientBuilder

A builder for a client of the Google Gemini API: apiKey and model are required (#1489).

A builder for a client of the Google Gemini API: apiKey and model are required (#1489).

Attributes

Source
Llm4s.scala
def ollama(): OllamaClientBuilder

A builder for an Ollama client: model is required, and the server is http://localhost:11434 unless set (#1489).

A builder for an Ollama client: model is required, and the server is http://localhost:11434 unless set (#1489).

Attributes

Source
Llm4s.scala
def openAi(): OpenAiClientBuilder

A builder for an OpenAI client: apiKey and model are required (#1489).

A builder for an OpenAI client: apiKey and model are required (#1489).

Attributes

Source
Llm4s.scala
def openAiCompatible(): OpenAiCompatibleClientBuilder

A builder for a client of any server that speaks OpenAI's chat-completions API - vLLM, LM Studio, LiteLLM, a gateway: baseUrl and model are required, the API key optional (#1489).

A builder for a client of any server that speaks OpenAI's chat-completions API - vLLM, LM Studio, LiteLLM, a gateway: baseUrl and model are required, the API key optional (#1489).

Attributes

Source
Llm4s.scala
def wrapAgent(agent: Agent): JAgent

Wraps an Agent built with Agent.builder, for what agentBuilder does not set: its other middleware (approvals, permissions), handoffs, subagents or runtime.

Wraps an Agent built with Agent.builder, for what agentBuilder does not set: its other middleware (approvals, permissions), handoffs, subagents or runtime.

Attributes

Source
Llm4s.scala

Wraps a Scala LLMClient you built yourself - a ReliableClient around a provider's client, your own implementation, or a test double - in a JLlmClient. Closing the wrapper closes client.

Wraps a Scala LLMClient you built yourself - a ReliableClient around a provider's client, your own implementation, or a test double - in a JLlmClient. Closing the wrapper closes client.

Attributes

Throws
NullPointerException

when client is null

Source
Llm4s.scala

Wraps a Scala EmbeddingClient you built yourself - over your own EmbeddingProvider, or a test double - in a JEmbeddingClient that asks for model and reports its dimensions.

Wraps a Scala EmbeddingClient you built yourself - over your own EmbeddingProvider, or a test double - in a JEmbeddingClient that asks for model and reports its dimensions.

Attributes

Throws
NullPointerException

when either argument is null

Source
Llm4s.scala