Llm4s
Entry-point factory for Java callers.
All methods return LlmResult so Java code never imports Scala Either or deals with implicit/given parameters directly. The Scala-level ModelRegistryService given is resolved internally and passed explicitly.
=== Quick-start (Java) ===
LlmResult<JLlmClient> r = Llm4s.createDefaultClient();
if (r.isSuccess()) {
JLlmClient client = r.get();
client.complete("Hello").ifSuccess(System.out::println);
}
=== A client configured in code === The providers this module brings each have a builder, so a client needs no application.conf and no Scala type: openAi, anthropic, gemini, ollama, openAiCompatible (vLLM, LM Studio, LiteLLM, a gateway) and azureOpenAi. Every other provider - one from its own module, such as llm4s-bedrock - takes its config object through createClient, or a section of application.conf through createDefaultClient.
LlmResult<JLlmClient> r = Llm4s.openAi()
.apiKey(apiKey) // read however your application keeps its secrets
.model("gpt-4o-mini")
.build();
Attributes
- Source
- Llm4s.scala
- Graph
-
- Supertypes
- Self type
-
Llm4s.type
Members list
Value members
Concrete methods
A builder for an agent called id over client, with a system prompt, tools, input and output guardrails (JGuardrails), a context window (JContextWindow) and streaming, set from Java (#2476). createAgent and createAgentWithTools are its zero-option cases. Neither argument is checked here: a null one is a failed result of build().
A builder for an agent called id over client, with a system prompt, tools, input and output guardrails (JGuardrails), a context window (JContextWindow) and streaming, set from Java (#2476). createAgent and createAgentWithTools are its zero-option cases. Neither argument is checked here: a null one is a failed result of build().
LlmResult<JAgent> agent = Llm4s.agentBuilder("support", client)
.systemPrompt("You are a support assistant.")
.inputGuardrails(List.of(JGuardrails.length(1, 10_000)))
.contextWindow(JContextWindow.lastMessages(20))
.build();
Attributes
- Source
- Llm4s.scala
A builder for the implementation of serviceInterface, an AI service (#2277): an interface whose methods are annotated with @SystemMessage, @UserMessage, @V and @MemoryId from org.llm4s.javaapi.services. Each call of a method renders its prompt from its arguments, asks the model - in one agent turn when the service has tools - and returns the answer as the method's return type. Experimental.
A builder for the implementation of serviceInterface, an AI service (#2277): an interface whose methods are annotated with @SystemMessage, @UserMessage, @V and @MemoryId from org.llm4s.javaapi.services. Each call of a method renders its prompt from its arguments, asks the model - in one agent turn when the service has tools - and returns the answer as the method's return type. Experimental.
interface Assistant {
@SystemMessage("You are a support agent for {{product}}. Answer briefly.")
String answer(@UserMessage String question, @V("product") String product);
}
Assistant assistant = Llm4s.aiService(Assistant.class).client(client).build();
Attributes
- Throws
-
NullPointerException
when
serviceInterfaceisnull - Source
- Llm4s.scala
A builder for an Anthropic client: apiKey and model are required (#1489).
A builder for an Anthropic client: apiKey and model are required (#1489).
Attributes
- Source
- Llm4s.scala
A builder for an Azure OpenAI client: endpoint, apiKey and model (the deployment) are required (#1489).
A builder for an Azure OpenAI client: endpoint, apiKey and model (the deployment) are required (#1489).
Attributes
- Source
- Llm4s.scala
Wraps a JLlmClient in a JAgent ready to accept natural-language queries, with no tools.
Wraps a JLlmClient in a JAgent ready to accept natural-language queries, with no tools.
Attributes
- Source
- Llm4s.scala
Wraps a JLlmClient in a JAgent that can call tools. Tools belong to the agent: an agent whose tools cannot be offered together (a name clash, an invalid schema) fails every run with the reason.
Wraps a JLlmClient in a JAgent that can call tools. Tools belong to the agent: an agent whose tools cannot be offered together (a name clash, an invalid schema) fails every run with the reason.
Attributes
- Source
- Llm4s.scala
createAgent with tools, whose model calls stream when streaming is set: then a JAgent.stream of it carries the answer's text deltas (AgentEvents.TextDelta()). Without it, the agent calls the model's complete, and a stream carries no deltas.
createAgent with tools, whose model calls stream when streaming is set: then a JAgent.stream of it carries the answer's text deltas (AgentEvents.TextDelta()). Without it, the agent calls the model's complete, and a stream carries no deltas.
Attributes
- Source
- Llm4s.scala
Wraps a JLlmClient in a JAgent that can call tools, defined from Java with JTool.builder or JTool.fromAnnotated (#1484). A null argument, a tool listed twice, two tools with one name, or tools the agent cannot offer together is a failed result, not an agent that fails every run.
Wraps a JLlmClient in a JAgent that can call tools, defined from Java with JTool.builder or JTool.fromAnnotated (#1484). A null argument, a tool listed twice, two tools with one name, or tools the agent cannot offer together is a failed result, not an agent that fails every run.
JAgent agent = Llm4s.createAgentWithTools(client, List.of(weather)).get();
Attributes
- Source
- Llm4s.scala
Creates a JLlmClient from an explicit ProviderConfig. Useful when the caller constructs the config programmatically rather than relying on environment variables.
Creates a JLlmClient from an explicit ProviderConfig. Useful when the caller constructs the config programmatically rather than relying on environment variables.
Attributes
- Source
- Llm4s.scala
createClient whose client reports every call (request outcome, duration, tokens and cost) to metrics, for example a MicrometerMetrics from llm4s-observability-micrometer over the application's MeterRegistry, or a PrometheusMetrics from llm4s-observability-prometheus. The Spring Boot starter uses it to attach the application's MetricsCollector bean. A null metrics records nothing; a null config is a failed result.
createClient whose client reports every call (request outcome, duration, tokens and cost) to metrics, for example a MicrometerMetrics from llm4s-observability-micrometer over the application's MeterRegistry, or a PrometheusMetrics from llm4s-observability-prometheus. The Spring Boot starter uses it to attach the application's MetricsCollector bean. A null metrics records nothing; a null config is a failed result.
Attributes
- Source
- Llm4s.scala
Creates a JLlmClient from the default provider: the section of application.conf that llm4s.providers.provider names. A provider's API key comes from its section's apiKey, else from llm4s.credentials (bound to the vendor's usual environment variable, e.g. OPENAI_API_KEY).
Creates a JLlmClient from the default provider: the section of application.conf that llm4s.providers.provider names. A provider's API key comes from its section's apiKey, else from llm4s.credentials (bound to the vendor's usual environment variable, e.g. OPENAI_API_KEY).
Attributes
- Source
- Llm4s.scala
Creates a JEmbeddingClient for the embedding model configured as chat providers are, in the application's application.conf: llm4s.embeddings.model, provider/model (for example openai/text-embedding-3-small), which EMBEDDING_MODEL sets; the provider's key comes from its llm4s.embeddings.<provider> block or its vendor variable, such as OPENAI_API_KEY.
Creates a JEmbeddingClient for the embedding model configured as chat providers are, in the application's application.conf: llm4s.embeddings.model, provider/model (for example openai/text-embedding-3-small), which EMBEDDING_MODEL sets; the provider's key comes from its llm4s.embeddings.<provider> block or its vendor variable, such as OPENAI_API_KEY.
Does not throw and sends no request: no model configured, an embedding provider that is not on the classpath or a missing key is a failed LlmResult of kind CONFIGURATION, whose message says what to set. A model its provider module does not list, such as an Ollama model you pulled yourself, is created: its dimensions() is empty, and its vectors are as long as the provider makes them.
Attributes
- Source
- Llm4s.scala
Creates a JEmbeddingClient for model, provider/model (for example openai/text-embedding-3-large or ollama/bge-m3), rather than the model llm4s.embeddings.model names - which need not be set. Everything else comes from configuration as for createDefaultEmbeddingClient: the provider's key from its llm4s.embeddings.<provider> block or its vendor variable, such as OPENAI_API_KEY, and its base URL and timeouts from that block.
Creates a JEmbeddingClient for model, provider/model (for example openai/text-embedding-3-large or ollama/bge-m3), rather than the model llm4s.embeddings.model names - which need not be set. Everything else comes from configuration as for createDefaultEmbeddingClient: the provider's key from its llm4s.embeddings.<provider> block or its vendor variable, such as OPENAI_API_KEY, and its base URL and timeouts from that block.
Does not throw and sends no request: a null model is a failed result of kind VALIDATION; a model not in the form provider/model, an embedding provider that is not on the classpath, or a missing key is one of kind CONFIGURATION, whose message says what to set. A model its provider module does not list is created, as for createDefaultEmbeddingClient.
Attributes
- Source
- Llm4s.scala
A builder for a client of the Google Gemini API: apiKey and model are required (#1489).
A builder for a client of the Google Gemini API: apiKey and model are required (#1489).
Attributes
- Source
- Llm4s.scala
A builder for an Ollama client: model is required, and the server is http://localhost:11434 unless set (#1489).
A builder for an Ollama client: model is required, and the server is http://localhost:11434 unless set (#1489).
Attributes
- Source
- Llm4s.scala
A builder for an OpenAI client: apiKey and model are required (#1489).
A builder for an OpenAI client: apiKey and model are required (#1489).
Attributes
- Source
- Llm4s.scala
A builder for a client of any server that speaks OpenAI's chat-completions API - vLLM, LM Studio, LiteLLM, a gateway: baseUrl and model are required, the API key optional (#1489).
A builder for a client of any server that speaks OpenAI's chat-completions API - vLLM, LM Studio, LiteLLM, a gateway: baseUrl and model are required, the API key optional (#1489).
Attributes
- Source
- Llm4s.scala
Wraps an Agent built with Agent.builder, for what agentBuilder does not set: its other middleware (approvals, permissions), handoffs, subagents or runtime.
Wraps an Agent built with Agent.builder, for what agentBuilder does not set: its other middleware (approvals, permissions), handoffs, subagents or runtime.
Attributes
- Source
- Llm4s.scala
Wraps a Scala LLMClient you built yourself - a ReliableClient around a provider's client, your own implementation, or a test double - in a JLlmClient. Closing the wrapper closes client.
Wraps a Scala LLMClient you built yourself - a ReliableClient around a provider's client, your own implementation, or a test double - in a JLlmClient. Closing the wrapper closes client.
Attributes
- Throws
-
NullPointerException
when
clientisnull - Source
- Llm4s.scala
Wraps a Scala EmbeddingClient you built yourself - over your own EmbeddingProvider, or a test double - in a JEmbeddingClient that asks for model and reports its dimensions.
Wraps a Scala EmbeddingClient you built yourself - over your own EmbeddingProvider, or a test double - in a JEmbeddingClient that asks for model and reports its dimensions.
Attributes
- Throws
-
NullPointerException
when either argument is
null - Source
- Llm4s.scala
