org.llm4s.spring

Members list

Type members

Classlikes

The auto-configured agent (llm4s.agent.*, #2279).

The auto-configured agent (llm4s.agent.*, #2279).

Attributes

Source
Llm4sProperties.scala
Supertypes
class Object
trait Matchable
class Any

Sizing of the executor behind LLM4STemplate's completeAsync (llm4s.async.*).

Sizing of the executor behind LLM4STemplate's completeAsync (llm4s.async.*).

Attributes

Source
Llm4sProperties.scala
Supertypes
class Object
trait Matchable
class Any

Health indicator settings (llm4s.health.*).

Health indicator settings (llm4s.health.*).

Attributes

Source
Llm4sProperties.scala
Supertypes
class Object
trait Matchable
class Any
final case class HealthSettings(provider: String, model: String, probe: Boolean, ttl: Duration, timeout: Duration, secrets: Seq[String])

What the indicator reports about and how it probes; the secrets are only used to redact messages.

What the indicator reports about and how it probes; the secrets are only used to redact messages.

Attributes

Companion
object
Source
LlmHealthProbe.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all

Attributes

Companion
class
Source
LlmHealthProbe.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
final class LLM4STemplate(client: JLlmClient, executor: ExecutorService)

Spring-friendly entry point to an LLM.

Spring-friendly entry point to an LLM.

complete and tryComplete block the calling thread. completeAsync never does: it is JLlmClient.completeAsync with the call on executor (the llm4sTaskExecutor bean), and the returned future completes from there. Cancelling the future with cancel(true) interrupts the provider call, which is how llm4s cancels work; a failure completes the future exceptionally with the LlmException, and so does an executor that refuses the call (its RejectedExecutionException the cause).

Attributes

Source
LLM4STemplate.scala
Supertypes
class Object
trait Matchable
class Any

Attributes

Companion
class
Source
Llm4sAutoConfiguration.scala
Supertypes
class Object
trait Matchable
class Any
Self type
@AutoConfiguration(value = _, before = _, beforeName = _, after = _, afterName = _) @Configuration(value = _, proxyBeanMethods = _, enforceUniqueMethods = _) @ConditionalOnClass(value = _, name = scala.Array.apply[java.lang.String]("org.llm4s.javaapi.Llm4s$")(scala.reflect.ClassTag.apply[java.lang.String](classOf[java.lang.String]))) @ConditionalOnProperty(value = _, prefix = "llm4s", name = scala.Array.apply[java.lang.String]("enabled")(scala.reflect.ClassTag.apply[java.lang.String](classOf[java.lang.String])), havingValue = "true", matchIfMissing = true) @EnableConfigurationProperties(value = scala.Array.apply[java.lang.Class[_ >: scala.Nothing <: . ]](scala.Predef.classOf[org.llm4s.spring.Llm4sProperties])(scala.reflect.ClassTag.apply[java.lang.Class[_ >: scala.Nothing <: . ]](classOf[java.lang.Class])))

Attributes

Companion
object
Source
Llm4sAutoConfiguration.scala
Supertypes
class Object
trait Matchable
class Any

Name of the executor bean behind completeAsync and the health probe; define a bean of this name to replace it.

Name of the executor bean behind completeAsync and the health probe; define a bean of this name to replace it.

Attributes

Source
Llm4sAutoConfiguration.scala
Supertypes
class Object
trait Matchable
class Any
Self type
@ConfigurationProperties(value = _, prefix = "llm4s", ignoreInvalidFields = _, ignoreUnknownFields = _)

Attributes

Source
Llm4sProperties.scala
Supertypes
class Object
trait Matchable
class Any
@AutoConfiguration(value = _, before = _, beforeName = _, after = scala.Array.apply[java.lang.Class[_ >: scala.Nothing <: . ]](scala.Predef.classOf[org.llm4s.spring.Llm4sAutoConfiguration])(scala.reflect.ClassTag.apply[java.lang.Class[_ >: scala.Nothing <: . ]](classOf[java.lang.Class])), afterName = _) @Configuration(value = _, proxyBeanMethods = _, enforceUniqueMethods = _) @ConditionalOnClass(value = _, name = scala.Array.apply[java.lang.String]("reactor.core.publisher.Flux", "org.llm4s.javaapi.reactor.Streams")(scala.reflect.ClassTag.apply[java.lang.String](classOf[java.lang.String]))) @ConditionalOnBean(value = scala.Array.apply[java.lang.Class[_ >: scala.Nothing <: . ]](scala.Predef.classOf[org.llm4s.javaapi.JLlmClient])(scala.reflect.ClassTag.apply[java.lang.Class[_ >: scala.Nothing <: . ]](classOf[java.lang.Class])), type = _, annotation = _, name = _, search = _, parameterizedContainer = _) @ConditionalOnProperty(value = _, prefix = "llm4s", name = scala.Array.apply[java.lang.String]("enabled")(scala.reflect.ClassTag.apply[java.lang.String](classOf[java.lang.String])), havingValue = "true", matchIfMissing = true)

The llm4sReactiveTemplate bean (#2279): the client's replies as Reactor Fluxes, for Spring WebFlux. Active when Reactor and llm4s-reactor are on the classpath, both optional dependencies of the starter. A bean of type ReactiveLLM4STemplate replaces it.

The llm4sReactiveTemplate bean (#2279): the client's replies as Reactor Fluxes, for Spring WebFlux. Active when Reactor and llm4s-reactor are on the classpath, both optional dependencies of the starter. A bean of type ReactiveLLM4STemplate replaces it.

Attributes

Source
Llm4sStreamingAutoConfiguration.scala
Supertypes
class Object
trait Matchable
class Any
@AutoConfiguration(value = _, before = _, beforeName = _, after = scala.Array.apply[java.lang.Class[_ >: scala.Nothing <: . ]](scala.Predef.classOf[org.llm4s.spring.Llm4sAutoConfiguration])(scala.reflect.ClassTag.apply[java.lang.Class[_ >: scala.Nothing <: . ]](classOf[java.lang.Class])), afterName = _) @Configuration(value = _, proxyBeanMethods = _, enforceUniqueMethods = _) @ConditionalOnClass(value = _, name = scala.Array.apply[java.lang.String]("org.springframework.web.servlet.mvc.method.annotation.SseEmitter")(scala.reflect.ClassTag.apply[java.lang.String](classOf[java.lang.String]))) @ConditionalOnBean(value = scala.Array.apply[java.lang.Class[_ >: scala.Nothing <: . ]](scala.Predef.classOf[org.llm4s.javaapi.JLlmClient])(scala.reflect.ClassTag.apply[java.lang.Class[_ >: scala.Nothing <: . ]](classOf[java.lang.Class])), type = _, annotation = _, name = _, search = _, parameterizedContainer = _) @ConditionalOnProperty(value = _, prefix = "llm4s", name = scala.Array.apply[java.lang.String]("enabled")(scala.reflect.ClassTag.apply[java.lang.String](classOf[java.lang.String])), havingValue = "true", matchIfMissing = true)

The llm4sSseTemplate bean (#2279): the client's replies as Spring MVC SseEmitters. Active when Spring MVC is on the classpath, which the starter declares provided. A bean of type SseLLM4STemplate replaces it.

The llm4sSseTemplate bean (#2279): the client's replies as Spring MVC SseEmitters. Active when Spring MVC is on the classpath, which the starter declares provided. A bean of type SseLLM4STemplate replaces it.

Attributes

Source
Llm4sStreamingAutoConfiguration.scala
Supertypes
class Object
trait Matchable
class Any
final class Llm4sTools(declared: List[JTool])

The agent tools the application's beans declare (#2279): each public method annotated with llm4s's @Tool (org.llm4s.javaapi.Tool) on a singleton bean, read as JTool.fromAnnotated reads it. The llm4sAgent bean offers them to its model; inject this bean to give them to an agent of your own.

The agent tools the application's beans declare (#2279): each public method annotated with llm4s's @Tool (org.llm4s.javaapi.Tool) on a singleton bean, read as JTool.fromAnnotated reads it. The llm4sAgent bean offers them to its model; inject this bean to give them to an agent of your own.

@Component
public class WeatherTools {
   @Tool(description = "The current weather in a city")
   public String weather(@Param(name = "city", description = "City name") String city) { ... }
}

Attributes

Companion
object
Source
Llm4sTools.scala
Supertypes
class Object
trait Matchable
class Any
object Llm4sTools

Attributes

Companion
class
Source
Llm4sTools.scala
Supertypes
class Object
trait Matchable
class Any
Self type
Llm4sTools.type
@AutoConfiguration(value = _, before = _, beforeName = _, after = scala.Array.apply[java.lang.Class[_ >: scala.Nothing <: . ]](scala.Predef.classOf[org.llm4s.spring.Llm4sAutoConfiguration])(scala.reflect.ClassTag.apply[java.lang.Class[_ >: scala.Nothing <: . ]](classOf[java.lang.Class])), afterName = _) @Configuration(value = _, proxyBeanMethods = _, enforceUniqueMethods = _) @ConditionalOnClass(value = _, name = scala.Array.apply[java.lang.String]("org.springframework.boot.actuate.health.HealthIndicator")(scala.reflect.ClassTag.apply[java.lang.String](classOf[java.lang.String]))) @ConditionalOnBean(value = scala.Array.apply[java.lang.Class[_ >: scala.Nothing <: . ]](scala.Predef.classOf[org.llm4s.javaapi.JLlmClient])(scala.reflect.ClassTag.apply[java.lang.Class[_ >: scala.Nothing <: . ]](classOf[java.lang.Class])), type = _, annotation = _, name = _, search = _, parameterizedContainer = _) @ConditionalOnProperty(value = _, prefix = "llm4s", name = scala.Array.apply[java.lang.String]("enabled")(scala.reflect.ClassTag.apply[java.lang.String](classOf[java.lang.String])), havingValue = "true", matchIfMissing = true)

Attributes

Source
LlmActuatorAutoConfiguration.scala
Supertypes
class Object
trait Matchable
class Any
@AutoConfiguration(value = _, before = _, beforeName = _, after = scala.Array.apply[java.lang.Class[_ >: scala.Nothing <: . ]](scala.Predef.classOf[org.llm4s.spring.Llm4sAutoConfiguration])(scala.reflect.ClassTag.apply[java.lang.Class[_ >: scala.Nothing <: . ]](classOf[java.lang.Class])), afterName = _) @Configuration(value = _, proxyBeanMethods = _, enforceUniqueMethods = _) @ConditionalOnClass(value = _, name = scala.Array.apply[java.lang.String]("org.springframework.boot.health.contributor.HealthIndicator")(scala.reflect.ClassTag.apply[java.lang.String](classOf[java.lang.String]))) @ConditionalOnMissingClass(value = scala.Array.apply[java.lang.String]("org.springframework.boot.actuate.health.HealthIndicator")(scala.reflect.ClassTag.apply[java.lang.String](classOf[java.lang.String]))) @ConditionalOnBean(value = scala.Array.apply[java.lang.Class[_ >: scala.Nothing <: . ]](scala.Predef.classOf[org.llm4s.javaapi.JLlmClient])(scala.reflect.ClassTag.apply[java.lang.Class[_ >: scala.Nothing <: . ]](classOf[java.lang.Class])), type = _, annotation = _, name = _, search = _, parameterizedContainer = _) @ConditionalOnProperty(value = _, prefix = "llm4s", name = scala.Array.apply[java.lang.String]("enabled")(scala.reflect.ClassTag.apply[java.lang.String](classOf[java.lang.String])), havingValue = "true", matchIfMissing = true)

LlmActuatorAutoConfiguration for Spring Boot 4: the same llmHealthIndicator bean, as an LlmBoot4HealthIndicator, when Spring Boot 4's spring-boot-health (which spring-boot-starter-actuator brings) is on the classpath and Spring Boot 3's org.springframework.boot.actuate.health.HealthIndicator is not. Spring Boot 3 has no org.springframework.boot.health.contributor.HealthIndicator and Spring Boot 4 no org.springframework.boot.actuate.health.HealthIndicator; the second condition keeps a Boot 3 application that has a stray Boot 4 spring-boot-health jar to LlmActuatorAutoConfiguration alone, so one indicator is registered.

LlmActuatorAutoConfiguration for Spring Boot 4: the same llmHealthIndicator bean, as an LlmBoot4HealthIndicator, when Spring Boot 4's spring-boot-health (which spring-boot-starter-actuator brings) is on the classpath and Spring Boot 3's org.springframework.boot.actuate.health.HealthIndicator is not. Spring Boot 3 has no org.springframework.boot.health.contributor.HealthIndicator and Spring Boot 4 no org.springframework.boot.actuate.health.HealthIndicator; the second condition keeps a Boot 3 application that has a stray Boot 4 spring-boot-health jar to LlmActuatorAutoConfiguration alone, so one indicator is registered.

Attributes

Source
LlmBoot4ActuatorAutoConfiguration.scala
Supertypes
class Object
trait Matchable
class Any
final class LlmBoot4HealthIndicator(client: JLlmClient, settings: HealthSettings, executor: ExecutorService, nanoClock: () => Long) extends HealthIndicator

The Spring Boot 4 health indicator, registered by LlmBoot4ActuatorAutoConfiguration. Spring Boot 4 moved HealthIndicator and Health from org.springframework.boot.actuate.health (spring-boot-actuator) to org.springframework.boot.health.contributor (spring-boot-health), so the Boot 3 LlmHealthIndicator cannot be one; this reports exactly what it does, from the same probe, settings and cache rules.

The Spring Boot 4 health indicator, registered by LlmBoot4ActuatorAutoConfiguration. Spring Boot 4 moved HealthIndicator and Health from org.springframework.boot.actuate.health (spring-boot-actuator) to org.springframework.boot.health.contributor (spring-boot-health), so the Boot 3 LlmHealthIndicator cannot be one; this reports exactly what it does, from the same probe, settings and cache rules.

Attributes

Source
LlmBoot4HealthIndicator.scala
Supertypes
trait HealthIndicator
trait HealthContributor
class Object
trait Matchable
class Any
final class LlmHealthIndicator(client: JLlmClient, settings: HealthSettings, executor: ExecutorService, nanoClock: () => Long) extends HealthIndicator

The Spring Boot 3 Actuator health indicator, registered by LlmActuatorAutoConfiguration. By default (llm4s.health.probe=false) it makes no provider call: it is UP with probe=disabled and the provider and model names, meaning "configured", not "reachable". With llm4s.health.probe=true it sends a one-token completion on executor, waits at most llm4s.health.probe-timeout, and caches the outcome for llm4s.health.probe-ttl. A failed or timed-out probe is DOWN with a redacted message.

The Spring Boot 3 Actuator health indicator, registered by LlmActuatorAutoConfiguration. By default (llm4s.health.probe=false) it makes no provider call: it is UP with probe=disabled and the provider and model names, meaning "configured", not "reachable". With llm4s.health.probe=true it sends a one-token completion on executor, waits at most llm4s.health.probe-timeout, and caches the outcome for llm4s.health.probe-ttl. A failed or timed-out probe is DOWN with a redacted message.

health() never throws InterruptedException: a thread interrupted while it waits for the probe gets DOWN with probe=cancelled, its interrupt flag set again, and the probe call is cancelled. That outcome is not cached: it is about the interrupted thread, not the provider, so the next check probes again.

On Spring Boot 4, whose HealthIndicator is another type, LlmBoot4HealthIndicator reports the same.

Attributes

Source
LlmHealthIndicator.scala
Supertypes
trait HealthIndicator
trait HealthContributor
class Object
trait Matchable
class Any
@AutoConfiguration(value = _, before = scala.Array.apply[java.lang.Class[_ >: scala.Nothing <: . ]](scala.Predef.classOf[org.llm4s.spring.Llm4sAutoConfiguration])(scala.reflect.ClassTag.apply[java.lang.Class[_ >: scala.Nothing <: . ]](classOf[java.lang.Class])), beforeName = _, after = _, afterName = scala.Array.apply[java.lang.String]("org.springframework.boot.actuate.autoconfigure.metrics.MetricsAutoConfiguration", "org.springframework.boot.actuate.autoconfigure.metrics.CompositeMeterRegistryAutoConfiguration", "org.springframework.boot.actuate.autoconfigure.metrics.export.simple.SimpleMetricsExportAutoConfiguration", "org.springframework.boot.micrometer.metrics.autoconfigure.MetricsAutoConfiguration", "org.springframework.boot.micrometer.metrics.autoconfigure.CompositeMeterRegistryAutoConfiguration", "org.springframework.boot.micrometer.metrics.autoconfigure.export.simple.SimpleMetricsExportAutoConfiguration")(scala.reflect.ClassTag.apply[java.lang.String](classOf[java.lang.String]))) @Configuration(value = _, proxyBeanMethods = _, enforceUniqueMethods = _) @ConditionalOnClass(value = _, name = scala.Array.apply[java.lang.String]("io.micrometer.core.instrument.MeterRegistry", "org.llm4s.metrics.MicrometerMetrics")(scala.reflect.ClassTag.apply[java.lang.String](classOf[java.lang.String]))) @ConditionalOnBean(value = _, type = scala.Array.apply[java.lang.String]("io.micrometer.core.instrument.MeterRegistry")(scala.reflect.ClassTag.apply[java.lang.String](classOf[java.lang.String])), annotation = _, name = _, search = _, parameterizedContainer = _) @ConditionalOnProperty(value = _, prefix = "llm4s", name = scala.Array.apply[java.lang.String]("enabled")(scala.reflect.ClassTag.apply[java.lang.String](classOf[java.lang.String])), havingValue = "true", matchIfMissing = true)

Records llm4s request, token, cost and error metrics in the application's Micrometer MeterRegistry, so they appear on /actuator/metrics (and in whatever the registry exports to) with no wiring.

Records llm4s request, token, cost and error metrics in the application's Micrometer MeterRegistry, so they appear on /actuator/metrics (and in whatever the registry exports to) with no wiring.

Active when llm4s-observability-micrometer is on the classpath and the context has a MeterRegistry bean, as Spring Boot Actuator provides. The starter declares that module provided, like Actuator itself, so an application opts in by adding the dependency. A MetricsCollector bean of the application's own replaces this one; the client bean reports to every MetricsCollector bean there is.

It runs after the auto-configurations that create the MeterRegistry, so @ConditionalOnBean sees it. They are named as strings, because Actuator is optional, and Spring Boot 4 moved them from org.springframework.boot.actuate.autoconfigure.metrics (spring-boot-actuator-autoconfigure) to org.springframework.boot.micrometer.metrics.autoconfigure (spring-boot-micrometer-metrics), so both names are listed; Spring ignores a name that is not on the classpath. LlmMetricsAutoConfigurationOrderingSpec fails if a Boot 3 name stops resolving on the Boot version the starter is tested against, or if the Boot 4 names are dropped.

Attributes

Source
LlmMetricsAutoConfiguration.scala
Supertypes
class Object
trait Matchable
class Any

Attributes

Source
ProviderConfigParser.scala
Supertypes
class Object
trait Matchable
class Any
Self type
final class ReactiveLLM4STemplate(client: JLlmClient)

A model's reply as a Reactor Flux<String> of its text (#2279), for Spring WebFlux: return it from a controller with produces = text/event-stream and each piece reaches the browser as the model writes it.

A model's reply as a Reactor Flux<String> of its text (#2279), for Spring WebFlux: return it from a controller with produces = text/event-stream and each piece reaches the browser as the model writes it.

The llm4sReactiveTemplate bean, registered when Reactor and llm4s-reactor are on the classpath. A class of its own, not methods of LLM4STemplate, so that an application without Reactor can still load that one. Each Flux is Streams.toFlux's: cold (each subscription makes its own provider call), failing with an LlmException, and a cancelled subscription - a client that disconnects - interrupts the provider call.

Attributes

Source
ReactiveLLM4STemplate.scala
Supertypes
class Object
trait Matchable
class Any
final class SseLLM4STemplate(client: JLlmClient)

A model's reply as a Spring MVC SseEmitter (#2279): return it from a controller and each piece of the reply's text reaches the browser as one server-sent event, as the model writes it.

A model's reply as a Spring MVC SseEmitter (#2279): return it from a controller and each piece of the reply's text reaches the browser as one server-sent event, as the model writes it.

The llm4sSseTemplate bean, registered when Spring MVC is on the classpath. The provider call runs on a thread of its own (JLlmClient.streamAsPublisher), so the request thread is free at once. The emitter completes after the last piece, and completes with the LlmException when the call fails. When the client disconnects or the request times out, the provider call is interrupted.

Attributes

Source
SseLLM4STemplate.scala
Supertypes
class Object
trait Matchable
class Any