org.llm4s.metrics

In-process cost and usage tracker that implements MetricsCollector.

Accumulates per-model request counts, token usage, and cost data in memory with lock-free thread safety via AtomicReference + CAS. Provides query methods for retrieving aggregated statistics without external infrastructure.

Cost is stored as BigDecimal to prevent floating-point accumulation drift, matching the pattern used by UsageSummary and ModelUsage.

Example usage:

val tracker = CostTracker.create()
val client = LLMConnect.getClient(config, tracker)
// ... make LLM calls ...
println(s"Total cost: $$${tracker.totalCost}")
println(tracker.summary)

Attributes

Companion: object
Supertypes: trait MetricsCollector

class Object

trait Matchable

class Any

Attributes

Companion: class
Supertypes: class Object

trait Matchable

class Any
Self type: CostTracker.type

@Stable

Stable categorization of LLM errors for metrics.

These are stable labels safe for use in metrics dimensions. Do not use exception class names as they may change.

Attributes

Companion: object
Supertypes: class Object

trait Matchable

class Any
Known subtypes: object Authentication

object Cancelled

object ExecutionError

object Network

object RateLimit

object ServiceError

object Timeout

object Unknown

object Validation
Show all

Attributes

Companion: trait
Supertypes: trait Sum

trait Mirror

class Object

trait Matchable

class Any
Self type: ErrorKind.type

@Experimental

A metric a library module declares once and records through MetricsCollector.increment, MetricsCollector.observe and MetricsCollector.set: its name, what it measures, and the names of its labels, in the order their values are passed. It is how a module above core (the agent runtime, say) reports metrics core knows nothing about, while every MetricsCollector backend exports them the same way.

The name is lower-case words separated by dots, such as llm4s.agent.runs.started, the Micrometer convention. A Prometheus backend turns the dots into underscores and adds the unit suffix: a counter is exposed as llm4s_agent_runs_started_total, a timer as <name>_seconds, a gauge under its name. So no name ends in total, nor a timer's in seconds: the backend adds them. A gauge names its own unit, such as llm4s.agent.queue.oldest.seconds.

Label values must come from a bounded set - an outcome, a status, an agent id fixed by the code - and carry no content. Never a thread, run, request or tenant id, a prompt or a message: each distinct value is a series kept for the life of the process. A backend caps the distinct values of each label (see MetricsCollector.MaxLabelValues) and records further values as other.

Built by Metric.counter, Metric.timer and Metric.gauge, which throw IllegalArgumentException for an invalid name, label or bucket: a metric is a constant of the code that declares it, so a bad one fails where it is declared, at class initialisation, rather than at its first recording.

Attributes

Companion: object
Supertypes: class Object

trait Matchable

class Any
Known subtypes: class Counter

class Gauge

class Timer

@Experimental

Attributes

Companion: trait
Supertypes: trait Sum

trait Mirror

class Object

trait Matchable

class Any
Self type: Metric.type

@Stable

Minimal algebra for collecting metrics about LLM operations.

The core metrics model tracks request latency, request outcome, token usage, estimated cost, retry attempts, circuit-breaker transitions, generic errors, and image generation usage. Modules above core declare metrics of their own as Metrics and record them through the generic increment, observe and set, so core names none of their concepts. Concrete implementations decide where these observations are stored or exported.

Implementations should be safe: failures must not propagate to callers. All methods should catch and log errors internally without throwing.

Example usage:

val startNanos = System.nanoTime()
client.complete(conversation) match {
 case Right(completion) =>
   val duration = FiniteDuration(System.nanoTime() - startNanos, NANOSECONDS)
   metrics.observeRequest(provider, model, Outcome.Success, duration)
   completion.usage.foreach { u =>
     metrics.addTokens(provider, model, u.promptTokens, u.completionTokens)
   }
 case Left(error) =>
   val duration = FiniteDuration(System.nanoTime() - startNanos, NANOSECONDS)
   val errorKind = ErrorKind.fromLLMError(error)
   metrics.observeRequest(provider, model, Outcome.Error(errorKind), duration)
}

Attributes

Companion: object
Supertypes: class Object

trait Matchable

class Any
Known subtypes: class CostTracker

class MicrometerMetrics

class PrometheusMetrics

Attributes

Companion: trait
Supertypes: class Object

trait Matchable

class Any
Self type: MetricsCollector.type

A MetricsCollector that records into a Micrometer io.micrometer.core.instrument.MeterRegistry.

Use it when the application already has a registry (Spring Boot's Actuator registry, a Quarkus registry, a PrometheusMeterRegistry of your own) and the LLM metrics should appear next to the rest:

val metrics = new MicrometerMetrics(registry)
val client  = LLMConnect.getClient(config, metrics)

The meter names follow the series that llm4s-observability-prometheus exposes, so a dashboard built on llm4s_requests_total or llm4s_request_duration_seconds keeps working when the registry is exported as Prometheus (Micrometer's Prometheus naming turns llm4s.requests into llm4s_requests_total). See MicrometerMetrics.Names for every meter and its tags; the full table is in the monitoring guide.

Properties:

No global state: the registry is the only state, and two collectors on two registries are independent.
Safe: like every MetricsCollector, no method throws. A failure inside Micrometer (for example a MeterFilter that rejects a meter) is logged at WARN and dropped.
Bounded tag cardinality: tags are provider, model, a fixed outcome or error label, a fixed token type, and a fixed set of image operations and circuit-breaker states. Anything outside those sets is recorded as other, and a blank provider or model as unknown. Prompts, user ids, request ids and other unbounded values are never tags. provider and model come from the caller, so they are capped too: after 64 distinct providers or 256 distinct models on one collector, further values are recorded as other (logged once at WARN), so arbitrary model strings cannot grow the registry without limit.
Thread-safe: Micrometer meters are, and the only mutable state this class holds is the two sets of provider and model tag values admitted under the caps above, which are concurrent sets guarded for the check-then-add.

Differences from PrometheusMetrics: this class also implements recordRetryAttempt, recordCircuitBreakerTransition and recordError, which the Prometheus collector leaves as no-ops, and it ignores negative or non-finite amounts instead of passing them to the registry. recordError goes to its own meter, MicrometerMetrics.Names.ReliabilityErrors, never to llm4s.errors: ReliableClient reports each failed attempt through the provider client's observeRequest and then the final failure through recordError, so sharing one counter would count the same failure twice when both report to this collector. llm4s.errors therefore means exactly what llm4s_errors_total means in the Prometheus module: failed requests and failed image generations.

This module is Beta: the meter names are intended to be stable, but they are not covered by the 1.x compatibility promise.

Value parameters

registry: the registry to record into; must not be null

Attributes

Companion: object
Supertypes: trait MetricsCollector

class Object

trait Matchable

class Any

Attributes

Companion: class
Supertypes: class Object

trait Matchable

class Any
Self type: MicrometerMetrics.type

@Stable

Outcome of an LLM request.

Use Outcome.Success when the operation completed normally and Outcome.Error when it failed with a categorized ErrorKind.

Attributes

Companion: object
Supertypes: class Object

trait Matchable

class Any
Known subtypes: class Error

object Success

Attributes

Companion: trait
Supertypes: trait Sum

trait Mirror

class Object

trait Matchable

class Any
Self type: Outcome.type

HTTP endpoint for exposing Prometheus metrics.

Wraps the Prometheus HTTPServer and provides lifecycle management. Use this to expose metrics at /metrics for Prometheus scraping.

Value parameters

port: Port the server is listening on
server: Underlying HTTP server

Attributes

Example

val registry = new PrometheusRegistry()
val metrics = new PrometheusMetrics(registry)
PrometheusEndpoint.start(9090, registry) match {
 case Right(endpoint) =>
   // Use `metrics` with LLM clients while Prometheus scrapes:
   // http://localhost:9090/metrics
   endpoint.stop()
 case Left(error) =>
   println(s"Failed to start Prometheus endpoint: ${error.message}")
}

Companion

object

Supertypes

class Object

trait Matchable

class Any

Attributes

Companion: class
Supertypes: class Object

trait Matchable

class Any
Self type: PrometheusEndpoint.type

Prometheus implementation of MetricsCollector.

Tracks request volumes, token usage, errors, and latency across different providers and models using Prometheus metrics.

Registered metric names:

llm4s_requests_total
llm4s_tokens_total
llm4s_cost_usd_total
llm4s_errors_total
llm4s_request_duration_seconds
llm4s_image_generations_total
llm4s_images_generated_total
llm4s_image_generation_duration_seconds
llm4s_image_generation_cost_usd_total
llm4s_image_generation_errors_total

Generic Metrics that other modules declare (the agent runtime's run, claim, queue and child-task metrics) are registered on their first recording, their name's dots turned into underscores: a counter llm4s.agent.runs.started is exposed as llm4s_agent_runs_started_total, a timer as a histogram <name>_seconds with the metric's buckets, a gauge under its name. Label values are bounded as MetricsCollector.increment describes.

All operations are wrapped in try-catch to ensure metric failures never propagate to callers. This implementation is thread-safe.

Example usage:

val registry = new PrometheusRegistry()
val metrics = new PrometheusMetrics(registry)

// Use with endpoint
PrometheusEndpoint.start(9090, registry).foreach { endpoint =>
 // ... use metrics ...
 endpoint.stop()
}

Value parameters

registry: Prometheus collector registry

Attributes

Companion: object
Supertypes: trait MetricsCollector

class Object

trait Matchable

class Any

Attributes

Companion: class
Supertypes: class Object

trait Matchable

class Any
Self type: PrometheusMetrics.type

org.llm4s.metrics

Members list

Type members

Classlikes

Attributes

Attributes

Attributes

Attributes

Attributes

Attributes

Attributes

Attributes

Value parameters

Attributes

Attributes

Attributes

Attributes

Value parameters

Attributes

Attributes

Value parameters

Attributes

Attributes