core
API
Generated with
core/org.llm4s/org.llm4s.metrics

org.llm4s.metrics

package org.llm4s.metrics

Members list

Type members

Classlikes

final class CostTracker extends MetricsCollector

In-process cost and usage tracker that implements MetricsCollector.

In-process cost and usage tracker that implements MetricsCollector.

Accumulates per-model request counts, token usage, and cost data in memory with lock-free thread safety via AtomicReference + CAS. Provides query methods for retrieving aggregated statistics without external infrastructure.

Cost is stored as BigDecimal to prevent floating-point accumulation drift, matching the pattern used by UsageSummary and ModelUsage.

Example usage:

val tracker = CostTracker.create()
val client = LLMConnect.getClient(config, tracker)
// ... make LLM calls ...
println(s"Total cost: $$${tracker.totalCost}")
println(tracker.summary)

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
object CostTracker

Attributes

Companion
class
Supertypes
class Object
trait Matchable
class Any
Self type
sealed trait ErrorKind

Stable categorization of LLM errors for metrics.

Stable categorization of LLM errors for metrics.

These are stable labels safe for use in metrics dimensions. Do not use exception class names as they may change.

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
Known subtypes
object Network
object RateLimit
object ServiceError
object Timeout
object Unknown
object Validation
Show all
object ErrorKind

Attributes

Companion
trait
Supertypes
trait Sum
trait Mirror
class Object
trait Matchable
class Any
Self type
ErrorKind.type
trait MetricsCollector

Minimal algebra for collecting metrics about LLM operations.

Minimal algebra for collecting metrics about LLM operations.

The core metrics model tracks request latency, request outcome, token usage, estimated cost, retry attempts, circuit-breaker transitions, generic errors, and image generation usage. Concrete implementations decide where these observations are stored or exported.

Implementations should be safe: failures must not propagate to callers. All methods should catch and log errors internally without throwing.

Example usage:

val startNanos = System.nanoTime()
client.complete(conversation) match {
 case Right(completion) =>
   val duration = FiniteDuration(System.nanoTime() - startNanos, NANOSECONDS)
   metrics.observeRequest(provider, model, Outcome.Success, duration)
   completion.usage.foreach { u =>
     metrics.addTokens(provider, model, u.promptTokens, u.completionTokens)
   }
 case Left(error) =>
   val duration = FiniteDuration(System.nanoTime() - startNanos, NANOSECONDS)
   val errorKind = ErrorKind.fromLLMError(error)
   metrics.observeRequest(provider, model, Outcome.Error(errorKind), duration)
}

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
Known subtypes
class CostTracker
object MetricsCollector

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type
sealed trait Outcome

Outcome of an LLM request.

Outcome of an LLM request.

Use Outcome.Success when the operation completed normally and Outcome.Error when it failed with a categorized ErrorKind.

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
Known subtypes
class Error
object Success
object Outcome

Attributes

Companion
trait
Supertypes
trait Sum
trait Mirror
class Object
trait Matchable
class Any
Self type
Outcome.type
final class PrometheusEndpoint

HTTP endpoint for exposing Prometheus metrics.

HTTP endpoint for exposing Prometheus metrics.

Wraps the Prometheus HTTPServer and provides lifecycle management. Use this to expose metrics at /metrics for Prometheus scraping.

Value parameters

port

Port the server is listening on

server

Underlying HTTP server

Attributes

Example 
val registry = new PrometheusRegistry()
val metrics = new PrometheusMetrics(registry)
PrometheusEndpoint.start(9090, registry) match {
 case Right(endpoint) =>
   // Use `metrics` with LLM clients while Prometheus scrapes:
   // http://localhost:9090/metrics
   endpoint.stop()
 case Left(error) =>
   println(s"Failed to start Prometheus endpoint: ${error.message}")
}
Companion
object
Supertypes
class Object
trait Matchable
class Any
object PrometheusEndpoint

Attributes

Companion
class
Supertypes
class Object
trait Matchable
class Any
Self type
final class PrometheusMetrics(val registry: PrometheusRegistry) extends MetricsCollector

Prometheus implementation of MetricsCollector.

Prometheus implementation of MetricsCollector.

Tracks request volumes, token usage, errors, and latency across different providers and models using Prometheus metrics.

Registered metric names:

  • llm4s_requests_total
  • llm4s_tokens_total
  • llm4s_cost_usd_total
  • llm4s_errors_total
  • llm4s_request_duration_seconds
  • llm4s_image_generations_total
  • llm4s_images_generated_total
  • llm4s_image_generation_duration_seconds
  • llm4s_image_generation_cost_usd_total
  • llm4s_image_generation_errors_total

All operations are wrapped in try-catch to ensure metric failures never propagate to callers. This implementation is thread-safe.

Example usage:

val registry = new PrometheusRegistry()
val metrics = new PrometheusMetrics(registry)

// Use with endpoint
PrometheusEndpoint.start(9090, registry).foreach { endpoint =>
 // ... use metrics ...
 endpoint.stop()
}

Value parameters

registry

Prometheus collector registry

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
object PrometheusMetrics

Attributes

Companion
class
Supertypes
class Object
trait Matchable
class Any
Self type
Generated with