Minimal algebra for collecting metrics about LLM operations.
The core metrics model tracks request latency, request outcome, token usage, estimated cost, retry attempts, circuit-breaker transitions, generic errors, and image generation usage. Modules above core declare metrics of their own as Metrics and record them through the generic increment, observe and set, so core names none of their concepts. Concrete implementations decide where these observations are stored or exported.
Implementations should be safe: failures must not propagate to callers. All methods should catch and log errors internally without throwing.
Example usage:
val startNanos = System.nanoTime()
client.complete(conversation) match {
case Right(completion) =>
val duration = FiniteDuration(System.nanoTime() - startNanos, NANOSECONDS)
metrics.observeRequest(provider, model, Outcome.Success, duration)
completion.usage.foreach { u =>
metrics.addTokens(provider, model, u.promptTokens, u.completionTokens)
}
case Left(error) =>
val duration = FiniteDuration(System.nanoTime() - startNanos, NANOSECONDS)
val errorKind = ErrorKind.fromLLMError(error)
metrics.observeRequest(provider, model, Outcome.Error(errorKind), duration)
}
Attributes
- Companion
- object
- Graph
-
- Supertypes
-
class Objecttrait Matchableclass Any
- Known subtypes
Members list
Value members
Abstract methods
Record token usage.
Record token usage.
Implementations typically record input and output tokens separately so dashboards can show prompt and completion usage independently.
Value parameters
- inputTokens
-
Number of input/prompt tokens
- model
-
Model name
- outputTokens
-
Number of output/completion tokens
- provider
-
Provider name
Attributes
Record an LLM request with its outcome and duration.
Record an LLM request with its outcome and duration.
The outcome dimension separates successful requests from failures. Error outcomes carry a stable ErrorKind such as ErrorKind.RateLimit, ErrorKind.Timeout, or ErrorKind.ServiceError so metrics backends can aggregate failures without depending on exception class names.
Value parameters
- duration
-
Request duration
- model
-
Model name (e.g., "gpt-4o", "claude-3-5-sonnet-latest")
- outcome
-
Success or Error with stable error kind
- provider
-
Provider name (e.g., "openai", "anthropic", "ollama")
Attributes
Record estimated cost in USD.
Record estimated cost in USD.
Use this after pricing metadata is available for a request or image generation operation. Implementations should treat the value as an additive counter.
Value parameters
- costUsd
-
Estimated cost in USD
- model
-
Model name
- provider
-
Provider name
Attributes
Concrete methods
Add amount to counter metric, with labelValues for its label names, in order.
Add amount to counter metric, with labelValues for its label names, in order.
The generic entry point for metrics a module above core declares (see Metric): the agent runtime's run, claim, queue and child-task metrics come through here, so a backend exports them with no dependency on that module. A backend drops a recording whose number of values is not the metric's number of labels, or whose amount is negative or not finite, logging it, and records a null or blank value as unknown and values past MetricsCollector.MaxLabelValues distinct ones per label as other.
Attributes
Record an image generation operation.
Record an image generation operation.
The outcome dimension has the same meaning as in observeRequest. imageCount records the number of generated images only for successful operations.
Value parameters
- duration
-
Request duration
- imageCount
-
Number of images generated
- model
-
Model name (e.g., "gpt-image-1", "dall-e-3")
- operation
-
Operation type: "generate" or "edit"
- outcome
-
Success or Error with stable error kind
- provider
-
Provider name (e.g., "openai", "stability-ai")
Attributes
Record circuit breaker state transition.
Record circuit breaker state transition.
Expected states are stable labels such as "open", "closed", and "half-open".
Value parameters
- newState
-
New circuit breaker state ("open", "closed", "half-open")
- provider
-
Provider name
Attributes
Record a generic error for metrics (when full request tracking not applicable).
Record a generic error for metrics (when full request tracking not applicable).
Use this for failures that occur outside a complete request timing scope, such as configuration, validation, or setup failures.
Value parameters
- errorKind
-
Type of error
- provider
-
Provider name
Attributes
Record estimated image generation cost in USD.
Record estimated image generation cost in USD.
Value parameters
- costUsd
-
Estimated cost in USD
- imageCount
-
Number of images
- model
-
Model name
- provider
-
Provider name
Attributes
Record a retry attempt for reliability tracking.
Record a retry attempt for reliability tracking.
The first retry after the original request fails should use attemptNumber = 1.
Value parameters
- attemptNumber
-
Which retry attempt (1 = first retry, 2 = second, etc.)
- provider
-
Provider name