org.llm4s.metrics.MetricsCollector
See theMetricsCollector companion object
trait MetricsCollector
Minimal algebra for collecting metrics about LLM operations.
Implementations should be safe: failures must not propagate to callers. All methods should catch and log errors internally without throwing.
Example usage:
val startNanos = System.nanoTime()
client.complete(conversation) match {
case Right(completion) =>
val duration = FiniteDuration(System.nanoTime() - startNanos, NANOSECONDS)
metrics.observeRequest(provider, model, Outcome.Success, duration)
completion.usage.foreach { u =>
metrics.addTokens(provider, model, u.promptTokens, u.completionTokens)
}
case Left(error) =>
val duration = FiniteDuration(System.nanoTime() - startNanos, NANOSECONDS)
val errorKind = ErrorKind.fromLLMError(error)
metrics.observeRequest(provider, model, Outcome.Error(errorKind), duration)
}
Attributes
- Companion
- object
- Graph
-
- Supertypes
-
class Objecttrait Matchableclass Any
- Known subtypes
-
class PrometheusMetrics
Members list
In this article