org.llm4s.agent.metrics
Members list
Type members
Classlikes
The agent runtime's metrics (Stage 5 O2, design 4.59), declared as core Metrics and recorded through any MetricsCollector - PrometheusMetrics, MicrometerMetrics or a MetricsCollector.compose of them - so neither backend depends on llm4s-agent. Two things record them:
The agent runtime's metrics (Stage 5 O2, design 4.59), declared as core Metrics and recorded through any MetricsCollector - PrometheusMetrics, MicrometerMetrics or a MetricsCollector.compose of them - so neither backend depends on llm4s-agent. Two things record them:
- a MeteredCheckpointer wrapped around the store the runtimes and workers share: runs, claims, commits and child tasks, read from what each call to the store did, so every run is counted once, by the process that ran it;
- an
AgentWorkergiven the collector: the run queue and the worker's own work.
Every label takes its values from a fixed set named on the metric, except graph (a graph's id) and agent (an agent's id), which the code that builds the agents fixes; backends cap every label's distinct values. No thread, run, entry or tenant id is ever a label, and no metric carries message content (Stage 5 rule 10).
The names below are Micrometer's; a Prometheus backend exposes llm4s.agent.runs.started as llm4s_agent_runs_started_total, a timer with the suffix _seconds, a gauge under its name with dots as underscores.
Attributes
- Supertypes
-
class Objecttrait Matchableclass Any
- Self type
-
AgentMetrics.type
A Checkpointer that records the agent runtime's run, claim, commit and child-task metrics (AgentMetrics, design 4.59) into metrics as it forwards every call to inner unchanged. Give it to every runtime and worker of a process in place of the store:
A Checkpointer that records the agent runtime's run, claim, commit and child-task metrics (AgentMetrics, design 4.59) into metrics as it forwards every call to inner unchanged. Give it to every runtime and worker of a process in place of the store:
val metrics = PrometheusMetrics.create()
for
store <- CheckpointerConfigLoader.open() // llm4s.agent.checkpointer
queue <- store.runQueue
metered = MeteredCheckpointer(store.checkpointer, metrics)
agent <- Agent.builder("assistant", client).withRuntime(new GraphRuntime(metered)).build()
worker <- AgentWorker(queue, metered, Seq(QueuedAgent(agent)), metrics = metrics)
yield worker
It reads what each call did, so it needs no hook in the runtime and counts every run once, in the process whose runtime committed it:
- '''claims''' (AgentMetrics.Claims) from
claim's answer - a grant namingtakenOverFromis a takeover, which the store names only within one tenant, so a takeover of another tenant's expired claim counts asacquired- and renewals (AgentMetrics.ClaimRenewals) fromrenew's; - '''commits''' (AgentMetrics.CommitDuration) timed around
commit, by result; - '''runs''' from the durable events a commit wrote:
RunStarted,RunResumedandRunRecoveredbegin a run (AgentMetrics.RunsStarted), and its terminal event ends it (AgentMetrics.RunsEnded, AgentMetrics.RunDuration: from the first event's timestamp to the last's, both stamped by this process). A run is named by its graph's id, read from the commit's checkpoint and remembered from its start for a terminal commit that writes none (a cancel's). At most MeteredCheckpointer.InFlightLimit runs are remembered at once; a run past that, or one that began before this checkpointer did, ends without a duration; - '''child tasks''': a child thread's
RunStarted- its first admission, its checkpoint naming its origin - starts one (AgentMetrics.TasksStarted), and the parent's durableagent.task_endedends it (AgentMetrics.TasksEnded, AgentMetrics.TaskDuration).
Every other call is forwarded as it is - awaitEventsAfter included, so a store's change notification is kept. A metric that fails to record never changes a call's answer, and no metric carries a thread, run or tenant id or any content (Stage 5 rule 10).
Attributes
- Companion
- object
- Supertypes
Attributes
- Companion
- class
- Supertypes
-
class Objecttrait Matchableclass Any
- Self type
-
MeteredCheckpointer.type