org.llm4s.agent.graph

Members list

Type members

Classlikes

@Experimental
trait AppliedKeys

A Transactional target's applied-keys table, fenced by the claim token of the run acting on it (design ยง4.24). The target keeps, for each key scope (one per thread), the newest FencingToken any operation on that scope has carried, and every operation on the scope - the effect given to ReplayGuard.transactional and forget - does this in one transaction:

A Transactional target's applied-keys table, fenced by the claim token of the run acting on it (design ยง4.24). The target keeps, for each key scope (one per thread), the newest FencingToken any operation on that scope has carried, and every operation on the scope - the effect given to ReplayGuard.transactional and forget - does this in one transaction:

  • a token older than the scope's newest is refused with a Left, changing nothing: a run that lost its claim can neither apply, nor read back, nor forget anything after its successor touched the target, whatever it saw in the thread's store before it was taken over;
  • otherwise the scope's newest token becomes this one, and the operation goes ahead. The effect records (key, outcome) with the effect, or, for a key already applied, returns the recorded outcome and writes nothing.

The newest token per scope is kept for as long as the target keeps data of that thread: a small record per thread, as the thread store's tombstones are. ReplayGuard.transactional trusts an applied key only while the thread's store holds the effect's begun record, and forgets it once the outcome is recorded there.

Attributes

Source
EffectReplay.scala
Supertypes
class Object
trait Matchable
class Any
Known subtypes
object Breakpoint

Attributes

Source
Breakpoint.scala
Supertypes
class Object
trait Matchable
class Any
Self type
Breakpoint.type

Where a static breakpoint holds a task: before its node runs, or after it ran.

Where a static breakpoint holds a task: before its node runs, or after it ran.

A run's RunConfig.interruptBefore and RunConfig.interruptAfter name the nodes. A task held before its node runs has done nothing; once its interrupt is answered it runs, and that breakpoint does not hold it again. A task held after its node ran has its update committed, so the state shows what it did, but its routes, static edges and join arrivals wait: once its interrupt is answered they are made, without running the node again. Either way the held task is parked like a task that suspended - one interrupt per task, keyed by its task id - and the run pauses at the end of the superstep, so breakpoints and typed interrupts are answered together, any non-empty subset at a time.

Attributes

Source
Breakpoint.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
final case class CachePolicy

Declares that a node's result depends only on its input, so a repeated input may be answered from memory without running the node.

Declares that a node's result depends only on its input, so a repeated input may be answered from memory without running the node.

The node must not depend on ThreadState, on the clock, or on an external system: a node that needs part of the state takes that part as input. The key is the node, its input schema version and the encoded input. A hit returns the stored Command and the node does not run, so a cached node should not rely on RunContext.emit or progress. Only a Continue result is stored: a failure or a suspension is not.

The cache belongs to the CompiledGraph instance and is shared by its runs. It is not checkpointed, so another instance, another process or a restart starts empty and recomputes. Entries beyond maxEntries are evicted least-recently-used first. Two tasks with one input running at the same time both run the node.

Every value is validated like RunBudgets: apply and the with* setters throw IllegalArgumentException; CachePolicy.of returns a ValidationError.

Value parameters

maxEntries

the most results kept for the node

ttl

how long an entry is served after it was stored; None means until evicted

Attributes

Companion
object
Source
CachePolicy.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object CachePolicy

Attributes

Companion
class
Source
CachePolicy.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
final case class Checkpoint

One durable point in a thread's execution - data only. Closures, codecs and update functions are rebound from the compiled graph on restore; every value inside carries the version of the codec that wrote it (VersionedJson).

One durable point in a thread's execution - data only. Closures, codecs and update functions are rebound from the compiled graph on restore; every value inside carries the version of the codec that wrote it (VersionedJson).

parent is the checkpoint this one supersedes. A checkpointer accepts a new checkpoint only if parent is the thread's latest, so two writers cannot both advance one thread: the latest checkpoint's id is the thread's version. fencingToken is the token of the claim whose run wrote it (RunClaim); None on checkpoints written before claims existed (format 4 and earlier).

Persist with Checkpoint.toJson and read with Checkpoint.fromJson, which migrates older formatVersions and refuses newer ones.

Attributes

Companion
object
Source
Checkpoint.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object Checkpoint

Attributes

Companion
class
Source
Checkpoint.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
Checkpoint.type

Whether a thread's latest checkpoint is mid-execution or finished its run.

Whether a thread's latest checkpoint is mid-execution or finished its run.

Attributes

Source
Checkpoint.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
trait Checkpointer

Durable storage for graph threads: each thread's checkpoint history, the pending writes recorded against its latest checkpoint, its durable event log, and the claim of the run that holds it. One store owns all of them so that a commit is atomic and fenced. Several GraphRuntimes, in one process or many, may share a store: the claim is what keeps two of them from running one thread at once.

Durable storage for graph threads: each thread's checkpoint history, the pending writes recorded against its latest checkpoint, its durable event log, and the claim of the run that holds it. One store owns all of them so that a commit is atomic and fenced. Several GraphRuntimes, in one process or many, may share a store: the claim is what keeps two of them from running one thread at once.

Implementations must be safe to call from several threads and processes, and must:

  • grant at most one live claim per thread (claim): a claim is live until its expiresAt, by the store's own clock, has passed; a request while one is live is refused with GraphError.ThreadBusy naming its holder, and an expired claim is replaced - taken over;
  • issue each claim a FencingToken strictly greater than every token issued for that thread before - across releases, takeovers and deleteThread - so a token is never reused;
  • apply a Commit entirely or not at all, and only if its token is the thread's current claim (GraphError.StaleClaim otherwise, checked first) - so a writer that lost its claim, even one whose token is the highest it has seen, records nothing after another run took over - and fence deleteThread by its token in the same way, atomically with the delete; a commit's Commit.fence and a delete's fence are another thread's claim, checked in that same atomic step;
  • accept a new checkpoint only when its parent is the thread's latest checkpoint id - the thread version the writer expects (GraphError.CheckpointConflict otherwise) - and its id is not already in the thread's history, and only pending writes naming the resulting latest checkpoint (GraphError.InvalidCommit);
  • keep every checkpoint a commit superseded in the thread's history, newest first in history and readable by id with checkpoint, until prune, a commit's retractTurn or deleteThread removes it - but keep pending writes for the latest checkpoint only;
  • number a thread's events consecutively in commit order, inside the commit, never reusing a number - including after compaction, pruning, a failed commit or deleteThread: a thread's first event is 1 unless a thread of the same id that numbered events was deleted, and then it is one past the deleted thread's next number n - n itself is skipped - so a new thread of the id numbers on past every event of the old one; once sweep has reclaimed a tombstone, a thread whose id has none starts at the store's reclamation mark instead of 1 - at or past the first number of every tombstone it reclaimed - so a reclaimed id's new thread still numbers on past the old one (design 4.53);
  • record on each event the token of the commit that stored it (EventRecord.token, EventRecord.committed), so a run's events can be told from those of an earlier run that had the same RunId;
  • refuse a read of the log after a number of a deleted thread of the id - from 1 up to the skipped n, exclusive (Checkpointer.deleted) - with GraphError.ThreadDeleted, from eventsAfter and awaitEventsAfter alike, also once sweep has reclaimed the deleted thread's tombstone, so a reader that was following the deleted thread never takes the new one for its continuation; a read from 0 reads the current thread from its first event, and a watch waiting in awaitEventsAfter from inside a log that is deleted gets that refusal within about its timeout (from its next call, for the polling default);
  • keep events until compactEvents or prune removes them, and report the earliest sequence still available when asked to replay from before it (GraphError.ReplayUnavailable);
  • keep a thread's effect records (EffectRecord), upserted by key by a commit's effects and remembered with the task they belong to (taskId, lineage), until a commit with a checkpoint releases them (Commit.releasesEffect, judged before that commit's own records are applied) or deleteThread removes them - never prune, never by age - and copy none of them anywhere; effect reads one;
  • store checkpoints, pending writes and effect records as data (Checkpoint.toJson, PendingWrite's ReadWriter, EffectRecord.toJson), so nothing executable survives a round trip;
  • keep what it stores apart from the values its callers hold: ujson values are mutable, so changing a checkpoint, pending write or event payload after committing it, or one a read or a commit returned, must change nothing stored or later read back;
  • return from awaitEventsAfter with what eventsAfter would, at once if the log already holds an event after afterSeq, and otherwise within about timeout - with the events of a commit made meanwhile through any store over the same storage, or empty;
  • tolerate the thread calling eventsAfter or awaitEventsAfter being interrupted at any point: cancelling a subscription interrupts its watch mid-read, so an interrupt must end that one call - with Left(CancelledError) or a failed result - and leave the store usable by every other thread and later call (see awaitEventsAfter);
  • list a thread's direct children (children) - the threads whose latest checkpoint's RunOrigin names it as parent - in the order they were first committed, paged, and never one deleteThread removed;
  • keep each thread's ThreadMetadata, replaced only by a commit carrying Commit.metadata and removed only by deleteThread, and list a tenant's root threads (threads) and its incomplete ones (incompleteThreads) - the thread registry (design 4.52) - by their latest checkpoint, their claim judged by the store's clock, and when the store last committed a checkpoint or metadata to them, by that clock, never one of another tenant, a child thread, one with no checkpoint, or one deleteThread removed;
  • answer latestOfRuns - the newest checkpoint any of a set of runs committed on a thread, the run-id lookup a run queue's entry classifies its attempts by (Stage 5 rule 4, design 4.55) - by an index on the run, and keep the checkpoint it answers with through prune while an open entry of the store's run queue names that run;
  • visit every thread it holds, tombstones included, through sweep, a bounded page at a time, pruning each as prune does and reclaiming old tombstones, but never what the sweep's SweepGuard protects;
  • return from every call within a bound it documents (#1919) - see below.

'''Every call returns within a bound.''' A store times out its own calls: it documents a bound, set by its configuration (such as SqliteCheckpointerConfig.busyTimeout), within which every call returns, whatever its storage is doing - waiting for a lock, for another connection's or process's write, for the store's own earlier calls, or for a network - and a call that reaches the bound fails, changing nothing. awaitEventsAfter returns within its timeout plus that bound. A store that cannot bound some call - a disk that stalls inside a write, say, which no driver can time out - documents which call, and why. Keep the bound well inside ClaimPolicy.ttl - ClaimPolicy.renewEvery: a renewal must land within that, and a store that serialises its calls queues a renewal behind a slow commit.

A GraphRuntime bounds what it can without the store: a run's result, its slot in the runtime and its claim are freed within twice ClaimPolicy.ttl of a lost claim being found (GraphError.ClaimLost), and of a closing commit - an Async run's drain, or an OnExit run's exit commit - sticking while the claim holds (GraphError.CommitTimedOut). Every other call made while a claim holds - an admission's claim commit, a Sync run's commits, its outcome's included, a change's write, and the release of a claim that was not lost - is waited for as long as it takes, and only the store's bound limits it. Nor can the runtime free what a call stuck in the store holds itself, so these last as long as the call does:

  • '''the calling thread.''' A run thread, or the virtual thread an admission, fork, updateState or deleteThread makes its store calls on, stays alive inside the call, and an Async run's writer thread with it, after the runtime has given the call up;
  • '''the thread's later commits.''' Commits on one thread are written one at a time in a runtime, so a commit - or a subscription's read of events another runtime committed, which a commit's hand-over makes under the same lock - holds up every later commit on that thread in that runtime, a new run's claim commit included, until it returns; never another thread's;
  • '''subscriptions behind a commit.''' A GraphRuntime.subscribe made while a run's claim commit is in the store, on a thread with nothing of the run handed over yet - a claim commit its admission gave up on after a lost claim included - waits for that commit to return, or for the subscription to be cancelled, before it delivers its replay; a run-scoped subscription never does. And with WatchPolicy.disabled, a replay made while a write a run gave up on is still in the store stops at the first event another runtime committed above what this runtime has handed over; it receives the rest when this runtime next commits on the thread, and a replay made once the write has returned has them all.

The holder of a claim may commit and renew it after expiresAt for as long as no other claim has replaced it: expiry only lets another run take the thread over. CheckpointerContract in llm4s-agent-testkit is the executable form of this contract; a store should pass it.

Attributes

Companion
object
Source
Checkpointer.scala
Supertypes
class Object
trait Matchable
class Any
Known subtypes
object Checkpointer

Attributes

Companion
trait
Source
Checkpointer.scala
Supertypes
class Object
trait Matchable
class Any
Self type
@Experimental
final case class ChildThread(threadId: ThreadId, origin: RunOrigin, status: CheckpointStatus, latest: String, updatedAt: Instant)

One direct child of a thread, as Checkpointer.children lists it: its id, its RunOrigin, and its latest checkpoint's status, id and creation time.

One direct child of a thread, as Checkpointer.children lists it: its id, its RunOrigin, and its latest checkpoint's status, id and creation time.

Attributes

Companion
object
Source
ChildThreads.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object ChildThread

Attributes

Companion
class
Source
ChildThreads.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
final case class ClaimPolicy

How a GraphRuntime holds its runs' claims: each claim lasts ttl from when the store grants or renews it, and is renewed every renewEvery from its grant - also while the run is still being admitted, and while fork or updateState holds a thread - until it is released. A process that dies stops renewing, so after at most ttl its threads can be recovered elsewhere.

How a GraphRuntime holds its runs' claims: each claim lasts ttl from when the store grants or renews it, and is renewed every renewEvery from its grant - also while the run is still being admitted, and while fork or updateState holds a thread - until it is released. A process that dies stops renewing, so after at most ttl its threads can be recovered elsewhere.

A run judges its own lease: it ends ttl after the moment, by the runtime's monotonic clock, just before the request that granted or last renewed the claim was sent. A run whose renewals fail or hang until that lease passes - its store is unreachable, or partitioned from it - stops at once, as one whose renewal or commit the store refuses as stale does: its tasks are interrupted, it commits nothing more, and it fails with GraphError.ClaimLost. A renewal that fails is tried again at the next beat while the lease lasts. Tasks already running when the loss is found may still complete a side effect before they see their interrupt.

A renewal must land within ttl - renewEvery of being due, or the run judges its lease passed and stops: keep renewEvery at most a third of ttl, as the default does, so a renewal can fail or be slow once and still be retried in time.

label, when set, names the runtime to an operator: it is logged with the run id when a claim is granted (at INFO; without a label, at DEBUG) and when a renewal fails or the claim is lost, so that the holder a GraphError.ThreadBusy names can be traced to the runtime holding it - a host, a pod, a process. It goes nowhere else: not to the store, and not into an error another caller gets.

Both durations must be positive and renewEvery shorter than ttl, and a label must not be blank: apply and the with* setters throw IllegalArgumentException otherwise, and ClaimPolicy.of returns a ValidationError.

Attributes

Companion
object
Source
RunClaim.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object ClaimPolicy

Attributes

Companion
class
Source
RunClaim.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
final case class ClaimRequest

What a run asks of Checkpointer.claim: who holds the claim, the tenant it holds it for (the run's RunConfig.tenantId; None and Some differ, as for a thread), and for how long it lasts from the moment the store grants it, by the store's clock. The store records the tenant with the claim, and names the claim's holder to a later request - in a refusal, or as RunClaim.takenOverFrom - only when that request is of the same tenant. apply and the with* setters throw IllegalArgumentException for a non-positive ttl; ClaimRequest.of returns it as a ValidationError.

What a run asks of Checkpointer.claim: who holds the claim, the tenant it holds it for (the run's RunConfig.tenantId; None and Some differ, as for a thread), and for how long it lasts from the moment the store grants it, by the store's clock. The store records the tenant with the claim, and names the claim's holder to a later request - in a refusal, or as RunClaim.takenOverFrom - only when that request is of the same tenant. apply and the with* setters throw IllegalArgumentException for a non-positive ttl; ClaimRequest.of returns it as a ValidationError.

Attributes

Companion
object
Source
RunClaim.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object ClaimRequest

Attributes

Companion
class
Source
RunClaim.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
final case class Command(update: StateUpdate, routes: List[Route])

A node's state update and routes. Static edges declared on the builder are additive: they are scheduled before the command's routes. A command with no routes and no outgoing edges ends that branch; the run completes when no branch has work left.

A node's state update and routes. Static edges declared on the builder are additive: they are scheduled before the command's routes. A command with no routes and no outgoing edges ends that branch; the run completes when no branch has work left.

Attributes

Companion
object
Source
GraphNode.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object Command

Attributes

Companion
class
Source
GraphNode.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
Command.type
final case class Commit

One atomic write to a thread, fenced by the writing run's claim. The checkpointer applies all of it or none of it:

One atomic write to a thread, fenced by the writing run's claim. The checkpointer applies all of it or none of it:

  • token must be the FencingToken of the thread's current claim (GraphError.StaleClaim otherwise, checked first): a writer that lost its claim records nothing.
  • checkpoint, if present, becomes the thread's latest, provided its parent is the current latest - the thread version the writer expects (GraphError.CheckpointConflict) - and its id is not already in the thread's history (GraphError.InvalidCommit); the previous latest stays in the history, and the pending writes recorded against it are dropped.
  • pendingWrites are recorded against the (resulting) latest checkpoint, and must name it - the version they expect (GraphError.InvalidCommit).
  • events are appended to the thread's durable log, each given the next sequence number.
  • retractTurn, which needs a checkpoint (GraphError.InvalidCommit otherwise), first removes from the history every checkpoint after the thread's latest settled one - the newest whose status is Completed or Failed, or all of them if there is none - so that nothing of a turn a guardrail blocked stays readable. The runtime sets it on the Failed checkpoint that closes a run a node blocked (NodeResult.Block).
  • effects are upserted by key into the thread's effect records (Checkpointer.effect), each remembered with its task's id and lineage. Before they are, a commit with a checkpoint releases every record for which releasesEffect holds, so replay evidence lives exactly as long as its task can run again (design ยง4.24). The runtime writes each record of a ReplayGuard as a commit of its own, with no checkpoint, pending write or event.
  • fence, if present as (other, otherToken), fences the commit by another thread's claim too: it applies only if otherToken is other's current claim, checked in the same atomic step as token and the write (one transaction, or under one lock), and otherwise is GraphError.StaleClaim(other, otherToken, otherCurrent) with nothing written - token checked first. The runtime fences every admission and settlement a parent drives on a child thread - its first commit, which writes its Checkpoint.origin, a reattachment's, a resume's and a settlement's - by the parent's claim (design 4.41), so a stale task of a parent taken over never registers or runs a child after a successor took the parent.
  • metadata, if present, replaces the thread's ThreadMetadata (design 4.52), fenced like the rest of the commit; None leaves it as it is. A thread keeps its metadata across commits until another replaces it, and loses it only with the thread (Checkpointer.deleteThread).

Attributes

Companion
object
Source
Checkpoint.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object Commit

Attributes

Companion
class
Source
Checkpoint.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
Commit.type
final class CompiledGraph[I, O]

An immutable, validated graph, run in supersteps.

An immutable, validated graph, run in supersteps.

Each superstep runs every ready task against the same committed ThreadState, then commits their updates and schedules the next frontier together:

  1. Tasks run in frontier order (possibly concurrently); none sees another's writes.
  2. If any task fails, the superstep commits nothing and the run fails with the first failure in frontier order.
  3. Each command is checked: every update is to a key in the node's declared write set, every route targets a node and join of this graph, and a task fans out to a join at most once.
  4. Updates apply task by task in frontier order, and in emission order within a task.
  5. The next frontier is, for each task in order, its static edges (declaration order) then its routes (route order, fan-out payloads in item order); then the targets of joins this commit released - static joins in declaration order, then dynamic activations in the order they opened.

A task that returns NodeResult.Suspend parks a continuation; its update commits with the superstep, and the run pauses after that superstep (RunResult.Suspended). A continuation stands in for the suspended task: when it completes it makes that task's join arrivals. The suspended task itself makes no arrival and fires none of its node's static edges.

A run's static breakpoints (RunConfig.interruptBefore, RunConfig.interruptAfter) park a task the same way: before its node runs, it is parked instead of run; after it ran, its update commits and its routes, static edges and join arrivals wait in its continuation. Either way the run pauses after the superstep, and the task's interrupt is answered with Breakpoint.proceed.

When the frontier is empty the run completes - unless a join is still waiting. A join waiting only for arrivals that parked continuations can make keeps the run suspended; a join waiting for an arrival nothing can make fails the run with GraphError.UnsatisfiedJoin.

Attributes

Source
CompiledGraph.scala
Supertypes
class Object
trait Matchable
class Any

Why a subscription ended with StreamEvent.Disconnected.

Why a subscription ended with StreamEvent.Disconnected.

Attributes

Source
RunEvent.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
enum Durability

When a run's checkpoints and events become durable. In every mode a durable event is delivered only after the commit that numbered it, so no subscriber sees a sequence number that a crash could later reuse. In every mode a run claims its thread with a synchronous commit before it runs anything, and makes everything durable before it returns - so a returned RunResult.Suspended has been persisted and can be resumed by another process.

When a run's checkpoints and events become durable. In every mode a durable event is delivered only after the commit that numbered it, so no subscriber sees a sequence number that a crash could later reuse. In every mode a run claims its thread with a synchronous commit before it runs anything, and makes everything durable before it returns - so a returned RunResult.Suspended has been persisted and can be resumed by another process.

Attributes

Source
GraphRuntime.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
final class DynamicJoin

A barrier attached to a fan-out. A Route.FanOut opens an activation recording the task id of every child it schedules; target is scheduled once all of them have completed. An empty fan-out releases immediately.

A barrier attached to a fan-out. A Route.FanOut opens an activation recording the task id of every child it schedules; target is scheduled once all of them have completed. An empty fan-out releases immediately.

Attributes

Source
GraphNode.scala
Supertypes
class Object
trait Matchable
class Any
@Experimental
final case class EffectKey

One external effect, the identity its replay evidence is kept under (design ยง4.24): the SHA-256 of the thread that performs it, a base key - the tool call's IdempotencyKey or the task's RunContext.taskKey - the operation (the kind of effect, such as file.edit) and a discriminator naming what it touches (a path, sorted source ids). Effects of one task differ in operation or discriminator, so they never suppress one another; a fork's re-run of its source's call is a new effect, because the thread is part of the key.

One external effect, the identity its replay evidence is kept under (design ยง4.24): the SHA-256 of the thread that performs it, a base key - the tool call's IdempotencyKey or the task's RunContext.taskKey - the operation (the kind of effect, such as file.edit) and a discriminator naming what it touches (a path, sorted source ids). Effects of one task differ in operation or discriminator, so they never suppress one another; a fork's re-run of its source's call is a new effect, because the thread is part of the key.

Built only by RunContext.effectKey and ToolContext.effectKey. value is the identity; operation stays readable so that refusals can name it. scope names the thread that performs the effect, without revealing it (the SHA-256 of its id): a Transactional target fences its keys per scope, because a thread's claim tokens only grow (AppliedKeys).

Attributes

Companion
object
Source
EffectReplay.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object EffectKey

Attributes

Companion
class
Source
EffectReplay.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
EffectKey.type
@Experimental
final class EffectRecord

The replay evidence of one effect, as the thread's Checkpointer keeps it: begun (an intent) or completed (an outcome). taskId and lineage name the task the evidence belongs to - for a continuation, the task it stands in for - by its id and a digest of what scheduled it; they decide when a commit releases it (Commit.releasesEffect). A store persists EffectRecord.toJson and reads it back with EffectRecord.fromJson; the body is the guard's.

The replay evidence of one effect, as the thread's Checkpointer keeps it: begun (an intent) or completed (an outcome). taskId and lineage name the task the evidence belongs to - for a continuation, the task it stands in for - by its id and a digest of what scheduled it; they decide when a commit releases it (Commit.releasesEffect). A store persists EffectRecord.toJson and reads it back with EffectRecord.fromJson; the body is the guard's.

Attributes

Companion
object
Source
EffectReplay.scala
Supertypes
class Object
trait Matchable
class Any
object EffectRecord

Attributes

Companion
class
Source
EffectReplay.scala
Supertypes
class Object
trait Matchable
class Any
Self type

A state operation as data: updates are encoded with the key's update codec.

A state operation as data: updates are encoded with the key's update codec.

Attributes

Source
Checkpoint.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all

A route as data: payloads are encoded with the target node's input codec.

A route as data: payloads are encoded with the target node's input codec.

Attributes

Source
Checkpoint.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
final case class EncodedSuspension(resumeNode: String, question: VersionedJson)

A suspended task's parked continuation: where it resumes and the question, as data.

A suspended task's parked continuation: where it resumes and the question, as data.

Attributes

Source
Checkpoint.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
final case class EventDraft(runId: String, checkpointId: Option[String], taskId: Option[String], nodeId: Option[String], timestamp: Instant, event: RunEvent)

A durable event before its commit assigns it a sequence number.

A durable event before its commit assigns it a sequence number.

Attributes

Source
RunEvent.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
final class EventHandlers[+R] extends StreamEvent => Unit

Typed handlers for a run's events, each registered with its EventType, so that a selection of mixed event types keeps each payload's type:

Typed handlers for a run's events, each registered with its EventType, so that a selection of mixed event types keeps each payload's type:

val handlers = EventHandlers
 .on(AgentEvents.TextDelta)(d => print(d.text))
 .on(Checked)(c => println(s"checked ${c.path}"))
 .otherwise {
   case StreamEvent.LiveGap(n) => println(s"[$n live events lost]")
   case _                      => ()
 }
agent.stream(threadId, query)(handlers)

An event is looked up once by its name and version - a durable RunEvent.Custom or a StreamEvent.Live alike - and its payload decoded once, by the one type registered for them. R is what a handler returns: Unit for a listener, or a caller's own type, to turn a run's events into a stream of one ADT (on(Checked)(MyEvent.Checked(_)), the effect modules' events).

An EventHandlers is a listener (StreamEvent => Unit): pass it to Agent.stream, AgentRun.subscribe or GraphRuntime.subscribe. As a listener it runs dispatch and discards the result; a handler that throws is a listener that throws.

A delegated child's events, which a subscription made with nested = true receives as StreamEvent.Nested (design 4.47), are matched with onChild: on never takes one, nor onChild a parent's own event.

Attributes

Companion
object
Source
EventHandlers.scala
Supertypes
trait StreamEvent => Unit
class Object
trait Matchable
class Any
object EventHandlers

Attributes

Companion
class
Source
EventHandlers.scala
Supertypes
class Object
trait Matchable
class Any
Self type
final case class EventRecord(threadId: String, seq: Long, runId: String, token: FencingToken, checkpointId: Option[String], taskId: Option[String], nodeId: Option[String], timestamp: Instant, event: RunEvent)

A committed durable event. seq is per thread, allocated in the commit that stored the event, contiguous and never reused; an event is delivered to subscribers only after that commit.

A committed durable event. seq is per thread, allocated in the commit that stored the event, contiguous and never reused; an event is delivered to subscribers only after that commit.

token is the FencingToken of the claim the commit was fenced by (Commit.token): one incarnation of a run. A RunId may be reused - a recovery under the same id, or any run given one with RunConfig.withRunId - so two runs on a thread can share an id, but never a token: the store issues each claim a token above every earlier one. A record written before tokens were recorded reads as FencingToken(0), which no store issues.

Attributes

Companion
object
Source
RunEvent.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object EventRecord

Attributes

Companion
class
Source
RunEvent.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
final class EventType[A]

A typed event: a payload type A with a stable name and version, sent durably with emit (a RunEvent.Custom, committed with the task) or live with progress (a StreamEvent.Live), and matched on either with unapply:

A typed event: a payload type A with a stable name and version, sent durably with emit (a RunEvent.Custom, committed with the task) or live with progress (a StreamEvent.Live), and matched on either with unapply:

val Ping = EventType[Ping]("my.ping", 1)
runtime.subscribe(threadId) {
 case Ping(p) => println(p)
 case _       => ()
}

unapply is None for another name or version, for a kernel event, and for a payload that does not decode (logged at DEBUG), so a listener never throws on an event it does not know.

Attributes

Companion
object
Source
EventType.scala
Supertypes
class Object
trait Matchable
class Any
object EventType

Attributes

Companion
class
Source
EventType.scala
Supertypes
class Object
trait Matchable
class Any
Self type
EventType.type
final class Execution

A graph run paused at a superstep boundary: the committed state, the ready frontier, the open join activations and any parked continuations. Advance it with CompiledGraph.step; answer its interrupts with CompiledGraph.resume; persist it with CompiledGraph.snapshot.

A graph run paused at a superstep boundary: the committed state, the ready frontier, the open join activations and any parked continuations. Advance it with CompiledGraph.step; answer its interrupts with CompiledGraph.resume; persist it with CompiledGraph.snapshot.

Attributes

Source
Execution.scala
Supertypes
class Object
trait Matchable
class Any
object FencingToken

Attributes

Source
GraphIds.scala
Supertypes
class Object
trait Matchable
class Any
Self type
final class GraphBuilder

Builds a typed graph. The builder issues every handle - nodes, joins - and validates the whole graph in compile, reporting every problem at once.

Builds a typed graph. The builder issues every handle - nodes, joins - and validates the whole graph in compile, reporting every problem at once.

Nodes that route to each other in a cycle are declared first and implemented afterwards:

val b      = GraphBuilder("counter", "v1")
val count  = StateKey.replace[Int]("count", 0)
val tick   = b.declare[Unit]("tick")
b.implement(tick, writes = Set(count)) { (_, state, _) =>
 NodeResult.fromResult(state.get(count).map { n =>
   val next = Command.empty.update(count, n + 1)
   if n + 1 < 3 then next.goto(tick) else next
 })
}
val graph = b.compile(tick)(_.get(count))

The builder is mutable and meant to be used from one thread while the graph is assembled; the CompiledGraph it produces is immutable.

Attributes

Companion
object
Source
GraphBuilder.scala
Supertypes
class Object
trait Matchable
class Any
object GraphBuilder

Attributes

Companion
class
Source
GraphBuilder.scala
Supertypes
class Object
trait Matchable
class Any
Self type
sealed trait GraphError extends LLMError

Errors raised while building, running or restoring a typed graph. Every case is a NonRecoverableError except GraphError.DeadlineExceeded - recover with a new budget continues that run - and GraphError.ThreadBusy, which a retry may get past once the thread is free.

Errors raised while building, running or restoring a typed graph. Every case is a NonRecoverableError except GraphError.DeadlineExceeded - recover with a new budget continues that run - and GraphError.ThreadBusy, which a retry may get past once the thread is free.

Attributes

Companion
object
Source
GraphError.scala
Supertypes
trait LLMError
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
Known subtypes
object GraphError

Attributes

Companion
trait
Source
GraphError.scala
Supertypes
trait Sum
trait Mirror
class Object
trait Matchable
class Any
Self type
GraphError.type
trait GraphNode[I]

A node's behaviour. It reads the superstep's committed snapshot - never another task's uncommitted writes - and returns its effects as data.

A node's behaviour. It reads the superstep's committed snapshot - never another task's uncommitted writes - and returns its effects as data.

Attributes

Source
GraphNode.scala
Supertypes
class Object
trait Matchable
class Any
object GraphRuntime

Runs compiled graphs on durable threads; see GraphRuntime.

Runs compiled graphs on durable threads; see GraphRuntime.

Attributes

Companion
class
Source
GraphRuntime.scala
Supertypes
class Object
trait Matchable
class Any
Self type
final class GraphRuntime(checkpointer: Checkpointer, clock: Clock, claims: ClaimPolicy, watch: WatchPolicy)

Runs compiled graphs on durable threads.

Runs compiled graphs on durable threads.

A thread's latest checkpoint says what may happen next:

  • none, Completed or Failed: start runs the graph with a new input (on a finished thread, over its committed state). Failed is a run that ended in a failure that is its outcome, not an interruption: a node returned NodeResult.Block, as a guardrail does. The thread is usable and recover has nothing to continue;
  • Running - work was scheduled when the last run stopped: recover continues it with no new input, reusing every pending write so completed tasks are not run again, and running failed or unstarted tasks again, each with its node's full RetryPolicy;
  • Suspended: resume answers any non-empty subset of the parked interrupts; unanswered ones stay parked, and the run suspends again if nothing else can proceed.

Each call's RunConfig brings the run's static breakpoints; one on a node the graph does not have is refused with a ValidationError before the thread is read. A task they hold parks like a task that suspended, so the thread ends Suspended and resume answers it with Breakpoint.proceed.

Any other call is refused (GraphError.IncompleteRun, GraphError.PendingInterrupts, GraphError.NothingToRecover, GraphError.NotSuspended) without changing the thread. A thread belongs to the tenant on its latest checkpoint, and a call with another tenantId is refused with GraphError.TenantMismatch before anything else is reported - the thread's status, or that a run is live on it - and the error names only the caller's tenant, so it learns nothing about the thread.

A thread whose latest checkpoint records a RunOrigin is a child thread, started by a tool call of its parent's run (design 4.41). It is driven only through that run: start, recover, resume, fork, updateState and deleteThread on it are refused with GraphError.ChildThread, after the tenant check and before any claim, while history, checkpoint, subscribe and children read it as any thread. Deleting a thread deletes its children first, and child retention (RunConfig.withChildRetention) bounds how many settled children a thread keeps. Each run's terminal (Completed or Failed) checkpoint records how it settled (Checkpoint.outcome).

Each call is a new run, and claims the thread in two steps before it runs anything. First it asks the store for a RunClaim: a lease with an expiry and a FencingToken, which the run renews while it executes (see ClaimPolicy) and releases when it ends. While another run's claim is live - in this runtime or in any other runtime or process sharing the store - the call fails with GraphError.ThreadBusy naming that run, so recover elsewhere refuses a live run and takes over a dead one only once its claim has expired. Then it commits a checkpoint whose parent is the latest it read; if another run advanced the thread in between, it fails with ThreadBusy too. Either way its input or answers are neither accepted nor discarded. Every commit of the run carries its claim's token, and the store refuses one whose claim was taken over (GraphError.StaleClaim), so a run that lost its claim - a process paused past ttl - cannot record anything after its successor started. The run also judges its own lease, by its monotonic clock, so it learns of the loss even when no renewal reaches the store: once the lease passes with no renewal landing, or a renewal or commit is refused as stale, it stops at once - its tasks interrupted, as a cancel interrupts them - commits nothing more, and fails with GraphError.ClaimLost. A loss found once the run has begun committing its outcome sends no interrupt, but still skips every commit not yet begun; and whatever the run is doing, it ends within twice ttl of the loss: if it has not begun to by then - its thread stuck in a store call that ignores interrupts - it is ended without its thread, ClaimLost, and the thread freed. Admission is bounded the same way: a claim's commit still in the store twice ttl after the loss is given up, and the call fails ClaimLost - and if that commit then lands before the claim's release does, nobody having taken the thread over, it leaves its Running checkpoint and its RunStarted, RunRecovered or RunResumed event although no run continues it: recover does, once the release lands; one the store refuses as stale - the claim lapsed, or another run took it over, before it landed - fails ClaimLost too, never GraphError.CheckpointWriteFailed. So are fork, updateState and deleteThread, which hold a claim while they change the thread: a write not begun at the loss is never sent, and the call returns GraphError.ThreadBusy within twice ttl of it. A recover that takes over an expired claim records the run it took the thread from (RunEvent.RunRecovered). A call on a thread whose run is still executing in this runtime fails with ThreadBusy before reading the thread at all.

Admission - the checks above, restoring the thread and committing the claim - runs on the caller's thread, but for the read under the claim and the claim's commit, which run on llm4s-admit-<threadId> while the caller waits, and never throws: a Left means no run exists, and the thread is free again. A throwable from the store, the clock or the graph during admission is GraphError.RunCrashed, and an interrupt is CancelledError with the flag set. The run itself executes on a virtual thread owned by the runtime, named llm4s-run-<threadId>, behind the returned RunHandle; the thread stays busy until that run thread exits. An unexpected throwable escaping the run ends it with GraphError.RunCrashed.

RunHandle.cancel cancels the run by interrupting its run thread: every task is interrupted and joined, tasks that finished first keep their pending writes, an interrupted task records nothing, and the run ends Failed with GraphError.Cancelled - naming the last checkpoint, which a RunEvent.RunCancelled event is committed against. The checkpoint stays Running, so recover continues the run. A cancel that interrupts a superstep's commit ends the run the same way even if the store reports that commit as failed (as a JDBC store may, rather than throwing InterruptedException): the run is cancelled, not failed by its store, and the error names the last checkpoint that was durable. Once the run has begun committing its outcome - its completed or suspended checkpoint, or the RunFailed of a failed run - a cancel is ignored - it sends no interrupt, so that commit is not disturbed - and the run ends with its outcome. Interrupting a thread blocked in RunHandle.await does not cancel the run.

A run's closing wait is bounded while it holds its claim too (#1919). An Async run waiting for its queued commits, or an OnExit run making its exit commit, whose commit stays in the store for twice claims.ttl with none returning, ends GraphError.CommitTimedOut without waiting for it, through the same settling as a lost run: its result is set, its thread freed in this runtime, and its claim released on a detached thread. The commit is an orphan, fenced by the claim as a lost run's late write is: it lands only if the store applies it before the release, and the run's own listeners never receive it. A drain that keeps making progress is never cut short, however long it takes in all. Every other store call is waited for as long as it takes: every Checkpointer must return from each call within a bound it documents.

A run whose RunBudgets.timeout is set has a deadline, fixed when its claim commits; its tasks read it, by the runtime's clock, as RunContext.deadline. When it passes, the run stops the same way, but ends with GraphError.DeadlineExceeded and commits RunEvent.RunTimedOut. The first of cancel and expiry to be recorded wins, so a run reports exactly one. DeadlineExceeded is recoverable: recover with a new budget continues the run.

subscribe replays a thread's committed events after a sequence number and then delivers new ones as their commits succeed - only to a caller of the thread's tenant, as every other read of a thread is (see subscribe) - in ascending order with no gaps or duplicates, followed by live progress as it happens. Each subscription has its own dispatcher thread and a queue of capacity events, so a slow listener never holds up a run: one that falls behind by more than capacity durable events is disconnected (DisconnectReason.Lagging), and live events that do not fit are dropped and counted (StreamEvent.LiveGap). See EventHub. Live events are never replayed, but a subscription receives every one sent after subscribe returns, also while it is still replaying: so subscribing and then calling start loses none of the run's. One made after start returns can miss the run's first live events; an Observer passed to start, recover or resume is subscribed during admission, before the claim commits, and so sees every event of the run (RunHandle.observation).

A subscription also sees the durable events of commits made through another runtime or process sharing the checkpointer, with the same durable guarantees (ascending seq, no gap or duplicate): each live subscription watches the store (WatchPolicy, Checkpointer.awaitEventsAfter), so they arrive as soon as the store reports them - at once from a store that is told of commits, within pollInterval from one that is polled. Live progress events are never stored, so another runtime's do not arrive.

'''Ordering across runtimes.''' Concurrently active subscriptions to a thread that stay caught up see one interleaving of durable and live events within one runtime: every event handed to several of them has the same relative placement for each. A subscription that attaches later misses earlier live events, and a slower one can lose live events (StreamEvent.LiveGap) or be disconnected as Lagging; the guarantee does not cover those cases. A commit made elsewhere reaches each subscription through that subscription's own store watch, through a hand-over's gap read, or, while the subscription is still in its initial replay (one created after the commit included), directly from that replay, so its position among this runtime's live events can differ from one subscription to another; each still sees the durable events in seq order. The guarantee holds when nothing outside this runtime commits to the thread; an external commit allows subscriptions to place it differently, not necessarily make them differ. A consumer that needs one order across runtimes orders only StreamEvent.Durable records by EventRecord.seq. It ignores StreamEvent.Live and StreamEvent.LiveGap, which have no seq, and treats StreamEvent.Disconnected as the end of the subscription, not as a record: it carries lastSeq and a reason.

Attributes

Companion
object
Source
GraphRuntime.scala
Supertypes
class Object
trait Matchable
class Any
final case class GraphShape(entry: String, nodes: Vector[String], edges: Map[String, Vector[String]], staticJoins: Vector[StaticJoin], dynamicJoins: Map[String, String], resumes: Vector[String], keys: Vector[String])

A compiled graph's declared structure, as data: what a GraphSnapshot records of the graph that wrote it, so a restore can tell what changed since. nodes, resumes, keys and each static join's sources are sorted; edges holds each node's static successors in declaration order, and only nodes that have some; staticJoins is in declaration order, which is the order joins released in one superstep schedule their targets. CompiledGraph.restore says which differences keep a snapshot restorable.

A compiled graph's declared structure, as data: what a GraphSnapshot records of the graph that wrote it, so a restore can tell what changed since. nodes, resumes, keys and each static join's sources are sorted; edges holds each node's static successors in declaration order, and only nodes that have some; staticJoins is in declaration order, which is the order joins released in one superstep schedule their targets. CompiledGraph.restore says which differences keep a snapshot restorable.

Attributes

Companion
object
Source
GraphShape.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object GraphShape

Attributes

Companion
class
Source
GraphShape.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
GraphShape.type
final case class GraphSnapshot(graphId: String, graphVersion: String, fingerprint: String, superstep: Int, state: Map[String, VersionedJson], frontier: Vector[PendingTask], staticJoins: Vector[StaticArrivals], dynamicJoins: Vector[Activation], parked: Vector[ParkedContinuation], paused: Boolean, shape: Option[GraphShape])

A serializable picture of an Execution - data only, no closures or codecs. Node inputs, questions and state values are encoded with the codecs of the graph that wrote them, and decoded and checked against the graph that restores them.

A serializable picture of an Execution - data only, no closures or codecs. Node inputs, questions and state values are encoded with the codecs of the graph that wrote them, and decoded and checked against the graph that restores them.

Each value records its codec's version and is migrated on restore; the snapshot itself is versioned by the Checkpoint that carries it. shape is the writing graph's declared structure, which CompiledGraph.restore compares with its own; None on a snapshot written before shapes were recorded (#1934), which is matched by fingerprint alone.

Attributes

Companion
object
Source
Execution.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object GraphSnapshot

Attributes

Companion
class
Source
Execution.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type

Attributes

Source
GraphIds.scala
Supertypes
class Object
trait Matchable
class Any
Self type
final class InMemoryCheckpointer(clock: Clock) extends Checkpointer

A Checkpointer in memory. Checkpoints and pending writes are stored as JSON and decoded on read, exactly as a database-backed store would, so nothing executable survives a round trip. Claims expire, and prune judges age, by clock, which also stamps each checkpoint and event with the time it was committed. Fencing tokens come from one counter for the whole store, so they never repeat for a thread, even after it is deleted; a deleted thread that numbered any event leaves a tombstone record of its next event number. awaitEventsAfter wakes on the store's own commits and deletes, so every runtime sharing the instance sees a commit as soon as it lands, without polling. Its calls wait for nothing but one another's brief turn at its lock, which no call holds across I/O or a wait, so each returns at once - awaitEventsAfter within its timeout - and it needs no timeout of its own to meet the contract's bound.

A Checkpointer in memory. Checkpoints and pending writes are stored as JSON and decoded on read, exactly as a database-backed store would, so nothing executable survives a round trip. Claims expire, and prune judges age, by clock, which also stamps each checkpoint and event with the time it was committed. Fencing tokens come from one counter for the whole store, so they never repeat for a thread, even after it is deleted; a deleted thread that numbered any event leaves a tombstone record of its next event number. awaitEventsAfter wakes on the store's own commits and deletes, so every runtime sharing the instance sees a commit as soon as it lands, without polling. Its calls wait for nothing but one another's brief turn at its lock, which no call holds across I/O or a wait, so each returns at once - awaitEventsAfter within its timeout - and it needs no timeout of its own to meet the contract's bound.

Attributes

Source
Checkpointer.scala
Supertypes
trait Checkpointer
class Object
trait Matchable
class Any
object InterruptId

Attributes

Source
GraphIds.scala
Supertypes
class Object
trait Matchable
class Any
Self type
object JoinId

Attributes

Source
GraphIds.scala
Supertypes
class Object
trait Matchable
class Any
Self type
JoinId.type
object NodeId

Attributes

Source
GraphIds.scala
Supertypes
class Object
trait Matchable
class Any
Self type
NodeId.type
final class NodeRef[I]

A handle to a node that consumes I, issued by a GraphBuilder.

A handle to a node that consumes I, issued by a GraphBuilder.

Handles are the only way to route: a node cannot name another by string. Goto takes a NodeRef[Unit] and Send a payload of the target's input type, so a mis-typed route does not compile. A handle from another builder is rejected when the graph compiles, or when a node returns it at run time.

Attributes

Source
GraphNode.scala
Supertypes
class Object
trait Matchable
class Any
enum NodeResult

What a node task produced.

What a node task produced.

Attributes

Companion
object
Source
GraphNode.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
Known subtypes
class Suspend[Q, A]
object NodeResult

Attributes

Companion
enum
Source
GraphNode.scala
Supertypes
trait Sum
trait Mirror
class Object
trait Matchable
class Any
Self type
NodeResult.type
final case class Observer(capacity: Int, listener: StreamEvent => Unit, nested: Boolean)

A listener subscribed to a run's thread during admission - before the run's claim commits, with no replay - so it receives every durable event of the run from its claim, and every live event, of which a GraphRuntime.subscribe made after start returns can miss the first: a subscription receives only the live events sent after it returns. capacity is the subscription's queue size, at least 2, as for GraphRuntime.subscribe. Its durable events start at the run's claim, as a RunHandle.subscribe's do: a commit of an earlier run on the thread handed over during admission - a lost run's late write, even one of a run with the same RunId - is not delivered, nor is a live event sent before the claim, nor one of an earlier claim sent after it. The run's own claim is recognised by its fencing token (EventRecord.token, StreamEvent.Live.token), which, unlike a run id, is never reused. The subscription is thread-scoped like any other: it keeps delivering later runs until cancelled, through RunHandle.observation. A refused admission abandons it, and its listener is never called. With nested, it also receives the events of the children the run delegates to, as StreamEvent.Nested (design 4.47).

A listener subscribed to a run's thread during admission - before the run's claim commits, with no replay - so it receives every durable event of the run from its claim, and every live event, of which a GraphRuntime.subscribe made after start returns can miss the first: a subscription receives only the live events sent after it returns. capacity is the subscription's queue size, at least 2, as for GraphRuntime.subscribe. Its durable events start at the run's claim, as a RunHandle.subscribe's do: a commit of an earlier run on the thread handed over during admission - a lost run's late write, even one of a run with the same RunId - is not delivered, nor is a live event sent before the claim, nor one of an earlier claim sent after it. The run's own claim is recognised by its fencing token (EventRecord.token, StreamEvent.Live.token), which, unlike a run id, is never reused. The subscription is thread-scoped like any other: it keeps delivering later runs until cancelled, through RunHandle.observation. A refused admission abandons it, and its listener is never called. With nested, it also receives the events of the children the run delegates to, as StreamEvent.Nested (design 4.47).

Attributes

Source
Observer.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
final case class PendingInterrupt(id: InterruptId, resumeNode: NodeId, question: Value, breakpoint: Option[BreakpointPhase])

A parked continuation, as reported by a suspended run. For a task a static breakpoint holds, breakpoint is its phase, resumeNode is the held task's own node, and question is the task's input before it ran, or JSON null after; answer it with Breakpoint.proceed.

A parked continuation, as reported by a suspended run. For a task a static breakpoint holds, breakpoint is its phase, resumeNode is the held task's own node, and question is the task's input before it ran, or JSON null after; answer it with Breakpoint.proceed.

Attributes

Source
GraphNode.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
final case class PendingWrite(checkpointId: String, taskId: String, nodeId: String, operations: Vector[EncodedOperation], routes: Vector[EncodedRoute], suspension: Option[EncodedSuspension], resultDigest: Option[String])

A completed (or suspended) task's result, recorded against the checkpoint whose frontier it ran in, before the superstep commits. On recovery the task is not run again: its command is decoded from here.

A completed (or suspended) task's result, recorded against the checkpoint whose frontier it ran in, before the superstep commits. On recovery the task is not run again: its command is decoded from here.

Attributes

Source
Checkpoint.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object Principal

Attributes

Source
GraphIds.scala
Supertypes
class Object
trait Matchable
class Any
Self type
Principal.type
@Experimental
object ReplayGuard

The only way the harness performs an effect outside thread state (design ยง4.24, runtime rule 4). A task can run more than once - recover, a retry, a re-run after approval or an answer - so each effect keeps evidence in the thread's store under its EffectKey: an intent before the effect and the outcome after it. A completed effect replays its stored outcome, decoded with A's codec; what a replay of an intent alone does depends on the target's ReplaySupport, one entry point each. The evidence lives exactly as long as the task can run again: the commit that ends that releases it (Commit.releasesEffect), never age.

The only way the harness performs an effect outside thread state (design ยง4.24, runtime rule 4). A task can run more than once - recover, a retry, a re-run after approval or an answer - so each effect keeps evidence in the thread's store under its EffectKey: an intent before the effect and the outcome after it. A completed effect replays its stored outcome, decoded with A's codec; what a replay of an intent alone does depends on the target's ReplaySupport, one entry point each. The evidence lives exactly as long as the task can run again: the commit that ends that releases it (Commit.releasesEffect), never age.

Every Left is either GraphError.EffectReplayRefused - a refusal the model should see, which a tool maps to ToolOutcome.Error - or a failure that leaves the outcome unknown or unrecorded (the store, or the effect itself), which a tool maps to ToolOutcome.Fatal and a hook to a failed node, so the task commits nothing and recover replays it from the evidence. An effect returns Left only when it did not take effect, or cannot tell: the guard never stores it.

Intents and outcomes are capped at maxOutcomeBytes encoded. An oversized intent refuses the effect before it runs; an oversized outcome is recorded as omitted, and its replay is refused, never run again.

Attributes

Source
EffectReplay.scala
Supertypes
class Object
trait Matchable
class Any
Self type
@Experimental

How a target lets the harness replay an effect on it; every external target declares one (design ยง4.24).

How a target lets the harness replay an effect on it; every external target declares one (design ยง4.24).

Attributes

Source
EffectReplay.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
final class ResumeRef[Q, A]

A handle to a node that continues suspended work: it consumes Resumed[Q, A]. Issued by GraphBuilder.declareResume; a node may suspend only to a resume handle. The question and answer codecs belong to the graph - a checkpoint stores the question as JSON and a resume supplies the answer as JSON, and both are decoded here.

A handle to a node that continues suspended work: it consumes Resumed[Q, A]. Issued by GraphBuilder.declareResume; a node may suspend only to a resume handle. The question and answer codecs belong to the graph - a checkpoint stores the question as JSON and a resume supplies the answer as JSON, and both are decoded here.

Attributes

Source
GraphNode.scala
Supertypes
class Object
trait Matchable
class Any
final case class Resumed[Q, A](question: Q, answer: A)

A suspended task's question with the answer it was resumed with.

A suspended task's question with the answer it was resumed with.

Attributes

Source
GraphNode.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
final case class RetentionPolicy

How much of a thread's history Checkpointer.prune keeps, by age or by size. Every limit is optional, and one that is not set removes nothing, so RetentionPolicy() keeps everything.

How much of a thread's history Checkpointer.prune keeps, by age or by size. Every limit is optional, and one that is not set removes nothing, so RetentionPolicy() keeps everything.

  • maxAge: a checkpoint, and an event, that the store committed more than maxAge ago is removed - by the store's clock, against the commit time the store recorded with it, never the checkpoint's createdAt or the event's timestamp: those are the producer's record, and a producer whose clock is skewed would make them too old or too young.
  • maxCheckpoints: only the newest maxCheckpoints checkpoints are kept.
  • maxEvents: only the newest maxEvents events are kept.

Whatever the limits, the thread's latest checkpoint and its pending writes are never removed, so pruning never changes what start, recover or resume does next. Pruning a checkpoint also removes the events recorded before the oldest checkpoint kept; maxAge and maxEvents may remove more events than that, including the latest checkpoint's. Removing events raises the thread's replay floor exactly as Checkpointer.compactEvents does (GraphError.ReplayUnavailable).

apply and the with* setters throw IllegalArgumentException for a non-positive maxAge or maxCheckpoints, or a negative maxEvents; RetentionPolicy.of returns a ValidationError.

Attributes

Companion
object
Source
RetentionPolicy.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all

Attributes

Companion
class
Source
RetentionPolicy.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
final case class RetryPolicy

How a node is run again after its own failure, inside the run.

How a node is run again after its own failure, inside the run.

A node that throws, or returns NodeResult.Fail, is run again against the same committed snapshot while its error passes retryOn and attempts remain. By default an error is retried by the library's one retry rule, org.llm4s.reliability.RetryPolicy.isRetryable: a recoverable error, except a client-error response and an optimistic-lock failure, which repeating the node cannot fix. A cancellation, a result the kernel rejects (an undeclared write, an invalid route) and a suspension are never retried.

The wait after the n-th failed attempt is initialBackoff * backoffFactor^(n-1), at most maxBackoff. It is an interruptible wait: cancelling the run, or its deadline expiring, ends the retries. The attempts belong to one run: recover runs a failed task again with its full policy.

Every value is validated on the way in: apply and the with* setters throw IllegalArgumentException; RetryPolicy.of returns a ValidationError for untrusted input.

Value parameters

backoffFactor

what each further wait is multiplied by; at least 1.0

initialBackoff

the wait after the first failed attempt

maxAttempts

the most times the node is run, the first included; 1 means no retry

maxBackoff

the longest wait

retryOn

whether an error is worth another attempt

Attributes

Companion
object
Source
RetryPolicy.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object RetryPolicy

Attributes

Companion
class
Source
RetryPolicy.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
enum Route

A routing decision returned by a node; scheduled for the next superstep.

A routing decision returned by a node; scheduled for the next superstep.

Attributes

Source
GraphNode.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
Known subtypes
class Send[I]
class FanOut[I]
final case class RunBudgets

Limits on one run; each start, recover or resume brings its own. Every value is validated on the way in: apply and the with* setters throw IllegalArgumentException for a non-positive one, and RunBudgets.of returns it as a ValidationError.

Limits on one run; each start, recover or resume brings its own. Every value is validated on the way in: apply and the with* setters throw IllegalArgumentException for a non-positive one, and RunBudgets.of returns it as a ValidationError.

maxTokens is the input and output tokens the run may use, counted from the thread's usage when the run was admitted: before each model call - the loop's own and every one a middleware makes through next or ModelContext.accounted - the run's tokens so far are compared with it, and at or above it the turn ends TokenLimitReached without calling the model. A model call in flight is never interrupted, so a run can overshoot the budget by one call. A completion without usage is charged an estimate of its whole request and answer. A retrying client (ReliableClient) is one call: its failed attempts carry no usage and are not counted, so set maxTokens with retries in mind. None, the default, sets no token budget. A plain graph's nodes never read it.

Attributes

Companion
object
Source
RunConfig.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object RunBudgets

Attributes

Companion
class
Source
RunConfig.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
RunBudgets.type
final case class RunClaim

A run's lease on a thread, as a Checkpointer granted it: the run that holds it, the FencingToken its commits must carry, and when it expires by the store's clock unless renewed. While it is live - expiresAt has not passed - no other claim on the thread is granted, so a run in another runtime or process is refused with GraphError.ThreadBusy. Once it has expired, another run may take the thread over, and the store then refuses this claim's token (GraphError.StaleClaim). tenantId is the tenant the claim was requested for (ClaimRequest.tenantId). takenOverFrom is the holder of the expired claim, never released, that this one replaced, named only when that claim was of the same tenant as this one: None when the thread was free, and when the replaced claim was another tenant's or recorded no tenant (a claim a store persisted before claims carried one), so that a takeover never reveals another tenant's RunId.

A run's lease on a thread, as a Checkpointer granted it: the run that holds it, the FencingToken its commits must carry, and when it expires by the store's clock unless renewed. While it is live - expiresAt has not passed - no other claim on the thread is granted, so a run in another runtime or process is refused with GraphError.ThreadBusy. Once it has expired, another run may take the thread over, and the store then refuses this claim's token (GraphError.StaleClaim). tenantId is the tenant the claim was requested for (ClaimRequest.tenantId). takenOverFrom is the holder of the expired claim, never released, that this one replaced, named only when that claim was of the same tenant as this one: None when the thread was free, and when the replaced claim was another tenant's or recorded no tenant (a claim a store persisted before claims carried one), so that a takeover never reveals another tenant's RunId.

To the store, the holder may still commit and renew past expiresAt until another claim replaces it: expiry only lets another run claim the thread. A GraphRuntime does not rely on that: a run judges its own lease, by its monotonic clock from just before it sent the request that granted or last renewed the claim, and stops with GraphError.ClaimLost once that lease has passed. That is no later than the store's expiry provided the store judges it by one clock that does not run faster than the runtime's monotonic clock and is not stepped forward; a constant offset does not matter. A store clock that runs ahead - stepped forward, or each process of a shared store on its own skewed clock - can expire the claim early, and another run can take the thread over while this one still runs: it then stops at its next refused renewal or commit, and fencing keeps every write safe.

Attributes

Companion
object
Source
RunClaim.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object RunClaim

Attributes

Companion
class
Source
RunClaim.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
RunClaim.type
final case class RunConfig

The identity and limits of one run. RunConfig() is evaluated per call, so each gets a fresh RunId. The tenant is part of the thread's identity: it is recorded on every checkpoint, and admission refuses a run whose tenantId differs from the latest checkpoint's (GraphError.TenantMismatch; None and Some differ). The principal is recorded on the run's RunStarted, RunRecovered and RunResumed events and is never checked.

The identity and limits of one run. RunConfig() is evaluated per call, so each gets a fresh RunId. The tenant is part of the thread's identity: it is recorded on every checkpoint, and admission refuses a run whose tenantId differs from the latest checkpoint's (GraphError.TenantMismatch; None and Some differ). The principal is recorded on the run's RunStarted, RunRecovered and RunResumed events and is never checked.

metadata, when not empty, is the thread's ThreadMetadata (design 4.52): a top-level start sets it in its claim commit, replacing what the thread had, and refuses metadata that breaks its limits with a ValidationError before it claims anything. recover, resume and a child thread's run leave it as it is; change it with GraphRuntime.updateMetadata.

interruptBefore and interruptAfter are the run's static breakpoints (BreakpointPhase): a task of a node in interruptBefore is held before it runs, and one of a node in interruptAfter after it ran, each as an interrupt of its own that GraphRuntime.resume answers with Breakpoint.proceed. They belong to the run, like its budgets: a resume or recover brings its own, and a continuation already past a breakpoint is not held by it again. Admission refuses a node the graph does not have with a ValidationError.

Attributes

Companion
object
Source
RunConfig.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object RunConfig

Attributes

Companion
class
Source
RunConfig.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
RunConfig.type
final class RunContext

What a running task knows: the run's RunConfig, its own RunPosition, and its event channels.

What a running task knows: the run's RunConfig, its own RunPosition, and its event channels.

emit records a durable custom event: it is committed with this task's result, given a per-thread sequence number in that commit, delivered only after the commit and replayed to later subscribers. It is discarded if the task fails. progress is live-only, for token deltas and similar high-volume progress, and carries a name and version like emit: delivered at once to current subscribers, never persisted or replayed. Outside a GraphRuntime both are no-ops.

Attributes

Source
RunConfig.scala
Supertypes
class Object
trait Matchable
class Any
enum RunEvent

What happened in a run. Durable: persisted in the thread's event log and replayable.

What happened in a run. Durable: persisted in the thread's event log and replayable.

The three events that begin a run record the run's tenantId and principal (as supplied in its RunConfig); the log reads events written before they existed as having neither.

Attributes

Companion
object
Source
RunEvent.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object RunEvent

Attributes

Companion
enum
Source
RunEvent.scala
Supertypes
trait Sum
trait Mirror
class Object
trait Matchable
class Any
Self type
RunEvent.type
trait RunHandle[O]

A run admitted by GraphRuntime.start, GraphRuntime.recover or GraphRuntime.resume. Its thread was claimed before the handle was returned, and the run executes on a virtual thread owned by the runtime, named llm4s-run-<threadId>, which holds the thread until it exits.

A run admitted by GraphRuntime.start, GraphRuntime.recover or GraphRuntime.resume. Its thread was claimed before the handle was returned, and the run executes on a virtual thread owned by the runtime, named llm4s-run-<threadId>, which holds the thread until it exits.

Attributes

Source
RunHandle.scala
Supertypes
class Object
trait Matchable
class Any
object RunId

Attributes

Source
GraphIds.scala
Supertypes
class Object
trait Matchable
class Any
Self type
RunId.type
@Experimental
final case class RunOrigin

Where a child thread came from (design 4.41): the thread whose run started it (parent), the tool call of that run that started it (call, the call's IdempotencyKey), the agent it runs (agent), when it was first admitted (startedAt, by the runtime's clock: GraphRuntime.startChild stamps it, whatever the caller passed), and how deep it is: 1 for a child of a top-level thread, one more for each level below.

Where a child thread came from (design 4.41): the thread whose run started it (parent), the tool call of that run that started it (call, the call's IdempotencyKey), the agent it runs (agent), when it was first admitted (startedAt, by the runtime's clock: GraphRuntime.startChild stamps it, whatever the caller passed), and how deep it is: 1 for a child of a top-level thread, one more for each level below.

A thread is a child thread exactly when its latest checkpoint records an origin (Checkpoint.origin), never because of its id's shape. The origin is written with the child's first checkpoint and copied onto every later one, as the tenant is.

trace is the trace context of the parent's tool call that started the child (design 4.58): every run of the child

  • its first, and each recovery or resume, on whichever machine - takes it as its span's parent, so the child stays in the trace of the run that started it. None on a child first admitted before it existed, whose runs continue the context of the call that drives them. A child thread is driven only through its parent's run: start, recover, resume, fork, updateState and deleteThread on it are refused with GraphError.ChildThread.

Attributes

Companion
object
Source
ChildThreads.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object RunOrigin

Attributes

Companion
class
Source
ChildThreads.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
RunOrigin.type
@Experimental
enum RunOutcome

How a run settled, recorded on the terminal (Completed or Failed) checkpoint that closed it (Checkpoint.outcome), so a settled run's result can be rebuilt from the thread alone, after a restart as well as live (design 4.41). A later run's first checkpoint carries none.

How a run settled, recorded on the terminal (Completed or Failed) checkpoint that closed it (Checkpoint.outcome), so a settled run's result can be rebuilt from the thread alone, after a restart as well as live (design 4.41). A later run's first checkpoint carries none.

The answer itself is in the thread's state, so Answered carries none. A graph that is not an agent turn records Answered when it completes and Failed when a node blocks it; any graph records Blocked for a GuardrailBlocked Block and OutputInvalid for a GraphError.OutputInvalid one.

Attributes

Companion
object
Source
ChildThreads.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object RunOutcome

Attributes

Companion
enum
Source
ChildThreads.scala
Supertypes
trait Sum
trait Mirror
class Object
trait Matchable
class Any
Self type
RunOutcome.type
final case class RunPosition

Where a task runs. checkpointId is the checkpoint whose frontier it belongs to; "" outside a GraphRuntime. fencingToken is the token of the run's claim on the thread (RunClaim), None outside a runtime: a node that writes to an external system able to fence can pass it on, so that system too refuses a run that has lost its claim.

Where a task runs. checkpointId is the checkpoint whose frontier it belongs to; "" outside a GraphRuntime. fencingToken is the token of the run's claim on the thread (RunClaim), None outside a runtime: a node that writes to an external system able to fence can pass it on, so that system too refuses a run that has lost its claim.

Attributes

Companion
object
Source
RunConfig.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object RunPosition

Attributes

Companion
class
Source
RunConfig.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
enum RunResult[+O]

The outcome of a run.

The outcome of a run.

Attributes

Source
Execution.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
Known subtypes
class Completed[O]
class Suspended
class Failed
enum RunStatus

Where a run is; see RunHandle.status.

Where a run is; see RunHandle.status.

Attributes

Source
RunHandle.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
final class SchemaVersion

The current version of a persisted JSON shape, with the migrations that bring older versions up to it. A state key's value, a state key's update and a node's input each carry one; their stable codec id is the owner's id with this version (<keyId>@<version>), and every encoded value records the version it was written with (see VersionedJson).

The current version of a persisted JSON shape, with the migrations that bring older versions up to it. A state key's value, a state key's update and a node's input each carry one; their stable codec id is the owner's id with this version (<keyId>@<version>), and every encoded value records the version it was written with (see VersionedJson).

A step keyed n migrates JSON written at version n to version n + 1. Steps are plain functions of the compiled graph, never persisted.

Attributes

Companion
object
Source
SchemaVersion.scala
Supertypes
class Object
trait Matchable
class Any
object SchemaVersion

Attributes

Companion
class
Source
SchemaVersion.scala
Supertypes
class Object
trait Matchable
class Any
Self type
@Experimental
object StateCheck

Attributes

Source
EffectReplay.scala
Supertypes
class Object
trait Matchable
class Any
Self type
StateCheck.type
final class StateKey[A, U]

A typed slot in a graph's thread state.

A typed slot in a graph's thread state.

A is the stored value and U the update a node emits for it. Every update - including a single writer's - is applied to the committed value with applyUpdate, in deterministic task and emission order; replacement is the special case U = A (see StateKey.replace). The function belongs to the compiled graph, never to a checkpoint: checkpoints hold values encoded with stateCodec and pending updates encoded with updateCodec, each tagged with the codec's SchemaVersion so an older checkpoint is migrated before it is decoded.

Keys are compared by identity. A graph rejects two distinct keys with the same id.

Attributes

Companion
object
Source
StateKey.scala
Supertypes
class Object
trait Matchable
class Any
object StateKey

Attributes

Companion
class
Source
StateKey.scala
Supertypes
class Object
trait Matchable
class Any
Self type
StateKey.type
object StateKeyId

Attributes

Source
GraphIds.scala
Supertypes
class Object
trait Matchable
class Any
Self type
StateKeyId.type
final class StateUpdate

An ordered list of state operations emitted by one node task.

An ordered list of state operations emitted by one node task.

Operations apply in emission order. combine is sequential composition: a.combine(b) applies a's operations and then b's, so for a StateKey.replace key the later value wins and for an operation-valued key both operations apply. Updates from different tasks in one superstep are never combined; the scheduler applies them task by task in frontier order.

Attributes

Companion
object
Source
StateUpdate.scala
Supertypes
class Object
trait Matchable
class Any
object StateUpdate

Attributes

Companion
class
Source
StateUpdate.scala
Supertypes
class Object
trait Matchable
class Any
Self type
final class StaticJoin

A barrier that schedules target once every source node has completed a task since its last release. Each source counts once per activation, however many of its tasks complete.

A barrier that schedules target once every source node has completed a task since its last release. Each source counts once per activation, however many of its tasks complete.

Attributes

Source
GraphNode.scala
Supertypes
class Object
trait Matchable
class Any
enum Step[+O]

The outcome of one superstep.

The outcome of one superstep.

Attributes

Source
Execution.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
Known subtypes
class Next
class Done[O]
final case class StoredCheckpoint(checkpoint: Checkpoint, pendingWrites: Vector[PendingWrite])

A thread's latest checkpoint with the pending writes recorded against it.

A thread's latest checkpoint with the pending writes recorded against it.

Attributes

Source
Checkpoint.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all

What a subscriber receives. A payload - of a StreamEvent.Live or a RunEvent.Custom - is a snapshot taken when the node emitted it, and one value may be shared by every subscriber: a listener must not mutate it, and copies it (ujson.copy) to change it.

What a subscriber receives. A payload - of a StreamEvent.Live or a RunEvent.Custom - is a snapshot taken when the node emitted it, and one value may be shared by every subscriber: a listener must not mutate it, and copies it (ujson.copy) to change it.

Attributes

Source
RunEvent.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
trait Subscription

A registration with GraphRuntime.subscribe or RunHandle.subscribe. It is scoped to a thread, not a run: it keeps delivering the events of every later run on the same thread, and its dispatcher - a virtual thread, parked while there is nothing to deliver - lives until cancel is called or the subscription is disconnected. Cancel every subscription you no longer need.

A registration with GraphRuntime.subscribe or RunHandle.subscribe. It is scoped to a thread, not a run: it keeps delivering the events of every later run on the same thread, and its dispatcher - a virtual thread, parked while there is nothing to deliver - lives until cancel is called or the subscription is disconnected. Cancel every subscription you no longer need.

It delivers the durable events of every commit on the thread, made through the GraphRuntime it was made on or through any other runtime or process sharing the same Checkpointer: the latter it reads from the store as they land (WatchPolicy). Live progress events reach it only from runs of its own runtime, because they are never stored.

Attributes

Source
RunEvent.scala
Supertypes
class Object
trait Matchable
class Any
@Experimental
trait SweepGuard

What a Checkpointer.sweep must not take from a thread, asked once per page before the page changes: for each thread of threads that something outside the store still depends on, the RunIds whose newest checkpoint on that thread must be kept. A thread in the answer - with an empty set or not - keeps its tombstone; for each RunId in its set, the newest checkpoint that run committed is kept, whatever the retention policy, while everything else of the thread is pruned as usual (Stage 5, rule 4: a run queue's open entry names the thread and each attempt's RunId, and the newest checkpoint of each is the evidence that the entry's input was admitted). A thread missing from the answer has nothing protected.

What a Checkpointer.sweep must not take from a thread, asked once per page before the page changes: for each thread of threads that something outside the store still depends on, the RunIds whose newest checkpoint on that thread must be kept. A thread in the answer - with an empty set or not - keeps its tombstone; for each RunId in its set, the newest checkpoint that run committed is kept, whatever the retention policy, while everything else of the thread is pruned as usual (Stage 5, rule 4: a run queue's open entry names the thread and each attempt's RunId, and the newest checkpoint of each is the evidence that the entry's input was admitted). A thread missing from the answer has nothing protected.

A Left ends the sweep's call with that error, changing nothing of the page. The guard is called outside the store's transactions and locks, so it may read another store, such as a run queue in the same database; it is called with at most SweepPolicy.batchSize threads.

Attributes

Companion
object
Source
RetentionSweep.scala
Supertypes
class Object
trait Matchable
class Any
object SweepGuard

Attributes

Companion
trait
Source
RetentionSweep.scala
Supertypes
class Object
trait Matchable
class Any
Self type
SweepGuard.type
@Experimental
final case class SweepPage

One call of Checkpointer.sweep: how many threads it visited (examined), how many tombstones it reclaimed (reclaimed), and the cursor of the next page - the last thread visited - when more threads follow, None once the sweep has visited the whole store.

One call of Checkpointer.sweep: how many threads it visited (examined), how many tombstones it reclaimed (reclaimed), and the cursor of the next page - the last thread visited - when more threads follow, None once the sweep has visited the whole store.

Attributes

Companion
object
Source
RetentionSweep.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object SweepPage

Attributes

Companion
class
Source
RetentionSweep.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
SweepPage.type
@Experimental
final case class SweepPolicy

What a store-wide retention sweep (Checkpointer.sweep, design 4.53) does to each thread it visits: it prunes the thread's history by retention, exactly as Checkpointer.prune does, and - when tombstoneAge is set - reclaims the tombstone of a deleted thread once the store deleted it more than tombstoneAge ago, by the store's clock. At most batchSize threads are visited by one call, so every call stays within the store's bound however many threads it holds; a caller pages through the store with the cursor each call returns.

What a store-wide retention sweep (Checkpointer.sweep, design 4.53) does to each thread it visits: it prunes the thread's history by retention, exactly as Checkpointer.prune does, and - when tombstoneAge is set - reclaims the tombstone of a deleted thread once the store deleted it more than tombstoneAge ago, by the store's clock. At most batchSize threads are visited by one call, so every call stays within the store's bound however many threads it holds; a caller pages through the store with the cursor each call returns.

SweepPolicy() changes nothing: it keeps every checkpoint and event and reclaims no tombstone.

apply and the with* setters throw IllegalArgumentException for a non-positive tombstoneAge or a batchSize outside 1 to SweepPolicy.MaxBatchSize; SweepPolicy.of returns a ValidationError.

Attributes

Companion
object
Source
RetentionSweep.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object SweepPolicy

Attributes

Companion
class
Source
RetentionSweep.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
object TaskId

Attributes

Source
GraphIds.scala
Supertypes
class Object
trait Matchable
class Any
Self type
TaskId.type
object TenantId

Attributes

Source
GraphIds.scala
Supertypes
class Object
trait Matchable
class Any
Self type
TenantId.type
@Experimental
final case class ThreadCursor(updatedAt: Instant, threadId: ThreadId)

Where a page of the thread registry ended: the last thread's ThreadSummary.updatedAt and id, which the next page starts after (ThreadQuery.after, Checkpointer.incompleteThreads). It names a position, not a thread, so it stays valid when that thread changes or is deleted.

Where a page of the thread registry ended: the last thread's ThreadSummary.updatedAt and id, which the next page starts after (ThreadQuery.after, Checkpointer.incompleteThreads). It names a position, not a thread, so it stays valid when that thread changes or is deleted.

Attributes

Source
ThreadRegistry.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object ThreadId

Attributes

Source
GraphIds.scala
Supertypes
class Object
trait Matchable
class Any
Self type
ThreadId.type
@Experimental
final case class ThreadMetadata

A thread's metadata: a small map of strings its caller sets, for finding the thread again (ThreadQuery.metadata), not for content - it is stored as it is, read by operators, and copied into no event or trace. It is set by a start whose RunConfig.metadata is not empty, and replaced with GraphRuntime.updateMetadata; in the store, by a fenced Commit.metadata. A deleted thread's metadata goes with it.

A thread's metadata: a small map of strings its caller sets, for finding the thread again (ThreadQuery.metadata), not for content - it is stored as it is, read by operators, and copied into no event or trace. It is set by a start whose RunConfig.metadata is not empty, and replaced with GraphRuntime.updateMetadata; in the store, by a fenced Commit.metadata. A deleted thread's metadata goes with it.

At most ThreadMetadata.MaxEntries entries; each key is 1 to ThreadMetadata.MaxKeyLength characters of A-Z a-z 0-9 _ . : / -, and each value at most ThreadMetadata.MaxValueLength characters, none of them a control character.

Attributes

Companion
object
Source
ThreadRegistry.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all

Attributes

Companion
class
Source
ThreadRegistry.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
@Experimental
final case class ThreadPage(threads: Vector[ThreadSummary], next: Option[ThreadCursor])

A page of the thread registry and where the next one starts: next is the last thread's cursor when more threads match, and None on the last page.

A page of the thread registry and where the next one starts: next is the last thread's cursor when more threads match, and None on the last page.

Attributes

Companion
object
Source
ThreadRegistry.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object ThreadPage

Attributes

Companion
class
Source
ThreadRegistry.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
ThreadPage.type
@Experimental
final case class ThreadQuery

A search of the thread registry (Checkpointer.threads): the root threads of one tenant - None and Some differ, as for claims: None finds only threads of no tenant - narrowed by every filter given, newest first by ThreadSummary.updatedAt, at most limit at a time (1 to ThreadQuery.MaxLimit), starting after after.

A search of the thread registry (Checkpointer.threads): the root threads of one tenant - None and Some differ, as for claims: None finds only threads of no tenant - narrowed by every filter given, newest first by ThreadSummary.updatedAt, at most limit at a time (1 to ThreadQuery.MaxLimit), starting after after.

  • status: only threads of that ThreadStatus;
  • agent: only threads whose latest checkpoint is of that graph (GraphSnapshot.graphId; an Agent's is its id);
  • metadata: only threads whose metadata holds every one of these entries (ThreadMetadata.matches);
  • updatedSince: only threads the store changed at or after this instant, by its clock.

Attributes

Companion
object
Source
ThreadRegistry.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object ThreadQuery

Attributes

Companion
class
Source
ThreadRegistry.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
final class ThreadState

The committed state of a graph thread: a typed key/value store over the graph's registered StateKeys. It is immutable, so every task in a superstep reads the same snapshot.

The committed state of a graph thread: a typed key/value store over the graph's registered StateKeys. It is immutable, so every task in a superstep reads the same snapshot.

Reads are checked: a key the graph did not register - or a different key instance sharing a registered key's id - is an GraphError.UnknownStateKey. A registered key with no stored entry reads as its initial value. The erasure to Any stays private to this boundary.

Attributes

Source
ThreadState.scala
Supertypes
class Object
trait Matchable
class Any
@Experimental

Where a thread is, as the thread registry reports it (Checkpointer.threads, design 4.52). It is derived, when the store is read, from the thread's latest checkpoint and its claim, judged by the store's clock:

Where a thread is, as the thread registry reports it (Checkpointer.threads, design 4.52). It is derived, when the store is read, from the thread's latest checkpoint and its claim, judged by the store's clock:

  • Deleted: a cascade delete of the thread has begun (Checkpoint.deleting) and not finished: the thread refuses every admission until a deleteThread finishes it. A thread deleteThread removed is not listed at all: its tombstone holds nothing of it, not even its tenant.
  • Running: a live claim holds the thread - a run, or a change such as updateState, fork or a delete, is under way on some machine - whatever its latest checkpoint says.
  • Idle: no live claim holds the thread and its latest checkpoint is incomplete (CheckpointStatus.Running): a run stopped before it finished - it crashed, was cancelled or was drained, which look alike here - and recover would continue it. Checkpointer.incompleteThreads lists these, oldest first.
  • Suspended, Completed, Failed: no live claim holds the thread, and its latest checkpoint has that status.

Attributes

Companion
object
Source
ThreadRegistry.scala
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object ThreadStatus

Attributes

Companion
enum
Source
ThreadRegistry.scala
Supertypes
trait Sum
trait Mirror
class Object
trait Matchable
class Any
Self type
@Experimental
final case class ThreadSummary

One thread as the registry lists it: its id and tenant, its agent (the graph id of its latest checkpoint), its ThreadStatus, its latest checkpoint's id, its metadata, and updatedAt - when the store last committed a checkpoint or metadata to it, by the store's clock (never a producer's timestamp).

One thread as the registry lists it: its id and tenant, its agent (the graph id of its latest checkpoint), its ThreadStatus, its latest checkpoint's id, its metadata, and updatedAt - when the store last committed a checkpoint or metadata to it, by the store's clock (never a producer's timestamp).

Attributes

Companion
object
Source
ThreadRegistry.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object ThreadSummary

Attributes

Companion
class
Source
ThreadRegistry.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
object ToolCallId

Attributes

Source
GraphIds.scala
Supertypes
class Object
trait Matchable
class Any
Self type
ToolCallId.type
object ToolName

Attributes

Source
GraphIds.scala
Supertypes
class Object
trait Matchable
class Any
Self type
ToolName.type

Bridges a thread's durable run events to a org.llm4s.trace.Tracing: each becomes a TraceEvent.CustomEvent named graph.<event> (RunStarted is graph.run_started), stamped with the record's timestamp. Its data holds threadId, runId, seq, checkpointId, taskId and nodeId (null when absent) and the event's own fields. Live progress is not traced.

Bridges a thread's durable run events to a org.llm4s.trace.Tracing: each becomes a TraceEvent.CustomEvent named graph.<event> (RunStarted is graph.run_started), stamped with the record's timestamp. Its data holds threadId, runId, seq, checkpointId, taskId and nodeId (null when absent) and the event's own fields. Live progress is not traced.

A subscription that falls behind is disconnected like any other: a StreamEvent.Disconnected(lastSeq, DisconnectReason.Lagging) ends tracing, and is only logged. Nothing re-attaches by itself: to keep tracing, the caller attaches again with afterSeq = lastSeq, which replays from the first event not traced.

Attach it to any graph's thread, kernel graphs included. An Agent built withTracing traces through AgentTracing instead, which adds usage and AgentRunEnded; use attach for graphs of your own.

Attributes

Source
TracingSubscriber.scala
Supertypes
class Object
trait Matchable
class Any
Self type
final case class VersionedJson(version: Int, value: Value)

A persisted value and the version of the codec that wrote it.

A persisted value and the version of the codec that wrote it.

Attributes

Source
SchemaVersion.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
final case class WatchPolicy

How a GraphRuntime's subscriptions learn of commits made through another runtime or process sharing its Checkpointer. Commits made through the runtime itself are handed to its subscriptions as they land, whatever this says.

How a GraphRuntime's subscriptions learn of commits made through another runtime or process sharing its Checkpointer. Commits made through the runtime itself are handed to its subscriptions as they land, whatever this says.

While enabled, each live subscription watches the store on a virtual thread of its own, named llm4s-watch-<threadId>, calling Checkpointer.awaitEventsAfter with pollInterval as the timeout, from the last event it has queued. A store that can be told of commits returns as soon as one lands; one that cannot - the default, and the SQLite store - is read once per pollInterval, so a commit made elsewhere reaches the subscription up to pollInterval after it lands. Only durable events cross runtimes: live progress (StreamEvent.Live) is never stored, so it reaches only the subscriptions of the runtime whose run sent it. When disabled, a subscription sees another runtime's commits only by subscribing again, which replays the log; a runtime that knows it is the only one using its store can disable it to save the watching thread and its reads.

'''Ordering.''' Each subscription reads on its own cursor, so a commit made elsewhere falls at a point among the runtime's live events that depends on which read reached that subscription (its initial replay, its watch, or a hand-over's gap read): two subscriptions can place it differently, though each delivers the durable events in seq order. Concurrently active subscriptions of one runtime that stay caught up share one live/durable interleaving, every event handed to several of them having the same relative placement for each, while nothing outside the runtime commits to the thread (#1920); an external commit allows them to differ, not necessarily make them differ. Where one order across runtimes matters, order durable events by seq (see GraphRuntime.subscribe).

pollInterval must be at least one millisecond - a shorter one would have every subscription read a polled store in a busy loop: apply and the with* setters throw IllegalArgumentException otherwise, and WatchPolicy.of returns a ValidationError.

Attributes

Companion
object
Source
WatchPolicy.scala
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object WatchPolicy

Attributes

Companion
class
Source
WatchPolicy.scala
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type

Types

opaque type FencingToken

The fencing token of a run's claim on a thread (RunClaim). A Checkpointer issues one with every claim, strictly greater than every token it issued for that thread before - including before a takeover, a release or a deleteThread - and accepts a commit only with the token of the thread's current claim, so a writer that lost its claim cannot commit after another took over.

The fencing token of a run's claim on a thread (RunClaim). A Checkpointer issues one with every claim, strictly greater than every token it issued for that thread before - including before a takeover, a release or a deleteThread - and accepts a commit only with the token of the thread's current claim, so a writer that lost its claim cannot commit after another took over.

Attributes

Source
GraphIds.scala
opaque type IdempotencyKey

The key a tool gives an external system so that a second run of the same tool call does not repeat its side effect: one key per model-issued call, the same in every run of that call (a retrying wrapper, an approval, an answered question, recover after a failure, a cancellation or a crash), and different for a call from another model request, even one that reuses the provider's call id.

The key a tool gives an external system so that a second run of the same tool call does not repeat its side effect: one key per model-issued call, the same in every run of that call (a retrying wrapper, an approval, an answered question, recover after a failure, a cancellation or a crash), and different for a call from another model request, even one that reuses the provider's call id.

The agent loop derives it with IdempotencyKey.ofCall from the model task that issued the call (its key, lineage included) and the call's id, name and arguments, and records it with the call, so it reaches the tool as ToolContext.idempotencyKey. It de-duplicates only where the external system honours it: the runtime runs a tool at least once, never exactly once (design ยง4.1).

Attributes

Source
GraphIds.scala
opaque type InterruptId

Identifies a parked continuation awaiting an answer; stable across processes.

Identifies a parked continuation awaiting an answer; stable across processes.

Attributes

Source
GraphIds.scala
opaque type JoinId

Stable identifier of a static or dynamic join barrier; persisted in snapshots.

Stable identifier of a static or dynamic join barrier; persisted in snapshots.

Attributes

Source
GraphIds.scala
opaque type NodeId

Stable identifier of a node in a compiled graph; persisted in snapshots.

Stable identifier of a node in a compiled graph; persisted in snapshots.

Attributes

Source
GraphIds.scala
opaque type Principal

Who a run acts for.

Who a run acts for.

Attributes

Source
GraphIds.scala
opaque type RunId

One execution attempt on a thread; start, recover and (later) resume each begin a new run.

One execution attempt on a thread; start, recover and (later) resume each begin a new run.

Attributes

Source
GraphIds.scala
opaque type StateCheck

A digest of what a Verifiable effect touches, as its target reads it: compared, never interpreted.

A digest of what a Verifiable effect touches, as its target reads it: compared, never interpreted.

Attributes

Source
EffectReplay.scala
opaque type StateKeyId

Stable identifier of a StateKey; persisted in snapshots.

Stable identifier of a StateKey; persisted in snapshots.

Attributes

Source
GraphIds.scala
opaque type TaskId

Identifier of one scheduled node execution; unique within a thread's history.

Identifier of one scheduled node execution; unique within a thread's history.

Attributes

Source
GraphIds.scala
opaque type TenantId

The tenant a thread belongs to.

The tenant a thread belongs to.

Attributes

Source
GraphIds.scala
opaque type ThreadId

A conversation or workflow thread: the address of its checkpoints and event log.

A conversation or workflow thread: the address of its checkpoints and event log.

Attributes

Source
GraphIds.scala
opaque type ToolCallId

The provider's identifier of one tool call; unique within a conversation.

The provider's identifier of one tool call; unique within a conversation.

Attributes

Source
GraphIds.scala
opaque type ToolName

The name of a tool, as the model calls it.

The name of a tool, as the model calls it.

Attributes

Source
GraphIds.scala