org.llm4s.agent.service
Members list
Type members
Classlikes
One attempt at an entry's run (Stage 5 rule 4): the RunId the attempt runs as, recorded durably before the attempt touches the thread, when the store recorded it, by its clock, and the lease holder that recorded it.
One attempt at an entry's run (Stage 5 rule 4): the RunId the attempt runs as, recorded durably before the attempt touches the thread, when the store recorded it, by its clock, and the lease holder that recorded it.
Attributes
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
What an attempt at a queue entry does to its thread (Stage 5 rule 4, design 4.55), decided after RunQueue.recordAttempt recorded the attempt's RunId and before the attempt touches the thread:
What an attempt at a queue entry does to its thread (Stage 5 rule 4, design 4.55), decided after RunQueue.recordAttempt recorded the attempt's RunId and before the attempt touches the thread:
Start: none of the entry's runs committed - its input was never admitted, an earlier attempt having died before its first commit - so the attempt admits it:run(or the typed agent's run) for a start,resumefor a resume.Recover: the attempt recovers the thread - its entry's admitted run is the thread's latest and incomplete (Running), so a worker died mid-run; or the entry is a queued recover, whose attempts recover the thread until one of them commits (the owner's explicit transfer of the thread's recovery to the queue).Close: the entry's input was admitted and the thread has moved on - its run reached rest, or the thread moved on through a directrecover, which applied the input once - so the attempt closes the entry withoutcome, running nothing. That includes an admitted entry (QueueEntry.admitted) whose checkpoints the lookup no longer finds: a direct run's blocked turn retracted them (Commit.retractTurn), so the input must not run again.
Attributes
- Companion
- object
- Supertypes
-
trait Enumtrait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Attributes
- Companion
- enum
- Supertypes
-
trait Sumtrait Mirrorclass Objecttrait Matchableclass Any
- Self type
-
AttemptPlan.type
What RunQueue.recordAttempt did (Stage 5 rule 4).
What RunQueue.recordAttempt did (Stage 5 rule 4).
Recorded: the attempt'sRunIdis recorded, the last ofentry.attempts; the attempt may now touch the thread, classifying it first (AttemptPlan).Exhausted: the entry has mademaxAttemptsattempts, so none is recorded. The worker settles the thread if the entry's input was admitted, then closes the entryFailed(AttemptsExhausted)- a dead letter.ThreadDeleted: the thread was deleted since the entry was submitted; the store closed the entryCancelled(ThreadDeleted)and recorded nothing, so nothing runs.
Attributes
- Supertypes
-
trait Enumtrait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
How a submission treats a thread that already has an open entry or a live run (design 4.55; W3 #2136 gives each its full behaviour). Reject, the default, refuses the submission with GraphError.ThreadBusy, as a direct run on a busy thread is refused (§5.3). Enqueue waits behind the thread's open entries, in submission order. Interrupt goes ahead of the thread's queued entries, which keep their order behind it; of two Interrupt entries the later goes first and the earlier is closed EntryOutcome.Interrupted before it runs (Stage 5 rule 5). Stopping the thread's running entry at its next safe point is W3's.
How a submission treats a thread that already has an open entry or a live run (design 4.55; W3 #2136 gives each its full behaviour). Reject, the default, refuses the submission with GraphError.ThreadBusy, as a direct run on a busy thread is refused (§5.3). Enqueue waits behind the thread's open entries, in submission order. Interrupt goes ahead of the thread's queued entries, which keep their order behind it; of two Interrupt entries the later goes first and the earlier is closed EntryOutcome.Interrupted before it runs (Stage 5 rule 5). Stopping the thread's running entry at its next safe point is W3's.
Attributes
- Supertypes
-
trait Enumtrait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Why an entry closed cancelled (design 4.55). ThreadDeleted: the thread was deleted after the entry was submitted, so the entry ran nothing (a deleted thread is never resurrected, rule 4). Requested: a cancel request stopped it (W3), with the requester's note.
Why an entry closed cancelled (design 4.55). ThreadDeleted: the thread was deleted after the entry was submitted, so the entry ran nothing (a deleted thread is never resurrected, rule 4). Requested: a cancel request stopped it (W3), with the requester's note.
Attributes
- Supertypes
-
trait Enumtrait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Attributes
- Supertypes
-
class Objecttrait Matchableclass Any
- Self type
-
EntryId.type
How an entry closed: its terminal state (design 4.55).
How an entry closed: its terminal state (design 4.55).
Done: the entry's run reached rest - completed or suspended - or the thread had moved on past the entry's admitted input (rule 4), at checkpointcheckpointIdwithstatus, if the thread had one.Failed: see FailureReason.Cancelled: see CancelReason.Interrupted: anInterruptentrybysuperseded it - before it ran, for a queuedInterruptentry, or at its next safe point (W3). It is not retried.
Attributes
- Companion
- object
- Supertypes
-
trait Enumtrait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Attributes
- Companion
- enum
- Supertypes
-
trait Sumtrait Mirrorclass Objecttrait Matchableclass Any
- Self type
-
EntryOutcome.type
Why an entry closed failed (design 4.55).
Why an entry closed failed (design 4.55).
AttemptsExhausted:attemptsattempts ran and none finished (a dead letter). If the input was admitted, the worker that found the attempts exhausted settled the thread first, so later entries start.RunFailed: the run ended in failure - a terminalFailedcheckpoint, or a refusal such asNotSuspendedfor a resume of a thread that is not waiting - with the error's message.Unrunnable: no worker could run the entry (no worker has its agent at a graph version that can restore its thread, W2), with why.
Attributes
- Supertypes
-
trait Enumtrait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
A page of a queued run's events (RunSubmitter.follow): events, those of the entry's attempts among the thread's events after the cursor the caller passed, ascending; next, the cursor to pass next (the last thread event read, whoever's it was); and entry as it was read after the events, so its attempts cover them. finished is true once the entry is closed and every event up to its close (QueueEntry.closedAtSeq) has been read: nothing more will come, however busy the thread stays with later runs.
A page of a queued run's events (RunSubmitter.follow): events, those of the entry's attempts among the thread's events after the cursor the caller passed, ascending; next, the cursor to pass next (the last thread event read, whoever's it was); and entry as it was read after the events, so its attempts cover them. finished is true once the entry is closed and every event up to its close (QueueEntry.closedAtSeq) has been read: nothing more will come, however busy the thread stays with later runs.
Attributes
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
The RunQueue of an InMemoryCheckpointer (InMemoryCheckpointer.runQueue): entries in memory, under the checkpointer's own lock, leases judged and closes stamped by its clock. It shares the checkpointer's lock so the checkpointer's deleteThread marks its open entries, and its prune reads their runs, in one step. awaitClosed wakes on closes rather than polling. Its calls take only each other's brief turn at the lock, so each returns at once.
The RunQueue of an InMemoryCheckpointer (InMemoryCheckpointer.runQueue): entries in memory, under the checkpointer's own lock, leases judged and closes stamped by its clock. It shares the checkpointer's lock so the checkpointer's deleteThread marks its open entries, and its prune reads their runs, in one step. awaitClosed wakes on closes rather than polling. Its calls take only each other's brief turn at the lock, so each returns at once.
Attributes
- Supertypes
A lease on an open entry, as RunQueue.take grants it: the worker holding it may record attempts at the entry, renew the lease and close the entry, for as long as no other lease replaced it - expiry, by the store's clock, only lets another worker take the entry over. token fences every call made under it.
A lease on an open entry, as RunQueue.take grants it: the worker holding it may record attempts at the entry, renew the lease and close the entry, for as long as no other lease replaced it - expiry, by the store's clock, only lets another worker take the entry over. token fences every call made under it.
Attributes
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
How a worker holds its queue entries' leases (design 4.55): each lease lasts ttl from when the store grants or renews it, by the store's clock, and is renewed every renewEvery - with the run's claim, W2 - so an entry whose worker died comes back to the queue at most ttl after its last renewal. Keep renewEvery at most a third of ttl, as the default does.
How a worker holds its queue entries' leases (design 4.55): each lease lasts ttl from when the store grants or renews it, by the store's clock, and is renewed every renewEvery - with the run's claim, W2 - so an entry whose worker died comes back to the queue at most ttl after its last renewal. Keep renewEvery at most a third of ttl, as the default does.
Both durations must be positive and renewEvery shorter than ttl: apply and the with* setters throw IllegalArgumentException otherwise, and LeasePolicy.of returns a ValidationError.
Attributes
- Companion
- object
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Attributes
- Companion
- class
- Supertypes
-
trait Producttrait Mirrorclass Objecttrait Matchableclass Any
- Self type
-
LeasePolicy.type
Attributes
- Supertypes
-
class Objecttrait Matchableclass Any
- Self type
-
LeaseToken.type
An entry RunQueue.take leased, with its lease.
An entry RunQueue.take leased, with its lease.
Attributes
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
A run queue entry, as a store reads it (design 4.55): one submitted run on a thread, durable, with the attempts made at it and, once closed, its EntryOutcome.
A run queue entry, as a store reads it (design 4.55): one submitted run on a thread, durable, with the attempts made at it and, once closed, its EntryOutcome.
An entry is open until it is closed, and only an open entry is ever taken or reclaimed-proof: a closed one is kept for the ticket lifetime after closedAt, then reclaimed. While open, position is how many of the thread's open entries run before it (0: it runs next, or is running), and leaseHolder and leaseExpiresAt say which worker holds it, if one does - a lease that has expired by the store's clock may be taken over.
Value parameters
- admitted
-
whether a run of one of
attemptshas committed a checkpoint on the thread - set by the store in that commit's own transaction, and kept when the checkpoint later leaves the history (a direct run'sCommit.retractTurnremoves it), so the input is known to be admitted even when the run-id lookup no longer finds it (AttemptPlan) - attempts
-
the attempts recorded, oldest first, each before it touched the thread
- closedAtSeq
-
the number of the thread's last event when the entry closed (
0for none): every event of the entry's runs is at or below it, so following the entry stops there (RunSubmitter.follow) - maxAttempts
-
the most attempts the entry may make, fixed at submission
Attributes
- Companion
- object
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Attributes
- Companion
- class
- Supertypes
-
trait Producttrait Mirrorclass Objecttrait Matchableclass Any
- Self type
-
QueueEntry.type
Errors a RunQueue answers with (design 4.55). Every case is non-recoverable: retrying the same call changes nothing.
Errors a RunQueue answers with (design 4.55). Every case is non-recoverable: retrying the same call changes nothing.
Attributes
- Companion
- object
- Supertypes
-
trait LLMErrortrait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
- Known subtypes
-
class EntryClosedclass LeaseLostclass SubmissionKeyReusedclass TicketExpiredclass TicketNotFound
Attributes
- Companion
- trait
- Supertypes
-
trait Sumtrait Mirrorclass Objecttrait Matchableclass Any
- Self type
-
QueueError.type
What a queued run does to its thread (design 4.55), in the JSON form the entry stores. The queue runs Agents and TypedAgent[I, O]s, the two entry points whose inputs have codecs; a raw CompiledGraph is not queued.
What a queued run does to its thread (design 4.55), in the JSON form the entry stores. The queue runs Agents and TypedAgent[I, O]s, the two entry points whose inputs have codecs; a raw CompiledGraph is not queued.
Start: a new turn withinput- an AgentInput for anAgent(QueuedInput.query), anIfor aTypedAgent[I, O](QueuedInput.typed).Resume: answers to the thread's pending interrupts, asAgent.resumetakes them.Recover: continue the thread's failed or interrupted run, asAgent.recoverdoes. A queued recover is the thread owner's explicit transfer of the thread's recovery to the queue: its first attempt recovers the thread whatever ran on it before (rule 4 otherwise forbids a worker from adopting a run started outside the queue).
The input is the caller's to store: the queue keeps it in its entry until the entry is reclaimed, but no metric, trace attribute or log line carries it (Stage 5 rule 10).
Attributes
- Companion
- object
- Supertypes
-
trait Enumtrait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Attributes
- Companion
- enum
- Supertypes
-
trait Sumtrait Mirrorclass Objecttrait Matchableclass Any
- Self type
-
QueuedInput.type
The durable run queue (Stage 5 W1, design 4.55): submitted runs, one entry each, that workers on any machine take, run and close. A store's queue lives in the same database as its checkpoints - InMemoryCheckpointer.runQueue, SqliteCheckpointer.runQueue, PostgresCheckpointer.runQueue - but is a separate SPI: no transaction spans an entry and a commit. Correctness rests on rule 4 instead - every attempt has its own RunId, recorded here before the attempt touches the thread, and an attempt classifies the thread by those ids (Checkpointer.latestOfRuns, AttemptPlan) - so an input is applied at most once however many attempts run.
The durable run queue (Stage 5 W1, design 4.55): submitted runs, one entry each, that workers on any machine take, run and close. A store's queue lives in the same database as its checkpoints - InMemoryCheckpointer.runQueue, SqliteCheckpointer.runQueue, PostgresCheckpointer.runQueue - but is a separate SPI: no transaction spans an entry and a commit. Correctness rests on rule 4 instead - every attempt has its own RunId, recorded here before the attempt touches the thread, and an attempt classifies the thread by those ids (Checkpointer.latestOfRuns, AttemptPlan) - so an input is applied at most once however many attempts run.
Implementations must be safe to call from several threads and processes, and must:
- '''submit''' an entry atomically: look up its SubmissionKey, if any, among the tenant's entries and insert only if there is none, so a retried submit gets the first one's ticket (Ticket.existing) and a different submission under a held key is refused (QueueError.SubmissionKeyReused); refuse a
Rejectsubmission to a thread with an open entry withGraphError.ThreadBusynaming no holder; close every queuedInterruptentry of the thread that has recorded no attemptInterrupted(by = the new entry)when anInterruptentry is submitted; and issue entry ids in submission order, never reused; - '''order''' each thread's open entries (rule 5): an entry that has recorded an attempt first, then
Interruptentries, then the rest, each by id; only a thread's first entry in that order - its head - is ever taken, and only while no other entry of the thread holds a live lease; - '''lease''' like a claim: take leases a head whose lease is absent or expired, by the store's clock, with a new LeaseToken greater than every one issued for the entry, and ends the expired lease of every other entry of the thread (a head an
Interruptentry displaced), so two workers never hold an actionable lease on one thread; a lease may be renewed, released, have attempts recorded under it and close its entry for as long as it is the entry's current one, expired or not, and every such call under another token is refused (QueueError.LeaseLost), as is any on a closed entry (QueueError.EntryClosed); taking is skip-locked where the store can, so workers do not contend for one entry; - '''record attempts''' durably before answering (recordAttempt): an attempt's
RunIdnaming no other run on the thread - no checkpoint of the thread's history and no attempt of any entry of the thread has it - so the run-id lookup finds only the attempt's own run; no attempt pastmaxAttempts(AttemptStart.Exhausted); and none once the thread was deleted while the entry was open, closing the entryCancelled(ThreadDeleted)instead (AttemptStart.ThreadDeleted); - '''learn of deletes''': the store's
Checkpointer.deleteThreadmarks every open entry of the thread in the delete's own transaction (or under its lock), whatever the deleted thread numbered, so no such entry runs on a new thread of the id; an entry submitted after the delete is not marked. The store'sCheckpointer.commitrefuses a checkpoint whose run is an attempt a marked entry recorded - while the entry is open and after it closes, until it is reclaimed - so a worker whose claim outlived its lease never writes the deleted thread's input; - '''learn of admissions''': the store's
Checkpointer.commitof a checkpoint whose run is an attempt of an open entry marks the entry QueueEntry.admitted in the commit's own transaction (or under its lock), and the mark stays when a later retraction removes the checkpoint, so AttemptPlan never starts an admitted input again; - '''keep the evidence''': while an entry is open, its store's
Checkpointer.prunekeeps the newest checkpoint each of the entry's recorded runs committed on its thread, solatestOfRunsnever loses an admitted run; - '''close''' an entry once, with its EntryOutcome, the store's time and the thread's last event number then (QueueEntry.closedAtSeq), and keep it for the ticket lifetime; reclaim removes closed entries older than that, oldest first, and never an open one;
- '''scope by tenant''' (rule 9): entry, awaitClosed and openEntries answer for the caller's tenant's entries only (
NoneandSomediffer), another tenant's being as unknown as a reclaimed one; - return from every call within a bound it documents, as
Checkpointerdoes.
RunQueueContract in llm4s-agent-testkit is the executable form of this contract.
Attributes
- Companion
- object
- Supertypes
-
class Objecttrait Matchableclass Any
- Known subtypes
-
class InMemoryRunQueue
The run queue's own settings (design 4.55), loaded from llm4s.agent.queue by RunQueueConfigLoader:
The run queue's own settings (design 4.55), loaded from llm4s.agent.queue by RunQueueConfigLoader:
maxAttempts: the most attempts an entry makes before it is closedFailed(AttemptsExhausted)(default RunQueueSettings.DefaultMaxAttempts). Fixed on each entry at submission, so every worker agrees on it.ticketLifetime: how long a closed entry - its input, attempts and outcome - is kept after it closed, by the store's clock, before the retention sweep reclaims it (default RunQueueSettings.DefaultTicketLifetime). Waiting on or following a reclaimed ticket answers QueueError.TicketExpired.lease: the LeasePolicy a worker leases entries with.
maxAttempts must be at least 1 and ticketLifetime positive: apply and the with* setters throw IllegalArgumentException otherwise, and RunQueueSettings.of returns a ValidationError.
Attributes
- Companion
- object
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Attributes
- Companion
- class
- Supertypes
-
trait Producttrait Mirrorclass Objecttrait Matchableclass Any
- Self type
-
RunQueueSettings.type
The submission API (design 4.55): queue a run for a worker on any machine to take, wait for its ticket's terminal state, or follow its events, over a store's RunQueue and the Checkpointer of the same store. Every call is tenant-scoped (rule 9): a ticket is answered only to its own tenant.
The submission API (design 4.55): queue a run for a worker on any machine to take, wait for its ticket's terminal state, or follow its events, over a store's RunQueue and the Checkpointer of the same store. Every call is tenant-scoped (rule 9): a ticket is answered only to its own tenant.
Entries are submitted with settings.maxAttempts; the worker side (W2) takes them.
Attributes
- Supertypes
-
class Objecttrait Matchableclass Any
A run to queue (design 4.55): input for thread threadId, run by the agent agentId for tenantId, treating a busy thread as policy says, with at most maxAttempts attempts (RunQueueSettings.maxAttempts when built by RunSubmitter). A submissionKey makes the submit idempotent: a second submit with the key of an entry the store still holds returns that entry's ticket, and a different submission under the key is refused (QueueError.SubmissionKeyReused).
A run to queue (design 4.55): input for thread threadId, run by the agent agentId for tenantId, treating a busy thread as policy says, with at most maxAttempts attempts (RunQueueSettings.maxAttempts when built by RunSubmitter). A submissionKey makes the submit idempotent: a second submit with the key of an entry the store still holds returns that entry's ticket, and a different submission under the key is refused (QueueError.SubmissionKeyReused).
apply and the with* setters throw IllegalArgumentException for maxAttempts below 1; Submission.of returns it as a ValidationError.
Attributes
- Companion
- object
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Attributes
- Companion
- class
- Supertypes
-
trait Producttrait Mirrorclass Objecttrait Matchableclass Any
- Self type
-
Submission.type
Attributes
- Supertypes
-
class Objecttrait Matchableclass Any
- Self type
-
SubmissionKey.type
What RunQueue.take may take: the head entry of a thread whose agent is one of agents, leased to holder for ttl from now, by the store's clock.
What RunQueue.take may take: the head entry of a thread whose agent is one of agents, leased to holder for ttl from now, by the store's clock.
Attributes
- Companion
- object
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Attributes
- Companion
- class
- Supertypes
-
trait Producttrait Mirrorclass Objecttrait Matchableclass Any
- Self type
-
TakeRequest.type
What RunQueue.submit answers (design 4.55): the entry entryId made for the submission on threadId, and its position among the thread's open entries when the store answered (0: it runs next). existing is true when a submission key named an entry the store already held, so this submit made none; its position is then the entry's current one, and None once that entry has closed.
What RunQueue.submit answers (design 4.55): the entry entryId made for the submission on threadId, and its position among the thread's open entries when the store answered (0: it runs next). existing is true when a submission key named an entry the store already held, so this submit made none; its position is then the entry's current one, and None once that entry has closed.
Attributes
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass AnyShow all
Types
A run queue entry's id (design 4.55): positive, issued by the store in submission order and never reused, so it is also the entry's place in the store-wide order of submissions. A Ticket names it; a caller keeps it to wait on or follow the entry's run, from any machine.
A run queue entry's id (design 4.55): positive, issued by the store in submission order and never reused, so it is also the entry's place in the store-wide order of submissions. A Ticket names it; a caller keeps it to wait on or follow the entry's run, from any machine.
Attributes
The fencing token of a lease on a queue entry (Lease): strictly greater than every lease token the store issued for that entry before, so a worker whose lease was taken over after it expired can neither renew it, record an attempt under it nor close the entry.
The fencing token of a lease on a queue entry (Lease): strictly greater than every lease token the store issued for that entry before, so a worker whose lease was taken over after it expired can neither renew it, record an attempt under it nor close the entry.
Attributes
A caller's key for one submission (design 4.55): unique per tenant, so a caller that retries a submit whose answer it lost - a dropped response, an ambiguous timeout - gets the ticket of the entry its first try made rather than a second entry. 1 to 200 characters. The key is held for as long as its entry is (the ticket lifetime after it closes); once the entry is reclaimed the key names nothing and may be used again.
A caller's key for one submission (design 4.55): unique per tenant, so a caller that retries a submit whose answer it lost - a dropped response, an ambiguous timeout - gets the ticket of the entry its first try made rather than a second entry. 1 to 200 characters. The key is held for as long as its entry is (the ticket lifetime after it closes); once the entry is reclaimed the key names nothing and may be used again.