InMemoryRunQueue

org.llm4s.agent.service.InMemoryRunQueue
@Experimental
final class InMemoryRunQueue extends RunQueue

The RunQueue of an InMemoryCheckpointer (InMemoryCheckpointer.runQueue): entries in memory, under the checkpointer's own lock, leases judged and closes stamped by its clock. It shares the checkpointer's lock so the checkpointer's deleteThread marks its open entries, and its prune reads their runs, in one step. awaitClosed wakes on closes rather than polling. Its calls take only each other's brief turn at the lock, so each returns at once.

Attributes

Graph
Supertypes
trait RunQueue
class Object
trait Matchable
class Any

Members list

Value members

Concrete methods

override def awaitClosed(tenantId: Option[TenantId], entryId: EntryId, timeout: FiniteDuration): Result[QueueEntry]

Waits on the queue's own closes rather than polling; see RunQueue.awaitClosed.

Waits on the queue's own closes rather than polling; see RunQueue.awaitClosed.

Attributes

Definition Classes
def close(lease: Lease, outcome: EntryOutcome): Result[QueueEntry]

Closes lease's entry with outcome and answers it closed; refused unless lease is current.

Closes lease's entry with outcome and answers it closed; refused unless lease is current.

Attributes

def entry(tenantId: Option[TenantId], entryId: EntryId): Result[QueueEntry]

Entry entryId, if it is tenantId's: QueueError.TicketExpired for an id the store issued and no longer holds for the tenant (reclaimed, or another tenant's), QueueError.TicketNotFound for one it never issued.

Entry entryId, if it is tenantId's: QueueError.TicketExpired for an id the store issued and no longer holds for the tenant (reclaimed, or another tenant's), QueueError.TicketNotFound for one it never issued.

Attributes

def openEntries(tenantId: Option[TenantId], threadId: ThreadId): Result[Vector[QueueEntry]]

tenantId's open entries on threadId, in the order they run (see the class doc); empty for an unknown thread.

tenantId's open entries on threadId, in the order they run (see the class doc); empty for an unknown thread.

Attributes

def reclaim(ticketLifetime: FiniteDuration, limit: Int): Result[Int]

Removes up to limit closed entries whose close is ticketLifetime or more ago, by the store's clock, oldest first, input and attempts included, and answers how many it removed; never an open entry. The store-wide retention sweep (W2) calls it.

Removes up to limit closed entries whose close is ticketLifetime or more ago, by the store's clock, oldest first, input and attempts included, and answers how many it removed; never an open entry. The store-wide retention sweep (W2) calls it.

Attributes

def recordAttempt(lease: Lease, runId: RunId): Result[AttemptStart]

Records an attempt at lease's entry running as runId, durably, before the caller touches the thread (rule 4); see AttemptStart. Refused unless lease is current, and with a ValidationError for a runId that already names a run on the thread - a checkpoint, or an attempt of any entry - since a reused RunId would name two runs and the run-id lookup would take the other for this attempt's. A worker gives each attempt RunId.random().

Records an attempt at lease's entry running as runId, durably, before the caller touches the thread (rule 4); see AttemptStart. Refused unless lease is current, and with a ValidationError for a runId that already names a run on the thread - a checkpoint, or an attempt of any entry - since a reused RunId would name two runs and the run-id lookup would take the other for this attempt's. A worker gives each attempt RunId.random().

Attributes

def release(lease: Lease): Result[Unit]

Gives the entry back without closing it - as a worker that defers to a live run on the thread does, recording no attempt - so it may be taken again at once. A lease that is not the current one changes nothing; a closed entry is QueueError.EntryClosed, so the worker knows it is terminal rather than requeued.

Gives the entry back without closing it - as a worker that defers to a live run on the thread does, recording no attempt - so it may be taken again at once. A lease that is not the current one changes nothing; a closed entry is QueueError.EntryClosed, so the worker knows it is terminal rather than requeued.

Attributes

def renew(lease: Lease, ttl: FiniteDuration): Result[Lease]

Extends lease to ttl from now, by the store's clock; refused unless it is the entry's current lease.

Extends lease to ttl from now, by the store's clock; refused unless it is the entry's current lease.

Attributes

def submit(submission: Submission): Result[Ticket]

Queues submission, or finds the entry its key already names, and answers its Ticket. Refusals: GraphError.ThreadBusy for a Reject submission to a thread with an open entry, and QueueError.SubmissionKeyReused.

Queues submission, or finds the entry its key already names, and answers its Ticket. Refusals: GraphError.ThreadBusy for a Reject submission to a thread with an open entry, and QueueError.SubmissionKeyReused.

Attributes

def take(request: TakeRequest): Result[Option[LeasedEntry]]

Leases the oldest takeable head - see the class doc - whose agent is one of request.agents, to request.holder for request.ttl from now, by the store's clock; None if there is none.

Leases the oldest takeable head - see the class doc - whose agent is one of request.agents, to request.holder for request.ttl from now, by the store's clock; None if there is none.

Attributes