CachePolicy

org.llm4s.agent.graph.CachePolicy
See theCachePolicy companion object
final case class CachePolicy

Declares that a node's result depends only on its input, so a repeated input may be answered from memory without running the node.

The node must not depend on ThreadState, on the clock, or on an external system: a node that needs part of the state takes that part as input. The key is the node, its input schema version and the encoded input. A hit returns the stored Command and the node does not run, so a cached node should not rely on RunContext.emit or progress. Only a Continue result is stored: a failure or a suspension is not.

The cache belongs to the CompiledGraph instance and is shared by its runs. It is not checkpointed, so another instance, another process or a restart starts empty and recomputes. Entries beyond maxEntries are evicted least-recently-used first. Two tasks with one input running at the same time both run the node.

Every value is validated like RunBudgets: apply and the with* setters throw IllegalArgumentException; CachePolicy.of returns a ValidationError.

Value parameters

maxEntries

the most results kept for the node

ttl

how long an entry is served after it was stored; None means until evicted

Attributes

Companion
object
Graph
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all

Members list

Value members

Concrete methods

def withTtl(t: FiniteDuration): CachePolicy
def withTtl(t: Option[FiniteDuration]): CachePolicy

Inherited methods

def productElementNames: Iterator[String]

Attributes

Inherited from:
Product
def productIterator: Iterator[Any]

Attributes

Inherited from:
Product