org.llm4s.llmconnect.caching.CacheConfig
See theCacheConfig companion object
Configuration for the CachingLLMClient semantic cache.
Uses the sealed-abstract-case-class pattern to prevent direct construction and disable the generated copy method; always construct via CacheConfig.create, which validates all fields and returns a typed error on invalid input.
Value parameters
- maxSize
-
Maximum number of entries in the in-memory cache. When the limit is reached the least-recently-used entry is evicted automatically.
- similarityThreshold
-
Minimum cosine similarity
[0.0, 1.0]for a cache hit. A value of1.0requires near-identical queries; lower values allow semantically similar but textually different queries to share a cached response. - ttl
-
Maximum age of a CacheEntry before it is considered expired and the cache is bypassed. Must be positive.
Attributes
- Companion
- object
- Graph
-
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass Any
Members list
In this article