llm4s-core
API
Generated with
llm4s-core/org.llm4s/org.llm4s.llmconnect/org.llm4s.llmconnect.middleware/CachingMiddleware

CachingMiddleware

org.llm4s.llmconnect.middleware.CachingMiddleware
class CachingMiddleware(embeddingClient: EmbeddingClient, embeddingModel: EmbeddingModelConfig, config: CacheConfig, tracing: Tracing, clock: Clock) extends LLMMiddleware

Middleware adapter for CachingLLMClient.

Allows using semantic caching as part of the middleware pipeline.

Attributes

Graph
Supertypes
class Object
trait Matchable
class Any

Members list

Value members

Concrete methods

override def name: String

Human-readable name for logging/debugging.

Human-readable name for logging/debugging.

Attributes

Definition Classes
override def wrap(next: LLMClient): LLMClient

Wrap the given LLMClient, returning a new client with added behavior.

Wrap the given LLMClient, returning a new client with added behavior.

Implementations should delegate all LLMClient methods to next, adding behavior before/after delegation as needed.

Value parameters

next

the next client in the pipeline

Attributes

Returns

a new LLMClient with this middleware's behavior added

Definition Classes
Generated with