org.llm4s.llmconnect.middleware
Members list
Type members
Classlikes
Middleware adapter for CachingLLMClient.
Middleware adapter for CachingLLMClient.
Allows using semantic caching as part of the middleware pipeline.
Attributes
- Supertypes
Strategy for redacting sensitive information from logs.
Strategy for redacting sensitive information from logs.
Attributes
- Companion
- object
- Supertypes
-
class Objecttrait Matchableclass Any
- Known subtypes
-
class RegexContentRedactor
Attributes
- Companion
- trait
- Supertypes
-
class Objecttrait Matchableclass Any
- Self type
-
ContentRedactor.type
Middleware that sanitizes and validates input prompts.
Middleware that sanitizes and validates input prompts.
Can enforce max length limits and reject inputs containing forbidden patterns.
Attributes
- Supertypes
Builder for composing middleware into a pipeline around a base LLMClient.
Builder for composing middleware into a pipeline around a base LLMClient.
Middleware is applied in FIFO order: the first middleware added is closest to the base client, the last added is the outermost wrapper.
Usage:
val client = LLMClientPipeline(openAIClient)
.use(LoggingMiddleware(logger)) // innermost
.use(MetricsMiddleware(collector))
.use(RateLimitingMiddleware(config)) // outermost
.build()
Attributes
- Companion
- object
- Supertypes
-
class Objecttrait Matchableclass Any
Attributes
- Companion
- class
- Supertypes
-
class Objecttrait Matchableclass Any
- Self type
-
LLMClientPipeline.type
A middleware that wraps an LLMClient to add cross-cutting behavior.
A middleware that wraps an LLMClient to add cross-cutting behavior.
Middleware instances are composable and applied in order via LLMClientPipeline. The last middleware added is the outermost wrapper (first to execute).
Attributes
- Supertypes
-
class Objecttrait Matchableclass Any
- Known subtypes
-
class CachingMiddlewareclass LoggingMiddlewareclass MetricsMiddlewareclass RateLimitingMiddlewareclass RequestIdMiddlewareShow all
Middleware that logs requests and responses.
Middleware that logs requests and responses.
Value parameters
- includeMessages
-
Whether to log full message content (careful with PII!)
- logger
-
SLF4J logger instance
- redactor
-
Optional redactor to mask sensitive data in logs
Attributes
- Supertypes
Middleware that collects metrics for LLM operations.
Middleware that collects metrics for LLM operations.
Replaces the provider-baked MetricsRecording trait with a composable layer. Records request duration, success/failure counts, token usage, and costs.
Attributes
- Supertypes
Base class for middleware implementations that wrap an underlying client. Provides default delegation for all LLMClient methods.
Base class for middleware implementations that wrap an underlying client. Provides default delegation for all LLMClient methods.
Most middleware only needs to override complete and streamComplete. Other methods (validate, close, configuration getters) are delegated automatically.
Attributes
- Supertypes
Middleware that enforces a local rate limit using a Token Bucket algorithm.
Middleware that enforces a local rate limit using a Token Bucket algorithm.
Prevents the application from overwhelming downstream providers or exceeding cost budgets.
Value parameters
- burstCapacity
-
Maximum burst size (default: same as RPM)
- requestsPerMinute
-
Maximum allowable requests per minute
Attributes
- Supertypes
Middleware that generates and propagates a unique Request ID for each LLM call.
Middleware that generates and propagates a unique Request ID for each LLM call.
Uses SLF4J MDC to make the request ID available to all loggers in the thread context. This is critical for correlating logs in distributed systems.
Attributes
- Supertypes