Configuration for document chunking.
An invalid combination (a non-positive size, a maxSize below targetSize, an overlap that is negative or not smaller than targetSize) is a programming error: the constructor throws IllegalArgumentException, as RAGConfig.withChunking is a chainable builder and cannot return a Left. Code that chunks with sizes from user input should call ChunkingUtils.chunkTextValidated, which reports them as a Left (#1318, item 4).
Value parameters
- maxSize
-
Maximum chunk size: a longer sentence or section is force-split at whitespace or line breaks. A single word longer than
maxSizeis kept whole, so its chunk can exceed it; SimpleChunker usestargetSizeonly. Sizes are UTF-16 units, and no chunker cuts inside a surrogate pair (#1711). - minChunkSize
-
Minimum size for a chunk (smaller chunks are merged)
- overlap
-
Characters to overlap between chunks
- preserveCodeBlocks
-
Keep code blocks intact if possible
- preserveHeadings
-
Include heading context in metadata
- targetSize
-
Target chunk size in characters (soft limit)
Attributes
- Companion
- object
- Graph
-
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass Any