DirectoryLoader

org.llm4s.rag.loader.DirectoryLoader
See theDirectoryLoader companion object
final case class DirectoryLoader(path: Path, extensions: Set[String], recursive: Boolean, metadata: Map[String, String], maxDepth: Int) extends DocumentLoader

Load all documents from a directory.

Supports recursive directory traversal and file filtering by extension. Each file is loaded using FileLoader, inheriting its version and hint detection.

Value parameters

extensions

File extensions to include (without leading dot)

maxDepth

Maximum recursion depth (0 = current directory only)

metadata

Additional metadata to attach to all documents

path

Path to the directory

recursive

Whether to recurse into subdirectories

Attributes

Companion
object
Graph
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all

Members list

Value members

Concrete methods

def description: String

Human-readable description of this loader.

Human-readable description of this loader.

Used for logging and debugging.

Attributes

override def estimatedCount: Option[Int]

Estimated number of documents (if known).

Estimated number of documents (if known).

Used for progress reporting and resource allocation. Returns None if count is unknown or expensive to compute.

Attributes

Definition Classes
def load(): Iterator[LoadResult]

Load documents from this source.

Load documents from this source.

Returns an iterator of LoadResult for streaming large document sets. Each result is either a successfully loaded document or a loading error. This allows processing to continue even when some documents fail.

Attributes

Returns

Iterator of load results (successes and failures)

def withExtension(ext: String): DirectoryLoader

Add an extension to filter

Add an extension to filter

Attributes

def withExtensions(exts: Set[String]): DirectoryLoader

Set extensions (replaces existing)

Set extensions (replaces existing)

Attributes

def withMaxDepth(depth: Int): DirectoryLoader

Set maximum recursion depth

Set maximum recursion depth

Attributes

def withMetadata(meta: Map[String, String]): DirectoryLoader

Add metadata to all documents

Add metadata to all documents

Attributes

def withRecursive(r: Boolean): DirectoryLoader

Enable/disable recursive traversal

Enable/disable recursive traversal

Attributes

Inherited methods

Combine this loader with another.

Combine this loader with another.

Creates a composite loader that loads from both sources.

Attributes

Inherited from:
DocumentLoader
def productElementNames: Iterator[String]

Attributes

Inherited from:
Product
def productIterator: Iterator[Any]

Attributes

Inherited from:
Product