SyncableSource

org.llm4s.rag.loader.SyncableSource

DocumentSource that supports change detection for incremental sync.

SyncableSource extends DocumentSource with version information, enabling RAG.sync() to detect which documents have changed.

Attributes

Get version information for change detection.

This should return quickly without reading the full document content. For S3, use the ETag; for filesystems, use content hash + mtime.

Estimated number of documents in this source, if known.

Used for progress reporting. Return None if unknown or expensive to compute.

Read document content as a stream.

Use this for large documents to avoid loading everything into memory. The caller is responsible for closing the returned stream.

Default implementation wraps readDocument; override for true streaming.

Human-readable description of this source.

Used for logging and debugging (e.g., "S3(my-bucket/docs/)")

List all document references in this source.

Returns an iterator for streaming large document sets. Each element is either a successful DocumentRef or an error.

Read document content into memory.

In this article

Generated with