org.llm4s.rag.loader.UrlLoader
See theUrlLoader companion object
final case class UrlLoader (urls : Seq [String ], headers : Map [String , String ], timeout : FiniteDuration , metadata : Map [String , String ], retryCount : Int ) extends DocumentLoader
Load documents from URLs.
Supports HTTP/HTTPS URLs with configurable timeouts and headers. Includes ETag-based version detection for efficient sync operations.
Value parameters
headers
HTTP headers to send with requests
metadata
Additional metadata to attach
retryCount
Number of retry attempts for failed requests
timeout
Connection and read timeout
urls
URLs to load
Attributes
Companion
object
Graph
Reset zoom Hide graph Show graph
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
Members list
Human-readable description of this loader.
Human-readable description of this loader.
Used for logging and debugging.
Attributes
Estimated number of documents (if known).
Estimated number of documents (if known).
Used for progress reporting and resource allocation. Returns None if count is unknown or expensive to compute.
Attributes
Definition Classes
Load documents from this source.
Load documents from this source.
Returns an iterator of LoadResult for streaming large document sets. Each result is either a successfully loaded document or a loading error. This allows processing to continue even when some documents fail.
Attributes
Returns
Iterator of load results (successes and failures)
The case class's form, with each credential-named header's value (Authorization, x-api-key, ...) masked.
The case class's form, with each credential-named header's value (Authorization, x-api-key, ...) masked.
Attributes
Definition Classes
Any
Combine this loader with another.
Combine this loader with another.
Creates a composite loader that loads from both sources.
Attributes
Inherited from:
DocumentLoader
Attributes
Inherited from:
Product
Attributes
Inherited from:
Product