org.llm4s.rag.loader.UrlLoader
See theUrlLoader companion object
final case class UrlLoader (urls : Seq [String ], headers : Map [String , String ], timeoutMs : Int , metadata : Map [String , String ], retryCount : Int ) extends DocumentLoader
Load documents from URLs.
Supports HTTP/HTTPS URLs with configurable timeouts and headers. Includes ETag-based version detection for efficient sync operations.
Value parameters
headers
HTTP headers to send with requests
metadata
Additional metadata to attach
retryCount
Number of retry attempts for failed requests
timeoutMs
Connection and read timeout in milliseconds
urls
URLs to load
Attributes
Companion
object
Graph
Reset zoom Hide graph Show graph
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
Members list
Human-readable description of this loader.
Human-readable description of this loader.
Used for logging and debugging.
Attributes
Estimated number of documents (if known).
Estimated number of documents (if known).
Used for progress reporting and resource allocation. Returns None if count is unknown or expensive to compute.
Attributes
Definition Classes
Load documents from this source.
Load documents from this source.
Returns an iterator of LoadResult for streaming large document sets. Each result is either a successfully loaded document or a loading error. This allows processing to continue even when some documents fail.
Attributes
Returns
Iterator of load results (successes and failures)
Combine this loader with another.
Combine this loader with another.
Creates a composite loader that loads from both sources.
Attributes
Inherited from:
DocumentLoader
Attributes
Inherited from:
Product
Attributes
Inherited from:
Product