llm4s-docs
API
Generated with
llm4s-docs/org.llm4s/org.llm4s.vectorstore/org.llm4s.vectorstore.elasticsearch

org.llm4s.vectorstore.elasticsearch

package org.llm4s.vectorstore.elasticsearch

Members list

Type members

Classlikes

sealed trait ElasticsearchAuth

How ElasticsearchVectorStore authenticates to the cluster. Credentials are never printed by toString.

How ElasticsearchVectorStore authenticates to the cluster. Credentials are never printed by toString.

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
Known subtypes
class ApiKey
class Basic
object NoAuth
object ElasticsearchAuth

Attributes

Companion
trait
Supertypes
trait Sum
trait Mirror
class Object
trait Matchable
class Any
Self type
final case class ElasticsearchConfig

What ElasticsearchVectorStore needs to reach one index of an Elasticsearch cluster.

What ElasticsearchVectorStore needs to reach one index of an Elasticsearch cluster.

Load it from configuration with org.llm4s.config.ElasticsearchVectorStoreConfigLoader, which reads llm4s.vectorstore.elasticsearch, or build one in code:

val config = ElasticsearchConfig("https://es.example.com:9200")
 .withIndex("docs")
 .withAuth(ElasticsearchAuth.ApiKey(encodedKey))

The store checks it with ElasticsearchConfig#validate when it is built.

Value parameters

auth

the credentials, ElasticsearchAuth.NoAuth by default

baseUrl

the cluster's root URL, optionally with a path prefix (a proxy), without a query. It must be https when credentials are sent; plain http is accepted for a loopback host (localhost, 127.0.0.0/8, ::1) or when there are no credentials to protect.

dimensions

the vector size the index is created with; when unset, the size of the first record upserted. When set, a record of another size is refused before it is sent.

index

the index holding the vectors; Elasticsearch's index-name rules apply (lower case, no \ / * ? " < > | , # or space, not starting with -, _ or +)

numCandidates

kNN num_candidates, the candidates each shard considers; when unset, max(100, 10 * topK) capped at 10,000

refreshOnWrite

whether writes wait for a refresh (refresh=wait_for), so that a search straight after an upsert or delete sees it. Turn it off for bulk ingestion.

similarity

the dense_vector similarity the index is created with

timeout

how long each HTTP request may take

Attributes

Companion
object
Supertypes
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object ElasticsearchConfig

Attributes

Companion
class
Supertypes
trait Product
trait Mirror
class Object
trait Matchable
class Any
Self type
enum ElasticsearchSimilarity(val wireName: String)

The similarity an Elasticsearch dense_vector field is indexed with, fixed when the index is created.

The similarity an Elasticsearch dense_vector field is indexed with, fixed when the index is created.

Elasticsearch turns each similarity into a non-negative _score of its own: (1 + cosine) / 2 for ElasticsearchSimilarity.Cosine and, for unit-length vectors, (1 + dot) / 2 for ElasticsearchSimilarity.DotProduct, both in [0, 1]; 1 / (1 + l2_norm^2) for ElasticsearchSimilarity.L2Norm, in (0, 1]. ElasticsearchSimilarity.MaxInnerProduct can score above 1. A org.llm4s.vectorstore.ScoredRecord score must lie in [0, 1], so the store clamps it.

Value parameters

wireName

the value of the mapping's similarity parameter

Attributes

Companion
object
Supertypes
trait Enum
trait Serializable
trait Product
trait Equals
class Object
trait Matchable
class Any
Show all
object ElasticsearchSimilarity

Attributes

Companion
enum
Supertypes
trait Sum
trait Mirror
class Object
trait Matchable
class Any
Self type
final class ElasticsearchVectorStore extends VectorStore

A org.llm4s.vectorstore.VectorStore over one Elasticsearch index, through its REST API and org.llm4s.http.Llm4sHttpClient: no Elasticsearch client library is involved.

A org.llm4s.vectorstore.VectorStore over one Elasticsearch index, through its REST API and org.llm4s.http.Llm4sHttpClient: no Elasticsearch client library is involved.

The index is created on the first upsert if it does not exist, with a dense_vector field sized to ElasticsearchConfig#dimensions (or to the first record) and indexed for kNN with the configured ElasticsearchSimilarity. Writes go through _bulk; search is an approximate kNN search whose metadata filter is applied inside the kNN (so topK results come back when that many match); the other reads are plain queries. Each org.llm4s.vectorstore.MetadataFilter becomes a query DSL clause: Equals a term, In a terms, HasKey an exists, Contains a wildcard (*substring*), and And, Or and Not a bool query.

Reads of an index that does not exist yet answer as an empty store, and clear() deletes the index, which the next upsert creates again. Every operation returns a Result: a transport failure is the client's org.llm4s.error.NetworkError or org.llm4s.error.TimeoutError, a 401 or 403 an org.llm4s.error.AuthenticationError, a 429 a org.llm4s.error.RateLimitError, any other non-2xx status a org.llm4s.error.ServiceError carrying the status and the (redacted) body.

list pages with from and size, so offset + limit may not exceed the index's max_result_window (10,000 by default). Contains is a leading-wildcard query, refused by a cluster that sets search.allow_expensive_queries: false.

Attributes

Companion
object
Supertypes
trait VectorStore
class Object
trait Matchable
class Any
object ElasticsearchVectorStore

Attributes

Companion
class
Supertypes
class Object
trait Matchable
class Any
Self type
Generated with