org.llm4s.rag.evaluation.RAGASEvaluator
See theRAGASEvaluator companion object
class RAGASEvaluator(llmClient: LLMClient, embeddingClient: EmbeddingClient, embeddingModelConfig: EmbeddingModelConfig, metrics: Seq[RAGASMetric], options: EvaluatorOptions, tracer: Option[Tracing])
Main RAGAS evaluator that orchestrates all metrics.
RAGAS (Retrieval Augmented Generation Assessment) evaluates RAG pipelines across four dimensions:
- Faithfulness: Are claims in the answer supported by context?
- Answer Relevancy: Does the answer address the question?
- Context Precision: Are relevant docs ranked at the top?
- Context Recall: Were all relevant docs retrieved?
The composite RAGAS score is the mean of all evaluated metric scores.
Value parameters
- embeddingClient
-
Embedding client for similarity calculations
- embeddingModelConfig
-
Configuration for the embedding model
- llmClient
-
LLM client for semantic evaluation (claim verification, relevance)
- metrics
-
Custom metrics to use (defaults to all four RAGAS metrics)
- options
-
Evaluation options (parallelism, timeouts)
- tracer
-
Optional tracer for cost tracking
Attributes
- Example
-
{ val evaluator = RAGASEvaluator(llmClient, embeddingClient, embeddingConfig) val sample = EvalSample( question = "What is the capital of France?", answer = "Paris is the capital of France.", contexts = Seq("Paris is the capital and largest city of France."), groundTruth = Some("The capital of France is Paris.") ) val result = evaluator.evaluate(sample) result match { case Right(eval) => println(s"RAGAS Score: $${eval.ragasScore}") eval.metrics.foreach { m => println(s" $${m.metricName}: $${m.score}") } case Left(error) => println(s"Evaluation failed: $${error.message}") }}
- Companion
- object
- Graph
-
- Supertypes
-
class Objecttrait Matchableclass Any
Members list
In this article