org.llm4s.rag.benchmark.BenchmarkSuite
See theBenchmarkSuite companion object
final case class BenchmarkSuite(name: String, description: String, experiments: Seq[RAGExperimentConfig], datasetPath: String, subsetSize: Option[Int], seed: Long)
A suite of benchmark experiments to run together.
Groups related experiments for systematic comparison. Provides pre-built suites for common comparison scenarios.
Value parameters
- datasetPath
-
Path to the evaluation dataset JSON file
- description
-
What this suite tests
- experiments
-
The experiments in this suite
- name
-
Suite identifier
- seed
-
Random seed for reproducible sample selection
- subsetSize
-
Optional limit on samples to evaluate (for quick tests)
Attributes
- Example
-
val suite = BenchmarkSuite.chunkingSuite("data/datasets/ragbench/test.json") val results = runner.runSuite(suite) - Companion
- object
- Graph
-
- Supertypes
-
trait Serializabletrait Producttrait Equalsclass Objecttrait Matchableclass Any
Members list
In this article