GroundTruthGenerator

org.llm4s.rag.benchmark.GroundTruthGenerator
See theGroundTruthGenerator companion object
class GroundTruthGenerator(llmClient: LLMClient, options: GeneratorOptions)

Generates ground truth evaluation datasets from documents using LLM.

Creates question-answer pairs with context that can be used for RAGAS evaluation. Supports multiple generation strategies for different testing scenarios.

Value parameters

llmClient

LLM client for generation

options

Generation options

Attributes

Example
val generator = GroundTruthGenerator(llmClient)
// Generate from documents
val dataset = generator.generateFromDocuments(
 documents = Seq(doc1, doc2, doc3),
 questionsPerDoc = 5,
 datasetName = "my-test-set"
)
// Save for later use
TestDataset.save(dataset, "data/generated/my-test-set.json")
Companion
object
Graph
Supertypes
class Object
trait Matchable
class Any

Members list

Value members

Concrete methods

def generateFromDirectory(dirPath: String, questionsPerDoc: Int, extensions: Set[String], datasetName: String): Result[TestDataset]

Generate from a directory of documents.

Generate from a directory of documents.

Value parameters

datasetName

Name for dataset

dirPath

Path to directory

extensions

File extensions to include

questionsPerDoc

QA pairs per document

Attributes

Returns

Generated TestDataset

def generateFromDocument(docId: String, content: String, count: Int): Result[Seq[EvalSample]]

Generate QA pairs from a single document.

Generate QA pairs from a single document.

Value parameters

content

Document content

count

Number of QA pairs to generate

docId

Document identifier

Attributes

Returns

Sequence of EvalSamples

def generateFromDocuments(documents: Seq[(String, String)], questionsPerDoc: Int, datasetName: String): Result[TestDataset]

Generate a test dataset from documents.

Generate a test dataset from documents.

Value parameters

datasetName

Name for the generated dataset

documents

Sequence of (id, content) pairs

questionsPerDoc

Number of QA pairs per document

Attributes

Returns

Generated TestDataset or error

def generateMultiHop(documents: Seq[(String, String)], count: Int): Result[Seq[EvalSample]]

Generate multi-hop questions that require information from multiple documents.

Generate multi-hop questions that require information from multiple documents.

Value parameters

count

Number of multi-hop questions to generate

documents

Documents to use as sources

Attributes

Returns

Sequence of EvalSamples with multiple contexts