BenchmarkSuite

The names of the product elements

The name of the type

Suite comparing different chunking strategies.

Tests: Simple, Sentence, Markdown chunkers with default parameters. Semantic chunking is excluded as it requires embedding client setup.

Comprehensive suite testing all major dimensions.

Combines chunking, fusion, and basic embedding comparisons. Use for thorough evaluation when time permits.

Create a custom suite from configurations.

Suite comparing different embedding providers.

Tests: OpenAI (small/large), Voyage, Ollama embeddings. Note: Requires API keys for cloud providers and local Ollama for local models.

Suite comparing different fusion strategies.

Tests: RRF with different k values, weighted scoring, and single-source modes.

Quick test suite with minimal experiments for fast validation.

Tests just two configurations with a small sample subset.