Skip to main content

vai benchmark

Run performance benchmarks for embeddings, reranking, asymmetric retrieval, quantization, cost, batch throughput, and end-to-end pipelines.

Synopsis​

vai benchmark <type> [options]

Description​

vai benchmark runs performance tests against the Voyage AI API using built-in sample data (no setup required). It measures latency, throughput, and quality across different models and configurations.

Benchmark Types​

TypeWhat It Measures
embedEmbedding latency across models
rerankReranking latency and score distribution
asymmetricCross-model similarity in the shared embedding space
quantizationQuality impact of int8/binary output types
costCost per 1M tokens across models
batchThroughput at different batch sizes
spaceStorage size at different dimensions
e2eEnd-to-end pipeline latency

Options​

FlagDescriptionDefault
<type>Benchmark type (required)—
--models <list>Comma-separated model listAll Voyage 4 models
--iterations <n>Iterations per measurementVaries by type
--jsonMachine-readable JSON output—
-q, --quietSuppress non-essential output—

Examples​

Benchmark embedding latency​

vai benchmark embed

Benchmark specific models​

vai benchmark embed --models voyage-4-large,voyage-4-lite

Asymmetric retrieval benchmark​

vai benchmark asymmetric

Cost comparison​

vai benchmark cost

JSON output for dashboards​

vai benchmark embed --json

Tips​

  • Benchmarks use built-in sample texts — no database or file setup required.
  • Run vai benchmark asymmetric to see how well different Voyage 4 models work together in the shared embedding space.
  • The quantization benchmark shows how int8/binary output types affect similarity scores compared to float.
  • vai eval — Evaluate retrieval quality (not just performance)
  • vai models — View model specs and pricing
  • vai estimate — Project costs for your workload