VelesDB Benchmark Kit

Benchmark suite comparing VelesDB against pgvector (HNSW).

Benchmark Guardrails

• Microbench numbers are host-specific and must be reported with CPU, OS, Rust toolchain, and feature flags.
• Internal SIMD, sparse, and VelesQL cache measurements should be run with --features internal-bench.
• Do not claim superiority over FAISS, Qdrant, SimSIMD, or other systems unless the dataset, recall target, hardware, and methodology are matched.
• For the latest controlled-host remediation run, see benchmarks/results/2026-03-10-perf-remediation-report.md.

Benchmark Modes

1. Docker vs Docker (Fair comparison)

docker-compose up -d --build  # Start both servers
python benchmark_docker.py --vectors 5000 --clusters 25

Database	Mode	What it measures
VelesDB	REST API (Docker)	Client-server via HTTP
pgvector	Docker + PostgreSQL	Client-server via SQL

2. Native vs Docker (Embedded advantage)

python benchmark_recall.py --vectors 10000

Database	Mode	What it measures
VelesDB	Native Python (PyO3)	Best-case embedded performance
pgvector	Docker + PostgreSQL	Client-server with SQL overhead

Quick Start

# 1. Start both servers (Docker required)
docker-compose up -d --build

# 2. Install dependencies
pip install -r requirements.txt

# 3. Run fair Docker benchmark
python benchmark_docker.py --vectors 5000 --clusters 25

Options

# Both scripts support:
--vectors 5000     # Dataset size
--dim 768          # Vector dimension  
--queries 100      # Number of queries
--clusters 25      # Data clusters (realistic)

# Docker benchmark only:
--velesdb-url http://localhost:8080

Methodology

Fair Comparison

Both databases are measured with total time including index construction:

• VelesDB: Insert + inline HNSW indexing
• pgvector: Raw INSERT + separate CREATE INDEX time

This ensures an apples-to-apples comparison of the complete ingestion pipeline.

Controlled-host microbenchmarks

Use these commands for host-local remediation runs:

cargo bench -p velesdb-core --features internal-bench --bench simd_benchmark -- 768 --noplot
cargo bench -p velesdb-core --features internal-bench --bench sparse_benchmark -- sparse_insert --noplot
cargo bench -p velesdb-core --features internal-bench --bench velesql_benchmark -- velesql_cache --noplot

Report the exact host and toolchain alongside the measured values.

HNSW Parameters (Adaptive)

VelesDB uses adaptive parameters based on dataset size:

Dataset Size	M	ef_construction	Target Recall
≤10K	32	400	≥98%
≤100K	64	800	≥95%
≤500K	96	1200	≥95%
≤1M	128	1600	≥95%

// Automatic parameter selection
let params = HnswParams::for_dataset_size(768, 100_000);
// Or for 1M scale
let params = HnswParams::million_scale(768);

When to Choose Each

Use Case	Recommendation
Bulk import speed	VelesDB ✅ (3.2x faster)
Embedded/Desktop apps	VelesDB ✅
Real-time (<10ms)	VelesDB ✅
Edge/IoT/WASM	VelesDB ✅
Existing PostgreSQL	pgvector ✅
SQL ecosystem	pgvector ✅

License

This benchmark suite is part of the VelesDB repository and is covered by the VelesDB Core License 1.0, like the rest of the tree.