Perf: Garnet Providers vs other Vector DBs (Zilliz, Pinecone, etc.)

The experiment inserts 1000 vector per sec in the background, and in the foreground measures Query throughput, p99 query latency and recall. Measurements were done with Zilliz's Vector DB Bench on Azure D32v6 VM on the Wikipedia-10M + Cohere Embeddings (768 D, float 32) dataset. Measurements for other vector DBs are taken from the benchmark website.

DiskANN3 + Garnet outperforms the rest of vector DBs by a large margin:

• Roughly 2x better QPS better than Zilliz (next best) and 5-10x better than the rest. Many Vector DBs cannot even catch up with 1000 vector inserts/second, let alone support
• Lowest and most stable p99 query latency (<5ms) despite high ingestion rate. Other entries have 10s or 100s of milliseconds latency that fluctuate
• Highest recall.
• Fastest ingestion overall. Ingestion would have been even faster if performed outside Vector DB bench.

The gap increases further at larger scale and higher dimensional embeddings.

Please see vectorset and diskann-garnet crates for code.