turbovec: Rust vector index cuts RAM to 4 GB

A 10 million document corpus that needs 31 GB as float32 can now fit in 4 GB. RyanCodrai’s turbovec aims to do that with a Rust vector index and Python bindings.

turbovec is a Rust vector index with Python bindings that compresses large corpora and speeds up search.

What changed

turbovec packages Google Research’s TurboQuant algorithm into a local vector index written in Rust, with a Python API on top. The project is built for online ingest, incremental saves, filtered search, and offline use.

The repo says vectors can be added without a training pass, parameter tuning, or rebuilds as the corpus grows. It also adds stable external IDs through IdMapIndex, plus write/load snapshots and a sync path that persists only changed data.

• Rust core with Python bindings
• TurboQuant-based compression and search
• Incremental sync(path) for crash-safe saves
• Allowlist filtering inside the search kernel
• Local-only deployment for air-gapped RAG stacks

Benchmarks in the repo compare FAISS IndexPQFastScan against turbovec on 100K vectors. The project reports faster search on both ARM and x86, with average gains of 3.4x to 3.5x at 4-bit and about 20% to 26% at 2-bit, depending on architecture and test cell.

Why it matters

For developers building RAG or similarity search systems, the pitch is simple: less memory, no training step, and faster local retrieval. That matters when embeddings must stay on-device, inside a VPC, or under tight latency budgets.

The Python API also lowers the barrier for teams already using LangChain, LlamaIndex, Haystack, or Agno. turbovec ships drop-in replacements for in-tree memory stores, so teams can swap the backend without rewriting the rest of the pipeline.

The bigger signal is operational. Incremental sync and allowlist filtering are the kinds of details that decide whether a vector index is convenient in a demo or usable in production.

The open question is not whether compression works, but how well this approach holds up once indexes get larger, filters get more selective, and workloads move beyond the repo’s benchmark set.