Revisiting RaBitQ and TurboQuant: A Symmetric Comparison of Methods, Theory, and Experiments

TL;DR

A technical-note rebuttal by the RaBitQ authors (Gao et al., ETH/NTU) to the ICLR 2026 TurboQuant paper from Google Research. Places both methods in a single comparison framework and argues three things: (i) at the method level, TurboQuant and RaBitQ share the same critical primitive — a random JL rotation followed by per-coordinate scalar quantization — which TurboQuant does not acknowledge; (ii) at the theory level, TurboQuant’s only bit-width guarantee is on variance, which via Chebyshev gives a 1/δ² dependence on failure probability — exponentially worse than RaBitQ’s sub-Gaussian tail (matching the Alon-Klartag 2017 optimal log(1/δ) lower bound); (iii) at the experimental level, under a reproducible symmetric setup TurboQuant performs worse than RaBitQ on most inner-product, ANN, and KV-cache settings, and the speed numbers in the original TurboQuant paper cannot be reproduced from the released implementation. The TurboQuant authors privately confirmed that their RaBitQ baseline ran single-threaded on CPU while TurboQuant ran on A100 GPU; this was not disclosed in the paper.

Key claims

TurboQuant and RaBitQ share the same preprocessing — sample one random orthogonal matrix, rotate every input vector with it, then quantize each coordinate scalar-wise — and both motivate the procedure as a Johnson-Lindenstrauss transform that makes the post-rotation coordinate distribution data-independent [§2.2.1, Table 1].
The methodological difference is the codebook: RaBitQ uses a uniform shifted-integer grid plus a per-vector rescale factor chosen to maximize cosine similarity; TurboQuant uses a non-uniform k-means codebook constructed to match the post-rotation 1D marginal (the Lloyd-Max condition on the beta distribution that arises from sphere coordinates) [§2.2.2, §2.2.4, Table 1].
RaBitQ’s inner-product estimator works directly on the bit code via native unsigned-integer arithmetic and supports incremental refinement (top bit gives a coarse estimate, lower bits refine); TurboQuant’s estimator requires a codebook lookup at decode time plus a QJL-based residual correction stage [§2.2.3, §2.2.5, Table 1].
RaBitQ’s bit-width to achieve additive error ε with failure probability δ scales as 1/ε² · log(1/δ), which matches the optimal Alon-Klartag (FOCS 2017) lower bound — Theorem 3.2 of Gao et al. 2025 gives the sub-Gaussian tail directly [§3, citing Alon-Klartag 2017].
TurboQuant’s published guarantee is a bound on the variance of its estimator (Theorem 2 of Zandieh et al. 2026). Converting that to a tail bound via Chebyshev forces the required bit-width to scale as 1/ε² · 1/δ², an exponentially worse dependence on the failure probability than RaBitQ’s log(1/δ) [§3, Lemma 3.1].
The TurboQuant paper’s RaBitQ baseline was run on a single-core CPU with multithreading disabled, via a Python reimplementation, while TurboQuant itself ran on an A100; this asymmetric setup was not disclosed in the paper but was confirmed in private email correspondence with the TurboQuant authors [§4.2, footnote 7].
When both methods run on the same A100 GPU, RaBitQ (GPU implementation) is approximately 1.5×–7× faster than TurboQuant at the d ∈ {200, 1536, 3072} dimensions reported in the original paper; with the faster_quant flag and Fast Walsh–Hadamard rotation enabled, RaBitQ (GPU⁺) is roughly 20× faster [§4.2, Table 2].
Even the CPU C++ implementation of RaBitQ achieves quantization times within the same order of magnitude as TurboQuant on an A100 [§4.2, Table 2].
The TurboQuant paper’s reported TurboQuant quantization times themselves cannot be reproduced from the released artifact: the authors measured up to two orders of magnitude slower wall-clock from the official implementation on the same A100 hardware [§4.2].
On inner-product estimation with the DBpedia Entities (1536-d) dataset, RaBitQ_prod produces tighter error distributions (lower std and max error) than TurboQuant_prod across most tested bit widths > 1; both methods are unbiased on average for the inner-product-oriented variants [§4.1, Figures 1–2].
Headline result, summarized from §1: “TurboQuant performs worse than RaBitQ in most tested settings of inner-product estimation, nearest-neighbor search and KV cache quantization” [§1, abstract].
The TurboQuant paper’s released implementation does not include optimized inner-product estimation or nearest-neighbor search routines — only a Python reference for IP estimation — making a fair end-to-end serving comparison impossible against RaBitQ’s FastScan / SIMD / GPU stack [§4 introductory paragraph].

Method

The note is structured as three parallel rebuttals to the TurboQuant ICLR 2026 paper. (1) Methodology. Both algorithms are written under one notation: random rotation → per-coordinate quantization → stored norm. Table 1 lays the steps side-by-side, showing that the only meaningful method-level differences are RaBitQ’s uniform shifted-integer grid + per-vector rescale (vs TurboQuant’s Lloyd-Max k-means codebook) and RaBitQ’s lookup-free integer-arithmetic estimator (vs TurboQuant’s codebook-lookup + QJL residual). (2) Theory. The authors restate the Alon-Klartag (2017) bit-width lower bound for additive-error inner-product sketches between unit vectors, then derive what each paper’s guarantee implies via standard concentration: RaBitQ has a sub-Gaussian tail directly giving log(1/δ); TurboQuant has only a variance bound, which Chebyshev converts to 1/δ². (3) Experiments. Reproductions on the same hardware reported by the TurboQuant paper (A100 GPU, dual-socket Xeon for CPU), using the official RaBitQ C++ library (with faster_quant disabled by default for like-for-like comparison) and the official TurboQuant PyTorch artifact from OpenReview. Datasets are the ones from the TurboQuant paper: DBpedia Entities for accuracy, GloVe-200 + DBpedia 1536/3072 for quantization speed, GloVe-100 + OpenAI3-{small,large} for ANN, and the original TurboQuant KV-cache benchmarks. All code is open-sourced at github.com/VectorDB-NTU/rabitq-turboquant-comparison for independent reruns.

Results

Quantization speed (4-bit, all dims, A100 GPU): RaBitQ (GPU) 0.009/0.065/0.152 s on d=200/1536/3072 vs TurboQuant 0.011/0.114/0.276 s — ≈1.2×/1.8×/1.8× faster. RaBitQ (GPU⁺, with FWHT) is 0.003/0.008/0.013 s — ≈3.7×/14×/21× faster [Table 2].
Quantization speed, CPU vs GPU cross-comparison: RaBitQ (CPU⁺ with FWHT) 0.085/0.143/0.218 s is within the same order of magnitude as TurboQuant on A100 — i.e. RaBitQ on CPU is roughly competitive with TurboQuant on GPU [Table 2].
Reproducibility of TurboQuant numbers: the authors’ measured TurboQuant times on the same A100 hardware are up to ≈100× slower than the numbers reported in the TurboQuant paper [§4.2].
Inner-product estimation accuracy (DBpedia 1536-d, 100k points × 1k queries): RaBitQ_prod has lower std and max error than TurboQuant_prod for bit-widths > 1; both are unbiased on average [§4.1].
ANN and KV-cache: “TurboQuant performs worse than RaBitQ in most tested settings” — the body extends through Figure 3 and the KV section (truncated in the fetched copy); at 2.5-bit KV quantization RaBitQ shows clear gains, and the two methods are comparable at 3.5-bit [§1, §4.3+].
Disclosure finding: the original TurboQuant paper’s RaBitQ baseline used a single-core CPU + disabled multithreading + Python reimplementation, while TurboQuant itself used an A100 GPU; this combination is not stated in the paper and was only confirmed via private email correspondence [§4.2, footnote 7 — direct quote from TurboQuant’s second author admitting the configuration].

Why it’s interesting

This is a rare public, technical-note-level rebuttal that affects how to read a paper already filed in the wiki — TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate. The TurboQuant page’s existing TL;DR and headline claims (6× memory reduction, 8× speedup, “beats Product Quantization and RaBitQ at smaller code sizes”) are exactly the numbers this note disputes; the conflict should be visible to anyone evaluating quantized KV-cache designs for serving. The interesting structural lesson is the same one Speculative Decoding: Performance or Illusion? makes in llm-inference-efficiency: published wall-clock comparisons in inference-efficiency papers are extremely sensitive to baseline implementation quality (single-threaded CPU vs multithreaded C++ vs GPU), and the “right” comparison framework — symmetric hardware, reproducible artifacts, optimized baselines on both sides — is the thing the field is missing more than it is missing new algorithms. The theory-vs-variance argument (Alon-Klartag log(1/δ) optimal vs Chebyshev 1/δ² suboptimal) is also a clean technical point worth knowing independent of the controversy: a variance bound on a quantization estimator is not the same caliber of guarantee as a sub-Gaussian tail, no matter how the abstract phrases it.