Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling

TL;DR

Four Over Six (4/6) is a small modification to the NVFP4 block-scaling algorithm: for each 16-value block, the encoder evaluates both a per-block scale of 6 (NVFP4’s default) and a per-block scale of 4, and picks whichever gives lower MSE against the original block. Because FP4 E2M1’s step size is largest near its max value (step 2 between 4 and 6), scaling to 4 better represents near-maximal values in some blocks at the cost of giving up the FP4 codes ±6. The selection adds <2% overhead at inference sequence lengths and <15% at training lengths on Blackwell, prevents divergence in NVFP4 transformer / hybrid / gated-attention pretraining where NVIDIA’s current state-of-the-art recipe diverges, and pairs cleanly with AWQ / SmoothQuant for PTQ. This is the MIT Han Lab / NVIDIA FP4-training paper whose follow-up (IF4) is already filed at Adaptive Block-Scaled Data Types (IF4 / IF3 / IF6).

Key claims

NVFP4 performance degradation is almost entirely attributable to FP4-value rounding error, not FP8 scale-factor error — keeping scale factors in BF16 barely moves the needle, but keeping FP4 values in BF16 fully recovers BF16 quality on Llama-3.1-8B [§2.2, Fig. 2(a)].
The error is concentrated on near-maximal values: ablating quantization above a threshold shows perplexity steadily degrades up to threshold ≈ 4 and then collapses sharply for values above 4, with smaller spikes at ≈2.5 and ≈3.5 — exactly the values where FP4’s step size jumps [§2.2, Fig. 2(b)].
Scaling a block to 4 instead of 6 maps 75% of the block max onto FP4 code 3 (vs 66.7% onto code 4 under scale-6), improving near-maximal representation at the cost of the FP4 codes ±6 and 50% of dynamic range [§3, Table 1, Table 2].
Scaling every block to 4 hurts: across Llama-3 and Qwen-3 at 1B/8B/70B/1.7B/8B/32B, all-4 underperforms all-6 NVFP4 on WikiText-2 / C4 perplexity. Adaptivity is necessary [§3.1, Table 3].
Three scale-selection rules were tested (smallest max error, smallest L1 norm of errors, smallest MSE); MSE wins in most cases and is adopted [§3.1, Table 4].
4/6 is implementable using Blackwell PTX cvt instructions with all quantized values, dequantized values, and per-element errors kept in the register file; overhead is under 2% at S≤2048 and under 15% at S=8192 [§3.2].
In NVFP4 pretraining of 340M and 1B models (transformer, hybrid w/ sliding-window attention, hybrid + Gated Attention), the current SOTA NVFP4 recipe diverges in several configurations; adding 4/6 prevents divergence and brings loss curves close to BF16 [§4.1, Fig. 4].
4/6 is recipe-agnostic for PTQ — drops into RTN, AWQ, and SmoothQuant. AWQ+4/6 improves all 6 Llama-3 / Qwen-3 models tested on WikiText-2 word perplexity and brings it 19.9% closer to BF16 on average; SmoothQuant+4/6 gives a smaller 5.3% improvement [§4.2, Table 5].
GPTQ + 4/6 hurts — increases the NVFP4↔BF16 gap by 34.6% on average — because GPTQ’s optimization process is unaware of the 4/6 branching; the authors flag this for future work [§4.2, Table 5].
4/6 cannot be ported to MXFP4: MXFP4 scale factors are E8M0 (each value 2× the previous), so the 50% larger scale factor a max-4 block needs is unrepresentable. NVFP4’s E4M3 scale factor is what makes 4/6 viable [§5.2].
Empirically, NVFP4+4/6 beats INT4 in most real-world settings despite INT4 having more uniform quantization error, because outliers (which NVFP4 represents almost exactly thanks to per-16 scale factors) dominate the practical quality budget [§5.1].
Rotation-based methods (QuaRot, SpinQuant) perform poorly under NVFP4 even with 4/6 — flagged as an open problem [§5.2].

Method

The encoder quantizes each 16-value block twice — once with the standard scale of 6 (amax/6), once with a scale of 4 (amax/4) — yielding quantized FP4 representations and their dequantized reconstructions. For each block the encoder computes the MSE between the original values and each reconstruction and picks the lower-MSE branch; the selected scale factor goes into the per-block FP8 E4M3 slot. The mathematical structure is identical to standard NVFP4 (Eqs. 1–3 in the paper) except the divisor in the block-level scale is per-block-adaptive rather than the constant 6.

On NVIDIA Blackwell, the kernel uses the cvt family of PTX instructions to quantize each block to packed FP4 with both scale factors in parallel and to dequantize FP4→FP16 for error computation, keeping all intermediate values in registers. Both candidate quantizations write to the same packed FP4 layout, so the only added state is per-block MSE bookkeeping. The training recipe inherits the rest of NVIDIA’s NVFP4 pretraining stack: stochastic rounding on gradients, random Hadamard transform on both inputs to the weight-gradient calculation, 2D block quantization on weight matrices, and BF16 forward+backward on the last 4 layers. All NVFP4 matmuls accumulate in FP32 and output in BF16; model weights are stored in FP32. For PTQ, 4/6 substitutes the inner FP4 linear layer, leaving the outer optimization pass (AWQ outlier smoothing, SmoothQuant clipping, GPTQ second-order updates) untouched.

Results

Pretraining divergence is fixed. On 340M transformer, 340M hybrid (dense + sliding-window attention), 1B transformer, 1B hybrid, and 1B hybrid + Gated Attention, NVIDIA’s SOTA recipe diverges or destabilizes in several configurations; +4/6 trains normally with loss curves close to BF16 [§4.1, Fig. 4]. Training used 4–8 B200/GB200 GPUs, FineWeb-Edu, AdamW, constant LR 3e-4, FSDP.
PTQ WikiText-2 word perplexity (lower is better) — BF16 / RTN+4/6 / GPTQ+4/6 / AWQ+4/6 / SmoothQuant+4/6:
- Llama-3.1-1B: 11.98 / 13.84 / 13.67 / 13.67 / 14.03 [Table 5].
- Llama-3.1-8B: 7.54 / 8.30 / 8.30 / 8.24 / 8.32 [Table 5].
- Llama-3.1-70B: 2.86 / 3.83 / — / 3.71 / 3.80 [Table 5].
- Qwen3-1.7B: 21.06 / 23.60 / 22.70 / 21.67 / 21.97 [Table 5].
- Qwen3-8B: 12.22 / 12.56 / 12.65 / 12.57 / 12.62 [Table 5].
- Qwen3-32B: 9.34 / 9.84 / 9.66 / 9.64 / 9.63 [Table 5].
- AWQ+4/6 averages 11.58 WikiText-2 / 32.36 C4 across all six models — the best NVFP4 PTQ combination tested.
Downstream task accuracy (BoolQ / ARC-E / ARC-C / HellaSwag averaged) for Qwen3-1.7B: BF16 62.8 → RTN 56.6 → RTN+4/6 59.8 → AWQ 56.2 → AWQ+4/6 59.2 [Table 7]. The +4/6 row consistently improves the average in nearly all cases across both Llama-3 and Qwen-3.
Quantization-error vignette [Table 1, Table 2]: for the block [10, 20, 30, 40], scale-6 gives MSE 4.33; scale-4 gives MSE 0. For [15, 30, 120, 180], scale-6 gives MSE 0; scale-4 gives MSE 96.25. The MSE rule picks correctly in both directions.
Kernel overhead [§3.2]: <2% at S≤2048, <15% at S=8192.

Why it’s interesting

This is the original “Four Over Six” paper from the same Cook / Han / NVIDIA group whose follow-up Adaptive Block-Scaled Data Types (IF4 / IF3 / IF6) is already filed — the latter explicitly enumerates 4/6’s two structural costs (wasted ±6 codes, 42.9% loss of tensor dynamic range) and replaces the “scale to 4” branch with a scaled-INT4 representation that preserves the full [0, 6] interval, encoded into the unused sign bit of NVFP4’s E4M3 scale factor. Filing both pins down the design trajectory: 4/6 is the algorithm-only intervention that needs no format change, IF4 is the format change that obviates 4/6. Anyone choosing between them is reading both.

This complements Attn-QAT: 4-Bit Attention With Quantization-Aware Training which makes the same end-to-end-FP4 push but from the attention side — Attn-QAT identifies that FlashAttention’s backward identity breaks under fake-quantized P/V and fixes it with a high-precision auxiliary output, while 4/6 identifies that NVFP4’s failure concentrates on near-max values and fixes it with adaptive block scaling. Both are training-time fixes that pair with their own B200 kernels. Together with FlashAttention-4: Algorithm and Kernel Pipelining Co-Design for Asymmetric Hardware Scaling, they are the early-2026 FP4-training-and-serving toolkit. The 4/6 finding also connects directly to A Unified View of Attention and Residual Sinks: Outlier-Driven Rescaling is Essential for Transformer Training‘s prediction that training-stability fixes should be evaluated against post-Hadamard activation distributions — the divergence-prevention behavior of 4/6 in the Gated-Attention 1B model is the data point that supports the architecture↔quantization coupling claimed there.