GPU-accelerated Weight Teleportation

TL;DR

humans& ai (with Miles / SGLang / NVIDIA) publish a Blackwell-native delta weight sync pipeline for frontier RL that reduces the sampler pause for a weight update to <0.23 s and end-to-end weight-sync time to <4 s on an Ampersand RL configuration (NVFP4 GLM-5.3, 445 GB, 64×B300 trainer). The design co-designs (a) distributed GPU compression of the per-step XOR mask across all trainer ranks with distributed ownership; (b) a double-compression scheme (GPU-friendly inner codec Snappy + outer Zstd on CPU) that gives both a small transfer payload (~0.6% of model size) and a receiver-side pipeline that keeps GPU HBM and compute free; (c) the Blackwell Decompression Engine (DE) consuming pinned CPU RAM directly into HBM, overlapped with the bitwise XOR application on SMs via double-buffering; and (d) a layer-at-a-time streaming design that needs no KV-cache eviction and no change to the inference engine’s memory pool. The pause becomes so short that the authors claim it beats intra-cluster P2P RDMA for most real workloads, and the same infrastructure doubles as inference deployment, fault-tolerance recovery, and a superior LoRA-adapter-swap alternative via one /update_weights_from_gpu_delta call.

Key claims

The XOR delta primitive is bit-level and codec-agnostic. The mask w_old ⊕ w_new has the same shape as the weights, but RL training signals are sparse, so the mask is mostly zero and compresses well under lossless codecs. Decompression is pointwise XOR application — no floating-point arithmetic, no rounding, works unchanged under low-precision (NVFP4) RL [§Delta update algebra].
The CPU-disk-delta baseline (Miles v0.1) is bottlenecked not by payload size but by three fixed costs: trainer compression latency, a disk round-trip for checkpoint patching, and a full checkpoint reload on the rollout side [§CPU disk delta weight sync, §Model checkpoint as a handoff].
Distributed trainer-side compression with ownership remapping amortizes compression across all existing training GPUs, cuts weight-gathering traffic (gather compressed payloads, not raw weights), and mitigates memory pressure from holding earlier weight versions. Each trainer rank owns a disjoint subset of the model for compression independent of its training-time sharding (which may duplicate under CP/DP) [§A strong baseline with Blackwell-native DE].
LZ4 is chosen as the single-codec GPU baseline over Snappy because LZ4’s longer copy-command encoding compresses long zero runs more compactly (the RL mask case), and is DE-supported. On the proxy workload trainer prep+compress is 0.834 s and receiver work is 6.041 s [§A strong baseline with Blackwell-native DE].
Two flaws remain in the single-codec baseline: (a) payload size — LZ4 produces 5.36 GB vs CPU-Zstd’s 0.876 GB, 6.1× more bytes; (b) HBM pressure — decompression input/output buffers for the full mask force KV-cache eviction, which also prevents overlapping rollout decoding with decompression [§The remaining trade-offs].
Double compression: GPU-friendly inner codec + Zstd outer wrapper. Snappy + Zstd compresses to 2.65 GB (13.4% of inner Snappy publication; 0.60% of model weights); LZ4 + Zstd less effective (75.2% of inner, 0.91% of model), hypothesized because Snappy’s simpler copy commands are easier for Zstd to re-compress [§Double compression with heterogeneous decompression, Table 2].
Heterogeneous decompression pipeline: CPU for outer Zstd, DE for inner codec. The outer Zstd stage runs on CPU during rollout decoding, staging the decompressed Snappy payload in pinned CPU RAM. DE then consumes pinned CPU RAM directly, outputting to GPU HBM via CU_MEM_LOCATION_TYPE_HOST_NUMA + CU_MEM_CREATE_USAGE_HW_DECOMPRESS allocations — this is not an ordinary pinned allocation and gives higher host-device bandwidth than a custom ring buffer [§Double compression with heterogeneous decompression; footnote on cuMemCreate].
DE-SM overlap via double buffering. DE is a fixed-function ASIC that runs concurrently with SM compute; a double-buffered pipeline streams one model layer through DE decompression while SMs apply bitwise XOR via a single Triton kernel launch over all remapped layer weights. Nsight Systems traces show DE pipeline saturation; Triton kernels hidden under DE setup/decompress loops [§Overlapping DE and SM].
No KV-cache eviction and no inference-engine memory-pool changes. The two layer-sized double buffers fit in the GPU memory already reserved for activations during inference, so the rollout engine’s KV cache stays resident across the sync [§Overlapping DE and SM].
Low-precision weights can absorb small high-precision updates (changes to low mantissa bits of a BF16 update land on quantized weight bits that may be stable), making the delta payload even smaller under NVFP4 training [§Working with low-precision RL].
Fused-kernel-aware weight layouts (e.g., gate-up weights paired in groups of 64 for the humans& NVFP4 decode fast path The 4-bitter Lesson: Balancing Stability and Performance in NVFP4 RL) break naive pointwise XOR; the fix is to remap the decompressed mask with the exact same physical layout so mask and weights realign bit-for-bit. Metadata is prepared on CPU; the Triton XOR kernel reads the prepared plan [§Working with low-precision RL].
End-to-end numbers (Ampersand NVFP4 GLM-5.3, 64×B300 trainer, 2×TP4-engine 8-GPU B300 rollout, 0.085% mutated weights proxy): trainer prep+compress 0.789 s (Snappy + Zstd, including extra Zstd pass, lower than LZ4 baseline 0.834 s); rollout pull + prep + outer-Zstd-decompress + stage 2.355 s without blocking inference; DE-SM streaming (the only inference-blocking stage) 219.6 ms [§Assembling the weight teleporter].
Auxiliary deltas for eval, fault tolerance, and resuming from trained checkpoints. The trainer maintains deltas against multiple base versions (last eval version, rollout/eval engine base version); because trainer-side ownership is distributed and sharded, per-CPU memory overhead stays small. Processing auxiliary deltas is overlapped with transmission/prep of the hot-path delta, so the <0.23 s pause is unchanged [§Eval, fault tolerance, and resuming from trained checkpoints, Table 3].
Larger accumulated deltas (e.g., resume-from-trained-checkpoint) switch to LZ4 + Zstd startup codec — larger final payload but smaller inner payload so the receiver staging buffer stays bounded; hot path continues to use Snappy + Zstd [§Eval, fault tolerance, and resuming from trained checkpoints].
Inference-deployment use case: ship compressed deltas across data centers, not full checkpoints, and let the inference engine pull via a single /update_weights_from_gpu_delta call starting from a base checkpoint. Opt-in release_state argument clears staging buffers / GPU kernels / CPU threads after apply for pure-inference deployments [§Inference deployment and beyond].
Positioned as a superior alternative to LoRA adapters for serving-side swap: full-parameter deltas compressed to ~0.6% of model size swap in <0.23 s; LoRA keeps payload small only by limiting rank [§Inference deployment and beyond].
The authors claim no further RL weight-sync work is needed: the decompression ASIC throughput is now the bottleneck, and even intra-cluster P2P RDMA is slower in most real workloads [§Conclusion].

Method

The pipeline is three cooperating components across trainer, transport, and rollout:

Trainer side. After one RL training step, each trainer GPU rank computes a bitwise XOR between its owned shards of w_old and w_new. Ownership for compression is remapped so that the full model is covered exactly once with no duplication (training-time CP/DP replication is collapsed). Each rank compresses its local mask with Snappy on-GPU, then wraps the result in a Zstd pass on the CPU; the per-shard compressed payloads are gathered into a publication. Memory pressure for holding w_old is distributed across all training ranks.

Transport. Compressed delta payloads are shipped (disk or network) to rollout engines. This is where disaggregated cross-DC deployment also plugs in: the same compressed publication is the inter-cluster artifact, replacing a multi-hundred-GB checkpoint transfer.

Rollout side. On delta arrival, the receiver:

CPU allocates pinned RAM with the DE-compatible flags (CU_MEM_LOCATION_TYPE_HOST_NUMA + CU_MEM_CREATE_USAGE_HW_DECOMPRESS) and prepares per-layer address-mapping metadata (which accounts for fused-kernel layouts like paired gate-up weights);
CPU runs Zstd decompression of the outer wrapper into the pinned staging buffer — this and step 1 run during active rollout decoding, using no GPU compute or HBM;
When the sync is applied: a double-buffered streaming loop iterates over model layers. For each layer, DE decompresses the inner Snappy payload from pinned CPU RAM into one HBM slot while SMs apply bitwise XOR from the other slot using a single Triton kernel launch. The two slots are sized for one model layer each; they fit in existing activation memory so the KV cache is not evicted;
On completion, if release_state=True, the staging buffers and prepared kernels are freed.

The implementation is open-sourced in Miles PR #3894 (orchestration + distributed compression + fault tolerance + delta checkpointing) and SGLang PR #42210 (heterogeneous decompression pipeline + DE-SM overlap + /update_weights_from_gpu_delta API). nvCOMP provides the Snappy / LZ4 / Zstd / Deflate implementations.

Results

Measured on the Ampersand production RL configuration (NVFP4 GLM-5.3, 445 GB canonical weight tensors, 0.085% mutated weights per training step proxy, 64×B300 trainer, two 8-GPU B300 rollout nodes with TP4 NVFP4 GLM-5.3 engines):

Config	Final publication size	Trainer prep + compress	Receiver non-blocking prep	Inference-blocking stage
CPU disk-delta Zstd baseline	0.876 GB	large (disk)	— (checkpoint reload blocks)	full reload
GPU LZ4 + DE (single codec)	5.36 GB	0.834 s	—	6.041 s
GPU Snappy + Zstd + DE (ship)	2.65 GB (0.60% of weights)	0.789 s	2.355 s (overlapped with rollout)	0.2196 s

The Snappy + Zstd pipeline reduces trainer-side time by 45 ms over LZ4 despite adding a Zstd pass, because Snappy is more GPU-friendly. End-to-end weight sync is <4 s. Table 2 reports compression ratios: Snappy publication 19.78 GB → 2.65 GB after Zstd (13.41%); LZ4 publication 5.36 GB → 4.03 GB after Zstd (75.21%). Nsight Systems traces confirm DE pipeline saturation during the streaming phase with Triton XOR kernels hidden under DE work.

Why it’s interesting

This is the most aggressive filed answer to the trainer↔rollout weight-sync bottleneck slime adds lossless delta weight sync for Megatron ↔ SGLang RL disaggregation (PR #1806) names as the frontier-RL wall-clock constraint. Where slime landed delta weight sync with disk + NCCL transports as a feature, humans& here push it to a hardware-saturated regime — the pause is bounded by Blackwell DE throughput, not by compression ratio, NCCL bandwidth, or GPU compute. The rollout-side design is particularly load-bearing: double compression with CPU-outer / GPU-inner stages means the GPU work is just DE + one Triton kernel per layer, which is why the KV cache survives and rollout decoding can continue during the long non-blocking prep phase.

Three cross-links to filed work sharpen the significance. First, the paper is a direct hardware-level continuation of The 4-bitter Lesson: Balancing Stability and Performance in NVFP4 RL: the 4-bitter-lesson established the numerical contract (bit-exact trainer/sampler quantization across TransformerEngine and FlashInfer) that NVFP4 RL needs; this paper adds the systems contract (near-zero-pause ownership-remapped bitwise-XOR delta sync) that NVFP4 RL at frontier scale needs. The fused-kernel layout remapping (paired gate-up weights in groups of 64) explicitly references the layout established in the companion humans& “faster-than-light NVFP4 decode” piece. Second, it rebalances the Fireworks + Cursor + slime consensus Scaling and Optimizing Frontier Model Training (Fireworks AI) Composer 2 Technical Report slime adds lossless delta weight sync for Megatron ↔ SGLang RL disaggregation (PR #1806) on delta weight sync: those stacks each ship delta sync as a feature; this one argues the stack has been tuned to the hardware ceiling. Third, the “release-on-inference” posture turns weight-sync infrastructure into a deployment primitive — one API call swaps in a fresh trained delta across data centers, a cheaper substitute for LoRA-adapter swap-in/swap-out with full-parameter expressive power.

The claim that intra-cluster P2P RDMA is now the slower alternative is strong and worth future verification against a controlled baseline, but the Nsight-measured DE saturation and the 0.2196 s pause number are concrete.