DriftWorld: Fast World Modeling through Drifting

TL;DR

DriftWorld is an action-conditioned video world model that swaps the multi-step denoising of diffusion-based WFMs for a single-forward-pass drifting generator, running at 30+ FPS on a single H100 across Bridge-V2, RT-1, Language Table, Push-T, and Robomimic — roughly 17× faster on average than diffusion-based baselines with matched or better rollout quality. The reported use cases are the two that a fast, accurate WFM most directly unlocks: inference-time policy improvement by ranking many candidate action sequences via rolled-out reward, and offline policy evaluation with correlation to real-world ground truth up to 0.9916 on Robomimic Lift. The project frames drifting not as a step-count optimization on top of a diffusion WFM but as a from-scratch generator that never had a step count to reduce.

Key claims

Multi-step sampling is the operational bottleneck that keeps diffusion-based world models from serving as fast policy rollout backends, since planning value scales with how many candidate action sequences can be searched at inference [§Abstract].
DriftWorld replaces iterative denoising with an action-conditioned drift learned at training time, generating a future-frame sequence from the current observation plus a candidate action sequence in a single forward pass [§Abstract].
DriftWorld runs at 30+ FPS on average across evaluated benchmarks, which is 17× faster on average than diffusion-based baselines [§Abstract].
Per-task inference speed on a single H100 is 33.3 FPS on Bridge-V2, 38.8 FPS on RT-1, 36.6 FPS on Language Table, 100.0 FPS on Robomimic, and 270.3 FPS on Push-T [§Visualization].
DriftWorld matches or outperforms action-conditioned world model baselines in visual generation quality at those speeds, evaluated qualitatively and quantitatively against ground-truth rollouts on Bridge-V2, RT-1, and Language Table [§Visualization].
Used as an inference-time policy improver — rolling out multiple candidate action proposals in the world model and selecting the highest-reward rollout — DriftWorld reaches SOTA decision-making performance on the evaluated benchmarks at a fraction of the inference time of diffusion-based WFMs [§Abstract; §Inference-Time Policy Improvement].
Used as an offline policy-evaluation simulator, DriftWorld’s rollout-based scores correlate with ground-truth real-world performance at r = 0.9916 (Robomimic Lift), 0.9250 (Robomimic Can), and 0.9515 (Push-T) [§Policy Evaluation].
Training a DriftWorld backbone requires only 1–2 GPUs, positioning drifting as substantially cheaper to train than diffusion-based WFMs at comparable rollout quality [tweet].
The interactive demo shows real-time control (arrow-key gripper motion) on Language Table and Push-T, with every frame generated by a single forward pass on an NVIDIA L4 GPU — direct evidence that the FPS numbers hold on interactive-hardware-class inference budgets [§Interactive Demo].

Method

DriftWorld inherits the drifting one-step generation paradigm (Generative Modeling via Drifting, A Unified View of Drifting and Score-Based Models) and specializes it to action-conditioned video prediction for robot manipulation. In the reference drifting recipe, a pushforward generator maps a noise sample to a data sample by following a learned drift field that vanishes at equilibrium with the data distribution; training regresses the generator against its own one-step-forwarded output, so the trained model outputs a clean sample in a single forward pass rather than iterating a denoiser. DriftWorld’s specialization is (i) conditioning on the current observation and a candidate action sequence rather than on a text prompt, and (ii) predicting a sequence of future frames as the pushforward target rather than a single image.

The public project page describes DriftWorld as trained on standard robotic-manipulation video datasets (Bridge-V2, RT-1, Language Table, Robomimic, Push-T) with the drifting objective on action-conditioned tuples (o_t, a_{t:t+H}) → (o_{t+1:t+H}). The interactive demo runs on an NVIDIA L4 GPU where each key press triggers a fresh single-forward-pass rollout, which is the operational form of the FPS claim: no denoising loop, no KV cache to unroll, no per-step diffusion schedule to traverse — one action sequence in, one video out.

For inference-time policy improvement, the deployed workflow is straightforward MPC-style action search: a base policy (which the page does not fully specify but frames as generic) proposes multiple candidate action sequences, DriftWorld rolls out each in a single forward pass, a task-reward function scores each rollout, and the highest-scoring first action is executed. The green-highlighted “chosen rollout” in the project page illustrates this. For offline policy evaluation, DriftWorld replaces the physical / physics-simulator environment entirely: policies are rolled out inside DriftWorld and scored, and the resulting scores are correlated against ground-truth real-world performance.

Method-level details not present on the project page (drift-field parameterization, whether Gaussian or Laplace kernel, model backbone size, whether the video is generated frame-parallel or chunked, exact conditioning-fusion architecture) are presumably in a companion paper not linked from the project site at time of filing.

Results

Speed. 30+ FPS average, 17× faster than diffusion-based WFM baselines on average, on a single H100 [§Abstract]. Per-task FPS: Bridge-V2 33.3, RT-1 38.8, Language Table 36.6, Robomimic 100.0, Push-T 270.3 [§Visualization].

Rollout quality. DriftWorld matches or beats action-conditioned WFM baselines in generation quality across Bridge-V2, RT-1, Language Table, Robomimic Can, Robomimic Lift, and Push-T (single-view and two-view settings) [§Visualization]. Absolute FVD / PSNR numbers are not on the project page.

Decision-making. SOTA decision-making performance on the evaluated benchmarks, framed as “far less inference time than diffusion-based WFM baselines” for the same or better task success [§Abstract].

Policy-evaluation correlation. Pearson r between DriftWorld-inside-simulator policy scores and ground-truth real-world scores is 0.9916 on Robomimic Lift, 0.9250 on Robomimic Can, and 0.9515 on Push-T [§Policy Evaluation]. The comparison partner named on the page is Ctrl-World, with DriftWorld reported as more faithful.

Training cost. 1–2 GPUs sufficient to train the model [tweet]. This is a claim about training scale (small enough for a single lab / single node) rather than about a specific parameter count.

Why it’s interesting

DriftWorld collides three threads the wiki has been tracking and picks a different point on each. First, on the world-model-as-policy-evaluator axis, it lands with RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation (KAIST, autoregressive DiT + VLM scorer, r = 0.989 on real-robot correlation) and Evaluating Gemini Robotics Policies in a Veo World Simulator (Google DeepMind, Veo fine-tune as a manipulation simulator) as the third distinct instance of the same claim: a learned video WFM’s dynamics is faithful enough that policy rankings inside it transfer to the real world. DriftWorld’s contribution to that thread is the throughput lever — 30+ FPS one-step rollout instead of an autoregressive DiT with a few-step denoising schedule, which is the constraint RoboWorld’s Step Forcing is explicitly trying to relax on the diffusion side.

Second, on the drifting-vs-diffusion axis, DriftWorld is the first filed application of Kaiming-lab-style drifting to a robotics WFM, downstream of the theoretical framing in A Unified View of Drifting and Score-Based Models (drifting is a teacher-free reverse-Fisher variant of DMD) and the original Generative Modeling via Drifting. The 17× speed claim is exactly the payoff drifting was designed to deliver — a one-step generator that avoids the DMD-teacher training cost — and applying it to action-conditioned video is a natural extension because policy rollout is the setting where the multi-step denoising cost is most punishing. The wiki’s diffusion-distillation concept has been dominated by teacher-based distillation (DMD, phased DMD, TDM-R1); DriftWorld is a datapoint for the teacher-free branch that survives the mode-dropping concern raised in the theoretical bridge paper, at least for the specific setting of action-conditioned rollout where mode diversity matters less than dynamics fidelity.

Third, on the fast-WFM-for-inference-time-planning axis, DriftWorld contrasts with Enabling Robust Cloth Manipulation via Inference-Time Simulator-in-the-Loop Refinement (SILR — MPPI over a hand-built physics simulator inside the control loop) by keeping the same MPC-with-many-rollouts structure but swapping the physics simulator for a learned WFM. It also contrasts with Test-Time Gradient Guidance of Flow Policies in Reinforcement Learning (test-time gradient guidance of flow policies) — DriftWorld improves the base policy by search rather than by test-time gradient — and with OSCAR: Omni-Embodiment Action-Conditioned World Model for Robotics (a slower diffusion action-conditioned WFM aimed at generalist embodied simulation).

The open question the project page leaves unanswered: does the drifting formulation actually preserve the action-observation dynamics causally (does executing action a in the WFM produce the same effect as in reality), or does it just produce visually-plausible rollouts that happen to correlate at rank level with ground truth? The 0.99 correlation is a rank-level claim, and RoboWorld’s Step Forcing recipe was designed specifically to address the dynamics-preservation failure mode. This is the natural follow-up for a paper release.