WorldDiT: A Unified Diffusion Architecture for World and Action Modeling

TL;DR

WorldDiT is a single diffusion transformer that jointly generates continuous action chunks and predicts normalized RGB patch targets from future camera frames during training — no separate pretrained VLM backbone, no separate video world model. Across four LIBERO simulation suites the sub-billion-parameter model is reported to sit on the Pareto frontier of total-parameters × mean-success among methods reporting all four suites. Positioned as a strong sub-billion-parameter baseline for scaling studies of unified action + visual-world-modeling DiTs.

Key claims

Action generation is coupled with visual world modeling inside one diffusion transformer: the same network produces continuous action chunks and predicts normalized RGB patch targets from future camera frames during training [Abstract].
Strong control performance is achievable without a large pretrained VLM as the action backbone — WorldDiT is trained from scratch as a unified DiT rather than adapted from a VLM [Abstract].
On the four LIBERO simulation suites, WorldDiT lies on the reported Pareto frontier of total model parameters versus mean success among methods that report all four suites [Abstract].
The paper positions itself as a sub-billion-parameter baseline for future scaling studies of unified diffusion architectures for world + action modeling [Abstract].

Method

WorldDiT is a single diffusion transformer trained with a joint objective. One head of the model produces continuous action chunks (the policy output); a second training-time supervision signal comes from predicting normalized RGB patch targets from future camera frames — i.e. the visual world-modeling target is a patch-level future-frame prediction rather than pixel-decoded rollout. The two targets share the same DiT backbone, so world-modeling supervision reshapes the same representations that produce actions. There is no VLM backbone attached; the entire stack is trained together as a unified architecture, and the auxiliary future-frame target is a training-time signal (its role at inference is not described in the abstract).

Results

Evaluated on four LIBERO simulation suites (Spatial, Object, Goal, Long — the standard four) and reported to occupy the Pareto frontier for total model parameters versus mean success among methods reporting all four suites [Abstract].
Total parameter count is sub-billion; specific numbers per suite are not disclosed in the abstract.
No real-robot deployment or wall-clock inference numbers are reported in the abstract.

Why it’s interesting

WorldDiT stakes out a from-scratch unified DiT recipe distinct from the two dominant sub-1B VLA design points abc-scalable-behavior-cloning-open-2026-06 (see VLA Models) tracks — the 2B-DiT-with-tiny-vision-encoder and the VLM-plus-small-action-head — by removing the VLM entirely and putting a future-frame patch-prediction target on the same backbone. Sibling in spirit to Causal World Modeling for Robot Control (LingBot-VA) (LingBot-VA) and LingBot-VA 2.0: Native Video-Action Pretraining for Generalizable Robot Control (native-from-scratch pretraining, unified world+action) but pushed to sub-billion parameters and evaluated only on LIBERO rather than real robots.
The future-RGB-patch auxiliary is the same class of supervision as From Foundation to Application: Improving VLA Models in Practice (LingBot-VLA 2.0) (LingBot-VLA 2.0’s “video representation model for semantic priors” head) and Understanding and Mitigating the Video-Action Generalization Gap via Temporal Ratio (Temporal Ratio’s Cosmos-Predict future frames), but with the world-modeling target co-trained on the same backbone from scratch rather than distilled from a frozen video model or attached as a proxy. Concrete data point for the “WFM-as-policy-internal-substrate” deployment pattern tracked on World Foundation Models, at the smallest scale filed so far.
Contrasts with the “unified-VLM-is-enough” position of Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models (Embodied-R1.5) which reaches 97.3 LIBERO overall from an 8B Qwen3-VL-derived embodied VLM. WorldDiT’s Pareto claim is at sub-billion parameters, so the comparison is at a very different point on the compute frontier — the paper as filed doesn’t report head-to-head numbers against Embodied-R1.5 or π0.5, only Pareto positioning within its filed peer set.