GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation

TL;DR

GigaWorld-1 (GigaBrain / GigaAI team) is a systematic study of world models as robot-policy evaluators, plus a benchmark (WMBench) and a purpose-built model derived from the study’s design roadmap. WMBench pairs real-robot teleoperation data with matched policy rollouts across diverse manipulation tasks so that world-model families, action encodings, rollout horizons, and evaluation metrics can be compared under controlled conditions [Abstract]. Using WMBench, the paper analyzes 7 video world models, 4 action-representation schemes, and 324,000+ simulated rollouts paired with real robot executions, augmented by CVPR 2026 GigaBrain Challenge submissions, curated synthetic trajectories, and 12,000+ hours of training video [Abstract]. The three headline findings — long-horizon action-faithful rollout consistency dominates evaluator quality (not short-term visual realism), pretraining gains come from balancing general world knowledge with robot-specific controllability (not just data scale), and evaluator-focused post-training + architectural choices strongly determine real-world alignment — are then compiled into a design roadmap and instantiated as GigaWorld-1, released with code, models, datasets, and toolkits [Abstract].

Key claims

Evaluating embodied robot foundation models remains the critical bottleneck for progress: unlike LLMs assessed via cheap digital benchmarks, robot policies require slow, costly, hardware-and-supervision-limited real-world rollouts, which is what drives interest in world models as surrogate policy evaluators [Abstract].
The key properties that make a world model reliable for policy assessment remain poorly understood at filing time, and the paper positions itself as the first systematic study specifically targeting this question rather than generic video-generation quality [Abstract].
WMBench is constructed from real-robot teleoperation data plus matched policy rollouts covering diverse manipulation tasks, enabling controlled comparisons across model families, action encodings, rollout horizons, and evaluation metrics [Abstract].
The evaluation scope is large: 7 video world models × 4 action-representation schemes, 324,000+ simulated policy rollouts paired with real robot executions, plus CVPR 2026 GigaBrain Challenge community submissions and 12,000+ hours of curated training video [Abstract].
Finding 1 (evaluator quality): evaluator quality is dominated by long-horizon, action-faithful rollout consistency, not by short-term visual realism [Abstract].
Finding 2 (pretraining): pretraining gains stem not only from data scale but from balancing general world knowledge with robot-specific controllability — a mixture claim rather than a pure-scale claim [Abstract].
Finding 3 (architecture / post-training): action encoding, memory design, and evaluator-focused post-training strongly determine alignment with real-world robot behavior [Abstract].
GigaWorld-1 is the paper’s own instantiation of the design roadmap derived from these findings — a world model specifically optimized for policy evaluation rather than general video generation [Abstract].
Code, models, datasets, and toolkits are released to enable scalable evaluation research for embodied foundation models [Abstract].

Method

The paper’s structure is study → roadmap → model. The study machinery is WMBench: a benchmark built from real-robot teleoperation data and matched policy rollouts on the same tasks, so that a candidate world model’s rollout can be compared against a ground-truth real execution for the same policy on the same initial condition. The controlled variables tracked in the ablation grid are (a) model family — 7 video world models spanning the current landscape; (b) action encoding — 4 schemes for how policy actions are conditioned into the world model; (c) rollout horizon — how far into the future the model is trusted; and (d) evaluation metric — the aggregation function that turns predicted vs. real trajectories into a per-model score. The design of WMBench is what makes the three headline claims falsifiable: absent a controlled matched-rollout setup, “long-horizon consistency beats short-term realism” would collapse into a preference for a specific metric.

The data stack has three tiers: (1) real-robot teleoperation and matched policy rollouts inside WMBench itself (~324K rollouts); (2) CVPR 2026 GigaBrain Challenge submissions folded in as a community-contributed evaluation surface; (3) 12,000+ hours of training videos plus curated synthetic trajectories that feed the pretraining half of the study.

GigaWorld-1, the model, is presented as the operationalization of the roadmap the study derives. Method-level details of GigaWorld-1’s backbone (parameter count, tokenizer, denoising schedule vs. autoregressive vs. hybrid), the specific action-encoding scheme that “wins”, the memory design, and the evaluator-focused post-training recipe are not extractable from the abstract; the paper releases them alongside code, models, datasets, and toolkits.

Results

The three findings are the headline results:

Long-horizon action-faithful rollout consistency dominates evaluator quality — short-term visual realism (the axis benchmarks like RISE-Video and PhyGenBench probe on general video generators) is not the axis that separates good policy evaluators from bad ones [Abstract].
Pretraining gains follow data mixture, not just scale — general world knowledge and robot-specific controllability need to be balanced, positioning the paper against pure-scale framings of world-model pretraining [Abstract].
Architectural choices matter — action encoding, memory design, and evaluator-focused post-training strongly determine real-world alignment [Abstract].

Per-model correlations against real robot outcomes, per-action-encoding ablations, per-horizon curves, and the WMBench leaderboard numbers themselves are in the paper body and not extracted here. The abstract does not report a single-number correlation coefficient in the style of RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation‘s Pearson r = 0.989 or SimFoundry: Modular and Automated Scene Generation for Policy Learning and Evaluation‘s Pearson r = 0.911, so cross-paper comparison on the correlation axis has to wait for the body-level numbers.

Why it’s interesting

GigaWorld-1 is the first filed benchmark + study paper aimed specifically at the “which world model is a good policy evaluator” question, not the more general “which world model has the best video-generation quality” question. The wiki has been steadily accumulating instances of the WFM-as-evaluator pattern — RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation (KAIST + Config, autoregressive DiT + Step Forcing, Pearson r = 0.989), Evaluating Gemini Robotics Policies in a Veo World Simulator (Google DeepMind, action-conditioned Veo fine-tune, 1600+ real evaluations), Scale Robot Policy Evaluation with Ray (Distributed Sim-Eval on Anyscale) (Anyscale, distributed sim-eval infrastructure), SimFoundry: Modular and Automated Scene Generation for Policy Learning and Evaluation (explicit-physics counter-recipe with Pearson r = 0.911) — but each of those asserts their WFM is a good evaluator without a controlled study of what makes a WFM a good evaluator. WMBench is the missing evaluation-of-evaluators layer, and the three headline findings give the World Foundation Models concept page a first empirical answer to its long-standing open question about which axis of WFM-ness matters for downstream use.

The Finding-1 claim — long-horizon action-faithful rollout consistency dominates short-term visual realism — is a direct empirical counter to the design pressure of general video-generation benchmarks tracked on Video Generation Benchmarks. RISE-Video, PhyGenBench, VideoPhy-2, and Physics-IQ all reward frame-level fidelity; a world model that wins them may still be a bad policy evaluator if it drifts under 30-second action-conditioned rollouts. GigaWorld-1 puts a concrete empirical stake in the ground here.

Finding 2 — pretraining gains come from balancing general world knowledge with robot-specific controllability — reads as an empirical validation of the mixture recipe that LingBot-Video: Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence argued for from the model-design side and that Data Pyramid for Embodied Manipulation argued for from the data-side taxonomy. That three independent 2026-07 papers converge on the same claim from very different starting points is worth noting.

The scale (324K rollouts, 12K+ hours of video, 7 WFMs, 4 action encodings, community-submitted CVPR 2026 GigaBrain Challenge entries) puts this alongside UMI-Bench 1.0: An Open and Reproducible Real-World Benchmark for Tabletop Robotic Manipulation with UMI Data and RoboArena: Distributed Real-World Evaluation of Generalist Robot Policies as one of the largest robotics-evaluation infrastructure releases on file — but on the simulated / centralized pole rather than the real-world / decentralized ones.