World Foundation Models

TL;DR

World foundation models (WFMs) are large pretrained models marketed as priors over the physical world — generating, predicting, or reasoning about video, geometry, and dynamics at scale. The filed papers cover the two ways the wiki currently sees them: (i) as first-party releases of WFM stacks (NVIDIA Cosmos Reason / Predict / Transfer, Meta VL-JEPA via Action100M), and (ii) as off-the-shelf backbones reused inside downstream systems — Cosmos Reason 2 plugged into Isaac GR00T N1.6 as the reasoning backbone for humanoid VLA, and VJEPA-2 plugged into WMReward as a frozen physics-plausibility reward for video-generation inference-time alignment. The unifying claim is that physics/dynamics understanding lives in predictive latent representations, not in general-purpose VLMs.

Key claims

• WFMs are positioned as the backbone layer of physical-AI stacks: NVIDIA bundles Cosmos Reason 2 (reasoning VLM), Cosmos Predict 2.5 and Cosmos Transfer 2.5 (synthetic-video generators) as the open layer beneath embodied policies (NVIDIA Unveils New Open Models, Data and Tools to Advance AI Across Every Industry).
• The same WFM can be reused as the perception/reasoning prior for a downstream VLA without retraining — Isaac GR00T N1.6 uses Cosmos Reason 2 for full-body humanoid control (NVIDIA Unveils New Open Models, Data and Tools to Advance AI Across Every Industry).
• Latent-predictive WFMs (the V-JEPA / VL-JEPA line) outperform image–text-pretrained encoders on motion-focused tasks at far fewer training samples — VL-JEPA ViT-L at 3.3B samples beats PE-Core ViT-G at 86B samples on average action-recognition top-1 (52.5 vs 44.7) and especially on SSv2 / EPIC-KITCHENS-100 / EgoExo4D (Action100M: A Large-scale Video Action Dataset Tab. 3).
• Latent-predictive WFMs work as off-the-shelf physics-plausibility reward models: VJEPA-2’s predictive surprise drives MAGI-1 24B from PhysicsIQ 55.22 → 62.00 on V2V (+6.78, 1st place at ICCV 2025) without any fine-tuning (Inference-time Physics Alignment of Video Generative Models with Latent World Models §3.1, Table 1).
• VLM-based judges are not a substitute for latent-WFM signals at this task — Qwen2.5-VL / Qwen3-VL scored as reward models perform near chance on PhysicsIQ and can regress baselines (Qwen2.5-VL(BoN) on MAGI-1 I2V: −4.78 vs +6.79 for VJEPA-based WMReward) (Inference-time Physics Alignment of Video Generative Models with Latent World Models §3.1, Table 1).
• The same predictive-latent prior that helps zero-shot action recognition (VL-JEPA on Action100M) is what makes VJEPA-2 a useful reward model — a single architectural family pays off in both directions (Action100M: A Large-scale Video Action Dataset Tab. 3; Inference-time Physics Alignment of Video Generative Models with Latent World Models §2.3).
• WFMs are typically released open: NVIDIA on HF + GitHub + build.nvidia.com; Meta’s VL-JEPA accompanied by Action100M (~205 GB, 147M segments) on HF (NVIDIA Unveils New Open Models, Data and Tools to Advance AI Across Every Industry; Action100M: A Large-scale Video Action Dataset §4).
• …but the flagship interactive-rollout WFMs ship closed: Google DeepMind’s Genie 3 is gated behind a Google AI Ultra subscription via a Google Labs web prototype (Project Genie), with no model weights, no architecture details, and no benchmarks disclosed (Project Genie: Experimenting with infinite, interactive worlds).
• The interactive-rollout WFM product surface centres on three primitives — text+image World Sketching (image-grounded preview pre-commit), World Exploration (real-time path generation as the user moves, 20–24 fps / 720p), and World Remixing (gallery + prompt-remix layer) (Project Genie: Experimenting with infinite, interactive worlds).
• Closed-flagship WFMs are specializable into vertical foundation models via post-training rather than from-scratch retraining: Waymo’s autonomous-driving simulator is built by post-training Genie 3 onto Waymo’s multi-sensor driving data, inheriting Genie’s broad world knowledge while extending the output modality from camera-only to camera + lidar (4D point clouds) — a modality the upstream model never trained on (The Waymo World Model: A New Frontier For Autonomous Driving Simulation).
• The post-trained vertical WFM exposes the same three control primitives the upstream flagship does (driving-action / counterfactual control, scene-layout control, language control), plus a domain-specific dashcam-to-simulation mode and an “efficient variant” for long-horizon rollouts — though no quantitative results, ablations, or benchmarks are disclosed (The Waymo World Model: A New Frontier For Autonomous Driving Simulation).
• Frontier video generators (Hailuo 2.3, Veo 3.1, Sora-2, Wan 2.6) fail the strict-reasoning check on most samples even while their per-frame Visual Quality stays >90% — RISE-Video reports a top-line strict accuracy ceiling of 22.5% across 11 representative TI2V models, with open-weight models like Wan 2.2-TI2V-5B and CogVideoX 1.5-5B dropping to 5.4% and 1.9% (RISE-Video: Can Video Generators Decode Implicit World Rules? Scoreboard). This separates “video generator as WFM” claims into a fidelity axis (largely solved) and a reasoning axis (largely open).
• The community treats “world model” as a cross-domain umbrella spanning game simulation, autonomous driving, embodied AI, science, NLP, and agents — with the term’s definition tracing back to Ha & Schmidhuber 2018 and LeCun’s “Path Towards Autonomous Machine Intelligence” position paper (Awesome World Models Definition of World Models).
• Foundation-layer normative position: generative-rollout world models should replace, not coexist with, explicit 3D representations (point clouds, radiance fields, SDFs, voxel grids) and SE(3) camera poses as the perception-side pre-training substrate for embodied AI; explicit 3D is a hand-crafted intermediate that the Bitter Lesson should be expected to dissolve. The same logic applies to camera-pose estimation (ego-motion is just the simplest action). 3D meshes are still reasonable at the human–machine interface layer (CAD, manufacturing output formats), but should not be baked into training architectures (The flavor of the bitter lesson for computer vision).
• Open problem named by the same source: the chicken-and-egg of action-conditional world models — training interactive WFMs requires paired action–observation data, which is exactly the resource embodied-AI lacks (excepting self-driving). Video generative pre-training is offered as a candidate scalable objective whose latents implicitly encode physics + human-skill structure, but it is not claimed to be sufficient (The flavor of the bitter lesson for computer vision).
• Organizational framing: at least one closed-flagship lab is publicly restructuring around the GWM thesis rather than around video generation per se — Runway announces an internal incubator (Runway Labs) explicitly chartered to prototype products on top of its GWM stack across film, healthcare, education, gaming, advertising, sports, retail, real estate and training (Introducing Runway Labs). This is a portfolio framing in contrast to DeepMind’s single-product Project Genie launch and Waymo’s single-vertical Genie-3 specialization.

Recent contributions

• [2026-11-05] DreamFree: Future Representation Distillation for Imagination-free World Action Models: DreamFree (Jo, Jang, Yu, Ye et al.; KAIST — shared authorship with World Action Models are Zero-shot Policies (DreamZero) DreamZero) adds an eleventh WFM deployment pattern to this page’s ledger: WFM-as-training-only-representation-teacher-for-current-frame-encoder — distinct from the ten already tracked (backbone, predictive-surprise reward, interruptible tool, policy-internal substrate, edit-prior via KV cache, frozen trace-WM + small action expert, value-model, inference-time guidance signal, self-regenerator for data-free continual, substrate for jointly-generated executable code). Frozen video-diffusion teacher processes current + future frames; a student visual encoder observing only the current frame is trained under the standard video generative objective plus an alignment loss that pushes its features toward the teacher’s future-aware representations. Teacher and video rollout are both discarded at inference — the student’s visual encoder alone drives a readout + action decoder at 91 ms / 11 Hz on H200 (17.8× faster than DreamZero’s full-WAM rollout at the same 14B backbone). Directly engages the “video co-training as training-side lever vs test-time imagination” axis Fast-WAM opened and Long-WAM contested: 5B → 14B RoboCasa scaling signature is +0.3pp for Fast-WAM and +13.2pp for DreamFree (58.5 → 71.7%, 74.4% with extra videos). First filed WAM recipe with a clean per-backbone-size scaling signature on the imagination-free side at 14B, plus the first filed instance where extra video-only data (no action labels) still improves policy performance under this pattern (Fast-WAM +24.1pp, DreamFree +2.7pp at 14B with extra video). Real-world DROID (50 trials × 10 tasks/policy) 64.0% success / 83.0% progress vs DreamZero 56.0 / 71.8 and π0.5 16.0 / 33.5.
• [2026-11-02] 3D World Simulator: Hybrid Particle-Voxel Dynamics for Multimodal 3D World Simulation: 3D World Simulator (Wang, Chou, K. Zhang, Y. Wang, B. Huang, Yu, Nabi, Yunzhu Li; Columbia + Berkeley + Michigan + NVIDIA + Analog Devices) stakes out the explicit-3D-latent cell of the WFM design space: a sparse voxel latent encoded from multi-view RGB-D point clouds with hybrid particle-voxel decoding, 3D-U-Net action-conditioned dynamics (no diffusion, no flow matching), 50+ FPS on an RTX 5090, and a factored foreground/background/embodiment setup where room (3DGS) and arms (3DGS + kinematic model) are separate assets that can be swapped post-hoc. Directly engages the “foundation-layer normative position: generative-rollout WFMs should replace, not coexist with, explicit 3D representations” claim recorded in the Key claims — this filing is a sharp counter-datapoint showing the explicit-3D cell delivering multi-view-consistent rendering, co-located tactile, policy-in-the-model evaluation, and real-robot transfer from model-only training data in one system. No quantitative sim-to-real numbers on the project page.
• [2026-11-02] The Bitter Lesson: Back-fills the canonical reference behind the “generative-rollout WFMs should dissolve explicit 3D intermediates” normative framing (seventh WFM angle) — Sutton 2019’s four-case historical argument (chess, Go, speech, vision) is what The flavor of the bitter lesson for computer vision applies to perception + embodied AI, now resolvable as a single wiki slug rather than an external link.
• [2026-10-29] ThinkWorld: World Imagination Reinforces Chain-of-Dynamics Reasoning: ThinkWorld (Chaehyun Kim) uses world imagination as the per-CoD reward signal: for each proposed Chain-of-Dynamics (sequence of discrete motion tokens connecting current to future state), predict the induced future and reinforce CoDs whose predictions better match the observed outcome. Adds a new deployment pattern for WFMs alongside WFM-as-policy (Dyna-2, LingBot-VA), WFM-as-planner (DriftWorld, RoboJEPA), and WFM-as-environment (Cosmos Simulation) — WFM-as-reward-signal for the reasoning trace of a VLA, with the key property that it operates on observations rather than action labels so action-free human videos can supply the reward channel.
• [2026-10-29] Dex-SuperMimic — Reproducing SuperMimic on Dexterous Hand Manipulation: Dex-SuperMimic ports SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model SuperMimic’s differentiable-world-model + BPTT policy learning from humanoid whole-body control to dexterous Wuji Hand in-hand cube reorientation + pen spin — concrete evidence the sim-state-space differentiable-WFM recipe transfers across task families via community reimplementation (built on supermimic_vibed + Wuji MJLab + InstinctMJ), with a translucent world-model ghost rendered in the Viser viewer alongside real-sim trajectories as a built-in debug surface for the WFM prediction.
• [2026-10-28] RynnVLA-Latent: Scaling Robotic Manipulation Foundation Models with 3D-Grounded Latent Actions: RynnLAM, the latent-action model behind RynnVLA-Latent (Alibaba DAMO), is trained on 30.1K hours of heterogeneous interaction video (human egocentric + robot teleop + simulation) with residual future-feature prediction + camera-compensated 3D-flow supervision. Yields 745.52M latent-action descriptors treated as a reusable pretraining substrate for downstream VLA policies — a video-trained “action prior” that is positioned alongside more generative video WFMs on the control-relevance axis rather than the pixel-fidelity axis.
• [2026-10-28] Ψ-0.6: Adaptive Streams of Thought in a Unified Vision-Language World Model: Ψ-0.6 (Aw, Kotar, Zhang et al., Stanford NeuroAILab) adds a new WFM deployment pattern to the ledger: WFM-as-counterfactual-rollout-plus-interpreter under one weight set. One 8B AR transformer both generates imagined scenes (action-conditioned future frames conditioned on language instructions like “move the box 0.8 m right”) and interprets them to answer VLM-style questions (“is the cup hidden?”), with simulation and interpretation sharing weights rather than factored across a generator and a judge. Distinct from the DiffThinker-style “pixel-space generator proposes / token-space reasoner verifies” split and from the frozen-WFM + small-action-expert line (µ₀, FLUX-mimic): one shared vocabulary over images / text / camera pose / depth / flow / point tracks, trained on simulated rooms, transfers to real robot tabletops at different scales/objects. Beats other world models on FlowBench motion prediction and Qwen3-VL on MindCube zero-shot. RL on task feedback jointly optimizes the selection, generation, and use of intermediate representations.
• [2026-10-28] Genie 4: A new generation of world simulation: Genie 4 productizes the LLM-in-the-loop pattern inside a shipped WFM: Gemini 3.8 Flash runs as an observe-plan-verify-adapt guiding agent inside the real-time simulation, decomposing user goals into step sequences, reviewing incoming frames to verify each step’s success, and adapting the plan when the world shifts or the user intervenes. First closed-flagship instance of the per-step LLM-guided-WFM pattern where the agent reads frames and rewrites the next action rather than invoking external tools.
• [2026-10-27] Google DeepMind announces Genie 4 — most realistic and interactive world model yet: Genie 4 launch (DeepMind) — closed-flagship interactive-WFM announcement with no weights, no architecture, no benchmarks, no availability tier disclosed. Filed as a market-positioning marker in the closed/open WFM race alongside Cosmos 3: Omnimodal World Models for Physical AI Cosmos 3 (open) and Advancing Open-source World Models (LingBot-World) LingBot-World (open). Reinforces that the flagship interactive-rollout WFMs continue to ship closed in 2026 regardless of the open-WFM momentum from NVIDIA, Robbyant, and SolarWM.
• [2026-10-27] AHa-3D: Agentic Tool Use for Real2Sim with GPT-6 Astra: AHa-3D (and the LiteReality-Agent NeurIPS 2026 companion) is the “agent-authored editable 3D scene from indoor video” cell of the Real2Sim spectrum, complementing SIMPACT and RoboSimStudio’s robot-task Real2Sim pipelines with a room-scale human-motion payload. The underlying bet is that frontier coding/agentic models with 3D vision tools out-perform end-to-end generative scene models on geometric accuracy and simulation compatibility (per LiteReality-Agent abstract).
• [2026-10-26] Agentic World Action Models: AW-0 extends the WAM joint-denoising lineage (World Action Models are Zero-shot Policies (DreamZero) DreamZero, Causal World Modeling for Robot Control (LingBot-VA) LingBot-VA, InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data InternW0-Δ, EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Data EgoWAM) with code as a third coupled generative stream denoised jointly with future video latents and action chunks inside one MoT (code expert = LLaDA diffusion LM + frozen Qwen3-VL ViT; video expert = pretrained Cosmos-Predict2.5 DiT; action expert = separate DiT). Load-bearing attention-ordering ablation: C → V → A (code first) beats V → C → A on 7/8 real-world tasks because semantic task structure shapes future prediction throughout denoising rather than only after the video is committed — first filed empirical answer with a controlled multi-stream comparison to where structured planning should sit relative to WAM future prediction. Qualitative future-video readout under the three ablations is unusually sharp: on “arrange the fruits so only mango is in the left bowl” AW-0 predicts the right wrist camera interacting with the mango, a natural-language-subgoal variant hesitates between apple and mango, and the no-code variant goes for the apple — direct evidence that explicit skill calls + object arguments give the video expert a more specific condition than language alone and that the action expert in turn reads more reliable predictive features. First filed WAM where the planning channel is executable at deployment time: in hybrid mode the decoded program is run Code-as-Policies-style against perception/control primitives with the action chunk as fallback, trading 17.7 pp on clean scenes for +8.6 pp on visual-randomization and preserving 46% of clean performance under randomization vs 25% for default mode. Pretrained on 2,000 h YAM bimanual + 1,000 h egocentric human manipulation under a frozen 50-skill library (23 YAM + 27 egocentric) distilled from the data itself. Adds a tenth WFM deployment pattern to this page’s ledger — WFM as substrate for jointly-generated executable planning code — distinct from the nine previously tracked (backbone, predictive-surprise reward, interruptible tool, policy-internal substrate, edit-prior via KV cache, frozen trace-WM + small action expert, value-model, inference-time guidance signal, self-regenerator for data-free continual learning).
• [2026-10-25] Andrew Carr — SuperMimic open reimplementations diverge during Bones Seed; four engineers compare notes: Reproducibility signal on the differentiable-WFM-for-humanoid-tracking pole (SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model): multiple open reimplementations diverge ~20% through SuperMimic’s Bones Seed curriculum. The failure signature is critic overestimation under BPTT absent SHAC-style TD-target clamping — a specifically differentiable-WFM-RL instability that the pixel-space (Cosmos 3, Dyna-2) and latent-predictive (JEPA) poles of the WFM cluster do not surface. One port (Sammo) reaches 86k iterations stable but is still short of the paper’s clip-failure count on the full 320k schedule, so the paper’s headline >1000× GPU-hour reduction remains unverified outside the authors’ pipeline.
• [2026-10-23] Toward Recursive Self-Improvement in Physical AI: Luo et al. survey positions world models as one of four categories of retained change in physical RSI (alongside policy updates, memories/skills/code, and learning procedures) — specifically “improve a policy inside fixed learned models” and “update the world model, then the policy; repeat with new physical data” as representative pathways. Supplies the physical-regime vocabulary for the WFM-as-improved-component case (sibling to the WFM-as-backbone / WFM-as-policy-internal-substrate / WFM-as-value-model deployment patterns already tracked on this page), and raises the question of whether the chicken-and-egg of action-conditional WFM training constrains the recursion of the WFM retained-change category specifically. First filed survey to decompose “WFM update” as one of four targets under a controlled frozen-knowledge comparison rather than treating it as a monolithic axis.
• [2026-10-23] NVIDIA Robotics announces Isaac Lab 3.0 — Newton multi-physics, OVRTX, RL Post Training, LEAPP deployment: Isaac Lab 3.0 wires Cosmos 3-Nano Transfer into the simulator-rendering pipeline as a WFM-backed sim-to-real appearance bridge — simulated frames pass through the WFM to acquire photorealistic distributions before being used downstream for training or policy deployment via LEAPP. Operational instance of the “WFM as data engine for physical AI” positioning from Cosmos 3: Omnimodal World Models for Physical AI Cosmos 3, now packaged as a first-party step in the robot-training stack rather than a standalone generator. No quantitative sim-to-real transfer results in the launch tweet.
• [2026-10-22] CW-WAM: A Structured Contact-Wrench World Action Model for Force-Sensitive Manipulation: CW-WAM (anonymous; cw-wam-project.github.io) extends the WAM joint-prediction target from (video, motion) to (video, motion, 6-D contact wrench) with pair-causal attention structuring motion ↔ wrench dependencies at the chunk-token level and a long fixed-length wrench memory preserving sustained payload tension / sustained resistance across the attention window. The predicted wrench chunk becomes the reference for a 20 Hz outer-loop admittance controller, with a 1 kHz low-level robot controller consuming interpolated corrected waypoints — a slow–fast factorization that keeps force feedback live between WAM queries. First filed WAM instance on the page where the predicted force channel is driving an admittance controller rather than being consumed as auxiliary supervision (InternW0: A Foundational Physical World Model for Efficient Real-World Interactions augments the WAM action target to [a_t; f_t; τ_t] as a self-prediction target under the action expert, with no admittance loop) or as an upstream-pretraining axis (FACT: A Multimodal Force-Action-Track Foundation Model for Human-to-Robot Dexterity FACT, EgoForce). Reports 25% → 70% on 1 mm-clearance square peg insertion, 40% → 95% on power-adapter seating, force-reference RMSE 4.8 N → 2.7 N (peg) and 2.0 N → 1.3 N (wiping) over a no-wrench WAM baseline. Adds a concrete answer to this cluster’s long-standing “which non-visual channel actually matters for contact-rich tasks and how should it enter the WAM” open question — on this task class, it’s the 6-D wrench, and it enters via joint chunk-level prediction + a slow–fast admittance controller.
• [2026-10-22] Interactive World Simulation with Cosmos for Physical AI: Cosmos Simulation (NVIDIA) ships the sharpest filed open-source instance of the shared bidirectional → causal → distillation post-training recipe applied end-to-end to a frontier open WFM across three unrelated simulator domains in one report — Cosmos 3 → Cosmos3-Nano-Sim-Bimanual (action-conditioned manipulation), Cosmos3-Nano-Sim-Transfer (control-guided video transfer with external simulator state), and Cosmos3-Nano-Sim-Auto (multi-view RGB+LiDAR AV). Open-source counterpart to Meet Odyssey-3: Our Most Powerful Foundation World Model Odyssey-3 Pro at the same recipe-shape; backbone-anchored complement to SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models SolarWM’s cross-backbone demonstration. Explicitly positions the WFM as the environment a downstream policy interacts with, distinct from the WFM-as-policy line (Dyna-2: A 1-Million-Hour Scaling Law for World-Action Models Dyna-2, Long-WAM: Scaling the Context of World-Action Models Long-WAM); closed-loop policy utility deferred as separate validation.
• [2026-10-21] SuperMimic: Scaling Humanoid Whole-Body Control with One Differentiable World Model: SuperMimic (Liao, Shao, Long, Zakka, Horowitz, Abbeel, Sreenath — UC Berkeley) stakes out a differentiable-dynamics WFM position distinct from both the generative-rollout pole (Cosmos 3, Genie 3, LingBot-World) and the latent-predictive pole (V-JEPA 2.1, LeVJEPA, LeWAM): the WFM is a learned dynamics approximator fit from logged rigid-body-simulator transitions whose load-bearing property is differentiability through short imagined rollouts, used to produce analytic policy gradients via BPTT for a motion-tracking policy. Co-trained end-to-end with the policy, replacing per-motion PPO specialists + distillation. Reports >1000× GPU-hour reduction vs a SOTA motion-tracker baseline (3 h on 1 GPU surpasses 7 days on 128 GPUs; 25/129,514 benchmark failures vs 179; zero failures after 9.8 h on 2 GPUs), physical Unitree G1 transfer across acrobatic motions, and flat world-model prediction error from 1 to 264 hours of motion with GPU-hours-to-target-return growing slowly over the same range. Direct counterpoint to Jiahua Dong — 30,000-hour egocentric world model learns hands cleanly, not objects; neither 100× data nor 4× model closes the gap‘s ~10⁴×-more-compute measurement on 30k-h pixel-space human-video WFMs by moving the dynamics approximator from pixel space into simulator state space where contacts and forces are observed. Sibling in framing to Dyna-2: A 1-Million-Hour Scaling Law for World-Action Models Dyna-2 at the opposite endpoint: Dyna-2 pays for a pixel-level WAM with million-hour human-video scale; SuperMimic pays with simulator-state differentiability at a tiny fraction of the compute.
• [2026-10-21] R²-WAM: Data-Free Continual Learning for World Action Model with Rectified Generative Replay: R²-WAM (Fang et al., UESTC + Shanghai AI Lab + Fudan + SJTU + Tongji + SII) adds a ninth WFM deployment pattern distinct from the eight already on the ledger: WFM as self-regenerator of its own training distribution for data-free continual learning. Before learning task k, the WAM is prompted with language instructions of prior tasks and generates rollouts from scratch; an agentic VLM evaluator truncates at task completion and discards non-successes; a physical-consistency filter scores each rollout by matching generated-action flow (robot-only sim) against 3D-tracker flow on the generated frames; survivors replay against sequential fine-tuning. Reports LIBERO 4-suite 92.7% SR within 1.9 points of real-data ER and vs 12.5% Seq-FT, with no previous-task real demonstrations. Load-bearing new WFM-cluster datapoint: a teacher-forced probe on LIBERO-Spatial as tasks accumulate shows the WAM’s inverse-dynamics channel collapses on prior tasks while video prediction on the same tasks stays visually plausible — asymmetric forgetting between the two WAM channels, with the generative channel surviving long enough to re-seed the discriminative channel. Direct contrast to Argus: An Open-Source Annotator for Robotics Data Argus (VLM-driven audit of real-robot datasets) and Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Model Xiaomi-U0 (WFM-as-data-engine augmenting real demos): R²-WAM uses the WAM to regenerate replacement for the real data, not to augment or audit it. Sibling in channel-ablation framing to EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Data EgoWAM but applied to self-generated past rather than human-video co-training.
• [2026-10-20] Flora: An Open Model Suite for Scaling Robotics Data Engines: Flora (NVIDIA GEAR) is the open-weight annotation substrate behind GR00T training — producing the timestamped captions and dense progress-reward labels needed to filter and balance world/action-model pretraining corpora at 200K hours/day. Load-bearing input to the WAM/WFM training pipeline rather than a WFM itself.
• [2026-10-19] World Action Models are Zero-shot Policies (DreamZero): DreamZero (NVIDIA; Ye, Ge, Zheng et al.) is the founding filed instance of the World Action Model recipe — 14B video-diffusion WFM jointly predicts future world states and actions, with the video channel as the dense substrate for OOD generalization and cross-embodiment / cross-task transfer. 7 Hz closed-loop control at 14B via model+system optimization. Reported >2× generalization over SOTA VLAs on new tasks/environments in real-robot experiments; 10–20 min of video-only cross-embodiment demonstrations yield +42% on unseen tasks. Predecessor to Zero-Shot to New Robots: Introducing Dream-1 Dream-1 and the shared-ancestor citation for the Dyna-2 / LingBot-VA / µ₀ WAM lineage.
• [2026-10-19] Zero-Shot to New Robots: Introducing Dream-1: Dream-1 (Dream Labs) productizes DreamZero’s WAM substrate into a public API with per-arm 6-DoF EE-pose output that reads every bimanual parallel-gripper robot as the same video distribution — zero-shot deployment on 20+ unseen embodiments including PR2 (2008). First filed WAM-as-API deployment surface; FaaS supplies the fine-tune loop when zero-shot falls short. Companion to World Action Models are Zero-shot Policies (DreamZero) DreamZero research result and sibling to Dyna-2: A 1-Million-Hour Scaling Law for World-Action Models Dyna-2 (million-hour-scale WAM substrate on human-video).
• [2026-10-16] Are We Overthinking One-Step Generation?: Priming Models (reconstruction pretraining → distributional post-training, one-step) is applied to action-conditioned world modeling on RT-1 and Bridge-V2 — a datapoint that the simplest from-scratch one-step recipe filed on the wiki extends to robotics WFMs. Complements the DriftWorld thread (which also skips the diffusion teacher but uses the Kaiming-lab drifting objective); Priming’s recipe is strictly simpler and claimed to beat Drifting-L/B by 6–12× in FLOPs on ImageNet, though direct WFM-head-to-head comparisons aren’t reported.
• [2026-10-15] HumanoidToolBench: Benchmarking Humanoid Tool Use from Selection to Mobile Execution: HumanoidToolBench supplies a tool-selection ↔ mobile-execution benchmark where FastWAM (6B WAM) outperforms every VLA (π0.5, Ψ0, GR00T N1.7) on execution — leads 6-of-9 standard-mode cells (e.g. standard L1 Break 86, L2 Break 69) while GR00T N1.7 leads selection (standard L0 77 vs FastWAM 64). The gap widens in decoy mode where same-category tool confusers are added, and persists on real Unitree G1 (FastWAM 5/10 and 4/10 vs GR00T 6/10 and 4/10 BallMove/BallRetrieve in standard mode). Consistent with Dyna-2: A 1-Million-Hour Scaling Law for World-Action Models Dyna-2 and Fast-WAM: Do World Action Models Need Test-time Future Imagination? FastWAM’s position that WAM backbones generalize to deployment regimes VLAs under-perform on — now sharpened to the humanoid + locomotion-coordinated tool-use regime.
• [2026-10-13] Gemini 4 Argon for Robotics: Argon-steering-Omni closes a prompt-rewriting loop around Gemini Omni (video world model) — inspect keyframes from the latest rollout, write the next timestamped <omni_prompt> action, iterate — a new cell on the WFM-steered-by-LLM axis distinct from SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulators SyncWorld’s rank-and-execute best-of-N and from World Pilot: Steering Vision-Language-Action Models with World-Action Priors World Pilot’s dual-path latent/action steering. Argon also authors MuJoCo + mujoco_warp scenes from a single photo / hand-drawn sketch / teleop episode as task-specific physics world models, extending the Argon-as-physics-world-author lineage (SIMPACT: Simulation-Enabled Action Planning using Vision-Language Models, EMPIRIC — agents that write physics-engine extensions as code to learn unknown dynamics EMPIRIC) to the humanoid + 500 Hz dexterous regime.
• [2026-10-13] InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data: InternW0-Δ (Shanghai AI Lab + collaborators) sharpens the Fast-WAM “video co-training as training-side lever” position with mask-level guarantees that no activation path exposes realized future frames to the action expert — Causal Imprint supervises against adjacent clean-latent differences on future frames and a Track4World 4D-aware teacher distills clip-level geometry/motion into cached student queries, both training-only. Unified >20K-h open corpus (robot + UMI + egocentric + Ego2Robot) under a canonical state-action layout; reports LIBERO-Plus 92.8%, RoboTwin 2.0 Clean2Random 71.9%, EBench 66.0, RoboDojo 23.9% (≈2× strongest prior WAM) at 152.8 ms RTX-5090 latency. Directly sidesteps Sitzmann’s action-conditional WFM chicken-and-egg by keeping clean futures and 4D teacher descriptors only in the training objective; sibling in framing to Causal World Modeling for Robot Control (LingBot-VA) (unified causal AR flow-matching) and μ₀: A Scalable 3D Interaction-Trace World Model (frozen 3D-trace WM + action expert) with a dual training-only supervision answer. Code, weights, infra, and processed data (where licenses permit) pledged open-source.
• [2026-10-13] The Will to Act — Natura-1: Natura-1 (Nature Will) stakes a single-sequence-model unification position on the WFM axis — one 50B MoE transformer handles language, vision, visual world modeling, and action generation jointly rather than factoring world model and policy into separate subsystems. Distinct from the WFM-as-peer line (World Pilot: Steering Vision-Language-Action Models with World-Action Priors World Pilot, μ₀: A Scalable 3D Interaction-Trace World Model µ₀, Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generation Qwen-RobotWorld), the WFM-as-policy-backbone line (Direct Video-Action Models — Causal Video Models Are Data-Efficient Robot Policy Learners DVA, RoboJEPA: Scaling Robotic Latent World Models RoboJEPA), and the two-head WAM+BC line (EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Data EgoWAM). Pretrained on 1M hours of egocentric human experience; the blog’s framing is explicitly that generalization emerges from human-video scale paid for by data + training + inference infrastructure co-design, not from a separate world-model module. Reports zero-shot execution of complex language-conditioned tasks (spatial constraints, multi-object relationships, negative conditions) and emergent recovery behaviors (card-overhang graspable-edge creation, rod-as-reach-tool, mid-air wrist rotation) not seen in robot demonstrations — qualitative support for the “implicit world model learned from human video scales to novel physical situations” claim.
• [2026-10-13] Jiahua Dong — 30,000-hour egocentric world model learns hands cleanly, not objects; neither 100× data nor 4× model closes the gap: Jiahua Dong reports a video WFM trained on 30,000 h of first-person Mecka data learns hand motion cleanly but object dynamics much worse — and the gap does not close under either 100× data (300 → 30,000 h) or 4× model (Cosmos 3 Nano 16B → Super 64B), with the measured rate implying ~10⁴× more compute to close it. Modifying the training signal to add extra noise in moving regions raises the object ceiling but object fidelity still trails hands. Pre-training on 30k h beats released Cosmos 3 across all metrics on downstream humanoid transfer. Author’s structural conclusion: contacts, forces, and hidden geometry are never visible in pixels — exactly what simulators represent well — so combining the two may be what’s needed. Sharpest filed single-paper evidence against the pure-pixel-rollout-WFM endpoint of the cluster’s central debate and the first filed scaled human-video-WM-vs-released-frontier-WFM head-to-head on humanoid downstream.
• [2026-10-10] LeWAM: End-to-End World and Action Modeling with JEPAs: LeWAM (Fu/Siebert/Halicki/Balestriero) extends the LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels LeWorldModel two-loss recipe with a flow-matching action head, trained end-to-end from pixels at 17M params on 1 GPU. Reports goal-conditioned visual planning success 28.6%→89.7% on contact-rich manipulation and 10.9%→46.2% on long-horizon tasks vs LeWM (CEM), at 32.3× faster planning (~19 ms/plan). Sharpens the latent-predictive JEPA pole against The Obsessed Encoder — Latent-Space Misallocation in JEPA-Style Self-Supervised Models Enigma’s episode-constant-feature collapse critique: action-regression is an independent anti-collapse signal (without SIGReg LeWM collapses to R²=0 on Push-T probes while LeWAM retains decodable R²=0.86 proprio / 0.90 object). Dynamics-only ablation (CEM over LeWAM latents, action head removed) beats LeWM-CEM on Cube +22.2 pp — the latent-dynamics representation itself is a lever, not just end-to-end action supervision.
• [2026-10-09] In-context Robot Learning Made Simple: A Democratized Recipe for Manipulation Tasks: SimpleICL mounts in-context robot learning on top of a Wan2.2-5B WAM backbone + a 1B action DiT (video DiT LoRA-trained, rest full-parameter) with a 6-token query bottleneck feeding both video and action DiT cross-attention. Ablation isolates future prediction as the dominant mechanism by which the policy actually attends to the demo video (removing it drops RoboTwin 2.0 avg success from 72.1% → 20.2%), reinforcing the EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Data EgoWAM / Modality-Autoregressive World-Action Models ModAR finding that the WAM head is the effective interface for human-video supervision — now extended to the specifically in-context conditioning stream rather than co-training.
• [2026-10-09] UniWAM: Unified World-Action Model: UniWAM folds the world generator in as one of three co-trained heads (alongside a physical reasoner and action predictor) rather than treating it as a frozen prior (μ₀: A Scalable 3D Interaction-Trace World Model µ₀) or a steering peer (World Pilot: Steering Vision-Language-Action Models with World-Action Priors World Pilot). Load-bearing design choice is future visual noise augmentation during post-training to reduce policy reliance on precise future prediction — directly addresses the “WAMs are expensive at inference” bottleneck by weakening the dependency rather than accelerating the generator. Reports a log-linear scaling law for unified human-robot co-training on the WAM-plus-action-plus-reasoner joint objective; Apache 2.0 open weights + code.
• [2026-10-08] Meet Odyssey-3: Our Most Powerful Foundation World Model: Odyssey’s October follow-up to its September Odyssey-3 launch lands the first filed closed-flagship interactive-WFM leaderboard-first on Physics-IQ: Odyssey-3 Pro reports 66.1 V2V (highest reported score) and 54.7 I2V, plus #1 on 3 of 4 WorldMark splits (first-person stylized 77.2, third-person real 79.0, third-person stylized 76.3) under WorldMark’s own captions — but loses first-person real to Lyra 2.0 at 84.4 vs 80.6, with LingBot-World and AlayaWorld top-3 across every first-person split. Also publishes training-recipe disclosure for the first time (multi-step video DiT taught with temporally-resolved prompts → teacher-forced AR extension with causal masking → DMD + adversarial distillation into a few-step real-time variant) and training-data composition (internet video + schema-verified event annotations, gameplay with input traces, simulated rigid-body interactions). New deployment claim: three-camera synchronized autonomous-driving video sequences from an Odyssey-3 checkpoint after only 100 training steps on a target sensor arrangement. Research preview now available (vs. the September “coming weeks”); weights still closed. Pushes the “pretrained video generators are not law-grounded world simulators” ceiling that Video Generation Benchmarks has tracked across PhyGenBench, VideoPhy-2, Physics-IQ, and Apple-π.
• [2026-10-08] S2PD: Serial-to-Parallel Diffusion for Physically and Logically Consistent Video Generation: S2PD argues that bidirectional video diffusion models cannot learn to follow physical laws or symbolic rules even under effectively unlimited in-distribution procedural-generator data, framing causal attention + AR sampling as a necessary ingredient for a world-model-grade video generator. The project also contributes a cheap LoRA retrofit (rank-32) of a pretrained WAN model to causal attention, lowering the cost of turning off-the-shelf bidirectional video DiTs into WFM-grade substrates.
• [2026-10-08] Long-WAM: Scaling the Context of World-Action Models: Long-WAM (NVIDIA + MIT + HKU + UCSD) is a real-time causal WAM built by continued-pretraining LongLive-2.0’s AR video foundation on ~10k window-equivalent hours of robot + egocentric video (RoVid-X, AgiBot World, EgoDex, EgoVerse, VITRA) and then adapting to joint future-video + action prediction. Key WFM-level claim: AR vs. bidirectional pretraining of the video base gates whether context length transfers to downstream policy quality — RoboCasa GR-1 success 63.3→78.7% for AR init over 0→19.2s context, no gain for bidirectional init. SOTA on LIBERO-Long, RoboTwin 2.0, DOMINO; 107.4 ms per action chunk on RTX 5090 including future-video latent prediction; real-robot deployment on Unitree G1 and YAM with 95% success on dynamic cup stacking (vs 0/20 for π0.5 and Fast-WAM).
• [2026-10-07] World Action Learning via Interaction-Centric Spectral Latent Guidance: WING-LAM is a video-based latent-action model pretrained on egocentric human video with the twist that camera motion and interaction motion are explicitly separated at training time (point tracks + hand-object masks → camera warp vs residual interaction displacement, with teacher-student distillation into an RGB-only encoder). Adds a datapoint to the “egocentric video is a WFM substrate but needs structured debiasing before use” position — contrasts with scale-first egocentric WFM recipes that absorb ego-camera noise implicitly.
• [2026-10-06] RealtimeWAM: One-Step Asynchronous World Action Models: RealtimeWAM (Lv, Du, Feng, Yong, Gu, Yang, Gong, Ren, Zhang, Wang; NTU + SenseTime) is the first filed WAM-serving recipe to attack both intra-expert iteration (multi-step action denoising) and inter-expert waiting (sequential video-then-action execution) in a shared MoT backbone simultaneously. Teacher-Anchored Consistency Distillation closes the local–global error gap of one-step action distillation by anchoring the student against a frozen teacher’s multi-step rollout endpoint; Cross-Expert Wavefront Pipelining overlaps the two experts via block-wise sharing of the video KV cache with synchronization deferred until action attention consumes each block. Reports ~25× end-to-end H100 speedup at <1% accuracy drop across LIBERO, LIBERO-Plus, and RoboTwin on both Fast-WAM and Faster-WAM variants — pushes WAM inference into VLA-class latency regimes alongside FLUX 3 Action: A 7B World Action Model for Robot Control FLUX 3 Action’s step-distillation + FP8 stack, with the inter-expert pipelining being a finer parallelism axis than any filed async-VLA recipe.
• [2026-10-05] Rho-1: Collapsing the Multimodal Stack: Rho-1 (Reka Labs) is the sharpest filed form of the “renderer + simulator + planner collapse into one WFM” thesis articulated by A Functional Taxonomy of World Models — one 19B network does interactive real-time video rollout, robotic policy (actions as native continuous tokens), and multimodal dialogue, with “the network predicting future camera observations is the exact network dictating joint actuation.” Direct architectural counter to the NVIDIA Cosmos Reason + Predict + GR00T split and to the frozen-WFM-plus-small-action-expert factorization. Base video rollout is 0.79× real-time with ~6s first-frame latency; 99→8 step distillation moves it into the interactive-steering regime where new instructions update the shared KV cache mid-rollout and the generation stream renders the new trajectory without a cut. Explicitly pairs with Reka’s RIDM (Reka Inverse Dynamics Model for Interactive World Models) for action-labelled web-video pretraining. No head-to-heads against Genie 3 / GWM-1 / Cosmos 3; 672×384 native resolution and 30s structural-drift ceiling, framed as a scaling-headroom claim rather than a finished WFM product.
• [2026-10-03] Reka Inverse Dynamics Model for Interactive World Models: Reka’s RIDM targets a specific blocker to training interactive world models on captured real-world video: the absence of action labels. It proposes a game-trained, optical-flow-bottlenecked inverse dynamics model that pseudo-labels real footage with W/A/S/D-style low-level commands (86% real-world 3-way action accuracy), bootstrapping action-conditioned WAM training without teleop.
• [2026-10-02] EMPIRIC — agents that write physics-engine extensions as code to learn unknown dynamics: EMPIRIC (Yichao Liang) is the first filed instance of a world model written as a readable Python subclass of a physics engine with Bayesian parameter uncertainty — a WFM-as-program branch distinct from both video-WFM and neural-simulator approaches. Also contributes RoboDisco, a 22-environment robot benchmark suite spanning glue/heat/wind/magnets/lasers/hidden-wiring for world-model learning and causal discovery.
• [2026-10-01] Argus: An Open-Source Annotator for Robotics Data: Argus publishes a dataset-level diagnostic specifically for WFM training substrate: 10% of Galaxea Open-World episodes achieve the goal then push it apart before recording ends (teaching goal-conditioned WFMs the wrong terminal state), 11% of MolmoAct2-BimanualYAM is sped-up (training WFMs on incorrect velocities), 1.5% of FastUMI has camera streams swapped (teaching WFMs that one gripper’s motion causes the other’s view to change). Each defect comes with a per-episode fix recipe (trim at Argus-reported goal frame, correct playback speed from leader-follower delay, swap streams). First filed instance of a VLM-driven per-episode quality harness released alongside the full annotation corpus for the WFM-pretraining substrate.
• [2026-10-01] Introducing Praxis-1: Runway announces Praxis-1 — their first open-weight world action model, built on the same large-scale video pretraining backing their Solaris / GWM Worlds 2 general world models. First filed instance of a closed-flagship video-generation lab publicly committing to open weights on the robotics-policy side rather than keeping the WFM/WAM stack closed as DeepMind (Genie 3), Odyssey (Odyssey-3), and Waymo (Genie-3 driving) have done. Pitch: video is effectively limitless (and becoming infinite as generative models reach parity with real video) which makes it the right pretraining substrate for cross-embodiment policies; teleop data cannot cover edge-case-heavy deployments at required volume. Currently testing with Noble Machines + Standard Bots + Ultra on each partner’s own hardware. The r = 0.95 real-vs-simulated policy-evaluation correlation Runway previously reported (see Runway GWM-1 launch broadcast — world model, avatars, audio, multi-shot editing / Runway’s accelerating-robot-policy-evaluation post) is cross-referenced as the evidence for video-WM-as-evaluator — if replicable at Praxis-1 scale, sharpens the RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation / Evaluating Gemini Robotics Policies in a Veo World Simulator / SimFoundry: Modular and Automated Scene Generation for Policy Learning and Evaluation WFM-as-policy-evaluator thread. Launch-signal only — no architecture, parameter count, action-head details, or benchmark numbers; weights deferred to “coming months.”
• [2026-09-30] SIMPACT: Simulation-Enabled Action Planning using Vision-Language Models: SIMPACT (Liu, Yao, Chen, Gao, Mao, Huang, Du) demonstrates a per-scene multi-physics simulator authored on-the-fly from a single RGB-D frame by a VLM orchestrating visual foundation models (GroundedSAM2 for segmentation, an image-to-3D model for mesh reconstruction, FoundationPose for 6-DoF pose) plus VLM-selected physics engines (MuJoCo for rigid bodies, projective-dynamics for stiff deformables, MPM for soft) with VLM-inferred material parameters. Puts an analytic multi-physics simulator on the world-model design surface alongside learned video WFMs — the same VLM then plans against this world model via in-context optimization over rollouts. Extension of the Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agents Agentic Real2Sim orchestration pattern from real-to-sim conversion to closed-loop action planning.
• [2026-09-29] Dyna Robotics announces Dyna-2.1 and Taku humanoid — from tasks to whole workflows: Dyna-2.1 + Taku operationalize the Dyna-2 WAM-as-embodied-policy-substrate thesis (Dyna-2: A 1-Million-Hour Scaling Law for World-Action Models) on a first-principles humanoid: the same 1M-h human-video pretraining prior now feeds a whole-body loco-manipulation controller trained with GPU-parallelized RL in Isaac Sim, with the claim that new whole-body tasks with different motion primitives require only 30 min – few hours of on-robot data. Sibling in framing to Delta Intelligence launches Δ₀ — whole-body humanoid model with latent world-action brain, 69-DoF controller, and 10,000+ h egocentric panoramic pretraining Δ₀’s latent-world-action-brain humanoid launch within the same week.
• [2026-09-28] World Labs is Joining AMD: Market-structure event: World Labs — Fei-Fei Li’s spatial-intelligence lab, publisher of Atlas + Marble and the renderer/simulator/planner taxonomy — signs a definitive agreement to be acquired by AMD (expected close end of 2026). Fei-Fei Li joins AMD as EVP + Chief Scientist reporting to Lisa Su; Justin Johnson and Ben Mildenhall continue leading the World Labs team. Framed as scaling spatial/physical intelligence by “getting closer to the hardware” and committing to an “end-to-end open AI ecosystem spanning hardware, software, platforms, and widely accessible open models.” Reads as AMD’s direct counter-move to NVIDIA’s Cosmos 3 + Physical AI Coalition omnimodal-WFM-plus-hardware bundle. Distinct from the prior organizational framings on the page (Runway Labs portfolio incubator, DeepMind single-product Project Genie, Waymo single-vertical Genie-3 specialization): this is the first filed acquisition of an independent WFM lab by a hardware vendor.
• [2026-09-28] Delta Intelligence launches Δ₀ — whole-body humanoid model with latent world-action brain, 69-DoF controller, and 10,000+ h egocentric panoramic pretraining: Δ₀’s “brain” is a latent world-action model that predicts visual futures and delta-action-conditioned control, paired with a 69-DoF whole-body controller — the WFM+controller factorization applied to whole-body humanoids at 10,000+ h pretraining scale. Sim-to-real appearance/geometry/physics reconstruction shows sim-vs-real dishwasher success parity (86% vs 89%) as training data grows, an unusually clean fidelity datapoint for evaluating WFMs as robot-training substrates.
• [2026-09-26] Training Object Permanence in World Models: WROP (Zhang, Yu, Luo et al.) opens a developmental cognitive-science axis for the cluster distinct from Newtonian-physics (PhyGenBench, VideoPhy-2, Physics-IQ) and implicit-world-rules (RISE-Video): 150 hand-designed object-permanence + solidity tasks across six cognitive categories, procedurally randomized in Blender into a 1.5M-sample training corpus + 300-question exam. Ships a matched 16B continuation model (PWM-WROP) that ranks 1st among continuation models and 3rd overall in a blind pairwise Elo across 14 video models, tied behind two reference-to-video systems — first filed instance in the cluster where a purpose-trained WFM competes head-to-head with frontier video models on a cognition-defined benchmark it was co-designed with.
• [2026-09-25] Contrastive World Models: Contrastive World Models (Bonnie Li, GDM → OAI) replaces Dreamer’s pixel reconstruction with a Deep InfoMax lower bound over local patch features of future observations, drops the pixel decoder entirely, and matches Dreamer + momentum-prediction baselines in the clean setting while substantially outperforming both once distractors or natural-video backgrounds are added. Sibling to LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels LeWorldModel and LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics LeVJEPA on the latent-predictive pole of the cluster’s central debate, but uses a contrastive rather than predictive-MSE objective — a classical answer to the predictable-feature collapse The Obsessed Encoder — Latent-Space Misallocation in JEPA-Style Self-Supervised Models flagged for JEPA-family models on exactly this distractor/natural-background regime.
• [2026-09-23] FLUX 3 Action: A 7B World Action Model for Robot Control: FLUX 3 Action (BFL) is the sibling architectural choice to FLUX 3 x mimic: The Next Generation of Video-Action Models FLUX-mimic on the same FLUX 3 backbone — where FLUX-mimic freezes the backbone and decodes actions from intermediate features (VAM), F3A keeps joint video+action denoising end-to-end (WAM) and preserves the imagination path through a stacked distillation pipeline (guidance distill + step distill + FP8) that hits VLA-class inference latencies. Sharpest datapoint yet on the joint-imagination-vs-VLA-latency debate — beats Cosmos 3 Nano (16B WAM) at 42.92% vs 36.8% RoboLab at less than half the parameters, and step-distilled runs at 32.29 ms/prediction (2.13 s of motion) FP8, competitive with π0.5 BF16 at 31.99 ms while still being a WAM. Also delivers the first filed answer to “does the fast policy’s quality determine whether frontier-LLM+fast-policy hybrids pay off”: Astra + F3A delegation reaches 90% task success at $8.77/success (29$12.28 vs $13.47). Reinforces the “generative-rollout video pretraining transfers to control” side of Sitzmann’s foundation-layer debate — video >95% of pretraining tokens is load-bearing (with-vs-without pretraining takes DROID→RoboLab from <1% to 11.6%).
• [2026-09-23] InstinctFlash — High-Performance Serving Runtime for Robotics Models: InstinctFlash (General Instinct, YC P26, AGPL-3.0) is the first filed serving runtime to treat video-world-model policies (Cosmos3 Edge DROID, Cosmos3 Nano DROID, DreamZero DROID, LingBot-VA) as first-class citizens alongside VLAs (π0.5, LingBot-VLA, GR00T N1.7) under one Runtime API — small but load-bearing datapoint that WAMs / video-world policies / VLAs share enough of a runtime contract to fit inside one planner + six-layer optimization DAG. Jetson Thor P50 speedups vs PyTorch: 3.24× Cosmos3 Edge DROID and 2.13× Cosmos3 Nano DROID under NUMERIC + UniPC4 / CFG3; 1.98× DreamZero DROID under FP8 + 16 steps + BEHAVIORAL-tier dynamic step cache; 5.36× LingBot-VA FP8 25V/50A and 33.78× stacked with 2V/4A distillation. DreamZero’s dynamic step cache is exposed as a first-class BEHAVIORAL-tier option under a strict precision-lattice policy — the first WFM-serving instance where the “how much correctness are you willing to trade for latency” contract is a planner-enforced tier rather than a per-flag toggle. Complements the recipe-side WFM entries with concrete numbers on what edge-serving latency looks like for a Cosmos3 Edge policy actually running on Jetson Thor.
• [2026-09-22] WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memory: WorldCrafter augments a video WFM with a jointly-trained implicit 3D-aware memory: a memory encoder plus a pose-conditioned readout module that produces target-view-specific tokens per query. Contrasts with the explicit-3D-cache branch (SPMem, Mirage) and with in-network-only recipes (Context Forcing, MALT, PackForcing). Streaming minute-scale exploration from single image or text via few-step distillation.
• [2026-09-21] LeRobot — Eidon AI winds down and open-sources 9TB egocentric housework dataset (tracker-pov) with 7-point IMU arm tracking: Additional first-person household video pretraining substrate: 1,274 h / 13,451 recordings / 27 homes with paired 7-point IMU arm tracking, released CC-BY-4.0 as Eidon AI shuts down.
• [2026-09-20] CausalWM: Causal Chain-of-Thought Reasoning for Embodied World Model: CausalWM (Aether Labs) is the second and larger-scale filed instance of ModAR’s modality-autoregressive WAM recipe — a 16B shared diffusion Transformer that stages future prediction as an explicit causal chain (optical flow → 3D pointmaps → future RGB) with a causal-across-modalities attention mask. Reported first place on the TriWorldBench leaderboard (TWB-Score 66.04 vs 65.66 for #2 across 36 published models), 2 firsts + 5 seconds across 19 individual metrics, and SOTA on PAI-Bench robot-domain language-conditioned generation under a 4/4/4 denoising-step budget. Trained on ~31K source hours (~20K retained) mixing human egocentric video, real-robot data, and simulation. Sibling to Modality-Autoregressive World-Action Models on modality ordering; the fact that the same load-bearing recipe (autoregressive across modalities) tops PAI-Bench-family leaderboards at 16B is the sharpest scale-side evidence yet that ordering is not an incidental design choice.
• [2026-09-18] PointZero: 3D Point Track Completion for Learning Transferable 3D Dynamics: PointZero (Duisterhof, Zhang, Hung, Wen, Birchfield, Li, Ramanan, Ichnowski — CMU + NVIDIA + Columbia) stakes out a third-interface position between pixel-rollout WFMs and latent-predictive JEPA WFMs: 3D point track completion as the pretraining objective. Given RGB-D + a few complete 3D context tracks, predict dense future 3D tracks of all observed points; supervision is robot-free by construction (labels are just 3D tracks). A 2.9M synthetic-frame corpus + Perceiver-IO/DINOv2 + denoising DiT (flow matching, JiT, and regression compared) yields a checkpoint that post-trains two ways: (a) action-conditioned dynamics on the PGND real-robot benchmark (best on 4/6 scenes; from-scratch collapses 2–3×), and (b) imitation learning with a small action head trained on 20 labeled + 100 actionless demos per task (best or joint-best on 6/7 sim + real manipulation tasks, beats DP3 on all 7). Sibling to µ₀’s frozen 3D-trace WM but with tracks as the pretraining supervision rather than a downstream interface; concrete partial vindication of Sitzmann’s generative-rollout thesis at the 3D-track granularity rather than pixels.
• [2026-09-16] Modality-Autoregressive World-Action Models: ModAR (CMU; Hung, Duisterhof, Ramanan, Ichnowski) is the first WAM to autoregressively denoise across modalities rather than in parallel or from a single fused representation — depth → DINO → point tracks → optional RGB, each conditioned on previously generated modalities. Sibling to Flex-π: A Multi-Stream World-Action Model with Compute Flexibility Flex-π (shared-latent parallel denoising of RGB+3D+DINO) and EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Data EgoWAM (single-target ablations across pixel-VAE / DINO / 3D-flow); adds the missing ordering axis. Empirical finding aligns with EgoWAM’s ranking: RGB is not a consistent contributor while point tracks, DINO, and depth all are. Trained from scratch, matches fine-tuned Flex-π at ~20× fewer FLOPs and no pretraining (75% vs 72% observed avg on the same data), a small-scale datapoint on the “video-model pretraining is a crutch” side of the Florence vs Sitzmann debate. Human-video co-training gives further improvement on three real-world bimanual tasks.
• [2026-09-15] Introducing Odyssey-3: A General-Purpose Physical Intelligence: Odyssey-3 launches as the third entry in Odyssey’s WFM line — an autoregressive diffusion transformer positioned as a single general-purpose backbone deployed across robot arms, humanoids, cars, drones, video games, and generated training environments via the frozen-WFM + small action decoder recipe. Third-party partners announced: Poke & Wiggle (cross-embodiment/viewpoint/control benchmarking) and Flexion (humanoid loco-manipulation, reportedly beating tested VLA baselines on lighting generalization with only tens of hours of teleop data). The cleanest numeric datapoint: simulation-only-trained driving policies reach ≈77% of the between-intervention distance of policies trained on real Indian-roads footage using 20 h of sim data. Weights not yet public (“in the coming weeks”), no benchmarks vs Cosmos 3 / Genie 3 / GWM-1.
• [2026-09-15] Odyssey teases 'World Models, part 3' launch tomorrow: Odyssey teases “World models, part 3” for release the following day — third entry in their interactive world-model line after Odyssey-2 Max and Agora-1.
• [2026-09-11] LiteReality-Agent goes simulation-ready — Layout Agent, articulated objects, MuJoCo integration, GPT-6 small-object recovery: LiteReality-Agent’s MuJoCo export turns RGB-D room scans into runnable physics simulations — an alternative substrate to WFM rollouts for downstream robot policy training and evaluation.
• [2026-09-10] Rhoda AI — Web-video pretraining scaling laws for robot policy learning (DVA): First public scaling law for the “video model as the policy” framing: Rhoda AI’s DVA scales real-robot task performance monotonically with both model size (4% → 85% across XS/S/M/L) and pre-training compute on general web video, with DINO FD as a pre-training-time proxy for post-training robot quality. Concrete evidence that action-free web-video prediction scales cleanly into a deployed policy signal.
• [2026-09-10] SyncWorld: Visual Calibration Enables World Models as Zero-Shot Simulators: SyncWorld (UMass/MIT/Tsinghua) introduces visual calibration episodes as an in-context adaptation mechanism for action-conditioned WFMs — a short paired video–action clip that exercises each controllable DoF lets a single pretrained DiT serve as a zero-shot simulator across unseen camera views / environments / robot embodiments (xArm, ARX5, KUKA, Franka Panda, UR5e) without any parameter update, and further supports training-free test-time policy improvement by re-ranking VLA-proposed action chunks against a VLM judge.
• [2026-09-09] Robot-Use Agents: Phillip Isola frames the “robot brains tech stack” — world models, behavior foundation models, continual learning — as an immature alternative that competes with cloud-LLM-as-puppeteer, arguing the varied bets in this space don’t yet have the LLM-tech-stack buildout that makes robot-use agents deployable at ecosystem scale today. Sharpest external framing yet that the WFM line is a slower deployment path than robot-use agents, orthogonal to any recipe-level WFM comparison.
• [2026-09-08] EgoWAM: World Action Models Beyond Pixels with In-the-Wild Egocentric Human Data: EgoWAM (Georgia Tech RL2) is the sharpest controlled study filed so far of what world-model target transfers under WAM co-training. Fixing the HPT backbone, action head, horizons, and data mixture and swapping only the world-prediction target across pixel VAE, DINO features, and camera-stabilized 3D motion flow across three real bimanual tasks (1,800 rollouts), pixel-VAE targets transfer weakly, DINO drives 4× OOD object/scene generalization, and 3D flow drives 20–30% in-domain spatial gains. Names three desiderata for the target — appearance abstraction, cross-embodiment consistency, ego-motion factoring — with pixel-VAE violating all three, DINO satisfying two, 3D flow satisfying all three by construction (Aria VIO poses fold into the 3D point tracker to camera-stabilize the flow signal). Sibling to Flow Matching in Feature Space for Stochastic World Modeling FlowWM and Back to the Features: DINO as a Foundation for Video World Models DINO-world in the DINO-target camp; adds the comparative datapoint those lack.
• [2026-09-08] OpenWAM — Extensible framework for video-action world models in robot learning: OpenWAM (Stanford SVL, AGPL-3.0 v0.1.0 pre-release) is a shared training + evaluation framework for video-action world models organized around a fixed five-contract composition boundary (ExperimentConfig → VariantPipeline → VisualTower → PolicyVariant → ActionDecoder), three maintained topologies, and six standard program selectors plus GJD — designed to make imagine-vs-no-imagine and program-schedule ablations (Fast-WAM: Do World Action Models Need Test-time Future Imagination? Fast-WAM style) routine on one codebase rather than requiring per-paper forks. Also ships reproducibility infrastructure (versioned result envelope with source state, argv, config hashes, checkpoint identity, dataset metadata, package versions, device details) from day one; parallel_stream runtime is explicitly LingBot-compatible (LingBot-VA 2.0: Native Video-Action Pretraining for Generalizable Robot Control).
• [2026-09-07] Lingxiao Guo — GPT-6 does end-to-end real2sim from multi-view RGB + robot actions: Lingxiao Guo’s GPT-6 real2sim demo is the frontier-model zero-shot counterpart to Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agents — end-to-end camera calibration + asset construction + MuJoCo system-ID + Blender rendering from RGB + action logs, no framework code, with the same rigid-simulator ceiling on compliant contact.
• [2026-09-04] ACE-Data-0: Human-Centric Ambient Capture as Embodied Data Engine: ACE-Data-0 (ACE Robotics + S-Lab NTU) contributes a synchronized ego + exo + tactile + audio + motion capture substrate designed for WFM training — every stream on one 60 Hz clock and one world frame, with measured (not estimated) object pose and contact, addressing the label-noise problem that constrains egocentric-video-driven WFM pretraining.
• [2026-09-03] SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models: SolarWM (Huang et al., MMLab@NTU + collaborators) is the first filed open interactive-WFM release that stacks the same three-stage recipe (bidirectional adaptation → teacher-forced AR init → DMD) across three unrelated video backbones (Wan2.2 5B/33B, LTX-2.5, MiniMax-H3) under shared camera-conditioning and inference interfaces — a backbone-agnostic answer to the recipe-vs-backbone entanglement other open WFMs (LingBot-World, RynnWorld-4D, AlayaWorld) address per-backbone. Ships a 1.43M-clip / 25 TB unified data engine (10 sources normalized to a frame-aligned contract) alongside training/inference code and four checkpoints; reports minute-to-hour rollouts from 5s-only training. Direct open-recipe counterpart to Genie 3 / Runway GWM-1 and complementary to H3-World: Turning Language Understanding into World Control (LoRA-adapts one of the same backbones).
• [2026-09-02] C³ache: Accelerating World Action Models with Cross Inference Chunk Cache: C³ache targets the inference cost of World Action Models specifically, arguing the imagine-then-execute paradigm’s expense is a first-class deployment problem worth its own systems technique. Empirically characterizes cross-chunk redundancy in Fast-WAM’s action expert (residuals at matched denoising steps are highly correlated across consecutive chunks under smooth robot behavior) and delivers up to 2.21× wall-clock speedup on LIBERO / 2.03× on RoboTwin 2.0 training-free, with negligible task-success degradation.
• [2026-09-02] H3-World: Turning Language Understanding into World Control: H3-World (Chen, Wang, Lin, Yang, Jin) turns the 33B MiniMax-H3 video generator into an interactive world model without a dedicated action module — 8K gameplay samples, 10K LoRA steps, 0.199% trainable parameters, temporal attention routing to bind language actions to their time intervals. First filed downstream repurposing of the H3 open-weights base (MiniMax H3 — Open-Weights Omni-Reference Video-Audio Generation Model Card) and a concrete instance of the Sitzmann thesis: a strong pretrained video generator already carries the control-relevant language structure, and adaptation only needs to time pre-existing conditioning, not create a new modality. Sharpens the frozen-WFM-plus-tiny-adapter line (µ₀, FLUX-mimic) by pushing the action interface entirely into the backbone’s existing language channel.
• [2026-09-01] Atlas: A World Model for Spatial Intelligence: Atlas (World Labs) is the first public instantiation of the A Functional Taxonomy of World Models renderer+simulator unification argument — a multimodal AR diffusion transformer pretrained from scratch on text/images/video/3D that emits both pixel views (renderer channel) and explicit 3D as point clouds / 3D Gaussian splats (simulator channel) from one architecture. Load-bearing primitive is a spatial context: each input image and depth map is bound to a 6-DoF camera pose, so AR conditioning lives in 3D coordinates. Reports beating specialized open-source 3D-reconstruction models on sparse-view benchmarks under a shared protocol, and outperforming recent video models on human-rater cinematic camera-motion adherence (with the gap widening on complex trajectories). Robotics real-to-sim workflow (24-frame cell-phone-video → simulate arbitrary robot embodiment/trajectory with matched RGB+depth) is the operational counterpart to Building Worlds That Train Robots — Real-to-Sim-to-Real (R2S2R) as a Scalable Engine for Training and Evaluating Robot Policies‘s R2S2R engine. First unified WFM on file that combines Genie-3-scale interactive video quality (1-min at 1440p) with native explicit-3D output at Marble-format Gaussian splats. Sharpest architectural counter to the Camera-Controlled Video Diffusion frozen-Wan-plus-adapter recipe: rather than injecting camera geometry through an adapter on a frozen backbone, Atlas rewrites the sequence primitive so camera pose is a first-class AR conditioning element.
• [2026-08-30] LaGSplat: Inferring Physics-Governed Interactive Simulation from Monocular Video Using Latent Lagrangian Gaussian Splatting: LaGSplat frames the WFM problem as an interactive differentiable simulator rather than a generative rollout: from a few seconds of monocular video it learns a dissipative Lagrangian in a shared latent q that also conditions a Gaussian-Splatting decoder, so user-applied forces (never measured or seen in training) enter the equations of motion via the Gaussian-primitive Jacobian pullback and get rendered in real time.
• [2026-08-29] PAWBench: How Far Are We from Probabilistically Aligned World Modeling?: PAWBench (Pu, Zhuo, Paul et al.) sharpens the “video generator as world model” definition from single-sample plausibility to distributional reproduction of valid physical behaviors under fixed conditioning. Across 11 systems on 50 scenarios, no model simultaneously covers the valid-outcome support and matches reference probabilities — surface plausibility (per-frame) decouples from distributional alignment (per-histogram). Complements FlowWM (Flow Matching in Feature Space for Stochastic World Modeling)‘s training-side stochastic-WM argument with the missing distribution-level evaluation shape; lands the same “current video models fail as physical simulators” thesis at Physion-Eval and RISE-Video, but along a new axis (distribution vs single-video).
• [2026-08-28] LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics: LeVJEPA (Kuhn, Maes, Serra, Le Lidec, LeCun, Balestriero, Buettner — DKFZ + Mila + Brown + NYU + AMI Labs) ports the LeJEPA / LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels two-loss (invariance MSE + SIGReg) recipe to video encoders, matching or exceeding V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning across ViT-S/B/L at 5.6×–20.8× less pretraining compute on identical data, and beating a compute-matched DINOv2 on Something-Something-v2 by ≈2× (30.4 vs 16.9) while trailing by 3.1 on ImageNet-1K — the sharpest filed head-to-head where a video-pretrained JEPA encoder finally overtakes an image-pretrained encoder on motion at equal FLOPs. Two structural results worth flagging: (i) 95% uniform random token dropping improves accuracy monotonically (33.9 → 47.6% IN1K), and (ii) block-causal attention comes at no accuracy cost, enabling KV-cached video encoders for downstream world-modeling.
• [2026-08-27] Zero-WAM: In-Context World-Action Modeling from Human Videos for Open-Ended Task Generalization: Zero-WAM extends the causal video-action WAM lineage (Causal World Modeling for Robot Control (LingBot-VA) LingBot-VA) with an in-context human-video conditioning stream + an in-context future chunk prediction (IFP) objective; reports +29.5 pp over LingBot-VA on 7 held-out RoboTwin 2.0 tasks — evidence that WAMs benefit substantially from the visual task-specification interface even under a matched backbone.
• [2026-08-27] Gemini Omni 1.1 Flash lets you build with more control: Gemini Omni 1.1 Flash — production update to the DeepMind closed-flagship WM product surface, adding chunked scene extension (10s of prior context, extendable to 40s total) alongside keyframe and video-reference conditioning; no benchmark numbers, no architectural detail.
• [2026-08-27] V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning: Technical report behind V-JEPA 2 — the foundational latent-predictive WFM; 1B ViT-g on VideoMix22M with 3D-RoPE and progressive-resolution cool-down, then a 300M action-conditioned predictor for zero-shot latent-MPC on Franka arms from 62 hours of DROID.
• [2026-08-26] Isaac 0.5: An Open-Weight Embodied Foundation Model: Isaac 0.5’s future-percept prediction is a WFM-style self-supervised video objective narrowed to task-relevant semantic content — predicts whether an object is grasped, a drawer is open, or a contact is about to occur, not future pixels. Targets are constructed automatically from future video observations without human annotation or action labels; construction and loss remain proprietary. Trained jointly with autoregressive/FAST action loss and continuous-action Flow Matching on 1M h general video + egocentric + UMI + robotics, and drives the 210× video-vs-teleop exchange claim. Related to but structurally different from Causal World Modeling for Robot Control (LingBot-VA) LingBot-VA’s unified causal AR flow-matching (Isaac keeps a shared representation but separates the continuous-action DiT route) and from Dyna-2: A 1-Million-Hour Scaling Law for World-Action Models Dyna-2’s video-only WAM (Isaac uses video as supervision on the shared representation rather than as the policy backbone itself).
• [2026-08-25] Introducing Index: Building The World's Largest and Most Diverse Physical Dataset: Figure Index positions itself as the world’s largest useful robot pre-training dataset via app-crowdsourced human video, with a controlled per-1,000-hour diversity target (373 tasks / 1,146 objects / 116 environments) that contrasts with quantity-first web-scraped approaches like Dyna-2.
• [2026-08-25] Accelerated Understanding — Physical AI for 4D Simulation: Accelerated Understanding company manifesto — stakes out a scientific-simulation-first position within the WFM cluster, distinct from generative-rollout (Sitzmann), latent-predictive (V-JEPA), and renderer/simulator/planner taxonomy (Fei-Fei Li) framings. Design contract: full-trajectory (non-autoregressive) 4D prediction, resolution-invariant, cross-physics-trained, with a directional-improvement head enabling closed-loop hill-climbing over engineering designs. Reported scale: hundreds of runs at up to 1T params, scaling experiments to 35T, up to 1T-token training / >5T-token inference context, 2–6 PB per average run. No technical details, benchmarks, or paper reference disclosed — filed as market/positioning marker. Founder lineage: FourCastNet (large-scale AI weather).
• [2026-08-25] Figure launches Index — crowdsourced app-based real-world video dataset for humanoid robots: Figure launches Index — an app-crowdsourced real-world video collection pipeline (16M+ videos, 30 min/s ingest across 108 countries) explicitly framed as the data substrate for training robot foundation models; the productized-collection counterpart to Dyna-2’s 1M-hour academic scaling law.
• [2026-08-21] Lightwheel AI open-sources EgoSuite-Open100K — 100K-hour egocentric human dataset with hand and body pose: Lightwheel AI’s EgoSuite-Open100K adds a 100K-h annotated egocentric human corpus to the workplace-domain pretraining substrate pool (factory floors, retail backrooms) — sibling to Build AI’s Egocentric-1M and Reka’s RekaDaily-10k but with ground-truth hand/body pose + subtask semantics instead of raw video.
• [2026-08-21] Hydra-0: Action Flow for Generalist World Modeling and Control: Hydra-0 (NVIDIA + Brown/Columbia; Li, Wen, Zhu, Wang, Du, Li, Konidaris, Birchfield et al.) makes pixel motion (action flow) the shared visual interface for a generalist world model — same signal semantics across embodiments, tasks, environments, and video-generation backbones. Reports 90.4% / 60.2% lower robot- / object-motion error vs an action-conditioned baseline and r = 0.96 replay-vs-reference success on RoboLab, joining RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation (r=0.989) and DriftWorld: Fast World Modeling through Drifting (r≥0.92) on the WFM-as-policy-evaluator axis. The inverse mode — predict compatible robot motion from a desired object flow lifted from a human demo, with a trained action head decoding to executable actions without task-specific expert robot demos — is a new interface primitive that folds LUCID-style object-flow-as-intent inside the same video-generation backbone.
• [2026-08-20] Going Beyond World Models & VLAs: Pete Florence (Generalist AI) explicitly names the early-2026 “world model moment” as a bandwagon and refuses the WFM label from the physical-interaction-foundation-model side — arguing that video-model pretraining, like VLM pretraining, is a “crutch” for robotics data scarcity that becomes optional at Generalist’s >500K-hour scale, and that ~99% from-scratch parameters is the correct answer once the data constraint is removed. A meta-position piece rather than a technical contribution — no benchmarks — but the sharpest filed articulation of the “don’t build a world model, build a native foundation model for physical interaction” framing.
• [2026-08-19] URDFormer: A Pipeline for Constructing Articulated Simulation Environments from Real-World Images: URDFormer (RSS 2024) is the 2024 predecessor of the video-driven real-to-sim WFM wave (SimFoundry, R2S2R, SAGE): a transformer maps a single RGB image to a URDF (bounding boxes, kinematic hierarchy, joint types), with paired training data generated by inverting a text-to-image diffusion model rather than reconstructing from video. Frames “generative simulator with articulated structure” as a WFM position that predates the reconstruction-based twin line, and pairs cleanly with SimFoundry’s cousin-generation recipe as the “how do you get paired data” step it left implicit.
• [2026-08-18] HarnessEval-W: Agentifying the Evaluation of Visual Worlds: HarnessEval-W (MirroS + collaborators) is a new benchmark-side entry: an agentic, tool-using evaluation harness that decomposes each case into sub-agent-owned subproblems and returns an evidence tree rather than a scalar. Applied to 18 world models across 330 cases (18 labs covered per launch tweet — ByteDance, Alibaba, Kuaishou, MiniMax, NVIDIA, Tencent, Ant Group, Lightricks). Live community-extensible benchmark alongside WorldModelGym, WMBench, and PAI-Bench in the “evaluator infrastructure for WFMs” thread.
• [2026-08-15] FACT: Failure-Aware Causal Training for World-Action Models: FACT (Peng et al., UCSD Xiaolong Wang group) adds a supervision-distribution axis to the WAM training-recipe board — complementary to Dyna-2’s video-co-training-on-unlabeled-human-video axis and Fast-WAM’s imagine-vs-no-imagine axis. Reorders the WAM interface from video-first + IDM-recovery to action-first + video-conditioned-on-action, so failed rollouts become legal supervision targets rather than being discarded. Sharpest quantitative form yet of the “success-biased hallucination” failure mode: a success-only WAM given a failed action still imagines success; FACT lifts failure-future PSNR 19.51 → 25.92 dB while preserving success-future prediction. Failure-fraction scaling law (32.7 → 44.7 → 57.3% at 0/50/100%) suggests the axis has headroom rather than saturating.
• [2026-08-15] Simulation Pre-training for Dexterity (SPD): SPD uses classical physics simulation (MuJoCo at 480 Hz with tuned contact parameters, Madrona-adapted parallel rendering) as the substrate for a human-teleoperated dexterous data engine — a counterpoint to neural-WFM data pipelines: rather than generating video/action rollouts from a learned world model, put a human into the physics simulator via VR and collect on-embodiment action-labeled trajectories. Six curated scenes with tuned contact parameters + the spd-vr VR teleop software are released, positioning classical-sim + human-in-the-loop as an alternative to WFM-generated synthetic data for the dexterous-manipulation regime.
• [2026-08-14] H2R-Bench: Benchmarking Human-to-Robot Manipulation Video Generation in World Models: H2R-Bench probes whether video world models can act as human-to-robot data engines — the promise that video WMs synthesize robot-centric manipulation clips from human observations. Evaluation of 11 SOTA video generators finds current WMs remain limited on embodiment consistency, functional interaction, and task execution, sharpening the case that surface video fidelity is not sufficient for the WFM-as-data-engine claim.
• [2026-08-14] DreamX-Phi 1.0: Action-Conditioned Video World Model for Robotic Manipulation: DreamX-Phi 1.0 (Alibaba DreamX Team) is an action-conditioned bimanual-manipulation video WM that routes per-arm SE(3) transformations through attention via a PRoPE-style relative encoding (the same primitive Cameras as Relative Positional Encoding uses for cameras), adds a depth branch + SAM3 masks distilled from a frozen V-JEPA teacher for scene/object faithfulness, and is compressed to a few-step student via DMD. Placed 1st on Track 1 and 2nd on Track 2 of the WorldArena 2.0 Challenge — datapoint that a smaller specialized diffusion WM with structured geometric attention is competitive with monolithic AR omnimodel entries like Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Model on the same benchmark.
• [2026-08-12] Flex-π: A Multi-Stream World-Action Model with Compute Flexibility: Flex-π (UW + Allen AI; Yan et al.) sharpens the WFM-as-policy-internal-substrate pattern with the first filed instance of runtime compute flexibility: a 6B shared-latent-space model jointly denoises RGB + 3D pointmap + DINOv3 semantic futures with actions, and per-stream latent dropout + cross-modality forcing produces 56 deployable input/output combinations from one checkpoint — a live speed–accuracy dial (60 ms action-only vs 193 ms full joint on RTX 5090) rather than per-configuration checkpoints. Structural encoder economy: the same frozen Wan-2.2 VAE encodes both RGB and 3D pointmaps, avoiding a geometry-specific encoder; DINOv3 semantics enter through a single linear projection. Ablation isolates cross-modality forcing as the load-bearing training-recipe piece (+47% relative on RoboTwin). Real-world YAM: 83.0% avg full joint on 5 tasks in-distribution vs strongest baseline 58.0; OOD gap widens to ~2.4×. Complements µ₀’s frozen-3D-trace-WM + small action expert and LingBot-VA’s unified causal AR flow-matching by keeping multiple latent future modalities jointly trained in one denoiser while making the modality set a runtime choice.
• [2026-08-12] Fast-WAM: Do World Action Models Need Test-time Future Imagination?: Fast-WAM (Tsinghua) is the sharpest controlled ablation to date of “does the WAM benefit come from the video-prediction training objective or from test-time future imagination?” — builds a shared-framework MoT WAM on Wan2.2-5B and instantiates two representative imagine-then-execute variants (Joint / IDM) plus a no-video-co-training control. Fast-WAM matches SOTA WAMs without embodied pretraining (97.6 LIBERO / 91.8 RoboTwin), stays within 1 pp of imagine-then-execute variants, but drops 4–8 pp when video co-training is removed — the video-co-training gap dominates the imagine-vs-no-imagine gap. Runs at 190 ms (>4× faster than IDM’s 810 ms). Small-scale, ablation-first counterpart to Dyna-2: A 1-Million-Hour Scaling Law for World-Action Models Dyna-2’s >1M-hour scaling law — same directional finding (video co-training is the lever, not test-time future generation) at ~1000× less data with much sharper isolation.
• [2026-08-10] Dyna-2: A 1-Million-Hour Scaling Law for World-Action Models: Dyna-2 (Dyna Robotics) is the largest filed WAM-as-embodied-policy pretraining run — >1M h egocentric human video, MoT video-diffusion architecture — and delivers the first cross-embodiment transfer scaling law: same human-only checkpoints show monotone offline-metric improvement on 39 held-out YAM robot tasks across four orders of magnitude of pretraining data with an inflection at 10k–100k h. The load-bearing objective finding for the WFM cluster: action-only supervision loses on 39/39 tasks at every scale, and joint denoising + video co-training on unlabeled human video is the specific lever that makes the transfer scale with data — sharpens the Sitzmann “video-generative-rollout as pre-training objective” thesis with a controlled ablation, and offers zero-shot deployment quality/throughput 87% vs the production VLA Dyna-1’s 46%.
• [2026-08-08] MiniWorld: Democratizing the Training of Video World Models from Scratch: MiniWorld is a from-scratch reproducible baseline (block-causal DiT + Flow Matching + Diffusion Forcing + rolling KV cache) trainable in a few days on 8 GPUs, filling the lightweight-baseline gap in the AR video WFM design space.
• [2026-08-07] World-to-Wrist: Task-Conditioned Future Wrist Modeling for Fine-Grained Robot Manipulation: W²-VLA narrows the “predict a latent future, then act” WAM recipe to just the wrist ROI in a frozen V-JEPA space, arguing that action-proximal, wrist-local foresight is a cheaper and more decision-relevant substitute for whole-scene video prediction on fine-grained contact-rich tasks.
• [2026-08-06] RekaDaily-10k: Collecting 10,000+ Hours of Egocentric Household Manipulation Data: RekaDaily-10k releases 10,312 h of unscripted first-person household video (Apache 2.0) as an open pretraining substrate for household-scoped video WMs — the training-side complement to Reka’s WorldModelGym evaluation release.
• [2026-08-04] Assemble Bench and CG-DAgger — NIST small-parts assembly benchmark on Isaac Lab Arena + code-guided DAgger with zero teleop: Cosmos Nano-DROID (an NVIDIA Cosmos-family world-action model) fails zero-shot on HUD’s DROID sim benchmark alongside π0.5-DROID — third-party evidence that frontier WAMs still struggle on hard contact-rich manipulation without task-specific post-training.
• [2026-08-03] MiniMax H3 — Open-Weights Omni-Reference Video-Audio Generation Model Card: MiniMax H3 model card documents an omni-modal generative system that ingests text + image + video + audio references and produces 15-second joint video+audio at up to 2K — Ref2VA supports mixed-modality reference conditioning (≤9 images, ≤3 video clips, ≤3 audio clips, ≤12 total files) via mask-driven inpainting. Positioned as a generation-side counterpart to interactive WFMs, without the closed-loop action interface.
• [2026-07-31] PhiZero: A World Model Built Around Physical Language: PhiZero splits a video WFM into an AR reasoner over discrete “physical language” tokens plus a diffusion renderer, arguing pixel-space WFMs leave world dynamics implicit; leads IQ-Score / Physics Score / IntPhys2 and enables zero-shot cross-embodiment transfer by reusing the transition sequence with a new first frame.
• [2026-07-31] Towards Video World Models: Xun Huang (Luma, Self-Forcing co-author) position essay proposing the sharpest filed specification for video world models — five required properties (causal, interactive, persistent, real-time, physically accurate) in dependency order, with the load-bearing structural argument that non-causal diffusion models are architecturally incapable of real-time latency regardless of throughput (a t-second-window non-causal generator has minimum latency t). Independently maps the internal-vs-external WM split (aligned with the LWM/JEPA vs simulator poles this page tracks) and stakes out a moderate hybrid taxonomy for external substrates (video WMs + physics engines + neural 3D/4D + composed recipes like WonderPlay) — a middle position between Fei-Fei Li’s renderer/simulator/planner functional split (A Functional Taxonomy of World Models) and Sitzmann’s “explicit 3D should dissolve” (The flavor of the bitter lesson for computer vision).
• [2026-07-30] Gemini Robotics 2 brings whole body intelligence to robots: Gemini Robotics 2 is the closed-flagship embodied-policy layer above the Gemini backbone — direct successor to Gemini Robotics 1.5, now with whole-body humanoid control (Apollo 2 walking + manipulating), 22-DoF five-fingered dexterity, ER 2 tracking multi-minute / hundreds-of-decisions horizons with explicit begin/end + key-event detection, and multi-robot orchestration across heterogeneous robots. On-Device 2 adapts to new bi-arm embodiments in a few hours with <200 examples via inherited “motion transfer.” No benchmark numbers in the blog; safety evaluations deferred to the Safety Technical Report.
• [2026-07-30] The Obsessed Encoder — Latent-Space Misallocation in JEPA-Style Self-Supervised Models: Enigma’s second research drop delivers the sharpest filed critique yet of the latent-predictive JEPA WFM pole. Reproduces a latent-space misallocation collapse in DINOv3, LeJEPA, and LeWorldModel under a matched-control template where a ≤12-bit predictable feature dominates the entire embedding while SIGReg / KoLeo / Sinkhorn-Knopp all report healthy statistics. On PushT-RandGoal (goal pose randomized across episodes — a natural real-robot setup where the goal is an episode-constant on-screen feature) LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels fails outright: prediction loss drives below the healthy baseline while planning success stays at chance. Existing filter-based defenses cannot recover because the “distractor” is the actual goal. Cited authors’ own follow-up stable-worldmodel: A Platform for Reproducible World Modeling Research and Evaluation noted LeWM’s mild-variation failures without an explanation; this post gives one. Complements Haiyu Wu — critique of reachability-augmented CEM planning in JEPA world models (with VISReg counter-example)‘s independent VISReg-swap counter-example on the same LeWM stack — together the two critiques land on the same open question: whether latent-predictive WFMs survive the low-entropy episode-constant features that dominate real robot observations.
• [2026-07-29] FilmBench: A Film-Grade Benchmark for Cinematic Video Generation: FilmBench benchmarks 9 T2V and 7 R2V models on a film-academy-designed rubric of 1,169 prompts (90.3% multi-shot) reverse-engineered from award-winning films — evidence that closed and open video foundation models still score well below web-benchmark levels on professionally-judged cinematic craft, with a single-to-multi-shot performance drop that widens for weaker models.
• [2026-07-29] Kaolin — A PyTorch Library for Accelerating 3D Deep Learning Research: Kaolin (NVIDIA, Apache 2.0) is the production PyTorch stack for representation-agnostic 3D deep learning — ships Simplicits + FreeForm physics, first-class 3DGS containers, differentiable rendering, and a USD physics schema for round-tripping simulation-ready assets.
• [2026-07-29] Simplicits: Mesh-Free, Geometry-Agnostic, Elastic Simulation: Simplicits (SIGGRAPH 2024) is the seminal representation-agnostic physics simulator — one elastodynamics engine that runs atop meshes, SDFs, point clouds, neural primitives, radiance fields, 3DGS, and volume meshes via a common occupancy-function interface.
• [2026-07-29] FreeForm: Reduced-Order Deformable Simulation from Particle-Based Skinning Eigenmodes: FreeForm (NVIDIA, CVPR 2026) is a mesh-free reduced-order elastodynamics method for meshes / point clouds / 3DGS that replaces Simplicits’ per-shape neural-field basis with an RKPM eigensystem solve, giving ~40× faster training and lower FEM error — a concrete physics-simulator backend for the “simulator-category” branch of the WFM taxonomy.
• [2026-07-28] GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation: GigaWorld-1 (GigaBrain / GigaAI) is the first filed systematic study of what makes a world model a good policy evaluator — a controlled ablation across 7 WFMs × 4 action encodings × 324K matched real/sim rollouts, plus the WMBench benchmark and a purpose-built model. Delivers a first empirical answer to this concept’s long-standing open question about which axis of WFM-ness matters for downstream use: long-horizon action-faithful rollout consistency dominates short-term visual realism; pretraining gains come from balancing general world knowledge with robot-specific controllability, not scale alone; action encoding + memory design + evaluator-focused post-training determine real-world alignment. Complements the single-model WFM-as-evaluator entries (RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation, Evaluating Gemini Robotics Policies in a Veo World Simulator, SimFoundry: Modular and Automated Scene Generation for Policy Learning and Evaluation) with the missing evaluation-of-evaluators layer.
• [2026-07-28] 1XWM: From Video to Action — 1X's video-pretrained world model as a NEO robot policy: 1X reframes 1XWM from policy evaluator (1X World Model (1XWM) — action-controllable world model for humanoid policy evaluation) to policy: a 14B text-conditioned video WM rolls out ~5 s futures on NEO, and a Depth-Anything-conditioned IDM (not a single DiT à la DreamGen) extracts robot actions. Reports two-handed coordination + human-interaction emerging from egocentric-human mid-training despite 98.5% of NEO post-training being single-hand pick-and-place, best-of-8 test-time compute improving success on ambiguous prompts, and a Scrub-dish real-world ablation where caption-upsampling and egocentric mid-training are jointly required for nonzero OOD success. Sharpest single-lab datapoint yet that the WFM-as-evaluator and WFM-as-policy patterns can coexist in the same production stack.
• [2026-07-28] Data Pyramid for Embodied Manipulation: Data Pyramid re-reads recent world-action models alongside embodied-brain and VLA models through their data recipes (selection, alignment, mixing) rather than architectures, and relates data composition to world-prediction capability specifically — a data-side counterpart to A Functional Taxonomy of World Models (World Labs functional taxonomy) and Survey notes on World Action Models / VLA for robotics (index) (Luma-internal direction survey). Positions “principled data recipes for robot learning” as an open challenge, arguing world-model quality follows data-mixture quality.
• [2026-07-28] Being-H0.8: A Latent Tactile World-Action Model at Scale: Being-H0.8 extends the latent World-Action Model formulation of Being-H0.7 from visual-only prediction to tactile-aware interaction learning — the deployable prior + training-only posterior share token layout, but posterior latents are now supervised by future visual + tactile evidence rather than future frames alone. The bet: rather than reconstructing future observations at pixel level, capture their task-relevant consequences in latent space and use predicted states to condition action generation. Distinct from the video-generation-as-world-model camp (RynnWorld-4D: 4D Embodied World Models for Robotic Manipulation) and from the frozen-WFM-plus-action-expert camp (WAM-TTT: Steering World-Action Models by Watching Human Play at Test Time) by making tactile a first-class future modality inside the same joint pretraining objective.
• [2026-07-27] τ₀-VLA: A Hierarchical Robot Foundation Model with World-Model-Guided Test-Time Computation: τ₀-VLA adds a ninth WFM deployment pattern alongside the existing eight (backbone, predictive-surprise reward, interruptible tool, policy-internal substrate, edit-prior via KV cache, frozen trace-WM + small action expert, WFM-as-value-model, WFM-as-inference-time-guidance-signal): WFM as pre-commitment subtask discriminator inside a beam-search loop over natural-language subtasks. A video world model predicts the terminal head-camera image induced by each candidate subtask before the high-level policy commits, and a value model scores those imagined outcomes to drive top-B pruning — cleanest filed separation yet between “WFM as post-commitment renderer” (π0.7) and “WFM as pre-commitment comparator” (τ₀-VLA). Concrete answer to Sitzmann’s action-conditional-WFM chicken-and-egg for the hierarchical-VLA regime: the world model only ever needs to predict terminal observations under short subtask segments, not full continuous action-observation trajectories, so paired data requirements collapse to (subtask, terminal-image) pairs harvested from the same 40,115 h demonstration corpus that trains the low-level policy.
• [2026-07-25] SuperMap: A Spatio-Temporal SLAM System for Visual-Language Navigation: SuperMap (CMU, RSS 2026) uses GroundingDINO + SAM2 as frozen open-vocabulary perception backbones and adds the missing persistent-identity + temporal-evolution memory layer above them — argues explicitly that foundation vision models operate at the frame level and need an external spatial memory to reason about scene evolution, positioned as complementary rather than a replacement.
• [2026-07-25] Light Origins tweet 2080861398712946779 (content not retrievable at filing time): Light Origins (Roger Jiang, ex-OpenAI RLHF/InstructGPT) positions the company as building an embodied pretraining substrate — LLM-scale pretraining applied to human behavior data as the foundation for cross-embodiment agents.
• [2026-07-24] Open Dreamer — Open-source Dreamer 4 world-model implementation in JAX/Flax: Open Dreamer (next-state; Marti Monso / Sacco / Hu) — first filed open-source reimplementation of Dreamer 4 (Hafner, Yan, Lillicrap 2025) in JAX/Flax, trained on Minecraft/VPT data with a two-stage causal-video-tokenizer + action-conditioned latent-dynamics pipeline and a live in-browser demo on Reactor. Occupies the classical Ha–Schmidhuber–Hafner latent-world-model pole (contrast: AlayaWorld / LingBot-World / HY-World 1.5 all adapt large pretrained video DiTs). No FVD numbers reported at filing; BC/RL agent training loop is on the roadmap but unimplemented.
• [2026-07-24] SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation: Successor to SANA-WM in the NVlabs SANA line — a general video generator (not action-conditioned) that carries forward the hybrid Gated-Linear + softmax substrate and adds Block AttnRes on top; positions the hybrid-linear-attention substrate as SANA’s shared backbone across the world-modeling and general-video-gen deployments.
• [2026-07-23] FLUX 3 x mimic: The Next Generation of Video-Action Models: FLUX-mimic (BFL + mimic robotics, deployed at Audi) is the sharpest industrial-deployment instance yet of the WFM-as-policy-internal-substrate pattern: a small action decoder reads intermediate features from FLUX 3’s video-prediction path and runs at <80 ms backbone latency / 101 ms end-to-end reaction time on a single RTX 5090. Reports outperforming prior VLAs with the backbone completely frozen (a regime where prior VLAs fail to succeed) and SOTA when jointly fine-tuned. Load-bearing claim: Self-Flow representation alignment (Self-Flow: Self-Supervised Flow Matching for Scalable Multi-Modal Synthesis) is what makes the world model decodable to a small action head — not just generation quality but disentangled feature quality matters. Sibling in framing to μ₀: A Scalable 3D Interaction-Trace World Model (µ₀) but with a much larger, generation-optimized backbone and a real production line attached.
• [2026-07-23] FLUX 3 — Real World Models: Towards Multimodal Flow Models as the Backbone of Visual Intelligence: FLUX 3 (BFL) is the scaled-up, production-facing incarnation of Self-Flow: Self-Supervised Flow Matching for Scalable Multi-Modal Synthesis — a single flow-matching multimodal backbone jointly trained on image + video + audio (video accounting for >95% of compute) and natively extended to action prediction, positioned by BFL as the model where “content creation” and “physical AI” become two applications of one model rather than two separate backbones. First filed instance of an open-model-lab-with-closed-flagship publicly landing this thesis with a real deployment (Audi via FLUX-mimic, see FLUX 3 x mimic: The Next Generation of Video-Action Models), alongside NVIDIA’s Cosmos 3 as the other omnimodal-backbone bet.
• [2026-07-23] RepWAM: World Action Modeling with Representation Visual-Action Tokenizers: RepWAM (Fudan/Robbyant/HKUST) is the standalone arxiv paper form of the RepViTok tokenizer that LingBot-VA 2.0 already cited — a semantic visual-action tokenizer built by aligning a ViT video-autoencoder latent to a frozen Perception Encoder, then learning latent actions as IDM+FDM transport-plus-residual operators inside that space. Direct ablation of “reconstruction-oriented WAN2.2 VAE vs semantically-aligned RepViTok” at fixed 1.3B WAM lifts RoboTwin 2.0 Easy/Hard averages 78.0/76.0 → 86.6/83.1 and PickFruit 20% → 30%, sharpening the “the latent space is the bottleneck” position with the cleanest single-component ablation on file. Also reports the semantic alignment eliminates reliance on video CFG (peaks at scale 1.0), a first filed inference-time efficiency signature of representation-aligned WAM latents.
• [2026-07-22] Agentic Real2Sim: Physics-based World Modeling with Vision-Language Agents: Agentic Real2Sim positions a physics-simulator + VLM-agent orchestration stack as an alternative to generative WFMs for producing policy-learning and policy-evaluation substrates: instead of rolling out a learned world model, it converts a real robot-interaction video into a runnable per-episode simulatable twin that preserves geometry, object states, and interactions. Three-domain coverage (rigid, deformable, humanoid) in one framework contrasts with the typically domain-specialised real2sim pipelines that feed WFM-adjacent training loops.
• [2026-07-22] Physion-Arc 1.0: Benchmarking Video Agents on Minute-Long Video Generation: Physion-Arc 1.0 evaluates six video-agent systems (Runway Agent 2.0, Luma, MiniMax, Kling, Utopai, TapNow) on minute-long multi-shot narrative video output — a data point on the “agent scaffolding vs raw model scale” axis: Runway Agent 2.0 (a closed agentic product) leads all announced dimensions, inverting the “closed models don’t dominate” pattern seen in physics-commonsense benchmarks (VideoPhy-2, Physics-IQ). Suggests cinematic/narrative quality rewards system-level orchestration in a way physics-commonsense does not.
• [2026-07-21] AlayaWorld: Interactive Long-Horizon World Modeling: AlayaWorld (Alaya Lab / Shanda, same team as WildWorld: A Large-Scale Dataset for Dynamic World Modeling with Actions and Explicit State toward Generative ARPG) is the most complete filed composite-context open-source interactive WFM: an LTX-2.3-based ~13B video DiT with a four-stream in-context prefix (pinned sink + Frame-Preservation temporal memory + GEN3C-style depth-projected spatial cache + I2V nearby frame) trained in three stages (bidirectional adaptation → autoregressive fine-tune with Helios drift simulation + error-bank residual replay → DMD+Self-Forcing++++CD distillation to 4 steps/chunk). Reported best overall on iWorld-Bench; code + weights + full technical report released. Direct open-weights alternative to Genie 3 / GWM-1 in the real-time interactive category.
• [2026-07-21] DriftWorld: Fast World Modeling through Drifting: DriftWorld (Yilun Du et al.) applies Kaiming-lab-style drifting one-step generators to action-conditioned video world modeling for robotics, reaching 30+ FPS on a single H100 (17× faster than diffusion-based WFM baselines) with 0.99 rank correlation to real-world policy performance on Robomimic Lift — a fast-WFM datapoint for both inference-time policy improvement (MPC-style rollout search) and offline policy evaluation.
• [2026-07-21] Apple-π: Benchmarking Thinking with Video Towards Law-Grounded Physical Intelligence: Apple-π adds a mechanics-focused, law-grounded benchmark-as-probe entry alongside RISE-Video: Can Video Generators Decode Implicit World Rules? and Do generative video models understand physical principles? (Physics-IQ): 400 videos × 10 classical-mechanics tasks with Perception → Formulation → Deduction stage decomposition and a physics-law-grounded objective evaluator (not VLM-judge naturalness, not pixel-mask overlap). Best of 11 evaluated video models scores 0.473; sharpens the “current video models are not law-grounded world simulators” thesis with the sharpest per-stage failure profile on file, and provides a diagnostic probe distinguishing perception-side, formulation-side, and generation-side breakage.
• [2026-07-20] Happy Oyster — Real-Time World Model for Interactive Creation: Happy Oyster (Alibaba Token Hub) — product-surface entry for a closed real-time interactive WFM with two operating modes (Directing: text/voice/image steering up to 3 min at 480p/720p with audio+video output; Wandering: first-person WASD exploration up to 1 min at 480p). No architecture, weights, or benchmarks disclosed. Distinguishing features vs Genie 3 / GWM-1 on file: baked-in joint audio-video generation and an explicit voice input channel; joins the closed-flagship real-time WFM product race alongside DeepMind’s Project Genie and Runway GWM-1.
• [2026-07-20] Introducing Cosmos 3 Edge: Cosmos 3 Edge is the first filed on-device variant of an omnimodal WFM that keeps the flagship’s full dual-tower AR + DM architecture — a 4B open-weights model with a 2B Nemotron reasoner, targeted at Jetson (incl. new T2000/T3000), DGX Spark, and RTX PRO/GeForce RTX. As a post-trained WAM it operates at 640×360, generates 32 actions per inference at 15 Hz on Jetson Thor, and is #1 on VANTAGE-Bench among ~4B open models. Adds an edge-scale WFM-as-backbone row alongside the existing flagship (Cosmos 3 Super/Nano) and closed on-device gaps — the same understanding + generation + action interface, just resized.
• [2026-07-20] Flow Matching in Feature Space for Stochastic World Modeling: FlowWM (Porcher/Carion/Alahari/Chen, FAIR + Inria) makes DINO-world stochastic: flow matching directly in a frozen DINOv3 encoder’s high-dimensional feature space, with a shallow-wide projection head (RAE-style) and a differentiable one-step projection that lets temporal-consistency and task-driven losses supervise the endpoint without ODE-through-time backprop. Introduces the FuturePerception Waymo benchmark that scores predicted features via linear-probe detection + depth heads rather than pixel FVD. Concrete counterpoint to VAE-latent stochastic WMs on real driving (AP(3) 21.7 vs 18.2 for WAN 2.2 VAE-WM, depth RMSE 0.078 vs 0.10), and to the deterministic DINO-world / DINO-WM line (F1 error 4.31 vs 17.1 on the synthetic Bouncing-Shapes benchmark where all modes are enumerable). First filed paper porting the RAE recipe to stochastic world modeling.
• [2026-07-19] RynnBrain 1.1 — Open Embodied Foundation Models (Alibaba DAMO Academy): RynnBrain 1.1 is the perception/planning tier of Alibaba DAMO’s Rynn* stack that pairs with the RynnWorld-4D world-model tier; extends the “unified embodied brain” position with metric-3D grounding + contact-point prediction, and pushes the model scale to 122B-A10B sparse MoE.
• [2026-07-18] Haiyu Wu — critique of reachability-augmented CEM planning in JEPA world models (with VISReg counter-example): Haiyu Wu (@HaiyuWu1) shares an unnamed JEPA WFM paper that adds a reachability score + K-step training rollouts to CEM planning over LeWM latents; Wu counters with a working example that swapping LeWM’s SIGReg for VISReg (arxiv 2606.02572) recovers the same or better planning success on TwoRoom and OGBench Cube — a datapoint that the reported “train-test objective misalignment” may be a latent-regularization artifact rather than a planning-objective problem, and a concrete instance of the swappable-JEPA-components design space bundled by x-jepa — Explorations into JEPA approaches (lucidrains).
• [2026-07-17] WAM-TTT: Steering World-Action Models by Watching Human Play at Test Time: WAM-TTT (Feng, Lyu, Wang et al.) makes the frozen WAM itself the substrate for user-preference / task-variant steering: rather than fine-tuning or adding a separate steering module, a lightweight adaptive key–value memory inside the WAM absorbs raw human-play videos via self-supervised video prediction at test time, aligned by a paired-human-robot meta-training stage with a KV memory reconstruction objective. Concrete evidence that a pretrained WAM can be steered efficiently without touching its weights — complements the “WAM-as-policy-peer” recipe of World Pilot: Steering Vision-Language-Action Models with World-Action Priors and the “frozen 3D-trace WM + small action expert” recipe of μ₀: A Scalable 3D Interaction-Trace World Model with a “frozen WAM + TTT-adapted memory” position.
• [2026-07-17] HDR: Hierarchical Denoising for Multi-Step Visual Reasoning: HDR reports that its hierarchical video reasoner transfers to real robot maze interaction after fine-tuning on only 50 real-world robot maze videos — a data point that coarse-to-fine hierarchical video priors are compact enough to serve as world models for embodied control with minimal domain data.
• [2026-07-16] VideoRAE: Taming Video Foundation Models for Generative Modeling via Representation Autoencoders: VideoRAE closes the loop on the “frozen video foundation model as latent space” thesis (V-JEPA 2, VideoMAEv2) by showing those latents can be decoded back to pixels via a lightweight projector + representation-aligned decoder — providing a generative video-model counterpart to the latent-predictive world-model line rather than a competitor to it.
• [2026-07-16] RxBrain: Embodied Cognition Foundation Model with Joint Language-Visual Reasoning and Imagination: RxBrain performs 4-frame future world-state prediction in the FLUX VAE latent space via parallel flow matching, with an internal mask preserving temporal causality across generated frames.
• [2026-07-16] FlowDAgger: Human-in-the-Loop Adaptation of Generative Robot Policies in Latent Space: FlowDAgger’s noise-space-steering adaptation recipe transfers across the VLA/WAM boundary: applied to Cosmos-Policy (a video-conditioned world-action model), the same action-inversion primitive lifts mean success 0.53 → 0.74 on 7 MetaWorld tasks — matched-magnitude to the π0.5 (VLA) result (0.53 → 0.79) on the same task set. Evidence that the interface between “generative policy” and “downstream adapter” is architecture-agnostic across action-head VLAs and video-conditioned WAMs, provided the base is trained as a flow-matching sampler.
• [2026-07-15] Pantograph introduces Pan-1 — Minecraft model with RL-based pretraining, arguing video games are the right testbed for robotics: Pantograph launches Pan-1 (Minecraft, RL-based pretraining) and explicitly argues video games are the right robotics testbed — reproducible, safe, open-ended, long-horizon — stakes out an RL-pretraining-on-games corner adjacent to NitroGen’s BC-on-controller-overlay-video corner, and a distinct bet from Sitzmann’s video-generative-rollout framing. Product-tweet marker; no benchmarks disclosed.
• [2026-07-14] Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Model: Xiaomi-Robotics-U0 is a 38B AR multimodal WFM continually trained from EMU3.5 that jointly optimizes T2I, editing, multi-view embodied scene generation, controllable transfer, and video rollout — ranks 1st on WorldArena and lifts π₀.₅ OOD success from 36.9% → 63.2% when used as a data engine, arguing the pretraining prior is what carries embodied generalization.
• [2026-07-14] Towards Generalizable Robotic Manipulation in Dynamic Environments (DOMINO / PUMA): PUMA (Fang et al., ECCV 2026) adds a compact-internal-WFM design point to the VLA-with-world-model recipe board: no separate video generator, no denoising loop — historical optical-flow perception (Grounded-SAM-2 masks) as input plus learnable world queries as an implicit object-centric-future-state bottleneck inside the VLA forward pass. Sibling to RynnWorld-4D’s tri-branch DiT + inverse dynamics, µ₀’s frozen-trace WM + action expert, and World Pilot’s dual-path steering, but with the prediction machinery collapsed into a fixed-size query set inside the policy; reports +6.3% absolute SR over baselines on the new DOMINO dynamic-manipulation benchmark.
• [2026-07-13] x-jepa — Explorations into JEPA approaches (lucidrains): lucidrains ships an exploratory PyTorch codebase (x-jepa) that bundles the recent latent-predictive-WFM pole — LeWorldModel, LeJEPA, VISReg, Next-Latent Prediction Transformers, Compositional Planning with Jumpy World Models, and Kimi Attention Residuals — into a single reimplementation, treating them as one composable design space and giving the wiki a portable minimal reference for the JEPA-end-to-end-from-pixels recipe.
• [2026-07-13] Video Generation Models are General-Purpose Vision Learners: GenCeption (Google DeepMind, ECCV 2026 — Kaiming He, Saining Xie, Zisserman, Sminchisescu et al.) is the sharpest filed empirical vindication of Sitzmann’s “generative-rollout video pretraining is the right substrate” position: a pre-trained text-to-video diffusion backbone, turned into a feed-forward text-steered vision model, matches or beats DepthAnything3, SAM3, D4RT, VGGT-Ω, Sapiens, and Lotus-2 on depth / normals / camera pose / expression-referring segmentation / 3D keypoints with 7×–500× less training data — and, critically, the paper reports that video-generation pretraining outperforms V-JEPA and VideoMAE under comparable settings. First filed head-to-head between generative-rollout and latent-predictive video pretraining on downstream perception with the generative side winning, updating the cluster’s central open question with a data point that leans toward the generation side.
• [2026-07-10] Understanding and Mitigating the Video-Action Generalization Gap via Temporal Ratio: Temporal Ratio adds an eighth WFM deployment pattern to the ledger — WFM-as-source-of-inference-time-guidance-signal — distinct from the seven already tracked (backbone, predictive-surprise reward, interruptible tool, policy-internal substrate, edit-prior via KV cache, frozen trace-WM + small action expert, value-model). Keeps the pretrained Cosmos-Predict 2.5-2B backbone in the VAM stack but reads out its internal attention statistics (Temporal Ratio = action-attention(future)/action-attention(current)) and feeds them back into the same policy’s guidance schedule at inference. Also a concrete data point on Sitzmann’s action-conditional-WFM chicken-and-egg: the video prior’s compositional generalization does survive robot-domain finetuning (autoregressive rollouts imagine plausible novel object–goal combinations) — the gap is entirely in how the action head consumes it, not in the WFM’s world knowledge.
• [2026-07-10] LingBot-VA 2.0: Native Video-Action Pretraining for Generalizable Robot Control: LingBot-VA 2.0 sharpens the WFM-as-policy-internal-substrate pattern (established by v1 Causal World Modeling for Robot Control (LingBot-VA)) with a natively-pretrained variant: no bidirectional-to-causal retrofit, no off-the-shelf video-generator base — the ViT visual tokenizer + latent-action tokenizer + causal MoE DiT are all trained from scratch under a joint objective aligning to a frozen Perception Encoder. Concrete partial answer to Sitzmann’s action-conditional-WFM chicken-and-egg: the latent-action tokenizer bootstraps action signal from unlabeled video via an inverse-dynamics+forward-dynamics bottleneck, so the world-model half never needs paired action-observation data. Foresight Reasoning + learned-forward-dynamics re-grounding is a concrete recipe for keeping an autoregressive WFM coupled to reality under async execution — an open sub-question Next Forcing: Causal World Modeling with Multi-Chunk Prediction and v1 both raise.
• [2026-07-09] From Foundation to Application: Improving VLA Models in Practice (LingBot-VLA 2.0): LingBot-VLA 2.0 adds a WFM-as-auxiliary-proxy datapoint to the recipe board: rather than a full generative rollout engine or a frozen trace-WFM feeding an action expert, it formulates future frame prediction as a proxy task supervised by two pretrained models — a video representation model for semantic priors and a depth estimator for geometric cues. Structurally lighter than PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulation‘s physics-reinforced rollout or Causal World Modeling for Robot Control (LingBot-VA)‘s unified AR flow-matching, and closer in spirit to representation-alignment losses than to world-model-in-the-loop planning.
• [2026-07-09] ProxyPose: 6-DoF Pose Tracking via Video-to-Video Translation: ProxyPose is a clean stress-test of the “video diffusion as universal perception backbone” claim: after fine-tuning only on synthetic (input clip, proxy-video) pairs, the same model transfers zero-shot across sensing modalities (event cameras, SPAD arrays) and object classes (rigid objects → non-rigid faces → camera pose on textureless scenes) — all downstream of a single video prior.
• [2026-07-09] LingBot-World 2.0 / LingBot-World-Infinity — Infinite Worlds with Versatile Interactions: LingBot-World 2.0 (Robbyant / Ant Group) — second-generation open interactive WFM, headlines causal pretraining (not just distillation) as the base objective, 720p / 60 fps with sub-second latency (higher fps than Genie 3’s public spec), unbounded interaction horizon, and a pilot+director agentic harness wrapping the rollout. License drops from v1’s Apache-2.0 to CC BY-NC-SA 4.0 non-commercial; tech report URL not yet resolvable at filing. Direct sequel to Advancing Open-source World Models (LingBot-World).
• [2026-07-08] LingBot-Video: Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligence: LingBot-Video is the video-foundation-model layer of the Robbyant LingBot stack (paired with LingBot-World, LingBot-VA, LingBot-VLA) — argues WFMs for embodied AI need physical-rationality + task-completion reward signals and robot-augmented pretraining corpora, not just aesthetic optimization on internet video; claims to be the first large-scale open-source MoE video foundation model built for embodiment.
• [2026-07-08] RynnWorld-4D: 4D Embodied World Models for Robotic Manipulation: RynnWorld-4D (Alibaba DAMO, Apache-2.0) co-generates synchronized RGB + depth + optical flow via a tri-branch DiT with cross-modal attention and 3D RoPE, then reads its internal 4D representations with a single-forward-pass inverse-dynamics head — a WFM deployment pattern that skips multi-step denoising to run as a real-time policy.
• [2026-07-08] Towards Practical World Model-based Reinforcement Learning for Vision-Language-Action Models: VLA-MBPO adds a sixth WFM deployment pattern alongside backbone / reward / interruptible-tool / policy-internal-substrate / edit-prior: UMM-as-joint-dynamics-and-reward-model. A single finetuned Bagel emits multi-view pixel next-frames + a discrete reward token in one forward pass, matching a dedicated video world model (Ctrl-World) on prediction fidelity at ~2× the speed and matching a dedicated VLM reward model (Qwen3-VL-8B, 98.4% vs 97.0% accuracy). Concrete evidence the “world model + separate reward model” two-stack pattern is redundant when the backbone is a pretrained UMM.
• [2026-07-06] Embodied.cpp: A Portable Inference Runtime of Embodied AI Models on Heterogeneous Robots: Embodied.cpp treats VLAs and World-Action Models as a single execution class under one five-layer runtime contract (input adapters / sequence builders / backbone / head plugins / deployment adapters) and one backend abstraction across CPU/CUDA/NPU — small structural-evidence entry for the “world-model peer + action policy share deployment infrastructure” thread; preliminary WAM memory result reduces a LingBot-VA Transformer block from 312.2 MiB to 88.1 MiB.
• [2026-07-04] RoboWorld: Fast and Reliable Neural Simulators for Generalist Robot Policy Evaluation: RoboWorld (KAIST) uses a DROID-trained AR video world model + VLM judge as a neural simulator for RoboArena policy eval; rankings across 8 policies × 4,186 rollouts correlate with the real leaderboard at Pearson r = 0.989.
• [2026-07-02] Markov AI gaming dataset — 500+ hours of Valorant / Minecraft / GTA gameplay screen recordings + keystrokes/mouse: Markov AI ships a 500+ hour open gaming dataset with paired keystroke/mouse action labels across three distributionally distinct genres (tactical FPS / sandbox / open-world) — a small but genre-diverse fine-tuning + evaluation substrate for action-conditioned world models over game screens, complementing NitroGen’s 40,000h gamepad-overlay corpus with native OS-level input labels.
• [2026-07-01] AdaJEPA: An Adaptive Latent World Model: AdaJEPA (Wang, Bounou, LeCun, Ren) proposes test-time adaptation of the JEPA world model inside the MPC loop — after executing an action chunk, the observed next-state transition is used as a self-supervised update signal on the predictor, and replanning uses the freshly-adapted model. Concrete recipe for the “action-conditional WFM under distribution shift” open sub-problem: pushes the shift-correction signal into the parameters at inference (contrast: ICWM pushes it into the context, D-MPC keeps the model frozen). Reports substantial planning-success gains with as few as one gradient step per replan. Complements the LeCun-lab lineage from LeWorldModel and V-JEPA 2.1.
• [2026-06-30] TrueLabel — Physical AI Dataset Directory: Robotics, Humanoid & Egocentric Data: TrueLabel Physical AI Dataset Directory — 658-entry third-party catalog of robotics / humanoid / egocentric / teleop / simulation datasets with license and modality metadata; sibling to datasets.bot, useful for sourcing the perception substrate WFMs are trained on.
• [2026-06-30] EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Data: EgoScale (NVIDIA GEAR) treats 20,854 h of egocentric human video as a scalable embodied pretraining substrate and shows the resulting representation transfers across both task (one-shot to unseen Fold Shirt / Unscrew Bottles) and embodiment (Sharpa 22-DoF → Unitree G1 7-DoF tri-finger). The framing — humans as another embodiment whose data is treated like any other pretraining source — is the strongest filed argument that “world-foundation prior over physical interaction” can be learned from egocentric video alone without a generative video objective.
• [2026-06-30] datasets.bot — Curated catalog of robotics and embodied-AI training datasets: datasets.bot — third-party catalog of robotics/embodied datasets tagged by modality and license; relevant when scoping WFM training corpora (egocentric video, simulator dumps, X-embodiment data) against the human-vs-robot-data substrate debate.
• [2026-06-30] ACE-Ego-0: Unifying Egocentric Human and Robotic Data for VLA Pretraining: ACE-Ego-0 supplies another datapoint on egocentric human video as a complementary pretraining substrate for embodied VLAs — 1.48K hours of pseudo-action-labeled egocentric data jointly trained with 4.53K hours of robot + sim under a reliability-aware loss; reports gains in both joint pretraining and SFT regimes on RoboCasa GR1 TableTop and RoboTwin 2.0.
• [2026-06-30] World Pilot: Steering Vision-Language-Action Models with World-Action Priors: World Pilot adds a refinement to the WFM-as-policy-peer pattern alongside µ₀ (frozen trace WM + action expert) and ImageWAM (KV-cache injection from a pretrained editor): the WAM steers the policy through two complementary paths simultaneously — a scene-evolution latent into the VLA’s perception layer (Latent Steering) and an anticipated trajectory into the action generator (Action Steering). Concrete partial answer to Sitzmann’s action-conditional-WFM chicken-and-egg: the perception-side latent transfers effectively from a video-pretrained world model that was not action-post-trained, so the dynamics prior does not require paired action–observation data. Reports 84.7% on LIBERO-Plus zero-shot OOD with largest margins on viewpoint / geometry / deformable / pose shifts.
• [2026-06-29] Flexion Reflect v1.0: The Path Towards Long-Horizon Autonomous Humanoid Work: Flexion Reflect v1.0 opts against a generative WFM as the spatial prior — uses a precomputed semantic map from a one-time building scan plus VLM-mediated natural-language queries and global path planning over it — a structural counter-recipe to the WFM-as-substrate framings filed elsewhere on the page.
• [2026-06-29] SimFoundry: Modular and Automated Scene Generation for Policy Learning and Evaluation: SimFoundry takes the modular foundation-model pipeline position against monolithic generative WFMs: rather than fine-tuning Veo or Cosmos into a robotics simulator, it composes segmentation + depth + 2D-to-3D + physics validation into a sim-ready digital twin, with the physics engine as the gate. Reports r = 0.911 sim-to-real Pearson correlation and MMRV 0.018 over 7 tasks × 5 policy architectures — the strongest filed signal that explicit physics simulators are competitive with generative WFMs as policy-evaluation substrates.
• [2026-06-29] World Value Models for Robotic Manipulation: WVM (ByteDance Seed + PKU + Tsinghua) adds a new WFM deployment pattern — WFM-as-value-model — distinct from the prior six on this page (backbone, predictive-surprise reward, interruptible tool, policy-internal substrate, edit-prior via KV cache, frozen trace-WM + small action expert): the world model is the substrate of a learned generalist value head that produces task-progression scores for downstream policy learning across embodiments. Reports SOTA Value-Order Correlation (0.95 avg expert, 0.78 avg suboptimal) and introduces Suboptimal-Value-Bench (800 human-frame-annotated multi-embodiment trajectories) as the first filed value-model evaluation on suboptimal data; baselines split sharply on suboptimal data (VLAC collapses to -0.37 VOC vs WVM’s 0.78).
• [2026-06-29] In-Context World Modeling for Robotic Control: ICWM (Fudan NLP) — the arxiv paper behind the HuggingPapers tweet 2070794187063099893 — In-Context World Modeling for Robotic Control (ICWM) stub, now resolvable. Candidate answer to Sitzmann’s action-conditional WFM chicken-and-egg (training interactive WFMs needs paired action–observation data, which embodied AI lacks): the robot generates its own short task-agnostic interaction prefix on the fly, and the model treats that prefix as in-context system identification rather than relying on demonstration prompts or pretraining-scale paired data. Same family as DAWN’s coupled world-action inference (The DAWN of World-Action Interactive Models) but pushed into the test-time-context regime.
• [2026-06-29] PhysisForcing: Physics Reinforced World Simulator for Robotic Manipulation: PhysisForcing adds two physics-informed auxiliary losses (pixel-trajectory + relational semantic alignment) during finetuning of video DiTs and lifts robotics-oriented R-Bench scores by 22.3% on Wan2.2-I2V-A14B and 9.2% on Cosmos3-Nano, with WorldArena closed-loop success going 16% → 24%.
• [2026-06-28] HuggingPapers tweet 2070794187063099893 — In-Context World Modeling for Robotic Control (ICWM): HuggingPapers tweet stub flagging “In-Context World Modeling for Robotic Control” (ICWM) — claims a world model that learns dynamics from a few seconds of self-generated interaction and zero-shot generalizes to unseen cameras / new morphologies without fine-tuning; underlying paper not linked from the tweet and not locatable at filing — kept as a re-resolve target. If the claim holds it is a candidate answer to Sitzmann’s action-conditional-WFM chicken-and-egg.
• [2026-06-27] Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond: First filed capability-centric survey for the WFM cluster — proposes a three-level hierarchy (L1 Predictor / L2 Simulator / L3 Evolver) crossed with four governing-law regimes (physical / digital / social / scientific), with testable boundary conditions for each level (L2 = long-horizon coherence + intervention sensitivity + constraint consistency; L3 = deployment-evidence-driven stack revision, distinct from closed-loop use). Synthesizes 400+ works / 100+ systems across model-based RL, video generation, web/GUI agents, multi-agent simulation, and AI-for-science. Orthogonal to Fei-Fei Li’s renderer/simulator/planner output-channel split — together they give the cluster a 2D vocabulary it previously lacked. Argues L3 model revision requires a symbolic substrate (sharp counter to Sitzmann’s “explicit 3D should dissolve” position); names “meta-world modeling” (learnable governing laws) as the open stage beyond L3.
• [2026-06-26] Survey notes on World Action Models / VLA for robotics (index): Luma-internal survey-notes index (Dong Guo + co-author) covering WAM / VLA directions with per-direction “what does this actually solve vs. only appear to solve” commentary — sits at the WFM↔policy boundary alongside Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models and A Functional Taxonomy of World Models.
• [2026-06-26] Remediating Alfred Hitchcock: From Parametric to Recursive Montage: Boutet de Monvel (Bibliotheca Hertziana – MPI for Art History) reframes “world models” as a contemporary phase in a 70-year history of cinematic montage — classical cut → parametric (Whitney’s analog-computer animation for Vertigo) → recursive (Chatonsky’s Readonlymemories VI, a ComfyUI chain of LLaVA-Phi + FLUX.1 Kontext + Wan 2.6 producing an in-principle infinite Vertigo). The conclusion delivers the first filed humanities-side critique of the sim-to-real bet: gaming/entertainment uses are legitimate, but deploying models trained on mediated representations as substrates for robotics and embodied agents is a category error — “driving a car is not like racing in Forza Horizon, cleaning a house is not like playing The Sims.” Stands as a critical counterweight to Sitzmann’s “explicit 3D should dissolve into generative-rollout WFMs” position, and complements VideoPhy-2 / RISE-Video / Dream.exe (which produce quantitative versions of the same structural gap).
• [2026-06-24] Qwen-AgentWorld: Language World Models for General Agents: Qwen-AgentWorld opens a third pole in the WFM cluster alongside pixel-rollout (Cosmos 3, Genie 3, Qwen-RobotWorld) and latent-predictive (V-JEPA 2.1, LeWorldModel): language world models that simulate the token-space dynamics of agentic environments (browsers, code interpreters, terminals) via long CoT reasoning. Trained on 10M+ real-environment trajectories across 7 domains through CPT → SFT → RL with hybrid rubric+rule rewards; 35B-A3B and 397B-A17B MoE variants. Reports gains over real-environment training when used as a decoupled simulator for agentic RL, plus warm-up benefit across 7 agentic benchmarks when used as a unified agent foundation model. Sibling to Qwen-RobotWorld (pixel-side); language-side analog to Cosmos 3’s omnimodal “world model that emits the environment response”.
• [2026-06-24] Next Forcing: Causal World Modeling with Multi-Chunk Prediction: Next Forcing adapts LLM-style multi-token prediction to causal video WAMs: auxiliary MCP heads denoise future chunks (next¹/²/³) during training, giving 2.3× faster convergence vs. LingBot-VA at 50 fps and new SOTA on RoboTwin (94.1/93.5 Clean/Random); also reports significant gains on PhyWorld and 2× inference speedup when heads are kept for parallel next-chunk prediction.
• [2026-06-23] ImageWAM: Do World Action Models Really Need Video Generation, or Just Image Editing?: ImageWAM adds a fifth WFM deployment pattern alongside WFM-as-backbone (Cosmos+GR00T), WFM-as-reward (VJEPA-2/WMReward), WFM-as-interruptible-tool (VoLo), and WFM-as-policy-internal-substrate (LingBot-VA): WFM-as-edit-prior consumed via KV cache — the world model is a pretrained image editor whose denoising KV cache feeds a flow-matching action expert, and the target frame is never actually decoded at inference, reportedly cutting FLOPs to 1/6 and latency to 1/4 of video-based WAMs.
• [2026-06-23] Jun Gao thread (4/5) on Cosmos2.5-2B finetuned with cleaned data + kinematic skeleton conditioning: Jun Gao reports Cosmos2.5-2B finetuned with cleaned data + kinematic skeleton conditioning beats baselines on action following, appearance, and motion consistency, trained on a single GH200.
• [2026-06-23] Causal World Modeling for Robot Control (LingBot-VA): LingBot-VA fuses the world-model and action policy into one autoregressive flow-matching network — adding a fourth WFM deployment pattern alongside WFM-as-backbone (NVIDIA Cosmos+GR00T), WFM-as-reward (VJEPA-2/WMReward), and WFM-as-interruptible-tool (VoLo): WFM-as-policy-internal-substrate, where the world model is part of the action-decoding network rather than a separate peer. Concrete answer to µ₀’s “frozen WFM + small action expert” factorization with the orthogonal “end-to-end joint training” recipe on Wan2.2-5B.
• [2026-06-22] Inside NVIDIA Halos for Robotics: A Full-Stack Functional Safety System for Physical AI: NVIDIA Halos for Robotics uses Isaac Sim as the SIL closed-loop testing harness for safety agents — a deployment-side role for world simulators that complements their pretraining-side use in policy-evaluation work (e.g. Evaluating Gemini Robotics Policies in a Veo World Simulator)
• [2026-06-21] HumanEgo: Zero-Shot Robot Learning from Minutes of Human Egocentric Videos: HumanEgo’s encoder-shared auxiliary heads (3D object-motion + 2D trace + latent consistency) construct a lightweight in-policy world model of hand-object interaction inside a flow-matching manipulation policy, rather than as a separate WFM peer — a structurally distinct WFM-inside-VLA recipe relative to the NVIDIA GR00T+Cosmos two-model split or μ₀’s frozen-trace world model + small action expert.
• [2026-06-19] EgoInfinity: A Web-Scale 4D Hand-Object Interaction Data Engine for Any-View Robot Retargeting and Video-to-Action Robot Learning: EgoInfinity is a perception-stack composition (segmentation + reconstruction + interaction-aware refinement + retargeting under cross-module metric calibration) that consumes any-view RGB and emits agent-agnostic 4D hand-object trajectories — a concrete instance of “world-perception modules composed into a metric-calibrated pipeline” feeding the WFM-for-robotics line.
• [2026-06-19] UMA: Unified Motion-Action Modeling for Heterogeneous Robot Learning: UMA is a non-pixel, non-action world-model interface that uses 3D object motion as the common ground across human video, real-robot, and simulation data, and toggles between policy / dynamics-predictor / adapter-target via mask pattern in a single masked generative model. Reports MSE 0.042 on motion-based dynamics modeling vs PointWorld 0.054, with the human-video and sim sources each providing measurable lift.
• [2026-06-19] HumanScale: Egocentric Human Video Can Outperform Real-Robot Data for Embodied Pretraining: HumanScale reports egocentric human video can invert (not just match) the teleop-pretraining advantage for embodied foundation models — 24% lower validation loss and 52.5% / 90% higher in-distribution / OOD real-robot success rates at matched pretraining volume, with a “carefully designed filtering and labeling pipeline” as the lever.
• [2026-06-19] MolmoMotion: Language-guided 3D motion forecasting: MolmoMotion is a sharp counter-datapoint to the pixel-rollout WFM frame: a Molmo-2-based 3D point-trajectory forecaster reportedly beats Wan2.2-5B and Cosmos Predict on PointMotionBench 3D ADE by 2–6× across HOT3D / WorldTrack / DAVIS, extending the “tracks beat pixels for motion” position of Forecasting Motion in the Wild to language-conditioned, object-grounded 3D forecasting with explicit robotics + video-gen downstream transfer.
• [2026-06-19] Diffusion Knows Transparency: Repurposing Video Diffusion for Transparent Object Depth and Normal Estimation: DKT (BAAI/USC/Tsinghua et al.) extends the DVD: Deterministic Video Depth Estimation with Generative Priors “video diffusion as the prior for video depth regression” recipe to transparent and highly reflective objects — a regime where stereo/ToF/discriminative-depth all fail by construction. Same LoRA-on-Wan family, different design choices (channel-concat RGB+depth latents inside the DiT, standard flow-matching loss, 5-step fixed inference), plus a new 11k-video / 1.32M-frame synthetic dataset (TransPhy3D) filtered by Qwen2.5-VL aesthetic scoring; reported zero-shot SOTA on ClearPose / DREDS / TransPhy3D-Test depth and ClearPose normals, with grasp success jumping from 0.46–0.48 to 0.73 on reflective/translucent tabletops. Strongest filed evidence that VDM-internalized “physics” extends past rigid-body motion into refraction/reflection, and a sharp cash-out of the Sitzmann thesis in a vertical where explicit-3D pipelines are structurally blocked.
• [2026-06-18] M*: A Modular, Extensible, Serving System for Multimodal Models: Stanford’s M* flags world-model rollout as a serving-optimization frontier. Reports up to 12.5× faster V-JEPA 2 rollout vs Meta’s native implementation on 1×H100, purely from treating the variable-horizon rollout as a first-class Loop primitive with a persistent KV cache across steps rather than recomputing the cache each iteration. No kernel or quantization changes — the speedup is structural. This is the largest single multiplier reported in the M* benchmark suite (vs ≈2.7× / ≈4× on Qwen3-Omni TTS, ≈1.3–2.6× on BAGEL), suggesting that world-model rollout is currently the most under-served loop type relative to AR text decode, and that the same continuous-batching + CUDA-graph machinery the wiki has tracked for LLM and diffusion serving has not yet been ported into the published world-model rollout recipes (Genie 3, Cosmos 3, V-JEPA 2). Implication for this page: serving-side recipes are now a missing axis in the world-model evaluation toolkit.
• [2026-06-18] Robot Learning 2026 Main Lectures — ETH Zürich (Oier Mees): ETH Zürich Robot Learning 2026 main-lectures playlist (Oier Mees) — the course’s foundation-models-for-robotics segment is a structured walkthrough of the world-foundation-model literature the concept tracks.
• [2026-06-18] ActiveMimic: Egocentric Video Pretraining with Active Perception: ActiveMimic adds a signal-design axis to the egocentric-pretraining substrate debate started by EgoVerse and HumanNet: treating the demonstrator’s camera motion as a first-class action (not nuisance noise) during pretraining is reported to close the long-standing gap to robot-data pretraining on real-world tasks.
• [2026-06-15] μ₀: A Scalable 3D Interaction-Trace World Model: µ₀ (Seungjae Lee, Jusuk Lee, Jia-Bin Huang, Furong Huang et al.) — third design path between pixel-rollout WFMs and direct-action VLAs: a world model whose interface is 3D trajectories of salient interaction points (objects, tools, hands, contacts), parameterized as B-spline control points and pretrained action-free from web video via the TraceExtract auto-supervision pipeline. Frozen + paired with embodiment-specific action experts, reported competitive with π₀ on downstream manipulation despite no action pretraining. Direct successor to DynaFLIP from the same group — dynamics pushed from the encoder layer into the world-model interface layer.
• [2026-06-15] VoLo: A Physical Orchestrator for Open-Vocabulary Long-Horizon Manipulation: VoLo treats a world-action model (WAM) as an interruptible tool inside a VLM orchestrator loop — a third deployment pattern alongside WFM-as-backbone (NVIDIA Unveils New Open Models, Data and Tools to Advance AI Across Every Industry) and WFM-as-reward (Inference-time Physics Alignment of Video Generative Models with Latent World Models); the orchestrator can monitor the WAM rollout and intervene before it completes.
• [2026-06-14] LEGS: Fine-Tuning Teleop-Free VLAs for Humanoid Loco-manipulation in an Embodied Gaussian Splatting World: LEGS uses 3D Gaussian Splatting as the photorealistic background substrate for a humanoid simulator, decoupling foreground motion from scene appearance so the same motion can re-render against new scenes/objects at ~15× lower cost than teleop re-collection — a concrete instantiation of the “simulator/world model as data engine for policies” lineage.
• [2026-06-14] RoboArena rolls back evaluations after benchmark hacking observed since April (Pranav Atreya announcement): RoboArena’s lead author publicly retracts the post-April leaderboard ordering after benchmark hacking was detected, with NVIDIA’s Cosmos among the entries removed (per the Slack pointer) — directly puncturing the “Cosmos 3 Nano leads RoboArena” headline claim in NVIDIA Launches Cosmos 3, the Open Frontier Foundation Model for Physical AI and How Cosmos 3 Helps Physical AI Think Before It Acts, and casting retroactive doubt on the Spirit-v1.6-vs-Cosmos3-Nano scoreboard reported in Spirit AI beats Nvidia on RoboArena robotics benchmark.
• [2026-06-11] Yann LeCun: World Models — Enabling the next AI revolution (ETH Frontiers of Embodied AI): Yann LeCun ETH “Frontiers of Embodied AI” lecture (29 May 2026), “World Models: Enabling the next AI revolution” — anchors the latent-predictive JEPA pole of the cluster’s central debate. Recording filed as a stub (no transcript available at filing time); pending /bud refresh once a transcript or write-up appears.
• [2026-06-10] AnchorWorld: Embodied Egocentric World Simulation with View-based Evolution Customization: AnchorWorld (Tsinghua + Kuaishou Kling) introduces anchor-view conditioning for egocentric interactive world simulation on Wan2.2 TI2V-5B: each anchor view bundles an RGB image, a 3D pose in a unified world coordinate, and a text evolution prompt, injected via in-context concat / pose-embedding addition / masked cross-attention; uses hybrid TPV+FPV training with projection-based action representation so the model learns full-body motion despite first-person view truncation; benchmarks against PlayerOne and Context-as-Memory (Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval) baselines on action control, static-scene consistency, dynamic evolution, and UE/real OOD generalization.
• [2026-06-09] The DAWN of World-Action Interactive Models: DAWN (COWARobot + SJTU + Hohai) formalizes World-Action Interactive Models (WAIMs): future world and future action are inferred as a coupled, recursively-refined pair rather than via parallel heads or a one-way predict-then-plan pipeline. Instantiated for autonomous driving with V-JEPA 2 Large + a token-space resampler, a causal-Transformer world predictor, and a DiT action denoiser that shares weights between proposal and refinement roles. Hits perception-free PDMS 89.1 on NAVSIM v1 and the best L2 (0.33 m avg) and collision rate (0.11%) on nuScenes; ablations isolate the gain to interactive coupling rather than latent compression or rollout alone, and show 2–3 s latent rollout captures most of the benefit — concrete instance of an action-conditional WFM that ducks Sitzmann’s chicken-and-egg by leveraging AV’s paired action-observation data.
• [2026-06-09] Robots Need More than VLA and World Models: Bou-Ammar et al. position paper arguing scaling VLA + WMs is insufficient for generalist robotics — proposes four missing “interfaces” (data autolabelling, embodiment retargeting, physics-grounded 3D world-model reasoning, and reward inference from video+language) as the actual bottleneck. Frames the WFM cluster as one of four required components rather than the central one; adds a vocabulary aid (the four-way interface decomposition) to the robotics-adjacent WFM literature.
• [2026-06-07] Scaling Robotic Manipulation via Structured World Models and Tactile Sensing — Yunzhu Li (Montreal Robotics): Yunzhu Li (Columbia) Montreal Robotics talk arguing scaling manipulation requires both structured world models (action-conditioned predictive models with explicit physical structure) and tactile/contact sensing — vision alone is the ceiling. Adds a fourth dimension to the wiki’s WFM framing — sensing modality — orthogonal to the renderer/simulator/planner split from Fei-Fei Li’s taxonomy. Position lecture, no quantitative results at filing time; transcript pending.
• [2026-06-06] Direct Video-Action Models — Causal Video Models Are Data-Efficient Robot Policy Learners: Rhoda AI introduces Direct Video-Action Models (DVA) — a from-scratch causal video model pre-trained on web video is used as the policy at every closed-loop control step, with a separate inverse-dynamics model translating predicted futures into end-effector motion; deployments report autonomous bimanual decanting (1.5 h uncut, 11 h of robot data) and Contico container breakdown (160 min uncut, 17 h of data). Strongest filed instance of the “video model as the policy” framing — concrete instantiation of the Sitzmann thesis, and complementary to the WFM-as-evaluator framing of 1XWM and Veo-as-Gemini-Robotics-eval.
• [2026-06-06] Learning Causal World Models: A Closed-Loop Recipe for Exploration, Representation, and Decision Making: Aether Labs position essay proposing a “causal WFM” recipe — closed-loop causality-guided exploration (probing for causal contrast), unified latent action (inverse-forward dynamics → cross-embodiment shared intervention language), and causal verifier (predictive dynamics energy used at inference to score video-generator rollouts for physical plausibility). Generalizes the Inference-time Physics Alignment of Video Generative Models with Latent World Models VJEPA-2-as-reward pattern from a single-signal reward to a structured dynamics-consistency critic, and answers one of Sitzmann’s open chicken-and-egg sub-problems with a concrete probing-as-data-collection prescription. Position essay, no benchmarks.
• [2026-06-06] Dream.exe: Can Video Generation Models Dream Executable Robot Manipulation?: Dream.exe operationalizes WFM-as-rollout as a benchmark — generated manipulation videos are extracted into trajectories and executed in a physics simulator; on 101 tasks × 8 models, several frontier video generators achieve measurable execution success and visual quality fails to predict the executability ranking.
• [2026-06-05] EgoTL: Egocentric Think-Aloud Chains for Long-Horizon Tasks: EgoTL evaluates world models on 60-second CoT-conditioned egocentric rollouts (CLIP-Score + VBench) and documents that locally photorealistic frames mask systematic long-horizon drift: object size/position change, occlusions go inconsistent, and cameras teleport across rooms — symptomatic of a missing explicit egocentric spatial backbone.
• [2026-06-05] Kairos-HomeWorld — whole-home 3D scene generation with object-level interactivity (ACE Robotics + CUHK MMLab + Shenzhen Loop Area Institute): ACE Robotics + CUHK MMLab + Shenzhen Loop Area Institute announce Kairos-HomeWorld — “first” unified framework for whole-home 3D scene generation with object-level interactivity from a single text prompt, plus an open-source 300K Chinese residential floor plans + 5K interactive home dataset purpose-built for embodied-AI robot training. Press coverage attributes a four-stage hierarchical pipeline (floorplan → 2D-to-3D lifting → recursive refinement → manipulable-object placement). Sibling to SAGE and WorldGen but with a whole-home + localized-corpus framing; product/press marker, no paper or code at filing time.
• [2026-06-03] VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generation: VideoPhy-2 stress-tests the “video models as world simulators” thesis on 197 real-world actions: Wan2.1-14B tops at 32.6% joint SA∧PC on full / 21.9% on hard, closed models (Sora, Ray2) trail top open models, and PC scores are nearly uncorrelated with aesthetics (r=0.09) or optical-flow motion quality (r=0.002), so surface-metric optimization doesn’t help.
• [2026-06-03] A Functional Taxonomy of World Models: Fei-Fei Li / World Labs position essay proposing a three-way functional split of “world model” grounded in the POMDP loop — renderer (action → observation pixels, optimizes visual fidelity), simulator (→ explicit state: geometry + physics + dynamics, serves humans and downstream programs), planner (observation → action). Argues simulation is the linchpin (most consequential, least mature) and that the three categories are converging into a unified world model; Marble (splats + collision meshes from one model) named as World Labs’ first move. Complements 3D as code (3D-as-durable-interface predecessor) and contrasts sharply with The flavor of the bitter lesson for computer vision (which argues explicit 3D / state should dissolve into generative-rollout WFMs, where this essay preserves state as a first-class output channel). First filed taxonomic vocabulary for the WFM cluster’s three-way split.
• [2026-06-03] NeuROK: Generative 4D Neural Object Kinematics: Object-level WFM datapoint on the explicit-physics-state branch — learns a low-dimensional kinematic latent space per object and rolls out dynamics via the Euler–Lagrange equation in that space, decoding back to geometry frame-by-frame. Composites generated 4D objects into a static 3D kitchen for an interactive world. Sits between Genesis World (classical multi-physics engine) and the neural-rollout WFMs (PAN, Genie 3) — explicit physics integrator, but the coordinates the integrator operates on are learned from a large 4D shape dataset.
• [2026-06-03] Ego1 — egocentric capture headset for Physical AI (General Intelligence Labs): Ego1 (General Intelligence Labs) — hardware + perception co-designed first-person headset targeting everyday manipulation as pretraining substrate for embodied/robotic models; data not yet released.
• [2026-06-03] Rethinking Video Generation Model for the Embodied World (ReVidGen / RBench / RoVid-X): RBench + RoVid-X (DAGroup PKU) — first filed robotics-oriented video-generation benchmark spanning 5 task domains × 4 embodiments, validated at Spearman 0.96 with human raters. Headline: robotics-specific models (Cosmos 2.5, DreamGen, Vidar, UnifoLM-WMA-0) trail general commercial video models (Wan 2.6 leads at 0.607); Sora v2 Pro ranks 17. Paired with RoVid-X, a 4M-clip 720p robotic video dataset with optical flow + diverse forms + diverse captions, framed as the data-side response to RBench’s diagnostic failures.
• [2026-06-03] HumanNet: Scaling Human-centric Video Learning to One Million Hours: HumanNet (PKU DAGroup / Ant) — ~1M-hour human-centric video corpus (first- and third-person) framed as embodied-pretraining substrate; validated by VLA ablation where 1,000 h egocentric video ≥ 100 h real-robot data and substantially closes the gap to 20,000 h real-robot baseline on the LingBot-VLA stack.
• [2026-06-03] Project Chrono — Open-Source Multi-Physics Simulation Engine: Project Chrono — classical BSD-3 multi-physics simulation library (rigid + FEM + granular DVI/DAE + fluid-solid + ODE) with a ROS2 sensor suite. Anchors the analytic-physics-engine baseline that recent neural WFMs are implicitly competing with on fidelity and generality.
• [2026-06-02] DynaFLIP: Rethinking Robotics Perception via Tri-Modal-Dynamics Guided Representation: DynaFLIP argues the vision encoder layer (not the world-model backbone, not the VLA head) is where robotics dynamics should be encoded, and trains a DINOv2/SigLIP-replacement encoder via image-transition × language × 3D-flow triplets — a third “WFM-adjacent” pattern alongside world-model-as-backbone and world-model-as-on-policy-advisor.
• [2026-06-02] Project Eden — persistent multiplayer world model that decouples world state from visual rendering (Tripo / VAST AI Research): Tripo / VAST AI Research announces Project Eden — a persistent multiplayer world model that decouples world state from rendering (research preview, demo video only, no architecture/benchmarks). Third independent 2026 datapoint for the “hoist shared state out of the network + condition learned renderer on it” pattern, alongside Odyssey’s Agora-1 (closed flagship) and MultiGen (open research). First filed multiplayer-WFM entry from a 3D-asset-focused lab.
• [2026-06-01] LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels: LeWorldModel (FAIR + Mila, LeCun & Balestriero) — first JEPA WFM trained stably end-to-end from raw pixels with only two losses (next-embedding prediction + SIGReg isotropic-Gaussian regularizer). 15M params, single GPU, 48× faster planning than DINOv2-backed WMs, beats PLDM on PushT by +18% and matches or beats DINO-WM even without DINO-WM’s proprioceptive input. Concrete instantiation of the Sitzmann “no explicit 3D, pixel-to-latent-prediction” thesis at research-budget scale; complements V-JEPA 2.1 (end-to-end + dense-token loss) and contrasts with DINO-world (frozen DINOv2 + learned dynamics only).
• [2026-06-01] Diffusion Model Predictive Control: D-MPC (DeepMind, NeurIPS 2024) — the cleanest filed factorized state-space WFM: separately-trained multi-step diffusion dynamics and action-proposal models with a sample-score-rank planner. Matches CQL/IQL on D4RL while enabling independent dynamics fine-tuning under hardware drift (Walker2d defective-actuator: 22.74 → 30.65 vs Diffuser 25.85 → 6.8). Counterweight to monolithic pixel-rollout WFMs (Genie 3, Cosmos Predict) and to joint state+action models (Diffuser); the inference-cost gap is closed by distilling the planner into a reactive MLP at 65.08 vs 65.98 normalized score.
• [2026-06-01] RenderFlow: Single-Step Neural Rendering via Flow Matching: RenderFlow (Disney Research, CVPR 2026) trains one flow-matching backbone for both directions of the rendering map: forward G-buffer → photoreal RGB in a single deterministic step, and inverse RGB → intrinsics via a lightweight adapter on the same weights. Reinforces the Generative World Renderer bidirectional-rendering framing — explicit G-buffer intermediates as a useful interface — against the Sitzmann no-explicit-3D position.
• [2026-06-01] EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the World: EgoVerse — first filed consortium-style human-data ecosystem (Georgia Tech / Stanford / UCSD / ETH / MIT / Meta RL / Mecka / Scale): 1,362 h / 80k episodes / 1,965 tasks / 240 scenes / 2,087 demonstrators, split into EgoVerse-A (controlled Aria captures, six shared flagship tasks, mirrored protocols) and EgoVerse-I (industry in-the-wild). Living dataset (EgoDB) with continuous ingestion plus a phone-capture path. Cross-lab co-training study reports human-data scaling helps when aligned with the robot objective — first filed datapoint of a consortium-coordinated human-data initiative as a counterweight to single-company scaling plays like Build AI’s Egocentric-1M.
• [2026-06-01] Nyx — GPU path-traced renderer plugin for Genesis World: Nyx renderer release — path-traced GPU renderer purpose-built for robotics evaluation with first-class 3D Gaussian splat support and the IBL+splat-under-path-tracing reconciliation problem framed as the hard one. Sharp explicit-rendering counterpoint to neural-rendering WFMs (Mirage LSD, Runway Gen-4.5) targeting the same “millions of frames that match a real camera” requirement.
• [2026-06-01] Genesis World — Simulation platform for general-purpose robotics & embodied AI learning: Open-source release (Apache 2.0) of the Genesis World multi-physics simulator — rigid + FEM + MPM + SPH + PBD + IPC in one scene/state with three swappable couplers, multi-platform (NVIDIA + AMD + macOS + CPU). The explicit-physics-engine counterpart to neural WFMs like HY-World 1.5 and PAN; physical-robotics analog to the terminal/coding env-as-package pattern in RL Environment Platforms.
• [2026-06-01] The Role of Simulation in Scalable Robotics, Genesis World 1.0, and the Path Forward: Genesis World 1.0 — Genesis AI’s open-source robotics simulation stack reports Pearson 0.8996 sim-to-real correlation and 45% smaller FID reality gap vs next-best simulator, explicitly arguing simulation’s first job is evaluation (not data generation) and that explicit-physics + path-traced rendering remains the trustworthy substrate for robotics iteration. The strongest filed counter-bet to the Sitzmann “explicit 3D dissolves into neural pre-training” thesis.
• [2026-06-01] PAI-Bench: A Comprehensive Benchmark For Physical AI: PAI-Bench provides a 2,808-case Physical AI benchmark covering video generation, conditional video generation, and video understanding across AV / robotics / industry / ego-view / physical common sense. Headline result: 15 evaluated VGMs (including Veo3, Wan2.2, Cosmos-Predict, MAGI-1, HunyuanVideo-I2V, CogVideoX, LTX) match real-source visual quality (~78.0) but no model reaches the source Domain Score of 89.8, with the Robotics sub-domain the hardest (Veo3 72.1 vs source 96.6). Quantifies what world-foundation-model proponents have argued must be the next push: physics, not visuals.
• [2026-06-01] NVIDIA Launches Cosmos 3, the Open Frontier Foundation Model for Physical AI: NVIDIA launches Cosmos 3 (Super 32B + Nano 8B, OpenMDW 1.1) — a mixture-of-transformers omnimodel pairing a reasoning transformer with a generation transformer that emits text, video, audio, and numerical action (joint angles, gripper positions, trajectory points). Collapses the prior Cosmos Reason 2 + Predict 2.5 + Transfer 2.5 + GR00T-VLA stack (NVIDIA Unveils New Open Models, Data and Tools to Advance AI Across Every Industry) into a single open model. Reported #1 on Artificial Analysis open-weights, Physics-IQ, R-Bench, PAI-Bench, VANTAGE-Bench, RoboLab, RoboArena — no closed-flagship comparison disclosed.
• [2026-06-01] Cosmos 3: Omnimodal World Models for Physical AI: NVIDIA Cosmos 3 — first filed open WFM that unifies VLM + image/video/audio generation + VLA/world-action policy under one Mixture-of-Transformers backbone (dual-tower AR + DM layers with joint attention), released under OpenMDW-1.1 in Super and Nano variants with five SDG datasets and the Cosmos-HUE benchmark; reported as best open-source T2I and I2V on Artificial Analysis and best policy on RoboArena at submission.
• [2026-06-01] How Cosmos 3 Helps Physical AI Think Before It Acts: NVIDIA Cosmos 3 launch (GTC Taipei, June 1 2026) — a single mixture-of-transformers omnimodel that pairs a reasoning block with an expert generation block, emitting text, video, ambient sound, AND native action trajectories (joint angles, gripper positions). Collapses the prior Cosmos Reason 2 + Predict 2.5 + Transfer 2.5 three-model stack into one model, and adds the NVIDIA Cosmos Coalition (Agile Robots, Black Forest Labs, Generalist, LTX, Runway, Skild AI) as an open-WFM alignment against closed flagships like Genie 3 and Runway GWM-1. Reports leaderboard-first claims on RoboLab, RoboArena, VANTAGE-Bench, TAR, Physics-IQ, R-Bench, PAI-Bench — no numbers disclosed, but a 7-leaderboard sweep at a single open release is itself notable.
• [2026-05-28] Featurising Pixels from Dynamic 3D Scenes with Linear In-Context Learners: LILA trains a DPT decoder on top of a frozen DINOv2 encoder via linear-in-context consistency over depth/flow cue maps, producing pixel-resolution feature grids that outperform FlowFeat / LoftUp on DAVIS-VOS, NYUv2 normals, and COCO-Stuff — extending the latent self-supervised video-representation line (V-JEPA 2.1, DINO-world) to dense pixel outputs.
• [2026-05-27] stable-worldmodel: A Platform for Reproducible World Modeling Research and Evaluation: stable-worldmodel (Meta FAIR / Brown / Mila incl. LeCun & Balestriero) — an open-source platform unifying Lance-based data loading (MP4 / HDF5 / LeRobot), reference WFM baselines and planning solvers, and environments with controllable visual/geometric/physical factors of variation for systematic OOD evaluation. First filed infrastructure paper aimed at the WFM cluster — proposes to pay the “fair-comparison tax” Sitzmann named.
• [2026-05-27] Planning at Inference: MCTS Test-Time Scaling for Long Video Generation: Demonstrates Cosmos-Predict2 as a substrate for inference-time MCTS scaling: applies modular search over continuations on top of NVIDIA’s WFM without retraining the base, generating coherent >20s rollouts.
• [2026-05-25] WildPose: A Unified Framework for Robust Pose Estimation in the Wild: Uses a frozen MASt3R backbone as a 3D-aware perception frontend for monocular camera pose estimation, then layers a learned update operator + differentiable bundle adjustment on top — a “freeze the backbone, add explicit optimization” recipe that contrasts with VGGT-Ω’s “scale end-to-end” bet.
• [2026-05-23] Introducing Runway Labs: Runway announces Runway Labs, an internal incubator for prototyping products on top of its GWM-1 stack, led by co-founder Alejandro Matamala Ortiz. Strategic/hiring announcement only — no technical disclosure. First filed datapoint of a closed-flagship lab publicly committing to a portfolio productization strategy for WFMs, in contrast to DeepMind’s single-product Project Genie launch and Waymo’s single-vertical Genie-3 specialization. Useful primarily as a market-signal marker.
• [2026-05-23] The flavor of the bitter lesson for computer vision: Vincent Sitzmann (MIT, Scene Representation Group) position essay arguing that conventional computer-vision intermediate representations — 3D reconstructions (NeRF, point clouds, SDFs), SE(3) camera poses, segmentation, optical flow — are historical artifacts of scoping tractable sub-problems, and that the Bitter Lesson predicts they will dissolve into end-to-end perception–action loops with video-generative world models as the most promising current pre-training objective. First filed foundation-layer normative argument for the WFM cluster — sharp contrast with the still-3D-heavy 4D Scene Generation design pattern. Notable that the author’s own group built much of the modern 3D-representation stack (SIREN, scene representation networks, light-field networks), so the argument against 3D carries unusual weight.
• [2026-05-23] Awesome World Models: A curated awesome-list (CC0, 1k+ GitHub stars in 30 days) cataloging world-model papers across 10 sections — definitions, surveys, game simulation (pixel & 3D-mesh), autonomous driving, embodied AI, science, positions, theory, general approaches, and evaluation. First filed navigational hub for the WFM cluster; useful as a map when surveying a sub-area, but does not synthesize. Confirms the cross-domain umbrella framing and pins canonical definitional sources (Ha & Schmidhuber 2018; LeCun 2022). The list catalogs surveys the wiki has not filed individually — including “A Comprehensive Survey on World Models for Embodied AI” (arXiv:2510.16732) and several driving-specific surveys.
• [2026-05-23] RISE-Video: Can Video Generators Decode Implicit World Rules?: RISE-Video — a reasoning-oriented TI2V benchmark (467 human-annotated samples × 8 knowledge categories × 4 metrics) that probes whether video generators “decode implicit world rules”. 11-model leaderboard separates a relaxed weighted score (cluster 49.5–79.4) from a strict accuracy (cluster 1.9–22.5), making this the first filed benchmark-as-probe of the WFM thesis on closed and open video generators. Frontier closed models (Hailuo 2.3 / Veo 3.1 / Sora-2) lead but are tightly bunched and never exceed 22.5% strict accuracy.
• [2026-05-23] The Waymo World Model: A New Frontier For Autonomous Driving Simulation: Waymo announces a Genie-3-derived driving simulator that emits synchronized camera + lidar (4D point clouds), exposes driving-action / scene-layout / language control, supports dashcam-to-sim conversion, and includes an efficient long-rollout variant. First filed datapoint of a closed-flagship WFM being post-trained into a vertical foundation model with a new output modality. No architecture, parameter counts, or benchmarks disclosed; pure product/research framing.
• [2026-05-22] Project Genie: Experimenting with infinite, interactive worlds: Google DeepMind productizes Genie 3 as Project Genie — a Google Labs prototype for US Google AI Ultra subscribers (Jan 29, 2026). Real-time path-ahead generation at 20–24 fps / 720p, 60-second cap, World Sketching / Exploration / Remixing UI. Product framing only; no architecture, weights, or benchmarks disclosed. The closed-source flagship counterpart to LingBot-World in the same time window.
• [2026-05-22] Inference-time Physics Alignment of Video Generative Models with Latent World Models: Demonstrates VJEPA-2 functioning as a frozen, off-the-shelf reward model for physics-plausibility — the world model’s predictive surprise outperforms VideoMAE pixel-reconstruction and VLM-judge rewards as a steering signal for video generation. Wins ICCV 2025 PhysicsIQ Challenge with MAGI-1 + ∇+BoN at 62.0% V2V.
• [2026-05-22] Action100M: A Large-scale Video Action Dataset: Releases the data backing Meta’s VL-JEPA — 147M annotated segments from 1.2M HowTo100M videos. VL-JEPA outperforms CLIP-style image–text encoders on motion-centric benchmarks at far fewer samples-seen, pinning the WFM win to predictive-latent training.
• [2026-05-22] NVIDIA Unveils New Open Models, Data and Tools to Advance AI Across Every Industry: NVIDIA’s CES 2026 omnibus release of Cosmos Reason 2 / Predict 2.5 / Transfer 2.5 (open WFM stack), Isaac GR00T N1.6 (humanoid VLA built on Cosmos Reason), and Alpamayo 1 (AV VLA). Establishes the “WFM as backbone, VLA as task-head” architectural pattern.

Open questions

• Which axis of WFM-ness actually matters for downstream use — generation quality (Cosmos Predict / Transfer for synthetic data, video models themselves) or prediction quality in latent space (VJEPA-2 as a reward, VL-JEPA as a representation)? The two filed Meta entries lean predictive; NVIDIA’s stack offers both; Waymo’s Genie-3-derived simulator leans generative-rollout; RISE-Video probes the generative side and finds it reasoning-bound at 22.5% strict accuracy. Sitzmann argues the generative-rollout side is the right pre-training objective and that latent-predictive WFMs are complementary but not sufficient — adding a normative tilt to a previously empirical question.
• What’s the right reward-model architecture for video generation? VJEPA-2 surprise beats VLM-judge by a large margin on PhysicsIQ but is text-agnostic — semantic-adherence drops slightly under WMReward on VideoPhy T2V. A composable physics + semantics reward is the obvious next step. RISE-Video uses an LMM-as-judge for scoring, but its own absolute reliability against human raters is not validated in the released artifact.
• Does the “WFM as backbone” pattern transfer beyond robotics/AV — e.g. can Cosmos Reason 2 or VJEPA-2 plug in as the perception/reasoning prior for generation-time systems (video editing, controllable T2V) the way they already do for policy-time systems (VLA)?
• How tightly coupled are the released WFMs to their training corpora? Action100M is HowTo100M-derived (instructional/English), so VL-JEPA’s motion-benchmark wins may not generalize to social/sports/ego non-procedural domains. Cosmos’s training data is not detailed publicly.
• What does the closed flagship (Genie 3 / Project Genie) actually do that the open Wan-based recipe (LingBot-World, Context Forcing, PFP, FlowAct-R1) doesn’t? Genie 3 claims “breakthrough consistency” with no numbers; LingBot-World reports specific VBench gains over Yume-1.5 / HY-World-1.5 but cannot be directly compared. RISE-Video does not include Genie 3 in its leaderboard, so the closed-flagship vs frontier-TI2V gap on reasoning is also unmeasured.
• How is camera→lidar modality transfer actually trained? Waymo’s post-training claim — that a Genie-3-style 2D-video prior can be adapted to emit structurally different modalities (4D point clouds) — is asserted but undocumented. If it works, this opens the door to specializing closed flagships into modalities they never trained on (audio, depth, segmentation, force) by analogous post-training procedures.
• Is the RISE-Video Reasoning-Alignment metric (LMM-judge over human-designed per-sample questions) actually a faithful operationalization of “decoded implicit world rules” — i.e., does it correlate with human strict-accuracy judgment? No filed paper measures this; if the metric is noisy, the 22.5% ceiling could be an artifact rather than a real capability boundary.
• The Awesome-World-Models repo’s surveys section catalogs ≥10 WFM-related survey papers the wiki has not filed — including dedicated surveys on world models for embodied AI (arXiv:2510.16732), 3D/4D world modeling (arXiv:2509.07996), and “From Masks to Worlds” (arXiv:2510.20668). Should any of these be filed individually as concept-anchoring synthesis sources, or treated as second-order references? Open until a reader needs synthesis the filed papers don’t cover.
• The chicken-and-egg of action-conditional WFMs (named by Sitzmann): training interactive world models requires paired action–observation data, which is precisely the resource embodied AI lacks except in self-driving. Video pre-training is one candidate bridge but not a proven one. Open sub-problems Sitzmann explicitly names: intrinsic motivation, exploration, long-horizon memory, continual learning, real-time control with large models.
• Does the wiki’s 4D-generation cluster — explicit 3D / 4D Gaussian state plus a frozen diffusion backbone (NeoVerse, VerseCrafter, SpaceTimePilot) — survive the Bitter-Lesson critique? Sitzmann’s essay is the strongest articulation on file of why this whole design family may be a local optimum. No filed paper directly compares an explicit-3D-state video generator against a no-3D generative-rollout WFM on the same benchmark; until that exists, the question is open.
• Does the portfolio-incubator strategy (Runway Labs) actually produce more durable products than the single-product (Project Genie) or single-vertical (Waymo) approaches? Filed strategy announcements are too early to tell — open until Runway Labs ships, fails to ship, or a competitor announces a comparable internal-incubator unit and the wiki can compare.

Papers

2026-05

• π*0.6: a VLA That Learns From Experience (RECAP) — π*0.6: a VLA That Learns From Experience (RECAP) (published 2026-05-25)
• Saining Xie joining AMI Labs (LeCun's new world-model lab) — announcement — Saining Xie joining AMI Labs (LeCun’s new world-model lab) — announcement (published 2026-05-23)
• Introducing Gemini Omni — Introducing Gemini Omni (published 2026-05-19)
• Agora-1: The Multi-Agent World Model — Agora-1: The Multi-Agent World Model (published 2026-05-18)
• VGGT-Ω: Scaling Feed-Forward Reconstruction for Static and Dynamic Scenes — VGGT-Ω: Scaling Feed-Forward Reconstruction for Static and Dynamic Scenes (published 2026-05-18)
• DeltaToken: 100× Token-Efficient Video Representation for World Models — DeltaToken: 100× Token-Efficient Video Representation for World Models (published 2026-05-16)
• SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer — SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer (published 2026-05-14)

2026-04

• A Very Big Video Reasoning Suite (VBVR) — A Very Big Video Reasoning Suite (VBVR) (published 2026-04-28)
• Omni: Context Unrolling in Omni Models (ByteDance Seed) — Omni: Context Unrolling in Omni Models (ByteDance Seed) (published 2026-04-24)
• Vista4D: Video Reshooting with 4D Point Clouds — Vista4D: Video Reshooting with 4D Point Clouds (published 2026-04-23)
• HumanScore: Benchmarking Human Motions in Generated Videos — HumanScore: Benchmarking Human Motions in Generated Videos (published 2026-04-22)
• Image Generators are Generalist Vision Learners (Vision Banana) — Image Generators are Generalist Vision Learners (Vision Banana) (published 2026-04-22)
• Introducing Odyssey-2 Max: Scaled World Simulation — Introducing Odyssey-2 Max: Scaled World Simulation (published 2026-04-21)
• HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds — HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds (published 2026-04-16)
• π0.7: A Steerable Robotic Foundation Model with Emergent Compositional Generalization — π0.7: A Steerable Robotic Foundation Model with Emergent Compositional Generalization (published 2026-04-16)
• Seedance 2.0: Advancing Video Generation for World Complexity — Seedance 2.0: Advancing Video Generation for World Complexity (published 2026-04-15)
• Lyra 2.0: Explorable Generative 3D Worlds — Lyra 2.0: Explorable Generative 3D Worlds (published 2026-04-15)
• Envisioning the Future, One Step at a Time (Myriad) — Envisioning the Future, One Step at a Time (Myriad) (published 2026-04-10)
• Black Forest Labs — 'visual intelligence and the open infrastructure that powers it' as the next step — Black Forest Labs — ‘visual intelligence and the open infrastructure that powers it’ as the next step (published 2026-04-10)
• Egocentric-1M — largest egocentric video dataset (Build AI / Eddy Xu announcement) — Egocentric-1M — largest egocentric video dataset (Build AI / Eddy Xu announcement) (published 2026-04-08)
• LatentUM: Unleashing the Potential of Interleaved Cross-Modal Reasoning via a Latent-Space Unified Model — LatentUM: Unleashing the Potential of Interleaved Cross-Modal Reasoning via a Latent-Space Unified Model (published 2026-04-02)
• VOID: Video Object and Interaction Deletion — VOID: Video Object and Interaction Deletion (published 2026-04-02)
• Generative World Renderer — Generative World Renderer (published 2026-04-02)
• Forecasting Motion in the Wild — Forecasting Motion in the Wild (published 2026-04-01)

2026-03

• Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory — Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory (published 2026-03-27)
• Unitree open-sources UnifoLM-WBT-Dataset — humanoid whole-body teleoperation dataset — Unitree open-sources UnifoLM-WBT-Dataset — humanoid whole-body teleoperation dataset (published 2026-03-27)
• OmniWeaving: Towards Unified Video Generation with Free-form Composition and Reasoning — OmniWeaving: Towards Unified Video Generation with Free-form Composition and Reasoning (published 2026-03-25)
• WildWorld: A Large-Scale Dataset for Dynamic World Modeling with Actions and Explicit State toward Generative ARPG — WildWorld: A Large-Scale Dataset for Dynamic World Modeling with Actions and Explicit State toward Generative ARPG (published 2026-03-24)
• Wan 2.7 upcoming features preview (video editing, V2V, real-time) — Wan 2.7 upcoming features preview (video editing, V2V, real-time) (published 2026-03-22)
• Physion-Eval: Evaluating Physical Realism in Generated Video via Human Reasoning — Physion-Eval: Evaluating Physical Realism in Generated Video via Human Reasoning (published 2026-03-20)
• The Pulse of Motion: Measuring Physical Frame Rate from Visual Dynamics — The Pulse of Motion: Measuring Physical Frame Rate from Visual Dynamics (published 2026-03-15)
• V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning — V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning (published 2026-03-15)
• DVD: Deterministic Video Depth Estimation with Generative Priors — DVD: Deterministic Video Depth Estimation with Generative Priors (published 2026-03-12)
• Introducing Runway Labs — Introducing Runway Labs (published 2026-03-11)
• MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Engines — MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Engines (published 2026-03-06)
• Beyond Language Modeling: An Exploration of Multimodal Pretraining — Beyond Language Modeling: An Exploration of Multimodal Pretraining (published 2026-03-03)
• 3D as code — 3D as code (published 2026-03-03)

2026-02

• Summer-22B: A Systematic Approach to Dataset Engineering and Training at Scale for Video Foundation Model — Summer-22B: A Systematic Approach to Dataset Engineering and Training at Scale for Video Foundation Model (published 2026-02-26)
• A Very Big Video Reasoning Suite — A Very Big Video Reasoning Suite (published 2026-02-23)
• The First Fully General Computer Action Model (FDM-1) — The First Fully General Computer Action Model (FDM-1) (published 2026-02-23)
• Solaris: Building a Multiplayer Video World Model in Minecraft — Solaris: Building a Multiplayer Video World Model in Minecraft (published 2026-02-23)
• Xmax X1 — Real-Time Interactive Video Model (Product Announcement) — Xmax X1 — Real-Time Interactive Video Model (Product Announcement) (published 2026-02-08)
• The Waymo World Model: A New Frontier For Autonomous Driving Simulation — The Waymo World Model: A New Frontier For Autonomous Driving Simulation (published 2026-02-06)
• RISE-Video: Can Video Generators Decode Implicit World Rules? — RISE-Video: Can Video Generators Decode Implicit World Rules? (published 2026-02-05)
• The flavor of the bitter lesson for computer vision — The flavor of the bitter lesson for computer vision (published 2026-02-01)

2026-01

• Advancing Open-source World Models (LingBot-World) — Advancing Open-source World Models (LingBot-World) (published 2026-01-29)
• Project Genie: Experimenting with infinite, interactive worlds — Project Genie: Experimenting with infinite, interactive worlds (published 2026-01-29)
• Inference-time Physics Alignment of Video Generative Models with Latent World Models — Inference-time Physics Alignment of Video Generative Models with Latent World Models (published 2026-01-15)
• Action100M: A Large-scale Video Action Dataset — Action100M: A Large-scale Video Action Dataset (published 2026-01-15)
• NVIDIA Unveils New Open Models, Data and Tools to Advance AI Across Every Industry — NVIDIA Unveils New Open Models, Data and Tools to Advance AI Across Every Industry (published 2026-01-05)
• NitroGen: An Open Foundation Model for Generalist Gaming Agents — NitroGen: An Open Foundation Model for Generalist Gaming Agents (published 2026-01-04)
• E-RayZer: Self-supervised 3D Reconstruction as Spatial Visual Pre-training — E-RayZer: Self-supervised 3D Reconstruction as Spatial Visual Pre-training (published 2026-01-01)
• PV-VAE: Video Generation with Predictive Latents — PV-VAE: Video Generation with Predictive Latents (published 2026-01-01)

2025-12

• Yume-1.5: A Text-Controlled Interactive World Generation Model — Yume-1.5: A Text-Controlled Interactive World Generation Model (published 2025-12-22)
• HY-World 1.5 (WorldPlay): A Systematic Framework for Interactive World Modeling with Real-Time Latency and Geometric Consistency — HY-World 1.5 (WorldPlay): A Systematic Framework for Interactive World Modeling with Real-Time Latency and Geometric Consistency (published 2025-12-17)
• Evaluating Gemini Robotics Policies in a Veo World Simulator — Evaluating Gemini Robotics Policies in a Veo World Simulator (published 2025-12-11)
• Sharp Monocular View Synthesis in Less Than a Second — Sharp Monocular View Synthesis in Less Than a Second (published 2025-12-11)
• Runway GWM-1 launch broadcast — world model, avatars, audio, multi-shot editing — Runway GWM-1 launch broadcast — world model, avatars, audio, multi-shot editing (published 2025-12-11)
• SIMA 2: A Generalist Embodied Agent for Virtual Worlds — SIMA 2: A Generalist Embodied Agent for Virtual Worlds (published 2025-12-04)
• TV2TV: A Unified Framework for Interleaved Language and Video Generation — TV2TV: A Unified Framework for Interleaved Language and Video Generation (published 2025-12-04)
• Data-regularized Reinforcement Learning for Diffusion Models at Scale — Data-regularized Reinforcement Learning for Diffusion Models at Scale (published 2025-12-03)
• RULER-Bench: Probing Rule-based Reasoning Abilities of Next-level Video Generation Models for Vision Foundation Intelligence — RULER-Bench: Probing Rule-based Reasoning Abilities of Next-level Video Generation Models for Vision Foundation Intelligence (published 2025-12-02)
• Generative Action Tell-Tales: Assessing Human Motion in Synthesized Videos — Generative Action Tell-Tales: Assessing Human Motion in Synthesized Videos (published 2025-12-01)
• D4RT: Efficiently Reconstructing Dynamic Scenes One D4RT at a Time — D4RT: Efficiently Reconstructing Dynamic Scenes One D4RT at a Time (published 2025-12-01)
• RELIC: Interactive Video World Models with Long-Horizon Memory — RELIC: Interactive Video World Models with Long-Horizon Memory (published 2025-12-01)
• Introducing Runway Gen-4.5 — Introducing Runway Gen-4.5 (published 2025-12-01)

2025-11

• Describe Anything Anywhere At Any Moment — Describe Anything Anywhere At Any Moment (published 2025-11-29)
• Captain Safari: A World Engine with Pose-Aligned 3D Memory — Captain Safari: A World Engine with Pose-Aligned 3D Memory (published 2025-11-28)
• STARFlow-V: End-to-End Video Generative Modeling with Normalizing Flows — STARFlow-V: End-to-End Video Generative Modeling with Normalizing Flows (published 2025-11-25)
• Infinite Context Length with Global but Constant Attention Memory — Infinite Context Length with Global but Constant Attention Memory (published 2025-11-24)
• WorldGen: From Text to Traversable and Interactive 3D Worlds — WorldGen: From Text to Traversable and Interactive 3D Worlds (published 2025-11-21)
• Solving Spatial Supersensing Without Spatial Supersensing — Solving Spatial Supersensing Without Spatial Supersensing (published 2025-11-20)
• Kandinsky 5.0: A Family of Foundation Models for Image and Video Generation — Kandinsky 5.0: A Family of Foundation Models for Image and Video Generation (published 2025-11-19)
• Depth Anything 3: Recovering the Visual Space from Any Views — Depth Anything 3: Recovering the Visual Space from Any Views (published 2025-11-13)
• Lumine: An Open Recipe for Building Generalist Agents in 3D Open Worlds — Lumine: An Open Recipe for Building Generalist Agents in 3D Open Worlds (published 2025-11-12)
• PAN: A World Model for General, Interactable, and Long-Horizon World Simulation — PAN: A World Model for General, Interactable, and Long-Horizon World Simulation (published 2025-11-12)
• Lucy Edit 1.1 Dev: Open-Weight Instruction-Guided Video Editing on Wan2.2 5B — Lucy Edit 1.1 Dev: Open-Weight Instruction-Guided Video Editing on Wan2.2 5B (published 2025-11-07)
• Cambrian-S: Towards Spatial Supersensing in Video — Cambrian-S: Towards Spatial Supersensing in Video (published 2025-11-06)
• SAGE: Scalable Agentic 3D Scene Generation for Embodied AI — SAGE: Scalable Agentic 3D Scene Generation for Embodied AI (published 2025-11-06)

2025-10

• Emu3.5: Native Multimodal Models are World Learners — Emu3.5: Native Multimodal Models are World Learners (published 2025-10-30)
• Build AI's early egocentric release — 400k action labels, 2.5k clips, 2× open-source dataset size (Eddy Xu tweet, Oct 22 2025) — Build AI’s early egocentric release — 400k action labels, 2.5k clips, 2× open-source dataset size (Eddy Xu tweet, Oct 22 2025) (published 2025-10-22)
• FlashWorld: High-quality 3D Scene Generation within Seconds — FlashWorld: High-quality 3D Scene Generation within Seconds (published 2025-10-15)
• What If: Understanding Motion Through Sparse Interactions — What If: Understanding Motion Through Sparse Interactions (published 2025-10-14)
• ChronoEdit: Towards Temporal Reasoning for Image Editing and World Simulation — ChronoEdit: Towards Temporal Reasoning for Image Editing and World Simulation (published 2025-10-05)
• Awesome World Models — Awesome World Models (published 2025-10-01)

2025-09

• Sora 2 is here — Sora 2 is here (published 2025-09-30)
• mmmbchang on Sora 2 physics — credit to teammate behind the breakthrough — mmmbchang on Sora 2 physics — credit to teammate behind the breakthrough (published 2025-09-30)
• NewtonGen: Physics-Consistent and Controllable Text-to-Video Generation via Neural Newtonian Dynamics — NewtonGen: Physics-Consistent and Controllable Text-to-Video Generation via Neural Newtonian Dynamics (published 2025-09-25)
• Gemini Robotics 1.5 brings AI agents into the physical world — Gemini Robotics 1.5 brings AI agents into the physical world (published 2025-09-25)
• Video models are zero-shot learners and reasoners — Video models are zero-shot learners and reasoners (published 2025-09-24)
• OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modeling — OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modeling (published 2025-09-15)
• SpatialVID: A Large-Scale Video Dataset with Spatial Annotations — SpatialVID: A Large-Scale Video Dataset with Spatial Annotations (published 2025-09-11)

2025-08

• DINOv3: Self-supervised learning for vision at unprecedented scale — DINOv3: Self-supervised learning for vision at unprecedented scale (published 2025-08-14)
• Yan: Foundational Interactive Video Generation — Yan: Foundational Interactive Video Generation (published 2025-08-12)
• Genie 3: A new frontier for world models — Genie 3: A new frontier for world models (published 2025-08-05)

2025-07

• Back to the Features: DINO as a Foundation for Video World Models — Back to the Features: DINO as a Foundation for Video World Models (published 2025-07-25)
• MirageLSD: The First Live-Stream Diffusion AI Video Model — MirageLSD: The First Live-Stream Diffusion AI Video Model (published 2025-07-17)
• NeuralOS: Towards Simulating Operating Systems via Neural Generative Models — NeuralOS: Towards Simulating Operating Systems via Neural Generative Models (published 2025-07-11)
• Moonvalley Marey — controllable AI video model with 360° camera, pose, and motion transfer — Moonvalley Marey — controllable AI video model with 360° camera, pose, and motion transfer (published 2025-07-08)
• Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrations — Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrations (published 2025-07-01)

2025-06

• facebookresearch/vjepa2 — official PyTorch code and checkpoints for V-JEPA 2 and V-JEPA 2.1 — facebookresearch/vjepa2 — official PyTorch code and checkpoints for V-JEPA 2 and V-JEPA 2.1 (published 2025-06-25)
• Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Condition — Hunyuan-GameCraft: High-dynamic Interactive Game Video Generation with Hybrid History Condition (published 2025-06-20)
• Sekai: A Video Dataset towards World Exploration — Sekai: A Video Dataset towards World Exploration (published 2025-06-18)
• Seedance 1.0: Exploring the Boundaries of Video Generation Models — Seedance 1.0: Exploring the Boundaries of Video Generation Models (published 2025-06-12)
• Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generation — Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generation (published 2025-06-11)
• Introducing the V-JEPA 2 world model and new benchmarks for physical reasoning — Introducing the V-JEPA 2 world model and new benchmarks for physical reasoning (published 2025-06-11)
• Product of Experts for Visual Generation — Product of Experts for Visual Generation (published 2025-06-10)
• Autoencoders for Diffusion: A Deep Dive — Autoencoders for Diffusion: A Deep Dive (published 2025-06-07)
• Video World Models with Long-term Spatial Memory — Video World Models with Long-term Spatial Memory (published 2025-06-05)
• Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generation — Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generation (published 2025-06-04)
• AGI Is Not Multimodal — AGI Is Not Multimodal (published 2025-06-04)
• Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval — Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval (published 2025-06-03)

2025-05

• Object Concepts Emerge from Motion — Object Concepts Emerge from Motion (published 2025-05-27)

2025-04

• Direct Motion Models for Assessing Generated Videos — Direct Motion Models for Assessing Generated Videos (published 2025-04-30)

2024-10

• 1X World Model (1XWM) — action-controllable world model for humanoid policy evaluation — 1X World Model (1XWM) — action-controllable world model for humanoid policy evaluation (published 2024-10-10)
• Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation (PhyGenBench) — Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation (PhyGenBench) (published 2024-10-07)

2023-12

• WonderJourney: Going from Anywhere to Everywhere — WonderJourney: Going from Anywhere to Everywhere (published 2023-12-06)

Unknown date

• Magica: AI UGC game engine (Dynamics Lab) — Magica: AI UGC game engine (Dynamics Lab) (published unknown)
• ECHO — Environment Prediction Loss for GRPO CLI Agents (microsoft/echo-rl) — ECHO — Environment Prediction Loss for GRPO CLI Agents (microsoft/echo-rl) (published unknown)
• Veo 3 Tech Report — Veo 3 Tech Report (published unknown)
• AI humans from Runway (Runway Characters / GWM Avatars product announcement) — AI humans from Runway (Runway Characters / GWM Avatars product announcement) (published unknown)