Skip to content
Bud's Wiki

SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models

TL;DR

Section titled “TL;DR”

SolarWM is a fully-open interactive video world model release that unifies data preparation, backbone-agnostic training, and long-horizon inference under one framework. The data engine ingests 1.43M canonical clips from 10 heterogeneous sources into a frame-aligned contract (visual observations + metric camera geometry + captions + quality metadata + selection decisions + provenance) that decouples source processing from mixture construction. Under shared camera-conditioning / training / inference interfaces, the same recipe instantiates four 5B–33B models on top of Wan2.2, LTX-2.5, and MiniMax-H3 while preserving each backbone’s native representations. A three-stage recipe (bidirectional adaptation → teacher-forced autoregressive initialization → distribution matching distillation) yields causal models that interact in real time over rollouts from minutes to hours despite training only on 5-second sequences.

Key claims

Section titled “Key claims”
  • Naive multi-source data mixing produces inconsistent supervision because datasets differ in temporal scale, camera geometry, visual quality, motion, and captioning styles [§Abstract].
  • A reconfigurable multi-source data engine can convert 1.43M canonical clips across 10 datasets into a unified, frame-aligned contract covering visual observations, metric camera geometry, captions, quality metadata, selection decisions, and provenance, decoupling source processing from mixture construction [§Abstract].
  • Under shared camera-conditioning, training, and inference interfaces, the same recipe instantiates four models spanning 5B–33B parameters on Wan2.2, LTX-2.5, and MiniMax-H3 while preserving each backbone’s native representations and objectives [§Abstract].
  • A three-stage training recipe — bidirectional adaptation → teacher-forced autoregressive initialization → distribution matching distillation — produces causal models that support real-time interaction [§Abstract].
  • The resulting causal models generalize from 5-second training sequences to rollouts spanning minutes to hours [§Abstract].

Method

Section titled “Method”

SolarWM addresses two coupled sources of variance that make cross-source, cross-backbone world-model training difficult: heterogeneity in raw data (temporal scale, camera geometry, visual quality, motion, caption style) and heterogeneity in generators (representations, architectures, objectives). The paper’s two-tier answer is (i) a data engine that decouples per-source processing from downstream mixture construction by emitting a frame-aligned contract per clip — pixels, metric camera geometry, captions, quality metadata, selection decisions, and provenance — and (ii) a backbone-native adaptation framework that exposes shared camera-conditioning, training, and inference interfaces so the same recipe can wrap Wan2.2, LTX-2.5, and MiniMax-H3 without touching their native representations or losses.

The training pipeline runs in three stages: bidirectional adaptation (align the pretrained backbone to the unified frame-aligned contract), teacher-forced autoregressive initialization (convert the bidirectional model to a causal roll-out sampler), and distribution matching distillation (compress the multi-step causal teacher into a few-step student that interacts in real time). The same three stages are applied across all four instantiated models (5B–33B on the three backbones). Rollouts remain stable from minutes to hours despite the 5-second training window.

Results

Section titled “Results”
  • 1.43M canonical clips consolidated from 10 datasets under the unified frame-aligned contract [§Abstract].
  • Four backbone-native model instantiations at 5B–33B parameters covering Wan2.2, LTX-2.5, and MiniMax-H3 [§Abstract].
  • Real-time interaction over minute-to-hour rollouts from models trained only on 5-second sequences [§Abstract].

Why it’s interesting

Section titled “Why it’s interesting”

SolarWM is the first filed open interactive-WFM release that treats cross-backbone portability as a first-class objective — three unrelated video generators (Wan2.2, LTX-2.5, MiniMax-H3) train under the same three-stage recipe with matched camera-conditioning and inference contracts. This complements RynnWorld-4D: 4D Embodied World Models for Robotic Manipulation‘s single-backbone tri-branch design and Advancing Open-source World Models (LingBot-World) LingBot-World’s single-recipe base by explicitly decoupling the recipe from the backbone, and answers one axis of Data Pyramid for Embodied Manipulation‘s data-recipe framing by shipping a reconfigurable multi-source data engine as a reusable artifact. The 5-second-training → minute-to-hour-rollout claim puts SolarWM in the same forgetting-vs-drifting design space as Context Forcing: Consistent Autoregressive Video Generation with Long Context, FlowAct-R1: Towards Interactive Humanoid Video Generation, and Agentic Context Management: Solving Agent Memory and Cost by Treating Them as Lifecycle and Architecture Problems — with the specific bet that the bidirectional-then-causal-then-DMD stack (Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation Causal Forcing) transfers across backbones. It also lands as a direct open-recipe counterpart to closed real-time interactive WFMs (Genie 3, Runway GWM-1) with concrete downstream instantiations on the MiniMax H3 — Open-Weights Omni-Reference Video-Audio Generation Model Card and LTX-2.5 open bases — sibling to H3-World: Turning Language Understanding into World Control which turned MiniMax-H3 into an interactive WM via LoRA + temporal attention routing rather than by rebuilding the training recipe.

See also

Section titled “See also”