Yume-1.5: A Text-Controlled Interactive World Generation Model

TL;DR

Yume-1.5 is a text-controlled interactive world generation model that produces long, explorable videos from a single image or text prompt with WASD keyboard control over the camera. It targets three weaknesses of prior interactive world models — large parameter counts, expensive multi-step inference, and exploding history context — via a long-video framework with unified context compression + linear attention, a streaming acceleration recipe using bidirectional-attention distillation and Self-Forcing, and an event/action-decomposed text-control scheme. It is the follow-up to the original Yume (arXiv:2507.17744) released by the Shanghai AI Lab / Fudan-affiliated team.

Key claims

Yume-1.5 generates interactive, autoregressive long videos from a single image or text prompt with WASD camera control [project page §Overview].
A Joint Temporal-Spatial-Channel Modeling (TSCM) module enables long-video generation by combining unified context compression with linear attention to keep memory cost bounded as the rollout grows [project page §TSCM].
A streaming acceleration strategy built on bidirectional-attention distillation + Self-Forcing plus an enhanced text-embedding scheme reduces inference cost and error accumulation in long sequences [project page §Streaming Acceleration].
Text control is achieved by decomposing each caption into separate Event and Action descriptions, enabling precise control over in-world dynamics rather than coarse scene re-prompts [project page §Text-Controlled Events].

Method

Yume-1.5 follows the now-standard interactive-world-model template — autoregressive video diffusion conditioned on (i) an initial image or text, (ii) per-step keyboard actions, (iii) running history — but replaces three components of that template. (1) The long-video backbone uses Temporal-Spatial-Channel Modeling: a unified context compression block paired with linear-attention sequence modeling, so the per-step compute and memory grow sub-quadratically in horizon length. (2) The few-step inference path is obtained by distilling the bidirectional-attention teacher into a causal streaming student in the Self-Forcing tradition (closing the train-inference gap by rolling out the student during training), augmented with a stronger text-embedding scheme. (3) For controllability, captions are factorized into an Event channel (what changes in the world) and an Action channel (what the camera/agent does), and both are injected as conditioning so the user can issue language-level world edits independently of the WASD camera stream.

The project page does not disclose parameter count, training data, full evaluation numbers, or whether the backbone is Wan-family / Seedance / proprietary; the arXiv preprint (2512.22096) is the source of record but was rate-limited at filing time.

Results

The project page is a release announcement and shows qualitative video demos of (a) long, smoothly continuing AR rollouts from a single image, (b) WASD-driven camera exploration, and (c) text-issued world events (e.g. weather, scene state changes) propagating through subsequent frames. No quantitative numbers (VBench, PhysicsIQ, user-study GSB, FPS, latency) are surfaced on the project page; these are in the arXiv paper which could not be fetched at filing time. The original Yume (2507.17744) is cited as the upstream system.

Why it’s interesting

This is the second filed datapoint of the same lineage as FlowAct-R1: Towards Interactive Humanoid Video Generation — a fully product-engineered streaming AR interactive world model with structured memory, distilled few-step inference, and explicit user controls — but for general-world rollout rather than humanoid-only. Architecturally, it stacks several axes the wiki’s Autoregressive Video Generation cluster has been tracking separately: linear attention + unified context compression (a sibling of Pretraining Frame Preservation in Autoregressive Video Memory Compression‘s memory-compression encoder), Self-Forcing-style bidirectional-attention distillation (Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion), and explicit text-control of events on top of action control (a controllability axis not previously combined with WASD-style action control in the filed papers). It is also one of the few interactive WFMs released by an academic-affiliated group rather than a closed lab — useful as an open counterpoint to Project Genie and the closed Genie-3-derived The Waymo World Model: A New Frontier For Autonomous Driving Simulation vertical specialization.