MiniWorld: Democratizing the Training of Video World Models from Scratch

TL;DR

MiniWorld is a fully open-source recipe for training streaming video world models from scratch on a single 8-GPU server in a few days, rather than post-training/distilling a bidirectional pretrained T2V. It uses a block-causal Video DiT trained with Flow Matching in a pretrained Video VAE latent space, extends Diffusion Forcing with a chunk-wise non-decreasing noise schedule and two-stage continued training, and pairs it with a rolling KV cache + pipelined asynchronous denoising for bounded-cost streaming inference. The pitch is a lightweight, transparent, reproducible baseline for autoregressive video world modeling.

Key claims

Training autoregressive video world models from scratch (rather than post-training a bidirectional pretrained T2V) is feasible and scalable, and avoids the mismatch between bidirectional pretraining and causal streaming inference [Abstract].
A block-causal Video DiT + Flow Matching in a pretrained Video VAE latent space is a viable minimal architecture for a from-scratch streaming video world model [Abstract].
Extending Diffusion Forcing with a chunk-wise non-decreasing noise schedule plus two-stage continued training improves temporal modeling and stability over vanilla Diffusion Forcing [Abstract].
A rolling KV cache combined with pipelined asynchronous denoising enables efficient streaming generation under bounded per-step compute [Abstract].
The full pipeline is trainable end-to-end within several days on a single 8-GPU server, with training/inference code and pretrained checkpoints released [Abstract].

Method

MiniWorld is a block-causal Video DiT operating in the latent space of a pretrained Video VAE, trained with Flow Matching. The generative recipe extends Diffusion Forcing: instead of a single noise level per sample, MiniWorld applies a chunk-wise non-decreasing noise schedule across temporal chunks and trains in two continued-training stages to stabilize temporal modeling. At inference the model runs autoregressively over chunks: a rolling KV cache retains a bounded window of past-chunk activations and denoising steps are pipelined asynchronously across chunks so wall-clock cost per new chunk is bounded. The core design goal is to get a functioning streaming video world model within a few days on 8 GPUs while releasing training and inference code plus checkpoints.

Results

The paper’s headline result is qualitative — a fully reproducible from-scratch training recipe with released code and checkpoints, trained within several days on a single 8-GPU server [Abstract]. Specific FVD/quality numbers vs. adapted-from-T2V baselines are not in the abstract; readers should consult the released report and repo for quantitative comparisons.

Why it’s interesting

MiniWorld is the reproducibility counterpart to the growing pile of from-scratch AR video world models filed here: it takes the same block-causal DiT + Diffusion Forcing + rolling-KV recipe used by heavier stacks like PhiZero: A World Model Built Around Physical Language and AlayaWorld: Interactive Long-Horizon World Modeling and shows it works at 8-GPU scale, which is the missing baseline the concept page for World Foundation Models has flagged. It also complements Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation‘s argument that bidirectional-teacher → AR-student distillation has a structural injectivity gap — MiniWorld sidesteps that gap entirely by never having a bidirectional teacher.