MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Engines

TL;DR

MultiGen is a diffusion game engine that decomposes video world-model generation into three explicit modules — Memory (a persistent, editable external state holding map geometry and per-player poses), Observation (next-frame diffusion conditioned on memory readouts + recent history), and Dynamics (action-conditioned pose updates) — instead of operating as a monolithic next-frame predictor. The external memory gives users a direct handle for level design (edit a 2D top-down minimap before inference) and, because it is a shared state independent of any single context window, naturally supports an arbitrary number of players: each client runs its own Observation + Dynamics modules while reading from and writing to the same memory. The system runs ~20 FPS for real-time interactive multiplayer in a Doom testbed, and the authors release a 30-minute 4-player timelapse demonstrating multi-minute stability.

Key claims

Two interactivity properties existing diffusion world models lack — editable, reproducible environment control and shared inference across multiple players — share a common missing primitive: an explicit external memory that persists beyond the model’s context window and is updated by user actions [§1].
The generation process is factored into Memory / Observation / Dynamics modules rather than a single next-frame predictor, with the memory module maintaining minimap geometry and active agent states as a persistent state read at every rollout step [project page §Approach].
Level design is exposed via an editable 2D minimap (coarse top-down geometry) that anchors generation; the model synthesizes detailed first-person observations consistent with the user-authored layout [§1, Fig. 1].
Multiplayer is architecturally free: each player runs their own Observation and Dynamics instances, sharing only the external memory, so the number of players can change without changing the model interface [project page §Multiplayer].
The system runs at ~20 FPS for real-time interactive multiplayer rollouts [project page §Multiplayer].
Doom is used as the testbed because it offers rich first-person dynamics while retaining a clear notion of level layout, making it well-suited for studying shared, long-horizon rollouts [§1].
The authors demonstrate stability over a 30-minute, 4-player gameplay timelapse [project page §Timelapse].

Method

The diffusion game engine is split into three modules with disjoint responsibilities. The Memory module is an explicit state object — the static 2D map (authored by the user as a coarse top-down minimap) plus the live set of active player poses — that persists across the entire rollout, independent of the diffusion model’s context window. The Observation module is the diffusion model proper: it takes recent observation history plus a readout from memory (which queries the minimap and agent set as a function of the current pose) and denoises the next first-person frame for one player. The Dynamics module consumes the latest action and observation and emits a pose update, which is written back into memory.

This factorization changes what a “next-frame predictor” has to remember. Long-horizon scene structure does not need to live in the diffusion model’s KV cache, because it lives in memory and is fetched on demand — so the diffusion model can keep a short observation history. It also changes how multiplayer works: with memory hoisted out, multiple agents can run their own observation and dynamics modules in parallel against the same shared state, with reads producing coherent cross-viewpoint observations (e.g. one player sees the structure another player just built) and writes resolving against the same authoritative pose set. The model is trained in Doom; level layout is communicated to the diffusion module exclusively through memory readouts derived from the editable minimap, which the project page positions as the “external memory blueprint.”

Results

The released materials are demonstration-level rather than benchmark-driven: a project page with comparison videos (level-design conditioning, multiplayer rendering, ablations), and a 30-minute 4-player timelapse standing in for a long-horizon stability evaluation. Headline operating point: ~20 FPS, arbitrary player count, multi-minute coherent rollouts in Doom. No quantitative tables or shared-benchmark numbers are present on the project page; the arxiv PDF and an independent web demo (play-multigen.com) are linked but not analyzed here.

Why it’s interesting

MultiGen sits at the same problem as Solaris: Building a Multiplayer Video World Model in Minecraft — joint, real-time, multi-player video world-modeling — but takes the opposite architectural side. Solaris keeps a single joint DiT that denoises both players’ observation streams together (multiplayer is baked into the model via per-player 3D RoPE + player-ID embeddings, with cross-attention/FFN/action-conditioning applied per player), and frames the contribution as fitting more agents into the context window. MultiGen instead hoists shared state out of the model entirely into an external memory, so each player is rendered by their own Observation module instance and adding players is a deployment-side decision rather than a model-side one. The two papers are the cleanest “in-context vs. out-of-context multi-agent state” pair on the wiki and should be read together.

The Memory/Observation/Dynamics factorization is also a direct counter-instance to The flavor of the bitter lesson for computer vision‘s Bitter-Lesson argument that generative-rollout WFMs should replace explicit intermediate representations — MultiGen reintroduces an explicit, hand-designed state representation (the minimap + pose set) precisely because it is what gives users editability and multiplayer coherence. Whether that is a transient scaffold or a durable design pattern is one of the open questions on World Foundation Models. As a closed-source flagship analogue, Project Genie: Experimenting with infinite, interactive worlds‘s Project Genie ships World Sketching (image-grounded pre-commit) and World Exploration primitives that look superficially like MultiGen’s editable-minimap idea, but without the disclosed multiplayer or shared-memory factorization; MultiGen is the open architectural recipe behind that surface.