InfiniteDiffusion: Bridging Learned Fidelity and Procedural Utility for Open-World Terrain Generation

TL;DR

InfiniteDiffusion is a training-free reformulation of diffusion sampling that converts any bounded diffusion model into a lazy, functionally stateless, infinite generator with O(1) random access by seed + coordinate. It generalizes MultiDiffusion to unbounded domains via a bounded LRU cache and order-invariant blending, so output depends only on (seed, requested region), not on traversal history. The companion system, Terrain Diffusion, is a SIGGRAPH 2026 paper applying this to cascaded planetary terrain — feature scales spanning hundreds of km, vertical range from −10000 m to ~9000 m via Laplacian encoding, shipped as an open-source Minecraft mod that runs in multiplayer on consumer GPUs at ~9× orbital velocity.

Key claims

InfiniteDiffusion frames the “infinite / stateless / learned-realism trilemma” as the design constraint: diffusion gives realism but is bounded, classical procedural noise is infinite+stateless but cannot learn, and autoregressive outpainting is learned+unbounded but requires global state that precludes determinism and random access [Body §1].
It generalizes MultiDiffusion as a drop-in replacement that makes the diffusion process a lazy computation generating only the requested region, with a bounded LRU cache as a performance optimization rather than persistent state [Body §1, comparison figure].
Quality is reported as “little to no degradation” versus MultiDiffusion in side-by-side comparisons, while gaining infinite extent, lazy evaluation, and order-invariance [Body §1].
Versus autoregression, InfiniteDiffusion claims O(1) random access (vs O(n)), order-invariant determinism (vs order-dependent), no error compounding, embarrassingly parallel sampling (vs sequential), and training-free deployment [Comparison table].
Terrain Diffusion uses a cascade of diffusion models to couple planetary context with local detail, generating features spanning hundreds of real-world km with continents on the order of millions of km² [Body §2].
A compact Laplacian encoding of heightmaps stabilizes outputs across the Earth-scale dynamic range (−10000 m to ~9000 m) inside a single world [Body §2].
The system runs locally on consumer hardware at “3× orbital velocity” in a Unity flying demo, and ships as a Minecraft mod with multiplayer support, seed-sharing, and instant teleport across millions of miles — concrete evidence that O(1) random access is real, not just asymptotic [Body §2, demos].

Method

InfiniteDiffusion treats the denoised image as a lazy, indexable function (seed, x, y) → pixel rather than a finite tensor. Concretely, it inherits MultiDiffusion’s per-step strategy of running the diffusion model on overlapping tiles and averaging their predicted velocity/noise in overlap regions, but drops the requirement that the canvas be pre-allocated. Tiles are materialized on demand when a region is queried; intermediate denoising states for nearby tiles are kept in a bounded LRU cache so that overlapping neighbors can be consistently blended without redoing earlier diffusion steps. Because the blending is order-invariant (averaging in overlaps) and the per-tile RNG is seeded by (global-seed, tile-coords), output for a fixed (seed, region) is bit-deterministic regardless of access order. An open-source “infinite tensor” framework handles constant-memory manipulation of these unbounded grids.

Terrain Diffusion stacks two ingredients on top: (1) a hierarchical cascade of diffusion models, where a coarse model produces planetary-scale context (continents, climate-band-like features) and finer models conditionally denoise local 1024×1024 patches at 100 km width; (2) a Laplacian (multi-resolution residual) encoding of heightmaps so that the network does not have to regress a single absolute height field whose dynamic range spans five orders of magnitude. The Minecraft and Unity demos consume the resulting infinite heightmap by direct coordinate queries.

Results

9× orbital velocity sustained on a consumer GPU during interactive flight in the Unity demo (cited as ~3× orbital velocity in the “comfortable flying” framing of the body text — the larger figure is the upper bound) [Body §2].
Single 1024×1024 patches span 100 km × 100 km of terrain; continents span millions of km² inside one stateless world [Body §2].
Vertical range −10000 m to +8848 m handled within a single model via Laplacian encoding [Body §2].
Ships as a runnable Minecraft mod (github.com/xandergos/terrain-diffusion-mc) with multiplayer, seed-sharing, and instant teleport — a strong existence proof for the O(1) random access claim [Body §2].
No quantitative quality metric (FID/PSNR) reported in the blog; comparison to MultiDiffusion is qualitative (“little to no degradation”) [Body §1].
Accepted to SIGGRAPH 2026 Conference Papers (DOI 10.1145/3799902.3811080) [BibTeX].

Why it’s interesting

This is a different shape of contribution from most of the wiki: a sampler-level reformulation that turns a bounded diffusion model into a procedural-noise replacement, without retraining, and an honest game-engine integration as evidence. Two threads on the wiki are relevant. First, the scene/world generation cluster (4D Scene Generation, WorldGen: From Text to Traversable and Interactive 3D Worlds, FlashWorld: High-quality 3D Scene Generation within Seconds) has been about getting one high-quality scene; InfiniteDiffusion attacks a different axis — making the scene unbounded and randomly addressable, at the cost of being 2D heightmap-only for now. Second, the diffusion-LM and autoregressive-video literature (Autoregressive Video Generation, TiDAR: Think in Diffusion, Talk in Autoregression, parallel-decoding) keeps hitting the same wall this paper names explicitly: autoregression compounds error and serializes generation, while a parallel-blended diffusion sampler avoids both. The O(1) random-access / order-invariance properties may be more broadly useful than terrain — e.g. for video world models that need consistent revisitation of arbitrary spacetime coordinates, where the equivalent of “shared by seed, teleport instantly” is currently expensive (Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval).