Skip to content
Bud's Wiki

Atlas: A World Model for Spatial Intelligence

TL;DR

Section titled “TL;DR”

Atlas is World Labs’ next-generation omni world model, pretrained from scratch as a multimodal autoregressive diffusion transformer operating on text, images, camera poses, and 3D depth maps in a shared spatial context where each visual token is grounded at an explicit 3D camera pose. One model spans camera-controlled generation (up to 1 minute at 1440p from a handful of reference images), sparse-view 3D reconstruction (point clouds and 3D Gaussian splats that outperform specialist reconstruction baselines), space-time simulation (bullet-time reframing from 3–5 phone cameras), robotics real-to-sim (photorealistic RGB + depth for any robot trajectory from a 24-frame phone scan), and text-to-image / 360-panorama generation. It powers future versions of Marble and the R2S2R stack, and enters early access with select partners.

Key claims

Section titled “Key claims”
  • Atlas is a single omni model that natively operates on text, images, camera poses, and 3D depth maps in one unified autoregressive-diffusion-transformer architecture, treating videos as sequences of images and each image as grounded at an explicit camera pose [§Model Architecture].
  • The core design primitive is a spatial context: reference images are placed at 3D positions inside the input sequence, and generation proceeds conditioned on that context — enabling both “smooth interpolation between two unrelated reference images” and hand-designed camera paths through a reconstructed world [§Generating with Spatial Context].
  • Atlas supports camera-controlled generation up to 1 minute of video at 1440p from a small number of reference images, using precise camera geometry as a native input format rather than text-based camera descriptions [§Controllable Long Videos].
  • A team of third-party human raters judged Atlas to follow the intended camera path better than a selection of top recent video models, with the gap widening as camera trajectories become more complex; competing models were fed cinematic camera terms via their text-prompt input since they lack native camera conditioning [§Benchmarks].
  • Atlas outperforms the best specialized open-source 3D reconstruction models on multiple state-of-the-art sparse-view 3D reconstruction benchmarks (all baselines reproduced under a common protocol), despite being a generalist omni model that also handles generation [§Benchmarks].
  • More input images monotonically shift Atlas’s behavior from imagination toward faithful reconstruction — the paper’s slogan is “the more it sees, the less it imagines” — and Atlas can absorb over one hundred images in its spatial context [§Reconstructing from Multiple Images].
  • Atlas natively emits explicit 3D — 2D image frames, 3D depth maps, point clouds, and Gaussian splats — and its splats use the same representation as Marble, enabling on-device high-resolution/high-framerate rendering [§Explicit 3D Outputs].
  • For robotics, Atlas jointly serves as reconstructor and sensor simulator: from a 24-frame phone video of a large environment it reconstructs the space and then generates photorealistic RGB + depth for any body-mounted camera along any simulated robot trajectory — “the world and the robot’s view of it come from the same model” [§Robotics Simulation].
  • “Bullet-time” multi-view reframing works from as few as 3–5 ordinary phone cameras on tripods and clamps, with no professional capture equipment [§Reframing Video].
  • Atlas exhibits compute scaling: over the course of development a series of models of increasing size and training compute was trained, and each new compute level unlocked new capabilities; the team expects this trend to hold with continued scaling [§Model Scaling].
  • Atlas is a rectified-flow (latent) diffusion model and blends LLM-side infrastructure (KV-caching, cache-aware routing, disaggregated serving) with modern video-model techniques (diffusion distillation, classifier-free guidance, shifted noise schedules, advances in VAE design) [§Model Architecture].

Method

Section titled “Method”

Atlas is a multimodal autoregressive diffusion transformer. Inputs and outputs are heterogeneous sequences whose elements are one of: text tokens, images, camera poses, or 3D depth maps; videos are represented as image sequences. Each image and depth-map element is conditioned on an explicit camera pose, making 3D spatial position a first-class component of the sequence rather than an emergent property. Generation is autoregressive at the element level (one image / depth map / text chunk at a time, conditioned on the prefix) and diffusion at the within-element level (each element is produced by gradually denoising a latent under rectified-flow training). This factorization is what the post calls “spatial context”: earlier elements in the sequence supply both content and 3D anchoring for what comes next, so “task” reduces to sequence layout — camera-controlled generation, sparse-view reconstruction, video reframing, robotics sensor simulation, and text-to-image all become different input–output sequence templates against the same weights.

Practical mechanics: (i) camera poses enter as a native input modality rather than as text descriptions of cinematic moves; (ii) reference images can be placed at arbitrary 3D positions in the context, and Atlas fills in transitions between them from its world knowledge; (iii) more input images monotonically bias the output from imagination toward faithful reconstruction; (iv) explicit 3D outputs come from jointly generating new views and their depth maps and combining them into point clouds or 3D Gaussian splats — the same splat representation Marble uses, so the outputs render on-device at high resolution and framerate. The architecture blends LLM-side infrastructure (KV-caching, cache-aware routing, disaggregated serving) with modern latent-diffusion techniques (rectified flow, diffusion distillation, CFG, shifted noise schedules, VAE design). No parameter counts, training-data description, or training-compute numbers are disclosed in the post.

Results

Section titled “Results”

Two quantitative evaluations are reported, both against strong specialist baselines. On camera-controlled generation, third-party human raters judged which model better followed an intended camera path across trials pairing a single input image with sequences of one to three cinematic camera motions (pan, truck, crane, etc.). Atlas is reported to beat “recent video models” with a lead that widens as trajectories become more complex; baselines received the camera path via their text-prompt interface using standard cinematic terminology since none of them accept cameras as native inputs. On sparse-view 3D reconstruction, the model receives a set of images with camera poses and predicts a 3D point for each input pixel; Atlas is reported to beat the best specialized open-source reconstruction models across “several state-of-the-art benchmarks” under a reproduced, common evaluation protocol. No specific model names, numeric scores, benchmark identifiers, or ablations are disclosed in the post. Qualitative capability demonstrations include: 1-minute 1440p video from a small set of reference images along a hand-designed camera path; sparse-view scene reconstruction that improves monotonically as inputs increase from 1 → 2 → 3 images (illustrated by piecewise assembly of a garden + cottage + main house scene, and of Stanford’s Main Quad from 2–25 ground-level views generating fly-through aerials); bullet-time reframing from 3–5 phone cameras; and robotics sensor simulation from a 24-frame phone scan of large environments (RGB + depth for arbitrary robot trajectories).

Why it’s interesting

Section titled “Why it’s interesting”

Atlas is the direct upstream model behind the R2S2R engine described in Building Worlds That Train Robots — Real-to-Sim-to-Real (R2S2R) as a Scalable Engine for Training and Evaluating Robot Policies and the concrete instantiation of the “simulator is the linchpin” thesis in A Functional Taxonomy of World Models — the same weights render pixels, output explicit 3D, and provide the sensor-simulation layer for policy training and evaluation, so all three of Fei-Fei Li’s renderer / simulator / planner output channels are supposed to come from one model. On the wiki’s central WFM debate, Atlas plants a flag on a third pole between latent-predictive JEPA-style models (LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels, V-JEPA 2) and monolithic pixel-rollout video generators (Project Genie: Experimenting with infinite, interactive worlds, Cosmos 3: Omnimodal World Models for Physical AI): explicit camera-pose grounding and native 3D depth-map outputs at the interface, but end-to-end learned generation rather than a bolted-on classical 3D pipeline. This is closest in spirit to HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds (HY-World 2.0 emits splats/meshes as first-class outputs) and to the compact-3D-interface μ₀: A Scalable 3D Interaction-Trace World Model (µ₀ uses 3D interaction traces as a shared substrate for policy), but pushes further by making spatial context (multiple reference images placed at 3D positions) the input interface — the “how do I steer this?” answer that Genie 3 / GWM-1 leave to text or coarse controls. Two concrete implications worth watching: (i) if the human-rated camera-following lead over text-prompted baselines holds under standardized eval (existing camera-controlled DiT baselines like CamCloneMaster: Enabling Reference-based Camera Control for Video Generation or ChronoEdit already accept camera inputs — the post’s baselines don’t), it argues that camera-as-native-modality is worth the architectural cost even outside long-horizon generation; (ii) the “up to 100+ images in context, more images → less imagination” curve is the sharpest filed operating point yet on the Context Length / Quality Trade-off in Video Generation axis for the reconstruction-vs-generation trade-off, and turns the number of input views into a runtime dial rather than a training-time choice. No parameter count, training data, or numeric benchmarks are disclosed, so this remains a product-and-position announcement rather than a technical paper — file as a marker for the R2S2R backbone and revisit when a tech report lands.

See also

Section titled “See also”