360Anything: Geometry-Free Lifting of Images and Videos to 360°

TL;DR

360Anything is a geometry-free DiT that lifts a perspective image or video to a gravity-aligned 360° panorama (or panorama video) without any camera or 3D supervision at inference time. The recipe concatenates conditioning tokens from the input perspective view with noisy panorama latents along the sequence dimension and lets the DiT infer intrinsics/extrinsics implicitly; a “Circular Latent Encoding” trick removes seam artifacts at the panorama wraparound. The panorama outputs can then be used as a free working-memory substrate — naive 3DGS on the original input video produces broken geometry, but training 3DGS on the synthesized panorama video gives clean fly-through reconstructions. The motivating thesis (highlighted by Jon Barron) is that panoramic representations sidestep an entire class of object-permanence failures in image/video generation, because there is no frame for objects to leave and re-enter.

Key claims

A monocular perspective image or video can be lifted to a gravity-aligned 360° panorama with no explicit camera or 3D conditioning — the DiT learns to infer the camera intrinsics and extrinsics directly from the data [project page §Method].
Conditioning is implemented by concatenating perspective-view tokens (orange) with the noisy panorama latent tokens (green) along the sequence dimension of a DiT; no cross-attention adapter or pose embedder is required [project page §Method].
Training panoramas must be canonicalized to gravity-aligned upright orientation in a two-step pipeline (raw → stabilized → canonicalized) so the model learns a single canvas convention regardless of input camera pose [project page §Method, Raw/Stabilized/Canonicalized comparison].
“Circular Latent Encoding” eliminates the seam artifact at the panorama left/right wraparound that would otherwise appear in latent-grid DiTs [project page §Method].
Training data is synthesized from canonicalized 360° video by projecting to perspective views with randomly sampled camera trajectories — the panorama itself supplies the supervision; the perspective conditioning is generated procedurally [project page §Method].
3DGS trained on a single input perspective video produces broken geometry due to low scene coverage; 3DGS trained on the 360Anything-outpainted panorama video gives clean fly-through reconstructions on the same inputs [project page §3D scene reconstruction].
The method compares against Imagine360 (NeurIPS 2025), Argus (ICCV 2025), and ViewPoint (NeurIPS 2025) on perspective-video → panorama-video lifting [project page §Comparisons, refs 1–3].
The image variant fine-tunes FLUX on synthetic 360° renderings of 3D scenes and generalizes to in-the-wild and OOD images [project page §Image generation].

Method

The training pipeline starts from a corpus of raw 360° videos and runs a two-step canonicalization (stabilization + gravity-alignment) to produce upright panorama videos. To get supervision pairs, each canonicalized panorama is projected to perspective views along randomly sampled camera trajectories — this becomes the model conditioning, with the original panorama as the target. The architecture is a single DiT that takes the perspective conditioning tokens and the noisy panorama latent tokens, concatenated along the sequence axis, and denoises the panorama in one stream. No separate camera embedder, no pose adapter, no explicit 3D representation — the model infers all camera parameters implicitly. The image variant uses the same recipe but fine-tunes FLUX on synthetic 360° renderings of 3D scenes; the video variant uses Wan as the backbone (citation [5] on the project page). “Circular Latent Encoding” handles the wraparound seam that would otherwise show at the panorama’s east/west boundary. At inference time, the model also runs as a feeder for 3DGS: outpaint the input perspective video to a panorama video, then fit a 3DGS on the panorama for downstream fly-through.

Results

The project page shows qualitative head-to-heads on two captions (“person walking down a busy city street,” “man in dark grey t-shirt gestures towards a large green tank”) against Imagine360, Argus, and ViewPoint, with red boxes highlighting the input-video region inside the output panorama [project page §Comparisons]. No quantitative table is included on the project page.
3D scene reconstruction qualitative results show that input-video-only 3DGS breaks at fly-through camera moves while panorama-trained 3DGS holds up [project page §3D scene reconstruction].
The model is reported as robust to “challenging conditioning videos with large object or camera motion” — it still infers the camera trajectory and produces stably canonicalized output panoramas [project page §Results, video gallery].
Image-side perspective→panorama generation generalizes from synthetic 3D scene renderings to in-the-wild and OOD perspective images after fine-tuning FLUX [project page §Image generation].

Why it’s interesting

This is a different bet from the rest of the Wan-2.1-adapter camera-control cluster the wiki is tracking. Instead of conditioning a video diffusion model on an explicit camera trajectory (ReCamMaster, SpaceTimePilot, VerseCrafter, NeoVerse), 360Anything changes the output canvas: by generating the full 360° sphere, it sidesteps the trajectory-control problem entirely — once you have a panorama video, any downstream camera path is a free crop. Jon Barron’s framing of this as an “under-appreciated observation” about object permanence is the load-bearing pitch: the panorama is a “working memory” for consistent world generation because nothing can leave the frame. For Luma’s environment-consistency problems specifically (gray’s framing in the post), this offers a different decomposition than the trajectory-conditioned recipes — generate the world once as a panorama, then render any view. The 3DGS-on-panorama-video result is the practical demonstration: an input video that breaks naive 3DGS becomes tractable once it’s been “completed” to 360°. Related cluster siblings on the wiki: Camera-Controlled Video Diffusion (the trajectory-conditioned alternative paradigm) and 4D Scene Generation (the downstream 3DGS application).