Depth Any Panoramas (DAP): A Foundation Model for Panoramic Depth Estimation

TL;DR

Depth Any Panoramas (DAP) is Insta360’s panoramic metric depth foundation model, built on a DINOv3-Large backbone with a distortion-aware decoder and a plug-and-play range-mask head that adapts metric output across indoor/outdoor scale variation. The data side is the bigger contribution: a three-stage progressive pipeline (Scene-Invariant Labeler → Realism-Invariant Labeler with PatchGAN discriminator selecting 300K indoor + 300K outdoor high-confidence pseudo-labels → final DAP training on labeled+pseudo-labeled data) combines public datasets, UE5-simulated synthetic data, text-to-image generated panoramas, and web-scraped real panoramas to bridge synthetic↔real and indoor↔outdoor domain gaps. Training uses multi-level geometric and sharpness-aware losses (SILog + depth-fidelity + gradient + normal + point-cloud). Reported zero-shot on Stanford2D3D, Matterport3D, and Deep360 shows robust metric predictions across diverse real-world panoramic scenes.

Key claims

A single panoramic depth model can produce robust metric depth across both indoor and outdoor distance regimes by combining a plug-and-play range mask head with sharpness- and geometry-centric losses [Methodology, Architecture figure].
Synthetic-only training does not transfer; bridging the synthetic↔real gap requires a PatchGAN-based realism discriminator that selects 300K indoor + 300K outdoor high-confidence pseudo-labeled samples from web-scraped real panoramas before the final supervised stage [Methodology, Stage 2].
A three-stage progressive pipeline — Scene-Invariant Labeler (synthetic indoor+outdoor pretraining) → Realism-Invariant Labeler (discriminator-gated pseudo-label curation) → DAP (semi-supervised training on the union) — is presented as the load-bearing recipe; each stage targets a different domain gap [Methodology, three-stage diagram].
DINOv3-Large is chosen as the visual backbone for its pretrained generalization; the depth decoder is “distortion-aware” to handle equirectangular distortion in panoramas [Architecture figure].
Training is supervised by an additive loss combining SILog (L_SILog), depth fidelity (L_DF), gradient (L_grad), surface-normal (L_normal), and per-point (L_pts) terms — i.e. multi-level geometric + sharpness supervision in one objective [Architecture figure].
A “range mask head” is added as plug-and-play to enable adaptive distance control across diverse scenes, addressing the indoor-meters-vs-outdoor-tens-of-meters scale mismatch [Architecture figure].
Data construction mixes four sources — public datasets, high-quality UE5 synthetic, text-to-image generated panoramas, and web-scraped real panoramic images — for both labeled and pseudo-labeled supervision [Abstract, Methodology].
Zero-shot evaluation is reported on Stanford2D3D, Matterport3D, and Deep360 with “strong performance and zero-shot generalization,” and “particularly robust and stable metric predictions in diverse real-world scenes” [Abstract].

Method

The pipeline progresses through three labelers. Stage 1 — Scene-Invariant Labeler: train on high-quality synthetic indoor+outdoor data (UE5 simulator + text-to-image-generated panoramas) to give a strong initial depth predictor that doesn’t overfit to one scale regime. Stage 2 — Realism-Invariant Labeler: the synthetic-trained predictor is run on web-scraped real panoramas; a PatchGAN discriminator then selects the 300K indoor + 300K outdoor highest-confidence predictions as pseudo-labels, throwing out cases where the synthetic→real domain gap is too large. Stage 3 — DAP: the final model is trained semi-supervised on the union of labeled (public + synthetic) and pseudo-labeled (curated real) data.

The DAP network itself is a DINOv3-Large visual backbone feeding a distortion-aware depth decoder. A plug-and-play range mask head sits in parallel, providing adaptive distance control so the same model can output metric depth at room scale (Stanford2D3D, Matterport3D) and at scene/outdoor scale (Deep360). Supervision is a sum of five losses: SILog (scale-invariant log), depth fidelity (L_DF), gradient (L_grad) for edge sharpness, surface normal (L_normal) for geometric consistency, and a per-point loss (L_pts). The project page does not disclose model size beyond “DINOv3-Large” or training compute.

Results

Qualitative comparisons on outdoor scenes (image + video), indoor scenes, fine-grained details, robustness, scenes-with-humans, and scale ability are shown on the project page; no quantitative table is provided in the page text [Qualitative Evaluation section].
Zero-shot performance on Stanford2D3D, Matterport3D, and Deep360 is reported as “strong” with “particularly robust and stable metric predictions in diverse real-world scenes” — no numeric metrics are exposed on the project page [Abstract].
The 300K + 300K curated-pseudo-label budget is the only specific data-volume claim disclosed [Methodology, Stage 2].

Why it’s interesting

This is the wiki’s first dedicated panorama depth model and a natural sibling to Depth Anything 3: Recovering the Visual Space from Any Views (DA3): both are DINO-backboned dense-depth foundation models that route the synthetic-vs-real domain gap through a curated pseudo-label stage. DA3 uses a DA2-style monocular teacher to align real ground truth with synthetic-trained predictions via RANSAC scale-shift; DAP uses a PatchGAN discriminator to gate which real-panorama predictions survive into the pseudo-label pool. The two papers converge on the same recipe — “synthetic-trained teacher → realism-curation gate → final mixed-data student” — applied to perspective-view and 360° depth respectively, which is a useful design pattern to track. On the panorama side, this is the depth-prediction counterpart to 360Anything: Geometry-Free Lifting of Images and Videos to 360°‘s panorama-generation pitch: 360Anything argues panoramas sidestep object-permanence failures because nothing leaves the frame; DAP supplies the metric-depth half of that pipeline, so the two together could plug into a panorama-native 3DGS or world-model stack. It also continues the Synthetic Training Data thread: like NeoVerse (NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videos) and DA3, DAP cannot train on synthetic alone — the realism-curation gate (PatchGAN discriminator) is the load-bearing piece, matching the page’s broader “verification gate dominates returns” claim.