Skip to content
Bud's Wiki

Depth Anything V4: Dynamic 4D Scene Reconstruction via Riemannian Flow Matching on 4D Gaussian Splatting

TL;DR

Section titled “TL;DR”

Depth Anything V4 (DAV4) reconstructs dynamic 4D scenes from monocular video by applying Riemannian Flow Matching (RFM) to 4D Gaussian Splatting parameters — defining probability paths directly on the non-Euclidean manifolds of scale, rotation, and opacity so every intermediate sampling state is a valid 4DGS. The paper’s methodological contribution is an ablation that isolates RFM’s contribution from test-time optimization (TTO) and pre-training: a deterministic MLP baseline with the same data, architecture, and TTO reaches F-score 0.762, while RFM reaches 0.806 — the +0.044 gap is claimed as RFM’s isolated contribution, a rare clean measurement in the 4D-generation cluster where “flow matching helps” is usually asserted rather than isolated. DAV4 uses no human-annotated depth labels as training losses and reports uncertainty via NLL and ECE.

Key claims

Section titled “Key claims”
  • Applying Riemannian Flow Matching to 4DGS parameters (scale, rotation, opacity) — with probability paths defined on the parameters’ native manifolds rather than in Euclidean coordinates — ensures every intermediate sampling state is a valid 4DGS [Abstract, §3].
  • RFM’s isolated contribution over a same-data, same-architecture, same-TTO deterministic MLP baseline is +0.044 F-score (0.762 → 0.806), the paper’s headline controlled-ablation number [Abstract].
  • DAV4 outperforms prior Depth Anything models and per-scene 4D-GS on dynamic reconstruction and novel-view synthesis while using no human-annotated depth labels as training losses [Abstract].
  • Pre-training cost is 360 GPU-hours, framed as amortizing across ≥10,000-scene deployments — the paper explicitly corrects prior cost-analysis framing [Abstract].
  • Uncertainty is reported via Negative Gaussian Log-Likelihood and Expected Calibration Error rather than qualitative confidence maps [Abstract].

Method

Section titled “Method”

DAV4 targets the dynamic 4DGS-from-monocular-video setting shared by NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videos and Shape-for-Motion: Precise and Consistent Video Editing with 3D Proxy but replaces the deterministic feed-forward Gaussianization head with a Riemannian Flow Matching sampler. Standard flow matching parameterizes probability paths as straight-line interpolations in Euclidean space; that is incompatible with 4DGS scale (positive real), rotation (SO(3)), and opacity (0-1 interval) parameters, which live on manifolds where a Euclidean linear interpolation between two valid endpoints can pass through invalid regions. RFM defines geodesic probability paths on each parameter’s native manifold so all intermediate states along the sampling trajectory are valid 4DGS. The paper isolates this design choice against a deterministic MLP head that receives the same features, is trained on the same data, and is followed by the same test-time optimization step, so the +0.044 F-score delta is attributable to RFM specifically rather than to data, capacity, or TTO.

Results

Section titled “Results”
  • Deterministic MLP baseline (same data + architecture + TTO): F-score 0.762 [Abstract].
  • RFM: F-score 0.806, +0.044 over the isolated baseline [Abstract].
  • Outperforms prior Depth Anything models and per-scene 4D-GS on dynamic reconstruction and novel-view synthesis [Abstract].
  • Pre-training compute: 360 GPU-hours [Abstract].
  • Uncertainty calibration reported via NLL and ECE (specific numbers not in the abstract) [Abstract].

Why it’s interesting

Section titled “Why it’s interesting”

DAV4 is the first filed paper in the 4D Scene Generation cluster to run a clean isolation ablation on flow-matching-vs-deterministic-regression for 4DGS parameter prediction — the +0.044 F-score number is the kind of same-data, same-architecture, same-TTO delta the cluster’s other members have not reported. It sits between two neighbors already on file: Depth Anything 3: Recovering the Visual Space from Any Views, the ByteDance Depth Anything 3 backbone that is deterministic-regression + optional 3D-Gaussian head, and NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videos, which “Gaussianizes VGGT” for feed-forward 4DGS but leaves the regression head deterministic. If the +0.044 gap replicates outside this setup, it would suggest the deterministic-head design shared by DA3 and NeoVerse is leaving quality on the table specifically because it ignores the manifold structure of 4DGS parameters — a design lesson the Riemannian-manifold-aware branch of Modular Manifolds (Muon on Stiefel) has been making in optimizer space. The 360-GPU-hour framing is also a data point for the same-cluster cost debate: NeoVerse and VerseCrafter both argue reconstruct-then-generate degrades gracefully; DAV4 argues the pretraining step itself is cheap enough to amortize once the deployment reaches 10k scenes.

See also

Section titled “See also”