E-RayZer: Self-supervised 3D Reconstruction as Spatial Visual Pre-training

TL;DR

E-RayZer (CVPR 2026, Adobe Research + CMU) is a self-supervised large 3D vision model that learns 3D-aware representations directly from unlabeled multi-view images by performing self-supervised 3D reconstruction with explicit geometry. Unlike its predecessor RayZer, which infers 3D indirectly via latent-space view synthesis, E-RayZer predicts explicit 3D Gaussians as the scene representation and renders target views via self-predicted cameras under a photometric loss — eliminating shortcut solutions that RayZer exhibited (grid-like interpolation artifacts, flickering surfaces). A DINO-similarity-based curriculum organizes training from easy-to-hard visual overlap to make convergence at scale tractable. The model matches or surpasses fully-supervised VGGT on pose estimation and reconstruction; its learned features beat DINOv3, CroCo v2, VideoMAE V2, and RayZer on 3D downstream tasks (multi-view depth, pose estimation, pairwise flow), positioning explicit-3D self-supervision as a viable spatial-pretraining objective.

Key claims

Self-supervised 3D reconstruction with explicit Gaussian geometry beats RayZer’s implicit latent-view-synthesis recipe — RayZer’s renderings exhibit grid-like artifacts in high-uncertainty / occluded / wide-baseline regions, evidence it learned unwanted interpolation shortcuts rather than 3D structure [§Comparisons on Novel View Synthesis].
A single self-supervised checkpoint matches or surpasses fully-supervised VGGT* (re-trained under the same settings) on pose estimation and reconstruction, with the gap widening as training data scales up [§Self-supervised E-RayZer Scales Well].
E-RayZer’s learned features outperform DINOv3, CroCo v2, VideoMAE V2, and RayZer when transferred (frozen-backbone or fully fine-tuned) to multi-view depth and pose estimation on ScanNet++ and BlendedMVS — datasets not in pretraining for any model [§Results on 3D Tasks].
On 2.5D pairwise flow (StaticThings3D, OOD synthetic), E-RayZer features beat the same baselines under flow supervision [§Results on 2.5D Task].
E-RayZer features show better object discrimination and spatial coherence than RayZer under PCA visualization — a qualitative signal that the explicit-geometry objective propagates back into the encoder [§Representation Learning].
A DINO-embedding visual-overlap curriculum is the load-bearing trick for scaling: direct from-scratch training fails, and DINO similarity beats the naive “frame interval” proxy as the easy-to-hard ranking signal [§Learning Curriculum].
Using E-RayZer as pretraining (then fine-tuning a supervised model on top) improves the supervised model itself, supporting a self-supervised pretrain + supervised fine-tune paradigm for spatial vision [§Self-supervised E-RayZer Scales Well].
The pipeline is fully self-supervised: E-RayZer first predicts camera poses + intrinsics for all input images, splits images into reference and target sets (RayZer-style), predicts 3D Gaussians from references, renders targets through self-predicted cameras, and trains under a photometric loss with no 3D annotations [§Model & Training].

Method

E-RayZer takes a set of unposed, unlabeled multi-view images and learns 3D-aware representations through three coupled predictions in a single forward pass: (a) per-image camera intrinsics + extrinsics, (b) a per-pixel 3D Gaussian (explicit scene representation) over the reference views, and (c) photometric renderings of the held-out target views using the self-predicted target cameras. Training is supervised only by the photometric loss between rendered target views and the held-out target images — no 3D ground truth, no posed inputs, no labeled depth. The split between reference and target views follows the RayZer recipe; the key change is that the scene is now an explicit point cloud of 3D Gaussians, not an implicit latent. This eliminates the “fake it through view interpolation” shortcut that RayZer exhibits in regions of high uncertainty, occlusion, and large viewpoint change.

Training from scratch this way often fails to converge, so the authors introduce a fine-grained curriculum: image pairs are ranked from high to low visual overlap, with DINO-embedding cosine similarity used as the overlap proxy (better than frame interval). The model is exposed to high-overlap (easy) samples first, then progressively to harder wide-baseline samples, harmonizing heterogeneous data sources without manual dataset weighting.

Results

Pose estimation: E-RayZer outperforms baselines on rotation/translation accuracy across the evaluation suite, with predicted poses (visualized after Procrustes alignment to ground truth) tracking ground-truth black markers tightly where RayZer and SPFSplat diverge [§Comparisons on Pose Estimation].
Novel view synthesis: RayZer renders look superficially high-quality but exhibit visible artifacts on examples 4 and 8 (high uncertainty / occlusion / large viewpoint change); E-RayZer is more stable thanks to its explicit geometry. SPFSplat fails on wide-baseline inputs [§Comparisons on NVS].
Representation transfer (3D tasks): On ScanNet++ + BlendedMVS multi-view depth and pose, E-RayZer features are best (bold) or second-best (underlined) on the reported settings across frozen-backbone and fully-finetuned protocols, beating DINOv3 / CroCo v2 / VideoMAE V2 / RayZer [§Representation Learning].
Representation transfer (2.5D / flow): On out-of-distribution StaticThings3D pairwise flow estimation under full flow supervision, E-RayZer features again top or near-top the rankings [§Representation Learning].
Scaling: Across multiple training-data scales, E-RayZer’s red-to-yellow ranking pattern shows monotone improvement with data; underlined cells mark cases where self-supervised E-RayZer beats supervised VGGT* trained on the same data scale [§Self-supervised E-RayZer Scales Well].
Headline quantitative numbers (PSNR, AUC, MPJPE, etc.) are tabulated on the project page but not reproduced in the marketing copy; numbers will be in the CVPR paper.

Why it’s interesting

E-RayZer plants a flag on the opposite side of the foundation-layer debate the wiki has been tracking. The flavor of the bitter lesson for computer vision argues that explicit 3D representations (point clouds, Gaussians, NeRFs) are hand-crafted intermediates the Bitter Lesson should dissolve in favor of end-to-end generative-rollout world models. E-RayZer is a direct counterbet: it shows that self-supervised training on an explicit-3D objective transfers better than DINOv3-style image SSL or CroCo-style cross-view-completion SSL on the spatial tasks Sitzmann cares about — and scales like a foundation model. This complements VGGT-Ω: Scaling Feed-Forward Reconstruction for Static and Dynamic Scenes (supervised scaling of the same geometry-foundation recipe) and Depth Anything 3: Recovering the Visual Space from Any Views (vanilla-DINOv2 + Dual-DPT minimal-prediction-target alternative) by removing the supervision requirement entirely. It is also the natural backbone candidate for the 4D Scene Generation cluster’s reconstruction-then-generate methods like NeoVerse: Enhancing 4D World Model with in-the-wild Monocular Videos (NeoVerse “Gaussianizes” VGGT into a feed-forward 4DGS reconstructor — E-RayZer is the same recipe with a self-supervised backbone). Jiaming’s framing — asking whether the field should return to 3D reconstruction — captures the moment: a year of generative video and predictive-latent WFMs has not closed the explicit-3D-spatial-pretraining angle.