This paper presents VGGT-360, a novel training-free framework for zero-shot, geometry-consistent panoramic depth estimation. Unlike prior view-independent training-free approaches, VGGT-360 reformulates the task as panoramic reprojection over multi-view reconstructed 3D models by leveraging the intrinsic 3D consistency of VGGT-like foundation models, thereby unifying fragmented per-view reasoning into a coherent panoramic understanding. To achieve robust and accurate estimation, VGGT-360 integrates three plug-and-play modules that form a unified panorama-to-3D-to-depth framework: (i) Uncertainty-guided adaptive projection slices panoramas into perspective views to bridge the domain gap between panoramic inputs and VGGT's perspective prior. It estimates gradient-based uncertainty to allocate denser views to geometry-poor regions, yielding geometry-informative inputs for VGGT. (ii) Structure-saliency enhanced attention strengthens VGGT's robustness during 3D reconstruction by injecting structure-aware confidence into its attention layers, guiding focus toward geometrically reliable regions and enhancing cross-view coherence. (iii) Correlation-weighted 3D model correction refines the reconstructed 3D model by reweighting overlapping points using attention-inferred correlation scores, providing a consistent geometric basis for accurate panoramic reprojection. Extensive experiments show that VGGT-360 outperforms both trained and training-free state-of-the-art methods across multiple resolutions and diverse indoor and outdoor datasets.
Figures & tables
Figure 1: Comparison between the conventional training-free panoramic depth estimation framework and our VGGT-360 . Unlike view-independent inference methods ( e.g . , 360MD [ 29 ] ), VGGT-360 reconstructs a globally coherent 3D representation via VGGT-like 3D foundation models and reprojects it to the panorama, unifying fragmented per-view predictions into consistent, cross-view correlated depth with superior performance.
Figure 2: Framework Overview of VGGT-360 . Given a panoramic image, we first perform uncertainty-guided adaptive projection to produce geometry-informative views for VGGT. With structure-saliency enhanced attention , VGGT reconstructs a structure-faithful 3D model, which is then refined by correlation-weighted 3D model correction and reprojected into a globally consistent panoramic depth map.
Figure 3: Pipeline of our uncertainty-guided adaptive projection . We first generate NB base views from the panorama, compute per-view uncertainty maps via edge-based scoring, and select the top- K most uncertain views (with NB=6 , K=1 in this example). These views are then augmented with neighboring projections to form a geometry-aware multi-view set as input for VGGT.
Figure 4: Comparison of results before and after applying our structure-saliency enhanced attention mechanism . Guided by our well-designed structure-aware confidence map , our VGGT-360 effectively removes artifacts and preserves geometric structures in weakly structured regions, which are easily affected by illumination cues and noise.
Figure 5: Pipeline of our correlation-weighted 3D model correction module . Each overlapping 3D point is assigned a correlation score derived from VGGT’s intra-frame attention. These scores are then used as reliability weights to refine the reconstructed 3D model and produce the final coherent ERP depth.
Test Dataset
Method
Backbone
Train → Test
Abs Rel ↓
δ1↑
δ2↑
δ3↑
Matterport3D
BiFuse++ [ 39 ]
ResNet34 [ 15 ]
M → M
0.112
0.881
0.966
0.987
EGFormer [ 46 ]
Transformer
M → M
0.147
0.816
0.939
0.974
HRDFuse [ 1 ]
ResNet34 [ 15 ]
M → M
0.117
0.867
0.962
0.985
Elite360D [ 2 ]
EfficientNet-B5 [ 34 ]
M → M
0.105
0.899
0.971
0.991
Depth Anywhere [ 41 ]
Depth Anything [ 44 ] , UniFuse [ 21 ]
M+ → M
0.089
0.911
0.975
0.991
Depth Anywhere [ 41 ]
Depth Anything [ 44 ] , BiFuse++ [ 39 ]
M+ → M
0.085
0.917
0.976
0.991
Table 1: Quantitative comparisons with SOTA methods on Matterport3D [ 9 ] , Stanford2D3D [ 3 ] , and Replica360-2K [ 32 , 29 ] test sets.
Figure 6: Comparison of reconstructed 3D models without (a) and with (b) our correlation-weighted 3D model correction . Our correction module significantly enhances surface continuity and removes artifacts in overlapping regions.
Figure 7: Qualitative comparisons with the state-of-the-art supervised method Depth-Anywhere (DA) [ 41 ] and the training-free method 360MD [ 29 ] across three indoor datasets: Matterport3D [ 9 ] , Stanford2D3D [ 3 ] , and Replica360-2K [ 32 , 29 ] .
Figure 8: Qualitative comparison with SOTA methods: supervised Depth-Anywhere (DA) [ 41 ] and training-free 360MD [ 29 ] , on outdoor panoramas from OmniPhotos [ 5 ] .
Figure 9: Ablation studies on Stanford2D3D [ 3 ] : effectiveness of the three proposed modules across various VGGT-like baselines.
Method
Abs Rel ↓
RMSE ↓
Time
Effect of Structure-Saliency Enhanced Attention
Baseline
0.080
0.354
1.41 s
Baseline + Mg
0.075
0.343
1.43s
Baseline + Mg + E
0.073
0.346
1.44s
Baseline + Mg + E + 1valid
0.072
0.340
1.45s
Effect of Correlation-Weighted 3D Model Correction
Table 2: Ablation of VGGT [ 40 ] baseline on Stanford2D3D [ 3 ] .
Figure 10: Ablation studies on Stanford2D3D [ 3 ] : impact of projection parameters (base views ( NB=6 - 12 ) and top- K neighbor augmentation ( K=0 - 3 )) on performance (a) and runtime (b).
Panoramic depth estimation captures the complete 360∘ scene geometry, being essential for robotics and AR/VR applications. While perspective depth models have achieved remarkable zero-shot generalization via large-scale training, panoramic methods lag behind, especially for open-world scenes, due to data scarcity. To bridge this gap, we introduce DA360, a panoramic-adapted version of Depth Anything V2. Our key insight is that the base DAV2 model, trained on perspective images to predict affine-invariant disparity, already exhibits good zero-shot performance on panoramas. Building on this, we design a lightweight adaptation framework that (i) learns a per-image shift from the ViT class token with scale-invariant supervision, transforming affine-invariant disparity into scale-invariant disparity that directly yields well-formed 3D point clouds, and (ii) integrates circular padding into the DPT decoder to eliminate seam artifacts, ensuring spatial coherence. Fine-tuned on a combination of synthetic indoor and outdoor panoramic data, DA360 is evaluated on standard real-world indoor benchmarks and our newly curated outdoor dataset, Metropolis. Results show that DA360 not only outperforms the original DAV2 by over 50% and 12% relative error reduction indoors and outdoors, but also surpasses prior specialized methods like PanDA by about 25--35% across all tests, establishing state-of-the-art zero-shot panoramic depth estimation.
Geometry estimation from perspective images has greatly advanced, maturing to the point where off-the-shelf foundation models are able to reconstruct 3D scene structure not only from multi-view imagery, but even from a single view. A natural extension is 3D reconstruction from panoramas, with the exciting prospect of recovering a full 360-degree scene from a single panoramic image. In this work, we introduce PaGeR (Panoramic Geometry Reconstruction), a framework to lift powerful 3D foundation models designed for perspective imagery to the panorama domain. Our strategy is to start from a pre-trained transformer for 3D reconstruction and turn it into a unified high-performance model that predicts scale-invariant depth, metric depth, surface normals, and sky masks from both perspective and omnidirectional images, in a single forward pass. By keeping architectural changes to a minimum and mixing perspective and panoramic images during training, PaGeR retains the rich 3D prior of the underlying foundation model while learning to also estimate geometrically consistent 360-degree scenes from single panoramas. We extensively test our method in both indoor and outdoor environments and find that it delivers state-of-the-art performance and excellent zero-shot performance across a wide range of scenes. Code, data and models are available here.
Recent depth foundation models trained on perspective imagery achieve strong performance, yet generalize poorly to 360∘ images due to the substantial geometric discrepancy between perspective and panoramic domains. Moreover, fully fine-tuning these models typically requires large amounts of panoramic data. To address this issue, we propose RePer-360, a distortion-aware self-modulation framework for monocular panoramic depth estimation that adapts depth foundation models while preserving powerful pretrained perspective priors. Specifically, we design a lightweight geometry-aligned guidance module to derive a modulation signal from two complementary projections (i.e., ERP and CP) and use it to guide the model toward the panoramic domain without overwriting its pretrained perspective knowledge. We further introduce a Self-Conditioned AdaLN-Zero mechanism that produces pixel-wise scaling factors to reduce the feature distribution gap between the perspective and panoramic domains. In addition, a cubemap-domain consistency loss further improves training stability and cross-projection alignment. By shifting the focus from complementary-projection fusion to panoramic domain adaptation under preserved pretrained perspective priors, RePer-360 surpasses standard fine-tuning methods while using only 1% of the training data. Under the same in-domain training setting, it further achieves an approximately 20% improvement in RMSE. The code is available at https://github.com/munimo/RePer360.
Cheng Guan, Chunyu Lin, Zhijie Shen +2
Institute of Information Science, Beijing Jiaotong University, Beijing 100044, China. · Visual Intelligence +X International Cooperation Joint Laboratory of MOE, Beijing 100044, China.