VGGT-360: Geometry-Consistent Zero-Shot Panoramic Depth Estimation

Jiayi Yuan; Haobo Jiang; De Wen Soh; Na Zhao

cs.CVMar 19, 2026

VGGT-360: Geometry-Consistent Zero-Shot Panoramic Depth Estimation

Authors: Jiayi Yuan, Haobo Jiang, De Wen Soh, Na Zhao

Organizations: Singapore University of Technology and Design · Nanyang Technological University

Abstract

This paper presents VGGT-360, a novel training-free framework for zero-shot, geometry-consistent panoramic depth estimation. Unlike prior view-independent training-free approaches, VGGT-360 reformulates the task as panoramic reprojection over multi-view reconstructed 3D models by leveraging the intrinsic 3D consistency of VGGT-like foundation models, thereby unifying fragmented per-view reasoning into a coherent panoramic understanding. To achieve robust and accurate estimation, VGGT-360 integrates three plug-and-play modules that form a unified panorama-to-3D-to-depth framework: (i) Uncertainty-guided adaptive projection slices panoramas into perspective views to bridge the domain gap between panoramic inputs and VGGT's perspective prior. It estimates gradient-based uncertainty to allocate denser views to geometry-poor regions, yielding geometry-informative inputs for VGGT. (ii) Structure-saliency enhanced attention strengthens VGGT's robustness during 3D reconstruction by injecting structure-aware confidence into its attention layers, guiding focus toward geometrically reliable regions and enhancing cross-view coherence. (iii) Correlation-weighted 3D model correction refines the reconstructed 3D model by reweighting overlapping points using attention-inferred correlation scores, providing a consistent geometric basis for accurate panoramic reprojection. Extensive experiments show that VGGT-360 outperforms both trained and training-free state-of-the-art methods across multiple resolutions and diverse indoor and outdoor datasets.

Figures & tables

Figure 1: Comparison between the conventional training-free panoramic depth estimation framework and our VGGT-360 . Unlike view-independent inference methods ( e.g . , 360MD [ 29 ] ), VGGT-360 reconstructs a globally coherent 3D representation via VGGT-like 3D foundation models and reprojects it to the panorama, unifying fragmented per-view predictions into consistent, cross-view correlated depth with superior performance.

Figure 2: Framework Overview of VGGT-360 . Given a panoramic image, we first perform uncertainty-guided adaptive projection to produce geometry-informative views for VGGT. With structure-saliency enhanced attention , VGGT reconstructs a structure-faithful 3D model, which is then refined by correlation-weighted 3D model correction and reprojected into a globally consistent panoramic depth map.

Figure 3: Pipeline of our uncertainty-guided adaptive projection . We first generate $N_{\mathcal{B}}$ base views from the panorama, compute per-view uncertainty maps via edge-based scoring, and select the top- $K$ most uncertain views (with $N_{\mathcal{B}}{=}6$ , $K{=}1$ in this example). These views are then augmented with neighboring projections to form a geometry-aware multi-view set as input for VGGT.

Figure 4: Comparison of results before and after applying our structure-saliency enhanced attention mechanism . Guided by our well-designed structure-aware confidence map , our VGGT-360 effectively removes artifacts and preserves geometric structures in weakly structured regions, which are easily affected by illumination cues and noise.

Figure 5: Pipeline of our correlation-weighted 3D model correction module . Each overlapping 3D point is assigned a correlation score derived from VGGT’s intra-frame attention. These scores are then used as reliability weights to refine the reconstructed 3D model and produce the final coherent ERP depth.

Test Dataset	Method	Backbone	Train $\to$ Test	Abs Rel $\downarrow$	$\boldsymbol{\delta_{1}}\uparrow$	$\boldsymbol{\delta_{2}}\uparrow$	$\boldsymbol{\delta_{3}}\uparrow$
Matterport3D	BiFuse++ [ 39 ]	ResNet34 [ 15 ]	M $\to$ M	0.112	0.881	0.966	0.987
	EGFormer [ 46 ]	Transformer	M $\to$ M	0.147	0.816	0.939	0.974
	HRDFuse [ 1 ]	ResNet34 [ 15 ]	M $\to$ M	0.117	0.867	0.962	0.985
	Elite360D [ 2 ]	EfficientNet-B5 [ 34 ]	M $\to$ M	0.105	0.899	0.971	0.991
	Depth Anywhere [ 41 ]	Depth Anything [ 44 ] , UniFuse [ 21 ]	M+ $\to$ M	0.089	0.911	0.975	0.991
	Depth Anywhere [ 41 ]	Depth Anything [ 44 ] , BiFuse++ [ 39 ]	M+ $\to$ M	0.085	0.917	0.976	0.991

Table 1: Quantitative comparisons with SOTA methods on Matterport3D [ 9 ] , Stanford2D3D [ 3 ] , and Replica360-2K [ 32 , 29 ] test sets.

Figure 6: Comparison of reconstructed 3D models without (a) and with (b) our correlation-weighted 3D model correction . Our correction module significantly enhances surface continuity and removes artifacts in overlapping regions.

Figure 7: Qualitative comparisons with the state-of-the-art supervised method Depth-Anywhere (DA) [ 41 ] and the training-free method 360MD [ 29 ] across three indoor datasets: Matterport3D [ 9 ] , Stanford2D3D [ 3 ] , and Replica360-2K [ 32 , 29 ] .

Figure 8: Qualitative comparison with SOTA methods: supervised Depth-Anywhere (DA) [ 41 ] and training-free 360MD [ 29 ] , on outdoor panoramas from OmniPhotos [ 5 ] .

Figure 9: Ablation studies on Stanford2D3D [ 3 ] : effectiveness of the three proposed modules across various VGGT-like baselines.

Effect of Structure-Saliency Enhanced Attention
Method	Abs Rel $\downarrow$	RMSE $\downarrow$	Time
Baseline	0.080	0.354	1.41 s
Baseline + $\mathbf{M}_{g}$	0.075	0.343	1.43s
Baseline + $\mathbf{M}_{g}$ + $\mathbf{E}$	0.073	0.346	1.44s
Baseline + $\mathbf{M}_{g}$ + $\mathbf{E}$ + $\mathbf{1}_{\mathrm{valid}}$	0.072	0.340	1.45s
Effect of Correlation-Weighted 3D Model Correction

Table 2: Ablation of VGGT [ 40 ] baseline on Stanford2D3D [ 3 ] .

Figure 10: Ablation studies on Stanford2D3D [ 3 ] : impact of projection parameters (base views ( $N_{\mathcal{B}}{=}6$ - $12$ ) and top- $K$ neighbor augmentation ( $K{=}0$ - $3$ )) on performance (a) and runtime (b).

Explore similar work

Depth Anything in $360^\circ$ : Towards Scale Invariance in the Wild
Dec 28, 2025Hualie Jiang, Ziyang Song, Zhiqiang Lou +2Depth Estimation Monocular Depth Estimation
Insta360 Research
Unified Panoramic Geometry Estimation via Multi-View Foundation Models
May 25, 2026Vukasin Bozic, Isidora Slavkovic, Dominik Narnhofer +43D Foundation Models 3D ViTs
ETH Zürich · Google · Athlence Sports +1
RePer-360: Releasing Perspective Priors for 360 $^\circ$ Depth Estimation via Self-Modulation
Mar 6, 2026Cheng Guan, Chunyu Lin, Zhijie Shen +2Domain Adaptation Depth Estimation
Institute of Information Science, Beijing Jiaotong University, Beijing 100044, China. · Visual Intelligence +X International Cooperation Joint Laboratory of MOE, Beijing 100044, China.

Dec 28, 2025J/K move · Enter open · S save

Depth Anything in $360^\circ$ : Towards Scale Invariance in the Wild

Hualie Jiang, Ziyang Song, Zhiqiang Lou, Rui Xu+1

Insta360 Research

Depth Estimation Monocular Depth Estimation Vision Foundation Model Adaptation

Open paper AskPDF arXiv

Panoramic depth estimation captures the complete 360

^\circ

scene geometry, being essential for robotics and AR/VR applications. While perspective depth models have achieved remarkable zero-shot generalization via large-scale training, panoramic methods lag behind, especially for open-world scenes, due to data scarcity. To bridge this gap, we introduce DA360, a panoramic-adapted version of Depth Anything V2. Our key insight is that the base DAV2 model, trained on perspective images to predict affine-invariant disparity, already exhibits good zero-shot performance on panoramas. Building on this, we design a lightweight adaptation framework that (i) learns a per-image shift from the ViT class token with scale-invariant supervision, transforming affine-invariant disparity into scale-invariant disparity that directly yields well-formed 3D point clouds, and (ii) integrates circular padding into the DPT decoder to eliminate seam artifacts, ensuring spatial coherence. Fine-tuned on a combination of synthetic indoor and outdoor panoramic data, DA360 is evaluated on standard real-world indoor benchmarks and our newly curated outdoor dataset, Metropolis. Results show that DA360 not only outperforms the original DAV2 by over 50% and 12% relative error reduction indoors and outdoors, but also surpasses prior specialized methods like PanDA by about 25--35% across all tests, establishing state-of-the-art zero-shot panoramic depth estimation.

Abstract

Explore similar work

Depth Anything in 360∘360^\circ360∘: Towards Scale Invariance in the Wild

Unified Panoramic Geometry Estimation via Multi-View Foundation Models

RePer-360: Releasing Perspective Priors for 360∘^\circ∘ Depth Estimation via Self-Modulation

Depth Anything in $360^\circ$ : Towards Scale Invariance in the Wild

RePer-360: Releasing Perspective Priors for 360 $^\circ$ Depth Estimation via Self-Modulation