Generalized panorama depth estimation has witnessed great advances in recent years. These methods typically achieve robust zero-shot generalization performance using large pretrained models, but their high computational cost makes real-time inference on embedded platforms challenging. In contrast, previous lightweight panorama models have mostly been developed for single-domain evaluations and cannot generalize well due to domain shifts. To reduce the gap between these two types of models, we introduce EG360, an efficient and generalizable panorama depth estimation model. Specifically, building on a lightweight depth model, we design an Adaptive Panorama-Perspective Distillation (APPD) strategy guided by scene geometry, along with a Spherical Cross-Level Feature Fusion (SCFF) module for global semantic perception. As a result, with only 6.593M parameters after reparameterization, EG360 achieves strong zero-shot performance on real-world benchmarks and runs at 35 FPS on a Jetson Orin NX 16GB in MAXN mode, whereas a previous ViT-small model runs at less than 2 FPS. Our code will also be made public for future research.
Figures & tables
Figure 1: Accuracy–computation trade-off for panoramic depth estimation. Mean AbsRel is averaged equally over Matterport3D, Stanford2D3D, and Metropolis. Filled marker area is positively proportional to measured inference FPS on a Jetson Orin NX 16GB (MAXN mode, PyTorch eager, AMP FP16, batch size 1). EG360 uses reparameterization for inference. PaGeR ran into out-of-memory (OOM) and is represented by a fixed-size hollow diamond.
Datasets
Indoor
Outdoor
Samples
Labeled Synthetic Datasets
Structured3D [ 59 ]
✓
18,222
PanoCARLA [ 36 ]
✓
112,212
TartanAirV2 [ 48 ]
✓
✓
432,381
Labeled total
562,815
Unlabeled Real-world Datasets
Table 1: Labeled synthetic and unlabeled real-world datasets used.
Stanford2D3D
Metropolis
Teacher
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
DAP [ 23 ]
0.071
0.965
0.185
0.733
MoGe2 [ 47 ]
0.150
0.774
0.127
0.862
Table 2: Comparison of panorama and perspective teachers. Metrics are averaged equally over six cube faces for Stanford2D3D and four horizontal faces for Metropolis, as its top and bottom faces lack valid ground-truth depth. Best results are bolded.
Figure 2: Visualization of geometry-adaptive teacher weighting. Faces dominated by planar surfaces, such as walls or ceilings, receive higher panorama teacher weights, while views with greater geometric variation receive lower weights (blue or green).
Figure 3: Diagram of Spherical Cross-Level Feature Fusion. Cross-attention with a spherical direction bias is followed by a local feed-forward network with panorama padding and depthwise convolution. Reshaping is omitted for brevity.
Matterport3D
Stanford2D3D
Metropolis
Method
Params (M)
GFLOPs ↓
Latency (ms) ↓
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
Foundation models
DA360-S † [ 18 ]
24.804
299.520
517.412
0.168
0.802
0.082
0.947
0.232
0.597
DAP [ 23 ]
365.050
2543.697
1868.322
0.137
0.840
0.072
0.963
0.195
0.734
DepthMaster [ 46 ]
385.191
11521.707
OOM
0.101
0.905
0.062
0.965
1.022
0.263
PaGeR [ 5 ]
1286.563
26200.188
OOM
0.116
0.889
0.064
0.972
0.224
0.500
Table 3: Comparison with existing method without ground-truth scale alignment. † denotes models retrained or finetuned with our recipe. Latency is measured on a Jetson Orin NX 16GB in MAXN mode, with AMP FP16 and batch size 1. EG360 uses structural reparameterization. OOM denotes out of memory. Bold denotes the best value in each column within each group.
Figure 4: Qualitative comparisons on Matterport3D (top) and Metropolis (bottom). The first column shows ground-truth depth and RGB images; the remaining columns show predictions above per-pixel AbsRel maps for UniFuse, EG360, DA360-S, and DAP. Error colors range from purple (low) to yellow (high). Pixels without ground truth depth are masked in the error map.
Matterport3D
Stanford2D3D
Metropolis
Method
Params (M)
GFLOPs ↓
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
Baseline
6.144
21.196
0.199
0.722
0.126
0.850
0.266
0.578
+ APPD
6.144
21.196
0.169
0.781
0.101
0.903
0.244
0.619
+ SCFF
6.593
23.492
0.195
0.733
0.113
0.876
0.263
0.582
EG360
6.593
23.492
0.173
0.784
0.094
0.923
0.237
0.632
Table 4: Overall ablation of APPD and SCFF on three real-world test datasets. EG360 includes both components. Bold denotes the best value in each column.
Matterport3D
Stanford2D3D
Metropolis
Distill Method
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
No distill
0.199
0.722
0.126
0.850
0.266
0.578
DAP only
0.176
0.771
0.101
0.901
0.257
0.601
MoGe2 only
0.176
0.762
0.146
0.762
0.235
0.628
DAP + MoGe2 (Avg)
0.168
0.780
0.109
0.889
0.238
0.630
Table 5: Teacher complementarity with our baseline student. Using both teachers leads to better results.
Matterport3D
Stanford2D3D
Metropolis
Batch size
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
1
0.184
0.758
0.115
0.875
0.240
0.630
2
0.175
0.769
0.114
0.872
0.242
0.613
4
0.173
0.778
0.110
0.883
0.244
0.618
8
0.168
0.780
0.109
0.889
0.238
0.630
16
0.166
0.782
0.108
0.890
0.239
0.628
Table 6: Effect of the batch size for unlabeled data under Avg distillation. The first column is the unlabeled batch size.
Matterport3D
Stanford2D3D
Metropolis
Distill Method
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
Avg
0.168
0.780
0.109
0.889
0.238
0.630
Random
0.172
0.770
0.113
0.874
0.240
0.625
Manual
0.166
0.777
0.107
0.885
0.238
0.604
APPD
0.169
0.781
0.101
0.903
0.244
0.619
Table 7: Joint teacher strategies. Avg averages the two teacher losses. Random selects one teacher with equal probability per unlabeled batch. Manual assigns DAP to ZInD and MoGe2 to HoliCity. APPD is our geometry-adaptive loss weighting.
Matterport3D
Stanford2D3D
Metropolis
Loss
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
L1
0.168
0.780
0.109
0.889
0.238
0.630
SI
0.228
0.680
0.155
0.805
0.246
0.603
LSS
0.176
0.748
0.111
0.901
0.466
0.095
Table 8: Pseudo-label loss type ablation under Avg teacher fusion.
Matterport3D
Stanford2D3D
Metropolis
Method
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
AbsRel ↓
δ1↑
UniFuse
0.234
0.663
0.155
0.754
0.297
0.500
+ APPD
0.197
0.745
0.119
0.867
0.250
0.603
BiFuse++
0.295
0.555
0.204
0.717
0.398
0.260
+ APPD
0.211
0.718
0.158
0.760
0.270
0.546
ACDNet
0.233
0.670
0.146
0.795
0.301
0.487
Table 9: Applying APPD to other lightweight models.
Figure 5: Qualitative results of applying APPD to our baseline model. The error is reduced in this real-world example.
Figure 6: Qualitative results of using SCFF. The error for the large shelf is lowered as SCFF allows long-range context modeling.
Configuration
Params (M)
GFLOPs ↓
AbsRel ↓
δ1↑
Baseline
6.144
21.196
0.126
0.850
Full
6.593
23.492
0.113
0.876
Full − D
6.593
23.492
0.125
0.864
Full − Cross
6.296
21.814
0.119
0.867
Full −3×3 DW Conv
6.589
23.478
0.122
0.868
Full − Cross + C4 Self
6.839
22.173
0.119
0.875
Table 10: SCFF ablation on Stanford2D3D using labeled data only. Full is the complete SCFF module. − D removes the spherical direction bias, − Cross removes cross-level attention, and −3×3 DW Conv removes the depthwise convolution in the panoramic FFN. Full − Cross + C4 Self/C3 Self replaces the cross attention with a self-attention block on C4/C3.
Panoramic depth estimation captures the complete 360∘ scene geometry, being essential for robotics and AR/VR applications. While perspective depth models have achieved remarkable zero-shot generalization via large-scale training, panoramic methods lag behind, especially for open-world scenes, due to data scarcity. To bridge this gap, we introduce DA360, a panoramic-adapted version of Depth Anything V2. Our key insight is that the base DAV2 model, trained on perspective images to predict affine-invariant disparity, already exhibits good zero-shot performance on panoramas. Building on this, we design a lightweight adaptation framework that (i) learns a per-image shift from the ViT class token with scale-invariant supervision, transforming affine-invariant disparity into scale-invariant disparity that directly yields well-formed 3D point clouds, and (ii) integrates circular padding into the DPT decoder to eliminate seam artifacts, ensuring spatial coherence. Fine-tuned on a combination of synthetic indoor and outdoor panoramic data, DA360 is evaluated on standard real-world indoor benchmarks and our newly curated outdoor dataset, Metropolis. Results show that DA360 not only outperforms the original DAV2 by over 50% and 12% relative error reduction indoors and outdoors, but also surpasses prior specialized methods like PanDA by about 25--35% across all tests, establishing state-of-the-art zero-shot panoramic depth estimation.
Recent depth foundation models trained on perspective imagery achieve strong performance, yet generalize poorly to 360∘ images due to the substantial geometric discrepancy between perspective and panoramic domains. Moreover, fully fine-tuning these models typically requires large amounts of panoramic data. To address this issue, we propose RePer-360, a distortion-aware self-modulation framework for monocular panoramic depth estimation that adapts depth foundation models while preserving powerful pretrained perspective priors. Specifically, we design a lightweight geometry-aligned guidance module to derive a modulation signal from two complementary projections (i.e., ERP and CP) and use it to guide the model toward the panoramic domain without overwriting its pretrained perspective knowledge. We further introduce a Self-Conditioned AdaLN-Zero mechanism that produces pixel-wise scaling factors to reduce the feature distribution gap between the perspective and panoramic domains. In addition, a cubemap-domain consistency loss further improves training stability and cross-projection alignment. By shifting the focus from complementary-projection fusion to panoramic domain adaptation under preserved pretrained perspective priors, RePer-360 surpasses standard fine-tuning methods while using only 1% of the training data. Under the same in-domain training setting, it further achieves an approximately 20% improvement in RMSE. The code is available at https://github.com/munimo/RePer360.
Cheng Guan, Chunyu Lin, Zhijie Shen +2
Institute of Information Science, Beijing Jiaotong University, Beijing 100044, China. · Visual Intelligence +X International Cooperation Joint Laboratory of MOE, Beijing 100044, China.
This paper presents VGGT-360, a novel training-free framework for zero-shot, geometry-consistent panoramic depth estimation. Unlike prior view-independent training-free approaches, VGGT-360 reformulates the task as panoramic reprojection over multi-view reconstructed 3D models by leveraging the intrinsic 3D consistency of VGGT-like foundation models, thereby unifying fragmented per-view reasoning into a coherent panoramic understanding. To achieve robust and accurate estimation, VGGT-360 integrates three plug-and-play modules that form a unified panorama-to-3D-to-depth framework: (i) Uncertainty-guided adaptive projection slices panoramas into perspective views to bridge the domain gap between panoramic inputs and VGGT's perspective prior. It estimates gradient-based uncertainty to allocate denser views to geometry-poor regions, yielding geometry-informative inputs for VGGT. (ii) Structure-saliency enhanced attention strengthens VGGT's robustness during 3D reconstruction by injecting structure-aware confidence into its attention layers, guiding focus toward geometrically reliable regions and enhancing cross-view coherence. (iii) Correlation-weighted 3D model correction refines the reconstructed 3D model by reweighting overlapping points using attention-inferred correlation scores, providing a consistent geometric basis for accurate panoramic reprojection. Extensive experiments show that VGGT-360 outperforms both trained and training-free state-of-the-art methods across multiple resolutions and diverse indoor and outdoor datasets.
Jiayi Yuan, Haobo Jiang, De Wen Soh +1
Singapore University of Technology and Design · Nanyang Technological University