Prior panorama stitching approaches heavily rely on pairwise feature correspondences and are unable to leverage geometric consistency across multiple views. This leads to severe distortion and misalignment, especially in challenging scenes with weak textures, large parallax, and repetitive patterns. Given that multi-view geometric correspondences can be directly constructed in 3D space, making them more accurate and globally consistent, we extend the 2D alignment task to the 3D photogrammetric space. We adopt a novel transformer-based architecture to achieve 3D awareness and aggregate global information across all views. It directly utilizes camera poses to guide image warping for global alignment in 3D space and employs a multi-feature joint optimization strategy to compute the seams. Additionally, to establish an evaluation benchmark and train our network, we constructed a large-scale dataset of real-world scenes. Extensive experiments show that our method significantly outperforms existing alternatives in alignment accuracy and perceptual quality.
Figures & tables
Figure 1: Comparison in repetitive patterns. GES-GSP [ 6 ] suffers from inaccurate pairwise feature correspondences, causing severe artifacts. Leveraging multi-view geometric consistency in 3D space effectively filters unreliable matches, enabling robust alignment. Our method achieves a 97.8% success rate in challenging scenes with weak textures, large parallax, and repetitive patterns.
Figure 2: Architecture Overview. (a) Perspective images are projected onto a common panoramic coordinate system using camera parameters. (b) Overlapping regions are extracted from the globally aligned images (a single image may overlap multiple neighbors). (c) The seam decoder is supervised with three weight maps and generates the seam mask for each image. (d) The final panorama is blended using the seam mask and the globally aligned images, supporting various projection formats.
Figure 3: Qualitative comparison of our method Pano360 to UDIS2 on challenging real-world scenarios. As shown in the first column, our method successfully handles the large parallax scene, while UDIS2 suffers from ghosting and misalignment. In the second column, our method stitches the images in the weak-textured fog scene, while UDIS2 fails. The third column presents a challenging case with repetitive patterns, unusual lighting, and a large FoV. Our method successfully recovers a panorama from 24 frames, whereas UDIS2 is limited to pairwise processing and suffers from accumulated errors, leading to severe geometric distortion and failure.
Figure 4: Visualization of the proposed Pano360 dataset. It showcases distinct and challenging scenes. Each case presents several source images, their seam masks, and the final panorama we generated.
Method
Scene (a)
Scene (b)
Scene (c)
QAq↑
QAa↑
BRIS ↓
NIQE ↓
QAq↑
QAa↑
BRIS ↓
NIQE ↓
QAq↑
QAa↑
BRIS ↓
NIQE ↓
AutoStitch [ 3 ]
3.82
3.20
40.98
4.55
3.24
2.74
43.50
5.94
3.28
2.81
49.84
5.01
APAP [ 40 ]
3.62
3.05
54.49
3.72
3.22
3.03
52.62
3.78
3.53
3.66
45.66
3.77
ELA [ 15 ]
3.88
3.17
46.15
3.71
3.35
2.60
40.69
5.02
3.75
3.20
41.54
3.88
GSP [ 4 ]
3.87
3.25
37.55
3.53
3.20
3.45
40.66
3.96
3.65
3.57
42.19
3.71
GES-GSP [ 6 ]
3.95
3.20
36.45
3.36
3.45
3.35
32.25
3.95
3.74
3.72
44.22
3.95
Table 1: Quantitative comparison on the Pano360 Dataset. We compare our method to different types of previous approaches and mark the top results as best and second best . None of these methods were trained on the Pano360 testing dataset. Methods marked with ‡ only accept pairwise image inputs. All experiments were conducted on a single NVIDIA A100 GPU.
Table 6
Figure 5: Qualitative comparison of panorama stitching on challenging in-the-wild images. Existing methods suffer from noticeable artifacts and distortions on buildings with repetitive patterns. However, our method achieves globally consistent alignment and remains free of these artifacts.
Figure 6: Ablation study for seam detection. (a) Our full method. (b-d) Ablation of different components. (e) Baseline using traditional graph-cut with pairwise computation. Methods (a-d) perform joint optimization strategy for the seam mask.
w. Lcam
w. Lproj
w. Lseam
QAquality↑
BRIS ↓
NIQE ↓
✗
✗
✗
2.76
62.47
5.31
✓
✗
✗
3.45
47.43
4.65
✓
✓
✗
3.68
43.71
3.97
✗
✗
✓
3.01
51.12
4.83
✓
✓
✓
4.09
37.96
3.37
Table 4: Ablation study for panorama stitching components , which shows that combining with pose-guided image warping, projection function, and seam mask detection yields the highest performance on Pano360.
Multi-view consistency studies how visual models keep geometry, appearance, identity, and predictions aligned across different views of the same scene.
Panoramic imagery offers a full 360° field of view and is increasingly common in consumer devices. However, it introduces non-pinhole distortions that challenge joint pose estimation and 3D reconstruction. Existing feed-forward models, built for perspective cameras, generalize poorly to this setting. We propose PanoVGGT, a permutation-equivariant Transformer framework that jointly predicts camera poses, depth maps, and 3D point clouds from one or multiple panoramas in a single forward pass. The model incorporates spherical-aware positional embeddings and a panorama-specific three-axis SO(3) rotation augmentation, enabling effective geometric reasoning in the spherical domain. To resolve inherent global-frame ambiguity, we further introduce a stochastic anchoring strategy during training. In addition, we contribute PanoCity, a large-scale outdoor panoramic dataset with dense depth and 6-DoF pose annotations. Extensive experiments on PanoCity and standard benchmarks demonstrate that PanoVGGT achieves competitive accuracy, strong robustness, and improved cross-domain generalization. Code and dataset will be released.
Panoramic 3D reconstruction enables the holistic recovery of complete 360 ∘ scene geometry, offering significant benefits for robotics and AR/VR applications. However, directly estimating geometry from ERP images is inherently challenged by spherical distortions. To reduce spherical projection distortions, a common strategy is to decompose panoramas into distortion-free perspective views, but independently processing these views often introduces cross-view geometric inconsistencies. To address these challenges, we introduce GeoPano, a feed-forward framework for single-view panoramic 3D reconstruction that recovers a geometrically accurate and consistent point cloud from a single panorama. Our approach is driven by two key innovations: First, we propose a panoramic-aware attention aggregator, featuring a streamlined attention architecture equipped with a novel Pano-Attention mechanism, which explicitly exploits the intrinsic spherical geometry of the panorama to guide cross-view interactions, thereby suppressing geometric inconsistencies. Furthermore, we formulate a hybrid supervision strategy to jointly enforce fine-grained local details in perspective space and global structural coherence in spherical space. Experiments demonstrate that our method significantly outperforms previous SOTA approaches in reconstruction accuracy, geometric consistency, and visual fidelity.
Jing OU, Zidong Cao, Liaoyuan Fan +7
The Hong Kong University of Science and Technology (Guangzhou), China · The Hong Kong University, Hong Kong · Sun Yat-sen University, Guangzhou, China
Geometry estimation from perspective images has greatly advanced, maturing to the point where off-the-shelf foundation models are able to reconstruct 3D scene structure not only from multi-view imagery, but even from a single view. A natural extension is 3D reconstruction from panoramas, with the exciting prospect of recovering a full 360-degree scene from a single panoramic image. In this work, we introduce PaGeR (Panoramic Geometry Reconstruction), a framework to lift powerful 3D foundation models designed for perspective imagery to the panorama domain. Our strategy is to start from a pre-trained transformer for 3D reconstruction and turn it into a unified high-performance model that predicts scale-invariant depth, metric depth, surface normals, and sky masks from both perspective and omnidirectional images, in a single forward pass. By keeping architectural changes to a minimum and mixing perspective and panoramic images during training, PaGeR retains the rich 3D prior of the underlying foundation model while learning to also estimate geometrically consistent 360-degree scenes from single panoramas. We extensively test our method in both indoor and outdoor environments and find that it delivers state-of-the-art performance and excellent zero-shot performance across a wide range of scenes. Code, data and models are available here.