cs.CVOct 26, 2026
SaveLearning Scene Coordinate Reconstruction from Unposed Images via Pose Graph Optimization
Organizations: dConstruct Robotics · National University of Singapore
Abstract
Learning-based structure-from-motion methods such as ACE-Zero have demonstrated strong performance in estimating cameraposes and scene coordinates from unordered image collections without requiring ground truth supervision. However, the lack of global and multi-view consistency constraints in ACE-Zero can lead to pose drift and misalignment, particularly in complex or ambiguous scenes. In this work, we propose a hybrid framework that integrates pose graph optimization (PGO) into ACE-Zero to refine camera poses and suppress incorrect refinements. We construct pose graphs directly from ACE-Zero outputs by extracting relative pose constraints from predicted scene coordinates. Furthermore, we introduce an uncertainty-aware optimization strategy by estimating confidence scores using geometric priors, including epipolar and optical flow consistencies across views. Our approach improves the robustness and accuracy of pose estimation, demonstrating that global geometric reasoning can effectively complement learning-based inference in structure-from-motion.
Figures & tables
Figure 1: Despite being initialized from (a) COLMAP generated camera poses, local refinements in (b) often introduce global inconsistencies. In contrast, (c) our hybrid-framework, which integrates pose graph optimization into ACE-Zero, enforces multi-view consistency and suppress incorrect refinements.
Figure 2: Qualitative example of the epipolar-based uncertainty metric. We overlay the reprojected points and their corresponding epipolar lines which are color-coded by error (green=low, red=high), indicating geometric alignment quality under perturbed pose. In (a), errors remain low with no perturbation applied. Introducing pose perturbations in (b) and (c) naturally increases the error. Interestingly, adding translation perturbations in (d) and (e) can sometimes reduce the error, indicating that epipolar error is sensitive to interactions between pose components and does not always correlate directly with pose accuracy. Consequently, we interpret it as a heuristic confidence indicator rather than a strict geometric validator.
Figure 3: Qualitative example of optical flow predictions. (a) One image from the input pair. (b) Predicted flow visualization where similar colours indicate similar motion directions. (c) UV correspondences overlaid on the image pair, demonstrating that optical flow provides robust pixel-level matches between views.
Figure 4: Qualitative comparison on Mip-NeRF 360 (top two rows) and Tanks and Temples datasets (bottom two rows) with VGGT-SLAM [ 27 ] and ACE-Zero [ 8 ] . While ACE-Zero performs well on small-scale scenes such as Mip-NeRF 360, it struggles with large, complex environments like Tanks and Temples, where our method maintains high rendering quality across diverse scenes.
Figure 5: Ablations on (a) design components, and sensitivity to (b) hyperparameter and (c) sparse training data.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Despite being initialized from (a) COLMAP generated camera poses, local refinements in (b) often introduce global inconsistencies. In contrast, (c) our hybrid-framework, which integrates pose graph optimization into ACE-Zero, enforces multi-view consistency and suppress incorrect refinements. Furthermore, we demonstrate that (d) ACE-Zero fails without COLMAP initialization, and (e) the low PSNR indicates that VGGT-SLAM lacks accuracy despite its ability to perform loop closure.
Figure 7: We provide several sample image frames from the office loop video that were used for the teaser example.
Figure 8: Qualitative comparison from the office loop video that were used for the teaser example. Although ACE-Zero, even when initialized with COLMAP-generated camera poses, tends to exhibit artifacts and blurriness in synthesized novel views, our proposed method consistently delivers high-fidelity renderings. This improvement reflects a more globally consistent and structurally coherent scene reconstruction.
| 7-Scenes | Mip-NeRF 360 | |
| 0.4 : 0.6 (Best) | 21.7 | 24.3 |
| 0.5 : 0.5 | 21.5 | 24.1 |
| 0.3 : 0.7 | 21.6 | 24.2 |
| 0.6 : 0.4 | 21.3 | 23.8 |
| 0.7 : 0.3 | 21.0 | 23.5 |
| 1.0 : 0.0 | 20.5 | 22.8 |
Table 9: Ablations on different ratios on 7-Scenes and Mip-NeRF 360 datasets. We report the pose accuracy via view synthesis as PSNR in dB.
| Configuration | 7-Scenes | Mip 360 |
| Vanilla PGO (with construction) | 18.5 | 21.0 |
| + Epipolar metric | 19.5 | 22.0 |
| + Optical flow metric | 20.1 | 22.3 |
| + Both metrics | 20.8 | 23.2 |
Table 10: Ablations on different uncertainty-aware configurations. We report the pose accuracy via view synthesis as PSNR in dB.
Figure 9: Ablations on sparse training data for both Mip-NeRF 360 and Tanks and Temples (T2) datasets. We show that our method consistently demonstrates better robustness to limited training data.
Figure 10: Qualitative example of the epipolar-based uncertainty metric (top row: 7-Scenes dataset; bottom row: Tanks and Temples dataset). We overlay reprojected points with their corresponding epipolar lines, color-coded by error (green = low, red = high), to indicate geometric alignment quality under perturbed poses. In the top row, the highest error occurs when no perturbation is applied, suggesting the original camera pose is inaccurate; introducing rotation perturbations reduces the error. In the bottom row, from left to right, we observe a similar trend as in Fig. 2 of the main text: errors remain low without perturbation, increase with pose perturbations, and occasionally decrease when translation perturbations are added. This behavior highlights that epipolar error is sensitive to interactions between pose components and does not always correlate directly with pose accuracy. Consequently, we treat it as a heuristic confidence indicator rather than a strict geometric validator.
Figure 11: Qualitative examples of optical flow predictions are shown (top row: 7-Scenes dataset; bottom row: Tanks and Temples dataset). From left to right: (i) an image from the input pair, (ii) predicted flow visualization where similar colors indicate similar motion directions, and (iii) UV correspondences overlaid on the image pair, illustrating that optical flow provides robust pixel-level matches across views.
Figure 12: Qualitative comparison on Mip-NeRF 360 (top two rows) and Tanks and Temples datasets (bottom two rows) with VGGT-SLAM [ 27 ] and ACE-Zero [ 8 ] . While ACE-Zero performs well on small-scale scenes such as Mip-NeRF 360, it struggles with large, complex environments like Tanks and Temples, where our method maintains high rendering quality across diverse scenes.
More in this topic
Camera Pose Estimation
Camera pose estimation recovers a camera’s position and orientation from images, visual features, scene geometry, and sometimes sensor measurements.
See all 237 papers