cs.ROOct 8, 2026

When pose graphs are not enough: a failure analysis of pose-graph-based camera estimation for Gaussian splatting

Authors: Senthil Palanisamy, Azwad Sabik, Alan Dolhasz, Mira Slavcheva

Organizations: Matterport

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Exterior Gaussian-splat twins for real-estate listings are captured to a fixed recipe: a consumer drone orbits a single property in several overlapping passes at stepped altitudes, with occasional vertical ascents, so the imagery is dense, sequential, GPS- and time-tagged, and revisits each facade from many angles. This structure is tantalizing: it suggests camera poses could be recovered by a dense pose-graph optimization (PGO) whose only unknowns are the poses, far fewer than the joint pose-and-point problem bundle adjustment (BA) solves. PGO is mature in robotics, but almost always as post-processing: it redistributes the drift of an already-decent trajectory once loop closures tie it together. Splatting captures push that regime to its limit, every frame closing loops with dozens of others, raising a clean question: does pose-only optimization reach BA accuracy? We built a system specialized to this capture structure, not a general-purpose SfM tool, to test it. It beat COLMAP on speed but never on quality; this paper is the candid post-mortem. The shortfall arises from several interacting causes, hand-set weights, greedy merging and a noisy GPS ruler, together with two limits inherent to the formulation: SE(3) edges carry no scale degree of freedom, so independently scaled sub-maps drift when merged, and a two-view edge stops short of the coupling one 3D point imposes on every frame that sees it. A viable design must add back Sim(3) constraints, absolute GPS priors, retained structure, and uncertainty-aware weighting

Figures & tables

Figure 1: The pose-only intuition. Bundle adjustment (left) couples the camera poses to a large landmark cloud through observation edges. Because the landmark block CC is block-diagonal and cheap to invert, BA marginalizes the landmarks via the Schur complement into a much smaller, pose-only reduced system S=B−EC−1E⊤S=B-EC^{-1}E^{\top} (right), solves it for the poses, and only then back-substitutes for the points. Since BA’s expensive core is already this pose-only system, a pose-only formulation tries to assemble SS directly from inter-pose relative-pose constraints (dashed), skipping the landmark detour. Both routes target the same SS ; this paper asks whether the shortcut recovers bundle-adjustment-grade poses.
Figure 2: System overview. The image stream is split into per-sequence chunks that are optimized independently as dense SE(3)\mathrm{SE}(3) pose graphs, then merged pairwise into a single graph. Every chunk-level solve is independent, which is what the design intended to parallelize across the cloud.
  dev. vs COLMAPruntime [min]splat PSNR ↑\uparrow / FID ↓\downarrow
Scene#imgRot ∘Trans mWallParCOLMAPoursCOLMAP
house 18240.52.097377016.71/0.9316.71/0.9318.38/0.33\mathbf{18.38/0.33}
house 27410.62.195335814.95/3.2414.95/3.2417.60/0.61\mathbf{17.60/0.61}
Table 1: Self-captured houses. Pose deviation is relative to COLMAP . “Wall” is measured single-machine runtime; “Par” is the parallel time, the runtime the (independent) front end achieves when its steps run in parallel.
Figure 3: Splat quality. On house 1 the pose graph places cameras close to COLMAP ( 0.5∘0.5^{\circ} , 22 m), yet the PGO reconstruction (b) is visibly softer than COLMAP ’s (a), the seam blur left by scale drift across chunk merges ( Sec. 5.2 ) that a shared point cloud would have prevented. Row (c) shows the same on a second scene (house 2): in the zoomed roof crops the PGO splat (center) smears the shingle texture, while the COLMAP splat (right) resolves individual shingles, matching ground truth (left).
SceneRot ∘↓{}^{\circ}\downarrowTrans m ↓\downarrow
CUHK_UPPER0.84.2
HAVFailed
LFLS1.136.3
Table 2: GauU-Scene V2 [ 25 ] : a reconstructed trajectory (left) and per-scene deviation from the lidar ground truth (right; medians, means skewed by large outliers; “Failed” = no usable reconstruction).
Figure 4: Qualitative output of the pose-only pipeline on a self-captured house: (a) the camera poses it estimates and (b) the resulting Gaussian-splat reconstruction. The poses are close to COLMAP and the splat is recognizable; the failures analyzed below are quality and consistency gaps, not gross breakage.
Figure 5: Camera centers, PGO versus COLMAP , both aligned to the GPS metric frame (house 2, 741741 images, a 148148 m scene with 104104 m of vertical ascent). Left: the two sets of centers overlaid. Right: every frame that disagrees by more than 11 m is highlighted in color, the rest green. The disagreement is pervasive, 90%90\% of frames exceed 11 m (median 1.21.2 m), and worst on the vertical-ascent frames, up to 2929 m. The two regimes fail differently: the near-linear ascent trajectories suffer nadir degeneracies, while the rest of the scene inherits scaling error from noisy GPS, either way enough to soften the PGO splat relative to COLMAP .
More in this topic

Camera Pose Estimation

Camera pose estimation recovers a camera’s position and orientation from images, visual features, scene geometry, and sometimes sensor measurements.

See all 241 papers

Explore similar work

CardsList
Aug 11, 2026

GS-CPE: Unified 6-Degree-of-Freedom Camera Pose Estimation via 3D Gaussian Splatting

Despite substantial progress in visual localization, from scene coordinate regression to direct camera pose regression, achieving both robust generalization and high accuracy remain challenging. This study introduces GS-CPE (Gaussian Splatting based Camera Pose Estimation), a coarse-to-fine framework for 6-DoF camera pose estimation that unifies geometry-based coarse pose estimation with robust 3D Gaussian Splatting (3DGS) warping based pose refinement. GS-CPE first estimates a coarse pose via retrieval-guided geometric pose estimation on a 3DGS scene representation, then refines it by minimizing a visibility aware masked RGB warping objective in a multi-scale optimization framework, with adaptive re-rendering. Extensive experiments on indoor and outdoor benchmarks including 7Scenes, Cambridge Landmarks, FAST-LIVO2 datasets, and a custom dataset demonstrate state-of-the-art performance, consistently outperforming in both accuracy and generalization.

Huaiyuan Weng, Chul Min Yeum, Su-Min Kang

Department of Civil and Environmental Engineering, University of Waterloo, 200 University Ave. W., Waterloo, ON N2L 3G1, Canada · School of Architecture, Soongsil University, 369 Sangdo-ro, Dongjak District, Seoul, South Korea

Camera Pose Estimation
AskPDF
Oct 20, 2026

CrossFit-GS: Out-of-Fold Pose Estimation for COLMAP-Free 3D Gaussian Splatting

Pose-free 3D Gaussian Splatting (3DGS) aims to bypass cumbersome Structure-from-Motion (SfM) by directly reconstructing scenes from unposed image sequences. In this work, we present CrossFit-GS, a deferred-commitment progressive framework that mitigates cumulative trajectory drift and breaks the self-reinforcing loop of distorted-map error propagation. Specifically, we partition the input sequence into interleaved subsets and maintain two complementary submaps. Each candidate pose is cross-refined against a frozen opposite submap that has never been optimized using the target frame, thereby providing an independent geometric reference for pose estimation. Checkerboard validation and spatial coverage safeguards further reject unreliable updates, and only verified poses are committed to their corresponding owner submap. Once the trajectory has been refined, the intermediate submaps are discarded and the final global representation is trained from scratch using the fixed refined poses. Extensive evaluations across multiple benchmarks demonstrate that CrossFit-GS substantially reduces cumulative trajectory drift and consistently improves both camera pose estimation and novel-view synthesis over existing methods.

Zijian Wu, Jinliang Wang, Mingfeng Jiang +3

Zhejiang Sci-Tech University · Tsinghua University

3D Gaussian Splatting3D ReconstructionCamera Pose EstimationNovel View Synthesis
AskPDF
Oct 7, 2026

DeltaSplat: Iterative Gaussian Refinement for Pose-Free Feed-Forward 3D Gaussian Splatting

Pose-free feed-forward 3D Gaussian Splatting (3DGS) reconstructs a scene from sparse, unposed images in a single network pass, removing the need for camera calibration and per-scene optimization. However, camera estimation errors propagate into the predicted Gaussians and compound the geometric and photometric inaccuracies of single-pass prediction. To correct these errors, we introduce DeltaSplat, a lightweight Gaussian refinement module for pose-free feed-forward 3DGS. It iteratively renders the current Gaussians at the input context views and predicts per-Gaussian updates from the resulting residuals. A 2D residual alone, however, underdetermines the 3D correction. DeltaSplat therefore conditions each update on per-pixel Plücker rays and rendered depth as a soft geometric prior. A dual-branch convolutional mixer efficiently encodes these inputs, and per-attribute heads decode the fused features into position, opacity, and color updates. The module adds only ~2.2% parameters to the backbone and remains fully feed-forward at inference. On DL3DV, DeltaSplat reaches 26.64 dB PSNR in the pose-free setting, improving its state-of-the-art backbone by 1.75 dB and surpassing even baselines supplied with ground-truth cameras; consistent gains hold across 6-24 views and all camera regimes.

Chanung Park, Seunghyeon Song, Joo Chan Lee +2

Department of Electrical and Computer Engineering, Sungkyunkwan University · Department of Artificial Intelligence, Sungkyunkwan University · Department of Artificial Intelligence, Yonsei University

3D Gaussian Splatting3D ReconstructionFeed-Forward 3D ReconstructionSparse-View 3D Reconstruction
AskPDF