Classical Bundle Adjustment (BA) relies on precise metric initialization and known camera intrinsics, and traditional Structure-from-Motion (SfM) pipelines struggle to exploit modern dense matchers because rigid track-building heuristics break down under their inherent noise. We present Track-Free Dense Structure from Motion (TFD-SfM), which jointly optimizes camera extrinsics, focal lengths, and geometry from a strict cold start. Our central idea is to relax the rigidities of classical SfM into soft, probabilistic decisions under Negative Log-Likelihood (NLL) objectives. Instead of forcing matched pixels onto a single shared 3D point, we model each match as a 3D Gaussian whose variance is optimized jointly with the geometry. Unreliable matches take on large variance and contribute little to the loss. This smooths the optimization landscape, so the solver converges even from identity cameras and uniform depths. A kinematic relative-pose parameterization stabilizes the cold start, and a dual-hypothesis mechanism resolves the mirror-symmetry ambiguity on object-centric scenes. Across DTU, Mip-NeRF360, ETH3D, and IMC2021, TFD-SfM is competitive with both classical and learning-based SfM baselines and leads on ETH3D, recovering accurate camera poses, intrinsics, and dense geometry from unordered, uncalibrated image collections. Projectpage: https://cphjsn.github.io/tfd-sfm/
Figures & tables
Figure 1: Comparison against COLMAP variants. TFD-SfM produces robust, dense 3D reconstructions across diverse scenarios: object-centric (blue), wide-baseline indoor (yellow), complex structures (green), and in-the-wild (orange). Gray insets report Relative Translation Accuracy (RTA@5 ∘ ) against standard and dense COLMAP baselines. TFD-SfM attains competitive metric accuracy with markedly denser structure.
Figure 2: System Overview. From unordered images, three stages: (1) View Graph Construction via LoFTR [ 37 ] (solid: spanning-tree edges, dash: auxiliary edges); (2) Finding Correspondences via DKM [ 9 ] ; and (3) TFD-SfM Optimization from a strict cold start. The temporal sequence ( t=0→200 s) shows the prior-free emergence of the outputs : optimized intrinsics, absolute poses, and a 3D reconstruction.
Figure 3: Cold-start convergence: toy ablation. Pose parametrization (independent abs-poses vs. chained relative-pose) × landmark model (one shared 3D point, i.e. classical BA, vs. per-observation 3D Gaussians); Left: reconstructions on the full 360∘ ring ( 24 cameras, σ=4 px). Right: mean pairwise relative-rotation error vs. arc span ( +15∘ per added camera), over noise σ∈{0,1,4,8} px and 5 seeds. Only chained poses and Gaussian landmarks (green) recover the ring; all other cells collapse.
Figure 4: The mirror-symmetry ambiguity. A scene and its reflection (concave surfaces appear convex) render near-identically, so the optimizer can fall into the reflected basin; TFD-SfM optimizes both hypotheses and keeps the lower-NLL one.
Figure 5
Figure 6: Qualitative Reconstructions. Estimated (red) and ground-truth (grey) camera poses and point clouds across the tested datasets (DTU, Mip-NeRF 360, ETH3D, and IMC2021). Left to right: Input, COLMAP, COLMAP+DKM, COLMAP+RoMa, MASt3R-SfM, VGGT, and TFD-SfM. While classical methods yield shattered geometries when forced to use dense matches, TFD-SfM consistently recovers coherent topology and accurate trajectories on par with foundation models.
Figure 7: Runtime analysis. (Left) Time per optimization step vs. correspondence count M (log–log axes). The dotted line marks linear scaling in M , and the dashed line marks the out-of-memory limit of a 95 GB GPU. (Right) End-to-end runtime vs. number of input views N on Mip-NeRF 360 for TFD-SfM (blue), COLMAP (red), COLMAP+DKM (green), MASt3R-SfM (orange), and VGGT (purple). Transparent lines show single scenes and solid lines the mean.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Spanning-tree robustness. RRA/RTA@5 ∘ versus the number of tree nodes whose parent is randomly reassigned (mean of 5 seeds).
Setting
Value
Optimizer
AdamW ( ϵ=10−15 , weight decay 0 )
Iterations
30,000
Learning rate (all groups)
0.0016
Betas (poses, depth, variance)
(0.9,0.999)
Betas (focal)
(0.5,0.999)
Depth & variance maps
32×32 per view, bilinear
Appendix
Table 4: Optimization hyperparameters and schedules.
Figure 10: Joint evolution of camera poses and 3D Gaussians. Initialized from a cold start, the camera poses, intrinsics, and 3D Gaussians are jointly optimized. Shown from left to right are representative scenes from the four evaluated datasets: DTU, Mip-NeRF 360, ETH3D, and IMC2021. Note: Due to rendering constraints, visualizations display a randomly subsampled set of 10,000 Gaussians per scene, depicting isotropic Gaussians as polygonal spheres rather than perfectly smooth spheres.
Figure 11: Evolution of the two hypothesis worlds and their corresponding loss Lworld . The mirrored world_1 (blue) is rejected into a higher-loss minimum; the true world_2 (red) descends to convergence. Solid lines: EMA over the raw stochastic data (semi-transparent).
Figure 12: Robustness to Controlled Noise. Relative Translation Accuracy (RTA@ 10∘ , left ) and Relative Rotation Accuracy (RRA@ 10∘ , right ) evaluated across varying noise magnitudes ( σ , x-axis) and ratios ( r , colored lines) on the ETH3D kicker scene. The method maintains high accuracy under both widespread mild noise and sparse gross outliers, degrading only when both parameters are extreme.
Figure 13: Evolution of Isotropic vs. Anisotropic 3D Gaussians during joint optimization. Top: isotropic Gaussians converge into coherent structure. Bottom: anisotropic Gaussians overfit early poses, stretching into degenerate shapes and failing.
Figure 14: Convergence Analysis on ETH3D. Mean Relative Rotation Accuracy (RRA@5 ∘ ) and Relative Translation Accuracy (RTA@5 ∘ ) across 30,000 iterations. Grey lines represent individual scenes in ETH3D dataset, while the blue line denotes the mean of them.
Figure 15: Visualization of Focal-Depth Ambiguity. Left: Input images. Right: Estimated poses (red), ground truth poses (grey), and the reconstructed scene.
Global Structure-from-Motion (SfM) is an efficient paradigm for recovering camera poses and sparse 3D structure from unordered images. However, its reliance on scale-ambiguous epipolar geometry makes global positioning sensitive to noisy baseline estimates and weak view-graph constraints, while false edges from visually ambiguous pairs can further degrade reconstruction. We propose DGSfM, a depth-aware global SfM pipeline that uses monocular depth maps as a scalable prior while preserving explicit multi-view optimization. For each image pair, we use a depth-aware relative pose solver to convert scale-ambiguous epipolar constraints into scale-aware relative pose constraints. We further improve robustness through view-graph filtering and depth-consistency-based correspondence pruning, which suppress false edges and matches that remain plausible under epipolar geometry alone. Finally, global scale averaging and depth-guided pose-point initialization align monocular depth maps into a common reconstruction scale and provide stable initialization for global positioning and bundle adjustment. Experiments on ETH3D and IMC2021 show that DGSfM consistently improves over strong global SfM baselines across sparse and dense matching front-ends, achieving substantial gains in pose accuracy. Code and project page: https://dgsfm.github.io/.
Sithu Aung, Viktor Kocur, Yaqing Ding +2
VRG, FEE, Czech Technical University in Prague · FMPH, Comenius University in Bratislava · Southeast University +1
We present DeepBA (DeepBundle Adjustment), a graph neural network (GNN) that recovers camera poses, intrinsics, and scene structure from 2D matches across an arbitrary number of views. Rather than solving a non-linear least-squares problem per scene, DeepBA amortizes bundle adjustment into a learned graph network that iteratively refines its estimate from geometric residuals. Where classical SfM computes its updates analytically and needs careful initialization, DeepBA learns them and converges from a cold start. It is trained with scale-invariant supervision (ground-truth rotation, translation direction, and intrinsics, plus reprojection self-consistency), so datasets with arbitrary metric units can be mixed without per-scene normalization. DeepBA is decoupled from the matcher. At 2.04M parameters, it outperforms prior learned SfM from matches on scene reconstruction and on most pose metrics, is competitive with classical SfM at moderate accuracy thresholds and overtakes it in wall time as the view count grows, and approaches monolithic image-to-geometry transformers such as VGGT (∼1.2B parameters) with greater modularity. Project~page: https://cphjsn.github.io/deep-ba/
Pak Hong Chui, Hector Andrade-Loarca, Daniel Cremers
Technical University of Munich, Munich Center for Machine Learning (MCML)
Camera pose estimation is a key step in 3D reconstruction and view-synthesis pipelines. We present a deep, global Structure-from-Motion framework based on learned view-graph aggregation. Our method employs a permutation-equivariant, edge-conditioned graph neural network that takes noisy pairwise relative poses as input and outputs globally consistent camera extrinsics. The network is trained without ground-truth supervision, relying solely on a relative-pose consistency objective. This is followed by 3D point triangulation and robust bundle adjustment. Our approach is efficient, scalable to more than a thousand images, and robust to graph density. We evaluate our method on MegaDepth, 1DSfM, Strecha, and BlendedMVS. These experiments demonstrate that our method achieves superior rotation and translation accuracy compared to deep track-centric methods while registering more images across many scenes, and competitive results compared to state-of-the-art classical pipelines, while being much faster.