Structure-from-Motion (SfM) is a fundamental 3D vision task for recovering camera parameters and scene geometry from multi-view images. While recent deep learning advances enable accurate Monocular Depth Estimation (MDE) from single images without depending on camera motion, integrating MDE into SfM remains a challenge. Unlike conventional triangulated sparse point clouds, MDE produces dense depth maps with significantly higher error variance. Inspired by modern RANSAC estimators, we propose Marginalized Bundle Adjustment (MBA) to mitigate MDE error variance leveraging its density. With MBA, we show that MDE depth maps are sufficiently accurate to yield SoTA or competitive results in SfM and camera relocalization tasks. Through extensive evaluations, we demonstrate consistently robust performance across varying scales, ranging from few-frame setups to large multi-view systems with thousands of images. Our method highlights the significant potential of MDE in multi-view 3D vision.
Figures & tables
Figure 1: Marginalized Bundle Adjustment (MBA). Our method registers monocular depth maps into a consistent 3D coordinate system. Red camera icons indicate the viewpoints of registered depth maps. Monocular depth provides a strong structural prior, yet its predictions are inherently high-variance, as reflected by the noisy appearance of the reconstructed point cloud. This makes classical Bundle Adjustment designed for accurate sparse point cloud ill-suited. We therefore introduce a RANSAC-inspired Bundle Adjustment objective that leverages the depth maps’ density to robustly accommodate their variance. Although depth-derived point clouds have lower visual fidelity, our experiments show that monocular depth already supports SoTA or competitive performance on diverse SfM benchmarks, as exemplified ScanNet [ 14 ] , IMC2021 [ 6 ] , and ETH3D [ 6 ] . It highlights significant potential of monocular depth models for multi-view vision tasks.
Dataset
ScanNet [ 15 ]
Tanks&Temples [ 30 ]
ETH3D [ 58 ]
IMC2021 [ 6 ]
7-Scenes [ 60 ]
Wayspots [ 8 ]
Images
391
1,106
76
25
8,000
1,157
Pairs
21,982
210,894
1,571
300
282,209
666,598
Table 1: Maximum Size of Pose Graph in each benchmarked dataset. The nodes and edges are its images and co-visible image pairs.
Figure 2: System Overview. With N RGBs, the system consumes dense depth maps and pairwise correspondence inferred by pretrained models. The system outputs intrinsics, extrinsics, and frame-wise depth affine corrections scalars. A sparse N×N pose graph is built from co-visible frames using correspondences. Dense inputs are subsampled into a data matrix of ∣E∣×κ×5 (graph edges count) for scalable multi-GPU optimization. After initialization, the Bundle Adjustment proceeds from coarse to fine. In coarse stage, the BA objective is evaluated and summed over “star-shaped” subgraph Gi of each frame i . One subgraph includes itself plus its co‑visible neighbors, marked as one colored row in coarse pose graph. Fine stage computes with full graph. The BA applies gradient descent for fixed iterations.
Figure 3: CDF and PDF of empirical residual distribution R . In Eq. 8 , our BA maximizes the area-under-the-curve of R ’s CDF curve up a maximum threshold. The BA automatically formulates a smoothed categorization of inliers versus outliers. The forward and backward computation ( Eqn. 10 & 11 ) at the residual r is succinctly defined as indexing the curve at F(r) and p(r) .
Figure 4: Dense Depth and Correspondence are inputs to our multi-view pose estimation system. We benchmark pose performance with depth map under diverse imaging conditions. Figures (a) to (e) include ScanNet [ 15 ] indoor images, T&T [ 30 ] and ETH3D [ 59 ] high resolution images, IMC2021 [ 6 ] internet-collected images, and Wayspots [ 8 ] flipped image. Fig. (f) visualizes dense correspondence.
Figure 5: Camera Re-localization on 7-Scenes and Wayspots. Green and blue mark predicted and groundtruth odometry. We present challenging sequences of repetitive, textureless images. The images exhibit (1) scale changes, (2) flipping, and (3) a lack of distinguishable depth references. Consequently, the model predicts sub-optimal depth maps. Surprisingly, despite these significant challenges, depth maps still support accurate camera poses.
Scene
COLMAP [ 56 ]
ACE-Zero [ 10 ]
FlowMap [ 62 ]
VGG-SfM [ 73 ]
DF-SfM [ 28 ]
MASt3R-SfM [ 18 ]
MBA (Ours)
RRA
RTA
RRA
RTA
RRA
RTA
RRA
RTA
RRA
RTA
RRA
RTA
RRA
RTA
courtyard
56.3
60.0
4.0
1.9
7.5
3.6
50.5
51.2
80.7
74.8
89.8
64.4
94.7
94.7
delivery area
34.0
28.1
27.4
1.9
29.4
23.8
22.0
19.6
82.5
82.0
83.1
81.8
83.1
83.0
electro
53.3
48.5
16.9
7.9
2.5
1.2
79.9
58.6
82.8
81.2
100.0
95.5
95.6
78.2
facade
92.2
90.0
74.5
64.1
15.7
16.8
57.5
48.7
80.9
82.6
74.3
75.3
100.0
99.2
kicker
87.3
86.2
26.2
16.8
1.5
1.5
100.0
97.8
93.5
91.0
100.0
100.0
100.0
98.9
Table 2: Structure-from-Motion benchmark on ETH3D dataset [ 58 ] in metrics RRA ( @5∘ ) and RTA ( @5∘ ) following MASt3R-SfM evaluation protocol. See performance with other MDE models at Supp.Tab. 13 . See detailed evaluation protocol at Supp.Sec. 7.5
Type
Method
AUC @3∘
AUC @5∘
AUC @10∘
Detector-Based
SIFT+NN + COLMAP [ 56 ] CVPR’16
24.87
34.47
45.94
SIFT + NN + PixSfM [ 41 ] ICCV’21
26.45
35.73
47.24
D2Net + NN + PixSfM [ 41 ] ICCV’21
10.27
13.12
17.25
R2D2 + NN + PixSfM [ 41 ] ICCV’21
32.44
42.55
55.01
SP + SG + PixSfM [ 41 ] ICCV’21
46.30
58.43
71.62
Detector-Free
LoFTR + PixSfM [ 41 ] ICCV’21
44.06
56.16
69.61
Table 3: Structure-from-Motion on IMC2021 [ 6 ] dataset in AUC metric. We are comparable or better than Dense-SfM, DF-SfM, Mast3r-SfM, PixSfM, VGG-SfM and VGGT without BA.
Method
Inference
RRA@5 ↑
RTA@5 ↑
ATE ↓
Registration ↑
Cut3R [ 75 ] CVPR’25
FeedForward
18.8
25.8
0.017
100.0
Spann3R [ 71 ] 3DV’25
22.1
30.7
0.016
100.0
SLAM3R [ 42 ] CVPR’25
20.3
24.7
0.015
100.0
VGGT-SLAM [ 43 ] arXiv’25
57.3
67.9
0.008
100.0
Light3R-SfM [ 21 ] CVPR’25
52.0
52.8
0.011
100.0
SAIL [ 16 ] arXiv’25
70.4
74.7
0.008
100.0
Table 4: Structure-from-Motion on Tanks&Temple [ 30 ] dataset.
Method
Depth
Corres.
ACC@3∘
ACC@5∘
ACC@10∘
COLMAP [ 56 ]
-
SuperPoint [ 17 ]
0.342
0.505
0.670
MBA (Ours)
ZoeDepth [ 5 ]
RoMa [ 20 ]
0.372
0.586
0.811
DUSt3R [ 79 ]
RoMa [ 20 ]
0.403
0.615
0.820
UniDepth [ 49 ]
RoMa [ 20 ]
0.407
0.612
0.823
DUSt3R [ 79 ]
MASt3R [ 34 ]
0.384
0.596
0.811
UniDepth [ 49 ]
MASt3R [ 34 ]
0.393
0.598
0.817
Table 5: Structure-from-Motion on the ScanNet dataset [ 15 ] .
Category
Method
Average (deg/cm)
FM
AS [ 55 ] PAMI’16
5.1/2.46
HLoc [ 53 ] CVPR’19
3.4/1.07
E2E
SC-wLS [ 81 ] ECCV’22
6.6/1.45
NeuMaps [ 65 ] CVPR’23
3.1/1.09
PixLoc [ 54 ] CVPR’21
2.9/0.98
SCR
ACE [ 8 ] CVPR’23
2.8/0.93
Table 6: Camera relocalization performance benchmark on the 7-Scenes dataset [ 60 ] . Per-scene results are in Supp.Tab. 15 .
Scene
MAREPO [ 11 ]
DSAC∗ [ 9 ]
ACE [ 8 ]
MBA (Ours)
Cubes
71.8%
83.8%
97.0%
75.1%
Bears
80.7%
82.6%
80.7%
100%
Winter Sign
0.0%
0.2%
1.0%
9.3%
Inscription
37.1%
54.1%
49.0%
28.3%
The Rock
99.8%
100%
100%
100%
Tendrils
29.3%
25.1%
34.9%
51.5%
Table 7: Camera relocalization benchmark on the Wayspots [ 8 ] .
MegaDepth
ScanNet
Method ↓ AUC @→
5∘↑
10∘↑
20∘↑
5∘↑
10∘↑
20∘↑
LoFTR [ 63 ] CVPR’21
52.8
69.2
81.2
22.1
40.8
57.6
PDC-Net+ [ 68 ] TPAMI’23
51.5
67.2
78.5
20.3
39.4
57.1
DKM [ 19 ] CVPR’23
60.4
74.9
85.1
29.4
50.7
68.3
PMatch [ 86 ] CVPR’23
61.4
75.7
85.7
29.4
50.1
67.4
RoMa [ 20 ] CVPR’24
62.6
76.7
86.3
31.8
53.4
70.9
Table 8: RoMa benchmark of two-view pose estimation with RANSAC on MegaDepth-1500 [ 38 ] and ScanNet-1500 [ 14 ] .
Dataset
Max Graph Size
Max Runtime
Images
Pairs
Ours
COLMAP
ScanNet [ 15 ]
391
43,964
2.98 h
0.91 h
ETH3D [ 58 ]
76
3,142
41 min
9 min
IMC2021 [ 6 ]
25
600
14 min
3 min
7-Scenes [ 60 ]
8,000
564,418
3.12 h
13.52 h
Wayspots [ 8 ]
1,157
1,333,196
4.72 h
Crash
Table 9: Max Pose Graph and its Runtime in each benchmarked dataset. We run experiments on ScanNet, ETH3D, and IMC2021 with single V100 GPU, and on 7-Scenes and Wayspots with eight.
Ablation
Method
RRA @5∘
RTA @5∘
Stages
Initialization
62.6
41.0
Coarse
97.1
87.9
Fine
97.3
90.2
Density
κ=30
91.7
78.6
κ=200
97.3
90.2
κ=500
99.3
92.0
Table 10: Ablations on ETH3D dataset.
Type
Method
ETH3D Dataset
AUC @1∘
AUC @3∘
AUC @5∘
Detector-Based
SIFT+NN + COLMAP [ 56 ] CVPR’16
26.71
38.86
42.14
SIFT + NN + PixSfM [ 41 ] ICCV’21
26.94
39.01
42.19
D2Net + NN + PixSfM [ 41 ] ICCV’21
34.50
49.77
53.58
R2D2 + NN + PixSfM [ 41 ] ICCV’21
43.58
62.09
66.89
SP + SG + PixSfM [ 41 ] ICCV’21
50.82
68.52
72.86
Table 11: Structure-from-Motion on ETH3D [ 59 , 58 ] dataset in metric AUC at multiple thresholds following DF-SfM.
ATE ↓
RTA@3 ↑
AUC-R&T @ 3 ↑
RTA@1 ↑
AUC-R&T @ 1 ↑
n_imgs
MBA (Ours)
FastMap [ 36 ]
GLOMAP [ 47 ]
COLMAP [ 56 ]
MBA (Ours)
FastMap [ 36 ]
GLOMAP [ 47 ]
COLMAP [ 56 ]
MBA (Ours)
FastMap [ 36 ]
GLOMAP [ 47 ]
COLMAP [ 56 ]
MBA (Ours)
FastMap [ 36 ]
GLOMAP [ 47 ]
COLMAP [ 56 ]
MBA (Ours)
FastMap [ 36 ]
GLOMAP [ 47 ]
COLMAP [ 56 ]
mipnerf360 (9)
215.6
5.0e-4
4.2e-4
3.3e-5
5.8e-5
99.6
99.9
100.0
100.0
85.5
97.4
98.2
97.2
87.3
99.8
100.0
99.7
66.1
92.4
94.6
91.9
tnt_advanced (6)
337.8
1.7e-2
6.4e-3
1.2e-2
1.2e-3
59.9
71.4
79.1
98.5
34.7
42.6
75.3
94.8
25.2
42.3
77.5
97.0
12.5
16.7
69.8
90.0
tnt_intermediate (8)
268.6
5.7e-3
7.8e-5
1.9e-5
2.6e-4
89.8
99.9
100.0
99.8
61.7
94.1
99.0
98.9
62.8
99.3
99.9
99.5
35.4
83.1
96.9
97.3
tnt_training (7)
470.1
3.8e-3
3.0e-3
1.1e-2
3.0e-4
88.8
87.8
88.7
99.9
63.2
77.2
87.9
99.5
63.6
82.1
88.6
99.9
31.9
60.5
86.3
98.7
nerf_osr (8)
402.8
1.4e-3
1.6e-3
1.1e-3
1.3e-3
89.8
91.7
92.0
92.1
69.3
70.9
71.9
71.7
54.4
71.1
71.9
71.7
35.0
43.2
45.2
44.7
Table 12: Pose accuracy comparison on MipNeRF360 [ 4 ] , Tanks and Temples [ 30 ] , NeRF-OSR [ 52 ] , DroneDeploy [ 50 ] , Urbanscene3D [ 39 ] , Mill-19 [ 70 ] , and Eyeful Tower [ 82 ] . We follow the evaluation protocol established by FastMap [ 36 ] for reporting pose accuracy metrics on these datasets. Note, [ 36 ] uses a different COLMAP groundtruth to Tab. 4 . For datasets with multiple scenes, we denote the average number of images as dataset-name(#scenes) . Results are listed separately for each scene in Mill-19 and Eyeful Tower. Metrics are color-coded with dark green for best performance and light green for competitive performance. Red denotes complete failures and gray indicates timeout (did not finish in one week).
Scene
COLMAP [ 56 ]
ACE-Zero [ 10 ]
FlowMap [ 62 ]
VGGSfM [ 73 ]
DF-SfM [ 28 ]
MASt3R-SfM [ 18 ]
Ours / DUSt3R [ 79 ]
Ours / ZoeDepth [ 5 ]
Ours / UniDepth [ 49 ]
RRA
RTA
RRA
RTA
RRA
RTA
RRA
RTA
RRA
RTA
RRA
RTA
RRA
RTA
RRA
RTA
RRA
RTA
courtyard
56.3
60.0
4.0
1.9
7.5
3.6
50.5
51.2
80.7
74.8
89.8
64.4
94.7
94.7
94.7
94.5
94.7
94.4
delivery area
34.0
28.1
27.4
1.9
29.4
23.8
22.0
19.6
82.5
82.0
83.1
81.8
83.1
83.0
87.8
82.0
83.1
83.1
electro
53.3
48.5
16.9
7.9
2.5
1.2
79.9
58.6
82.8
81.2
100.0
95.5
95.6
78.2
91.9
78.5
93.0
77.2
facade
92.2
90.0
74.5
64.1
15.7
16.8
57.5
48.7
80.9
82.6
74.3
75.3
100.0
99.2
100.0
97.4
80.9
86.0
kicker
87.3
86.2
26.2
16.8
1.5
1.5
100.0
97.8
93.5
91.0
100.0
100.0
100.0
98.9
100.0
98.5
100.0
98.0
Table 13: Structure-from-Motion ablation with other Monocular Depth Models ZoeDepth and UniDepth on ETH3D dataset [ 59 , 58 ] .
Method
Depth
Corres.
Calibrated
Uncalibrated
Acc@3∘
Acc@5∘
Acc@10∘
Acc@3∘
Acc@5∘
Acc@10∘
COLMAP [ 56 ]
-
SuperPoint [ 17 ]
0.398
0.589
0.783
0.342
0.505
0.670
MBA (Ours)
ZoeDepth [ 5 ]
RoMa [ 20 ]
0.396
0.614
0.823
0.372
0.586
0.811
DUSt3R [ 79 ]
RoMa [ 20 ]
0.426
0.631
0.830
0.403
0.615
0.820
UniDepth [ 49 ]
RoMa [ 20 ]
0.432
0.636
0.833
0.407
0.612
0.823
DUSt3R [ 79 ]
MASt3R [ 34 ]
0.432
0.639
0.837
0.384
0.596
0.811
Table 14: Extended Structure-from-Motion Results on the ScanNet dataset [ 15 ] , with calibrated and uncalibrated cases.
Category
Method
Chess
Fire
Heads
Office
Pumpkin
Kitchen
Stairs
Average
FM
AS [ 55 ] PAMI’16
4/1.96
3/1.53
2/1.45
9/3.61
8/3.10
7/3.37
3/2.22
5.1/2.46
HLoc [ 53 ] CVPR’19
2/0.79
2/0.87
2/0.92
3/0.91
5/1.12
4/1.25
6/1.62
3.4/1.07
E2E
SC-wLS [ 81 ] ECCV’22
3/0.76
5/1.09
3/1.92
6/0.86
8/1.27
9/1.43
12/2.80
6.6/1.45
NeuMaps [ 65 ] CVPR’23
2/0.81
3/1.11
2/1.17
3/0.98
4/1.11
4/1.33
4/1.12
3.1/1.09
PixLoc [ 54 ] CVPR’21
2/0.80
2/0.73
1/0.82
3/0.82
4/1.21
3/1.20
5/1.30
2.9/0.98
SCR
ACE [ 8 ] CVPR’23
1.9/0.7
1.9/0.9
0.9/0.6
2.7/0.8
4.2/1.1
4.2/1.3
3.9/1.1
2.8/0.93
Table 15: Extended Camera Relocalization Results on the 7-Scenes dataset [ 60 ] , with per-scene performance.
Department of Computer Science and Artificial Intelligence, Dongguk University, Seoul 04620, Republic of Korea · 42dot, Seongnam-si, Gyeonggi-do 13449, Republic of Korea