Feed-forward 3D reconstruction models predict dense point maps from multiple images in a single pass. Without explicit geometric constraints, however, their predictions are often inconsistent across views and lack fine-grained precision. Recent methods incorporate geometric priors such as camera pose or depth, but use them merely as auxiliary inputs, process each view independently, or refine the geometry only where the priors are available. We present Refin3R, which refines feed-forward point maps with Structure-from-Motion (SfM) points, propagating SfM geometry over the image grid. In its dual-branch architecture, a 2D branch extends SfM geometry beyond its coverage using geometry-aware backbone features, while a 3D branch reasons over the SfM points of all views, capturing multi-view structure that the image grid alone cannot represent. The two branches exchange information bidirectionally to jointly refine each view, yielding dense and accurate point maps. Across ScanNet++, ETH3D, Tanks&Temples, and out-of-domain human and surgical scenes, Refin3R consistently outperforms feed-forward and geometry-guided methods in point map accuracy while running faster than the strongest geometry-guided baseline.
Figures & tables
Figure 2: Overview of Refin3R. A feed-forward model provides dense depth Dff and geometry-aware features F , and SfM provides partial but accurate depth Dsfm (Sec. 3.2 ). For each view, a 2D U-Net refines the dense depth on the image grid, while a sparse 3D U-Net refines the SfM points of all views that project into its image, with both branches exchanging information (Secs. 3.3 and 3.4 ). A spatial propagation network (SPN) fuses their estimates into the refined depth (Sec. 3.5 ), which is back-projected into the final point map.
Figure 3: Bidirectional 2D–3D interaction. The 2D feature map of each view and the voxel features of its grid exchange information along pixel columns. Top : In 2D → 3D injection, the image feature of each pixel is broadcast to all occupied voxels in its column, providing the 3D branch with image context. Bottom : In 3D → 2D gathering, the voxel features in each column are average-pooled along the depth axis and fused into the image feature map, providing multi-view evidence back to the 2D branch. Only voxels containing points of the current view (blue) are pooled, while those formed solely by points of other views (red) are excluded.
ScanNet++ [ 61 ]
ETH3D [ 42 ]
T&T [ 21 ]
4 imgs
8 imgs
16 imgs
4 imgs
8 imgs
16 imgs
4 imgs
8 imgs
16 imgs
Feed-forward
Depth Anything 3 [ 27 ]
1.6/8.8
0.9/7.1
0.6/4.7
8.9/31.9
8.0/29.6
7.0/27.1
1.1/8.7
0.7/7.1
0.5/6.3
Pi3X (RGB only) [ 55 ]
14.7/50.4
15.3/53.9
14.3/54.8
9.8/34.8
6.1/28.8
4.8/26.4
5.1/25.2
3.9/23.5
3.4/22.7
VGGT- Ω [ 50 ]
2.1/16.3
1.3/13.5
1.0/11.9
4.1/25.6
3.1/22.2
2.2/17.2
2.0/16.0
1.5/14.6
0.9/9.9
Geometry-conditioned Feed-forward
Table 1: Multi-view 3D reconstruction on standard test sets. We report AUC@1/5 cm (% ↑ ). ScanNet++ is within-domain , and ETH3D and Tanks&Temples are cross-domain . The best and second-best results for each metric within each dataset are shown in bold and underline , respectively.
4D-DRESS [ 53 ]
MV-dVRK [ 4 ]
@1 cm
@5 cm
@1 mm
@5 mm
Feed-forward
Depth Anything 3
2.1
6.3
17.2
50.7
Pi3X (RGB only)
3.6
11.8
3.9
21.1
VGGT- Ω
42.0
81.6
17.7
54.0
Geometry-conditioned Feed-forward
Table 2: Out-of-domain evaluation. AUC (% ↑ ) on 4D-DRESS and MV-dVRK, with thresholds in centimeters and millimeters, respectively. The best and second-best results for each metric within each dataset are shown in bold and underline , respectively.
Figure 4: Qualitative comparison. Each row shows one example from ScanNet++, ETH3D, Tanks&Temples, 4D-DRESS, and MV-dVRK, with only one input view visualized. We compare a feed-forward model using only images (DA3), monocular depth completion (Any2Full), a geometry-conditioned model (Pi3X), and feed-forward methods with multi-view geometry (BLASt3R, GGPT), where Any2Full, Pi3X, and GGPT use the same SfM prior as ours. AUC@10 (cm; mm for MV-dVRK) is shown in each reconstruction.
ScanNet++
ETH3D
@1 / @5 / @10 ↑
@1 / @5 / @10 ↑
Feed-forward (DA3)
0 1.0 / 0 6.9 / 15.1
0 8.0 / 29.5 / 43.6
GGPT (w/ DA3)
27.0 / 55.5 / 67.5
29.5 / 59.8 / 71.0
(a)
2D branch, geometry only
34.3 / 64.7 / 76.6
42.8 / 67.4 / 76.3
(b)
+ image features
35.4 / 66.3 / 77.9
44.7 / 68.8 / 77.2
(c)
+ 3D branch & interaction
36.7 / 67.2 / 78.4
46.2 / 69.7 / 77.9
Table 3: Ablation study. AUC (%) at 1, 5, and 10 cm on ScanNet++ and ETH3D, averaged over the 4-, 8-, and 16-view splits. Rows (a)–(c) add one component at a time, and the last two rows compare how the depth of the 3D branch reaches the output on top of (c). † Applied at test time only, without retraining.
Method
Time per scene (s) ↓
Mem. (GB) ↓
4 views
8 views
16 views
16 views
Any2Full
0.78
1.58
3.17
5.0
MapAnything
0.70
1.37
2.76
12.3
Pi3X
0.46
0.88
1.88
10.9
GGPT
4.36
7.20
12.92
6.2
Ours
0.17
0.39
0.86
5.1
Table 4: Efficiency. Inference time and peak GPU memory on ETH3D, measured on an NVIDIA RTX A6000. The last row includes the DA3 encoder.