With recent advances, Feed-forward Reconstruction Models (FFRMs) have demonstrated great potential in reconstruction quality and adaptiveness to multiple downstream tasks. However, the excessive reliance on multi-view geometric annotations, e.g. 3D point maps and camera poses, makes the fully-supervised training scheme of FFRMs difficult to scale up. In this paper, we propose Reliev3R, a weakly-supervised paradigm for training FFRMs from scratch without cost-prohibitive multi-view geometric annotations. Relieving the reliance on geometric sensory data and compute-exhaustive structure-from-motion preprocessing, our method draws 3D knowledge directly from monocular relative depths and image sparse correspondences given by zero-shot predictions of pretrained models. At the core of Reliev3R, we design an ambiguity-aware relative depth loss and a trigonometry-based reprojection loss to facilitate supervision for multi-view geometric consistency. Training from scratch with the less data, Reliev3R catches up with its fully-supervised sibling models, taking a step towards low-cost 3D reconstruction supervisions and scalable FFRMs.
Figures & tables
Figure 1: In this paper, we propose Reliev3R, the first learning paradigm to train a F eed- f orward R econstruction M odel (FFRM) from scratch without reliance on multi-view geometric annotations. To learn multi-view geometric knowledge, Reliev3R engages pseudo monocular relative depths and pseudo image sparse correspondences with multi-view geometry constraints as a weakly-supervised learning objective. As shown above in the figure, Reliev3R surpasses early FFRMs ( e.g . , FLARE [ 43 ] ) and weakly-supervised camera pose estimation models ( e.g . , AnyCam [ 33 ] ) in the overall performance.
Figure 2: Similar to prior FFRMs, Reliev3R performs geometry reconstruction in a single forward pass given a group of images. However, the supervision of Reliev3R doesn’t depend on any multl-view geometric annotations ( e.g . , ground truth of point maps and camera poses which are registered in 3D world coordinate). Specifically, instead of directly predicting the point maps registered in world coordinate, Reliev3R predicts view-wise depth maps, which are regularized with pseudo relative depth. To learn the registration of depth maps and camera poses in world coordinate, Reliev3R draws supervision signal from pseudo image correspondences. Pseudo annotations used by Reliev3R are produced with pretrained expert models (see Sec. 4.1 ). Experiments in Sec. 4 demonstrate that Reliev3R catches up with and even surpasses some of FFRMs supervised with multi-view geometric annotations.
Figure 3: Point map and camera pose visualization on DL3DV-benchmark dataset [ 16 ] . As shown above in the figure, Reliev3R delivers a visually comparable construction accuracy with the π3 [ 32 ] while surpassing FLARE [ 43 ] in the overall performance. The camera pose of FLARE above is adopted directly from the model prediction instead of solved with PnP [ 6 ] based on point maps.
Method
Training with DL3DV-10K
MVG Annotations
Point Map
Camera Pose
Depth Map
rel ↓
τ↑
ATE ↓
AUC ↑
rel ↓
τ↑
Map-Anything [ 13 ]
Yes
w
0.051
0.911
0.004
87.088
0.045
0.896
π3 [ 32 ]
No
w
0.056
0.927
0.003
92.746
0.054
0.916
VGGT [ 28 ]
Yes
w
0.061
0.905
0.003
94.795
0.059
0.899
CUT3R [ 29 ]
Yes
w
0.073
0.815
0.005
89.432
0.072
0.801
π3†
Alone
w
0.057
0.897
0.013
63.314
0.047
0.890
Table 1: Evaluation on DL3DV-benchmark dataset [ 16 ] for 8-view reconstruction. Reliev3R outperforms early FFRMs trained with MVS annotations like MVDUSt3R [ 24 ] and FLARE [ 43 ] , demonstrating the advantage and potential of our method. We report the absolute relative error (rel) and inlier ratio at a relative threshold of 10% ( τ ) to evaluate the reconstructed point maps and depth maps. We also report the average aligned trajectory error (ATE) and area under curve at an error threshold of 30 ° to evaluate camera pose estimation. π3† denotes the result of training π3 [ 32 ] from scratch on DL3DV-10K [ 16 ] dataset. We display if based methods are trained with/without multi-view geometric (MVG) annotations, and classify them by whether DL3DV-10K dataset is used for training into ‘Yes’, ‘Alone’ and ‘No’. SOTAs trained with 20 times of data for Reliev3R are colored as gray . Best , second best and third best are colored respectively.
Method
Point Map
Camera Pose
Depth Map
rel ↓
τ↑
ATE ↓
AUC ↑
rel ↓
τ↑
π3 [ 32 ]
0.027
0.972
0.002
86.941
0.027
0.970
π3†
0.232
0.678
0.028
17.789
0.220
0.171
AnyCam [ 33 ]
0.438
0.448
0.084
15.376
0.286
0.095
Reliev3R(ours)
0.172
0.594
0.030
15.711
0.124
0.583
Table 2: Zero-shot evaluation on ScanNet++ [ 39 ] dataset, which has a different focal length with DL3DV-10K dataset [ 16 ] that unseen during the training of Reliev3R and π3† . The same metrics are reported as in Tab. 1 . While consistently outperforming AnyCam [ 33 ] , Reliev3R matches against π3† in overall performance and holds significant advantage in depth prediction.
Figure 4: Visualization of point maps for zero-shot evaluation on ScanNet++ dataset [ 4 ] , which has a different focal length from DL3DV-10K dataest [ 16 ] . Reliev3R holds its advantage over AnyCam [ 33 ] even in zero-shot circumstance. Furthermore, Reliev3R presents a notable zero-shot performance on par with fully-supervised π3† training on the same datasets.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Method
Point Map
Camera Pose
Depth Map
rel ↓
τ↑
ATE ↓
AUC ↑
rel ↓
τ↑
RayZer
×
×
0.786
0.362
×
×
Synth Fully-Sup.
0.277
0.475
0.042
17.075
0.208
0.357
+Reliev3R
0.137
0.667
0.033
34.240
0.106
0.679
Appendix
Table 3: We train π3 from-scratch (random initial weights) in a fully-supervised manner on a down-sampled mixture of synthetic datasets, including ASE dataset, TartanAir dataset, MVS-Synth dataset, Blended-MVS dataset and etc . following VGGT. The training runs for 20k iterations with a total batch size of 128. The result is denoted as ‘Synth Fully-Sup’. in the table. Then we subsequently finetune this checkpoint on DL3DV dataset for 20k steps using the weak supervision proposed by Reliev3R. The result is denoted as ‘+Reliev3R’ in the table. We report the same metrics as Table 1 in the paper, which are evaluated on DL3DV-benchmark dataset. We also evaluate camera pose estimation for RayZer on this benchmark with its recently open-sourced checkpoint.
Figure 5: Visualization of input (multi-view images), output (multi-view depth maps and confidence maps) and pseudo labels (multi-view correspondences, monocular depth maps) of Reliev3R.
Figure 6: Visualization of input (multi-view images), output (multi-view depth maps and confidence maps) and pseudo labels (multi-view correspondences, monocular depth maps) of Reliev3R.
Figure 7: Visualization of input (multi-view images), output (multi-view depth maps and confidence maps) and pseudo labels (multi-view correspondences, monocular depth maps) of Reliev3R.
Figure 8: Visualization of input (multi-view images), output (multi-view depth maps and confidence maps) and pseudo labels (multi-view correspondences, monocular depth maps) of Reliev3R.
Point Map
Camera Pose
Depth Map
rel ↓
τ↑
ATE ↓
AUC ↑
rel ↓
τ↑
α=0.2
0.143
0.607
0.022
42.116
0.137
0.606
α=0.5
0.137
0.626
0.023
46.703
0.129
0.628
α=1.0 (Reliev3R)
0.122
0.663
0.018
49.426
0.115
0.657
α=2.0
0.127
0.651
0.020
48.566
0.121
0.647
Appendix
Table 4: Ablation study on different confidence weight α on DL3DV-benchmark [ 16 ] dataset. α=1.0 is adopted in the paper to train Reliev3R. Absolute relative error (rel) and inlier ratio at a relative threshold of 10% ( τ ) are reported to evaluate point maps and depth maps. Average aligned trajectory error (ATE) and area under curve at an error threshold of 30° (AUC) are reported to evluate camera pose estimation.
Recent advances in vision foundation models have revolutionized geometry reconstruction and semantic understanding. Yet, most of the existing approaches treat these capabilities in isolation, leading to redundant pipelines and compounded errors. This paper introduces FF3R, a fully annotation-free feed-forward framework that unifies geometric and semantic reasoning from unconstrained multi-view image sequences. Unlike previous methods, FF3R does not require camera poses, depth maps, or semantic labels, relying solely on rendering supervision for RGB and feature maps, establishing a scalable paradigm for unified 3D reasoning. In addition, we address two critical challenges in feedforward feature reconstruction pipelines, namely global semantic inconsistency and local structural inconsistency, through two key innovations: (i) a Token-wise Fusion Module that enriches geometry tokens with semantic context via cross-attention, and (ii) a Semantic-Geometry Mutual Boosting mechanism combining geometry-guided feature warping for global consistency with semantic-aware voxelization for local coherence. Extensive experiments on ScanNet and DL3DV-10K demonstrate FF3R's superior performance in novel-view synthesis, open-vocabulary semantic segmentation, and depth estimation, with strong generalization to in-the-wild scenarios, paving the way for embodied intelligence systems that demand both spatial and semantic understanding.
Chaoyi Zhou, Run Wang, Feng Luo +4
Microsoft · Clemson University · Texas A&M University
Feed-forward 3D reconstruction models are efficient but rigid: once trained, they perform inference in a zero-shot manner and cannot adapt to the test scene. As a result, visually plausible reconstructions often contain errors, particularly under occlusions, specularities, and ambiguous cues. To address this, we introduce Free Geometry, a framework that enables feed-forward 3D reconstruction models to self-evolve at test time without any 3D ground truth. Our key insight is that, when the model receives more views, it produces more reliable and view-consistent reconstructions. Leveraging this property, given a testing sequence, we mask a subset of frames to construct a self-supervised task. Free Geometry enforces cross-view feature consistency between representations from full and partial observations, while maintaining the pairwise relations implied by the held-out frames. This self-supervision allows for fast recalibration via lightweight LoRA updates, taking less than 2 minutes per dataset on a single GPU. Our approach consistently improves state-of-the-art foundation models, including Depth Anything 3 and VGGT, across 4 benchmark datasets, yielding an average improvement of 3.73% in camera pose accuracy and 2.88% in point map prediction. Code is available at https://github.com/hiteacherIamhumble/Free-Geometry .
Yuhang Dai, Xingyi Yang
The Hong Kong Polytechnic University, Hong Kong SAR
Recent feed-forward geometry foundation models have demonstrated impressive generalization by recovering depth and poses in a single forward pass. However, these models are typically constrained by a global coordinate frame assumption. This dependency becomes a significant bottleneck for long-context and streaming reconstruction, as it forces the network to maintain an arbitrary temporal origin and handle translation magnitudes that grow unbounded over time. Our solution, which we call R3, employs relative regression. We employ a lightweight MLP to predict confidence-weighted relative constraints. These confidences serve as a unified anchor: weighting losses during training and guiding pose aggregation during inference. R3 supports both full-context offline reconstruction and causal, bounded-memory streaming. Our evaluation in both offline and streaming settings validates the effectiveness of our relative mechanism. Project page: https://kevinxu02.github.io/r3-site
Congrong Xu, Huachen Gao, Xingyu Chen +3
University of Michigan · Westlake University · NVIDIA