Organizations: Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology, Shenzhen Graduate School, Peking University · The Hong Kong University of Science and Technology · Pengcheng Laboratory
Feed-forward 3D reconstruction models have achieved remarkable progress through large-scale training and strong learned priors. However, they remain fragile when applied to dynamic in-the-wild videos, where non-rigid motion, occlusions, and illumination changes are prevalent. A key limitation lies in their reliance on expensive depth and camera pose annotations, which hinders scaling to diverse real-world dynamic data and restricts generalization. To address this issue, we propose PURE, an unlabeled post-training refinement framework that improves depth and camera estimation without requiring ground-truth supervision. Instead of relying on annotations, PURE elegantly distills robust structural knowledge from state-of-the-art 2D priors into 4D feed-forward architectures. PURE leverages relative depth constraints, affine-invariant geometry distillation, and pose refinement based on feature correspondences to stabilize geometric predictions under challenging dynamic conditions. Extensive experiments on both synthetic and real-world dynamic benchmarks demonstrate that PURE delivers consistent improvements across multiple state-of-the-art architectures. Furthermore, our data-efficient paradigm effectively mitigates domain shifts, successfully unlocking the architectural limits of these base models in challenging, in-the-wild dynamic scenes.
Figures & tables
Figure 1. Our annotation-free post-training significantly improves feed-forward 3D reconstruction in dynamic scenes, yielding more accurate and temporally consistent geometry compared to the original pre-trained models. Side-by-side reconstructions before and after unlabeled post-training.
Figure 2. MoGe provides accurate frame-wise relative geometry, while feed-forward 3D models ensure temporal scale consistency. We unify both by transferring monocular geometric cues while preserving temporal consistency. Frame-wise geometry from a monocular teacher compared with a feed-forward model.
Figure 3. Overview of PURE , our annotation-free post-training framework. Given an unlabeled image sequence, a pre-trained feed-forward model predicts depth and camera poses. We refine the predictions via two complementary branches: (1) local geometry refinement using relative depth and affine-invariant constraints with intrinsic distillation; and (2) pose refinement using robust correspondence-based constraints from GIM matches, enforcing cross-frame geometric consistency. Diagram of the geometry-distillation and correspondence branches in PURE.
Name
#Scene
Avg.
Total
Type
#Img
#Img
SpatialVID
1.8k
∼ 490
882k
Real(Unlabeled)
Sekai-Real
4k
900
3.6M
Real(Unlabeled)
OmniWorld
121
∼ 4k
484k
Synthetic
VKITTI
100
425
42k
Synthetic
BEDLAM
30
250
7.5k
Synthetic
Table 1. Statistics of datasets for fine-tuning.
Method
Sintel
Bonn
Dycheck
ATE ↓
Trans ↓
Rot ↓
ATE ↓
Trans ↓
Rot ↓
ATE ↓
Trans ↓
Rot ↓
VGGT
0.148
0.044
0.455
0.015
0.010
0.633
0.022
0.007
0.672
Ours (VGGT)
0.061
0.039
0.300
0.010
0.010
0.630
0.015
0.005
0.614
Pi3
0.060
0.069
0.654
0.013
0.010
0.632
0.011
0.004
0.613
Ours (Pi3)
0.058
0.066
0.668
0.013
0.010
0.630
0.010
0.004
0.615
DA3
0.078
0.046
0.336
0.008
0.008
0.626
0.007
0.003
0.619
Table 3. Camera pose estimation on Sintel, Bonn and Dycheck. We report ATE, and RPE for translation ( Trans ) and rotation ( Rot ). Best results are highlighted.
Figure 4. Qualitative comparison results. The first row (a) shows the reconstruction results produced by the original models, while the second row (b) presents the reconstructions after post-training with our proposed method, exhibiting improved geometric consistency and reconstruction quality, particularly in dynamic regions. Qualitative reconstructions before and after post-training on dynamic scenes.
Dataset
Depth (Abs Rel ↓ )
Camera Pose (ATE ↓ )
Before
After
Before
After
Sintel
0.157
0.147
0.048
0.037
Bonn
0.052
0.041
0.015
0.012
KITTI
0.054
0.049
-
-
Table 4. Effect of annotation-free post-training on VGGT. We report depth and camera pose performance before and after post-training on three benchmarks.
Method
Depth
Camera Pose
Abs Rel ↓
δ<1.25↑
ATE ↓
Trans ↓
Rot ↓
Base Model
0.113
0.852
0.022
0.007
0.672
w/o all
0.084
0.892
0.019
0.006
0.656
w/o Lorder
0.062
0.957
0.018
0.005
0.635
w/o Llocal
0.075
0.935
0.016
0.005
0.640
Ours (Full)
0.055
0.976
0.015
0.005
0.614
Table 5. Ablation study on Dycheck dataset. We evaluate the impact of different loss terms on depth estimation and camera trajectory accuracy.
Figure 5. Scaling behavior with increasing post-training data. Performance improves significantly as more unlabeled data is introduced, but gradually saturates at larger scales due to the fixed model capacity. Bar chart of relative depth error as the unlabeled training set grows.
Feed-forward 3D reconstruction models are efficient but rigid: once trained, they perform inference in a zero-shot manner and cannot adapt to the test scene. As a result, visually plausible reconstructions often contain errors, particularly under occlusions, specularities, and ambiguous cues. To address this, we introduce Free Geometry, a framework that enables feed-forward 3D reconstruction models to self-evolve at test time without any 3D ground truth. Our key insight is that, when the model receives more views, it produces more reliable and view-consistent reconstructions. Leveraging this property, given a testing sequence, we mask a subset of frames to construct a self-supervised task. Free Geometry enforces cross-view feature consistency between representations from full and partial observations, while maintaining the pairwise relations implied by the held-out frames. This self-supervision allows for fast recalibration via lightweight LoRA updates, taking less than 2 minutes per dataset on a single GPU. Our approach consistently improves state-of-the-art foundation models, including Depth Anything 3 and VGGT, across 4 benchmark datasets, yielding an average improvement of 3.73% in camera pose accuracy and 2.88% in point map prediction. Code is available at https://github.com/hiteacherIamhumble/Free-Geometry .
Yuhang Dai, Xingyi Yang
The Hong Kong Polytechnic University, Hong Kong SAR
Recent depth foundation models like Depth Anything 3 (DA3) achieve remarkable multi-view depth estimation but assume static 3D scenes, limiting their applicability to real-world dynamic environments. Existing training-free 4D methods like Easi3R and VGGT4D rely on correspondence-trained backbones whose attention encodes cross-frame matching, a property absent in depth-only models like DA3. We present Dyna3, a training-free framework that extends DA3 for 4D dynamic scene reconstruction without any fine-tuning. Our key insight is that DA3's cross-view features, though trained only for depth consistency, implicitly encode motion-discriminative signals when combined with best-match feature search across frames. Its static surfaces find consistent matches globally, while dynamic objects cannot. We further adopt vision-language models (VLM) to automatically generate scene-specific semantic prompts for SAM 3, enabling precise instance-level segmentation that distinguishes which objects move from what objects exist. For reconstruction, we decouple the scene into a cross-frame aligned static background and per-frame dynamic point clouds. Experiments on four datasets demonstrate that Dyna3 surpasses correspondence-trained methods with +5.5pp J-Mean over state-of-the-art VGGT4D on dynamic object segmentation, while achieving up to 13x faster pose estimation and 3x faster 4D reconstruction with 4 to 8x lower memory. Dyna3 could therefore enable much denser temporal sampling that prior methods cannot support.
Xinhao Xiang, Weiyang Li, Zhijie Zheng +2
IFM Lab, University of California, Davis, CA, USA · Genies Inc., CA, USA
Current feed-forward 3D/4D reconstruction systems rely on dense geometry and pose supervision -- expensive to obtain at scale and particularly scarce for dynamic real-world scenes. We present Flow3r, a framework that augments visual geometry learning with dense 2D correspondences (`flow') as supervision, enabling scalable training from unlabeled monocular videos. Our key insight is that the flow prediction module should be factored: predicting flow between two images using geometry latents from one and pose latents from the other. This factorization directly guides the learning of both scene geometry and camera motion, and naturally extends to dynamic scenes. In controlled experiments, we show that factored flow prediction outperforms alternative designs and that performance scales consistently with unlabeled data. Integrating factored flow into existing visual geometry architectures and training with ∼800K unlabeled videos, Flow3r achieves state-of-the-art results across eight benchmarks spanning static and dynamic scenes, with its largest gains on in-the-wild dynamic videos where labeled data is most scarce.