Organizations: Guangdong Provincial Key Laboratory of Ultra High Definition Immersive Media Technology, Shenzhen Graduate School, Peking University · The Hong Kong University of Science and Technology · Pengcheng Laboratory
Feed-forward 3D reconstruction models have achieved remarkable progress through large-scale training and strong learned priors. However, they remain fragile when applied to dynamic in-the-wild videos, where non-rigid motion, occlusions, and illumination changes are prevalent. A key limitation lies in their reliance on expensive depth and camera pose annotations, which hinders scaling to diverse real-world dynamic data and restricts generalization. To address this issue, we propose PURE, an unlabeled post-training refinement framework that improves depth and camera estimation without requiring ground-truth supervision. Instead of relying on annotations, PURE elegantly distills robust structural knowledge from state-of-the-art 2D priors into 4D feed-forward architectures. PURE leverages relative depth constraints, affine-invariant geometry distillation, and pose refinement based on feature correspondences to stabilize geometric predictions under challenging dynamic conditions. Extensive experiments on both synthetic and real-world dynamic benchmarks demonstrate that PURE delivers consistent improvements across multiple state-of-the-art architectures. Furthermore, our data-efficient paradigm effectively mitigates domain shifts, successfully unlocking the architectural limits of these base models in challenging, in-the-wild dynamic scenes.
Figures & tables
Figure 1. Our annotation-free post-training significantly improves feed-forward 3D reconstruction in dynamic scenes, yielding more accurate and temporally consistent geometry compared to the original pre-trained models. Side-by-side reconstructions before and after unlabeled post-training.
Figure 2. MoGe provides accurate frame-wise relative geometry, while feed-forward 3D models ensure temporal scale consistency. We unify both by transferring monocular geometric cues while preserving temporal consistency. Frame-wise geometry from a monocular teacher compared with a feed-forward model.
Figure 3. Overview of PURE , our annotation-free post-training framework. Given an unlabeled image sequence, a pre-trained feed-forward model predicts depth and camera poses. We refine the predictions via two complementary branches: (1) local geometry refinement using relative depth and affine-invariant constraints with intrinsic distillation; and (2) pose refinement using robust correspondence-based constraints from GIM matches, enforcing cross-frame geometric consistency. Diagram of the geometry-distillation and correspondence branches in PURE.
Name
#Scene
Avg.
Total
Type
#Img
#Img
SpatialVID
1.8k
∼ 490
882k
Real(Unlabeled)
Sekai-Real
4k
900
3.6M
Real(Unlabeled)
OmniWorld
121
∼ 4k
484k
Synthetic
VKITTI
100
425
42k
Synthetic
BEDLAM
30
250
7.5k
Synthetic
Table 1. Statistics of datasets for fine-tuning.
Method
Sintel
Bonn
Dycheck
ATE ↓
Trans ↓
Rot ↓
ATE ↓
Trans ↓
Rot ↓
ATE ↓
Trans ↓
Rot ↓
VGGT
0.148
0.044
0.455
0.015
0.010
0.633
0.022
0.007
0.672
Ours (VGGT)
0.061
0.039
0.300
0.010
0.010
0.630
0.015
0.005
0.614
Pi3
0.060
0.069
0.654
0.013
0.010
0.632
0.011
0.004
0.613
Ours (Pi3)
0.058
0.066
0.668
0.013
0.010
0.630
0.010
0.004
0.615
DA3
0.078
0.046
0.336
0.008
0.008
0.626
0.007
0.003
0.619
Table 3. Camera pose estimation on Sintel, Bonn and Dycheck. We report ATE, and RPE for translation ( Trans ) and rotation ( Rot ). Best results are highlighted.
Figure 4. Qualitative comparison results. The first row (a) shows the reconstruction results produced by the original models, while the second row (b) presents the reconstructions after post-training with our proposed method, exhibiting improved geometric consistency and reconstruction quality, particularly in dynamic regions. Qualitative reconstructions before and after post-training on dynamic scenes.
Dataset
Depth (Abs Rel ↓ )
Camera Pose (ATE ↓ )
Before
After
Before
After
Sintel
0.157
0.147
0.048
0.037
Bonn
0.052
0.041
0.015
0.012
KITTI
0.054
0.049
-
-
Table 4. Effect of annotation-free post-training on VGGT. We report depth and camera pose performance before and after post-training on three benchmarks.
Method
Depth
Camera Pose
Abs Rel ↓
δ<1.25↑
ATE ↓
Trans ↓
Rot ↓
Base Model
0.113
0.852
0.022
0.007
0.672
w/o all
0.084
0.892
0.019
0.006
0.656
w/o Lorder
0.062
0.957
0.018
0.005
0.635
w/o Llocal
0.075
0.935
0.016
0.005
0.640
Ours (Full)
0.055
0.976
0.015
0.005
0.614
Table 5. Ablation study on Dycheck dataset. We evaluate the impact of different loss terms on depth estimation and camera trajectory accuracy.
Figure 5. Scaling behavior with increasing post-training data. Performance improves significantly as more unlabeled data is introduced, but gradually saturates at larger scales due to the fixed model capacity. Bar chart of relative depth error as the unlabeled training set grows.