Reconstructing dynamic 4D scenes requires jointly estimating correspondence, geometry, object motion, and camera motion. Existing feed-forward methods typically predict dense task-specific maps or independently process source-target pairs, leading to unnecessary computation for sparse queries and limited feature reuse across different frame pairs. We present UniQuery4R, a query-conditioned framework that encodes a multi-frame clip once and selects the source view, target view, and continuous source-image coordinate only at decoding time via source-to-target cross-attention. Each query jointly predicts target correspondence, target-time 3D position, and scene flow, along with source depth, while camera parameters are estimated per view. This design allows the encoded clip to be reused across arbitrary source-target selections and supports both sparse inference and dense reconstruction through batched queries, without learned temporal embeddings tied to a fixed clip length. We further introduce a direction-magnitude parameterization of scene flow with separate supervision for moving and static points. Among the evaluated methods, UniQuery4R achieves the best macro-average results on WorldTrack for both scene-flow estimation and dynamic-point reconstruction.
Figures & tables
Figure 1: Conceptual overview. A continuous source-pixel query over a jointly encoded clip predicts target correspondence, target-time 3D position, scene flow, and source depth, while camera parameters are estimated per view.
Method
Input Clip Length
Query Source
Key / Value
Supported Output
V-DPM
Variable
N/A
N/A
Dense
D4RT
Fixed (48 frames)
Input embedding
All-frame patches
Sparse / Dense
UniQuery4R (ours)
Variable
Sampled from Fs
Ft patches
Sparse / Dense
Table 1: Paradigm comparison with D4RT and V-DPM. UniQuery4R differs in temporal horizon, query formation, and target-side attention. “Variable” means no learned fixed clip length (memory-bounded in practice).
Figure 2: (a) Overall architecture. A clip is patch-embedded with camera tokens and jointly encoded by frame/global attention. At decoding time, q=(u,v,s,t) selects source/target; the multi-scale query decoder yields qs and target features Mt , then qs cross-attends to Mt ( ×4 ) to form qc . Src → Tgt MLPs predict P , ΔP , and f from qc ; a Src-only MLP predicts local depth from qs ; a camera head reads per-view tokens. (b) Multi-scale query decoder with gated fusion that builds qs from the source pyramid.
Output
Sym.
Q?
Description
Local, per-query — from qc
Dynamic point
P∈R3
Y
target-time pos. (ref.-cam.)
Scene flow
ΔP∈R3
Y
s→t displ. (ref.-cam.)
2D corresp.
f∈R2
Y
target-image coordinate
Conf./prec.
(cP,cf)
Y
point and correspondence
Local, per-query — from qs
Table 2: UniQuery4R outputs for query q=(u,v,s,t) . “Q?” denotes query dependence; camera parameters are predicted per view.
Figure 5
Method
ADT
NRGBD
Sintel
A 5 ↑
A 30 ↑
ATE ↓
R t ↓
R r ↓
A 5 ↑
A 30 ↑
ATE ↓
R t ↓
R r ↓
A 5 ↑
A 30 ↑
ATE ↓
R t ↓
R r ↓
VGGT ( Wang et al. 2025 )
0.134
0.724
0.219
0.269
3.457
0.676
0.926
0.047
0.071
0.994
0.212
0.475
0.100
0.050
0.588
DA3-Giant ( Lin et al. 2026b )
0.145
0.741
0.160
0.223
2.221
0.772
0.944
0.034
0.045
0.848
0.214
0.495
0.065
0.044
0.373
VGGT- Ω ( Wang et al. 2026a )
0.446
0.851
0.087
0.091
2.107
0.704
0.933
0.040
0.059
1.008
0.286
0.583
0.028
0.022
0.230
Any4D ( Karhade et al. 2026 )
0.070
0.597
0.346
0.432
4.566
0.348
0.833
0.105
0.163
2.247
0.034
0.290
0.144
0.104
2.226
V-DPM ( Sucar et al. 2026 )
0.400
0.861
0.076
0.103
2.374
0.664
0.925
0.045
0.066
1.113
0.175
0.505
0.082
0.054
0.520
Table 5: Camera pose (A 5 /A 30 : AUC@5/@30) and trajectory (ATE; R t /R r : RPE t /RPE r ) on ADT, NRGBD, and Sintel. Best bolded; second-best underlined.
Output
From
Ch.
Act.
Notes
P
qc
3
inv-log
target-time 3D
cP
qc
1
—
logit
v~,m~
qc
3+1
dir.–mag.
→ΔP
f
qc
2
identity
abs. UV
cf
qc
4
—
logits
d,cd
qs
1+1
exp /—
source depth
Table A.1: Query prediction heads.
Term
Default
Dataset overrides
Source depth Ld
1
DL3DV/Waymo: 0 ; VKITTI2/Stereo4D: 0
Target point LP
10
DL3DV: .1 ; Waymo/VKITTI2: 0 ; Stereo4D: .1
Displacement LΔ
50
Waymo/VKITTI2/Stereo4D: 30
Direction/static magnitude
1
none
2D correspondence Lf
5
PointOdyssey: 0
Camera Lπ
1
outer multiplier 10
Table A.2: Loss weights used to train the final model. Camera translation, quaternion, and FoV components have internal weights 1 , 10 , and 0.5 , respectively. The four refinement predictions are weighted by {0.63,0.62,0.6,1} and then averaged.
Source
#samples
Weight
PointOdyssey
111
0.4
Kubric-4D
12,928
1.0
CoTracker3Kubric
5,827
1.0
Internal dataset
2,596
1.0
Hypersim
758
0.1
DL3DV
54,903
0.1
Table A.3: Training mixture (indexed samples and relative weights).
Setting
Value
Hardware
16× NVIDIA H20
Iterations
150 k
Numerical precision
bfloat16
Batch size (per GPU / global)
1 / 16
Backbone learning rate
10−5
Other-module learning rate
10−4
Table A.4: Optimization hyperparameters for the final model.
Encode vs. clip length
Encode vs. resolution
Encode/decode vs. query count
T
Enc. (ms)
H×W
Enc. (ms)
Mode
Q (chunks)
Enc. (ms)
Query dec. (ms)
2
118.90±14.48
2522
107.62±13.36
S
256 (1)
357.79
5.36±0.10
4
200.06±10.73
3642
197.37±5.29
S
1,024 (1)
359.38
8.33±0.05
8
359.76±8.92
5042
359.76±8.92
S
4,096 (1)
358.54
21.80±1.15
12
544.36±4.79
5182
356.40±6.90
S
8,192 (1)
361.27
36.85±0.32
16
765.93±6.41
S
16,384 (1)
366.20
69.66±0.66
Table B.1: Measured runtime points underlying the scaling results in the main paper. The clip-length sweep fixes H=W=504 and Q=4096 ; the resolution sweep fixes T=8 and Q=4096 . The query sweep uses T=8 ; S denotes sparse decoding at 5042 , while D denotes dense decoding at the indicated resolution. Parentheses report the number of chunks with a maximum chunk size of 30k. Results use FP16 on one NVIDIA A800 with five warm-up runs, 20 measured repeats, and CUDA synchronization.
Figure B.1: First-layer source-to-target cross-attention over increasing temporal gaps. Each row fixes a source query and visualizes its normalized attention on progressively more distant target frames.
Encoder initialization
APD ↑
EPE ↓
VGGT- Ω ( Wang et al. 2026a )
80.88
0.1857
DA3-Giant ( Lin et al. 2026b )
81.29
0.1821
Internal foundation model
83.62
0.1597
Table B.2: WorldTrack dynamic-point macro-average under matched training ( 150 k iterations, 8× A800) with different encoder initializations.
Method
EPE ↓
SpaTrackerV2 ( Xiao et al. 2025 )
5.5187
St4RTrack ( Feng et al. 2025 )
3.8485
TraceAnything ( Liu et al. 2026 )
7.3385
Any4D ( Karhade et al. 2026 )
5.2362
V-DPM ( Sucar et al. 2026 )
2.7367
4RC ( Luo et al. 2026 )
3.2957
Table B.3: Dynamic-point EPE (m) on SynthVerse ( 50 sequences ×64 frames; WorldTrack protocol). Zero-shot evaluation excludes SynthVerse training; the last row uses the official training split with benchmark sequences held out.
Figure C.1: Additional 3D tracking on diverse dynamic scenes. Each color denotes one fixed source query over time.
Figure C.2: Qualitative 2D correspondence for selected source–target pairs.
Figure C.3: Motion masks from thresholded consecutive scene flow ( γ=0.02 m).
Figure C.4: Static-scene reconstruction with estimated cameras.
We present Depth Anything V4 (DAV4), a framework for dynamic 4D scene reconstruction from monocular video. Our key contribution is the application of Riemannian Flow Matching (RFM) to 4D Gaussian Splatting parameters, defining probability paths directly on non-Euclidean manifolds (scale, rotation, opacity), ensuring all intermediate states are valid. Through controlled experiments, we isolate RFM's contribution from test-time optimization (TTO) and pre-training. A deterministic MLP baseline with the same data, architecture, and TTO achieves F-score 0.762; RFM achieves 0.806 - the +0.044 gain is RFM's isolated contribution. We provide corrected computational cost analysis: pre-training is 360 GPU-hours, amortizing for large-scale deployment (over 10,000 scenes). Uncertainty is quantified via Negative Gaussian Log-Likelihood and Expected Calibration Error. DAV4 outperforms prior Depth Anything models and per-scene 4D-GS on dynamic reconstruction and novel-view synthesis, while using no human-annotated depth labels as training losses.
Jiaming Fan, Jian Lu, Jinling Jia +1
College of Artificial Intelligence, Nanjing University of Posts and Telecommunications · College of Automation, Nanjing University of Posts and Telecommunications · Institute of Ad-vanced Technology for Carbon Neutrality, Nanjing University of Posts and Telecommunications
Previous feed-forward 4D reconstruction methods either predict per-frame static point clouds, ignoring foreground motion, or estimate point cloud trajectories while being limited to small camera motions. This restricts their ability to aggregate observations over time and reconstruct complete dynamic scenes under large viewpoint changes. To address this limitation, we propose OmniX, a feed-forward 4D reconstruction framework that predicts dense 3D point trajectories for every pixel from videos with large camera motion. OmniX decouples dynamic motion modeling from static geometry prediction and represents motion using a compact set of dynamic tokens. By leveraging the sparse and low-rank structure of 3D motion, these tokens generate trajectory fields for all pixels across all images while efficiently preserving global interactions. To facilitate training, we further build an automatic UE5-based 4D data engine and introduce a large-scale dataset containing 80K scenes and 1.28M multi-view videos with full geometric annotations. OmniX achieves state-of-the-art performance on dense 3D point trajectory prediction and 3D point tracking, while also demonstrating competitive results on video depth estimation and camera pose estimation.
Yanqin Jiang, Tengfei Wang, Zhengwei Wang +6
MAIS, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Tencent Hunyuan +3
Dynamic scene reconstruction from monocular video remains a fundamental challenge in computer vision. Existing feed-forward methods predict 3D Gaussians pixel-wise for each frame, suffering from duplicated Gaussians and view-dependent biases that hinder effective learning of scene motion. We present C4G, a feed-forward 4D reconstruction framework built upon a compact set of timestamp-conditioned learnable Gaussian query tokens. Each token aggregates corresponding features across the full temporal context and decodes a 3D Gaussian whose position is modulated by the target timestamp, enabling globally coherent motion modeling without per-scene optimization. To capture fine-grained details, we further introduce a video diffusion model-based rendering enhancement module. Since our framework effectively aggregates features into Gaussians, we extend this capability to feature lifting, producing a 4D feature field that supports point tracking and dynamic scene understanding. C4G achieves strong novel-view synthesis performance using significantly fewer Gaussians and without requiring camera poses, while exhibiting stronger motion modeling and robustness to large temporal gaps.