Dense 4D reconstruction from unposed images remains a critical challenge, with current methods relying on slow test-time optimization or fragmented, task-specific feedforward models. We introduce UFO-4D, a unified feedforward framework to reconstruct a dense, explicit 4D representation from just a pair of unposed images. UFO-4D directly estimates dynamic 3D Gaussian Splats, enabling the joint and consistent estimation of 3D geometry, 3D motion, and camera pose in a feedforward manner. Our core insight is that differentiably rendering multiple signals from a single Dynamic 3D Gaussian representation offers major training advantages. This approach enables a self-supervised image synthesis loss while tightly coupling appearance, depth, and motion. Since all modalities share the same geometric primitives, supervising one inherently regularizes and improves the others. This synergy overcomes data scarcity, allowing UFO-4D to outperform prior work by up to 3 times in joint geometry, motion, and camera pose estimation. Our representation also enables high-fidelity 4D interpolation across novel views and time. Please visit our project page for visual results: https://ufo-4d.github.io/
Figures & tables
Figure 1: Given a pair of unposed images, the proposed UFO-4D outputs dynamic 3D Gaussians in the canonical space and relative camera pose in a feedforward manner. This explicit 4D representation can solve various downstream tasks such as 3D geometry (point, depth) and motion (scene flow, optical flow). Besides, it can interpolate image, geometry, and motion at novel view and time.
Figure 2: Network architecture. Given a pair of input images It and It+1 and camera intrinsics K , UFO-4D outputs parameters for dynamic 3D Gaussians and relative camera pose in a feed-forward manner. Given the estimates, UFO-4D can render image, point, and motion at any interpolated time and view. While the intrinsic token needs the real camera intrinsics, the pose token is a learnable parameter and does not require the inference-time pose input.
Figure 3: (a) Each Gaussian is translated with its motion to represent 3D scene at time t+Δt . (b) Point and motion as well as an image are rasterized together.
Method
Stereo4D
Bonn
KITTI
Sintel final
EPE
Abs. Rel.
δ<1.25
EPE
Abs. Rel.
δ<1.25
EPE
Abs. Rel.
δ<1.25
EPE
Abs. Rel.
δ<1.25
MASt3R ( Leroy et al., 2024 )
1.8869
0.3141
66.5000
0.4870
0.1739
76.4857
2.3036
0.0865
92.04
3.9972
0.3659
61.0337
MonST3R ( Zhang et al., 2025a )
1.5038
0.1907
74.7522
0.1881
0.0620
96.2170
2.797
0.107
87.6071
3.6067
0.3405
57.6134
DynaDUSt3R ( Jin et al., 2025 )
0.811
0.112
86.07
0.4236
0.0840
92.7514
4.8892
0.1158
83.3925
4.3620
0.3013
61.0385
ZeroMSF ( Liang et al., 2025b )
1.6619
0.2200
76.1773
0.2077
0.0787
91.5787
3.3921
0.107
92.2784
3.637
0.3332
63.8576
St4RTrack ( Feng et al., 2025 )
1.4668
0.2093
71.2561
0.181
0.0674
95.1704
4.7307
0.1252
82.8568
4.1447
0.3574
57.1376
Table 1: Geometry estimation : We report end-point error (EPE) for pointmap accuracy and absolute relative error (Abs. Rel.) and δ<1.25 for depth accuracy. Lower is better for EPE and Abs. Rel., and higher is better for δ<1.25 .
Method
Stereo4D
KITTI
forward
backward
forward
EPE3D↓
δ3D0.05↑
EPE3D↓
δ3D0.05↑
EPE3D↓
δ3D0.05↑
DynaDUSt3R ( Jin et al., 2025 )
0.1751
51.46
0.161
52.16
0.4625
1.0925
ZeroMSF ( Liang et al., 2025b )
0.164
45.6312
–
–
0.442
16.20
St4RTrack ( Feng et al., 2025 )
0.3725
26.6157
–
–
0.7506
4.0082
UFO-4D (Ours)
0.0488
83.0618
0.0500
82.6700
0.1372
47.7964
Table 2: Motion estimation. We evaluate the scene flow accuracy on the Stereo4D test split ( Jin et al., 2025 ) and KITTI Scene Flow 2015 Training ( Menze et al., 2018 ) . Our approach substantially outperforms others, achieving the best numbers on all metrics.
Figure 4: Qualitative comparison of depth and projected 2D optical flow on Stereo4D, Bonn, and KITTI. For motion on KITTI, it visualizes motion relative to the camera, as GT is defined. Unlike DynaDUSt3R, ZeroMSF and St4RTrack, which suffers from residual motions in static region and inaccurate motion on object boundaries, UFO-4D exhibits clear motion boundaries and separation between moving objects and background. More qualitative results are in Section A.1 .
Method
Stereo4D ( Jin et al., 2025 )
Bonn ( Palazzolo et al., 2019 )
Sintel ( Butler et al., 2012 )
ATE
RPEtrans
RPErot
ATE
RPEtrans
RPErot
ATE
RPEtrans
RPErot
MonST3R ( Zhang et al., 2025a )
0.0458
0.0647
0.8047
0.0031
0.0043
0.5436
0.0290
0.0410
1.0796
St4RTrack ( Feng et al., 2025 )
0.0602
0.0851
0.7289
0.0024
0.0034
0.3997
0.0231
0.0326
0.8835
UFO-4D (Ours)
0.0101
0.0142
0.1794
0.0020
0.0028
0.2709
0.0122
0.0172
0.2980
Table 3: Pose estimation. We report standard metrics, ATE and RPE translation ( RPEtrans ) and rotation ( RPErot ) for pose estimation on Stereo4D test, Bonn, and Sintel final, Our feed-forward approach outperforms methods relying on iterative solvers ( e. g . MonST3R, St4RTrack) on all metrics.
Figure 5: Opacity as learnable confidence : Opacity maps show the model’s behavior in a (dis)occlusion scenario. Our model learns to assign high confidence (opacity) to disoccluded regions, and for mutually-visible regions, it selects only one corresponding Gaussian from the two views, enabling an efficient and compact 4D representation.
Alias
Configuration
Image (PSNR ↑ )
Point (EPE ↓ )
Point rast. (EPE ↓ )
Motion (EPE ↓ )
Motion rast. (EPE ↓ )
(a)
Full model
23.929
0.827
0.841
0.069
0.064
(b)
W/o image gradient.
24.268
0.903
0.899
0.072
0.070
(c)
W/o motion and point rendering
20.675
0.911
0.943
0.071
0.069
(d)
W/o motion, point, and image rendering
–
0.886
–
0.071
–
Table 4: Loss ablation. We report image reconstruction in PSNR and accuracy of Gaussian center (Point), rasterized point (Point rast.), Gaussian velocity (Motion), and rasterized motion (Motion rast.) in end-point error (EPE). Integrating the photometric loss gradient ( Eq. 7c ) with all heads boosts overall performance. Losses on rendered motion ( Eq. 6b ) and point map ( Eq. 6c ) are crucial for achieving high accuracy.
Figure 6: Qualitative comparison on loss ablation. Gradient backpropagation from the image synthesis loss and rendering losses on point and motion helps UFO-4D improve motion and point estimates, especially on object and motion boundaries.
Output representation
Equivalent or closest models
Stereo4D
KITTI
Bonn
Motion EPE
Point EPE
Motion EPE
Point EPE
Point EPE
Dynamic 3DGS
UFO-4D (Ours)
0.058
0.781
0.244
3.553
0.211
Per-pixel point and motion
DynaDUSt3R, ZeroMSF
0.057
0.809
0.283
4.567
0.196
Per-pixel point
MonST3R
–
0.804
–
4.562
0.201
Table 5: Architecture comparison. For apples-to-apples comparison, we train other methods with different output representation on our training protocol. Our method achieves the best accuracy except for Bonn on both point and motion end-point errors (lower the better).
Figure 7: 4D interpolation on DAVIS. Given an input pair on the left, the righthand side shows rasterized images, depth, and motion maps at the canonical camera coordinate at interpolated time between the two input frames. Our method also clearly segments out moving objects.
Appendix figures & tables15 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A: Qualitative comparison on Stereo4D. We visualize depth and projected 2D optical flow. UFO-4D consistently outperforms other methods in depth and motion accuracy. Notably, in scenes with significant camera rotation (top two rows), our method robustly estimates 3D geometry and isolates 3D motion of dynamic objects in the canonical space. In contrast, other methods struggle to disentangle camera ego-motion, resulting in residual motion artifacts on static background.
Figure B: Qualitative comparison on KITTI. We visualize motion in camera coordinates, consistent with the ground truth of KITTI. Our method outperforms competitors with notably clearer motion boundaries, whereas ZeroMSF and St4RTrack frequently show erroneous motion estimation.
Figure C: Qualitative comparison on Bonn. Comparing to other methods, our method outputs clear 3D motion disentangled from camera motion. ZeroMSF and St4RTrack suffer from non-zero residual motion in background. DynaDUSt3R tends to show unclear motion boundaries.
Figure D: Visualization of annotation of Stereo4D. The first two columns visualize two input frames, and the last columns visualizes 3D motion annotation defined at the canonical camera coordinate. While Stereo4D provides large-scale real-world annotated data, the label includes noisy annotation, e. g . , non-zero 3D motion for static region.
Method
Stereo4D
Bonn
KITTI
Sintel final
EPE
Abs. Rel.
δ<1.25
EPE
Abs. Rel.
δ<1.25
EPE
Abs. Rel.
δ<1.25
EPE
Abs. Rel.
δ<1.25
MASt3R ( Leroy et al., 2024 )
1.6575
0.2897
65.5990
0.4839
0.1726
76.6771
2.2467
0.0856
92.15
3.9065
0.3595
60.6771
MonST3R ( Zhang et al., 2025a )
1.3798
0.1939
72.5653
0.1871
0.0616
96.2591
2.741
0.1057
87.9555
3.5138
0.3376
57.2272
DynaDUSt3R ( Jin et al., 2025 )
0.683
0.103
87.94
0.4230
0.0835
92.8303
4.8054
0.1125
84.0951
4.2958
0.2946
60.8804
ZeroMSF ( Liang et al., 2025b )
1.6737
0.2412
72.7655
0.2067
0.0782
91.6802
3.2652
0.103
92.6609
3.560
0.3266
63.8047
St4RTrack ( Feng et al., 2025 )
1.2691
0.1975
71.3659
0.180
0.0669
95.2358
4.6610
0.1236
83.1346
4.0965
0.3493
56.8702
Appendix
Table A: Geometry estimation (under per-valid-pixel averaging protocol) : Using the same evaluation metric in DynaDUSt3R, our method consistently outperforms other methods.
Method
Stereo4D
KITTI
forward
backward
forward
EPE3D
δ3D0.05
EPE3D
δ3D0.05
EPE3D
δ3D0.05
DynaDUSt3R ( Jin et al., 2025 )
0.118
65.19
0.106
64.47
0.467
1.09
ZeroMSF ( Liang et al., 2025b )
0.067
67.38
–
–
0.452
13.84
St4RTrack ( Feng et al., 2025 )
0.233
42.56
–
–
0.768
3.36
UFO-4D (Ours)
0.023
91.78
0.025
91.28
0.139
46.71
Appendix
Table B: Motion estimation (under per-valid-pixel averaging protocol) We evaluate the scene flow accuracy on the Stereo4D test split ( Jin et al., 2025 ) and KITTI Scene Flow 2015 Training ( Menze et al., 2018 ) . Our approach consistently outperforms others on all metrics.
Dataset proportion
Stereo4D
KITTI
Bonn
ST
PO
VK
Image ( ↑ )
Point ( ↓ )
Motion ( ↓ )
Image ( ↑ )
Point ( ↓ )
Motion ( ↓ )
Image ( ↑ )
Point ( ↓ )
0.7
0.15
0.15
23.547
0.065
0.891
18.845
0.273
3.427
18.568
0.314
0.85
–
0.15
23.419
0.065
0.839
18.791
0.298
4.102
18.056
0.344
0.85
0.15
–
23.587
0.064
0.874
18.037
0.355
6.214
18.610
0.291
–
0.5
0.5
21.918
0.074
1.292
19.363
0.267
2.761
19.824
0.273
1.0
–
–
23.929
0.064
0.841
18.610
0.311
6.138
18.913
0.316
Appendix
Table C: Dataset ablation. Given three training datasets, Stereo4D (ST), PointOdyssey (PO), and Virtual KITTI 2 (VK), we try a different mixture of each dataset, train our model, and report performance on three evaluation datasets, Stere4D, KITTI, and Bonn, reporting PSNR for image reconstruction and end-point error (EPE) for point and motion. The best accuracy on Stereo4D is achieved by solely training on Stereo4D. Adding Virtual KITTI2 improves especially motion accuracy on KITTI but hurts point accuracy on Bonn. Adding PointOdyssey improves accuracy on Bonn but hurt that on KITTI as trade-off. All experiment is done in a small scale training.
Method
Stereo4D
KITTI
Bonn
PSNR ↑
Point EPE ↓
Motion EPE ↓
PSNR ↑
Point EPE ↓
Motion EPE ↓
PSNR ↑
Point EPE ↓
Initialization from MASt3R (baseline)
24.519
0.781
0.058
19.606
3.553
0.244
26.606
0.211
Initialization from MonST3R
24.200
0.868
0.059
19.506
3.260
0.258
26.551
0.155
Appendix
Table D: Training with different initialization. We compare our model trained on different pretrained weights, report image PSNR, 3D point EPE, and 3D motion EPE on Stereo4D ( Jin et al., 2025 ) , KITTI ( Menze et al., 2018 ) , and Bonn ( Palazzolo et al., 2019 ) datasets. Initialization from MASt3R gives overall better accuracy whereas that from MonST3R improves 3D point accuracy on KITTI and Bonn.
Figure E: Visualization of the scale of 3D Gaussian. For each dataset, KITTI, Bonn, and Stereo4D, we visualize Gaussian scale (the bottom row) along with an input image and predicted depth of our method. The Gaussian scale (x,y,z) is visualized in the (R,G,B) color space. For textureless region such as sky or wall, our method predicts larger scale of gaussian whereas small-scale 3D Gaussians are dominant in high-frequency texture region. Bright color means larger Gaussians.
Figure F: Comparison with DynaDUSt3R on Bonn. Unlike per-pixel point representation, our representation gives subtle inaccuracy in geometry on textureless region ( e. g . wall). Brighter means higher errors.
Scale Range
Point (EPE) ↓
Depth (Abs. Rel.) ↓
0−0.5
0.189
0.073
0.5−1.0
0.151
0.059
1.0−1.5
0.200
0.068
1.5−2.0
0.189
0.067
2.0−2.5
0.185
0.071
2.5−3.0
0.205
0.085
Appendix
Table E: Impact of Gaussian scale on geometry accuracy on Bonn. We report Point EPE and Depth Abs. Rel. across different ranges of estimated Gaussian scales. We observe that depth accuracy decreases along with the increased scale of estimated 3D Gaussians.
Figure G: Visualization of cross-attention maps of the camera pose token. We observe that at specific blocks (Blocks 8th, 11th, and 12th), the camera pose token more attends to image tokens located in static regions ( i. e . , high attention scores in brighter colors), which provide reliable cues for relative camera pose estimation. Moving objects tend to have darker colors, i. e . , lower attention scores. As a reference at each second row, we also visualize the cross-attention maps for a static scene configuration, where identical images are used as input.
Output representation
Pose estimator
Stereo4D
KITTI
Bonn
Motion EPE
Point EPE
Motion EPE
Point EPE
Point EPE
Dynamic 3DGS ( Ours )
Pose head
0.057
0.781
0.244
3.553
0.211
Per-pixel point and motion
Pose head
0.060
0.811
0.245
4.396
0.206
PnP-RANSAC
0.058
0.809
0.283
4.567
0.196
Appendix
Table F: Ablation on Pose and Representation. We compare our method with per-pixel baselines using different pose estimators (pose head vs. PnP-RANSAC). Our Dynamic 3DGS representation with the feedforward pose head achieves the best accuracy on Stereo4D and KITTI.
Figure H: Qualitative comparison of our method and baseline with per-pixel point representation on KITTI. Unlike per-pixel representation model, our method yields much better accuracy especially on planar road surface, benefiting from our accurate pose estimation and photometric losses on rendered images. In error maps, darker means more accurate.
Method
Stereo4D ( Jin et al., 2025 )
Bonn ( Palazzolo et al., 2019 )
Sintel ( Butler et al., 2012 )
ATE
RPEtrans
RPErot
ATE
RPEtrans
RPErot
ATE
RPEtrans
RPErot
MonST3R ( Zhang et al., 2025a )
0.0458
0.0647
0.8047
0.0031
0.0043
0.5436
0.0290
0.0410
1.0796
St4RTrack ( Feng et al., 2025 )
0.0602
0.0851
0.7289
0.0024
0.0034
0.3997
0.0231
0.0326
0.8835
UFO-4D (Ours, with PnP+RANSAC)
0.0120
0.0170
0.2052
0.0024
0.0033
0.2933
0.0168
0.0238
0.3315
UFO-4D (Ours, with the pose head)
0.0101
0.0142
0.1794
0.0020
0.0028
0.2709
0.0122
0.0172
0.2980
Appendix
Table G: Pose estimation with PnP+RANSAC. Our method with PnP+RANSAC also outperforms direct competitors, validating the higher accuracy of our estimated geometry than others. Also our direct feedforward estimator achieves ∼ 16.6% higher accuracy than this baseline, further demonstrating its robustness over noise-sensitive iterative solvers. ATE and RPE translation ( RPEtrans ) and rotation ( RPErot ) are reported.
Generating interactive and dynamic 4D scenes from a single static image remains a core challenge. Most existing generate-then-reconstruct and reconstruct-then-generate methods decouple geometry from motion, causing spatiotemporal inconsistencies and poor generalization. To address these, we present MoGe4D (Motion and Geometry-Aware image-to-4D Synthesis), a geometry-conditioned framework for single-image 4D synthesis that models a scene as dense 4D point trajectories. Instead of treating geometry and dynamics as two disconnected stages, our method starts from an initial geometric prior inferred from the input image and predicts future time-varying trajectories in a diffusion process, improving spatiotemporal coherence while preserving structural stability. To support this task, we first introduce TrajScene-60K, a large-scale dataset of 60,000 video samples with dense 4D point trajectories, addressing the scarcity of high-quality training data for scene-level 4D generation. Built on this, our diffusion-based 4D Scene Trajectory Generator (4D-STraG) predicts geometry-consistent and motion-plausible trajectory fields conditioned on the input image, with a depth-guided motion normalization strategy to reduce scale ambiguity and a Motion Perception Module (MPM) to inject motion-aware priors. We further propose a 4D View Synthesis Module (4D-ViSM) to render the generated 4D representation into videos under arbitrary camera trajectories. Experiments show that MoGe4D produces high-quality 4D scenes with strong temporal coherence, favorable geometry-aware consistency, and compelling novel-view synthesis from a single image. Code: https://github.com/Zhangyr2022/MoGe4D.
Yanran Zhang, Ziyi Wang, Wenzhao Zheng +3
Department of Automation, Tsinghua University, China · GigaAI
Reconstructing dynamic 4D scenes requires jointly estimating correspondence, geometry, object motion, and camera motion. Existing feed-forward methods typically predict dense task-specific maps or independently process source-target pairs, leading to unnecessary computation for sparse queries and limited feature reuse across different frame pairs. We present UniQuery4R, a query-conditioned framework that encodes a multi-frame clip once and selects the source view, target view, and continuous source-image coordinate only at decoding time via source-to-target cross-attention. Each query jointly predicts target correspondence, target-time 3D position, and scene flow, along with source depth, while camera parameters are estimated per view. This design allows the encoded clip to be reused across arbitrary source-target selections and supports both sparse inference and dense reconstruction through batched queries, without learned temporal embeddings tied to a fixed clip length. We further introduce a direction-magnitude parameterization of scene flow with separate supervision for moving and static points. Among the evaluated methods, UniQuery4R achieves the best macro-average results on WorldTrack for both scene-flow estimation and dynamic-point reconstruction.
Tiancheng Chen, Sheng Tang, Wenhua Jin +4
Kosmo Research · Automotive Engineering Department, Jilin University · School of Software, Tsinghua University
Recent feed-forward 3D gaussian splatting methods have made dramatic progress on individual aspects of 3D scene reconstruction, but no existing method jointly addresses dynamic content, multi-view input, and unknown camera poses in a single feed-forward pass. Methods that handle dynamics either require accurate camera poses or accept only monocular input; pose-free multi-view methods address only static scenes; and per-scene optimization methods bridge some of these gaps but at minutes-to-hours cost per scene. We introduce NoPo4D, the first feed-forward system that addresses this empty quadrant. Building on a pretrained geometry backbone and recent 4D Gaussian frameworks, NoPo4D introduces a velocity decomposition that splits Gaussian motion into per-pixel image-plane shifts and depth changes, allowing direct supervision from pseudo ground-truth optical flow on the 2D component. This sidesteps both the differentiable rendering that couples prior posed methods to pose accuracy and the 3D motion ground truth that prior pose-free methods require. The system is rounded out by a bidirectional motion encoder for cross-view and cross-frame feature aggregation, and view-dependent opacity that mitigates cross-view and cross-timestep Gaussian misalignments. On four multi-view dynamic benchmarks, NoPo4D consistently outperforms prior feed-forward baselines, and with an optional post-optimization stage surpasses per-scene optimization methods, while running orders of magnitude faster.