UFO-4D: Unposed Feedforward 4D Reconstruction from Two Images

Junhwa Hur; Charles Herrmann; Songyou Peng; Philipp Henzler; Zeyu Ma; Todd Zickler; Deqing Sun

cs.CVFeb 27, 2026

UFO-4D: Unposed Feedforward 4D Reconstruction from Two Images

Authors: Junhwa Hur, Charles Herrmann, Songyou Peng, Philipp Henzler, Zeyu Ma, Todd Zickler, Deqing Sun

Organizations: Google · Princeton University · Harvard University

Abstract

Dense 4D reconstruction from unposed images remains a critical challenge, with current methods relying on slow test-time optimization or fragmented, task-specific feedforward models. We introduce UFO-4D, a unified feedforward framework to reconstruct a dense, explicit 4D representation from just a pair of unposed images. UFO-4D directly estimates dynamic 3D Gaussian Splats, enabling the joint and consistent estimation of 3D geometry, 3D motion, and camera pose in a feedforward manner. Our core insight is that differentiably rendering multiple signals from a single Dynamic 3D Gaussian representation offers major training advantages. This approach enables a self-supervised image synthesis loss while tightly coupling appearance, depth, and motion. Since all modalities share the same geometric primitives, supervising one inherently regularizes and improves the others. This synergy overcomes data scarcity, allowing UFO-4D to outperform prior work by up to 3 times in joint geometry, motion, and camera pose estimation. Our representation also enables high-fidelity 4D interpolation across novel views and time. Please visit our project page for visual results: https://ufo-4d.github.io/

Figures & tables

Figure 1: Given a pair of unposed images, the proposed UFO-4D outputs dynamic 3D Gaussians in the canonical space and relative camera pose in a feedforward manner. This explicit 4D representation can solve various downstream tasks such as 3D geometry (point, depth) and motion (scene flow, optical flow). Besides, it can interpolate image, geometry, and motion at novel view and time.

Figure 2: Network architecture. Given a pair of input images $\mathbf{I}_{t}$ and $\mathbf{I}_{t+1}$ and camera intrinsics $\mathbf{K}$ , UFO-4D outputs parameters for dynamic 3D Gaussians and relative camera pose in a feed-forward manner. Given the estimates, UFO-4D can render image, point, and motion at any interpolated time and view. While the intrinsic token needs the real camera intrinsics, the pose token is a learnable parameter and does not require the inference-time pose input.

Panel of Figure 3 — Figure 3: (a) Each Gaussian is translated with its motion to represent 3D scene at time $t+\Delta t$ . (b) Point and motion as well as an image are rasterized together.

Method	Stereo4D			Bonn			KITTI			Sintel final
Method	EPE	Abs. Rel.	$\delta{<}1.25$	EPE	Abs. Rel.	$\delta{<}1.25$	EPE	Abs. Rel.	$\delta{<}1.25$	EPE	Abs. Rel.	$\delta{<}1.25$
MASt3R ( Leroy et al., 2024 )	$1.8869$	$0.3141$	$66.5000$	$0.4870$	$0.1739$	$76.4857$	2.3036	0.0865	92.04	$3.9972$	$0.3659$	$61.0337$
MonST3R ( Zhang et al., 2025a )	$1.5038$	$0.1907$	$74.7522$	$0.1881$	0.0620	96.2170	$2.797$	$0.107$	$87.6071$	3.6067	$0.3405$	$57.6134$
DynaDUSt3R ( Jin et al., 2025 )	0.811	0.112	86.07	$0.4236$	$0.0840$	$92.7514$	$4.8892$	$0.1158$	$83.3925$	$4.3620$	0.3013	$61.0385$
ZeroMSF ( Liang et al., 2025b )	$1.6619$	$0.2200$	$76.1773$	$0.2077$	$0.0787$	$91.5787$	$3.3921$	$0.107$	92.2784	3.637	$0.3332$	63.8576
St4RTrack ( Feng et al., 2025 )	$1.4668$	$0.2093$	$71.2561$	0.181	$0.0674$	$95.1704$	$4.7307$	$0.1252$	$82.8568$	$4.1447$	$0.3574$	$57.1376$

Table 1: Geometry estimation : We report end-point error (EPE) for pointmap accuracy and absolute relative error (Abs. Rel.) and

\delta{<}1.25

for depth accuracy. Lower is better for EPE and Abs. Rel., and higher is better for

\delta{<}1.25

Method	Stereo4D				KITTI
	forward		backward		forward
	$\text{EPE}_{\text{3D}}\downarrow$	$\delta^{0.05}_{\text{3D}}\uparrow$	$\text{EPE}_{\text{3D}}\downarrow$	$\delta^{0.05}_{\text{3D}}\uparrow$	$\text{EPE}_{\text{3D}}\downarrow$	$\delta^{0.05}_{\text{3D}}\uparrow$
DynaDUSt3R ( Jin et al., 2025 )	$0.1751$	51.46	0.161	52.16	$0.4625$	$1.0925$
ZeroMSF ( Liang et al., 2025b )	0.164	$45.6312$	–	–	0.442	16.20
St4RTrack ( Feng et al., 2025 )	$0.3725$	$26.6157$	–	–	$0.7506$	$4.0082$
UFO-4D (Ours)	0.0488	83.0618	0.0500	82.6700	0.1372	47.7964

Table 2: Motion estimation. We evaluate the scene flow accuracy on the Stereo4D test split ( Jin et al., 2025 ) and KITTI Scene Flow 2015 Training ( Menze et al., 2018 ) . Our approach substantially outperforms others, achieving the best numbers on all metrics.

Panel of Figure 4 — Figure 4: Qualitative comparison of depth and projected 2D optical flow on Stereo4D, Bonn, and KITTI. For motion on KITTI, it visualizes motion relative to the camera, as GT is defined. Unlike DynaDUSt3R, ZeroMSF and St4RTrack, which suffers from residual motions in static region and inaccurate motion on object boundaries, UFO-4D exhibits clear motion boundaries and separation between moving objects and background. More qualitative results are in Section A.1 .

Method	Stereo4D ( Jin et al., 2025 )			Bonn ( Palazzolo et al., 2019 )			Sintel ( Butler et al., 2012 )
Method	ATE	$\text{RPE}_{\text{trans}}$	$\text{RPE}_{\text{rot}}$	ATE	$\text{RPE}_{\text{trans}}$	$\text{RPE}_{\text{rot}}$	ATE	$\text{RPE}_{\text{trans}}$	$\text{RPE}_{\text{rot}}$
MonST3R ( Zhang et al., 2025a )	$0.0458$	$0.0647$	$0.8047$	$0.0031$	$0.0043$	$0.5436$	$0.0290$	$0.0410$	$1.0796$
St4RTrack ( Feng et al., 2025 )	$0.0602$	$0.0851$	$0.7289$	$0.0024$	$0.0034$	$0.3997$	$0.0231$	$0.0326$	$0.8835$
UFO-4D (Ours)	0.0101	0.0142	0.1794	0.0020	0.0028	0.2709	0.0122	0.0172	0.2980

Table 3: Pose estimation. We report standard metrics, ATE and RPE translation (

\text{RPE}_{\text{trans}}

) and rotation (

\text{RPE}_{\text{rot}}

) for pose estimation on Stereo4D test, Bonn, and Sintel final, Our feed-forward approach outperforms methods relying on iterative solvers ( e. g . MonST3R, St4RTrack) on all metrics.

Figure 5: Opacity as learnable confidence : Opacity maps show the model’s behavior in a (dis)occlusion scenario. Our model learns to assign high confidence (opacity) to disoccluded regions, and for mutually-visible regions, it selects only one corresponding Gaussian from the two views, enabling an efficient and compact 4D representation.

Alias	Configuration	Image (PSNR $\uparrow$ )	Point (EPE $\downarrow$ )	Point rast. (EPE $\downarrow$ )	Motion (EPE $\downarrow$ )	Motion rast. (EPE $\downarrow$ )
(a)	Full model	23.929	0.827	0.841	0.069	0.064
(b)	W/o image gradient.	24.268	$0.903$	0.899	$0.072$	$0.070$
(c)	W/o motion and point rendering	$20.675$	$0.911$	$0.943$	0.071	0.069
(d)	W/o motion, point, and image rendering	–	0.886	–	0.071	–

Table 4: Loss ablation. We report image reconstruction in PSNR and accuracy of Gaussian center (Point), rasterized point (Point rast.), Gaussian velocity (Motion), and rasterized motion (Motion rast.) in end-point error (EPE). Integrating the photometric loss gradient ( Eq. 7c ) with all heads boosts overall performance. Losses on rendered motion ( Eq. 6b ) and point map ( Eq. 6c ) are crucial for achieving high accuracy.

Panel of Figure 6 — Figure 6: Qualitative comparison on loss ablation. Gradient backpropagation from the image synthesis loss and rendering losses on point and motion helps UFO-4D improve motion and point estimates, especially on object and motion boundaries.

Output representation	Equivalent or closest models	Stereo4D		KITTI		Bonn
Output representation	Equivalent or closest models	Motion EPE	Point EPE	Motion EPE	Point EPE	Point EPE
Dynamic 3DGS	UFO-4D (Ours)	$0.058$	0.781	0.244	3.553	$0.211$
Per-pixel point and motion	DynaDUSt3R, ZeroMSF	0.057	$0.809$	$0.283$	$4.567$	0.196
Per-pixel point	MonST3R	–	$0.804$	–	$4.562$	$0.201$

Table 5: Architecture comparison. For apples-to-apples comparison, we train other methods with different output representation on our training protocol. Our method achieves the best accuracy except for Bonn on both point and motion end-point errors (lower the better).

Figure 7: 4D interpolation on DAVIS. Given an input pair on the left, the righthand side shows rasterized images, depth, and motion maps at the canonical camera coordinate at interpolated time between the two input frames. Our method also clearly segments out moving objects.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 13 — Figure A: Qualitative comparison on Stereo4D. We visualize depth and projected 2D optical flow. UFO-4D consistently outperforms other methods in depth and motion accuracy. Notably, in scenes with significant camera rotation (top two rows), our method robustly estimates 3D geometry and isolates 3D motion of dynamic objects in the canonical space. In contrast, other methods struggle to disentangle camera ego-motion, resulting in residual motion artifacts on static background.

Panel of Figure 14 — Figure B: Qualitative comparison on KITTI. We visualize motion in camera coordinates, consistent with the ground truth of KITTI. Our method outperforms competitors with notably clearer motion boundaries, whereas ZeroMSF and St4RTrack frequently show erroneous motion estimation.

Panel of Figure 15 — Figure C: Qualitative comparison on Bonn. Comparing to other methods, our method outputs clear 3D motion disentangled from camera motion. ZeroMSF and St4RTrack suffer from non-zero residual motion in background. DynaDUSt3R tends to show unclear motion boundaries.

Panel of Figure 16 — Figure D: Visualization of annotation of Stereo4D. The first two columns visualize two input frames, and the last columns visualizes 3D motion annotation defined at the canonical camera coordinate. While Stereo4D provides large-scale real-world annotated data, the label includes noisy annotation, e. g . , non-zero 3D motion for static region.

Method	Stereo4D			Bonn			KITTI			Sintel final
Method	EPE	Abs. Rel.	$\delta{<}1.25$	EPE	Abs. Rel.	$\delta{<}1.25$	EPE	Abs. Rel.	$\delta{<}1.25$	EPE	Abs. Rel.	$\delta{<}1.25$
MASt3R ( Leroy et al., 2024 )	$1.6575$	$0.2897$	$65.5990$	$0.4839$	$0.1726$	$76.6771$	2.2467	0.0856	92.15	$3.9065$	$0.3595$	$60.6771$
MonST3R ( Zhang et al., 2025a )	$1.3798$	$0.1939$	$72.5653$	$0.1871$	0.0616	96.2591	$2.741$	$0.1057$	$87.9555$	3.5138	$0.3376$	$57.2272$
DynaDUSt3R ( Jin et al., 2025 )	0.683	0.103	87.94	$0.4230$	$0.0835$	$92.8303$	$4.8054$	$0.1125$	$84.0951$	$4.2958$	0.2946	$60.8804$
ZeroMSF ( Liang et al., 2025b )	$1.6737$	$0.2412$	$72.7655$	$0.2067$	$0.0782$	$91.6802$	$3.2652$	$0.103$	92.6609	3.560	$0.3266$	63.8047
St4RTrack ( Feng et al., 2025 )	$1.2691$	$0.1975$	$71.3659$	0.180	$0.0669$	$95.2358$	$4.6610$	$0.1236$	$83.1346$	$4.0965$	$0.3493$	$56.8702$

Appendix

Table A: Geometry estimation (under per-valid-pixel averaging protocol) : Using the same evaluation metric in DynaDUSt3R, our method consistently outperforms other methods.

Method	Stereo4D				KITTI
Method	forward		backward		forward
	$\text{EPE}_{\text{3D}}$	$\delta^{0.05}_{\text{3D}}$	$\text{EPE}_{\text{3D}}$	$\delta^{0.05}_{\text{3D}}$	$\text{EPE}_{\text{3D}}$	$\delta^{0.05}_{\text{3D}}$
DynaDUSt3R ( Jin et al., 2025 )	$0.118$	$65.19$	0.106	64.47	$0.467$	$1.09$
ZeroMSF ( Liang et al., 2025b )	0.067	67.38	–	–	0.452	13.84
St4RTrack ( Feng et al., 2025 )	$0.233$	$42.56$	–	–	$0.768$	$3.36$
UFO-4D (Ours)	0.023	91.78	0.025	91.28	0.139	46.71

Appendix

Table B: Motion estimation (under per-valid-pixel averaging protocol) We evaluate the scene flow accuracy on the Stereo4D test split ( Jin et al., 2025 ) and KITTI Scene Flow 2015 Training ( Menze et al., 2018 ) . Our approach consistently outperforms others on all metrics.

Dataset proportion			Stereo4D			KITTI			Bonn
ST	PO	VK	Image ( $\uparrow$ )	Point ( $\downarrow$ )	Motion ( $\downarrow$ )	Image ( $\uparrow$ )	Point ( $\downarrow$ )	Motion ( $\downarrow$ )	Image ( $\uparrow$ )	Point ( $\downarrow$ )
0.7	0.15	0.15	$23.547$	$0.065$	$0.891$	18.845	0.273	3.427	$18.568$	$0.314$
0.85	–	0.15	$23.419$	$0.065$	0.839	$18.791$	$0.298$	$4.102$	$18.056$	$0.344$
0.85	0.15	–	23.587	0.064	$0.874$	$18.037$	$0.355$	$6.214$	$18.610$	0.291
–	0.5	0.5	$21.918$	$0.074$	$1.292$	19.363	0.267	2.761	19.824	0.273
1.0	–	–	23.929	0.064	0.841	$18.610$	$0.311$	$6.138$	18.913	$0.316$

Appendix

Table C: Dataset ablation. Given three training datasets, Stereo4D (ST), PointOdyssey (PO), and Virtual KITTI 2 (VK), we try a different mixture of each dataset, train our model, and report performance on three evaluation datasets, Stere4D, KITTI, and Bonn, reporting PSNR for image reconstruction and end-point error (EPE) for point and motion. The best accuracy on Stereo4D is achieved by solely training on Stereo4D. Adding Virtual KITTI2 improves especially motion accuracy on KITTI but hurts point accuracy on Bonn. Adding PointOdyssey improves accuracy on Bonn but hurt that on KITTI as trade-off. All experiment is done in a small scale training.

Method	Stereo4D			KITTI			Bonn
Method	PSNR $\uparrow$	Point EPE $\downarrow$	Motion EPE $\downarrow$	PSNR $\uparrow$	Point EPE $\downarrow$	Motion EPE $\downarrow$	PSNR $\uparrow$	Point EPE $\downarrow$
Initialization from MASt3R (baseline)	24.519	0.781	0.058	19.606	$3.553$	0.244	26.606	$0.211$
Initialization from MonST3R	$24.200$	$0.868$	$0.059$	$19.506$	3.260	$0.258$	$26.551$	0.155

Appendix

Table D: Training with different initialization. We compare our model trained on different pretrained weights, report image PSNR, 3D point EPE, and 3D motion EPE on Stereo4D ( Jin et al., 2025 ) , KITTI ( Menze et al., 2018 ) , and Bonn ( Palazzolo et al., 2019 ) datasets. Initialization from MASt3R gives overall better accuracy whereas that from MonST3R improves 3D point accuracy on KITTI and Bonn.

Panel of Figure 21 — Figure E: Visualization of the scale of 3D Gaussian. For each dataset, KITTI, Bonn, and Stereo4D, we visualize Gaussian scale (the bottom row) along with an input image and predicted depth of our method. The Gaussian scale $(x,y,z)$ is visualized in the $(R,G,B)$ color space. For textureless region such as sky or wall, our method predicts larger scale of gaussian whereas small-scale 3D Gaussians are dominant in high-frequency texture region. Bright color means larger Gaussians.

Panel of Figure 22 — Figure F: Comparison with DynaDUSt3R on Bonn. Unlike per-pixel point representation, our representation gives subtle inaccuracy in geometry on textureless region ( e. g . wall). Brighter means higher errors.

Scale Range	Point (EPE) $\downarrow$	Depth (Abs. Rel.) $\downarrow$
$0-0.5$	0.189	0.073
$0.5-1.0$	0.151	0.059
$1.0-1.5$	0.200	0.068
$1.5-2.0$	0.189	0.067
$2.0-2.5$	0.185	0.071
$2.5-3.0$	0.205	0.085

Appendix

Table E: Impact of Gaussian scale on geometry accuracy on Bonn. We report Point EPE and Depth Abs. Rel. across different ranges of estimated Gaussian scales. We observe that depth accuracy decreases along with the increased scale of estimated 3D Gaussians.

Panel of Figure 24 — Figure G: Visualization of cross-attention maps of the camera pose token. We observe that at specific blocks (Blocks 8th, 11th, and 12th), the camera pose token more attends to image tokens located in static regions ( i. e . , high attention scores in brighter colors), which provide reliable cues for relative camera pose estimation. Moving objects tend to have darker colors, i. e . , lower attention scores. As a reference at each second row, we also visualize the cross-attention maps for a static scene configuration, where identical images are used as input.

Output representation	Pose estimator	Stereo4D		KITTI		Bonn
Output representation	Pose estimator	Motion EPE	Point EPE	Motion EPE	Point EPE	Point EPE
Dynamic 3DGS ( Ours )	Pose head	0.057	0.781	0.244	3.553	$0.211$
Per-pixel point and motion	Pose head	$0.060$	$0.811$	$0.245$	$4.396$	$0.206$
Per-pixel point and motion	PnP-RANSAC	$0.058$	$0.809$	$0.283$	$4.567$	0.196

Appendix

Table F: Ablation on Pose and Representation. We compare our method with per-pixel baselines using different pose estimators (pose head vs. PnP-RANSAC). Our Dynamic 3DGS representation with the feedforward pose head achieves the best accuracy on Stereo4D and KITTI.

Panel of Figure 26 — Figure H: Qualitative comparison of our method and baseline with per-pixel point representation on KITTI. Unlike per-pixel representation model, our method yields much better accuracy especially on planar road surface, benefiting from our accurate pose estimation and photometric losses on rendered images. In error maps, darker means more accurate.

Method	Stereo4D ( Jin et al., 2025 )			Bonn ( Palazzolo et al., 2019 )			Sintel ( Butler et al., 2012 )
Method	ATE	$\text{RPE}_{\text{trans}}$	$\text{RPE}_{\text{rot}}$	ATE	$\text{RPE}_{\text{trans}}$	$\text{RPE}_{\text{rot}}$	ATE	$\text{RPE}_{\text{trans}}$	$\text{RPE}_{\text{rot}}$
MonST3R ( Zhang et al., 2025a )	$0.0458$	$0.0647$	$0.8047$	$0.0031$	$0.0043$	$0.5436$	$0.0290$	$0.0410$	$1.0796$
St4RTrack ( Feng et al., 2025 )	$0.0602$	$0.0851$	$0.7289$	$0.0024$	$0.0034$	$0.3997$	$0.0231$	$0.0326$	$0.8835$
UFO-4D (Ours, with PnP+RANSAC)	$0.0120$	$0.0170$	$0.2052$	$0.0024$	$0.0033$	$0.2933$	$0.0168$	$0.0238$	$0.3315$
UFO-4D (Ours, with the pose head)	0.0101	0.0142	0.1794	0.0020	0.0028	0.2709	0.0122	0.0172	0.2980

Appendix

Table G: Pose estimation with PnP+RANSAC. Our method with PnP+RANSAC also outperforms direct competitors, validating the higher accuracy of our estimated geometry than others. Also our direct feedforward estimator achieves

\sim

16.6% higher accuracy than this baseline, further demonstrating its robustness over noise-sensitive iterative solvers. ATE and RPE translation (

\text{RPE}_{\text{trans}}

) and rotation (

\text{RPE}_{\text{rot}}

) are reported.

4D Reconstruction

4D reconstruction recovers changing 3D scenes, shapes, and motion over time from images, videos, or other sparse measurements.

See all 139 papers

Geometry-Aware Single-Image 4D Synthesis via Dense Trajectory Generation
Dec 4, 2025Yanran Zhang, Ziyi Wang, Wenzhao Zheng +3Video Diffusion Models 3D Scene Generation
Department of Automation, Tsinghua University, China · GigaAI
UniQuery4R: Unified 4D Scene Reconstruction from a Single Query
Aug 18, 2026Tiancheng Chen, Sheng Tang, Wenhua Jin +4Scene Flow Estimation Dynamic Scene Reconstruction
Kosmo Research · Automotive Engineering Department, Jilin University · School of Software, Tsinghua University
No Pose, No Problem in 4D: Feed-Forward Dynamic Gaussians from Unposed Multi-View Videos
May 21, 2026Matteo Balice, Yanik Kunzi, Chenyangguang Zhang +33D Gaussian Splatting 4D Reconstruction
Politecnico di Milano · ETH Zürich · ETH AI Center

Dec 4, 2025J/K move · Enter open · S save

Geometry-Aware Single-Image 4D Synthesis via Dense Trajectory Generation

Yanran Zhang, Ziyi Wang, Wenzhao Zheng, Zheng Zhu+2

Department of Automation, Tsinghua University, China · GigaAI

Video Diffusion Models 3D Scene Generation Temporal Consistency in Video Generation 4D Scene Generation

Open paper AskPDF arXiv

Generating interactive and dynamic 4D scenes from a single static image remains a core challenge. Most existing generate-then-reconstruct and reconstruct-then-generate methods decouple geometry from motion, causing spatiotemporal inconsistencies and poor generalization. To address these, we present MoGe4D (Motion and Geometry-Aware image-to-4D Synthesis), a geometry-conditioned framework for single-image 4D synthesis that models a scene as dense 4D point trajectories. Instead of treating geometry and dynamics as two disconnected stages, our method starts from an initial geometric prior inferred from the input image and predicts future time-varying trajectories in a diffusion process, improving spatiotemporal coherence while preserving structural stability. To support this task, we first introduce TrajScene-60K, a large-scale dataset of 60,000 video samples with dense 4D point trajectories, addressing the scarcity of high-quality training data for scene-level 4D generation. Built on this, our diffusion-based 4D Scene Trajectory Generator (4D-STraG) predicts geometry-consistent and motion-plausible trajectory fields conditioned on the input image, with a depth-guided motion normalization strategy to reduce scale ambiguity and a Motion Perception Module (MPM) to inject motion-aware priors. We further propose a 4D View Synthesis Module (4D-ViSM) to render the generated 4D representation into videos under arbitrary camera trajectories. Experiments show that MoGe4D produces high-quality 4D scenes with strong temporal coherence, favorable geometry-aware consistency, and compelling novel-view synthesis from a single image. Code: https://github.com/Zhangyr2022/MoGe4D.

Abstract

4D Reconstruction

Explore similar work

Geometry-Aware Single-Image 4D Synthesis via Dense Trajectory Generation

UniQuery4R: Unified 4D Scene Reconstruction from a Single Query

No Pose, No Problem in 4D: Feed-Forward Dynamic Gaussians from Unposed Multi-View Videos