MoRe: Motion-aware Feed-forward 4D Reconstruction Transformer

Juntong Fang; Zequn Chen; Weiqi Zhang; Donglin Di; Xuancheng Zhang; Chengmin Yang; Yu-Shen Liu

cs.CVMar 5, 2026

MoRe: Motion-aware Feed-forward 4D Reconstruction Transformer

Authors: Juntong Fang, Zequn Chen, Weiqi Zhang, Donglin Di, Xuancheng Zhang, Chengmin Yang, Yu-Shen Liu

Organizations: School of Software, Tsinghua University, Beijing, China · Li Auto

Abstract

Reconstructing dynamic 4D scenes remains challenging due to the presence of moving objects that corrupt camera pose estimation. Existing optimization methods alleviate this issue with additional supervision, but they are mostly computationally expensive and impractical in real-time applications. To address these limitations, we propose MoRe, a feedforward 4D reconstruction network that efficiently recovers dynamic 3D scenes from monocular videos. Built upon a strong static reconstruction backbone, MoRe employs an attention-forcing strategy to disentangle dynamic motion from static structure. To further enhance robustness, we fine-tune the model on large-scale, diverse datasets encompassing both dynamic and static scenes. Moreover, our grouped causal attention captures temporal dependencies and adapts to varying token lengths across frames, ensuring temporally coherent geometry reconstruction. Extensive experiments on multiple benchmarks demonstrate that MoRe achieves high-quality dynamic reconstructions with exceptional efficiency.

Figures & tables

Figure 2: Method Overview. During training, an attention-forcing mechanism aligns the attention weights with ground-truth motion masks, enabling the model to effectively disentangle dynamic motion from static scene structure. For streaming reconstruction task, MoRe is based on a causal transformer where global attention is replaced by aggregated causal attention.

Figure 3: Attention Map Visualization. We visualize the attention map of the camera token within VGGT [ 39 ] and observe that the model tends to confuse moving objects with static background regions, which accounts for the degradation in prediction accuracy.

Figure 4: Grouped Causal Attention. Unlike traditional causal attention, our design allows image tokens within the same frame to attend to each other regardless of their ordering. This formulation enables the model to preserve causal temporal reasoning while maintaining spatial consistency within each frame.

Figure 5: Streaming Inference pipeline. Leveraging causal attention, our model can efficiently process streaming input in an online manner. To enhance camera pose accuracy, we apply a bundle-adjustment-like post-processing step after the entire sequence has been processed. Specifically, for each frame, we duplicate the camera token and perform inference again using the previously cached key-value pairs.

Figure 6: Qualititive Comparison of Our Full Attention Model with Other Methods. MoRe delivers outstanding performance in real-world scenes, outperforming other methods through its precise and robust geometry estimation.

		Sintel			Bonn			TUM-dynamics			ScanNet (static)
Method	type	ATE $\downarrow$	RPE ${}_{\text{trans}}$ $\downarrow$	RPE ${}_{\text{rot}}$ $\downarrow$	ATE $\downarrow$	RPE ${}_{\text{trans}}$ $\downarrow$	RPE ${}_{\text{rot}}$ $\downarrow$	ATE $\downarrow$	RPE ${}_{\text{trans}}$ $\downarrow$	RPE ${}_{\text{rot}}$ $\downarrow$	ATE $\downarrow$	RPE ${}_{\text{trans}}$ $\downarrow$	RPE ${}_{\text{rot}}$ $\downarrow$
MapAnything [ 16 ]	FA	0.2104	0.0919	2.7396	0.0248	0.0132	0.6927	0.0244	0.0161	0.3871	0.0603	0.0280	0.8490
VGGT [ 39 ]	FA	0.1715	0.0617	0.4695	0.0141	0.0100	0.6323	0.0109	0.0092	0.2992	0.0347	0.0151	0.3758
MoRe(FA)	FA	0.0877	0.0580	0.3899	0.0138	0.0099	0.6267	0.0115	0.0088	0.2980	0.0375	0.0147	0.3847
Spann3R [ 38 ]	Streaming	0.3313	0.1111	4.4952	0.0344	0.0212	2.2539	0.0421	0.0333	0.8120	0.0651	0.0339	0.9348
CUT3R [ 40 ]	Streaming	0.2163	0.0756	0.6518	0.0420	0.0094	0.6825	0.0438	0.0134	0.4210	0.0929	0.0223	0.5811

Table 1: Camera Pose Estimation on Sintel [ 3 ] , TUM-dynamics [ 33 ] , Bonn [ 27 ] , and ScanNet [ 5 ] datasets. FA refers to full attention.

Figure 7: Qualitative Comparison of our Stream Model with Other Methods. MoRe strikes an optimal balance between reconstruction quality and computational efficiency, delivering high-fidelity results at competitive inference speeds.

		Sintel		Bonn		TUM-dynamics		kitti
Method	type	Abs Rel $\downarrow$	$\delta<1.25\uparrow$	Abs Rel $\downarrow$	$\delta<1.25\uparrow$	Abs Rel $\downarrow$	$\delta<1.25\uparrow$	Abs Rel $\downarrow$	$\delta<1.25\uparrow$
Flare [ 51 ]	FA	0.729	0.336	0.116	0.897	0.152	0.790	0.356	0.570
MapAnything [ 16 ]	FA	0.632	0.461	0.125	0.859	0.241	0.813	0.277	0.742
VGGT [ 39 ]	FA	0.387	0.584	0.055	0.970	0.132	0.901	0.073	0.961
MoRe(FA)	FA	0.335	0.645	0.055	0.971	0.120	0.902	0.066	0.967
Spann3R [ 38 ]	Streaming	0.740	0.364	0.229	0.729	0.264	0.645	0.422	0.358

Table 2: Video Depth Estimation on Sintel [ 3 ] , TUM-dynamics [ 33 ] , Bonn [ 27 ] and kitti [ 12 ] .

Method	ATE $\downarrow$	RPE ${}_{\text{trans}}$ $\downarrow$	RPE ${}_{\text{rot}}$ $\downarrow$	ATE $\downarrow$	RPE ${}_{\text{trans}}$ $\downarrow$	RPE ${}_{\text{rot}}$ $\downarrow$
	Sintel			TUM-dynamics
w/o attention forcing	0.163	0.092	0.660	0.028	0.014	0.329
w/o BA-like refinement	0.155	0.085	0.619	0.027	0.014	0.321
full	0.147	0.082	0.616	0.026	0.013	0.320

Table 3: Ablation on Camera Pose Estimation

Method	Abs Rel $\downarrow$	$\delta<1.25\uparrow$	Abs Rel $\downarrow$	$\delta<1.25\uparrow$	Abs Rel $\downarrow$	$\delta<1.25\uparrow$
	Sintel		Bonn		kitti
w/o GCA	0.277	0.592	0.070	0.953	0.079	0.949
w/	0.254	0.637	0.068	0.961	0.072	0.966

Table 4: Ablation on Video Depth Estimation

Figure 8: Qualitative Results of Motion Mask Extraction. Our method robustly captures moving objects across diverse scenes and objects.

Method	FPS
VGGT [ 39 ]	7.32
Spann3R [ 38 ] (224×224)	13.55
CUT3R [ 40 ]	16.58
Stream3R α [ 17 ]	23.48
Stream3R β [ 17 ]	12.95
Stream3R β -W[5] [ 17 ] (window=5)	32.93

Table 5: Inference speed comparison (FPS), tested on KITTI [ 12 ] .

Figure 9: Qualitative Comparison of Our Model with Other Methods. We present an extensive set of visual results showcasing reconstruction quality, motion handling, and robustness under challenging dynamic scenarios.

Method	Co3Dv2
	RRA@30 $\uparrow$	RTA@30 $\uparrow$	AUC@30 $\uparrow$
Fast3R [ 46 ]	97.49	91.11	73.43
CUT3R [ 40 ]	96.19	92.69	75.82
FLARE [ 51 ]	96.38	93.76	73.99
VGGT [ 39 ]	98.96	97.13	88.59
$\pi^{3}$ [ 43 ]	99.05	97.33	88.41

Table 6: Camera Pose Estimation Comparison on Co3Dv2 [ 30 ] .

Figure 10: Attention Map Comparison. We visualize the attention map on Dynamic Replica [ 34 ] and DAVIS [ 28 ] dataset. Our motion-aligned training suppresses undesired attention from camera tokens to dynamic objects, yielding cleaner and more structured attention patterns.

	Sintel			TUM-dynamics
Method	ATE $\downarrow$	RPE ${}_{\text{trans}}$ $\downarrow$	RPE ${}_{\text{rot}}$ $\downarrow$	ATE $\downarrow$	RPE ${}_{\text{trans}}$ $\downarrow$	RPE ${}_{\text{rot}}$ $\downarrow$
w/ KL loss	0.185	0.084	0.707	0.029	0.015	0.350
Ours	0.147	0.082	0.616	0.026	0.013	0.320

Table 7: Ablation on Loss Function for Motion Alignment.

Explore similar work

4DVGGT-D: 4D Visual Geometry Transformer with Improved Dynamic Depth Estimation
May 12, 2026Ying Zang, Xuanyi Liu, Yidong Han +94D Reconstruction Depth Estimation
4Huzhou University · 2Peking University · 1KOKONI 3D, Moxin Technology +4
MotionScale: Reconstructing Appearance, Geometry, and Motion of Dynamic Scenes with Scalable 4D Gaussian Splatting
Mar 31, 2026Haoran Zhou, Gim Hee Lee4D Reconstruction Dynamic Scene Reconstruction
Department of Computer Science, National University of Singapore
Robust 4D Visual Geometry Transformer with Uncertainty-Aware Priors
Apr 10, 2026Ying Zang, Yidong Han, Chaotao Ding +8Dynamic Scene Reconstruction 4D Reconstruction
Huzhou University · KOKONI 3D, Moxin Technology · Univeristy of Science and Technology of China +3

May 12, 2026J/K move · Enter open · S save

4DVGGT-D: 4D Visual Geometry Transformer with Improved Dynamic Depth Estimation

Ying Zang, Xuanyi Liu, Yidong Han, Deyi Ji+8

4Huzhou University · 2Peking University · 1KOKONI 3D, Moxin Technology+4

4D Reconstruction Depth Estimation Monocular 3D Reconstruction Representation Disentanglement Dynamic Scene Reconstruction

Open paper AskPDF arXiv

Reconstructing dynamic 4D scenes from monocular videos is a fundamental yet challenging task. While recent 3D foundation models provide strong geometric priors, their performance significantly degrades in dynamic environments. This degradation stems from a fundamental tension: the inherent coupling of camera ego-motion and object motion within global attention mechanisms. In this paper, we propose a novel, training-free progressive decoupling framework that disentangles dynamics from statics in a principled, coarse-to-fine manner. Our core insight is to resolve the tension by first stabilizing the camera pose, followed by geometric refinement. Specifically, our approach consists of three synergistic components: (1) a Dynamic-Mask-Guided Pose Decoupling module that isolates pose estimation from dynamic interference, yielding a stable motion-free reference frame; (2) a Topological Subspace Surgery mechanism that orthogonally decomposes the depth manifold, safely preserving dynamic objects while injecting refined, mask-aware geometry into static regions; and (3) an Information-Theoretic Confidence-Aware Fusion strategy that formulates depth integration as a heteroscedastic Bayesian inference problem, adaptively blending multi-pass predictions via inverse-variance weighting. Extensive experiments on standard 4D reconstruction benchmarks demonstrate that our method achieves consistent and substantial improvements across principal point-cloud metrics. Notably, our approach shows competitive performance in robust 4D scene reconstruction without requiring fine-tuning, suggesting the potential of mathematically grounded dynamic-static disentanglement.

Abstract

Explore similar work

4DVGGT-D: 4D Visual Geometry Transformer with Improved Dynamic Depth Estimation

MotionScale: Reconstructing Appearance, Geometry, and Motion of Dynamic Scenes with Scalable 4D Gaussian Splatting

Robust 4D Visual Geometry Transformer with Uncertainty-Aware Priors