MotionScale: Reconstructing Appearance, Geometry, and Motion of Dynamic Scenes with Scalable 4D Gaussian Splatting

Haoran Zhou; Gim Hee Lee

cs.CVMar 31, 2026

MotionScale: Reconstructing Appearance, Geometry, and Motion of Dynamic Scenes with Scalable 4D Gaussian Splatting

Authors: Haoran Zhou, Gim Hee Lee

Organizations: Department of Computer Science, National University of Singapore

Abstract

Realistic reconstruction of dynamic 4D scenes from monocular videos is essential for understanding the physical world. Despite recent progress in neural rendering, existing methods often struggle to recover accurate 3D geometry and temporally consistent motion in complex environments. To address these challenges, we propose MotionScale, a 4D Gaussian Splatting framework that scales efficiently to large scenes and extended sequences while maintaining high-fidelity structural and motion coherence. At the core of our approach is a scalable motion field parameterized by cluster-centric basis transformations that adaptively expand to capture diverse and evolving motion patterns. To ensure robust reconstruction over long durations, we introduce a progressive optimization strategy comprising two decoupled propagation stages: 1) A background extension stage that adapts to newly visible regions, refines camera poses, and explicitly models transient shadows; 2) A foreground propagation stage that enforces motion consistency through a specialized three-stage refinement process. Extensive experiments on challenging real-world benchmarks demonstrate that MotionScale significantly outperforms state-of-the-art methods in both reconstruction quality and temporal stability. Project page: https://hrzhou2.github.io/motion-scale-web/.

Figures & tables

Figure 2: Overview of MotionScale. Our method adopts a scalable motion field that progressively captures object motions through an adaptive control mechanism, enabling efficient splitting and refinement of motion components. For optimization, the background is updated through region sampling, camera refinement, and shadow handling, while the foreground propagation employs a three-stage refinement to propagate motion across long temporal windows for consistent 4D reconstruction.

Figure 3: Comparison of dynamic scene reconstruction results on challenging real-world videos from DAVIS dataset. We compare MotionScale with Shape of Motion [ 43 ] and GFlow [ 45 ] on several dynamic scenes containing complex object motions, occlusions, and large appearance variations. For the top rows, we show rendered results under two different viewpoints for each compared method.

Method	DyCheck NVS			NVIDIA NVS
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
T-NeRF [ 10 ]	15.60	0.55	0.55	20.76	0.59	0.17
HyperNeRF [ 30 ]	15.99	0.59	0.51	20.05	0.57	0.18
DynIBaR [ 22 ]	13.41	0.48	0.55	-	-	-
Deformable-GS [ 54 ]	11.92	0.49	0.66	-	-	-
4D Gaussians [ 49 ]	13.11	0.39	0.73	17.69	0.48	0.38

Table 1: Comparison of novel view synthesis results on DyCheck [ 10 ] and NVIDIA [ 55 ] datasets.

Method	3D Point Tracking			2D Point Tracking
Method	EPE $\downarrow$	$\delta_{\text{3D}}^{.05}\uparrow$	$\delta_{\text{3D}}^{.10}\uparrow$	AJ $\uparrow$	$<\delta_{\text{avg}}\uparrow$	OA $\uparrow$
HyperNeRF [ 30 ]	0.182	28.4	45.8	10.1	19.3	52.0
DynIBaR [ 22 ]	0.252	11.4	24.6	5.4	8.7	37.7
Deformable-GS [ 54 ]	0.151	33.4	55.3	14.0	20.9	63.9
DynMF [ 20 ]	0.188	22.9	53.8	5.5	9.5	60.5
CoTracker [ 16 ] +DA [ 51 ]	0.202	34.3	57.9	24.1	33.9	73.0

Table 2: Comparison of point-based tracking performance on the DyCheck [ 10 ] dataset.

Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	AJ $\uparrow$	$<\delta_{\text{avg}}\uparrow$	OA $\uparrow$
Ours (Full)	17.98	0.70	0.40	37.7	50.6	87.4
Global Bases	16.70	0.63	0.45	34.2	46.6	86.1
w/o Adaptive Control	17.21	0.67	0.42	34.9	47.0	86.6
w/o Pose Ref.	17.45	0.67	0.41	-	-	-
w/o Shadow	16.26	0.60	0.50	-	-	-
w/o FG Propagation	16.97	0.64	0.42	34.4	46.9	86.4

Table 3: Ablation studies on the DyCheck dataset.

Panel of Figure 4 — Figure 4: Visual comparison of ablation results.

Explore similar work

RiGS: Rigid-aware 4D Gaussian Splatting from a Single Monocular Video
May 22, 2026Chenyu Wu, Wanhua Li, Zhu-Tian Chen +13D Gaussian Splatting Monocular 3D Reconstruction
Harvard University · Zhejiang University · Nanyang Technological University +1
MoRe: Motion-aware Feed-forward 4D Reconstruction Transformer
Mar 5, 2026Juntong Fang, Zequn Chen, Weiqi Zhang +44D Reconstruction Dynamic Scene Reconstruction
School of Software, Tsinghua University, Beijing, China · Li Auto
Ground4D: Consistency-Aware 4D Reconstruction from Monocular Video
Jun 27, 2026Qing Zhao, Weijian Deng, Pengxu Wei +1Multi-View Consistency 3D Gaussian Splatting
School of Computer Science and Engineering, Sun Yat-sen University, China · Tsinghua Shenzhen International Graduate School, Tsinghua University, China

May 22, 2026J/K move · Enter open · S save

RiGS: Rigid-aware 4D Gaussian Splatting from a Single Monocular Video

Chenyu Wu, Wanhua Li, Zhu-Tian Chen, Hanspeter Pfister

Harvard University · Zhejiang University · Nanyang Technological University+1

3D Gaussian Splatting Monocular 3D Reconstruction Dynamic Scene Reconstruction Novel View Synthesis

Open paper AskPDF arXiv

Reconstructing dynamic 3D scenes from monocular videos is a fundamental yet highly challenging task, as real-world motions often involve both long-term smooth transformations and short-term complex deformations. Existing methods either struggle to maintain temporal consistency or fail to capture high-frequency dynamics due to limited motion modeling capacity. In this work, we present Rigid-aware 4D Gaussian Splatting (RiGS), which simultaneously captures motions across multiple temporal scales. Specifically, RiGS introduces three types of Gaussian primitives: static, rigid, and transient, which represent static backgrounds, long-term low-frequency motions, and short-term high-frequency dynamics, respectively. An object-wise dynamic mask is proposed to aggregate long-range spatiotemporal motion information and guide the decomposition of static and dynamic regions. To jointly model motion across scales, rigid Gaussians are allowed to transition into transient Gaussians based on their temporal duration, and both are optimized under scene flow guidance, providing dense 3D motion supervision. Extensive experiments demonstrate that RiGS achieves state-of-the-art performance on novel view synthesis benchmarks. Code is available at \hyperlink{https://github.com/ladvu/RiGS}{https://github.com/ladvu/RiGS}.

Abstract

Explore similar work

RiGS: Rigid-aware 4D Gaussian Splatting from a Single Monocular Video

MoRe: Motion-aware Feed-forward 4D Reconstruction Transformer

Ground4D: Consistency-Aware 4D Reconstruction from Monocular Video