This paper addresses the problem of dynamic scene surface reconstruction using Gaussian Splatting (GS), aiming to recover temporally consistent geometry. While existing GS-based dynamic surface reconstruction methods can yield superior reconstruction, they are typically limited to either a single object or objects with only small deformations, struggling to maintain temporally consistent surface reconstruction of large deformations over time. We propose ``4DSurf'', a novel and unified framework for generic dynamic surface reconstruction that does not require specifying the number or types of objects in the scene, can handle large surface deformations and temporal inconsistency in reconstruction. The key innovation of our framework is the introduction of Gaussian deformations induced Signed Distance Function Flow Regularization that constrains the motion of Gaussians to align with the evolving surface. To handle large deformations, we introduce an Overlapping Segment Partitioning strategy that divides the sequence into overlapping segments with small deformations and incrementally passes geometric information across segments through the shared overlapping timestep. Experiments on two challenging dynamic scene datasets, Hi4D and CMU Panoptic, demonstrate that our method outperforms state-of-the-art surface reconstruction methods by 49% and 19% in Chamfer distance, respectively, and achieves superior temporal consistency under sparse-view settings.
Figures & tables
Figure 1: Left : Using only a sparse set of input videos. Right : Our approach sets a new state-of-the-art benchmark in surface reconstruction for dynamic scenes on the CMU Panoptic dataset [ 13 ] , compared with recent dynamic surface reconstruction methods (Neural SDF-Flow [ 26 ] , Sparse2DGS [ 46 ] , GauSTAR [ 56 ] , D-2DGS [ 55 ] , and ST-2DGS [ 42 ] ).
Figure 2: Overview: (a) Overall Training Pipeline. We first divide the sequence into N segments, each containing K+1 timesteps with one overlapping virtual timestep. For the 1st segment, the initialization is derived from the visual hull reconstructed from all frames of its first timestep. After training the first segment, the Gaussians of virtual timestep serves as the initialization for training the next segment. Each segment maintains its own canonical space and Gaussian Velocity Field. (b) Gaussian Velocity Field. Given the Gaussian center μi in the canonical space and a specific timestep t , the Gaussian Velocity Field Fθ(⋅) predicts its velocity v(μi,t) , angular velocity ω(μi,t) and expansion velocity e(μi,t) at timestep t . These are then converted to position μit , rotation qit , and scale ξit , which are fed into the differentiable rasterizer for image rendering. (c) SDF Approximation. Following previous works [ 7 , 28 ] , we compute the distance between the center and its corresponding depth point to estimate the signed distance.
Figure 3: Incremental Motion Tuning (IMT). After training the Gaussian Velocity Field of the 1st segment, for the later Nth segment ( N≥2 ), its Gaussian Velocity Field FθN is initialized from the θN−1 of previous segment and fine-tuned using LoRA ΔθN .
Methods
Band1
Ian3
Haggling b2
Pizza1
Acc ↓
Comp ↓
Overall ↓
Acc ↓
Comp ↓
Overall ↓
Acc ↓
Comp ↓
Overall ↓
Acc ↓
Comp ↓
Overall ↓
NDR [ 2 ]
15.9
23.7
19.8
21.8
20.7
21.3
12.5
22.8
17.7
17.7
25.0
21.3
Tensor4D [ 35 ]
17.1
29.2
23.2
15.4
22.8
19.1
13.7
25.3
19.5
18.3
23.5
22.9
Neural SDF-Flow [ 26 ]
13.0
21.4
17.2
14.1
17.5
15.8
8.3
18.6
13.5
11.5
20.6
16.1
4DGS [ 45 ]
12.4
22.4
17.3
8.8
17.2
13.0
9.0
19.9
14.4
12.1
22.1
17.1
SC-GS [ 10 ]
12.2
22.2
17.2
8.4
17.3
12.8
8.3
19.2
13.8
11.7
22.1
16.9
Table 1: CMU Panoptic [ 13 ] comparisons. We evaluate performance with Chamfer Distance (unit: mm). The top three results for each metric are highlighted with — , — , and — , respectively. Ours consistently achieves the best performance on the Overall metric.
Cheers37
Talk22
Football18
Fight17
Basketball13
Backhug02
Methods
Acc ↓
Comp ↓
Overall ↓
Acc ↓
Comp ↓
Overall ↓
Acc ↓
Comp ↓
Overall ↓
Acc ↓
Comp ↓
Overall ↓
Acc ↓
Comp ↓
Overall ↓
Acc ↓
Comp ↓
Overall ↓
4DGS [ 45 ]
3.05
1.43
2.24
3.73
3.75
3.74
3.05
2.22
2.63
2.53
2.94
2.73
2.96
2.53
2.74
3.05
5.36
4.21
SC-GS [ 10 ]
1.58
0.97
1.27
1.62
3.21
2.41
1.47
1.81
1.64
1.58
2.65
2.11
1.72
1.90
1.81
1.87
5.16
3.51
FreeTimeGS [ 43 ]
2.34
1.22
1.78
3.21
2.21
2.71
2.20
1.64
1.92
3.71
2.14
2.93
3.04
3.10
3.07
2.61
3.81
3.21
Sparse2DGS [ 46 ]
2.66
0.96
1.81
3.21
1.86
2.54
1.94
0.98
1.46
1.27
0.75
1.01
4.87
2.10
3.48
2.00
1.59
1.79
GauSTAR [ 56 ]
2.10
2.52
2.31
2.74
1.96
2.35
2.74
0.49
1.62
3.32
2.25
2.79
4.15
1.04
2.59
2.18
1.99
2.09
Table 2: Hi4D [ 54 ] comparisons. We evaluate performance using Chamfer Distance (unit: cm). The top three results for each metric are highlighted in — , — , and — , respectively. Ours significantly outperforms all baselines on the Overall metric.
Figure 4: Qualitative results on CMU Panoptic [ 13 ] . We compare our methods with three baselines (Dynamic-2DGS [ 55 ] , Sparse2DGS [ 46 ] , FreeTimeGS [ 43 ] ) at two timesteps of the Band1 and Ian3 scene. Bounding boxes highlight major differences.
Figure 5: Qualitative results on Hi4D [ 54 ] . We compare our methods with three baselines (Dynamic-2DGS [ 55 ] , Sparse2DGS [ 46 ] , FreeTimeGS [ 43 ] ) at two timesteps of the Basketball13 and Fight17 scene. Bounding boxes highlight major differences.
GVF
SF-Reg
I-Segment
O-Segment
IMT-64
Acc ↓
Comp ↓
Overall ↓
(a)
√
1.75
1.23
1.49
(b)
√
√
1.18
0.86
1.02
(c)
√
√
√
1.06
0.47
0.77
(d)
√
√
√
0.89
0.45
0.67
(e)
√
√
√
√
0.91
0.48
0.70
Table 3: Ablation Studies on the Hi4D dataset [ 54 ] . We calculate the average of the three metrics (Acc, Comp, and Overall) for the six scenes. GVF: Gaussian Velocity Field. SF-Reg: SDF-Flow Regularization. I-Segment: Independent Segment Partitioning. O-Segment: Overlapping Segment Partitioning. IMT-64: Incremental Motion Tuning with LoRA rank 64.
Figure 6: Qualitative comparison of each ablation study on scene Cheers37 . Subscripts correspond to rows in Tab. 3 . ( a ): Gaussian Velocity Field with normal & depth regularization from [ 9 ] , photorealistic and mask losses. ( b ): Add SDF Flow Regularization based on ( a ). ( c ): Training ( b ) with Independent Segment Partitioning. ( d ): Training ( b ) with Overlapping Segment Partitioning. ( e ): Adopting Incremental Motion Tuning (LoRA rank 64) on ( d ).
Methods
Acc STD ↓
Comp STD ↓
Overall STD ↓
Dynamic-2DGS [ 55 ]
1.50
1.30
1.19
GauSTAR [ 56 ]
0.53
4.39
2.70
Sparse2DGS [ 46 ]
0.95
0.50
0.68
Ours w IMT-64
0.28
0.37
0.28
Ours wo IMT
0.22
0.17
0.18
Table 4: Temporal stability comparison on the Hi4D dataset [ 54 ] . STD: standard deviation. It shows the average STD of the three metrics (Acc, Comp, and Overall) across six scenes. The best and the second-best are highlighted in bold and underlined .
Method
Acc ↓
Comp ↓
Overall ↓
Ours w IMT-16
1.07
0.70
0.89
Ours w IMT-32
1.00
0.64
0.82
Ours w IMT-64
0.91
0.48
0.70
Ours wo IMT
0.89
0.45
0.67
Table 5: Different LoRA ranks comparison on Hi4D dataset [ 54 ] . It shows the average of the three metrics (Acc, Comp, and Overall) in six scenes under different LoRA ranks.
Figure 7: LoRA rank and storage analysis on scene Backhug02 . Left : As the number of segments increases, Ours wo IMT exhibits growing storage of Gaussian velocity fields, while Ours w IMT- { LoRA-rank } effectively curb the storage growth. Right : For different LoRA ranks, Ours w IMT maintains strong performance (even at rank 16), achieving competitive results to Ours wo IMT .
While novel view synthesis (NVS) for dynamic scenes has seen significant progress, reconstructing temporally consistent geometric surfaces remains a challenge. Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) offer powerful dynamic scene rendering capabilities; however, relying solely on photometric optimization often leads to geometric ambiguities. This results in discontinuous surfaces, severe artifacts, and broken surfaces over time. To address these limitations, we present DySurface, a novel framework that bridges the effectiveness of explicit Gaussians with the geometric fidelity of implicit Signed Distance Functions (SDFs) in dynamic scenes. Our approach tackles the structural discrepancy between the forward deformation of 3DGS (canonical→dynamic) and the backward deformation required for volumetric SDF rendering (dynamic→canonical). Specifically, we propose the VoxGS-DSDF branch that leverages deformed Gaussians to construct a dynamic sparse voxel grid, providing explicit geometric guidance to the implicit SDF field. This explicit anchoring effectively regularizes the volumetric rendering process, significantly improving surface reconstruction quality, with watertight boundaries and detailed representations. Quantitative and qualitative experiments demonstrate that DySurface significantly outperforms state-of-the-art baselines in geometric accuracy while maintaining competitive rendering performance.
Reconstructing dynamic 3D scenes with photorealistic detail and strong temporal coherence remains a significant challenge. Existing Gaussian splatting approaches for dynamic scene modeling often rely on per-frame optimization, which can overfit to instantaneous states instead of capturing underlying motion dynamics. To address this, we present Mango-GS, a multi-frame, node-guided framework for high-fidelity 4D reconstruction. Mango-GS leverages a temporal Transformer to model motion dependencies within a short window of frames, producing temporally consistent deformations. For efficiency, temporal modeling is confined to a sparse set of control nodes. Each node is represented by a decoupled canonical position and a latent code, providing a stable semantic anchor for motion propagation and preventing correspondence drift under large motion. Our framework is trained end-to-end, enhanced by an input masking strategy and two multi-frame losses to improve robustness. Extensive experiments demonstrate that Mango-GS achieves state-of-the-art reconstruction quality and real-time rendering speed, enabling high-fidelity reconstruction and interactive rendering of dynamic scenes.
Dynamic scene reconstruction and novel view synthesis are fundamental to next-generation visual intelligence applications such as virtual reality, robotics, and digital twins. However, high-fidelity reconstruction of complex, time-varying scenes from arbitrary viewpoints remains a significant challenge. Existing dynamic 3DGS methods suffer from computational inefficiency, since they model all Gaussians as dynamic components. While recent decomposition-based approaches address this issue, they still struggle with degraded reconstruction quality and prolonged training time. To mitigate these limitations, we propose a novel dynamic reconstruction framework built upon an efficient static-dynamic decomposition strategy using a Feed-Forward Gaussian Splatting encoder and an optical flow model. By eliminating redundant computations on static regions, our method achieves state-of-the-art performance, outperforming existing baselines across rendering quality, training and rendering speed, and storage efficiency. Notably, on the Neural 3D dataset, our framework requires only 10 minutes for training and achieves a rendering speed of over 700 FPS on a single NVIDIA RTX 5090 GPU at resolution of 1352x1014. Furthermore, our decomposition strategy eliminates the need for COLMAP preprocessing and enables deterministic initialization, thereby enhancing both efficiency and reproducibility.
Youngtae Han, Sung-hwan Han, Youngmin Yi
Department of Artificial Intelligence Engineering, Sogang University