This paper addresses the problem of dynamic scene surface reconstruction using Gaussian Splatting (GS), aiming to recover temporally consistent geometry. While existing GS-based dynamic surface reconstruction methods can yield superior reconstruction, they are typically limited to either a single object or objects with only small deformations, struggling to maintain temporally consistent surface reconstruction of large deformations over time. We propose ``4DSurf'', a novel and unified framework for generic dynamic surface reconstruction that does not require specifying the number or types of objects in the scene, can handle large surface deformations and temporal inconsistency in reconstruction. The key innovation of our framework is the introduction of Gaussian deformations induced Signed Distance Function Flow Regularization that constrains the motion of Gaussians to align with the evolving surface. To handle large deformations, we introduce an Overlapping Segment Partitioning strategy that divides the sequence into overlapping segments with small deformations and incrementally passes geometric information across segments through the shared overlapping timestep. Experiments on two challenging dynamic scene datasets, Hi4D and CMU Panoptic, demonstrate that our method outperforms state-of-the-art surface reconstruction methods by 49% and 19% in Chamfer distance, respectively, and achieves superior temporal consistency under sparse-view settings.
Figures & tables
Figure 1: Left : Using only a sparse set of input videos. Right : Our approach sets a new state-of-the-art benchmark in surface reconstruction for dynamic scenes on the CMU Panoptic dataset [ 13 ] , compared with recent dynamic surface reconstruction methods (Neural SDF-Flow [ 26 ] , Sparse2DGS [ 46 ] , GauSTAR [ 56 ] , D-2DGS [ 55 ] , and ST-2DGS [ 42 ] ).
Figure 2: Overview: (a) Overall Training Pipeline. We first divide the sequence into N segments, each containing K+1 timesteps with one overlapping virtual timestep. For the 1st segment, the initialization is derived from the visual hull reconstructed from all frames of its first timestep. After training the first segment, the Gaussians of virtual timestep serves as the initialization for training the next segment. Each segment maintains its own canonical space and Gaussian Velocity Field. (b) Gaussian Velocity Field. Given the Gaussian center μi in the canonical space and a specific timestep t , the Gaussian Velocity Field Fθ(⋅) predicts its velocity v(μi,t) , angular velocity ω(μi,t) and expansion velocity e(μi,t) at timestep t . These are then converted to position μit , rotation qit , and scale ξit , which are fed into the differentiable rasterizer for image rendering. (c) SDF Approximation. Following previous works [ 7 , 28 ] , we compute the distance between the center and its corresponding depth point to estimate the signed distance.
Figure 3: Incremental Motion Tuning (IMT). After training the Gaussian Velocity Field of the 1st segment, for the later Nth segment ( N≥2 ), its Gaussian Velocity Field FθN is initialized from the θN−1 of previous segment and fine-tuned using LoRA ΔθN .
Methods
Band1
Ian3
Haggling b2
Pizza1
Acc ↓
Comp ↓
Overall ↓
Acc ↓
Comp ↓
Overall ↓
Acc ↓
Comp ↓
Overall ↓
Acc ↓
Comp ↓
Overall ↓
NDR [ 2 ]
15.9
23.7
19.8
21.8
20.7
21.3
12.5
22.8
17.7
17.7
25.0
21.3
Tensor4D [ 35 ]
17.1
29.2
23.2
15.4
22.8
19.1
13.7
25.3
19.5
18.3
23.5
22.9
Neural SDF-Flow [ 26 ]
13.0
21.4
17.2
14.1
17.5
15.8
8.3
18.6
13.5
11.5
20.6
16.1
4DGS [ 45 ]
12.4
22.4
17.3
8.8
17.2
13.0
9.0
19.9
14.4
12.1
22.1
17.1
SC-GS [ 10 ]
12.2
22.2
17.2
8.4
17.3
12.8
8.3
19.2
13.8
11.7
22.1
16.9
Table 1: CMU Panoptic [ 13 ] comparisons. We evaluate performance with Chamfer Distance (unit: mm). The top three results for each metric are highlighted with — , — , and — , respectively. Ours consistently achieves the best performance on the Overall metric.
Cheers37
Talk22
Football18
Fight17
Basketball13
Backhug02
Methods
Acc ↓
Comp ↓
Overall ↓
Acc ↓
Comp ↓
Overall ↓
Acc ↓
Comp ↓
Overall ↓
Acc ↓
Comp ↓
Overall ↓
Acc ↓
Comp ↓
Overall ↓
Acc ↓
Comp ↓
Overall ↓
4DGS [ 45 ]
3.05
1.43
2.24
3.73
3.75
3.74
3.05
2.22
2.63
2.53
2.94
2.73
2.96
2.53
2.74
3.05
5.36
4.21
SC-GS [ 10 ]
1.58
0.97
1.27
1.62
3.21
2.41
1.47
1.81
1.64
1.58
2.65
2.11
1.72
1.90
1.81
1.87
5.16
3.51
FreeTimeGS [ 43 ]
2.34
1.22
1.78
3.21
2.21
2.71
2.20
1.64
1.92
3.71
2.14
2.93
3.04
3.10
3.07
2.61
3.81
3.21
Sparse2DGS [ 46 ]
2.66
0.96
1.81
3.21
1.86
2.54
1.94
0.98
1.46
1.27
0.75
1.01
4.87
2.10
3.48
2.00
1.59
1.79
GauSTAR [ 56 ]
2.10
2.52
2.31
2.74
1.96
2.35
2.74
0.49
1.62
3.32
2.25
2.79
4.15
1.04
2.59
2.18
1.99
2.09
Table 2: Hi4D [ 54 ] comparisons. We evaluate performance using Chamfer Distance (unit: cm). The top three results for each metric are highlighted in — , — , and — , respectively. Ours significantly outperforms all baselines on the Overall metric.
Figure 4: Qualitative results on CMU Panoptic [ 13 ] . We compare our methods with three baselines (Dynamic-2DGS [ 55 ] , Sparse2DGS [ 46 ] , FreeTimeGS [ 43 ] ) at two timesteps of the Band1 and Ian3 scene. Bounding boxes highlight major differences.
Figure 5: Qualitative results on Hi4D [ 54 ] . We compare our methods with three baselines (Dynamic-2DGS [ 55 ] , Sparse2DGS [ 46 ] , FreeTimeGS [ 43 ] ) at two timesteps of the Basketball13 and Fight17 scene. Bounding boxes highlight major differences.
GVF
SF-Reg
I-Segment
O-Segment
IMT-64
Acc ↓
Comp ↓
Overall ↓
(a)
√
1.75
1.23
1.49
(b)
√
√
1.18
0.86
1.02
(c)
√
√
√
1.06
0.47
0.77
(d)
√
√
√
0.89
0.45
0.67
(e)
√
√
√
√
0.91
0.48
0.70
Table 3: Ablation Studies on the Hi4D dataset [ 54 ] . We calculate the average of the three metrics (Acc, Comp, and Overall) for the six scenes. GVF: Gaussian Velocity Field. SF-Reg: SDF-Flow Regularization. I-Segment: Independent Segment Partitioning. O-Segment: Overlapping Segment Partitioning. IMT-64: Incremental Motion Tuning with LoRA rank 64.
Figure 6: Qualitative comparison of each ablation study on scene Cheers37 . Subscripts correspond to rows in Tab. 3 . ( a ): Gaussian Velocity Field with normal & depth regularization from [ 9 ] , photorealistic and mask losses. ( b ): Add SDF Flow Regularization based on ( a ). ( c ): Training ( b ) with Independent Segment Partitioning. ( d ): Training ( b ) with Overlapping Segment Partitioning. ( e ): Adopting Incremental Motion Tuning (LoRA rank 64) on ( d ).
Methods
Acc STD ↓
Comp STD ↓
Overall STD ↓
Dynamic-2DGS [ 55 ]
1.50
1.30
1.19
GauSTAR [ 56 ]
0.53
4.39
2.70
Sparse2DGS [ 46 ]
0.95
0.50
0.68
Ours w IMT-64
0.28
0.37
0.28
Ours wo IMT
0.22
0.17
0.18
Table 4: Temporal stability comparison on the Hi4D dataset [ 54 ] . STD: standard deviation. It shows the average STD of the three metrics (Acc, Comp, and Overall) across six scenes. The best and the second-best are highlighted in bold and underlined .
Method
Acc ↓
Comp ↓
Overall ↓
Ours w IMT-16
1.07
0.70
0.89
Ours w IMT-32
1.00
0.64
0.82
Ours w IMT-64
0.91
0.48
0.70
Ours wo IMT
0.89
0.45
0.67
Table 5: Different LoRA ranks comparison on Hi4D dataset [ 54 ] . It shows the average of the three metrics (Acc, Comp, and Overall) in six scenes under different LoRA ranks.
Figure 7: LoRA rank and storage analysis on scene Backhug02 . Left : As the number of segments increases, Ours wo IMT exhibits growing storage of Gaussian velocity fields, while Ours w IMT- { LoRA-rank } effectively curb the storage growth. Right : For different LoRA ranks, Ours w IMT maintains strong performance (even at rank 16), achieving competitive results to Ours wo IMT .