High-fidelity reconstruction of dynamic scenes is an important yet challenging problem. While recent 4D Gaussian Splatting (4DGS) has demonstrated the ability to model temporal dynamics, it couples Gaussian motion and geometric attributes within a single covariance formulation, which limits its expressiveness for complex motions and often leads to visual artifacts. To address this, we propose VeGaS, a novel velocity-based 4D Gaussian Splatting framework that decouples Gaussian motion and geometry. Specifically, we introduce a Galilean shearing matrix that explicitly incorporates time-varying velocity to flexibly model complex non-linear motions, while strictly isolating the effects of Gaussian motion from the geometry-related conditional Gaussian covariance. Furthermore, a Geometric Deformation Network is introduced to refine Gaussian shapes and orientations using spatio-temporal context and velocity cues, enhancing temporal geometric modeling. Extensive experiments on public datasets demonstrate that VeGaS achieves state-of-the-art performance.
Figures & tables
Figure 1: Graphic illustration of the proposed VeGaS in comparison with 4DGS. (a) Both methods share the same random initialization. (b) During training, VeGaS fits trajectories (as illustrated by the blue curves) using a decoupled motion and geometry formulation, whereas 4DGS adopts a coupled modeling scheme. (c) At inference, VeGaS employs time-varying velocity and geometry to capture complex trajectories and deformations, while 4DGS assumes constant velocity and time-invariant geometry. (d) Qualitative results show that VeGaS yields higher rendering fidelity, whereas 4DGS exhibits noticeable artifacts.
Figure 2: Overview of our proposed velocity-based decoupling of motion and geometry for 4D Gaussian Splatting. (a) Original velocity v0 is transformed using the time-varying velocity v(t) calculated from the shearing matrix, enabling original 4D Gaussians to move along non-linear trajectories in continuous time. (b) The deformation network predicts geometric transformations of Gaussians at any time based on velocity information, time query t , and 4D Gaussian center μ4D . (c) Combining the velocity and geometry transformations, the rendered images are obtained through differentiable rasterization of transformed Gaussians at each frame.
Methods
Venue
PSNR ↑
SSIM ↑
LPIPS ↓
Neural Volumes Lombardi et al. (2019)
TOG
22.80
0.94
0.295
LLFF Mildenhall et al. (2019)
TOG
23.24
0.92
0.235
DyNeRF Li et al. (2022b)
CVPR
29.58
0.98
0.099
HexPlane Cao and Johnson (2023)
CVPR
31.70
0.98
0.075
K-Planes-explicit Fridovich-Keil et al. (2023)
CVPR
30.88
0.98
-
K-Planes-hybrid Fridovich-Keil et al. (2023)
CVPR
31.63
0.98
-
Table 1: Comparison with state-of-the-art methods on the Neu3DV dataset. + denotes results reproduced under our experimental setting, as the original protocol adopts a different train/test split.
Figure 3: Qualitative result on the Neural 3D Video dataset. Our method exhibits noticeably higher visual quality compared to others.
Methods
Venue
PSNR ↑
SSIM ↑
LPIPS ↓
T-NeRF Pumarola et al. (2021)
CVPR
29.51
0.95
0.08
D-NeRF Pumarola et al. (2021)
CVPR
29.67
0.95
0.07
TiNeuVox Fang et al. (2022)
SIGGRAPH Asia
32.67
0.97
0.04
K-Planes-explicit Fridovich-Keil et al. (2023)
CVPR
31.05
0.97
-
K-Planes-hybrid Fridovich-Keil et al. (2023)
CVPR
31.61
0.97
-
V4D Gan et al. (2023)
TVCG
33.72
0.98
0.02
Table 2: Quantitative comparison of methods on the monocular synthetic D-NeRF dataset.
Figure 4: Qualitative results on the D-NeRF dataset, where our method captures finer details.
Methods
PSNR ↑
SSIM ↑
LPIPS ↓
FPS ↑
Storage (MB) ↓
4DGS Yang et al. (2023)
32.01
0.97
0.055
114
2195
+ velocity
32.07
0.97
0.050
202
1124
+ geometric modeling
32.17
0.97
0.052
134
1088
VeGaS (Ours Full)
32.68
0.98
0.045
115
1124
Table 3: Ablation study on the components of our design.
flame_salmon
# Anchors
PSNR ↑
SSIM ↑
LPIPS ↓
trex
# Anchors
PSNR ↑
SSIM ↑
LPIPS ↓
2
29.25
0.960
0.065
2
31.30
0.993
0.011
4
29.54
0.962
0.063
4
31.43
0.993
0.011
6
29.88
0.964
0.061
6
31.51
0.993
0.011
8
29.30
0.961
0.062
8
31.34
0.993
0.011
10
29.15
0.955
0.072
10
31.26
0.992
0.012
Table 4: Ablation study on the impact of anchor number.
Figure 5: Visualization of ablation studies on the Neu3DV dataset, where we individually remove velocity and geometric modeling to evaluate their impact on the model’s performance. Continuous video frames are extracted to observe the effects of removing these components.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Additional qualitative result on the Neural 3D Video dataset. Our method exhibits noticeably higher visual quality compared to others.