Dynamic 3D Gaussian Splatting (3DGS) achieves photorealistic reconstruction of time-varying scenes, and recent physics-aware extensions improve extrapolation by explicitly predicting velocity fields. However, these extensions merely fit vector fields to visual deformations without satisfying Lagrangian mechanics, leading to three major issues: (i) physically inconsistent trajectories, (ii) lack of time-reversibility, and (iii) geometric collapse during long-term extrapolation. In this paper, we propose LagrangeGS, which formulates dynamic 3DGS as a non-conservative Lagrangian system. While this Lagrangian formulation fundamentally solves (i), a direct application of general LNNs to dynamic 3DGS requires a large velocity-Hessian inversion for millions of Gaussian particles. To overcome this computational bottleneck, we approximate the velocity-Hessian as an identity matrix, decoupling particle dynamics for computational tractability. For (ii), we restrict the non-conservative forces to be explicitly time independent, enabling consistent backward integration. Finally, to address (iii), we introduce local rigid alignment that regularizes particle trajectories. Extensive evaluations on dynamic scene benchmarks demonstrate that LagrangeGS enables stable long-term extrapolation, consistent time reversal, and counterfactual physics-based editing without retraining.
Figures & tables
Figure 1: LagrangeGS formulates dynamic scenes as a non-conservative Lagrangian system parameterized by potential energy U , initial velocity v0 , and non-conservative forces Q , enabling (a) long-term extrapolation, (b) time reversal, and (c) physics-based editing.
Figure 2: Overview of LagrangeGS. (i) Gaussian particles are partitioned into distinct objects via pre-trained model trajectory. (ii) TD-LNN computes per-particle accelerations q¨=−∇qU+Q , combining the decoupled particle dynamics for avoiding intractable computation and the time-independent system for exact backward integration. (iii) To prevent geometric collapse, LRA binds the decoupled particles into locally rigid structures.
Figure 3: Overview of the physics manipulations. (i) Extrapolation: future states ( 0<τ ) are computed by forward integration. (ii) Time reversal: Past states ( τ<0 ) are recovered by backward integration using a negative time step −Δτ . (iii) Counterfactual physics-based editing: Dynamic properties are manipulated by scaling the initial velocity ( α ), learned potential ( β ), and time-independent non-conservative force ( γ ).
Figure 4: Learned trajectories of FreeGave and LagrangeGS with the potential U(q) for the ”Fan” and ”Falling Ball” scenes. While the rollouts of FreeGave exhibit erratic fluctuations, LagrangeGS ensures consistent trajectories satisfying Lagrangian mechanics. Without structural priors, U(q) captures the underlying physics: a rotationally symmetric potential for the Fan, and a uniform gravitational gradient for the Falling Ball.
Figure 5: Extrapolation on the Fan scene. Training spans t=0.0 – 0.75 . Upon entering extrapolation, DefGS instantly suffers from unnatural motion or complete stagnation, while FreeGave exhibits geometric drift at t=1.80 and catastrophic volume divergence at t=2.25 . Distilling both backbones via LagrangeGS (Ours) mitigates these artifacts, sustaining physically consistent and smooth rotation across the entire extended horizon.
Figure 6: Extrapolation on the Factory scene. Training spans t=0.0 – 0.75 . Upon extrapolation, DefGS immediately degenerates into an indistinct haze, while FreeGave suffers from geometric collapse for t≥1.80 . LagrangeGS prevents this long-rollout collapse for both backbones, preserving the can geometry.
Dynamic Object Dataset
Dynamic Indoor Scene Dataset
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
T-NeRF [ 17 ]
13.818
0.739
0.324
22.242
0.700
0.363
D-NeRF [ 17 ]
14.660
0.737
0.312
20.791
0.692
0.349
NSFF [ 12 ]
-
-
-
24.163
0.795
0.289
TiNeuVox [ 3 ]
19.612
0.940
0.073
21.029
0.770
0.281
NVFi [ 8 ]
27.594
0.972
0.036
29.745
0.876
0.204
Table 1: Short-term extrapolation performance on dynamic scene benchmarks. LagrangeGS improves the extrapolation performance of DefGS. Conversely, when applied to FreeGave, photometric metrics exhibit a marginal drop. This is a plausible consequence of strict physical regularization: by satisfying Lagrangian mechanics across all particles, LagrangeGS deliberately prunes non-physical visual artifacts such as shadows.
Dynamic Object Dataset
Dynamic Indoor Scene Dataset
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
FreeGave
31.987
0.990
0.007
35.019
0.966
0.051
Full
LagrangeGS
28.940
0.977
0.020
33.097
0.955
0.059
diff
(-3.047)
(-0.013)
(+0.013)
(-1.922)
(-0.011)
(+0.008)
FreeGave
24.874
0.970
0.027
25.757
0.823
0.214
t<0.5
LagrangeGS
24.601
0.962
0.032
24.713
0.821
0.218
Table 2: Extrapolation under truncated observation windows ( t<0.5 , t<0.3 ). By artificially shortening the training horizon (originally tmax∼0.75 ), the performance gap between FreeGave and LagrangeGS narrows as the training window is shortened. Especially for the extreme truncation ( t<0.3 ) in the Dynamic Object dataset, LagrangeGS outperforms FreeGave, demonstrating its resilience in long-term extrapolation under data-sparse regimes.
Figure 7: Time-reversal rollout on the ”Telescope” scene via LagrangeGS. Past states ( t<0 ) are recovered exactly by integrating the same trained TD-LNN backward with a negative time step −Δt , ensuring visual and structural consistency with forward rollouts ( t>0 ). Red arrows denote per-particle velocity vectors; their exact directional inversion across t=0 confirms that LagrangeGS captures genuine continuous mechanics rather than a learned animation shortcut.
Dynamic Object Dataset
Dynamic Indoor Scene Dataset
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Full
28.940
0.977
0.020
33.097
0.955
0.059
w/o stage3
27.608
0.975
0.018
27.804
0.934
0.079
w/o stage2,3
27.471
0.974
0.021
29.011
0.938
0.077
w/o LRA
26.964
0.974
0.022
34.261
0.957
0.057
w/o Force Q
24.971
0.965
0.032
33.303
0.956
0.054
Table 3: Ablation study. The Stage 2 (TD-LNN) and Stage 3 (Gaussian parameter) fine-tuning contribute to visual scores. While LRA is essential for preventing geometric collapse, its absence marginally inflates PSNR in the Indoor dataset; this merely reflects unconstrained particles overfitting to non-rigid visual phenomena like moving shadows. Furthermore, because the Indoor scenes exhibit strictly conservative motions, the non-conservative force Q is redundant, meaning its removal avoids over-parameterization and slightly benefits PSNR.
Figure 8: Inference-time counterfactual physics-based editing on the ”Falling Ball” scene. The dynamics are systematically manipulated without retraining by rescaling the learned potential field ( U′=β⋅U ). The baseline ( β=1.0 ) reproduces the true observed trajectory. Decreasing β simulates a weakened gravitational environment, slowing the descent, while increasing β strengthens the effective gravity field. This predictable and continuous modulation exposes the learned U(q) as an explicitly editable physical property rather than an uninterpretable visual latent feature.
3D Gaussian Splatting has become a de facto scene representation for novel view synthesis, yet robustly learning 3D Gaussian primitives from visual input remains challenging. Standard optimization relies on gradient-based updates, but a common issue is the gradient vanishing phenomenon: a pixel far from a Gaussian primitive often has diminishing gradient magnitudes to influence primitive attributes, resulting in suboptimal scene reconstruction. In this paper, we propose a method to address gradient vanishing with a piecewise truncated gradient formulation that improves the optimization stability and robustness to initializations. We show that our method consistently improves 3D Gaussian Splatting with random and COLMAP initializations while being generalizable across static and dynamic Gaussian Splatting. As a by-product, we also examine the limitations of current benchmarks for dynamic scenes, and introduce a novel dataset for benchmarking dynamic Gaussian Splatting using synthetic 3D scenes. We demonstrate the effectiveness of our method in both static and dynamic settings for the public benchmarks and our proposed dataset.
Dynamic 3D Gaussian Splatting (3DGS) methods reconstruct time-varying scenes from synchronized multi-camera video using photometric supervision. When a moving object becomes fully occluded from all training cameras, this supervision vanishes: the Gaussians representing it receive no gradient signal and degrade. Existing approaches to incomplete observations in neural reconstruction rely on learned generative priors that prioritize visual plausibility over physical correctness. We propose PersistGS, a method that restores object permanence during occlusion by coupling differentiable rigid body simulation with 3D Gaussian Splatting. Our approach decomposes the scene into per-object Gaussians and collision meshes, estimates friction and velocity from the observed pre-occlusion trajectory via differentiable simulation, and uses the resulting SE(3) trajectory to position object Gaussians throughout the occlusion period. Because the predicted trajectory satisfies the governing equations of rigid body dynamics, it faithfully captures contact events (bounces, friction-based deceleration, direction changes) that kinematic extrapolation cannot model. We introduce a centroid silhouette loss that isolates positional gradients from appearance noise, yielding 40% lower trajectory error than photometric supervision. We evaluate using cameras withheld from training that observe the object during its occlusion. Experiments on synthetic scenes show that PersistGS outperforms constant velocity extrapolation by +2.46dB PSNR and comes within 0.19dB of a ground-truth trajectory upper bound.
Reconstructing dynamic 3D scenes from multi-view videos is a foundational task for robotics, AR/VR, and digital twins. While 3D Gaussian Splatting (3DGS) provides state-of-the art scene representation, its dynamic adaptations (4DGS) face significant challenges. Early MLP-based deformation models excel at interpolating within training frames but fail at temporal extrapolation --- important feature for forecasting in predictive systems. To address this, physically grounded approaches have been proposed to enable robust future forecasting, while, concurrently, semantic 4DGS methods have been developed for scene analysis and language-driven prompting. However, there is currently no framework that unites both physical consistency and semantic understanding. In this paper, we propose the first, to the best of our knowledge, unified 4DGS approach that is both physically grounded and semantically aware. Furthermore, to optimize computational efficiency, we introduce a novel rigid-body constraint mechanism. Instead of predicting the physical dynamics for every Gaussian particle, our model predicts the motion of a single anchor Gaussian per semantic object, updating the remaining Gaussians via relative transformations. This architectural design significantly accelerates novel view synthesis compared to an unconstrained baseline, while enabling object-accurate extrapolation and intuitive semantic interaction. Our project page: https://be2rlab.github.io/r5dgs/.