Dynamic reconstruction from casual video must preserve valid subject motion while suppressing transient occluders, shadows, and volumetric effects. Existing methods either suppress valid motion through static-scene assumptions or model all observations jointly, producing ghosting and unstable geometry. We introduce ReCan-GS, a 4D Gaussian Splatting framework that separates coherent scene content from inconsistent observations. It derives a Jacobian-based deformation-sensitivity score from the learned deformation field, using it as an empirical proxy for reconstruction ambiguity. The score guides separation into dynamic Core Gaussians and per-view Distractor Gaussians, or post-hoc pruning of sensitive Core Gaussians. The same criterion supports removal of spatial occluders, photometric artifacts, and evolving smoke or fog without masks, pretrained segmenters, or medium-specific supervision. Experiments on synthetic and real dynamic scenes and synthetic smoke/fog scenes demonstrate its usefulness across the evaluated distractor regimes.
Figures & tables
Property
SWAG [ 4 ]
GS-W [ 46 ]
Wild-GS [ 42 ]
Splatfacto-W [ 41 ]
WildGaussians [ 12 ]
SpotlessSplats [ 28 ]
DeSplat [ 36 ]
Ours
No pre-trained model
✓
✗
✗
✓
✗
✓
✓
✓
Fast 3DGS rendering
✗
✗
✗
✓
✓
✓
✓
✓
Explicit scene separation
✗
✗
✗
✗
✗
✓
✓
✓
Dynamic scene reconstruction
✗
✗
✗
✗
✗
✗
✗
✓
Uncertainty Metric
✗
✗
✗
✗
✗
✗
✗
✓
Fog/Smoke Removal
✗
✗
✗
✗
✗
✗
✗
✓
Table 1: Feature comparison. Our method uniquely supports distractor removal in fully dynamic scenes without pretraining.
Figure 2: An overview of our ReCan-GS pipeline. The scene is modeled with two components: dynamic Core Gaussians ( G ) and per-view Distractor Gaussians ( Gdist ). An MLP deforms the core Gaussians over time ( t ). A Jacobian-based uncertainty signal, derived from the deformation field, guides the optimization to penalize unstable motion in the core layer, effectively forcing transient effects into the distractor layer.
Figure 3: Qualitative results on dynamic scenes with distractors. Columns 2–3 are dynamic reconstruction methods that jointly model all motion, often producing ghosting from transient occluders. Columns 4–5 are standard distractor-removal methods that assume a static scene and therefore struggle with valid dynamic content. ReCan-GS explicitly decomposes inconsistent observations into a distractor layer and recovers a clean, stable core reconstruction.
Figure 4: Smoke removal results. ReCan-GS separates slow-moving volumetric distractors from the core scene by detecting deformation-space instability, enabling clean smoke-free reconstructions where appearance-based methods leave residual haze.
Method
Squatting
Bouncing Balls
Jeep
Bottle
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Static distractor-removal methods
WildGaussians [ 12 ]
24.3
0.88
0.15
–
–
–
25.4
0.90
0.18
18.2
0.67
0.33
SLS-MLP [ 28 ]
22.9
0.87
0.15
15.3
0.84
0.25
25.8
0.90
0.17
17.8
0.65
0.35
Robust3DGS [ 32 ]
24.4
0.90
0.11
24.6
0.97
0.07
26.6
0.92
0.07
17.3
0.67
0.30
DeSplat [ 36 ]
25.0
0.89
0.12
14.3
0.87
0.16
26.5
0.89
0.17
17.1
0.67
0.30
Table 2: Quantitative results on dynamic scenes with distractors, grouped into static distractor-removal and dynamic reconstruction methods. Higher is better for PSNR/SSIM and lower for LPIPS; best results per scene are in bold . Real-scene holdout frames are distractor-free, so these metrics do not fully measure retained distractor geometry; see Table 3 .
VLM judges
Scene
Method
Human
GPT-5
Claude
Gemini
Bottle
4DGS [ 39 ]
3.41 (82.0)
2.09 (35.5)
3.15 (70.0)
3.12 (75.1)
DeformableGS [ 44 ]
3.37 (76.7)
2.60 (56.7)
3.23 (73.8)
3.10 (70.4)
ReCan-GS (Ours)
1.33 (9.3)
1.21 (5.6)
1.81 (26.0)
1.71 (24.7)
Jeep
4DGS [ 39 ]
3.43 (76.7)
3.42 (82.8)
3.68 (83.2)
3.63 (87.8)
DeformableGS [ 44 ]
3.29 (72.7)
3.38 (78.6)
3.87 (78.8)
3.46 (82.6)
Table 3: Retained distractor geometry. Three human annotators and three VLM judges rate per frame how strongly the distractor is reproduced ( 1 = absent, 5 = fully reproduced); mean severity and (% of frames rated ≥3 ). VLMs score all distractor-present frames ( Bottle : 534, Jeep : 345); the Human column averages three annotators over the same 50 frames per method, who agree closely (mean pairwise quadratic-weighted κ=0.85 , Spearman ρ=0.87 over 900 paired ratings). On Jeep , no annotator rated any ReCan-GS frame ≥3 . Per-annotator scores are on the project website.
Figure 5: Uncertainty vs. Motion and Coverage. Uncertainty decreases as the number of camera views increases, indicating that the metric captures reconstructability rather than raw motion magnitude.
Figure 6: Jacobian uncertainty vs. motion magnitude. Motion magnitude responds strongly to valid subject motion, whereas the Jacobian score more selectively localizes the transient distractor.
Method
Muscle Car
Chair
Hotdog
Lego
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
PSNR ↑
SSIM ↑
LPIPS ↓
SLS-MLP [ 28 ]
5.4
0.01
0.71
6.3
0.05
0.72
5.6
0.14
0.76
4.3
0.06
0.80
DeSplat [ 36 ]
11.2
0.19
0.46
8.6
0.05
0.63
8.6
0.19
0.60
8.8
0.14
0.59
Robust3DGS [ 32 ]
10.9
0.17
0.52
8.6
0.05
0.67
8.6
0.14
0.65
8.2
0.09
0.67
AsymGS [ 14 ]
11.1
0.18
0.48
8.7
0.06
0.65
8.6
0.15
0.65
8.5
0.09
0.66
Ours
16.8
.69
.26
10.9
.33
.50
14.8
.69
.30
12.5
.58
.39
Table 4: Fog/smoke removal on synthetic scenes with fog-free ground truth. Appearance-based methods misidentify volumetric fog as stable background; ReCan-GS removes it without any fog-specific design. Best results in bold .
Figure 7: Hierarchical Sensitivity. Post-hoc pruning removes the top 0%, 5%, 7%, and 15% most uncertain Core Gaussians, ranked by accumulated deformation sensitivity. Without pruning, transient hands and moving cast shadows remain. Removing the top 5% primarily suppresses the hands, while pruning the top 7% and 15% additionally suppresses moving cast shadows.
Reconstructing dynamic 3D scenes from multi-view videos is a foundational task for robotics, AR/VR, and digital twins. While 3D Gaussian Splatting (3DGS) provides state-of-the art scene representation, its dynamic adaptations (4DGS) face significant challenges. Early MLP-based deformation models excel at interpolating within training frames but fail at temporal extrapolation --- important feature for forecasting in predictive systems. To address this, physically grounded approaches have been proposed to enable robust future forecasting, while, concurrently, semantic 4DGS methods have been developed for scene analysis and language-driven prompting. However, there is currently no framework that unites both physical consistency and semantic understanding. In this paper, we propose the first, to the best of our knowledge, unified 4DGS approach that is both physically grounded and semantically aware. Furthermore, to optimize computational efficiency, we introduce a novel rigid-body constraint mechanism. Instead of predicting the physical dynamics for every Gaussian particle, our model predicts the motion of a single anchor Gaussian per semantic object, updating the remaining Gaussians via relative transformations. This architectural design significantly accelerates novel view synthesis compared to an unconstrained baseline, while enabling object-accurate extrapolation and intuitive semantic interaction. Our project page: https://be2rlab.github.io/r5dgs/.
Dynamic 3D Gaussian Splatting (3DGS) methods reconstruct time-varying scenes from synchronized multi-camera video using photometric supervision. When a moving object becomes fully occluded from all training cameras, this supervision vanishes: the Gaussians representing it receive no gradient signal and degrade. Existing approaches to incomplete observations in neural reconstruction rely on learned generative priors that prioritize visual plausibility over physical correctness. We propose PersistGS, a method that restores object permanence during occlusion by coupling differentiable rigid body simulation with 3D Gaussian Splatting. Our approach decomposes the scene into per-object Gaussians and collision meshes, estimates friction and velocity from the observed pre-occlusion trajectory via differentiable simulation, and uses the resulting SE(3) trajectory to position object Gaussians throughout the occlusion period. Because the predicted trajectory satisfies the governing equations of rigid body dynamics, it faithfully captures contact events (bounces, friction-based deceleration, direction changes) that kinematic extrapolation cannot model. We introduce a centroid silhouette loss that isolates positional gradients from appearance noise, yielding 40% lower trajectory error than photometric supervision. We evaluate using cameras withheld from training that observe the object during its occlusion. Experiments on synthetic scenes show that PersistGS outperforms constant velocity extrapolation by +2.46dB PSNR and comes within 0.19dB of a ground-truth trajectory upper bound.
We present a 4D Gaussian Splatting framework for the Sparse-View Track of the SIGGRAPH Asia 2026 Volumetric Video Challenge, which requires dynamic scene reconstruction from only six cameras with wide baselines. To achieve robust dynamic reconstruction under such sparse views, our framework integrates three components. (1) Region-adaptive spatial priors: We use foreground masks to guide Gaussian initialization and mask voting to control densification separately for the dynamic foreground and static background. Background geometry is regularized using monocular depth aligned to metric scale. (2) Motion-consistent temporal priors: We provide supervision at intermediate times through frame interpolation and constrain projected Gaussian motion with estimated optical flow. (3) Generative assistance: We place virtual cameras in the widest angular gaps and restore their rendered images using a diffusion-based model conditioned on camera poses. The restored images are iteratively incorporated into training as pseudo-supervision. On the validation set, our framework improves full-frame PSNR from 25.60 dB for the baseline to 29.75 dB. On the official test benchmark, it achieves 30.04 dB full-frame PSNR and 27.88 dB foreground PSNR, ranking first overall in the Sparse-View Track.
Shengqi Wang, Zhengxian Yang, Kaiwen Tian +6
Department of Automation, Tsinghua University, Beijing, China · JD.com, Beijing, China · Beijing MEET YUAN Co., Ltd, Beijing, China