Dusting the Phantom: Gaussian Allocation for Fading Dynamic Content

Minhyeok Roh; Minseo Lee; Jun Young Jeong; Byeonghyeon Lee; Lucas Yunkyu Lee; Jaesik Park; Eunbyung Park

cs.CVOct 15, 2026

Dusting the Phantom: Gaussian Allocation for Fading Dynamic Content

Authors: Minhyeok Roh, Minseo Lee, Jun Young Jeong, Byeonghyeon Lee, Lucas Yunkyu Lee, Jaesik Park, Eunbyung Park

Organizations: Yonsei University · Electronics and Telecommunications Research Institute (ETRI) · Pohang University of Science and Technology (POSTECH) · Seoul National University

Abstract

Dynamic Gaussian representations have made high-quality, real-time volumetric video increasingly practical, yet challenging motion can still break their reconstruction: small fast-moving objects may disappear, and fine textures in highly dynamic regions are often lost. In particular, simply training longer or using more Gaussians may still fail to recover small, fast-moving objects. We argue that the key bottleneck is not representational capacity itself, but where and when Gaussian primitives are allocated. Existing allocation strategies rely on current Gaussian locations and often overlook the spatiotemporal destinations of relocated Gaussians. To address these limitations, we introduce two complementary strategies. First, Seed Injection identifies under-reconstructed dynamic regions using temporal appearance changes and reconstruction errors, and initializes new Gaussians at geometrically plausible 3D locations through multi-view observations, independent of existing Gaussian locations. Second, Temporal Biased Relocation uses time-resolved optimization signals to guide the spatiotemporal placement of relocated Gaussians. Both strategies maintain a fixed Gaussian budget without modifying the underlying representation. Experiments across multiple dynamic-scene datasets demonstrate improved reconstruction quality, particularly in highly dynamic regions, recovering small, rapidly moving objects and fine-grained appearance details.

Figures & tables

Figure 1: Reconstruction under highly challenging dynamics. Existing 4DGS methods can (a) completely miss rapidly moving objects and (b) lose fine details under fast motion. Our method addresses both failure cases, recovering missing objects while better preserving fine details.

Figure 2: Overview of Seed Injection. We identify under-reconstructed dynamic regions from synchronized multi-view observations and sample candidate pixels from the resulting seed map. The sampled observations are lifted into 3D using multi-view geometry to initialize new Gaussian primitives independently of the existing Gaussian distribution. After optimization, the injected Gaussians provide additional support for dynamic content that is poorly represented or missing in the original reconstruction.

Method	PSNR $\uparrow$	Masked PSNR $\uparrow$	DSSIM 1 $\downarrow$	DSSIM 2 $\downarrow$	LPIPS $\downarrow$
DyNeRF † ( Li et al., 2022 )	29.58	–	–	0.020	0.083
K-Planes † ( Fridovich-Keil et al., 2023 )	31.63	–	–	0.018	–
NeRFPlayer † ( Song et al., 2023 )	30.96	–	0.034	–	0.111
Deformable-3DGS † ( Yang et al., 2024b )	31.15	–	0.030	–	0.049
SpaceTimeGS † ( Li et al., 2024 )	32.05	–	0.026	0.014	0.044
RealTime4DGS ( Yang et al., 2024a )	31.97	31.90	0.027	0.015	0.051

Table 1: Quantitative comparison on the N3DV dataset. Results marked with

\dagger

are sourced from the original papers. Best and second-best results are highlighted in bold and underlined , respectively.

Method	PSNR $\uparrow$	Masked PSNR $\uparrow$	DSSIM 1 $\downarrow$	DSSIM 2 $\downarrow$	LPIPS $\downarrow$
Dynamic 3DGS ( Luiten et al., 2024 )	28.72	27.13	0.051	0.024	0.095
RealTime4DGS ( Yang et al., 2024a )	28.01	25.84	0.044	0.023	0.111
Ex4DGS ( Lee et al., 2024 )	24.86	23.64	0.054	0.029	0.120
4DGaussian ( Wu et al., 2024 )	26.92	23.68	0.047	0.024	0.116
FreeTimeGS ( Wang et al., 2025 )	28.34	26.68	0.039	0.019	0.110
FreeTimeGS + Ours	29.05	27.28	0.032	0.016	0.063

Table 2: Quantitative comparison on the CMU Panoptic dataset.

Method	PSNR $\uparrow$	Masked PSNR $\uparrow$	DSSIM 1 $\downarrow$	DSSIM 2 $\downarrow$	LPIPS $\downarrow$
RealTime4DGS ( Yang et al., 2024a )	20.46	24.45	0.154	0.114	0.249
Ex4DGS ( Lee et al., 2024 )	19.74	22.00	0.179	0.132	0.335
4DGaussian ( Wu et al., 2024 )	20.80	22.03	0.163	0.114	0.308
FreeTimeGS ( Wang et al., 2025 )	20.84	27.37	0.131	0.097	0.170
FreeTimeGS + Ours	20.98	27.47	0.128	0.095	0.160

Table 3: Quantitative comparison on the Custom dataset.

Figure 3: Qualitative comparison on the CMU Panoptic Juggle sequence. Our method better reconstructs the rapidly moving juggling balls while preserving fine details of the hands.

Method	PSNR $\uparrow$	Masked PSNR $\uparrow$	DSSIM 1 $\downarrow$	DSSIM 2 $\downarrow$	LPIPS $\downarrow$	Training Time (m) $\downarrow$
FreeTimeGS ( Wang et al., 2025 )	28.22	27.58	0.045	0.023	0.161	5.69
FreeTimeGS w/ Seed Injection	28.53	27.82	0.045	0.022	0.160	5.85
FreeTimeGS w/ Temporal Biased Relocation	28.57	28.02	0.034	0.017	0.080	5.74
FreeTimeGS + Ours	28.98	28.24	0.034	0.017	0.082	5.91

Table 4: Ablation study of our proposed components on the CMU Juggle, Football, and Basketball sequences.

Figure 4: Qualitative ablation of Seed Injection and Temporal Biased Relocation on the CMU Panoptic Juggle sequence. Both components improve the reconstruction of dynamic content, while their combination provides the most complete reconstruction.

Figure 5: Visualization of Seed Injection on the Custom dataset. Red markers indicate the projected centers of newly injected Gaussians, which are concentrated around rapidly moving objects such as the glove, bat and baseball.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Effect of increased Gaussian capacity and training iterations. Even with twice the number of Gaussians (3M) and training iterations (0.2M), FreeTimeGS fails to recover the rapidly moving ball, while our method successfully reconstructs it with 1.5M Gaussians and 0.1M iterations.

Figure 7: Construction of the dynamic error-based seed map. The reconstruction error map is computed from the difference between the ground-truth and rendered images, while the dynamic mask is obtained from the intersection of temporal difference masks from adjacent frames. The final seed map is obtained by element-wise multiplication of the reconstruction error map and the dynamic mask, highlighting under-reconstructed dynamic regions.

Figure 8: Qualitative ablation on the number of sampled seeds $N_{\mathrm{seed}}$ on the Custom Baseball dataset. We compare different values of $N_{\mathrm{seed}}$ across three representative frames containing small and rapidly moving objects. Increasing $N_{\mathrm{seed}}$ leads to more consistent recovery of challenging dynamic content, including the baseball, bat, and glove.

Figure 9: Qualitative ablation on the maximum number of Gaussians injected per frame, $N_{\max}$ , on the Custom Baseball dataset. We compare $N_{\max}\in\{16,32,64,256\}$ across three frames. Even with $N_{\max}=16$ , challenging dynamic content, including the rapidly moving baseball, glove, and bat, is successfully reconstructed.

Figure 10: Qualitative ablation on the number of camera views $C$ used for multi-view seed generation on the Custom Baseball dataset frame 9. Using only two views results in inaccurate reconstruction of the rapidly moving baseball, whereas four or more views successfully recover the object with similar reconstruction quality.

Figure 11: Number of relocation operations during training on the CMU Panoptic dataset. FTGS denotes FreeTimeGS and TBR denotes Temporal Biased Relocation.

Figure 12: Qualitative results on the N3DV dataset.

Figure 13: Qualitative results on the CMU Panoptic dataset.

Figure 14: Qualitative results on the Custom dataset.

Dynamic Scene Reconstruction

Dynamic scene reconstruction builds time-varying 3D models of moving objects and environments from video or multi-view images.

See all 288 papers

Temporally Aware Densification for Dynamic 3D Gaussian Splatting
Jun 22, 2026Vikram Sandu, Mayurdeep Pathak, Rajiv Soundararajan3D Gaussian Splatting Adaptive Gaussian Densification
Indian Institute of Science, Bengaluru, India
Sparse-View 4D Gaussian Splatting via Spatiotemporal Priors and Generative Assistance
Oct 3, 2026Shengqi Wang, Zhengxian Yang, Kaiwen Tian +6Sparse-View 4D Reconstruction Dynamic Scene Reconstruction
Department of Automation, Tsinghua University, Beijing, China · JD.com, Beijing, China · Beijing MEET YUAN Co., Ltd, Beijing, China
GrainGS: Gradient-Decoupled Gaussian Splatting for Efficient Dynamic Novel View Synthesis
Jul 23, 2026Jiahao He, Yihua Shao, Zhengkai Zhao +63D Gaussian Splatting Dynamic Scene Reconstruction
College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, Jiangsu, China · Institute of Automation, Chinese Academy of Sciences, Beijing, China · Department of Computing, Hong Kong Polytechnic University, Hong Kong, China +4

Jun 22, 2026J/K move · Enter open · S save

Temporally Aware Densification for Dynamic 3D Gaussian Splatting

Vikram Sandu, Mayurdeep Pathak, Rajiv Soundararajan

Indian Institute of Science, Bengaluru, India

3D Gaussian Splatting Adaptive Gaussian Densification Dynamic Scene Reconstruction

Open paper AskPDF arXiv

Despite modeling temporal motion, dynamic 3D Gaussian Splatting (3DGS) methods still inherit a static densification strategy that is ill-suited for dynamic scenes. This neglect of temporal behavior leads to under-reconstructed and blurry dynamic regions, as short-lived Gaussians receive sparse supervision and fail to densify effectively. We propose a Visibility-Aware Densification (VAD) framework that integrates temporal visibility into the densification process, ensuring that Gaussians are refined based on their actual temporal presence. A Temporally-Adaptive Thresholding (TAT) mechanism further adjusts each Gaussian's densification threshold according to its temporal lifespan, promoting balanced refinement of both static and dynamic regions. Finally, a Temporal Offset Warping (TOW) design enhances deformation capacity around temporal centers, extending the lifespan of highly dynamic Gaussians and facilitating more effective densification. Our approach achieves substantial improvements in the visual quality of dynamic regions, outperforming existing methods across three dynamic multi-view benchmark datasets. Moreover, the proposed VAD module generalizes across diverse dynamic 3DGS methods, consistently improving dynamic reconstruction as a plug-and-play component.

Abstract

Dynamic Scene Reconstruction

Explore similar work

Temporally Aware Densification for Dynamic 3D Gaussian Splatting

Sparse-View 4D Gaussian Splatting via Spatiotemporal Priors and Generative Assistance

GrainGS: Gradient-Decoupled Gaussian Splatting for Efficient Dynamic Novel View Synthesis