Organizations: Sun Yat-sen University, Guangzhou, China · Texas A&M University, College Station, USA · Cardiff University, Wales, UK · Centre for Perceptual and Interactive Intelligence, Hong Kong, China · Shenzhen Loop Area Institute, Shenzhen, China
3D Gaussian Splatting (3DGS) has demonstrated remarkable performance in novel view synthesis and 3D scene reconstruction, but its quality often degrades in real-world environments due to transient distractors, such as moving objects and varying shadows. Existing methods commonly introduce semantic priors from pre-trained vision models either to group pixels into coherent regions or to define perceptual error metrics. However, semantic grouping is often misaligned with the binary static/transient distinction, while perceptual features can be fragile under appearance perturbations introduced during 3DGS optimization. We propose 3DGS-HPC, a framework that addresses these issues by combining two complementary principles: a patch-wise classification strategy that leverages local spatial consistency for robust region-level decisions, and a hybrid classification metric that adaptively integrates photometric and perceptual cues for more reliable separation. Extensive experiments demonstrate the superiority and robustness of our method in mitigating distractors to improve 3DGS-based novel view synthesis. Our project page is https://cnhaox.github.io/3DGS-HPC/ .
Figures & tables
Figure 1Figure 2Figure 3Figure 4Figure 5Figure 6
Figure 1: Overview of Hybrid Patch-wise Classification (HPC) . Given training images disturbed by transient distractors, our method (a) optimizes 3DGS following standard practice, (b) classifies static and transient regions using a novel hybrid metric that combines the strengths of photometric and perceptual cues, and (c) performs classification at the patch level to exploit local spatial consistency, thereby enhancing robustness and efficiency without relying on external semantic segmentation or detection priors for region grouping.
Figure 2: Visualization of Different Partitioning Strategies. When grouping pixels into multiple semantically consistent regions with blue borders, existing methods (b-d) fail to faithfully represent transient regions ( e.g . , pedestrians and shadows) due to semantic mismatch between different tasks, while our patch-wise partition strategy (e) can accurately extract these regions.
Figure 3: Visualization of Different Error Metrics. Given training images with distractors and corresponding rendered images, traditional photometric metrics (L1) produce noisy static maps due to low-level appearance ambiguities ( e.g . , similar colors). Instead, perceptual metrics based on vision foundation models (DINOv2) produce clean static maps with clear boundaries.
Figure 4: Comparison of different classification approaches with different error metrics. Bottom row: Although perceptual error metrics are effective in capturing semantic differences, they can produce anomalous results in textureless regions ( e.g . , green wall and white tablecloth) due to visual perturbability ( e.g . , blurring) between training and rendered images. Top row: In contrast, while inherently noisier, photometric error metrics demonstrate greater reliability in estimating the overall proportion of transient pixels within an image, thus guiding perception error metrics for more accurate classification.
Figure 5: Quantitative and Qualitative Comparisons on the RobustNeRF Dataset ( Sabour et al., 2023 ) . The first , second , and third best values are highlighted. Quantitatively, our method not only improves the performance of native 3DGS, but also outperforms previous methods on most quality metrics. Qualitatively, our method can better preserve static details while ignoring distractors.
Figure 6: Quantitative and Qualitative Comparisons on the On-the-go Dataset ( Ren et al., 2024 ) . The first , second , and third best values are highlighted. Our method consistently achieves top-three quantitative results in scenes with varying distractor ratios. In terms of visual quality, our method can better reconstruct static content, while previous methods suffer from missing or distorted details.
Figure 7: Comparisons of Partitioning Strategies. Pixel-wise method (a) produces noisy static maps, while other methods (b-d) confuse the desktop with transient shadows. Instead, our method (e) accurately distinguishes between transient and static elements.
Figure 8: Comparisons of Classification Metrics. (a) fails to distinguish distractors ( e.g . , clothing) that closely resemble static backgrounds in color, while (b) tends to incorrectly remove certain static regions ( e.g . , wall surfaces). A naive combination (d) does not yield meaningful improvements. In contrast, our method (e) effectively separates the transient region from the static one.
Figure 9: Performance of Different Patch Sizes (PS). Patch sizes that are extremely small ( <8 ) or large ( >16 ) will lead to suboptimal quantitative results, and the resulting mask will fail to accurately label and filter out transient distractors ( e.g . , people).
Figure 10: Performance of Different Warm-up Steps (WS). When the number of warm-up steps exceeds 1,000 , the model is at risk of “memorizing” transient distractors, leading to artifacts in the rendered results.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Vision Model
Extracted Feature Layers
VGG-16 ( Simonyan and Zisserman, 2014 )
{conv1_2, conv2_2, conv3_3, conv4_3, conv5_3}
ResNet-18 ( He et al., 2016 )
{layer1, layer2, layer3, layer4}
DINOv2 ViT-S/14 ( Oquab et al., 2023 )
{layer2, layer5, layer8, layer11}
Appendix
Table 1: Settings for Our Perceptual Error Metrics.
RobustNeRF (Avg.)
On-the-go (Avg.)
PSNR ↑
SSIM ↑
LPIPS ↓
Train hrs. ↓
PSNR ↑
SSIM ↑
LPIPS ↓
Train hrs. ↓
(a)
Refinement Variant
29.74
.912
.074
2.53
23.69
.848
.111
2.09
(b)
Ours
29.81
.913
.074
0.22
23.86
.849
.106
0.25
Appendix
Table 2: Performance of Our Patch-wise Classification vs . Its Refinement Variant. Our method can significantly reduce training time and have slightly better performance in visual metrics.
Avg. Train Time (min) ↓
Avg. GPU Mem (GB) ↓
SLS-mlp ( Sabour et al., 2025 )
11.91
3.60
WildGaussians ( Kulhanek et al., 2024 )
24.10
16.02
T-3DGS ( Markin et al., 2024 )
26.75
4.36
Ours (VGG)
16.35
2.09
Ours (ResNet)
13.29
1.98
Ours (DINOv2)
17.52
2.05
Appendix
Table 3: Quantitative Efficiency Comparisons between Different Methods. Our method outperforms most other methods in training time and memory usage.
GT Mask Type
Avg. PSNR ↑
Avg. SSIM ↑
Avg. LPIPS ↓
(1)
Unchanged
29.66
.906
.074
(2)
Dilate 5 pixels
30.01
.908
.074
(3)
Convert to 16×16 patches
29.93
.906
.077
(4)
Misclassify 10% of (3)
29.35
.901
.081
Appendix
Table 4: Quantitative Comparisons between Different Types of GT Masks. Inaccurate object boundaries in masks can still yield comparable results, whereas region misclassification cannot.
Method
Avg. PSNR ↑
Avg. SSIM ↑
Avg. LPIPS ↓
3DGS
15.93
.624
.545
3DGS w/ GT Masks (Upper Bound)
28.84
.992
.117
Ours
24.85
.820
.229
Appendix
Table 5: Extreme Stress Test where Distractors Occupy More Than 50% of the View. Our method can successfully recover the appearance by avoiding catastrophic failures.
Figure 11: Comparisons on Challenging Cases. Our method achieves better visual quality in most regions ( e.g . , walls and floors) with default settings, while it can further capture frequently occluded details ( e.g . , shoes and plaid patterns) by increasing warm-up steps ( † ).
Figure 12: Comparisons on the Phototourism Dataset ( Martin-Brualla et al., 2021 ) . The first , second , and third best values are highlighted. Our method can easily adapt to scenes with appearance variations by assigning embeddings to each image and Gaussian.
Figure 13: A Failure Case. Without prior knowledge from the generative model, 3DGS and its variants cannot correctly predict the scene content that has never been observed before.
Figure 14: Qualitative Results of View Synthesis on Different Datasets.
Figure 15: Qualitative Results of Static Map Generation on the RobustNeRF Dataset.
Figure 16: Qualitative Results of Static Map Generation on the On-the-go Dataset.
Recent advances in 3D Gaussian Splatting (3DGS) have enabled impressive real-time photorealistic rendering. However, conventional training pipelines inherently assume full multi-view consistency among input images, which makes them sensitive to distractors that violate this assumption and cause visual artifacts. In this work, we revisit an underexplored aspect of 3DGS: its inherent ability to suppress inconsistent signals. Building on this insight, we propose PDF-GS (Progressive Distractor Filtering for Robust 3D Gaussian Splatting), a framework that amplifies this self-filtering property through a progressive multi-phase optimization. The progressive filtering phases gradually remove distractors by exploiting discrepancy cues, while the following reconstruction phase restores fine-grained, view-consistent details from the purified Gaussian representation. Through this iterative refinement, PDF-GS achieves robust, high-fidelity, and distractor-free reconstructions, consistently outperforming baselines across diverse datasets and challenging real-world conditions. Moreover, our approach is lightweight and easily adaptable to existing 3DGS frameworks, requiring no architectural changes or additional inference overhead, leading to a new state-of-the-art performance. The code is publicly available at https://github.com/kangrnin/PDF-GS.
3D Gaussian Splatting (3DGS) enables efficient training and fast novel view synthesis in static environments. To address challenges posed by transient objects, distractor-free 3DGS methods have emerged and shown promising results when dense image captures are available. However, their performance degrades significantly under sparse input conditions. This limitation primarily stems from the reliance on the color residual heuristics to guide the training, which becomes unreliable with limited observations. In this work, we propose a framework to enhance distractor-free 3DGS under sparse-view conditions by incorporating rich prior information. Specifically, we first adopt the geometry foundation model VGGT to estimate camera parameters and generate a dense set of initial 3D points. Then, we harness the attention maps from VGGT for efficient and accurate semantic entity matching. Additionally, we utilize Vision-Language Models (VLMs) to further identify and preserve the large static regions in the scene. We also demonstrate how these priors can be seamlessly integrated into existing distractor-free 3DGS methods. Extensive experiments confirm the effectiveness and robustness of our approach in mitigating transient distractors for sparse-view 3DGS training.
Yi Gu, Zhaorui Wang, Jiahang Cao +4
The Hong Kong University of Science and Technology (Guangzhou) · University of Macau
In-the-wild 3D Gaussian Splatting remains challenging due to transient distractors and illumination-induced cross-view appearance inconsistencies. Existing methods mainly rely on image-level masking to suppress unreliable supervision, but masking alone cannot fully eliminate residual occlusions or resolve illumination-induced inconsistencies, both of which can introduce conflicting cross-view gradients. These unresolved conflicts may destabilize Gaussian optimization and lead to visible reconstruction artifacts. We propose a conflict-aware 3DGS framework that addresses this problem from both image-space supervision and gradient-level optimization. Semantic Consistency-Guided Masking learns pixel-wise consistency scores to adaptively refine prior masks and suppress unreliable supervision before gradient formation. A dual-view Conflict-Aware Gradient Harmonization strategy further reconciles view-specific gradients by mutually rotating them into an orthogonal configuration, reducing negative directional interference across views. We also introduce conflict-aware densification and pruning to stabilize Gaussian growth and remove persistently conflicting primitives. Extensive experiments on standard in-the-wild benchmarks demonstrate that our method achieves state-of-the-art rendering quality under complex transient distractors and cross-view inconsistencies.
Yulei Kang, Tianze Zhu, Jian-Fang Hu +2
Sun Yat-sen University · Northeastern University · Guangdong Province Key Laboratory of Information Security Technology, China +1