cs.CVMar 8, 2026
Save3DGS-HPC: Distractor-free 3D Gaussian Splatting with Hybrid Patch-wise Classification
Organizations: Sun Yat-sen University, Guangzhou, China · Texas A&M University, College Station, USA · Cardiff University, Wales, UK · Centre for Perceptual and Interactive Intelligence, Hong Kong, China ·
Abstract
3D Gaussian Splatting (3DGS) has demonstrated remarkable performance in novel view synthesis and 3D scene reconstruction, but its quality often degrades in real-world environments due to transient distractors, such as moving objects and varying shadows. Existing methods commonly introduce semantic priors from pre-trained vision models either to group pixels into coherent regions or to define perceptual error metrics. However, semantic grouping is often misaligned with the binary static/transient distinction, while perceptual features can be fragile under appearance perturbations introduced during 3DGS optimization. We propose 3DGS-HPC, a framework that addresses these issues by combining two complementary principles: a patch-wise classification strategy that leverages local spatial consistency for robust region-level decisions, and a hybrid classification metric that adaptively integrates photometric and perceptual cues for more reliable separation. Extensive experiments demonstrate the superiority and robustness of our method in mitigating distractors to improve 3DGS-based novel view synthesis. Our project page is https://cnhaox.github.io/3DGS-HPC/ .
Figures & tables
Figure 1: Overview of Hybrid Patch-wise Classification (HPC) . Given training images disturbed by transient distractors, our method (a) optimizes 3DGS following standard practice, (b) classifies static and transient regions using a novel hybrid metric that combines the strengths of photometric and perceptual cues, and (c) performs classification at the patch level to exploit local spatial consistency, thereby enhancing robustness and efficiency without relying on external semantic segmentation or detection priors for region grouping.
Figure 2: Visualization of Different Partitioning Strategies. When grouping pixels into multiple semantically consistent regions with blue borders, existing methods (b-d) fail to faithfully represent transient regions ( e.g . , pedestrians and shadows) due to semantic mismatch between different tasks, while our patch-wise partition strategy (e) can accurately extract these regions.
Figure 3: Visualization of Different Error Metrics. Given training images with distractors and corresponding rendered images, traditional photometric metrics (L1) produce noisy static maps due to low-level appearance ambiguities ( e.g . , similar colors). Instead, perceptual metrics based on vision foundation models (DINOv2) produce clean static maps with clear boundaries.
Figure 4: Comparison of different classification approaches with different error metrics. Bottom row: Although perceptual error metrics are effective in capturing semantic differences, they can produce anomalous results in textureless regions ( e.g . , green wall and white tablecloth) due to visual perturbability ( e.g . , blurring) between training and rendered images. Top row: In contrast, while inherently noisier, photometric error metrics demonstrate greater reliability in estimating the overall proportion of transient pixels within an image, thus guiding perception error metrics for more accurate classification.
Figure 5: Quantitative and Qualitative Comparisons on the RobustNeRF Dataset ( Sabour et al., 2023 ) . The first , second , and third best values are highlighted. Quantitatively, our method not only improves the performance of native 3DGS, but also outperforms previous methods on most quality metrics. Qualitatively, our method can better preserve static details while ignoring distractors.
Figure 6: Quantitative and Qualitative Comparisons on the On-the-go Dataset ( Ren et al., 2024 ) . The first , second , and third best values are highlighted. Our method consistently achieves top-three quantitative results in scenes with varying distractor ratios. In terms of visual quality, our method can better reconstruct static content, while previous methods suffer from missing or distorted details.
Figure 7: Comparisons of Partitioning Strategies. Pixel-wise method (a) produces noisy static maps, while other methods (b-d) confuse the desktop with transient shadows. Instead, our method (e) accurately distinguishes between transient and static elements.
Figure 8: Comparisons of Classification Metrics. (a) fails to distinguish distractors ( e.g . , clothing) that closely resemble static backgrounds in color, while (b) tends to incorrectly remove certain static regions ( e.g . , wall surfaces). A naive combination (d) does not yield meaningful improvements. In contrast, our method (e) effectively separates the transient region from the static one.
Figure 9: Performance of Different Patch Sizes (PS). Patch sizes that are extremely small ( ) or large ( ) will lead to suboptimal quantitative results, and the resulting mask will fail to accurately label and filter out transient distractors ( e.g . , people).
Figure 10: Performance of Different Warm-up Steps (WS). When the number of warm-up steps exceeds , the model is at risk of “memorizing” transient distractors, leading to artifacts in the rendered results.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| Vision Model | Extracted Feature Layers |
| VGG-16 ( Simonyan and Zisserman, 2014 ) | {conv1_2, conv2_2, conv3_3, conv4_3, conv5_3} |
| ResNet-18 ( He et al., 2016 ) | {layer1, layer2, layer3, layer4} |
| DINOv2 ViT-S/14 ( Oquab et al., 2023 ) | {layer2, layer5, layer8, layer11} |
Table 1: Settings for Our Perceptual Error Metrics.
| RobustNeRF (Avg.) | On-the-go (Avg.) | ||||||||
| PSNR | SSIM | LPIPS | Train hrs. | PSNR | SSIM | LPIPS | Train hrs. | ||
| (a) | Refinement Variant | 29.74 | .912 | .074 | 2.53 | 23.69 | .848 | .111 | 2.09 |
| (b) | Ours | 29.81 | .913 | .074 | 0.22 | 23.86 | .849 | .106 | 0.25 |
Table 2: Performance of Our Patch-wise Classification vs . Its Refinement Variant. Our method can significantly reduce training time and have slightly better performance in visual metrics.
| Avg. Train Time (min) | Avg. GPU Mem (GB) | |
| SLS-mlp ( Sabour et al., 2025 ) | 11.91 | 3.60 |
| WildGaussians ( Kulhanek et al., 2024 ) | 24.10 | 16.02 |
| T-3DGS ( Markin et al., 2024 ) | 26.75 | 4.36 |
| Ours (VGG) | 16.35 | 2.09 |
| Ours (ResNet) | 13.29 | 1.98 |
| Ours (DINOv2) | 17.52 | 2.05 |
Table 3: Quantitative Efficiency Comparisons between Different Methods. Our method outperforms most other methods in training time and memory usage.
| GT Mask Type | Avg. PSNR | Avg. SSIM | Avg. LPIPS | |
| (1) | Unchanged | 29.66 | .906 | .074 |
| (2) | Dilate 5 pixels | 30.01 | .908 | .074 |
| (3) | Convert to patches | 29.93 | .906 | .077 |
| (4) | Misclassify of (3) | 29.35 | .901 | .081 |
Table 4: Quantitative Comparisons between Different Types of GT Masks. Inaccurate object boundaries in masks can still yield comparable results, whereas region misclassification cannot.
| Method | Avg. PSNR | Avg. SSIM | Avg. LPIPS |
| 3DGS | 15.93 | .624 | .545 |
| 3DGS w/ GT Masks (Upper Bound) | 28.84 | .992 | .117 |
| Ours | 24.85 | .820 | .229 |
Table 5: Extreme Stress Test where Distractors Occupy More Than 50% of the View. Our method can successfully recover the appearance by avoiding catastrophic failures.
Figure 11: Comparisons on Challenging Cases. Our method achieves better visual quality in most regions ( e.g . , walls and floors) with default settings, while it can further capture frequently occluded details ( e.g . , shoes and plaid patterns) by increasing warm-up steps ( ).
Figure 12: Comparisons on the Phototourism Dataset ( Martin-Brualla et al., 2021 ) . The first , second , and third best values are highlighted. Our method can easily adapt to scenes with appearance variations by assigning embeddings to each image and Gaussian.
Figure 13: A Failure Case. Without prior knowledge from the generative model, 3DGS and its variants cannot correctly predict the scene content that has never been observed before.
Figure 14: Qualitative Results of View Synthesis on Different Datasets.
Figure 15: Qualitative Results of Static Map Generation on the RobustNeRF Dataset.
Figure 16: Qualitative Results of Static Map Generation on the On-the-go Dataset.
More in this topic
3D Gaussian Splatting
3D Gaussian Splatting represents scenes as optimizable 3D Gaussian primitives for fast, photorealistic rendering from new viewpoints.
See all 1,013 papers