3D Gaussian Splatting for Fine-grained Urban Scene Reconstruction via Semantic-guided Density Control and Loss Design

Yueling Liao; San Jiang; Xing Zhang; Zhongwen Hu

cs.CVOct 12, 2026

3D Gaussian Splatting for Fine-grained Urban Scene Reconstruction via Semantic-guided Density Control and Loss Design

Authors: Yueling Liao, San Jiang, Xing Zhang, Zhongwen Hu

Organizations: School of Computer Science, China University of Geosciences (Wuhan), Wuhan, 430074, China · School of Architecture and Urban Planning, Shenzhen University, Shenzhen, 518060, China · Guangdong Key Laboratory of Urban Informatics, Shenzhen University, Shenzhen, 518060, China · MNR Key Laboratory for Geo-Environmental Monitoring of Great Bay Area, Shenzhen University, Shenzhen, 518060, China

Abstract

3D Gaussian Splatting (3DGS) has been becoming a promising 3D reconstruction technique because of its advantage of high-fidelity rendering effect with high efficiency. However, it still suffers from coarse density control and semantically blind loss functions when reconstructing the fine-grained structures in complex urban scenes. Thus, this study proposes SG-GS, a semantic-guided 3DGS method for fine-grained structure reconstruction. It takes the semantic segmentation deduced by the Segment Anything Model (SAM) as a constraint and systematically integrates it into both Gaussian density control and loss function design. On the one hand, a semantic-constrained adaptive density control mechanism is constructed, which dynamically allocates gradient thresholds to different semantic regions of the scene and introduces an additional boundary-enhanced densification strategy, so that new Gaussians are directed to structurally critical regions. On the other hand, a multi-feature guided loss function is designed, and fine-grained supervision signals are gradually introduced through a phased progressive training strategy, alleviating the artifacts caused by the lack of semantic constraints in the basic loss. Experiments on three datasets show that the proposed method consistently improves reconstruction accuracy. Compared with the original 3DGS, the PSNR increases by approximately 2.7% to 5.2%, while artifacts on linear structures and boundary blurring are effectively mitigated. The method can serve the fine-grained 3D reconstruction of urban scenes.

Figures & tables

Figure 1: Overview of the proposed SG-GS framework. SAM instance masks are classified into four region types and converted into a pixel importance weight map (blue); the region labels drive the semantic-guided adaptive density control (orange) while the weight map drives the multi-feature guided loss with a progressive schedule (green); the optimized Gaussians are rendered by the differentiable rasterizer (purple).

Figure 2: Visualization of SAM semantic region classification.

Figure 3: Illustration of the Gaussian–region mapping.

Figure 4: Adaptive-threshold density control strategy.

Figure 5: Weight schedule curves of the phased progressive training of MF-Loss.

Name	Zeche	Derrick	DroneSplat
Number of images	147	292	24
POS data	Yes	Yes	Yes
Number of sparse points ( $\times 10^{4}$ )	15	20.8	1.6
Image size (pixels)	8,300 $\times$ 5,500	6,000 $\times$ 4,000	1,920 $\times$ 1,080
Train/test split	128/19	255/37	—

Table 1: Details of the datasets.

Figure 6: Example images of the datasets.

Parameter	Value
Rendering resolution (-r)	1
Training iterations	30,000
SH degree	3
Base gradient threshold $\tau$	0.0002
Region threshold scaling factors $\alpha$	0.5 / 0.7 / 0.9 / 1.5
Gradient magnitude threshold $\theta_{g}$	30

Table 2: Parameter settings.

Metric	Description
Structural Similarity Index (SSIM)	Similarity measure based on image structural information
Peak Signal-to-Noise Ratio (PSNR)	Objective criterion based on the mean squared error
Learned Perceptual Image Patch Similarity (LPIPS)	Image similarity measure based on human perception

Table 3: Evaluation metrics.

Figure 7: Intermediate results of semantic preprocessing on the three datasets.

Figure 8: The comparison of Gaussian distribution of 3DGS and SG-GS.

Figure 9: Progressively zoomed comparison of the Gaussian distributions inside the ROIs.

Figure 10: 3D shapes of the Gaussians inside the zoomed 3D cube window.

Method	Zeche			Derrick			DroneSplat
Method	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
3DGS	24.26	0.79	0.20	25.84	0.87	0.14	34.98	0.96	0.07
2DGS	24.36	0.76	0.24	25.56	0.85	0.18	31.55	0.94	0.11
FastGS	24.07	0.77	0.23	26.25	0.85	0.21	33.05	0.93	0.13
Scaffold-GS	26.20	0.81	0.19	26.35	0.86	0.16	33.68	0.94	0.10
SG-GS (ours)	24.93	0.81	0.18	27.18	0.90	0.11	36.51	0.98	0.05

Table 4: Quantitative comparison results.

Figure 11: Qualitative results on the Derrick dataset.

Figure 12: Qualitative results on the Zeche dataset.

Figure 13: Qualitative results on the DroneSplat dataset.

Panel of Figure 14 — Figure 14: Evolution of the number of Gaussians and PSNR with the number of iterations.

Model	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	#Gaussians ( $\times 10^{4}$ )
3DGS (baseline)	24.26	0.79	0.20	304.10
+SG-ADC	24.75	0.81	0.18	411.93
+MF-Loss	24.60	0.80	0.20	309.49
+SG-ADC+MF-Loss (full)	24.93	0.81	0.18	423.60

Table 5: Overall ablation results.

Threshold strategy	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	#Gaussians ( $\times 10^{4}$ )
Globally uniform threshold	24.58	0.81	0.19	392.53
Region-adaptive threshold	24.93	0.81	0.18	423.60

Table 6: Ablation results of the adaptive threshold.

Loss function	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
L1+D-SSIM (baseline loss)	24.26	0.79	0.20
+Weight-map weighting	24.78	0.81	0.19
+Edge-preservation loss	24.58	0.81	0.18
+Frequency-domain loss (full MF-Loss)	24.93	0.81	0.18

Table 7: Ablation results of the multi-feature guided loss.

Training strategy	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
One-shot introduction	18.90	0.74	0.29
Progressive introduction	24.93	0.81	0.18

Table 8: Ablation results of the training strategy.

Figure 15: Qualitative comparison of the training strategy ablation.

Urban Scene Reconstruction

See all 14 papers

GS4City: Hierarchical Semantic Gaussian Splatting via City-Model Priors
Apr 13, 2026Qilin Zhang, Jinyu Zhu, Olaf Wysocki +23D Scene Representation 3D Scene Understanding
Technical University of Munich (TUM) · Munich Center for Machine Learning (MCML) · Karlsruhe Institute of Technology (KIT) +1
GaussianDS: Depth-supervised Semantic Gaussian Splatting for Scene Understanding
Aug 22, 2026Yufei Zhang, Chenlu Zhan, Hongwei Wang3D Gaussian Splatting 3D Scene Understanding
Zhejiang University
PointGauss: Point Cloud-Guided Multi-Object Segmentation for Gaussian Splatting
Aug 1, 2025Wentao Sun, Yiping Chen, John S. Zelek +13D Gaussian Splatting 3DGS Segmentation
University of Waterloo, Waterloo, ON N2L 3G1, Canada · Sun Yat-Sen University, Zhuhai 519082, China

Apr 13, 2026J/K move · Enter open · S save

GS4City: Hierarchical Semantic Gaussian Splatting via City-Model Priors

Qilin Zhang, Jinyu Zhu, Olaf Wysocki, Benjamin Busam+1

Technical University of Munich (TUM) · Munich Center for Machine Learning (MCML) · Karlsruhe Institute of Technology (KIT)+1

3D Scene Representation 3D Scene Understanding 3D Semantic Segmentation Semantic Segmentation

Open paper AskPDF arXiv

Recent semantic 3D Gaussian Splatting (3DGS) methods primarily rely on 2D foundation models, often yielding ambiguous boundaries and limited support for structured urban semantics. While city models such as CityGML encode hierarchically organized semantics together with building geometry, these labels cannot be directly mapped to Gaussian primitives. We present GS4City, a hierarchical semantic Gaussian Splatting method that incorporates city-model priors for urban scene understanding. GS4City derives reliable image-aligned masks from Level of Detail (LoD) 3 CityGML models via two-pass raycasting, explicitly using parent-child relations to validate and recover fine-grained facade elements. It then fuses these geometry-grounded masks with foundation-model predictions to establish scene-consistent instance correspondences, and learns a compact identity encoding for each Gaussian under joint 2D identity supervision and 3D spatial regularization. Experiments on the TUM2TWIN and Gold Coast datasets show that GS4City effectively incorporates structured building semantics into Gaussian scene representations, outperforming existing 2D-driven semantic 3DGS baselines, including LangSplat and Gaga, by up to 15.8 IoU points in coarse building segmentation and 14.2 mIoU points in fine-grained semantic segmentation. By bridging structured city models and photorealistic Gaussian scene representations, GS4City enables semantically queryable and structure-aware urban reconstruction. Code is available at https://github.com/Jinyzzz/GS4City.

Abstract

Urban Scene Reconstruction

Explore similar work

GS4City: Hierarchical Semantic Gaussian Splatting via City-Model Priors

GaussianDS: Depth-supervised Semantic Gaussian Splatting for Scene Understanding

PointGauss: Point Cloud-Guided Multi-Object Segmentation for Gaussian Splatting