GS4City: Hierarchical Semantic Gaussian Splatting via City-Model Priors

Qilin Zhang; Jinyu Zhu; Olaf Wysocki; Benjamin Busam; Boris Jutzi

cs.CVApr 13, 2026

GS4City: Hierarchical Semantic Gaussian Splatting via City-Model Priors

Authors: Qilin Zhang, Jinyu Zhu, Olaf Wysocki, Benjamin Busam, Boris Jutzi

Organizations: Technical University of Munich (TUM) · Munich Center for Machine Learning (MCML) · Karlsruhe Institute of Technology (KIT) · CV4DT, University of Cambridge

Abstract

Recent semantic 3D Gaussian Splatting (3DGS) methods primarily rely on 2D foundation models, often yielding ambiguous boundaries and limited support for structured urban semantics. While city models such as CityGML encode hierarchically organized semantics together with building geometry, these labels cannot be directly mapped to Gaussian primitives. We present GS4City, a hierarchical semantic Gaussian Splatting method that incorporates city-model priors for urban scene understanding. GS4City derives reliable image-aligned masks from Level of Detail (LoD) 3 CityGML models via two-pass raycasting, explicitly using parent-child relations to validate and recover fine-grained facade elements. It then fuses these geometry-grounded masks with foundation-model predictions to establish scene-consistent instance correspondences, and learns a compact identity encoding for each Gaussian under joint 2D identity supervision and 3D spatial regularization. Experiments on the TUM2TWIN and Gold Coast datasets show that GS4City effectively incorporates structured building semantics into Gaussian scene representations, outperforming existing 2D-driven semantic 3DGS baselines, including LangSplat and Gaga, by up to 15.8 IoU points in coarse building segmentation and 14.2 mIoU points in fine-grained semantic segmentation. By bridging structured city models and photorealistic Gaussian scene representations, GS4City enables semantically queryable and structure-aware urban reconstruction. Code is available at https://github.com/Jinyzzz/GS4City.

Figures & tables

Figure 1: GS4City augments a photorealistic Gaussian scene with structured CityGML priors to produce hierarchical urban understanding. It enables appearance-robust segmentation, geometry-aligned boundaries, open-vocabulary querying, and city-model metadata integration within a unified 3DGS scene.

Figure 2: Overview of GS4City. Starting from an aligned LoD3 city model, calibrated images, and a pre-trained 3DGS scene, the framework first derives hierarchical city-model masks by Two-Pass Raycasting ( Section 3.1 ). It then generates and filters foundation-model masks, establishes scene-consistent correspondences between building entities, image masks, and Gaussian primitives through Mask Fusion and Instance Linking ( Section 3.2 ), and finally learns Gaussian Identity Encoding ( Section 3.3 ) for multi-level building parsing, semantic querying, and interactive urban-scene visualization.

Figure 3: Foundation-model mask filtering. Raw image-based masks are compared with the city-model building-support region. High-overlap masks are disambiguated by a vision-language model into building-related versus foreground candidates, yielding cleaned image-based masks for cross-view association.

Figure 4: Qualitative results on representative scenes from TUM2TWIN and Gold Coast at both the coarse and fine-grained semantic levels. The first column shows the reference RGB views together with the extended ground truth derived from labeled point clouds and auxiliary 2D pseudo labels. The remaining columns show the predictions of LangSplat, Gaga, and GS4City. GS4City provides more complete Building/Non-Building separation at the coarse level and more accurate fine-grained segmentation of challenging building parts, such as windows and roofs, while avoiding incorrect semantic assignments in non-building regions, such as water surfaces.

Dataset	Method	Coarse Level				Fine-Grained Level
Dataset	Method	IoU $\uparrow$	Acc. $\uparrow$	Prec. $\uparrow$	Rec. $\uparrow$	mIoU $\uparrow$	Door	Roof	Wall	Window	Car	Person	Tree
TUM2TWIN	LangSplat	67.5	71.2	86.2	75.8	21.0	0.1	3.3	39.3	12.9	61.7	4.5	30.5
	Gaga	77.3	80.8	91.2	80.9	22.3	0.0	1.1	59.3	1.2	78.6	6.1	18.3
	GS4City (Ours)	93.1	94.6	96.2	96.6	30.5	3.6	19.8	59.2	25.6	82.3	9.0	18.8
Gold Coast	LangSplat	53.9	71.6	86.7	59.5	30.2	0.0	5.0	36.4	0.0	27.3	–	83.8
	Gaga	80.5	87.7	87.6	91.1	23.5	0.0	20.6	2.2	0.0	33.3	–	72.9

Table 1: Overall results at the coarse and fine-grained semantic levels. Coarse-level metrics evaluate Building/Non-Building grouping, while the fine-grained part reports overall mIoU together with per-class IoU. All values are reported in %.

Dataset	Method	Per-class Prec. $\uparrow$
Dataset	Method	Door	Roof	Wall	Wind.	Car	Pers.	Tree
TUM2TWIN	LangSplat	0.1	7.6	70.4	22.5	61.7	4.7	43.9
	Gaga	0.0	2.8	76.4	30.0	78.6	23.0	41.3
	GS4City (Ours)	5.4	21.7	81.6	39.9	82.3	39.1	55.0
Gold Coast	LangSplat	–	23.5	57.1	0.0	27.3	–	86.1
	Gaga	–	23.7	17.3	3.4	33.3	–	99.2

Table 2: Per-class precision at the fine-grained semantic level. Classes absent from a dataset split are marked as “–”. All values are reported in %.

Method	$m_{\mathrm{view}}$	mIoU $\uparrow$	Time (min) $\downarrow$	mIoU/min $\uparrow$
LangSplat	–	15.8	39.2	0.40
Gaga	–	18.7	43	0.43
GS4City (Ours)	1	25.8	84.7	0.30
GS4City (Ours)	3	24.0	55.0	0.44
GS4City (Ours)	5	21.9	51.8	0.42

Table 3: Ablation on the minimal-view filter in cross-view association on a challenging TUM2TWIN subset with strong tree occlusions. LangSplat and Gaga are included as reference baselines. The reported mIoU is measured at the fine-grained semantic level.

3D Scene Representation

3D scene representation studies how to encode geometry, appearance, semantics, and motion for reconstruction, rendering, generation, and interaction.

See all 235 papers

3D Gaussian Splatting for Fine-grained Urban Scene Reconstruction via Semantic-guided Density Control and Loss Design
Oct 12, 2026Yueling Liao, San Jiang, Xing Zhang +1Urban Scene Reconstruction 3D Gaussian Splatting
School of Computer Science, China University of Geosciences (Wuhan), Wuhan, 430074, China · School of Architecture and Urban Planning, Shenzhen University, Shenzhen, 518060, China · Guangdong Key Laboratory of Urban Informatics, Shenzhen University, Shenzhen, 518060, China +1
PointGauss: Point Cloud-Guided Multi-Object Segmentation for Gaussian Splatting
Aug 1, 2025Wentao Sun, Yiping Chen, John S. Zelek +13D Gaussian Splatting 3DGS Segmentation
University of Waterloo, Waterloo, ON N2L 3G1, Canada · Sun Yat-Sen University, Zhuhai 519082, China
Agentic Building-Aware Satellite Gaussian Splatting for Auditable Urban DSM Reconstruction
Sep 22, 2026Wentao Sun, Zhengsen Xu, Yiping Chen +23D Reconstruction Urban Scene Reconstruction
University of Waterloo, Department of Systems Design Engineering, Waterloo, Canada · University of Calgary, Department of Geomatics Engineering, Calgary, Canada · Sun Yat-sen University, School of Geospatial Engineering and Science, Zhuhai, China

Oct 12, 2026J/K move · Enter open · S save

3D Gaussian Splatting for Fine-grained Urban Scene Reconstruction via Semantic-guided Density Control and Loss Design

Yueling Liao, San Jiang, Xing Zhang, Zhongwen Hu

School of Computer Science, China University of Geosciences (Wuhan), Wuhan, 430074, China · School of Architecture and Urban Planning, Shenzhen University, Shenzhen, 518060, China · Guangdong Key Laboratory of Urban Informatics, Shenzhen University, Shenzhen, 518060, China+1

Urban Scene Reconstruction 3D Gaussian Splatting 3D Reconstruction Adaptive Gaussian Densification

Open paper AskPDF arXiv

3D Gaussian Splatting (3DGS) has been becoming a promising 3D reconstruction technique because of its advantage of high-fidelity rendering effect with high efficiency. However, it still suffers from coarse density control and semantically blind loss functions when reconstructing the fine-grained structures in complex urban scenes. Thus, this study proposes SG-GS, a semantic-guided 3DGS method for fine-grained structure reconstruction. It takes the semantic segmentation deduced by the Segment Anything Model (SAM) as a constraint and systematically integrates it into both Gaussian density control and loss function design. On the one hand, a semantic-constrained adaptive density control mechanism is constructed, which dynamically allocates gradient thresholds to different semantic regions of the scene and introduces an additional boundary-enhanced densification strategy, so that new Gaussians are directed to structurally critical regions. On the other hand, a multi-feature guided loss function is designed, and fine-grained supervision signals are gradually introduced through a phased progressive training strategy, alleviating the artifacts caused by the lack of semantic constraints in the basic loss. Experiments on three datasets show that the proposed method consistently improves reconstruction accuracy. Compared with the original 3DGS, the PSNR increases by approximately 2.7% to 5.2%, while artifacts on linear structures and boundary blurring are effectively mitigated. The method can serve the fine-grained 3D reconstruction of urban scenes.

Abstract

3D Scene Representation

Explore similar work

3D Gaussian Splatting for Fine-grained Urban Scene Reconstruction via Semantic-guided Density Control and Loss Design

PointGauss: Point Cloud-Guided Multi-Object Segmentation for Gaussian Splatting

Agentic Building-Aware Satellite Gaussian Splatting for Auditable Urban DSM Reconstruction