TreeGaussian: Tree-Guided Cascaded Contrastive Learning for Hierarchical Consistent 3D Gaussian Scene Segmentation and Understanding

Jingbin You; Zehao Li; Hao Jiang; Xinzhu Ma; Shuqin Gao; Honglong Zhao; Congcong Zheng; Tianlu Mao

cs.CVMar 31, 2026

TreeGaussian: Tree-Guided Cascaded Contrastive Learning for Hierarchical Consistent 3D Gaussian Scene Segmentation and Understanding

Authors: Jingbin You, Zehao Li, Hao Jiang, Xinzhu Ma, Shuqin Gao, Honglong Zhao, Congcong Zheng, Tianlu Mao,

Organizations: Institute of Computing Technology, Chinese Academy of Sciences, ICT · University of Chinese Academy of Sciences, UCAS · Beihang University, BUAA

Abstract

3D Gaussian Splatting (3DGS) has emerged as a real-time, differentiable representation for neural scene understanding. However, existing 3DGS-based methods struggle to represent hierarchical 3D semantic structures and capture whole-part relationships in complex scenes. Moreover, dense pairwise comparisons and inconsistent hierarchical labels from 2D priors hinder feature learning, resulting in suboptimal segmentation. To address these limitations, we introduce TreeGaussian, a tree-guided cascaded contrastive learning framework that explicitly models hierarchical semantic relationships and reduces redundancy in contrastive supervision. By constructing a multi-level object tree, TreeGaussian enables structured learning across object-part hierarchies. In addition, we propose a two-stage cascaded contrastive learning strategy that progressively refines feature representations from global to local, mitigating saturation and stabilizing training. A Consistent Segmentation Detection (CSD) mechanism and a graph-based denoising module are further introduced to align segmentation modes across views while suppressing unstable Gaussian points, enhancing segmentation consistency and quality. Extensive experiments, including open-vocabulary 3D object selection, 3D point cloud understanding, and ablation studies, demonstrate the effectiveness and robustness of our approach.

Figures & tables

Figure 1: Motivation illustration. (a) Flat contrastive learning isolates feature spaces for object wholes and parts, limiting their hierarchical interaction. (b) Fused contrastive learning merges feature spaces but suffers from oversaturation and instability due to dense pairwise comparisons. (c) Cascaded contrastive learning (our method) preserves semantic hierarchy while minimizing contrastive redundancy, enabling efficient and stable optimization. Solid arrows indicate implicit contrastive interactions among clusters; dashed arrows denote explicit hierarchical indexing relationships.

Figure 2: Overview of our method. (a) Constructing object-tree from multi-view images using SAM to capture structured relationships between object parts and wholes. (b) Two-stage cascaded contrastive learning strategy to progressively optimize the instance feature of each Gaussian point. (c) Graph-based denoising is applied to each language-mapped instance cluster for improving the multi-view rendering quality.

Figure 3: Consistent Segmentation Detection (CSD) for local contrastive learning. The blue curve shows the raw Split Number and the red curve shows the smoothed reference across views. Views where the blue curve lies above the red reference are treated as over segmentation (apply only $\mathcal{L}_{pull}^{2}$ ), while views where it lies below are treated as under segmentation (apply only $\mathcal{L}_{push}^{2}$ ). When the blue curve is close to the reference, we treat it as optimal segmentation (apply $\mathcal{L}_{pull}^{2}+\mathcal{L}_{push}^{2}$ ).

Scale	Methods	Type	mIoU(%)	mAcc(%)
whole	LangSplat [ 33 ]	Language distillation	9.66	12.41
whole	LEGaussians [ 37 ]	Language distillation	16.21	23.82
whole	OmniSeg3D-GS [ 44 ]	Fused contrastive learning	38.55	63.29
whole	OpenGaussian [ 40 ]	Flat contrastive learning	38.36	51.43
whole	InstanceGaussian [ 23 ]	Flat contrastive learning	45.30	58.44
whole	Ours	Cascaded contrastive learning	51.78	70.51

Table 1: Quantitative comparison of open-vocabulary 3D object selection on the Lerf_ovs dataset. We report mIoU(%) and [email protected](%) scores.

Figure 4: Qualitative comparison of the rendered instance feature maps. Our method achieves better global feature consistency across objects (cup and spoon) at the whole scale and exhibits clearer feature separation at the part scale.

Figure 5: Qualitative comparison of rendered local objects. Our method produces cleaner and more accurate segmentation results compared to baselines, effectively reducing noise and preserving fine-grained structures.

Figure 6: Qualitative comparison of click-based 3D object selection. Compared to OpenGaussian, our method produces more accurate and hierarchically consistent results at both whole and part scales.

Methods	19 Classes			15 Classes			10 Classes
Methods	whole	part	subpart	whole	part	subpart	whole	part	subpart
OmniSeg3D-GS [ 44 ]	20.56	15.29	12.11	23.74	17.83	18.79	32.41	25.73	18.79
OpenGaussian [ 40 ]	24.73	18.12	14.73	30.13	20.91	17.24	38.29	33.50	25.77
InstanceGaussian [ 23 ]	40.66	27.20	17.20	42.51	23.76	19.93	47.94	30.63	24.24
Ours	41.61	28.99	18.32	42.57	29.28	20.01	49.80	38.76	27.66

Table 2: Quantitative comparison on ScanNet-v2 dataset. We report mIoU(%) scores.

Methods	19 Classes			15 Classes			10 Classes
Methods	whole	part	subpart	whole	part	subpart	whole	part	subpart
OmniSeg3D-GS [ 44 ]	38.51	25.39	23.13	36.02	29.06	30.49	48.32	38.91	30.49
OpenGaussian [ 40 ]	41.54	28.89	24.66	48.25	31.89	27.87	55.19	45.17	39.88
InstanceGaussian [ 23 ]	54.01	40.73	27.78	59.15	37.63	29.07	64.01	48.89	37.50
Ours	54.38	42.11	28.82	59.94	42.95	32.61	65.41	54.55	40.35

Table 3: Quantitative comparison on ScanNet-v2 dataset. We report mAcc(%) scores.

Scale	w/o $\mathcal{L}_{local}$		w/o Denoising		Ours
Scale	mIoU	mAcc	mIoU	mAcc	mIoU	mAcc
whole	47.05	62.92	48.23	66.63	51.78	70.51
part	42.37	59.02	43.45	59.66	44.10	60.44

Table 4: Quantitative ablation results of Local Contrastive Loss and Graph-based Denoising on the Lerf_ovs dataset. We report the mIoU(%) and [email protected](%) scores.

Scale	w/o $\mathcal{L}_{pull}^{2}$		w/o $\mathcal{L}_{push}^{2}$		w/o CSD		Ours
Scale	mIoU	mAcc	mIoU	mAcc	mIoU	mAcc	mIoU	mAcc
whole	56.63	69.49	48.22	69.49	47.51	59.32	62.61	77.97
part	52.14	69.49	52.51	69.49	51.54	69.49	53.68	74.58

Table 5: Quantitative ablation results on the teatime scene, evaluating the effect of each local contrastive learning component.

Figure 7: Qualitative ablation results with Consistent Segmentation Detection (CSD) mechanism and Graph-based Gaussian Points Denoising. The results demonstrate that CSD reduces oversegmentation and enhances consistency, while the denoising module effectively suppresses distant clutter and improves clarity.

Scene	Annotated Object Categories
figurines	jake, pirate hat, pikachu, rubber duck with hat, porcelain hand, red apple, tesla door handle, waldo, bag, toy cat statue, miffy, green apple, pumpkin, rubics cube, old camera, rubber duck with buoy, red toy chair, pink ice cream, spatula, green toy chair , and toy elephant .
teatime	sheep, yellow pouf, stuffed bear, coffee mug, tea in a glass, apple, coffee, hooves, bear nose, dall-e brand, plate, paper napkin, three cookies , and bag of cookies .
ramen	nori, sake cup, kamaboko, corn, spoon, egg, onion segments, plate, napkin, bowl, glass of water, hand, chopsticks , and wavy noodles .
waldo_kitchen	stainless steel pots, dark cup, refrigerator, frog cup, pot, spatula, plate, spoon, toaster, ottolenghi, plastic ladle, sink, ketchup, cabinet, red cup, pour-over vessel, knife , and yellow desk .

Table S1 : Annotated object categories for each scene in the Lerf_ovs dataset.

19 Classes	wall, floor, cabinet, bed, chair, sofa, table, door, window, bookshelf, picture, counter, desk, curtain, refrigerator, shower curtain, toilet, sink, bathtub
15 Classes	wall, floor, cabinet, bed, chair, sofa, table, door, window, bookshelf, counter, desk, toilet, sink, bathtub
10 Classes	wall, floor, bed, chair, sofa, table, door, window, toilet, sink

Table S2 : Annotated semantic categories for 10 scenes in the ScanNet-v2 dataset.

Figure S1 : More detailed visualization of multi-level object-tree construction

Methods	Type	mIoU (whole)	mAcc (whole)	mIoU (part)	mAcc (part)
GarField	Fused	49.91	65.62	40.15	59.14
Ours	Cascaded	51.78	70.51	44.10	60.44

Table S3 : Comparison with an additional NeRF-based method on Lerf_ovs dataset.

Dataset	SAM (s/img)	Scale	Training time (min)	VRAM (GB)	CSD	Denoising
Lerf_ovs	26.85	whole / part	30 / 50	7.6 / 10.9	$<$ 1.0 min	1.2 s/cluster
ScanNet-v2	19.15	whole / part / subpart	20 / 30 / 30	9.9 / 14.4 / 16.8	$<$ 1.0 min	0.8 s/cluster

Table S4 : Computational cost breakdown of our pipeline on Lerf_ovs and ScanNet-v2. We report SAM runtime per image, training time and VRAM at different semantic scales, and the overhead of CSD and denoising.

Metric	w/o $\mathcal{L}_{pull}^{2}$	w/o $\mathcal{L}_{push}^{2}$	w/o CSD	w/o Denoising	Ours
mIoU	27.61	27.34	26.70	28.24	28.99
mAcc	38.82	38.41	37.78	39.75	42.11

Table S5 : Additional ablation results on the ScanNet-v2 dataset with part scale.

Scale	figurines	ramen	teatime	waldo_kitchen	Overall
whole	56/56 (100.0%)	62/71 (87.3%)	57/59 (96.6%)	14/22 (63.6%)	189/208 (90.9%)
part	53/56 (94.6%)	62/71 (87.3%)	57/59 (96.6%)	14/22 (63.6%)	186/208 (89.4%)
whole+part	56/56 (100.0%)	64/71 (90.1%)	59/59 (100.0%)	16/22 (72.7%)	195/208 (93.8%)

Table S6 : CLIP query success rate on Lerf_ovs dateset. We report the number and percentage of valid samples per scene under different scales.

$\tau$	0.00	0.05	0.10	0.15	0.20
w/ position clustering	53.68	53.10	52.57	51.77	50.51
w/o position clustering	48.60	47.89	45.88	42.71	40.33

Table S7 : Robustness to feature perturbation. We inject Gaussian noise of magnitude

\tau

into global instance features before first-stage clustering and evaluate the downstream mIoU on teatime scene. Position-aware clustering consistently yields higher mIoU and degrades more gracefully as

\tau

increases.

Figure S2 : Qualitative robustness under feature perturbation. Visualization on teatime at increasing noise magnitudes ( $\tau\in\{0.00,0.10,0.20\}$ ). Position-aware clustering produces more stable masks under stronger perturbations.

Figure S3 : More results with the rendered instance feature maps.

Figure S4 : More results with the rendered objects across different scenes.

Figure S5 : More results of point cloud clusters, demonstrating the 3D point-level segmentation ability of our method.

Figure S6 : More results with the rendered objects across different views, demonstrating the multi-view consistency of our method.

3D Semantic Segmentation

3D semantic segmentation assigns meaning to points, voxels, meshes, or reconstructed scene elements for spatial understanding.

See all 137 papers

PointGauss: Point Cloud-Guided Multi-Object Segmentation for Gaussian Splatting
Aug 1, 2025Wentao Sun, Yiping Chen, John S. Zelek +13D Gaussian Splatting 3DGS Segmentation
University of Waterloo, Waterloo, ON N2L 3G1, Canada · Sun Yat-Sen University, Zhuhai 519082, China
PePESeg3D: Perception Prior Enhances Multi-Scale Segmentation for 3D Gaussian Splatting
Sep 23, 2026Sungjae Choi, Seunghee Koh, Junmo Kim3D Gaussian Splatting 3D Reconstruction
School of Electrical Engineering Korea Advanced Institute of Science and Technology (KAIST) Daejeon, South Korea
Relation-Centric Open-Vocabulary 3D Gaussian Segmentation
Jul 1, 2026Eunsung Cha, Hyunjoon Lee, Jaesik Park3D Gaussian Splatting Open-Vocabulary 3D Segmentation
Seoul National University, Republic of Korea

Aug 1, 2025J/K move · Enter open · S save

PointGauss: Point Cloud-Guided Multi-Object Segmentation for Gaussian Splatting

Wentao Sun, Yiping Chen, John S. Zelek, Jonathan Li

University of Waterloo, Waterloo, ON N2L 3G1, Canada · Sun Yat-Sen University, Zhuhai 519082, China

3D Gaussian Splatting 3DGS Segmentation 3D Semantic Segmentation Remote Sensing 3D Instance Segmentation

Open paper AskPDF arXiv

While 3D Gaussian Splatting (3DGS) has established new standards for high-fidelity 3D scene modeling, interpreting massive, unstructured Gaussian primitives into meaningful geospatial entities remains a critical challenge for remote sensing and urban applications. Existing 2D-to-3D distillation methods suffer from projection ambiguities, geometric inconsistencies, and computational bottlenecks when applied to complex, large-scale topographies. Furthermore, current benchmarks lack natively aligned 2D-3D ground truth tailored for splatting-based representations. To address these limitations, we propose PointGauss, a unified 3D-native framework for robust semantic parsing and instance segmentation. By treating Gaussian primitives as unstructured point sets, PointGauss leverages Point Transformer V3 (PTv3) to extract scale-invariant geometric features directly from Gaussian attributes, effectively resolving projection ambiguities. To ensure scalability in building-scale scenes, we introduce an adaptive region-of-interest cropping strategy and an instance-aware distance-constrained rasterization pipeline for pixel-level, view-consistent projection. Additionally, we present SplatSeg-360, the first rigorous cross-scale benchmark tailored for 3DGS, comprising 32 complex scenes with full 360

^\circ

coverage and over 6,300 natively aligned 2D-3D masks. Extensive experiments demonstrate that PointGauss operates in real-time and achieves state-of-the-art performance. Notably, it attains approximately 90% 3D-mIoU in large-scale building scenarios and roughly 80% 2D-mIoU in view-consistent 2D instance segmentation, outperforming baseline methods by 16%. (Code)

Abstract

3D Semantic Segmentation

Explore similar work

PointGauss: Point Cloud-Guided Multi-Object Segmentation for Gaussian Splatting

PePESeg3D: Perception Prior Enhances Multi-Scale Segmentation for 3D Gaussian Splatting

Relation-Centric Open-Vocabulary 3D Gaussian Segmentation