DIGS: Distribution-Informed Gaussian Splatting for Training-Free Open-Vocabulary 3D Segmentation

Zhifan Zeng; Wenbo Xiao; Arcot Sowmya; Changming Sun

cs.CVOct 12, 2026

DIGS: Distribution-Informed Gaussian Splatting for Training-Free Open-Vocabulary 3D Segmentation

Authors: Zhifan Zeng, Wenbo Xiao, Arcot Sowmya, Changming Sun

Organizations: UNSW Sydney · CSIRO

Abstract

Open-vocabulary 3D Gaussian segmentation typically binds semantic features to the per-Gaussian state via per-scene gradient training (1-4 h) or the more recent closed-form distillation. Both commit each primitive to a single feature; therefore, boundary Gaussians spanning multiple 2D instances are forced into a hard decision that degrades retrieval accuracy and yields unstable mask boundaries at render time. We present DIGS, a training-free framework with two stages, class-agnostic mapping and multi-modal encoding, that decouples instance discovery from linguistic naming. At the Gaussian level, our distribution-informed representation maintains an explicit top- $M$ label distribution per primitive, updated by a tempered Dirichlet rule; preservation of multi-peaked label distributions at boundary primitives allows a late-argmax renderer to recover clean instance boundaries. A bidirectional state machine aggregates these distributions into a persistent class-agnostic instance ontology. The instance-level encoding stage then fuses the CLIP visual embedding of each instance's isolated 3D footprint with the CLIP-text embedding of an LLM-generated attribute description, disambiguating visually similar instances. On a single A100, DIGS completes mapping and instance encoding on LERF-OVS in 1.0-3.2 min per scene, depending on whether LLM fusion is enabled, and serves native 3D queries in 5.4 ms. It reaches 71.95% mean mIoU on LERF-OVS, outperforming both the training-free SFS and training-based LaGa baselines. DIGS also achieves the highest mean mIoU on LERF-Mask and 3D-OVS and surpasses Gaga and ObjectGS on Replica under their respective evaluation protocols. Code is available at https://github.com/ZephyrZos/DIGS.

Figures & tables

Figure 1: DIGS at a glance. (a) Three per-Gaussian semantic representations (One-hot ID, Feature Vector, and distribution-informed per-Gaussian voting (ours)) and their rendered boundaries. (b) LERF-OVS performance landscape (with our approach performing the best). (c) Qualitative comparison with adversarial query (Waldo_kitchen, yellow desk ).

Figure 2: DIGS architecture. (a) Class-agnostic ontology mapping : per-frame SAM masks are Hungarian-matched to the Late-Argmax-rendered label map; the global label map drives the Tempered Dirichlet Update (injection bound $\eta_{i}^{(t)}\!\cdot\!W_{i}^{(t)}<\tau$ , Proposition 1 ) on the semantic state machine, maintaining each Gaussian’s top- $M$ Hit Accumulator $\hat{H}_{g_{p,k}}$ and label distribution $\hat{\pi}_{g_{p,k}}$ alongside per-instance footprint $\mathcal{G}_{j}$ and lifecycle counters. (b) Multi-modal instance encoding : Auto-View Selection yields paired context / isolated renderings of $\mathcal{G}_{j}$ , whose CLIP visual embedding is linearly fused with the CLIP-text embedding of an LLM-generated attribute description. (c) Open-vocabulary query : a text prompt is CLIP-text-encoded and matched to the per-instance embeddings by cosine similarity, with no per-query retraining.

Figure 3: Qualitative comparison on LERF-OVS against LangSplat, LangSplatV2, LaGa, and the training-free comparator SFS. Two prompts per scene across the four LERF-OVS scenes.

Method	Train	Map	VRAM	Per-prompt	3D-only
LangSplat [ 5 ]	$\checkmark$	60	6.2	–	–
LEGaussians [ 7 ]	$\checkmark$	78	11.0	–	–
LangSplatV2 [ 6 ]	$\checkmark$	204	16.8	71.4	–
LaGa [ 10 ]	$\checkmark$	225	12.5	101.7	72.8
SFS [ 11 ]	$\times$	5.8	13.3	161.0	8.0
DIGS (map)	$\times$	0.91	12.4	–	–

Table 1: Per-scene computational cost on LERF-OVS. Units: Map (min), VRAM (GB), Per-prompt, and 3D-only (ms).

Method	Figurines		Ramen		Teatime		Waldo_kitchen		Mean
	mIoU	mAcc	mIoU	mAcc	mIoU	mAcc	mIoU	mAcc	mIoU	mAcc
LangSplat ∗ [ 5 ]	10.16	8.93	7.92	11.27	11.38	20.34	9.18	9.09	9.66	12.41
LangSplat ‡ [ 5 ]	46.44	73.21	48.96	67.61	60.85	84.75	39.65	63.64	48.98	72.30
LEGaussians ∗ [ 7 ]	17.99	23.21	15.79	26.76	19.27	27.12	11.78	18.18	16.21	23.82
OpenGaussian ∗ [ 8 ]	39.29	55.36	31.01	42.25	60.44	76.27	22.70	31.82	38.36	51.43
LaGa [ 10 ]	64.10	–	55.60	–	70.90	–	65.60	–	64.00	–

Table 2: LERF-OVS (mIoU / mAcc, %). OpenGaussian (OG) 3D-native protocol unless ‡ (LangSplat-family 2D). ∗ re-eval by OG [ 8 ] ; † our reproduction; unmarked: paper-reported.

Table 6

Figure 5: CLIP-only versus LLM-enhanced retrieval on LERF-OVS.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

$\lambda$	Figurines	Ramen	Teatime	Waldo_kitchen	Mean
0.0	67.61	43.23	48.66	37.12	49.16
0.2	75.97	43.25	52.58	37.30	52.28
0.3	75.97	51.69	52.58	37.30	54.39
0.5	75.97	60.40	60.26	41.02	59.41
0.7	77.60	58.38	77.65	43.75	64.34
0.8	80.01	50.30	77.65	43.75	62.93

Appendix

Table 10:

\lambda

sweep on LERF-OVS, embedding-fusion path with retrieval thresholds disabled (mIoU, %).

$\Delta_{\min}$	Figurines	Ramen	Teatime	Waldo_kitchen	Mean
0.00	77.60	58.38	77.65	43.75	64.34
0.05	80.58	60.81	77.65	48.33	66.84
0.10	78.71	64.68	77.65	57.85	69.72
0.12	78.71	64.68	77.65	66.76	71.95
0.15	78.71	57.94	74.41	57.43	67.12
0.17	75.96	55.17	68.72	56.02	63.97

Appendix

Table 11:

\Delta_{\min}

sweep at

\sigma_{\text{abs}}{=}0.25

on LERF-OVS at

\lambda{=}0.7

(mIoU, %).

\Delta_{\min}{=}0

is equivalent to disabling the filter pair. Deployed

\Delta_{\min}{=}0.12

$M$	Figurines		Ramen		Teatime		Waldo_kitchen		Mean
	mIoU	mAcc	mIoU	mAcc	mIoU	mAcc	mIoU	mAcc	mIoU	mAcc
1	65.32	85.71	55.25	71.83	74.81	86.44	50.18	68.18	61.39	78.04
2	70.66	87.50	54.78	73.24	71.10	84.75	53.37	68.18	62.48	78.42
3	74.81	91.07	56.15	73.24	74.70	84.75	61.72	81.82	66.85	82.72
4	74.78	91.07	51.33	60.56	75.55	84.75	54.62	72.73	64.07	77.28
5	74.76	91.07	51.58	71.83	75.48	84.75	54.46	72.73	64.07	80.09

Appendix

Table 12: Top-

M

sweep on LERF-OVS (mIoU / mAcc, %).

$\tau$	Figurines		Ramen		Teatime		Waldo_kitchen		Mean
	mIoU	mAcc	mIoU	mAcc	mIoU	mAcc	mIoU	mAcc	mIoU	mAcc
1	73.59	92.86	38.89	53.52	74.96	84.75	54.20	72.73	60.41	75.96
2	74.81	91.07	45.41	61.97	74.97	84.75	50.10	68.18	61.32	76.49
5	74.81	91.07	56.15	73.24	74.70	84.75	61.72	81.82	66.85	82.72
10	74.80	91.07	51.60	66.20	74.55	84.75	53.47	72.73	63.61	78.69
20	71.48	87.50	52.03	70.42	71.97	84.75	49.13	68.18	61.15	77.71

Appendix

Table 13:

\tau

sweep on LERF-OVS (mIoU / mAcc, %).

Scene	raw	pass	filt	mIoU off	mIoU on	$\Delta$
Figurines	511	31	443	74.00	74.81	$+0.81$
Ramen	255	20	161	44.21	56.15	$+11.95$
Teatime	511	15	461	58.20	74.70	$+16.50$
Waldo_kitchen	1023	67	709	54.90	61.72	$+6.82$
Mean	575	33	444	57.83	66.85	$+9.02$

Appendix

Table 14: Stability Filter counts on LERF-OVS. raw = total instance proposals discovered after mapping; pass = instances meeting the strict criterion

M_{i}>50\land m_{i}<30

; filt = instances zeroed out.

Class	Naive $\eta$	Tempered $\eta$	$\Delta$ IoU
Ramen
wavy noodles	0.00	85.62	$+85.62$
kamaboko	35.27	82.30	$+47.02$
bowl	38.27	56.22	$+17.95$
napkin	30.59	41.21	$+10.62$
corn	39.95	26.45	$-13.49$

Appendix

Table 15: Per-class

\Delta

IoU (tempered

\eta

-

naive

\eta

) on the two adversarial scenes.

Figure 6: 3D-OVS qualitative results across all five scenes.

Figure 7: Naive $\eta$ versus tempered $\eta_{i}^{(t)}$ across all four LERF-OVS scenes. One representative prompt per scene. Tempered $\eta$ produces visibly cleaner instance footprints in every panel.

Figure 8: F1: Transparency couples voting weight to background— transparent box (Figurines) and transparent cup (Ramen).

Figure 9: F2: SAM-granularity caps part-level prompts— waldo , pirate hat , and hooves / dall-e brand (paired on the same figurine) all retrieve the whole parent instance.

Open-Vocabulary 3D Segmentation

Open-vocabulary 3D segmentation assigns language-queryable semantic and instance masks to 3D scenes, including objects unseen during training.

See all 70 papers

Hybrid Gaussians for Robust Open-Vocabulary 3D Segmentation with Multi-View Object Association and Boundary Refinement
Sep 23, 2026Xueqi Qiu, Yueming Sun, Tianyu Zhang +2Language-Embedded 3DGS Open-Vocabulary 3D Segmentation
Department of Computer Science, Durham University, Durham, United Kingdom · Shanghai Jiao Tong University, Shanghai, China
Relation-Centric Open-Vocabulary 3D Gaussian Segmentation
Jul 1, 2026Eunsung Cha, Hyunjoon Lee, Jaesik Park3D Gaussian Splatting Open-Vocabulary 3D Segmentation
Seoul National University, Republic of Korea
OP2GS: Object-Aware 3D Gaussian Splatting with Dual-Opacity Primitives
May 19, 2026Guiyu Liu, Niklas Vaara, Janne Mustaniemi +23D Gaussian Splatting Open-Vocabulary 3D Segmentation
Center for Machine Vision and Signal Analysis, University of Oulu, Finland · Aalto University, Finland

Sep 23, 2026J/K move · Enter open · S save

Hybrid Gaussians for Robust Open-Vocabulary 3D Segmentation with Multi-View Object Association and Boundary Refinement

Xueqi Qiu, Yueming Sun, Tianyu Zhang, Yuxuan Xia+1

Department of Computer Science, Durham University, Durham, United Kingdom · Shanghai Jiao Tong University, Shanghai, China

Language-Embedded 3DGS Open-Vocabulary 3D Segmentation Boundary-Aware Segmentation 3D VLMs

Open paper AskPDF arXiv

Open-vocabulary 3D segmentation localizes objects from free-form text queries, but remains challenging in real image sequences: incomplete or noisy 2D supervision destabilizes multi-view identity assignment, while full-scene semantic learning weakens object-level discriminability. We introduce Hybrid Gaussians, a unified 3D representation jointly modeling object association and language-aligned semantics. Its Multi-View Object Association mechanism combines Observation Fusion and Semantic Contrastive Learning to improve identity consistency and semantic discrimination. Boundary Reconstruction Optimization further refines local boundary structure to improve contour quality. Experiments on LERF and 3D-OVS demonstrate strong quantitative and qualitative performance. Our method achieves 59.1% mIoU on LERF, yielding a 13.4% relative gain over the baseline. Project page: https://nora202.github.io/hybridgaussians.

Abstract

Open-Vocabulary 3D Segmentation

Explore similar work

Hybrid Gaussians for Robust Open-Vocabulary 3D Segmentation with Multi-View Object Association and Boundary Refinement

Relation-Centric Open-Vocabulary 3D Gaussian Segmentation

OP2GS: Object-Aware 3D Gaussian Splatting with Dual-Opacity Primitives