RelateGS: Reliable Semantic and Relational Evidence for 3D Scene Understanding with Gaussian Splatting

Seongjun Choi; Sangyoun Lee

cs.CVOct 24, 2026

RelateGS: Reliable Semantic and Relational Evidence for 3D Scene Understanding with Gaussian Splatting

Authors: Seongjun Choi, Sangyoun Lee

Organizations: Yonsei University, School of Electrical and Electronic Engineering

Abstract

Language-aware 3D Gaussian Splatting (3DGS) has advanced language-driven 3D scene understanding by incorporating semantic and relational cues into reconstructed scenes. However, relation-aware grounding depends not only on representing such information, but also on whether the underlying scene-side evidence is reliable and sufficient. Multi-view semantic aggregation can incorporate weak observations, while object relations can remain under-observed along the available camera trajectory, leading to ambiguous evidence for target disambiguation. We present RelateGS, a structured language-aware Gaussian representation that selectively refines semantic and relational evidence for 3D visual grounding. RelateGS suppresses unreliable observations during semantic feature uplifting and consolidates multi-view object observations into a persistent 3D graph. The graph further identifies under-observed object relations and guides targeted re-querying of informative existing frames to complement their evidence. Experiments on Sr3D+ and Nr3D demonstrate competitive 3D grounding performance, improving the previous best by 7.9 and 1.7 percentage points at [email protected], respectively.

Figures & tables

Figure 1: Motivation of RelateGS. Existing methods can suffer from (A) unreliable semantic evidence due to weak multi-view observations and (B) insufficient relational evidence due to limited viewpoint coverage. RelateGS addresses these issues by (A) filtering weak observations during semantic feature uplifting and (B) complementing under-observed relations through graph-guided re-querying of informative existing views.

Figure 2: Overview of RelateGS. Given a reconstructed 3D Gaussian scene and posed RGB-D observations, RelateGS selectively refines the scene-side evidence used for relation-aware grounding. At the Gaussian level, weak multi-view observations are filtered during semantic uplifting. At the object level, multi-view observations are consolidated into a persistent graph, whose current state identifies under-observed object pairs and guides targeted re-querying of informative existing frames.

Method	Easy		Hard		Overall
Method	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$
LangSplat [ 14 ]	4.7	1.5	2.1	1.1	4.5	2.3
OpenGaussian [ 26 ]	7.3	5.1	3.6	1.9	7.0	4.9
ConceptGraph [ 4 ]	13.0	6.8	16.0	1.3	13.3	6.2
OpenFusion [ 28 ]	14.0	2.4	1.3	1.3	12.6	2.4
GaussianGraph [ 22 ]	19.1	7.7	16.3	5.6	18.2	7.4

Table 1: Comparison on the Sr3D+ benchmark. We report [email protected] and [email protected] (%) for the Easy, Hard, and overall splits.

Method	Easy		Hard		Overall
Method	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$
LangSplat [ 14 ]	8.5	1.7	3.4	1.2	7.4	1.5
OpenGaussian [ 26 ]	10.1	5.9	7.4	3.1	9.5	4.7
ConceptGraph [ 4 ]	18.7	9.2	9.1	2.0	16.0	7.2
OpenFusion [ 28 ]	12.9	1.4	5.1	1.5	10.7	1.4
GaussianGraph [ 22 ]	20.7	12.1	10.9	6.3	17.2	8.6

Table 2: Comparison on the Nr3D benchmark. We report [email protected] and [email protected] (%) for the Easy, Hard, and overall splits.

Point-cloud-based
Method	Replica		ScanNet
Method	mIoU $\uparrow$	mAcc $\uparrow$	mIoU $\uparrow$	mAcc $\uparrow$
ConceptFusion [ 6 ]	10.07	16.15	9.72	15.41
ConceptGraph [ 4 ]	20.72	31.54	16.42	27.60
HOV-SG [ 24 ]	23.16	29.85	22.43	43.81
3DGS-based

Table 3: Semantic segmentation results on Replica [ 19 ] and ScanNet [ 3 ] . We compare RelateGS with point-cloud and 3DGS-based scene representations.

Method	ramen			teatime			waldo_kitchen			figurines
Method	[email protected] $\uparrow$	[email protected] $\uparrow$	mIoU $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	mIoU $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	mIoU $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$	mIoU $\uparrow$
LangSplat [ 14 ]	11.09	7.25	8.94	21.73	19.05	14.42	10.92	10.08	10.79	9.00	7.31	14.84
LEGaussian [ 18 ]	31.54	26.76	15.79	30.49	27.12	19.27	25.76	18.18	11.78	29.41	23.21	17.99
OpenGaussian [ 26 ]	40.85	21.13	25.41	79.66	71.19	59.00	40.15	35.45	25.15	78.57	71.43	57.50
Occam’s LGS [ 2 ]	54.93	45.07	44.49	89.83	81.36	71.03	81.82	59.09	59.44	80.36	60.71	59.85
GaussianGraph [ 22 ]	43.66	28.63	29.58	79.96	76.27	62.70	51.42	36.36	35.85	81.24	75.61	62.00

Table 4: Open-vocabulary localization and segmentation results on the LERF-OVS benchmark. We report [email protected], [email protected], and mIoU (%) for each scene. Best and second-best results are highlighted in bold and underline, respectively.

Method	CA	RQ	Selection	[email protected] $\uparrow$	[email protected] $\uparrow$
Initial Object Graph	–	–	–	21.34	12.12
+ Confidence-Aware Semantics	✓	–	–	21.72	13.10
+ Random Frame Re-query	✓	✓	Random	19.78	11.62
+ Graph-Guided Frame Re-query	✓	✓	Graph	22.22	15.28

Table 5: Ablation of selective evidence refinement on Sr3D+. CA denotes confidence-aware semantic uplifting and RQ denotes relational re-querying. Random and Graph indicate the strategy used to select re-query observations. All RQ variants use the same additional VLM query budget.

Figure 3: Qualitative comparison of relation-aware 3D visual grounding on ScanNet. The ground-truth column shows the target object in green and distractor objects in red. For each method, green denotes the ground-truth target, yellow indicates a correctly grounded prediction, and red indicates an incorrect prediction. Compared with language-embedded Gaussian representations and RelateGS without relational re-querying (w/o RQ), the full RelateGS more reliably resolves targets from relational expressions by complementing under-observed relational evidence.

Figure 4: Qualitative comparison of open-vocabulary localization on LERF-OVS. Given the same language query, we compare the activation maps and localized regions produced by Occam’s LGS and RelateGS. Confidence-aware semantic uplifting suppresses weak multi-view observations, yielding more concentrated language responses around the referred object.

3D Scene Understanding

3D scene understanding builds geometric and semantic representations of environments for spatial reasoning, perception, and interaction.

See all 295 papers

ReLaGS: Relational Language Gaussian Splatting
Mar 18, 2026Yaxu Xie, Abdalla Arafa, Alireza Javanmardi +5Language-Embedded 3DGS 3D SGG
German Research Center for Artificial Intelligence (DFKI) · RPTU University Kaiserslautern-Landau · University of Modena and Reggio Emilia
RecallGS: Geometry-Grounded Recall for Online Semantic Gaussian Splatting
Oct 22, 2026Thanh-Khoi Nguyen, Thien-Phuc Tran, Minh-Triet TranOpen-Vocabulary 3D Segmentation 3DGS Segmentation
University of Science, Ho Chi Minh City, Vietnam · Viet Nam National University, Ho Chi Minh City, Vietnam
DSSR-3D: Decoupled Reasoning for View-Dependent Referring in 3D Gaussians
Sep 3, 2026Thanh-Khoi Nguyen, Thien-Phuc Tran, Minh-Triet Tran3D Referring Segmentation 3D Visual Grounding
University of Science, Ho Chi Minh City, Vietnam · Viet Nam National University, Ho Chi Minh City, Vietnam

Mar 18, 2026J/K move · Enter open · S save

ReLaGS: Relational Language Gaussian Splatting

Yaxu Xie, Abdalla Arafa, Alireza Javanmardi, Christen Millerdurai+4

German Research Center for Artificial Intelligence (DFKI) · RPTU University Kaiserslautern-Landau · University of Modena and Reggio Emilia

Language-Embedded 3DGS 3D SGG Open-Vocabulary 3D Segmentation

Open paper AskPDF arXiv

Achieving unified 3D perception and reasoning across tasks such as segmentation, retrieval, and relation understanding remains challenging, as existing methods are either object-centric or rely on costly training for inter-object reasoning. We present a novel framework that constructs a hierarchical language-distilled Gaussian scene and its 3D semantic scene graph without scene-specific training. A Gaussian pruning mechanism refines scene geometry, while a robust multi-view language alignment strategy aggregates noisy 2D features into accurate 3D object embeddings. On top of this hierarchy, we build an open-vocabulary 3D scene graph with Vision Language derived annotations and Graph Neural Network-based relational reasoning. Our approach enables efficient and scalable open-vocabulary 3D reasoning by jointly modeling hierarchical semantics and inter/intra-object relationships, validated across tasks including open-vocabulary segmentation, scene graph generation, and relation-guided retrieval. Project page: https://dfki-av.github.io/ReLaGS/

Abstract

3D Scene Understanding

Explore similar work

ReLaGS: Relational Language Gaussian Splatting

RecallGS: Geometry-Grounded Recall for Online Semantic Gaussian Splatting

DSSR-3D: Decoupled Reasoning for View-Dependent Referring in 3D Gaussians