ReLaGS: Relational Language Gaussian Splatting

Yaxu Xie; Abdalla Arafa; Alireza Javanmardi; Christen Millerdurai; Jia Cheng Hu; Shaoxiang Wang; Alain Pagani; Didier Stricker

cs.CVMar 18, 2026

ReLaGS: Relational Language Gaussian Splatting

Authors: Yaxu Xie, Abdalla Arafa, Alireza Javanmardi, Christen Millerdurai, Jia Cheng Hu, Shaoxiang Wang, Alain Pagani, Didier Stricker

Organizations: German Research Center for Artificial Intelligence (DFKI) · RPTU University Kaiserslautern-Landau · University of Modena and Reggio Emilia

Abstract

Achieving unified 3D perception and reasoning across tasks such as segmentation, retrieval, and relation understanding remains challenging, as existing methods are either object-centric or rely on costly training for inter-object reasoning. We present a novel framework that constructs a hierarchical language-distilled Gaussian scene and its 3D semantic scene graph without scene-specific training. A Gaussian pruning mechanism refines scene geometry, while a robust multi-view language alignment strategy aggregates noisy 2D features into accurate 3D object embeddings. On top of this hierarchy, we build an open-vocabulary 3D scene graph with Vision Language derived annotations and Graph Neural Network-based relational reasoning. Our approach enables efficient and scalable open-vocabulary 3D reasoning by jointly modeling hierarchical semantics and inter/intra-object relationships, validated across tasks including open-vocabulary segmentation, scene graph generation, and relation-guided retrieval. Project page: https://dfki-av.github.io/ReLaGS/

Figures & tables

Figure 2: ReLaGS Overview . Given a reconstructed Gaussian scene, redundant primitives are first pruned to improve geometric accuracy. Heuristic clustering under multi-level SAM supervision then forms a hierarchical scene structure, where each cluster is assigned a CLIP-based language feature with outlier rejection. Finally, open-vocabulary inter- and intra-object scene graphs are obtained either by lifting LLM-derived relations for semantic diversity or by using a pretrained graph network for efficient offline inference.

Figure 3: Illustration of proposed two improvement methods for hierarchical scene construction and two example scene graphs. (a): Low contribution Gaussian points (red) are removed to improve scene geometry. (b): Outlier features (e.g., due to occluded or inconsistent masks) are filtered before aggregation, producing a more coherent and consistent embedding for the target. (c): The spatial relationships are predicted by our GNN. (d): The more semantic-enriched relationship lifted with LLM, the root object is marked as -1.

Method	Object		Predicate		Scene agno.
Method	R@5	R@10	R@3	R@5	Scene agno.
ConceptGraphs [ 10 ]	0.37	0.46	0.74	0.79	✗
RelationField [ 19 ]	0.69	0.80	0.76	0.82	✗
Ours (VLM)	0.68	0.79	0.10	0.35	✗
Open3DSG [ 18 ]	0.56	0.61	0.58	0.65	✓
Ours (pred.)	0.68	0.79	0.79	0.87	✓

Table 1: Results of 3D scene graph prediction on 3DSSG [ 37 ] . “Scene agnostic” denotes methods without per-scene training.

Figure 4: Qualitative results of open vocabulary object segmentation. We show results on LERF dataset for segmentation mask on 2D view. With multi-hierarchy querying search and 3D scene graph for relation guidance, our method shows strong improvement against THGS.

Method	mIoU	Scene agno.
Lerf [ 16 ]	0.25	✗
OpenNeRF [ 8 ]	0.45	✗
LangSplat [ 30 ]	0.49	✗
RelationField [ 19 ]	0.53	✗
THGS [ 7 ]	0.29	✓
Ours w/o MH	0.54	✓

Table 2: Relationship-Guided 3D Instance Segmentation on ScanNet++ [ 50 ] . Our method achieves the highest mean IoU, even operating only on single hierarchy (w/o MH).

Method	LERF-OVS mIoU (%)				Mean	T-F
Method	Fig.	Ramen	Teatime	Waldo	Mean	T-F
LangSplatV2 [ 23 ]	56.4	51.4	72.2	59.1	59.9	✗
LAGA [ 3 ]	64.1	55.6	70.9	65.6	64.0	✗
THGS [ 7 ]	57.3	43.5	68.3	50.7	54.9	✓
Occam’s [ 5 ]	58.6	51.0	70.2	65.3	61.3	✓
VALA [ 39 ]	59.9	51.5	70.2	65.1	61.7	✓

Table 3: Open-vocabulary segmentation on LERF-OVS [ 16 ] . Our method achieves the highest mean IoU among both training and training-free approaches.

Methods	19 cls		15 cls		10 cls		T-F
Methods	mIoU	mAcc	mIoU	mAcc	mIoU	mAcc	T-F
LangSplat [ 30 ]	3.78	9.11	5.35	13.20	8.40	22.06	✗
OpenGau. [ 46 ]	24.73	41.54	30.13	48.25	38.29	55.19	✗
LAGA [ 3 ]	32.50	49.10	35.50	53.50	42.60	63.20	✗
THGS [ 7 ]	34.39	50.74	39.61	57.07	46.38	64.74	✓
Occam’s [ 5 ]	31.93	48.93	34.25	53.71	45.16	64.39	✓

Table 4: Open-vocabulary semantic segmentation on ScanNet [ 6 ] . Our method achieves the highest mean performance among both training and training-free approaches on 15 and 10 classes subset.

Variant	Figurines	Ramen	Teatime	Kitchen	Mean
Base	52.05	47.19	76.77	47.5	55.88
Base+ $M.$	59.16	47.41	80.98	60.59	62.04
Base+ $M.$ + $R.$ (Full)	64.69	51.15	80.98	60.6	64.36

Table 5: Ablation on LeRF-OVS dataset for open-vocabulary segmentation.

M.

is Maximum Weight Pruning and

R.

is Robust Outlier-Aware Feature Aggregation .

$\tau_{lang}$	Figurines	Ramen	Teatime	Waldo Kitchen	Mean
1	61.85	43.3	76.02	63.37	60.78
2	63.39	51.07	73.8	60.62	62.22
3	64.69	51.15	80.98	60.60	64.36
4	63.3	47.4	80.98	60.60	63.07

Table 6: Ablation on different values of

\tau_{lang}

on LeRF-OVS.

Method	Time (min) / Storage (MB) / GPU Memory (GB)
Method	Scene Rec.	Language Distill.	Scene Graph	Overall
RF [ 19 ]	60/500/32 (inseparable)			60/500/32
Ours	11/52.2/3.2	1.5/10.6/7.5	0.1/2.2/3.0	12.6/65/7.5

Table 7: Runtime and resource comparison against RelationField [ 19 ] on 3DSSG scenes. Our method is 4.7

\times

faster and 7.6

\times

more memory efficient and a maximum of 7.5GB of GPU memory.

Method	Object mIoU	Part mIoU
THGS	63.2	45
Ours	65.3	60.3

Table 8: Hierarchical query evaluation on LERF. We manually categorize LERF queries into object-level and part-level queries and report mIoU for each category. Our method improves performance across both groups, with particularly large gains on part-level queries, demonstrating the ability of the proposed representation to support fine-grained hierarchical language queries.

Method	19 Classes		15 Classes		10 Classes
Method	mIoU	mAcc	mIoU	mAcc	mIoU	mAcc
THGS	39.33	54.23	43.61	61.55	52.52	70.50
Ours	41.26	57.29	45.24	62.75	54.04	71.48

Table 9: ScanNet Results with Revised Evaluation Protocol. Quantitative results on ScanNet using densified Gaussian reconstruction and nearest-point assignment (L2). THGS* refers to the THGS method re-run by us on all scenes under this protocol. Our method consistently outperforms THGS* across all class subsets.

Variants	Object		Predicate
Variants	R@5	R@10	R@3	R@5
Base	0.57	0.69	0.75	0.85
Base+ $M.$	0.65	0.75	0.77	0.87
Base+ $M.$ + $R.$	0.68	0.79	0.79	0.87

Table 10: Ablation on 3DSSG dataset for 3D scene graph prediction. M. is Maximum Weight Pruning and R. is Robust Outlier-Aware Feature Aggregation .

Variants	Object		Predicate
Variants	R@5	R@10	R@3	R@5
L=1, $n\mapsto e$	0.68	0.79	0.79	0.87
L=2, $n\mapsto e$	0.68	0.79	0.77	0.84
L=1, $0\mapsto e$	0.68	0.79	0.75	0.88

Table 11: Ablation on 3DSSG dataset for 3D scene graph prediction. L is the number of transformer layer, and

0\mapsto e

means using zero vector instead of using node feature pair to initialize edge feature

n\mapsto e

$\tau_{contrib}$	mIoU $\uparrow$	SSIM $\uparrow$	PSNR $\uparrow$	LPIPS $\downarrow$
-	57.84	0.8478	23.576	0.2492
$5\times 10^{-4}$	64.36	0.8478	23.576	0.2492
$5\times 10^{-3}$	59.27	0.8478	23.576	0.2492
$5\times 10^{-2}$	57.53	0.8462	23.526	0.2516
$5\times 10^{-1}$	39.18	0.4967	10.905	0.5185

Table 12: Ablation Study on weight pruning.

Method	3D-OVS mIoU (%)					Mean
Method	Bed	Bench	Lawn	Room	Sofa	Mean
LangSplatV2 [ 23 ]	93.0	94.9	96.1	92.3	96.6	94.6
LAGA [ 3 ]	96.8	92.8	97.0	93.0	96.9	95.3
Occam’s [ 5 ]	96.8	94.8	97.0	96.5*	88.8	95.0
THGS [ 7 ] $\dagger$	68.9	67.6	73.9	68.4*	54.6	66.7
Ours	95.4	94.4	91.2	94.8*	86.9	92.5

Table 13: Open-vocabulary segmentation on 3D-OVS [ 25 ] . Methods with

*

are evaluated on testing-data that has an annotation error in the 3D-OVS testset corrected.

\dagger

means we run the open-source code under our environment.

Figure 5: Qualitative results on LERF, ScanNet++, ScanNet and 3D-OVS.

Figure 6: Visualization of multi-hierarchy scene reconstruction on LERF and 3D-OVS datasets.

Figure 7: Examples of our SoM+LLM scene graph annotation on 2D images from 3DSSG dataset.

Language-Embedded 3DGS

Language-embedded 3D Gaussian Splatting attaches language-aligned features to 3D scenes for open-vocabulary querying and semantic understanding.

See all 24 papers

CausalSplat: Towards Comprehensive Hierarchical Reasoning in 3D Gaussian Splatting
Aug 11, 2026Jiayu Ding, Meilu Song, Yun Chen +23D Gaussian Splatting Spatial Reasoning Benchmarks
Hunan University College of Computer Science and Electronic Engineering Changsha, China
Beyond Isolated Objects: Relationship-aware Open Vocabulary Scene Understanding via 3D Scene Graph Analysis
Jul 6, 2026Xianhao Chen, Jiarui Hu, Yuanbo Yang +5Graph Attention Networks Open-Vocabulary 3D Segmentation
State Key Lab of CAD&CG, Zhejiang University, China · Zhejiang University, China
RelationVGGT: Visual Geometry Transformers for 3D Spatial Relation Segmentation
Oct 1, 2026Minsu Kim, Jaesung Choe, Jiwoo Lee +23D ViTs 3D Spatial Reasoning
Yonsei University · NVIDIA

Aug 11, 2026J/K move · Enter open · S save

CausalSplat: Towards Comprehensive Hierarchical Reasoning in 3D Gaussian Splatting

Jiayu Ding, Meilu Song, Yun Chen, Wei Gao+1

Hunan University College of Computer Science and Electronic Engineering Changsha, China

3D Gaussian Splatting Spatial Reasoning Benchmarks Open-Vocabulary 3D Segmentation 3D Spatial Reasoning

Open paper AskPDF arXiv

While 3D Gaussian Splatting (3DGS) has advanced open vocabulary scene understanding, existing methods remain confined to explicit queries. They struggle to interpret implicit intents, complex spatial constraints, and commonsense reasoning required for practical embodied interactions. To address this gap, we introduce the task of reasoning 3D Gaussian segmentation and construct two benchmarks, Causal-LERF and Causal-ScanNet. These benchmarks systematically evaluate commonsense, spatial, affordance, and counterfactual reasoning. Evaluations reveal that current state of the art methods perform poorly on these reasoning challenges. Therefore, we propose CausalSplat, a framework that integrates vision-language models with 3D scene graphs to disentangle explicit structural perception from implicit logical inference. Extensive experiments demonstrate that CausalSplat achieves state of the art performance on our reasoning benchmarks while showing strong generalizability on standard referring and open vocabulary 3D segmentation tasks. Project Page: https://jiayuding031020.github.io/CausalSplat

Abstract

Language-Embedded 3DGS

Explore similar work

CausalSplat: Towards Comprehensive Hierarchical Reasoning in 3D Gaussian Splatting

Beyond Isolated Objects: Relationship-aware Open Vocabulary Scene Understanding via 3D Scene Graph Analysis

RelationVGGT: Visual Geometry Transformers for 3D Spatial Relation Segmentation