DISC: Dense Integrated Semantic Context for Large-Scale Open-Set Semantic Mapping

Felix Igelbrink; Lennart Niecksch; Martin Atzmueller; Joachim Hertzberg

cs.CVMar 4, 2026

DISC: Dense Integrated Semantic Context for Large-Scale Open-Set Semantic Mapping

Authors: Felix Igelbrink, Lennart Niecksch, Martin Atzmueller, Joachim Hertzberg

Organizations: the German Research Center for Artificial Intelligence (DFKI), Osnabrück, Germany · the Department of Computer Sciene, Osnabrück University, Osnabrück, Germany

Abstract

Open-set semantic mapping enables language-driven robotic perception, but current instance-centric approaches are bottlenecked by context-depriving and computationally expensive crop-based feature extraction. To overcome this fundamental limitation, we introduce DISC (Dense Integrated Semantic Context), featuring a novel single-pass, distance-weighted extraction mechanism. By deriving high-fidelity CLIP embeddings directly from the vision transformer's intermediate layers, our approach eliminates the latency and domain-shift artifacts of traditional image cropping, yielding pure, mask-aligned semantic representations. To fully leverage these features in large-scale continuous mapping, DISC is built upon a fully GPU-accelerated architecture that replaces periodic offline processing with precise, on-the-fly voxel-level instance refinement. We evaluate our approach on standard benchmarks (Replica, ScanNet) and a newly generated large-scale-mapping dataset based on Habitat-Matterport 3D (HM3DSEM) to assess scalability across complex scenes in multi-story buildings. Extensive evaluations demonstrate that DISC significantly surpasses current state-of-the-art zero-shot methods in both semantic accuracy and query retrieval, providing a robust, real-time capable framework for robotic deployment. The full source code, data generation and evaluation pipelines will be made available at https://github.com/DFKI-NI/DISC.

Figures & tables

Panel of Figure 1 — Fig. 1 : Example mapping results on a HM3DSEM scene. Top: The tracked instances of the map, randomly colored. Bottom: The resulting semantic segmentation with the $\text{top}_{k}$ semantic classes.

Panel of Figure 2 — Fig. 2 : Visualization of our single-pass dense feature extraction. (a) The original RGB input frame. (b) Weighted dense cosine similarity heatmap weighted for the open-vocabulary query "an image of a chair" . The extracted patch features provide precise semantic grounding without requiring image crops.

		Replica			ScanNet
	Methods	mAcc	mIoU	fmIoU	mAcc	mIoU	fmIoU
Privileged	OpenFusion [ 26 ]	0.41	0.30	0.58	0.67	0.53	0.64
Zero-Shot	ConceptFusion [ 15 ]	0.29	0.11	0.14	0.49	0.26	0.31
	OpenMask3D [ 27 ]	-	-	-	0.34	0.18	0.20
	ConceptGraphs [ 6 ]	0.36	0.18	0.15	0.52	0.26	0.29
	BBQ [ 7 ]	0.38	0.27	0.48	0.56	0.34	0.36

TABLE I : 3D open-set semantic segmentation benchmark results on subsets of the Replica [ 24 ] and ScanNet [ 25 ] datasets.

Method	Acc@5	Acc@10	Acc@25	Acc@100	Acc@250	Acc@500	AUC ${}^{\text{top}}_{k}$
VLMaps [ 31 ]	0.05	0.17	0.54	15.32	26.01	40.02	56.20
ConceptGraphs [ 6 ]	18.11	24.01	33.00	55.17	70.85	81.55	84.07
HOV-SG [ 5 ]	18.43	25.73	36.41	56.46	69.95	80.86	84.88
DISC (Ours)	22.22	33.76	45.18	59.85	71.76	81.86	85.15

TABLE II :

AUC_{top-k}

results on the subset of HM3DSEM used by HOV-SG [ 5 ] . HOV-SG states to only consider hungarian matches with

IoU>0.5

, which we found differs in their implementation, which evaluates on all pairs after hungarian matching.

Fig. 3 : Generated trajectory from HM3DSEM scene 00800 . Red: navigation mesh. Blue lines: generated trajectory. Teal points: coverage analysis from ray tracing as down sampled voxel grid.

Panel of Figure 6 — Fig. 4 : (a) Ratio of covered objects ( $Area>50\%$ of surface covered) and ratio of covered surface for our generated trajectories for HM3DSEM . High variance is explained by the fact that the largest connected component of the navigation mesh from Habitat only covers the scene partially (Largest Island/Total Area). (b) Tour length plotted in relation to the navigable area, with color encoding the resulting dataset size (= simulation steps required).

ID	Category	Region	Model Voxels	Covered Voxels	Coverage (%)
113	chair	2	9773	8340	85.34
114	table	2	23818	22922	96.24
115	flower vase	2	2656	2599	97.85
116	ceiling	3	317994	239945	75.46
117	wall	3	22406	15816	70.59
118	wall	3	10972	10237	93.30

TABLE III : An excerpt of the surface coverage analysis details how well regions and object instances have been covered for 00891 .

Fig. 5 : Performance of the mapping pipeline on the exemplary HM3DSEM scene 00849 .

Split	mAcc	mIoU	fmIoU	Acc@5	Acc@10	AUC ${}^{\text{top}}_{k}$
val	0.34	0.20	0.45	21.45	31.63	85.93
train	0.33	0.17	0.42	18.87	28.61	84.10
combined	0.33	0.18	0.42	19.09	29.64	84.41

TABLE IV : Comparison of dense zero-shot segmentation and open-vocabulary retrieval metrics across the large scale HM3DSEM dataset.

	Method	mAcc	mIoU	fmIoU	Acc@5	Acc@10
patch features	ViT-B/16	0.30	0.20	0.41	25.78	35.13
	ViT-L/14	0.34	0.20	0.45	21.96	33.87
	ViT-H/14	0.34	0.22	0.39	16.32	24.64
	ConvNeXt-L	0.30	0.33	0.17	13.06	18.93
	EVA02-L/14	0.25	0.15	0.33	13.60	20.62
crop features	ViT-B/16	0.28	0.15	0.36	16.11	23.00

TABLE V : Comparison of dense zero-shot segmentation and open-vocabulary retrieval metrics across various CLIP backbone architectures using crop-based and single-pass patch feature extraction.

Explore similar work

FUS3DMaps: Scalable and Accurate Open-Vocabulary Semantic Mapping by 3D Fusion of Voxel- and Instance-Level Layers
May 5, 2026Timon Homberger, Finn Lukas Busch, Jesús Gerardo Ortega Peimbert +2Open-Vocabulary 3D Segmentation 3D Semantic Segmentation
Division of Robotics, Perception, and Learning, KTH Royal Institute of Technology, Sweden
OpenSplatGraph: From Dense Semantic Maps to Structured Scene Graphs for Open-Vocabulary Robot Perception
Oct 6, 2026Binh Long Nguyen, Kien Nguyen, Clinton Fookes +1Language-Embedded 3DGS Open-Vocabulary 3D Segmentation
School of Electrical Engineering and Robotics, Queensland University of Technology (QUT), Brisbane, QLD 4000, Australia · CSIRO Robotics, CSIRO, Brisbane, QLD 4069, Australia
RoboAtlas: Contextual Active SLAM
Jun 24, 2026Alexander Schperberg, Shivam K. Panda, Abraham P. Vinod +2Simultaneous Localization and Mapping Robot Navigation
Mitsubishi Electric Research Laboratories, Cambridge, MA 02139, USA · Department of Mechanical and Aerospace Engineering, University of California, Los Angeles, Los Angeles, CA 90095, USA

May 5, 2026J/K move · Enter open · S save

FUS3DMaps: Scalable and Accurate Open-Vocabulary Semantic Mapping by 3D Fusion of Voxel- and Instance-Level Layers

Timon Homberger, Finn Lukas Busch, Jesús Gerardo Ortega Peimbert, Quantao Yang+1

Division of Robotics, Perception, and Learning, KTH Royal Institute of Technology, Sweden

Open-Vocabulary 3D Segmentation 3D Semantic Segmentation Semantic Mapping 3D Scene Understanding

Open paper AskPDF arXiv

Open-vocabulary semantic mapping enables robots to spatially ground previously unseen concepts without requiring predefined class sets. Current training-free methods commonly rely on multi-view fusion of semantic embeddings into a 3D map, either at the instance-level via segmenting views and encoding image crops of segments, or by projecting image patch embeddings directly into a dense semantic map. The latter approach sidesteps segmentation and 2D-to-3D instance association by operating on full uncropped image frames, but existing methods remain limited in scalability. We present FUS3DMaps, an online dual-layer semantic mapping method that jointly maintains both dense and instance-level open-vocabulary layers within a shared voxel map. This design enables further voxel-level semantic fusion of the layer embeddings, combining the complementary strengths of both semantic mapping approaches. We find that our proposed semantic cross-layer fusion approach improves the quality of both the instance-level and dense layers, while also enabling a scalable and highly accurate instance-level map where the dense layer and cross-layer fusion are restricted to a spatial sliding window. Experiments on established 3D semantic segmentation benchmarks as well as a selection of large-scale scenes show that FUS3DMaps achieves accurate open-vocabulary semantic mapping at multi-story building scales. Additional material and code will be made available: https://githanonymous.github.io/FUS3DMaps/.

Abstract

Explore similar work

FUS3DMaps: Scalable and Accurate Open-Vocabulary Semantic Mapping by 3D Fusion of Voxel- and Instance-Level Layers

OpenSplatGraph: From Dense Semantic Maps to Structured Scene Graphs for Open-Vocabulary Robot Perception

RoboAtlas: Contextual Active SLAM