PanOVOcc: Panoramic Embodied Open-Vocabulary Occupancy Mapping with Long-term Spatial Voxel Memory

Di Kuang; Mengfei Duan; Yuhang Wang; Weixing Peng; Kailun Yang

cs.CVSep 21, 2026

PanOVOcc: Panoramic Embodied Open-Vocabulary Occupancy Mapping with Long-term Spatial Voxel Memory

Authors: Di Kuang, Mengfei Duan, Yuhang Wang, Weixing Peng, Kailun Yang

Organizations: School of Artificial Intelligence and Robotics, Hunan University, China · National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, China

Abstract

Persistent semantic occupancy mapping is essential for embodied scene understanding. However, perspective-based systems provide limited spatial coverage, while existing panoramic methods primarily predict local volumes from single observations. We introduce PanOVOcc, a training-free framework for persistent open-vocabulary semantic occupancy mapping from panoramic sequences. PanOVOcc unifies panoramic SLAM, open-vocabulary perception, and long-term spatial voxel memory within an online architecture, continuously integrating geometric and semantic evidence into a global, language-queryable map. To facilitate systematic evaluation of this setting, we establish Pan-Replica and Pan-Holo360D, two benchmarks pairing continuous panoramic RGB-D sequences with scene-level semantic occupancy ground truth across synthetic and real-world scenes. Compared with the strongest evaluated baseline for each metric, PanOVOcc improves occupancy IoU and semantic mIoU by absolute +20.03 and +7.06 on Pan-Replica, and by +43.26 and +20.16 on Pan-Holo360D, respectively. The source code and the established benchmarks will be available at https://github.com/bakereet/PanOVOcc.

Figures & tables

Fig. 1: Overview of the benchmarks and PanOVOcc framework. Pan-Replica and Pan-Holo360D provide panoramic RGB-D sequences as mapping inputs and scene-level semantic occupancy ground truth for evaluation. Given panoramic RGB-D sequences, PanOVOcc obtains geometric priors from SLAM-estimated camera poses, and semantic priors from open-vocabulary segmentation. These priors are incrementally integrated through online 3D fusion to construct a persistent open-vocabulary semantic occupancy map. Abbreviations: Occ: Occupancy; OV: Open-Vocabulary; GT: Ground Truth.

Benchmark	Obs.	Temporal	Scope	Domain
EmbodiedOcc-ScanNet [ 1 ]		Sequence	Scene	In.-Real
ReplicaOcc [ 2 ]		Sequence	Scene	In.-Synth.
QuadOcc [ 6 ]		Single	Local	Out.-Real
Human360Occ [ 6 ]		Single	Local	Out.-Synth.
PanoMMOcc [ 7 ]		Single	Local	Out.-Real
HIOcc [ 3 ]		Both	Local / Scene	In.-Real

TABLE I: Comparison with representative embodied and panoramic occupancy benchmarks. / : perspective / panoramic observations. In. / Out.: Indoor / Outdoor.

Panel of Figure 3 — Fig. 2: Overview of Pan-Holo360D. Scene-level semantic occupancy GT for seven real-world indoor scenes selected from Holo360D [ 18 ] .

Method	Obs.	Anno.	Pose	IoU	mIoU	$\blacksquare$ ceiling	$\blacksquare$ floor	$\blacksquare$ wall	$\blacksquare$ wind.	$\blacksquare$ chair	$\blacksquare$ bed	$\blacksquare$ sofa	$\blacksquare$ table
Supervised learning
EmbodiedOcc [ 1 ]		✓	✓	18.02	5.58	3.19	6.34	10.05	3.44	4.26	7.78	6.81	2.76
EmbodiedOcc++ [ 4 ]		✓	✓	19.41	5.96	3.04	7.89	10.61	4.26	4.31	6.34	7.06	4.18
VEOcc [ 5 ]		✓	✓	20.83	17.02	19.08	12.42	13.83	7.96	21.26	8.96	34.29	18.41
GPOcc [ 39 ]		✓	✓	19.64	15.72	21.14	19.98	9.75	7.22	14.53	10.96	29.87	12.30
Training-free

TABLE II: Quantitative comparison on Pan-Replica. Perspective methods receive twelve directional views generated from each panoramic observation, providing approximately

360^{\circ}

horizontal coverage.

				Overlap-8 Vocabulary								Open-Vocabulary
Method	Obs.	IoU	mIoU (8)	$\blacksquare$ ceiling	$\blacksquare$ floor	$\blacksquare$ wall	$\blacksquare$ wind.	$\blacksquare$ chair	$\blacksquare$ bed	$\blacksquare$ sofa	$\blacksquare$ table	$\blacksquare$ curtain	$\blacksquare$ door	$\blacksquare$ tv	$\blacksquare$ lighti.
Supervised learning
EmbodiedOcc [ 1 ]		18.21	5.16	0.15	13.35	9.65	4.10	3.08	4.30	3.12	3.49	✗	✗	✗	✗
EmbodiedOcc++ [ 4 ]		18.02	4.99	0.16	13.21	10.15	3.07	2.81	3.81	2.95	3.74	✗	✗	✗	✗
VEOcc [ 5 ]		23.01	12.49	15.64	7.81	18.72	13.41	11.47	4.10	15.85	12.89	✗	✗	✗	✗
GPOcc [ 39 ]		22.03	14.56	29.20	11.15	17.24	12.88	8.54	8.20	21.51	7.76	✗	✗	✗	✗

TABLE III: Quantitative comparison on Pan-Holo360D. We report semantic occupancy performance on eight categories shared with Pan-Replica and four additional open-vocabulary categories.

Panel of Figure 6 — Fig. 3: Observation-level spatial coverage. Perspective and panoramic observations are compared along the same trajectory circling the room in Indoor_015 . (a) Cumulative coverage visualizations at matched observation steps $N=1,80,110$ . (b) Cumulative coverage over the complete sequence. Vertical dotted lines and markers correspond to the snapshots in (a).

Variant	PSVF	GSC	RPU	IoU $\uparrow$	mIoU $\uparrow$
w/o PSVF	✗	✓	✓	53.02	23.51
w/o GSC	✓	✗	✓	76.75	31.51
w/o RPU	✓	✓	✗	69.69	30.54
Full model	✓	✓	✓	76.75	32.09

TABLE IV: Component ablation of PanOVOcc on Pan-Replica. PSVF, GSC, RPU denote Panoramic Signed Voxel Fusion, Geometry-Aware Semantic Consolidation, and Reversible Persistent Update.

Panel of Figure 8 — Fig. 4: Qualitative comparison of semantic occupancy mapping results on (a) Pan-Replica and (b) Pan-Holo360D. Compared with existing methods, e.g. , EmbodiedOcc [ 1 ] , EmbodiedOcc++ [ 4 ] , VEOcc [ 5 ] , GPOcc [ 39 ] , and FreeOcc [ 2 ] , our approach more accurately reconstructs scene geometry and maintains strong performance in larger-scale environments. For clearer visualization, top-layer voxels are removed to reduce occlusion.

	Pan-Replica		Pan-Holo360D
Method	IoU $\uparrow$	mIoU $\uparrow$	IoU $\uparrow$	mIoU $\uparrow$
Default setting
ours	77.88	33.54	65.64	33.71
(a) SLAM variants
OpenVSLAM [ 42 ]	60.25	27.15	58.35	29.35
OpenVSLAM dense [ 43 ]	62.29	27.65	41.67	20.39

TABLE V: Effect of upstream perception modules. The default configuration uses ODGS-SLAM [ 32 ] and OOOPS [ 23 ] .

Explore similar work

FreeOcc: Training-Free Embodied Open-Vocabulary Occupancy Prediction
Apr 30, 2026Zeyu Jiang, Changqing Zhou, Xingxing Zuo +1Semantic Occupancy Mapping 3D Occupancy Prediction
The Hong Kong University of Science and Technology (Guangzhou) · MBZUAI
GEM-Occ: From Visual Geometry Evidence to Embodied Semantic Occupancy Memory
Jul 6, 2026Hu Zhu, Bohan Li, Xianda Guo +8Semantic Occupancy Mapping 3D Occupancy Prediction
The Hong Kong Polytechnic University · Eastern Institute of Technology · Shanghai Jiao Tong University +3
Monocular Open Vocabulary Occupancy Prediction for Indoor Scenes
Feb 26, 2026Changqing Zhou, Yueru Luo, Han Zhang +23D Occupancy Prediction Semantic Occupancy Mapping
The Hong Kong University of Science and Technology (Guangzhou) · The Chinese University of Hong Kong, Shenzhen

Apr 30, 2026J/K move · Enter open · S save

FreeOcc: Training-Free Embodied Open-Vocabulary Occupancy Prediction

Zeyu Jiang, Changqing Zhou, Xingxing Zuo, Changhao Chen

The Hong Kong University of Science and Technology (Guangzhou) · MBZUAI

Semantic Occupancy Mapping 3D Occupancy Prediction

Open paper AskPDF arXiv

Existing learning-based occupancy prediction methods rely on large-scale 3D annotations and generalize poorly across environments. We present FreeOcc, a training-free framework for open-vocabulary occupancy prediction from monocular or RGB-D sequences. Unlike prior approaches that require voxel-level supervision and ground-truth camera poses, FreeOcc operates without 3D annotations, pose ground truth, or any learning stage. FreeOcc incrementally builds a globally consistent occupancy map via a four-layer pipeline: a SLAM backbone estimates poses and sparse geometry; a geometrically consistent Gaussian update constructs dense 3D Gaussian maps; open-vocabulary semantics from off-the-shelf vision-language models are associated with Gaussian primitives; and a probabilistic Gaussian-to-occupancy projection produces dense voxel occupancy. Despite being entirely training-free and pose-agnostic, FreeOcc achieves over

2\times

improvements in IoU and mIoU on EmbodiedOcc-ScanNet compared to prior self-supervised methods. We further introduce ReplicaOcc, a benchmark for indoor open-vocabulary occupancy prediction, and show that FreeOcc transfers zero-shot to novel environments, substantially outperforming both supervised and self-supervised baselines. Project page: https://the-masses.github.io/freeocc-web/.

Abstract

Explore similar work

FreeOcc: Training-Free Embodied Open-Vocabulary Occupancy Prediction

GEM-Occ: From Visual Geometry Evidence to Embodied Semantic Occupancy Memory

Monocular Open Vocabulary Occupancy Prediction for Indoor Scenes