EmbPASS: Towards Cross-Embodiment Open Panoramic Segmentation

Pujun Guo; Yuanfan Zheng; Fei Teng; Mengfei Duan; Guoqiang Zhao; Yuheng Zhang; Kai Luo; Kailun Yang

cs.CVOct 2, 2026

EmbPASS: Towards Cross-Embodiment Open Panoramic Segmentation

Authors: Pujun Guo, Yuanfan Zheng, Fei Teng, Mengfei Duan, Guoqiang Zhao, Yuheng Zhang, Kai Luo, Kailun Yang

Organizations: School of Artificial Intelligence and Robotics, Hunan University, China · National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, China

Abstract

Panoramic images provide a complete 360-degree field of view, enabling comprehensive scene understanding for embodied perception. However, heterogeneous embodied platforms exhibit substantial differences in observation viewpoints and spatial layouts, giving rise to cross-embodiment observation shifts that pose additional challenges to consistent and reliable panoramic perception, while systematic studies of this problem remain limited. To bridge this gap, we introduce a new task, termed Cross-Embodiment Open Panoramic Segmentation. Meanwhile, we establish EmbPASS, a multi-platform panoramic semantic segmentation benchmark spanning Vehicle, Drone, Wearable, and Quadruped platforms under a unified semantic taxonomy, providing a testbed for systematically studying cross-embodiment panoramic perception. We further propose EPONet, an open-vocabulary panoramic semantic segmentation network that integrates Relation-Aware Metric Adapter (RAMA) and Content-Adaptive Semantic Transfer (CAST) to enhance spatial modeling and semantic transfer under heterogeneous embodied observations. Extensive experiments show that EPONet achieves the best platform-balanced performance on EmbPASS with 35.82% mIoU, outperforming the strongest baseline by 1.10%, while remaining competitive on existing panoramic segmentation benchmarks. The source code and EmbPASS benchmark will be made publicly available at https://github.com/guopj1/EmbPASS.

Figures & tables

Dataset	Scene	# Eval.	# Cls.	Acq. Platform	Platform Coverage
Stanford2D3D [ 12 ]	Indoor	1,413	13	Stat. scanner
Matterport3D [ 13 ]	Indoor	772	20	Stat. scanner
DensePASS [ 15 ]	Outdoor	100	19	Street-view sys.
WildPASS [ 14 ]	Outdoor	500	8	Street-view sys.
EmbPASS	Outdoor	1,000	19	Embodied platf.

TABLE I: Comparison of panoramic semantic segmentation benchmarks. Abbreviations: Stat. (stationary), Sys. (system), Platf. (platform), (stationary scanner), (vehicle), (drone), (wearable), (quadruped).

Panel of Figure 2 — Fig. 1: Dataset statistics and cross-platform analysis of EmbPASS. (a) Platform-level statistics, including the number of images, spatial resolution, and spherical latitude coverage. (b) Platform-specific semantic composition, measured by the fraction of valid labeled pixels belonging to each semantic group. (c) Class-wise spherical latitude distributions across the four platform subsets, where color intensity denotes the normalized pixel fraction within each spherical latitude bin.

Fig. 2: Overview of EPONet. (a) Overall framework with a frozen CLIP model and a trainable Side branch. (b) Relation-Aware Metric Adapter (RAMA) for relation-aware metric prediction and structured spatial aggregation. (c) Content-Adaptive Semantic Transfer (CAST) for content-adaptive reassembly and residual transfer of pretrained CLIP features.

TABLE II: Comparison with open-vocabulary segmentation methods on EmbPASS. We report mIoU and open mIoU (%). Avg. denotes the platform-balanced average over the four target platforms. #Params. is the number of learnable parameters in millions. Best and second-best results are shown in bold and underlined, respectively.

Panel of Figure 5 — Fig. 3: Qualitative comparison on EmbPASS. From top to bottom, we show the input image, predictions from SAN [ 24 ] , OOOPS with RERP [ 9 ] , our method, and the ground truth. Dashed boxes highlight representative regions for visual comparison.

Fig. 4: Controlled projection analysis across projection-center latitudes. Positive

\Delta\mathrm{mIoU}

favors RAMA.

Method	mIoU	Params. (M)
SAN [ 24 ]	33.82	8.37
$+\;$ Conv	34.15	8.65
$+\;$ DCN [ 38 ]	34.43	8.69
$+\;$ DCNv2 [ 39 ]	34.43	8.71
$+\;$ DCNv3 [ 40 ]	34.46	8.56
$+\;$ DAO [ 9 ]	34.09	8.56

TABLE V: Comparison of spatial aggregation operators on EmbPASS. Results are averaged across the four platforms.

Explore similar work

Panoramic Affordance Prediction
Mar 16, 2026Zixin Zhang, Chenfei Liao, Hongfei Zhang +10Robotic Perception Fine-Grained Visual Perception
HKUST(GZ) · SJTU · MBZUAI +3
Seeing Beyond: Extrapolative Domain Adaptive Panoramic Segmentation
Mar 16, 2026Yuanfan Zheng, Kunyu Peng, Xu Zheng +1Semantic Segmentation Domain Adaptation
Hunan University · Karlsruhe Institute of Technology · INSAIT, Sofia University “St. Kliment Ohridski” +1
Panoramic Out-of-Distribution Segmentation
May 6, 2025Mengfei Duan, Yuheng Zhang, Yihong Cao +5OOD Detection Semantic Segmentation
The authors are with the School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, Changsha 410082, China

Mar 16, 2026J/K move · Enter open · S save

Panoramic Affordance Prediction

Zixin Zhang, Chenfei Liao, Hongfei Zhang, Harold Haodong Chen+9

HKUST(GZ) · SJTU · MBZUAI+3

Robotic Perception Fine-Grained Visual Perception

Open paper AskPDF arXiv

Affordance prediction serves as a critical bridge between perception and action in embodied AI. However, existing research is confined to pinhole camera models, which suffer from narrow Fields of View (FoV) and fragmented observations, often missing critical holistic environmental context. In this paper, we present the first exploration into Panoramic Affordance Prediction, utilizing 360-degree imagery to capture global spatial relationships and holistic scene understanding. To facilitate this novel task, we first introduce PAP-12K, a large-scale benchmark dataset containing over 1,000 ultra-high-resolution (12k, 11904 x 5952) panoramic images with over 12k carefully annotated QA pairs and affordance masks. Furthermore, we propose PAP, a training-free, coarse-to-fine pipeline inspired by the human foveal visual system to tackle the ultra-high resolution and severe distortion inherent in panoramic images. PAP employs recursive visual routing via grid prompting to progressively locate targets, applies an adaptive gaze mechanism to rectify local geometric distortions, and utilizes a cascaded grounding pipeline to extract precise instance-level masks. Experimental results on PAP-12K reveal that existing affordance prediction methods designed for standard perspective images suffer severe performance degradation and fail due to the unique challenges of panoramic vision. In contrast, PAP framework effectively overcomes these obstacles, significantly outperforming state-of-the-art baselines and highlighting the immense potential of panoramic perception for robust embodied intelligence.

Abstract

Explore similar work

Panoramic Affordance Prediction

Seeing Beyond: Extrapolative Domain Adaptive Panoramic Segmentation

Panoramic Out-of-Distribution Segmentation