Panoramic Out-of-Distribution Segmentation

Mengfei Duan; Yuheng Zhang; Yihong Cao; Fei Teng; Kai Luo; Jiaming Zhang; Kailun Yang; Zhiyong Li

cs.CVMay 6, 2025

Panoramic Out-of-Distribution Segmentation

Authors: Mengfei Duan, Yuheng Zhang, Yihong Cao, Fei Teng, Kai Luo, Jiaming Zhang, Kailun Yang, Zhiyong Li

Organizations: The authors are with the School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, Changsha 410082, China

Abstract

Panoramic imaging enables capturing 360° images with an ultra-wide Field-of-View (FoV) for dense omnidirectional perception, which is critical to applications such as autonomous driving and augmented reality. However, current panoramic semantic segmentation methods fail to identify outliers, and pinhole out-of-distribution segmentation (pinhole-OoS) models perform unsatisfactorily in the panoramic domain due to pixel distortions and background clutter. To address these issues, we introduce a new task, Panoramic Out-of-Distribution Segmentation (PanOoS), with the aim of achieving comprehensive and safe scene understanding. Furthermore, we propose the first solution, POS, which adapts to the characteristics of panoramic images through text-guided prompt distribution learning. Specifically, POS integrates a disentanglement strategy designed to materialize the cross-domain generalization capability of CLIP. The proposed Prompt-based Restoration Attention (PRA) optimizes semantic decoding by prompt guidance and self-adaptive correction, while Bilevel Prompt Distribution Learning (BPDL) refines the manifold of per-pixel mask embeddings via semantic prototype supervision. Besides, to compensate for the scarcity of PanOoS datasets, we establish two benchmarks: DenseOoS, which features diverse outliers in complex environments, and QuadOoS, captured by a quadruped robot with a panoramic annular lens system. Extensive experiments demonstrate the effectiveness of POS, with AuPRC increasing and FPR95 decreasing by 34.25 and 21.42 percentage points, respectively, compared with RbA on DenseOoS, outperforming state-of-the-art pinhole-OoS methods. Moreover, POS achieves leading closed-set segmentation capabilities and advances the development of panoramic understanding. Code and datasets will be available at https://github.com/MengfeiD/PanOoS.

Figures & tables

Fig. 1 : PanOoS enhances comprehensive and safe omnidirectional scene perception. \scriptsize{1}⃝: pinhole image with narrow FoV; \scriptsize{2}⃝: $360^{\circ}$ panoramic image; \scriptsize{3}⃝: segmentation result. The tiger in the image is an outlier on the street.

Fig. 2 : Comparison of panoramic semantic and out-of-distribution segmentation paradigms. (a) Panoramic semantic segmentation methods adapt the model from pinhole to panoramas through domain adaptation, yet remain limited to in-distribution dense predictions; (b) Pinhole- OoS methods combine semantic segmentation with OoD scoring to detect outliers, but are restricted to narrow pinhole FoV; (c) POS achieves segmentation in the distribution space of panoramic images via prompt distribution learning. The pinhole images denote the training-domain samples, including labeled inlier data and auxiliary outlier-exposure data, while the panoramic images denote the target-domain evaluation data.

Fig. 3 : Impact of pixel distortions and background clutter on state-of-the-art pinhole- OoS methods. The white dashed boxes highlight representative challenging regions, while the red solid contours indicate anomalous objects or regions. Existing pinhole- OoS methods, e.g. , Mask2Anomaly and RbA, fail to adapt to pixel distortions and background clutter, resulting in a higher number of false positives. In contrast, the proposed POS effectively enhances semantic perception of panoramic pixels.

Fig. 4 : Overview of POS. POS consists of an image encoder, a text encoder, a pixel decoder, and a Transformer decoder equipped with Prompt-based Restoration Attention (PRA). During outlier exposure fine-tuning, Bilevel Prompt Distribution Learning (BPDL) regularizes the embedding manifold of per-pixel mask embeddings with class-level and distribution-level prompt priors. See Fig. 6 for details of BPDL. During inference, POS jointly processes image features and prompt embeddings to predict masks and class probabilities, which are further integrated into per-pixel class logits for final OoD scoring.

Fig. 5 : Prompt-based Restoration Attention continuously employs masked attention and cross attention between image features and prompt embeddings while refining semantic clarity through query semantic self-adaptive correction.

Fig. 6 : Bilevel Prompt Distribution Learning. The distribution prompts, text prompts for each class, and pixel embeddings are denoted by stars, diamonds, and circles, respectively. The dashed circles indicate equidistant lines from the semantic center, and the dashed lines represent the decision boundary separating in- and out-of-distribution regions.

Fig. 7 : In-distribution Learning. Enhancing intra-class compactness: the similarity of pixel embeddings within the same class is reinforced, resulting in a more compact intra-class distribution. Improving inter-class separation: the embedding distance between different classes is expanded, thereby strengthening the discriminability of inter-class features.

Fig. 8 : Distribution Learning. Isolation of out-of-distribution pixels: A distinct boundary is established in the feature space to separate out-of-distribution regions from known class regions, thereby mitigating overconfident predictions on outliers. Aggregation of out-of-distribution pixels: Through semantic guidance, out-of-distribution pixel embeddings are clustered into specific semantic spaces, preventing their dispersion and interference with the feature distribution of known classes.

Datasets	Year	Domain	Annotation	Test Images	OoD realism	No shift
Road Anomaly [ 59 ]	2019	Pinhole	outlier	60	✓	✘
Fishyscapes L&F [ 51 ]	2019		outlier	275	✓	(✓)
Fishyscapes Static [ 51 ]	2019		outlier	1000	✘	✓
StreetHazards [ 56 ]	2019		all	1500	(✓)	✘
SMIYC RA-21 [ 50 ]	2021		outlier	100	✓	✘
SMIYC RO-21 [ 50 ]	2021		outlier	327	✓	✘

TABLE I : Comparison of different OoS datasets. Referring to POC [ 84 ] , we qualitatively compare datasets on two main axes, and score them as good ( ✓ ), medium ( (✓) ) or bad ( ✘ ).

Fig. 9 : Distribution details of DenseOoS. DenseOoS contains $30$ distinct outliers, divided into $5$ main categories: Animals, Garbage, Furniture, Vehicles & Tools, and Others.

Fig. 10 : Examples from the DenseOoS dataset. Image backgrounds are highly cluttered, and distortions are severe. Outliers appear in semantically plausible regions. Zoom in for a better view.

Panel of Figure 12 — Fig. 11 : (a) Quadruped robot platform with a panoramic annular camera; (b) is the data statistics; (c) Y-axis pixel positions of quadruped robot motion trajectories. While in motion, the quadruped robot experiences significant vertical jitter.

Fig. 12 : Examples from the QuadOoS dataset. The images come with challenges, including uneven exposure, color inconsistencies due to the broader FoV, and increased motion blur. Numerous small outliers are scattered across the scene.

Method	DenseOoS			QuadOoS		Mean
Method	AuPRC $\uparrow$	FPR 95 $\downarrow$	mIoU $\uparrow$	AuPRC $\uparrow$	FPR 95 $\downarrow$	AuPRC $\uparrow$	FPR 95 $\downarrow$
PEBAL [ 31 ]	16.39	7.59	46.29	11.34	39.25	13.87	23.42
DenseHybrid [ 32 ]	24.44	7.74	45.19	19.09	33.86	21.77	20.80
EAM [ 23 ]	17.97	6.46	61.58	33.05	79.66	25.51	43.06
Mask2Anomaly [ 25 ]	36.28	8.79	48.10	25.93	92.55	31.10	50.67
RbA [ 24 ]	51.31	21.87	57.54	34.12	90.34	42.71	56.11

TABLE II : Panoramic out-of-distribution segmentation quantitative evaluation. POS shows significant improvement over baseline per-pixel and mask-transformer-based pinhole- OoS methods on PanOoS . Higher values for AuPRC and mIoU are better, whereas for FPR 95 lower values are better. The best and second best results are bold and underlined , respectively, and the line in-between the table divides the per-pixel architectures and mask-transformer-based methods.

Method	mIoU
DAFormer [ 33 ]	44.53
Trans4PASS (Tiny) [ 10 ]	44.93
Trans4PASS (Small) [ 10 ]	49.35
Trans4PASS+ (Tiny) [ 1 ]	49.29
Trans4PASS+ (Small) [ 1 ]	50.11
360SFUDA++ [ 48 ]	50.46

TABLE III : Quantitative closed-set segmentation results on DenseOoS. POS achieves the highest mIoU among the compared methods.

Method	AuPRC	FPR 95
PEBAL [ 31 ]	62.37	28.29
DenseHybrid [ 32 ]	-	-
EAM [ 23 ]	69.40	7.70
Mask2Anomaly [ 25 ]	79.70	13.45
RbA [ 24 ]	85.42	6.92
Ours	89.39	6.25

TABLE IV : Pinhole out-of-distribution segmentation results on Road Anomaly. The best and second best results are bold and underlined , respectively; “-” indicates the unavailability of benchmark results.

PRA	BPDL	AuPRC	FPR 95	mIoU
		76.88	0.84	61.73
✓		83.94	0.46	62.50
✓	✓	85.56	0.45	63.30

TABLE V : Component-wise ablation study of POS. PRA: Prompt-based Restoration Attention; BPDL: Bilevel Prompt Distribution Learning.

MA	CA	SAC	AuPRC	FPR 95	mIoU
✓			81.24	0.67	58.03
✓	✓		81.98	0.53	62.48
✓	✓	✓	85.56	0.45	63.30

TABLE VI : Structural ablation of Prompt-based Restoration Attention. MA: Masked Attention; CA: Cross Attention; SAC: Self-adaptive Correction.

Panel of Figure 19 — Fig. 13 : Attention maps of queries in PRA after different query stages, resembling a confusion matrix in which each query has the highest semantic similarity with itself. When querying only with image features, the semantic information of queries is relatively vague. However, after further interaction with the prompt embeddings, the semantics become clearer, and the self-adaptive correction further mitigates the impact of pixel distortion on semantic decoding.

$\mathcal{L}_{pixel}$	$\mathcal{L}_{ind}$	$\mathcal{L}_{inDir}$	$\mathcal{L}_{outlier}$	AuPRC	FPR 95	mIoU
✓				81.72	0.54	62.69
✓		✓	✓	82.61	0.50	62.67
✓	✓		✓	83.61	0.49	63.04
✓	✓	✓		83.37	0.49	63.06
	✓	✓	✓	83.08	0.50	62.89
✓	✓	✓	✓	85.56	0.45	63.30

TABLE VII : Ablation study of different loss contributions in bilevel prompt distribution learning loss

\mathcal{L}_{bpdl}

Panel of Figure 21 — Fig. 14 : Sensitivity analysis of $s$ , $d$ , $\alpha$ and $\lambda_{bpdl}$ .

TABLE VIII : Ablation study on outlier exposure fine-tuning. Lower NLL and ECE indicate better probabilistic prediction quality and calibration, respectively. Params denotes the number of parameters updated during OE fine-tuning.

Fig. 15 : OoD score statistics on DenseOoS and QuadOoS. (a) and (b) show the relative-frequency distributions of ID and OoD pixel scores. (c) shows the relationship between the logarithm of GT OoD-component area and mean OoD score, together with the Spearman correlation

\rho

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Class	ID	Number
stroller	19	19
trolley	20	20
garbage bag	21	55
wheelie bin	22	41
suitcase	23	16
skateboard	24	13

Appendix

TABLE B.1 : Details of the DenseOoS dataset.

Fig. B.1 : Examples of outliers included in the QuadOoS dataset. Including but not limited to the above types.

Method	road	sidewalk	building	wall	fence	pole	traffic light	traffic sign	vegetation	terrain	sky	person	rider	car	truck	bus	train	motorcycle	bicycle	Metric
DAFormer [ 33 ]	48.94	31.78	84.12	26.51	31.06	24.92	6.83	7.73	71.86	40.63	91.23	44.13	2.1	78.02	55.02	25.24	84.89	61.37	29.59	44.53
Trans4PASS (S) [ 10 ]	74.28	30.68	84.79	28.45	36.60	27.90	15.40	15.11	78.99	33.71	93.28	49.69	6.79	82.00	46.42	51.79	72.84	68.34	40.63	49.35
Trans4PASS+ (S) [ 1 ]	74.74	34.17	85.43	23.46	42.78	28.45	17.14	18.08	79.64	29.41	93.00	48.74	5.40	82.76	50.54	44.91	91.49	61.49	40.20	50.11
360SFUDA++ [ 48 ]	72.29	45.24	84.91	32.11	38.18	29.83	19.28	17.98	76.02	27.23	92.01	53.14	18.74	81.35	44.42	45.38	82.84	60.93	36.92	50.46
Ours	87.67	59.56	89.68	50.88	45.10	42.62	34.02	22.73	81.21	45.19	94.59	72.24	14.42	86.62	64.88	69.16	95.27	82.98	63.92	63.30

Appendix

TABLE C.1 : Panoramic semantic segmentation results on the DenseOoS benchmark. The per-class results are reported as IoU, and the metric is mIoU.

Features	Layer	AuPRC	FPR 95	mIoU
$f_{4}$	1	85.56	0.45	63.30
	3	71.18	0.68	57.72
	6	76.46	0.64	57.33
$f_{2}$ , $f_{3}$ , $f_{4}$	3	72.38	0.94	58.96
	6	63.60	1.01	58.66
	9	65.91	1.06	58.79

Appendix

TABLE C.2 : Comparison of different features in Transformer Decoder. The

f_{2}

and

f_{3}

are feature maps with stride

8(c_{8})

and

16(c_{16})

processed by DeformAttn layers, respectively.

Method	Efficiency Metrics						DenseOoS
Method	Input Res.	Total Params (M) $\downarrow$	Trainable Params (M) $\downarrow$	TFLOPs $\downarrow$	FPS $\uparrow$	Mem. (GB) $\downarrow$	AuPRC $\uparrow$	FPR95 $\downarrow$	mIoU $\uparrow$
Ours	$400{\times}2048$	361.27	9.49	3.25	12.92	4.21	85.56	0.45	63.30

Appendix

TABLE C.3 : Efficiency analysis of POS when inferring with RTX 3090 on DenseOoS images.

Dataset	Size	Inst.	Median Area (%)	AuPRC	FPR 95	Inst. Recall $\uparrow$
DenseOoS	Small	250	0.142	50.75	0.35	96.22
	Medium	500	0.353	77.05	0.43	95.16
	Large	250	0.743	76.56	0.49	94.93
	Overall	1000	0.353	85.56	0.45	95.37
QuadOoS	Small	184	0.014	0.16	97.18	74.41
	Medium	367	0.061	10.38	90.51	88.10

Appendix

TABLE C.4 : Performance of POS across different outlier-instance sizes on DenseOoS and QuadOoS. Instances are grouped independently for each dataset according to their relative GT areas: Small (bottom

25\%

), Medium (middle

50\%

), and Large (top

25\%

). Median Area denotes the median instance area relative to the full panoramic image. Inst. Recall is the macro-averaged per-instance recall at the dataset-level threshold corresponding to

95\%

OoD pixel recall. Overall reports performance on the complete dataset.

Category	Images	Median Area (%)	AuPRC	FPR 95	Inst. Recall $\uparrow$
Animals	392	0.275	85.65	0.34	97.97
Garbage	145	0.354	80.21	0.63	91.24
Furniture	159	0.363	90.65	0.29	95.76
Vehicles & Tools	141	0.585	83.62	0.64	89.40
Others	163	0.373	92.47	0.23	97.58
Overall	1000	0.353	85.56	0.45	95.37

Appendix

TABLE C.5 : Category-wise OoD detection performance of POS on DenseOoS. The five super-categories follow the DenseOoS taxonomy. Pixel-level metrics are computed using all valid ID and OoD pixels from images belonging to each category.

Fig. D.1 : Qualitative comparisons of PanOoS . When existing pinhole- OoS methods are transferred to PanOoS , they tend to misidentify complex background regions as outliers. Although EAM [ 23 ] has a low false positive rate, it incorrectly identifies bicycles as outliers and nearly overlooks the actual outliers. In contrast, Mask2Anomaly [ 25 ] and RbA [ 24 ] suffer from significant false positives in the background regions, whereas POS delivers more precise PanOoS across the $360^{\circ}$ .

Fig. D.2 : Visualization results of PanOoS on DenseOoS. POS better distinguishes outliers from inliers and produces smoother outlier maps with fewer false positives. Zoom in for a better view.

Fig. D.3 : Visualization results of PanOoS on QuadOoS. Uneven exposure, defocus, and motion blur caused by dense motion pose extra challenges to all methods. The pinhole- OoS methods have serious omissions and false positives. However, POS achieves precise segmentation of outliers. Zoom in for a better view.

Fig. E.1 : Failure case analysis in PanOoS . Brighter colors indicate higher OoD confidence. The white dashed boxes highlight challenging regions, while the red solid contours indicate anomalous objects or regions.

Explore similar work

Seeing Beyond: Extrapolative Domain Adaptive Panoramic Segmentation
Mar 16, 2026Yuanfan Zheng, Kunyu Peng, Xu Zheng +1Semantic Segmentation Domain Adaptation
Hunan University · Karlsruhe Institute of Technology · INSAIT, Sofia University “St. Kliment Ohridski” +1
Denoise and Align: Towards Source-Free UDA for Robust Panoramic Semantic Segmentation
Mar 26, 2026Yaowen Chang, Zhen Cao, Xu Zheng +2Semantic Segmentation Source-Free Domain Adaptation
Wuhan University · HKUST (Guangzhou) · Wuhan University of Technology
PanoSeg3R: Feed-Forward 3D Semantic Segmentation for Panoramic Images with an Automatic Data Curation Pipeline
Sep 19, 2026Heechan Yoon, Dongki Jung, Phuc Nguyen +23D Semantic Segmentation
University of Maryland, College Park, USA

Mar 16, 2026J/K move · Enter open · S save

Seeing Beyond: Extrapolative Domain Adaptive Panoramic Segmentation

Yuanfan Zheng, Kunyu Peng, Xu Zheng, Kailun Yang

Hunan University · Karlsruhe Institute of Technology · INSAIT, Sofia University “St. Kliment Ohridski”+1

Semantic Segmentation Domain Adaptation

Open paper AskPDF arXiv

Cross-domain panoramic semantic segmentation has attracted growing interest as it enables comprehensive 360° scene understanding for real-world applications. However, it remains particularly challenging due to severe geometric Field of View (FoV) distortions and inconsistent open-set semantics across domains. In this work, we formulate an open-set domain adaptation setting, and propose Extrapolative Domain Adaptive Panoramic Segmentation (EDA-PSeg) framework that trains on local perspective views and tests on full 360° panoramic images, explicitly tackling both geometric FoV shifts across domains and semantic uncertainty arising from previously unseen classes. To this end, we propose the Euler-Margin Attention (EMA), which introduces an angular margin to enhance viewpoint-invariant semantic representation, while performing amplitude and phase modulation to improve generalization toward unseen classes. Additionally, we design the Graph Matching Adapter (GMA), which builds high-order graph relations to align shared semantics across FoV shifts while effectively separating novel categories through structural adaptation. Extensive experiments on four benchmark datasets under camera-shift, weather-condition, and open-set scenarios demonstrate that EDA-PSeg achieves state-of-the-art performance, robust generalization to diverse viewing geometries, and resilience under varying environmental conditions. The code is available at https://github.com/zyfone/EDA-PSeg.

Abstract

Explore similar work

Seeing Beyond: Extrapolative Domain Adaptive Panoramic Segmentation

Denoise and Align: Towards Source-Free UDA for Robust Panoramic Semantic Segmentation

PanoSeg3R: Feed-Forward 3D Semantic Segmentation for Panoramic Images with an Automatic Data Curation Pipeline