Seeing Beyond: Extrapolative Domain Adaptive Panoramic Segmentation

Yuanfan Zheng; Kunyu Peng; Xu Zheng; Kailun Yang

cs.CVMar 16, 2026

Seeing Beyond: Extrapolative Domain Adaptive Panoramic Segmentation

Authors: Yuanfan Zheng, Kunyu Peng, Xu Zheng, Kailun Yang

Organizations: Hunan University · Karlsruhe Institute of Technology · INSAIT, Sofia University “St. Kliment Ohridski” · HKUST(GZ)

Abstract

Cross-domain panoramic semantic segmentation has attracted growing interest as it enables comprehensive 360° scene understanding for real-world applications. However, it remains particularly challenging due to severe geometric Field of View (FoV) distortions and inconsistent open-set semantics across domains. In this work, we formulate an open-set domain adaptation setting, and propose Extrapolative Domain Adaptive Panoramic Segmentation (EDA-PSeg) framework that trains on local perspective views and tests on full 360° panoramic images, explicitly tackling both geometric FoV shifts across domains and semantic uncertainty arising from previously unseen classes. To this end, we propose the Euler-Margin Attention (EMA), which introduces an angular margin to enhance viewpoint-invariant semantic representation, while performing amplitude and phase modulation to improve generalization toward unseen classes. Additionally, we design the Graph Matching Adapter (GMA), which builds high-order graph relations to align shared semantics across FoV shifts while effectively separating novel categories through structural adaptation. Extensive experiments on four benchmark datasets under camera-shift, weather-condition, and open-set scenarios demonstrate that EDA-PSeg achieves state-of-the-art performance, robust generalization to diverse viewing geometries, and resilience under varying environmental conditions. The code is available at https://github.com/zyfone/EDA-PSeg.

Figures & tables

Figure 1: Extrapolative Domain Adaptation (EDA) extends beyond the training FoV and known semantic categories, facilitating the transfer of knowledge from pinhole supervision to unlabeled 360° perception. It addresses both cross-view geometric distortions and the semantic extrapolation to unknown categories.

Figure 2: Illustration of the proposed EDA - PS eg, which tackles cross-view and semantic extrapolation. The framework incorporates two key components: the Graph Matching Adapter (GMA), which constructs a high-order graph to align domain shared class graph nodes, and the Euler-Margin Attention (EMA), which models features with the Euler formula to enhance angle invariance under unseen viewpoints.

Figure 3: (a) EulerFormer [ 47 ] fails to build a semantic angle constraint, limiting generalization to unseen views. (b) Our method employs Euler-Margin Projection to constrain the angle within the interval, and utilizes Amplitude and Phase Modulation to adjust the class distribution for known and unknown class separation.

Figure 4: Feature distributions of the car category visualized via t-SNE across multiple datasets, illustrating the effects of Camera Types (Pinhole, Panoramic, Real, and Synthetic) and varying Weather Conditions (Fog, Rain, Cloudy, Sunny, Night, and Snow).

Method	Road	S.walk	Build.	Wall	Fence	Tr.light	Veget.	Terrain	Sky	Car	Bus	M.cycle	Bicycle	Common	Private	H-Score
OSBP* [ 44 ]	39.29	24.94	84.90	34.11	40.23	27.70	78.34	16.00	26.52	53.05	21.89	51.76	34.34	41.00	7.01	11.97
UAN* [ 58 ]	46.78	26.59	82.61	36.57	43.02	18.97	79.09	21.09	43.51	65.06	13.30	48.97	20.31	41.99	0.93	1.81
UniOT* [ 21 ]	63.59	31.37	86.79	42.35	38.04	18.90	79.82	37.86	81.45	68.89	37.91	59.10	31.67	52.13	0.00	0.00
DAFormer [ 18 ]	65.37	34.95	85.75	35.89	37.61	30.46	79.16	36.36	84.46	69.77	31.61	62.26	37.07	53.13	0.00	0.00
MIC [ 20 ]	51.04	15.66	84.28	36.06	35.34	30.55	76.44	19.56	73.52	72.81	28.31	55.48	29.70	46.83	0.00	0.00
HRDA [ 19 ]	68.84	34.89	85.71	38.39	39.64	28.12	79.58	38.74	90.78	68.23	29.37	59.62	32.53	53.42	0.00	0.00

Table 1: Comparisons under Camera Shift { Pin2Pan, Real2Real } for Cityscapes

\rightarrow

DensePASS ( C2D ) open-set domain adaptation in outdoor scenarios. * indicates an experiment based on DAFormer [ 18 ] .

Method	Road	S.walk	Build.	Wall	Fence	Pole	Tr.light	Tr.sign	Veget.	Terrain	Sky	Person	Car	Common	Private	H-Score
OSBP* [ 44 ]	34.68	0.20	23.01	3.17	2.37	0.74	0.00	0.27	25.58	4.69	56.97	0.17	3.91	11.98	5.77	7.79
UAN* [ 58 ]	51.01	2.22	33.48	1.52	7.33	7.73	0.00	0.59	29.75	12.07	58.77	10.0	6.72	17.01	0.83	1.59
UniOT* [ 21 ]	47.14	2.73	32.15	1.64	5.12	7.93	0.00	0.20	27.71	1.01	50.80	21.97	2.24	15.44	0.00	0.00
DAFormer [ 18 ]	52.20	4.44	34.42	3.24	10.16	5.80	0.00	0.12	56.07	6.33	63.67	37.26	2.31	21.23	0.00	0.00
MIC [ 20 ]	48.23	3.6	46.16	0.04	10.52	4.77	0.00	0.20	25.68	3.52	65.18	3.27	2.83	16.46	0.00	0.00
HRDA [ 19 ]	53.18	7.29	36.77	3.04	9.35	5.09	0.00	0.09	46.23	5.41	61.59	17.58	3.09	19.13	0.00	0.00

Table 2: Comparisons under Camera Shift { Syn2Real, Pan2Pan } for SynPASS

\rightarrow

DensePASS ( S2D ) open-set domain adaptation in outdoor scenarios. * indicates an experiment based on DAFormer [ 18 ] .

Method	Road	S.walk	Build.	Wall	Fence	Pole	Tr.light	Tr.sign	Veget.	Terrain	Sky	Person	Car	Common	Private	H-Score
OSBP* [ 44 ]	75.30	35.06	40.29	0.00	11.27	15.14	8.60	21.40	43.69	47.07	57.05	32.83	43.37	33.16	0.00	0.00
UAN* [ 58 ]	67.23	26.45	31.54	0.06	6.25	19.58	20.71	18.37	44.69	39.72	27.94	22.41	25.22	26.94	0.73	1.42
UniOT* [ 21 ]	88.10	57.96	56.18	0.07	14.98	19.95	18.89	14.41	42.46	47.94	87.44	16.95	61.36	40.52	0.00	0.00
DAFormer [ 18 ]	95.69	51.33	58.01	0.02	9.67	22.30	8.77	14.88	53.18	43.83	88.08	32.80	64.98	41.81	0.00	0.00
MIC [ 20 ]	97.38	59.72	53.17	0.52	11.56	20.57	24.20	19.46	56.61	44.62	84.15	28.69	64.90	43.51	0.00	0.00
HRDA [ 19 ]	91.13	54.68	49.92	0.24	10.33	20.03	20.70	13.25	41.23	37.74	80.60	29.12	67.33	39.72	0.00	0.00

Table 3: Comparisons under different Camera Shift (Pin2Pan,Syn2Syn ) and Weather Shift { Sunny }

\rightarrow

{ Sunny , Cloudy, Fog, Rain, Night } for GTA

\rightarrow

SynPASS ( G2S ) open-set domain adaptation in outdoor scenarios. * indicates an experiment based on DAFormer [ 18 ] .

Method	Road	S.walk	Build.	Wall	Fence	Pole	Tr.light	Tr.sign	Veget.	Terrain	Sky	Person	Car	Common	Private	H-Score
OSBP* [ 44 ]	52.79	0.89	6.72	0.80	0.41	0.04	0.00	0.00	32.79	0.00	73.7	0.21	0.14	12.96	5.02	7.24
UAN* [ 58 ]	67.50	7.27	26.60	0.97	1.26	14.20	0.00	0.00	50.96	13.10	67.25	0.23	0.34	19.21	0.13	0.26
UniOT* [ 21 ]	57.98	2.00	20.36	0.03	2.09	4.51	0.00	0.00	59.15	0.51	87.66	8.53	4.52	19.03	0.00	0.00
MIC [ 20 ]	62.32	10.60	60.77	0.00	1.17	3.18	0.00	0.00	60.02	4.71	80.46	26.67	11.82	24.75	0.00	0.00
DAFormer [ 18 ]	66.49	6.90	30.34	0.07	3.12	1.96	0.00	0.00	75.47	19.46	93.97	33.34	0.01	25.47	0.00	0.00
HRDA [ 19 ]	66.40	8.37	31.28	2.05	1.66	5.77	0.00	0.00	57.23	19.00	75.79	37.27	14.81	24.59	0.00	0.00

Table 4: Comparisons under Camera Shift (Syn2Real, Pan2Pin ) and Weather Shift { Fog, Rain, Night, Sunny , Cloudy }

\rightarrow

{ Fog, Rain, Night, Sunny , Snow } for SynPASS

\rightarrow

ACDC ( S2A ) open-set domain adaptation in outdoor scenarios. * indicates DAFormer-based results [ 18 ] .

Figure 5: T-SNE visualization of source and target domains in EDA-PS eg. (a) The prototype exhibits a relatively mixed unknown class. (b) Ours is the relatively separable unknown class.

Cityscapes $\rightarrow$ DensePASS ( C2D )
Exp.	GMA	EMA	Common	Private	H-Score
①	-	-	52.56	8.57	14.74
②	✓		55.15	14.67	23.18
③		✓	56.12	13.00	21.11
④	✓	✓	56.81	18.86	28.32

Table 5: Ablation study of the components in our framework.

Cityscapes $\rightarrow$ DensePASS ( C2D )
Exp.	Method	Common	Private	H-Score
①	GMA w/o Unknown	54.72	7.78	13.62
②	GMA w/o Matching	52.75	4.76	8.73
③	GMA w/o Affinity	54.06	11.59	19.09
④	GMA (Full)	55.15	14.67	23.18

Table 6: Ablation analysis of the GMA module.

Cityscapes $\rightarrow$ DensePASS ( C2D )
Exp.	Weight $\gamma$	Common	Private	H-Score
①	-	52.56	8.57	14.74
②	1.00	47.16	15.45	23.28
③	0.10	55.15	14.67	23.18
④	0.05	54.57	7.90	13.80
⑤	0.01	51.72	4.59	8.43

Table 7: Sensitivity of the GMA module loss to the factor

\gamma

Cityscapes $\rightarrow$ DensePASS ( C2D )
Exp.	Method	Common	Private	H-Score
①	Baseline	52.56	8.57	14.74
②	Self-Attention [ 49 ]	55.45	10.95	18.28
③	Eulerformer [ 47 ]	55.09	7.20	12.74
④	Deformable MLP [ 61 ]	55.89	7.68	13.51
⑤	Euler-Margin (Ours)	56.12	13.0	21.11

Table 8: Effectiveness analysis of the EMA module, added at the same position as in the baseline with the same layers.

Figure 6: Conceptual comparison of the domain adaptation setting.

Domain Adaptation Setting	Label Shift	Weather Shift	FoV Shift
Unsupervised Domain Adaptation [ 14 ]	✗	✗	N/A
Panoramic Domain Adaptation [ 70 ]	✗	✗	✓
Open-Set Domain Adaptation [ 13 ]	✓	✗	N/A
Open Compound Domain Adaptation [ 35 ]	✗	✓	N/A
Panoramic Open-Set Domain Adaptation (Ours)	✓	✓	✓

Table 9: Comparison of domain adaptation settings with the domain shift.

Component	Setting
Framework	Two-branch open-set UDA: source-supervised + target self-training.
Backbone	DAFormer with MiT-B5 encoder and Graph decoder.
Auxiliary Net	MobileSAM for pseudo-mask refinement.
Input Size	$512\times 512$ (crop, resize, flip, normalize).
Normalization	ImageNet mean=[123.7,116.3,103.5], std=[58.4,57.1,57.4].
Pseudo-labels	Refined by MobileSAM [ 59 ] .

Table 12: Key Settings of the Proposed EDA-PSeg Framework

Cityscapes $\rightarrow$ DensePASS ( C2D )
Exp.	Method	Common	Private	H-Score
①	Baseline	52.56	8.57	14.74
①	Self-Att	55.45	10.95	18.28
①	Self-Att+Margin Projection	55.64	15.06	23.71
②	EMA w/o Margin Projection	55.03	7.56	13.30
③	EMA w/o Learnable Scale	55.50	13.07	21.16

Table 13: Ablation analysis of the EMA module.

Method	#Params	FLOPs	MACs	Time
OSBP [ 44 ]	85.15 M	59.45 G	29.73 G	26.86 ms
UAN [ 58 ]	85.29 M	59.45 G	29.73 G	26.67 ms
UniOT [ 21 ]	85.22 M	59.45 G	29.73 G	27.03 ms
DMLP [ 61 ]	90.15 M	59.45 G	29.73 G	27.12 ms
MIC [ 20 ]	85.68 M	14.86 G	7.43 G	26.62 ms
DAF [ 18 ]	85.15 M	59.45 G	29.73 G	27.76 ms

Table 14: Comparison of the number of parameters (M), FLOPs (G), MACs (G), and test time per image (ms). Experiments are conducted on a single RTX 4090 and AMD Ryzen 9 5950X CPU.

Cityscapes $\rightarrow$ DensePASS ( C2D )
Exp.	Layer	Common	Private	H-Score
①	-	52.56	8.57	14.74
②	2	56.12	13.00	21.11
③	4	54.80	9.94	16.83
⑤	8	54.99	6.04	10.88

Table 15: Sensitivity analysis of the self-attention layer in the EMA module. Note that, since the Euler encoding in EMA requires both real and imaginary components, the dimensionality must be even.

Cityscapes $\rightarrow$ DensePASS ( C2D )
Exp.	Threshold	Common	Private	H-Score
①	0.4	53.03	1.66	3.22
②	0.5	56.02	6.17	11.11
③	0.6	56.81	18.86	28.32
④	0.7	54.59	9.66	16.41

Table 16: Sensitivity analysis under different threshold values. Results are reported on Cityscapes

\rightarrow

DensePASS ( C2D ).

Figure 7: Qualitative comparison between our method and existing OSDA and UDA approaches.

Figure 8: Illustration of the match matrix derived from the graph affinity and the matching ground-truth in the GMA module.

Figure 9: T-SNE visualization of data from seen and unseen viewpoints.

Figure 10: Qualitative results for EMA. We visualize feature representations using Principal Component Analysis (PCA) and the L2 norm.

Figure 11: Societal implications of panoramic vision technology.

Explore similar work

Panoramic Out-of-Distribution Segmentation
May 6, 2025Mengfei Duan, Yuheng Zhang, Yihong Cao +5OOD Detection Semantic Segmentation
The authors are with the School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, Changsha 410082, China
Denoise and Align: Towards Source-Free UDA for Robust Panoramic Semantic Segmentation
Mar 26, 2026Yaowen Chang, Zhen Cao, Xu Zheng +2Semantic Segmentation Source-Free Domain Adaptation
Wuhan University · HKUST (Guangzhou) · Wuhan University of Technology
MC-PanDA++: Simpler, Stronger, and More Robust Domain-Adaptive Panoptic Segmentation
Sep 30, 2026Ivan Martinović, Josip Šarić, Yuki M. Asano +1Panoptic Segmentation Unsupervised Domain Adaptation
Faculty of Electrical Engineering and Computing, University of Zagreb, Unska 3, 10000 Zagreb, Croatia. · Faculty of Computer and Information Science, University of Ljubljana, Večna Pot 113, 1000 Ljubljana, Slovenia. · Fundamental AI Lab, University of Technology Nuremberg, Dr.-Luise-Herzberg-Straße, 90461 Nürnberg, Germany.

May 6, 2025J/K move · Enter open · S save

Panoramic Out-of-Distribution Segmentation

Mengfei Duan, Yuheng Zhang, Yihong Cao, Fei Teng+4

The authors are with the School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, Changsha 410082, China

OOD Detection Semantic Segmentation Prompt Learning

Open paper AskPDF arXiv

Panoramic imaging enables capturing 360° images with an ultra-wide Field-of-View (FoV) for dense omnidirectional perception, which is critical to applications such as autonomous driving and augmented reality. However, current panoramic semantic segmentation methods fail to identify outliers, and pinhole out-of-distribution segmentation (pinhole-OoS) models perform unsatisfactorily in the panoramic domain due to pixel distortions and background clutter. To address these issues, we introduce a new task, Panoramic Out-of-Distribution Segmentation (PanOoS), with the aim of achieving comprehensive and safe scene understanding. Furthermore, we propose the first solution, POS, which adapts to the characteristics of panoramic images through text-guided prompt distribution learning. Specifically, POS integrates a disentanglement strategy designed to materialize the cross-domain generalization capability of CLIP. The proposed Prompt-based Restoration Attention (PRA) optimizes semantic decoding by prompt guidance and self-adaptive correction, while Bilevel Prompt Distribution Learning (BPDL) refines the manifold of per-pixel mask embeddings via semantic prototype supervision. Besides, to compensate for the scarcity of PanOoS datasets, we establish two benchmarks: DenseOoS, which features diverse outliers in complex environments, and QuadOoS, captured by a quadruped robot with a panoramic annular lens system. Extensive experiments demonstrate the effectiveness of POS, with AuPRC increasing and FPR95 decreasing by 34.25 and 21.42 percentage points, respectively, compared with RbA on DenseOoS, outperforming state-of-the-art pinhole-OoS methods. Moreover, POS achieves leading closed-set segmentation capabilities and advances the development of panoramic understanding. Code and datasets will be available at https://github.com/MengfeiD/PanOoS.

Abstract

Explore similar work

Panoramic Out-of-Distribution Segmentation

Denoise and Align: Towards Source-Free UDA for Robust Panoramic Semantic Segmentation

MC-PanDA++: Simpler, Stronger, and More Robust Domain-Adaptive Panoptic Segmentation