Seeing Beyond: Extrapolative Domain Adaptive Panoramic Segmentation

Yuanfan Zheng; Kunyu Peng; Xu Zheng; Kailun Yang

cs.CVMar 16, 2026

Seeing Beyond: Extrapolative Domain Adaptive Panoramic Segmentation

Authors: Yuanfan Zheng, Kunyu Peng, Xu Zheng, Kailun Yang

Organizations: Hunan University · Karlsruhe Institute of Technology · INSAIT, Sofia University “St. Kliment Ohridski” · HKUST(GZ)

Abstract

Cross-domain panoramic semantic segmentation has attracted growing interest as it enables comprehensive 360° scene understanding for real-world applications. However, it remains particularly challenging due to severe geometric Field of View (FoV) distortions and inconsistent open-set semantics across domains. In this work, we formulate an open-set domain adaptation setting, and propose Extrapolative Domain Adaptive Panoramic Segmentation (EDA-PSeg) framework that trains on local perspective views and tests on full 360° panoramic images, explicitly tackling both geometric FoV shifts across domains and semantic uncertainty arising from previously unseen classes. To this end, we propose the Euler-Margin Attention (EMA), which introduces an angular margin to enhance viewpoint-invariant semantic representation, while performing amplitude and phase modulation to improve generalization toward unseen classes. Additionally, we design the Graph Matching Adapter (GMA), which builds high-order graph relations to align shared semantics across FoV shifts while effectively separating novel categories through structural adaptation. Extensive experiments on four benchmark datasets under camera-shift, weather-condition, and open-set scenarios demonstrate that EDA-PSeg achieves state-of-the-art performance, robust generalization to diverse viewing geometries, and resilience under varying environmental conditions. The code is available at https://github.com/zyfone/EDA-PSeg.

Figures & tables

Figure 1: Extrapolative Domain Adaptation (EDA) extends beyond the training FoV and known semantic categories, facilitating the transfer of knowledge from pinhole supervision to unlabeled 360° perception. It addresses both cross-view geometric distortions and the semantic extrapolation to unknown categories.

Figure 2: Illustration of the proposed EDA - PS eg, which tackles cross-view and semantic extrapolation. The framework incorporates two key components: the Graph Matching Adapter (GMA), which constructs a high-order graph to align domain shared class graph nodes, and the Euler-Margin Attention (EMA), which models features with the Euler formula to enhance angle invariance under unseen viewpoints.

Figure 3: (a) EulerFormer [ 47 ] fails to build a semantic angle constraint, limiting generalization to unseen views. (b) Our method employs Euler-Margin Projection to constrain the angle within the interval, and utilizes Amplitude and Phase Modulation to adjust the class distribution for known and unknown class separation.

Figure 4: Feature distributions of the car category visualized via t-SNE across multiple datasets, illustrating the effects of Camera Types (Pinhole, Panoramic, Real, and Synthetic) and varying Weather Conditions (Fog, Rain, Cloudy, Sunny, Night, and Snow).

Method	Road	S.walk	Build.	Wall	Fence	Tr.light	Veget.	Terrain	Sky	Car	Bus	M.cycle	Bicycle	Common	Private	H-Score
OSBP* [ 44 ]	39.29	24.94	84.90	34.11	40.23	27.70	78.34	16.00	26.52	53.05	21.89	51.76	34.34	41.00	7.01	11.97
UAN* [ 58 ]	46.78	26.59	82.61	36.57	43.02	18.97	79.09	21.09	43.51	65.06	13.30	48.97	20.31	41.99	0.93	1.81
UniOT* [ 21 ]	63.59	31.37	86.79	42.35	38.04	18.90	79.82	37.86	81.45	68.89	37.91	59.10	31.67	52.13	0.00	0.00
DAFormer [ 18 ]	65.37	34.95	85.75	35.89	37.61	30.46	79.16	36.36	84.46	69.77	31.61	62.26	37.07	53.13	0.00	0.00
MIC [ 20 ]	51.04	15.66	84.28	36.06	35.34	30.55	76.44	19.56	73.52	72.81	28.31	55.48	29.70	46.83	0.00	0.00
HRDA [ 19 ]	68.84	34.89	85.71	38.39	39.64	28.12	79.58	38.74	90.78	68.23	29.37	59.62	32.53	53.42	0.00	0.00

Table 1: Comparisons under Camera Shift { Pin2Pan, Real2Real } for Cityscapes

\rightarrow

DensePASS ( C2D ) open-set domain adaptation in outdoor scenarios. * indicates an experiment based on DAFormer [ 18 ] .

Method	Road	S.walk	Build.	Wall	Fence	Pole	Tr.light	Tr.sign	Veget.	Terrain	Sky	Person	Car	Common	Private	H-Score
OSBP* [ 44 ]	34.68	0.20	23.01	3.17	2.37	0.74	0.00	0.27	25.58	4.69	56.97	0.17	3.91	11.98	5.77	7.79
UAN* [ 58 ]	51.01	2.22	33.48	1.52	7.33	7.73	0.00	0.59	29.75	12.07	58.77	10.0	6.72	17.01	0.83	1.59
UniOT* [ 21 ]	47.14	2.73	32.15	1.64	5.12	7.93	0.00	0.20	27.71	1.01	50.80	21.97	2.24	15.44	0.00	0.00
DAFormer [ 18 ]	52.20	4.44	34.42	3.24	10.16	5.80	0.00	0.12	56.07	6.33	63.67	37.26	2.31	21.23	0.00	0.00
MIC [ 20 ]	48.23	3.6	46.16	0.04	10.52	4.77	0.00	0.20	25.68	3.52	65.18	3.27	2.83	16.46	0.00	0.00
HRDA [ 19 ]	53.18	7.29	36.77	3.04	9.35	5.09	0.00	0.09	46.23	5.41	61.59	17.58	3.09	19.13	0.00	0.00

Table 2: Comparisons under Camera Shift { Syn2Real, Pan2Pan } for SynPASS

\rightarrow

DensePASS ( S2D ) open-set domain adaptation in outdoor scenarios. * indicates an experiment based on DAFormer [ 18 ] .

Method	Road	S.walk	Build.	Wall	Fence	Pole	Tr.light	Tr.sign	Veget.	Terrain	Sky	Person	Car	Common	Private	H-Score
OSBP* [ 44 ]	75.30	35.06	40.29	0.00	11.27	15.14	8.60	21.40	43.69	47.07	57.05	32.83	43.37	33.16	0.00	0.00
UAN* [ 58 ]	67.23	26.45	31.54	0.06	6.25	19.58	20.71	18.37	44.69	39.72	27.94	22.41	25.22	26.94	0.73	1.42
UniOT* [ 21 ]	88.10	57.96	56.18	0.07	14.98	19.95	18.89	14.41	42.46	47.94	87.44	16.95	61.36	40.52	0.00	0.00
DAFormer [ 18 ]	95.69	51.33	58.01	0.02	9.67	22.30	8.77	14.88	53.18	43.83	88.08	32.80	64.98	41.81	0.00	0.00
MIC [ 20 ]	97.38	59.72	53.17	0.52	11.56	20.57	24.20	19.46	56.61	44.62	84.15	28.69	64.90	43.51	0.00	0.00
HRDA [ 19 ]	91.13	54.68	49.92	0.24	10.33	20.03	20.70	13.25	41.23	37.74	80.60	29.12	67.33	39.72	0.00	0.00

Table 3: Comparisons under different Camera Shift (Pin2Pan,Syn2Syn ) and Weather Shift { Sunny }

\rightarrow

{ Sunny , Cloudy, Fog, Rain, Night } for GTA

\rightarrow

SynPASS ( G2S ) open-set domain adaptation in outdoor scenarios. * indicates an experiment based on DAFormer [ 18 ] .

Method	Road	S.walk	Build.	Wall	Fence	Pole	Tr.light	Tr.sign	Veget.	Terrain	Sky	Person	Car	Common	Private	H-Score
OSBP* [ 44 ]	52.79	0.89	6.72	0.80	0.41	0.04	0.00	0.00	32.79	0.00	73.7	0.21	0.14	12.96	5.02	7.24
UAN* [ 58 ]	67.50	7.27	26.60	0.97	1.26	14.20	0.00	0.00	50.96	13.10	67.25	0.23	0.34	19.21	0.13	0.26
UniOT* [ 21 ]	57.98	2.00	20.36	0.03	2.09	4.51	0.00	0.00	59.15	0.51	87.66	8.53	4.52	19.03	0.00	0.00
MIC [ 20 ]	62.32	10.60	60.77	0.00	1.17	3.18	0.00	0.00	60.02	4.71	80.46	26.67	11.82	24.75	0.00	0.00
DAFormer [ 18 ]	66.49	6.90	30.34	0.07	3.12	1.96	0.00	0.00	75.47	19.46	93.97	33.34	0.01	25.47	0.00	0.00
HRDA [ 19 ]	66.40	8.37	31.28	2.05	1.66	5.77	0.00	0.00	57.23	19.00	75.79	37.27	14.81	24.59	0.00	0.00

Table 4: Comparisons under Camera Shift (Syn2Real, Pan2Pin ) and Weather Shift { Fog, Rain, Night, Sunny , Cloudy }

\rightarrow

{ Fog, Rain, Night, Sunny , Snow } for SynPASS

\rightarrow

ACDC ( S2A ) open-set domain adaptation in outdoor scenarios. * indicates DAFormer-based results [ 18 ] .

Figure 5: T-SNE visualization of source and target domains in EDA-PS eg. (a) The prototype exhibits a relatively mixed unknown class. (b) Ours is the relatively separable unknown class.

Cityscapes $\rightarrow$ DensePASS ( C2D )
Exp.	GMA	EMA	Common	Private	H-Score
①	-	-	52.56	8.57	14.74
②	✓		55.15	14.67	23.18
③		✓	56.12	13.00	21.11
④	✓	✓	56.81	18.86	28.32

Table 5: Ablation study of the components in our framework.

Cityscapes $\rightarrow$ DensePASS ( C2D )
Exp.	Method	Common	Private	H-Score
①	GMA w/o Unknown	54.72	7.78	13.62
②	GMA w/o Matching	52.75	4.76	8.73
③	GMA w/o Affinity	54.06	11.59	19.09
④	GMA (Full)	55.15	14.67	23.18

Table 6: Ablation analysis of the GMA module.

Cityscapes $\rightarrow$ DensePASS ( C2D )
Exp.	Weight $\gamma$	Common	Private	H-Score
①	-	52.56	8.57	14.74
②	1.00	47.16	15.45	23.28
③	0.10	55.15	14.67	23.18
④	0.05	54.57	7.90	13.80
⑤	0.01	51.72	4.59	8.43

Table 7: Sensitivity of the GMA module loss to the factor

\gamma

Cityscapes $\rightarrow$ DensePASS ( C2D )
Exp.	Method	Common	Private	H-Score
①	Baseline	52.56	8.57	14.74
②	Self-Attention [ 49 ]	55.45	10.95	18.28
③	Eulerformer [ 47 ]	55.09	7.20	12.74
④	Deformable MLP [ 61 ]	55.89	7.68	13.51
⑤	Euler-Margin (Ours)	56.12	13.0	21.11

Table 8: Effectiveness analysis of the EMA module, added at the same position as in the baseline with the same layers.

Figure 6: Conceptual comparison of the domain adaptation setting.

Domain Adaptation Setting	Label Shift	Weather Shift	FoV Shift
Unsupervised Domain Adaptation [ 14 ]	✗	✗	N/A
Panoramic Domain Adaptation [ 70 ]	✗	✗	✓
Open-Set Domain Adaptation [ 13 ]	✓	✗	N/A
Open Compound Domain Adaptation [ 35 ]	✗	✓	N/A
Panoramic Open-Set Domain Adaptation (Ours)	✓	✓	✓

Table 9: Comparison of domain adaptation settings with the domain shift.

Component	Setting
Framework	Two-branch open-set UDA: source-supervised + target self-training.
Backbone	DAFormer with MiT-B5 encoder and Graph decoder.
Auxiliary Net	MobileSAM for pseudo-mask refinement.
Input Size	$512\times 512$ (crop, resize, flip, normalize).
Normalization	ImageNet mean=[123.7,116.3,103.5], std=[58.4,57.1,57.4].
Pseudo-labels	Refined by MobileSAM [ 59 ] .

Table 12: Key Settings of the Proposed EDA-PSeg Framework

Cityscapes $\rightarrow$ DensePASS ( C2D )
Exp.	Method	Common	Private	H-Score
①	Baseline	52.56	8.57	14.74
①	Self-Att	55.45	10.95	18.28
①	Self-Att+Margin Projection	55.64	15.06	23.71
②	EMA w/o Margin Projection	55.03	7.56	13.30
③	EMA w/o Learnable Scale	55.50	13.07	21.16

Table 13: Ablation analysis of the EMA module.

Method	#Params	FLOPs	MACs	Time
OSBP [ 44 ]	85.15 M	59.45 G	29.73 G	26.86 ms
UAN [ 58 ]	85.29 M	59.45 G	29.73 G	26.67 ms
UniOT [ 21 ]	85.22 M	59.45 G	29.73 G	27.03 ms
DMLP [ 61 ]	90.15 M	59.45 G	29.73 G	27.12 ms
MIC [ 20 ]	85.68 M	14.86 G	7.43 G	26.62 ms
DAF [ 18 ]	85.15 M	59.45 G	29.73 G	27.76 ms

Table 14: Comparison of the number of parameters (M), FLOPs (G), MACs (G), and test time per image (ms). Experiments are conducted on a single RTX 4090 and AMD Ryzen 9 5950X CPU.

Cityscapes $\rightarrow$ DensePASS ( C2D )
Exp.	Layer	Common	Private	H-Score
①	-	52.56	8.57	14.74
②	2	56.12	13.00	21.11
③	4	54.80	9.94	16.83
⑤	8	54.99	6.04	10.88

Table 15: Sensitivity analysis of the self-attention layer in the EMA module. Note that, since the Euler encoding in EMA requires both real and imaginary components, the dimensionality must be even.

Cityscapes $\rightarrow$ DensePASS ( C2D )
Exp.	Threshold	Common	Private	H-Score
①	0.4	53.03	1.66	3.22
②	0.5	56.02	6.17	11.11
③	0.6	56.81	18.86	28.32
④	0.7	54.59	9.66	16.41

Table 16: Sensitivity analysis under different threshold values. Results are reported on Cityscapes

\rightarrow

DensePASS ( C2D ).

Figure 7: Qualitative comparison between our method and existing OSDA and UDA approaches.

Figure 8: Illustration of the match matrix derived from the graph affinity and the matching ground-truth in the GMA module.

Figure 9: T-SNE visualization of data from seen and unseen viewpoints.

Figure 10: Qualitative results for EMA. We visualize feature representations using Principal Component Analysis (PCA) and the L2 norm.

Figure 11: Societal implications of panoramic vision technology.

Explore similar work

May 6, 2025

Panoramic Out-of-Distribution Segmentation

Panoramic imaging enables capturing 360° images with an ultra-wide Field-of-View (FoV) for dense omnidirectional perception, which is critical to applications such as autonomous driving and augmented reality. However, current panoramic semantic segmentation methods fail to identify outliers, and pinhole out-of-distribution segmentation (pinhole-OoS) models perform unsatisfactorily in the panoramic domain due to pixel distortions and background clutter. To address these issues, we introduce a new task, Panoramic Out-of-Distribution Segmentation (PanOoS), with the aim of achieving comprehensive and safe scene understanding. Furthermore, we propose the first solution, POS, which adapts to the characteristics of panoramic images through text-guided prompt distribution learning. Specifically, POS integrates a disentanglement strategy designed to materialize the cross-domain generalization capability of CLIP. The proposed Prompt-based Restoration Attention (PRA) optimizes semantic decoding by prompt guidance and self-adaptive correction, while Bilevel Prompt Distribution Learning (BPDL) refines the manifold of per-pixel mask embeddings via semantic prototype supervision. Besides, to compensate for the scarcity of PanOoS datasets, we establish two benchmarks: DenseOoS, which features diverse outliers in complex environments, and QuadOoS, captured by a quadruped robot with a panoramic annular lens system. Extensive experiments demonstrate the effectiveness of POS, with AuPRC increasing and FPR95 decreasing by 34.25 and 21.42 percentage points, respectively, compared with RbA on DenseOoS, outperforming state-of-the-art pinhole-OoS methods. Moreover, POS achieves leading closed-set segmentation capabilities and advances the development of panoramic understanding. Code and datasets will be available at https://github.com/MengfeiD/PanOoS.

Mengfei Duan, Yuheng Zhang, Yihong Cao +5

The authors are with the School of Artificial Intelligence and Robotics and the National Engineering Research Center of Robot Visual Perception and Control Technology, Hunan University, Changsha 410082, China

OOD Detection Semantic Segmentation Prompt Learning

Ask PDF

Mar 26, 2026

Denoise and Align: Towards Source-Free UDA for Robust Panoramic Semantic Segmentation

Panoramic semantic segmentation is pivotal for comprehensive 360° scene understanding in critical applications like autonomous driving and virtual reality. However, progress in this domain is constrained by two key challenges: the severe geometric distortions inherent in panoramic projections and the prohibitive cost of dense annotation. While Unsupervised Domain Adaptation (UDA) from label-rich pinhole-camera datasets offers a viable alternative, many real-world tasks impose a stricter source-free (SFUDA) constraint where source data is inaccessible for privacy or proprietary reasons. This constraint significantly amplifies the core problems of domain shift, leading to unreliable pseudo-labels and dramatic performance degradation, particularly for minority classes. To overcome these limitations, we propose the DAPASS framework. DAPASS introduces two synergistic modules to robustly transfer knowledge without source data. First, our Panoramic Confidence-Guided Denoising (PCGD) module generates high-fidelity, class-balanced pseudo-labels by enforcing perturbation consistency and incorporating neighborhood-level confidence to filter noise. Second, a Contextual Resolution Adversarial Module (CRAM) explicitly addresses scale variance and distortion by adversarially aligning fine-grained details from high-resolution crops with global semantics from low-resolution contexts. DAPASS achieves state-of-the-art performances on outdoor (Cityscapes-to-DensePASS) and indoor (Stanford2D3D) benchmarks, yielding 55.04% (+2.05%) and 70.38% (+1.54%) mIoU, respectively.

Yaowen Chang, Zhen Cao, Xu Zheng +2

Wuhan University · HKUST (Guangzhou) · Wuhan University of Technology

Semantic Segmentation Source-Free Domain Adaptation Unsupervised Domain Adaptation Pseudo-Labeling Domain-Adversarial Learning

Ask PDF

Sep 30, 2026

MC-PanDA++: Simpler, Stronger, and More Robust Domain-Adaptive Panoptic Segmentation

Unsupervised domain adaptation (UDA) reduces the annotation burden in panoptic segmentation by leveraging a cost-effectively labeled source domain (e.g., synthetic) and an unlabeled target domain to bridge the distribution gap. Existing panoptic UDA methods rely on teacher-student consistency learning built upon suboptimal per-pixel segmentation architectures. In contrast, state-of-the-art mask transformers are rarely adopted due to their pronounced vulnerability to confirmation bias in consistency learning, where erroneous teacher predictions are reinforced during training. Our earlier approach, MC-PanDA, mitigates this issue through fine-grained confidence estimation, which suppresses gradients from unreliable masks while sampling informative yet reliable locations for loss computation. However, this method entails a complex multi-stage training and requires careful hyperparameter tuning. This work presents MC-PanDA++, which addresses these limitations by introducing: (i) self-supervised vision encoders that provide a stronger and more robust initialization, further reducing the reliance on human annotations, (ii) per-class, self-adapting mask-wide loss scaling that stabilizes training and enables the usage of a single set of hyperparameters across domains, and (iii) a single-stage training pipeline that decreases overall conceptual complexity. Together, these improvements result in a conceptually simpler, better-performing, and more robust method for domain-adaptive panoptics. Source code: https://github.com/martinovicivan/MC-PanDA

Ivan Martinović, Josip Šarić, Yuki M. Asano +1

Faculty of Electrical Engineering and Computing, University of Zagreb, Unska 3, 10000 Zagreb, Croatia. · Faculty of Computer and Information Science, University of Ljubljana, Večna Pot 113, 1000 Ljubljana, Slovenia. · Fundamental AI Lab, University of Technology Nuremberg, Dr.-Luise-Herzberg-Straße, 90461 Nürnberg, Germany.

Panoptic Segmentation Unsupervised Domain Adaptation Teacher-Student Learning

Ask PDF