ASCoR: Asymmetric Semantic-Spatial Learning for Remote Sensing Domain Generalization

Zitong Wang; Shuchang Lyu; Meng Li; Qiang Wang; Guangliang Cheng; Zhenwei Shi

cs.CVNov 2, 2026

ASCoR: Asymmetric Semantic-Spatial Learning for Remote Sensing Domain Generalization

Authors: Zitong Wang, Shuchang Lyu, Meng Li, Qiang Wang, Guangliang Cheng, Zhenwei Shi

Organizations: Beihang University, Xueyuan Road No.37, Haidian district, Beijing, 100191, , China · University of Science and Technology Beijing, Xueyuan Road No.30, Haidian district, Beijing, 100083, , China · Guangdong Electronic Information Engineering Research Institute, University of Electronic Science and Technology of China, Zongbu 2nd Road No. 17, Dongguan, 523808, , China · University of Liverpool, Foundation Building, Brownlow Hill,, Liverpool, L693BX, , UK

Abstract

Source-only domain generalization (DG) for remote sensing semantic segmentation seeks reliable predictions in unseen domains without target-domain data during training. Although vision foundation models offer transferable knowledge, category recognition remains sensitive to source-specific visual cues, while task-specific adaptation may weaken pretrained transferability. We propose ASCoR, an asymmetric semantic-spatial framework built on SAM3. Its text-conditioned semantic branch uses class-name prompts to provide explicit category guidance. Probe-Once Dynamic Adaptive LoRA distributes a fixed adaptation budget across encoder blocks to limit source-specific specialization. A complementary task-specific dense branch learns multi-scale spatial representations using a separate frozen SAM3 image encoder. A lightweight reliability router integrates the two branches through bounded spatial correction based on local prediction reliability. All components are trained using source-domain supervision only. Experiments on the ISPRS, LoveDA, and OpenEarthMap benchmarks cover nine cross-domain transfers. ASCoR achieves a state-of-the-art mean IoU of 68.61% across six ISPRS transfers, outperforming the previous best method by 2.69 percentage points. It also achieves the best results on all three LoveDA/OpenEarthMap transfers, with 59.07%, 47.95%, and 49.38% mIoU on Rural-to-Urban, Urban-to-Rural, and Urban-to-OpenEarthMap, respectively. Ablation studies, feature visualizations, and spatial reliability analyses support the effectiveness and complementarity of the proposed components. Code: https://github.com/WangZitong42/ASCoR.

Figures & tables

Figure 1: Motivation and paradigm comparison for source-only remote sensing domain generalization. (a) Limitations of representative visual-driven DG approaches in transferring category semantics and exploiting task-specific dense representations. (b) ASCoR combines text-conditioned semantic guidance with complementary dense spatial learning through an asymmetric SAM3 framework and reliability-guided correction. Earth image credit: NASA/Apollo 17, via Wikimedia Commons (public domain).

Figure 2: Architecture and three-stage training of ASCoR. TCSB combines category prompts with Probe-Once Dynamic Adaptive LoRA. TSDB learns complementary spatial representations through trainable adaptation, structure, and decoding modules upon frozen pretrained SAM3 weights. After both branches are frozen, the reliability router learns bounded spatial correction using source-domain supervision.

Domain shift	Source domain	Target domain	Task	Train	Eval.	Image size	Classes
Geographic	Potsdam IRRG	Vaihingen IRRG	P(i)2V	3456	398	$512\!\times\!512$	6
Geographic	Vaihingen IRRG	Potsdam IRRG	V2P(i)	344	2016	$512\!\times\!512$	6
Spectral	Potsdam RGB	Potsdam IRRG	P(r)2P(i)	3456	2016	$512\!\times\!512$	6
Spectral	Potsdam IRRG	Potsdam RGB	P(i)2P(r)	3456	2016	$512\!\times\!512$	6
Geographic + spectral	Potsdam RGB	Vaihingen IRRG	P(r)2V	3456	398	$512\!\times\!512$	6
Geographic + spectral	Vaihingen IRRG	Potsdam RGB	V2P(r)	344	2016	$512\!\times\!512$	6

Table 1: Datasets and protocols for the nine source-only DG transfers.

Figure 3: Overall performance comparison across nine source-only DG transfers on ISPRS, LoveDA, and OpenEarthMap. Each axis is normalized by the highest mIoU among the compared methods for that task; numerical annotations report the original mIoU (%).

Method	Backbone	Task	Surf	Bldg	Vegt	Tree	Car	Clut	mIoU	Task	Surf	Bldg	Vegt	Tree	Car	Clut	mIoU
SAM3 (baseline)	SAM3	P(i)2V	9.47	83.46	20.32	48.36	60.25	2.12	37.33	V2P(i)	17.17	70.15	9.37	35.27	62.51	4.51	33.17
DeepLabV3+	R101		9.15	60.94	17.60	52.28	7.61	2.54	25.02		29.32	39.45	29.17	24.92	37.42	0.00	26.72
SegFormer	MiT-B5		60.58	74.72	27.68	63.69	33.38	7.56	44.60		66.01	70.94	48.53	13.19	65.98	3.00	44.61
DAFormer	MiT-B5		61.48	75.53	42.87	66.57	34.52	9.99	48.49		62.22	70.86	43.09	13.86	64.31	6.95	43.55
HRDA	MiT-B5		64.14	76.65	42.06	65.11	28.72	13.52	48.37		68.64	77.24	46.33	11.05	64.12	6.48	45.64
CrossEarth	ViT-L		83.60	91.70	63.80	70.20	59.50	36.30	67.50		77.10	80.80	61.30	38.90	82.90	8.40	58.20

Table 2: Per-class IoU and mIoU (%) on six ISPRS transfers. Bold and underlined values indicate the best and second-best results. Surf: impervious surface; Bldg: building; Vegt: low vegetation; Clut: clutter/background. SAM3 denotes the zero-shot baseline.

Method	Bkgd	Bldg	Rd	Wtr	Barr	Frst	Agri	mIoU (%)
Performance comparison: LoveDA Rural $\rightarrow$ LoveDA Urban
SAM3 (baseline)	17.87	53.01	55.71	58.74	34.85	50.69	43.31	44.88
DeepLabV3+ R101	34.56	48.35	38.70	56.61	34.02	36.27	45.92	42.06
SegFormer-B5	34.06	52.63	54.30	71.79	43.53	38.81	52.77	49.70
DAFormer	35.49	52.74	51.51	73.10	45.03	47.13	53.20	51.17
HRDA	38.80	51.19	48.06	72.47	46.01	34.60	56.65	49.68

Table 3: Per-class IoU and mIoU (%) on LoveDA and OpenEarthMap. Bold and underlined values rank first and second within each transfer. Bkgd: background; Bldg: building; Rd: road; Wtr: water; Barr: barren; Frst: forest; Agri: agriculture.

TCSB	AR	TSDB	Router	U2R	P(r)2P(i)	P(r)2V	Avg.
–	–	–	–	27.94	33.17	37.33	32.81
$\checkmark$	–	–	–	45.65	76.78	65.10	62.51
$\checkmark$	$\checkmark$	–	–	46.88	76.88	64.69	62.82
–	–	$\checkmark$	–	35.01	69.98	51.74	52.24
$\checkmark$	$\checkmark$	$\checkmark$	–	47.45	77.07	65.37	63.30
$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	47.95	77.24	65.98	63.72

Table 4: Component ablation across three representative transfers. Values are mIoU (%).

TCSB	AR	TSDB	Router	Bkgd	Bldg	Road	Water	Barren	Forest	Agri	mIoU
–	–	–	–	2.27	36.29	43.41	53.12	10.68	13.68	36.10	27.94
$\checkmark$	–	–	–	54.04	60.53	49.46	63.86	11.05	24.43	56.16	45.65
$\checkmark$	$\checkmark$	–	–	49.53	55.23	48.57	66.02	17.71	35.76	55.35	46.88
–	–	$\checkmark$	–	52.06	46.33	36.76	61.61	12.91	7.38	28.02	35.01
$\checkmark$	$\checkmark$	$\checkmark$	–	51.50	56.46	49.07	66.41	17.53	35.91	55.28	47.45
$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$	53.41	57.75	49.62	66.85	17.38	35.82	54.79	47.95

Table 5: Class-wise component ablation on LoveDA Urban

\rightarrow

Rural. Values are IoU and mIoU (%).

$\mathcal{L}_{\mathrm{fuse}}$	$\mathcal{L}_{\mathrm{util}}$	$\mathcal{L}_{\mathrm{cons}}$	mIoU (%)	Gain (pp)
$\checkmark$	–	–	47.93	–
$\checkmark$	$\checkmark$	–	47.94	+0.01
$\checkmark$	$\checkmark$	$\checkmark$	47.95	+0.02

Table 6: Ablation of the reliability-router objective on LoveDA Urban

\rightarrow

Rural. Both branches and all inference settings remain fixed.

Figure 4: Qualitative comparison on the R2U, U2R, and U2OEM transfers. Columns show the input image, ground truth (GT), zero-shot SAM3, uniform fixed-rank LoRA, and ASCoR, illustrating segmentation differences under urban–rural and cross-dataset shifts.

Model/component	Total	Trainable	Ratio	Active
	(M)	(M)	(%)	(M)
TCSB	853.09	12.583	1.475	3.146
TSDB	868.22	27.715	3.192	27.715
Complete framework	1721.32	40.302	2.341	30.865

Table 7: Parameter analysis. “Trainable” counts physically allocated task-specific parameters, following the conventional PEFT reporting protocol. “Active” counts parameters participating in the final forward computation.

Fusion strategy	$P(r)\to{}P(i)$	$P(r)\to{}V$	Average	Overhead
Fixed $w_{B}=0.1$	525.51	522.33	523.92	–
Reliability-guided	527.02	523.83	525.42	+0.29%

Table 8: Inference latency of fixed and learned dual-branch fusion. The reported time includes both branches and the corresponding fusion operation.

Figure 5: Qualitative comparison across the six directed ISPRS transfers among Potsdam IRRG, Potsdam RGB, and Vaihingen IRRG. Columns show the input image, ground truth (GT), DeepLabV3+, SegFormer, HRDA, DAFormer, CrossEarth, EarthAdapter, and ASCoR, covering geographic, spectral, and combined domain shifts.

Figure 6: Feature responses of pretrained SAM3 and ASCoR on $R\to{}U$ at blocks 3, 15, 23, and 31. The bottom row shows their differences; warmer colors indicate stronger responses.

Figure 8: UMAP of TCSB (left) and TSDB (right) on $P(r)\to{}V$ . Colors indicate semantic classes; circles and triangles denote source and target samples, respectively.

Domain Generalization for Semantic Segmentation

See all 21 papers

CrossEarth-SAR: A SAR-Centric and Billion-Scale Geospatial Foundation Model for Domain Generalizable Semantic Segmentation
Mar 12, 2026Ziqi Ye, Ziyang Gong, Ning Liao +10Domain Generalization for Semantic Segmentation Remote Sensing Image Segmentation
Fudan University · Shanghai Innovation Institute · Shanghai Jiao Tong University +7
OSMDA: OpenStreetMap-based Domain Adaptation for Remote Sensing VLMs
Mar 12, 2026Stefan Maria Ailuro, Mario Markov, Mohammad Mahdi +3Remote Sensing VLMs VLM Adaptation
INSAIT, Sofia University "St. Kliment Ohridski" · INSAIT, Sofia University “St. Kliment Ohridski”
LASA: Language-and-Source-Anchored Alignment for Domain Generalized Semantic Segmentation
Aug 9, 2026Jinhong Zhu, Weiqi Yan, Shengchuan Zhang +1Domain Generalization Domain Generalization for Semantic Segmentation
Xiamen University · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China · Xiamen, Fujian, China

Mar 12, 2026J/K move · Enter open · S save

CrossEarth-SAR: A SAR-Centric and Billion-Scale Geospatial Foundation Model for Domain Generalizable Semantic Segmentation

Ziqi Ye, Ziyang Gong, Ning Liao, Xiaoxing Hu+9

Fudan University · Shanghai Innovation Institute · Shanghai Jiao Tong University+7

Domain Generalization for Semantic Segmentation Remote Sensing Image Segmentation Geospatial Foundation Models Domain Generalization Remote Sensing Benchmarks Mixture-of-Experts Models

Open paper AskPDF arXiv

Synthetic Aperture Radar (SAR) enables global, all-weather earth observation. However, owing to diverse imaging mechanisms, domain shifts across sensors and regions severely hinder its semantic generalization. To address this, we present CrossEarth-SAR, the first billion-scale SAR vision foundation model built upon a novel physics-guided sparse mixture-of-experts (MoE) architecture incorporating physical descriptors, explicitly designed for cross-domain semantic segmentation. To facilitate large-scale pre-training, we develop CrossEarth-SAR-200K, a weakly and fully supervised dataset that unifies public and private SAR imagery. We also introduce a benchmark suite comprising 22 sub-benchmarks across 8 distinct domain gaps, establishing the first unified standard for domain generalization semantic segmentation on SAR imagery. Extensive experiments demonstrate that CrossEarth-SAR achieves state-of-the-art results on 20 benchmarks, surpassing previous methods by over 10% mIoU on some benchmarks under multi-gap transfer. All code, benchmark and datasets will be publicly available.

Abstract

Domain Generalization for Semantic Segmentation

Explore similar work

CrossEarth-SAR: A SAR-Centric and Billion-Scale Geospatial Foundation Model for Domain Generalizable Semantic Segmentation

OSMDA: OpenStreetMap-based Domain Adaptation for Remote Sensing VLMs

LASA: Language-and-Source-Anchored Alignment for Domain Generalized Semantic Segmentation