Prior-guided Fusion of Multimodal Features for Change Detection from Optical-SAR Images

Xuanguang Liu; Lei Ding; Yujie Li; Chenguang Dai; Zhenchao Zhang; Mengmeng Li; Ziyi Yang; Yifan Sun

cs.CVApr 7, 2026

Prior-guided Fusion of Multimodal Features for Change Detection from Optical-SAR Images

Authors: Xuanguang Liu, Lei Ding, Yujie Li, Chenguang Dai, Zhenchao Zhang, Mengmeng Li, Ziyi Yang, Yifan Sun,

Organizations: Institute of Geospatial Information, Information Engineering University, Zhengzhou, China · Academy of Digital China (Fujian), Fuzhou University, Fuzhou, China · The School of Electronics and Communication Engineering, Sun Yat-sen University, Shenzhen, China · The Department of Information Engineering and Computer Science, University of Trento, Trento, Italy

Abstract

Multimodal change detection (MMCD) identifies changed areas in multimodal remote sensing data, demonstrating significant application value in land use monitoring and urban sustainable development. However, literature MMCD approaches exhibit limitations in both cross-modal interaction and exploiting modality-specific characteristics. This leads to insufficient modeling of fine-grained change information, thus hindering the precise detection of semantic changes. To address these problems, we propose STSF-Net, a framework designed for MMCD between optical and SAR images. STSF-Net jointly models modality-specific and spatio-temporal common features to enhance change representations. Specifically, modality-specific features are exploited to capture genuine semantic change signals, while spatio-temporal common features are embedded to suppress pseudo-changes caused by differences in imaging mechanisms. Furthermore, we introduce an optical and SAR feature fusion strategy that adaptively adjusts multimodal feature importance based on semantic priors obtained from visual foundation models. Finally, we introduce the novel Delta-SN6 dataset, the first openly-accessible multiclass MMCD benchmark consisting of very-high-resolution fully polarimetric SAR and optical images. Experimental results on Delta-SN6, BRIGHT, and Wuhan datasets demonstrate that our method outperforms the state-of-the-art by 3.21%, 0.87%, and 1.32% in mIoU, respectively.

Figures & tables

Fig. 1: An overview of the proposed OSCD architecture. It contains three core steps: (i) modality-specific feature extraction, (ii) spatio-temporal common feature alignment and (iii) multimodal feature fusion guided by change priors.

Fig. 2: Details of the Modal-specific feature Extractor.

Fig. 3: Extraction of cross-modal common features with spatio-temporal correlations.

Fig. 4: Workflow for fusing modality-specific and common features guided by change priors.

Dataset	Modality	Resolution	Image Size	Temporal Span	Change Category
BRIGHT	SAR-optical	0.3-1 m	1024×1024	2014-2020	Building
Wuhan	SAR-optical	2-10 m	256×256	2015-2022	Building, Road
Delta-SN6	SAR-optical optical-optical	0.5 m	900×900	2007-2019	Building, Road, Water

Table 1: Comparison of Delta-SN6 with existing OSCD datasets

Fig. 5: The Delta-SN6 dataset provides multimodal and multi-temporal remote sensing images along with ground truth labels, designed for the OSCD task.

Method	Reference	Recall	Precision	OA	$F_{1}^{bcd}$	mIoU
DeepLabV3+	ECCV’18	52.80	54.54	87.25	53.66	61.45
SiamCRNN	TGRS’20	48.65	33.49	79.30	39.67	51.27
SiamAttnUNet	ISPRS’21	53.53	58.18	88.12	55.76	62.91
DTCDN	ISPRS’21	58.03	42.90	83.32	49.33	57.30
ChangeOS	RSE’21	42.80	57.42	87.56	49.04	59.63
ICIF-Net	TGRS’22	44.41	58.13	86.08	50.35	57.84

Table 2: Evaluation of OSCD accuracy results (

\%

) on the Wuhan dataset. The best and second-best results are in bold and underlined, respectively.

Fig. 6: Qualitative results provided by different methods in the testing areas (a)-(d) on the Wuhan dataset.

Methods	$F_{1}^{bcd}$	$F_{1}^{clf}$	OA	mIoU	IoU per class
Methods	$F_{1}^{bcd}$	$F_{1}^{clf}$	OA	mIoU	Background	Intact	Damaged	Destroyed
UNet	86.01	62.94	94.60	59.22	95.46	67.35	28.44	45.63
DeepLabV3+	82.61	70.33	93.60	59.37	94.82	63.96	32.19	46.53
SiamCRNN	88.77	68.71	95.42	63.37	96.32	71.45	35.06	50.67
SiamAttnUNet	87.21	70.22	94.99	62.82	95.83	69.45	36.03	49.96
ChangeOS	89.60	71.88	95.84	65.98	96.54	73.85	38.99	54.53

Table 3: Evaluation of building damage accuracy results (

\%

) of different methods on the BRIGHT dataset.

Fig. 7: Qualitative results provided by different methods in the testing areas (a)-(d) on the BRIGHT dataset.

Methods	$F_{1}^{bcd}$	$F_{1}^{clf}$	OA	mIoU	IoU per class
Methods	$F_{1}^{bcd}$	$F_{1}^{clf}$	OA	mIoU	BG	AB	AR	AW	DB	DR	DW
UNet	83.28	26.37	98.53	53.44	98.54	62.10	57.89	77.47	78.09	0.00	0.00
DeepLabV3+	90.83	22.32	98.58	51.88	98.58	64.21	29.04	73.57	78.87	2.95	15.95
SiamCRNN	82.17	54.20	98.36	57.04	98.38	52.83	28.25	70.20	78.83	15.09	55.73
SiamAttnUNet	82.80	88.15	98.54	74.19	98.53	61.81	54.10	74.34	79.37	74.95	76.20
ChangeOS	78.57	86.53	98.10	66.92	98.13	55.83	43.02	67.59	71.69	61.16	71.01

Table 4: Evaluation of OSCD accuracy results (

\%

) on the Delta-SN6 dataset. BG, AB, AR, AW, DB, DR and DW refer to background, added building, added road, added water, disappeared building, disappeared road and disappeared water, respectively.

Fig. 8: Qualitative results provided by different methods in the testing areas (a)-(d) on the Delta-SN6 dataset.

Methods	Proposed Module			BRIGHT Dataset (%)				Delta-SN6 Dataset (%)
Methods	FIM	GSFM	PGFFM	F1 bcd	F1 clf	OA	mIoU	F1 bcd	F1 clf	OA	mIoU
Baseline	$\times$	$\times$	$\times$	88.14	70.87	95.39	63.99	79.83	87.46	98.34	71.42
STSF-Net-V1	$\checkmark$	$\times$	$\times$	88.51	71.81	95.44	64.76	81.63	87.38	98.49	74.45
STSF-Net-V2	$\times$	$\checkmark$	$\times$	88.85	72.21	95.73	65.52	89.75	90.88	99.17	85.87
STSF-Net-V3	$\times$	$\times$	$\checkmark$	88.21	74.54	95.43	66.28	91.82	91.91	99.32	87.45
STSF-Net-V4	$\checkmark$	$\checkmark$	$\times$	88.63	73.32	95.69	65.64	92.40	92.42	99.35	88.21

Table 5: Ablation study results (

\%

) of the proposed modules on BRIGHT and Delta-SN6 datasets.

Fig. 9: Qualitative comparison of the OSCD results obtained using different network modules on the BRIGHT and Delta-SN6 datasets. STSF-Net-V1: baseline+FIM, STSF-Net-V2: baseline+GSFM, STSF-Net-V3: baseline+PGFFM, STSF-Net-V4: baseline+FIM+GSFM, STSF-Net-V5: baseline+FIM+PGFFM, STSF-Net-V6: baseline+GSFM+PGFFM.

Methods	Trainable Params (M)	$F_{1}^{bcd}$	$F_{1}^{clf}$	OA	mIoU	IoU per class
Methods	Trainable Params (M)	$F_{1}^{bcd}$	$F_{1}^{clf}$	OA	mIoU	BG	AB	AR	AW	DB	DR	DW
Siamese SwinTransformer	56.58	91.76	95.05	99.31	87.82	99.31	78.37	76.14	91.26	89.28	90.15	90.24
Siamese SAM2	0.72	90.87	94.02	99.23	85.84	99.24	77.41	72.53	89.17	87.62	86.96	87.95
SAM2 + SwinTransformer	29.01	94.60	95.27	99.54	91.33	99.55	85.68	81.42	92.95	92.77	94.62	92.33

Table 6: Performance comparison of different encoder architectures on the Delta-SN6 dataset.

Fig. 10: Comparison of feature distributions of optical and SAR before and after STCFM processing.

Methods	BRIGHT Dataset				Delta-SN6 Dataset
Methods	$F_{1}^{bcd}$	$F_{1}^{clf}$	OA	mIoU	$F_{1}^{bcd}$	$F_{1}^{clf}$	OA	mIoU
STSF-Net(w/o. PGFFM)	88.63	73.32	95.69	65.64	89.63	93.27	99.12	84.93
STSF-Net(w. PGFFM with SAM)	89.74	73.37	95.95	66.86	92.72	95.09	99.37	88.16
STSF-Net(w. PGFFM with DINOv3)	88.85	73.21	95.90	66.72	93.11	92.65	99.40	88.51
STSF-Net(w. PGFFM with SAM2)	91.71	75.83	96.10	67.91	94.60	95.27	99.54	91.33

Table 7: Performance comparison of different methods on BRIGHT and Delta-SN6 datasets

Fig. 11: Visualized change in feature response intensity before and after PGFFM addition on BRIGHT and Delta-SN6 datasets.

Type	Methods	PGFFM	BRIGHT				Delta-SN6
Type	Methods	PGFFM	$\mathbf{F_{1}^{bcd}}$	$\mathbf{F_{1}^{clf}}$	$\mathbf{OA}$	$\mathbf{mIoU}$	$\mathbf{F_{1}^{bcd}}$	$\mathbf{F_{1}^{clf}}$	$\mathbf{OA}$	$\mathbf{mIoU}$
CNN	SiamCRNN	$\times$	88.77	$68.71$	$95.42$	$63.37$	$89.44$	$92.93$	$99.11$	$83.82$
CNN	SiamCRNN	$\checkmark$	89.26	72.01	95.76	65.57	90.67	94.46	99.25	86.79
Transformer	DamageFormer	$\times$	89.92	$72.22$	95.95	$66.09$	$91.38$	$94.81$	$99.37$	$88.12$
Transformer	DamageFormer	$\checkmark$	89.46	72.71	$95.92$	66.49	93.09	$95.41$	99.41	89.36
Mamba	GSTM-SCD	$\times$	90.51	$73.96$	$95.88$	$66.83$	$91.42$	$95.31$	$99.32$	$87.04$

Table 8: Performance comparison of different methods with and without PGFFM on BRIGHT and Delta-SN6 datasets.

Fig. 12: Visualization of specific features, common features, and the fused change feature on the Delta-SN6 dataset.

Methods	Reference	Params (M)	FLOPs (G)	Inf. time (ms)
UNet	MICCAI’15	31.05	219.45	394.50
DeepLabV3+	ECCV’18	42.40	140.15	112.35
SiamAttnUNet	ISPRS’21	60.95	494.76	345.68
SiamCRNN	TGRS’20	54.37	135.02	118.78
DamageFormer	IGARSS’22	48.16	202.50	216.15
ChangeOS	RSE’21	50.60	117.53	11.7

Table 9: Comparison of computational efficiency for different methods.

Remote Sensing Change Detection

Remote sensing change detection finds and interprets differences across Earth-observation images acquired at different times.

See all 89 papers

Revealing Inter-Channel Modality Discriminability: Channel Selective Interaction Fusion Network for Optical-SAR Object Detection
Oct 20, 2026Haojie Li, Yuanfei Huang, Lingfei Song +2Object Detection Multimodal Object Detection
School of Artificial Intelligence, Beijing Normal University · Beijing Key Laboratory of Artificial Intelligence for Education · Engineering Research Center of Intelligent Technology and Educational Application, Ministry of Education
TAR: Text Semantic Assisted Cross-modal Image Registration Framework for Optical and SAR Images
May 12, 2026Zhuoyu Cai, Dou Quan, Ning Huyan +3Remote Sensing Synthetic Aperture Radar
Key Laboratory of Intelligent Perception and Image Understanding of Ministry of Education of China, School of Artificial Intelligence, Xidian University, Xi’an 710071, China
M4-SAR: A Multi-Resolution, Multi-Polarization, Multi-Scene, Multi-Source Dataset and Benchmark for optical-SAR Object Detection
May 16, 2025Chao Wang, Wei Lu, Xiang Li +2Remote Sensing Object Detection
Nanjing University of Science and Technology, Nanjing 210094, China · Anhui University, Hefei 230601, China · Nankai University, Tianjin 300350, China

Oct 20, 2026J/K move · Enter open · S save

Revealing Inter-Channel Modality Discriminability: Channel Selective Interaction Fusion Network for Optical-SAR Object Detection

Haojie Li, Yuanfei Huang, Lingfei Song, Jia Li+1

School of Artificial Intelligence, Beijing Normal University · Beijing Key Laboratory of Artificial Intelligence for Education · Engineering Research Center of Intelligent Technology and Educational Application, Ministry of Education

Object Detection Multimodal Object Detection Multimodal Fusion Synthetic Aperture Radar Cross-Modal Attention

Open paper AskPDF arXiv

Synthetic aperture radar (SAR) and optical imagery provide complementary structural and appearance information for remote sensing object detection. However, substantial cross-modal discrepancies and indiscriminate channel interaction inevitably introduce redundant or conflicting features. Since different feature channels encode heterogeneous semantic information and contribute unequally to cross-modal fusion, selectively highlighting task-relevant channels while suppressing redundant ones is essential. To address this issue, we propose the Channel Selective Interaction Fusion Network (CSIF-Net), which performs selective channel-level cross-modal interaction. CSIF-Net first employs a frequency decomposition module to construct frequency-aware SAR inputs, facilitating the separation of structural information from noise-related responses. It then models cross-modal channel relationships through a cross-attention channel selection module and dynamically partitions the features into shared and modality-specific components. Finally, a dual-branch fusion module performs cross-modal interaction and channel realignment for shared features while independently enhancing SAR-specific features, thereby exploiting complementary information and suppressing modality interference. Extensive experiments on the OGSOD 1.0 and M4-SAR datasets demonstrate that CSIF-Net achieves state-of-the-art performance in both horizontal and oriented object detection, surpassing the existing methods by 4.01% and 3.18% gains, respectively.

Abstract

Remote Sensing Change Detection

Explore similar work

Revealing Inter-Channel Modality Discriminability: Channel Selective Interaction Fusion Network for Optical-SAR Object Detection

TAR: Text Semantic Assisted Cross-modal Image Registration Framework for Optical and SAR Images

M4-SAR: A Multi-Resolution, Multi-Polarization, Multi-Scene, Multi-Source Dataset and Benchmark for optical-SAR Object Detection