Semantic-Guided Fusion Network for Multi-Source Remote Sensing Image Classification

Yuwei Zhao; Chuanzheng Gong; Baogui Huan; Feng Gao; Junyu Dong; Qian Du

eess.IVSep 16, 2026

Semantic-Guided Fusion Network for Multi-Source Remote Sensing Image Classification

Authors: Yuwei Zhao, Chuanzheng Gong, Baogui Huan, Feng Gao, Junyu Dong, Qian Du

Organizations: State Key Laboratory of Physical Oceanography, Ocean University of China, Qingdao 266100, China · Department of Electrical and Computer Engineering, Mississippi State University, Starkville, MS 39762 USA

Abstract

Multi-source remote sensing image classification has attracted increasing attention due to the complementary spectral, structural, and geometric information. However, existing methods still suffer from two limitations: insufficient semantic contextual modeling and unreliable feature fusion caused by slight spatial misalignment. To address these issues, we propose a Semantic-Guided Fusion Network (SGFNet) for multi-source remote sensing image classification. Specifically, the Semantic Mixing Convolution Block (SMCB) is designed to dynamically generate semantic-aware convolution kernels according to contextual relationships among feature representations. In addition, the Frequency Modulated Fusion Block (FMFB) is introduced to perform cross-modal interaction in the frequency domain, which effectively alleviates the influence of slight spatial misalignment and improves complementary information fusion. Extensive experiments conducted on the Augsburg and Houston 2018 datasets demonstrate that the proposed SGFNet consistently outperforms several state-of-the-art methods. The codes are publicly available at https://github.com/oucailab/SGFNet .

Figures & tables

Fig. 1: Overview of the Semantic-Guided Fusion Network (SGFNet). It adopts a dual-branch hierarchical architecture composed of multiple stacked Semantic-Guided Fusion Modules (SGFMs). Within each SGFM, two Semantic Mixing Convolution Blocks (SMCB) are introduced to model semantic-aware contextual representations. Frequency Modulated Fusion Block (FMFB) is inserted between the two branches.

Fig. 2: Illustration of the Semantic Mixing Convolution Block (SMCB).

Fig. 3: Illustration of the Frequency Modulated Fusion Block (FMFB).

Fig. 4: Classification results of different methods on the Houston2018 dataset.

Class	S 2 ENet	DFINet	ExViT	DFFNet	MSFM	GCCQTNet	SGFNet
Forest	98.10	97.38	90.04	96.17	97.17	95.26	97.18
Residential area	99.08	98.37	95.44	96.64	98.15	96.80	98.28
Industrial area	12.19	61.31	34.58	37.70	50.26	68.20	61.41
Low plants	91.78	92.63	90.68	94.21	95.52	91.35	95.48
Allotment	45.12	49.33	51.82	53.54	53.35	41.68	65.01
Commercial area	1.22	3.54	28.63	11.36	2.63	13.97	8.55

TABLE I: Experimental results on the Augsburg dataset. The bold and underline denote the best and second results.

Fig. 5: Parameter analysis. (a) The relationship between OA and the number of SGFM. (b) The relationship between OA and the number of principal components after PCA for HSI.

$K$	5	7	9	11
Augsburg	91.64	92.58	92.50	92.02
Houston2018	93.26	94.10	94.03	93.47

TABLE II: OA (%) of SGFNet under different dynamic kernel sizes

K

in SMCB.

Class	S 2 ENet	DFINet	ExViT	DFFNet	MSFM	GCCQTNet	SGFNet
Healthy grass	99.15	94.29	93.34	92.63	90.38	91.71	92.07
Stressed grass	84.35	92.71	93.44	92.44	94.04	94.22	93.18
Artificial turf	100.0	100.0	100.0	100.0	100.0	100.0	100.0
Evergreen trees	98.13	98.92	95.87	99.38	99.34	98.58	99.76
Deciduous trees	86.39	97.77	96.85	99.16	99.11	98.71	99.94
Bare earth	99.33	100.0	100.0	99.96	100.0	99.96	100.0

TABLE III: Experimental results on the Houston2018 dataset. The bold and underline denote the best and second best results.

Model Variant	Augsburg	Houston2018
Baseline	88.54	89.65
w/o SMCB	89.88	91.01
w/o FMFB	91.29	92.37
SGFNet (Full model)	92.38	94.17

TABLE IV: Ablation study of the proposed SGFNet.

Shift (pixels)	0	1	2	3	5
DFFNet	91.21	90.98	90.09	89.36	88.61
MSFMamba	92.38	92.14	92.07	91.81	90.35
SGFNet (Ours)	94.17	94.03	93.96	93.64	92.77

TABLE V: OA (%) under spatial shifts on the Houston2018 dataset.

Explore similar work

A Spatial-Spectral-Frequency Interactive Network for Multimodal Remote Sensing Classification
Oct 6, 2025Hao Liu, Yunhao Gao, Wei Li +3Remote Sensing Image Understanding Frequency-Domain Feature Learning
Department of Information Engineering and Computer Science, University of Trento, 38123 Trento, Italy · School of Information and Electronics, Beijing Institute of Technology, Beijing 100081, China · Beijing Key Laboratory of Fractional Signals and Systems, Beijing Institute of Technology, Beijing 100081, China +3
Representative Spectral Correlation Network for Multi-source Remote Sensing Image Classification
Apr 30, 2026Chuanzheng Gong, Feng Gao, Junyan Lin +2Remote Sensing Image Understanding Feature Selection
State Key Laboratory of Physical Oceanography, Ocean University of China, Qingdao 266100, China · Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China · Department of Electrical and Computer Engineering, Mississippi State University, Starkville, MS 39762 USA
Parameter-Efficient Modality-Balanced Symmetric Fusion for Multimodal Remote Sensing Semantic Segmentation
Mar 18, 2026Haocheng Li, Juepeng Zheng, Shuangxi Miao +4Remote Sensing Image Segmentation Multimodal Fusion
the College of Land Science and Technology, China Agricultural University, and also with the Key Laboratory of Remote Sensing for Agri-Hazards, Ministry of Agriculture and Rural Affairs, Beijing 100083, China · the School of Artificial Intelligence, Sun Yat-Sen University, Zhuhai, China · Henan Polytechnic University, and also with the Key Laboratory of Spatio-Temporal Information and Ecological Restoration of Mines, Ministry of Natural Resources of the People’s Republic of China, Jiaozuo 454003, China +4

Oct 6, 2025J/K move · Enter open · S save

A Spatial-Spectral-Frequency Interactive Network for Multimodal Remote Sensing Classification

Hao Liu, Yunhao Gao, Wei Li, Mingyang Zhang+2

Department of Information Engineering and Computer Science, University of Trento, 38123 Trento, Italy · School of Information and Electronics, Beijing Institute of Technology, Beijing 100081, China · Beijing Key Laboratory of Fractional Signals and Systems, Beijing Institute of Technology, Beijing 100081, China+3

Remote Sensing Image Understanding Frequency-Domain Feature Learning Remote Sensing Multimodal Fusion

Open paper AskPDF arXiv

Deep learning-based methods have achieved significant success in remote sensing Earth observation data analysis. Numerous feature fusion techniques address multimodal remote sensing image classification by integrating global and local features. However, these techniques often struggle to extract structural and detail features from heterogeneous and redundant multimodal images. With the goal of introducing frequency domain learning to model key and sparse detail features, this paper introduces the spatial-spectral-frequency interaction network (S

^2

Fin), which integrates pairwise fusion modules across the spatial, spectral, and frequency domains. Specifically, we propose a high-frequency sparse enhancement transformer that employs sparse spatial-spectral attention to optimize the parameters of the high-frequency filter. Subsequently, a two-level spatial-frequency fusion strategy is introduced, comprising an adaptive frequency channel module that fuses low-frequency structures with enhanced high-frequency details, and a high-frequency resonance mask that emphasizes sharp edges via phase similarity. In addition, a spatial-spectral attention fusion module further enhances feature extraction at intermediate layers of the network. Experiments on four benchmark multimodal datasets with limited labeled data demonstrate that S

^2

Fin performs superior classification, outperforming state-of-the-art methods. The code is available at https://github.com/HaoLiu-XDU/SSFin.

Abstract

Explore similar work

A Spatial-Spectral-Frequency Interactive Network for Multimodal Remote Sensing Classification

Representative Spectral Correlation Network for Multi-source Remote Sensing Image Classification

Parameter-Efficient Modality-Balanced Symmetric Fusion for Multimodal Remote Sensing Semantic Segmentation