Cross-Layer Refinement and Frequency-aware Fusion Network for Multi-Source Remote Sensing Image Classification

Wenhan Xiao; Chuanzheng Gong; Feng Gao; Junyu Dong; Qian Du

eess.IVNov 2, 2026

Cross-Layer Refinement and Frequency-aware Fusion Network for Multi-Source Remote Sensing Image Classification

Authors: Wenhan Xiao, Chuanzheng Gong, Feng Gao, Junyu Dong, Qian Du

Organizations: State Key Laboratory of Physical Oceanography, Ocean University of China, Qingdao 266100, China. · Department of Electrical and Computer Engineering, Mississippi State University, Starkville, MS 39762 USA.

Abstract

Multi-source remote sensing image classification benefits from complementary spectral and structural information. However, identifying discriminative cross-modal features remains challenging. This letter proposes a Cross-Layer Refinement and Frequency-aware Fusion Network (CRFNet) within a lightweight dual-branch architecture. Specifically, a Cross-Layer Refinement Attention (CLRA) module propagates selected attention relationships across successive layers and employs top relation selection to progressively strengthen discriminative spatial dependencies while suppressing irrelevant responses. Furthermore, a Frequency-aware Complementary Fusion (FCF) module combines frequency-wise interactions with cross-modal channel attention in the Fourier domain. The resulting representations are then converted into spatial-domain gates to selectively modulate complementary features. Experiments on the Houston 2013 and Augsburg datasets demonstrate that the proposed CRFNet consistently outperforms several state-of-the-art methods. The codes are publicly available at https://github.com/oucailab/CRFNet

Figures & tables

Fig. 1: Overview of the proposed Cross-Layer Refinement and Frequency-aware Fusion Network (CRFNet) for multi-source remote sensing classification. The dual branches extract features from HSI and SAR/LiDAR data, respectively. Cross-Layer Refinement Attention (CLRA) blocks progressively refine features through cross-layer interactions, while Frequency-aware Complementary Fusion (FCF) modules integrate complementary information across modalities at successive stages.

Fig. 2: Illustration of the Cross-Layer Refinement Attention (CLRA).

Fig. 3: Illustration of the Frequency-aware Complementary Fusion (FCF) module.

Fig. 4: Parameter analysis of CRFNet. (a) Number of CLRA blocks. (b) Ratio in top-

k

selection.

Class	S 2 ENet	DFINet	AsyFF	ExViT	HCT	MACN	RSCNet	CRFNet
Forest	98.10	97.38	97.56	90.04	97.26	97.84	96.07	96.77
Residential	99.08	98.37	99.16	95.44	98.63	98.73	96.40	97.94
Industrial	12.19	61.31	61.38	34.58	36.61	43.94	78.38	56.48
Low plants	91.78	92.63	83.47	90.68	93.54	94.11	94.08	96.62
Allotment	45.12	49.33	42.64	51.82	51.43	52.39	58.32	61.38
Commercial	1.22	3.54	5.58	28.63	7.97	6.80	1.96	3.24

TABLE I: Classification performance of different methods on the Augsburg dataset. The bold and underline denote the best and second results.

Class	S 2 ENet	DFINet	AsyFF	ExViT	HCT	MACN	RSCNet	CRFNet
Healthy grass	81.48	82.91	82.72	82.72	81.10	82.90	82.53	82.81
Stressed grass	84.68	85.15	83.93	85.15	86.09	100.0	79.98	94.83
Synthetic grass	99.01	99.60	100.0	99.21	99.80	99.80	99.60	100.0
Trees	92.14	92.33	91.95	91.95	98.01	98.11	99.15	97.16
Soil	100.0	99.81	100.0	100.0	100.0	100.0	99.23	100.0
Water	100.0	100.0	95.80	98.60	95.10	95.80	98.60	95.80

TABLE II: Classification performance of different methods on the Houston2013 dataset. The bold and underline denote the best and second results.

Fig. 5: Qualitative comparison of the classification maps on the Augsburg dataset, with enlarged views of a selected region. (a) Ground truth. (b) S 2 ENet. (c) DFINet. (d) AsyFF. (e) ExViT. (f) HCT. (g) MACN. (h) RSCNet. (i) Proposed CRFNet.

Metric	S 2 ENet	DFINet	AsyFF	ExViT	HCT	MACN	RSCNet	CRFNet
Params (M)	1.7055	0.8460	2.0987	1.8807	4.7811	2.0199	2.8812	0.3492
FLOPs (G)	0.2065	0.12464	0.1789	0.2897	0.0437	0.1672	0.3346	0.0821

TABLE III: Comparison of model complexity on the Augsburg dataset.

CLRA	FCF	Augsburg	Houston2013
$\times$	$\times$	90.71	89.36
$\times$	$\checkmark$	91.51	92.24
$\checkmark$	$\times$	91.24	92.03
$\checkmark$	$\checkmark$	92.04	93.75

TABLE IV: Ablation experiments. OA (%) is reported.

Remote Sensing Image Understanding

Remote sensing image understanding uses AI to interpret satellite and aerial imagery for objects, land cover, change, hazards, and spatial relationships.

See all 211 papers

A Spatial-Spectral-Frequency Interactive Network for Multimodal Remote Sensing Classification
Oct 6, 2025Hao Liu, Yunhao Gao, Wei Li +3Remote Sensing Image Understanding Frequency-Domain Feature Learning
Department of Information Engineering and Computer Science, University of Trento, 38123 Trento, Italy · School of Information and Electronics, Beijing Institute of Technology, Beijing 100081, China · Beijing Key Laboratory of Fractional Signals and Systems, Beijing Institute of Technology, Beijing 100081, China +3
Semantic-Guided Fusion Network for Multi-Source Remote Sensing Image Classification
Sep 16, 2026Yuwei Zhao, Chuanzheng Gong, Baogui Huan +3Remote Sensing Remote Sensing Image Understanding
State Key Laboratory of Physical Oceanography, Ocean University of China, Qingdao 266100, China · Department of Electrical and Computer Engineering, Mississippi State University, Starkville, MS 39762 USA
Representative Spectral Correlation Network for Multi-source Remote Sensing Image Classification
Apr 30, 2026Chuanzheng Gong, Feng Gao, Junyan Lin +2Remote Sensing Image Understanding Feature Selection
State Key Laboratory of Physical Oceanography, Ocean University of China, Qingdao 266100, China · Department of Computing, The Hong Kong Polytechnic University, Hong Kong, China · Department of Electrical and Computer Engineering, Mississippi State University, Starkville, MS 39762 USA

Oct 6, 2025J/K move · Enter open · S save

A Spatial-Spectral-Frequency Interactive Network for Multimodal Remote Sensing Classification

Hao Liu, Yunhao Gao, Wei Li, Mingyang Zhang+2

Department of Information Engineering and Computer Science, University of Trento, 38123 Trento, Italy · School of Information and Electronics, Beijing Institute of Technology, Beijing 100081, China · Beijing Key Laboratory of Fractional Signals and Systems, Beijing Institute of Technology, Beijing 100081, China+3

Remote Sensing Image Understanding Frequency-Domain Feature Learning Remote Sensing Multimodal Fusion

Open paper AskPDF arXiv

Deep learning-based methods have achieved significant success in remote sensing Earth observation data analysis. Numerous feature fusion techniques address multimodal remote sensing image classification by integrating global and local features. However, these techniques often struggle to extract structural and detail features from heterogeneous and redundant multimodal images. With the goal of introducing frequency domain learning to model key and sparse detail features, this paper introduces the spatial-spectral-frequency interaction network (S

^2

Fin), which integrates pairwise fusion modules across the spatial, spectral, and frequency domains. Specifically, we propose a high-frequency sparse enhancement transformer that employs sparse spatial-spectral attention to optimize the parameters of the high-frequency filter. Subsequently, a two-level spatial-frequency fusion strategy is introduced, comprising an adaptive frequency channel module that fuses low-frequency structures with enhanced high-frequency details, and a high-frequency resonance mask that emphasizes sharp edges via phase similarity. In addition, a spatial-spectral attention fusion module further enhances feature extraction at intermediate layers of the network. Experiments on four benchmark multimodal datasets with limited labeled data demonstrate that S

^2

Fin performs superior classification, outperforming state-of-the-art methods. The code is available at https://github.com/HaoLiu-XDU/SSFin.

Abstract

Remote Sensing Image Understanding

Explore similar work

A Spatial-Spectral-Frequency Interactive Network for Multimodal Remote Sensing Classification

Semantic-Guided Fusion Network for Multi-Source Remote Sensing Image Classification

Representative Spectral Correlation Network for Multi-source Remote Sensing Image Classification