CoFusion: Multispectral and Hyperspectral Image Fusion via Spectral Coordinate Attention

Baisong Li

cs.CVApr 12, 2026

CoFusion: Multispectral and Hyperspectral Image Fusion via Spectral Coordinate Attention

Authors: Baisong Li

Organizations: School of Integrated Circuits, Tsinghua University, Beijing 100084, China

Abstract

Multispectral and Hyperspectral Image Fusion (MHIF) aims to reconstruct high-resolution images by integrating low-resolution hyperspectral images (LRHSI) and high-resolution multispectral images (HRMSI). However, existing methods face limitations in modeling cross-scale interactions and spatial-spectral collaboration, making it difficult to achieve an optimal trade-off between spatial detail enhancement and spectral fidelity. To address this challenge, we propose CoFusion: a unified spatial-spectral collaborative fusion framework that explicitly models cross-scale and cross-modal dependencies. Specifically, a Multi-Scale Generator (MSG) is designed to construct a three-level pyramidal architecture, enabling the effective integration of global semantics and local details. Within each scale, a dual-branch strategy is employed: the Spatial Coordinate-Aware Mixing module (SpaCAM) is utilized to capture multi-scale spatial contexts, while the Spectral Coordinate-Aware Mixing module (SpeCAM) enhances spectral representations through frequency decomposition and coordinate mixing. Furthermore, we introduce the Spatial-Spectral Cross-Fusion Module (SSCFM) to perform dynamic cross-modal alignment and complementary feature fusion. Extensive experiments on multiple benchmark datasets demonstrate that CoFusion consistently outperforms state-of-the-art methods, achieving superior performance in both spatial reconstruction and spectral consistency.

Figures & tables

Figure 1: Overall architecture of the proposed CoFusion framework. The framework features a three-level pyramidal pipeline driven by the Multi-Scale Generator (MSG) and employs dual-path coordinate-aware modules for collaborative spatial-spectral reconstruction.

Figure 2: Detailed structure of SpaCAM. It adaptively refines spatial details by leveraging depthwise separable convolutions with multiple dilation rates and the Gated Spatial Block (GSB) mechanism.

Figure 3: Schematic of SpeCAM and its core Spectral Coordinate Mixing (SCM) operator. This module employs frequency-domain decoupling and a coordinate-based traversal strategy to capture deep spectral correlations.

Figure 4: Architecture of SSCFM. It utilizes spatial and spectral branches to achieve fine-grained inter-modal interaction and feature alignment.

Method	Scale	Chikusei				CAVE				PaviaU
Method	Scale	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	ERGAS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	ERGAS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	SAM $\downarrow$	ERGAS $\downarrow$
HSRnet ( Hu et al., 2022b )	$\times 2$	51.5324	0.9951	2.0982	1.5893	50.5410	0.9926	2.4608	2.0012	39.4947	0.9662	3.9012	2.4531
Fusformer ( Hu et al., 2022a )	$\times 2$	52.3104	0.9954	2.0596	1.5698	51.3352	0.9935	2.3294	1.9692	40.3442	0.9703	3.5187	2.2514
PSRT ( Deng et al., 2023 )	$\times 2$	52.3821	0.9957	2.0091	1.4692	51.4128	0.9941	2.1795	1.8891	40.4956	0.9766	2.3996	2.0812
U2Net ( Peng et al., 2023 )	$\times 2$	52.4657	0.9961	2.0003	1.4098	51.4816	0.9948	2.1297	1.8496	40.7347	0.9789	2.3092	1.5398
3DT-Net ( Ma et al., 2023 )	$\times 2$	52.4155	0.9974	1.9498	1.4195	51.5043	0.9958	2.0496	1.8294	39.7224	0.9784	2.4692	1.7896

Table 1: Quantitative comparison of various methods on the Chikusei, CAVE, and PaviaU datasets across

\times 2

\times 4

, and

\times 8

scaling factors. Bold indicates the best performance, while underline denotes the second-best.

Figure 5: Pseudo-color images with spectral bands 67-15-13 (first patch) and 64-58-16 (third patch) as RGB of $\times$ 4 fusion results, and corresponding SAM error maps produced by all methods on the Chikusei dataset.

Figure 6: Pseudo-color images with spectral bands 20-10-3 ( balloons ) and 6-25-11 ( Chart and Stuffed Toy ) as RGB of $\times$ 4 fusion results, and corresponding SAM error maps produced by all methods on the CAVE dataset.

Figure 7: Pseudo-color images with spectral bands 57-27-9 as RGB of $\times$ 4 fusion results, and corresponding SAM error maps produced by all methods on the PaviaU dataset.

Method	Chikusei			CAVE			PaviaU
Method	$D_{\lambda}\downarrow$	$D_{s}\downarrow$	$QNR\uparrow$	$D_{\lambda}\downarrow$	$D_{s}\downarrow$	$QNR\uparrow$	$D_{\lambda}\downarrow$	$D_{s}\downarrow$	$QNR\uparrow$
HSRnet ( Hu et al., 2022b )	0.0842	0.0915	0.8321	0.0921	0.1045	0.8142	0.1124	0.1256	0.7785
Fusformer ( Hu et al., 2022a )	0.0795	0.0882	0.8456	0.0884	0.0998	0.8265	0.1085	0.1198	0.7892
PSRT ( Deng et al., 2023 )	0.0768	0.0854	0.8532	0.0852	0.0965	0.8348	0.1042	0.1145	0.7965
U2Net ( Peng et al., 2023 )	0.0685	0.0792	0.8698	0.0765	0.0885	0.8512	0.0945	0.1052	0.8124
3DT-Net ( Ma et al., 2023 )	0.0712	0.0765	0.8645	0.0798	0.0862	0.8465	0.0982	0.1025	0.8056

Table 2: Quantitative evaluation on real-world benchmarks (Chikusei, CAVE, and PaviaU) under

\times 4

scaling factors. The bold and underlined results represent the best and second-best performance, respectively.

Figure 8: Spectral reconstruction error analysis for three datasets. The subplots from left to right display pixel-level spectral response curves at specific locations: (a) region (79, 63) of the Chikusei dataset; (b) pixel (98, 128) in the “ Chart and Stuffed Toy ” scene of the CAVE dataset; and (c) pixel (64, 72) of the PaviaU dataset. The results show that the proposed method (red line) maintains minimal deviation from the ground truth (GT) across the entire spectral range, validating its superior spectral fidelity.

Figure 9: Model efficiency comparison for

\times 4

fusion on the PaviaU dataset. The bar chart compares the parameter count (in millions, M) and inference latency (in milliseconds, ms, measured on an Intel Xeon CPU). Note that both axes utilize a logarithmic scale to accommodate the wide range of values.

Variants	PSNR (dB) $\uparrow$	SSIM $\uparrow$	Params (M) $\downarrow$	Latency (ms) $\downarrow$
w/o SpaCAM	37.1245	0.9612	3.58	4.12
w/o SpeCAM	37.5862	0.9694	4.15	4.85
w/o SSCFM	37.9518	0.9752	4.62	5.28
Full Model	38.3195	0.9824	4.91	5.65

Table 3: Ablation study of CoFusion modules on the PaviaU dataset for

\times 4

fusion task. The bold and underlined results represent the best and second-best performance, respectively.

Explore similar work

Dual-Domain Manifold Modeling for Hyperspectral Image Fusion
Jul 28, 2026Chengxin Xie, Qiya Song, Yangbangyan Jiang +2Hyperspectral Imaging Manifold Learning
College of Information Science and Engineering, Hunan Normal University, Changsha, China · Institute of Information Engineering, Chinese Academy of Sciences,Beijing, 100000, Beijing, China · School of Artificial Intelligence and Robotics, Hunan University, Changsha, China
Unregistered Spectral Image Fusion: Unmixing, Adversarial Learning, and Recoverability
Mar 23, 2026Jiahui Song, Sagar Shrestha, Xiao FuMulti-Source Image Fusion Hyperspectral Image Super-Resolution
The authors are with the School of Electrical Engineering and Computer Science, Oregon State University, Corvallis, OR 97331, United States.
ASSR-Net: Anisotropic Structure-Aware and Spectrally Recalibrated Network for Hyperspectral Image Fusion
Apr 7, 2026Qiya Song, Hongzhi Zhou, Lishan Tan +2Multi-Source Image Fusion Hyperspectral Imaging
School of Information Science and Engineering, Hunan Normal University, Changsha, Hunan 410081, China. · the School of Robotics, Hunan University, Changsha, Hunan 410082, China.

Jul 28, 2026J/K move · Enter open · S save

Dual-Domain Manifold Modeling for Hyperspectral Image Fusion

Chengxin Xie, Qiya Song, Yangbangyan Jiang, Renwei Dian+1

College of Information Science and Engineering, Hunan Normal University, Changsha, China · Institute of Information Engineering, Chinese Academy of Sciences,Beijing, 100000, Beijing, China · School of Artificial Intelligence and Robotics, Hunan University, Changsha, China

Hyperspectral Imaging Manifold Learning

Open paper AskPDF arXiv

Achieving a coherent integration of spectral richness and spatial fidelity remains a central objective in hyperspectral image fusion. However, existing hyperspectral image fusion methods struggle to effectively model geometric constraints. In the spatial domain, weak spatial-spectral interaction limits geometry-aware feature learning and suppresses high-frequency structural information, resulting in low-frequency bias and structural degradation. In the spectral domain, local manifold structures induced by spectral similarity are insufficiently exploited, limiting intrinsic pixel relationship modeling and fine-grained spectral reconstruction. To address these challenges, we propose a dual-domain manifold modeling (DDMM) framework. Specifically, we introduce a Topology-Aware Transformer (TPFormer) that combines global attention with neighborhood propagation, jointly modeling spatial topology and pixel-level feature manifold relationships to capture intrinsic spatial-spectral structures and improve topology-aware representation learning. Furthermore, a Frequency-Decoupled Spatial-Spectral Collaborative Fusion (FDSCF) module is devised, in which features are projected into the frequency domain via the discrete cosine transform and explicitly decoupled into low- and high-frequency components. Guided by a low-rank structural prior and spectral-driven spatial enhancement, FDSCF selectively enhances geometry-aware high-frequency features, strengthening spatia-spectral coupling and recovering sharper edges and finer textures. Extensive experiments on multiple benchmark datasets demonstrate that DDMM achieves superior overall performance over SoTA methods in terms of spatial structure preservation and spectral reconstruction.

Abstract

Explore similar work

Dual-Domain Manifold Modeling for Hyperspectral Image Fusion

Unregistered Spectral Image Fusion: Unmixing, Adversarial Learning, and Recoverability

ASSR-Net: Anisotropic Structure-Aware and Spectrally Recalibrated Network for Hyperspectral Image Fusion