CAWM-Mamba: A unified model for infrared-visible image fusion and compound adverse weather restoration

Huichun Liu; Xiaosong Li; Zhuangfan Huang; Tao Ye; Yang Liu; Haishu Tan

cs.CVMar 3, 2026

CAWM-Mamba: A unified model for infrared-visible image fusion and compound adverse weather restoration

Authors: Huichun Liu, Xiaosong Li, Zhuangfan Huang, Tao Ye, Yang Liu, Haishu Tan

Organizations: School of Physics and Optoelectronic Engineering, Foshan University, Foshan 528225, China · Guangdong-HongKong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, Foshan 528225, China · School of Mechanical Electronic and Information Engineering, China University of Mining and Technology, Beijing 100083, China

Abstract

Multimodal Image Fusion (MMIF) integrates complementary information from various modalities to produce clearer and more informative fused images. MMIF under adverse weather is particularly crucial in autonomous driving and UAV monitoring applications. However, existing adverse weather fusion methods generally only tackle single types of degradation such as haze, rain, or snow, and fail when multiple degradations coexist (e.g., haze+rain, rain+snow). To address this challenge, we propose Compound Adverse Weather Mamba (CAWM-Mamba), the first end-to-end framework that jointly performs image fusion and compound weather restoration with unified shared weights. Our network contains three key components: (1) a Weather-Aware Preprocess Module (WAPM) to enhance degraded visible features and extracts global weather embeddings; (2) a Cross-modal Feature Interaction Module (CFIM) to facilitate the alignment of heterogeneous modalities and exchange of complementary features across modalities; and (3) a Wavelet Space State Block (WSSB) that leverages wavelet-domain decomposition to decouple multi-frequency degradations. WSSB includes Freq-SSM, a module that models anisotropic high-frequency degradation without redundancy, and a unified degradation representation mechanism to further improve generalization across complex compound weather conditions. Extensive experiments on the AWMM-100K benchmark and three standard fusion datasets demonstrate that CAWM-Mamba consistently outperforms state-of-the-art methods in both compound and single-weather scenarios. In addition, our fusion results excel in downstream tasks covering semantic segmentation and object detection, confirming the practical value in real-world adverse weather perception. The source code will be available at https://github.com/Feecuin/CAWM-Mamba.

Figures & tables

Figure 1: The overall architecture of CAWM-Mamba. It consists of three main components: WAPM, CFIM, and WSSB. Within WSSB, Freq-SSM captures anisotropic high-frequency features, while CDSM learns a unified degradation representation for robust generalization.

Figure 2: The distribution of degradation interference in the wavelet domain. The left image is a visible light image with degradation interference, while the right image shows the distribution in the wavelet domain.

Figure 3: Freq-SSM scanning mechanism

Figure 4: Comparison of fusion results of different methods in three scenarios with composite degradation interference.

Source	Methods	publication	Baseline	$Q_{MI}$	$Q_{NICE}$	$Q_{G}$	$Q_{M}$	$Q_{P}$	$Q_{S}$	$Q_{CV}$	SSIM	$Q_{abf}$	Avg.Rank
Haze&Rain	FusionBooster ( Cheng et al., 2025b )	IJCV’25	AdaIR	0.2606	0.8032	0.2353	0.3804	0.1241	0.5374	1815.6884	0.1819	0.2811	8.11
	GIF-Net ( Cheng et al., 2025a )	CVPR’25		0.2749	0.8032	0.2943	0.3876	0.1911	0.7261	1793.9563	0.3100	0.3173	4.78
	MaeFuse ( Li et al., 2025b )	TIP’25		0.2283	0.8031	0.2885	0.4344	0.1464	0.7394	1200.3360	0.2960	0.3617	5.67
	SAGE ( Wu et al., 2025 )	CVPR’25		0.2318	0.8033	0.2242	0.4389	0.1382	0.6878	1133.0998	0.2170	0.3143	6.00
	SHIP ( Zheng et al., 2024 )	CVPR’24		0.2341	0.8034	0.2793	0.4873	0.1519	0.6756	1058.1555	0.2147	0.3843	4.56
	TIM ( Liu et al., 2024b )	TPAMI’24		0.1569	0.8026	0.1569	0.3264	0.0599	0.3869	1797.5505	0.0958	0.2368	10.78

Table 1: Comparison of quantitative results of different methods in compound adverse weather scenes. The best scores are in bold, while the second-best scores are in blue.

Figure 5: Comparison of fusion results of different methods in three scenarios with single degradation interference.

Source	Methods	Publication	Baseline	$Q_{MI}$	$Q_{NICE}$	$Q_{G}$	$Q_{M}$	$Q_{P}$	$Q_{S}$	$Q_{CV}$	SSIM	$Q_{abf}$	Avg.Rank
Rain	FusionBooster ( Cheng et al., 2025b )	IJCV’25	AWRaCLe	0.3065	0.8037	0.2830	0.4086	0.1646	0.6247	812.3392	0.2473	0.3517	8.33
	GIF-Net ( Cheng et al., 2025a )	CVPR’25		0.2927	0.8036	0.3037	0.3712	0.2071	0.8060	664.2364	0.3499	0.3605	6.67
	MaeFuse ( Li et al., 2025b )	TIP’25		0.3126	0.8041	0.3372	0.4446	0.1888	0.8337	580.1806	0.3680	0.4280	4.67
	SAGE ( Wu et al., 2025 )	CVPR’25		0.4134	0.8062	0.2304	0.5201	0.1872	0.7899	462.2776	0.2809	0.3729	4.89
	SHIP ( Zheng et al., 2024 )	CVPR’24		0.3966	0.8056	0.3777	0.6662	0.2299	0.8049	452.2582	0.3347	0.4923	3.11
	TIM ( Liu et al., 2024b )	TPAMI’24		0.2659	0.8039	0.1877	0.3498	0.0780	0.4702	922.9942	0.1190	0.2774	10.33

Table 2: Comparison of quantitative results of different methods in single adverse weather scenes. The best scores are in bold, while the second-best scores are in blue.

Figure 6: Comparison of fusion results of different methods in real-world scenarios

Figure 7: Detection comparisons of different methods.

Methods	Publication	Baseline	People	Car	Bus	Lamp	Motorcycle	Truck	[email protected]	AP@[0.5:0.95]
VI	-	-	0.26	0.767	0.715	0.554	0.538	0.592	0.571	0.339
IR	-	-	0.605	0.619	0.319	0.013	0.299	0.227	0.347	0.203
FusionBooster ( Cheng et al., 2025b )	IJCV’25	AdaIR	0.799	0.887	0.846	0.681	0.666	0.775	0.776	0.484
GIF-Net ( Cheng et al., 2025a )	CVPR’25		0.795	0.903	0.922	0.772	0.672	0.792	0.809	0.506
MaeFuse ( Li et al., 2025b )	TIP’25		0.799	0.885	0.904	0.720	0.669	0.798	0.796	0.503
SAGE ( Wu et al., 2025 )	CVPR’25		0.806	0.903	0.908	0.782	0.657	0.810	0.811	0.516

Table 3: Comparing quantitative results of different methods in object detection task. The best scores are in bold, while the second-best scores are in blue.

Figure 8: Segmentation comparisons of different methods.

Figure 9: Comparison of fusion results of different methods under three standard scenarios

Methods	Baseline	unlabelled	car	person	bike	curve	car_stop	guardrail	color_cone	bump	mIoU
VI	-	96.71	74.57	25.61	49.03	38.92	60.95	70.2	52.99	68.88	59.76
IR	-	96.12	61.17	69.77	23.08	34.16	23.67	0.0	20.63	27.39	39.55
FusionBooster ( Cheng et al., 2025b )	AdaIR	98.22	87.33	72.95	63.76	56.12	68.88	88.35	59.68	66.15	72.52
GIF-Net ( Cheng et al., 2025a )		98.17	86.65	70.63	65.23	55.32	67.77	75.67	61.38	64.67	71.72
MaeFuse ( Li et al., 2025b )		98.30	87.72	72.13	66.45	61.91	69.2	82.94	61.32	74.9	74.99
SAGE ( Wu et al., 2025 )		98.11	86.21	67.56	64.68	52.15	66.84	88.95	60.05	73.34	72.21

Table 4: Comparing quantitative results of different methods in semantic segmentation task. The best scores are in bold, while the second-best scores are in blue.

Source	Methods	Publication	$Q_{MI}$	$Q_{NICE}$	$Q_{G}$	$Q_{M}$	$Q_{P}$	$Q_{S}$	$Q_{CV}$	SSIM	$Q_{abf}$
LLVIP	FusionBooster ( Cheng et al., 2025b )	IJCV’25	0.3395	0.8052	0.4502	0.2281	0.2525	0.7267	891.8374	0.4050	0.5161
	GIF-Net ( Cheng et al., 2025a )	CVPR’25	0.3224	0.8045	0.3916	0.1829	0.2734	0.7641	656.5176	0.4249	0.4783
	MaeFuse ( Li et al., 2025b )	TIP’25	0.3144	0.8045	0.4466	0.1880	0.2995	0.7809	655.3157	0.4415	0.5279
	SAGE ( Wu et al., 2025 )	CVPR’25	0.3824	0.8060	0.5297	0.3433	0.4280	0.8498	432.6362	0.4742	0.6496
	SHIP ( Zheng et al., 2024 )	CVPR’24	0.5886	0.8131	0.6628	1.3962	0.4931	0.8741	388.4069	0.4786	0.7351
	TIM ( Liu et al., 2024b )	TPAMI’24	0.2635	0.8039	0.3053	0.1590	0.2348	0.6358	764.1014	0.2861	0.3669

Table 5: Comparing quantitative results of different methods in standard scenes. The best scores are in bold, while the second-best scores are in blue.

	Haze&Rain						Rain&Snow						Haze&Snow
Methods	$Q_{MI}$	$Q_{NICE}$	$Q_{M}$	$Q_{P}$	$Q_{CV}$	$Q_{abf}$	$Q_{MI}$	$Q_{NICE}$	$Q_{M}$	$Q_{P}$	$Q_{CV}$	$Q_{abf}$	$Q_{MI}$	$Q_{NICE}$	$Q_{M}$	$Q_{P}$	$Q_{CV}$	$Q_{abf}$
w/o WAPM	0.323	0.804	0.682	0.233	250.234	0.491	0.450	0.808	0.734	0.233	179.341	0.505	0.338	0.805	0.674	0.234	238.599	0.505
w/o Freq-SSM	0.335	0.805	0.730	0.228	204.546	0.500	0.449	0.808	0.729	0.228	176.947	0.499	0.347	0.805	0.702	0.232	221.822	0.506
w/o CFIM	0.265	0.804	0.556	0.206	424.822	0.448	0.425	0.807	0.708	0.206	189.912	0.487	0.290	0.804	0.529	0.191	512.925	0.450
w/o WAPM & CDSM	0.314	0.804	0.688	0.233	248.967	0.494	0.463	0.808	0.741	0.230	199.260	0.494	0.344	0.805	0.656	0.234	262.215	0.499
CAWM-Mamba	0.360	0.805	0.743	0.240	194.344	0.509	0.477	0.809	0.799	0.249	183.127	0.514	0.368	0.806	0.712	0.236	206.410	0.511

Table 6: Quantitative comparative structures in ablation studies. The best scores are in bold.

	Haze						Rain						Snow
Methods	$Q_{MI}$	$Q_{NICE}$	$Q_{M}$	$Q_{P}$	$Q_{CV}$	$Q_{abf}$	$Q_{MI}$	$Q_{NICE}$	$Q_{M}$	$Q_{P}$	$Q_{CV}$	$Q_{abf}$	$Q_{MI}$	$Q_{NICE}$	$Q_{M}$	$Q_{P}$	$Q_{CV}$	$Q_{abf}$
w/o WAPM	0.219	0.803	0.465	0.299	936.957	0.409	0.415	0.807	0.686	0.201	588.402	0.437	0.457	0.808	0.580	0.318	338.571	0.511
w/o Freq-SSM	0.220	0.803	0.480	0.294	957.493	0.437	0.379	0.806	0.663	0.202	490.035	0.454	0.462	0.809	0.584	0.310	313.490	0.508
w/o CFIM	0.222	0.803	0.392	0.272	1265.871	0.272	0.391	0.806	0.646	0.201	556.927	0.435	0.418	0.807	0.531	0.278	350.504	0.485
w/o WAPM & CDSM	0.216	0.803	0.429	0.244	1335.691	0.350	0.404	0.806	0.664	0.215	584.111	0.441	0.453	0.808	0.600	0.321	316.826	0.511
CAWM-Mamba	0.501	0.811	0.882	0.486	159.615	0.634	0.431	0.807	0.758	0.236	451.176	0.502	0.532	0.812	0.746	0.349	211.109	0.566

Table 7: Quantitative comparative structures in ablation studies. The best scores are in bold.

Metric	CAWM-Mamba	AWFusion	AWMFuse	Text-IF	TG-ECNet
Params	31.95M	19.36 M	137.29M	89.01M	160.863M
FLOPs	371.01G	4167.20G	1145.5G	1518.88G	-

Table 8: The performance of different single weather restoration fusion methods on the quantity of model parameters and MACs.

Explore similar work

Jun 25, 2026

Multi-modality Image Fusion under Adverse Weather: Mask-Guided Feature Restoration and Interaction

Multi-modality image fusion (MMIF) enhances scene representation by exploiting complementary cues from different modalities. Adverse weather, however, causes significant image degradation, disrupting feature representation and requiring simultaneous feature restoration and cross-modal complementarity. Existing methods often struggle with effective representation learning under such conditions, limiting their practical performance. To address these challenges, we propose a mask-guided MMIF method that integrates feature restoration and interaction. We first introduce "Pseudo Ground Truth" to simplify training, promoting faster and more effective feature learning. Then, we design a mask generation mechanism based on the mapping relationship between the fused result and the source images, quantifying the relative contribution of each modality during the fusion process. By incorporating the proposed mask-guided cross-modal cross-attention mechanism, the network is encouraged to selectively attend to informative features during modality interaction, mitigating the risk of overfitting to the static distribution of the "Pseudo Ground Truth". Additionally, we propose a mask-guided learning strategy and a task-coupled degradation-aware learning strategy to balance feature restoration and interaction. Extensive experiments on synthetic and real-world datasets demonstrate that our method surpasses state-of-the-art approaches in visual quality, quantitative metrics, and downstream tasks. The source code is available at https://github.com/ixilai/AMG-Fuse.

Xilai Li, Xiaosong Li, Haishu Tan +3

Foshan University, Foshan, China · China University of Mining and Technology, Beijing, China · Kunming University of Science and Technology, Kunming, China

Multi-Source Image Fusion Cross-Modal Attention Image Restoration Multimodal Fusion

Ask PDF

Apr 10, 2026

Degradation-Robust Fusion: An Efficient Degradation-Aware Diffusion Framework for Multimodal Image Fusion in Arbitrary Degradation Scenarios

Complex degradations like noise, blur, and low resolution are typical challenges in real world image fusion tasks, limiting the performance and practicality of existing methods. End to end neural network based approaches are generally simple to design and highly efficient in inference, but their black-box nature leads to limited interpretability. Diffusion based methods alleviate this to some extent by providing powerful generative priors and a more structured inference process. However, they are trained to learn a single domain target distribution, whereas fusion lacks natural fused data and relies on modeling complementary information from multiple sources, making diffusion hard to apply directly in practice. To address these challenges, this paper proposes an efficient degradation aware diffusion framework for image fusion under arbitrary degradation scenarios. Specifically, instead of explicitly predicting noise as in conventional diffusion models, our method performs implicit denoising by directly regressing the fused image, enabling flexible adaptation to diverse fusion tasks under complex degradations with limited steps. Moreover, we design a joint observation model correction mechanism that simultaneously imposes degradation and fusion constraints during sampling to ensure high reconstruction accuracy. Experiments on diverse fusion tasks and degradation configurations demonstrate the superiority of the proposed method under complex degradation scenarios.

Yu Shi, Yu Liu, Zhong-Cheng Wu +3

Department of Biomedical Engineering, Hefei University of Technology · Faculty of Information Engineering and Automation, Kunming University of Science and Technology · School of Information Science and Technology, University of Science and Technology of China

Multi-Source Image Fusion Diffusion Models

Ask PDF

Aug 7, 2026

Degradation-Aware Prompt Learning with Cross-Modal Compensation for Adverse Weather Removal

Adverse weather causes diverse and complex image degradations, severely compromising the reliability of computer vision systems. Existing all-in-one restoration models attempt to address multiple degradation types within a unified framework, but often lack explicit spatial and semantic modeling of degradation characteristics, limiting their adaptability to diverse weather conditions. To address this limitation, we propose a Degradation-Aware Cross-Modal Prompt Compensation Network (DCMPC-Net) that leverages cross-modal degradation cues from a pretrained vision-language model to condition restoration features within a unified backbone. Specifically, our DCMPC-Net mainly consists of the Cross-Modal Prompt Generator (CMPG), Prompt-Guided Attention Alignment Module (PGAAM), and Dual Feature Compensation Module (DFCM). The CMPG integrates textual embeddings with visual features to produce degradation-aware prompts that encode degradation-related semantic and contextual cues. These prompts are injected into the decoder via a PGAAM, which adaptively aligns semantic information with degraded regions to facilitate context-aware restoration. To further enhance structural fidelity, DFCM is introduced that disentangles degradation artifacts from scene structures, thereby improving the reconstruction of fine textures and detailed content. By integrating cross-modal semantic guidance with spatial alignment and structural enhancement, DCMPC-Net achieves robust and perceptually consistent restoration across diverse weather conditions. Extensive experiments show that DCMPC-Net outperforms state-of-the-art methods in both task-specific and unified settings, achieving superior accuracy and visual fidelity.

Wanshu Fan, Yunzhe Zhang, Yue Shen +5

School of Software Engineering, Dalian University, Dalian, China · School of Mathematical Sciences, Dalian University of Technology, Dalian, China · The Hong Kong Polytechnic University, Hong Kong, China +2

Prompt Learning All-in-One Image Restoration Image Restoration

Ask PDF