CAWM-Mamba: A unified model for infrared-visible image fusion and compound adverse weather restoration

Huichun Liu; Xiaosong Li; Zhuangfan Huang; Tao Ye; Yang Liu; Haishu Tan

cs.CVMar 3, 2026

CAWM-Mamba: A unified model for infrared-visible image fusion and compound adverse weather restoration

Authors: Huichun Liu, Xiaosong Li, Zhuangfan Huang, Tao Ye, Yang Liu, Haishu Tan

Organizations: School of Physics and Optoelectronic Engineering, Foshan University, Foshan 528225, China · Guangdong-HongKong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, Foshan 528225, China · School of Mechanical Electronic and Information Engineering, China University of Mining and Technology, Beijing 100083, China

Abstract

Multimodal Image Fusion (MMIF) integrates complementary information from various modalities to produce clearer and more informative fused images. MMIF under adverse weather is particularly crucial in autonomous driving and UAV monitoring applications. However, existing adverse weather fusion methods generally only tackle single types of degradation such as haze, rain, or snow, and fail when multiple degradations coexist (e.g., haze+rain, rain+snow). To address this challenge, we propose Compound Adverse Weather Mamba (CAWM-Mamba), the first end-to-end framework that jointly performs image fusion and compound weather restoration with unified shared weights. Our network contains three key components: (1) a Weather-Aware Preprocess Module (WAPM) to enhance degraded visible features and extracts global weather embeddings; (2) a Cross-modal Feature Interaction Module (CFIM) to facilitate the alignment of heterogeneous modalities and exchange of complementary features across modalities; and (3) a Wavelet Space State Block (WSSB) that leverages wavelet-domain decomposition to decouple multi-frequency degradations. WSSB includes Freq-SSM, a module that models anisotropic high-frequency degradation without redundancy, and a unified degradation representation mechanism to further improve generalization across complex compound weather conditions. Extensive experiments on the AWMM-100K benchmark and three standard fusion datasets demonstrate that CAWM-Mamba consistently outperforms state-of-the-art methods in both compound and single-weather scenarios. In addition, our fusion results excel in downstream tasks covering semantic segmentation and object detection, confirming the practical value in real-world adverse weather perception. The source code will be available at https://github.com/Feecuin/CAWM-Mamba.

Figures & tables

Figure 1: The overall architecture of CAWM-Mamba. It consists of three main components: WAPM, CFIM, and WSSB. Within WSSB, Freq-SSM captures anisotropic high-frequency features, while CDSM learns a unified degradation representation for robust generalization.

Figure 2: The distribution of degradation interference in the wavelet domain. The left image is a visible light image with degradation interference, while the right image shows the distribution in the wavelet domain.

Figure 3: Freq-SSM scanning mechanism

Figure 4: Comparison of fusion results of different methods in three scenarios with composite degradation interference.

Source	Methods	publication	Baseline	$Q_{MI}$	$Q_{NICE}$	$Q_{G}$	$Q_{M}$	$Q_{P}$	$Q_{S}$	$Q_{CV}$	SSIM	$Q_{abf}$	Avg.Rank
Haze&Rain	FusionBooster ( Cheng et al., 2025b )	IJCV’25	AdaIR	0.2606	0.8032	0.2353	0.3804	0.1241	0.5374	1815.6884	0.1819	0.2811	8.11
	GIF-Net ( Cheng et al., 2025a )	CVPR’25		0.2749	0.8032	0.2943	0.3876	0.1911	0.7261	1793.9563	0.3100	0.3173	4.78
	MaeFuse ( Li et al., 2025b )	TIP’25		0.2283	0.8031	0.2885	0.4344	0.1464	0.7394	1200.3360	0.2960	0.3617	5.67
	SAGE ( Wu et al., 2025 )	CVPR’25		0.2318	0.8033	0.2242	0.4389	0.1382	0.6878	1133.0998	0.2170	0.3143	6.00
	SHIP ( Zheng et al., 2024 )	CVPR’24		0.2341	0.8034	0.2793	0.4873	0.1519	0.6756	1058.1555	0.2147	0.3843	4.56
	TIM ( Liu et al., 2024b )	TPAMI’24		0.1569	0.8026	0.1569	0.3264	0.0599	0.3869	1797.5505	0.0958	0.2368	10.78

Table 1: Comparison of quantitative results of different methods in compound adverse weather scenes. The best scores are in bold, while the second-best scores are in blue.

Figure 5: Comparison of fusion results of different methods in three scenarios with single degradation interference.

Source	Methods	Publication	Baseline	$Q_{MI}$	$Q_{NICE}$	$Q_{G}$	$Q_{M}$	$Q_{P}$	$Q_{S}$	$Q_{CV}$	SSIM	$Q_{abf}$	Avg.Rank
Rain	FusionBooster ( Cheng et al., 2025b )	IJCV’25	AWRaCLe	0.3065	0.8037	0.2830	0.4086	0.1646	0.6247	812.3392	0.2473	0.3517	8.33
	GIF-Net ( Cheng et al., 2025a )	CVPR’25		0.2927	0.8036	0.3037	0.3712	0.2071	0.8060	664.2364	0.3499	0.3605	6.67
	MaeFuse ( Li et al., 2025b )	TIP’25		0.3126	0.8041	0.3372	0.4446	0.1888	0.8337	580.1806	0.3680	0.4280	4.67
	SAGE ( Wu et al., 2025 )	CVPR’25		0.4134	0.8062	0.2304	0.5201	0.1872	0.7899	462.2776	0.2809	0.3729	4.89
	SHIP ( Zheng et al., 2024 )	CVPR’24		0.3966	0.8056	0.3777	0.6662	0.2299	0.8049	452.2582	0.3347	0.4923	3.11
	TIM ( Liu et al., 2024b )	TPAMI’24		0.2659	0.8039	0.1877	0.3498	0.0780	0.4702	922.9942	0.1190	0.2774	10.33

Table 2: Comparison of quantitative results of different methods in single adverse weather scenes. The best scores are in bold, while the second-best scores are in blue.

Figure 6: Comparison of fusion results of different methods in real-world scenarios

Figure 7: Detection comparisons of different methods.

Methods	Publication	Baseline	People	Car	Bus	Lamp	Motorcycle	Truck	[email protected]	AP@[0.5:0.95]
VI	-	-	0.26	0.767	0.715	0.554	0.538	0.592	0.571	0.339
IR	-	-	0.605	0.619	0.319	0.013	0.299	0.227	0.347	0.203
FusionBooster ( Cheng et al., 2025b )	IJCV’25	AdaIR	0.799	0.887	0.846	0.681	0.666	0.775	0.776	0.484
GIF-Net ( Cheng et al., 2025a )	CVPR’25		0.795	0.903	0.922	0.772	0.672	0.792	0.809	0.506
MaeFuse ( Li et al., 2025b )	TIP’25		0.799	0.885	0.904	0.720	0.669	0.798	0.796	0.503
SAGE ( Wu et al., 2025 )	CVPR’25		0.806	0.903	0.908	0.782	0.657	0.810	0.811	0.516

Table 3: Comparing quantitative results of different methods in object detection task. The best scores are in bold, while the second-best scores are in blue.

Figure 8: Segmentation comparisons of different methods.

Figure 9: Comparison of fusion results of different methods under three standard scenarios

Methods	Baseline	unlabelled	car	person	bike	curve	car_stop	guardrail	color_cone	bump	mIoU
VI	-	96.71	74.57	25.61	49.03	38.92	60.95	70.2	52.99	68.88	59.76
IR	-	96.12	61.17	69.77	23.08	34.16	23.67	0.0	20.63	27.39	39.55
FusionBooster ( Cheng et al., 2025b )	AdaIR	98.22	87.33	72.95	63.76	56.12	68.88	88.35	59.68	66.15	72.52
GIF-Net ( Cheng et al., 2025a )		98.17	86.65	70.63	65.23	55.32	67.77	75.67	61.38	64.67	71.72
MaeFuse ( Li et al., 2025b )		98.30	87.72	72.13	66.45	61.91	69.2	82.94	61.32	74.9	74.99
SAGE ( Wu et al., 2025 )		98.11	86.21	67.56	64.68	52.15	66.84	88.95	60.05	73.34	72.21

Table 4: Comparing quantitative results of different methods in semantic segmentation task. The best scores are in bold, while the second-best scores are in blue.

Source	Methods	Publication	$Q_{MI}$	$Q_{NICE}$	$Q_{G}$	$Q_{M}$	$Q_{P}$	$Q_{S}$	$Q_{CV}$	SSIM	$Q_{abf}$
LLVIP	FusionBooster ( Cheng et al., 2025b )	IJCV’25	0.3395	0.8052	0.4502	0.2281	0.2525	0.7267	891.8374	0.4050	0.5161
	GIF-Net ( Cheng et al., 2025a )	CVPR’25	0.3224	0.8045	0.3916	0.1829	0.2734	0.7641	656.5176	0.4249	0.4783
	MaeFuse ( Li et al., 2025b )	TIP’25	0.3144	0.8045	0.4466	0.1880	0.2995	0.7809	655.3157	0.4415	0.5279
	SAGE ( Wu et al., 2025 )	CVPR’25	0.3824	0.8060	0.5297	0.3433	0.4280	0.8498	432.6362	0.4742	0.6496
	SHIP ( Zheng et al., 2024 )	CVPR’24	0.5886	0.8131	0.6628	1.3962	0.4931	0.8741	388.4069	0.4786	0.7351
	TIM ( Liu et al., 2024b )	TPAMI’24	0.2635	0.8039	0.3053	0.1590	0.2348	0.6358	764.1014	0.2861	0.3669

Table 5: Comparing quantitative results of different methods in standard scenes. The best scores are in bold, while the second-best scores are in blue.

	Haze&Rain						Rain&Snow						Haze&Snow
Methods	$Q_{MI}$	$Q_{NICE}$	$Q_{M}$	$Q_{P}$	$Q_{CV}$	$Q_{abf}$	$Q_{MI}$	$Q_{NICE}$	$Q_{M}$	$Q_{P}$	$Q_{CV}$	$Q_{abf}$	$Q_{MI}$	$Q_{NICE}$	$Q_{M}$	$Q_{P}$	$Q_{CV}$	$Q_{abf}$
w/o WAPM	0.323	0.804	0.682	0.233	250.234	0.491	0.450	0.808	0.734	0.233	179.341	0.505	0.338	0.805	0.674	0.234	238.599	0.505
w/o Freq-SSM	0.335	0.805	0.730	0.228	204.546	0.500	0.449	0.808	0.729	0.228	176.947	0.499	0.347	0.805	0.702	0.232	221.822	0.506
w/o CFIM	0.265	0.804	0.556	0.206	424.822	0.448	0.425	0.807	0.708	0.206	189.912	0.487	0.290	0.804	0.529	0.191	512.925	0.450
w/o WAPM & CDSM	0.314	0.804	0.688	0.233	248.967	0.494	0.463	0.808	0.741	0.230	199.260	0.494	0.344	0.805	0.656	0.234	262.215	0.499
CAWM-Mamba	0.360	0.805	0.743	0.240	194.344	0.509	0.477	0.809	0.799	0.249	183.127	0.514	0.368	0.806	0.712	0.236	206.410	0.511

Table 6: Quantitative comparative structures in ablation studies. The best scores are in bold.

	Haze						Rain						Snow
Methods	$Q_{MI}$	$Q_{NICE}$	$Q_{M}$	$Q_{P}$	$Q_{CV}$	$Q_{abf}$	$Q_{MI}$	$Q_{NICE}$	$Q_{M}$	$Q_{P}$	$Q_{CV}$	$Q_{abf}$	$Q_{MI}$	$Q_{NICE}$	$Q_{M}$	$Q_{P}$	$Q_{CV}$	$Q_{abf}$
w/o WAPM	0.219	0.803	0.465	0.299	936.957	0.409	0.415	0.807	0.686	0.201	588.402	0.437	0.457	0.808	0.580	0.318	338.571	0.511
w/o Freq-SSM	0.220	0.803	0.480	0.294	957.493	0.437	0.379	0.806	0.663	0.202	490.035	0.454	0.462	0.809	0.584	0.310	313.490	0.508
w/o CFIM	0.222	0.803	0.392	0.272	1265.871	0.272	0.391	0.806	0.646	0.201	556.927	0.435	0.418	0.807	0.531	0.278	350.504	0.485
w/o WAPM & CDSM	0.216	0.803	0.429	0.244	1335.691	0.350	0.404	0.806	0.664	0.215	584.111	0.441	0.453	0.808	0.600	0.321	316.826	0.511
CAWM-Mamba	0.501	0.811	0.882	0.486	159.615	0.634	0.431	0.807	0.758	0.236	451.176	0.502	0.532	0.812	0.746	0.349	211.109	0.566

Table 7: Quantitative comparative structures in ablation studies. The best scores are in bold.

Metric	CAWM-Mamba	AWFusion	AWMFuse	Text-IF	TG-ECNet
Params	31.95M	19.36 M	137.29M	89.01M	160.863M
FLOPs	371.01G	4167.20G	1145.5G	1518.88G	-

Table 8: The performance of different single weather restoration fusion methods on the quantity of model parameters and MACs.

Explore similar work

Multi-modality Image Fusion under Adverse Weather: Mask-Guided Feature Restoration and Interaction
Jun 25, 2026Xilai Li, Xiaosong Li, Haishu Tan +3Multi-Source Image Fusion Cross-Modal Attention
Foshan University, Foshan, China · China University of Mining and Technology, Beijing, China · Kunming University of Science and Technology, Kunming, China
Degradation-Robust Fusion: An Efficient Degradation-Aware Diffusion Framework for Multimodal Image Fusion in Arbitrary Degradation Scenarios
Apr 10, 2026Yu Shi, Yu Liu, Zhong-Cheng Wu +3Multi-Source Image Fusion Diffusion Models
Department of Biomedical Engineering, Hefei University of Technology · Faculty of Information Engineering and Automation, Kunming University of Science and Technology · School of Information Science and Technology, University of Science and Technology of China
Degradation-Aware Prompt Learning with Cross-Modal Compensation for Adverse Weather Removal
Aug 7, 2026Wanshu Fan, Yunzhe Zhang, Yue Shen +5Prompt Learning All-in-One Image Restoration
School of Software Engineering, Dalian University, Dalian, China · School of Mathematical Sciences, Dalian University of Technology, Dalian, China · The Hong Kong Polytechnic University, Hong Kong, China +2

Jun 25, 2026J/K move · Enter open · S save

Multi-modality Image Fusion under Adverse Weather: Mask-Guided Feature Restoration and Interaction

Xilai Li, Xiaosong Li, Haishu Tan, Tao Ye+2

Foshan University, Foshan, China · China University of Mining and Technology, Beijing, China · Kunming University of Science and Technology, Kunming, China

Multi-Source Image Fusion Cross-Modal Attention Image Restoration Multimodal Fusion

Open paper AskPDF arXiv

Multi-modality image fusion (MMIF) enhances scene representation by exploiting complementary cues from different modalities. Adverse weather, however, causes significant image degradation, disrupting feature representation and requiring simultaneous feature restoration and cross-modal complementarity. Existing methods often struggle with effective representation learning under such conditions, limiting their practical performance. To address these challenges, we propose a mask-guided MMIF method that integrates feature restoration and interaction. We first introduce "Pseudo Ground Truth" to simplify training, promoting faster and more effective feature learning. Then, we design a mask generation mechanism based on the mapping relationship between the fused result and the source images, quantifying the relative contribution of each modality during the fusion process. By incorporating the proposed mask-guided cross-modal cross-attention mechanism, the network is encouraged to selectively attend to informative features during modality interaction, mitigating the risk of overfitting to the static distribution of the "Pseudo Ground Truth". Additionally, we propose a mask-guided learning strategy and a task-coupled degradation-aware learning strategy to balance feature restoration and interaction. Extensive experiments on synthetic and real-world datasets demonstrate that our method surpasses state-of-the-art approaches in visual quality, quantitative metrics, and downstream tasks. The source code is available at https://github.com/ixilai/AMG-Fuse.

Abstract

Explore similar work

Multi-modality Image Fusion under Adverse Weather: Mask-Guided Feature Restoration and Interaction

Degradation-Robust Fusion: An Efficient Degradation-Aware Diffusion Framework for Multimodal Image Fusion in Arbitrary Degradation Scenarios

Degradation-Aware Prompt Learning with Cross-Modal Compensation for Adverse Weather Removal