Multi-Modal Image Fusion via Intervention-Stable Feature Learning

Xue Wang; Zheng Guan; Wenhua Qian; Chengchao Wang; Runzhuo Ma

cs.CVMar 24, 2026

Multi-Modal Image Fusion via Intervention-Stable Feature Learning

Authors: Xue Wang, Zheng Guan, Wenhua Qian, Chengchao Wang, Runzhuo Ma

Organizations: School of Information Science and Engineering, Yunnan University · School of Artificial Intelligence, Nanyang Normal University · Department of Electrical and Electronic Engineering, Hong Kong Polytechnic University

Abstract

Multi-modal image fusion integrates complementary information from different modalities into a unified representation. Current methods predominantly optimize statistical correlations between modalities, often capturing dataset-induced spurious associations that degrade under distribution shifts. In this paper, we propose an intervention-based framework inspired by causal principles to identify robust cross-modal dependencies. Drawing insights from Pearl's causal hierarchy, we design three principled intervention strategies to probe different aspects of modal relationships: i) complementary masking with spatially disjoint perturbations tests whether modalities can genuinely compensate for each other's missing information, ii) random masking of identical regions identifies feature subsets that remain informative under partial observability, and iii) modality dropout evaluates the irreplaceable contribution of each modality. Based on these interventions, we introduce a Causal Feature Integrator (CFI) that learns to identify and prioritize intervention-stable features maintaining importance across different perturbation patterns through adaptive invariance gating, thereby capturing robust modal dependencies rather than spurious correlations. Extensive experiments demonstrate that our method achieves SOTA performance on both public benchmarks and downstream high-level vision tasks.

Figures & tables

Figure 1: Comparison with SOTA method in training framework and performance. (a) General designs of existing methods, which often rely on empirical trial-and-error to fit all source features. (b) Our framework actively probes modal dependencies through structured interventions. (c) The superiority of our method is validated by its advantages in static metrics (CC, PSNR), efficiency analysis (FPS), semantic segmentation (mIoU), object detection (mAP), and qualitative comparisons.

Figure 2: The framework of the proposed method. It employs a U-Net-like Siamese architecture that incorporates CFI within the decoder to enable robust multi-modal fusion. By leveraging three complementary intervention strategies, the model learns to identify intervention-stable features that represent genuine cross-modal complementarity rather than spurious statistical co-occurrences arising from dataset biases.

Figure 3: An illustrative example of feature visualization. Through the learnable invariance gates, the model progressively focuses on intervention-stable features across successive iterations, prioritizing regions that maintain consistent importance under perturbations to ensure effective integration of complementary modalities.

Figure 4: Qualitative comparison with SOTA methods on the IVIF benchmarks.

Method	TNO					MSRS					M 3 FD
Method	AG	SF	PSNR	CC	$\mathcal{Q}_{abf}$	AG	SF	PSNR	CC	$\mathcal{Q}_{abf}$	AG	SF	PSNR	CC	$\mathcal{Q}_{abf}$
TIMFusion	4.019	3.901	61.41	0.379	0.397	3.747	4.378	59.48	0.525	0.424	4.272	5.176	61.85	0.425	0.489
Conti	3.860	3.987	61.12	0.446	0.437	3.737	4.504	64.26	0.603	0.570	4.476	5.545	61.11	0.479	0.499
SAGE	3.817	3.745	59.81	0.317	0.444	3.431	4.373	65.59	0.594	0.529	4.590	5.443	61.88	0.557	0.510
MUFusion	4.713	3.867	61.73	0.448	0.319	3.160	3.482	63.85	0.589	0.422	4.220	4.155	61.72	0.451	0.372
LUT-Fuse	3.810	3.830	61.23	0.436	0.403	3.801	4.494	64.24	0.602	0.579	4.080	5.019	61.31	0.470	0.446

Table 1: Quantitative results of our proposed method vs. SOTA methods on the IVIF benchmarks. The best value is highlighted with Bold .

Figure 5: Visualization comparison of ablation study

Case	AG	SF	PSNR	CC	$\mathcal{Q}_{abf}$
w/o CFI	5.764	5.972	60.21	0.544	0.428
w/o $\mathcal{L}_{inv}$	5.179	5.728	58.08	0.573	0.331
w/o $\mathcal{L}_{nec}$	4.016	4.018	61.39	0.393	0.368
w/o $\mathcal{L}_{nec}\&\mathcal{L}_{inv}$	3.361	3.478	59.85	0.524	0.312
w/o $Int$	5.332	5.348	63.95	0.598	0.524
Ours	6.136	6.244	63.62	0.605	0.467

Table 2: Ablation experiment results. The best value is highlighted with Bold .

Figure 6: Quantitative results of ATE.

Figure 7: Qualitative comparison with SOTA IVIF methods in object detection and semantic segmentation tasks.

Method	mAP	Peo	Car	Bus	Mot	Tru	Lam
TIMFusion	0.796	0.802	0.900	0.846	0.653	0.816	0.756
Conti	0.809	0.789	0.910	0.875	0.650	0.834	0.797
SAGE	0.815	0.783	0.908	0.870	0.704	0.813	0.814
MUFusion	0.804	0.798	0.906	0.881	0.660	0.792	0.788
LUT-Fuse	0.804	0.772	0.905	0.881	0.654	0.836	0.775
LRRNet	0.811	0.780	0.911	0.878	0.694	0.804	0.798

Table 3: Quantitative results of our proposed method vs. SOTA methods on the object detection. The best value is highlighted with Bold .

Method	Unl	Car	Per	Bik	Cur	Roa	mIOU
TIMFusion	0.975	0.826	0.597	0.597	0.389	0.450	0.639
Conti	0.982	0.878	0.699	0.671	0.540	0.646	0.736
SAGE	0.982	0.871	0.678	0.680	0.548	0.621	0.730
MUFusion	0.981	0.866	0.667	0.679	0.507	0.634	0.722
LUT-Fuse	0.982	0.878	0.687	0.681	0.564	0.623	0.736
LRRNet	0.982	0.880	0.676	0.679	0.548	0.642	0.734

Table 4: Quantitative results of our proposed method vs. SOTA methods on the semantic segmentation. The best value is highlighted with Bold .

Method	AG	SF	PSNR	CC	$\mathcal{Q}_{abf}$
TIMFusion	5.122	5.840	61.37	0.800	0.467
Conti	7.205	9.473	63.98	0.857	0.637
SAGE	5.406	6.860	63.91	0.861	0.376
MUFusion	4.777	5.234	62.21	0.861	0.355
LUT-Fuse	6.549	8.467	64.34	0.852	0.605
LRRNet	3.753	4.564	63.92	0.834	0.211

Table 5: Quantitative results of our proposed method vs. SOTA methods on the MIF. The best value is highlighted with Bold .

Figure 8: Qualitative comparison with SOTA methods on the MIF.

Multi-Source Image Fusion

Multi-source image fusion combines complementary images to produce clearer, more informative results for perception, reconstruction, and analysis.

See all 80 papers

Multi-modality Image Fusion under Adverse Weather: Mask-Guided Feature Restoration and Interaction
Jun 25, 2026Xilai Li, Xiaosong Li, Haishu Tan +3Multi-Source Image Fusion Cross-Modal Attention
Foshan University, Foshan, China · China University of Mining and Technology, Beijing, China · Kunming University of Science and Technology, Kunming, China
Win-Win Fusion: Multi-Modal Image Fusion with Visual-Semantic Collaboration
Oct 9, 2026Hao Zhang, Xuhui Zuo, Jiayi MaMulti-Source Image Fusion
Electronic Information School, Wuhan University, Wuhan 430072, China
When Integral Meets Decomposition: A Signal-Level Self-Supervised Feature Decompose Paradigm for Multi-Modal Image Fusion
Sep 30, 2026Zeyu Wang, Jiayu Wang, Haiyu Song +1Multi-Source Image Fusion Multimodal Disentangled Representation Learning
College of Computer Science and Engineering, Dalian Minzu University, Dalian, China · School of Artificial Intelligence and Robotics, Hunan University, Hunan, China · Department of Automation, Tsinghua University, Beijing, China

Jun 25, 2026J/K move · Enter open · S save

Multi-modality Image Fusion under Adverse Weather: Mask-Guided Feature Restoration and Interaction

Xilai Li, Xiaosong Li, Haishu Tan, Tao Ye+2

Foshan University, Foshan, China · China University of Mining and Technology, Beijing, China · Kunming University of Science and Technology, Kunming, China

Multi-Source Image Fusion Cross-Modal Attention Image Restoration Multimodal Fusion

Open paper AskPDF arXiv

Multi-modality image fusion (MMIF) enhances scene representation by exploiting complementary cues from different modalities. Adverse weather, however, causes significant image degradation, disrupting feature representation and requiring simultaneous feature restoration and cross-modal complementarity. Existing methods often struggle with effective representation learning under such conditions, limiting their practical performance. To address these challenges, we propose a mask-guided MMIF method that integrates feature restoration and interaction. We first introduce "Pseudo Ground Truth" to simplify training, promoting faster and more effective feature learning. Then, we design a mask generation mechanism based on the mapping relationship between the fused result and the source images, quantifying the relative contribution of each modality during the fusion process. By incorporating the proposed mask-guided cross-modal cross-attention mechanism, the network is encouraged to selectively attend to informative features during modality interaction, mitigating the risk of overfitting to the static distribution of the "Pseudo Ground Truth". Additionally, we propose a mask-guided learning strategy and a task-coupled degradation-aware learning strategy to balance feature restoration and interaction. Extensive experiments on synthetic and real-world datasets demonstrate that our method surpasses state-of-the-art approaches in visual quality, quantitative metrics, and downstream tasks. The source code is available at https://github.com/ixilai/AMG-Fuse.

Abstract

Multi-Source Image Fusion

Explore similar work

Multi-modality Image Fusion under Adverse Weather: Mask-Guided Feature Restoration and Interaction

Win-Win Fusion: Multi-Modal Image Fusion with Visual-Semantic Collaboration

When Integral Meets Decomposition: A Signal-Level Self-Supervised Feature Decompose Paradigm for Multi-Modal Image Fusion