UniFusion: A Unified Image Fusion Framework with Robust Representation and Source-Aware Preservation

Xingyuan Li; Songcheng Du; Yang Zou; HaoYuan Xu; Zhiying Jiang; Jinyuan Liu

cs.CVMar 15, 2026

UniFusion: A Unified Image Fusion Framework with Robust Representation and Source-Aware Preservation

Authors: Xingyuan Li, Songcheng Du, Yang Zou, HaoYuan Xu, Zhiying Jiang, Jinyuan Liu

Organizations: Zhejiang University · Northwest Polytechnical University · Dalian University of Technology · Dalian Maritime University

Abstract

Image fusion aims to integrate complementary information from multiple source images to produce a more informative and visually consistent representation, benefiting both human perception and downstream vision tasks. Despite recent progress, most existing fusion methods are designed for specific tasks (i.e., multi-modal, multi-exposure, or multi-focus fusion) and struggle to effectively preserve source information during the fusion process. This limitation primarily arises from task-specific architectures and the degradation of source information caused by deep-layer propagation. To overcome these issues, we propose UniFusion, a unified image fusion framework designed to achieve cross-task generalization. First, leveraging DINOv3 for modality-consistent feature extraction, UniFusion establishes a shared semantic space for diverse inputs. Second, to preserve the understanding of each source image, we introduce a reconstruction-alignment loss to maintain consistency between fused outputs and inputs. Finally, we employ a bilevel optimization strategy to decouple and jointly optimize reconstruction and fusion objectives, effectively balancing their coupling relationship and ensuring smooth convergence. Extensive experiments across multiple fusion tasks demonstrate UniFusion's superior visual quality, generalization ability, and adaptability to real-world scenarios. Code is available at https://github.com/dusongcheng/UniFusion.

Figures & tables

Figure 2: Overview of the proposed UniFusion framework.

Dataset	M 3 FD Dataset				T&R Dataset				MEFB Dataset
Method	MI $\uparrow$	VIF $\uparrow$	$Q_{abf}\uparrow$	$Q_{y}\uparrow$	MI $\uparrow$	VIF $\uparrow$	$Q_{abf}\uparrow$	$Q_{y}\uparrow$	MI $\uparrow$	VIF $\uparrow$	CC $\uparrow$	PSNR $\uparrow$
CDDFuse [ 62 ]	3.776	0.839	0.610	0.978	3.102	0.732	0.489	0.945	6.575	1.430	0.837	56.809
CoCoNet [ 30 ]	2.621	0.721	0.378	0.826	2.576	0.602	0.362	0.781	4.607	1.091	0.858	58.007
DeFuse [ 27 ]	2.924	0.612	0.332	0.959	2.991	0.667	0.393	0.968	5.238	1.164	0.813	58.209
LRRNet [ 19 ]	2.763	0.629	0.495	0.950	2.766	0.577	0.352	0.781	5.946	1.110	0.885	58.970
MGDN [ 9 ]	2.832	0.750	0.618	0.961	2.810	0.701	0.552	0.968	4.935	1.148	0.838	58.516

Table 1: Quantitative comparison on M 3 FD, TNO, RoadScene, and MEFB datasets. The best and second best results are highlighted in bold and underline .

Figure 3: Quantitative comparison on MIF (top) and MFIF (bottom) datasets. The plots illustrate the distribution of all test samples across five evaluation metrics, where “–” and “ $\circ$ ” indicate the median and mean values, respectively.

Figure 4: Visual comparison of infrared and visual image fusion results with SOTA methods on M 3 FD (top) and T $\&$ R (bottom) datasets.

Figure 5: Visual comparison of medical image fusion results with SOTA methods on MIF dataset.

Figure 6: Visual comparison of multi-exposure image fusion and multi-focus image fusion results with SOTA methods on MEFB and MFIF datasets.

Figure 7: Visual comparison of fusion results for different ablation variants on representative samples from the M 3 FD dataset.

Figure 8: Visualization of encoded features and fusion results with and without the reconstruction alignment module (“Ours” vs “w/o Rec”). The feature maps are extracted from the encoders for the visible and infrared modalities, respectively.

Dataset	M 3 FD Dataset				MEFB Dataset				MFIF Dataset
Configurations	MI $\uparrow$	VIF $\uparrow$	$Q_{abf}\uparrow$	$Q_{y}\uparrow$	MI $\uparrow$	VIF $\uparrow$	CC $\uparrow$	PSNR $\uparrow$	MI $\uparrow$	VIF $\uparrow$	CC $\uparrow$	$Q_{abf}\uparrow$
w/o Adapter	3.646	0.863	0.520	0.911	5.512	1.232	0.489	57.934	5.375	0.969	0.959	0.532
w/o DINOv3 encoder	3.681	0.879	0.602	0.946	5.709	1.334	0.362	58.537	5.624	0.901	0.947	0.491
w/o Reconstruction	3.846	0.870	0.587	0.932	6.434	1.396	0.393	58.779	5.838	1.013	0.961	0.579
w/o Bilevel Optimization	3.924	0.876	0.621	0.959	6.374	1.424	0.393	58.960	6.021	1.032	0.964	0.583
Ours	4.268	0.899	0.637	0.982	6.861	1.484	0.906	59.219	6.253	1.159	0.973	0.685

Table 2: Ablation studies on M 3 FD, MEFB and MFIF datasets. The best and second best results are highlighted in bold .

Explore similar work

UniDiffFusion: A Unified Diffusion Framework for Multi-Task and Degradation-Robust Image Fusion
Aug 22, 2026Xingxin Xu, Siqi Zhao, Xin Li +3Multi-Source Image Fusion Infrared-Visible Image Fusion
School of New Media and Communication, Tianjin University · School of Artificial Intelligence, Tianjin University · School of Automation, Southeast University +1
Degradation-Robust Fusion: An Efficient Degradation-Aware Diffusion Framework for Multimodal Image Fusion in Arbitrary Degradation Scenarios
Apr 10, 2026Yu Shi, Yu Liu, Zhong-Cheng Wu +3Multi-Source Image Fusion Diffusion Models
Department of Biomedical Engineering, Hefei University of Technology · Faculty of Information Engineering and Automation, Kunming University of Science and Technology · School of Information Science and Technology, University of Science and Technology of China
AD-SRF: Attack-Defense Semantic Reinforcement for Multi-Modal Image Fusion
Oct 9, 2026Hao Zhang, Zhiqian Zheng, Linfeng Tang +2Multi-Source Image Fusion
Electronic Information School, Wuhan University, Wuhan, 430072, China · Suzhou Institute, Wuhan University, Suzhou, 215123, China · School of Robotics, Wuhan University, Wuhan, 430072, China

Aug 22, 2026J/K move · Enter open · S save

UniDiffFusion: A Unified Diffusion Framework for Multi-Task and Degradation-Robust Image Fusion

Xingxin Xu, Siqi Zhao, Xin Li, Xinjie Yao+2

School of New Media and Communication, Tianjin University · School of Artificial Intelligence, Tianjin University · School of Automation, Southeast University+1

Multi-Source Image Fusion Infrared-Visible Image Fusion Multimodal Fusion

Open paper AskPDF arXiv

General image fusion aims to integrate complementary information from multiple source images, but existing methods often rely on task-specific models and struggle to maintain robust performance under diverse degradation conditions. In this paper, we propose UniDiffFusion, a unified diffusion framework for multi-task and degradation-robust image fusion. UniDiffFusion leverages the strong generative prior of a pretrained diffusion model to establish a shared fusion backbone across heterogeneous fusion tasks, while introducing task- and degradation-aware conditional adaptation to accommodate their distinct information-selection requirements. Specifically, we employ task prompt modulation to progressively adapt the shared diffusion representations to different fusion objectives, and develop a degradation prompt router to dynamically retrieve degradation-aware priors and restore corrupted source features before fusion. Furthermore, an application prompt bank is introduced to incorporate task-oriented semantic guidance for downstream applications, such as object detection and semantic segmentation, without altering the shared fusion and restoration pathways. The proposed framework is trained in a progressive manner to decouple fusion learning, degradation-aware restoration, and application-specific adaptation, thereby reducing interference among heterogeneous objectives. Extensive experiments on visible-infrared, multi-exposure, and multi-focus image fusion demonstrate that UniDiffFusion achieves superior fusion quality and robustness under both clean and degraded conditions. Moreover, UniDiffFusion consistently improves downstream detection and semantic segmentation performance, demonstrating its effectiveness as a unified diffusion framework for both perceptual fusion and task-oriented vision.

Abstract

Explore similar work

UniDiffFusion: A Unified Diffusion Framework for Multi-Task and Degradation-Robust Image Fusion

Degradation-Robust Fusion: An Efficient Degradation-Aware Diffusion Framework for Multimodal Image Fusion in Arbitrary Degradation Scenarios

AD-SRF: Attack-Defense Semantic Reinforcement for Multi-Modal Image Fusion