SGPDFuse: Semantically-Guided Physics-Disentanglement General Multi-Modal Image Fusion

Haozhen Wei; Chengjun Jiang; Yutong Guo; Xinrui Ju; Xingyuan Li; Xiang Chen; Jinyuan Liu

cs.CVAug 29, 2026

SGPDFuse: Semantically-Guided Physics-Disentanglement General Multi-Modal Image Fusion

Authors: Haozhen Wei, Chengjun Jiang, Yutong Guo, Xinrui Ju, Xingyuan Li, Xiang Chen, Jinyuan Liu

Organizations: Dalian University of Technology Dalian, Liaoning, China · City University of Hong Kong Hong Kong, China · Zhejiang University Hangzhou, Zhejiang, China

Abstract

Multimodal image fusion (MMIF) aims to integrate complementary sensor data into a single representation that preserves intrinsic scene reality while eliminating environmental interferences. Most existing approaches rely on blind feature aggregation, which excels at signal accumulation but fails to distinguish essential content from physical degradations. We propose SGPDFuse, which bridges this gap by mapping inputs into a physics-disentangled structural representation via a Semantic-Physical Parametric Bridge (SPPB) built on pretrained vision foundation models, utilizing the Intrinsic-Variation principle to decouple invariant scene attributes from transient environmental factors. To guide this decomposition, we introduce a Semantic Alignment mechanism: we explicitly anchor the fused representation to salient semantic features in the same foundation model feature space via cosine similarity to preserve critical targets, while enforcing physical texture fidelity through Gram-matrix regularization to strictly eliminate unnatural artifacts. Extensive experiments demonstrate that SGPDFuse achieves state-of-the-art performance across infrared-visible, multi-focus, and multi-exposure benchmarks using a single architecture.

Figures & tables

Figure 1. Paradigm shift in multi-modal fusion. (a) Traditional methods conflate intrinsic scene content with variation factors. (b) Our method recovers sharper detail. (c) SPPB predicts a variation map $\mathcal{W}$ from DINOv3 features, separating $\Phi$ from $\Psi$ .

Figure 2. Illustration of our Reconstruction Alignment (RecA) loss for multi-modal image fusion.

Figure 3. Overview of SGPDFuse. A dual-branch encoder extracts base and detail features from $I_{A}$ , $I_{B}$ ; detail features are augmented with DINOv3-LoRA. The SPPB predicts a variation map $\mathcal{W}$ , guiding disentanglement into intrinsic ( $\Phi$ ) and variation ( $\Psi$ ) components. Only $\Phi$ enters fusion; $\Psi$ is discarded. A semantic alignment loss $\mathcal{L}_{\text{sem}}$ supervises in DINO feature space.

Figure 4. Qualitative comparison of fusion quality with other state-of-the-art methods on diverse multimodal fusion tasks. Magnified views demonstrate that SGPDFuse consistently achieves superior visual quality with enhanced detail preservation.


Methods	$\text{M}^{3}\text{FD}$ Dataset				T & R Dataset				MFIF Dataset				MEFB Dataset
Methods	CC $\uparrow$	PSNR $\uparrow$	TE $\uparrow$	SSIM $\uparrow$	CC $\uparrow$	PSNR $\uparrow$	TE $\uparrow$	SSIM $\uparrow$	CC $\uparrow$	PSNR $\uparrow$	TE $\uparrow$	SSIM $\uparrow$	CC $\uparrow$	PSNR $\uparrow$	TE $\uparrow$	SSIM $\uparrow$
SwinFusion ( Ma et al., 2022 )	0.52	62.65	7.10	0.44	0.61	61.41	7.16	0.54	0.97	75.73	7.29	0.91	0.89	59.05	4.90	0.80
TC-MoA ( Zhu et al., 2024 )	0.48	61.92	6.93	0.49	0.58	62.89	6.86	0.52	0.97	74.87	7.29	0.90	0.86	58.92	4.88	0.80
GIFNet ( Cheng et al., 2025 )	0.58	63.36	6.74	0.51	0.64	63.06	6.87	0.54	0.95	68.17	7.14	0.83	0.89	58.12	4.90	0.56
CCF ( Cao et al., 2024 )	0.53	63.27	6.89	0.69	0.57	63.43	6.94	0.59	0.90	68.81	7.13	0.73	0.85	58.65	4.55	0.50

Table 1. Quantitative comparisons of fusion metrics with other SOTA fusion methods on

\text{M}^{3}\text{FD}

, TNO & RoadScene, MFIF and MEFB datasets. Boldface denotes the best while underline denotes the second best results.


Methods	$\text{M}^{3}\text{FD}$ Dataset							FMB Dataset
Methods	People	Car	Bus	Light	Moto	Trunk	mAP	Car	Trunk	T-Light	T-Sign	Motor	Pole	mIoU
SwinFusion ( Ma et al., 2022 )	0.283	0.495	0.624	0.107	0.178	0.400	0.348	0.821	0.505	0.268	0.568	0.684	0.403	0.665
TC-MoA ( Zhu et al., 2024 )	0.288	0.501	0.624	0.122	0.155	0.405	0.349	0.830	0.490	0.366	0.703	0.644	0.415	0.687
GIFNet ( Cheng et al., 2025 )	0.280	0.496	0.610	0.106	0.141	0.429	0.344	0.834	0.579	0.327	0.687	0.690	0.419	0.695
CCF ( Cao et al., 2024 )	0.257	0.481	0.608	0.105	0.106	0.405	0.327	0.813	0.464	0.317	0.694	0.663	0.416	0.678

Table 2. Quantitative comparisons of downstream task performance with other existing fusion methods for object detection on

\text{M}^{3}\text{FD}

and semantic segmentation on FMB dataset. Boldface denotes the best while underline denotes the second best results.

Figure 5. (a)Qualitative comparisons of object detection performance in smoke scene. (b)Qualitative comparisons of semantic segmentation performance in nighttime scene.


Variant	CC $\uparrow$	PSNR $\uparrow$	TE $\uparrow$	SSIM $\uparrow$

Study on SPPB
(A) Full model	0.56	63.57	7.16	0.76
(B) w/o SPPB ( $\mathcal{W}=0.5$ )	0.52	61.38	7.10	0.72
(C) w/o IV Disentanglement	0.53	61.45	7.10	0.73

Table 3. Ablation study on

\text{M}^{3}\text{FD}

. Bold denotes the best result.

Explore similar work

When Integral Meets Decomposition: A Signal-Level Self-Supervised Feature Decompose Paradigm for Multi-Modal Image Fusion
Sep 30, 2026Zeyu Wang, Jiayu Wang, Haiyu Song +1Multi-Source Image Fusion Multimodal Disentangled Representation Learning
College of Computer Science and Engineering, Dalian Minzu University, Dalian, China · School of Artificial Intelligence and Robotics, Hunan University, Hunan, China · Department of Automation, Tsinghua University, Beijing, China
AD-SRF: Attack-Defense Semantic Reinforcement for Multi-Modal Image Fusion
Oct 9, 2026Hao Zhang, Zhiqian Zheng, Linfeng Tang +2Multi-Source Image Fusion
Electronic Information School, Wuhan University, Wuhan, 430072, China · Suzhou Institute, Wuhan University, Suzhou, 215123, China · School of Robotics, Wuhan University, Wuhan, 430072, China
Win-Win Fusion: Multi-Modal Image Fusion with Visual-Semantic Collaboration
Oct 9, 2026Hao Zhang, Xuhui Zuo, Jiayi MaMulti-Source Image Fusion
Electronic Information School, Wuhan University, Wuhan 430072, China

Sep 30, 2026J/K move · Enter open · S save

When Integral Meets Decomposition: A Signal-Level Self-Supervised Feature Decompose Paradigm for Multi-Modal Image Fusion

Zeyu Wang, Jiayu Wang, Haiyu Song, Haoran Duan

College of Computer Science and Engineering, Dalian Minzu University, Dalian, China · School of Artificial Intelligence and Robotics, Hunan University, Hunan, China · Department of Automation, Tsinghua University, Beijing, China

Multi-Source Image Fusion Multimodal Disentangled Representation Learning Self-Supervised Learning Multimodal Fusion

Open paper AskPDF arXiv

Multimodal image fusion (MMIF) aims to integrate complementary information from different modalities into a high-quality fused image and support downstream tasks. Recently, feature decomposition has become an important paradigm by separating source images into common and modality-specific unique features. However, existing methods lack clear supervision because ground-truth (GT) decomposition feature maps are unavailable. They usually combine multiple image-level metrics as losses, which are inherently incomplete and may conflict since each pixel couples attributes such as texture, edge, and contour. To address this, we propose a 1D signal-level self-supervised feature decomposition paradigm. Our core insight is to reformulate feature decomposition from unclear 2D image-level supervision into an integral-driven 1D signal-level optimization problem. This objective-level reformulation uses the 1D signal form to compute the integral constraint. The decomposer is optimized by the integral area between common and original signals, enabling more stable optimization with a clear optimization objective. Our model follows a two-stage SSL framework. Stage I designs dual pretext tasks for integral-driven decomposition at the signal level and structure-preserving reconstruction at the image level. Stage II fuses unique features and combines them with common features to reconstruct the fused image. Experiments on representative MMIF tasks show state-of-the-art (SOTA) performance. Code: github.com/Wangjiayu0512/SIDFusion.

Abstract

Explore similar work

When Integral Meets Decomposition: A Signal-Level Self-Supervised Feature Decompose Paradigm for Multi-Modal Image Fusion

AD-SRF: Attack-Defense Semantic Reinforcement for Multi-Modal Image Fusion

Win-Win Fusion: Multi-Modal Image Fusion with Visual-Semantic Collaboration