SGPDFuse: Semantically-Guided Physics-Disentanglement General Multi-Modal Image Fusion

Multimodal image fusion (MMIF) aims to integrate complementary information from different modalities into a high-quality fused image and support downstream tasks. Recently, feature decomposition has become an important paradigm by separating source images into common and modality-specific unique features. However, existing methods lack clear supervision because ground-truth (GT) decomposition feature maps are unavailable. They usually combine multiple image-level metrics as losses, which are inherently incomplete and may conflict since each pixel couples attributes such as texture, edge, and contour. To address this, we propose a 1D signal-level self-supervised feature decomposition paradigm. Our core insight is to reformulate feature decomposition from unclear 2D image-level supervision into an integral-driven 1D signal-level optimization problem. This objective-level reformulation uses the 1D signal form to compute the integral constraint. The decomposer is optimized by the integral area between common and original signals, enabling more stable optimization with a clear optimization objective. Our model follows a two-stage SSL framework. Stage I designs dual pretext tasks for integral-driven decomposition at the signal level and structure-preserving reconstruction at the image level. Stage II fuses unique features and combines them with common features to reconstruct the fused image. Experiments on representative MMIF tasks show state-of-the-art (SOTA) performance. Code: github.com/Wangjiayu0512/SIDFusion.

Existing image fusion methods struggle to balance visual quality and semantic attributes due to the inherent conflict between their optimization objectives. To address this challenge, we propose an attack-defense semantic reinforcement framework for multi-modal image fusion, termed AD-SRF. By decoupling the optimization of these two aspects and then losslessly reintegrating them, this framework injects rich semantic attributes into the fused image without compromising its visual quality. First, we develop a transformer-based restorative visual fusion network as a strong visual baseline. Equipped with the cross-modal retrieval module and the information restoration-fusion collaborative loss, it produces highly visible fused images even under degraded conditions. Second, we introduce an attack-defense game mechanism to strengthen the semantic attributes of the fused images. The attacker learns perturbation patterns that impair downstream segmentation, whereas the defender restores missing semantics from the original inputs and reintegrates them into the fused image through an imperceptible semantic map. Through iterative adversarial interaction, this mechanism drives the defender to continually strengthen the fused image's semantic attributes, boosting downstream segmentation performance without compromising visual quality. Extensive experiments demonstrate that AD-SRF achieves a win-win performance in both visual quality and semantic attributes. The code is publicly available at https://github.com/HaoZhang1018/AD-SRF.

Existing multi-modal image fusion (MMIF) methods generally face an optimization gap between the visual and semantic domains, limiting the enhancement of perception capabilities. To overcome this long-standing barrier, this work proposes W2F, a new visual-semantic mutual reinforcing paradigm for MMIF. First, we reinforce the foundation for visual-semantic collaboration by amplifying feature complementarity through two targeted designs. Specifically, the segmentation-guided fusion loss introduces object-aware structural guidance into visual fusion, while the embedded information restoration prior strengthens the ability of visual fusion to provide reliable perceptual evidence for the semantic branch under degraded conditions. Then, we introduce a strong multi-modal information interaction mechanism composed of interactive gated mixed attention (IGM-Att) and progressive coupling attention (PC-Att). These modules ensure comprehensive scene representation from both local and global perspectives, providing informational support for visual-semantic collaboration. Finally, we construct a collaborative mechanism to couple visual fusion and semantic fusion. It employs a feature search module to extract complementary key features from both tasks, enriching the representational potential of visual and semantic information. Meanwhile, a gradient correction strategy ensures collaboration from the optimization perspective, enhancing performance for both visual and semantic fusion through consistency learning. Extensive experiments demonstrate that W2F surpasses state-of-the-art methods in both visual appeal and decision accuracy. The code is publicly available at https://github.com/HaoZhang1018/W2F.