cs.CVApr 10, 2026

Customized Fusion: A Closed-Loop Dynamic Network for Adaptive Multi-Task-Aware Infrared-Visible Image Fusion

Authors: Zengyi Yang, Yu Liu, Juan Cheng, Zhiqin Zhu, Yafei Zhang, Huafeng Li

Organizations: Department of Biomedical Engineering, Hefei University of Technology · College of Automation, Chongqing University of Post and Telecommunications · Faculty of Information Engineering and Automation, Kunming University of Science and Technology

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Infrared-visible image fusion aims to integrate complementary information for robust visual understanding, but existing fusion methods struggle with simultaneously adapting to multiple downstream tasks. To address this issue, we propose a Closed-Loop Dynamic Network (CLDyN) that can adaptively respond to the semantic requirements of diverse downstream tasks for task-customized image fusion. Specifically, CLDyN introduces a closed-loop optimization mechanism that establishes a semantic transmission chain to achieve explicit feedback from downstream tasks to the fusion network through a Requirement-driven Semantic Compensation (RSC) module. The RSC module leverages a Basis Vector Bank (BVB) and an Architecture-Adaptive Semantic Injection (A2SI) block to customize the network architecture according to task requirements, thereby enabling task-specific semantic compensation and allowing the fusion network to actively adapt to diverse tasks without retraining. To promote semantic compensation, a reward-penalty strategy is introduced to reward or penalize the RSC module based on task performance variations. Experiments on the M3FD, FMB, and VT5000 datasets demonstrate that CLDyN not only maintains high fusion quality but also exhibits strong multi-task adaptability. The code is available at https://github.com/YR0211/CLDyN.

Figures & tables

Figure 1: Comparison of processing paradigms between existing downstream task-aware image fusion methods (a) and the proposed closed-loop dynamic network (b).
Figure 2: Overview of the adaptive multi-task-aware infrared-visible image fusion network. The network forms a semantic transmission chain, where semantic features from multiple downstream tasks guide the RSC module to perform task-specific compensation for the VFN. The reward-penalty strategy optimizes the compensation process by evaluating task performance before and after semantic compensation.
Figure 3: Architecture of the A2SI block. The A2SI block comprises six projection layers ProjiProj_{i} ( i=1,…,6i=1,\ldots,6 ) and several semantic extraction branches with dynamic convolutional architectures based on task semantics.
Figure 4: Qualitative comparison between the proposed method and the “task network retraining” methods.
Figure 5: Qualitative comparison between the proposed method and the “joint training” methods.
Figure 6: Qualitative (a) and quantitative (b) comparison between the proposed method and IDF-TDDT.
Figure 7: Qualitative and quantitative comparison between pre-compensation and post-compensation results.
Figure 8: Qualitative comparison between the full model and its ablated variants.
Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Network architecture of VFN. The VFN (a) consists of a Feature Extraction Blocks (FEB) (b) and a Fusion Feature Reconstruction Block (FRB) (c).
Figure 10: Qualitative comparison between the proposed method and existing state-of-the-art approaches. The first and second columns show the visible and infrared source images, respectively, while the third to tenth columns display the fused results of different comparison methods. The top two rows, middle two rows, and bottom two rows correspond to samples from the M 3 FD, FMB, and VT5000 datasets, respectively.
Figure 11: Qualitative comparison between the full model and the ablation models (Model IV and Model V).
e2{e_{2}}ODSegSOD
mAP 50→95↑{}_{50\rightarrow 95}\uparrowmIoU ↑\uparrowmF β↑{}_{\beta}\uparrowE m↑{}_{m}\uparrow
1280.627360.180.81180.9083
2560.630460.340.81290.9087
5120.629260.250.81300.9092
Appendix
Table 11: Quantitative analysis of the hyperparameter e2{e_{2}} across multiple downstream tasks. The hyperparameter settings of our proposed method are highlighted in bold , while the best performance for each metric is marked with a Red background.
ModelsA2SIBVBParams (M)FLOPs (G)
IV✓✗0.111.88
V✗✓0.35173.04
Full✓✓0.46174.06
Appendix
Table 12: Parameter counts and computational costs of the core components in the proposed method and their corresponding ablated variants.
Figure 12: Training loss curves of the proposed method.
Complex scenariosODSegSOD
mAP 50→95↑{}_{50\rightarrow 95}\uparrowmIoU ↑\uparrowmF β↑{}_{\beta}\uparrowE m↑{}_{m}\uparrow
✓0.628459.730.81000.9074
✗0.630460.340.81290.9087
Appendix
Table 13: Quantitative results under complex scenarios (exposure attenuation α∈[0.4,0.8]\alpha\in[0.4,0.8] ).

Explore similar work

CardsList

  1. CRT-HMAR: Causal Requirement Tracing-Guided Hierarchical Multi-Agent Regulation for Open-Task-Aware Infrared-Visible Image Fusion

    Oct 7, 2026Zengyi Yang, Shuai Yuan, Zhong-Cheng Wu +3Infrared-Visible Image FusionMulti-Task Optimization

    School of Instrument Science and Opto-electronics Engineering, Hefei University of Technology, Hefei 230009, China · Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China

  2. Fusion in Your Way: Aligning Image Fusion with Heterogeneous Demands via Direct Preference Optimization

    May 7, 2026Weijian Su, Songqian Zhang, Yuqi Han +3Diffusion Model AlignmentMulti-Source Image Fusion

    School of Computer Science and Technology, Dalian University of Technology · Key Laboratory of Social Computing and Cognitive Intelligence (Dalian University of Technology), Ministry of Education · Institute of Image Processing and Understanding, North Minzu University

  3. P2Fusion: Prompt-based Progressive Infrared-Visible Image Fusion via Dual-Prior Distillation

    Aug 13, 2026Yi Shi, Huichao Xie, Yuqing Wang +7Multi-Source Image FusionInfrared-Visible Image Fusion

    Northwestern Polytechnical University, Xi’an, China · Hefei University of Technology, Hefei, China · Rocket Force University of Engineering, Xi’an, China +1