Infrared-Visible Image Fusion

Ask about this topic

Overview

Infrared-visible image fusion combines thermal information, which can reveal warm objects in darkness or clutter, with visible-light detail such as edges and textures. Methods include convolutional and Transformer networks, frequency- and wavelet-based fusion, registration for misaligned sensors, and newer diffusion or prompt-guided models that adapt to scene conditions. Researchers assess fused-image quality on benchmarks and test whether fusion improves tasks such as object detection and semantic segmentation. A central challenge is balancing complementary information without introducing artifacts, especially when sensors are misaligned or conditions degrade the images; human-aligned evaluation and reliable gains across tasks remain open concerns.

Latest papers

CardsList

41 papers6 papers in the last four weeks, up 100% on the four weeks before. 0.0% of all new papers.

Nov 2, 2026

ProtoFusion: Protocol-Guided Prompt Refinement for Generative Infrared and Visible Image Fusion

Infrared and visible image fusion integrates complementary information from both modalities to provide a more comprehensive representation of the scene. We propose ProtoFusion, a protocol-guided generative framework for infrared and visible image fusion. Instead of relying on fixed fusion operators, hand-crafted objectives, or manually specified one-shot prompts, ProtoFusion formulates fusion as protocol-constrained generation. It represents fusion intent with a structured protocol, consisting of environment state cards, grounding principles, and a style profile, which explicitly specifies which evidence should be preserved or suppressed and which properties the fused result should satisfy. To optimize this protocol, we introduce protocol-guided prompt refinement, a closed-loop learning framework that iteratively performs generation, evaluation, and revision under multimodal evidence. Once learned, the optimized protocol is fixed at inference time and directly guides fusion generation. To address residual source inconsistencies, we further incorporate structured belief construction, which organizes modality reliability, scene evidence, and cross-modal conflicts into a structured representation for conservative correction after generation. Extensive experiments on standard benchmarks show that ProtoFusion improves quantitative performance, achieves favorable results in aesthetic evaluation, and produces visually coherent fusion outputs. Project Page: {https://erfect2020.github.io/ProtoFusion/}.

Pengwei Liang, Junjun Jiang, Junhui Hou

Department of Computer Science, City University of Hong Kong, Hong Kong SAR, China · School of Computer Science and Technology, Harbin Institute of Technology, Harbin, 150001, China

Infrared-Visible Image FusionPrompt OptimizationReliability-Aware Multimodal Fusion
AskPDF
Nov 1, 2026

NatureFusion: Rethinking Infrared and Visible Image Fusion as Natural Complementary Enhancement

In the absence of ground truth, most existing infrared and visible image fusion methods pursue fidelity to the source images and directly superimpose two physically distinct modalities. As a result, salient infrared targets appear as bright patches of thermal radiation pasted onto the scene. These patches obscure the original color and texture of the targets, so the fused images look unnatural to human observers and deviate from the data distribution learned by vision models pretrained on visible images. Therefore, we redefine infrared and visible image fusion as natural complementary enhancement rather than source fidelity, and propose NatureFusion, which exploits cross-modal complementary cues to infer a clear and natural visible-style image from low-quality visible and infrared image pairs. Specifically, to address the lack of ground truth, we employ a generative foundation model as the teacher to directly generate visually natural fused images for supervision. Furthermore, to fully distill the teacher's generative capability while reducing computational cost, we fine-tune a pretrained generative model with conditional flow matching through a light-weight fusion adapter. Last but not least, to further improve downstream performance without sacrificing natural appearance, we freeze the fusion adapter and introduce a detector-supervised task adapter that strengthens the complementary contribution of infrared. Extensive comparisons with seven multi-modal fusion and seven single-modality enhancement methods demonstrate that NatureFusion generates visually natural fused images with the best perceptual quality. Detection and segmentation models pretrained on visible images achieve substantially higher accuracy on our fused images than on those of any competing method.

Jianfeng Ding, Linfeng Tang, Hao Zhang +6

the National Engineering Research Center for Multimedia Software (NERCMS), School of Computer Science, Wuhan University, Wuhan 430072, China · the School of Robotics, Wuhan University, Wuhan 430072, China · the Electronic Information School, Wuhan University, Wuhan 430072, China +1

Infrared-Visible Image FusionMulti-Source Image Fusion
AskPDF
Oct 27, 2026

TriDifNet: A Unified Three-Task Diffusion Network for End-to-End RGB-T Perception

Infrared--visible (IR--VIS) multi-modal perception supports robust scene understanding under challenging illumination. Infrared and visible images are often misaligned due to platform motion, viewpoint changes, and modality-dependent distortions, causing registration errors to propagate into subsequent fusion and detection stages. We propose TriDifNet, a unified three-task diffusion network for end-to-end RGB-T perception. To the best of our knowledge, TriDifNet is the first diffusion-based network that jointly optimizes registration, fusion, and detection for misaligned RGB-T pedestrian perception. TriDifNet consists of three diffusion-based modules, i.e., the Registration Diffusion Module (RegDM), the Fusion Diffusion Module (FusDM), and the Detection Diffusion Module (DetDM). RegDM aligns the unaligned infrared image to the optical reference by estimating a dense deformation field. FusDM performs residual conditional diffusion to generate a detection-oriented fused representation with optical structural cues and complementary infrared saliency. DetDM formulates pedestrian detection as an iterative denoising process in bounding-box space. To couple the three tasks, TriDifNet introduces a closed-loop optimization mechanism, where detection semantics guide fusion and registration, and the fusion result further regularizes geometric alignment. A stage-wise progressive training strategy and a lightweight Nash multi-task learning (Nash-MTL)-based late-stage refinement are further adopted to balance registration, fusion, and detection optimization. Extensive experiments on the KAIST multispectral pedestrian benchmark show that TriDifNet achieves superior downstream detection performance under misaligned RGB-T settings. Against strong system-level competitors, TriDifNet achieves 65.44% Recall, 57.55% mAP50_{50}, and 26.83% mAP50:95_{50:95}.

Qianchao He, Jing Liu, Yunhao Li +1

School of Automation Science and Engineering, Faculty of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an 710049, China · School of Electrical and Control Engineering, Xi’an University of Science and Technology, Xi’an 710054, China

Object DetectionInfrared-Visible Image FusionCross-Modal Image Registration
AskPDF
Oct 21, 2026

Early versus Late RGB-Infrared Fusion for Counter-UAV Detection: A Comparative Study

This paper presents a novel strategy for UAV detection through visible (RGB) and infrared (IR) fusion, in order to improve accuracy and precision in harsh environments and weather conditions. In this research, two opposite approaches have been studied and implemented: the first concerns image-level (early) fusion, which merges two different spectra into a single image to which one detector is applied. On the other hand, the second approach, called decision-level (late) fusion, focuses on blending together the output bounding boxes of two different detection pipelines, aiming to create a common result that can be applied to both spectra simultaneously. Both strategies have been extensively tested in a dedicated environment, which included alternating low and high brightness scenes, different backgrounds (sky and ground), distances (5, 25 and 60 m), altitudes and zoom levels (1X, 2X and 4X). These tests allowed for the assessment of actual detection capabilities and system reliability in real-world applications. Furthermore, the two fusion strategies were evaluated and compared over a set of aligned images in order to highlight the better approach of the two. The results indicate that early fusion achieves better raw performance, since it incorporates the thermal information from the beginning of the training process. However, late fusion presents an innovative and useful detection workflow that can be applied in cases of sensor or frame misalignment. Both positive and negative aspects have also been pointed out and discussed, in order to give a more comprehensive overview. Finally, the paper outlines implementation considerations and practical recommendations based on operational priorities. Overall, the findings contribute to the research in RGB-IR fusion modalities, with the goal of enhancing UAV detection capabilities.

Alberto De Zan, Denis Tavaris, Gian Luca Foresti +1

Department of Mathematics, Computer Science, and Physics, University of Udine, Via delle Scienze 206, 33100 Udine, Italy

UAV DetectionInfrared-Visible Image FusionMultimodal Object Detection
AskPDF
Oct 19, 2026

Multi-View Structural Representations for Physically-Consistent TeX Generation

Infrared and visible images provide complementary cues for low-light image generation, and physics-based methods can better exploit this complementarity. However, existing physics-based methods still rely heavily on paired data. To address these limitations, we propose Multi-View Routing Mixture-of-Experts (MVR-MoE), a framework for physically consistent TeX generation that learns multi-view structural representations. MVR-MoE mines intrinsic structural cues from infrared and visible images to organize them into multi-view representations, while the proposed multi-view hyper connections expand the feature space and preserve both shared evidence and modality-specific details. A hierarchical routing mechanism further regulates feature utilization at both the structural-view level and the expert level, enabling adaptive exploitation of heterogeneous cues. The predicted temperature, emissivity, and texture maps are constrained by the Stefan-Boltzmann law to ensure physically consistent. Extensive experiments on challenging datasets demonstrate that MVR-MoE achieves superior robustness and discriminability while remaining lightweight.

Daoheng Li, Jianjun Qian*, Mengkai Yan +1

Nanjing University of Science and Technology, Nanjing, Jiangsu, China · Hohai University, Nanjing, Jiangsu, China

Image GenerationPhysics-Informed Generative ModelingMulti-View Feature FusionInfrared-Visible Image FusionMulti-View Image Synthesis
AskPDF
Oct 16, 2026

PCDFusion: Proposal-Context-Detail Bayesian Rendering for Infrared-Visible Image Fusion

Infrared-visible image fusion (IVIF) aims to combine visible and thermal infrared observations for human perception and downstream vision tasks. Visible and thermal infrared observations are physically complementary but unreliable in different ways. Existing deep IVIF methods mainly improve feature interaction and reconstruction, but often lack an explicit local rule for deciding when visible appearance should dominate and when thermal responses should contribute. To address this issue, we reformulate IVIF as local Bayesian posterior rendering, where fusion is controlled by posterior action weights rather than direct modality mixing. In this paper, we propose Proposal-Context-Detail Fusion (PCDFusion), a visible-anchored framework that implements this formulation through three rendering stages. PCDFusion preserves potentially useful thermal responses, updates exposure-degraded visible regions, and renders reliable band-limited infrared structures into the final luminance. Extensive experiments on public IVIF benchmarks show that PCDFusion improves fusion quality, target recovery, and downstream detection and segmentation performance under challenging illumination. The code is available at https://github.com/BIT-RuiLiu/PCDFusion.git.

Rui Liu, Lin Gu, Hesong Li +1

School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China · Research Institute of Electrical Communication, Tohoku University, Sendai 980-8577, Japan

Infrared-Visible Image FusionMulti-Source Image FusionThermal Imaging
AskPDF
Oct 7, 2026

CRT-HMAR: Causal Requirement Tracing-Guided Hierarchical Multi-Agent Regulation for Open-Task-Aware Infrared-Visible Image Fusion

Infrared and visible (IR-VIS) image fusion integrates complementary multimodal information into a single fused image to support downstream vision tasks. However, existing methods are typically tailored to seen tasks within a fixed task set and struggle to generalize to unseen tasks, which restricts their applicability in real-world open-task scenarios. To address this issue, this paper proposes CRT-HMAR, a Causal Requirement Tracing-Guided Hierarchical Multi-Agent Regulation Framework for open-task-aware IR-VIS image fusion. CRT-HMAR introduces a Causal Requirement Tracing Task Localization mechanism, which actively intervenes in key image information and observes task-network response variations to map task-specific semantic preferences into image-level causal requirement maps. Based on these maps, a requirement analysis agent aggregates task-specific requirement knowledge to adaptively guide requirement-customized image fusion. Moreover, CRT-HMAR incorporates History-Analysis Multi-Objective Balancing and Task-Level-Correction Conflict Mitigation mechanisms, jointly constructing a hierarchical regulation chain of "requirement interpretation - task balancing - conflict mitigation". Through multiple collaborative agents, CRT-HMAR dynamically regulates key processes including open-task requirement modeling, multi-task balanced optimization, and gradient conflict mitigation. Extensive experiments on open-task scenarios involving five downstream tasks demonstrate that CRT-HMAR significantly improves generalization to unseen tasks while maintaining the performance and balance of seen tasks. Overall, CRT-HMAR shifts IR-VIS image fusion from task-oriented modeling toward requirement-oriented modeling, promoting its extension from closed-task settings to real-world open-task scenarios.

Zengyi Yang, Shuai Yuan, Zhong-Cheng Wu +3

School of Instrument Science and Opto-electronics Engineering, Hefei University of Technology, Hefei 230009, China · Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China

Infrared-Visible Image FusionMulti-Task Optimization
AskPDF
Sep 25, 2026

SAGE: Source-Anchored Guidance via Frequency Equalization for Hierarchical RGB-T Alignment and Fusion

Spatial misregistration and cross-modal discrepancies often cause ghosting, structural blurring, and content imbalance in RGB-T fusion. Existing methods typically decouple appearance adaptation, geometric alignment, and information fusion, limiting dependency propagation across stages. We propose Source-Anchored Guidance via Frequency Equalization for Hierarchical RGB-T Alignment and Fusion (SAGE), a unified framework integrating frequency equalization, hierarchical alignment, and subband fusion. SAGE employs invertible joint encoding and source-specific low-frequency modulation to derive structural and gain guidance while preserving source information. Hierarchical frequency collaborative alignment estimates global affine geometry from low-frequency approximations and transfers geometric and contextual cues to high-frequency correlation reasoning for reliability-aware residual refinement. Guided subband fusion jointly aggregates the aligned frequency coefficients under propagated source and alignment guidance, coordinates complementary low- and high-frequency information, and reconstructs the fused image through the inverse wavelet transform. Extensive experiments on RGB-T datasets with real-world and synthetic misalignments demonstrate consistently competitive performance in alignment and fusion, validating the effectiveness of source-anchored guidance for weakly registered RGB-T images.

Timing Li, Yiming Sun, Boan Tao +3

Multi-Source Image FusionInfrared-Visible Image FusionCross-Modal Image RegistrationFrequency-Domain Image Processing
AskPDF
Sep 11, 2026

Aligned Radiometric RGB-Thermal Fusion for UAV Facade Anomaly Screening

Unmanned aerial vehicle facade inspection can combine red, green, and blue (RGB) imagery with thermal measurements to screen surface and subsurface anomalies. However, geometric discrepancies between the sensors and thermal image rendering can obscure spatial correspondence and weak temperature contrasts. This article presents a sensor-level pipeline comprising per-sensor correction, RGB-to-thermal registration, common-support cropping, and signed local contrast encoding of 16-bit radiometric measurements. The encoding preserves the distinction between locally hotter and colder regions and supplies the fourth input channel of a compact single-stream detector. We introduce M3T, a dataset of 674 paired RGB and radiometric thermal samples from five facade-inspection projects covering eight component and anomaly categories. The median residual registration error is 3.384 pixels, and a controlled-displacement analysis characterizes how the local contrast response changes under controlled displacement. Project-grouped four-fold evaluation yields mean average precision of 0.168 over intersection-over-union thresholds from 0.5 to 0.95, using 28.50 billion floating-point operations per image. A separate single-split ablation shows improved delamination detection over RGB-only and alternative thermal inputs, although aggregate accuracy does not improve over RGB alone. Evaluation on RGBT-Tiny shows mixed performance with rendered thermal imagery. These results characterize the category-specific benefits and limitations of aligned radiometric contrast for compact facade screening.

Yuan Yang, Shulei Li, Haobo Liang

Hong Kong Center for Construction Robotics, Hong Kong SAR, China

Thermal ImagingInfrared-Visible Image FusionMultispectral Object DetectionCross-Modal Image RegistrationUAV Remote SensingAnomaly Detection
AskPDF
Sep 11, 2026

RoES: Rotational Equivariant Selective-frequency Fusion for Multimodal Images

Infrared-visible image fusion facilitates robust multimodal perception by integrating complementary textural nuances from visible sensors with thermal signatures from infrared systems. Due to the task's inherently ill-posed nature, existing methods heavily rely on structural priors but typically enforce rotation equivariance uniformly across all features. Such a holistic approach overlooks a critical distinction where low-frequency shared structures strictly adhere to equivariant constraints while high-frequency modality-specific details require greater flexibility to preserve unique information. To bridge this gap, we propose RoES, a Rotational Equivariant Selective-frequency fusion network. Instead of employing static decomposition, we introduce a trainable rotation-enhanced updater/predictor module to dynamically decouple low- and high-frequency components. The resulting representations are then processed through a dual-branch fusion module tailored for spectral consistency. Specifically, a rotation-equivariant Mamba is employed to capture long-range structural dependencies in the low-frequency domain, while a polar spectral attention-based Dual-Fourier block refines high-frequency details under explicit low-frequency guidance. Extensive experiments demonstrate that RoES consistently achieves state-of-the-art performance in both fusion quality and downstream object detection, establishing a robust solution for multimodal fusion by reconciling frequency-selective features with equivariant constraints. The source code is available at https://github.com/BryceLosky/RoES-Fusion.

Jiabao Wang, Wenjian Liu, Yaoming Cai +5

Zhongnan University of Economics and Law Wuhan, Hubei, China · City University of Macau Macau, China · University of Illinois Chicago Chicago, Illinois, United States +3

Multi-Source Image FusionInfrared-Visible Image FusionEquivariant Neural NetworksFrequency-Domain Image ProcessingMultimodal Fusion
AskPDF
Sep 1, 2026

IT-TextFusion: Iterative Text-Image Interaction with Text-Guided Residual Refinement for Degradation-Aware Image Fusion

Text-guided image fusion has recently emerged as an effective paradigm for integrating multi-modal information while enabling flexible and task-oriented fusion control. However, existing text-guided fusion methods often rely on shallow semantic-visual interaction and limited attention mechanisms, which restrict their ability to robustly handle complex degradations and fully exploit textual guidance. In this paper, we propose an iterative text-guided image fusion framework that incorporates text-conditioned feature interaction across multiple fusion and refinement stages. The proposed method integrates deepest-level Cross-Attention, multi-scale Cross-Gate Fusion, and stage-specific text-conditioned modulation, allowing the global text embedding to condition hierarchical feature fusion and residual refinement. By repeatedly injecting the pooled text embedding across hierarchical decoder and refinement stages, the proposed framework provides degradation-aware global semantic conditioning while preserving complementary information from the visible and infrared modalities. Experiments on several benchmark datasets show that the proposed method improves several information-preservation and perceptual-quality metrics, while exhibiting metric-dependent trade-offs on some datasets.

Siyang Liu, Peiyi Zhou, Tianle Jin +3

Chair of Robotics, Artificial Intelligence and Real-time Systems, Technical University of Munich, Munich, Germany · School of Automation, Southeast University, Nanjing, China

Multi-Source Image FusionInfrared-Visible Image FusionMulti-Scale Feature FusionCross-Modal Attention
AskPDF
Aug 29, 2026

SGPDFuse: Semantically-Guided Physics-Disentanglement General Multi-Modal Image Fusion

Multimodal image fusion (MMIF) aims to integrate complementary sensor data into a single representation that preserves intrinsic scene reality while eliminating environmental interferences. Most existing approaches rely on blind feature aggregation, which excels at signal accumulation but fails to distinguish essential content from physical degradations. We propose SGPDFuse, which bridges this gap by mapping inputs into a physics-disentangled structural representation via a Semantic-Physical Parametric Bridge (SPPB) built on pretrained vision foundation models, utilizing the Intrinsic-Variation principle to decouple invariant scene attributes from transient environmental factors. To guide this decomposition, we introduce a Semantic Alignment mechanism: we explicitly anchor the fused representation to salient semantic features in the same foundation model feature space via cosine similarity to preserve critical targets, while enforcing physical texture fidelity through Gram-matrix regularization to strictly eliminate unnatural artifacts. Extensive experiments demonstrate that SGPDFuse achieves state-of-the-art performance across infrared-visible, multi-focus, and multi-exposure benchmarks using a single architecture.

Haozhen Wei, Chengjun Jiang, Yutong Guo +4

Dalian University of Technology Dalian, Liaoning, China · City University of Hong Kong Hong Kong, China · Zhejiang University Hangzhou, Zhejiang, China

Multi-Source Image FusionInfrared-Visible Image Fusion
AskPDF
Aug 22, 2026

UniDiffFusion: A Unified Diffusion Framework for Multi-Task and Degradation-Robust Image Fusion

General image fusion aims to integrate complementary information from multiple source images, but existing methods often rely on task-specific models and struggle to maintain robust performance under diverse degradation conditions. In this paper, we propose UniDiffFusion, a unified diffusion framework for multi-task and degradation-robust image fusion. UniDiffFusion leverages the strong generative prior of a pretrained diffusion model to establish a shared fusion backbone across heterogeneous fusion tasks, while introducing task- and degradation-aware conditional adaptation to accommodate their distinct information-selection requirements. Specifically, we employ task prompt modulation to progressively adapt the shared diffusion representations to different fusion objectives, and develop a degradation prompt router to dynamically retrieve degradation-aware priors and restore corrupted source features before fusion. Furthermore, an application prompt bank is introduced to incorporate task-oriented semantic guidance for downstream applications, such as object detection and semantic segmentation, without altering the shared fusion and restoration pathways. The proposed framework is trained in a progressive manner to decouple fusion learning, degradation-aware restoration, and application-specific adaptation, thereby reducing interference among heterogeneous objectives. Extensive experiments on visible-infrared, multi-exposure, and multi-focus image fusion demonstrate that UniDiffFusion achieves superior fusion quality and robustness under both clean and degraded conditions. Moreover, UniDiffFusion consistently improves downstream detection and semantic segmentation performance, demonstrating its effectiveness as a unified diffusion framework for both perceptual fusion and task-oriented vision.

Xingxin Xu, Siqi Zhao, Xin Li +3

School of New Media and Communication, Tianjin University · School of Artificial Intelligence, Tianjin University · School of Automation, Southeast University +1

Multi-Source Image FusionInfrared-Visible Image FusionMultimodal Fusion
AskPDF
Aug 13, 2026

P2Fusion: Prompt-based Progressive Infrared-Visible Image Fusion via Dual-Prior Distillation

Infrared-visible image fusion (IVIF) is pivotal for multimodal perception, yet reconciling the inherent information disparity between thermal and textural features remains a fundamental challenge. Existing prior-guided methods often rely on static constraints that induce optimization conflicts or utilize extrinsic semantic priors from large-scale foundation models (e.g., CLIP/DINO), which frequently fail to exploit the intrinsic modality characteristics essential for high-fidelity fusion. To address these issues, we propose P2Fusion, a prior-guided distillation-based framework that reformulates IVIF via dual intrinsic prompts. Instead of imposing hard-coded penalties, we distill image-intrinsic priors, thermal saliency and spatial quality, into learnable dynamic regulators. Specifically, a Teach-to-Fuse mechanism provides dual-granularity progressive guidance, coupled with a Gated Dynamic Expert Recalibration (GDER) module for decoupled feature refinement. This design enables the network to adaptively mediate modal competition through expert specialization. Extensive experiments demonstrate that P2Fusion achieves state-of-the-art performance across five mainstream datasets. Notably, our framework demonstrates consistent performance advantages in fusion quality, achieving state-of-the-art results in 14 out of 20 key evaluation metrics across 5 benchmarks. Furthermore, it effectively contributes to the robustness of downstream perception, such as +3.2% mAP on MSRS, +0.5% mAP on M3FD and +0.9% mAP on DroneVehicle for object detection. Our code will be available at https://github.com/YiShi99/P2Fusion

Yi Shi, Huichao Xie, Yuqing Wang +7

Northwestern Polytechnical University, Xi’an, China · Hefei University of Technology, Hefei, China · Rocket Force University of Engineering, Xi’an, China +1

Multi-Source Image FusionInfrared-Visible Image FusionMultimodal Fusion
AskPDF
Aug 2, 2026

Ranking Infrared-Visible Fusion the Way Humans Do: A Learned Pairwise Preference Measure

Human pairwise comparison provides a direct basis for perceptual infrared-visible image fusion assessment, but dense annotation becomes costly as method pools grow. We present the Learned Perceptual Image Fusion Measure (LPIFM), among the earliest learned fusion assessors trained directly on dense human A/B/Tie comparisons. LPIFM jointly examines both source images and both fused candidates, combining a shared hierarchical encoder, triadic interaction, and a tie-aware objective to predict comparative preference and perceptual indifference. We construct and publicly release all 6,300 unordered comparisons among 25 methods on 21 VIFB scenes, collected through blinded, randomized annotation and expert adjudication. Across four VIFB evaluation settings, LPIFM achieves 79.2-84.0% agreement with human pairwise judgments and Spearman correlations of 0.941-0.977 with human-derived method rankings. On full method pools, accuracy exceeds the strongest of 19 conventional metrics by 16.3-21.1 pp. Consistency diagnostics show 99.98-100% candidate-swap agreement and no observed decisive preference cycles. External experiments on EVAFusion further demonstrate rapid adaptation to a different fusion-evaluation preference protocol. After only three epochs of fine-tuning, LPIFM surpasses all 19 conventional metrics in accuracy, macro-F1, and ranking correlation. LPIFM provides a scalable instrument for human-aligned fusion assessment, with the preference corpus, model weights, and code publicly available.

Haoran Liu, Mingzhe Liu, Peng Li +1

School of Artificial Intelligence and Electronic Engineering, Sichuan Technology and Business University, Chengdu 611745, China · College of Nuclear Technology and Automation Engineering, Chengdu University of Technology, Chengdu 610059, China · National Synchrotron Radiation Laboratory, University of Science and Technology of China, Hefei 230029, China

Pairwise Preference LearningMulti-Source Image FusionPairwise ComparisonInfrared-Visible Image FusionImage Quality Assessment
AskPDF
Aug 1, 2026

Unleashing the Power of Text: Text-Guided Flow Matching for Image Fusion under Complex Degradations

Infrared-visible image fusion under realistic degradation scenarios is a challenging task, as degradations not only cause a loss of reliable modality-specific information in observed images but also hinder the fusion process. Recent studies indicate that text can provide prior information about degradation characteristics, complementing the limited evidence available from corrupted input images and facilitating fusion. However, existing methods typically inject fixed global text representations into visual features, making it difficult for textual guidance to adapt to spatially varying degradations, local structures, and thermal saliency. To this end, we propose TGFusion, a text-guided latent-space flow matching framework that unifies degradation suppression and cross-modal fusion. TGFusion encodes task, degradation, and generation cues into structured prompts. To fully exploit these priors, we design a Prompt-conditioned Multi-stream Joint Flow Transformer that represents text as an independent semantic stream alongside fusion, visible, and infrared streams. Joint attention enables token-level bidirectional interaction and layer-wise updating among semantic and visual representations, allowing degradation semantics to dynamically guide reliable information selection and fusion latent generation. Extensive experiments on public benchmarks and complex degradation scenarios demonstrate that TGFusion achieves superior or competitive performance in perceptual quality, image naturalness, structural-detail preservation, and infrared-saliency retention, while remaining robust across diverse single and compound degradations.

Axi Niu, Jieheng Li, Kang Zhang +3

School of Computer Science, Northwestern Polytechnical University, Xi'an, China · School of Computer Science, Northwestern Polytechnical University, Xi’an, Shaanxi, China · School of Electrical Engineering, KAIST, Daejeon, Republic of Korea +2

Flow MatchingMulti-Source Image FusionInfrared-Visible Image FusionMultimodal Fusion
AskPDF
Jul 31, 2026

Toward Robust and 3D-Aware RGB-NIR Imaging in the Dark

Robust low-light imaging remains challenging for the community. Recent studies have explored fusing Near-Infrared (NIR) with noisy RGB to achieve improved enhancement, yet most methods depend on carefully curated training data pairs, with limited robustness under different scenarios. This paper offers a new perspective for RGB-NIR low-light imaging by incorporating 3D-aware neural modeling. Without using clean RGB supervision, a powerful model can be optimized to implicitly fuse extremely noisy RGB observations with NIR cues in 3D space, effectively recovering clean RGB images. The proposed model obviates the requirement for clean RGB data collection, generalizes across different noise levels. Extensive evaluations on synthetic and real data demonstrate its superiority. Codes available: https://github.com/MyNiuuu/3DarkFusion

Muyao Niu, Mingze Ma, Yifan Zhan +5

The University of Tokyo Tokyo, Japan · Adelaide University Adelaide, Australia · School of Artificial Intelligence (SAI) Shanghai Jiao Tong University Shanghai, China +1

Image EnhancementInfrared-Visible Image FusionLow-Light Image Enhancement
AskPDF
Jul 27, 2026

BeyondFusion: Self-Aligned Latent Diffusion for Calibration-Free Infrared Super-Resolution and Infrared-Visible Fusion

Mobile infrared-visible imaging typically pairs a compact infrared sensor with a high-resolution visible camera for complementary perception. While cross-sensor misalignment caused by different optics, viewpoints, fields of view, and exposure timings hinders practical deployment. In this paper, we propose BeyondFusion, a unified latent diffusion framework for calibration-free visible-guided infrared super-resolution and infrared-visible fusion tasks. The proposed framework supports both task-specific training and joint training where two tasks are optimized and executed as two readouts of the same generative process. Instead of relying on explicit registration or geometric warping, BeyondFusion introduces a cross-modal self-aligning (CMSA) module into the denoising U-Net. CMSA reorganizes infrared and visible latent tokens into a shared attention space to learn content-adaptive cross-modal correspondence during the denoising process. Together with misalignment augmentation module, the model is facilitated to exploit visible structural and semantic cues while preserving thermal consistency, enabling high-frequency infrared reconstruction and informative fused-image generation under uncalibrated conditions. Extensive experiments on public benchmarks and a mobile infrared-visible imaging system show strong performance across aligned inputs, low-resolution infrared observations, synthetic misalignments, and real mobile captures with unsynchronized sensors. Ablation studies, unified training analysis, and downstream pedestrian detection further validate the effectiveness of BeyondFusion for calibration-free multimodal imaging.

Minchong Chen, Xiaoyun Yuan, Minyu Cao +4

MoE Key Lab of Artificial Intelligence, Institute of AI, School of Computer Science, Shanghai Jiao Tong University, Shanghai, China · Tsinghua University, Beijing, China · Institute of Modern Optics, Nankai University, Tianjin, China

Diffusion Models for Image RestorationMulti-Source Image FusionInfrared-Visible Image FusionCross-Modal AlignmentRemote Sensing Infrared Image Super-ResolutionImage Super-Resolution
AskPDF
Jul 26, 2026

ConFusion: Continuous Fusion Space Learning for Fine-Grained Controllable Infrared and Visible Image Fusion

Controllable infrared-visible image fusion aims to integrate complementary thermal and structural information with flexible region-aware modulation, producing fused images that adapt to diverse user requirements and downstream tasks. However, existing methods typically rely on predefined discrete control conditions, leading to a sparse space that fails to support fine-grained modulation demands. To address this, we propose ConFusion, a novel framework that learns the continuous fusion space via Gaussian-conditioned spatial-aware modulation, enabling instance-level fine-grained controllable infrared and visible image fusion. ConFusion employs a dual-branch architecture to disentangle modality-invariant and modality-specific representations under joint reconstruction and text-guided semantic alignment. Gaussian-conditioned instance modulation variables coupled with Grounded SAM-based instance masks guide instance-level fine-grained modulation through the Mask-Guided Specific Feature Modulator, while the Text-Driven Invariant Feature Enhancer improves semantic consistency and enhances fusion. During inference, the multimodal large language model parses user intents into instance-level modulation variables to guide image fusion. Extensive experiments show that ConFusion achieves state-of-the-art performance across multiple metrics in both fusion quality and downstream tasks, while supporting fine-grained controllable image fusion. Our code is available at https://github.com/HeyufeiAnto/Confusion

Guo Yurong, He Yufei, Li Yonghao +3

North China Electric Power University Baoding, Hebei, China · Beijing University of Posts and Telecommunications Beijing, China

Multi-Source Image FusionInfrared-Visible Image FusionMultimodal Fusion
AskPDF
Jul 22, 2026

Current Injection Spiking Neural Network for Infrared and Visible Image Fusion

Infrared and visible image fusion (IVIF) integrates the complementary information of two modalities into a single image with richer scene content. While existing methods are largely built on artificial neural networks (ANNs), which densely compute over all activations, spiking neural networks (SNNs) communicate through sparse binary spikes and compute only where and when a spike occurs, offering a route to more energy-efficient fusion. However, directly applying SNNs to IVIF creates a fundamental tension: cross-modal fusion relies on fine-grained responses from both modalities, whereas binary spikes can discard complementary cues that remain below the firing threshold. The membrane potential retains these subthreshold responses before firing, letting both modalities jointly shape the output when integrated at this stage. Building on this, we propose CIS-Fuse, a spiking network that performs cross-modal fusion directly at the membrane-potential level. At its core is the current injection spiking (CIS) operator, which injects one modality as a gated auxiliary current into the driving neuron of the other, so the two integrate before spike firing, with a per-channel learnable injection strength that adaptively regulates the modulation magnitude. Building on CIS, we construct a bidirectional cross-modal fusion (BCMF) module and deploy it on a dual-branch architecture with asymmetric stacking depths, where the two branches develop a clear functional specialization. Extensive experiments on four IVIF benchmarks and on downstream detection and segmentation show that CIS-Fuse achieves fusion quality on par with state-of-the-art ANN-based methods while inheriting the energy efficiency of spike-based computation, with roughly an order of magnitude lower inference energy than the similarly-sized ANN-based DCEvo. Code will be released upon publication.

Rui Zhao, Zhuoyuan Li, Wenrui Li +4

College of Computing and Data Science, Nanyang Technological University, Singapore · Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong · Department of Computer Science and Technology, Harbin Institute of Technology, Harbin, China +2

Efficient Neural Network InferenceMulti-Source Image FusionInfrared-Visible Image FusionSpiking Neural NetworksMultimodal Fusion
AskPDF
Jun 13, 2026

Text-Driven Fusion for Infrared and Visible Images: Achieving Image Scene Adaptation on Hyperbolic Space

Infrared and visible image fusion aims to integrate complementary modalities, while existing Euclidean methods impose rigid distance metrics that distort multi-modal interactions and parent-to-child semantic hierarchies. To overcome these limitations, we introduce a text-driven fusion framework empowered by hyperbolic manifold learning. During training, BLIP-extracted text prompts serve as topological anchors within the hyperbolic space, guiding vision-attribute alignment through hyperbolic embeddings that naturally accommodate varying semantic granularities. By exploiting the exponential volume growth dictated by the Poincaré ball's negative curvature, this approach seamlessly embeds hierarchical trees to encode coarse-to-fine semantics without metric saturation, while the vast peripheral space prevents texture distortion during cross-modal fusion. At inference, the fusion process autonomously adapts to input content using the learned text-attribute priors, completely eliminating the need for textual input. Experimental results show our method outperforms state-of-the-art approaches on benchmark datasets, with code available at https://github.com/Shaoyun2023/TEDFusion.

Huan Kang, Hui Li, Tianyang Xu +3

School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, China · Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, Guildford, UK

Multi-Source Image FusionInfrared-Visible Image FusionCross-Modal Alignment
AskPDF
Jun 6, 2026

FMRFusion: Frequency-Aware Multi-View Representation Learning for Heterogeneous Image Fusion

Infrared and visible image fusion aims to generate a composite image that retains significant target information and preserves detailed textures, integrating two heterogeneous modalities. Previous image fusion methods typically adopt a single-module stacking approach to extract features from the two modalities. However, these approaches may result in incomplete learning of their distinct characteristics, thereby limiting the fusion effectiveness and constrain ing robustness in real-world heterogeneous data scenarios. To address these challenges, we propose FMRFusion, a frequency-aware multi-view representation learning network for Heterogeneous Image Fusion. A Multi-Scale Struc tural Perception Module is introduced to effectively capture discriminative structures, extracting fine-grained local structures and essential contextual information. A bilinear frequency decomposition mechanism is employed to sepa rate features into high-frequency and low-frequency components, enabling joint modeling of local details and global representations across different frequency domains. Moreover, a Cross-View Complementary Interaction is incorpo rated to explicitly model and fuse the complementary characteristics between reflected light information and radiative intensity responses, facilitating effective cross-view interaction. We further improve the Performance of the fused results by flow matching, which progressively refines the fused features by learning the transformation from coarse data to high-quality representations. Extensive experiments conducted on multiple benchmark datasets demonstrate that FMRFusion achieves superior and consistent performance across a range of fusion tasks, especially in nighttime scenarios

Tao Zhoua, Yunlong Liu, Qinghui Chen +6

Control Science and Engineering, Shandong University, Jinan, 250061, China · Key Laboratory of Machine Intelligence and System Control, Ministry of Education, Jinan, 250061, China · School of Computer Science and Engineering, Macau University of Science and Technology, Macau, 999078, China +1

Multi-Source Image FusionInfrared-Visible Image FusionMulti-Scale Feature Fusion
AskPDF
May 14, 2026

Multimodal Object Detection Under Sparse Forest-Canopy Occlusion

Reliable detection of humans beneath forest canopy remains a difficult remote-sensing challenge due to sparse, structured, and viewpoint-dependent occlusion. This paper presents a multimodal proof-of-concept pipeline that integrates three complementary approaches: (i) experimental evaluation of LiDAR returns through vegetation to assess the feasibility of active sensing, (ii) visible--thermal image fusion using a multi-scale transform and sparse-representation framework to enhance human saliency, and (iii) synthetic-aperture image formation via Airborne Optical Sectioning (AOS) to suppress canopy clutter. A YOLOv5 detector is fine-tuned on the Teledyne FLIR thermal dataset and evaluated on thermal and fused imagery. Results show that the tested terrestrial LiDAR configuration provides limited penetration for object-level detection, while visible--thermal fusion improves target visibility in low-contrast scenes and AOS enhances ground-plane detection in synthetic forest imagery. The fine-tuned YOLOv5 achieves a mean average precision of ∼\sim0.83 on the top three FLIR classes. These findings establish an initial baseline for UAV-deployable search-and-rescue and surveillance systems operating in forested environments, and motivate future work on dedicated forest datasets and real-time multimodal integration.

Nitik Jain, Mangal Kothari

Robotics & AI, Johns Hopkins University, USA. This research was carried out entirely while the author was with the Department of Aerospace Engineering, IIT Kanpur. · This research was carried out entirely while the author was with the Department of Aerospace Engineering, IIT Kanpur. · Senior Principal Flight Control Engineer, ADASI, EDGE Group, Abu Dhabi, UAE. This research was carried out entirely while the author was with the Department of Aerospace Engineering, IIT Kanpur.

Infrared-Visible Image FusionObject DetectionUAV Remote Sensing
AskPDF
May 13, 2026

Uncertainty-aware Spatial-Frequency Registration and Fusion for Infrared and Visible Images

Infrared and Visible Image Fusion (IVIF) has shown promise in visual tasks under challenging environments, but fusion under unregistered conditions faces inherent misalignments. Current studies to solve them either predict the deformation parameters coarse-to-fine (i.e., coarse registration and fine registration) or estimate the deformation fields in multi-scales for registration. Though straightforward, they overlook the cumulative errors in registration, which contaminate the fusion stage and severely deteriorate the resulting images. We introduce the Spatial-Frequency Registration and Fusion (SFRF) framework, which incorporates uncertainty estimation and infrared thermal radiation distribution consistency into a unified pipeline to handle the error accumulation for robust registration and fusion across both spatial and frequency domains. Specifically, SFRF constructs a Multi-scale Iterative Registration (MIR) framework that iteratively refines the deformation field across scales, leveraging uncertainty estimation at each stage to mitigate error accumulation and enhance alignment accuracy dynamically. To ensure the accurate alignment of infrared thermal distributions during registration, thermal radiation distribution consistency is employed as a frequency-domain supervisory signal, promoting global consistency in the frequency domain. Based on the spatial-frequency alignment, SFRF further adopts a Dual-branch Spatial-Frequency Fusion (DSFF) module, which incorporates spatial geometric features and frequency distribution information to reconstruct visually appealing images. SFRF achieves impressive performance across diverse datasets.

Xingyuan Li, Haoyuan Xu, Xingyue Zhu +4

Dalian University of Technology · Northwestern Polytechnical University · Dalian Maritime University

Multi-Source Image FusionInfrared-Visible Image FusionFrequency-Domain Image ProcessingCross-Modal Image RegistrationDeformable Image Registration
AskPDF
May 7, 2026

Bringing Multimodal Large Language Models to Infrared-Visible Image Fusion Quality Assessment

Infrared-Visible image fusion (IVIF) aims to integrate thermal information and detailed spatial structures into a single fused image to enhance perception. However, existing evaluation approaches tend to over-optimize both hand-crafted no-reference statistics and full-reference metrics that treat the source images as pseudo ground truths. Recent IVIF reward-modelling efforts learn from human ratings but use scalar regression on aggregated scores, neither leveraging the reasoning of Multimodal Large Language Models (MLLMs) nor encoding per-image perceptual ambiguity in their supervision, but naively introducing MLLMs with discrete one-hot supervision likewise collapses fused images of similar quality into different rating levels. To address this, we introduce FuScore, which utilizes an MLLM to mimic human visual perception by producing continuous quality score, rather than discrete level predictions, enabling fine-grained discrimination among fused images of similar quality. We exploit the agreement among four IVIF-specific sub-dimensions to construct a per-image soft label whose sharpness reflects how consensual the overall judgment is. We further introduce a tripartite objective combining per-image distributional supervision, within-source-pair Thurstone fidelity for method-level ordering, and cross-source-pair Thurstone fidelity for scene-level ordering across scenes. Extensive experiments demonstrate that FuScore achieves state-of-the-art correlation with human visual preferences.

Yuchen Guo, Junli Gong, Yao Lu +3

Northwestern University · Northeastern University · University of Washington +2

Infrared-Visible Image FusionLearning to RankMultimodal Large Language ModelsImage Quality Assessment
AskPDF
May 7, 2026

Fusion in Your Way: Aligning Image Fusion with Heterogeneous Demands via Direct Preference Optimization

As a key technique in multi-modal processing, infrared and visible image fusion (IVIF) plays a crucial role in integrating complementary spectral information for visual enhancement and downstream vision tasks. Despite remarkable progress, existing methods struggle to flexibly accommodate heterogeneous demands. Achieving adaptive fusion that aligns with various preferences from both human and machine vision remains an open and challenging problem. To address this challenge, we propose DPOFusion, a direct preference optimization (DPO) framework integrating the property-aligned latent diffusion model (PALDM) and the preference-controllable latent diffusion model (PCLDM), enabling task-guided, preference-adaptive IVIF for both human and machine vision. The PALDM leverages a latent fusion prior and a joint conditional loss to generate diverse candidate fusion results with various properties. PCLDM is subsequently fine-tuned via instance direct preference optimization (IDPO), enabling direct control of the final fusion results with heterogeneous preference signals. Experimental results demonstrate that our framework not only attains precise preference alignment among humans, vision-language models, and task-driven networks, but also sets a new benchmark for adaptive fusion quality and task-oriented transferability.

Weijian Su, Songqian Zhang, Yuqi Han +3

School of Computer Science and Technology, Dalian University of Technology · Key Laboratory of Social Computing and Cognitive Intelligence (Dalian University of Technology), Ministry of Education · Institute of Image Processing and Understanding, North Minzu University

Diffusion Model AlignmentMulti-Source Image FusionInfrared-Visible Image FusionLatent Diffusion ModelsDirect Preference OptimizationMultimodal Fusion
AskPDF
May 7, 2026

Adding Thermal Awareness to Visual Systems in Real-Time via Distilled Diffusion Models

Purely RGB-based vision models often fail to provide reliable cues in challenging scenarios such as nighttime and fog, leading to degraded performance and safety risks. Infrared imaging captures heat-emitting sources and provides critical complementary information, but existing high-fidelity fusion methods suffer from prohibitive latency, rendering them impractical for real-time edge deployment. To address this, we propose FusionProxy, a real-time image fusion module designed as a fully independent, plug-and-play component with diffusion level quality. FusionProxy exploits two complementary statistics of a teacher sample ensemble: per-pixel variance in raw image space, used to weight pixel-level supervision, and per-pixel variance inside frozen foundation backbones, used to route feature-level alignment spatially. Once trained, FusionProxy can be directly integrated into any visual perception system without joint optimization. Extensive experiments demonstrate that our method achieves superior performance on static recognition tasks and significantly enhances robustness in dynamic tasks, including closed-loop autonomous driving. Crucially, FusionProxy achieves real-time inference speeds on diverse platforms, from high-end GPUs to commodity hardware, providing a flexible and generalizable solution for all-day perception.

Yuchen Guo, Junli Gong, Wenjun Dong +2

Northwestern University · Northeastern University · Hong Kong Baptist University +1

Multi-Source Image FusionInfrared-Visible Image FusionEdge InferenceAutonomous Driving PerceptionMultimodal Sensor Fusion
AskPDF
May 3, 2026

EAPFusion: Intrinsic Evolving Auxiliary Prior Guidance for Infrared and Visible Image Fusion

Infrared-visible image fusion aims to create an information-rich fused image by integrating the complementary thermal saliency from infrared sensing and fine textures from visible imaging. Such accurate fusion is essential for real-world perception applications in complex scenes, including nighttime autonomous driving, search and rescue, and surveillance, and can further benefit downstream tasks such as semantic segmentation. However, most existing fusion methods rely upon static trained weights that cannot adapt to scene-specific content at inference time, and often suffer from a granularity mismatch when coarse auxiliary semantics are injected, which makes it difficult to simultaneously highlight targets and preserve details. In this work, we propose EAPFusion to address these issues by using self-evolving intrinsic priors instead of relying on external auxiliary models. Concretely, EAPFusion maintains a compact set of intrinsic priors and progressively updates them across scales. These evolved priors are utilized to dynamically generate convolutional kernels, shifting the paradigm from fixed, pre-trained filters to instance-adaptive parameters via prior-conditioned dynamic convolution. Furthermore, we design a channel-level fusion module that shuffles and interleaves infrared and visible channels, applying local channel mixing to boost cross-modal complementarity. Experiments on different datasets, including cross-dataset evaluation and semantic segmentation, show that the proposed method achieves state-of-the-art quantitative and qualitative fusion results, and consistently boosts downstream performance. Code is coming soon.

Zhenyu Sun, Luobin Zhang, Axi Niu +2

Northwestern Polytechnical University, Xi’an, China

Dynamic ConvolutionMulti-Source Image FusionInfrared-Visible Image FusionMulti-Scale Feature FusionConvolutional Neural NetworksMultimodal Fusion
AskPDF
May 1, 2026

Combined Dictionary Unfolding Network with Gradient-Adaptive Fidelity for Transferable Multi-Source Fusion

Deep Unfolding Network-based methods have emerged as effective solutions for multi-source image fusion by combining model-driven iterative optimization with data-driven deep learning. However, most existing deep unfolding image fusion methods are derived from alternating minimization, which updates the features of different modalities separately. This design introduces considerable computational and memory overhead, limiting deployment on resource-constrained edge devices. To address this issue, we propose CDNet, a lightweight Combined Dictionary Unfolding Network for multi-source image fusion. Rather than introducing a new sparse coding prior or empirically compressing an existing fusion network, CDNet translates the unique-common decomposition prior of coupled dictionary learning into a structurally constrained joint unfolding architecture. The resulting CDBlock follows a block-sparse interaction topology and performs a model-derived joint update of common and modality-specific representations, thereby streamlining feature learning and improving efficiency.In addition, we design a compact High- and Low-frequency Image Fidelity loss for unsupervised training without ground-truth images. We evaluate CDNet on four tasks, including multi-exposure image fusion, infrared and visible image fusion, medical image fusion, and infrared and visible image fusion for semantic segmentation. Experimental results show that CDNet achieves competitive or superior fusion performance with high efficiency. For infrared and visible image fusion, CDNet outperforms competing methods on four of six metrics on the TNO dataset and five of six metrics on the RoadScene dataset. In particular, it surpasses the second-best method by 1.23 dB and 1.59 dB in PSNR on TNO and RoadScene, respectively.

Ge Luo, Jun-Jie Huang, Qi Yu +6

College of Computer Science and Technology, National University of Defense Technology, Changsha, China · College of Surveying and Geoinformatics, Tongji University, Shanghai, China · School of Computer and Information, Hefei University of Technology, Hefei 230009, China

Multi-Source Image FusionRepresentation LearningInfrared-Visible Image FusionDeep Unfolding
AskPDF
Apr 10, 2026

Customized Fusion: A Closed-Loop Dynamic Network for Adaptive Multi-Task-Aware Infrared-Visible Image Fusion

Infrared-visible image fusion aims to integrate complementary information for robust visual understanding, but existing fusion methods struggle with simultaneously adapting to multiple downstream tasks. To address this issue, we propose a Closed-Loop Dynamic Network (CLDyN) that can adaptively respond to the semantic requirements of diverse downstream tasks for task-customized image fusion. Specifically, CLDyN introduces a closed-loop optimization mechanism that establishes a semantic transmission chain to achieve explicit feedback from downstream tasks to the fusion network through a Requirement-driven Semantic Compensation (RSC) module. The RSC module leverages a Basis Vector Bank (BVB) and an Architecture-Adaptive Semantic Injection (A2SI) block to customize the network architecture according to task requirements, thereby enabling task-specific semantic compensation and allowing the fusion network to actively adapt to diverse tasks without retraining. To promote semantic compensation, a reward-penalty strategy is introduced to reward or penalize the RSC module based on task performance variations. Experiments on the M3FD, FMB, and VT5000 datasets demonstrate that CLDyN not only maintains high fusion quality but also exhibits strong multi-task adaptability. The code is available at https://github.com/YR0211/CLDyN.

Zengyi Yang, Yu Liu, Juan Cheng +3

Department of Biomedical Engineering, Hefei University of Technology · College of Automation, Chongqing University of Post and Telecommunications · Faculty of Information Engineering and Automation, Kunming University of Science and Technology

Infrared-Visible Image FusionMulti-Source Image FusionMultimodal FusionMulti-Task Learning
AskPDF