EvaNet: Towards More Efficient and Consistent Infrared and Visible Image Fusion Assessment

Chunyang Cheng; Tianyang Xu; Xiao-Jun Wu; Tao Zhou; Hui Li; Zhangyong Tang; Josef Kittler

cs.CVApr 3, 2026

EvaNet: Towards More Efficient and Consistent Infrared and Visible Image Fusion Assessment

Authors: Chunyang Cheng, Tianyang Xu, Xiao-Jun Wu, Tao Zhou, Hui Li, Zhangyong Tang, Josef Kittler

Organizations: C. Cheng and Z. Tang are with the Wuxi School of Medicine and also with the School of Food Science and Technology, Jiangnan University, Wuxi 214122, P.R. China. · T. Xu, X.-J. Wu, T. Zhou and H. Li are with the School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi 214122, P.R. China. · Josef Kittler is with the Centre for Vision, Speech and Signal Processing, University of Surrey, Guildford GU2 7XH, U.K.

Abstract

Evaluation is essential in image fusion research, yet most existing metrics are directly borrowed from other vision tasks without proper adaptation. These traditional metrics, often based on complex image transformations, not only fail to capture the true quality of the fusion results but also are computationally demanding. To address these issues, we propose a unified evaluation framework specifically tailored for image fusion. At its core is a lightweight network designed efficiently to approximate widely used metrics, following a divide-and-conquer strategy. Unlike conventional approaches that directly assess similarity between fused and source images, we first decompose the fusion result into infrared and visible components. The evaluation model is then used to measure the degree of information preservation in these separated components, effectively disentangling the fusion evaluation process. During training, we incorporate a contrastive learning strategy and inform our evaluation model by perceptual scene assessment provided by a large language model. Last, we propose the first consistency evaluation framework, which measures the alignment between image fusion metrics and human visual perception, using both independent no-reference scores and downstream tasks performance as objective references. Extensive experiments show that our learning-based evaluation paradigm delivers both superior efficiency (up to 1,000 times faster) and greater consistency across a range of standard image fusion benchmarks. Our code will be publicly available at https://github.com/AWCXV/EvaNet.

Figures & tables

Fig. 1: An illustration of the consistency and the efficiency issues (sub-figure (a) and (b)) of existing image fusion metrics. Image fusion evaluation relies heavily on traditional signal processing techniques ( e.g. , discrete cosine transform (DCT) and wavelet transform) or metrics adapted from other vision tasks. Without appropriate adjustment, these metrics lack consistency, i.e. , better visualisation cannot always correspond higher metric values. Moreover, the evaluation phase in current image fusion settings incurs high processing time costs. The proposed EvaNet addresses these issues effectively by significantly reducing the evaluation time and enhancing consistency.

Fig. 2: The speed (milliseconds per image) imbalance between inference and evaluation in image fusion. Traditional metrics rely on separate, computationally intensive procedures to perform different assessments, significantly slowing down the evaluation phase. In contrast, the proposed EvaNet generates multiple evaluation results simultaneously within a single forward pass, offering acceleration by a factor of up to 1000, and as will be demonstrated, also considerably improved metric consistency.

Category	IVIF Algorithm	IVIF Metric	MFIF Metric
Traditional	[ 19 ] , [ 20 ] , [ 21 ] …	[ 22 ] , Qabf, VIF…	SSIM, PSNR, …
CNN	[ 8 ] , [ 23 ] , [ 24 ] …	Unexplored	2024’ TPAMI [ 18 ]
Transformer	[ 25 ] , [ 26 ] , [ 27 ] …	-	-
Mamba	[ 9 ] , [ 28 ] …	-	-
Diffusion	[ 29 ] , [ 30 ] …	-	-

TABLE I: Statistics of recent Infrared and Visible Image Fusion (IVIF) and Multi-Focus Image Fusion (MFIF) algorithms and metrics.

Fig. 3: Overview of the proposed EvaNet framework. Our method replaces traditional image fusion assessment processes by a lightweight learning-based network to significantly improve evaluation efficiency. In addition, a divide-and-conquer strategy is used to disentangle and independently measure the information preserved from each source modality. The environment branch, as part of the three-branch design, introduces an adaptive penalty mechanism to mitigate modality imbalance in challenging fusion scenarios.

Fig. 4: The network architecture of the proposed EvaNet (see Sec. III-B ). The model consists of two main components. The left part shows the modality-specific decomposition, implemented using two lightweight information probes [ 11 ] . The right part illustrates the surrogate metric prediction process, consisting of three branches: two modality branches correspond to the infrared and visible inputs, while the environment branch, guided by a large language model (LLM), predicts an adaptive weighting factor ( $ENV$ ) to eliminate any modality imbalance during evaluation.

Fig. 5: An illustration of perceptual scene environment assessment. Left: some examples showing that the LLVIP-train dataset contains multiple image pairs with limited scene diversity, where foreground objects vary but the background remains consistent. Right: environment label generation process using a Generative Pre-trained Transformer (GPT) [ 45 ] , which estimates scene conditions such as illumination and obscuration to guide the training of the environment branch in EvaNet.

Fig. 6: An overview of the training strategy based on contrastive learning. The decomposed infrared or visible image (obtained from the fusion result, e.g. , $\hat{I}^{u}_{\textrm{vis}}$ ) and its corresponding original input ( $I^{u}_{\textrm{vis}}$ ) form a positive sample pair, guiding the model to learn the similarity range of relevant images. In contrast, the original image paired with an unrelated randomly selected image ( $I^{v}_{\textrm{vis}}$ ) forms a negative sample, encouraging the network to suppress metric predictions for unrelated content.

Setting	Traditional Metrics	EvaNet (Ours)
Platform	MATLAB	PyTorch
CPU	Intel i7-6850K	Intel i7-6850K
GPU	–	NVIDIA RTX 3090
Batch size	1	1
Image resolution	640 $\times$ 512	640 $\times$ 512
I/O time included	No	No

TABLE II: Comparison of runtime evaluation settings between traditional image fusion metrics and the proposed EvaNet. All experiments are conducted under an identical workstation.

Dataset	PSNR	CC	Qabf	VIF	SSIM	$\textrm{FMI}_{\textrm{pixel}}$	$\textrm{FMI}_{\textrm{dct}}$	$\textrm{FMI}_{\textrm{w}}$	Sum (8 metrics)	EvaNet (8 metrics)
LLVIP-test (part)	0.2s	1.0s	27.2s	330.4s	9.6s	516.1s	2770.4s	4039.4s	7694.3s ( $>$ 2 hours)	1.6s ( $<$ 1 min)
LLVIP-test (all)	2.9s	13.5s	341.3s	4560.6s	111.5s	8121.4s	38571.3s	51094.5s	102817.0s ( $>$ 24 hours)	10.8s ( $<$ 1 min)

TABLE III: The speed (in seconds) of generating the assessment scores using eight different metrics and our EvaNet, measured across part (250 pairs) and all fused images (3463 pairs) from the LLVIP dataset. ( Bold : Best performance)

Method	Setting	Manner	Efficiency	Model Size
Vanilla (N metrics)	Traditional	N-Exe.	30 s / img	-
TextAttn (N metrics)	Traditional	2N-Exe.	60 s / img	1552.67 MB
EvaNet (N metrics)	Leaning-based	1 Exe.	6 ms / img	1.15 MB

TABLE IV: A summary of the computational characteristics of existing image fusion evaluation methods when delivering

N

assessment metrics.

Fig. 7: Typical and zero-input decomposition results produced by the Information Probe. (a) and (b) show successful disentanglement of modality-specific content from two representative fused images with different degrees of thermal information preservation. (c) presents the zero-decomposition case, where an all-zero input yields two clean blank outputs without artefacts, verifying the robustness of the probe and its non-reliance on memorised patterns. At the right side of (a), we present intuitive prediction results of our EvaNet for 8 image fusion assessments, i.e. , VIF, Qabf, SSIM, CC, PSNR, $\textrm{FMI}_{\textrm{dct}}$ , $\textrm{FMI}_{\textrm{wave}}$ , and $\textrm{FMI}_{\textrm{pixel}}$ , for the decomposed image and the source input.

Setting	$\Delta$ vs. (a)	$\textrm{VIF}^{*}$	$\textrm{Qabf}^{*}$	$\textrm{SSIM}^{*}$	$\textrm{CC}^{*}$	$\textrm{PSNR}^{*}$	$\textrm{FMI}_{\textrm{d}}^{*}$	$\textrm{FMI}_{\textrm{w}}^{*}$	$\textrm{FMI}_{\textrm{p}}^{*}$
(a) Traditional	-	0.728	0.685	0.650	0.635	0.611	0.619	0.603	0.655
(b) Dec. only	2.27%	0.673	0.620	0.623	0.664	0.727	0.713	0.647	0.614
(c) Con. only	-0.33%	0.725	0.642	0.648	0.615	0.619	0.633	0.618	0.664
(d) Env. only	-1.37%	0.655	0.692	0.657	0.658	0.622	0.564	0.617	0.643
(e) w/o Env.	7.47%	0.710	0.682	0.717	0.724	0.716	0.691	0.677	0.639
(f) w/o Con.	-0.80%	0.587	0.605	0.683	0.665	0.730	0.625	0.608	0.614

TABLE V: Ablation experiments of the different components of the proposed EvaNet when applied to 8 different image fusion metrics. “

\Delta

vs. (a) ” indicates the average metric performance change compared to the setting a.

Fig. 8: The fusion results of two representative approaches A (SDNet) and B (ReCoNet). Below the fused images, we present the ablation results, highlighting the impact of the environment perception branch within our EvaNet. “X→Y” denotes the change in the corresponding value after enabling environment-aware evaluation. The proposed environment-aware design can effectively encourage and penalise results with or without an appropriate illumination condition. The environmental condition predicted by the perception branch is shown in each source image.

Fig. 9: Metric consistency scores obtained using environment labels generated by different large language models and a human-labeled baseline. More advanced LLMs yield higher overall consistency, while all LLM-based supervisions outperform human annotation.

Dataset	LLVIP Dataset			RoadScene Dataset			TNO Dataset
Metric	Vanilla	+TextAttn	+EvaNet	Vanilla	+TextAttn	+EvaNet	Vanilla	+TextAttn	+EvaNet
$\textrm{MC}_{\textrm{deep}}$ (VIF)	0.634	0.659	0.724	0.753	0.736	0.629	0.621	0.622	0.638
$\textrm{MC}_{\textrm{deep}}$ (Qabf)	0.611	0.607	0.717	0.542	0.582	0.709	0.609	0.620	0.644
$\textrm{MC}_{\textrm{deep}}$ (SSIM)	0.649	0.560	0.726	0.584	0.603	0.655	0.603	0.601	0.645
$\textrm{MC}_{\textrm{deep}}$ (CC)	0.696	0.608	0.724	0.590	0.589	0.630	0.673	0.614	0.655
$\textrm{MC}_{\textrm{deep}}$ (PSNR)	0.609	0.559	0.737	0.601	0.612	0.633	0.667	0.663	0.645

TABLE VI: The quantitative results of

\textrm{MC}_{\textrm{deep}}

on different image fusion datasets. ( Bold : Better performance compared to the vanilla metrics)

Fig. 10: Metric-wise attribution visualisation of “Conv3” for two fusion results on the LLVIP dataset. Brighter regions indicate higher contribution to the predicted metric. Different metrics attend to distinct image regions in accordance with their underlying definitions.

Fig. 11: The qualitative results produced by the EvaNet when predicting representative fusion results on the LLVIP dataset. From top to bottom, MetaFusion [ 33 ] , ReCoNet [ 50 ] , and FusionBooster [ 11 ] .

Fig. 12: The qualitative results produced by the EvaNet when predicting the representative fusion results on the RoadScene dataset. From top to bottom, MetaFusion, TarDAL [ 7 ] , and CoCoNet [ 53 ] .

Fig. 13: The qualitative results produced by the EvaNet when predicting representative fusion results on the TNO dataset. From top to bottom, EMMA [ 52 ] , IFCNN [ 46 ] , and DDFM [ 29 ] .

Fig. 14: The quantitative results of the ablation study of the number of predicted metrics, evaluated using

\textrm{MC}_{\textrm{deep}}

. For the first five independent metrics, increasing the number of simultaneous predictions does not significantly compromise consistency. This suggests that EvaNet maintains reliable performance even with a more comprehensive output configuration.

Fig. 15: Detection results visualisation of two representative image fusion methods (LRRNet [ 19 ] and CoCoNet [ 53 ] ) and the corresponding ranking difference relating to PSNR with or without EvaNet.

Fig. 16: Segmentation results visualisation of two representative image fusion methods (EMMA [ 52 ] and SDNet [ 47 ] ) and the corresponding ranking difference in Qabf with or without EvaNet.

Fig. 17: The quantitative results reporting metric consistency using CLIP-IQA as the reference model, i.e. ,

\textrm{MC}_{\textrm{clip}}

values for different metrics.

Task	Detection			Segmentation
Method	Vanilla	+TextAtt	+EvaNet	Vanilla	+TextAtt	+EvaNet
$\textrm{MC}_{\textrm{ds}}(\textrm{VIF})$	0.590	0.575	0.752	0.591	-	0.682
$\textrm{MC}_{\textrm{ds}}(\textrm{Qabf})$	0.604	0.615	0.720	0.595	-	0.632
$\textrm{MC}_{\textrm{ds}}(\textrm{SSIM})$	0.707	0.607	0.724	0.689	-	0.737
$\textrm{MC}_{\textrm{ds}}(\textrm{FMI}_{\textrm{d}})$	0.740	0.634	0.654	0.666	-	0.563
$\textrm{MC}_{\textrm{ds}}(\textrm{FMI}_{\textrm{w}})$	0.642	0.607	0.658	0.695	-	0.554

TABLE VII: The quantitative results of MC-DS obtained for different downstream vision tasks. ( Bold : Better performance compared to the vanilla metrics;’-’ suggests that the metric does not support this dataset)

Fig. 18: The results of comparison of the proposed EvaNet and the original image quality assessments on the MSRS dataset.

Fig. 19: A comparison of the EN values of the source images and the two state-of-the-art (Text-IF [ 51 ] and GIFNet [ 8 ] ) image fusion results. Without an appropriate adaptation, the vanilla version of EN indicates that simply outputting the infrared modality is better than fusing two modalities together.

Fig. 20: Qualitative results obtained for the daytime scene and nighttime scene by fusion method A (U2Fusion [ 60 ] ) and fusion method B (FBooster [ 11 ] ) on the MSRS dataset. The ranking change of the mutual information pixel level feature distributions is presented below these fused images.

Metric	EN	SD	EI	SF
$\textrm{MC}_{\textrm{deep}}$	0.659→ 0.679	0.715→0.621	0.643→0.625	0.622→ 0.635
$\textrm{MC}_{\textrm{clip}}$	0.676→0.617	0.628→ 0.726	0.585→ 0.695	0.676→ 0.702

TABLE VIII: The MC value results produced by our EvaNet for reference-free image quality assessments.

Explore similar work

Mar 4, 2026

Bridging Human Evaluation to Infrared and Visible Image Fusion

Infrared and visible image fusion (IVIF) integrates complementary modalities to enhance scene perception. Current methods predominantly focus on optimizing handcrafted losses and objective metrics, often resulting in fusion outcomes that do not align with human visual preferences. This challenge is further exacerbated by the ill-posed nature of IVIF, which severely limits its effectiveness in human perceptual environments such as security surveillance and driver assistance systems. To address these limitations, we propose a feedback reinforcement framework that bridges human evaluation to infrared and visible image fusion. To address the lack of human-centric evaluation metrics and data, we introduce the first large-scale human feedback dataset for IVIF, containing multidimensional subjective scores and artifact annotations, and enriched by a fine-tuned large language model with expert review. Based on this dataset, we design a domain-specific reward function and train a reward model to quantify perceptual quality. Guided by this reward, we fine-tune the fusion network through Group Relative Policy Optimization, achieving state-of-the-art performance that better aligns fused images with human aesthetics. Code is available at https://github.com/ALKA-Wind/EVAFusion.

Jinyuan Liu, Xingyuan Li, Qingyun Mei +5

School of Mechanical Engineering, Dalian University of Technology · College of Computer Science and Technology, Zhejiang University · School of Software Technology & DUT-RU International School of ISE, Dalian University of Technology +1

Infrared-Visible Image Fusion Multi-Source Image Fusion Human-in-the-Loop Evaluation

Ask PDF

May 7, 2026

Bringing Multimodal Large Language Models to Infrared-Visible Image Fusion Quality Assessment

Infrared-Visible image fusion (IVIF) aims to integrate thermal information and detailed spatial structures into a single fused image to enhance perception. However, existing evaluation approaches tend to over-optimize both hand-crafted no-reference statistics and full-reference metrics that treat the source images as pseudo ground truths. Recent IVIF reward-modelling efforts learn from human ratings but use scalar regression on aggregated scores, neither leveraging the reasoning of Multimodal Large Language Models (MLLMs) nor encoding per-image perceptual ambiguity in their supervision, but naively introducing MLLMs with discrete one-hot supervision likewise collapses fused images of similar quality into different rating levels. To address this, we introduce FuScore, which utilizes an MLLM to mimic human visual perception by producing continuous quality score, rather than discrete level predictions, enabling fine-grained discrimination among fused images of similar quality. We exploit the agreement among four IVIF-specific sub-dimensions to construct a per-image soft label whose sharpness reflects how consensual the overall judgment is. We further introduce a tripartite objective combining per-image distributional supervision, within-source-pair Thurstone fidelity for method-level ordering, and cross-source-pair Thurstone fidelity for scene-level ordering across scenes. Extensive experiments demonstrate that FuScore achieves state-of-the-art correlation with human visual preferences.

Yuchen Guo, Junli Gong, Yao Lu +3

Northwestern University · Northeastern University · University of Washington +2

Infrared-Visible Image Fusion Learning to Rank Multimodal Large Language Models Image Quality Assessment

Ask PDF

Aug 2, 2026

Ranking Infrared-Visible Fusion the Way Humans Do: A Learned Pairwise Preference Measure

Human pairwise comparison provides a direct basis for perceptual infrared-visible image fusion assessment, but dense annotation becomes costly as method pools grow. We present the Learned Perceptual Image Fusion Measure (LPIFM), among the earliest learned fusion assessors trained directly on dense human A/B/Tie comparisons. LPIFM jointly examines both source images and both fused candidates, combining a shared hierarchical encoder, triadic interaction, and a tie-aware objective to predict comparative preference and perceptual indifference. We construct and publicly release all 6,300 unordered comparisons among 25 methods on 21 VIFB scenes, collected through blinded, randomized annotation and expert adjudication. Across four VIFB evaluation settings, LPIFM achieves 79.2-84.0% agreement with human pairwise judgments and Spearman correlations of 0.941-0.977 with human-derived method rankings. On full method pools, accuracy exceeds the strongest of 19 conventional metrics by 16.3-21.1 pp. Consistency diagnostics show 99.98-100% candidate-swap agreement and no observed decisive preference cycles. External experiments on EVAFusion further demonstrate rapid adaptation to a different fusion-evaluation preference protocol. After only three epochs of fine-tuning, LPIFM surpasses all 19 conventional metrics in accuracy, macro-F1, and ranking correlation. LPIFM provides a scalable instrument for human-aligned fusion assessment, with the preference corpus, model weights, and code publicly available.

Haoran Liu, Mingzhe Liu, Peng Li +1

School of Artificial Intelligence and Electronic Engineering, Sichuan Technology and Business University, Chengdu 611745, China · College of Nuclear Technology and Automation Engineering, Chengdu University of Technology, Chengdu 610059, China · National Synchrotron Radiation Laboratory, University of Science and Technology of China, Hefei 230029, China

Pairwise Preference Learning Multi-Source Image Fusion Pairwise Comparison Infrared-Visible Image Fusion Image Quality Assessment

Ask PDF