NatureFusion: Rethinking Infrared and Visible Image Fusion as Natural Complementary Enhancement

Jianfeng Ding; Linfeng Tang; Hao Zhang; Han Xu; Xunpeng Yi; Yibing Zhang; Meiqi Gong; Zhongyuan Wang

cs.CVNov 1, 2026

NatureFusion: Rethinking Infrared and Visible Image Fusion as Natural Complementary Enhancement

Authors: Jianfeng Ding, Linfeng Tang, Hao Zhang, Han Xu, Xunpeng Yi, Yibing Zhang, Meiqi Gong, Zhongyuan Wang,

Organizations: the National Engineering Research Center for Multimedia Software (NERCMS), School of Computer Science, Wuhan University, Wuhan 430072, China · the School of Robotics, Wuhan University, Wuhan 430072, China · the Electronic Information School, Wuhan University, Wuhan 430072, China · the School of Automation, Southeast University, Nanjing 210096, China

Abstract

In the absence of ground truth, most existing infrared and visible image fusion methods pursue fidelity to the source images and directly superimpose two physically distinct modalities. As a result, salient infrared targets appear as bright patches of thermal radiation pasted onto the scene. These patches obscure the original color and texture of the targets, so the fused images look unnatural to human observers and deviate from the data distribution learned by vision models pretrained on visible images. Therefore, we redefine infrared and visible image fusion as natural complementary enhancement rather than source fidelity, and propose NatureFusion, which exploits cross-modal complementary cues to infer a clear and natural visible-style image from low-quality visible and infrared image pairs. Specifically, to address the lack of ground truth, we employ a generative foundation model as the teacher to directly generate visually natural fused images for supervision. Furthermore, to fully distill the teacher's generative capability while reducing computational cost, we fine-tune a pretrained generative model with conditional flow matching through a light-weight fusion adapter. Last but not least, to further improve downstream performance without sacrificing natural appearance, we freeze the fusion adapter and introduce a detector-supervised task adapter that strengthens the complementary contribution of infrared. Extensive comparisons with seven multi-modal fusion and seven single-modality enhancement methods demonstrate that NatureFusion generates visually natural fused images with the best perceptual quality. Detection and segmentation models pretrained on visible images achieve substantially higher accuracy on our fused images than on those of any competing method.

Figures & tables

Fig. 1: Comparison with multi-modal fusion and single-modality enhancement methods. Panels (a) and (b) show the visible and infrared inputs, (c) to (e) show the results of degradation-aware fusion, task-driven fusion, and single-modality enhancement methods, and (f) shows the result of NatureFusion. Red and blue boxes mark enlarged person and background regions, and gold boxes show person detections of a frozen RetinaNet with confidence scores of at least 0.25. (g) to (i) Comparison with seven multi-modal fusion methods in MUSIQ on M3FD, person $\mathrm{AP}_{50:95}$ on SMOD, and $\mathrm{mIoU}_{2}$ over Car and Person on MFNet. Detection and segmentation each use seven frozen models pretrained on visible images. Bars show means, and whiskers show ranges across images and models.

Fig. 2: Training and inference pipeline of NatureFusion. Flames and snowflakes denote trainable and frozen parameters, respectively.

Fig. 3: Qualitative comparison with multi-modal fusion methods.

Fig. 4: Qualitative comparison with single-modality enhancement methods.

Fig. 5: Person detections from the same frozen RetinaNet on an LLVIP scene. Blue boxes and numbers show predictions with confidence of at least 0.25. (a) Visible (upper right) and infrared (lower left) images are detected separately and then combined. Red circles enlarge the person region.

Fig. 6: Mean person AP versus IoU threshold. Fusion and Enhancement bands span the minimum to maximum AP of the multi-modal fusion and single-modality enhancement methods, respectively, and Margin shades where NatureFusion exceeds the best of them. Lower panels plot

\Delta

AP, NatureFusion minus the best compared method.

Method	SMOD		LLVIP		M3FD
Method	$\mathrm{AP}_{50:95}\uparrow$	$R_{\mathrm{miss}}\uparrow$	$\mathrm{AP}_{50:95}\uparrow$	$R_{\mathrm{miss}}\uparrow$	$\mathrm{AP}_{50:95}\uparrow$	$R_{\mathrm{miss}}\uparrow$
Retinexformer [ 12 ]	34.03	2.4	31.73	1.9	30.51	0.0
DiffLL [ 15 ]	29.82	0.5	29.72	2.6	33.64	0.3
QuadPrior [ 17 ]	34.87	1.3	35.31	2.7	35.15	0.0
DarkIR [ 14 ]	35.75	3.4	33.92	3.0	33.64	0.1
HVI-CIDNet [ 13 ]	34.45	2.4	30.70	1.9	33.93	0.2

TABLE IV: Detection comparison with single-modality enhancement methods.

Fig. 7: Frozen segmentation on MFNet by OCRNet-HR48. The left panel shows the visible and infrared inputs, with the red box marking the enlarged region, and each method’s crop is shown above its prediction.

Method	$\mathrm{mIoU}_{9}\uparrow$				$\mathrm{mIoU}_{2}\uparrow$
Method	PSPNet	DLV3+	SegFormer	Mean	PSPNet	DLV3+	SegFormer	Mean
Visible	46.07	45.82	48.48	46.79	62.12	62.75	65.34	63.41
Infrared	42.18	39.74	43.98	41.97	70.00	70.83	71.48	70.77
Text-IF [ 2 ]	49.78	45.88	49.96	48.54	72.42	73.47	74.02	73.30
OmniFuse [ 4 ]	45.75	46.77	48.88	47.13	65.91	64.98	70.79	67.23
HCLFuse [ 29 ]	47.49	42.72	47.93	46.04	66.73	67.22	71.47	68.47

TABLE VII: Retrained segmentation on MFNet.

Stages	Setting	$\mathrm{AP}_{50:95}\uparrow$	$\mathrm{AP}_{50}\uparrow$	$\mathrm{AP}_{75}\uparrow$	$\mathrm{AP}_{90}\uparrow$	$R_{\mathrm{miss}}\uparrow$
Test	NatureFusion, w/o IR	32.33	63.69	32.20	2.23	23.2
I	Teacher w/o IR	25.99	58.46	22.21	1.62	21.6
I	Teacher w/ IR	25.69	58.19	20.64	1.20	29.8
II	Fusion-only, VI+VI	32.12	63.62	31.89	2.42	24.1
II	Fusion-only, VI+IR	34.87	67.57	34.85	3.03	29.0
II+III	Fusion + Task, VI+VI	32.90	65.04	32.18	1.81	25.1

TABLE VIII: Role of infrared at each stage on LLVIP. w/o IR replaces infrared with an all-zero image at inference only, and VI+VI replaces infrared with the visible image in both training and inference. The best value in each column is marked in bold red.

Panel of Figure 11 — Fig. 8: $R_{\mathrm{miss}}$ under different infrared settings.

Variant	LLVIP
Variant	$\mathrm{AP}_{50:95}\uparrow$	$\mathrm{AP}_{50}\uparrow$	$\mathrm{AP}_{75}\uparrow$	$R_{\mathrm{miss}}\uparrow$	NIQE $\downarrow$
Zero-shot (VI+IR)	34.62	67.22	33.17	28.0	3.216
Fusion-only	34.44	67.30	33.87	29.2	2.774
$w=1$	34.85	67.36	34.65	29.2	2.763
w/o $\mathcal{L}_{\mathrm{det}}$	34.82	67.75	34.16	31.4	2.747
w/o $\mathcal{L}_{\mathrm{anc}}$	35.02	68.00	34.42	31.3	2.747

TABLE IX: Component ablation on LLVIP. Zero-shot is the Distilled backbone without adapters, and other variants report the mean of two training runs.

$w$	$\mathrm{AP}_{55}\uparrow$	$\mathrm{AP}_{70}\uparrow$	$\mathrm{AP}_{90}\uparrow$	$\mathrm{AP}_{95}\uparrow$	$R_{\mathrm{miss}}\uparrow$
1	77.89	65.59	11.51	1.70	43.1
1.5	78.00	66.21	11.90	1.63	43.4
2	78.69	68.23	12.01	1.76	45.0
3	78.50	68.08	11.81	1.63	44.9

TABLE X: Sensitivity to the amplification factor

w

on M3FD.

Panel of Figure 14 — Fig. 10: Comparison under registration errors on LLVIP with (a) aligned inputs and (b) infrared shifted right by 64 pixels.

Panel of Figure 15 — Fig. 12: Teacher and student outputs on an LLVIP scene.

Fig. 14: Mean generation time per image at native resolution on a single RTX 3090 (log scale).

Infrared-Visible Image Fusion

Infrared-visible image fusion combines thermal cues with visible-light detail to create images for human viewing and machine perception.

See all 41 papers

ProtoFusion: Protocol-Guided Prompt Refinement for Generative Infrared and Visible Image Fusion
Nov 2, 2026Pengwei Liang, Junjun Jiang, Junhui HouInfrared-Visible Image Fusion Prompt Optimization
Department of Computer Science, City University of Hong Kong, Hong Kong SAR, China · School of Computer Science and Technology, Harbin Institute of Technology, Harbin, 150001, China
P2Fusion: Prompt-based Progressive Infrared-Visible Image Fusion via Dual-Prior Distillation
Aug 13, 2026Yi Shi, Huichao Xie, Yuqing Wang +7Multi-Source Image Fusion Infrared-Visible Image Fusion
Northwestern Polytechnical University, Xi’an, China · Hefei University of Technology, Hefei, China · Rocket Force University of Engineering, Xi’an, China +1
EAPFusion: Intrinsic Evolving Auxiliary Prior Guidance for Infrared and Visible Image Fusion
May 3, 2026Zhenyu Sun, Luobin Zhang, Axi Niu +2Dynamic Convolution Multi-Source Image Fusion
Northwestern Polytechnical University, Xi’an, China

Nov 2, 2026J/K move · Enter open · S save

ProtoFusion: Protocol-Guided Prompt Refinement for Generative Infrared and Visible Image Fusion

Pengwei Liang, Junjun Jiang, Junhui Hou

Department of Computer Science, City University of Hong Kong, Hong Kong SAR, China · School of Computer Science and Technology, Harbin Institute of Technology, Harbin, 150001, China

Infrared-Visible Image Fusion Prompt Optimization Reliability-Aware Multimodal Fusion

Open paper AskPDF arXiv

Infrared and visible image fusion integrates complementary information from both modalities to provide a more comprehensive representation of the scene. We propose ProtoFusion, a protocol-guided generative framework for infrared and visible image fusion. Instead of relying on fixed fusion operators, hand-crafted objectives, or manually specified one-shot prompts, ProtoFusion formulates fusion as protocol-constrained generation. It represents fusion intent with a structured protocol, consisting of environment state cards, grounding principles, and a style profile, which explicitly specifies which evidence should be preserved or suppressed and which properties the fused result should satisfy. To optimize this protocol, we introduce protocol-guided prompt refinement, a closed-loop learning framework that iteratively performs generation, evaluation, and revision under multimodal evidence. Once learned, the optimized protocol is fixed at inference time and directly guides fusion generation. To address residual source inconsistencies, we further incorporate structured belief construction, which organizes modality reliability, scene evidence, and cross-modal conflicts into a structured representation for conservative correction after generation. Extensive experiments on standard benchmarks show that ProtoFusion improves quantitative performance, achieves favorable results in aesthetic evaluation, and produces visually coherent fusion outputs. Project Page: {https://erfect2020.github.io/ProtoFusion/}.

Abstract

Infrared-Visible Image Fusion

Explore similar work

ProtoFusion: Protocol-Guided Prompt Refinement for Generative Infrared and Visible Image Fusion

P2Fusion: Prompt-based Progressive Infrared-Visible Image Fusion via Dual-Prior Distillation

EAPFusion: Intrinsic Evolving Auxiliary Prior Guidance for Infrared and Visible Image Fusion