PCDFusion: Proposal-Context-Detail Bayesian Rendering for Infrared-Visible Image Fusion
Organizations: School of Computer Science and Technology, Beijing Institute of Technology, Beijing 100081, China · Research Institute of Electrical Communication, Tohoku University, Sendai 980-8577, Japan
Abstract
Infrared-visible image fusion (IVIF) aims to combine visible and thermal infrared observations for human perception and downstream vision tasks. Visible and thermal infrared observations are physically complementary but unreliable in different ways. Existing deep IVIF methods mainly improve feature interaction and reconstruction, but often lack an explicit local rule for deciding when visible appearance should dominate and when thermal responses should contribute. To address this issue, we reformulate IVIF as local Bayesian posterior rendering, where fusion is controlled by posterior action weights rather than direct modality mixing. In this paper, we propose Proposal-Context-Detail Fusion (PCDFusion), a visible-anchored framework that implements this formulation through three rendering stages. PCDFusion preserves potentially useful thermal responses, updates exposure-degraded visible regions, and renders reliable band-limited infrared structures into the final luminance. Extensive experiments on public IVIF benchmarks show that PCDFusion improves fusion quality, target recovery, and downstream detection and segmentation performance under challenging illumination. The code is available at https://github.com/BIT-RuiLiu/PCDFusion.git.
Figures & tables
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
| Role | Probabilistic operation | PCDFusion realization | Consequence |
|---|---|---|---|
| Decision | Bernoulli rendering action | Explicit proposal/context/detail action semantics | Defined local inference target |
| Prior | Analytic IR-response, exposure, or structure cue | Physical zero-evidence reference | |
| Evidence | Relative support in log Bayes-factor coordinates | Neural score | Amortized log-evidence correction |
| Update | Prior odds evidence factor | Normalized action posterior | |
| Rendering | Expectation over two actions | Posterior-weighted fusion | |
| Learning | Shared estimator of observation evidence | End-to-end optimization of through rendering losses | Amortized across pixels and images |
| Asset | Type | Repository / Source | License / Terms |
|---|---|---|---|
| SwinFusion JAS’22 | Baseline implementation | https://github.com/Linfeng-Tang/SwinFusion.git | Not specified |
| TarDAL CVPR’22 | Baseline implementation / M3FD source | https://github.com/JinyuanLiu-CV/TarDAL.git | GPL-3.0 |
| CDDFuse CVPR’23 | Baseline implementation | https://github.com/Zhaozixiang1228/MMIF-CDDFuse.git | Not specified |
| Diff-IF Inf. Fusion’24 | Baseline implementation | https://github.com/XunpengYi/Diff-IF.git | MIT |
| EMMA CVPR’24 | Baseline implementation | https://github.com/Zhaozixiang1228/MMIF-EMMA.git | Not specified |
| DCEvo CVPR’25 | Baseline implementation | https://github.com/Beate-Suy-Zhang/DCEvo.git | Not specified |
| Method | EN | AG | MI | SD | SCD | VIF | Qabf | SSIM | SF | CE |
|---|---|---|---|---|---|---|---|---|---|---|
| SwinFusion JAS’22 | 6.620 | 3.562 | 4.393 | 42.980 | 1.686 | 1.032 | 0.647 | 0.721 | 11.096 | 0.052562 |
| TarDAL CVPR’22 | 6.211 | 2.926 | 2.669 | 32.781 | 1.375 | 0.716 | 0.416 | 0.540 | 9.547 | 0.065997 |
| CDDFuse CVPR’23 | 6.699 | 3.749 | 4.705 | 43.368 | 1.618 | 1.084 | 0.676 | 0.709 | 11.567 | 0.053406 |
| Diff-IF Inf. Fusion’24 | 6.669 | 3.696 | 4.970 | 42.596 | 1.624 | 1.097 | 0.685 | 0.711 | 11.460 | 0.054479 |
| EMMA CVPR’24 | 6.715 | 3.786 | 4.129 | 44.568 | 1.628 | 1.016 | 0.632 | 0.711 | 11.574 | 0.052032 |
| DCEvo CVPR’25 | 6.622 | 3.725 | 4.157 | 42.926 | 1.662 | 1.066 | 0.688 | 0.721 | 11.470 | 0.050816 |
| Method | EN | AG | MI | SD | SCD | VIF | Qabf | SSIM | SF | CE |
|---|---|---|---|---|---|---|---|---|---|---|
| SwinFusion JAS’22 | 6.683 | 4.052 | 4.270 | 35.352 | 1.560 | 1.032 | 0.665 | 0.743 | 13.515 | 0.000238 |
| TarDAL CVPR’22 | 6.963 | 3.541 | 3.423 | 40.316 | 1.573 | 0.807 | 0.462 | 0.745 | 11.718 | 0.000369 |
| CDDFuse CVPR’23 | 6.778 | 4.306 | 4.148 | 37.043 | 1.622 | 1.061 | 0.672 | 0.745 | 14.577 | 0.000230 |
| Diff-IF Inf. Fusion’24 | 6.632 | 4.047 | 4.985 | 34.191 | 1.368 | 1.047 | 0.660 | 0.735 | 13.869 | 0.000229 |
| EMMA CVPR’24 | 6.775 | 4.684 | 3.947 | 36.805 | 1.504 | 1.031 | 0.645 | 0.735 | 15.006 | 0.000224 |
| DCEvo CVPR’25 | 6.673 | 4.051 | 4.512 | 34.749 | 1.503 | 1.058 | 0.686 | 0.747 | 13.771 | 0.000218 |
| Method | EN | AG | MI | SD | SCD | VIF | Qabf | SSIM | SF | CE |
|---|---|---|---|---|---|---|---|---|---|---|
| SwinFusion JAS’22 | 6.808 | 4.606 | 4.020 | 35.776 | 1.560 | 0.898 | 0.607 | 0.717 | 13.672 | 0.031855 |
| TarDAL CVPR’22 | 6.985 | 4.265 | 3.130 | 38.785 | 1.523 | 0.708 | 0.427 | 0.708 | 12.672 | 0.029532 |
| CDDFuse CVPR’23 | 6.907 | 4.861 | 3.774 | 37.206 | 1.646 | 0.917 | 0.610 | 0.718 | 14.753 | 0.031766 |
| Diff-IF Inf. Fusion’24 | 6.791 | 4.567 | 4.852 | 34.927 | 1.322 | 0.894 | 0.589 | 0.701 | 14.115 | 0.031833 |
| EMMA CVPR’24 | 6.927 | 5.335 | 3.753 | 38.253 | 1.494 | 0.897 | 0.590 | 0.703 | 15.235 | 0.031770 |
| DCEvo CVPR’25 | 6.842 | 4.620 | 4.298 | 35.582 | 1.491 | 0.917 | 0.633 | 0.720 | 14.109 | 0.031699 |
| Method | Road | Side. | Build. | Light | Sign | Veg. | Sky | Per. | Car | Truck | Bus | Motor. | Pole | mIoU |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SwinFusion JAS’22 | 89.22 | 48.98 | 83.16 | 43.22 | 80.68 | 87.16 | 94.80 | 70.58 | 84.23 | 48.77 | 57.57 | 36.28 | 50.23 | 58.33 |
| TarDAL CVPR’22 | 89.50 | 50.05 | 83.24 | 42.30 | 81.20 | 88.21 | 95.00 | 73.31 | 81.85 | 35.82 | 26.36 | 37.18 | 49.76 | 55.59 |
| CDDFuse CVPR’23 | 90.18 | 50.97 | 84.38 | 40.85 | 81.80 | 88.40 | 95.16 | 72.22 | 82.90 | 22.14 | 55.70 | 33.60 | 51.56 | 56.66 |
| Diff-IF Inf. Fusion’24 | 89.66 | 43.81 | 83.64 | 45.24 | 82.04 | 87.02 | 94.70 | 70.69 | 83.57 | 45.22 | 75.99 | 32.26 | 50.82 | 58.98 |
| EMMA CVPR’24 | 89.61 | 49.32 | 84.57 | 49.42 | 81.24 | 88.15 | 95.29 | 70.75 | 82.58 | 42.99 | 59.41 | 32.61 | 51.07 | 58.47 |
| DCEvo CVPR’25 | 89.72 | 48.01 | 84.39 | 46.98 | 81.01 | 88.14 | 95.27 | 71.50 | 84.55 | 43.67 | 49.08 | 33.89 | 49.73 | 57.73 |
| Method | mAP 50 | mAP 50:95 | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Per. | Car | Bus | Motor. | Lamp | Truck | mAP | Per. | Car | Bus | Motor. | Lamp | Truck | mAP | |
| SwinFusion JAS’22 | 82.28 | 91.25 | 89.10 | 66.22 | 79.83 | 78.75 | 81.24 | 48.90 | 68.13 | 71.31 | 40.82 | 42.03 | 58.00 | 54.86 |
| TarDAL CVPR’22 | 81.94 | 90.88 | 87.43 | 59.35 | 78.67 | 76.25 | 79.09 | 48.80 | 66.51 | 69.55 | 34.00 | 38.87 | 56.38 | 52.35 |
| CDDFuse CVPR’23 | 81.93 | 91.57 | 90.26 | 68.36 | 81.59 | 74.18 | 81.31 | 48.79 | 68.44 | 71.80 | 38.00 | 43.44 | 54.39 | 54.15 |
| Diff-IF Inf. Fusion’24 | 82.81 | 91.38 | 88.46 | 66.32 | 77.91 | 79.14 | 81.00 | 49.77 | 67.43 | 70.08 | 38.47 | 39.82 | 58.68 | 54.04 |
| EMMA CVPR’24 | 81.70 | 91.11 | 90.46 | 63.35 | 81.59 | 77.36 | 80.93 | 48.35 | 68.03 | 71.02 | 38.26 | 43.70 | 57.20 | 54.43 |
| Metric | SwinFusion | TarDAL | CDDFuse | Diff-IF | EMMA | DCEvo | Deno-IF | SAGE | UPFusion | PCDFusion |
|---|---|---|---|---|---|---|---|---|---|---|
| Over-exposed IR-edge retention (%) | 16.8 | 27.0 | 51.3 | 47.4 | 24.0 | 62.9 | 49.0 | 25.7 | 17.3 | 66.3 |
| Reliable-sky preservation (%) | 100.0 | 0.1 | 99.5 | 99.6 | 97.5 | 99.5 | 7.3 | 0.0 | 99.9 | 99.5 |
| Fusion method | FMB mIoU (%) | M3FD mAP 50:95 (%) | ||||
|---|---|---|---|---|---|---|
| VIS-trained | IR-trained | Mean | VIS-trained | IR-trained | Mean | |
| SwinFusion | 54.18 | 40.39 | 47.28 | 25.96 | 8.46 | 17.21 |
| TarDAL | 51.76 | 42.46 | 47.11 | 24.48 | 10.04 | 17.26 |
| CDDFuse | 56.09 | 40.79 | 48.44 | 26.54 | 8.35 | 17.44 |
| Diff-IF | 53.61 | 40.01 | 46.81 | 23.30 | 10.34 | 16.82 |
| EMMA | 55.65 | 39.59 | 47.62 | 30.01 | 5.93 | 17.97 |
| Method | Fusion level | Test-time inputs | Training protocol | Params (M) | GFLOPs | mAP 50:95 (%) |
|---|---|---|---|---|---|---|
| PCDFusion + YOLOv10x | Pixel/image | One fused image | 300 epochs | 36.27 | 508.62 | 61.05 |
| M2D-LIF | Feature | Paired VIS+IR | 200-epoch teacher + 100-epoch student | 36.51 | 122.42 | 61.26 |
| LoME (width ) | Feature | Paired VIS+IR | 300 epochs | 36.18 | 49.05 | 59.38 |
| Stage | Variant | AG | SCD | VIF | Qabf | Qcb | mIoU | mAP |
| Proposal | Uniform: | 3.863 | 1.672 | 1.110 | 0.723 | 0.591 | 58.88 | 55.86 |
| Proposal | Hard IR: | 3.861 | 1.671 | 1.109 | 0.722 | 0.589 | 58.95 | 55.72 |
| Proposal | Soft IR (Full): | 3.839 | 1.671 | 1.117 | 0.723 | 0.593 | 60.05 | 55.98 |
| Detail | Edge competition: | 3.855 | 1.674 | 1.104 | 0.721 | 0.585 | 58.95 | 55.83 |
| Detail | IR structure only: | 3.848 | 1.677 | 1.110 | 0.722 | 0.587 | 58.84 | 55.70 |
| Detail | IR intensity: | 3.859 | 1.678 | 1.105 | 0.721 | 0.586 | 58.90 | 55.95 |
| Parameter | Value | AG | VIF | Qabf | mIoU | mAP |
|---|---|---|---|---|---|---|
| Proposal temperature | 3.813 ( ) | 1.115 ( ) | 0.720 ( ) | 60.11 ( ) | 55.94 ( ) | |
| 3.832 ( ) | 1.119 ( ) | 0.723 ( ) | 60.10 ( ) | 55.88 ( ) | ||
| (default) | 3.839 ( ) | 1.117 ( ) | 0.723 ( ) | 60.05 ( ) | 55.98 ( ) | |
| 3.830 ( ) | 1.104 ( ) | 0.720 ( ) | 60.03 ( ) | 55.90 ( ) | ||
| 3.794 ( ) | 1.080 ( ) | 0.714 ( ) | 59.99 ( ) | 55.95 ( ) | ||
| Detail scale | 3.729 ( ) | 1.109 ( ) | 0.714 ( ) | 59.90 ( ) | 55.88 ( ) |
| Stage | Posterior-action map used for rendering | Local task gain | FMB mIoU |
|---|---|---|---|
| Proposal | Learned posterior | 0.071 | 60.05 |
| Prior only | 0.069 | 59.86 | |
| Spatially shuffled | 50.21 | ||
| Inverted | 0.003 | 58.68 | |
| Zero map | 0.000 | 58.58 | |
| Context | Learned posterior | 0.013 | 60.05 |
| Metric | SwinFusion JAS’22 | TarDAL CVPR’22 | CDDFuse CVPR’23 | Diff-IF Inf. Fusion’24 | EMMA CVPR’24 | DCEvo CVPR’25 | Deno-IF NIPS’25 | SAGE CVPR’25 | UPFusion AAAI’26 | PCDFusion Ours |
|---|---|---|---|---|---|---|---|---|---|---|
| Params (M) | 0.97 | 0.30 | 1.19 | 23.74 | 1.52 | 2.00 | 1.43 | 0.14 | 154.82 | 4.60 |
| GFLOPs | 604.42 | 155.07 | 945.04 | 6101.12 | 68.46 | 1561.87 | 221.15 | 34.11 | 2676.36 | 235.34 |
| Time (ms) | 997.82 | 5.04 | 126.70 | 780.17 | 13.17 | 178.92 | 112.68 | 2.12 | 584.92 | 92.51 |
| FPS | 1.00 | 198.55 | 7.89 | 1.28 | 75.92 | 5.59 | 8.87 | 471.34 | 1.71 | 10.81 |
| Dataset | Estimate | Pixel failure | Class-normalized mFailure | ||||
|---|---|---|---|---|---|---|---|
| Pearson | Spearman | MSE | Pearson | Spearman | MSE | ||
| MSRS | Darkness-only: | 0.762 | 0.982 | 0.1372 | -0.111 | -0.035 | 0.2231 |
| Brightness-only: | -0.762 | -0.982 | 0.3381 | 0.111 | 0.035 | 0.1936 | |
| Distance-to-mid: | 0.418 | 0.087 | 0.1825 | 0.815 | 0.928 | 0.0998 | |
| Ours prior: | 0.463 | 0.087 | 0.1014 | 0.879 | 0.928 | 0.0314 | |
| Ours posterior: | 0.646 | 0.147 | 0.0552 | 0.890 | 0.904 | 0.0658 | |
Infrared-Visible Image Fusion
Infrared-visible image fusion combines thermal cues with visible-light detail to create images for human viewing and machine perception.
See all 38 papers