RADIANT: Redundancy-Aware Information Allocation for Universal Multi-Modal Fusion
Organizations: Research Institute, Newland Digital Technology, Fuzhou, China
Abstract
Universal multi-modal fusion often fails not because modalities are insufficiently aligned, but because the fusion process allocates information poorly: redundant evidence is repeatedly amplified, modality-unique evidence is over-smoothed during alignment, and task-dependent synergistic evidence is insufficiently activated. We present RADIANT (Redundancy-Aware Dynamic Information Allocation Network), a task-general framework that organizes fusion around three complementary roles: suppressing redundant cross-modal overlap, preserving modality-unique evidence, and composing task-conditioned synergy through cross-modal interaction. Specifically, stable matrix-exponential projections are used to reduce interference among shared and modality-unique streams, while task memory with hard positive updates selects which synergistic interactions should be activated for the current fusion regime. Extensive experiments across six heterogeneous fusion tasks show consistent improvements over the compared methods. On FLIR, RADIANT improves downstream detection [email protected] from 91.2% to 96.8%, a gain of 5.6 percentage points (6.1% relative) over GIF. RADIANT also exhibits strong direct cross-dataset transfer, with further gains from inference-time task-memory adaptation while keeping all network weights frozen. Additional ablations and sensitivity studies support the robustness of the allocation mechanism.
Figures & tables
| Variant | Residual | EI |
| Orthogonal factor only ( / fixed basis) | 0.021 | 65.31 |
| Full sequential parameterization | 0.034 | 66.08 |
| (a1) MFIF: Lytro | (a2) MFIF: MFI-WHU | (c) MEIF: DSCIE | (d) NIR-VIS: Scene | |||||||||||||||
| Method | EI | VIF | SCD | AG | EI | VIF | SCD | AG | Method | EI | VIF | SCD | AG | Method | EI | VIF | SCD | AG |
| U2Fusion | 67.20 | 1.39 | 0.84 | 6.34 | 79.11 | 1.50 | 0.56 | 7.88 | U2Fusion | 83.00 | 1.69 | 0.49 | 8.71 | IFCNN | 82.13 | 0.90 | 1.14 | 8.72 |
| UNIFusion* | 70.14 | 1.30 | 0.60 | 6.77 | 66.57 | 1.01 | 0.29 | 7.19 | SPD-MEF* | 78.17 | 1.72 | 0.49 | 8.12 | U2Fusion | 80.73 | 1.07 | 1.19 | 8.51 |
| SDNet | 62.98 | 1.12 | 0.75 | 6.16 | 72.98 | 1.16 | 0.63 | 7.98 | MEF-GAN* | 80.21 | 1.59 | 0.63 | 8.06 | SDNet | 76.70 | 0.86 | 0.72 | 8.26 |
| MUFusion | 70.40 | 1.34 | 1.22 | 6.67 | 77.72 | 1.36 | 1.11 | 7.92 | MUFusion | 70.18 | 1.64 | 0.96 | 7.19 | MURF* | 41.71 | 0.41 | 0.19 | 4.22 |
| ZMFF* | 58.97 | 1.11 | 0.36 | 5.48 | 57.90 | 1.03 | 0.33 | 5.49 | IID-MEF* | 59.12 | 1.12 | 0.36 | 6.13 | Text-IF | 88.19 | 1.49 | 1.45 | 9.16 |
| Task | ||
| LLVIP | ||
| Harvard |
| Method | Technique | Size (MB) | GFLOPs | FPS |
| DDFM | Diffusion model | 2211.00 | 1840.49 | 0.3 |
| Text-IF | ViLT | 580.81 | 82.85 | 8.2 |
| TC-MoA | MAE, MoE | 426.98 | 524.28 | 2.1 |
| GIF | Fusion only | 3.30 | 9.96 | 38 |
| RADIANT | Allocation + Task Memory | 3.108 | 9.42 | 42 |
| NI | Task memory | Mamba | Swin | Multi-scale interaction | EI | VIF | SCD | AG | % |
| 52.31 | 0.61 | 1.42 | 5.18 | – | |||||
| 60.12 | 0.70 | 1.55 | 6.30 | +14.9 | |||||
| 58.22 | 0.68 | 1.51 | 6.01 | +11.3 | |||||
| 62.74 | 0.73 | 1.60 | 6.58 | +19.9 | |||||
| 63.01 | 0.74 | 1.61 | 6.69 | +20.5 | |||||
| 66.08 | 0.78 | 1.70 | 7.09 | +26.3 |
| Full (trained on TNO) | Transfer (LLVIP TNO) | ||||||||
| Method | EI | VIF | SCD | AG | EI | VIF | SCD | AG | Ratio |
| CDDFuse | 43.83 | 0.91 | 1.66 | 4.55 | 31.25 | 0.64 | 1.18 | 3.25 | 71.3% |
| GIF | 52.30 | 0.99 | 1.66 | 5.24 | 40.17 | 0.76 | 1.27 | 4.02 | 76.8% |
| RADIANT | 55.33 | 1.06 | 1.78 | 5.62 | 49.35 | 0.95 | 1.59 | 5.01 | 89.2% |
| Stream Used | SSIM | PSNR | [email protected] |
| only | 0.847 | 24.3 | 89.2% |
| only | 0.412 | 15.8 | 71.4% |
| only | 0.298 | 12.1 | 93.1% |
| 0.923 | 28.7 | – | |
| – | – | 95.8% | |
| Full model | 0.941 | 30.2 | 96.8% |
Multimodal Fusion
Multimodal fusion combines information from different data types to improve prediction, perception, and decision-making.
See all 810 papers