Overview

Multimodal fusion studies how to combine inputs such as images, text, audio, medical scans, sensor streams, and structured records so models can use their complementary information. Common approaches include early or late fusion, cross-attention and contrastive alignment, and adaptive gates or mixtures of experts that weight modalities by task, sample, or estimated reliability. Methods are tested across settings such as BraTS medical segmentation, CMU-MOSI and CMU-MOSEI sentiment analysis, RGB-thermal detection, and clinical prediction, with growing attention to missing, noisy, or misaligned inputs. A central open tension is that additional modalities can help when they add useful evidence, but can also introduce noise or let a dominant modality suppress weaker ones; robust gains and generalization beyond benchmark conditions remain difficult to establish.

Latest papers

CardsList

805 papers56 papers in the last four weeks, down 35% on the four weeks before. 0.4% of all new papers.

  1. InertiaMPDD: Multimodal Personalized Depression Detection via Emotional Inertia-Inspired Dynamics Modeling

    Nov 12, 2026Changzeng Fu, Songyang Wang, Chang Gao +9Depression DetectionLatent Dynamics Modeling

  2. RADIANT: Redundancy-Aware Information Allocation for Universal Multi-Modal Fusion

    Nov 12, 2026Shenghai Wu, Yue LinMultimodal Fusion

  3. A Time-Frequency-Condition Multimodal Fusion Foundation Model for Universal Rotating-Machinery Fault Diagnosis under Multi-Element Heterogeneous Scenarios

    Nov 11, 2026Xinyu Zou, Jian Ma, Liang Ma +3Industrial Fault DiagnosisBearing Fault Diagnosis

  4. LUMIR: Decoupled Multimodal Representation Learning from Heterogeneous Data for Next-POI Recommendation

    Nov 10, 2026Zehui Wang, Wolfram Höpken, Dietmar JannachMultimodal RecommendationCross-Modal Representation Learning

  5. Multimodal Road Closure Mapping via Cross-Modal Graph Attention

    Nov 9, 2026Izzeddin Teeti, Vaibhav Kango, Nidhi Sridhar +1Multimodal FusionMultimodal Graph Learning

  6. CSG-Nav: Confidence-Aware Semantic-Geometric Fusion for Robust Exploration and Accurate Zero-Shot Object-Goal Navigation

    Nov 9, 2026Shoubin Chen, Rui Xu, Zehao Wu +4Object-Goal NavigationVisual Robot Navigation

  7. FMA-Net: Flexible Modality Adaptation for Robust Multimodal Face Presentation Attack Detection

    Nov 9, 2026Yuhao Gao, Yichen Shi, Zitong Yu +5Face Presentation Attack DetectionMissing-Modality Learning

  8. STAR-VF: Spatio-Temporal Aware Representation Condensation for Data-Efficient Video Fusion

    Nov 7, 2026Yuchen Xian, Yunqiu Xu, Guosheng Lin +1Multimodal FusionDataset Condensation

  9. Guided Co-Attention with Adaptive Gating for Multimodal Medical Data Fusion

    Nov 7, 2026Muntaqim Ahmed Raju, Priyanka Siddappa, Md Shifat Haider Al Amin +2Multimodal FusionGated Multimodal Fusion

  10. CXR-CMFA: A Cross-Modal Feature Aggregation Network for Chest X-Ray Pathology Classification

    Nov 6, 2026Talukder Hasnat Zadid, Jahin Sultana, Felipe Bochini Brisola +4Chest X-Ray ClassificationMedical VLMs

  11. MM-DyGraph: A Dataset, Benchmark, and Model for Multimodal Dynamic Graphs

    Nov 6, 2026Zongyuan Wu, Pengjie Wang, Lingshan Chen +5Multimodal Graph LearningMultimodal Dataset

  12. HMMF-Net: A Lightweight Hierarchical Multi-Frame Multimodal Fusion Network for Accurate Fish Feeding Intensity Assessment

    Nov 6, 2026Phi-Hung Hoang, Van-Manh Tran, Nam-Thuan Trinh +1AquacultureMultimodal Fusion

  13. Mind the Divergence: Divergence-Guided Reliability-Aware Fusion and Adaptive LLM Prompting for Conversational Emotion Recognition

    Nov 6, 2026Jingwen Wang, Jiao Li, Yuhang Qian +2Emotion Recognition in ConversationsReliability-Aware Multimodal Fusion

  14. Interpretable Multimodal MRI-Clinical Fusion for Functional Outcome Prediction After Acute Ischemic Stroke

    Nov 5, 2026Sina Raeisadigh, Henning Müller, Abderrahmane HedjoudjeClinical Outcome PredictionMultimodal Fusion

  15. Multimodal Information Interactions Bottleneck for Multimodal Self-Supervised Learning

    Nov 5, 2026Liangjian Wen, Jiangtao Zheng, Jiang Duan +8Multimodal SSLMultimodal Learning

  16. InfMasking: A Unified Variational Framework for Synergistic Multimodal Learning

    Nov 5, 2026Liangjian Wen, Qun Dai, Jiang Duan +7Multimodal LearningMultimodal Fusion

  17. ELPAC: Endpoint-Anchored Latent Progression with Stage-Varying Multimodal Coordination

    Nov 5, 2026Shengxian Ding, Yifei Zhang, Xinyuan Tian +2Disease Progression ModelingLatent Variable Models

  18. Multimodal Fusion Methods for Whole-Slide Images and Clinical Data in Inflammatory Bowel Disease

    Nov 3, 2026Ankana Banerjee, Mark van der Giezen, Ole Gunnar Aasprong +4Multimodal FusionData Fusion

  19. Learning Behaviorally Relevant Dynamics in Multimodal Neural Recordings

    Nov 3, 2026Parastoo Azizeddin, Eray Erturk, Maryam M. ShanechiMultimodal FusionDynamical Systems

  20. Fuse To Align: Multimodal Representation Learning for Retrieval via Hypergraph Regularization

    Nov 2, 2026Anindya Nag, Ambuj Mehrish, Sebastiano VasconCross-Modal Representation LearningCross-Modal Retrieval

  21. Cross-Layer Refinement and Frequency-aware Fusion Network for Multi-Source Remote Sensing Image Classification

    Nov 2, 2026Wenhan Xiao, Chuanzheng Gong, Feng Gao +2Remote Sensing Image UnderstandingMultimodal Fusion

  22. Shared-Pattern Collaborative Network for Multi-Source Remote Sensing Data Classification

    Nov 2, 2026Chenfang Yang, Chuanzheng Gong, Feng Gao +2Remote SensingMultimodal Classification

  23. HistoTx: Early fusion of H&E images and spatial transcriptomics at varying spatial transcriptomics resolution with self-supervised learning

    Nov 2, 2026Alice Driessen, Emil Malmsten, Steffen Wolf +11Multimodal FusionMultimodal Learning

  24. MUPA2{}^2S: Multimodal Unified Perception with Asymmetric Attention for Stress Assessment

    Nov 1, 2026Stefanos Gkikas, Thomas Kassiotis, Yu Fang +8Multimodal Fusion

  25. PhoneValue-Bench: A Multimodal Benchmark and Datasets for Consumer Smartphone Recommerce Valuation

    Oct 31, 2026Zihan Deng, Jun GaoMultimodal DatasetMultimodal Model Evaluation

  26. DiFuse: Fusing Native Diffusion Representations for Spatial Reasoning in Unified Multimodal Models

    Oct 29, 2026Ruixin Fan, Weining Wang, Jiahui Sun +2Multimodal Spatial ReasoningUnified Multimodal Models

  27. MOSAIC-Flow: Orthogonal-Axis and Status-Conditioned Fusion for Missing-Sequence Completion in Brain Tumor MRI

    Oct 29, 2026Pengli Zhu, Zhihao Peng, Yixuan YuanBrain MRI3D Medical Imaging

  28. MemoryFusion: Cross-Temporal Memory Learning for Multimodal Video Fusion

    Oct 29, 2026Meiqi Gong, Hao Zhang, Jiayi MaMultimodal FusionTemporal Consistency