Overview

Multimodal fusion studies how to combine inputs such as images, text, audio, medical scans, sensor streams, and structured records so models can use their complementary information. Common approaches include early or late fusion, cross-attention and contrastive alignment, and adaptive gates or mixtures of experts that weight modalities by task, sample, or estimated reliability. Methods are tested across settings such as BraTS medical segmentation, CMU-MOSI and CMU-MOSEI sentiment analysis, RGB-thermal detection, and clinical prediction, with growing attention to missing, noisy, or misaligned inputs. A central open tension is that additional modalities can help when they add useful evidence, but can also introduce noise or let a dominant modality suppress weaker ones; robust gains and generalization beyond benchmark conditions remain difficult to establish.

Latest papers

CardsList

805 papers47 papers in the last four weeks, down 47% on the four weeks before. 0.4% of all new papers.

  1. Alpha-TCM: A Nine-Layer Framework for TCM-Guided Multimodal Inference and Structured-Prior Auditing

    Oct 28, 2026Yucheng Liu, Everett X. WangMultimodal ReasoningMultimodal Fusion

  2. Brain-Behavior Modeling from Multimodal MRI using Opportunistic Pretraining and Common-Cohort Adaptation

    Oct 28, 2026Md. Tanvir Rahman, Taslima Ferdaus Shuva, Nabil Anan Orka +2Multimodal FusionMultimodal Pretraining

  3. 3D-SceneQ: Empowering 3D LLM With Query-Guided Adaptive Pruning and Multi-Modal Feature Enhancement

    Oct 27, 2026Yongqi Shan, Lu Zhang, Jiazuo Yu +23D Scene Understanding3D VLMs

  4. MM-Net: A Multimodal Multi-Task Network for Sleep Staging and Sleep Apnea Assessment After Ischemic Stroke

    Oct 27, 2026Md Wahiduzzaman Suva, Esm E Moula Chowdhury Abha, Md Imtiaj Alam Sajin +2Sleep Stage ClassificationMultimodal Learning

  5. When Slides Met Scans: Cross-Attention for Renal Cancer Prognosis

    Oct 26, 2026Anusha Krishnamoorthy, Basudha PalCancer Survival PredictionMultimodal Survival Analysis

  6. When Fusion Order Matters: Insights on Progressive Multimodal Representation Learning for Cross-modal Retrieval

    Oct 26, 2026Haoxiang Zhao, Bin Wang, Yaofei Wang +3Multimodal FusionCross-Modal Retrieval

  7. Efficient Multi-modal Video Fusion Network with Dynamic Region-Guided Temporal Interaction

    Oct 26, 2026Jingxue Huang, Xiaosong Li, Yi LinMultimodal Fusion

  8. I2^2-KAN: Interpretable Intra- and Inter-Modal Multi-Pathway Representation Learning for Multimodal Sentiment Analysis

    Oct 22, 2026Xiaoting Xie, Feng Qiao, Xianwei Zhang +5Multimodal Sentiment AnalysisMultimodal Fusion

  9. ICGF-CXRMate: Image-Conditioned Clinical Context Compression and Gated Fusion for Chest X-ray Report Generation

    Oct 22, 2026Vu Minh Tran, Thuc Huynh, Son Thanh Luu +1Gated Multimodal FusionRadiology Report Generation

  10. AGF-Net: A Robust Multimodal Emotion Recognition Model for Asynchronous Audio-Visual Signals Based on Predictive Dynamic Fusion with Negative Learning

    Oct 22, 2026Chuan Han Li, Yun Chien ChengMultimodal Emotion RecognitionMultimodal Fusion

  11. GAMF-Net: Grouped Availability-Aware Multimodal Fusion for Smartphone Skin Lesion Classification

    Oct 20, 2026Yue Zhuo, Jingjia Yan, Fangyu Liu +4Skin Lesion ClassificationMultimodal Classification

  12. HypTI: Hyperbolic Geometry-Aware Multimodal Feature Fusion Framework for Dementia Detection Based on Tabular-Image Data

    Oct 20, 2026Jianfei Xu, Huizhi Liang, Judith Harrison +3Multimodal FusionMultimodal Classification

  13. Revealing Inter-Channel Modality Discriminability: Channel Selective Interaction Fusion Network for Optical-SAR Object Detection

    Oct 20, 2026Haojie Li, Yuanfei Huang, Lingfei Song +2Object DetectionMultimodal Object Detection

  14. Progressive multi-scale tumor-state memory for incomplete multimodal brain tumor segmentation

    Oct 19, 2026Milad Rostamian, Mohammadreza Mohammadi, Mohsen SoryaniMissing-Modality Medical Image SegmentationMedical Image Segmentation

  15. RGB--Event Object Detection Across Operating Frequencies via Reliability-Supervised Routing and Asymmetric-Resolution Fusion

    Oct 19, 2026Peijun Shi, Jiannan Zhao, Xiujin Zhu +3RGB-Event Sensor FusionReliability-Aware Multimodal Fusion

  16. CheXFuse: Guided Differential Joint Attention Fusion for Robust Multimodal Chest X-Ray Classification in Data-Scarce Medical Settings

    Oct 19, 2026Muntaqim Ahmed Raju, Priyanka Siddappa, Md Shifat Haider Al Amin +2Multimodal FusionChest X-Ray Classification

  17. Shared-Private Cross-Modal Discrepancy Learning for Imbalanced Multi-Source Audio-Visual Deepfake Detection

    Oct 18, 2026Qiwu Wen, Jahangir AlamMultimodal Deepfake DetectionMultimodal Fusion

  18. DeliMF: Gated Cross-Attention Fusion of Structured Time Series and Clinical Text for Early Intensive Care Delirium Prediction

    Oct 17, 2026Yibo Yang, Zerui Huang, Haomiao Yang +1Clinical PredictionClinical Risk Prediction

  19. MSCR: Jointly Balancing Utilization Across Modalities and Discovering Synergistic Information

    Oct 17, 2026Xinyu Chen, Liangjian Wen, Jiang Duan +7Multimodal FusionModality Reliance in Multimodal Models

  20. Joint, Decoupled, or Agent: When Each Multimodal Alignment Paradigm Wins

    Oct 16, 2026Yi Fang, Gaurav Srivastava, Sirui Ding +1Multimodal FusionMultimodal Model Evaluation

  21. Enhancing RAG-based Medical VQA with Gated Multimodal Representations

    Oct 16, 2026Daniela Jimenez-Pano, Hugo Jair Escalante, Delia Irazú Hernández-FaríasGated Multimodal FusionMedical Visual Question Answering

  22. ODSS-Net: Orthogonally Decoupled Shared-Specific Representations for Multi-Modal Object Re-Identification

    Oct 16, 2026Zhifeng Ma, Yuhong Feng, Chongyi Huang +2Multimodal Disentangled Representation LearningMultimodal Fusion

  23. Reliability-Aware Multimodal Sentiment Prediction Under Local Modality Missingness

    Oct 14, 2026Minghao Kong, Deng Yao, Xi Zhang +4Multimodal Sentiment AnalysisMissing-Modality Learning

  24. Context-aware Bayesian Fusion for Multimodal Object Detection

    Oct 13, 2026Arunachalam Venkatachalam, Mhafuzul Islam, Marwan Yusuf +3Multimodal Object DetectionObject Detection

  25. GeoMER: A Joint Euclidean-Hyperbolic Geometry Framework for Multimodal Emotion Recognition

    Oct 13, 2026Ting Zhu, Huizhi Liang, Boguslaw Obara +2Multimodal Emotion RecognitionMultimodal Fusion

  26. Parameter-Efficient Multimodal Brain Tumor Segmentation With Stage-Wise Shared Adapters

    Oct 13, 2026Milad Rostamian, Mohammadreza Mohammadi, Ender Konukoglu +1Brain Tumor SegmentationCross-Modality Medical Image Segmentation