Overview

Multimodal fusion studies how to combine inputs such as images, text, audio, medical scans, sensor streams, and structured records so models can use their complementary information. Common approaches include early or late fusion, cross-attention and contrastive alignment, and adaptive gates or mixtures of experts that weight modalities by task, sample, or estimated reliability. Methods are tested across settings such as BraTS medical segmentation, CMU-MOSI and CMU-MOSEI sentiment analysis, RGB-thermal detection, and clinical prediction, with growing attention to missing, noisy, or misaligned inputs. A central open tension is that additional modalities can help when they add useful evidence, but can also introduce noise or let a dominant modality suppress weaker ones; robust gains and generalization beyond benchmark conditions remain difficult to establish.

Latest papers

CardsList

805 papers50 papers in the last four weeks, down 43% on the four weeks before. 0.4% of all new papers.

  1. DiFuse: Fusing Native Diffusion Representations for Spatial Reasoning in Unified Multimodal Models

    Oct 29, 2026Ruixin Fan, Weining Wang, Jiahui Sun +2Multimodal Spatial ReasoningUnified Multimodal Models

  2. MOSAIC-Flow: Orthogonal-Axis and Status-Conditioned Fusion for Missing-Sequence Completion in Brain Tumor MRI

    Oct 29, 2026Pengli Zhu, Zhihao Peng, Yixuan YuanBrain MRI3D Medical Imaging

  3. MemoryFusion: Cross-Temporal Memory Learning for Multimodal Video Fusion

    Oct 29, 2026Meiqi Gong, Hao Zhang, Jiayi MaMultimodal FusionTemporal Consistency

  4. Alpha-TCM: A Nine-Layer Framework for TCM-Guided Multimodal Inference and Structured-Prior Auditing

    Oct 28, 2026Yucheng Liu, Everett X. WangMultimodal ReasoningMultimodal Fusion

  5. Brain-Behavior Modeling from Multimodal MRI using Opportunistic Pretraining and Common-Cohort Adaptation

    Oct 28, 2026Md. Tanvir Rahman, Taslima Ferdaus Shuva, Nabil Anan Orka +2Multimodal FusionMultimodal Pretraining

  6. 3D-SceneQ: Empowering 3D LLM With Query-Guided Adaptive Pruning and Multi-Modal Feature Enhancement

    Oct 27, 2026Yongqi Shan, Lu Zhang, Jiazuo Yu +23D Scene Understanding3D VLMs

  7. MM-Net: A Multimodal Multi-Task Network for Sleep Staging and Sleep Apnea Assessment After Ischemic Stroke

    Oct 27, 2026Md Wahiduzzaman Suva, Esm E Moula Chowdhury Abha, Md Imtiaj Alam Sajin +2Sleep Stage ClassificationMultimodal Learning

  8. When Slides Met Scans: Cross-Attention for Renal Cancer Prognosis

    Oct 26, 2026Anusha Krishnamoorthy, Basudha PalCancer Survival PredictionMultimodal Survival Analysis

  9. When Fusion Order Matters: Insights on Progressive Multimodal Representation Learning for Cross-modal Retrieval

    Oct 26, 2026Haoxiang Zhao, Bin Wang, Yaofei Wang +3Multimodal FusionCross-Modal Retrieval

  10. Efficient Multi-modal Video Fusion Network with Dynamic Region-Guided Temporal Interaction

    Oct 26, 2026Jingxue Huang, Xiaosong Li, Yi LinMultimodal Fusion

  11. I2^2-KAN: Interpretable Intra- and Inter-Modal Multi-Pathway Representation Learning for Multimodal Sentiment Analysis

    Oct 22, 2026Xiaoting Xie, Feng Qiao, Xianwei Zhang +5Multimodal Sentiment AnalysisMultimodal Fusion

  12. ICGF-CXRMate: Image-Conditioned Clinical Context Compression and Gated Fusion for Chest X-ray Report Generation

    Oct 22, 2026Vu Minh Tran, Thuc Huynh, Son Thanh Luu +1Gated Multimodal FusionRadiology Report Generation

  13. AGF-Net: A Robust Multimodal Emotion Recognition Model for Asynchronous Audio-Visual Signals Based on Predictive Dynamic Fusion with Negative Learning

    Oct 22, 2026Chuan Han Li, Yun Chien ChengMultimodal Emotion RecognitionMultimodal Fusion

  14. GAMF-Net: Grouped Availability-Aware Multimodal Fusion for Smartphone Skin Lesion Classification

    Oct 20, 2026Yue Zhuo, Jingjia Yan, Fangyu Liu +4Skin Lesion ClassificationMultimodal Classification

  15. HypTI: Hyperbolic Geometry-Aware Multimodal Feature Fusion Framework for Dementia Detection Based on Tabular-Image Data

    Oct 20, 2026Jianfei Xu, Huizhi Liang, Judith Harrison +3Multimodal FusionMultimodal Classification

  16. Revealing Inter-Channel Modality Discriminability: Channel Selective Interaction Fusion Network for Optical-SAR Object Detection

    Oct 20, 2026Haojie Li, Yuanfei Huang, Lingfei Song +2Object DetectionMultimodal Object Detection

  17. Progressive multi-scale tumor-state memory for incomplete multimodal brain tumor segmentation

    Oct 19, 2026Milad Rostamian, Mohammadreza Mohammadi, Mohsen SoryaniMissing-Modality Medical Image SegmentationMedical Image Segmentation

  18. RGB--Event Object Detection Across Operating Frequencies via Reliability-Supervised Routing and Asymmetric-Resolution Fusion

    Oct 19, 2026Peijun Shi, Jiannan Zhao, Xiujin Zhu +3RGB-Event Sensor FusionReliability-Aware Multimodal Fusion

  19. CheXFuse: Guided Differential Joint Attention Fusion for Robust Multimodal Chest X-Ray Classification in Data-Scarce Medical Settings

    Oct 19, 2026Muntaqim Ahmed Raju, Priyanka Siddappa, Md Shifat Haider Al Amin +2Multimodal FusionChest X-Ray Classification

  20. Shared-Private Cross-Modal Discrepancy Learning for Imbalanced Multi-Source Audio-Visual Deepfake Detection

    Oct 18, 2026Qiwu Wen, Jahangir AlamMultimodal Deepfake DetectionMultimodal Fusion

  21. DeliMF: Gated Cross-Attention Fusion of Structured Time Series and Clinical Text for Early Intensive Care Delirium Prediction

    Oct 17, 2026Yibo Yang, Zerui Huang, Haomiao Yang +1Clinical PredictionClinical Risk Prediction

  22. MSCR: Jointly Balancing Utilization Across Modalities and Discovering Synergistic Information

    Oct 17, 2026Xinyu Chen, Liangjian Wen, Jiang Duan +7Multimodal FusionModality Reliance in Multimodal Models

  23. Joint, Decoupled, or Agent: When Each Multimodal Alignment Paradigm Wins

    Oct 16, 2026Yi Fang, Gaurav Srivastava, Sirui Ding +1Multimodal FusionMultimodal Model Evaluation

  24. Enhancing RAG-based Medical VQA with Gated Multimodal Representations

    Oct 16, 2026Daniela Jimenez-Pano, Hugo Jair Escalante, Delia Irazú Hernández-FaríasGated Multimodal FusionMedical Visual Question Answering

  25. ODSS-Net: Orthogonally Decoupled Shared-Specific Representations for Multi-Modal Object Re-Identification

    Oct 16, 2026Zhifeng Ma, Yuhong Feng, Chongyi Huang +2Multimodal Disentangled Representation LearningMultimodal Fusion