Overview

Multimodal learning asks how information from different sources can be combined to improve tasks such as diagnosis, retrieval, forecasting, and robot perception. Common approaches include feature fusion and cross-attention, contrastive alignment, self-supervised pretraining, and mixture-of-experts models that balance shared information with modality-specific cues. Foundation models increasingly support understanding and generation across several modalities, while domain-specific systems use paired or partially paired data and may use extra inputs during training even when they are unavailable at deployment. A central tension is that modalities can complement one another but can also be missing, noisy, misaligned, or unevenly used, so research focuses on robust fusion and evaluations that test whether models truly rely on the relevant evidence.

Latest papers

CardsList

358 papers21 papers in the last four weeks, down 53% on the four weeks before. 0.1% of all new papers.

  1. Hyperbolic Multimodal Continual Learning: A Closest-Admissible Solution

    Sep 24, 2026Jiahong Liu, Ming Shen, Xiaohao Liu +4Cross-Modal AlignmentHyperbolic Representation Learning

  2. PhyMo: A Physical-Field Modality for Multimodal AI4Physics

    Sep 23, 2026Henan Sun, Haitao Hu, Jin Liu +4AI for ScienceCross-Modal Representation Learning

  3. LingLan: An Advancing Traditional Chinese Medicine Diagnosis LLM with Multimodal Data

    Sep 22, 2026Zheng Chen, Zhicheng Du, Haoxuan Li +2Medical DiagnosisTraditional Chinese Medicine

  4. Incentive Noise and Structural Prior Infusion for Multi-modal Object Re-Identification

    Sep 21, 2026Weixiang Zhou, Yuhao Wang, Xingguo Xu +4Multimodal RobustnessCross-Modal Alignment

  5. GLR-MM: Graph-Based Global-Local Reconstruction for Robust Multimodal Chest X-ray and EHR Representation Learning under Missing Modalities

    Sep 20, 2026Surbhi Sharma, Nikhil Manali, Devesh MaheshwariMissing-Modality LearningGraph Neural Networks

  6. From Models to Systems: A Comprehensive Survey of Efficient Multimodal Learning

    Sep 16, 2026Pan Wang, Siwei Song, Hui Ji +12Efficient Multimodal InferenceMultimodal Optimization

  7. MCLC-NET: Multimodal Continual Learning for Leaf Counting

    Sep 16, 2026Ruchi Bhatt, Pratibha Kumari, Shreya Bansal +3Plant PhenotypingObject Counting

  8. Multimodal Conditioning of Fine-Tuned Stable Diffusion XL for Controllable and Culturally Faithful Ulos Motif Generation

    Sep 16, 2026Humasak Simanjuntak, Tamara Yunika Sianipar, Bronson T. M Siallagan +2Latent Diffusion ModelsImage Generation

  9. Hub-Spectral Activation of Latent Multimodal Knowledge

    Sep 15, 2026Ying Guo, Haidong Chen, Linrui Xu +7Cross-Modal Representation LearningCross-Modal Alignment

  10. AsyncCouple-Flow: Asynchronous Cross-Modal Coupling and Flow Matching for Spatio-Temporal Forecasting

    Sep 15, 2026Zhixiang Wu, Yining Liu, Bo Zhao +4Missing-Modality LearningSpatiotemporal Forecasting

  11. Through the Eyes of the Beholder: Biometric and Demographic Conditioning for Multimodal Sexism Detection

    Sep 14, 2026Ana-Maria Luisa Mocanu, Sebastian Mocanu, Ciprian-Octavian Truică +1Multimodal ClassificationMultimodal Learning

  12. PRI-Net: A Lightweight Multimodal Framework for 3D UAV Localization

    Sep 13, 2026Zhixuan Chen, Jialiang Lu, Zhong Ye +2Robot LocalizationCamera-LiDAR Fusion

  13. M3-Former: Multimodal Transformer with Mixture-of-Experts for Long-Term Vessel Trajectory Prediction

    Sep 11, 2026Wenzhe Jin, Haina TangMixture of ExpertsVessel Trajectory Prediction

  14. Reading the Whole Heart: Latent-Attention Masked Autoencoders for Multimodal Cardiac Representation Learning

    Sep 10, 2026Andrea Agostini, Simon Böhi, Moritz Vandenhirtz +7Masked AutoencodersHealthcare

  15. Enabling Vision and Cross-Modal Learning for Multimodal Stroke Recurrence Prediction: An Interpretable Two-Step Framework

    Sep 10, 2026Christian Gapp, Elias Tappeiner, Martin Welk +8Multimodal LearningMultimodal Fusion

  16. Morphological Decoupling-Based Skeletal Classification for Clinical Assessment of Malocclusion

    Sep 9, 2026Zhichun Jin, Zhicheng He, Hao Xu +4Graph Neural NetworksOcclusion Reasoning

  17. EEGBind: Detecting Source-Level Interictal Epileptiform Discharges via EEG-Centric Multimodal Binding

    Sep 9, 2026Muchen Li, Anglin Liu, Xuetian Gao +2ElectroencephalographyMultimodal Fusion

  18. Studying Image Tokenizers as Visual Languages in Unified Multimodal Models

    Sep 8, 2026Siting Li, Zhengyang Wang, Simon Shaolei Du +2Image TokenizationUnified Multimodal Models

  19. Dreaming in Flow: Bidirectional Self-Refinement for Unified Multimodal Models

    Sep 8, 2026Ke Hao, Yuanzhi Liang, Tingxi Chen +5Multimodal GroundingUnified Multimodal Models

  20. NeuCME: Toward Dynamic Multimodal Continual Learning via Neural Combinatorics of Multiple Experts

    Sep 7, 2026Kai Guo, Chuanbin Liu, Peng Hu +2Multimodal LearningCatastrophic Forgetting

  21. A Comprehensive Review of Multimodal Facial State Analysis: Tasks, Methods, and Resources

    Sep 6, 2026Xuri Ge, Tianshuo Zhang, Ruihan Li +6Facial Expression RecognitionFacial Action Unit Detection

  22. SMILE: Self-Explainable Multimodal Information Bottleneck for Medical Diagnosis

    Sep 4, 2026Yuqing Yang, Alexander Schmatz, Zhaozhao Ma +3Multimodal Model InterpretabilityMedical Diagnosis

  23. Urban Boundaries, Social Barriers: A Benchmark and Vision-Centric Framework for Mapping Gated Communities and Equity Implications

    Sep 3, 2026Minwei Zhao, Weiming Zhang, Jiawang Du +4Urban PlanningRemote Sensing Benchmarks

  24. NeoRed: A Knowledge-Logic-Alignment Multimodal Large Language Model for Neonatal Respiratory Disease Diagnosis

    Sep 3, 2026Yinan Liu, Hongtai Xia, Haoran Xu +4Radiology Report GenerationChest X-Ray Classification

  25. MARS: What Retrieval Signals Are Hidden in Multimodal Large Language Models for Text-Video Retrieval?

    Sep 2, 2026Uicheol Jung, Juyoung Hong, Geuntaek Lim +1Video-Text RetrievalMulti-Vector Retrieval

  26. Synergistic Information Disentanglement for Omni-modal Slide Representation Learning in Computational Pathology

    Sep 2, 2026Mingxin Liu, Chengfei Cai, Anwen Lu +5Multimodal SSLMedical Image Representation Learning