Overview

Multimodal learning asks how information from different sources can be combined to improve tasks such as diagnosis, retrieval, forecasting, and robot perception. Common approaches include feature fusion and cross-attention, contrastive alignment, self-supervised pretraining, and mixture-of-experts models that balance shared information with modality-specific cues. Foundation models increasingly support understanding and generation across several modalities, while domain-specific systems use paired or partially paired data and may use extra inputs during training even when they are unavailable at deployment. A central tension is that modalities can complement one another but can also be missing, noisy, misaligned, or unevenly used, so research focuses on robust fusion and evaluations that test whether models truly rely on the relevant evidence.

Latest papers

CardsList

358 papers21 papers in the last four weeks, down 53% on the four weeks before. 0.1% of all new papers.

  1. HRIL: Learning Multimodal Synergy via Higher-Order Tensor Modeling

    Oct 8, 2026Qun Dai, Liangjian Wen, Jiang Duan +7Multimodal SSLMultimodal Learning

  2. PXtal: Learning to Align Powder X-Ray Diffraction and Crystal Structures under Information Asymmetry across Modalities

    Oct 7, 2026Zhuoran Yang, Christopher M. Collins, Bei Peng +3Cross-Modal AlignmentMultimodal Learning

  3. Temporal Visuo-Tactile Learning for Dexterous Grasp Stability

    Oct 7, 2026Ken Nakahara, Aleksei Buvailik, Prokhor Kotov +1Robotic GraspingTactile Sensing for Robotic Manipulation

  4. Unpaired Canonical Correlation Analysis

    Oct 7, 2026Nir Ben-Ari, Ronen Talmon, Uri ShahamCanonical Correlation AnalysisMulti-View Learning

  5. Mu-DisCoCat: A Variational Pipeline for Compositional Generalization on Quantum Processors

    Oct 6, 2026Mina Abbaszadeh, Matilda Karabina Moore, Raem Haq +2Quantum Machine LearningVariational Quantum Circuits

  6. On the Geometry of Multimodal Saturation: Riemannian VICReg

    Oct 5, 2026Nessim Ben Abbes, Duc Han Le, Sabri Mtibaa +1Self-Supervised LearningGeometric Representation Learning

  7. Multimodal Dual-Encoder Retrieval for Automated ICD Coding

    Oct 3, 2026Abhinav Bohra, Anuj BohraAutomated Medical CodingMultimodal Learning

  8. Transcriptome-informed multi-modal AI for predicting neoadjuvant therapy response from breast cancer biopsies

    Oct 2, 2026Jungkyu Park, Dhruva Biswas, Joseph Cappadona +22Clinical Outcome PredictionComputational Pathology

  9. ZeroMAG: Zero-Shot Multimodal Adapter Generation for Plug-and-Play EEG Foundation Models

    Oct 2, 2026Yubo Wang, Jingying Ma, Xinliang Zhou +8Multimodal LearningEEG Foundation Models

  10. FiberGeoText: A Vision-Language Model for Population- Level Organization of Superficial White Matter

    Oct 2, 2026Yuqian Chen, R. Jarrett Rushmore, Guikun Chen +5NeuroimagingMedical Image Representation Learning

  11. Harnessing Domain Specialists in Multimodal Mixture-of-Experts for Efficient Adaptation

    Oct 1, 2026Damiano Marsili, Raphi Kang, Aditya Mehta +2Mixture of ExpertsMixture-of-Experts Models

  12. Surface-volume self-supervised representation learning of brain MRI for genetic discovery

    Oct 1, 2026Tian Xia, Nuo Chen, Zihao Zhu +4Self-Supervised LearningMedical Image Representation Learning

  13. SpikeMoE: Brain-Inspired Competitive Routing for Flexible Spiking Mixture-of-Experts

    Oct 1, 2026Xiaoli Liu, Yujie Liang, Jialin Li +2Spiking Neural NetworksExpert Routing

  14. Can We Anticipate Violence? Multimodal Learning from Pre-Incident Behavioral Cues

    Sep 30, 2026Sindhuja Penchala, Mohammed Yusuf Mujawar, Noorbakhsh Amiri Golilarz +2Human Behavior PredictionMultimodal Learning

  15. Mutual Equilibrium: Multimodal Representation Learning through Reciprocal Feedback

    Sep 30, 2026Ho-min Park, Byungkon KangMultimodal GroundingMultimodal Embedding

  16. What to Attend, What to Keep: Skill-Conditioned Visuotactile Representation with Progress-Guided Event Memory

    Sep 29, 2026Amir-Hossein Shahidzadeh, Seungjae Lee, Eadom Dessalene +5Robot Task Progress EstimationContact-Rich Robotic Manipulation

  17. Beyond Layers: Position-Resolved Gradient Conflict and Position-Aware Modulation for Unified Multimodal Models

    Sep 29, 2026Shuyang Jiang, Fucheng Deng, Yuchuan Luo +1Unified Multimodal ModelsGradient Interference

  18. Seeing What Should Be Heard: Diagnosing and Repairing Cross-Modal Shortcuts in Omni-Modal LLMs

    Sep 29, 2026Yueran Ma, Ronghao LinAudio QAMultimodal Large Language Models

  19. Towards Breaking the Learning System Wall Using Multimodal Tutoring Transcriptions

    Sep 29, 2026Danielle R. Thomas, Marie Cynthia Abijuru Kamikazi, Ashish Gurung +5Intelligent Tutoring SystemsMultimodal Learning

  20. AdaKerNet: Neural Kernel Decoding for Task-Adaptive Prediction with Multimodal Large Models

    Sep 28, 2026Konstantinos D. Polyzos, Eleni Oikonomou, Tara JavidiMultimodal Large Language ModelsKernel Methods

  21. Paired Multimodal Scaling Laws

    Sep 28, 2026Marcus Ma, Shrikanth NarayananMultimodal ClassificationScaling Laws

  22. Structured Latent Modeling for Supervised Multimodal Information Decomposition

    Sep 28, 2026Wanting Huang, Sanvesh Srivastava, Weiran WangMultimodal Disentangled Representation LearningMultimodal Learning

  23. Advancing Wildlife Conservation through Multimodal Animal Re-Identification with Environmental Metadata

    Sep 28, 2026Yuzhuo Li, Di Zhao, Tingrui Qiao +3Animal Re-IdentificationWildlife Monitoring

  24. T-MoXAI: A Hierarchical Explainability Framework for Temporal Multimodal Data

    Sep 27, 2026Ali Inha, Mo Vali, Saaliha Vali +2Explainable Time Series ForecastingMultimodal Model Interpretability

  25. FORUM: Frozen Outputs Reconciled Using Model Agreement for Visual Grounding

    Sep 27, 2026Taiyo Sato, Takamasa Sanda, Keisuke Maeda +3Referring Expression ComprehensionVisual Grounding

  26. Learning Through Game: Skewed Transfer of Tabular Knowledge to Strengthen Image Model

    Sep 26, 2026Longfei Huang, Shangdong Yang, Yang YangCross-Modal Representation TransferMultimodal Learning

  27. UniAR: A Unified Framework for Autism Recognition Enhanced by Multi-View Prompt Learning

    Sep 25, 2026Lei Xin, Zeheng Wang, Jiayin Zhu +6Autism Spectrum DisorderPrompt-Based Multimodal Learning

  28. A Multimodal Dataset for Survival Prediction in Resected Pancreatic Ductal Adenocarcinoma

    Sep 24, 2026Anh-Tien Nguyen, Mawuko Tettey, Jacqueline Michelle Metsch +14Cancer Survival PredictionOncology