Overview

Multimodal learning asks how information from different sources can be combined to improve tasks such as diagnosis, retrieval, forecasting, and robot perception. Common approaches include feature fusion and cross-attention, contrastive alignment, self-supervised pretraining, and mixture-of-experts models that balance shared information with modality-specific cues. Foundation models increasingly support understanding and generation across several modalities, while domain-specific systems use paired or partially paired data and may use extra inputs during training even when they are unavailable at deployment. A central tension is that modalities can complement one another but can also be missing, noisy, misaligned, or unevenly used, so research focuses on robust fusion and evaluations that test whether models truly rely on the relevant evidence.

Latest papers

CardsList

359 papers19 papers in the last four weeks, down 59% on the four weeks before. 0.1% of all new papers.

  1. Prior-Fitted and Frozen Scorers for Multimodal Contrastive Learning

    Oct 10, 2026Tillmann Rheude, Roland Eils, Benjamin WildMultimodal Contrastive LearningMultimodal Learning

  2. HRIL: Learning Multimodal Synergy via Higher-Order Tensor Modeling

    Oct 8, 2026Qun Dai, Liangjian Wen, Jiang Duan +7Multimodal SSLMultimodal Learning

  3. PXtal: Learning to Align Powder X-Ray Diffraction and Crystal Structures under Information Asymmetry across Modalities

    Oct 7, 2026Zhuoran Yang, Christopher M. Collins, Bei Peng +3Cross-Modal AlignmentMultimodal Learning

  4. Temporal Visuo-Tactile Learning for Dexterous Grasp Stability

    Oct 7, 2026Ken Nakahara, Aleksei Buvailik, Prokhor Kotov +1Robotic GraspingTactile Sensing for Robotic Manipulation

  5. Unpaired Canonical Correlation Analysis

    Oct 7, 2026Nir Ben-Ari, Ronen Talmon, Uri ShahamCanonical Correlation AnalysisMulti-View Learning

  6. Mu-DisCoCat: A Variational Pipeline for Compositional Generalization on Quantum Processors

    Oct 6, 2026Mina Abbaszadeh, Matilda Karabina Moore, Raem Haq +2Quantum Machine LearningVariational Quantum Circuits

  7. On the Geometry of Multimodal Saturation: Riemannian VICReg

    Oct 5, 2026Nessim Ben Abbes, Duc Han Le, Sabri Mtibaa +1Self-Supervised LearningGeometric Representation Learning

  8. Multimodal Dual-Encoder Retrieval for Automated ICD Coding

    Oct 3, 2026Abhinav Bohra, Anuj BohraAutomated Medical CodingMultimodal Learning

  9. Transcriptome-informed multi-modal AI for predicting neoadjuvant therapy response from breast cancer biopsies

    Oct 2, 2026Jungkyu Park, Dhruva Biswas, Joseph Cappadona +22Clinical Outcome PredictionComputational Pathology

  10. ZeroMAG: Zero-Shot Multimodal Adapter Generation for Plug-and-Play EEG Foundation Models

    Oct 2, 2026Yubo Wang, Jingying Ma, Xinliang Zhou +8Multimodal LearningEEG Foundation Models

  11. FiberGeoText: A Vision-Language Model for Population- Level Organization of Superficial White Matter

    Oct 2, 2026Yuqian Chen, R. Jarrett Rushmore, Guikun Chen +5NeuroimagingMedical Image Representation Learning

  12. Harnessing Domain Specialists in Multimodal Mixture-of-Experts for Efficient Adaptation

    Oct 1, 2026Damiano Marsili, Raphi Kang, Aditya Mehta +2Mixture of ExpertsMixture-of-Experts Models

  13. Surface-volume self-supervised representation learning of brain MRI for genetic discovery

    Oct 1, 2026Tian Xia, Nuo Chen, Zihao Zhu +4Self-Supervised LearningMedical Image Representation Learning

  14. SpikeMoE: Brain-Inspired Competitive Routing for Flexible Spiking Mixture-of-Experts

    Oct 1, 2026Xiaoli Liu, Yujie Liang, Jialin Li +2Spiking Neural NetworksExpert Routing

  15. Can We Anticipate Violence? Multimodal Learning from Pre-Incident Behavioral Cues

    Sep 30, 2026Sindhuja Penchala, Mohammed Yusuf Mujawar, Noorbakhsh Amiri Golilarz +2Human Behavior PredictionMultimodal Learning

  16. Mutual Equilibrium: Multimodal Representation Learning through Reciprocal Feedback

    Sep 30, 2026Ho-min Park, Byungkon KangMultimodal GroundingMultimodal Embedding

  17. What to Attend, What to Keep: Skill-Conditioned Visuotactile Representation with Progress-Guided Event Memory

    Sep 29, 2026Amir-Hossein Shahidzadeh, Seungjae Lee, Eadom Dessalene +5Robot Task Progress EstimationContact-Rich Robotic Manipulation

  18. Beyond Layers: Position-Resolved Gradient Conflict and Position-Aware Modulation for Unified Multimodal Models

    Sep 29, 2026Shuyang Jiang, Fucheng Deng, Yuchuan Luo +1Unified Multimodal ModelsGradient Interference

  19. Seeing What Should Be Heard: Diagnosing and Repairing Cross-Modal Shortcuts in Omni-Modal LLMs

    Sep 29, 2026Yueran Ma, Ronghao LinAudio QAMultimodal Large Language Models

  20. Towards Breaking the Learning System Wall Using Multimodal Tutoring Transcriptions

    Sep 29, 2026Danielle R. Thomas, Marie Cynthia Abijuru Kamikazi, Ashish Gurung +5Intelligent Tutoring SystemsMultimodal Learning

  21. AdaKerNet: Neural Kernel Decoding for Task-Adaptive Prediction with Multimodal Large Models

    Sep 28, 2026Konstantinos D. Polyzos, Eleni Oikonomou, Tara JavidiMultimodal Large Language ModelsKernel Methods

  22. Paired Multimodal Scaling Laws

    Sep 28, 2026Marcus Ma, Shrikanth NarayananMultimodal ClassificationScaling Laws

  23. Structured Latent Modeling for Supervised Multimodal Information Decomposition

    Sep 28, 2026Wanting Huang, Sanvesh Srivastava, Weiran WangMultimodal Disentangled Representation LearningMultimodal Learning

  24. Advancing Wildlife Conservation through Multimodal Animal Re-Identification with Environmental Metadata

    Sep 28, 2026Yuzhuo Li, Di Zhao, Tingrui Qiao +3Animal Re-IdentificationWildlife Monitoring

  25. T-MoXAI: A Hierarchical Explainability Framework for Temporal Multimodal Data

    Sep 27, 2026Ali Inha, Mo Vali, Saaliha Vali +2Explainable Time Series ForecastingMultimodal Model Interpretability

  26. FORUM: Frozen Outputs Reconciled Using Model Agreement for Visual Grounding

    Sep 27, 2026Taiyo Sato, Takamasa Sanda, Keisuke Maeda +3Referring Expression ComprehensionVisual Grounding

  27. Learning Through Game: Skewed Transfer of Tabular Knowledge to Strengthen Image Model

    Sep 26, 2026Longfei Huang, Shangdong Yang, Yang YangCross-Modal Representation TransferMultimodal Learning

  28. UniAR: A Unified Framework for Autism Recognition Enhanced by Multi-View Prompt Learning

    Sep 25, 2026Lei Xin, Zeheng Wang, Jiayin Zhu +6Autism Spectrum DisorderPrompt-Based Multimodal Learning