Multimodal Learning

Ask about this topic

Momentum

14 papers in the last four weeks, up 250% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 113

All topics
CardsList

  1. InstMoE: Adaptive Multimodal Routing with Specialized Experts

    Oct 4, 2026Guimin Hu, Xiang He, Yingjian Li +3Multimodal Learning

    Guangdong University of Technology · University of Copenhagen · Pengcheng Laboratory +1

  2. Can We Anticipate Violence? Multimodal Learning from Pre-Incident Behavioral Cues

    Sep 30, 2026Sindhuja Penchala, Mohammed Yusuf Mujawar, Noorbakhsh Amiri Golilarz +2Fine-Grained Video UnderstandingViolence Detection

    Department of Computer Science, The University of Alabama, Tuscaloosa, AL 35487, USA

  3. VISTA: Internalizing Collective Visual Experience via On-Policy Distillation for Active Multimodal Agents

    Sep 29, 2026Zheng Jiang, Houde Qian, Yiming Chen +5Multimodal AgentsMultimodal Learning

    Tsinghua University

  4. Task-Oriented Visual Feature Compression via Residual Vector Quantization for Device-Edge Multimodal Inference

    Sep 29, 2026Luning Pang, Cheng Yuan, Jiawei Shao +2Residual Vector QuantizationMultimodal Learning

    Department of Electronic Engineering, and Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing 100084, China · Institute of Artificial Intelligence (TeleAI), China Telecom, Beijing 100033, China

  5. Structured Latent Modeling for Supervised Multimodal Information Decomposition

    Sep 28, 2026Wanting Huang, Sanvesh Srivastava, Weiran WangMultimodal LearningMultimodal Classification

    Department of Computer Science University of Iowa · Department of Statistics University of Iowa

  6. ReSight-SMC: Two-Stage Power Sampling via Island SMC with Visual Scouts

    Sep 28, 2026Yaowen Zhang, Xiangyu Qiu, Junyi Hu +5Recent Vision-Language ModelsMultimodal Learning

    Independent Researcher · University of Electronic Science and Technology of China · Tsinghua University

  7. PARSEE-VAD: Efficient Training-Free Online Video Anomaly Detection via Proposition-Aware Reasoning and Streaming Evidence Escalation

    Sep 27, 2026Ji Wang, Shuangqing Zhang, Guo-Sen Xie +1Training-Free Video Anomaly DetectionMultimodal Learning

    Global Institute of Future Technology Shanghai Jiao Tong University Shanghai, 200240, China · School of Intelligence Science and Technology Nanjing University Suzhou, 215163, China · School of Computer Science and Engineering Nanjing University of Science and Technology Nanjing, 210094, China

  8. Hyperbolic Multimodal Continual Learning: A Closest-Admissible Solution

    Sep 24, 2026Jiahong Liu, Ming Shen, Xiaohao Liu +4Multimodal LearningReplay-Based Continual Learning

    The Chinese University of Hong Kong · National University of Singapore · Yale University +1

  9. Confidence Falls Short: Asymmetric Certainty Gains from Optimization Hinder Multimodal Classification

    Sep 23, 2026Longfei Huang, Xiangyu Wu, Yang YangMultimodal LearningMultimodal Classification

    Nanjing University of Science and Technology · Alibaba Group

  10. VCMM: Variance-Calibrated Momentum for Multimodal Learning

    Sep 23, 2026Zhongjing Gu, Chenyang Huang, Yufa Feng +3Multimodal LearningModalities

  11. GeoBalance: Geometry-Aware Monitoring and Reconstruction with Asymmetric Optimization for Balanced Multimodal Learning

    Sep 20, 2026Zechang Xiong, Da Li, Rong Yin +7Multimodal LearningMultimodal Classification

    Beihang University, Beijing, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · Amap, Alibaba Group, Beijing, China +3

  12. From Models to Systems: A Comprehensive Survey of Efficient Multimodal Learning

    Sep 16, 2026Pan Wang, Siwei Song, Hui Ji +12Multimodal LearningMultimodal Model

    University of Pittsburgh · New York University · Carnegie Mellon University +6

  13. TwinICL: Diagnosing Multimodal In-Context Learning through Paired Counterfactuals

    Sep 14, 2026Zihan Xue, Po-Yi Lu, Serhii Honcharenko +5Multimodal LearningIn-Context Learning

    University of California, Los Angeles · 2National Taiwan University · 3Texas A&M University +2

  14. Exploring Diffusion Transformers for Cross-Modal Augmentation in Multimodal Brain State Decoding

    Sep 12, 2026Ziwei Wang, Xingyi He, Hongbin Wang +3Cross-Modal AttentionMultimodal Learning

    School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, China

  15. Contribution-Aware Bandwidth Allocation for Multimodal Split Learning

    Sep 1, 2026Iason Ofeidis, Leandros TassiulasMultimodal LearningPower Allocation

    Department of Electrical and Computer Engineering, Yale University, New Haven, CT, USA

  16. Modality Disentangled Learning for Incomplete Multimodal Emotion Recognition: A Primitive Memory Distillation Perspective

    Aug 31, 2026Jiaqi Zhang, Zheng Pang, Mengting Li +9Multimodal LearningEmotion Recognition

    Jiangsu University · Griffith University · National University of Singapore +8

  17. MAG: MAnifold Guided Semi-Supervised Multi-modal In-Context Learning

    Aug 13, 2026Zirui Cheng, Xun Xu, Tiankai Chen +7Multimodal Large Language ModelsMultimodal Learning

    College of Design and Engineering, National University of Singapore · Institute for Infocomm Research, ASTAR, Singapore

  18. Multimodal Item Parameter Estimation using Simulated Response Probabilitie

    Aug 10, 2026Christopher Ormerod, YoungKoung KimItem Response TheoryMultiple-Choice Questions

    College Board

  19. iStructTab: Structured Feature Sequencing for Multimodal Learning of Image and Tabular Data

    Aug 5, 2026Al Zadid Sultan Bin Habib, Md Younus Ahamed, Prashnna Gyawali +2Multimodal LearningTabular Data

    Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV 26506, USA · Scientific Computing and Imaging Institute & Department of Biomedical Informatics, The University of Utah, Salt Lake City, UT 84112, USA

  20. CIGTSurv: Clinical Information Guided Tri-modal Survival Prediction with Local Prototype Association and Global Feature Alignment

    Aug 4, 2026Jing Dai, Qibin Zhang, Weiwei Zhou +4Survival PredictionsMultimodal Clinical Data

    Cancer Hospital of Dalian University of Technology, Shenyang, China · School of Biomedical Engineering, Faculty of Medicine, Dalian University of Technology, Dalian, China · Institute of Pathology, Technical University of Munich, TUM School of Medicine and Health, Munich, Germany +2

  21. Understanding and Overcoming Cross-modal Fusion Bias in Multimodal Anomaly Detection From A Fisher Information Perspective

    Aug 2, 2026Kaifang Long, Lianbo Ma, Liming Liu +1Cross-Modal FusionMultimodal Anomaly Detection

    Northeastern University Shenyang, China · CATL Ningde, China

  22. Understanding Submodular Information Measure Based Objectives for Representation Learning: A Variance and Separation Perspective

    Jul 30, 2026Rishabh Iyer, Truong Pham, Anay MajeeMutual InformationRepresentational Similarity Analysis

    The University of Texas at Dallas, Richardson, TX, USA · Adobe, San Jose, CA, USA

  23. Regularizing modality contribution drift in multimodal continual learning

    Jul 29, 2026Zhen Zhang, Jielei Chu, Bin Liu +1Multimodal LearningContinual Learning

    School of Computing and Artificial Intelligence, Southwest Jiaotong University

  24. DICA: Dual-Indicator Guided Contrastive Alignment in Multimodal Large Language Models

    Jul 27, 2026Hao Yang, Jin Wang, Xuejie ZhangMultimodal Large Language ModelsMultimodal Learning

    School of Information Science and Engineering Yunnan University Kunming, China

  25. FedTaste: Topology-Aware Structural Transfer for Multimodal Federated Learning with Missing Modalities

    Jul 25, 2026Haochen Liang, Jie Zhang, Hideya OchiaiMultimodal LearningMissing Modalities

    The University of Tokyo Tokyo, Japan · Great Bay University Dongguan, China

  26. Beyond Independent Optimization: Compression, MoE Routing, and Quantization Interactions in Multimodal Edge Intelligence

    Jul 23, 2026Jay Gor, Karm Dave, Akshita Abrol +3Multimodal LearningIntelligent Edge

    Nirma University · Singapore Institute of Technology, Singapore · Department of CE-AI and Big Data, Marwadi University

  27. Adaptive Confidence-weighted Expansion for Trustworthy Multi-Omics Multimodal Fusion

    Jul 22, 2026Mohammad Raahemi, Ali Sekhavati, Alireza Maleki +1Multimodal FusionMultimodal Learning

    Faculty of Engineering, University of Ottawa, Ottawa ON K1N 1A2, Canada · RIV Lab, Department of Computer Engineering, Bu-Ali Sina University, Hamedan, Iran · School of Computing and Communications, Lancaster University, Lancaster, UK

  28. MultiLoReFT: Decoupling Shared and Modality-Specific Subspaces in Multimodal Learning via Low-Rank Representation Fine-Tuning

    Jul 18, 2026Sana Tonekaboni, Viktoria Schuster, Caroline UhlerMultimodal LearningModalities

    Massachusetts Institute of Technology, Cambridge, MA, USA · Eric and Wendy Schmidt Center, Broad Institute of MIT and Harvard, Cambridge, MA, USA · Vector Institute, Toronto, Canada +1

  29. Capacity and Redundancy Trade-offs in Multi-Task Learning

    Jul 17, 2026Asif KhanMulti--Task LearningRedundancy

    1Harvard Medical School, Boston, USA

  30. ShapKO: Shapley-Adaptive Modality Knockout for Robust Multimodal Learning

    Jul 10, 2026Nusrat Binta Nizam, Fengbei Liu, Sunwoo Kwak +3Multimodal LearningMultimodal Clinical Data

    Cornell University, New York, NY, USA · Cornell Tech, New York, NY, USA · Weill Cornell Medicine, New York, NY, USA

  31. One-Shot Multimodal Learning from Demonstration with Force-Constrained Elastic Maps

    Jul 10, 2026Brendan Hertel, Jonathan Spanos, Navya Garg +1Ground Reaction ForcesReal-World Manipulation Tasks

    Persistent Autonomy and Robot Learning (PeARL) Lab, University of Massachusetts Lowell, Lowell, MA 01854, USA.

  32. The Importance of Encoder Choice:A Tabular-Image Study

    Jul 8, 2026Ilia Koloiarov, Diego Coello de Portugal Mecke, Vijaya Krishna Yalavarthi +2Multimodal LearningEncoders

    ISMLL & VWFS DARC · University of Hildesheim · Hildesheim, Germany +1

  33. General Incomplete Multimodal Learning via Dynamic Quality Perception

    Jul 8, 2026Xiangyu Meng, Shicai WeiMultimodal LearningMissing Modalities

    University of Electronic Science and Technology of China

  34. Integrating Neural Encoders in Bayesian Generalized Linear Mixed Models for Multimodal Data

    Jul 6, 2026Yuankang Zhao, Youngsoo Baek, Felipe A. Medeiros +2Generalized Linear ModelMultimodal Learning

    Department of Biostatistics & Bioinformatics, Duke University · Department of Ophthalmology, University of Miami

  35. Global Logic and Local Search: Dual-Stream Multimodal In-Context Learning for Verifiable Industrial Anomaly Detection

    Jul 4, 2026Runzhi Deng, Yundi Hu, Yiming Zhong +5Multimodal Industrial Anomaly DetectionReference-Guided Multimodal In-Context Verification

    Nanjing University, Nanjing, Jiangsu, China · China Mobile Zijin Innovation Institute, Nanjing, Jiangsu, China · Southeast University, Nanjing, Jiangsu, China

  36. CL-Anomaly: Layer-Adaptive Mixture-of-Experts with Multimodal Large Language Model for Continual Learning in Anomaly Detection

    Jul 3, 2026Wen Dong, Zhao Wang, Shuangqing Zhang +5Replay-Based Continual LearningLogical Anomalies

    Nanjing University, Nanjing, China · China Mobile Zijin Innovation Institute, Nanjing, China · Nanjing University of Science and Technology, Nanjing, China

  37. Hidden Forgetting in Continual Multimodal Learning: When Accuracy Survives but Grounding Fails

    Jul 2, 2026Qianyu Chen, Canran Xiao, Runxuan TangMultimodal LearningCatastrophic Forgetting

    1Nanyang Technological University · 2Shenzhen Campus of Sun Yat-sen University

  38. FedLAB: Traceable Semantic Codebooks for Federated Multimodal Graph Foundation Learning

    Jun 30, 2026Zekai Chen, Kairui Yang, Xuaner Chen +4Federated Graph LearningGraph Foundation Model

    Beijing Institute of Technology, Beijing, China

  39. Fusing Complementary Multi-view Features for Screen-Based Eye Tracking

    Jun 30, 2026Chang Liu, Jiaqi Liu, Chengwen Zhang +6Gaze Target EstimationMultimodal Dataset

    Tsinghua University · Georgia Institute of Technology

  40. Learning from Reliable Latent Prompts for Visual Recognition with Missing Modalities

    Jun 29, 2026Taixi Chen, Nancy GuoMultimodal LearningMissing Modalities

    School of Computing State University of New York at Binghamton

  41. Residual-Guided Expert Specialization for Incomplete Multimodal Learning

    Jun 29, 2026Seunghun Baek, Jihwan Park, Jaeyoon Sim +3Missing ModalitiesMultimodal Learning

    Pohang University of Science and Technology (POSTECH), South Korea

  42. Modeling Local, Global, and Cross-Modal Context in Multimodal 3D MRI

    Jun 25, 2026Minh Duc Do, Tillmann Rheude, Noel Kronenberg +2Magnetic Resonance ImagingMultimodal Learning

    1Berlin Institute of Health at Charité - Universitätsmedizin Berlin · 2Health Data Science Unit, Heidelberg University Hospital and BioQuant · 3Intelligent Medicine Institute, Fudan University +1

  43. MultiMem: Measuring and Mitigating Memorization in Multi-Modal Contrastive Learning

    Jun 20, 2026Wenhao Wang, Franziska Boenisch, Michael Backes +1Multimodal Contrastive LearningMultimodal Memory

    CISPA Helmholtz Center for Information Security

  44. Resolving Multi-Target Association in OFDM-based ISAC via Vision-aided Multi-Modal Learning

    Jun 20, 2026Meng Hua, Chenghong Bian, Deniz GunduzIntegrated Sensing And CommunicationDoppler Velocity Log

    Department of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, U.K. · Department of Electrical and Computer Engineering, Hong Kong University of Science and Technology, Hong Kong.

  45. Beyond Time Series: Spatial Reasoning for Epidemic Forecasting via Multimodal Learning

    Jun 20, 2026Diana Guadalupe Gomez, Chenwei Wu, Zhiyi Wang +2Epidemiological ModelsSpatio-Temporal Forecasting

    University of Michigan Ann Arbor, MI, USA

  46. SAGE: An Expert-Annotated South Asian GI Endoscopy Dataset for Multimodal Learning and Hallucination Analysis

    Jun 20, 2026Niyoj Oli, Sachin Acharya, Sandesh Pokhrel +7Colorectal CancerColonoscopy

    Nepal Applied Mathematics and Informatics Institute for Research, Nepal · University of Utah, USA · GastroIntestinal Department, Dhulikhel Hospital, Nepal +3

  47. Revisiting LLM Adaptation for 3D CT Report Generation: A Study of Scaling and Diagnostic Priors

    Jun 15, 2026Vanshali Sharma, Andrea M. Bejar, Halil Ertugrul Aktas +4Large Language Model AdaptationCone-Beam Computed Tomography

    Department of Radiology Northwestern University Chicago, USA · Department of Computer Science Aalto University, Finland · Department of Computer Science University of South Dakota South Dakota, USA

  48. SPICE: Synergy and Partial Information Based Curriculum Evolution

    Jun 15, 2026Ankush Pratap Singh, Houwei Cao, Yong LiuMultimodal LearningCurriculum

    New York Institute of Technology · New York, NY, USA · New York University

  49. Unsupervised Learning for Missing Modalities in Multimodal Learning

    Jun 14, 2026Hassan Ismkhan, Hamid BouchahciaMissing ModalitiesMultimodal Learning

    Bournemouth University Bournemouth, UK

  50. Multimodal Graph Negative Learning

    Jun 11, 2026Zhengyu Wu, Xu Wang, Hongchao Qin +4Multimodal LearningProcess-Level Supervision

  51. Latent World Recovery for Multimodal Learning with Missing Modalities

    Jun 10, 2026Hui Wang, Tianyu Ren, Joseph Butler +3Missing ModalitiesMultimodal Learning

    Queen’s University Belfast, Belfast, United Kingdom

  52. Information-Theoretic Decomposition for Multimodal Interaction Learning

    Jun 10, 2026Zequn Yang, Yake Wei, Haotian Ni +2Multimodal LearningInformation-Theoretic Limits

    Gaoling School of Artificial Intelligence, Renmin University of China, Beijing · Beijing Key Laboratory of Research on Large Models and Intelligent Governance · Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE +2

  53. When to Align, When to Predict: A Phase Diagram for Multimodal Learning

    Jun 9, 2026Ilay Kamai, Hugues Van Assel, Aviv Regev +2Multimodal LearningMultimodal Representations

    Technion · Genentech · Brown University

  54. Listen, Look, and Learn: Learning Without Forgetting through SAM-Audio

    Jun 9, 2026Avi Gupta, Nilotpal Sinha, Vishnu Raj +4Class-Incremental LearningCatastrophic Forgetting

    IIIT Delhi · Dolby Laboratories · IIT Kanpur +1

  55. PRISM: Topology-Aware Cross-Modal Imputation for Modality-Deficient Federated Graph Learning

    Jun 8, 2026Zekai Chen, Miao Zhang, Jiayang Xing +4Federated Graph LearningMissing Modalities

    Beijing Institute of Technology, Beijing, 100081, China

  56. Self-supervised Learning Matters: A Simple Ensemble Solution for Micro-Gesture Recognition

    Jun 8, 2026Tingyi Liu, Kun Li, Fei Wang +5Self-Supervised LearningHand Gesture Recognition

    Hefei University of Technology, Hefei, China · United Arab Emirates University, Al Ain, United Arab Emirates · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, Hefei, China +5

  57. LongMoE: Longitudinal Multimodal Learning via Trajectory-Aware Mixture-of-Experts

    Jun 6, 2026Maxx Richard Rahman, Prakhar Kumar, Wolfgang MaassMultimodal Clinical DataMissing Modalities

    German Research Centre for Artificial Intelligence (DFKI) Saarbrücken, Germany

  58. Textual Supervision Enhances Geospatial Representations in Vision-Language Models

    Jun 5, 2026Marcelo Sartori Locatelli, Fernando Tonucci, Jea Kwon +5Spatial SupervisionGeospatial Reasoning

    Max Planck Institute for Security and Privacy (MPI-SP), Bochum, Germany · Universidade Federal de Minas Gerais, Belo Horizonte, Brazil · Changwon National University, Changwon, South Korea +1

  59. Differentiable Efficient Operator Search

    Jun 3, 2026Xiaohuan Pei, Jiyuan Zhang, Yuanfan Guo +4Multimodal Foundation ModelMultimodal Learning

    1The University of Sydney · 2ByteDance · 3Shanghai Jiao Tong University +1

  60. Multi-Modal Machine Learning for Breast Cancer Recurrence Prediction

    Jun 1, 2026Jiahao Shao, Xudong Wang, Anam Nawaz Khan +3Multidisciplinary Tumor BoardsTumor

    Department of Industrial & Systems Engineering, The University of Tennessee, Knoxville, TN 37996, USA · The University of Tennessee Medical Center, Knoxville, TN 37920, USA