Multimodal Learning

Ask about this topic

Recent momentum

-18%

49 papers in the last 28 days · 0.8% of indexed attention

Twelve weeks of publication activity for this topic as it is defined today.

Also in Multimodal

Neighbouring directions

Cross-Modal LearningMulti-Modal Large Language ModelsMultimodal ReasoningMultimodal BenchmarksUnified Multimodal ModelsCross-Modal AttentionInput ModalityModality Gap

Weekly history

Recent digests

What was published in this topic, kept on the site without email delivery.

Period ending 2026-09-21

18 new papers

A weekly snapshot of new work published in Multimodal Learning.

456 papers

Latest in Multimodal Learning

Browse all topics →
CardsList

  1. Ablation Study of Multimodal Perception, Language Grounding, and Control for Human-Robot Interaction in an Object Detection and Grasping Task

    May 1, 2026Zi Tian, Guanting ShenHuman-Robot InteractionPerception-Action Loop

  2. Learning Multimodal Energy-Based Model with Multimodal Variational Auto-Encoder via MCMC Revision

    May 1, 2026Jiali Cui, Zhiqiang Lao, Heather YuVariational AutoencoderMarkov Chain Monte Carlo

  3. Multimodal Large Language Models as Synthetic Participants in Video-Based Studies: An Evaluation

    May 1, 2026Prabal Shrestha, Bohan Jiang, Haoning Xue +2Video Multimodal Large Language ModelsUser Simulation

  4. Towards Customized Multimodal Role-Play

    May 1, 2026Chao Tang, Jianzong Wu, Qingyu Shi +5Multimodal LearningGroup

  5. InteractWeb-Bench: Can Multimodal Agent Escape Blind Execution in Interactive Website Generation?

    Apr 30, 2026Qiyao Wang, Haoran Hu, Longze Chen +4WebMock-Interface Synthesis

  6. Attribution-Guided Multimodal Deepfake Detection via Cross-Modal Forensic Fingerprints

    Apr 29, 2026Wasim Ahmad, Wei Zhang, Xuerui MaoExplainable Deepfake DetectionDeepfake Detection

  7. Benchmarking Complex Multimodal Document Processing Pipelines: A Unified Evaluation Framework for Enterprise AI

    Apr 29, 2026Saurabh K. Singh, Sachin RajMultimodal DocumentsLarge Language Model Pipelines

  8. A Multimodal Pre-trained Network for Integrated EEG-Video Seizure Detection

    Apr 29, 2026Tong Lu, Ke Xu, Zimo Zhang +12Interictal Epileptiform DischargesMultimodal Learning

  9. SWAN: World-Aware Adaptive Multimodal Networks for Runtime Variations

    Apr 28, 2026Jason Wu, Shir-Kang Scott Jin, Yuyang Yuan +4Multimodal Deep LearningMultimodal Learning

  10. Toward Multimodal Conversational AI for Age-Related Macular Degeneration

    Apr 28, 2026Ran Gu, Benjamin Hou, Mélanie Hébert +5OphthalmologyMultimodal Clinical Data

  11. Learning Generalizable Multimodal Representations for Software Vulnerability Detection

    Apr 28, 2026Zeming Dong, Yuejun Guo, Qiang Hu +5Vulnerability Detection & RepairCode Quality

  12. HotComment: A Benchmark for Evaluating Popularity of Online Comments

    Apr 28, 2026Yafeng Wu, Yunyao Zhang, Liliang Ye +4PopularityReddit

  13. DualFact+: A Multimodal Fact Verification Framework for Procedural Video Understanding

    Apr 28, 2026Cennet Oguz, Yasser Hamidullah, Josef van Genabith +1Multilingual Fact-CheckingVideo Captioning

  14. M3^3-VQA: A Benchmark for Multimodal, Multi-Entity, Multi-Hop Visual Question Answering

    Apr 28, 2026Jiatong Ma, Longteng Guo, Yuchen Liu +4Knowledge-Based Visual Question AnsweringMultimodal Reasoning

  15. Beyond Accuracy: Benchmarking Cross-Task Consistency in Unified Multimodal Models

    Apr 27, 2026Weixing Wang, Liudvikas Zekas, Anton Hackl +5Unified Multimodal ModelsMultimodal Learning

  16. Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligence

    Apr 27, 2026NVIDIA, :, Amala Sanjay Deshmukh +216Omni-ModalOmni

  17. MIMIC: A Generative Multimodal Foundation Model for Biomolecules

    Apr 27, 2026Siavash Golkar, Jake Kovalic, Irina Espejo Morales +28DnaMultimodal Foundation Model

  18. QEVA: A Reference-Free Evaluation Metric for Narrative Video Summarization with Multimodal Question Answering

    Apr 27, 2026Woojun Jung, Junyeong KimSummarizationVideo-Language Models

  19. DeepTaxon: An Interpretable Retrieval-Augmented Multimodal Framework for Unified Species Identification and Discovery

    Apr 27, 2026Jiawei Wang, Ming Lei, Yaning Yang +8TaxonomyMultimodal Retrieval

  20. Hierarchical Prototype-based Domain Priors for Multiple Instance Learning in Multimodal Histopathology Analysis

    Apr 27, 2026Xuemei Qiu, Dawei Fan, Yebin Huang +2Computational PathologyMultiple Instance Learning

  21. Exploring Audio Hallucination in Egocentric Video Understanding

    Apr 26, 2026Ashish Seth, Xinhao Mei, Changsheng Zhao +9Audio-Visual ReasoningHallucination Rate

  22. OceanPile: A Large-Scale Multimodal Ocean Corpus for Foundation Models

    Apr 25, 2026Yida Xue, Ningyu Zhang, Tingwei Wu +5Multimodal DatasetMulti-Modal Large Language Models

  23. Robust Audio-Text Retrieval via Cross-Modal Attention and Hybrid Loss

    Apr 25, 2026Meizhu Liu, Matthew Rowe, Amit Agarwal +8Audio RepresentationCross-Modal Attention

  24. Au-M-ol: A Unified Model for Medical Audio and Language Understanding

    Apr 25, 2026Meizhu Liu, Nistha Mitra, Paul Li +3Large Audio Language ModelsAudio

  25. Contrastive Learning for Multimodal Human Activity Recognition with Limited Labeled Data

    Apr 25, 2026Long Jing, Zhixiong Yang, Yajun Zhang +1Human Activity RecognitionRobust Multimodal Learning

  26. One Identity, Many Roles: Multimodal Entity Coreference for Enhanced Video Situation Recognition

    Apr 25, 2026Balaji Darur, Amanmeet Garg, Makarand TapaswiVideo DatasetCoreference Resolution

  27. RAG4Outcome: A Retrieval-Augmented Multimodal Framework for Prognostic Prediction in Chronic Osteomyelitis

    Apr 24, 2026Daqian Shi, Pei Han, Jishizhan Chen +4PrognosticsMultimodal Clinical Data

  28. Inter-Stance: A Dyadic Multimodal Corpus for Conversational Stance Analysis

    Apr 24, 2026Xiang Zhang, Xiaotian Li, Taoyue Wang +9Multimodal DatasetMultimodal Learning

  29. MTT-Bench: Predicting Social Dominance in Mice via Multimodal Large Language Models

    Apr 24, 2026Yunquan Chen, Haoyu ChenDominant ParadigmSocial Reasoning

  30. Can Multimodal Large Language Models Truly Understand Small Objects?

    Apr 24, 2026Fujun Han, Junan Chen, Xintong Zhu +4Multi-Modal Large Language ModelsSmall Language Models

  31. Source-Modality Monitoring in Vision-Language Models

    Apr 23, 2026Etha Tianze Hua, Tian Yun, Ellie PavlickInput ModalityMultimodal Learning

  32. DM3^3-Nav: Decentralized Multi-Agent Multimodal Multi-Object Semantic Navigation

    Apr 23, 2026Amin Kashiri, Atharva Jamsandekar, Yasin YazıcıoğluSocial Robot NavigationNavigation

  33. The Expense of Seeing: Attaining Trustworthy Multimodal Reasoning Within the Monolithic Paradigm

    Apr 22, 2026Karan GoyalMultimodal ReasoningMultimodal Learning

  34. Breaking the Illusion: When Positive Meets Negative in Multimodal Decoding

    Apr 22, 2026Yubo Jiang, Yitong An, Xin Yang +7Modality-Specific EncodersIllusions

  35. Sema: Semantic Transport for Real-Time Multimodal Agents

    Apr 22, 2026Jiaying Meng, Bojie LiSemantic CommunicationsMultimodal Agents

  36. Multi-Perspective Evidence Synthesis and Reasoning for Unsupervised Multimodal Entity Linking

    Apr 22, 2026Mo Zhou, Jianwei Wang, Kai Wang +3Multimodal Learning

  37. Text Steganography with Dynamic Codebook and Multimodal Large Language Model

    Apr 22, 2026Jianxin Gao, Ruohan Lei, Wanli PengSteganographyCodebooks

  38. ATIR: Towards Audio-Text Interleaved Contextual Retrieval

    Apr 22, 2026Tong Zhao, Chenghao Zhang, Yutao Zhu +1Audio RepresentationMultimodal Retrieval

  39. AROMA: Augmented Reasoning Over a Multimodal Architecture for Virtual Cell Genetic Perturbation Modeling

    Apr 22, 2026Zhenyu Wang, Geyan Ye, Wei Liu +1Single-Cell Perturbation PredictionSearch-Augmented Reasoning

  40. Bridging Traditional Explainability Methods and Multimodal Multilingual Models: An XAI-Based Analysis

    Apr 21, 2026Paweł Pozorski, Jakub Muszyński, Maria GanzhaMultimodal LearningExplainability

  41. Multimodal embodiment-aware navigation transformer

    Apr 21, 2026Louis Dezons, Quentin Picard, Rémi Marsal +2Embodied NavigationObject Goal Navigation

  42. DR-MMSearchAgent: Deepening Reasoning in Multimodal Search Agents

    Apr 21, 2026Shengqin Wang, Wentao Yan, Huichi Zhou +4Multimodal Search AgentsMultimodal Agents

  43. How Far Are Video Models from True Multimodal Reasoning?

    Apr 21, 2026Xiaotian Zhang, Jianhui Wei, Yuan Wang +9Large-Scale Video Generation ModelsVideo Understanding

  44. Diff-SBSR: Learning Multimodal Feature-Enhanced Diffusion Models for Zero-Shot Sketch-Based 3D Shape Retrieval

    Apr 21, 2026Hang Cheng, Fanhe Dong, Long ZengSketchesText-To-Image Diffusion Models

  45. ProjLens: Unveiling the Role of Projectors in Multimodal Model Safety

    Apr 21, 2026Kun Wang, Cheng Qian, Miao Yu +6Backdoor AttacksLens

  46. DUALVISION: RGB-Infrared Multimodal Large Language Models for Robust Visual Reasoning

    Apr 20, 2026Abrar Majeedi, Zhiyuan Ruan, Ziyi Zhao +3Multi-Modal Large Language ModelsVisual Reasoning

  47. MathNet: a Global Multimodal Benchmark for Mathematical Reasoning and Retrieval

    Apr 20, 2026Shaden Alshammari, Kevin Wen, Abrar Zainal +5Olympiad-Level ProblemMath Problems

  48. Learning Invariant Modality Representation for Robust Multimodal Learning from a Causal Inference Perspective

    Apr 20, 2026Sijie Mai, Shiqin HanRobust Multimodal LearningCausal Representation Learning

  49. Revisiting Change VQA in Remote Sensing with Structured and Native Multimodal Qwen Models

    Apr 20, 2026Yakoub Bazi, Mohamad M. Al Rahhal, Mansour Zuair +1Remote SensingKnowledge-Based Visual Question Answering

  50. Omni-Embed-Audio: Leveraging Multimodal LLMs for Robust Audio-Text Retrieval

    Apr 20, 2026HaeJun Yoo, Yongseop Shin, Insung Lee +2Audio RepresentationMultimodal Learning

  51. MACS: Modality-Aware Capacity Scaling for Efficient Multimodal MoE Inference

    Apr 19, 2026Bo Li, Chuan Wu, Shaolin ZhuMixture-Of-Expert InferenceMultimodal Benchmarks

  52. LFRAG: Layout-oriented Fine-grained Retrieval-Augmented Generation on Multimodal Document Understanding

    Apr 18, 2026Yifan Zhu, Yu Mi, Yue Lu +2Multi-Source Retrieval-Augmented GenerationMultimodal Documents

  53. Cross-Modal Bayesian Low-Rank Adaptation for Uncertainty-Aware Multimodal Learning

    Apr 17, 2026Habibeh Naderi, Behrouz Haji Soleimani, Stan MatwinParameter-Efficient Fine-TuningMultimodal Learning

  54. Beyond Feature Fusion: Contextual Bayesian PEFT for Multimodal Uncertainty Estimation

    Apr 17, 2026Habibeh Naderi, Behrouz Haji Soleimani, Stan MatwinParameter-Efficient Fine-TuningMultimodal Classification

  55. FineCog-Nav: Integrating Fine-grained Cognitive Modules for Zero-shot Multimodal UAV Navigation

    Apr 17, 2026Dian Shao, Zhengzheng Xu, Peiyang Wang +4Aerial Vision-Language NavigationMenu

  56. CAVERS: Multimodal SLAM Data from a Natural Karstic Cave with Ground Truth Motion Capture

    Apr 16, 2026Giacomo Franchini, David Rodríguez-Martínez, Alfonso Martínez-Petersen +2Simultaneous Localization And MappingMultimodal Dataset

  57. RaTA-Tool: Retrieval-based Tool Selection with Multimodal Large Language Models

    Apr 16, 2026Gabriele Mattioli, Evelyn Turri, Sara Sarto +4Multi-Modal Large Language ModelsMulti-Turn Tool-Use Agent

  58. G-MIXER: Geodesic Mixup-based Implicit Semantic Expansion and Explicit Semantic Re-ranking for Zero-Shot Composed Image Retrieval

    Apr 16, 2026Jiyoung Lim, Heejae Yang, Jee-Hyong LeeComposed Image RetrievalMultimodal Retrieval

  59. SynMulti: Synthetic-to-Real Learning for Multimodal Video Understanding

    Apr 14, 2026Tanzila Rahman, Renjie Liao, Leonid SigalVideo Multimodal Large Language ModelsVideo Understanding

  60. M2HRI: An LLM-Driven Multimodal Multi-Agent Framework for Personalized Human-Robot Interaction

    Apr 13, 2026Shaid Hasan, Breenice Lee, Sujan Sarker +1Human-Robot InteractionMultimodal Agents