Multimodal Large Language Models

Ask about this topic

Momentum

90 papers in the last four weeks, up 61% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 877

All topics
CardsList

  1. Where to Focus: Query-Modulated Multimodal Keyframe Selection for Long Video Understanding

    Apr 19, 2026Shaoguang Wang, Weiyu Guo, Ziyang Chen +2Video UnderstandingKeyframe Selection

    Thrust of Artificial Intelligence, HKUST (Guangzhou) Guangzhou, China · Department of CSE, HKUST Hong Kong SAR, China

  2. SpatialImaginer: Towards Adaptive Visual Imagination for Spatial Reasoning

    Apr 19, 2026Yian Li, Yang Jiao, Bin Zhu +4Spatial ReasoningMultimodal Large Language Models

    College of Computer Science and Artificial Intelligence, Fudan University · School of Computing and Information Systems, Singapore Management University · School of Computer Science and Technology, East China Normal University +2

  3. Instinct vs. Reflection: Unifying Token and Verbalized Confidence in Multimodal Large Models

    Apr 19, 2026Yunkai Dang, Yifan Jiang, Yizhu Jiang +3Confidence EstimationMultimodal Large Language Models

    School of Artificial Intelligence, Nanjing University.

  4. RemoteShield: Enable Robust Multimodal Large Language Models for Earth Observation

    Apr 19, 2026Rui Min, Liang Yao, Shiyu Miao +5Remote SensingEarth Observation

    1Hohai University · 2Nanjing University · 3Southeast University

  5. Enhancing Zero-shot Personalized Image Aesthetics Assessment with Profile-aware Multimodal LLM

    Apr 19, 2026Chun Wang, Chenfeng Wei, Chenyang Liu +1Zero-Shot RobustnessZero-Shot

    Mashang Consumer Finance Co., Ltd., Chongqing, China · Xi’an Jiaotong-Liverpool University, Suzhou, China · Beijing University of Posts and Telecommunications, Beijing, China

  6. Modeling Multi-Dimensional Cognitive States in Large Language Models under Cognitive Crowding

    Apr 19, 2026Lin Zhong, Siyu Zhu, Zizhen Yuan +5Multimodal Large Language ModelsCognitive Science

    Harbin Institute of Technology, Shenzhen, China · City University of Macau, Macao SAR, China · Peng Cheng Laboratory, Shenzhen, China

  7. EvoComp: Learning Visual Token Compression for Multimodal Large Language Models via Semantic-Guided Evolutionary Labeling

    Apr 18, 2026Jiafei Song, Fengwei Zhou, Jin Qu +7Token CompressionMultimodal Large Language Models

    OPPO CTG

  8. Evaluating Multimodal LLMs for Inpatient Diagnosis: Real-World Performance, Safety, and Cost Across Ten Frontier Models

    Apr 18, 2026Bruce A. Bassett, Amy Rouillard, Sitwala Mundia +8Multimodal Clinical DataMultimodal Large Language Models

    School of Computer Science and Applied Mathematics, University of the Witwatersrand, Johannesburg, South Africa · Wits MIND Institute, University of the Witwatersrand, Johannesburg, South Africa · Grai Labs, Cape Town, South Africa +3

  9. DOSE: Data Selection for Multi-Modal LLMs via Off-the-Shelf Models

    Apr 18, 2026Biao Wu, Yiwu Zhong, Meng Fang +1Multimodal Large Language ModelsMultimodal Learning

    Australian Artificial Intelligence Institute · Peking University · University of Liverpool

  10. MNAFT: modality neuron-aware fine-tuning of multimodal large language models for image translation

    Apr 18, 2026Bo Li, Ningyuan Deng, Tianyu Dong +3Unpaired Image-To-Image TranslationContrastive Language-Image Pre-Training Model

    School of Software, Tsinghua University, Beijing, China · Baidu Inc., Beijing, China · School of Information Resource Management, Renmin University of China,Beijing, China +2

  11. PivotMerge: Bridging Heterogeneous Multimodal Pre-training via Post-Alignment Model Merging

    Apr 18, 2026Zibo Shao, Baochen Xiong, Xiaoshan Yang +4Cross-Modal AlignmentMultimodal Pretraining

    State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · Pengcheng Laboratory, Shenzhen 518066, China · School of Artificial Intelligence, University of Chinese Academy Sciences, Beijing 100049, China +1

  12. Beyond Text-Dominance: Understanding Modality Preference of Omni-modal Large Language Models

    Apr 18, 2026Xinru Yan, Boxi Cao, Yaojie Lu +4Multimodal Large Language ModelsOmni-Modal

    University of Chinese Academy of Sciences, Beijing, China · Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences, Beijing, China

  13. Reducing Peak Memory Usage for Modern Multimodal Large Language Model Pipelines

    Apr 17, 2026Junwan Kim, Hyunkyung BaeMultimodal Large Language ModelsLLM Inference Optimization

    New York University

  14. Chain-of-Thought Degrades Visual Spatial Reasoning Capabilities of Multimodal LLMs

    Apr 17, 2026Sai Srinivas Kancheti, Aditya Sanjiv Kanade, Vineeth N. Balasubramanian +1Spatial ReasoningStable Spatial Understanding

    IIT, Hyderabad · Microsoft Research India

  15. Mind's Eye: A Benchmark of Visual Abstraction, Transformation and Composition for Multimodal LLMs

    Apr 17, 2026Rohit Sinha, Aditya Kanade, Sai Srinivas Kancheti +2Visual ReasoningMultimodal Large Language Models

    CSE Dept., IIT Hyderabad Hyderabad, India · Microsoft Research Bengaluru, India

  16. ReactBench: A Benchmark for Topological Reasoning in MLLMs on Chemical Reaction Diagrams

    Apr 17, 2026Qiang Xu, Shengyuan Bai, Yu Wang +6Chemical StructuresVisual Reasoning

    International Digital Economy Academy, Shenzhen, China · University of Wisconsin–Madison

  17. RefereeBench: Are Video MLLMs Ready to be Multi-Sport Referees

    Apr 17, 2026Yichen Xu, Yuanhang Liu, Chuhan Wang +5Video Multimodal Large Language ModelsLong-Video Benchmarks

    Renmin University of China Beijing, China · Sichuan University Chengdu, China

  18. Towards Robust Endogenous Reasoning: Unifying Drift Adaptation in Non-Stationary Tuning

    Apr 17, 2026Xiaoyu Yang, En Yu, Wei Duan +1Large Language Model AdaptationMultimodal Large Language Models

    Australian Artificial Intelligence Institute (AAII), Faculty of Engineering and Information Technology, University of Technology Sydney, Australia

  19. From Reactive to Proactive: Assessing the Proactivity of Voice Agents via ProVoice-Bench

    Apr 16, 2026Ke Xu, Yuhao Wang, Yu WangProactiveVoice Agents

    Shanghai Jiao Tong University

  20. RaTA-Tool: Retrieval-based Tool Selection with Multimodal Large Language Models

    Apr 16, 2026Gabriele Mattioli, Evelyn Turri, Sara Sarto +4Multimodal Search AgentsMultimodal Large Language Models

    University of Modena and Reggio Emilia, Italy · University of Pisa, Italy

  21. MirrorBench: Evaluating Self-centric Intelligence in MLLMs by Introducing a Mirror

    Apr 16, 2026Shengyu Guo, Tongrui Ye, Jianbo Zhang +3Embodied Artificial IntelligenceSelf-Referential Bias

    Shanghai AI Lab

  22. Confidence under Visual Token Pruning: Removed Evidence and Risk-Controlled Token Budgets for MLLMs

    Apr 13, 2026Kaizhen Tan, Yang Feng, Heqing Du +3Visual Token PruningMultimodal Large Language Models

    Carnegie Mellon University, Pittsburgh, PA, USA · Columbia University, New York, NY, USA

  23. FusionBERT: Multi-View Image--3D Retrieval via Cross-Attention Visual Fusion and Normal-Aware 3D Encoder

    Apr 2, 2026Wei Li, Yufan Ren, Hanqing Jiang +6Cross-View FusionProgressive Attention Fusion

    IROOTECH TECHNOLOGY, Hangzhou, Zhejiang, China · Wolf 1069 b Lab, Sany Group, Hangzhou, Zhejiang, China · Zhejiang University, Hangzhou, Zhejiang, China +4

  24. Steerable Visual Representations

    Apr 2, 2026Jona Ruthardt, Manu Gaur, Deva Ramanan +2Visual RepresentationsVision Encoders

    University of Technology Nuremberg · Carnegie Mellon University · International Institute of Information Technology, Hyderabad

  25. Colon-Bench: An Agentic Workflow for Scalable Dense Lesion Annotation in Full-Procedure Colonoscopy Videos

    Mar 26, 2026Abdullah Hamdi, Changchun Yang, Xin GaoPolyp SegmentationColonoscopy

    King Abdullah University of Science and Technology (KAUST)

  26. VideoTIR: Accurate Understanding for Long Videos with Efficient Tool-Integrated Reasoning

    Mar 26, 2026Zhe Gao, Shiyu Shen, Taifeng Chai +7Video UnderstandingLong Videos

    Nanjing University, Nanjing, China · Nankai University, Tianjin, China · East China Normal University, Shanghai, China +3

  27. ReLope: From Hidden-State Probing to a Decision Module for Multimodal LLM Routing

    Mar 25, 2026Yaopei Zeng, Congchao Wang, Blake JianHang Chen +1Large Language Model RoutingMultimodal Large Language Models

    Pennsylvania State University · Google · Amazon AGI

  28. SpecEyes: Accelerating Agentic Multimodal LLMs via Speculative Perception and Planning

    Mar 24, 2026Haoyu Huang, Jinfa Huang, Zhongwei Wan +3Multimodal AgentsMultimodal Large Language Models

    Xiamen University, Xiamen, Fujian, China · University of Rochester, Rochester, NY, USA · The Ohio State University, Columbus, OH, USA

  29. DetPO: In-Context Learning with Multi-Modal LLMs for Few-Shot Object Detection

    Mar 24, 2026Gautam Rajendrakumar Gare, Neehar Peri, Matvei Popov +3Open-Vocabulary Object DetectionObject Detection

    Carnegie Mellon University · Roboflow

  30. Clinical Cognition Alignment for Gastrointestinal Diagnosis with Multimodal LLMs

    Mar 21, 2026Huan Zheng, Yucheng Zhou, Tianyi Yan +6ColonoscopyMultimodal Clinical Data

    SKL-IOTSC, CIS, University of Macau · 2Shanghai Jiao Tong University · 3COWARobot Co. Ltd. +1

  31. When Demonstrations Fail: Diagnosing the Limits of In-Context Learning in Large Audio-Language Models with Progressive Cue Removal

    Mar 20, 2026Yen-Ting Piao, Jay Chiehen Liao, Wei-Tang Chien +5Large Audio Language ModelsMultimodal Large Language Models

    National Taiwan University, Taiwan

  32. ProactiveBench: Benchmarking Proactiveness in Multimodal Large Language Models

    Mar 19, 2026Thomas De Min, Subhankar Roy, Stéphane Lathuilière +2Multimodal Large Language ModelsProactive

    University of Trento · University of Bergamo · Inria, Univ. Grenoble Alpes, CNRS, LJK +1

  33. On Optimizing Multimodal Jailbreaks for Spoken Language Models

    Mar 19, 2026Aravind Krishnan, Karolina Stańczak, Dietrich KlakowLarge Language Model JailbreaksJailbreak Attacks

    Saarland University, Germany · DFKI GmbH, Saarbrücken, Germany · ETH AI Center, Switzerland

  34. EvoGuard: An Extensible Agentic RL-based Framework for Practical and Evolving AI-Generated Image Detection

    Mar 18, 2026Chenyang Zhu, Maorong Wang, Jun Liu +2Ai-Generated Image DetectionMultimodal Large Language Models

    The University of Tokyo Tokyo, Japan · National Institute of Informatics Tokyo, Japan

  35. Beyond Medical Diagnostics: How Medical Multimodal Large Language Models Think in Space

    Mar 14, 2026Quoc-Huy Trinh, Xi Ding, Yang Liu +7Medical ImagesMultimodal Large Language Models

  36. CoMMET: A Psychologically Grounded Benchmark for Evaluating Theory of Mind in Multimodal LLMs

    Mar 12, 2026Ruirui Chen, Weifeng Jiang, Chengwei Qin +3Multimodal Large Language ModelsSocial Reasoning

    Institute of High Performance Computing (IHPC) and 2Centre for Frontier AI Research (CFAR), Agency for Science, Technology and Research (A*STAR), Singapore · Nanyang Technological University, Singapore · Hong Kong University of Science and Technology (Guangzhou), China

  37. Explicit Logic Channel for Validation and Enhancement of MLLMs on Zero-Shot Tasks

    Mar 12, 2026Mei Chee Leong, Ying Gu, Hui Li Tan +2Multimodal Large Language ModelsValidation

    Institute for Infocomm Research (I2R) · Present address: Institute of Advanced Intelligence and Computing (IAIC), Agency for Science, Technology and Research (A*STAR), Singapore

  38. Omni-Diffusion: Unified Multimodal Understanding and Generation with Masked Discrete Diffusion

    Mar 6, 2026Lijiang Li, Zuwei Long, Yunhang Shen +6Multimodal Diffusion ModelsMultimodal Foundation Model

    Nanjing University · Tencent Youtu Lab · CASIA

  39. Design-MLLM: A Reinforcement Alignment Framework for Verifiable and Aesthetic Interior Design

    Mar 4, 2026Yuxuan Yang, Xiaotong Mao, Jingyao WangMllm-BasedMultimodal Large Language Models

    Nanjing Forestry University, Nanjing, China · Université de Lorraine, Nancy, France · Laboratoire Réactions et Génie des Procédés, Nancy, France +2

  40. MUSE: A Run-Centric Platform for Multimodal Unified Safety Evaluation of Large Language Models

    Mar 3, 2026Zhongxi Wang, Yueqian Lin, Jingyang Zhang +3Large Language Model SafetyMultimodal Large Language Models

    1Duke University · 2Virtue AI

  41. IDProxy: CTR Prediction with Multimodal LLMs for Cold-Start Recommendation at Xiaohongshu

    Mar 2, 2026Yubin Zhang, Haiming Xu, Guillaume Salha-Galvan +6Click-Through Rate PredictionCold-Start

    Xiaohongshu Inc. Shanghai, China · Shanghai Jiao Tong University Shanghai, China · Fudan University Shanghai, China

  42. Advancing Multimodal Judge Models through a Capability-Oriented Benchmark and MCTS-Driven Data Generation

    Feb 28, 2026Zeyu Chen, Huanjin Yao, Ziwang Zhao +1Large Language Model JudgesLlm-As-A-Judge

  43. EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Models

    Feb 27, 2026Yiyang Fang, Wenke Huang, Pei Fu +5Multimodal Large Language ModelsEmotion Recognition

    School of Computer Science, Wuhan University. · MiLM Plus, Xiaomi Inc.

  44. OmniGAIA: Towards Native Omni-Modal AI Agents

    Feb 26, 2026Xiaoxi Li, Wenxiang Jiao, Jiarui Jin +10Omni-ModalMultimodal Agents

    1Renmin University of China · 2Xiaohongshu Inc. · 3Peking University +2

  45. BrepCoder: A Unified Multimodal Large Language Model for Multi-task B-rep Reasoning

    Feb 25, 2026Mingi Kim, Yongjun Kim, Jungwoo Kang +1Computer-Aided Design ObjectsMultimodal Large Language Models

    Chungnam National University, Daejeon, Republic of Korea

  46. Magic-MM-Embedding: Towards Visual-Token-Efficient Universal Multimodal Embedding with MLLMs

    Feb 5, 2026Qi Li, Yanzhe Zhao, Yongxin Zhou +4Multimodal EmbeddingsMultimodal Large Language Models

    Honor Device Co., Ltd., China

  47. Superman: Unifying Skeleton and Vision for Human Motion Perception and Generation

    Feb 2, 2026Xinshun Wang, Peiming Li, Ziyi Wang +5Human Motion GenerationSkeleton

    State Key Laboratory of General Artificial Intelligence, Peking University, Shenzhen Graduate School · Sony R&D Center 3Nanyang Technological University

  48. A Multimodal Large Language Model-Driven Framework for Context-Aware UAV Emergency Landing Site Selection

    Feb 1, 2026Chunliang Hua, Lei Zhang, Jiayang Sun +2Cooperative Air CombatLandslides

    School of Information Science and Engineering, Southeast University, Nanjing 211189, China · LASER, International Digital Economy Academy, Shenzhen 510085, China · Department of Electronic Engineering, East China Normal University, Shanghai 200241, China

  49. Omni-RRM: Advancing Omni Reward Modeling via Automatic Rubric-Grounded Preference Synthesis

    Jan 31, 2026Zicheng Kong, Dehua Ma, Zhenbo Xu +9Large Language Model AlignmentProgress Reward Modeling

    Beijing University of Posts and Telecommunications, Beijing, P.R. China · ShiFang Technology Inc., Hangzhou, P.R. China · Tsinghua University, Beijing, P.R. China +1

  50. TimeBlind: A Spatio-Temporal Compositionality Benchmark for Video LLMs

    Jan 30, 2026Baiqi Li, Kangyi Zhao, Ce Zhang +3Spatio-Temporal ReasoningMultimodal Large Language Models

    University of North Carolina at Chapel Hill · University of Pittsburgh · Carnegie Mellon University

  51. SONIC-O1: A Real-World Benchmark for Evaluating Multimodal Large Language Models on Audio-Video Understanding

    Jan 29, 2026Ahmed Y. Radwan, Christos Emmanouilidis, Hina Tabassum +2Audio-Visual ReasoningLarge Audio Language Models

    Vector Institute for Artificial Intelligence, MaRS Centre, Toronto, ON M5G 1L7, Canada · University of Groningen, Nijenborgh 4, 9747 AG Groningen, Netherlands · York University, 4700 Keele Street, Toronto, ON M3J 1P3, Canada

  52. AMVICC: A Novel Benchmark for Cross-Modal Failure Mode Profiling for VLMs and IGMs

    Jan 20, 2026Aahana Basappa, Pranay Goel, Anusri Karra +3Multimodal Large Language ModelsVisual Reasoning

    Centennial High School, Frisco, Texas, USA · Algoverse AI Research, Palo Alto, Cali- fornia, USA · Lebanon Trail High School, Frisco, Texas, USA +1

  53. ChartAttack: Testing the Vulnerability of LLMs to Malicious Prompting in Chart Generation

    Jan 19, 2026Jesus-German Ortiz-Barajas, Jonathan Tonglet, Vivek Gupta +1Multimodal Large Language ModelsAttacker

    INSAIT · KU Leuven · Arizona State University +2

  54. M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understanding

    Jan 13, 2026Juntao Jiang, Jiangning Zhang, Yali Bi +7Medical ImagesDiagnostic Benchmark

    1Zhejiang University · University of Science and Technology of China · 3East China Normal University +2

  55. BabyVision: Visual Reasoning Beyond Language

    Jan 10, 2026Liang Chen, Weichu Xie, Yiyan Liang +27Recent Vision-Language ModelsVisual Reasoning

    1UniPat AI · 6Peking University · 3Alibaba Group +8

  56. Towards Multi-modal Multi-turn Safety: From Agentic Interaction to Strategic Alignment

    Jan 8, 2026Han Zhu, Jiale Chen, Chengkun Cai +8Safety AlignmentMultimodal Large Language Models

    Hong Kong University of Science and Technology · University College London · AISpeech +1

  57. MOSS Transcribe Diarize Technical Report

    Jan 4, 2026MOSI. AI, :, Donghua Yu +22Speaker DiarizationTranscript

    MOSI.AI*

  58. SpatialMosaic: A Multiview VLM Dataset for Partial Visibility

    Dec 29, 2025Kanghee Lee, Jungi Hong, Sion Lee +4Multimodal Large Language ModelsVisibility

    Seoul National University · Kyung Hee University · University College London +1

  59. SGM: Safety Glasses for Multimodal Large Language Models via Neuron-Level Detoxification

    Dec 17, 2025Hongbo Wang, AprilPyone MaungMaung, Isao EchizenToxicityLarge Language Model Safety

    1Graduate School of Information Science and Technology, The University of Tokyo, Japan · 2Information and Society Research Division, National Institute of Informatics, Japan · 3The Graduate University for Advanced Studies, SOKENDAI, Japan

  60. Ophiuchus: Incentivizing Tool-augmented "Think with Images" for Joint Medical Segmentation, Understanding and Reasoning

    Dec 16, 2025Yankai Jiang, Yujie Zhang, Peng Zhang +5Semi-Supervised Medical Image SegmentationMultimodal Large Language Models

    Zhejiang University · Shanghai Artificial Intelligence Laboratory · Shanghai Innovation Institute +5