Multimodal Large Language Models

Also known as MLLM

Ask about this topic

Overview

Multimodal large language models connect language generation and reasoning to visual, audio, video, and sometimes sensor or structured data. Common approaches align pretrained modality encoders with a language model using projectors or adapters, or train models to process modalities jointly; instruction tuning, reinforcement learning, retrieval, and tool use extend their capabilities. Evaluation spans visual question answering, document reading, long-video and spatial reasoning, and specialized settings such as medical imaging, where models still often miss fine-grained evidence or fail to use it consistently. A central tension is how to improve cross-modal grounding, temporal and physical reasoning, and safety while reducing the high inference costs of processing long or detailed inputs.

Latest papers

CardsList

1,096 papers106 papers in the last four weeks, up 23% on the four weeks before. 0.6% of all new papers.

  1. MAD: Modality-Adaptive Decoding for Mitigating Cross-Modal Hallucinations in Multimodal Large Language Models

    Jan 29, 2026Sangyun Chung, Se Yeon Kim, Youngchae Chee +1Multimodal RobustnessHallucination in Language Models

    Integrated Vision Language Lab, KAIST, South Korea

  2. AVMeme Exam: A Multimodal Multilingual Multicultural Benchmark for LLMs' Contextual and Cultural Knowledge and Thinking

    Jan 25, 2026Xilin Jiang, Qiaolin Wang, Junkai Wu +30Audio-Visual UnderstandingMultimodal Large Language Models

    Columbia University · University of Washington · University of Illinois Urbana-Champaign +5

  3. Social Caption: Evaluating Social Understanding in Multimodal Models

    Jan 21, 2026Leena Mathur, Bhaavanaa Thumu, Youssouf Kebe +1Multimodal Large Language ModelsMultimodal Reasoning

    School of Computer Science, Carnegie Mellon University

  4. AMVICC: A Novel Benchmark for Cross-Modal Failure Mode Profiling for VLMs and IGMs

    Jan 20, 2026Aahana Basappa, Pranay Goel, Anusri Karra +3Reasoning EvaluationMultimodal Large Language Models

    Centennial High School, Frisco, Texas, USA · Algoverse AI Research, Palo Alto, Cali- fornia, USA · Lebanon Trail High School, Frisco, Texas, USA +1

  5. ChartAttack: Testing the Vulnerability of LLMs to Malicious Prompting in Chart Generation

    Jan 19, 2026Jesus-German Ortiz-Barajas, Jonathan Tonglet, Vivek Gupta +1Data VisualizationAdversarial Attacks on VLMs

    INSAIT · KU Leuven · Arizona State University +2

  6. Generative Semantic Multi-Object Tracking: A Large-Scale Benchmark and an MLLM-Driven Reasoning Framework

    Jan 10, 2026Pan Liao, Feng Yang, Di Wu +3Multimodal Large Language ModelsMLLM Reasoning

  7. BabyVision: Visual Reasoning Beyond Language

    Jan 10, 2026Liang Chen, Weichu Xie, Yiyan Liang +27VLM EvaluationVisual Reasoning

    1UniPat AI · 6Peking University · 3Alibaba Group +8

  8. Towards Multi-modal Multi-turn Safety: From Agentic Interaction to Strategic Alignment

    Jan 8, 2026Han Zhu, Jiale Chen, Chengkun Cai +8Language Model Safety EvaluationMultimodal Large Language Models

    Hong Kong University of Science and Technology · University College London · AISpeech +1

  9. FastSLM: Hierarchical Temporal Abstraction for Efficient Long-Form Speech Adaptation

    Jan 8, 2026Junseok Lee, Chang-Jae ChunMultimodal Large Language ModelsSpeech Processing

    OKESTRO · Sejong University

  10. ELF: A Family of Encoder-Free ECG-Language Models

    Jan 5, 2026William Han, Tony Chen, Chaojing Duan +6HealthcareElectrocardiography

    Carnegie Mellon University, USA · Allegheny Health Network, USA · University of California, Los Angeles, USA +1

  11. NarrativeTrack: Evaluating Entity-Centric Reasoning for Narrative Understanding

    Jan 3, 2026Hyeonjeong Ha, Jinjin Ge, Bo Feng +2VLM EvaluationVideo Understanding

    Apple · University of Illinois Urbana-Champaign

  12. BrepLLM: Enabling Large Language Models to Understand Boundary Representations

    Dec 18, 2025Liyuan Deng, Hao Guo, Yongkang Dai +5Multimodal Large Language Models3D Representation Learning

    Northwestern Polytechnical University · National University of Singapore · Hong Kong University of Science and Technology +1

  13. EchoingPixels: Aliasing-Resistant Joint Token Reduction for Audio-Visual LLMs

    Dec 11, 2025Chao Gong, Depeng Wang, Zhipeng Wei +3Rotary Positional EmbeddingsMultimodal Large Language Models

    1Fudan University · 2Ant Group · 3UC Berkley

  14. Enhancing Large Language Model-Based Systems for End-to-End Circuit Analysis Problem Solving

    Dec 10, 2025Liangliang Chen, Weiyu Sun, Huiru Xie +2LLM Answer VerificationEducational Technology

    School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA 30332 USA

  15. Visual Species Recognition with Large Multimodal Models as Post-Hoc Correctors

    Dec 10, 2025Tian Liu, Anwesha Basu, James Caverlee +1Large Vision-Language ModelsMultimodal Large Language Models

    Department of Computer Science and Engineering, Texas A&M University · Department of AI, University of Macau

  16. VKnowU: Evaluating Visual Knowledge Understanding in Multimodal LLMs

    Nov 25, 2025Tianxiang Jiang, Sheng Xia, Yicheng Xu +5VLM EvaluationMultimodal Large Language Models

    University of Science and Technology of China · 2Shanghai AI Laboratory · 3Nanjing University +2

  17. Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearning

    Nov 25, 2025Zhen Zeng, Leijiang Gu, Zhangling Duan +4VLM UnlearningMultimodal Large Language Models

    Hefei University of Technology, China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, China · University of Amsterdam, the Netherlands

  18. Beyond Description: Cognitively Benchmarking Fine-Grained Action for Embodied Agents

    Nov 24, 2025Dayong Liu, Chao Xu, Weihong Chen +5AI Agent EvaluationMultimodal Large Language Models

    Zhejiang University · IROOTECH TECHNOLOGY · Wolf 1069 b Lab, Sany Group +2

  19. VisReason: A Large-Scale Dataset for Visual Chain-of-Thought Reasoning

    Nov 21, 2025Lingxiao Li, Yifan Wang, Xinyan Gao +3Visual Spatial ReasoningVisual Reasoning

    Boston University, Boston, MA, USA · Stony Brook University, Stony Brook, NY, USA · MMLab, The Chinese University of Hong Kong, Hong Kong SAR, China

  20. Can MLLMs Read the Room? A Multimodal Benchmark for Assessing Deception in Multi-Party Social Interactions

    Nov 20, 2025Caixin Kang, Yifei Huang, Liangyang Ouyang +3Multimodal Deception DetectionDeception in Language Models

    The University of Tokyo

  21. A Comprehensive Study on Visual Token Redundancy for Discrete Diffusion-based Multimodal Large Language Models

    Nov 19, 2025Duo Li, Zuhao Yang, Xiaoqin Zhang +2Multimodal Large Language ModelsDiffusion Language Model Inference

    CCDS, NTU, Singapore · CCST, ZJUT, China · Terminus AI Lab, UCAS, China

  22. LLM4Delay: Flight Delay Prediction via Cross-Modality Adaptation of Large Language Models and Aircraft Trajectory Representation

    Oct 24, 2025Thaweerath Phisannupawong, Joshua Julian Damanik, Han-Lim ChoiAir Traffic ManagementIntelligent Transportation Systems

    Department of Aerospace Engineering, Korea Advanced Institute of Science and Technology, Daejeon, 34141, Republic of Korea.

  23. Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Brain

    Oct 20, 2025Yulin Luo, Chun-Kai Fan, Menghang Dong +19Long-Horizon Robotic ManipulationMultimodal Large Language Models

    State Key Laboratory of Multimedia Information Processing, School of Computer2026 Science, Peking University · Institute for Brain and Intelligence, Fudan University · University of Science andJul Technology Beijing +2

  24. MCA: Modality Composition Awareness for Robust Composed Multimodal Retrieval

    Oct 17, 2025Qiyu Wu, Shuyang Cui, Satoshi Hayakawa +3Multimodal RobustnessMultimodal Large Language Models

    Sony Group Corporation · Sony AI

  25. Constructive Distortion: Improving MLLMs with Attention-Guided Image Warping

    Oct 10, 2025Dwip Dalal, Gautam Vashishtha, Utkarsh Mishra +6Object Hallucination in VLMsMultimodal Large Language Models

    University of Illinois Urbana–Champaign · Skan AI · Texas A&M University +1

  26. LM Fight Arena: Benchmarking Large Multimodal Models via Game Competition

    Oct 10, 2025Yushuo Zheng, Tongrui Ye, Zicheng Zhang +3Multimodal Large Language ModelsAI Agent Benchmarks

    Shanghai Jiao Tong University, Shanghai, China · Shanghai AI Lab, Shanghai, China · Tongji University, Shanghai, China

  27. How to Teach Large Multimodal Models New Skills

    Oct 9, 2025Zhen Zhu, Yiming Gong, Yao Xiao +2Multimodal Large Language ModelsContinual Learning for VLMs

    University of Illinois Urbana-Champaign · Google DeepMind · Carnegie Mellon University

  28. Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMs

    Oct 1, 2025Sanghwan Kim, Rui Xiao, Stephan Alaniz +2Multimodal Large Language ModelsLong-Video Understanding

    1Technical University of Munich · 2Munich Center for Machine Learning · 20263Helmholtz Munich +2

  29. LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models

    Sep 28, 2025Shubhang Bhatnagar, Andy Xu, Kar-Han Tan +1Mixed-Precision QuantizationLLM Quantization

    University of Illinois Urbana-Champaign · Work done as an intern at HP Inc. · University of California, Los Angeles +1

  30. Evidence Recomposition and Predictive Context Residualization for Visual Attribution in Multimodal Large Language Models

    Sep 26, 2025Jiawei Liang, Jianjie Huang, Ruoyu Chen +4Feature AttributionMultimodal Large Language Models

    Shenzhen Campus of Sun Yat-sen University, Shenzhen 518107, China · Zhongguan-cun Academy, Beijing 100094, China · University of Chinese Academy of Sciences, Beijing 100049, China +2