Multimodal Large Language Models

Also known as MLLM

Overview

Multimodal large language models connect language generation and reasoning to visual, audio, video, and sometimes sensor or structured data. Common approaches align pretrained modality encoders with a language model using projectors or adapters, or train models to process modalities jointly; instruction tuning, reinforcement learning, retrieval, and tool use extend their capabilities. Evaluation spans visual question answering, document reading, long-video and spatial reasoning, and specialized settings such as medical imaging, where models still often miss fine-grained evidence or fail to use it consistently. A central tension is how to improve cross-modal grounding, temporal and physical reasoning, and safety while reducing the high inference costs of processing long or detailed inputs.

Latest papers

CardsList

1,147 papers50 papers in the last four weeks, down 57% on the four weeks before. 0.4% of all new papers.

  1. LoC-Path: Learning to Compress for Pathology Multimodal Large Language Models

    Dec 5, 2025Qingqiao Hu, Weimin Lyu, Meilong Xu +5Multimodal Large Language ModelsEfficient Multimodal Inference

  2. VKnowU: Evaluating Visual Knowledge Understanding in Multimodal LLMs

    Nov 25, 2025Tianxiang Jiang, Sheng Xia, Yicheng Xu +5VLM EvaluationMultimodal Large Language Models

  3. Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearning

    Nov 25, 2025Zhen Zeng, Leijiang Gu, Zhangling Duan +4VLM UnlearningMultimodal Large Language Models

  4. Beyond Description: Cognitively Benchmarking Fine-Grained Action for Embodied Agents

    Nov 24, 2025Dayong Liu, Chao Xu, Weihong Chen +5AI Agent EvaluationMultimodal Large Language Models

  5. VisReason: A Large-Scale Dataset for Visual Chain-of-Thought Reasoning

    Nov 21, 2025Lingxiao Li, Yifan Wang, Xinyan Gao +3Visual Spatial ReasoningVisual Reasoning

  6. A Comprehensive Study on Visual Token Redundancy for Discrete Diffusion-based Multimodal Large Language Models

    Nov 19, 2025Duo Li, Zuhao Yang, Xiaoqin Zhang +2Multimodal Large Language ModelsDiffusion Language Model Inference

  7. LLM4Delay: Flight Delay Prediction via Cross-Modality Adaptation of Large Language Models and Aircraft Trajectory Representation

    Oct 24, 2025Thaweerath Phisannupawong, Joshua Julian Damanik, Han-Lim ChoiAir Traffic ManagementIntelligent Transportation Systems

  8. Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Brain

    Oct 20, 2025Yulin Luo, Chun-Kai Fan, Menghang Dong +19Long-Horizon Robotic ManipulationMultimodal Large Language Models

  9. MCA: Modality Composition Awareness for Robust Composed Multimodal Retrieval

    Oct 17, 2025Qiyu Wu, Shuyang Cui, Satoshi Hayakawa +3Multimodal RobustnessMultimodal Large Language Models

  10. Constructive Distortion: Improving MLLMs with Attention-Guided Image Warping

    Oct 10, 2025Dwip Dalal, Gautam Vashishtha, Utkarsh Mishra +6Object Hallucination in VLMsMultimodal Large Language Models

  11. LM Fight Arena: Benchmarking Large Multimodal Models via Game Competition

    Oct 10, 2025Yushuo Zheng, Tongrui Ye, Zicheng Zhang +3Multimodal Large Language ModelsAI Agent Benchmarks

  12. How to Teach Large Multimodal Models New Skills

    Oct 9, 2025Zhen Zhu, Yiming Gong, Yao Xiao +2Multimodal Large Language ModelsContinual Learning for VLMs

  13. MLLMEraser: Achieving Test-Time Unlearning in Multimodal Large Language Models through Activation Steering

    Oct 5, 2025Chenlu Ding, Jiancan Wu, Leheng Sheng +4MLLM UnlearningMachine Unlearning

  14. Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMs

    Oct 1, 2025Sanghwan Kim, Rui Xiao, Stephan Alaniz +2Multimodal Large Language ModelsLong-Video Understanding

  15. LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models

    Sep 28, 2025Shubhang Bhatnagar, Andy Xu, Kar-Han Tan +1Mixed-Precision QuantizationLLM Quantization

  16. Evidence Recomposition and Predictive Context Residualization for Visual Attribution in Multimodal Large Language Models

    Sep 26, 2025Jiawei Liang, Jianjie Huang, Ruoyu Chen +4Feature AttributionMultimodal Large Language Models

  17. Measuring Epistemic Humility in Multimodal Large Language Models

    Sep 11, 2025Bingkui Tong, Jiaer Xia, Sifeng Shang +1VLM HallucinationMultimodal Large Language Models

  18. AnomalyLMM: Bridging Generative Knowledge and Discriminative Retrieval for Text-Based Person Anomaly Search

    Sep 4, 2025Hao Ju, Hu Zhang, Zhedong ZhengZero-Shot Anomaly DetectionMultimodal Large Language Models

  19. Inverse-LLaVA: Rethinking Multimodal Alignment via Text-to-Vision Mapping

    Aug 17, 2025Xuhui Zhan, Tyler DerrCross-Modal LearningCross-Modal Representation Learning

  20. Can Multi-modal (reasoning) LLMs detect document manipulation?

    Aug 14, 2025Zisheng Liang, Kidus Zewde, Rudra Pratap Singh +7Multimodal Large Language ModelsMLLM Evaluation

  21. When Language Overrules: Revealing Text Dominance in Multimodal Large Language Models

    Aug 14, 2025Huyu Wu, Meng Tang, Xinhan Zheng +1Modality Reliance in Multimodal ModelsMultimodal Large Language Models

  22. SyncLoop: A Multimodal Dual-Loop Framework for Self-Improving Mathematical Reasoning

    Jul 22, 2025Xiuwei Chen, Wentao Hu, Hanhui Li +9Synthetic Data GenerationMultimodal Large Language Models

  23. Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey

    Jul 21, 2025Jindong Li, Yali Fu, Jiahong Liu +5Multimodal Large Language ModelsVector Quantization

  24. Region-Aware Multimodal Large Language Model via SlowFast Tokenization and Pseudo-Mask Guidance for 3D CT Report Generation

    Jun 29, 2025Sunggu Kyung, Jinyoung Seo, Hyunseok Lim +7Radiology Report Generation3D Medical Imaging

  25. Speculative Decoding Reimagined for Multimodal Large Language Models

    May 20, 2025Luxi Lin, Zhihang Lin, Zhanpeng Zeng +1Efficient Multimodal InferenceSpeculative Decoding

  26. HumaniBench: A Human-Centric Framework for Large Multimodal Models Evaluation

    May 16, 2025Shaina Raza, Aravind Narayanan, Vahid Reza Khazaie +6Algorithmic FairnessMultimodal Large Language Models

  27. FORGE: Forensic Reasoning with Grounded Evidence

    Mar 20, 2025Rohit Kundu, Shan Jia, Vishal Mohanty +2VLM AdaptationExplainable Deepfake Detection