Multimodal Large Language Models

Also known as MLLM

Overview

Multimodal large language models connect language generation and reasoning to visual, audio, video, and sometimes sensor or structured data. Common approaches align pretrained modality encoders with a language model using projectors or adapters, or train models to process modalities jointly; instruction tuning, reinforcement learning, retrieval, and tool use extend their capabilities. Evaluation spans visual question answering, document reading, long-video and spatial reasoning, and specialized settings such as medical imaging, where models still often miss fine-grained evidence or fail to use it consistently. A central tension is how to improve cross-modal grounding, temporal and physical reasoning, and safety while reducing the high inference costs of processing long or detailed inputs.

Latest papers

CardsList

1,147 papers50 papers in the last four weeks, down 57% on the four weeks before. 0.4% of all new papers.

  1. WeMM-Video: Pushing the Frontiers of Open-Source Omni-Modal Video Understanding

    Nov 19, 2026Changli Tang, Jie Yang, Tianyi Wang +9Multimodal Large Language ModelsMultimodal Foundation Models

  2. Support-Guided Triadic Decoding for Hallucination Mitigation in Multimodal Dialogue with LVLMs

    Nov 19, 2026Yiheng Li, Wenhao Wang, Xingyi Li +4VLM Hallucination MitigationMultimodal Hallucination

  3. FG-GraphBench: Benchmarking and Enhancing Fine-Grained Visual Understanding with Multimodal Scene Graphs

    Nov 19, 2026Sicheng Liu, Xunkai Li, Ziyu Han +3Fine-Grained Visual PerceptionMultimodal Large Language Models

  4. Choosing the Right Representations: Task-Aware Fusion for Multimodal Materials Language Models

    Nov 18, 2026Huu Binh Ta, Alex Morehead, N. Benjamin Erichson +2Multimodal Large Language ModelsMultimodal Fusion

  5. NaviGATE: Noise-Aware Dual-View Generation for Multimodal Knowledge Graph Reasoning

    Nov 18, 2026Jie Peng, Yongxue Shan, Yongfu Zha +2Multimodal KGsKG Completion

  6. WM^MLLM: Turning MLLMs into World Models

    Nov 18, 2026Baiang Li, Enze Xie, Jincheng Yu +5World ModelsInteractive World Models

  7. RFBench: A Simulator-Grounded Multimodal Benchmark for Passive RF Layout Design and Understanding

    Nov 17, 2026Jiarui Cui, Boxun YanMultimodal DatasetMultimodal Large Language Models

  8. Beyond Attribute-Centric Fairness: Modelling and Mitigating Unified Bias Representation in Omni-modal Large Language Models

    Nov 17, 2026Runqi Lin, Suqin Yuan, Zihao Fu +1Social Bias in Language ModelsLanguage Model Debiasing

  9. PACD: Provenance-Aware Consensus Decoding for Mitigating Cross-Modal Hallucinations in Omnimodal Large Language Models

    Nov 17, 2026Chang Sun, Yijun Tian, Zhenwei Tang +4Multimodal HallucinationMultimodal Large Language Models

  10. Cross-Modal StateKV Injection for Communication in Multi-LLM Systems across Temporal Segments

    Nov 17, 2026Xiaoqian Zhang, Zhihao ZhangMultimodal Large Language ModelsTemporal Video Understanding

  11. A Visual Information Extraction System Using Multimodal Large Language Models for Scanned Cognitive and Behavioral Assessments

    Nov 17, 2026Enshuo Hsu, Linh Nguyen, Sunyang Fu +5Document Information ExtractionClinical Information Extraction

  12. FaceJev: Direct Visual Decisions for Face Forgery Detection

    Nov 16, 2026Siran Peng, Tianshuo Zhang, Yibo Xue +6Image Forgery DetectionDeepfake Detection

  13. Beyond Token Stacking: Multi-scale Visual Modulation and Adaptive Fused-feature Calibration for MLLMs

    Nov 16, 2026Ye Xiao, Shesheng Song, Yi Ren +5Multimodal Large Language ModelsEfficient Multimodal Inference

  14. MemDoctor: Stage-wise Error Attribution and Repair for MLLM Agent Memory

    Nov 16, 2026Ruiyang Jiang, Xinyang Liao, Weikai Huang +3LLM Agent MemoryMemory-Augmented Language Model Agents

  15. Words Speak Louder? Exploring the Language Shortcut of LLM-Driven Incomplete Multimodal Learning

    Nov 16, 2026Qiangchang Wang, Chong Xiao, Xiaokun Bai +5Multimodal LearningMissing-Modality Learning

  16. FRAAC: A French Multimodal Large Language Model and Dataset forPictogram-Based Augmentative and Alternative Communication

    Nov 15, 2026Alexandre Audibert, François Portet, Benoit Favre +1Multimodal DatasetMultimodal Large Language Models

  17. TIVS: Mapping the Sensitivity Landscape of Visual Token Transformations in MLLMs

    Nov 15, 2026Yikang Qiao, Jianye Zeng, Duan Huang +1Multimodal Large Language ModelsVisual Evidence Sensitivity in VLMs

  18. Beyond Paired Inputs: Learning to Leverage RGB-T Cross-Modal Relations in Multimodal Large Language Models

    Nov 14, 2026Tianyi Zhao, Qihao Xu, Jiawen Xi +5Multimodal Large Language ModelsCross-Modal Learning

  19. CoPre-MoE: Consolidating and Preserving MoE for Multimodal Continual Instruction Tuning

    Nov 13, 2026Tao Hu, Ling-Xiang Zhang, Da-Wei ZhouMixture-of-Experts Fine-TuningContinual Learning for VLMs

  20. MAST: Medoid-Anchored Subspace Steering for Visual Grounding in Multimodal LLMs

    Nov 11, 2026Zhitao Chen, Zhenxin Cai, Jingyuan Li +1Activation SteeringMultimodal Large Language Models

  21. TAMEing the Open-World Personalization: Towards Open-Set Personalized MLLM Assistant

    Nov 10, 2026Rongpei Hong, Jian Lang, Ting Zhong +1LLM PersonalizationPersonalized Language Models

  22. Beyond Modality Preference: Controlling Multimodal Arbitration under Modality Conflict in MLLMs

    Nov 9, 2026Zhaoying Pan, Shenyu Lu, Jun Hui Wong +2Multimodal Large Language ModelsModality Reliance in Multimodal Models

  23. EmoHarness: Boosting Multimodal Emotion Understanding with Meta-Harness Evolving

    Nov 9, 2026Fenghai Li, Bingxuan Li, Jiawen Zhang +4Multimodal Emotion RecognitionMultimodal Large Language Models

  24. MaRiO: Multi-agent Collaborative Reasoning via Shared Observations in MLLMs

    Nov 7, 2026Nathaniel Redmond, Fidel Omar Tito Cruz, Devansh Sharma +4Multi-Agent ReasoningMultimodal Large Language Models

  25. LEMMO: A Large Electromagnetic Model leveraging Pretrained LLMs for Million-Sample Signal Understanding

    Nov 6, 2026Mingkai Li, Jiajun Wu, Yihan Mao +6Multimodal Large Language Models

  26. Measurement-Aware Multimodal LLM for No-Reference Low-Dose CT Quality Assessment

    Nov 5, 2026Kerem Basim, Tolga Demiroglu, Mehmet Ozan Unal +2Medical Image Quality AssessmentImage Quality Assessment

  27. The Geometry of Simulated Perspectives: Dissociating Representation and Behaviour in a Multimodal Language Model

    Nov 5, 2026Mathis Immertreu, Fitim Abdullahu, Thomas Kinfe +4Representation Geometry in VLMsMultimodal Large Language Models

  28. ProQuant: Progressive Quantization-aware Training for Edge MLLMs

    Nov 5, 2026Yufei Xue, Yushi Huang, Jiawei Shao +5LLM QuantizationQuantization-Aware Training