Multimodal Large Language Models

Also known as MLLM

Follow this topic Ask about this topic

Momentum

67 papers in the last four weeks, up 91% on the four weeks before. 0.7% of all new papers.

Jul 13Week of Sep 28

Latest papers 676

All topics

Enhancing Visual Question Answering with Multimodal LLMs via Chain-of-Question Guided Retrieval-Augmented Generation
May 5, 2026Quanxing Xu, Ling Zhou, Xian Zhong +3Multimodal RAG Visual Question Answering
School of Computer Science and Engineering, Macau University of Science and Technology, Macao SAR 999078, China · Hubei Key Laboratory of Transportation Internet of Things, School of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan, Hubei 430070, China · State Key Laboratory of Maritime Technology and Safety, Wuhan University of Technology, Wuhan 430063, China +3
Can Multimodal Large Language Models Understand Pathologic Movements? A Pilot Study on Seizure Semiology
May 5, 2026Lina Zhang, Tonmoy Monsoor, Mehmet Efe Lorasdagi +8Video Understanding Multimodal Large Language Models
University of California, Los Angeles · Mercedes-Benz AG · Zhejiang University
Bolek: A Multimodal Language Model for Molecular Reasoning
May 4, 2026Frederic Grabowski, Jacek Szczerbiński, Maciej Jaśkowski +4LLM Grounding Scientific Reasoning
1Ingenix.ai, Warsaw, Poland · Laboratory of Bioinformatics and Computational Genomics, Warsaw University of Technology, Warsaw, Poland
Visual Latents Know More Than They Say: Unsilencing Latent Reasoning in MLLMs
May 4, 2026Xin Zhang, Qiqi Tao, Jiawei Du +2Inference-Time Optimization Test-Time Optimization
Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore · Institute of High Performance Computing, Agency for Science, Technology and Research, Singapore · Singapore University of Technology and Design +1
From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs
May 4, 2026Le Zhang, Jihan Yang, Soundarya Krishnan +11Spatial Reasoning Benchmarks Visual Spatial Reasoning
1Mila - Qu´ebec AI Institute, UdeM · 2NYU · 3Apple +1
Referring Multiple Regions with Large Multimodal Models via Contextual Latent Steering
May 3, 2026Yun Xing, Hanyuan Liu, Jiahao Nie +1Multimodal Large Language Models Vision-Language Grounding
Nanyang Technological University
Mitigating Multimodal LLMs Hallucinations via Relevance Propagation at Inference Time
May 3, 2026Itai Allouche, Joseph KeshetMultimodal Grounding Inference-Time Optimization
Department of Electrical and Computer Engineering, Technion, Haifa, Israel
Injecting Distributional Awareness into MLLMs via Reinforcement Learning for Deep Imbalanced Regression
May 2, 2026Yao Du, Shanshan Song, Xiaomeng LiMultimodal Large Language Models RL Fine-Tuning
The Hong Kong University of Science and Technology, Hong Kong SAR, China.
DiagramNet: An End-to-End Recognition Framework and Dataset for Non-Standard System-Level Diagrams
May 2, 2026Jincheng Lou, Ruohan Xu, Jiapeng Li +6Multi-Agent LLM Systems Electronic Design Automation
School of IC, Peking University, Beijing, China · College of Artificial Intelligence, Xi’an Jiaotong University, Xi’an, China · Department of Precision Instruments, Tsinghua University, Beijing, China +4
OralMLLM-Bench: Evaluating Cognitive Capabilities of Multimodal Large Language Models in Dental Practice
May 2, 2026Rongyang Wang, Shuang Zhou, Jiashuo Wang +2Medical Imaging Cognitive Modeling
Department of Orthodontics, Beijing Stomatological Hospital, School of Stomatology, Capital Medical University, Fanjiacun Road #9, Fengtai District, Beijing 100070, China · Division of Computational Health Sciences, Department of Surgery, University of Minnesota, Minneapolis, MN, USA · Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR, China +1
Valley3: Scaling Omni Foundation Models for E-commerce
May 2, 2026Zeyu Chen, Guanghao Zhou, Min Yang +6Unified Multimodal Models Multimodal Pretraining
Valley Team, ByteDance Group
EmoMM: Benchmarking and Steering MLLM for Multimodal Emotion Recognition under Conflict and Missingness
May 1, 2026Yueru Sun, Yimeng Zhang, Haoyu Gu +5Missing-Modality Learning Multimodal Large Language Models
School of Future Technology, South China University of Technology, Guangzhou, China
Let ViT Speak: Generative Language-Image Pre-training
May 1, 2026Yan Fang, Mengcheng Lan, Zilong Huang +7Cross-Modal Learning Vision Transformer
1Beijing Jiaotong University · 2ByteDance · 3Nanyang Technological University
Generating Statistical Charts with Validation-Driven LLM Workflows
May 1, 2026Pavlin G. Poličar, Andraž Pevcin, Blaž ZupanData Visualization Multimodal Large Language Models
University of Ljubljana · Faculty of Computer and Information Science · Veˇcna pot 113, 1000 Ljubljana, Slovenia
DIYHealth Suite: Dataset, Model, and Benchmark for Health Management at Home
May 1, 2026Changshuo Liu, Junran Wu, Zhongle Xie +9Healthcare Multimodal Large Language Models
School of Computing, National University of Singapore, Singapore
Multimodal Large Language Models as Synthetic Participants in Video-Based Studies: An Evaluation
May 1, 2026Prabal Shrestha, Bohan Jiang, Haoning Xue +2Multimodal Large Language Models Language Model Bias Evaluation
Department of Computer Science, Boise State University, Boise, ID, USA · School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ, USA · Department of Communication, University of Utah, Salt Lake City, UT, USA
GaMMA: Towards Joint Global-Temporal Music Understanding in Large Multimodal Models
May 1, 2026Zuyao You, Zhesong Yu, Mingyu Liu +3Multimodal Pretraining Multimodal Large Language Models
1Fudan University · 2ByteDance
Are Multimodal LLMs Ready for Clinical Dermatology? A Real-World Evaluation in Dermatology
May 1, 2026Roy Jiang, Hyunjae Kim, Zhenyue Qin +8Healthcare Clinical Triage
Department of Dermatology, Yale School of Medicine, Yale University, New Haven, CT, USA · Department of Bioinformatics and Data Science, Yale School of Medicine, Yale University, New Haven, CT, USA · Yale School of Medicine, Yale University, New Haven, CT, USA
Beyond SFT-to-RL: Pre-alignment via Black-Box On-Policy Distillation for Multimodal RL
Apr 30, 2026Sudong Wang, Weiquan Huang, Xiaomin Yu +9Multimodal Large Language Models Multimodal Reasoning
Hong Kong University of Science and Technology (Guangzhou) · Nanyang Technological University · Tsinghua University +3
From Mirage to Grounding: Towards Reliable Multimodal Circuit-to-Verilog Code Generation
Apr 30, 2026Guang Yang, Xing Hu, Xiang Chen +1Multimodal Grounding Code Generation
State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, Hangzhou, China · School of Artificial Intelligence and Computer Science, Nantong University, Nantong, China
MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction
Apr 30, 2026Junbo Cui, Bokai Xu, Chongyi Wang +33Efficient Multimodal Inference Unified Multimodal Models
MiniCPM-o Team, OpenBMB
GLM-5V-Turbo: Toward a Native Foundation Model for Multimodal Agents
Apr 29, 2026GLM-V Team, :, Wenyi Hong +95Multimodal Large Language Models Multimodal Reasoning
State Beyond Appearance: Diagnosing and Improving State Consistency in Dial-Based Measurement Reading
Apr 29, 2026Yuanze Hu, Gen Li, Yuqin Lan +5Multimodal Robustness Cross-Modal Alignment
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University, Beijing, China · Beihang University, Beijing, China · Fudan University, Shanghai, China +1
Assessing Y-Axis Influence: Bias in Multimodal Language Models on Chart-to-Table Translation
Apr 27, 2026Seok Hwan Song, Azher Ahmed Efat, Wallapak TavanapongData Visualization Multimodal Large Language Models
Department of Computer Science, Iowa State University, Ames, Iowa, USA
Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligence
Apr 27, 2026NVIDIA, :, Amala Sanjay Deshmukh +216Efficient Multimodal Inference Multimodal Large Language Models
X2SAM: Any Segmentation in Images and Videos
Apr 27, 2026Hao Wang, Limeng Qiao, Chi Zhang +4Image Segmentation Video Object Segmentation
Sun Yat-Sen University · 2 Peng Cheng Laboratory · 3 Meituan Inc
Aligned Multi-View Scripts for Universal Chart-to-Code Generation
Apr 27, 2026Zhihan Zhang, Lizi LiaoData Visualization Code Generation
School of Computing and Information Systems, Singapore Management University
ShredBench: Evaluating the Semantic Reasoning Capabilities of Multimodal LLMs in Document Reconstruction
Apr 26, 2026Zichun Guo, Yuling Shi, Wenhao Zeng +6Image Reconstruction Multimodal Large Language Models
Xidian University · Shanghai Jiao Tong University · Alibaba Qwen +1
EmoTrans: A Benchmark for Understanding, Reasoning, and Predicting Emotion Transitions in Multimodal LLMs
Apr 25, 2026He Hu, Tengjin Weng, Zebang Cheng +5Multimodal Large Language Models Multimodal Model Evaluation
Shenzhen University Shenzhen, China · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) Shenzhen, China · Queen Mary University of London +3
UpstreamQA: A Modular Framework for Explicit Reasoning on Video Question Answering Tasks
Apr 25, 2026Jason Nguyen, Ameet Rao, Alexander Chang +2Multimodal Large Language Models Video QA
Lincoln North Star High School · The Charter School of Wilmington · Greenwich High School +2

May 5, 2026, cs.CVJ/K move · Enter open · S save

Enhancing Visual Question Answering with Multimodal LLMs via Chain-of-Question Guided Retrieval-Augmented Generation

Quanxing Xu, Ling Zhou, Xian Zhong, Xiaohua Huang+2

School of Computer Science and Engineering, Macau University of Science and Technology, Macao SAR 999078, China · Hubei Key Laboratory of Transportation Internet of Things, School of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan, Hubei 430070, China · State Key Laboratory of Maritime Technology and Safety, Wuhan University of Technology, Wuhan 430063, China+3

Multimodal RAG Visual Question Answering Multimodal Large Language Models Multimodal CoT Reasoning Knowledge-Based VQA

Open paper AskPDF arXiv

With advances in multimodal research and deep learning, Multimodal Large Language Models (MLLMs) have emerged as a powerful paradigm for a wide range of multimodal tasks. As a core problem in vision-language research, Visual Question Answering (VQA) has increasingly employed MLLMs to improve performance, particularly in open-domain settings where external knowledge is essential. In this work, we aim to further enhance retrieval-based VQA by more effectively integrating MLLMs with structured reasoning and knowledge acquisition. We introduce a logical prompting strategy that fuses Chain-of-Thought (CoT) reasoning with Visual Question Decomposition (VQD), termed CoVQD, to guide retrieval toward more accurate and relevant knowledge for MLLM inference. Building on this idea, we propose a new framework, CoVQD-guided RAG (CgRAG), which enables MLLMs to access more comprehensive and coherent external knowledge while benefiting from structured visual-text reasoning guidance, thereby improving generalization and reliability in complex cross-domain VQA scenarios. Extensive experiments on E-VQA, InfoSeek, and OKVQA benchmarks demonstrate the effectiveness of the proposed method.

Momentum

Related topics

Latest papers 676