Multimodal Large Language Models

Follow this topic Ask about this topic

Momentum

90 papers in the last four weeks, up 61% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 877

All topics

DiagramNet: An End-to-End Recognition Framework and Dataset for Non-Standard System-Level Diagrams
May 2, 2026Jincheng Lou, Ruohan Xu, Jiapeng Li +6Diagrams Table Reasoning Datasets
School of IC, Peking University, Beijing, China · College of Artificial Intelligence, Xi’an Jiaotong University, Xi’an, China · Department of Precision Instruments, Tsinghua University, Beijing, China +4
OralMLLM-Bench: Evaluating Cognitive Capabilities of Multimodal Large Language Models in Dental Practice
May 2, 2026Rongyang Wang, Shuang Zhou, Jiashuo Wang +2Occlusal Constraint Multimodal Clinical Data
Department of Orthodontics, Beijing Stomatological Hospital, School of Stomatology, Capital Medical University, Fanjiacun Road #9, Fengtai District, Beijing 100070, China · Division of Computational Health Sciences, Department of Surgery, University of Minnesota, Minneapolis, MN, USA · Department of Computing, The Hong Kong Polytechnic University, Hong Kong SAR, China +1
Beyond Perceptual Shortcuts: Causal-Inspired Debiasing Optimization for Generalizable Video Reasoning in Lightweight MLLMs
May 2, 2026Jingze Wu, Quan Zhang, Hongfei Suo +2Video Understanding Multimodal Large Language Models
Sun Yat-sen University, China
EmoMM: Benchmarking and Steering MLLM for Multimodal Emotion Recognition under Conflict and Missingness
May 1, 2026Yueru Sun, Yimeng Zhang, Haoyu Gu +5Emotion Recognition Multimodal Large Language Models
School of Future Technology, South China University of Technology, Guangzhou, China
Generating Statistical Charts with Validation-Driven LLM Workflows
May 1, 2026Pavlin G. Poličar, Andraž Pevcin, Blaž ZupanChart Large Language Model Workflows
University of Ljubljana · Faculty of Computer and Information Science · Veˇcna pot 113, 1000 Ljubljana, Slovenia
Scaling Video Understanding via Compact Latent Multi-Agent Collaboration
May 1, 2026Kerui Chen, Jinglu Wang, Jianrong Zhang +3Fine-Grained Video Understanding Multimodal Large Language Models
Microsoft Research Asia · Zhejiang University · Guanming Lab.
Multimodal Large Language Models as Synthetic Participants in Video-Based Studies: An Evaluation
May 1, 2026Prabal Shrestha, Bohan Jiang, Haoning Xue +2Video Multimodal Large Language Models Multimodal Large Language Models
Department of Computer Science, Boise State University, Boise, ID, USA · School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ, USA · Department of Communication, University of Utah, Salt Lake City, UT, USA
Are Multimodal LLMs Ready for Clinical Dermatology? A Real-World Evaluation in Dermatology
May 1, 2026Roy Jiang, Hyunjae Kim, Zhenyue Qin +8Multimodal Clinical Data Melanoma
Department of Dermatology, Yale School of Medicine, Yale University, New Haven, CT, USA · Department of Bioinformatics and Data Science, Yale School of Medicine, Yale University, New Haven, CT, USA · Yale School of Medicine, Yale University, New Haven, CT, USA
SpecVQA: A Benchmark for Spectral Understanding and Visual Question Answering in Scientific Images
Apr 30, 2026Jialu Shen, Han Lyu, Suyang Zhong +5Textvqa Multimodal Large Language Models
DP Technology
MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction
Apr 30, 2026Junbo Cui, Bokai Xu, Chongyi Wang +33Omni-Modal Full-Duplex Speech Models
MiniCPM-o Team, OpenBMB
COHERENCE: Benchmarking Fine-Grained Image-Text Alignment in Interleaved Multimodal Contexts
Apr 30, 2026Bingli Wang, Huanze Tang, Haijun Lv +5Multimodal Large Language Models Image-Text Alignment
1Southeast University · 2Shanghai AI Laboratory
State Beyond Appearance: Diagnosing and Improving State Consistency in Dial-Based Measurement Reading
Apr 29, 2026Yuanze Hu, Gen Li, Yuqin Lan +5Measures Multimodal Large Language Models
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University, Beijing, China · Beihang University, Beijing, China · Fudan University, Shanghai, China +1
Multimodal LLMs are not all you need for Pediatric Speech Language Pathology
Apr 29, 2026Darren Fürst, Sebastian Steindl, Ulrich SchäferSpeech Language Models Dysarthric Speech
Ostbayerische Technische Hochschule Amberg-Weiden
SIEVES: Selective Prediction Generalizes through Visual Evidence Scoring
Apr 28, 2026Hector G. Rodriguez, Marcus RohrbachVisual Evidence Multimodal Large Language Models
TU Darmstadt & hessian.AI, Germany
Learning from Medical Entity Trees: An Entity-Centric Medical Data Engineering Framework for MLLMs
Apr 28, 2026Jianghang Lin, Haihua Yang, Deli Yu +6Medical Ontology Multimodal Clinical Data
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, China · ByteDance · Northeastern University
OmniVTG: A Large-Scale Dataset and Training Paradigm for Open-World Video Temporal Grounding
Apr 28, 2026Minghang Zheng, Zihao Yin, Yi Yang +2Video Temporal Grounding Multimodal Large Language Models
Wangxuan Institute of Computer Technology, Peking University · Central Media Technology Institute, Huawei Technologies Ltd. · State Key Laboratory of General Artificial Intelligence, Peking University +1
M $^3$ -VQA: A Benchmark for Multimodal, Multi-Entity, Multi-Hop Visual Question Answering
Apr 28, 2026Jiatong Ma, Longteng Guo, Yuchen Liu +4Knowledge-Based Visual Question Answering Multimodal Reasoning
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences
Assessing Y-Axis Influence: Bias in Multimodal Language Models on Chart-to-Table Translation
Apr 27, 2026Seok Hwan Song, Azher Ahmed Efat, Wallapak TavanapongChart Tabular Learning
Department of Computer Science, Iowa State University, Ames, Iowa, USA
X2SAM: Any Segmentation in Images and Videos
Apr 27, 2026Hao Wang, Limeng Qiao, Chi Zhang +4Video Object Segmentation Multimodal Large Language Models
Sun Yat-Sen University · 2 Peng Cheng Laboratory · 3 Meituan Inc
Robust Grounding with MLLMs Against Occlusion and Small Objects via Language-Guided Semantic Cues
Apr 27, 2026Beomchan Park, Seongho Kim, Hyunjun Kim +2Multimodal Large Language Models Grounding
Integrated Vision Language Lab., KAIST, Republic of Korea
ShredBench: Evaluating the Semantic Reasoning Capabilities of Multimodal LLMs in Document Reconstruction
Apr 26, 2026Zichun Guo, Yuling Shi, Wenhao Zeng +6Visual Document Retrieval Multimodal Large Language Models
Xidian University · Shanghai Jiao Tong University · Alibaba Qwen +1
EmoTrans: A Benchmark for Understanding, Reasoning, and Predicting Emotion Transitions in Multimodal LLMs
Apr 25, 2026He Hu, Tengjin Weng, Zebang Cheng +5Emotion Recognition Multimodal Large Language Models
Shenzhen University Shenzhen, China · Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ) Shenzhen, China · Queen Mary University of London +3
OceanPile: A Large-Scale Multimodal Ocean Corpus for Foundation Models
Apr 25, 2026Yida Xue, Ningyu Zhang, Tingwei Wu +5Multimodal Dataset Ocean
College of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China. · School of Software Technology, Zhejiang University, Ningbo 315048, China. · State Key Laboratory of Ocean Sensing, Hangzhou 311200, China. +2
Au-M-ol: A Unified Model for Medical Audio and Language Understanding
Apr 25, 2026Meizhu Liu, Nistha Mitra, Paul Li +3Automatic Speech Recognition Multimodal Clinical Data
Oracle AI Science · Neuramill
SOLAR-RL: Semi-Online Long-horizon Assignment Reinforcement Learning
Apr 24, 2026Jichao Wang, Liuyang Bian, Yufeng Zhou +9Long-Horizon Task Planning Multimodal Large Language Models
1vivo AI Lab · 2Zhejiang Lab · 5Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences +2
CGC: Compositional Grounded Contrast for Fine-Grained Multi-Image Understanding
Apr 24, 2026Lihao Zheng, Zhenwei Shao, Yu Zhou +5Fine-Grained Perception Multimodal Large Language Models
School of Computer Science and Technology, Hangzhou Dianzi University · Base Model, Li Auto
MTT-Bench: Predicting Social Dominance in Mice via Multimodal Large Language Models
Apr 24, 2026Yunquan Chen, Haoyu ChenBehavioral Foundation Models Species
Department of Communication Systems, KTH Royal Institute of Technology, Stockholm, Sweden · CMVS, University of Oulu, Finland
SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environments
Apr 24, 2026Chih-Ting Liao, Xi Xiao, Chunlei Meng +6Spatial Memory Spatial Reasoning
The University of New South Wales · The University of Alabama at Birmingham · Fudan University +4
CNSL-bench: Benchmarking the Sign Language Understanding Capabilities of MLLMs on Chinese National Sign Language
Apr 24, 2026Rui Zhao, Xuewen Zhong, Xiaoyun Zheng +2Sign Language Translation Multilingual Benchmark
School of Informatics, Xiamen University, China · 2Key Lab of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian-Taiwan (XMU), Ministry of Culture and Tourism, China · 3National Language Resources Monitoring and Research Center for Education and Teaching Media, Xiamen University, China
Can Multimodal Large Language Models Truly Understand Small Objects?
Apr 24, 2026Fujun Han, Junan Chen, Xintong Zhu +4Multimodal Large Language Models Small Models
Shanghai AI Laboratory · The Chinese University of Hong Kong, Shenzhen · University of Science and Technology of China +2
Towards Temporal Compositional Reasoning in Long-Form Sports Videos
Apr 24, 2026Siyu Cao, Lu Zhang, Ruizhe Zeng +1Video Temporal Grounding Temporal Reasoning
MAIS, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · Nanjing Artificial Intelligence Research of IA, Nanjing, 211100, China
Do MLLMs Understand Pointing? Benchmarking and Enhancing Referential Reasoning in Egocentric Vision
Apr 23, 2026Chentao Li, Zirui Gao, Mingze Gao +3Egocentric Vision Multimodal Large Language Models
Department of Automation, Tsinghua University · Academy of Art & Design, Tsinghua University
Beyond Single Plots: A Benchmark for Question Answering on Multi-Charts
Apr 23, 2026Azher Ahmed Efat, Seok Hwan Song, Wallapak TavanapongChart Multimodal Query
Department of Computer Science, Iowa State University, Ames, Iowa, USA
Can MLLMs "Read" What is Missing?
Apr 23, 2026Jindi Guo, Chaozheng Huang, Xi FangMultimodal Large Language Models Question-Answer Pairs
DP Technology
Leveraging Multimodal LLMs for Built Environment and Housing Attribute Assessment from Street-View Imagery
Apr 22, 2026Siyuan Yao, Siavash Ghorbany, Kuangshi Ai +6Real Estate Data Multimodal Large Language Models
University of Notre Dame, Notre Dame, IN, 46556, USA.
Thinking Like a Botanist: Challenging Multimodal Language Models with Intent-Driven Chain-of-Inquiry
Apr 22, 2026Syed Nazmus Sakib, Nafiul Haque, Shahrear Bin Amin +4Diagnostic Benchmark Visual Reasoning
Department of Robotics and Mechatronics Engineering, University of Dhaka, Dhaka, Bangladesh · Department of Computer Science and Engineering, University of Dhaka, Dhaka, Bangladesh · Department of Agronomy, Gazipur Agricultural University, Gazipur, Bangladesh +1
Where and What: Reasoning Dynamic and Implicit Preferences in Situated Conversational Recommendation
Apr 22, 2026Dongding Lin, Jian Wang, Yongqi Li +1Preference Alignment Learning Multimodal Large Language Models
Department of Computing, The Hong Kong Polytechnic University · College of Computer Science, Sichuan University
Render-in-the-Loop: Vector Graphics Generation via Visual Self-Feedback
Apr 22, 2026Guotao Liang, Zhangcheng Wang, Juncheng Hu +5Scalable Vector Graphic Visual Generation
School of Software, Beihang University, China · 3 4Paradigm, China · Department of Computer Science, The University of Hong Kong, China
SSL-R1: Self-Supervised Visual Reinforcement Post-Training for Multimodal Large Language Models
Apr 22, 2026Jiahao Xie, Alessio Tonioni, Nathalie Rauschmayr +2Multimodal Large Language Models Recent Vision-Language Models
Max Planck Institute for Informatics, SIC · VIA Research Center · Google
Exploring Spatial Intelligence from a Generative Perspective
Apr 22, 2026Muzhi Zhu, Shunyao Jiang, Huanyi Zheng +9Stable Spatial Understanding Multimodal Large Language Models
Zhejiang University, State Key Laboratory of CAD & CG · Ant Group · Zhejiang University of Technology +1
CHASM: Unveiling Covert Advertisements on Chinese Social Media
Apr 22, 2026Jingyi Zheng, Tianyi Hu, Yule Liu +5Social Media Posts Content Moderation
Hong Kong University of Science and Technology (Guangzhou) · Aarhus Univeristy
CCTVBench: Contrastive Consistency Traffic VideoQA Benchmark for Multimodal LLMs
Apr 22, 2026Xingcheng Zhou, Hao Guo, Rui Song +5Traffic Counterfactuals
Technical University of Munich · University of California, Los Angeles
X-PCR: A Benchmark for Cross-modality Progressive Clinical Reasoning in Ophthalmic Diagnosis
Apr 22, 2026Gui Wang, Zehao Zhong, YongSong Zhou +6Multimodal Clinical Data Ophthalmology
School of Computer Science and Software Engineering, Shenzhen University · 2Tsinghua University · 5Wenzhou Medical University +3
Text Steganography with Dynamic Codebook and Multimodal Large Language Model
Apr 22, 2026Jianxin Gao, Ruohan Lei, Wanli PengSteganography Large Language Model Watermarks
China Agricultural University Beijing, China
Detecting AI-Generated Content on Social Media with Multi-modal Language Models
Apr 21, 2026Chenyang Yang, Shen Yan, Yibo Yang +13Ai-Generated Content Detection Generative Artificial Intelligence
Carnegie Mellon University · Meta
Unveiling Fine-Grained Visual Traces: Evaluating Multimodal Interleaved Reasoning Chains in Multimodal STEM Tasks
Apr 21, 2026Jing Jin, Hao Liu, Yan Bai +9Multimodal Large Language Models Electron Microscopy
1Tsinghua University · 3Central South University · 2Meituan Inc +3
SafetyALFRED: Evaluating Safety-Conscious Planning of Multimodal Large Language Models
Apr 21, 2026Josue Torres-Fonseca, Naihao Deng, Yinpei Dai +5Hazard Safety-Critical Scenarios
University of Michigan · Boise State University
DT2IT-MRM: Debiased Preference Construction and Iterative Training for Multimodal Reward Modeling
Apr 21, 2026Zhihong Zhang, Jie Zhao, Xiaojian Huang +5Multimodal Large Language Models Multimodal Dataset
University of Science and Technology of China
Seeing Candidates at Scale: Multimodal LLMs for Visual Political Communication on Instagram
Apr 21, 2026Michael Achmann-Denkler, Mario Haim, Christian WolffSocial Media Posts Multimodal Large Language Models
Media Informatics Group University of Regensburg Regensburg, Germany · Department of Media and Communication Ludwig-Maximilians-Universität Munich, Germany
Bridging Traditional Explainability Methods and Multimodal Multilingual Models: An XAI-Based Analysis
Apr 21, 2026Paweł Pozorski, Jakub Muszyński, Maria GanzhaMultimodal Large Language Models Explainability
student record book number 327304 · student record book number 327246 · thesis supervisor
mllm-shap: A Shapley Value Explainability Platform for Text-Audio Multimodal Large Language Models
Apr 21, 2026Jakub Muszyński, Paweł Pozorski, Maria GanzhaMultimodal Large Language Models Mllm-Based
Warsaw University of Technology, Warsaw, Poland
ProjLens: Unveiling the Role of Projectors in Multimodal Model Safety
Apr 21, 2026Kun Wang, Cheng Qian, Miao Yu +6Multimodal Large Language Models Projection Heads
3Nanyang Technological University · 2Beijing University of Aeronautics and Astronautics · University of Science and Technology of China +2
DUALVISION: RGB-Infrared Multimodal Large Language Models for Robust Visual Reasoning
Apr 20, 2026Abrar Majeedi, Zhiyuan Ruan, Ziyi Zhao +3Recent Vision-Language Models Multimodal Large Language Models
University of Wisconsin-Madison · Amazon · Northeastern University
Autonomous Skeletal Landmark Localization towards Agentic C-Arm Control
Apr 20, 2026Jay Jung, Ahmad Arrabi, Jax Luo +2One-Shot Medical Landmark Detection Multimodal Large Language Models
University of Vermont, Burlington, 05405, Vermont, U.S.A. · Cleveland Clinic, Cleveland, 44195, Ohio, U.S.A.
Omni-Embed-Audio: Leveraging Multimodal LLMs for Robust Audio-Text Retrieval
Apr 20, 2026HaeJun Yoo, Yongseop Shin, Insung Lee +2Text-To-Audio Embedder
Sogang University
EVE: Verifiable Self-Evolution of MLLMs via Executable Visual Transformations
Apr 20, 2026Yongrui Heng, Chaoya Jiang, Han Yang +2Self-Evolution Multimodal Large Language Models
School of Control Science and Engineering, Shandong University Jinan, China · Zeekr, Geely Auto Shanghai, China
Multiplication in Multimodal LLMs: Computation with Text, Image, and Audio Inputs
Apr 20, 2026Samuel G. Balter, Ethan Jerzak, Connor T. JerzakMultimodal Large Language Models Modalities
University of Texas at Austin · National University of Singapore (NUS)
MM-JudgeBias: A Benchmark for Evaluating Compositional Biases in MLLM-as-a-Judge
Apr 20, 2026Sua Lee, Sanghee Park, Jinbae ImLlm-As-A-Judge Multimodal Large Language Models
Seoul National University · NAVER Cloud AI · KAIST AI
Retrieval-Augmented Multimodal Model for Fake News Detection
Apr 20, 2026Yiheng Li, Weihai Lu, Hanyi Yu +1Fake News Detection Fake News
University of International Business and Economics · Beijing, China · Peking University +4
Correct When Paired, Wrong When Split: Decoupling and Editing Modality-Specific Neurons in MLLMs
Apr 20, 2026Tingchao Fu, Wenkai Wang, Fanxiao Li +6Multimodal Large Language Models
School of Information Science and Engineering, Yunnan University · School of Software, Yunnan University · 3National University of Singapore +1

May 2, 2026, cs.AIJ/K move · Enter open · S save

DiagramNet: An End-to-End Recognition Framework and Dataset for Non-Standard System-Level Diagrams

Jincheng Lou, Ruohan Xu, Jiapeng Li, Junyin Pi+5

School of IC, Peking University, Beijing, China · College of Artificial Intelligence, Xi’an Jiaotong University, Xi’an, China · Department of Precision Instruments, Tsinghua University, Beijing, China+4

Diagrams Table Reasoning Datasets Diagnostic Benchmark Multimodal Large Language Models Chip Design Chain-of-Thought Reasoning

Open paper AskPDF arXiv

System-level diagrams encode the architectural blueprint of chip design, specifying module functions, dataflows, and interface protocols. However, non-standardized symbols and the scarcity of structured training data hinder existing multimodal large language models (MLLMs) from recognizing these diagrams. To address this gap, we introduce DiagramNet, the first multimodal dataset for system-level diagrams, comprising 10,977 connection annotations and 15,515 chain-of-thought QA pairs across four tasks: Listing, Localization, Connection, and Circuit QA. Building on this dataset, we propose a progressive training pipeline together with a decoupled multi-agent workflow that decomposes complex visual reasoning into Perception, Reasoning, and Knowledge stages. On the DiagramNet benchmark, integrating our 3B-parameter model with the proposed workflow surpasses the 2025 EDA Elite Challenge winner and outperforms GPT-5, Claude-Sonnet-4, and Gemini-2.5-Pro by over 2x in end-to-end evaluation. Notably, the workflow generalizes beyond our model, boosting Task 1 performance by 128.7x for Gemini-2.5-Pro and 12.4x for GPT-5. Furthermore, with only 60 images for detector adaptation, the method transfers effectively to AMSBench, achieving zero-shot connectivity reasoning on par with GPT-5 and Claude-Sonnet-4 while surpassing the AMS state-of-the-art method Netlistify.

Momentum

Related topics

Latest papers 877