Multimodal Large Language Models

Follow this topic Ask about this topic

Momentum

90 papers in the last four weeks, up 61% on the four weeks before. 0.9% of all new papers.

Jul 13Week of Sep 28

Latest papers 877

All topics

VKnowU: Evaluating Visual Knowledge Understanding in Multimodal LLMs
Nov 25, 2025Tianxiang Jiang, Sheng Xia, Yicheng Xu +5Video Multimodal Large Language Models Multimodal Large Language Models
University of Science and Technology of China · 2Shanghai AI Laboratory · 3Nanjing University +2
Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearning
Nov 25, 2025Zhen Zeng, Leijiang Gu, Zhangling Duan +4Large Language Model Unlearning Multimodal Large Language Models
Hefei University of Technology, China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, China · University of Amsterdam, the Netherlands
WaymoQA: A Multi-View Visual Question Answering Dataset for Safety-Critical Reasoning in Autonomous Driving
Nov 25, 2025Seungjun Yu, Seonho Lee, Namho Kim +5Waymo Open Motion Dataset Autonomous Driving
KAIST Seoul, Republic of Korea · Hanyang University Seoul, Republic of Korea
Thinking Ahead: Foresight Intelligence in MLLMs and World Model
Nov 24, 2025Zhantao Gong, Liaoyuan Fan, Qing Guo +3Recent Vision-Language Models Foresight
College of Software, Nankai University, China · The University of Hong Kong, China · NKIARI, Shenzhen Futian, China +2
Beyond Description: Cognitively Benchmarking Fine-Grained Action for Embodied Agents
Nov 24, 2025Dayong Liu, Chao Xu, Weihong Chen +5Embodied Agents Fine-Grained Actions
Zhejiang University · IROOTECH TECHNOLOGY · Wolf 1069 b Lab, Sany Group +2
RoadBench: Benchmarking MLLMs on Fine-Grained Spatial Understanding and Reasoning under Urban Road Scenarios
Nov 22, 2025Jun Zhang, Xin Zhang, Jie Feng +5Multimodal Large Language Models
Department of Electronic Engineering, BNRist, Tsinghua University, Beijing, China · Zhongguancun Academy · Amap, Alibaba Group, Beijing, China
VisReason: A Large-Scale Dataset for Visual Chain-of-Thought Reasoning
Nov 21, 2025Lingxiao Li, Yifan Wang, Xinyan Gao +3Multimodal Large Language Models Recent Vision-Language Models
Boston University, Boston, MA, USA · Stony Brook University, Stony Brook, NY, USA · MMLab, The Chinese University of Hong Kong, Hong Kong SAR, China
SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards
Nov 10, 2025Hunar Batra, Haoqin Tu, Hardy Chen +3Spatial Reasoning Spatial Grounding
University of Oxford · University of California, Santa Cruz
SEPS: Semantic-enhanced Patch Slimming Framework for fine-grained cross-modal alignment
Nov 3, 2025Xinyu Mao, Junsi Li, Haoji Zhang +2Cross-Modal Alignment Multimodal Large Language Models
School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China.
GUI-AIMA: Aligning Intrinsic Multimodal Attention with a Context Anchor for GUI Grounding
Nov 2, 2025Shijie Zhou, Viet Dac Lai, Hao Tan +4Graphical User Interface Multimodal Large Language Models
University at Buffalo, USA · Adobe Research, USA
Robobench: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models as Embodied Brain
Oct 20, 2025Yulin Luo, Chun-Kai Fan, Menghang Dong +19Robocasa Evaluation Benchmarks
State Key Laboratory of Multimedia Information Processing, School of Computer2026 Science, Peking University · Institute for Brain and Intelligence, Fudan University · University of Science andJul Technology Beijing +2
Constructive Distortion: Improving MLLMs with Attention-Guided Image Warping
Oct 10, 2025Dwip Dalal, Gautam Vashishtha, Utkarsh Mishra +6Multimodal Large Language Models Cross-Modal Attention
University of Illinois Urbana–Champaign · Skan AI · Texas A&M University +1
Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMs
Oct 1, 2025Sanghwan Kim, Rui Xiao, Stephan Alaniz +2Multimodal Large Language Models Recent Vision-Language Models
1Technical University of Munich · 2Munich Center for Machine Learning · 20263Helmholtz Munich +2
LUQ: Layerwise Ultra-Low Bit Quantization for Multimodal Large Language Models
Sep 28, 2025Shubhang Bhatnagar, Andy Xu, Kar-Han Tan +1Large Language Model Quantization Multimodal Large Language Models
University of Illinois Urbana-Champaign · Work done as an intern at HP Inc. · University of California, Los Angeles +1
Evidence Recomposition and Predictive Context Residualization for Visual Attribution in Multimodal Large Language Models
Sep 26, 2025Jiawei Liang, Jianjie Huang, Ruoyu Chen +4Multimodal Large Language Models Token-Level Uncertainty
Shenzhen Campus of Sun Yat-sen University, Shenzhen 518107, China · Zhongguan-cun Academy, Beijing 100094, China · University of Chinese Academy of Sciences, Beijing 100049, China +2
Bridging the Gap in Ophthalmic AI: MM-Retinal-Reason Dataset and OphthaReason Model toward Dynamic Multimodal Reasoning
Aug 22, 2025Ruiqi Wu, Yuang Yao, Na Su +11Multimodal Reasoning Ophthalmology
School of Computer Science and Engineering, Southeast University, Nanjing, China · Department of Ophthalmology, The First Affiliated Hospital of Nanjing Medical University, Nanjing, China · School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing, China
AD-FM: Multimodal LLMs for Anomaly Detection via Multi-Stage Reasoning and Fine-Grained Reward Optimization
Aug 6, 2025Jingyi Liao, Yongyi Su, Rong-Cheng Tu +6Vision-Language Model Adaptation Multimodal Large Language Models
1Nanyang Technological University · Institute for Infocomm Research (I2R), A*STAR · 3South China University of Technology
Test-time Prompt Refinement for Text-to-Image Models
Jul 22, 2025Mohammad Abdul Hafeez Khan, Yash Jain, Siddhartha Bhattacharyya +1Modern Text-To-Image Models Image Generation
Florida Institute of Technology, USA · Microsoft Research, USA
Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey
Jul 21, 2025Jindong Li, Yali Fu, Jiahong Liu +5Large Language Model Quantization Multimodal Large Language Models
Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · Jilin University, Changchun, China · The Chinese University of Hong Kong, Hong Kong, China +2
Region-Aware Multimodal Large Language Model via SlowFast Tokenization and Pseudo-Mask Guidance for 3D CT Report Generation
Jun 29, 2025Sunggu Kyung, Jinyoung Seo, Hyunseok Lim +7Radiology Report Generation Cone-Beam Computed Tomography
Department of Convergence Medicine, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea · University of Ulsan College of Medicine, Seoul, Republic of Korea · Department of Radiology and Research Institute of Radiology, University of Ulsan College of Medicine, Asan Medical Center, Seoul, Republic of Korea
Evidence-Grounded Trustworthy Multimodal Reasoning and Evaluation Benchmark in Complex Urban Scenes
Jun 11, 2025Zhaoyang Wei, Bowen Jiang, Xumeng Han +6Multimodal Reasoning Visual Reasoning
University of Chinese Academy of Sciences(UCAS) · Tencent CDG
Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning
Jun 8, 2025Weiwen Xu, Hou Pong Chan, Long Li +15Multimodal Clinical Data Multimodal Large Language Models
LASA Team · DAMO Academy, Alibaba Group
The NTNU System at the S&I Challenge 2025 SLA Open Track
Jun 5, 2025Hong-Yun Lin, Tien-Hong Lo, Yu-Hsuan Fang +4Wav2Vec Transcript
Department of Computer Science and Information Engineering, National Taiwan Normal University, Taiwan · Institute of AI Interdisciplinary Applied Technology, National Taiwan Normal University, Taiwan
Seeing Isn't Orienting: A Cognitively Informed Hierarchical Benchmark for Object Orientation in MLLMs
May 27, 2025Nazia Tasnim, Keanu Nichols, Yuting Yan +4Spatial Reasoning Object-Centric Representations
Boston University · Work done while at Boston University; currently at Johns Hopkins University
Decoding the Multimodal Mind: Generalizable Brain-to-Text Translation via Multimodal Alignment and Adaptive Routing
May 15, 2025Chunyu Ye, Yunhao Zhang, Jingyuan Sun +3Brain-Computer Interface Multimodal Alignment
1State Key Laboratory of Multimodal Artificial Intelligence System, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Department of Computer Science, The University of Manchester +1
Multimodal Language Models as Text-to-Image Model Evaluators
May 1, 2025Jiahui Chen, Candace Ross, Reyhane Askari-Hemmat +5Modern Text-To-Image Models Text-To-Image
1FAIR at Meta · University of Texas at Austin · ∗Work done at Meta +3
CompArt: Operationalizing Aesthetic Alignment in Text-to-Image Generation via Principles of Art
Mar 15, 2025Zhe Jin, Tat-Seng ChuaArt Value Alignment
National University of Singapore Singapore
SpurLens: Automatic Detection of Spurious Cues in Multimodal LLMs
Mar 11, 2025Parsa Hosseini, Sumit Nawathe, Mazda Moayeri +2Multimodal Large Language Models Large Language Models Fail
Department of Computer Science University of Maryland
Histopathology Multi-modal Embedding for Pathology Composed Retrieval
Feb 11, 2025Qifeng Zhou, Wenliang Zhong, Thao M. Dang +4Pathology Foundation Models Multimodal Clinical Data
The University of Texas at Arlington, Arlington TX 76010, USA
A Systematic Survey of Semantic Role Labeling in the Era of Pretrained Language Models
Feb 9, 2025Huiyao Chen, Meishan Zhang, Jing Li +4Multimodal Large Language Models Natural Language Processing
1Harbin Institute of Technology (Shenzhen) 2 Shenzhen Loop Area Institute (SLAI) · 1Harbin Institute of Technology (Shenzhen) · University of Oslo +1
Divide and Conquer: A Hybrid Strategy Defeats Multimodal Large Language Models
Dec 21, 2024Yanxu Mao, Peipei Liu, Tiehan Cui +3Large Language Model Jailbreaks Jailbreak Attacks
School of Computer Science and Technology, Shanghai Jiao Tong University, Shanghai, China · Zhongguancun Laboratory, Beijing, China · School of Software, Henan University, Kaifeng, China +1
MLLM-LLaVA-FL: Multimodal Large Language Model Assisted Federated Learning
Sep 9, 2024Jianyi Zhang, Hao Frank Yang, Ang Li +5Multimodal Large Language Models Heterogeneity
Duke University · Johns Hopkins University · University of Maryland College Park +1
Multimodal Large Language Model driven Radiology Report Generation with Clinical Knowledge Enhancement
Mar 11, 2024Miaojing Shi, Tianyu Cen, Zijie Yue +3Radiology Report Generation Multimodal Clinical Data
College of Electronic and Information Engineering, Tongji University, Shanghai, China
MAD: Modality-Adaptive Decoding for Mitigating Cross-Modal Hallucinations in Multimodal Large Language Models
Date pendingSangyun Chung, Se Yeon Kim, Youngchae Chee +1Multimodal Large Language Models Modality-Specific Encoders
Integrated Vision Language Lab, KAIST, South Korea
Do Multimodal LLMs See Before They Read? Diagnosing Contextual Sycophancy
Date pendingYi-Cheng Lai, Hen-Hsen HuangMultimodal Large Language Models Sycophancy
Institute of Information Science, Academia Sinica Taipei, Taiwan
Slot2Text: Object-Centric Visual Tokenization for Efficient and Spatially Traceable Surgical MLLMs
Date pendingGuiqiu Liao, Matjaz Jogan, Daniel A. HashimotoLong Visual-Token Sequences Multimodal Large Language Models
GRASP Laboratory, University of Pennsylvania · PCASO Laboratory, Department of Surgery, University of Pennsylvania · Department of Computer and Information Science, University of Pennsylvania
SaFeR-Steer: Evolving Multi-Turn MLLMs via Synthetic Bootstrapping and Feedback Dynamics
Date pendingHaolong Hu, Hanyu Li, Tiancheng He +6Mllm-Based Large Language Model Alignment
Huazhong University of Science and Technology · Beijing University of Posts and Telecommunications · West China Biomedical Big Data Center, Sichuan University +2

Nov 25, 2025, cs.CVJ/K move · Enter open · S save

VKnowU: Evaluating Visual Knowledge Understanding in Multimodal LLMs

Tianxiang Jiang, Sheng Xia, Yicheng Xu, Linquan Wu+4

University of Science and Technology of China · 2Shanghai AI Laboratory · 3Nanjing University+2

Video Multimodal Large Language Models Multimodal Large Language Models

Open paper AskPDF arXiv

While Multimodal Large Language Models (MLLMs) have become adept at recognizing objects, they often lack the intuitive, human-like understanding of the world's underlying physical and social principles. This high-level vision-grounded semantics, which we term visual knowledge, forms a bridge between perception and reasoning, yet remains an underexplored area in current MLLMs. To systematically evaluate this capability, we present VKnowU, a comprehensive benchmark featuring 1,680 questions in 1,249 videos, covering 8 core types of visual knowledge spanning both world-centric (e.g., intuitive physics) and human-centric (e.g., subjective intentions). Evaluation of 28 SOTA MLLMs reveals that leading models still fall short of human performance, with particularly notable gaps in the world-centric. To bridge this gap, we introduce a new dataset, VKnowQA, and VideoKnow+, a baseline model that explicitly incorporates visual knowledge into MLLMs. VideoKnow+ follows a structured See-Think-Answer paradigm and adopts reinforcement learning with visual knowledge reward, achieving a +3.7% improvement on VKnowU and consistent gains on MVBench (+5.4%), Video-MME (+7.0%), and MMVU (+5.7%). Our work highlights visual knowledge as a missing cornerstone for developing more generalizable MLLMs that can not only see but also truly understand our worlds.

Momentum

Related topics

Latest papers 877