Multimodal Large Language Models

Also known as MLLM

Follow this topic Ask about this topic

Momentum

67 papers in the last four weeks, up 91% on the four weeks before. 0.7% of all new papers.

Jul 13Week of Sep 28

Latest papers 676

All topics

RegRet: Enhancing Region-Level Retrieval in Large Multimodal Models
Sep 15, 2026Xun Liang, Honghui Yang, Weihang Pan +6Multimodal IR Multimodal Large Language Models
State Key Lab of CAD&CG, Zhejiang University · Xiaohongshu Inc. · School of Software Technology, Zhejiang University
ReDraft, Don't Just Distill: Reference-Driven Revision for Continual VLLM Post-Training
Sep 15, 2026Zhihao Zhang, Mingqi Wu, Qiaole Dong +15Fine-Tuning Multimodal Large Language Models
Fudan University · Shanghai Artificial Intelligence Laboratory · Tencent
A multimodal large language model for evidence-based autism spectrum disorder screening
Sep 15, 2026Jun Chen, Qi Zhao, Yunliang Jiang +8Autism Spectrum Disorder Multimodal Large Language Models
National Special Education Resource Center for Children with Autism, Zhejiang Normal University, China · School of Computer Science and Technology, Zhejiang Normal University, China · School of Mathematics and Statistics, Xi’an Jiaotong University, China +5
Circuit-MLLM: Topological Logic-Guided Latent-Space Visual Reasoning for Circuit Schematic Understanding
Sep 14, 2026Jinyuan Deng, Yuqi Jiang, Wenjing Huang +3Multimodal Large Language Models Multimodal Reasoning
Zhejiang University, Hangzhou, China
AdaVSkip: Adaptive Visual Token Skipping Across Layers For Efficient MLLMs Inference
Sep 14, 2026Yuyao Sun, Tao Deng, Shuang Li +1Efficient Multimodal Inference Multimodal Large Language Models
Beihang University
Not All Prompts Are Equal: Exploration-Guided Prompt Scaffolding for Multimodal Reinforcement Post-Training
Sep 14, 2026Yuanhao Yue, Qianli Ma, Chengyu Wang +3Multimodal Large Language Models Curriculum RL
Alibaba Cloud Computing · Shanghai Jiao Tong University · Fudan University +1
Calibrated Ambiguity in Multimodal Language Models: Humans reach for cultural references, while models describe the picture
Sep 14, 2026Cody Kommers, Mingrui Ye, Evelyn Gius +4Pragmatic Reasoning in Language Models Multimodal Large Language Models
King’s College London London, United Kingdom · University of Edinburgh Edinburgh, United Kingdom
Func-R1: Incentivizing Mathematical Function Reasoning in Multimodal Large Language Models
Sep 13, 2026Mingze Yin, Xiaohan Wang, Dian Li +8Multimodal Large Language Models Mathematical Reasoning
College of Computer Science and Technology, Zhejiang University · Tencent 3AI Thrust, HKUST (GZ) · Zhongguancun Academy +3
EMMI: Edge Multi-Modal Intelligence for Communication-Efficient MLLM Inference via Fused Representation Compression
Sep 11, 2026Motahare Mounesan, Irfan KhanEfficient Multimodal Inference Edge Computing
Texas A&M University
Bidirectional Multimodal Fusion of Sky Images and Time-Series for Solar Forecasting with Large Language Models
Sep 11, 2026Ken Chen, Maneesha Perera, Wei Wang +3Multivariate Time Series Forecasting Multimodal Large Language Models
Department of Mechanical Engineering, The University of Melbourne, Melbourne, Australia · Department of Mechanical Engineering, Sri Lanka Institute of Information Technology, Sri Lanka
OmniHallu: Unified Hallucination Detection for Cross-Modal Comprehension and Generation in Multimodal Large Language Models
Sep 11, 2026Jianjiang Yang, Peihang Li, Shanqing Xu +3Hallucination Detection in VLMs Hallucination Detection
The University of Manchester · The University of Hong Kong · Huazhong University of Science and Technology +2
OmniKVQuant: KV Cache Quantization for Omni-LLMs
Sep 11, 2026Suho Yoo, Hyunjong Ok, Jongmin Choi +2Efficient Multimodal Inference KV Caching
KAIST · POSTECH
$S^3$ -Bench: Evaluating Speech Interaction Models as Scientific Voice Assistants
Sep 9, 2026Heyang Liu, Jiayi Huang, Wenyang Xiao +8Audio QA Spoken Dialogue Systems
Shanghai Jiao Tong University · Ant Group · Tongji University
From Coordinates to Candidate Regions: Temporal Change Localization via Region Selection in Remote Sensing Multimodal LLMs
Sep 8, 2026Juwan Chung, Sungjune Park, Yeongyun Kim +1Multimodal Large Language Models Remote Sensing Change Detection
Integrated Vision Language Lab, KAIST, South Korea
InfluenceField: A Differentiable Field with Interventionally Identifiable Causal Structure for Multimodal World Modeling
Sep 7, 2026Zihao Yang, Zijia Wang, Zhiqiu HuangCausal Representation Learning Causal World Models
Mathematical Institute, University of Oxford · Department of Computer Science, University of Oxford · School of Mathematical Sciences, University of Nottingham Ningbo China
MCPO: Modality-Contrastive Preference Optimization for Multimodal Chain-of-Thought Compression
Sep 4, 2026Guangheng Yang, Zhenliang Ni, Zhenkai Wu +4Efficient Multimodal Inference Multimodal Large Language Models
Huawei Technologies Ltd. Shanghai, China
InSituMeasure: Probing Situated Measurement Grounding in Industrial Scenes with Multimodal Large Language Models
Sep 3, 2026Chao Shen, Xinyuan Li, Yunfan Zhou +4Multimodal Grounding Multimodal Robustness
Shanghai Key Laboratory of Scalable Computing and Systems · School of Computer Science, Shanghai Jiao Tong University · Qwen Team, Alibaba Group
GraFT: A Training-Free Framework for Spatial Reasoning in Multimodal Large Language Models via 3D Scene Graphs
Sep 3, 2026Junqing Du, Fernando Ropero, Erkin Turkoz +23D Spatial Reasoning Multimodal Large Language Models
Riemann Lab, Huawei Technologies
Beauty is in the AI of the beholder: MLLMs systematically overrate facial attractiveness
Sep 2, 2026Santiago Grandas, Juan Sebastian Cely-Acosta, Mohit Mendiratta +2Multimodal Large Language Models Human-AI Agreement
Qoves Inc., Wilmington, DE, United States
YesTrack: Referring Multi-Object Tracking via MLLM-based Yes/No Verification
Sep 2, 2026Quansheng Hu, Qin Sun, Qiansen Dai +4Multi-Object Tracking Multimodal Large Language Models
University of Electronic Science and Technology of China, Chengdu, China · Shenzhen Institute for Advanced Study, UESTC, Shenzhen, China
Transfer Safety Awareness for Cross-Modal Safety Drift in Multimodal Large Language Models
Sep 2, 2026Tianqi Xiao, Shiyao Cui, Minghao Zhang +2Multimodal Large Language Models Representation Engineering
1The Conversational AI (CoAI) group, DCST, Tsinghua University · 3Northwestern Polytechnical University · School of Cyberspace Security, Beijing University of Posts and Telecommunications +1
DocHop: Benchmarking Out-of-domain Multi-hop Reasoning in Information-Dense Documents
Sep 2, 2026Zhuoran Yu, Le Thien Phuc Nguyen, Jaden Park +5Multimodal Large Language Models Chart QA
University of Wisconsin-Madison · Massachusetts Institute of Technology · Stanford University +2
Evaluating Multimodal LLMs as Generalist Vision-Language-Action Agents for Drone Control: Commanding, Approaching, Tracking and Searching
Sep 1, 2026Jaewoo Park, Minyoung Lee, Sukmin Seo +11Aerial Robotics Language Model-Based Control
Drone AI Team, NAVER Cloud
S $^2$ Prune: Spatially Structured Visual Token Pruning for Multimodal Large Language Models
Sep 1, 2026Yuanyuan Jia, Shunpu Tang, Qianqian YangEfficient VLM Inference Multimodal Large Language Models
College of Information Science and Electronic Engineering, Zhejiang University Hangzhou, China
SinkPruner: Sink-Free Visual Token Pruning for Multimodal Large Language Models
Sep 1, 2026Shiyu Li, Zi-Yuan Hu, Shijia Huang +3Efficient VLM Inference Multimodal Large Language Models
1The Chinese University of Hong Kong · 2Weitu AI · 3Peking University
Forbid Your Attention: Fooling Multimodal Large Language Models by Selectively Removing Intrinsic Focus in Spectral Domain
Sep 1, 2026Daizong Liu, Junhao Dong, Zhiyuan Ma +6Adversarial Attacks on VLMs Adversarial Attacks
Institute for Math & AI, Wuhan University, Wuhan, 430072, China · College of Computing and Data Science, Nanyang Technological University, Singapore · School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, 430074, China +3
Same Semantics, Different Outcome: On the Modality Robustness of Multimodal LLMs under Knowledge Conflict
Sep 1, 2026Jungyeon Lee, Yejin Yoon, Taeuk KimMultimodal Robustness Knowledge Conflicts in Language Models
Hanyang University, Seoul, Republic of Korea
Distributed Implicit Harm: A Compositional Safety Blind Spot in MLLM-Based Video Moderation
Aug 31, 2026Ruotong Wang, Zihao Zhu, Siwei Lyu +2Multimodal Large Language Models Video Content Moderation
The Chinese University of Hong Kong, Shenzhen · Kling Team, Kuaishou Technology · State University of New York at Buffalo
MMDS-Bench: Benchmarking Multimodal Large Language Models on Dynamic Stance in Social Media Interactions
Aug 31, 2026Yuzhe Ding, Kang He, Li Zheng +5Multimodal Large Language Models Multimodal Reasoning
School of Cyber Science and Engineering, Wuhan University · Shanghai Innovation Institute
GUIDE: Guiding Internal Evidence with Language Instructions
Aug 31, 2026Soyeon Caren Han, Hyunsuk Chung, Jinwoo Kim +2Multimodal Robustness Multimodal Large Language Models
School of Computing and Information Systems, The University of Melbourne, Melbourne, Australia · Department of Computer Science and Engineering, Korea University, Seoul, Republic of Korea · Brain Science Institute, Korea Institute of Science and Technology, Seoul, Republic of Korea +1

Sep 15, 2026, cs.CVJ/K move · Enter open · S save

RegRet: Enhancing Region-Level Retrieval in Large Multimodal Models

Xun Liang, Honghui Yang, Weihang Pan, Ruisi Zhao+5

State Key Lab of CAD&CG, Zhejiang University · Xiaohongshu Inc. · School of Software Technology, Zhejiang University

Multimodal IR Multimodal Large Language Models Multimodal Contrastive Learning

Open paper AskPDF arXiv

Region-level retrieval aims to align user-specified image regions with relevant regions or textual descriptions, playing a crucial role in realworld applications such as e-commerce product search and RAG. Although recent Large Multimodal Models (LMMs) have made significant strides in multimodal retrieval, they primarily focus on global-level tasks and struggle to capture effective region-level representations. To bridge this gap, we present RegRet, an LMM-based Region-level Retrieval framework that enhances the regional representations without compromising overall global retrieval performance. At its core, RegRet integrates a Region-Aware Encoder to capture detailed regional features while balancing them with the global background context. To further enhance the fine-grained understanding and discriminability of representations, we design a multi-stage training pipeline that includes detailed localized captioning and regional contrastive learning tasks. In addition, considering the absence of region-level contrastive training data and the limited diversity of evaluation tasks in current benchmarks, we introduce the REGMB benchmark. It comprises 225k contrastive pairs, covering four multimodal retrieval tasks. Extensive experiments validate the effectiveness of our approach. RegRet outperforms strong baselines in the zero-shot setting. Further training with contrastive learning leads to an average improvement of more than 20% on both REGMB and public benchmarks, while achieving comparable or better results on global-level retrieval tasks.

Momentum

Related topics

Latest papers 676