Multimodal Large Language Models

Also known as MLLM

Follow this topic Ask about this topic

Momentum

67 papers in the last four weeks, up 91% on the four weeks before. 0.7% of all new papers.

Jul 13Week of Sep 28

Latest papers 676

All topics

VisLens: Single-Pass Interpretable Visual Search for Multimodal LLMs
Aug 31, 2026Jingyi He, Sanghwan Kim, Zeynep AkataEfficient Multimodal Inference Multimodal Large Language Models
Technical University of Munich · Helmholtz Munich · Munich Center for Machine Learning
TUE-Detector: A Tool-Using Expert MLLM-Based Detector for AI-Generated Videos
Aug 31, 2026Yichen Wu, Haoxuan Qu, Yongxing Dai +5Multimodal Large Language Models Deepfake Detection
University of Nottingham · Lancaster University · Peking University
Foundation and Multimodal Large Language Models for Face Presentation and Morph Attack Detection
Aug 30, 2026Hatef Otroshi Shahreza, Asif Hussain Khan, Peter Lorenz +2Image Forgery Detection Vision Foundation Model Adaptation
Idiap Research Institute, Switzerland · Université de Lausanne, Switzerland
Modality Fault Lines: Structural Corruptions Reveal Fragile Omni-Modal Reasoning
Aug 29, 2026Zhaolu Kang, Meixin Wu, Yu Xue +6Multimodal Robustness Multimodal Large Language Models
Tencent · Peking University · Zhejiang University
Edit2TikZ: A Comprehensive and Challenging Benchmark for Scientific Figure Editing with TikZ
Aug 13, 2026Zongyun Zhang, Jiacheng Ruan, Xian Gao +5Text-Guided Image Editing Code Generation
Shanghai Jiao Tong University
Reasoning for Social Audio-Visual Question Answering: Where Do We Stand?
Aug 13, 2026Koen P. de Vries, Xavier Alameda-Pineda, Estefanía Talavera +1Supervised Fine-Tuning Multimodal Large Language Models
Inria at Univ. Grenoble Alpes, CNRS, LJK, France · University of Twente, The Netherlands
TennisVAR: A Stroke-Evidence-Grounded Multimodal Large Language Model for Tactical Reasoning in Tennis Videos
Aug 13, 2026Yifan Mei, Qingling Shi, Changli Wu +3Sports Video Analysis Multimodal Large Language Models
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University · Shanghai Innovation Institute · School of Artificial Intelligence, Shanghai Jiao Tong University
Beyond Correctness: Benchmarking and Aligning Response Behaviors in Hybrid-Thinking MLLMs
Aug 13, 2026Xinming Wang, Weinong Wang, Hongming Yang +13RL for Language Models LLM Alignment
Institute of Automation, Chinese Academy of Sciences · Tencent · Zhongguancun Academy +3
MAG: MAnifold Guided Semi-Supervised Multi-modal In-Context Learning
Aug 13, 2026Zirui Cheng, Xun Xu, Tiankai Chen +7In-Context Example Selection Multimodal ICL
College of Design and Engineering, National University of Singapore · Institute for Infocomm Research, ASTAR, Singapore
Context Blindness in DPO: Mitigating Object Hallucination in MLLMs via Context-Calibrated Preference Optimization
Aug 12, 2026Byungoh Ko, Jinyoung Park, Jongha Kim +3Direct Preference Optimization Object Hallucination in VLMs
Korea University, Seoul, Republic of Korea · KAIST, Daejeon, Republic of Korea
AWARe: Mitigating Catastrophic Forgetting via Activation-Weighted Adaptive REtention
Aug 12, 2026Juncheng Liao, Jinfan Lv, Guoming Wang +3Continual Learning for LLMs LLM Fine-Tuning
School of Software Technology, Zhejiang University, Hangzhou, China · College of Intelligent Robotics and Advanced Manufacturing, Fudan University, Shanghai, China · School of Artificial Intelligence, Sun Yat-Sen University, Guangzhou, China +1
Advancing MLLM-based UAV Image Understanding and Reasoning: A Benchmark and a Training-Free Multi-Agent System
Aug 12, 2026Haoyu Zhang, Shuoxun Zhang, Peng Ye +5Visual Question Answering Multi-Agent LLM Systems
Embedded Deep Learning and Visual Analysis Laboratory, College of Future Information Technology, Fudan University, Shanghai 200433, China · Embedded Deep Learning and Visual Analysis Laboratory, College of Future Information Technology, Fudan University, Shanghai 200433, China, and also with Shanghai Artificial Intelligence Laboratory, Shanghai 200232, China, and also with The Chinese University of Hong Kong, Hong Kong · College of Future Information Technology, Fudan University, Shanghai 200433, China +1
LEMUR: Latent Entropy-aware Multimodal Unlearning via Visual-anchored Reasoning Redirection
Aug 12, 2026Xinhao Zhong, Yuxia Qiao, Junhao Li +3VLM Unlearning Multimodal Large Language Models
3Pengcheng Laboratory · 1Harbin Institute of Technology, Shenzhen · 2Tsinghua University
MultiModal Code-Switching: Interleaving Visual Objects into Language for Explicit Object-Level Alignment
Aug 11, 2026Changhao Xiang, Shangyu Xing, Zhen Wu +2Multimodal Grounding Multimodal Pretraining
National Key Laboratory for Novel Software Technology, Nanjing University, China
E $^3$ mo-Bench: A Scalable Benchmark for Multimodal Evoked and Expressed Emotion Understanding via Bayesian Pairwise Alignment
Aug 11, 2026Lancheng Gao, Ziheng Jia, Shengyan Li +4Multimodal Large Language Models Multimodal Emotion Recognition
Shanghai Jiao Tong University
VERDICT: Training-Free Step-Wise Verification of Multimodal Reasoning via Disagreement-Aware Consensus
Aug 11, 2026Rohit Sinha, Kunal Tilaganji, Tanuja Ganu +3Multimodal Large Language Models Multimodal CoT Reasoning
Indian Institute of Technology, Hyderabad · Microsoft Research
Multimodal Item Parameter Estimation using Simulated Response Probabilitie
Aug 10, 2026Christopher Ormerod, YoungKoung KimEducational Assessment Multimodal Large Language Models
College Board
ADOPD: Reference-Privileged On-Policy Distillation for MLLM-Based Industrial Anomaly Detection
Aug 10, 2026Jingtai He, Shiyuan Meng, Wenchao Meng +1Industrial Anomaly Detection Multimodal Large Language Models
Zhejiang University
Hallucination-Free GUI Grounding via Regression-Free Layout-Aware Matching
Aug 10, 2026Yuke Li, Xuehan HouGUI Grounding Multimodal Large Language Models
School of Electronic and Computer Engineering, Peking University, Shenzhen, China
Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models
Aug 10, 2026Puneet Mathur, Manan Suri, Dinesh ManochaEfficient Multimodal Inference Multimodal Memory
University of Maryland, College Park
Improving Generalization Robustness of Multimodal RLVR
Aug 9, 2026Pengfei Zhou, Zhiwei Tang, Xiaopeng Peng +11Multimodal Robustness Multimodal Large Language Models
1National University of Singapore, HPC-AI Lab · 2DAMO Academy, Alibaba Group · 3Hupan Lab +7
Deferred Audio Pruning with Local Audio-Visual Dynamics for Omni-LLMs
Aug 9, 2026Kyeongyoon Lee, Hongyeob Kim, Youngeun Kim +1Efficient Multimodal Inference Audio Token Compression
Sungkyunkwan University · AWS AI Labs
FitAQA: A Benchmark of Fitness Action Quality Assessment for Multimodal Large Language Models
Aug 9, 2026Kaili Zheng, Kaiwen Wang, Xun Zhu +3Multimodal Large Language Models Action Quality Assessment
Department of Electronic Engineering, Tsinghua University · Xinjiang Region Sports Science Research Center · College of AI, Tsinghua University +1
Aero Realtime: Fully Aligned Input-Output Streams for Low-Latency Streaming Multimodal Generation
Aug 9, 2026Kaichen Zhang, Wei Huang, Keming Wu +2Efficient Multimodal Inference Multimodal Large Language Models
1The University of Hong Kong · 2LMMs-Lab · 3Tsinghua University
NeuPAT: Neuron-aware Plasticity Allocation Tuning for Language-Preserving MLLMs
Aug 8, 2026Jiayue Jin, Jingwei Zhang, Chen Wang +2Multimodal Large Language Models Multimodal Instruction Tuning
College of Intelligent Robotics and Advanced Manufacturing, Fudan University · Zhongguancun Academy · Tianjin University +3
TongGuOCR: A Layout-Aware and Token-Augmented OCR MLLM for Chinese Historical Documents
Aug 8, 2026Zhongheng Zhou, Yi Sun, Huiguo He +6Multimodal Large Language Models Optical Character Recognition
School of Electronic and Information Engineering, South China University of Technology, Guangzhou 510641, China · Huawei Technologies Co., Ltd., Dongguan 511700, China
MME-Safety: A Fine-grained Benchmark for Safety Evaluation of MLLMs
Aug 8, 2026Yilian Shi, Yueming Lyu, Haoxiang Tan +5LLM Safety Benchmarks Multimodal Robustness
Nanjing University
An AI4AI Framework for Visual Token Pruning
Aug 7, 2026Zhen Liu, Wenli Huang, Wei Song +3Automated Algorithm Discovery Efficient VLM Inference
Xi’an Jiaotong University · Ningbo University of Technology · 3North China University of Technology +3
LMM Modality Transfer: A Pre-requisite for Autonomous GIS Agents
Aug 7, 2026Ivan Majic, Zexian Huang, Franziska Hübl +5Cross-Modal Alignment Multimodal Large Language Models
Graz University of Technology, Austria · University of Vienna, Austria · University of Liverpool, UK
Debias in Text, Believe Your Eyes: Text-Anchored Cross-Modal Transfer for Visual Counter-Commonsense Reasoning
Aug 7, 2026Chen Ling, Hanqian Li, Dongnan Liu +9Cross-Modal Learning Visual Reasoning
Zhejiang University · Ant Group · The Hong Kong University of Science and Technology (Guangzhou) +2

Aug 31, 2026, cs.CVJ/K move · Enter open · S save

VisLens: Single-Pass Interpretable Visual Search for Multimodal LLMs

Jingyi He, Sanghwan Kim, Zeynep Akata

Technical University of Munich · Helmholtz Munich · Munich Center for Machine Learning

Efficient Multimodal Inference Multimodal Large Language Models

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) struggle with fine-grained Visual Search, the task of locating small or rare objects in high-resolution images. Existing remedies fall into two families: (1) Training-free methods based on attention or confidence scores are accurate but slow, since they require multiple MLLM queries per example. (2) Reinforcement Learning (RL) trained tool-use models are faster at inference but opaque, since their tool calls remain uncontrollable and hard to interpret. To overcome this, we propose \emph{VisLens} (Visual Focus via Logit Lens), a Visual Search method built on the logit lens, which decodes the semantics held in a hidden state by projecting it through the LLM head. VisLens further uses a lightweight tuned-lens that maps early hidden states into the final hidden state space, so visual tokens can be read out from early layers. These tokens are matched to target words in the query to generate a crop of the relevant region, which is fed back in alongside the original image to produce the final answer. The whole process, from decoding to the final answer, completes in a single forward pass without repeated queries. VisLens matches or exceeds prior baselines while delivering a substantial latency advantage, running

8.5

9.9\times

faster than Thyme and up to

22.2\times

faster than training-free multi-pass search methods.

Momentum

Related topics

Latest papers 676