Large Vision-Language Models

Also known as LVLM

Follow this topic Ask about this topic

Momentum

30 papers in the last four weeks, up 36% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 253

All topics

OccamToken: Efficient VLM Inference with Training-Free and Budget-Adaptive Token Pruning
May 28, 2026Geng Li, Guohao Chen, Ting Chen +6Efficient VLM Inference Large Vision-Language Models
1Nanyang Technological University (NTU)
Self-Prophetic Decoding to Unlock Visual Search in LVLMs
May 27, 2026Zhendong He, Qiyuan Dai, Guanbin Li +2Large Vision-Language Models VLM Reasoning
School of Computer Science and Engineering, Sun Yat-sen University · ShanghaiTech University
Risk-aware Selective Prompting for Hallucination Mitigation in Large Vision-Language Models
May 27, 2026Yuang Huang, Yafeng Zhang, Yu ZilanVision-Language Models Hallucination in Language Models
Shanghai Jiao Tong University · iFLYTEK · Tsinghua University
Structure-Guided Visual Perturbation Neutralization for LVLMs
May 27, 2026Yuanhe Zhang, Xueting Wang, YanBin Ren +6VLM Robustness Adversarial Attacks on VLMs
1Beijing University of Posts and Telecommunications · University of Science and Technology of China · 3JIUTIAN Research +2
Leveraging Visual Signals for Robust Token-Level Uncertainty in Vision-Language Generation
May 26, 2026Joseph Hoche, David Brellmann, Gianni FranchiVision-Language Models Large Vision-Language Models
AMIAD, Pôle Recherche, Palaiseau · Safran Tech · ENSTA Paris
LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence
May 25, 2026Xiang An, Yin Xie, Feilong Tang +27VLM Evaluation Temporal Video Grounding
Glint Lab · AIM for Health Lab · MVP Lab
Beyond Appearance: Can Multimodal Large Language Models Exploit Vertical Structure for Remote Sensing Natural Scene Understanding?
May 25, 2026Jing Huang, Duanchu Wang, Junjie Yang +5VLM Evaluation Remote Sensing Image Understanding
School of Software Engineering, Xi’an Jiaotong University, Xi’an 710049, China · Shenyang Institute of Automation, University of Chinese Academy of Sciences, Shenyang, China · State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi’an Jiaotong University
Adversarial Orthogonal Disentanglement for LVLM Hallucination Mitigation
May 25, 2026Ruoxi Cheng, Haoxuan Ma, Zhengfei Hai +6Adversarial Representation Learning Vision-Language Models
Fudan University · Tencent · Nanjing University +3
VaaWIT: Visual-Aware Adaptation of Large Language Models for Multilingual Web Image Translation
May 23, 2026Bo Li, Ronghao Chen, Ningyuan Deng +3VLM Adaptation Large Vision-Language Models
Tsinghua University Baidu Inc. Beijing, China · QuantaAlpha Beijing, China · The Hong Kong University of Science and the Technology Hong Kong, China +1
PGT: Procedurally Generated Tasks for improving visual grounding in MLLMs
May 22, 2026Rim Assouel, Amir Bar, Michal Drozdzal +1Visual Spatial Reasoning Synthetic Data Generation
1Mila - Québec AI Institute · 4McGill University · 3FAIR at Meta Superintelligence Labs +2
CVSearch: Empowering Multimodal LLMs with Cognitive Visual Search for High-Resolution Image Perception
May 22, 2026Liupeng Li, Haoqian Kang, Zhenyu Lu +4Efficient VLM Inference Large Vision-Language Models
Harbin Institute of Technology, Shenzhen, China · Peng Cheng Laboratory, Shenzhen, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.
Exposing Vulnerabilities in Visible-Infrared VLMs: A Unified Geometric Adversarial Framework with Cross-Task Transferability
May 21, 2026Xiang Chen, Yuxian Dong, Chao Li +6VLM Robustness Adversarial Patch Attacks
Reducing Object Hallucination in LVLMs via Emphasizing Image-negative Tokens
May 20, 2026Meng Shen, Minghao Wu, Deepu RajanVision-Language Models Hallucination in Language Models
Nanyang Technological University · Monash University
Mitigating Hallucinations in Large Vision-Language Models via Causal Route Gating
May 20, 2026Zhe Cheng, Wenyu Chen, Fode Zhang +1VLM Robustness Hallucination in Language Models
Center of Statistical Research, School of Statistics and Data Science, Southwestern University of Finance and Economics, Chengdu, China. · Department of Biomedical Engineering, College of Design and Engineering, National University of Singapore, Singapore.
Finding the Correct Visual Evidence Without Forgetting: Mitigating Hallucination in LVLMs via Inter-Layer Visual Attention Discrepancy
May 20, 2026Yutong Xie, Zhenglin Hua, Ran Wang +3Vision-Language Models Hallucination in Language Models
School of Computer Science and Engineering, Southeast University, Nanjing, China · School of Artificial Intelligence, Shenzhen University, Shenzhen, China · National Engineering Laboratory for Big Data Systems Computing Technology, Shenzhen University, Shenzhen, China +3
Breaking Modality Heterogeneity in Low-Bit Quantization for Large Vision-Language Models
May 19, 2026Yi Zhong, Haotong Qin, Xindong Zhang +2Vision-Language Models VLM Quantization
VCIP, College of Computer Science, Nankai University · D-ITET, ETH Zürich · OPPO Research Institute +1
Why We Look Where We Look: Emergent Human-like Fixations of a Foveated Visual Language Model Maximizing Scene Understanding
May 18, 2026Shravan Murlidaran, Ziqi Wen, Sana Shehabi +1Visual Attention Large Vision-Language Models
Psychological & Brain Sciences, University of California, Santa Barbara & 93106, USA. · Computer Science, University of California, Santa Barbara & 93106, USA. · Electrical and Computer Engineering, University of California, Santa Barbara & 93106, USA.
When a Zero-Shooter Cheats: Improving Age Estimation via Activation Steering
May 17, 2026Erik Imgrund, Pia Hanfeld, Klim Kireev +1VLM Robustness Large Vision-Language Models
BIFOLD & TU Berlin
DriveSafe: A Framework for Risk Detection and Safety Suggestions in Driving Scenarios
May 16, 2026Sainithin Artham, Shankar Gangisetty, Avijit Dasgupta +1VLMs for Autonomous Driving Autonomous Driving Safety Evaluation
IIIT-Hyderabad, India
A Cross-Modal Prompt Injection Attack against Large Vision-Language Models with Image-Only Perturbation
May 15, 2026Hao Yang, Zhuo Ma, Yang Liu +3VLM Robustness Adversarial Attacks on VLMs
Xidian University Xi’an, Shaanxi, China
EntropyScan: Towards Model-level Backdoor Detection in LVLMs via Visual Attention Entropy
May 15, 2026Xuanyu Ge, Zhongqi Wang, Jie Zhang +2Large Vision-Language Models Backdoor Detection
China University of Geosciences · University of the Chinese Academy of Sciences · Institute of Computing Technology, Chinese Academy of Sciences
LRCP: Low-Rank Compressibility Guided Visual Token Pruning for Efficient LVLMs
May 15, 2026Hongyu Lu, Feng Zhang, Wenwei Jin +5Efficient VLM Inference Large Vision-Language Models
1Xiaohongshu · 2Harbin Institute of Technology · 3Fudan University
AGC: Adaptive Geodesic Correction for Adversarial Robustness on Vision-Language Models
May 15, 2026Zhiwei Li, Jiacheng Xue, Weining Wang +4VLM Robustness Adversarial Robustness
1NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences · School of Computer Science and Engineering, Central South University · University of Chinese Academy of Sciences
Deep Pre-Alignment for VLMs
May 14, 2026Tianyu Yu, Kechen Fang, Zihao Wan +5Vision-Language Models Large Vision-Language Models
1Tsinghua University · 2Taobao & Tmall Group of Alibaba · 3Shanghai Qi Zhi Institute.
ATLAS: Agentic or Latent Visual Reasoning? One Word is Enough for Both
May 14, 2026Ziyu Guo, Rain Liu, Xinyan Chen +1Visual Reasoning Large Vision-Language Models
Meta AI · The Chinese University of Hong Kong
MHSA: A Lightweight Framework for Mitigating Hallucinations via Steered Attention in LVLMs
May 14, 2026Wei Ding, Yilin Li, Yudong Zhang +4Vision-Language Models LLM Hallucination Mitigation
Tsinghua University Beijing, China · Tsinghua University, Tencent Beijing, China · Tencent Beijing, China +2
Do We Really Need External Tools to Mitigate Hallucinations? SIRA: Shared-Prefix Internal Reconstruction of Attribution
May 14, 2026Tian Qin, Junzhe Chen, Yuqing Shi +3Efficient VLM Inference Hallucination in Language Models
Tsinghua University · The University of Sydney · Stanford University +1
CoRDS: Coreset-based Representative and Diverse Selection for Streaming Video Understanding
May 14, 2026Ailar Mahdizadeh, Puria Azadi, Muchen Li +2Streaming Video Understanding KV Caching
University of British Columbia · Vector Institute
To See is Not to Learn: Protecting Multimodal Data from Unauthorized Fine-Tuning of Large Vision-Language Model
May 14, 2026Chengshuai Zhao, Zhen Tan, Dawei Li +2VLM Robustness Multimodal Robustness
School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ, USA · Department of Computer Science and Engineering, Texas A&M University, College Station, TX, USA.
Training Long-Context Vision-Language Models Effectively with Generalization Beyond 128K Context
May 13, 2026Zhaowei Wang, Lishu Luo, Haodong Duan +9Visual Question Answering Vision-Language Models
1CSE Department, HKUST · 2ByteDance Seed · *Work done at ByteDance Seed

May 28, 2026, cs.CVJ/K move · Enter open · S save

OccamToken: Efficient VLM Inference with Training-Free and Budget-Adaptive Token Pruning

Geng Li, Guohao Chen, Ting Chen, Shilin Shan+5

1Nanyang Technological University (NTU)

Efficient VLM Inference Large Vision-Language Models Visual Token Pruning Visual Token Compression

Open paper AskPDF arXiv

Vision-language models (VLMs) rely on long visual token sequences for visual understanding, making the prefill stage expensive in both computation and memory. Most existing pruning methods follow an absolute-ranking paradigm, assigning importance scores to visual tokens and retaining a fixed top-K subset. In this work, we argue that this paradigm is fundamentally brittle: attention sinks distort token importance rankings, while image redundancy and query-dependent visual evidence make fixed token budgets unreliable across inputs. We propose OccamToken, a training-free framework that replaces absolute token ranking with register-anchored relative evidence testing. Instead of asking which tokens are globally important, OccamToken evaluates whether a visual token provides information beyond a register-based reference. Our key insight is that register tokens naturally absorb low-information attention patterns, making them a stable reference for identifying genuinely informative visual evidence. Based on this principle, OccamToken performs both image-adaptive redundancy pruning and query-adaptive relevance pruning through dynamic thresholds derived from register attention. Across LLaVA-NeXT, LLaVA-v1.5, and Qwen3-VL, OccamToken consistently improves the accuracy-efficiency trade-off without additional training. Notably, on LLaVA-NeXT, it reduces 2,880 visual tokens to approximately 40 while preserving over 93% of the original accuracy, enabling stable visual token compression even in the extreme 1.4% retention regime.

Momentum

Related topics

Latest papers 253