Vision-Language Models

Also known as VLM

Follow this topic Ask about this topic

Momentum

55 papers in the last four weeks, up 120% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 491

All topics

DiffPrune: differentiable information throttling for token pruning in vision-language models
Aug 3, 2026Landi He, Mingde Yao, Shawn Young +1Vision-Language Models Efficient VLM Inference
Shenzhen University of Advanced Technology · CUHK MMLab, CPII under InnoHK
Recompute or Reuse? Diagnosing and Mitigating Textual Shortcuts in VLM Self-Reflection
Aug 3, 2026Wenxiao Fan, Jingling Fu, Fang Li +9Vision-Language Models VLM Reasoning
School of Computer Science, Beijing Institute of Technology · Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University
SpatioLM: Towards General Physical Spatial Intelligence in Vision-Language Models
Aug 3, 2026Jing Wu, Jianhua Wu, Jiayi Guan +5Vision-Language Models Visual Spatial Reasoning
Xiaomi EV, Beijing, China · College of Automotive and Energy Engineering, Tongji University, Shanghai, China · Independent Researcher
DAVET: Denoising-Aware Visual Evidence Trajectory Allocation for Diffusion Vision-Language Models
Aug 3, 2026Yongkang Zhou, Xiang Xia, Cheng Yan +2Vision-Language Models Efficient VLM Inference
University of Science and Technology of China Hefei, China
Can Urban Blight Be Accessed with Vision-language Models: A Case Study in Detroit
Aug 3, 2026Xiaohao Yang, Aohua Tian, Derek Van Berkel +2VLM Evaluation Vision-Language Models
School for Environment and Sustainability; School of Information University of Michigan, Ann Arbor, USA
Linear Multi-Timescale Retention as a Memory-Efficient Vision-Language Bridge
Aug 3, 2026Ashfak Yeafi, Mehedi Hasan, Md Khairul IslamMemory-Augmented VLMs Vision-Language Models
Department of Electrical and Electronic Engineering Khulna University of Engineering & Technology Khulna-9203, Bangladesh · Department of Computer Science and Engineering Brac University, Dhaka, Bangladesh · Department of Mathematics and Computer Science Hobart and William Smith Colleges Geneva, NY, USA
Remember-R1: Mitigating Long-Context Visual Forgetting through Reinforcement Learning
Aug 2, 2026Jianmin Chen, Jiaqi Tang, Wei Wei +9Vision-Language Models Long-Context Modeling
Hong Kong University of Science and Technology Hong Kong, China · Northwestern Polytechnical University Xi’an, China · Zhejiang University Hangzhou, China
Evaluating VLMs on Multimodal Aristotelian Persuasion Tasks
Aug 2, 2026Khondoker Ittehadul IslamVLM Evaluation Vision-Language Models
Saarland University · Department of Language Science & Technology
Decoy Images Amplify Caption-Mediated Defenses Against Encoded Jailbreaks
Aug 2, 2026Haoyu Zhang, Xiangchen Guan, Shibo Zheng +2VLM Robustness Adversarial Attacks on VLMs
MonitorVLM-v2: A Deployed Vision-Language Framework for Real-Time Safety Violation Detection
Aug 2, 2026Jiang Wu, Sichao Wu, Yinsong Ma +2Vision-Language Models Efficient VLM Inference
School of Mechanical Engineering, University of Science and Technology Beijing · The Laboratory for Computational Sensing and Robotics, Johns Hopkins University
Relax Within, Balance Across: Geometry-Guided Load Balancing for Vision-Language Mixture-of-Experts
Aug 1, 2026Ziang Wu, Peng Jin, Qishen Yin +4Expert Load Balancing Vision-Language Models
2Peng Cheng Laboratory · School of Software and Microelectronics, Peking University · 1Shenzhen Graduate School, Peking University +2
Through the LENS: Local Geometric Decomposition of Vision-Language Model Representations
Aug 1, 2026Shalom Kachko, Raz Lapid, Margarita Vald +2Vision-Language Models VLM Interpretability
1Ben-Gurion University of the Negev · 2Intuit Inc. · 3Deepkeep
Attention-Steered Vision-Language Models for Sign Language Translation
Jul 31, 2026Meibo Hu, Guohao Sun, Annemarie D. Ross +2Sign Language Translation Vision-Language Models
Rochester Institute of Technology · National Technical Institute for the Deaf / Rochester Institute of Technology · University of Virginia
Sycophancy Undermines Epistemic Vigilance in Cooperative Vision-Language Tasks
Jul 31, 2026Rupak Sarkar, Neha Srikanth, Saloni Gupta +3VLM Evaluation Vision-Language Models
University of Maryland, College Park · Army Research Lab
Counting the Cost of War Under Satellite Embargo: Zero-Shot Estimation of Impacted Infrastructure
Jul 31, 2026Saleh Sakib Ahmed, M. Sohel RahmanRemote Sensing Image Understanding Vision-Language Models
Department of Computer Science and Engineering, Bangladesh University of Engineering and Technology, Palashi, Dhaka, 1000, Bangladesh.
ReToken: Improving Long-Context VLMs with Visual Retrieval Token
Jul 30, 2026Yao Xiao, Reuben Tan, Zhen Zhu +3Vision-Language Models Vision-Language Retrieval
University of Illinois at Urbana-Champaign · Microsoft Research · Google DeepMind
MarkushGlyph and OCSRGlyph: Improved Chemical Structure Recognition
Jul 30, 2026Alex Andonian, Samuel G Rodriques, Andrew D White +1Vision-Language Models Optical Character Recognition
1Edison Scientific
Scaling Vision-Language Models Is Not Enough to Mitigate Bias
Jul 30, 2026Ioannis Sarridis, Ioannis Kompatsiaris, Symeon PapadopoulosVLM Evaluation VLM Robustness
Information Technologies Institute, CERTH Thessaloniki, Greece
MemeBench: What LVLMs Miss When Interpreting Culture-Dependent Memes
Jul 30, 2026Weihang Wang, Kainan Tu, Jielei Zhang +9VLM Evaluation Vision-Language Models
1Bilibili · 2Fudan University · 3Peking University
Prior Directions: Why GUI Grounding Gets Locked in the Past
Jul 29, 2026Weile Gong, Zijian Lu, Mingcai Chen +3Vision-Language Models GUI Grounding
Nanjing University of Posts and Telecommunications, Nanjing, China
Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model
Jul 27, 2026Senqiao Yang, Kaichen Zhang, Zhaoyang Jia +20Vision-Language Models Efficient VLM Inference
Microsoft Mage Team
UltraViT: Latency-Optimized On-device Vision Encoder for Large Vision-Language Models
Jul 25, 2026Ioannis Maniadis Metaxas, Adrian Bulat, Alberto Baldrati +4Vision-Language Models Efficient VLM Inference
Samsung AI Cambridge · Technical University of Iasi · Queen Mary University of London
What CLIP Knows but Cannot Say: Recovering Negation from Frozen Intermediate Features
Jul 25, 2026Chen-Yi Lu, Yueh-Shao Chen, Somali ChaterjiVision-Language Models VLM Adaptation
Purdue University, West Lafayette, IN 47907, USA
Similarity Is Not Logic: Factored Inference for Dual-Encoder Vision-Language Models
Jul 25, 2026Sultan Alshehri, Zhantao Yang, Han Zhang +1Vision-Language Models VLM Reasoning
Carnegie Mellon University, Pittsburgh, PA, USA.
Small Vision-Language Models Know When They Are Wrong But Cannot Say So: A Two-Model Study of Stated versus Internal Confidence Under Realistic Image Degradation
Jul 24, 2026M M Asif FerdousVLM Robustness Vision-Language Models
Independent Researcher
SCALE: Self-Supervised Constraint-Aware Layout GEneration for Local P&R DRV Fixing at Advanced Nodes
Jul 23, 2026Chia-Tung Ho, Haoyu Yang, Guanglei Zhou +6Vision-Language Models Self-Supervised Learning
1NVIDIA · 2Duke University
ENTRAP-VL: A Taxonomic Probe for Dual Contextual Entrainment in Vision-Language Models
Jul 22, 2026Karan Goyal, Afreen Hossain, Debojyoti Das +1VLM Evaluation Vision-Language Models
IIIT Delhi, India · Dr. Ambedkar Institute of Technology, India · Heritage Institute of Technology, India +1
An Interactive Vision Language Platform for Cognitive Remediation in Schizophrenia
Jul 21, 2026Nassira Ait Mehdi, Milissa Temmam, Slimane LarabiVision-Language Models Mental Health
RIIMA Laboratory, Computer Science Faculty, USTHB University, 16111, Algeria
No Training, Better Flights: Test-Time Scaled VLMs for UAV Navigation
Jul 21, 2026Feinan Cheng, Dongliang Xu, Wenli Nong +4Vision-Language Models UAV Navigation
Shandong University Weihai, China · China University of Petroleum (East China) Qingdao, China · MBZUAI Abu Dhabi, United Arab Emirates
Linguistic Context Recodes Visual Representations in Vision-Language Models
Jul 21, 2026Brian Song, Michael A. Lepori, Ellie PavlickVision-Language Models VLM Interpretability
Brown University

Aug 3, 2026, cs.CVJ/K move · Enter open · S save

DiffPrune: differentiable information throttling for token pruning in vision-language models

Landi He, Mingde Yao, Shawn Young, Lijian Xu

Shenzhen University of Advanced Technology · CUHK MMLab, CPII under InnoHK

Vision-Language Models Efficient VLM Inference Token Pruning Visual Token Pruning

Open paper AskPDF arXiv

Visual token pruning reduces the computational cost of Vision-Language Models (VLMs) by removing redundant visual tokens. The key is to learn a score that measures whether a token is useful. Existing methods typically rely on Gumbel-Softmax to approximate discrete selection during training. Such selectors make the score depend on the behavior of a relaxed pruning operator, not directly on the consequence of information loss. In this paper, we propose DiffPrune, which gives token scores a direct meaning. During training, DiffPrune keeps all tokens and weakens each token's information according to its score. If weakening a token hurts the task, the scorer is pushed to protect it; if not, the token can receive a lower score. Because the loss is differentiated through this actual information-throttling path, the scorer avoids the unstable surrogate path of relaxed token selection. DiffPrune implements this idea with an Information Throttler, which injects variance-preserving noise into visual tokens, where high-score tokens remain close to their original representations, while low-score tokens carry less original information. At inference, the throttler is removed, and hard top-K pruning is applied using the learned scores. Across ten VLM benchmarks, DiffPrune retains 96.5% of full-model accuracy while accelerating LLM prefill by 2.85x, with only 0.69 ms inference overhead. Code will be publicly available.

Momentum

Related topics

Latest papers 491