Efficient VLM Inference

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

38 papers in the last four weeks, up 100% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 322

All topics

LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding
May 26, 2026Shihao Wang, Shilong Liu, Yuanguo Kuang +10Efficient VLM Inference Vision-Language Grounding
The Hong Kong Polytechnic University · NVIDIA · Princeton University +2
MobileExplorer: Accelerating On-Device Inference for Mobile GUI Agents via Online Exploration
May 26, 2026Runxi Huang, Liyu Zhang, Shengzhong Liu +1Mobile GUI Automation Efficient VLM Inference
Hong Kong University of Science and Technology · Shanghai Jiao Tong University
STORM: Internalized Modeling for Spatial-Temporal Reasoning in Video-Language Models
May 25, 2026Yiming Liang, Yixiao Chen, Yiyang Zhou +8Efficient VLM Inference Latent Visual Reasoning
1Purdue · 2Harvard · 3UNC +3
MuCRASP: Multimodal Chain-of-thought Reasoning aware Structured Pruning
May 25, 2026Aritra Dutta, Somak AdityaVision-Language Models Efficient VLM Inference
Indian Institute of Technology, Kharagpur
CVSearch: Empowering Multimodal LLMs with Cognitive Visual Search for High-Resolution Image Perception
May 22, 2026Liupeng Li, Haoqian Kang, Zhenyu Lu +4Efficient VLM Inference Large Vision-Language Models
Harbin Institute of Technology, Shenzhen, China · Peng Cheng Laboratory, Shenzhen, China · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China.
CHASD: Language Increment-Calibrated Contrastive Decoding against Hallucination in LVLMs
May 22, 2026Xiaoyi Huang, Kejia Zhang, Zhiming LuoHallucination Detection in VLMs Efficient VLM Inference
Institute of Artificial Intelligence, Xiamen University · Department of Artificial Intelligence, Xiamen University
Fast-dDrive: Efficient Block-Diffusion VLM for Autonomous Driving
May 22, 2026Kewei Zhang, Jin Wang, Sensen Gao +9VLMs for Autonomous Driving End-to-End Autonomous Driving
1Peking University · 3The University of Hong Kong · 2NVIDIA +1
GA-VLN: Geometry-Aware BEV Representation for Efficient Vision-Language Navigation
May 21, 2026Jiahao Yang, Zihan Wang, Xiangyang Li +4Efficient VLM Inference Geometric Representation Learning
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, Beijing · University of Chinese Academy of Sciences, Beijing · School of Computing, National University of Singapore +2
Focus-then-Context: Subject-Centric Progressive Visual Token Reduction for Vision-Language Models
May 20, 2026Yulin Zhao, Zheng ZhangVision-Language Models Efficient VLM Inference
Harbin Institute of Technology, Shenzhen, China · ShenZhen Loop Area Institute
EventPrune: Cascaded Event-Assisted Token Pruning for Efficient First-Person Dynamic Spatial Reasoning
May 19, 2026Pengtao Ma, Ziliang Zhou, Ciyu Ruan +7Visual Spatial Reasoning Efficient VLM Inference
Shenzhen International Graduate School, Tsinghua University · Harbin Institute of Technology · Tsinghua University +1
Rotation-Aligned Key Channel Pruning for Efficient Vision-Language Model Inference
May 19, 2026Beomseok Kang, Dongwon Jo, Jiwon Song +2Vision-Language Models Efficient VLM Inference
Seoul National University
Wasserstein Equilibrium Decoding for Reliable Medical Visual Question Answering
May 18, 2026Luca Hagen, Johanna P. Müller, Weitong Zhang +2Healthcare Efficient VLM Inference
Friedrich-Alexander University Erlangen-Nürnberg, Erlangen, Germany · Imperial College London, London, United Kingdom · University College London, London, United Kingdom
A More Word-like Image Tokenization for MLLMs
May 18, 2026Hyun Lee, Hyemin Jeong, Yejin Kim +4Image Tokenization Efficient VLM Inference
Seoul National University · Ewha Womans University
An Efficient Streaming Video Understanding Framework with Agentic Control
May 18, 2026Jinming Liu, Jianguo Huang, Zhaoyang Jia +7Streaming Video Understanding Adaptive Model Routing
1Shanghai Jiao Tong University · 2Eastern Institute of Technology, Ningbo, China · 3Microsoft Research Asia
SafeLens: Deliberate and Efficient Video Guardrails with Fast-and-Slow Screening
May 17, 2026Shahriar Kabir Nahin, Hadi Askari, Muhao Chen +1Adaptive Inference Efficient VLM Inference
University of South Florida · University of California, Davis
t-gems: text-guided exit modules for decreasing clip image encoder
May 17, 2026Alberto Presta, Grzegorz Stefanski, Michal Byra +1Cross-Modal Learning Vision-Language Models
Samsung AI Center Warsaw · Institute of Fundamental Technological Research, PAS, Warsaw
FastOCR: Dynamic Visual Fixation via KV Cache Pruning for Efficient Document Parsing
May 17, 2026Zihan Tang, Leqi Shen, Hui Chen +7Visual Attention Efficient VLM Inference
Zihan Tang1∗† · Leqi Shen1∗ · Hui Chen1 +7
LiteFrame: Efficient Vision Encoders Unlock Frame Scaling in Video LLMs
May 17, 2026Jihwan Kim, Nikhil Parthasarathy, Danfeng Qin +5Efficient VLM Inference Video-Language Models
1Google DeepMind · 2Seoul National University
HEED: Density-Weighted Residual Alignment for Hybrid Vision-Language Model Distillation
May 16, 2026Yihao Liang, Niraj K. JhaCross-Modal Knowledge Distillation Efficient VLM Inference
Princeton University
Thinking with Patterns: Breaking the Perceptual Bottleneck in Visual Planning via Pattern Induction
May 16, 2026Yichang Jian, Boyuan Xiao, Zhenyuan Huang +2Efficient VLM Inference Latent Visual Reasoning
State Key Lab of CAD& CG Zhejiang University · Zhejiang University
LRCP: Low-Rank Compressibility Guided Visual Token Pruning for Efficient LVLMs
May 15, 2026Hongyu Lu, Feng Zhang, Wenwei Jin +5Efficient VLM Inference Large Vision-Language Models
1Xiaohongshu · 2Harbin Institute of Technology · 3Fudan University
RoiMAM: Region-of-Interest Medical Attention Model for Efficient Vision-Language Understanding
May 15, 2026Jiayan Yang, Zhuoyu Wu, Wenqi FangHealthcare Efficient VLM Inference
Shenzhen Institutes of Advanced Technology, Chinese Academy of Science · CyPhi(ΨΦ) AI Research Lab, School of IT, Monash University, Malaysia Campus
KVCapsule: Efficient Sequential KV Cache Compression for Vision-Language Models with Asymmetric Redundancy
May 14, 2026Yingbing Huang, Tharun Adithya Srikrishnan, Steven K. Reinhardt +1Vision-Language Models Efficient VLM Inference
University of Illinois Urbana-Champaign · AMD
Do We Really Need External Tools to Mitigate Hallucinations? SIRA: Shared-Prefix Internal Reconstruction of Attribution
May 14, 2026Tian Qin, Junzhe Chen, Yuqing Shi +3Efficient VLM Inference Hallucination in Language Models
Tsinghua University · The University of Sydney · Stanford University +1
INAR-VL: Input-Aware Routing for Edge-Cloud Vision-Language Inference
May 13, 2026Ahmed Šabanović, Paul Joe Maliakel, Ivona BrandićEfficient VLM Inference Edge-Cloud Computing
TU Wien
GRIP-VLM: Group-Relative Importance Pruning for Efficient Vision-Language Models
May 13, 2026Mingzhe Huang, Weijun Wang, Xin Ding +7Efficient VLM Inference Visual Token Pruning
Institute for AI Industry Research (AIR), Tsinghua University · University of Science and Technology of China · Nanjing University +1
CLIP Tricks You: Training-free Token Pruning for Efficient Pixel Grounding in Large VIsion-Language Models
May 13, 2026Sangin Lee, Yukyung ChoiEfficient VLM Inference Vision-Language Grounding
Dept. of Artificial Intelligence and Robotics, Sejong University, Seoul, Republic of Korea · Artificial Intelligence and Robotics Institute (AIRI), Sejong University, Seoul, Republic of Korea
AdaFocus: Adaptive Relevance-Diversity Sampling with Zero-Cache Look-back for Efficient Long Video Understanding
May 13, 2026Xiao Yang, Yingzhe Ma, Haoxuan Yu +2Efficient VLM Inference Long-Video Understanding
University of Electronic Science and Technology of China · China
VIP: Visual-guided Prompt Evolution for Efficient Dense Vision-Language Inference
May 12, 2026Hao Zhu, Shuo Jin, Wenbin Liao +4Image Segmentation Vision-Language Models
Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, China · XJTLU +1
OTT-Vid: Optimal Transport Temporal Token Compression for Video Large Language Models
May 12, 2026Minseok Kang, Minhyeok Lee, Jungho Lee +6Efficient VLM Inference Video-Language Models
Yonsei University · LG Electronics · KIST

May 26, 2026, cs.CVJ/K move · Enter open · S save

LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding

Shihao Wang, Shilong Liu, Yuanguo Kuang, Xinyu Wei+9

The Hong Kong Polytechnic University · NVIDIA · Princeton University+2

Efficient VLM Inference Vision-Language Grounding Parallel Decoding Object Detection Visual Grounding

Open paper AskPDF arXiv

Vision-language models (VLMs) commonly formulate visual grounding and detection as a coordinate-token generation problem, serializing each 2D box into multiple 1D tokens that are learned and decoded largely independently. This token-by-token decoding mismatches the coupled structure of box geometry and creates a practical inference bottleneck due to strictly sequential generation. We introduce LocateAnything, a unified generative grounding and detection framework based on Parallel Box Decoding (PBD). By decoding geometric elements such as bounding boxes and points as atomic units in a single step, LocateAnything preserves intra-box geometric coherence and unlocks substantial parallelism. We show that PBD improves both decoding throughput and localization accuracy. We further develop a scalable data engine and curate LocateAnything-Data, a large-scale dataset with more than 138 million training samples, substantially increasing data diversity for high-precision localization. Extensive evaluations show that LocateAnything advances the speed-accuracy frontier, achieving significantly higher decoding throughput while improving high-IoU localization quality across diverse benchmarks. The results highlight the complementary benefits of Parallel Box Decoding and large-scale training data in enabling efficient and precise unified visual grounding and detection.

Momentum

Related topics

Latest papers 322