Efficient VLM Inference

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

38 papers in the last four weeks, up 100% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 322

All topics

Query-based Cross-Modal Projector Bolstering Mamba Multimodal LLM
Jun 3, 2026SooHwan Eom, Jay Shim, Gwanhyeong Koo +4Mamba Efficient VLM Inference
Korea Advanced Institute of Science and Technology / Korea, Republic of · University of Illinois in Urbana-Champaign / United States of America · Korea University / Korea, Republic of
Frames2LoRA: Parametric Video Internalization for Vision-Language Models
Jun 3, 2026Manan Suri, Sarvesh Baskar, Dinesh ManochaVLM Adaptation Efficient VLM Inference
University of Maryland, College Park
When Attention Collapses: Stage-Aware Visual Token Pruning from Structure to Semantics
Jun 2, 2026Jiahui Wang, Kai Zhang, Mai Han +1Vision-Language Models Efficient VLM Inference
1Shandong University · 2National University of Singapore (Suzhou) Research Institute
TGV-KV: Text-Grounded KV Eviction for Vision-Language Models
Jun 2, 2026Jizhihui Liu, Ruizi Han, Miao Zhang +4Vision-Language Models Efficient VLM Inference
Harbin Institute of Technology, Shenzhen · Peng Cheng Laboratory
ToolGate: Token-Efficient Pre-Call Control for Tool-Augmented Vision-Language Agents
Jun 2, 2026Anjie Liu, Yan Song, Zhixun Chen +3Tool Use in VLMs Tool-Augmented Language Model Agents
The Hong Kong University of Science and Technology (Guangzhou) · University College London · AI Lab, The Yangtze River Delta
SCOPE: Real-Time Natural Language Camera Agent at the Edge
Jun 1, 2026Nikolaj Hindsbo, Sina Ehsani, Pragyana MishraEfficient VLM Inference AI Control
Armada AI Bellevue, Washington, USA
AdaCodec: A Predictive Visual Code for Video MLLMs
Jun 1, 2026Haowen Hou, Zhen Huang, Zheming Liang +8Predictive Coding Efficient VLM Inference
Shanghai Jiao Tong University · Shanghai Innovation Institute
InfoMerge: Information-aware Token Compression for Efficient Video Large Language Models
Jun 1, 2026Xinxin Liu, Shiwei Gan, Xiao Liu +3Efficient VLM Inference Video-Language Models
State Key Laboratory of Novel Software Technology, Nanjing University Nanjing 210023, China
MOSS-Video-Preview: Toward Real-Time Video Understanding via Cross-Attention
Jun 1, 2026Pengyu Wang, Chenkun Tan, Shaojun Zhou +18Streaming Video Understanding Efficient VLM Inference
Fudan University · Shanghai Innovation Institute
EvoCut: Multi-Layer Evolution-Aware Visual Token Compression for Efficient Large Vision-Language Models
Jun 1, 2026Hongyu Lu, Feng Zhang, Wenwei Jin +5Efficient VLM Inference Large Vision-Language Models
Harbin Institute of Technology · Xiaohongshu · Fudan University
FlowNar: Scalable Streaming Narration for Long-Form Videos
May 30, 2026Zeyun Zhong, Manuel Martin, Chengzhi Wu +4Streaming Video Understanding Efficient VLM Inference
Karlsruhe Institute of Technology (KIT) · Fraunhofer IOSB · Lamarr Institute for Machine Learning and Artificial Intelligence +1
LASER: Loss-Aware Singular-value Decomposition and Rank Allocation for Efficient Low-Precision Vision-Language Models
May 30, 2026Haiyu Wang, Yutong Wang, Leshu Li +2Vision-Language Models Efficient VLM Inference
Tandon School of Engineering, New York University · Courant Institute of Mathematical Sciences, New York University · Brookhaven National Laboratory
ETC: Extreme Token Compression via Task-aware Visual Information Distillation in VLMs
May 30, 2026Yiling Gao, Hongchen Wei, Zhenzhong ChenEfficient VLM Inference Large Vision-Language Models
School of Remote Sensing and Information Engineering, Wuhan University
DREAM-S: Speculative Decoding with Searchable Drafting and Target-Aware Refinement for Multimodal Generation
May 30, 2026Zining Liu, Yunhai Hu, Tianhua Xia +4Efficient VLM Inference Speculative Decoding
New York University · Cerebras Systems Inc.
Zamba2-VL Technical Report
May 29, 2026Hassan Shapourian, Kasra Hejazi, Olabode M. Sule +1Vision-Language Models Efficient VLM Inference
Zyphra, San Francisco, CA
Linear Scaling Video VLMs for Long Video Understanding
May 29, 2026Cristobal Eyzaguirre, Jiajun Wu, Juan Carlos NieblesStreaming Video Understanding Efficient VLM Inference
Stanford University
YARD: Y-Architecture Register Decoding for Efficient Hallucination Mitigation in Large Vision-Language Models
May 29, 2026Ting Chen, Geng Li, Guohao Chen +5Efficient VLM Inference Object Hallucination in VLMs
Guangdong University of Technology · Nanyang Technological University · Shenzhen TENCLASS Technology Co., Ltd.
PEEK: Picking Essential frames via Efficient Knowledge distillation
May 29, 2026Killian Steunou, Anas Filali Razzouki, Khalil Guetari +2Efficient VLM Inference Video Captioning
Télécom SudParis — SAMOVAR Institut Polytechnique de Paris Palaiseau, France · Moments Lab 69 Avenue Pierre Grenier Boulogne-Billancourt, France
Simple Token-Efficient Vision-Language Model for Case-level Pathology Synoptic Report Generation
May 29, 2026Zhiyuan Yang, Jiahao Cheng, Vincent Quoc-Huy Trinh +1Computational Pathology Medical Report Generation
Department of Computer Science and Software Engineering (CSSE), Concordia University, Montreal, Canada · Axe Cancer, Centre de recherche du CHUM, Université de Montréal, Montreal, Canada · Institut de recherche en immunologie et cancérologie (IRIC), Université de Montréal
PARCEL: Pool-Anchored Resampling with Conditioned Elastic Queries for Efficient Vision-Language Understanding
May 28, 2026Selim Kuzucu, Alessio Tonioni, Vasile Lup +3Image Tokenization Efficient VLM Inference
Max Planck Institute for Informatics, SIC · Google · Technical University of Munich
VisualThink-VLA: Visual Intermediate Reasoning for Effective and Low-Latency Vision-Language-Action Policies
May 28, 2026Mingjian Gao, Wenqiao Zhang, Yuqian Yuan +9Efficient VLM Inference Efficient VLA Model Inference
1Zhejiang University · 2Cornell University · 3National University of Singapore +1
EarlyTom: Early Token Compression Completes Fast Video Understanding
May 28, 2026Hesong Wang, Xin Jin, Lu Lu +4Video Understanding Efficient VLM Inference
Zhejiang University · Westlake University · Alibaba Cloud Computing
SAFE-Pruner: Semantic Attention-Guided Future-Aware Token Pruning for Efficient Vision-Language-Action Manipulation
May 28, 2026Shilin Ma, Chubin Zhang, Changyuan Wang +6Efficient VLM Inference Robotic Manipulation
Tsinghua Shenzhen International Graduate School, Tsinghua University · GigaAI
OccamToken: Efficient VLM Inference with Training-Free and Budget-Adaptive Token Pruning
May 28, 2026Geng Li, Guohao Chen, Ting Chen +6Efficient VLM Inference Large Vision-Language Models
1Nanyang Technological University (NTU)
AsymVLM: Asymmetric Token Pruning for Efficient Vision-Language Model Inference
May 28, 2026Yilin Feng, Ahmed Burak Gulhan, Mahmut Taylan KandemirLLM Pruning Vision-Language Models
The Pennsylvania State University
ElegantVLA: Learning When to Think for Efficient Vision-Language-Action Models
May 28, 2026Ye Li, Huanan Liu, Kangye Ji +7Efficient VLM Inference Efficient VLA Model Inference
Tsinghua University · University of Illinois at Urbana-Champaign
Pocket-Dentist: On-Device Dental Image Understanding via Efficient Multimodal Large Language Models
May 28, 2026Kai Bian, Xucheng Guo, Bin Chen +4Medical Imaging Efficient VLM Inference
The University of Auckland, New Zealand · Shandong University, China · The University of Melbourne, Australia
CIVIC: End-to-End Sequence Compactness for Efficient Vision-Language Models
May 27, 2026Fengze Yang, Bo Yu, Xuewen Luo +2Efficient VLM Inference KV-Cache Compression
Department of Civil & Environmental Engineering University of Utah 201 Presidents’ Cir, Salt Lake City, UT 84112, USA
Beyond Surrogate Gradients: Fully Differentiable Token Pruning for Vision-Language Models
May 27, 2026Landi He, Mingde Yao, Shawn Young +1Vision-Language Models Efficient VLM Inference
Shenzhen University of Advanced Technology · CUHK MMLab, CPII under InnoHK
Bounded-Compute Multimodal Regression for Product-Rating Prediction
May 26, 2026William Leach, Ru He, Sizhuo Ma +4Vision-Language Models VLM Adaptation
Snap Inc.

Jun 3, 2026, cs.CLJ/K move · Enter open · S save

Query-based Cross-Modal Projector Bolstering Mamba Multimodal LLM

SooHwan Eom, Jay Shim, Gwanhyeong Koo, Haebin Na+3

Korea Advanced Institute of Science and Technology / Korea, Republic of · University of Illinois in Urbana-Champaign / United States of America · Korea University / Korea, Republic of

Mamba Efficient VLM Inference Multimodal Large Language Models Cross-Modal Attention Visual Token Compression

Open paper AskPDF arXiv

The Transformer's quadratic complexity with input length imposes an unsustainable computational load on large language models (LLMs). In contrast, the Selective Scan Structured State-Space Model, or Mamba, addresses this computational challenge effectively. This paper explores a query-based cross-modal projector designed to bolster Mamba's efficiency for vision-language modeling by compressing visual tokens based on input through the cross-attention mechanism. This innovative projector also removes the need for manually designing the 2D scan order of original image features when converting them into an input sequence for Mamba LLM. Experimental results across various vision-language understanding benchmarks show that the proposed cross-modal projector enhances Mamba-based multimodal LLMs, boosting both performance and throughput.

Momentum

Related topics

Latest papers 322