Efficient VLM Inference

VLM: Vision-Language Model

Overview

The field studies how to make vision-language models answer faster and use less memory and energy, especially for high-resolution images, long videos, and edge devices. Common approaches prune or merge visual tokens and frames, adapt resolution or retrieve regions on demand, reuse or compress KV caches, and use quantization, efficient kernels, routing, or speculative decoding. Methods are evaluated on tasks such as VQAv2, TextVQA, OCRBench, VideoMME, and LongVideoBench, measuring accuracy alongside time-to-first-token, throughput, and memory use. The central tension is preserving fine-grained evidence and reliable grounding while reducing computation: fewer tokens do not always mean lower end-to-end latency, and generating long answers can dominate cost on some devices.

Latest papers

CardsList

530 papers38 papers in the last four weeks, down 47% on the four weeks before. 0.3% of all new papers.

  1. ApET: Approximation-Error Guided Token Compression for Efficient VLMs

    Feb 23, 2026Qiankun Ma, Ziyao Zhang, Haofei Wang +3Visual Token CompressionEfficient VLM Inference

  2. QuickGrasp: Responsive Video-Language Querying Service via Accelerated Tokenization and Edge-Augmented Inference

    Feb 23, 2026Miao Zhang, Ruixiao Zhang, Jianxin Shi +3Efficient VLM InferenceVLM Inference

  3. Seeing Farther and Smarter: Value-Guided Multi-Path Reflection for VLM Policy Optimization

    Feb 22, 2026Yanting Yang, Shenyuan Gao, Qingwen Bu +2VLMs for RoboticsLarge Language Model-Based Robot Planning

  4. OTPrune: Distribution-Aligned Visual Token Pruning via Optimal Transport

    Feb 22, 2026Xiwen Chen, Wenhui Zhu, Gen Li +9Visual Token PruningEfficient VLM Inference

  5. DUET-VLM: Dual stage Unified Efficient Token reduction for VLM Training and Inference

    Feb 21, 2026Aditya Kumar Singh, Hitesh Kandala, Pratik Prabhanjan Brahma +2Efficient VLM InferenceMultimodal Token Compression

  6. HSD: Training-Free Acceleration for Document Parsing Vision-Language Models with Hierarchical Speculative Decoding

    Feb 13, 2026Wenhui Liao, Hongliang Li, Pengyu Xie +15Efficient VLM InferenceSpeculative Decoding

  7. Think Proprioceptively: State-Grounded Visual Token Selection for VLA Policies

    Feb 6, 2026Fangyuan Wang, Peng Zhou, Jiaming Qi +4Efficient VLM InferenceVisuomotor Policy Learning

  8. SPARC: Separating Perception And Reasoning Circuits for Test-time Scaling of VLMs

    Feb 6, 2026Niccolo Avogaro, Nayanika Debnath, Li Mi +6Visual Question AnsweringEfficient VLM Inference

  9. Gated Relational Alignment via Confidence-based Distillation for Efficient VLMs

    Jan 30, 2026Yanlong Chen, Amirhossein Habibian, Luca Benini +1Cross-Modal Knowledge DistillationVision-Language Models

  10. Efficient-VLN: A Simple yet Strong Baseline for Efficient Vision-Language Navigation

    Dec 11, 2025Duo Zheng, Shijia Huang, Yanyang Li +1KV CachingEfficient VLM Inference

  11. MMG-Vid: Maximizing Marginal Gains at Segment-level and Token-level for Efficient Video LLMs

    Aug 28, 2025Junpeng Ma, Qizhe Zhang, Ming Lu +4Efficient VLM InferenceVideo Token Compression

  12. AdaptInfer: Adaptive Token Pruning for Vision-Language Model Inference via Dynamical Text Guidance

    Aug 8, 2025Weichen Zhang, Zhui Zhu, Ningbo Li +5Efficient VLM InferenceVisual Token Pruning

  13. SalQ-VLM: Fine-Grained Saliency-Guided Quantization for Vision-Language Models

    Aug 5, 2025Yufei Xue, Yushi Huang, Lunjie Zhu +2Vision-Language ModelsVLM Quantization

  14. StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modeling

    Jul 7, 2025Meng Wei, Chenyang Wan, Xiqian Yu +9Efficient VLM InferenceEfficient VLAs

  15. ToDRE: Effective Visual Token Pruning via Token Diversity and Task Relevance

    May 24, 2025Duo Li, Zuhao Yang, Xiaoqin Zhang +2Efficient VLM InferenceLarge Vision-Language Models

  16. LaViDa: A Large Diffusion Language Model for Multimodal Understanding

    May 22, 2025Shufan Li, Konstantinos Kallidromitis, Hritik Bansal +7Multimodal UnderstandingVision-Language Models