Efficient VLM Inference

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

38 papers in the last four weeks, up 100% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 322

All topics

A Modern Multimodal Assistant on a 6 GB 2011 GPU: Stage-Validated, All-GPU CUDA Inference for Fermi
Jul 16, 2026A. C. Opus, J. Q. LuEfficient Transformer Inference GPU Kernel Optimization
Department of Physics, University of Puerto Rico, Mayaguez, PR 00680, USA
Attention-Free and Lightweight Token Reduction for Efficient Vision-Language Models
Jul 15, 2026Xuanyi Hao, Zuoyuan Zhang, Zhibo Wang +4Vision-Language Models Efficient VLM Inference
State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou 310027, China · Hong Kong University of Science and Technology, Hong Kong SAR, China · School of Artificial Intelligence, Nanchang University, Nanchang 330038, China
VisCo: Leveraging Large Language Models as Intrinsic Encoders for Visual Token Compression
Jul 14, 2026Yupeng Zheng, Kai Zou, Bin Liu +1Efficient VLM Inference Large Vision-Language Models
Anhui Province Key Laboratory of Digital Security, University of Science and Technology of China Hefei, China
TIGER: Text-Conditioned Visual Gated Routing with Acceptance Alignment for Multimodal Speculative Decoding
Jul 13, 2026Quynh Vo, Cong-Duy Nguyen, Ponhvoan Srey +2Efficient VLM Inference Speculative Decoding
National University of Singapore · Nanyang Technological University · Center of AI Research, VinUniversity
Spectral Heat Flow for Conservative Token Condensation in Vision-Language Models
Jul 12, 2026Zhaoyang Li, Yanjun Li, Wangkai Li +2Efficient VLM Inference Visual Token Pruning
School of Information Science and Technology / National Key Laboratory of Deep Space Exploration, University of Science and Technology of China.
Seeing is Free, Speaking is Not: Uncovering the True Energy Bottleneck in Edge VLM Inference
Jul 10, 2026Junfei Zhan, Haoxun Shen, Mingang Guo +2Efficient VLM Inference Edge Inference
Department of Electrical and Systems Engineering University of Pennsylvania Philadelphia, PA, USA · Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences Shenzhen, China · College of Information Science and Technology Jinan University Guangzhou, China
SigLIP-HD by Fine-to-Coarse Supervision
Jul 10, 2026Lihe Yang, Zhen Zhao, Hengshuang ZhaoEfficient VLM Inference Large Vision-Language Models
The University of Hong Kong · Shanghai AI Laboratory
GeoTrace: Geometry-Aware Trajectory Token Compression for Video Large Language Models
Jul 10, 2026Guohuan Xie, Mengqi Lei, Chuan Shi +3Efficient VLM Inference Video-Language Models
BNRist, THUIBCS, BLBCI, School of Software, Tsinghua University · Yangtze Delta Region Institute, Tsinghua University · Beijing University of Posts and Telecommunications, Beijing, China
MOSAIC: Adaptive Inter-layer Composition for Efficient Heterogeneous Vision-Language Models
Jul 10, 2026Yuncheng Yang, Feiyang Ye, Shixian Luo +7Vision-Language Models Efficient VLM Inference
LiAuto Inc.
Rethinking Small VLM Quantization: From Component-Wise Analysis to Hardware-Aware Edge Deployment
Jul 9, 2026Hyeju Shin, Chorwon Kim, Ryangsoo Kim +2Mixed-Precision Quantization Vision-Language Models
1ETRI, Republic of Korea.
AVA-VLM: Adaptive Visual Attention-Vision Language Model for In-the-Wild Construction Site Monitoring
Jul 7, 2026Younggun Kim, Taeheon Kim, Youngseo Kim +1VLM Robustness Efficient VLM Inference
Department of Civil and Environmental Engineering, University of California, Los Angeles, 405 Hilgard Avenue, Los Angeles, 90095, CA, USA · AI+KCIR Global Resilience Research Center, Suwon, 16419, Gyeonggi-do, South Korea · SmartInside AI Co., Ltd., Suwon, 16419, Gyeonggi-do, South Korea +1
HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better
Jul 6, 2026Gengluo Li, Xingyu Wan, Shangpin Peng +20Vision-Language Models Efficient VLM Inference
Institute of Information Engineering, Chinese Academy of Sciences · Large Language Model Department, Tencent · Nankai University
TORINO: Token Reduction via Interpretable Concept Overlap in Vision-Language Models
Jul 6, 2026Riccardo Renzulli, Gabriele Spadaro, Shruthi Gowda +2Efficient VLM Inference Visual Token Pruning
University of Turin, Italy · Eindhoven University of Technology, The Netherlands · LTCI, T´el´ecom Paris, Institut Polytechnique de Paris, France
CritiqueDriveVLM: From Verifier-Guided Reinforcement Learning to Latent Thought Distillation for Autonomous Driving
Jul 5, 2026Zhaohong Liu, Hao Ye, Xianlin Zhang +1VLMs for Autonomous Driving Efficient VLM Inference
State Key Laboratory of Networking and Switching Technology · Beijing University of Posts and Telecommunications, Beijing, China · School of Digital Media & Design Arts
Seeing Once is Enough? Online Geometry-Aware Token Pruning for 3D Question Answering
Jul 5, 2026Ruei-Chi Lai, Bolivar Solarte, Chin-Hsuan Wu +2Efficient VLM Inference Multimodal QA
National Tsing Hua University · Industrial Technology Research Institute ITRI · University of Toronto +1
RADIO1D: Elastic Representations for Condensed Vision Modeling
Jul 3, 2026Greg Heinrich, Mike Ranzinger, Collin McCarthy +6Vision-Language Models Efficient VLM Inference
1NVIDIA
Combating Textual Noise and Redundancy: Entropy-Aware Dense Visual Token Pruning
Jul 2, 2026Xuehui Wang, Xuankun Yang, Wei ShenEfficient VLM Inference Multimodal Token Compression
Shanghai Jiao Tong University, Shanghai, China
ReQuest: Rethinking-based Question-Aware Frame Selection for Long-Form Video QA
Jul 2, 2026Minkuk Kim, Suyong Yun, Young Tae Kim +3Efficient VLM Inference Video QA
Kyung Hee University, Republic of Korea · Electronics and Telecommunications Research Institute (ETRI), Republic of Korea
Segmenting, Fast and Slow: Real-Time Open-Vocabulary Video Instance Segmentation with Dual-Path Processing
Jun 30, 2026Luca Barsellotti, Martin Sundermeyer, Mattia Segu +5Video Object Segmentation Instance Segmentation
⋆Work done during an internship at Google. · Google · TU Munich; Munich Center for Machine Learning +1
ERA: Entropy-Guided Visual Token Pruning with Rectified Attention for Efficient MLLMs
Jun 30, 2026Yuhao Wang, Mu Qiao, Haiwen Diao +5Efficient VLM Inference Multimodal Large Language Models
School of Future Technology, Dalian University of Technology, Dalian 116024, China · S-Lab, Nanyang Technological University, Singapore 639798 · OPPO Research Institute, Shenzhen 518000, China +1
Attend, Transform, or Silence: Operator-Level Visual Skipping for Efficient Multimodal LLM Inference
Jun 30, 2026Zhaoyang Luo, Runmin Dong, Miao Yang +4Efficient VLM Inference Multimodal Large Language Models
Tsinghua Shenzhen International Graduate School, Shenzhen, China · Sun Yat-sen University, Zhuhai, China · Tsinghua University, Beijing, China +1
Reasoning-aware Speculative Decoding for Efficient Vision-Language-Action Models in Autonomous Driving
Jun 30, 2026Anh Dung Dinh, Simon Khan, Flora SalimEfficient VLM Inference Speculative Decoding
School of Computer Science and Engineering UNSW Sydney · Air Force Research Laboratory
VisReflect: Latent Visual Reflection for Fine-Grained Perception in Long Visual Context
Jun 29, 2026Xiaoqian Shen, Mohamed ElhoseinyEfficient VLM Inference Attention Sinks
King Abdullah University of Science and Technology, Saudi Arabia
Fast Enough to Act: Spatio-Temporal Visual Token Merging for Low-Latency Robotic VLMs and VLAs
Jun 28, 2026Junzhou Chen, Jindong Wang, Gang ZhouEfficient VLM Inference Vision-Language-Action Models
Department of Computer Science, William & Mary, Williamsburg, VA 23188, USA · Department of Data Science, William & Mary, Williamsburg, VA 23188, USA
X-Mind: Efficient Visual Chain-of-Thought via Predictive World Model for End-to-End Driving
Jun 27, 2026Bohao Zhao, Chengrui Wei, Guangfeng Jiang +17World Model Learning End-to-End Autonomous Driving
PWM Team, XPeng Inc.
LocalNav: Distilling Frontier VLMs and Embodied RL for On-Device Object Goal Navigation
Jun 26, 2026Nicolas Baumann, Liam Boyle, Pu Deng +5Efficient VLM Inference VLM Distillation
Center for Project-Based Learning, 2Integrated Systems Laboratory, 3Computer Vision and Geometry ETH Zurich
MVPruner: Dynamic Token Pruning for Accelerating Multi-view Vision-Language Models in Autonomous Driving
Jun 26, 2026Nan Yang, Zhanwen Liu, Linfeng Zhang +4Vision-Language Models VLMs for Autonomous Driving
School of Information Engineering, Chang’an University, China · School of Artificial Intelligence, Shanghai Jiaotong University, China
TOPS: First-Principles Visual Token Pruning via Constructing Token Optimal Preservation Sets for Efficient MLLM Inference
Jun 25, 2026Tinghao Wang, Yichen Guo, Rui Huang +11Efficient VLM Inference Multimodal Large Language Models
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Nanyang Technological University · University of Electronic Science and Technology of China +1
GeMoE: Gating Entropy is All You Need for Uncertainty-aware Adaptive Routing in MoE-based Large Vision-Language Models
Jun 24, 2026Chaoxiang Cai, Minghe Weng, Jie Li +4Efficient VLM Inference Large Vision-Language Models
School of Software Technology, Zhejiang University · College of Computer Science and Technology, Zhejiang University
Brevity is the Soul of Inference Efficiency: Inducing Concision in VLMs via Data Curation
Jun 24, 2026DatologyAI, :, Matthew L. Leavitt +8Efficient VLM Inference Text Generation
DatologyAI Team1

Jul 16, 2026, cond-mat.otherJ/K move · Enter open · S save

A Modern Multimodal Assistant on a 6 GB 2011 GPU: Stage-Validated, All-GPU CUDA Inference for Fermi

A. C. Opus, J. Q. Lu

Department of Physics, University of Puerto Rico, Mayaguez, PR 00680, USA

Efficient Transformer Inference GPU Kernel Optimization Efficient VLM Inference Multimodal Foundation Models VLM Inference

Open paper AskPDF arXiv

A companion study ran a 35B mixture-of-experts model on a 2011 NVIDIA Tesla C2075 (Fermi, sm_20, 6GB) as a GPU-prefill/CPU-decode hybrid, because the 4-bit model did not fit in device memory (arXiv:2606.24031). This report keeps the hardware and asks what a model that fits can do: we deploy MiniCPM-V-4.6, a modern multimodal assistant pairing a SigLIP2 vision encoder and window-attention merger (16x visual token compression) with a compact hybrid gated-delta-net backbone, entirely on the GPU. Three results. (i) An all-GPU engine built on measured foundations: projections that dequantize 8-bit weights once and call the vendor SGEMM still in the last Fermi toolchain (64% of FP32 peak; our best hand-written GEMM hit 37%, wrongly called the ceiling); a chunked delta-rule rewrite of the recurrent layers, 2.8x faster than the sequential scan once attribution exposed one bad kernel; and a measured negative: 4-bit weights make decode slower than 8-bit here, since Fermi issues nibble-unpacking shifts at half rate. (ii) The vision side is a port with a proof obligation: we translate tower, merger, and projector to sm_20 CUDA, validating every stage against a locally generated reference forward (full tower 1.4e-5). One failure, position-embedding bucketization differing on exact rational ties, generalizes to a rule: float tie-breaking in index arithmetic is implementation-defined; call the reference operator, do not reimplement it. (iii) Long context exposes an O(N^2) wall short benchmarks hide: prefill falls from 114 tok/s at 2k tokens to 21 at 10k in a naive attention kernel; per-head vendor-GEMM calls writing into the existing score buffer (zero extra memory) restore a flat profile (408 at 2k, 361 at 10k; 17x), verified by exact needle retrieval from 60% depth. The same rewrite cuts image encoding 6x, to 0.93s. The system answers an image question end-to-end in 1.7s.

Momentum

Related topics

Latest papers 322