Efficient VLM Inference

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

38 papers in the last four weeks, up 100% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 322

All topics

Toward Low-Latency Vision-Language Models with Doubly-Correct Predictions in Egocentric Visual Understanding
Jun 23, 2026Qitong Wang, Fan Du, Pranav Maneriker +2Efficient VLM Inference Egocentric Vision
Dolby Laboratories, Inc., 1275 Market St, San Francisco, California, 94103, USA · Department of Computer and Information Sciences at the University of Delaware, Newark, DE 19711, USA · Dolby Laboratories, Inc.
video-SALMONN-R $^3$ : Learning to ReWatch, ReAsk, and ReAnswer for Efficient Video Understanding
Jun 23, 2026Yixuan Li, Guangzhi Sun, Yudong Yang +1Efficient VLM Inference Video QA
Tsinghua University · University of Cambridge
P-MTP: Efficient Document Parsing via Multi-Token Prediction with Progressive Depth Scaling
Jun 23, 2026Le Xiang, Chenxi Zhai, Shu Wei +5Multi-Token Prediction Efficient VLM Inference
Department of Computer Vision Technology (VIS), Baidu Inc., China · Tsinghua University, Shenzhen International Graduate School, China
Accelerating Multimodal Large Language Models with Prior-Corrected Token Reduction
Jun 23, 2026Zengjie Chen, Yuxiang Cai, Jingcai Guo +3Efficient VLM Inference Multimodal Large Language Models
School of Software Technology, Zhejiang University, Ningbo, China · Zhejiang Key Laboratory of Digital-Intelligence Service Technology, China · Hong Kong Polytechnic University, Hong Kong SAR +1
UniFS: Unified Fast-to-Slow Hierarchical Architecture for Vision-Language-Action Models
Jun 22, 2026Lin Sun, Zhiwei Guan, Conglin Wang +7Efficient VLM Inference Efficient VLA Model Inference
JD.com, China · Yiwu Research Institute of Fudan University, China · Tianjin University, China
Look Before You Zoom: Adaptive Routing for the Resolution-Context Trade-off in Visual RAG
Jun 20, 2026Oanh N. Tran, Thanh Quoc Hung Le, Oscar Chew +2Multimodal RAG Efficient VLM Inference
VinUni-Illinois Smart Health Center, VinUniversity · Texas A&M University
FleetAgent: Teleoperation Assistant for Autonomous Fleets via Vectorized V2N Messages
Jun 19, 2026Juntong Peng, Qi Chen, Deyuan Qu +3Vision-Language Models VLMs for Autonomous Driving
College of Engineering, Purdue University, West Lafayette, IN, USA · Toyota InfoTech Labs, Mountain View, CA, USA
TimeProVe: Propose, then Verify for Efficient Long Video Temporal Reasoning in Activities of Daily Living
Jun 18, 2026Arkaprava Sinha, Dominick Reilly, Siddharth Krishnan +2Temporal Video Grounding Efficient VLM Inference
University of North Carolina, Charlotte
ViCoStream: Streaming VideoLLMs Can Run Beyond 100 FPS with Stage-Wise Coordinated Inference
Jun 18, 2026Yang Tan, Junlong Tong, Linan Yue +3Streaming Video Understanding Efficient VLM Inference
Southeast University · Eastern Institute of Technology, Ningbo · Shanghai Jiao Tong University
SAFE-Cascade: Cost-Adaptive Vision-Language Routing for Chart Question Answering
Jun 17, 2026Ayush Dwivedi, Qixin Wang, Ashvi Soni +5Visual Question Answering Efficient VLM Inference
Walmart USA · University of Arkansas USA
CABLE: Cloud-Assisted Bandwidth-efficient LMM-based Encoding for V2X Systems
Jun 17, 2026Haohua Que, Zhipeng Bao, Qianyi Wu +1Efficient VLM Inference Edge Computing
College of Engineering, University of Georgia, Athens, GA 30602, USA
Invertible Neural Network Adapter for One-Step Flow Matching in Robot Manipulation
Jun 17, 2026Yu Zhang, Kangyi Ji, Yongxiang Zou +3Invertible Neural Networks Flow Matching
Moving Beyond Diversity: Visual Token Pruning as Subspace Reconstruction for Efficient VLMs
Jun 17, 2026Jaeyeon Lee, Shunjie Wen, Dong-Wan ChoiEfficient VLM Inference Feature Selection
Inha University
ActiveSAM: Fast and Accurate Open-Vocabulary Semantic Segmentation with Frozen SAM 3
Jun 15, 2026Tran Dinh Tien, Zhiqiang ShenEfficient VLM Inference Open-Vocabulary Semantic Segmentation
VILA Lab, Mohamed bin Zayed University of Artificial Intelligence
DCP-Prune: Ultra-Low Token Pruning with Distribution Consistency Preservation
Jun 15, 2026Xifeng Xue, Xiaokang Wang, Zirui Li +2Efficient VLM Inference Visual Token Pruning
College of Computer Science, Nankai University, Tianjin, China · Nanjing University of Posts and Telecommunications, Nanjing, China
Focus When Necessary: Adaptive Routing and Collaborative Grounding for Training-Free Visual Grounding
Jun 15, 2026Yifan Wang, Peiming Li, Shiyu Li +5Efficient VLM Inference Large Vision-Language Models
1Tencent BAC · School of Electronic and Computer Engineering, Peking University · School of Mathematics and Statistics, University of Glasgow +1
Acting While Understanding: Asynchronous Semantic-Action Decoupling for Real-Time Vision-Language-Action Models
Jun 13, 2026Shenhao Yan, Ge Wang, Qi Liu +7Efficient VLM Inference Efficient VLA Model Inference
Northeastern University · Ising AI · CUHK-Shenzhen
Elastic Queries Reinforcement Learning: Self-Aware Policy Execution for VLA Models
Jun 12, 2026Ge Wang, Xinyu Tan, Xiang Li +11RL for Robotics Efficient VLM Inference
Ising AI · CUHK-Shenzhen · PKU
One Layer's Trash is Another Layer's Treasure: Adaptive Layer-wise Visual Token Selection in LVLMs
Jun 12, 2026Yongru Chen, Kai Zhang, Zeliang Zong +4Efficient VLM Inference Large Vision-Language Models
Hikvision Research Institute · East China Normal University · Peking University
ReactVLA: Fast and Lightweight Reactive Robot Manipulation via Improved Mean Flow Action Generation
Jun 12, 2026Yanzhao Guo, Wenkai Chen, Jianwei ZhangLanguage-Conditioned Robot Manipulation Efficient VLM Inference
Shanghai Jiao Tong University · Technical Aspects of Multimodal Systems (TAMS), Department of Informatics, Universität Hamburg
RT-VLA: Real-Time Vision-Language-Action Models via Knowledge Distillation
Jun 12, 2026Xiangyu Huang, Zhenlin Hua, Han Zhou +2End-to-End Autonomous Driving Efficient VLM Inference
Carnegie Mellon University
Reroute, Don't Remove: Recoverable Visual Token Routing for Vision-Language Models
Jun 10, 2026Cheng-Yu Yang, Shao-Yuan Lo, Yu-Lun LiuEfficient VLM Inference Visual Token Pruning
National Yang Ming Chiao Tung University · National Taiwan University
From Content to Knowledge: Lightning Fast Long-Video Understanding with Neural Knowledge Representations
Jun 10, 2026Yuchen Guan, Xiao Li, Zongyu Guo +4Efficient VLM Inference Long-Video Understanding
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Microsoft Research Asia
AVIS: Adaptive Test-Time Scaling for Vision-Language Models
Jun 10, 2026Ahmadreza Jeddi, Minh Ngoc Le, Amirhossein Kazerouni +8Test-Time Adaptation of VLMs Efficient VLM Inference
AI Center-Toronto, Samsung Electronics · University of Toronto · Vector Institute +1
Thinking Without Images: Internalizing Visual Manipulation with On-Policy Self-Distillation
Jun 7, 2026Yishuo Cai, Jiahui Liu, Yuanxin Liu +9Tool Use in VLMs Efficient VLM Inference
1State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · 2Central South University · University of Science and Technology of China +2
Learnable Token Sparsification for Efficient Gigapixel Whole Slide Image Reasoning
Jun 7, 2026Jingzhi Chen, Landi He, Zhuo Chen +2Efficient VLM Inference Medical VLMs
Shenzhen University of Advanced Technology
vla.cpp: A Unified Inference Runtime for Vision-Language-Action Models
Jun 6, 2026Khanh D. Nguyen, Hung T. Ho, Chinh T. Nguyen +5AI Accelerator Inference Efficient VLM Inference
VinRobotics, Vietnam · Max Planck Research School for Intelligent Systems, Germany · University of Stuttgart, Germany +3
RhinoVLA Technical Report
Jun 5, 2026Huixi Technology, :, Chen Zhang +13Efficient VLM Inference Efficient VLA Model Inference
Huixi Technology
SS-TPT: Stability and Suitability-Guided Test-Time Prompt Tuning for Adversarially Robust Vision-Language Models
Jun 5, 2026Sunoh Kim, Daeho UmVLM Robustness VLM Adaptation
Dankook University, Yongin, South Korea · University of Seoul, Seoul, South Korea
MemoVAD: Resource-Efficient Video Anomaly Detection via Dynamic Semantic Memory in Edge Computing Scenarios
Jun 4, 2026Guo Li, Jiandian Zeng, Yang Li +3Streaming Video Understanding Efficient VLM Inference
Institute of Artificial Intelligence and Future Networks, Beijing Normal University, Zhuhai, China · School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu, China · Engineering Research Center of Cloud-Edge Intelligent Collaboration on Big Data, Ministry of Education, Beijing Normal University, Zhuhai, Guangdong, China

Jun 23, 2026, cs.ROJ/K move · Enter open · S save

Toward Low-Latency Vision-Language Models with Doubly-Correct Predictions in Egocentric Visual Understanding

Qitong Wang, Fan Du, Pranav Maneriker, Jihui Jin+1

Dolby Laboratories, Inc., 1275 Market St, San Francisco, California, 94103, USA · Department of Computer and Information Sciences at the University of Delaware, Newark, DE 19711, USA · Dolby Laboratories, Inc.

Efficient VLM Inference Egocentric Vision VLMs for Robotics Neural Network Pruning

Open paper AskPDF arXiv

The rapid rise of Vision-Language Models (VLMs) in egocentric visual understanding has made low-latency inference in human-robot collaborative (HRC) tasks increasingly critical. Weight pruning techniques developed for VLMs to shrink model size and computation can be readily applied to satisfy the efficiency demands of on-board processing and real-time interactive robotics. Moreover, safe human-robot interaction demands pruning strategies that preserve doubly-correct predictions; outputs must be both accurate and evidentially grounded to mitigate risks and ensure user trust. In this paper, we present a new study of VLM pruning through the lens of doubly-correct prediction. Our experiments surprisingly show that existing pruning methods often preserve the right evidence localization but undermine correct prediction. To address this, we propose a rationale-informed pruning strategy that better aligns evidence with decisions. Benchmark results on egocentric video datasets demonstrate that our method not only achieves the highest prediction accuracy but also outperforms existing approaches in attaining doubly-correct predictions. We aim to stimulate research on efficient and reliable VLMs, ensuring accuracy-driven advances align with the transparency, auditability, and safety required for responsible human-robot interaction and embodied intelligence.

Momentum

Related topics

Latest papers 322