Efficient VLM Inference

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

38 papers in the last four weeks, up 100% on the four weeks before. 0.4% of all new papers.

Jul 13Week of Sep 28

Latest papers 322

All topics

VespaSeg: A Resource-Aware Ground-then-Segment Pipeline for Referring Expression Segmentation
Aug 2, 2026Savindu Dilshan WickramasinghePromptable Image Segmentation Efficient VLM Inference
University of Moratuwa · Department of Electronic and Telecommunication Engineering, University of Moratuwa Moratuwa, Sri Lanka
MonitorVLM-v2: A Deployed Vision-Language Framework for Real-Time Safety Violation Detection
Aug 2, 2026Jiang Wu, Sichao Wu, Yinsong Ma +2Vision-Language Models Efficient VLM Inference
School of Mechanical Engineering, University of Science and Technology Beijing · The Laboratory for Computational Sensing and Robotics, Johns Hopkins University
FreqNav: Stage-Wise Frequency Routing for Object-Oriented Aerial Vision-Language Navigation
Aug 2, 2026Yin Tang, Jiawei Ma, Jiahao Li +4Efficient VLM Inference UAV Navigation
Big Data Institute, Central South University · Department of Computer Science, City University of Hong Kong · School of Computer Science and Engineering, Central South University +1
Coverage-Driven Adaptive Keyframe Selection for Video Understanding
Aug 1, 2026Junyang Zhang, Puhan Luo, Chen Tang +2Efficient VLM Inference Large Vision-Language Models
University of Science and Technology of China, Hefei, China
Latent-Centroid Steering: Single-Pass Classifier-Free Guidance for Command-Aligned Autonomous Driving
Jul 31, 2026Meibo Hu, Jiamian Wang, Pichao Wang +1Classifier-Free Guidance VLMs for Autonomous Driving
Rochester Institute of Technology · NVIDIA
LAST: The Last Query Token Guides Visual Token Pruning for Edge-Cloud Collaborative MLLM Inference
Jul 30, 2026Feng Yang, Xinrui Ju, Keyang Zhang +6Efficient VLM Inference Visual Token Pruning
1City University of Hong Kong · 2Peng Cheng Laboratory
Thinking Once Is Enough: Intermediate-Layer Evidence Routing for High-Resolution VQA
Jul 30, 2026Zhongkuan Mao, Xianjie Liu, Tianyu Meng +9Visual Question Answering Efficient VLM Inference
National Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University
Calibrate Before Reason: Robust Visual Token Reduction against Semantic Drift in VLMs
Jul 30, 2026Jiasheng Li, Zhong Ji, Yan Zhang +1VLM Robustness Efficient VLM Inference
Tianjin University, Tianjin, China · Tsinghua University, Beijing, China
Beyond Accuracy: Auditing Spatial Provenance in Visual Token Pruning for OCR-Critical MLLM Inference
Jul 29, 2026Feixiang Liu, Qiang Qiu, Hao Zhang +1VLM Evaluation Efficient VLM Inference
State Key Laboratory of AI Safety, Institute of Computing Technology, CAS · University of Chinese Academy of Sciences
MedARC: Training-Free Adaptive Redundancy Compression of Visual Tokens for 3D Medical Vision-Language Models
Jul 29, 2026Yitao Zhu, Mengjun Liu, Yingji Fu +2Efficient VLM Inference Medical VLMs
Department of Health Technology and Informatics, The Hong Kong Polytechnic University, Hong Kong · School of Integrated Circuits and Electronics, Beijing Institute of Technology, China · 3Mental Health Research Center, The Hong Kong Polytechnic University, Hong Kong +1
Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Model
Jul 27, 2026Senqiao Yang, Kaichen Zhang, Zhaoyang Jia +20Vision-Language Models Efficient VLM Inference
Microsoft Mage Team
CADER: Confidence-Aware Dynamic Evidence Reasoning for Long-Video Understanding
Jul 27, 2026Jinlong Yang, Wenhao Zhang, Kuanwei Lin +1Tool Use in VLMs Efficient VLM Inference
School of Computer Science, Northwestern Polytechnical University · School of Electronic and Computer Engineering, Peking University · Department of Computer Science and Technology, Tsinghua University
MAViE: A Multi-scale Adaptive Vision Encoder for Fine-grained Visual Perception and Efficient Multimodal Reasoning
Jul 27, 2026Shaofei LeiEfficient VLM Inference Multi-Scale Feature Fusion
GOTS: Greedy Orthogonal Token Selection for High-Resolution Vision-Language Models
Jul 27, 2026Jun Ling, Tao Huang, Junzhuo Liu +2Efficient VLM Inference Large Vision-Language Models
School of Computer Science and Engineering, University of Electronic Science and Technology of China
PathSelect: Sequential Token Selection for Whole Slide Pathology
Jul 26, 2026Jingzhi Chen, Landi He, Zehong Chen +2Efficient VLM Inference Medical VLMs
Shenzhen University of Advanced Technology
MemVLN: Episodic and Procedural Memory for Vision-and-Language Navigation
Jul 26, 2026Yuqi Liu, Shengju Qian, Tianyuan Qu +5Memory-Augmented VLMs Efficient VLM Inference
1The Chinese University of Hong Kong · 2LIGHTSPEED · 3The University of Hong Kong +1
UltraViT: Latency-Optimized On-device Vision Encoder for Large Vision-Language Models
Jul 25, 2026Ioannis Maniadis Metaxas, Adrian Bulat, Alberto Baldrati +4Vision-Language Models Efficient VLM Inference
Samsung AI Cambridge · Technical University of Iasi · Queen Mary University of London
WaveZip: Wavelet-Driven Space-Time Decoupling for Video Token Condensation
Jul 25, 2026Yuhui Zeng, Wang Chen, Jinfa Huang +5Efficient VLM Inference Video Token Compression
Media Analytics and Computing Lab, Xiamen University, Xiamen, China · Department of Computer Science, University of Rochester, Rochester, NY, USA
Structured Redundancy Modeling for Efficient Visual Token Pruning in High-Resolution MLLMs
Jul 25, 2026Jouwon Song, Woohyeong Kim, Kyeongbo KongEfficient VLM Inference Large Vision-Language Models
LG Electronics, Seoul, Republic of Korea · Pusan National University, Busan, Republic of Korea
LayoutLite: Token-Level Implicit Layout Analysis for Efficient Document OCR
Jul 24, 2026Xudong Liu, Bicheng Wan, Yulin JinEfficient VLM Inference Document Layout Analysis
1Yuanli Technology (Beijing) Co., Ltd. · 2Beijing Normal University
When Are Reasoning-Based Guardrails Not Efficient? ResponseGuard: A Fast Vision-Language Guard for Real-Time Moderation
Jul 23, 2026Dongbin NaVLM Evaluation Efficient VLM Inference
POSTECH
Look Less, Think Faster: Joint Token-Compute Adaptation for Multimodal LLMs
Jul 22, 2026Pengcheng Wang, Zhiquan Wang, Jayoung Lee +5Efficient Multimodal Inference Efficient VLM Inference
Purdue University, West Lafayette, IN, USA · University of Wisconsin–Madison, Madison, WI, USA · NVIDIA, Santa Clara, CA, USA
SOPD-SocialNav: Selective On-Policy Distillation for Vision-Language Social Navigation
Jul 22, 2026Xinyu Zhang, Zishuo Wang, Ling XiaoEfficient VLM Inference VLM Distillation
Graduate School of Information Science and Technology, Hokkaido University, Sapporo, Japan
BLUE: Semantics-Preserving Video Compression for Efficient Vision-Language Surveillance Analytics
Jul 21, 2026Shubham Baid, Akash James, Sahil Chachra +2Efficient VLM Inference Video Compression
KGraph AI Solutions Pvt. Ltd., Bangalore, India – 560016
HPD-Parsing: Hierarchical Parallel Document Parsing
Jul 21, 2026Shu Wei, Jingjing Wu, Lingshu Zhang +10Efficient VLM Inference Document Parsing
Bounding Boxes to Improve Small Language Model Performance on Vision-Based Grading Tasks
Jul 21, 2026Lachlan McGinnessVLM Evaluation Educational Assessment
Australian National University, Acton, ACT, Australia
StepX-Edge: An On-Device UI Vision-Language Model via Architecture-Training-Deployment Co-Design
Jul 20, 2026Yin Wang, Haotian Hu, Jineng Han +4Vision-Language Models VLM Quantization
StepX Team
Searching for Task-Specific Vision Paths: Evolutionary Block Pruning Across Vision-Language Models
Jul 19, 2026Tarun TomarEfficient VLM Inference Neural Network Pruning
Efficient Frame Selection for Long Videos at Test Time with Attention-Based MLLM Selectors
Jul 17, 2026Yilin Wang, Xiangxi Zheng, Dongxing Mao +6Efficient VLM Inference Multimodal Large Language Models
ZJU · NJU · CSU +2
VideoChat3: Fully Open Video MLLM for Efficient and Generalist Video Understanding
Jul 16, 2026Xinhao Li, Yuhan Zhu, Xiangyu Zeng +24Video Understanding Streaming Video Understanding
Nanjing University · Nanyang Technological University · Shanghai AI Laboratory +1

Aug 2, 2026, cs.ROJ/K move · Enter open · S save

VespaSeg: A Resource-Aware Ground-then-Segment Pipeline for Referring Expression Segmentation

Savindu Dilshan Wickramasinghe

University of Moratuwa · Department of Electronic and Telecommunication Engineering, University of Moratuwa Moratuwa, Sri Lanka

Promptable Image Segmentation Efficient VLM Inference Referring Expression Segmentation Referring Image Segmentation Vision-Language Grounding

Open paper AskPDF arXiv

Referring expression segmentation requires language conditioned localization and pixel-accurate masks, but monolithic models can be costly to deploy. We present VespaSeg, a modular pipeline that grounds a text query with a compact vision-language model and converts the predicted box to a mask with MobileSAM. We study Florence-2-base, Florence-2-large, and Moondream2 grounders together with targeted adaptation of the grounding and segmentation stages. Under a repository-specific RefCOCO validation protocol containing the first expression for each of 3,811 referenced-object records, the adapted Florence-2-base pipeline obtains 73.64 mean intersection over union (mIoU) and 84.60 precision at IoU 0.5. On an NVIDIA RTX 6000 Ada GPU it processes 22.8 cached-image queries per second with 2.20 GB mean allocated GPU memory. A matched 500-query comparison gives 73.73 mIoU for Florence-2-base and 72.82 for Florence-2-large, while the base model is 1.70 times faster and uses 1.17 GB less allocated memory. Ablations show that ground-truth-box adaptation raises MobileSAM mIoU from 82.22 to 86.61 and that reducing the Florence-2 output-token budget from 64 to 32 preserves accuracy. These results support compact, modular grounding and segmentation, while also exposing the need for evaluation on the complete standard RefCOCO expression splits and deployment hardware.

Momentum

Related topics

Latest papers 322