VLM Inference

VLM: Vision-Language Model

Ask about this topic

Overview

Vision-language model inference is the process of using trained models to interpret images or video and generate answers, descriptions, or actions. Research improves reliability with training-free decoding and attention interventions that strengthen visual evidence, and improves perception through query-guided region selection and temporal frame retrieval. To reduce cost and delay, methods prune visual tokens, reuse caches, route queries across models, or coordinate inference across edge devices and cloud services. The central tension is preserving grounding and answer quality while reducing computation and latency, especially for long videos, high-resolution images, and real-time use.

Latest papers

CardsList

82 papers13 papers in the last four weeks, up 117% on the four weeks before. 0.1% of all new papers.

  1. VTAgent: Agentic Keyframe Anchoring for Evidence-Aware Video TextVQA

    May 6, 2026Haibin He, Maoyuan Ye, Jing Zhang +2Video QAVideo Frame Selection

    School of Computer Science, National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence, and Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University, Wuhan, Hubei, China

  2. Query-Dependent Use of Generated Descriptions for Reliable Visual Question Answering

    May 3, 2026Zeshang Li, Shuoyang ZhangVisual Question AnsweringHallucination in Language Models

    University of International Relations

  3. EdgeFM: Efficient Edge Inference for Vision-Language Models

    Apr 30, 2026Mengling Deng, Yuanpeng Chen, Sheng Yang +12Efficient VLM InferenceEdge Inference

    Go Further. AI · School of Data Science, Fudan University · RUYi Dynamics Co., Ltd +1

  4. Instruction-Evidence Contrastive Dual-Stream Decoding for Grounded Vision-Language Reasoning

    Apr 28, 2026Yashwant Pravinrao Bangde, Debaditya RoyVLM InferenceObject Hallucination in VLMs

    Department of Computer Science and Engineering Indian Institute of Technology Kharagpur Kharagpur, India 721302

  5. Global Context or Local Detail? Adaptive Visual Grounding for Hallucination Mitigation

    Apr 27, 2026Yubo Jiang, Xin Yang, Abudukelimu Wuerkaixi +7Hallucination in Language ModelsObject Hallucination in VLMs

    School of Astronautics, Beihang University, Beijing 102206, China · Longcat Interaction Team, Meituan, Beijing 100102, China · Tianmushan Laboratory, Beihang University, Hangzhou 311115, China

  6. Revisiting Greedy Decoding for Visual Question Answering: A Calibration Perspective

    Apr 25, 2026Boqi Chen, Xudong Liu, Yunke Ao +1Visual Question AnsweringLanguage Model Calibration

    1ETH Zurich · University of Toronto · 3MBZUAI

  7. Breaking the Illusion: When Positive Meets Negative in Multimodal Decoding

    Apr 22, 2026Yubo Jiang, Yitong An, Xin Yang +7Vision-Language ModelsHallucination in Language Models

    School of Astronautics, Beihang University, Beijing 102206, China · Longcat Interaction Team, Meituan, Beijing 100102, China · Tianmushan Laboratory, Beihang University, Hangzhou 311115, China

  8. Aligning What Vision-Language Models See and Perceive with Adaptive Information Flow

    Apr 17, 2026Chengxin Liu, Wonseok Choi, Chenshuang Zhang +1Vision-Language AlignmentVLM Inference

    KAIST · POSTECH

  9. Revisiting Compositionality in Dual-Encoder Vision-Language Models: The Role of Inference

    Apr 13, 2026Imanol Miranda, Ander Salaberria, Eneko Agirre +1Compositional Reasoning in VLMsVision-Language Models

    HiTZ Center – Ixa, University of the Basque Country (UPV/EHU)

  10. Text-Guided 6D Object Pose Rearrangement via Closed-Loop VLM Agents

    Apr 10, 2026Sangwon Baik, Gunhee Kim, Mingi Choi +1Language-Conditioned Robot ManipulationObject Pose Estimation

    Seoul National University · RLWRLD

  11. CodecSight: Leveraging Video Codec Signals for Efficient Streaming VLM Inference

    Apr 7, 2026Yulin Zou, Wenyan Chen, Yan Chen +6Efficient VLM InferenceVLM Inference

    NTU Singapore Singapore · Beihang University China · A*STAR IAIC Singapore +1

  12. Reinforce to Learn, Elect to Reason: A Dual Paradigm for Video Reasoning

    Apr 6, 2026Songyuan Yang, Weijiang Yu, Jilin Ma +5Video ReasoningVLM Inference

    National University of Defense Technology · Sun Yat-sen University

  13. Bridging the Missing-Modality Gap: Improving Text-Only Calibration of Vision Language Models

    Apr 3, 2026Mingyeong Kim, Jungwon Choi, Chaeyun Jang +1Modality Gap in VLMsVLM Inference

    Kim Jaechul Graduate School of AI, KAIST

  14. Look Twice: Training-Free Evidence Highlighting for Knowledge-based Visual Question Answering

    Apr 1, 2026Marco Morini, Sara Sarto, Marcella Cornia +2Knowledge-Based VQAVLM Inference

    University of Modena and Reggio Emilia, Italy

  15. A ROS 2 Wrapper for Florence-2: Multi-Mode Local Vision-Language Inference for Robotic Systems

    Apr 1, 2026J. E. Domínguez-VidalVLMs for RoboticsVLM Inference

  16. Evidence Packing for Cross-Domain Image Deepfake Detection with LVLMs

    Mar 18, 2026Yuxin Liu, Fei Wang, Kun Li +4Deepfake DetectionVLM Inference

    Anhui University · IAI, Hefei Comprehensive National Science Center · Hefei University of Technology +1

  17. UVLM: A Universal Vision-Language Model Loader for Reproducible Multimodal Benchmarking

    Mar 14, 2026Joan Perez, Giovanni FuscoVLM EvaluationVLM Inference

    Urban Geo Analytics, France · Université Côte d’Azur–CNRS–AMU–Avignon Université, ESPACE, France

  18. Authorize-on-Demand: Dynamic Authorization with Legality-Aware Intellectual Property Protection for VLMs

    Mar 5, 2026Lianyu Wang, Meng Wang, Huazhu Fu +1VLM Inference

    The Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education, China · Centre for Innovation and Precision Eye Health, Yong Loo Lin School of Medicine, NUS, Singapore · Department of Ophthalmology, Yong Loo Lin School of Medicine, NUS, Singapore +1

  19. QuickGrasp: Responsive Video-Language Querying Service via Accelerated Tokenization and Edge-Augmented Inference

    Feb 23, 2026Miao Zhang, Ruixiao Zhang, Jianxin Shi +3Efficient VLM InferenceVLM Inference

    School of Computing Science, Simon Fraser University, Canada · Computer Science Department, University of Illinois Urbana-Champaign, United States · College of Cryptology and Cyber Science, Nankai University, China +1

  20. Mitigating Cross-Image Information Leakage in Multi-Image Understanding with Large Vision-Language Models

    Aug 19, 2025Yeji Park, Minyoung Lee, Sanghyuk Chun +1Large Vision-Language ModelsVLM Reasoning

    Sogang University · Princeton University · NAVER AI Lab

  21. AdaptInfer: Adaptive Token Pruning for Vision-Language Model Inference via Dynamical Text Guidance

    Aug 8, 2025Weichen Zhang, Zhui Zhu, Ningbo Li +5Efficient VLM InferenceVisual Token Pruning

    Global Innovation Exchange, Tsinghua University, Beijing, China · Department of Automation, Tsinghua University, Beijing, China · School of Computer Science, Peking University, Beijing, China

  22. Mitigating Hallucinations via Inter-Layer Consistency Aggregation in Large Vision-Language Models

    May 18, 2025Kai Tang, Jinhao You, Yichen Guo +8Vision-Language ModelsHallucination in Language Models

    Peking University · University of Pennsylvania · Nanyang Technological University +4