cs.LGMar 30, 2026

Towards Efficient Large Vision-Language Models: A Comprehensive Survey on Inference Strategies

Authors: Surendra Pathak, Bo Han

Organizations: Computer Science Department, George Mason University, Fairfax, VA - 22030, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Although Large Vision Language Models (LVLMs) have demonstrated impressive multimodal reasoning capabilities, their scalability and deployment are constrained by massive computational requirements. In particular, the massive amount of visual tokens from high-resolution input data aggravates the situation due to the quadratic complexity of attention mechanisms. To address these issues, the research community has developed several optimization frameworks. This paper presents a comprehensive survey of the current state-of-the-art techniques for accelerating LVLM inference. We introduce a systematic taxonomy that categorizes existing optimization frameworks into four primary dimensions: visual token compression, memory management and serving, efficient architectural design, and advanced decoding strategies. Furthermore, we critically examine the limitations of these current methodologies and identify critical open problems to inspire future research directions in efficient multimodal systems.

Figures & tables

Fig. 1: Schematic of a standard Large Vision Language Model (LVLM) architecture. The vision encoder and text encoder convert the visual and text inputs to corresponding tokens. Vision tokens further transform through the projector before being fused with the text tokens. The aggregated tokens are then passed to the backbone language model for autoregressive generation.

Explore similar work

CardsList

  1. Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospects

    Apr 7, 2026Jun Zhang, Yicheng Ji, Feiyang Ren +7Efficient VLM InferenceLarge Vision-Language Models

    The State Key Laboratory of Blockchain and Data Security, Zhejiang University · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security

  2. Attention-aware Inference Optimizations for Large Vision-Language Models with Memory-efficient Decoding

    Mar 25, 2026Fatih Ilhan, Gaowen Liu, Ramana Rao Kompella +5Efficient VLM InferenceLarge Vision-Language Models

    Georgia Institute of Technology · Cisco Research

  3. Empirical Recipes for Efficient and Compact Vision-Language Models

    Mar 17, 2026Jiabo Huang, Zhizhong Li, Sina Sajadmanesh +2Vision-Language ModelsEfficient VLM Inference

    Sony AI.