LVLM Pruning

LVLM: Large Vision-Language Model

Follow this topic Ask about this topic

Latest papers

35 papers6 papers in the last four weeks, up 50% on the four weeks before. 0.0% of all new papers.

ASAP: Attention-Shift-Aware Pruning for Efficient LVLM Inference
Mar 15, 2026Surendra Pathak, Bo HanEfficient VLM Inference Visual Token Pruning
George Mason University, Fairfax, VA, USA
Language-Guided Token Compression with Reinforcement Learning in Large Vision-Language Models
Mar 11, 2026Sihan Cao, Jianwei Zhang, Pengcheng Zheng +7Efficient VLM Inference Visual Token Pruning
School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China · School of Robotics and Advanced Manufacture, Harbin Institute of Technology, Shenzhen, China · College of Information and Control Engineering, Xi’an University of Architecture and Technology, Xi’an, China
HiPP-Prune: Hierarchical Preference-Conditioned Structured Pruning for Vision-Language Models
Mar 6, 2026Lincen Bai, Hedi Tabia, Raul Santos-RodriguezLVLM Pruning Neural Network Pruning
Université Paris-Saclay · University of Bristol
OTPrune: Distribution-Aligned Visual Token Pruning via Optimal Transport
Feb 22, 2026Xiwen Chen, Wenhui Zhu, Gen Li +9Visual Token Pruning Efficient VLM Inference
Morgan Stanley · Clemson University · Arizona State University +4
Beyond Size: How Gradients Shape Pruning Decisions in Large Language Models
Nov 8, 2023Rocktim Jyoti Das, Mukul Ranjan, Mingjie Sun +2LVLM Pruning LLM Pruning
Mohamed bin Zayed University of AI · Carnegie Mellon University

Mar 15, 2026J/K move · Enter open · S save

ASAP: Attention-Shift-Aware Pruning for Efficient LVLM Inference

Surendra Pathak, Bo Han

George Mason University, Fairfax, VA, USA

Efficient VLM Inference Visual Token Pruning LVLM Pruning Visual Token Compression

Open paper AskPDF arXiv

While Large Vision-Language Models (LVLMs) demonstrate exceptional multi-modal capabilities, the quadratic computational cost of processing high-resolution visual tokens remains a critical bottleneck. Though recent token reduction strategies attempt to accelerate inference, such methods inadequately exploit attention values and fail to address token redundancy. More critically, they overlook the ``attention shift'' phenomenon inherent in LVLMs, which skews token attention scores. In this work, we propose ASAP, a novel training-free, KV-Cache-compatible pruning recipe that comprehensively addresses these limitations. First, we mitigate the attention shift by utilizing a dynamic bidirectional soft attention mask, ensuring the selection of genuinely informative tokens rather than naive attention-based selection. Second, we posit that high semantic redundancy within the token set degrades performance. We therefore introduce a weighted soft merging component that merges semantically similar tokens, preserving only the most feature-dense visual patches for subsequent layers. ASAP achieves virtually lossless compression of visual context, retaining 99.02% of the original LLaVA-NeXT-7B performance while aggressively slashing computational FLOPs by ~80%.

Latest papers

ASAP: Attention-Shift-Aware Pruning for Efficient LVLM Inference

Language-Guided Token Compression with Reinforcement Learning in Large Vision-Language Models

HiPP-Prune: Hierarchical Preference-Conditioned Structured Pruning for Vision-Language Models

OTPrune: Distribution-Aligned Visual Token Pruning via Optimal Transport

Beyond Size: How Gradients Shape Pruning Decisions in Large Language Models