ResPrune: Text-Conditioned Subspace Reconstruction for Visual Token Pruning in Large Vision-Language Models

Xu Li; Yi Zheng; Yuxuan Liang; Zhe Liu; Xiaolei Chen; Haotian Chen; Rui Zhu; Xiangyang Xue

cs.LGMar 22, 2026

ResPrune: Text-Conditioned Subspace Reconstruction for Visual Token Pruning in Large Vision-Language Models

Authors: Xu Li, Yi Zheng, Yuxuan Liang, Zhe Liu, Xiaolei Chen, Haotian Chen, Rui Zhu, Xiangyang Xue

Organizations: College of Computer Science and Artificial Intelligence, Fudan University, Shanghai 200433, China

Abstract

Large Vision-Language Models (LVLMs) rely on dense visual tokens to capture fine-grained visual information, but processing all these tokens incurs substantial computational and memory overhead during inference. To address this issue, we propose ResPrune, a training-free visual token pruning framework that enables efficient LVLM inference by selecting a compact yet informative subset of visual tokens. ResPrune formulates visual token pruning as a subspace reconstruction problem and employs a greedy subspace expansion strategy guided by residual energy, allowing it to preserve the geometric structure of the original visual token space. To further incorporate cross modal alignment, the selection process is conditioned on textual relevance, encouraging the retention of tokens that are both informative and instruction-relevant. The proposed method is lightweight and model-agnostic, and can be seamlessly integrated into existing LVLM pipelines without retraining or architectural modifications. Extensive experiments on multiple LVLM backbones, including LLaVA-1.5, LLaVA-NeXT, and Qwen2.5-VL, demonstrate that ResPrune consistently outperforms existing pruning approaches across a wide range of benchmarks, while achieving effective reductions in computation, memory consumption, and inference latency.

Figures & tables

Fig. 1: Comparison of visual token pruning behaviors across different methods under varying textual queries for a fixed image. Masked patches indicate pruned tokens, while unmasked patches represent the retained ones. Answer-relevant regions are highlighted with colored bounding boxes.

Retain 192 visual tokens (pruning ratio = 66.7%)
Method	Rel. Perf.	MME	GQA	SQA-I	POPE	TextVQA	VizWiz	VQA-v2	MMB-en	MM-Vet
LLaVA-1.5-7B [ 2 ]	100%	1862	61.9	69.5	85.9	58.2	50.0	78.4	64.7	31.3
ToMe (ICLR2023) [ 26 ]	89.9	1563	54.3	65.2	72.4	52.1	50.0	68.0	60.5	26.8
FastV (ECCV2024) [ 13 ]	89.5	1612	52.7	67.3	64.8	52.5	50.8	67.1	61.2	27.7
PruMerge (ICCV2025) [ 10 ]	91.5	1632	54.3	67.9	71.3	54.3	50.1	70.6	59.6	-
SparseVLM (ICML2025) [ 14 ]	96.6	1721	57.6	69.1	83.6	56.1	50.5	75.6	62.5	31.5

TABLE I: Performance comparison on LLaVA-1.5-7B under different visual token pruning ratios. “Rel. Perf.” denotes the average relative performance with respect to the full-token baseline where higher values indicate better performance retention. Best results under each pruning ratio are highlighted in bold.

Retain up to 640 visual tokens (pruning ratio = 66.7%)
Method	Rel. Perf.	MME	GQA	SQA-I	POPE	TextVQA	VizWiz	VQA-v2	MMB-en
LLaVA-NeXT-7B [ 3 ]	100%	1842	64.3	70.2	86.5	61.3	55.2	81.3	67.9
FastV (ECCV2024) [ 13 ]	94.7	1807	58.9	67.4	79.5	58.1	53.9	77.0	63.1
PruMerge (ICCV2025) [ 10 ]	96.6	1790	60.8	67.8	85.3	54.9	57.9	78.2	64.6
PDrop (CVPR2025) [ 15 ]	95.7	1782	60.0	66.7	83.8	57.8	53.8	79.1	64.1
SparseVLM (ICML2025) [ 14 ]	96.9	1772	61.2	67.6	85.3	59.7	53.6	79.2	65.9

TABLE II: Performance comparison on LLaVA-NeXT-7B under different visual token pruning ratios. “Rel. Perf.” denotes the average relative performance with respect to the full-token baseline where higher values indicate better performance retention. Best results under each pruning ratio are highlighted in bold.

Pruning ratio = 66.7%
Method	Rel. Perf.	MME	POPE	TextVQA	MMB-en	GQA
Qwen2.5-VL-7B	100%	2304	86.1	84.8	82.8	62.2
FastV (ECCV2024)	92.3	2072	82.2	77.9	75.7	58.0
DivPrune (CVPR2025)	96.7	2198	85.6	80.1	82.6	59.0
ResPrune (Ours)	98.4	2320	87.3	80.3	82.5	59.9
Pruning ratio = 77.8%

TABLE III: Performance comparison on Qwen2.5-VL-7B under different visual token pruning ratios. “Rel. Perf.” denotes the average relative performance with respect to the full-token baseline where higher values indicate better performance retention. Best results under each pruning ratio are highlighted in bold.

Retain 128 visual tokens (pruning ratio = 77.8%)
Ablation Setting	Rel. Perf.	MME	GQA	POPE	TextVQA	MMB
LLaVA-1.5-7B	100%	1862	61.9	85.9	58.2	64.7
Full Method	98.4	1792	60.1	87.6	57.8	63.0
Setting-1	97.5	1754	59.8	87.5	57.1	62.6
Setting-2	96.6	1773	59.7	86.7	54.6	62.5
Setting-3	82.4	1668	55.6	83.5	25.4	59.5

TABLE IV: Ablation study of the main components in ResPrune. Setting-1 removes text preprocessing, Setting-2 removes textual relevance, and Setting-3 removes subspace reconstruction. “Rel. Perf.” denotes relative performance compared to the full-token baseline.

Pruning ratio = 77.8%
Alpha	Rel. Perf.	MME	GQA	POPE	TextVQA	MMB
LLaVA-1.5-7B	100%	1862	61.9	85.9	58.2	64.7
Alpha = 0.25	97.7	1752	60.1	87.7	57.4	62.4
Alpha = 0.5	98.1	1778	59.9	87.7	57.9	62.7
Alpha = 0.75	98.4	1792	60.1	87.6	57.8	63.0
Alpha = 1.0	98.3	1801	60.0	87.5	57.7	62.6

TABLE V: Ablation study of the textual guidance strength

\alpha

on LLaVA-1.5-7B and Qwen2.5-VL-7B. “Rel. Perf.” denotes relative performance compared to the full-token baseline.

Retain 128 visual tokens (pruning ratio = 77.8%)
Setting	Rel. Perf.	MME	GQA	POPE	TextVQA	MMB
LLaVA-1.5-7B	100%	1862	61.9	85.9	58.2	64.7
Setting-1	98.4	1792	60.1	87.6	57.8	63.0
Setting-2	97.8	1778	59.9	87.7	56.6	62.9
Setting-3	95.4	1775	59.6	87.5	51.2	61.8

TABLE VI: Ablation study on different formulations of textual relevance computation. Details of each setting are described in Section IV-C3 . “Rel. Perf.” denotes the average relative performance with respect to the full-token baseline.

Pruning ratio = 77.8%, CLS token present
Setting	Rel. Perf.	MME	GQA	POPE	TextVQA	MMB
LLaVA-1.5-7B	100%	1862	61.9	85.9	58.2	64.7
Setting-1	98.4	1792	60.1	87.6	57.8	63.0
Setting-2	98.0	1773	60.0	87.3	57.7	62.7
Setting-3	97.7	1755	59.9	87.8	57.7	62.1
Setting-4	97.7	1753	59.9	87.5	57.9	62.2

TABLE VII: Ablation study on different seed token selection strategies. Details of each setting are described in Section IV-C4 . “Rel. Perf.” denotes the average relative performance with respect to the full-token baseline.

Method	FLOPs (T)	KV-cache (MB)	GPU Memory (GB)	Throughput (tokens/sec.)	Evaluation Time (min.:sec.)	Performance
Retain up to 2880 visual tokens
LLaVA-NeXT-7B	30.6	1084.7	19.8	16.1	9:05	1842
Retain up to 640 visual tokens (pruning ratio = 66.7%)
ResPrune	9.6 ( $\downarrow 68.6\%$ )	361.2 ( $\downarrow 66.7\%$ )	17.8 ( $\downarrow 10.1\%$ )	33.5 ( $2.08\times$ )	4:39 ( $\downarrow 48.8\%$ )	1821 ( $\downarrow 1.1\%$ )
Retain up to 320 visual tokens (pruning ratio = 88.9%)
ResPrune	3.1 ( $\downarrow 89.9\%$ )	120.4 ( $\downarrow 88.9\%$ )	17.7 ( $\downarrow 10.6\%$ )	36.7 ( $2.28\times$ )	3:28 ( $\downarrow 61.8\%$ )	1780 ( $\downarrow 3.3\%$ )

TABLE VIII: Practical efficiency analysis of ResPrune on LLaVA-NeXT-7B measured on the MME benchmark. Relative changes with respect to the full-token baseline are shown in parentheses and highlighted in green.

Fig. 2: Qualitative visualization of ResPrune’s pruning results on LLaVA-1.5-7B under different visual token budgets. Masked patches correspond to pruned visual tokens, while unmasked patches indicate retained tokens. Answer-relevant regions are highlighted with red bounding boxes.

Visual Token Pruning

Visual token pruning reduces the image and video patches processed by transformers to lower inference cost while preserving task-relevant information.

See all 269 papers

VPRune: Efficient Training-free Pre-LLM Visual Token Pruning
Sep 21, 2026Guangchuan Lv, Dianxing Shi, Dingjie FuEfficient VLM Inference Large Vision-Language Models
Northeastern University · Beihang University · Huazhong University of Science and Technology
Hierarchical and Query-Aware Visual Token Pruning for High-Resolution LVLMs
Oct 16, 2026Luther Wei, Ling Zou, Kang-Yang Huang +2Visual Token Pruning Efficient VLM Inference
Department of Computer Science and Information Engineering, National Taiwan University, Taipei, Taiwan · MediaTek Inc., Hsinchu, Taiwan
ACPruner: Visual Token Pruning as Biased Attention Coverage Maximization in LVLMs
Sep 28, 2026Xu Li, Yuxuan Liang, Yi Zheng +6Efficient VLM Inference Large Vision-Language Models
College of Computer Science and Artificial Intelligence Fudan University, Shanghai, China

Sep 21, 2026J/K move · Enter open · S save

VPRune: Efficient Training-free Pre-LLM Visual Token Pruning

Guangchuan Lv, Dianxing Shi, Dingjie Fu

Northeastern University · Beihang University · Huazhong University of Science and Technology

Efficient VLM Inference Large Vision-Language Models Visual Token Pruning Visual Token Compression LVLM Pruning

Open paper AskPDF arXiv

Visual token pruning is a promising approach to reducing the inference cost of large vision-language models (LVLMs), yet aggressive token reduction often causes substantial performance degradation. We identify three key factors behind this degradation: text-guided selection bias, information loss from discarded tokens, and positional distortion caused by sequence compaction. Based on these observations, we propose VPRune, a training-free pre-LLM pruning framework consisting of visual-only diversity selection, similarity-guided token recycling, and position-preserving restoration. Experiments on FastVLM-1.5B across multiple vision-language benchmarks demonstrate that VPRune achieves a favorable accuracy--compression trade-off, with particularly pronounced advantages under aggressive compression. Furthermore, evaluations on edge-device show that VPRune effectively reduces end-to-end inference latency while maintaining superior task performance, demonstrating its practicality for resource-constrained LVLM deployment.

Abstract

Visual Token Pruning

Explore similar work

VPRune: Efficient Training-free Pre-LLM Visual Token Pruning

Hierarchical and Query-Aware Visual Token Pruning for High-Resolution LVLMs

ACPruner: Visual Token Pruning as Biased Attention Coverage Maximization in LVLMs