Language-Guided Token Compression with Reinforcement Learning in Large Vision-Language Models

Sihan Cao; Jianwei Zhang; Pengcheng Zheng; Jiaxin Yan; Caiyan Qin; Yalan Ye; Wei Dong; Peng Wang

cs.CVMar 11, 2026

Language-Guided Token Compression with Reinforcement Learning in Large Vision-Language Models

Authors: Sihan Cao, Jianwei Zhang, Pengcheng Zheng, Jiaxin Yan, Caiyan Qin, Yalan Ye, Wei Dong, Peng Wang, +2 more

Organizations: School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China · School of Robotics and Advanced Manufacture, Harbin Institute of Technology, Shenzhen, China · College of Information and Control Engineering, Xi’an University of Architecture and Technology, Xi’an, China

Abstract

Large Vision-Language Models (LVLMs) incur substantial inference costs due to the processing of a vast number of visual tokens. Existing methods typically struggle to model progressive visual token reduction as a multi-step decision process with sequential dependencies and often rely on hand-engineered scoring rules that lack adaptive optimization for complex reasoning trajectories. To overcome these limitations, we propose TPRL, a reinforcement learning framework that learns adaptive pruning trajectories through language-guided sequential optimization tied directly to end-task performance. We formulate visual token pruning as a sequential decision process with explicit state transitions and employ a self-supervised autoencoder to compress visual tokens into a compact state representation for efficient policy learning. The pruning policy is initialized through learning from demonstrations and subsequently fine-tuned using Proximal Policy Optimization (PPO) to jointly optimize task accuracy and computational efficiency. Our experimental results demonstrate that TPRL removes up to 66.7% of visual tokens and achieves up to a 54.2% reduction in FLOPs during inference while maintaining a near-lossless average accuracy drop of only 0.7%. Code is released at mypink{https://github.com/MagicVicCoder/TPRL}.

Figures & tables

Figure 1: Overview of the TPRL framework consisting of three synergistic stages. (a) Autoencoder training for state compression learns a compact latent representation to reduce the feature dimensionality of each visual token. (b) Learning from demonstrations provides a foundational initialization for the pruning policy by imitating high-quality trajectories to ensure stable convergence. (c) RL training using PPO fine-tunes the policy through language-guided sequential optimization to achieve an optimal balance between task performance and computational efficiency.

Figure 2: Impact of state compression dimension $d_{l}$ on relative accuracy across RL training iterations.

Figure 3: Comparison of RL optimization methods.

Method	Token	GQA	MMB	MME	POPE	SEED	SQA I	VQA v2	Avg.
Upper Bound Model
LLaVA-1.5-7B [ 26 ]	576	62.0 (100%)	64.6 (100%)	1553.4 (100%)	85.9 (100%)	60.3 (100%)	70.4 (100%)	78.9 (100%)	- (100%)
Methods without RL
ToMe [ 5 ]	576	192	54.3 (87.7%)	60.5 (93.7%)	1485 (95.6%)	72.4 (84.3%)	53.1 (88.1%)	65.2 (92.6%)	67.9 (86.1%)	- (90.5%)
ToMe [ 5 ]	128	52.4 (84.5%)	53.3 (82.5%)	1343 (86.5%)	62.8 (73.1%)	50.9 (84.4%)	59.6 (84.7%)	63.0 (79.8%)	- (85.4%)
PruMerge+ [ 34 ]	144	61.3 (98.7%)	64.9 (100.5%)	1462.4 (94.1%)	84.0 (97.8%)	-	68.3 (97.0%)	76.8 (97.3%)	- (97.6%)

Table 1: Performance comparison on multiple benchmarks with the LLaVA-1.5-7B backbone. We report absolute scores and the compression retention rates to Upper Bound model. (Token) denotes the average visual token count per LLM layer. Methods marked with (*) means the retained token count is adaptively determined and we calculate the average token count during inference across all benchmarks.

Method	Token	GQA	MMB	MME	POPE	SQA I	Avg.
LLaVA-1.5-VL-13B	576	63.3	68.64	1522	85.99	72.88	-
LLaVA-1.5-VL-13B		100%	100%	100%	100%	100%	100%
FastV [ 8 ]	192	54.2	64.8	1290	64.1	69.9	-
		85.7%	94.5%	84.7%	75.2%	92.1%	86.4%
	128	48.5	61.2	1150	58.2	68.0	-
		76.6%	89.2%	75.6%	67.7%	93.3%	80.5%

Table 2: Performance comparison on multiple benchmarks with LLaVA-1.5-VL-13B backbone.

Method	Token	GQA	MME	SQA I	VQA v2	Avg.
Qwen-2.5-VL-3B [ 3 ]	256	61.5	1450	70.2	77.4	-
Qwen-2.5-VL-3B [ 3 ]		100%	100%	100%	100%	100%
FastV [ 8 ]	144	54.7	1305	64.6	73.5	-
		89.0%	90.0%	92.0%	95.0%	91.5%
	128	49.5	1232	61.1	67.7	-
		80.5%	85.0%	87.0%	87.5%	85.0%

Table 3: Performance comparison on multiple benchmarks with Qwen-VL-2.5 backbone.

Method	Avg.	Accuracy	Storage	$\Delta$	Latency	$\Delta$	FLOPs	$\Delta$
Method	Token	Accuracy	Memory (MB)	$\Delta$	(ms) $\downarrow$	$\Delta$	(T) $\downarrow$	$\Delta$
Upper Bound	576	100%	302.4	-	57.8	-	9.6	-
FastV [ 8 ]	192	86.0%	100.8	66.7%	23.1	60.0%	2.0	79.2%
SparseVLM [ 45 ]	192	94.2%	100.8	66.7%	26.0	55.0%	3.9	59.4%
Ours	192*	99.3%	100.9	66.6%	26.7	53.8%	4.4	54.2%
	144*	98.4%	75.7	75.0%	22.6	60.9%	3.7	61.4%

Table 4: Efficiency analysis including cache storage memory, latency and the FLOPs.

\Delta

denotes the reduction ratio.

LLM Status	VQA v2	MMB	MME	POPE
Frozen	98.8%	99.5%	98.5%	98.6%
Fine-tuned	99.6%	102.3%	99.4%	99.7%

Table 5: Ablation study on LLM fine-tuning.

$T_{\max}$	1	2	3	4	5
Rel. Acc (%)	95.2	98.8	99.6	96.2	93.7

Table 7: Impact of

T_{\max}

on VQA-v2.

Figure 4: Qualitative visualization of sequential pruning trajectories via mask superimposition. The agent progressively filters backgrounds and redundant details to isolate task-critical information through our multi-step MDP formulation.

Explore similar work

VPRune: Efficient Training-free Pre-LLM Visual Token Pruning
Sep 21, 2026Guangchuan Lv, Dianxing Shi, Dingjie FuEfficient VLM Inference Large Vision-Language Models
Northeastern University · Beihang University · Huazhong University of Science and Technology
ToDRE: Effective Visual Token Pruning via Token Diversity and Task Relevance
May 24, 2025Duo Li, Zuhao Yang, Xiaoqin Zhang +2Efficient VLM Inference Large Vision-Language Models
CCDS, NTU, Singapore · CCST, ZJUT, China · Terminus AI Lab, UCAS, China
GRIP-VLM: Group-Relative Importance Pruning for Efficient Vision-Language Models
May 13, 2026Mingzhe Huang, Weijun Wang, Xin Ding +7Efficient VLM Inference Visual Token Pruning
Institute for AI Industry Research (AIR), Tsinghua University · University of Science and Technology of China · Nanjing University +1

Sep 21, 2026J/K move · Enter open · S save

VPRune: Efficient Training-free Pre-LLM Visual Token Pruning

Guangchuan Lv, Dianxing Shi, Dingjie Fu

Northeastern University · Beihang University · Huazhong University of Science and Technology

Efficient VLM Inference Large Vision-Language Models Visual Token Pruning Visual Token Compression

Open paper AskPDF arXiv

Visual token pruning is a promising approach to reducing the inference cost of large vision-language models (LVLMs), yet aggressive token reduction often causes substantial performance degradation. We identify three key factors behind this degradation: text-guided selection bias, information loss from discarded tokens, and positional distortion caused by sequence compaction. Based on these observations, we propose VPRune, a training-free pre-LLM pruning framework consisting of visual-only diversity selection, similarity-guided token recycling, and position-preserving restoration. Experiments on FastVLM-1.5B across multiple vision-language benchmarks demonstrate that VPRune achieves a favorable accuracy--compression trade-off, with particularly pronounced advantages under aggressive compression. Furthermore, evaluations on edge-device show that VPRune effectively reduces end-to-end inference latency while maintaining superior task performance, demonstrating its practicality for resource-constrained LVLM deployment.

Abstract

Explore similar work

VPRune: Efficient Training-free Pre-LLM Visual Token Pruning

ToDRE: Effective Visual Token Pruning via Token Diversity and Task Relevance

GRIP-VLM: Group-Relative Importance Pruning for Efficient Vision-Language Models