TwinPrune: Density-Aware Two-Phase Token Pruning for Vision-Language Models

Kai Liu; Anqi Li; Junxian Li; Zhixin Wang; Zhikai Chen; Renjing Pei; Yulun Zhang

cs.CVOct 13, 2026

TwinPrune: Density-Aware Two-Phase Token Pruning for Vision-Language Models

Authors: Kai Liu, Anqi Li, Junxian Li, Zhixin Wang, Zhikai Chen, Renjing Pei, Yulun Zhang

Organizations: Shanghai Jiao Tong University · Huawei Technologies Ltd

Abstract

We propose a unified spatial-density signal to score token importance for both inference phases: bipartite merge for prefill and KV eviction for decode, yielding measurable gains in each stage. To isolate the two phases, we split benchmarks by scoring logic. Prefill probes (MMBench, POPE, ScienceQA) evaluate only the first decoded token; their accuracy reflects prefill logits argmax before any decode KV eviction. Full-pipeline probes (TextVQA, DocVQA, MM-Vet) assess the complete generated sequence and capture both phases. We identify a critical evaluation pitfall: decode pruning claims validated solely on prefill probes measure prefill quality, not decode performance. Experiments show discarding all visual KV during decode barely affects prefill-probe accuracy but degrades TextVQA and MM-Vet drastically; this insensitivity stems from the scoring rule, not compression intensity. Evaluated on seven model variants (InternVL-3.5, Qwen3-VL, LLaVA-1.5) and three full-pipeline benchmarks, our method achieves a three-way Pareto frontier across compute, memory and accuracy. Density-based prefill compression maintains strong accuracy, scales linearly with visual token count, and delivers 100x faster token selection at high resolution. For decode, density top-k KV eviction consistently outperforms strong baselines H2O, SnapKV and StreamingLLM on TextVQA and MM-Vet across three retention ratios. At 0.125 keep ratio on TextVQA, it surpasses H2O by 9.6 points; InternVL-3.5 models (2B to 38B) maintain a 7 to 10 point advantage. Compression scales favorably with model size: every measured 8B to 38B model transition reduces accuracy loss instead of worsening it. We suggest MM-Vet or DocVQA as mandatory full-pipeline validation for any decode-phase claims. Our code is available at https://github.com/Kai-Liu001/TwinPrune

Figures & tables

Figure 1: Two-Stage Pruning in TwinPrune. TwinPrune improves VLM inference through a two-stage pruning framework using a single unified pruning signal. The left panel evaluates visual token pruning in the prefilling stage, showing better accuracy–speedup trade-offs, while the right panel evaluates KV cache pruning on long-output benchmarks in the decode stage. Both panels report results on InternVL-3.5-8B under the same evaluation setting.

Figure 2: Prefill-probe benchmarks are structurally insensitive to decode-phase KV eviction across three VLM families. Left: accuracy vs. visual KV kept on InternVL-3.5-8B. Right: prediction mismatch at Dall grouped by (benchmark, model). Details in Sec. 3 .

Phase	Signal · Keep	MMBench	POPE	SQA	TextVQA
Prefill	Density · $75$ / $50$ / $25\%$	$0$	$0$ – $0.03$	–	$3.1$ / $7.0$ / $14.9$
Prefill	Random · $25\%$	$0$	$0$	–	$32.8$
Prefill	Attention · $25\%$	$0$	$0.03$	–	$25.1$
Decode	Density · $25$ / $12.5$ / $6.25\%$	$0.54$	$0.32$ – $0.35$	$0.20$	$15.9$ / $23.0$ / $31.1$
Decode	Density · $0\%$ (drop all)	0.54	0.38	0.20	45.5

Table 1: Prediction-mismatch % vs. no-compression baseline on InternVL-3.5-8B. Decode KV eviction flips up to

46\%

of TextVQA predictions but

\leq 0.54\%

on prefill-probe benches, since

y_{1}

depends only on prefill logits (Eq. 1 ). Prefill rows fix

L{=}15

(varying

L

shifts levels but not the scoring-rule gap). Appendix extends to all 3 archs.

Figure 3: TwinPrune pipeline. A single spatial density signal $\rho$ , defined as each visual token’s average cosine dissimilarity to its $w{\times}w$ neighborhood (b), drives both phases. Prefill bipartite-merges low- $\rho$ source tokens into high- $\rho$ destination anchors, cutting FLOPs (c). Decode ranks visual KV by $\rho$ and keeps the top- $k$ with original RoPE positions, cutting KV memory (d). One signal, two levers, and a three-axis Pareto trade-off over compute, KV, and accuracy (a).

Method	Venue	TPL %	MMBench	POPE	SQA	TextVQA	Avg-4	Speedup
Baseline (no compression)		100.0	0.8707	0.8903	0.9539	0.6493	0.8411	$1.00\times$
Block 1: low-TPL — $L\!=\!15$ , prune $50\%$ , TPL $=65.2\%$
Random	–	65.2	0.8862	0.9155	0.9782	0.6202	0.8500	$\mathbf{1.28\times}$
ToMe	ICLR’23 [ 4 ]	65.2	0.8584	0.8913	0.9346	0.5325	0.8042	$1.16\times$
FastV	ECCV’24 [ 6 ]	65.2	0.8584	0.8868	0.9326	0.5150	0.7982	$\mathbf{1.28\times}$
PruMerge	ICCV’25 [ 23 ]	65.2	0.8584	0.8943	0.9410	0.5937	0.8219	$1.21\times$

Table 2: Per-benchmark prefill compression on InternVL-3.5-8B ; same

(L,r)

within each block. Bold marks per-column best; daggers (†) flag structurally invalid output (acc

\approx\!0

, App. ). Block 4 in App. . Speedup : measured prefill speedup vs. uncompressed baseline at batch

=

8, HD mode (12 tiles,

N_{\text{vis}}{=}3072

) on RTX A6000.

Figure 4: Decode KV Eviction on TextVQA. On InternVL-3.5-8B, density-guided eviction Pareto-dominates random and attention-value-norm in the memory–accuracy trade-off. Right inset: Qwen3-VL-4B, LLaVA-1.5-7B, and InternVL-3.5-38B show the same monotonic density trend.

	TextVQA			MM-Vet			DocVQA
Method (keep $k$ )	$50\%$	$25\%$	$12.5\%$	$50\%$	$25\%$	$12.5\%$	$50\%$	$25\%$	$12.5\%$
Baseline (no eviction)	$0.733$			$0.756$			$0.642$
StreamingLLM [ 29 ]	0.637	0.573	0.507	0.667	0.561	0.514	0.468	0.384	0.324
H2O [ 33 ]	0.697	0.629	0.552	0.662	0.571	0.497	0.555	0.438	0.347
SnapKV [ 13 ]	0.707	0.643	0.573	0.654	0.585	0.513	0.577	0.469	0.377
Density (ours)	0.726	0.697	0.648	0.708	0.634	0.551	0.575	0.466	0.382

Table 3: Decode-phase KV eviction comparison on InternVL-3.5-8B at three keep ratios. Density top-

k

wins all six TextVQA and MM-Vet cells. On DocVQA, SnapKV narrowly leads at keep

\geq 25\%

but density reclaims the lead at

12.5\%

keep. Bold marks per-column best. TextVQA uses

n{=}5000

exact-match. MM-Vet uses

n{=}218

. DocVQA uses

n{=}5349

with ANLS scoring.

Efficient VLM Inference

Efficient vision-language model inference reduces the compute, memory, latency, and energy needed to process images, video, and text.

See all 485 papers

VPRune: Efficient Training-free Pre-LLM Visual Token Pruning
Sep 21, 2026Guangchuan Lv, Dianxing Shi, Dingjie FuEfficient VLM Inference Large Vision-Language Models
Northeastern University · Beihang University · Huazhong University of Science and Technology
AsymVLM: Asymmetric Token Pruning for Efficient Vision-Language Model Inference
May 28, 2026Yilin Feng, Ahmed Burak Gulhan, Mahmut Taylan KandemirLLM Pruning Vision-Language Models
The Pennsylvania State University
Are We Evaluating VLM Token Pruning Right? Looking Beyond Accuracy and Token Count
Oct 17, 2026Geng Li, Guohao Chen, Shilin Shan +1Visual Token Pruning VLM Evaluation
Nanyang Technological University

Sep 21, 2026J/K move · Enter open · S save

VPRune: Efficient Training-free Pre-LLM Visual Token Pruning

Guangchuan Lv, Dianxing Shi, Dingjie Fu

Northeastern University · Beihang University · Huazhong University of Science and Technology

Efficient VLM Inference Large Vision-Language Models Visual Token Pruning Visual Token Compression LVLM Pruning

Open paper AskPDF arXiv

Visual token pruning is a promising approach to reducing the inference cost of large vision-language models (LVLMs), yet aggressive token reduction often causes substantial performance degradation. We identify three key factors behind this degradation: text-guided selection bias, information loss from discarded tokens, and positional distortion caused by sequence compaction. Based on these observations, we propose VPRune, a training-free pre-LLM pruning framework consisting of visual-only diversity selection, similarity-guided token recycling, and position-preserving restoration. Experiments on FastVLM-1.5B across multiple vision-language benchmarks demonstrate that VPRune achieves a favorable accuracy--compression trade-off, with particularly pronounced advantages under aggressive compression. Furthermore, evaluations on edge-device show that VPRune effectively reduces end-to-end inference latency while maintaining superior task performance, demonstrating its practicality for resource-constrained LVLM deployment.

Abstract

Efficient VLM Inference

Explore similar work

VPRune: Efficient Training-free Pre-LLM Visual Token Pruning

AsymVLM: Asymmetric Token Pruning for Efficient Vision-Language Model Inference

Are We Evaluating VLM Token Pruning Right? Looking Beyond Accuracy and Token Count