We propose a unified spatial-density signal to score token importance for both inference phases: bipartite merge for prefill and KV eviction for decode, yielding measurable gains in each stage. To isolate the two phases, we split benchmarks by scoring logic. Prefill probes (MMBench, POPE, ScienceQA) evaluate only the first decoded token; their accuracy reflects prefill logits argmax before any decode KV eviction. Full-pipeline probes (TextVQA, DocVQA, MM-Vet) assess the complete generated sequence and capture both phases. We identify a critical evaluation pitfall: decode pruning claims validated solely on prefill probes measure prefill quality, not decode performance. Experiments show discarding all visual KV during decode barely affects prefill-probe accuracy but degrades TextVQA and MM-Vet drastically; this insensitivity stems from the scoring rule, not compression intensity. Evaluated on seven model variants (InternVL-3.5, Qwen3-VL, LLaVA-1.5) and three full-pipeline benchmarks, our method achieves a three-way Pareto frontier across compute, memory and accuracy. Density-based prefill compression maintains strong accuracy, scales linearly with visual token count, and delivers 100x faster token selection at high resolution. For decode, density top-k KV eviction consistently outperforms strong baselines H2O, SnapKV and StreamingLLM on TextVQA and MM-Vet across three retention ratios. At 0.125 keep ratio on TextVQA, it surpasses H2O by 9.6 points; InternVL-3.5 models (2B to 38B) maintain a 7 to 10 point advantage. Compression scales favorably with model size: every measured 8B to 38B model transition reduces accuracy loss instead of worsening it. We suggest MM-Vet or DocVQA as mandatory full-pipeline validation for any decode-phase claims. Our code is available at https://github.com/Kai-Liu001/TwinPrune
Figures & tables
Figure 1: Two-Stage Pruning in TwinPrune. TwinPrune improves VLM inference through a two-stage pruning framework using a single unified pruning signal. The left panel evaluates visual token pruning in the prefilling stage, showing better accuracy–speedup trade-offs, while the right panel evaluates KV cache pruning on long-output benchmarks in the decode stage. Both panels report results on InternVL-3.5-8B under the same evaluation setting.
Figure 2: Prefill-probe benchmarks are structurally insensitive to decode-phase KV eviction across three VLM families. Left: accuracy vs. visual KV kept on InternVL-3.5-8B. Right: prediction mismatch at Dall grouped by (benchmark, model). Details in Sec. 3 .
Phase
Signal · Keep
MMBench
POPE
SQA
TextVQA
Prefill
Density · 75 / 50 / 25%
0
0 – 0.03
–
3.1 / 7.0 / 14.9
Prefill
Random · 25%
0
0
–
32.8
Prefill
Attention · 25%
0
0.03
–
25.1
Decode
Density · 25 / 12.5 / 6.25%
0.54
0.32 – 0.35
0.20
15.9 / 23.0 / 31.1
Decode
Density · 0% (drop all)
0.54
0.38
0.20
45.5
Table 1: Prediction-mismatch % vs. no-compression baseline on InternVL-3.5-8B. Decode KV eviction flips up to 46% of TextVQA predictions but ≤0.54% on prefill-probe benches, since y1 depends only on prefill logits (Eq. 1 ). Prefill rows fix L=15 (varying L shifts levels but not the scoring-rule gap). Appendix extends to all 3 archs.
Figure 3: TwinPrune pipeline. A single spatial density signal ρ , defined as each visual token’s average cosine dissimilarity to its w×w neighborhood (b), drives both phases. Prefill bipartite-merges low- ρ source tokens into high- ρ destination anchors, cutting FLOPs (c). Decode ranks visual KV by ρ and keeps the top- k with original RoPE positions, cutting KV memory (d). One signal, two levers, and a three-axis Pareto trade-off over compute, KV, and accuracy (a).
Method
Venue
TPL %
MMBench
POPE
SQA
TextVQA
Avg-4
Speedup
Baseline (no compression)
100.0
0.8707
0.8903
0.9539
0.6493
0.8411
1.00×
Block 1: low-TPL — L=15 , prune 50% , TPL =65.2%
Random
–
65.2
0.8862
0.9155
0.9782
0.6202
0.8500
1.28×
ToMe
ICLR’23 [ 4 ]
65.2
0.8584
0.8913
0.9346
0.5325
0.8042
1.16×
FastV
ECCV’24 [ 6 ]
65.2
0.8584
0.8868
0.9326
0.5150
0.7982
1.28×
PruMerge
ICCV’25 [ 23 ]
65.2
0.8584
0.8943
0.9410
0.5937
0.8219
1.21×
Table 2: Per-benchmark prefill compression on InternVL-3.5-8B ; same (L,r) within each block. Bold marks per-column best; daggers (†) flag structurally invalid output (acc ≈0 , App. ). Block 4 in App. . Speedup : measured prefill speedup vs. uncompressed baseline at batch = 8, HD mode (12 tiles, Nvis=3072 ) on RTX A6000.
Figure 4: Decode KV Eviction on TextVQA. On InternVL-3.5-8B, density-guided eviction Pareto-dominates random and attention-value-norm in the memory–accuracy trade-off. Right inset: Qwen3-VL-4B, LLaVA-1.5-7B, and InternVL-3.5-38B show the same monotonic density trend.
TextVQA
MM-Vet
DocVQA
Method (keep k )
50%
25%
12.5%
50%
25%
12.5%
50%
25%
12.5%
Baseline (no eviction)
0.733
0.756
0.642
StreamingLLM [ 29 ]
0.637
0.573
0.507
0.667
0.561
0.514
0.468
0.384
0.324
H2O [ 33 ]
0.697
0.629
0.552
0.662
0.571
0.497
0.555
0.438
0.347
SnapKV [ 13 ]
0.707
0.643
0.573
0.654
0.585
0.513
0.577
0.469
0.377
Density (ours)
0.726
0.697
0.648
0.708
0.634
0.551
0.575
0.466
0.382
Table 3: Decode-phase KV eviction comparison on InternVL-3.5-8B at three keep ratios. Density top- k wins all six TextVQA and MM-Vet cells. On DocVQA, SnapKV narrowly leads at keep ≥25% but density reclaims the lead at 12.5% keep. Bold marks per-column best. TextVQA uses n=5000 exact-match. MM-Vet uses n=218 . DocVQA uses n=5349 with ANLS scoring.
Visual token pruning is a promising approach to reducing the inference cost of large vision-language models (LVLMs), yet aggressive token reduction often causes substantial performance degradation. We identify three key factors behind this degradation: text-guided selection bias, information loss from discarded tokens, and positional distortion caused by sequence compaction. Based on these observations, we propose VPRune, a training-free pre-LLM pruning framework consisting of visual-only diversity selection, similarity-guided token recycling, and position-preserving restoration. Experiments on FastVLM-1.5B across multiple vision-language benchmarks demonstrate that VPRune achieves a favorable accuracy--compression trade-off, with particularly pronounced advantages under aggressive compression. Furthermore, evaluations on edge-device show that VPRune effectively reduces end-to-end inference latency while maintaining superior task performance, demonstrating its practicality for resource-constrained LVLM deployment.
Guangchuan Lv, Dianxing Shi, Dingjie Fu
Northeastern University · Beihang University · Huazhong University of Science and Technology
Vision-Language Models (VLMs) process thousands of visual tokens per image alongside comparatively few text tokens, yet existing compression methods treat both modalities uniformly. We observe that the two modalities have fundamentally different properties: vision tokens are spatially redundant and dominate prefill, while text tokens are causally dependent and accumulate during decoding. Based on this asymmetry, we propose and empirically evaluate AsymVLM, which applies aggressive pruning to vision tokens before prefill using a learned importance scorer with per-sample adaptive budgeting, and temporal threshold-based eviction to text tokens only when they exceed a fixed budget. Our experiments indicate that AsymVLM achieves the highest FLOPs savings (up to 54%) among state-of-the-art methods while outperforming existing approaches by 2--3% on document and chart understanding tasks where visual information is spatially localized and query-specific, and maintaining competitive accuracy on holistic benchmarks. In text-dominated scenarios, our eviction strategy substantially outperforms standard LLM cache compression methods by adapting to the short-context nature of VLM.
Yilin Feng, Ahmed Burak Gulhan, Mahmut Taylan Kandemir
Visual token pruning aims to reduce inference cost while preserving the capabilities of the original VLM. Yet existing evaluations rely on aggregate task accuracy and retained token count as proxies for pruning quality and actual efficiency. We reveal two limitations: 1) Aggregate accuracy can obscure whether a pruning method truly identifies critical visual tokens; many examples remain solvable under arbitrary token removal and models can exploit textual shortcuts when visual evidence is degraded. 2) Retained token count is an unreliable proxy for efficiency, as identical token budgets can incur different end-to-end costs depending on where and how pruning is performed. Consequently, comparisons based on aggregate accuracy and retained token counts fail to recover the global performance-efficiency Pareto frontier across methods. We introduce PAA-Bench, a plug-and-play evaluation framework for existing benchmarks, to evaluate whether pruning succeeds through meaningful token selection and whether this success translates into real speedup. To determine when token choice actually matters, we propose Random-Pass@k to measure how often an example remains correct under random pruning, enabling comprehensive evaluation across different levels of selection difficulty. To compare efficiency fairly, we introduce Preservation-Constrained Speedup (PCS) to report the maximum end-to-end speedup each method achieves under a shared preservation target. Experiments across 8 methods, 3 VLM backbones, and 9 benchmarks show that stronger token compression does not necessarily yield greater speedup, method differences widen on examples where token selection matters, and efficiency rankings change with the preservation target.