Diffusion vision-language models (DVLMs) process their entire input at every denoising step. Since visual tokens make up 89-97% of this input on the benchmarks we study, most of the decoding computation is spent repeatedly processing the same image. Visual token pruning methods partially relieve this burden by removing part of the visual tokens, but they lose the removed information and still process the remaining image at every step. To measure how often the image is actually needed, we compare LLaDA-V's predictions with and without the image at every decoding step and find that removing the image changes a token being unmasked at only 22.8% of steps on average and at as few as 7.9%. We therefore propose LOWN, a decoding method that decides at every step whether to process the image and how many tokens to unmask. LOWN predicts the masked tokens without the image and processes the image only when this prediction disagrees with the last image-conditioned prediction. Agreement certifies the earlier prediction for reuse, and the number of certified tokens sets how many are unmasked in that step. We further observe that the need for the image differs across benchmarks, so LOWN's adaptive decision is needed rather than a fixed schedule. Without training or a tuned threshold, LOWN retains 99.6% of LLaDA-V's score on ten benchmarks at 35.3% of the decoding FLOPs on average. LOWN also generalizes across LLaDA-based DVLMs, composes orthogonally with visual token pruning, and runs under key-value caching for further efficiency gains. Code and project page are available at https://hyeon9698.github.io/LOWN/
Figures & tables
Figure 1: Look Only When Needed (LOWN). (a) Decoding with a diffusion VLM. Token pruning reduces the visual tokens processed during decoding, whereas LOWN decides at each step whether the visual tokens need to be processed at all and otherwise runs a TEXT-ONLY forward. (b) Composition of the input, averaged over ten benchmarks. Pruning reduces how many visual tokens each forward processes, LOWN whether a forward processes them at all. (c) Decoding FLOPs speedup. Each pruning method is shown at LOWN’s score (Appendix C.3 ), and LOWN + Token Pruning uses RedVTP ( r=50% ). (d) Quality relative to LLaDA-V when every method is run at the operating point closest to LOWN’s decoding FLOPs from above. LOWN keeps 99.6% on average.
Figure 2: Visual dependence along the decoding trajectory of LLaDA-V. (a) Share of masked positions at which the FULL and TEXT-ONLY top-1 predictions disagree, over token position (front to back) and denoising step (early to late). (b) Share of steps on which the image changed at least one unmasked decision, per benchmark, with 95% confidence intervals.
Figure 3: Overview of LOWN. TEXT-ONLY agreement certifies anchor tokens for image-free unmasking, and an insufficient certified prefix triggers a FULL forward that refreshes the anchor and unmasks the positions where the two forwards disagree first.
Figure 4: Performance and efficiency on LLaDA-V . Gray dashed lines denote the baseline. Triangles and squares denote RedVTP and ZOO-Prune retention settings, respectively. Arrows connect each fixed-pruning setting to its composition with LOWN. KLASS is matched to LOWN’s FLOPs.
Math
Science
Chart
MM
Doc
Cap
LLaVA-
DeCap
Method
Vista
QA
QA
MME
Bench
VQA
Arena
Bench
GQA
Bench
Avg.
LLaDA-V (Vanilla)
59.10
76.50
78.16
1719.49
70.88
83.91
− 88.67
58.20
53.31
33.18
100.0%
FLOPs matched to LOWN
51.70
70.70
74.32
1469.78
56.70
83.39
− 91.00
56.90
50.97
33.03
93.0%
RedVTP
( × 0.354)
( × 0.286)
( × 0.641)
( × 0.256)
( × 0.280)
( × 0.520)
( × 0.411)
( × 0.505)
( × 0.260)
( × 0.411)
( × 0.392)
53.70
73.87
76.44
1582.41
67.27
78.73
− 93.17
54.30
50.16
31.02
94.2%
Table 1: Performance at matched decoding FLOPs on LLaDA-V . The upper block compares standalone methods at LOWN’s budget, and the lower block at the budget of LOWN with RedVTP ( r=50% ). Competitors run on a discrete hyperparameter grid, so each cell uses the cheapest setting with at least LOWN’s FLOPs (Appendix C.3 ). Cells show scores with FLOPs relative to LLaDA-V below. Best scores are bold, second best underlined, and lowest FLOPs bold.
Method
MathVista
LLaVA-Bench
DeCapBench
Score ↑
Time (s) ↓
FLOPs ↓
Score ↑
Time (s) ↓
FLOPs ↓
Score ↑
Time (s) ↓
FLOPs ↓
LLaDA-V
59.10
7.56
1.0000
58.20
6.99
1.0000
33.18
14.64
1.0000
Fast-dLLM
57.30
2.14
0.0856
63.00
2.02
0.0876
35.19
4.22
0.1104
Fast-dLLM + LOWN
57.10
1.23
( − 42.4%)
0.0457
( − 46.6%)
58.00
1.25
( − 38.1%)
0.0489
( − 44.1%)
33.35
2.10
( − 50.1%)
0.0453
( − 58.9%)
Table 2: Composition of LOWN and Fast-dLLM on LLaDA-V.
Figure 5: Performance and efficiency on LaViDa . Gray dashed lines denote the baseline. Triangles, squares, and diamonds denote RedVTP, ZOO-Prune, and D 3 ToM, respectively. Composition markers use RedVTP at 75% and ZOO-Prune at 33% retention.
EgoSchema
MLVU
Method
Score ↑
FLOPs ↓
Score ↑
FLOPs ↓
VidLaDA
60.6
1.000
41.8
1.000
LOWN (Ours)
60.2
0.255
( − 74.5%)
43.3
0.272
( − 72.8%)
Table 3: Video-understanding results of LOWN on VidLaDA-8B.
MathVista
LLaVA-Bench
DeCapBench
LOWN (Ours)
56.00 ( × 0.325)
57.40 ( × 0.454)
34.43 ( × 0.391)
Period 2
57.00 ( × 0.541)
56.10 ( × 0.516)
33.43 ( × 0.522)
Period 4
52.00 ( × 0.321)
49.00 ( × 0.274)
27.79 ( × 0.284)
Period 8
48.50 ( × 0.211)
41.30 ( × 0.154)
19.09 ( × 0.164)
Random
53.40 ( × 0.349)
53.80 ( × 0.456)
30.48 ( × 0.389)
Table 4: Comparison of LOWN with fixed and random image-access schedules on LLaDA-V. Cells report score and relative FLOPs.
Adaptive
Math
Science
Chart
MM
Doc
Cap
LLaVA-
DeCap
Visual routing
acceptance
Vista
QA
QA
MME
Bench
VQA
Arena
Bench
GQA
Bench
✗
✗
59.10
76.50
78.16
1719.5
70.88
83.91
− 88.67
58.20
53.31
33.18
[0.5pt/1.5pt]
57.60
77.39
77.76
1624.6
68.30
83.50
− 86.83
57.90
52.53
33.02
✓
✗
0.5040 ( − 49.6%)
0.3554 ( − 64.5%)
0.7330 ( − 26.7%)
0.3383 ( − 66.2%)
0.3759 ( − 62.4%)
0.6743 ( − 32.6%)
0.6833 ( − 31.7%)
0.7002 ( − 30.0%)
0.3203 ( − 68.0%)
0.6840 ( − 31.6%)
[0.5pt/1.5pt]
56.00
78.83
78.36
1655.1
69.93
82.31
− 85.67
57.40
53.24
34.43
✓
✓
0.3252 ( − 35.5%)
0.2428 ( − 31.7%)
0.5759 ( − 21.4%)
0.2307 ( − 31.8%)
0.2701 ( − 28.1%)
0.4517 ( − 33.0%)
0.3885 ( − 43.1%)
0.4543 ( − 35.1%)
0.2045 ( − 36.2%)
0.3909 ( − 42.9%)
Table 5: Ablation of visual routing and adaptive acceptance on LLaDA-V. Each setting reports scores and, below, relative FLOPs with the reduction from the preceding setting in parentheses.
Figure 6: Qualitative comparison with visual token pruning (RedVTP). In LOWN’s output, blue underlines mark tokens unmasked on FULL forward steps, while the remaining tokens are unmasked on TEXT-ONLY forward steps.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Benchmark
Visual tokens (%)
Unchanged decisions
Image-dependent steps
Already top-1
MathVista
89.0
89.0 [88.6, 89.3]
17.9 [17.2, 18.5]
90.2 [89.9, 90.6]
ScienceQA
91.6
91.8 [91.0, 92.5]
8.2 [7.5, 9.0]
88.2 [87.2, 89.0]
ChartQA
96.1
75.5 [74.2, 76.7]
24.5 [23.3, 25.8]
88.4 [87.4, 89.3]
MME
97.0
89.4 [88.5, 90.3]
10.6 [9.7, 11.5]
89.6 [88.7, 90.4]
MMBench
91.3
92.1 [91.3, 92.8]
7.9 [7.2, 8.7]
88.6 [87.6, 89.5]
DocVQA
96.6
71.8 [70.9, 72.7]
43.8 [42.4, 45.2]
86.4 [85.7, 87.1]
Appendix
Table 6: Per-benchmark observations on LLaDA-V. Visual tokens is their mean percentage of the FULL input. Unchanged decisions is the share of unmasked positions at which the FULL and TEXT-ONLY top-1 predictions agree. Image-dependent steps is the share of steps on which the image changed at least one unmasked decision. Already top-1 is the share of final tokens that were the top-1 prediction at their position before the step that unmasked them. Brackets give 95% confidence intervals, and the last row is the unweighted mean.
Figure 7: Where and when the image changes a prediction, all ten benchmarks. Share of masked positions at which the FULL and TEXT-ONLY top-1 predictions disagree, over token position (front to back) and denoising step (early to late), as in Figure 2(a) .
Benchmark
Visual tokens (%)
Unchanged decisions
Image-dependent steps
Already top-1
MathVista
71.1
94.0 [93.8, 94.1]
10.4 [10.1, 10.7]
89.8 [89.6, 90.0]
ScienceQA
76.4
98.7 [98.6, 98.8]
1.3 [1.2, 1.4]
93.7 [93.5, 94.0]
ChartQA
88.8
83.6 [83.2, 84.0]
16.4 [16.0, 16.8]
92.6 [92.4, 92.9]
MME
87.9
96.1 [95.9, 96.3]
3.9 [3.7, 4.1]
93.7 [93.5, 94.0]
MMBench
78.4
96.8 [96.6, 96.9]
3.2 [3.1, 3.4]
93.7 [93.5, 93.9]
DocVQA
88.3
89.4 [89.3, 89.6]
15.2 [14.9, 15.4]
91.6 [91.4, 91.7]
Appendix
Table 7: Per-benchmark observations on LaViDa, in the format of Table 6 , with the response lengths and step counts of the LLaDA-V measurement and over the full evaluation sets. Brackets give 95% confidence intervals, and the last row is the unweighted mean.
Benchmark
Samples
Mean ρ (%)
MathVista
1,000
10.07
ScienceQA
2,017
7.92
ChartQA
2,500
2.54
MME
2,374
3.15
MMBench
4,329
7.04
DocVQA
5,349
3.02
Appendix
Table 8: Mean TEXT-ONLY/FULL forward-cost ratio on LLaDA-V. For each sample, we compute ρi=C(pi+oi)/C(pi+vi+oi) using its prompt length pi , visual-token count vi , and configured response length oi . Each row averages these ratios over the benchmark’s samples, and C is the analytical forward cost in Equation 4 .
MathVista
ScienceQA
ChartQA
MME
MMBench
DocVQA
CapArena
LLaVA-B.
GQA
DeCapB.
FLOPs matched to LOWN
RedVTP r
30
20
60
20
20
50
40
50
20
40
ZOO-Prune r
30
20
80
30
20
60
50
60
20
50
KLASS (τ,κ)
(0.2, 0.2)
(0.2, 1)
(0.2, 1)
(0.2, 0.3)
(0.2, 0.4)
(0.2, 0.1)
(0.2, 0.5)
(0.2, 1)
(0.2, 1)
(0.2, 0.4)
FLOPs matched to LOWN + RedVTP ( r=50% )
RedVTP r
20
10
30
10
20
25
20
25
10
20
Appendix
Table 9: Settings behind the competitor cells of Table 1 (FLOPs matched) and Table 10 (time matched). For RedVTP, r is the visual-token retention percentage. For ZOO-Prune, the released code sets the budget to r% of a fixed 2,880 tokens rather than of each image’s visual tokens, so the fraction actually retained varies with the image. For KLASS, (τ,κ) denotes its confidence and KL thresholds. The second and fourth blocks use the budget of LOWN composed with RedVTP at r=50% .
Math
Science
Chart
MM
Doc
Cap
LLaVA-
DeCap
Method
Vista
QA
QA
MME
Bench
VQA
Arena
Bench
GQA
Bench
Avg.
LLaDA-V (Vanilla)
59.10
76.50
78.16
1719.49
70.88
83.91
− 88.67
58.20
53.31
33.18
100.0%
Time matched to LOWN
51.70
65.49
72.20
1275.08
40.12
83.39
− 88.50
57.60
48.05
33.88
88.7%
RedVTP
( × 0.394)
( × 0.374)
( × 0.674)
( × 0.300)
( × 0.366)
( × 0.566)
( × 0.507)
( × 0.612)
( × 0.318)
( × 0.504)
( × 0.461)
53.70
72.48
76.00
1582.41
68.73
78.73
− 94.67
56.70
50.50
31.15
94.6%
Appendix
Table 10: Performance at matched generation time on LLaDA-V . The upper block compares standalone methods at LOWN’s generation time, and the lower block at that of LOWN with RedVTP ( r=50% ). Competitors run on a discrete hyperparameter grid, so each cell uses the setting with the shortest measured time at or above LOWN’s (Table 9 ), and LOWN gives the highest normalized average in both blocks. Cells show scores with mean generation time relative to LLaDA-V below. Best scores are bold, second best underlined, and lowest times bold.
Configuration before LOWN
FLOPs ratio ↓
Reduction ↑
Quality index ↑
LLaDA-V
LLaDA-V (unpruned)
0.353
64.7%
99.6
RedVTP (50% retention)
0.424
57.6%
100.9
ZOO-Prune (33% retention)
0.394
60.6%
98.7
LaViDa
LaViDa (unpruned)
0.174
82.6%
99.7
Appendix
Table 11: Relative gains from adding LOWN on LLaDA-V and LaViDa. Each row is measured against the listed configuration without LOWN. FLOPs ratio is the mean of the per-benchmark FLOPs ratios, and reduction is one minus this mean. Quality is the mean normalized index with the reference set to 100.
CapArena
LLaVA-Bench
DeCapBench
Method
Score ↑
FLOPs ↓
Score ↑
FLOPs ↓
Score ↑
FLOPs ↓
LLaDA-V ( T steps)
− 88.67
1.000
58.20
1.000
33.18
1.000
LLaDA-V ( T/2 )
− 92.33
0.500
45.40
0.500
32.17
0.500
LLaDA-V ( T/4 )
− 99.00
0.250
26.20
0.250
25.84
0.250
LOWN (Ours)
− 85.67
0.388
57.40
0.454
34.43
0.391
Appendix
Table 12: LLaDA-V with fewer denoising steps on CapArena, LLaVA-Bench, and DeCapBench. T/2 and T/4 run half and a quarter of the baseline’s T steps on the inputs of Table 1 . Scores use the judges of Table 1 , and FLOPs are relative to LLaDA-V. DeCapBench averages the captions its official evaluator judges valid, 399, 398, 392, and 399 of 400 for the four rows.
Figure 8: Performance and efficiency on LaViDa, all ten benchmarks. The markers follow Figure 5 .
Math
Science
Chart
MM
Doc
Cap
LLaVA-
DeCap
Method
Vista
QA
QA
MME
Bench
VQA
Arena
Bench
GQA
Bench
Avg.
LaViDa (Vanilla)
43.80
72.34
67.16
1353.03
76.95
63.54
− 93.17
38.80
56.69
30.18
100.0%
FLOPs matched to LOWN
36.60
71.34
20.40
987.28
72.23
18.55
− 99.33
28.60
50.14
22.58
73.9%
RedVTP
r=25% ( × 0.432)
r=25% ( × 0.448)
r=10% ( × 0.244)
r=10% ( × 0.248)
r=25% ( × 0.435)
r=10% ( × 0.225)
r=20% ( × 0.347)
r=10% ( × 0.226)
r=25% ( × 0.367)
r=25% ( × 0.389)
( × 0.336)
39.20
71.05
39.56
1220.76
73.76
30.62
− 95.67
30.20
52.23
28.55
84.3%
Appendix
Table 13: Performance at matched decoding FLOPs on LaViDa , in the format of Table 1 . Methods are compared at LOWN’s budget. Each cell shows the benchmark score, with the selected retention r and FLOPs relative to the LaViDa baseline below. Competitors run on a discrete grid of retention ratios, so each uses its cheapest measured setting whose FLOPs are at least LOWN’s (Appendix C.3 ). D 3 ToM uses constant 10% retention on MathVista, LLaVA-Bench, CapArena, and DeCapBench and its adaptive schedule on the remaining tasks. MME reports the perception score. Best scores are bold, second best underlined, and lowest FLOPs bold.
Figure 9: Additional qualitative comparisons with visual token pruning (RedVTP). The upper three examples are short-answer questions, and the lower three are open-ended captioning. Green marks correct answers and content consistent with the image, and red marks incorrect answers and content inconsistent with the image.
Figure 10: Qualitative comparison with visual token pruning (RedVTP) and its pruned image. The upper right darkens the visual tokens removed by RedVTP, and the inset enlarges the region around the x-axis tick 4.0. In LOWN’s output, blue underlines mark tokens unmasked on FULL forward steps, while the remaining tokens are unmasked on TEXT-ONLY forward steps.
Figure 11: A limitation example on DocVQA. In LOWN’s output, blue underlines mark tokens unmasked on FULL forward steps, while the remaining tokens are unmasked on TEXT-ONLY forward steps. FLOPs are relative to LLaDA-V on this input.
Adaptive acceptance
Math
Science
Chart
MM
Doc
Cap
LLaVA-
DeCap
Visual routing
Disagreement
Whole prefix
Vista
QA
QA
MME
Bench
VQA
Arena
Bench
GQA
Bench
✗
✗
✗
59.10
76.50
78.16
1719.5
70.88
83.91
− 88.67
58.20
53.31
33.18
[0.5pt/1.5pt]
57.60
77.39
77.76
1624.6
68.30
83.50
− 86.83
57.90
52.53
33.02
✓
✗
✗
0.5040 ( − 49.6%)
0.3554 ( − 64.5%)
0.7330 ( − 26.7%)
0.3383 ( − 66.2%)
0.3759 ( − 62.4%)
0.6743 ( − 32.6%)
0.6833 ( − 31.7%)
0.7002 ( − 30.0%)
0.3203 ( − 68.0%)
0.6840 ( − 31.6%)
[0.5pt/1.5pt]
57.10
78.19
78.32
1688.5
69.85
82.33
− 81.17
57.50
53.43
34.71
✓
✓
✗
0.3910 ( − 22.4%)
0.3006 ( − 15.4%)
0.5891 ( − 19.6%)
0.2755 ( − 18.6%)
0.3236 ( − 13.9%)
0.4789 ( − 29.0%)
0.4471 ( − 34.6%)
0.5067 ( − 27.6%)
0.2469 ( − 22.9%)
0.4450 ( − 34.9%)
Appendix
Table 14: Table 5 with adaptive acceptance split into disagreement-first order on FULL steps and whole-prefix acceptance on TEXT-ONLY steps (Section 4.2 ), on LLaDA-V. Each setting reports scores and, below, relative FLOPs with the reduction from the preceding setting in parentheses.
Benchmark
Coverage
ϵ(1)
ϵ(2)
ϵ(4)
ϵ(8)
Rref
Accept. error
RLOWN
MathVista
58.6
7.3
9.5
12.2
15.4
0.284
2.2
0.325
ScienceQA
63.5
6.4
8.3
9.2
8.8
0.218
4.7
0.243
ChartQA
47.0
10.6
14.0
16.6
15.2
0.490
4.2
0.576
MME
63.8
5.9
7.6
10.0
13.6
0.193
3.9
0.231
MMBench
65.3
6.3
7.3
7.5
6.9
0.217
3.9
0.270
DocVQA
43.4
11.0
15.5
20.2
25.3
0.517
3.0
0.452
Appendix
Table 15: Certificate reliability on LLaDA-V. Eligible positions are the masked positions that were already masked Δ steps earlier. Coverage is the share of eligible positions whose TEXT-ONLY prediction agrees with a one-step-old anchor. ϵ(Δ) is the share of agreeing positions at which a fresh FULL forward predicts a different token when the anchor is Δ steps old. Rref=f∗+(1−f∗)ρ+1/T is the cost of the oracle schedule of Section 5.3 , which runs a FULL forward only on the fraction f∗ of steps where the image changes an unmasked decision and pays ρ elsewhere, plus one initial FULL forward. For ChartQA, whose baseline stops at the end-of-sequence token after 7.0 of its 16 steps on average, f∗ and T are taken over the steps the baseline actually runs ( f∗=32.3% ). Acceptance error is the share of the tokens LOWN unmasks on TEXT-ONLY steps that a fresh FULL forward on the state they were accepted from would predict differently, and RLOWN is LOWN’s measured cost. Costs are relative to the baseline and rates are in percent.
Benchmark
Age 1
Age ≤2
Max age
MathVista
93.8
99.5
5
ScienceQA
92.9
99.4
3
ChartQA
94.9
99.9
3
MME
94.2
99.5
4
MMBench
91.8
99.4
3
DocVQA
96.7
99.6
4
Appendix
Table 16: Anchor age of the tokens LOWN accepts on TEXT-ONLY steps, in LOWN’s own runs on LLaDA-V. Age is the number of LOWN steps since the FULL forward that produced the anchor. Shares are in percent, and the last row pools the accepted tokens of all benchmarks.
Visual token pruning reduces the computational cost of Vision-Language Models (VLMs) by removing redundant visual tokens. The key is to learn a score that measures whether a token is useful. Existing methods typically rely on Gumbel-Softmax to approximate discrete selection during training. Such selectors make the score depend on the behavior of a relaxed pruning operator, not directly on the consequence of information loss. In this paper, we propose DiffPrune, which gives token scores a direct meaning. During training, DiffPrune keeps all tokens and weakens each token's information according to its score. If weakening a token hurts the task, the scorer is pushed to protect it; if not, the token can receive a lower score. Because the loss is differentiated through this actual information-throttling path, the scorer avoids the unstable surrogate path of relaxed token selection. DiffPrune implements this idea with an Information Throttler, which injects variance-preserving noise into visual tokens, where high-score tokens remain close to their original representations, while low-score tokens carry less original information. At inference, the throttler is removed, and hard top-K pruning is applied using the learned scores. Across ten VLM benchmarks, DiffPrune retains 96.5% of full-model accuracy while accelerating LLM prefill by 2.85x, with only 0.69 ms inference overhead. Code will be publicly available.
Landi He, Mingde Yao, Shawn Young +1
Shenzhen University of Advanced Technology · CUHK MMLab, CPII under InnoHK
Diffusion vision-language models (dVLMs) iteratively denoise masked responses while conditioning each denoising step on visual evidence, making visual conditioning a substantial recurring inference cost. Unlike autoregressive decoding, diffusion generation repeatedly revisits the entire response as uncertainty evolves. Our analysis reveals that visual evidence demand is strongly step-dependent, motivating adaptive allocation across denoising steps. Existing inference acceleration methods operate through decoding-side strategies or visual token compression via pruning and merging, but do not explicitly treat visual evidence as a resource whose demand evolves across the diffusion process. Therefore, we present Denoising-Aware Visual Evidence Trajectory Allocation (DAVET), a training-free framework that allocates visual evidence according to the evolving generation state. Starting from a phase-conditioned evidence trajectory, the proposed allocation policy uses operation demand to set an evidence reserve whose allocation at each denoising step is modulated by trajectory risk. DAVET realizes the resulting budgets through a hierarchy of evidence views constructed from a single visual encoding, separating when and how much evidence is needed from how the evidence views are constructed. Evaluated on two representative dVLMs, LLaDA-V and LaViDa, across multiple visual-understanding benchmarks, DAVET achieves an average speedup of 1.55× with an average relative performance drop of 1.86%, showing that denoising-aware visual evidence allocation can reduce visual conditioning cost while largely preserving generation quality.
Yongkang Zhou, Xiang Xia, Cheng Yan +2
University of Science and Technology of China Hefei, China
Large Vision-Language Models (LVLMs) face significant computational inefficiencies caused by the large number of visual tokens. Existing visual token pruning methods mainly focus on either retaining individually important tokens or selecting mutually diverse ones. In this work, we revisit visual token pruning from a coverage perspective and formulate it as a biased attention coverage maximization problem. The key idea is to select a compact token subset whose encoder-side outgoing attention can jointly cover the image while assigning higher coverage priority to more informative regions. From this perspective, we propose ACPruner, a training-free visual token pruning framework for efficient LVLM inference. ACPruner first estimates token importance by combining intra-modal saliency and inter-modal relevance, then derives token-wise coverage from attention patterns within the vision encoder, and finally performs greedy selection to maximize the proposed coverage objective. Extensive experiments across multiple LVLM backbones, including LLaVA-1.5-7B/13B, LLaVA-NeXT-7B/13B, Qwen2.5-VL-7B, and LLaVA-OneVision-7B, show that ACPruner consistently achieves strong performance retention while delivering substantial end-to-end inference speedups.
Xu Li, Yuxuan Liang, Yi Zheng +6
College of Computer Science and Artificial Intelligence Fudan University, Shanghai, China