STAR-Pro: Stage-Wise Token Adaptive Reduction with Progressive Refinement for Efficient Large Vision-Language Models

Yichen Guo; Tinghao Wang; Qizhe Zhang; Lingbei Meng; Yuan Zhang; Jiajun Cao; Hao Jiang; Chenwei Wu

cs.CVSep 5, 2026

STAR-Pro: Stage-Wise Token Adaptive Reduction with Progressive Refinement for Efficient Large Vision-Language Models

Authors: Yichen Guo, Tinghao Wang, Qizhe Zhang, Lingbei Meng, Yuan Zhang, Jiajun Cao, Hao Jiang, Chenwei Wu,

Organizations: State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Nanyang Technological University, Singapore · University of Electronic Science and Technology of China · De Artificial Intelligence Lab ·

Abstract

Large vision-language models (LVLMs) achieve strong multimodal understanding, but the hundreds to thousands of visual tokens they process impose substantial computational overhead, motivating training-free visual token pruning. In this work, we conduct two complementary analyses of visual token pruning. First, we measure the feature-space coverage of tokens retained before cross-modal fusion and find that aggressive pruning discards substantial visual information. Second, we track text-to-visual attention across decoder layers and find that the visual tokens considered important change substantially with depth, making one-shot pruning decisions unreliable. Together, these findings show that effective pruning should preserve broad visual coverage before fusion and progressively refine the retained tokens as cross-modal evidence evolves during fusion. We therefore propose STAR-Pro (STage-Wise Adaptive Token Reduction with Progressive Refinement), a training-free two-stage framework. Its Adaptive Stage applies pivoted QR to construct an over-budget feature-coverage candidate pool, while its Progressive Stage uses evolving text-to-visual attention at selected decoder layers to prune a nested survivor set under a target layer-average token budget. Extensive experiments across seven LVLMs spanning multiple architectures and 18 image and video benchmarks demonstrate the effectiveness of STAR-Pro under aggressive pruning. On LLaVA-Video-7B, STAR-Pro reduces visual tokens by 90.5%, retains 92.7% of baseline performance, and achieves a $2.24\times$ measured inference speedup. Code is available at https://github.com/EasonAI-5589/starpro.

Figures & tables

Figure 1: STAR-Pro at ${\sim}90\%$ visual token pruning. Each radar axis is normalised to the strongest plotted method (100); below each panel: STAR-Pro vs. the strongest baseline in the corresponding main table, both as percentages of baseline performance.

Figure 2: Empirical Study. (a) Empirical Study on Adaptive Stage. We measure the visual feature retention ratio $\eta{=}\|P_{S}X\|_{F}^{2}/\|X\|_{F}^{2}$ of the $K$ retained tokens. Across all budgets, STAR-Pro preserves more visual information than CDPruner and DivPrune, with a larger advantage under more aggressive pruning. The PCA insets visualize the first six selections of each method for one image: STAR-Pro spans the largest region of the feature space and reaches the most distinct feature clusters. (b) Empirical Study on Progressive Stage. At each decoder layer $\ell$ , we measure the IoU continuity of the top- $K$ attended set $S_{\ell}$ with layers a distance $d$ away, defined as $\tfrac{1}{2}[\mathrm{IoU}(S_{\ell},S_{\ell-d})+\mathrm{IoU}(S_{\ell},S_{\ell+d})]$ . Across gaps $d{=}1$ – $5$ , continuity stays well below $1$ , confirming visual token importance changes with depth (LLaVA-1.5-7B, $K{=}64$ , mean over 500 MME images).

Figure 3: Overview of STAR-Pro. The Adaptive Stage (§ 3.2 ) uses pivoted QR to build a feature-coverage candidate pool. At scheduled pruning layers, the Progressive Stage (§ 3.3 ) uses evolving text-to-visual attention to progressively refine the pool.

Method	LLaVA-1.5-7B			LLaVA-1.5-13B			LLaVA-NeXT-7B			LLaVA-NeXT-13B
Method	576 tokens			576 tokens			2880 tokens			2880 tokens
Compression	77.8%	88.9%	94.4%	77.8%	88.9%	94.4%	77.8%	88.9%	94.4%	77.8%	88.9%	94.4%
FastV (ECCV24)	92.5	75.6	–	94.1	85.0	–	95.4	80.4	–	96.4	86.9	–
PDrop (CVPR25)	94.8	75.2	–	97.8	88.7	–	97.0	83.1	–	98.5	91.5	–
SparseVLM (ICML25)	96.4	87.8	–	98.0	93.1	–	98.9	94.0	–	99.9	96.8	–
VisionZip (CVPR25)	96.7	93.1	87.9	96.3	93.1	87.4	99.4	95.8	90.5	94.8	96.4	92.6

Table 1: Evaluation on LLaVA Series. All numbers report Rel. (%), performance relative to the corresponding baseline. Ties at the displayed one-decimal precision are marked jointly best. “–”: not applicable.

Method	MLVU	LVB	VMME	Acc.	Rel.
Retain $64\times 32$ Tokens ( $\downarrow$ 81.1%)
FastV (ECCV24)	58.5	52.4	56.0	55.6	87.7
SparseVLM (ICML25)	60.7	53.7	59.0	57.8	91.1
CDPruner (NeurIPS25)	63.0	56.5	60.5	60.0	94.6
DivPrune (CVPR25)	61.5	56.4	59.3	59.1	93.1
VScan (TMLR26)	62.4	57.8	60.9	60.4	95.2

Table 2: Evaluation on Video. MLVU/LVB/VMME report m-avg/val/w/o-subtitle. Acc. is the equal average of the three reported benchmark scores; Rel. (%) normalizes Acc. by the unpruned baseline’s three-benchmark mean (63.4).

Method	AI2D	POPE	HallBench	MME	MMB ${}^{\text{EN}}$	MMB ${}^{\text{CN}}$	MMStar	SQA	Acc.	Rel.
Qwen3-VL-8B-Instruct (Total: 1,296 Tokens)
Baseline	84.1	89.7	56.2	2406	86.3	86.1	67.1	94.6	85.6	100
Layer-Average Budget $T{=}256$ ( $\downarrow$ 80.2%)
FastV (ECCV24)	69.3	81.7	43.0	1913	77.3	76.7	51.1	82.0	72.1	84.3
CDPruner (NeurIPS25)	75.1	87.6	41.9	2034	79.7	78.7	50.9	83.7	74.9	87.6
DivPrune (CVPR25)	80.0	89.4	46.9	2236	83.6	82.3	58.3	88.0	80.0	93.6

Table 3: Evaluation on Advanced Architectures. Budgets are layer-average visual-token counts. Acc. denotes the average across eight benchmarks (MME divided by 20); Rel. (%) is performance relative to the corresponding baseline.

Timed component	Latency (ms)		$\Delta$
	Full	STAR-Pro	ms	%
Vision encoder + projector	662.8	659.1	3.7	0.37
Adaptive Stage: pivoted QR	0.0	40.4	$-40.4$	$-3.99$
Auxiliary selection ops.	0.0	0.2	$-0.2$	$-0.02$
Decoder prefill	968.8	95.9	872.9	86.15
Residual runtime	188.9	11.9	177.0	17.47

Table 4: Adaptive-Stage overhead on LLaVA-Video-7B at

T{=}16

. End-to-end speedup: 2.24

\times

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Decoder layers	Nominal budget $T$	Adaptive pool $2T$	Progressive checkpoints
LLaVA-1.5-7B	32	128 / 64 / 32	256 / 128 / 64	$\ell_{12}{\rightarrow}74/37/17,\ \ell_{20}{\rightarrow}36/18/10$
LLaVA-1.5-13B	40	128 / 64 / 32	256 / 128 / 64	$\ell_{15}{\rightarrow}64/32/16,\ \ell_{30}{\rightarrow}32/16/8$
LLaVA-NeXT-7B	32	640 / 320 / 160	1,280 / 640 / 320	$\ell_{12}{\rightarrow}367/182/91,\ \ell_{20}{\rightarrow}182/92/46$
LLaVA-NeXT-13B	40	640 / 320 / 160	1,280 / 640 / 320	$\ell_{15}{\rightarrow}320/160/80,\ \ell_{30}{\rightarrow}160/80/40$
LLaVA-Video-7B	28	32 / 16	64 / 32	$\ell_{12}{\rightarrow}11/5,\ \ell_{21}{\rightarrow}4/3$
Qwen3-VL-8B-Instruct	36	256 / 128	512 / 256	$\ell_{14}{\rightarrow}128/64,\ \ell_{24}{\rightarrow}64/32$

Appendix

Table 5: Architecture-specific pruning checkpoints used by STAR-Pro. Decoder layers are named one-based: an entry

\ell{\rightarrow}m

prunes after layer

\ell

, so the following layer receives

m

visual tokens. In the zero-based implementation this is the boundary before decoder index

\ell

. Each schedule starts from the

2T

candidate pool produced by the Adaptive Stage. Slash-separated survivor counts correspond in order to the slash-separated budgets in the same row. For LLaVA-Video, token counts are reported per frame; integer allocation gives observed layer averages of 31.96 and 16.07 for the nominal 32- and 16-token budgets.

Method	SEED	GQA	SQA ${}^{\text{IMG}}$	VQA ${}^{\text{Text}}$	POPE	MME	MMB ${}^{\text{EN}}$	MMB ${}^{\text{CN}}$	AI2D	MMMU	Acc.	Rel.
Upper Bound: All 576 tokens (100%)
Baseline	66	61.9	69.5	58.2	85.9	1508.3	64.7	58.1	55.5	35	63.02	100.0
Retain 128 Tokens ( $\downarrow$ 77.8%)
FastV (ECCV24)	59.66	54	69.2	56.4	68.2	1376.5	63.0	55.9	53.6	34.3	58.31	92.5
PDrop (CVPR25)	59.74	57.1	70.1	56.7	77.5	1434.1	62.3	55.3	52.7	34.1	59.72	94.8
SparseVLM (ICML25)	61.72	57.3	69.0	56.3	83.1	1401.3	62.6	56.9	54.7	35.9	60.76	96.4

Appendix

Table 6: Performance comparison on LLaVA-1.5-7B. Acc. denotes the average across 10 benchmarks, with MME divided by 20. Rel. (%) is performance relative to the corresponding baseline.

Method	SEED	GQA	SQA ${}^{\text{IMG}}$	VQA ${}^{\text{Text}}$	POPE	MME	MMB ${}^{\text{EN}}$	MMB ${}^{\text{CN}}$	AI2D	MMMU	Acc.	Rel.
Upper Bound: All 576 tokens (100%)
Baseline	68.22	63.3	72.8	61.2	86	1531.2	68.5	63.5	60.8	36.4	65.73	100.0
Retain 128 Tokens ( $\downarrow$ 77.8%)
FastV (ECCV24)	55.5	58.3	74.2	58.6	75.5	1460.6	66.1	62.3	58.1	36.9	61.85	94.1
PDrop (CVPR25)	65.6	61	73.3	60.2	83.4	1494.3	66.7	62.7	59.1	36.1	64.28	97.8
SparseVLM (ICML25)	65.3	59.6	74.3	59.3	85	1487.9	68.4	62.6	58.1	37.4	64.44	98.0

Appendix

Table 7: Performance comparison on LLaVA-1.5-13B. Acc. denotes the average across 10 benchmarks, with MME divided by 20. Rel. (%) is performance relative to the corresponding baseline.

Method	SEED	GQA	SQA ${}^{\text{IMG}}$	VQA ${}^{\text{Text}}$	POPE	MME	MMB ${}^{\text{EN}}$	MMB ${}^{\text{CN}}$	AI2D	MMMU	Acc.	Rel.
Upper Bound: All 2880 tokens (100%)
Baseline	69.66	62.5	67.5	60.3	86.8	1511.8	65.8	57.3	64.7	35.2	64.54	100.0
Retain 640 Tokens ( $\downarrow$ 77.8%)
FastV (ECCV24)	64.25	58.9	67.4	58.1	79.5	1412.6	63.1	53.5	65.1	35	61.55	95.4
PDrop (CVPR25)	64.68	60	66.7	57.8	83.8	1475.9	64.1	55.2	64.6	35.1	62.58	97.0
SparseVLM (ICML25)	67.17	61.2	67.6	59.7	85.3	1456.8	65.9	58.6	64.7	35.1	63.81	98.9

Appendix

Table 8: Performance comparison on LLaVA-NeXT-7B. Acc. denotes the average across 10 benchmarks, with MME divided by 20. Rel. (%) is performance relative to the corresponding baseline.

Method	SEED	GQA	SQA ${}^{\text{IMG}}$	VQA ${}^{\text{Text}}$	POPE	MME	MMB ${}^{\text{EN}}$	MMB ${}^{\text{CN}}$	AI2D	MMMU	Acc.	Rel.
Upper Bound: All 2880 tokens (100%)
Baseline	71.58	64.4	73.1	63.2	85.3	1539.5	68.5	61.2	70.1	35.7	67.0	100.0
Retain 640 Tokens ( $\downarrow$ 77.8%)
FastV (ECCV24)	67.4	60.9	71.7	60.7	80.2	1516.7	65.5	59.9	67.7	35.8	64.6	96.4
PDrop (CVPR25)	68.8	62.8	71.7	62.1	84.4	1559.1	66.6	60.8	69	35.9	66.0	98.5
SparseVLM (ICML25)	69.7	62.7	72.5	62.8	85.6	1562.7	68.5	64.0	68.1	37.6	67.0	99.9

Appendix

Table 9: Performance comparison on LLaVA-NeXT-13B. Acc. denotes the average across 10 benchmarks, with MME divided by 20. Rel. (%) is performance relative to the corresponding baseline.

Method	MLVU	LongVideoBench			Video-MME				Acc.	Rel.
Metric	m-avg	val	perception	relation	w/o sub	short	medium	long
Baseline ( $64\times 169$ tokens, 100%)
Baseline	67.7	59.0	65.0	53.8	63.6	76.6	61.2	53.1	63.4	100.0
Retain $64\times 32$ Tokens ( $\downarrow$ 81.1%)
FastV (ECCV24)	58.5	52.4	57.0	48.5	56.0	63.8	55.9	48.4	55.6	87.7
SparseVLM (ICML25)	60.7	53.7	58.1	49.9	59.0	69.8	56.9	50.3	57.8	91.1

Appendix

Table 10: Full LLaVA-Video-7B results with 64 frames per video. Acc. uses the same three benchmark-level scores as the main paper (MLVU m-avg, LongVideoBench val, and Video-MME w/o subtitle); Rel. (%) normalizes Acc. by the corresponding unpruned mean (63.4). The remaining columns provide the associated sub-metrics.

Panel of Figure 6 — Figure 6: Pruning-schedule robustness on LLaVA-1.5-7B at $T{=}64$ . (a) Mean-4 over the 100 measured pairs, shown directly (left) and with the CDPruner Mean-4 reference plane at $68.1$ (right); the star marks the deployed $(12,20)$ schedule. (b) Cross-sections of the same surface, with regions at or above the reference shown in green and regions below it shown in red.

Figure 7: Robustness and stability to the candidate-pool multiplier

\alpha

. Each panel plots STAR-Pro’s aggregate Rel. (%) as

\alpha

sweeps

\{1,1.5,2,2.5,3\}

, for one model (rows) at one layer-averaged token budget (columns). The star marks

\alpha{=}2

, the deployed configuration used throughout the paper. All plotted points—including the

\alpha{=}2

star—are the per-model candidate-multiplier sweep values reported in Tables 13 , 14 , 15 and 16 , where the

\alpha{=}2

row is highlighted in each. The dashed line is the best competing method at the same budget in the main comparison tables. Rel. is the 10-benchmark Acc. (MME/20) normalized by each model’s unpruned baseline; exact values are listed in those tables.

Arm	Configuration	$T{=}128$ ( $\downarrow$ 77.8%)				$T{=}64$ ( $\downarrow$ 88.9%)				$T{=}32$ ( $\downarrow$ 94.4%)
Arm	Configuration	POPE	GQA	VQA ${}^{\text{T}}$	MME	POPE	GQA	VQA ${}^{\text{T}}$	MME	POPE	GQA	VQA ${}^{\text{T}}$	MME
(a) Adaptive Stage selection operator ( $\alpha{=}2$ , fixed diagnostic Progressive schedule)
Random	no structure	85.13	59.84	54.21	1447.6	82.88	57.97	51.43	1372.5	77.40	54.64	48.17	1265.2
Stride	uniform grid	85.16	60.13	54.47	1471.3	82.94	57.73	51.08	1380.8	77.40	55.41	48.12	1221.2
(b) Number of Progressive Stage pruning layers $P$ ( $\alpha{=}2$ )
$P{=}0$	Adaptive only	87.21	59.68	56.79	1422.5	86.12	58.13	56.03	1384.7	83.94	55.72	54.28	1353.1

Appendix

Table 11: Matched-compute controls on LLaVA-1.5-7B. Every arm has the nominal layer-average visual-token budget

T

. Block (a) reports Random and Stride candidate pools under a fixed diagnostic Progressive schedule. Block (b) varies the number of Progressive-Stage pruning layers;

P{=}0

denotes Adaptive-only selection, and the listed

P\in\{3,4\}

placements are diagnostic equal-depth schedules. The Ours row uses query-agnostic pivoted-QR coverage with the deployed

P{=}2

\ell{=}(12,20)

schedule and reports the corresponding main-paper scores. MME is unnormalised and should be read alongside POPE, GQA, and VQA

{}^{\text{Text}}

Schedule	Config ( $2T{\to}K_{1}{\to}K_{2}$ )	SEED	GQA	SQA ${}^{\text{IMG}}$	VQA ${}^{\text{Text}}$	POPE	MME	MMB ${}^{\text{EN}}$	MMB ${}^{\text{CN}}$	AI2D	MMMU	Acc.	Rel.
LLaVA-1.5-7B — deployed pruning layers $\ell_{12},\ell_{20}$ , $\alpha{=}2$
Layer-average budget $T{=}128$ ( $\downarrow$ 77.8%)
Depl.	$256{\to}74{\to}36$	64.4	60.8	68.9	57.2	87.3	1456.3	62.4	57.7	55.0	36.7	62.32	98.9
Token-var.	$256{\to}80{\to}32$	64.4	60.8	68.7	57.4	87.2	1464.3	62.7	57.7	55.1	36.8	62.41	99.0
Layer-average budget $T{=}64$ ( $\downarrow$ 88.9%)
Depl.	$128{\to}37{\to}18$	62.87	59.41	68.62	56.26	87.52	1421.4	60.97	56.44	53.69	35.11	61.20	97.1

Appendix

Table 12: Robustness to the per-layer token allocation. For each model the pruning layers and

\alpha{=}2

are fixed at their deployed values; only the per-layer retained counts change. Depl. is the deployed schedule and Token-var. a moderately more front-heavy re-allocation (

K_{1}{:}K_{2}\approx 2.5{:}1

) with the same layers and the same exact layer-average budget. The Config column gives

2T{\to}K_{1}{\to}K_{2}

. Acc. is the average over the 10 benchmarks with MME divided by 20; Rel. (%) is Acc. relative to the corresponding unpruned baseline.

Config.	$T_{\text{avg}}$	SEED	GQA	SQA ${}^{\text{IMG}}$	VQA ${}^{\text{Text}}$	POPE	MME	MMB ${}^{\text{EN}}$	MMB ${}^{\text{CN}}$	AI2D	MMMU	Acc.	Rel.
Layer-average budget $T{=}128$ ( $\downarrow$ 77.8%)
$\alpha{=}1$	128	62.99	59.68	68.67	56.79	87.21	1388.6	61.31	56.19	54.05	35.44	61.18	97.1
$\alpha{=}1.5$	128	63.87	60.10	68.62	57.57	87.47	1421.7	62.16	57.39	54.66	36.00	61.89	98.2
STAR-Pro ( $\alpha{=}2$ )	128	64.4	60.8	68.9	57.2	87.3	1456.3	62.4	57.7	55.0	36.7	62.32	98.9
$\alpha{=}2.5$	128	65.07	58.31	68.47	54.95	87.06	1480.1	63.52	58.33	54.79	36.33	62.08	98.5
$\alpha{=}3$	144	64.10	50.99	68.52	50.61	80.00	1469.4	62.84	57.13	55.21	36.00	59.89	95.0

Appendix

Table 13: Candidate-multiplier sweep on LLaVA-1.5-7B with the two Progressive-Stage checkpoints fixed at

\{12,20\}

. The highlighted STAR-Pro (

\alpha{=}2

) row is the deployed configuration used throughout the paper; its scores are taken from the main comparison table ( Table 6 ).

T_{\text{avg}}

is the realized layer-average visual-token count (it exceeds

T

only where the strict average constraint is infeasible, at

\alpha{=}3

). Acc. is the 10-benchmark average with MME/20, and Rel. (%) normalizes Acc. by the full-token baseline.

Config.	$T_{\text{avg}}$	SEED	GQA	SQA ${}^{\text{IMG}}$	VQA ${}^{\text{Text}}$	POPE	MME	MMB ${}^{\text{EN}}$	MMB ${}^{\text{CN}}$	AI2D	MMMU	Acc.	Rel.
Layer-average budget $T{=}128$ ( $\downarrow$ 77.8%)
$\alpha{=}1$	128	64.34	59.21	73.08	58.65	86.83	1761.8	66.33	62.03	58.00	36.44	65.30	99.3
$\alpha{=}1.5$	128	65.50	59.93	73.08	59.53	87.01	1493.7	66.24	62.11	57.71	36.11	64.19	97.7
STAR-Pro ( $\alpha{=}2$ )	128	66.1	60.2	72.7	59.5	87.2	1492.7	67.0	63.1	57.9	35.4	64.37	97.9
$\alpha{=}2.5$	128	66.97	60.88	73.23	59.87	86.94	1493.4	67.35	62.29	59.49	36.33	64.80	98.6
$\alpha{=}3$	128	67.22	58.73	73.33	57.48	86.30	1488.1	67.09	62.29	59.29	36.33	64.25	97.7

Appendix

Table 14: Candidate-multiplier sweep on LLaVA-1.5-13B with the two Progressive-Stage checkpoints fixed at

\{15,30\}

. The highlighted STAR-Pro (

\alpha{=}2

) row is the deployed configuration; its scores are taken from the main comparison table ( Table 7 ).

T_{\text{avg}}

is the realized layer-average visual-token count. Acc. is the 10-benchmark average with MME/20, and Rel. (%) normalizes Acc. by the full-token baseline.

Config.	$T_{\text{avg}}$	SEED	GQA	SQA ${}^{\text{IMG}}$	VQA ${}^{\text{Text}}$	POPE	MME	MMB ${}^{\text{EN}}$	MMB ${}^{\text{CN}}$	AI2D	MMMU	Acc.	Rel.
Layer-average budget $T{=}640$ ( $\downarrow$ 77.8%)
$\alpha{=}1$	640	67.66	61.97	67.97	58.19	87.31	1472.0	65.48	57.90	65.22	35.33	64.06	99.3
$\alpha{=}1.5$	640	68.53	62.62	68.37	59.51	87.56	1491.5	65.31	57.99	65.09	34.89	64.44	99.9
STAR-Pro ( $\alpha{=}2$ )	640	68.95	62.4	68.8	59.1	87.6	1479.1	66.0	58.0	65.6	32.2	64.26	99.6
$\alpha{=}2.5$	640	69.06	59.30	69.01	53.40	87.95	1488.2	65.65	58.08	64.93	34.33	63.61	98.6
$\alpha{=}3$	720	67.27	49.05	68.72	43.81	81.91	1471.8	63.69	56.01	64.35	34.33	60.27	93.4

Appendix

Table 15: Candidate-multiplier sweep on LLaVA-NeXT-7B with the two Progressive-Stage checkpoints fixed at

\{12,20\}

. The highlighted STAR-Pro (

\alpha{=}2

) row is the deployed configuration; its scores are taken from the main comparison table ( Table 8 ).

T_{\text{avg}}

is the realized layer-average visual-token count (it exceeds

T

only where the strict average constraint is infeasible, at

\alpha{=}3

). Acc. is the 10-benchmark average with MME/20, and Rel. (%) normalizes Acc. by the full-token baseline.

Config.	$T_{\text{avg}}$	SEED	GQA	SQA ${}^{\text{IMG}}$	VQA ${}^{\text{Text}}$	POPE	MME	MMB ${}^{\text{EN}}$	MMB ${}^{\text{CN}}$	AI2D	MMMU	Acc.	Rel.
Layer-average budget $T{=}640$ ( $\downarrow$ 77.8%)
$\alpha{=}1$	640	69.22	63.50	72.19	60.79	86.61	1529.1	67.26	63.75	68.17	38.00	66.59	99.4
$\alpha{=}1.5$	640	70.36	64.30	72.88	61.75	86.78	1528.3	68.11	63.14	68.91	38.00	67.06	100.1
STAR-Pro ( $\alpha{=}2$ )	640	71.33	64.2	72.8	62.2	87.4	1545.2	68.5	63.1	69.8	35.8	67.2	100.3
$\alpha{=}2.5$	640	71.38	63.72	73.03	62.30	87.00	1548.0	67.60	62.63	69.46	36.89	67.14	100.2
$\alpha{=}3$	640	71.30	60.51	72.88	55.15	86.42	1554.3	67.77	62.46	69.62	35.44	65.93	98.4

Appendix

Table 16: Candidate-multiplier sweep on LLaVA-NeXT-13B with the two Progressive-Stage checkpoints fixed at

\{15,30\}

. The highlighted STAR-Pro (

\alpha{=}2

) row is the deployed configuration; its scores are taken from the main comparison table ( Table 9 ).

T_{\text{avg}}

is the realized layer-average visual-token count. Acc. is the 10-benchmark average with MME/20, and Rel. (%) normalizes Acc. by the full-token baseline.

Efficient VLM Inference

Efficient vision-language model inference reduces the compute, memory, latency, and energy needed to process images, video, and text.

See all 472 papers

Shallow to Deep: Aligning Token Pruning with Stage-wise Roles in LVLMs
Sep 22, 2026Shuo Zhang, Jintao Tong, Yixiong Zou +2Visual Token Pruning Efficient VLM Inference
School of Computer Science and Technology, Huazhong University of Science and Technology
ACPruner: Visual Token Pruning as Biased Attention Coverage Maximization in LVLMs
Sep 28, 2026Xu Li, Yuxuan Liang, Yi Zheng +6Efficient VLM Inference Large Vision-Language Models
College of Computer Science and Artificial Intelligence Fudan University, Shanghai, China
VPRune: Efficient Training-free Pre-LLM Visual Token Pruning
Sep 21, 2026Guangchuan Lv, Dianxing Shi, Dingjie FuEfficient VLM Inference Large Vision-Language Models
Northeastern University · Beihang University · Huazhong University of Science and Technology

Sep 22, 2026J/K move · Enter open · S save

Shallow to Deep: Aligning Token Pruning with Stage-wise Roles in LVLMs

Shuo Zhang, Jintao Tong, Yixiong Zou, Yuhua Li+1

School of Computer Science and Technology, Huazhong University of Science and Technology

Visual Token Pruning Efficient VLM Inference Large Vision-Language Models

Open paper AskPDF arXiv

Large Vision-Language Models (LVLMs) incur high computational costs from redundant visual tokens. Although training-free attention-based multi-layer pruning in the vision encoder stage has been explored as an effective strategy, we find that pruning in shallow layers consistently degrades performance. In this paper, we aim to understand this problem and seek a solution. By analyzing attention patterns across network depth, we find that shallow layers primarily function as edge detectors with chaotic attention maps, while deeper layers transition through local subject recognition and unstable semantic aggregation. To address the misalignment between pruning strategies and network stages, we propose STD, a hierarchical token pruning framework that adapts token selection mechanisms to the functional role of each network stage. STD employs High-Frequency Spectral Analysis in shallow layers to deterministically preserve structural edges, uses Gaussian-Smoothed Attention in intermediate layers to maintain spatial coherence, and introduces a Stability-Adaptive Trigger in deep layers to execute pruning only during semantically stable phases. Extensive experiments show that STD outperforms state-of-the-art pruning methods by 1.1% on LLaVA-1.5-7B with 88.9% token reduction, while also being plug-and-play and highly effective when combined with other methods, and by 2.1% on LLaVA-NeXT-7B with 94.4% reduction, delivering a 3.9x speed-up in the prefilling stage. Our code will be released at https://github.com/Twilight03/STD.

Abstract

Efficient VLM Inference

Explore similar work

Shallow to Deep: Aligning Token Pruning with Stage-wise Roles in LVLMs

ACPruner: Visual Token Pruning as Biased Attention Coverage Maximization in LVLMs

VPRune: Efficient Training-free Pre-LLM Visual Token Pruning