MedSTP: Preserving Subspace and Transition Evidence for Medical VQA Token Pruning

Mengchen Fan; Qizhen Lan; Xi Xiao; Lin Zhao; Liang He; Tianyang Wang; Xiaoqian Jiang; Pulin Che

cs.CVOct 15, 2026

MedSTP: Preserving Subspace and Transition Evidence for Medical VQA Token Pruning

Authors: Mengchen Fan, Qizhen Lan, Xi Xiao, Lin Zhao, Liang He, Tianyang Wang, Xiaoqian Jiang, Pulin Che,

Organizations: University of Alabama at Birmingham · UTHealth Houston · Northeastern University · ETH Zurich

Abstract

Medical vision-language models (VLMs) for visual question answering (VQA) often pass long visual-token sequences to the language model, making visual-token pruning attractive but risky for medical images, where answers may depend on localized boundary and texture evidence. We study training free pre-decoding pruning and treat pruning as a subspace stable evidence preservation problem. The retained tokens should remain close to the dominant visual evidence subspace induced by the full token set. To this end, we propose Subspace-Transition Token Pruning (\method{}), which estimates each token's spectral support from principal component space and constructs a compact representation subspace. To better reflect medical image structure, \method{} further uses a bounded token-grid transition prior to promote boundary and texture sensitive tokens. We evaluate \method{} on the closed-ended SLAKE and OmniMedVQA benchmarks and on RadImageNet-VQA, which provides both closed-ended and open-ended evaluations, using Lingshu-7B and Qwen3-VL-8B backbones. \method{} delivers consistently strong performance on both the closed-ended and open-ended RadImageNet-VQA evaluations. These results show that \method{} is an effective training free pruning framework for medical domain VQA.

Figures & tables

Figure 1: Qualitative comparison under the same compact token budget. The dashed red circle marks the relevant abnormal region; MedSTP keeps boundary and texture evidence that generic pruning may miss.

Figure 2: Overview of MedSTP for medical VQA token pruning. Given a medical image-question input, the frozen VLM encoder produces visual tokens before decoding. MedSTP constructs a compact retained subspace from principal projection participation and incorporates a bounded token-grid transition multiplier. The selected compact subset is passed to the frozen VQA decoder for answer prediction.

Model	Method	OS Acc. $\uparrow$	RJ Acc. $\uparrow$	LC Acc. $\uparrow$	ES Acc. $\uparrow$	Avg. Acc. $\uparrow$	KL $\downarrow$
Lingshu-7B	Full tokens	73.80 $\pm$ 1.15	83.90 $\pm$ 1.20	86.82 $\pm$ 1.19	95.94 $\pm$ 1.78	80.62 $\pm$ 0.69	0.000 $\pm$ 0.000
	Random	22.29 $\pm$ 1.09	64.62 $\pm$ 1.56	33.21 $\pm$ 1.66	69.92 $\pm$ 4.14	38.69 $\pm$ 0.84	1.063 $\pm$ 0.017
	BTP	33.26 $\pm$ 1.23	64.19 $\pm$ 1.56	52.24 $\pm$ 1.76	89.43 $\pm$ 2.77	48.69 $\pm$ 0.87	0.810 $\pm$ 0.016
	FastV	17.15 $\pm$ 0.99	56.57 $\pm$ 1.61	26.62 $\pm$ 1.56	61.79 $\pm$ 4.38	32.26 $\pm$ 0.81	1.103 $\pm$ 0.016
	SparseVLM	18.18 $\pm$ 1.01	57.63 $\pm$ 1.61	26.74 $\pm$ 1.56	62.60 $\pm$ 4.36	33.07 $\pm$ 0.82	1.083 $\pm$ 0.016
	VisPruner	45.13 $\pm$ 1.30	72.88 $\pm$ 1.45	54.98 $\pm$ 1.75	79.67 $\pm$ 3.63	56.65 $\pm$ 0.86	0.659 $\pm$ 0.015

Table 1: OmniMedVQA result at 10% keep ratio. The OmniMedVQA categories are OS = organ/structure, RJ = radiology/joint/body region, LC = lesion/cellular/tissue morphology, and ES = eye/skin. Avg. is the overall accuracy. All values except KL are percentages.

Model	Method	OR Acc. $\uparrow$	AB Acc. $\uparrow$	PO Acc. $\uparrow$	OT Acc. $\uparrow$	Avg. Acc. $\uparrow$	KL $\downarrow$
Lingshu-7B	Full tokens	88.40 $\pm$ 0.95	72.78 $\pm$ 1.74	94.81 $\pm$ 2.53	86.43 $\pm$ 1.51	83.92 $\pm$ 0.75	0.000 $\pm$ 0.000
	Random	65.65 $\pm$ 1.40	25.84 $\pm$ 1.71	70.13 $\pm$ 5.22	75.00 $\pm$ 1.91	56.93 $\pm$ 1.01	0.641 $\pm$ 0.017
	BTP	80.12 $\pm$ 1.18	59.94 $\pm$ 1.92	93.51 $\pm$ 2.81	81.01 $\pm$ 1.73	75.23 $\pm$ 0.88	0.190 $\pm$ 0.008
	FastV	64.60 $\pm$ 1.41	26.76 $\pm$ 1.73	62.34 $\pm$ 5.52	68.41 $\pm$ 2.05	55.01 $\pm$ 1.02	0.611 $\pm$ 0.015
	SparseVLM	65.30 $\pm$ 1.41	26.45 $\pm$ 1.72	61.04 $\pm$ 5.56	70.35 $\pm$ 2.01	55.64 $\pm$ 1.02	0.610 $\pm$ 0.015
	VisPruner	74.63 $\pm$ 1.28	47.71 $\pm$ 1.95	87.01 $\pm$ 3.83	77.91 $\pm$ 1.83	68.38 $\pm$ 0.95	0.337 $\pm$ 0.012

Table 2: SLAKE result with official question group breakdown at 10% keep ratio. The SLAKE categories are OR = organ, AB = abnormality, PO = position, and OT = other.

Model	Method	Closed-ended				Open-ended
Model	Method	CT Acc. $\uparrow$	MRI Acc. $\uparrow$	Avg. Acc. $\uparrow$	KL $\downarrow$	Selene $\uparrow$	Phi-4 $\uparrow$	Avg. $\uparrow$
Lingshu-7B	Full tokens	82.16 $\pm$ 1.85	94.77 $\pm$ 0.93	89.40 $\pm$ 0.97	0.000 $\pm$ 0.000	72.40 $\pm$ 1.41	68.10 $\pm$ 1.47	70.25 $\pm$ 1.39
	Random	47.42 $\pm$ 2.41	63.76 $\pm$ 2.01	55.59 $\pm$ 1.57	0.866 $\pm$ 0.029	44.20 $\pm$ 1.57	41.90 $\pm$ 1.56	43.05 $\pm$ 1.54
	BTP	68.31 $\pm$ 2.25	71.95 $\pm$ 1.88	70.40 $\pm$ 1.44	0.543 $\pm$ 0.024	49.40 $\pm$ 1.58	48.10 $\pm$ 1.58	48.75 $\pm$ 1.55
	FastV	42.96 $\pm$ 2.40	33.45 $\pm$ 1.97	37.50 $\pm$ 1.53	1.247 $\pm$ 0.024	29.10 $\pm$ 1.44	28.30 $\pm$ 1.43	28.70 $\pm$ 1.41
	SparseVLM	43.90 $\pm$ 2.40	34.15 $\pm$ 1.98	38.30 $\pm$ 1.54	1.236 $\pm$ 0.024	28.40 $\pm$ 1.43	28.00 $\pm$ 1.42	28.20 $\pm$ 1.40

Table 3: RadImageNet-VQA results at a 10% visual-token keep ratio. Selene and Phi-4 are method-blind semantic judges.

Figure 3: Across most pruning rates, MedSTP keeps the retained-token eigenspace closer to the full-token visual eigenspace than the competing pruners. Subspace preservation diagnostic on SLAKE with Lingshu-7B; lower drift is better, and 0% pruning is the zero-drift identity case.

Figure 4: As retained-token budgets shrink, MedSTP stays nearer the low-drift, high-accuracy corner than the competing pruners. Subspace drift versus answer accuracy on SLAKE with Lingshu-7B; accuracy is reported as a percentage, and larger markers indicate larger retained-token budgets.

Panel of Figure 5 — Figure 5: VQA accuracy under increasing pruning strength with Lingshu-7B. All methods use matched pruning insertion points and fixed visual-token budgets. Dashed horizontal lines denote full-token accuracy.

Dataset	Variant	OR Acc. $\uparrow$	AB Acc. $\uparrow$	PO Acc. $\uparrow$	OT Acc. $\uparrow$	Avg. Acc. $\uparrow$	KL $\downarrow$
SLAKE	w/o trans.	80.99 $\pm$ 1.16	55.96 $\pm$ 1.94	94.81 $\pm$ 2.53	84.50 $\pm$ 1.59	75.36 $\pm$ 0.88	0.203 $\pm$ 0.010
SLAKE	MedSTP (ours)	81.78 $\pm$ 1.14	57.34 $\pm$ 1.93	93.51 $\pm$ 2.81	87.79 $\pm$ 1.44	76.78 $\pm$ 0.86	0.169 $\pm$ 0.009
Dataset	Variant	OS Acc. $\uparrow$	RJ Acc. $\uparrow$	LC Acc. $\uparrow$	ES Acc. $\uparrow$	Avg. Acc. $\uparrow$	KL $\downarrow$
OmniMedVQA	w/o trans.	62.68 $\pm$ 1.23	78.81 $\pm$ 1.33	64.41 $\pm$ 1.80	87.80 $\pm$ 2.95	68.55 $\pm$ 0.80	0.359 $\pm$ 0.011
OmniMedVQA	MedSTP (ours)	64.54 $\pm$ 1.21	80.19 $\pm$ 1.30	70.20 $\pm$ 1.72	82.11 $\pm$ 3.46	70.83 $\pm$ 0.79	0.298 $\pm$ 0.011

Table 4: Transition prior ablation with Lingshu-7B at 10% keep ratio.

Figure 6: Moderate transition weighting works best on OmniMedVQA at 10% keep ratio with Lingshu-7B:

\beta=0.1-0.3

gives higher accuracy, while larger weights reduce accuracy.

Visual Token Pruning

Visual token pruning reduces the image and video patches processed by transformers to lower inference cost while preserving task-relevant information.

See all 276 papers

MedPruner: Training-Free Hierarchical Token Pruning for Efficient 3D Medical Image Understanding in Vision-Language Models
Mar 12, 2026Shengyuan Liu, Zanting Ye, Yunrui Lin +6Efficient VLM Inference 3D Medical Imaging
Chinese University of Hong Kong · Westlake University · Southern Medical University +4
OccamToken: Efficient VLM Inference with Training-Free and Budget-Adaptive Token Pruning
May 28, 2026Geng Li, Guohao Chen, Ting Chen +6Efficient VLM Inference Large Vision-Language Models
1Nanyang Technological University (NTU)
How Much Can We Prune? Exploring the Performance Limits of Visual Token Pruning
Oct 13, 2026Janghoon Cho, Shubhankar Borse, Reza Pourreza +4Visual Token Pruning LVLM Pruning
Qualcomm AI Research∗

Mar 12, 2026J/K move · Enter open · S save

MedPruner: Training-Free Hierarchical Token Pruning for Efficient 3D Medical Image Understanding in Vision-Language Models

Shengyuan Liu, Zanting Ye, Yunrui Lin, Chen Hu+5

Chinese University of Hong Kong · Westlake University · Southern Medical University+4

Efficient VLM Inference 3D Medical Imaging Medical VLMs Visual Token Pruning

Open paper AskPDF arXiv

While specialized Medical Vision-Language Models (VLMs) have achieved remarkable success in interpreting 2D and 3D medical modalities, their deployment for 3D volumetric data remains constrained by significant computational inefficiencies. Current architectures typically suffer from massive anatomical redundancy due to the direct concatenation of consecutive 2D slices and lack the flexibility to handle heterogeneous information densities across different slices using fixed pruning ratios. To address these challenges, we propose MedPruner, a training-free and model-agnostic hierarchical token pruning framework specifically designed for efficient 3D medical image understanding. MedPruner introduces a two-stage mechanism: an Inter-slice Anchor-based Filtering module to eliminate slice-level temporal redundancy, followed by a Dynamic Information Nucleus Selection strategy that achieves adaptive token-level compression by quantifying cumulative attention weights. Extensive experiments on three 3D medical benchmarks and across three diverse medical VLMs reveal massive token redundancy in existing architectures. Notably, MedPruner enables models such as MedGemma-1.5 to maintain or even exceed their original performance while retaining fewer than 5% of visual tokens, thereby reducing visual-token overhead and validating the necessity of dynamic token selection for practical clinical deployment. Our code is available at https://github.com/CUHK-AIM-Group/MedPruner.

Abstract

Visual Token Pruning

Explore similar work

MedPruner: Training-Free Hierarchical Token Pruning for Efficient 3D Medical Image Understanding in Vision-Language Models

OccamToken: Efficient VLM Inference with Training-Free and Budget-Adaptive Token Pruning

How Much Can We Prune? Exploring the Performance Limits of Visual Token Pruning