MI-Pruner: Crossmodal Mutual Information-guided Token Pruner for Efficient MLLMs

Jiameng Li; Aleksei Tiulpin; Matthew B. Blaschko

cs.CVApr 3, 2026

MI-Pruner: Crossmodal Mutual Information-guided Token Pruner for Efficient MLLMs

Authors: Jiameng Li, Aleksei Tiulpin, Matthew B. Blaschko

Organizations: ESAT-PSI, KU Leuven, Belgium · Faculty of Medicine, University of Oulu, Finland · Weill Cornell Medicine, USA

Abstract

For multimodal large language models (MLLMs), visual information is relatively sparse compared with text. As a result, research on visual pruning emerges for efficient inference. Current approaches typically measure token importance based on the attention scores in the visual encoder or in the LLM decoder, then select visual tokens with high attention scores while pruning others. In this paper, we pursue a different and more surgical approach. Instead of relying on mechanism-specific signals, we directly compute Mutual Information (MI) between visual and textual features themselves, prior to their interaction. This allows us to explicitly measure crossmodal dependency at the feature levels. Our MI-Pruner is simple, efficient and non-intrusive, requiring no access to internal attention maps or architectural modifications. Experimental results demonstrate that our approach outperforms previous attention-based pruning methods with minimal latency.

Figures & tables

Figure 1: Pruning visualization on LLaVA1.5-7B with different budgets. Our MI-Pruner consistently identifies and preserves the queried regions, whereas other methods partially miss relevant information. Tokens from top to bottom: 64, 128, 256.

Figure 2: Overview. Previous methods prune tokens by attention scores from vision encoder or LLM decoder. Our MI-Pruner calculates Mutual Information between visual and textual embeddings in the projection space, achieving optimal performance with minimal latency.

Figure 3: A toy model of MI-based pruning. We construct similarity matrices to get conditional probability and marginal probability, then calculate crossmodal PMI (top) and internal PMI (bottom). All [ vis ] tokens are flattened for illustration.

keep 64
Methods	GQA	SQA	TextVQA	MMVet	MME P	POPE	P F1	P Yes
LLaVA1.5-7B	61.97	66.80	59.09	$30.46_{\pm 7.58}$	1509.13	85.18	0.86	0.56
FastV ( Chen et al., 2024a )	50.80	51.10	47.80	24.99 ±7.01	1019.60	48.50	0.48	0.35
SparseVLM ( Zhang et al., 2025e )	52.70	62.20	51.80	23.04 ±7.88	1221.10	75.45	0.75	0.37
VisionZip ( Yang et al., 2025a )	55.10	69.00	55.50	31.04 ±7.08	1365.60	77.34	0.76	0.37
VisPruner ( Zhang et al., 2025b )	55.40	69.10	55.80	32.09 ±7.11	1369.90	82.61	0.80	0.38

Table 1: Performance comparison of different pruning methods on LLaVA1.5-7B. Methods before - - - are attention-based. We use color to distinguish best and second best .

Methods	GQA	MME P	POPE	P F1	P Yes	Methods	GQA	MME P	POPE	P F1	P Yes
Qwen3VL-2B	59.55	1494.98	89.89	0.90	0.47	Qwen3VL-8B	61.53	1730.47	88.84	0.88	0.44
keep 50%
Random	49.96	1438.19	88.76	0.88	0.46	Random	53.02	1422.31	88.25	0.87	0.43
Similarity	50.87	1454.34	89.26	0.89	0.48	Similarity	53.17	1461.25	88.95	0.88	0.45
Attention	50.09	1462.74	89.54	0.89	0.47	Attention	52.93	1462.84	88.85	0.88	0.44
MI-Pruner	51.30	1478.26	89.78	0.90	0.48	MI-Pruner	53.48	1493.04	89.22	0.89	0.45

Table 2: Performance comparison of different pruning methods on Qwen3VL-2B and -8B. ”Attention” refers to the sum of attention scores from other image tokens. MI-Pruner and MI-Attention achieve SOTA performance for both scales.

Figure 4: Performance on Qwen2VL series (GQA).

Panel of Figure 5 — Figure 5: Pruning visualization on Qwen3VL series. Our method retains the semantic-relative patches regarding prompts adaptively.

Methods	Mem (GB)	Latency (ms)	Mem (GB)	Latency (ms)
	keep 128		keep 64
SparseVLM	18.08	96.36 ±0.35	18.11	93.67 ±0.31
VisPruner	14.35	92.04 ±0.44	14.35	89.48 ±0.32
CDPruner	14.73	111.12 ±0.65	14.65	91.41 ±0.41
DART	14.05	92.62 ±0.39	13.96	88.41 ±0.37
MI-Pruner λ=1	14.01	82.08 ±0.32	13.93	77.06 ±0.36

Table 3: Efficiency comparison on POPE (LLaVA1.5-7B).

Methods	TGIF		MSVD		MSRVTT
Methods	Acc	Score	Acc	Score	Acc	Score
Video-LLaVA	18.90	2.54	72.00	3.95	57.50	3.50
keep 227
FastV	14.30	2.42	68.90	3.90	53.00	3.40
VisPruner	15.90	2.41	69.30	3.92	55.60	3.45
MI-Pruner	15.70	2.44	70.60	3.94	56.40	3.46

Table 4: Video-QA performance on Video-LLaVA-7B.

Panel of Figure 6 — Figure 6: Ablation study on scoring functions (LLaVA1.5-7B).

$\tau$	GQA 64	GQA 32	SQA 64	SQA 32
0.01	55.26	55.39	69.06	69.02
0.1	56.88	55.78	69.81	69.26
0.5	54.32	52.23	68.56	68.99

Table 5: Ablation study on temperature (LLaVA1.5-7B).

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

	GQA 64	GQA 32	SQA 64	SQA 32
MinMax	55.37	52.95	67.56	67.01
Softmax	56.88	55.78	69.81	69.26

Appendix

Table 6: Ablation study on Normalization (LLaVA1.5-7B).

keep 64
Methods	GQA	SQA	TextVQA	MME P
LLaVA1.5-13B	61.97	72.73	61.24	1524.19
Random	53.45	70.85	50.74	1257.35
Similarity	53.63	71.64	51.58	1338.78
FastV	53.70	56.80	47.10	1275.41
SparseVLM	50.60	69.00	22.70	1289.92

Appendix

Table 7: Results on large-scale models, LLaVA1.5-13B and Qwen3VL-30B.

Methods	Instruct+Sampl.	Thinking+Sampl.
keep 50%
Random	52.79	32.93
Similarity	53.11	34.13
MI-Pruner	53.20	34.97

Appendix

Table 8: GQA results on hybrid sampling (Qwen3VL-8B).

Methods	Mem (GB)	Latency (ms)
keep 32
SparseVLM	18.12	93.33 ±0.44
VisPruner	14.35	89.98 ±0.39
DART	13.94	87.56 ±0.47
CDPruner	14.61	83.22 ±0.38
MI-Attention	14.35	77.36 ±0.40

Appendix

Table 9: Efficiency comparison on POPE (LLaVA1.5-7B).

Panel of Figure 7 — Figure 7: Dialogue examples on Qwen2.5VL-7B (GQA).

Methods	Random			Popular			Adversarial			Average
Methods	Acc $\uparrow$	F1 $\uparrow$	Yes	Acc $\uparrow$	F1 $\uparrow$	Yes	Acc $\uparrow$	F1 $\uparrow$	Yes	Acc $\uparrow$	F1 $\uparrow$	Yes
LLaVA1.5-7B	89.60	0.90	0.51	86.20	0.87	0.55	79.73	0.82	0.61	85.18	0.86	0.56
keep 128
Random	85.23	0.83	0.37	84.07	0.82	0.38	80.80	0.79	0.41	83.37	0.81	0.39
Similarity	87.43	0.86	0.42	86.30	0.85	0.43	82.60	0.82	0.46	85.44	0.84	0.44
MI-Attention	88.31	0.87	0.41	86.63	0.86	0.42	84.11	0.83	0.44	86.35	0.85	0.42

Appendix

Table 10: POPE performance comparison under 3 settings on LLaVA1.5-7B. We report accuracy (Acc), F1 scores and Yes-rate.

Methods	Random			Popular			Adversarial			Average
Methods	Acc $\uparrow$	F1 $\uparrow$	Yes	Acc $\uparrow$	F1 $\uparrow$	Yes	Acc $\uparrow$	F1 $\uparrow$	Yes	Acc $\uparrow$	F1 $\uparrow$	Yes
Qwen3VL-2B	92.37	0.92	0.44	89.53	0.89	0.47	87.77	0.88	0.49	89.89	0.90	0.47
keep 50%
Random	91.67	0.91	0.44	88.47	0.88	0.46	86.13	0.86	0.48	88.76	0.88	0.46
Attention	92.33	0.92	0.44	89.40	0.89	0.47	86.90	0.87	0.50	89.54	0.89	0.47
Similarity	92.63	0.92	0.45	88.77	0.89	0.49	86.37	0.87	0.51	89.26	0.89	0.48

Appendix

Table 11: POPE performance comparison under 3 settings on Qwen3VL. We report accuracy (Acc), F1 scores and Yes-rate.

Efficient Multimodal Inference

Efficient multimodal inference reduces the compute, memory, latency, and energy needed to process image, video, audio, text, and sensor inputs.

See all 227 papers

MiCo: Mutual Information Coverage Optimization through Semantic Erasure Modeling for Efficient MLLM Inference
Sep 28, 2026Tinghao Wang, Yichen Guo, Qizhe Zhang +11Efficient Multimodal Inference Multimodal Large Language Models
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · University of Electronic Science and Technology of China · Nanyang Technological University +2
SinkPruner: Sink-Free Visual Token Pruning for Multimodal Large Language Models
Sep 1, 2026Shiyu Li, Zi-Yuan Hu, Shijia Huang +3Efficient VLM Inference Multimodal Large Language Models
1The Chinese University of Hong Kong · 2Weitu AI · 3Peking University
TOPS: First-Principles Visual Token Pruning via Constructing Token Optimal Preservation Sets for Efficient MLLM Inference
Jun 25, 2026Tinghao Wang, Yichen Guo, Rui Huang +9Efficient VLM Inference Multimodal Large Language Models
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · University of Electronic Science and Technology of China · Nanyang Technological University +1

Sep 28, 2026J/K move · Enter open · S save

MiCo: Mutual Information Coverage Optimization through Semantic Erasure Modeling for Efficient MLLM Inference

Tinghao Wang, Yichen Guo, Qizhe Zhang, Yuan Zhang+10

State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · University of Electronic Science and Technology of China · Nanyang Technological University+2

Efficient Multimodal Inference Multimodal Large Language Models Visual Token Pruning

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) have demonstrated impressive performance in multimodal understanding, but processing large numbers of visual tokens results in high computational costs. While many methods have been proposed to reduce the number of visual tokens, most of them rely on heuristics and are prone to discarding substantial visual information during pruning, leading to degradation in model performance. In this work, by using a semantic erasure model, we derive a general mutual information coverage objective from task log-loss and propose MiCo, a training-free two-stage pruning method. MiCo first uses visual signals to select a representative candidate pool before visual tokens enter the language model, then performs task-aware subset selection within it. At each stage, suitable observable proxies instantiate the derived objective as a monotone submodular coverage function, which MiCo greedily optimizes under the token budget. MiCo is evaluated on diverse MLLMs ranging from 7B to 13B parameters across a broad range of image and video benchmarks spanning general visual reasoning, fine-grained OCR and grounding, hallucination detection, and long-video understanding. MiCo consistently achieves the best performance across nearly all evaluated models under all pruning ratios. On LLaVA-NEXT-13B, MiCo uses only 5.6% visual tokens, retains 97.5% of baseline performance, and achieves a 3.8-fold inference speedup. Our experiments demonstrate the effectiveness of MiCo and our mutual information coverage objective for visual token pruning. Our code is available at https://github.com/MAX-7237/MiCo.

Abstract

Efficient Multimodal Inference

Explore similar work

MiCo: Mutual Information Coverage Optimization through Semantic Erasure Modeling for Efficient MLLM Inference

SinkPruner: Sink-Free Visual Token Pruning for Multimodal Large Language Models

TOPS: First-Principles Visual Token Pruning via Constructing Token Optimal Preservation Sets for Efficient MLLM Inference