VIG: Visual Information Gain as a Reward Signal for Multimodal Chain-of-Thought Compression

Wen Luo; Xiaohan Yi; Xiaotao Huang; Liqun Huang

cs.CVAug 22, 2026

VIG: Visual Information Gain as a Reward Signal for Multimodal Chain-of-Thought Compression

Authors: Wen Luo, Xiaohan Yi, Xiaotao Huang, Liqun Huang

Organizations: School of Software Engineering, Huazhong University of Science and Technology · Tsinghua University

Abstract

Multimodal large reasoning models often rely on long Chain-of-Thought (CoT) traces in which a substantial fraction of tokens, such as repeated visual descriptions, self-reflection, and other visually-disengaged filler, inflate inference cost without contributing to the answer. Existing CoT compression methods optimize output length but never measure whether a reasoning token is actually grounded in the image. We propose VIG (Visual Information Gain), an information-theoretic GRPO reward that scores each reasoning token by how much the image reduces its predictive uncertainty. VIG is computed online from two forward passes of the same policy, one with and one without the image, so no reference chains, external annotations, or auxiliary reward models are needed. Across six main multimodal reasoning benchmarks and three Qwen3-VL-Thinking model sizes (2B/4B/8B), plus an additional R1-Onevision-Bench evaluation on 8B, VIG consistently improves the accuracy--efficiency trade-off, supporting our central claim: efficient multimodal reasoning emerges from raising visual information density, where every reasoning token earns its place by anchoring to the image, rather than from imposing a length budget. Our source code is available at https://github.com/chaser682/vig.

Figures & tables

Figure 1: Accuracy–efficiency Pareto frontier on Qwen3-VL-8B-Thinking. Each point reports the average accuracy (AvgACC, %) and efficiency (AvgEff, ACC per 1k tokens) of one method across the six multimodal benchmarks in Table 1 . VIG (

\bigstar

) Pareto-dominates the evaluated baselines (Base, GRPO, L1, ThinkPrune, TALE) on both axes, defining a new accuracy–efficiency Pareto frontier (red dashed lines).

Figure 2: Length penalties trim the chain; only visually grounded reasoning removes the redundancy at its source. Token counts and final answers from Base, L1, and our VIG model on the same geometry question (gold answer A, $2\,\text{cm}$ ).

Figure 3: VIG framework overview. Two forward passes over the generated reasoning chain—one with the image and one with visual tokens removed—yield a token-level entropy difference, which is aggregated into $R_{\mathrm{VIG}}$ and used as a GRPO reward.

Method	WeMath		MMMU		MathVision		DynaMath		MMK12		Geo3K		Summary
Method	ACC	TLen	ACC	TLen	ACC	TLen	ACC	TLen	ACC	TLen	ACC	TLen	AvgACC	AvgTLen	AvgEff
Qwen3-VL-2B-Thinking	51.72	3120	36.78	2855	15.39	3928	51.60	2252	33.25	3683	33.78	3087	37.09	3154.2	12.71
GRPO	57.36	2850	45.00	2614	17.17	3825	54.27	2046	36.70	3592	38.60	2929	41.52	2975.9	15.29
L1	61.67	2241	43.33	2561	17.14	3668	53.65	1926	34.30	3429	35.94	2670	41.01	2749.0	16.74
TALE	53.39	3033	40.00	2667	17.24	3872	54.01	2096	35.20	3595	36.11	3040	39.32	3050.5	14.08
ThinkPrune	60.80	2438	42.56	2372	17.53	3708	55.53	1863	39.70	3413	37.44	2679	42.26	2745.5	17.17

Table 1: Main results on six benchmarks. Bold and underline denote the best and second-best results within each model block; tied values share the same marking.

Method	Physics (278)		Mathematics (327)		Biology (134)		Chemistry (105)		Deduction (98)		Overall
Method	ACC	TLen	ACC	TLen	ACC	TLen	ACC	TLen	ACC	TLen	AvgACC	AvgTLen	AvgEff
Qwen3-VL-8B-Thinking	53.96	2793	45.26	2894	56.72	2121	60.95	2315	17.35	3953	48.30	2799.8	18.48
GRPO	61.87	2444	47.09	2775	62.69	1845	64.76	2125	24.49	3836	53.29	2583.1	22.62
L1	61.51	2338	48.32	2708	59.70	1844	68.57	2101	21.43	3676	53.29	2508.9	23.00
TALE	56.83	2149	48.62	2565	61.94	1621	68.57	1647	23.47	3444	52.55	2297.1	26.41
ThinkPrune	59.35	2416	46.48	2808	58.21	1671	63.81	1993	26.53	3797	51.80	2542.8	22.99

Table 2: Performance comparison on R1-Onevision-Bench. Bold and underline denote the best and second-best results.

Method	RealWorldQA		OCRBench
Method	ACC	TLen	ACC	TLen
Qwen3-VL-4B-Thinking	67.84	784	77.70	587
GRPO	68.10	635	77.30	483
L1	67.19	1116	76.00	779
ThinkPrune	68.10	453	77.50	367
VIG (Ours)	68.63	374	78.30	201

Table 3: Evaluation results on non-mathematical visual tasks with Qwen3-VL-4B-Thinking. All models use the same checkpoints as the 4B block of Table 1 . Bold and underline denote the best and second-best results.

Factor	Variant	AvgACC	AvgTLen	AvgEff
Reward	w/o VIG	62.88	1975.4	36.44
	w/ VIG	64.92	1524.3	52.42
Vision pass	masked image	62.98	1885.0	38.70
	no image	64.92	1524.3	52.42
$w_{v}$	0.5	63.06	1676.1	45.04
	1.0	64.92	1524.3	52.42

Table 4: Ablation results on Qwen3-VL-8B-Thinking. All values are averaged over the six benchmarks of Table 1 .

Figure 4: Average per-token VIG decreases as reasoning chains lengthen ( $r=-0.569$ , $p<0.001$ ): long chains are dominated by visually-disengaged tokens, exactly the redundancy VIG suppresses.

Figure 5: Accuracy and mean chain length stratified by per-sample average VIG (WeMath,

n=1{,}740

): higher visual density co-occurs with both higher accuracy and shorter chains. This association is descriptive; it is partly mediated by chain length and item difficulty (§ 4.4 ).

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

(a) Reward ablation
Reward	AvgACC	AvgTLen
GRPO (no visual term)	69.33	1751
Confidence-only ( $-H_{\text{vis}}$ )	68.21	1441
VIG (ours)	71.37	1297

Appendix

Table 5: Signal-integrity controls on Qwen3-VL-8B-Thinking. All arms share identical data, seed, and step count. (a) Reward ablation, 5-benchmark macro-average. (b) VIG vs. GRPO split by whether the base model solves the item without the image, 6-benchmark macro-average.

(a) VIG-8B chains
Deleted	10%	20%	30%	40%	50%
lowest-VIG	80.92	79.94	78.22	75.17	72.36
random	81.84	80.92	80.06	79.25	77.41
highest-VIG	82.07	81.95	81.78	81.32	80.75

Appendix

Table 6: Quantile-deletion probe (WeMath,

n=1{,}740

). Sentences are ranked by VIG, a fraction is deleted, and the answer is re-decoded from the surviving prefix with the image still in context. Bold marks the most damaging condition per block.

Panel of Figure 6 — Figure 6: Sentence-level VIG within complete VIG-8B chains, shown beside the input image. Sentences that read a quantity off the figure score highest ( $+1.16$ , $+0.73$ ), while symbolic manipulation scores near zero even when answer-bearing (“Divide both sides by 2”, $+0.01$ ).

Method	s/it	Wall-clock	GPU-h	Rel.
GRPO	31.8	4h 25m	35.3	$1.00\times$
L1	33.5	4h 39m	37.2	$1.05\times$
ThinkPrune	32.3	4h 29m	35.8	$1.01\times$
VIG (Ours)	37.2	5h 10m	41.3	$1.17\times$

Appendix

Table 7: Training cost on Qwen3-VL-8B-Thinking (8

\times

H20, 500 GRPO steps). Rel. is the relative wall-clock cost vs. plain GRPO.

Panel of Figure 7 — Figure 7: Additional cases ( $\vdots$ : omitted sentences). (a) Reading the ruler scores $+0.90$ , the answer-bearing arithmetic $+0.004$ . (b) A failure case: every sentence is visually dependent, yet the model misreads the protractor as $50^{\circ}$ instead of $40^{\circ}$ . High VIG certifies image-driven generation, not a correct reading.

Figure 8: An additional qualitative comparison between Base, L1, and VIG. VIG produces a shorter reasoning chain while preserving the key visual evidence needed for the final answer.

Figure 9: A second qualitative example comparing Base, L1, and VIG. Compared with the baseline methods, VIG better concentrates its reasoning on the visually relevant information and avoids unnecessary self-reflection.

Multimodal CoT Reasoning

Multimodal chain-of-thought reasoning studies how AI models combine visual, audio, and textual evidence through intermediate reasoning to solve complex tasks.

See all 143 papers

Look Back, Think Ahead: Visual Memory on Demand for Efficient Multimodal Reasoning
Oct 8, 2026Yicheng Xue, Han Wu, Jufeng Yang +4VLM Reasoning Efficient Multimodal Inference
City University of Hong Kong · Zhejiang University · Peking University +2
MCPO: Modality-Contrastive Preference Optimization for Multimodal Chain-of-Thought Compression
Sep 4, 2026Guangheng Yang, Zhenliang Ni, Zhenkai Wu +4Efficient Multimodal Inference Multimodal Large Language Models
Huawei Technologies Ltd. Shanghai, China
TVI-CoT: Text-Visual Interleaved Chain-of-Thought Reasoning for Multimodal Understanding
Jun 7, 2026Lianyu Hu, Xiaoyu Ma, Zeqin Liao +1Multimodal Large Language Models Multimodal Reasoning
College of Computing and Data Science of Nanyang Technological University, Singapore.

Oct 8, 2026J/K move · Enter open · S save

Look Back, Think Ahead: Visual Memory on Demand for Efficient Multimodal Reasoning

Yicheng Xue, Han Wu, Jufeng Yang, Minjing Dong+3

City University of Hong Kong · Zhejiang University · Peking University+2

VLM Reasoning Efficient Multimodal Inference Memory-Augmented VLMs Visual Token Compression Visual Token Pruning

Open paper AskPDF arXiv

Processing long visual token sequences from high-resolution images makes multi-step reasoning computationally expensive for multimodal Large Language Models (MLLMs). Existing one-shot pruning and aggregation methods compress visual tokens into a fixed context before decoding. However, visual evidence needs can shift as reasoning unfolds, making it difficult for a fixed compressed context to retain all the details needed across stages. To address this challenge, we propose ViMoD, a lightweight framework that maintains a compact visual context while preserving access to original fine-grained evidence as reasoning needs evolve. Deformable Aggregation of Region-wise Tokens (DART) learns content-adaptive groups and aggregation capacities, constructing compact Coarse representations linked to recoverable original Fine tokens. Temporal Routing for Adaptive Contextual Evidence (TRACE) integrates decoding history to anticipate upcoming evidence needs and select, retain, or replace active Fine-token groups. Selected Fine tokens augment the persistent Coarse context in the frozen backbone, enabling stage-specific evidence access without continuously attending to all visual tokens. On Qwen3-VL-4B, ViMoD outperforms all evaluated baselines on all eight reasoning benchmarks at a 20% target visual token budget, improving the mean normalized score by 39.0% over the strongest evaluated one-shot baseline. These gains are achieved with only 0.0546% additional trainable parameters relative to the frozen backbone.

Abstract

Multimodal CoT Reasoning

Explore similar work

Look Back, Think Ahead: Visual Memory on Demand for Efficient Multimodal Reasoning

MCPO: Modality-Contrastive Preference Optimization for Multimodal Chain-of-Thought Compression

TVI-CoT: Text-Visual Interleaved Chain-of-Thought Reasoning for Multimodal Understanding