Reflect to Inform: Boosting Multimodal Reasoning via Information-Gain-Driven Verification

Shuai Lv; Chang Liu; Feng Tang; Yujie Yuan; Aojun Zhou; Kui Zhang; Xi Yang; Yangqiu Song

cs.CVMar 27, 2026

Reflect to Inform: Boosting Multimodal Reasoning via Information-Gain-Driven Verification

Authors: Shuai Lv, Chang Liu, Feng Tang, Yujie Yuan, Aojun Zhou, Kui Zhang, Xi Yang, Yangqiu Song

Organizations: University of Science and Technology of China, Hefei, China · Huawei Foundation Model Department, Shanghai, China · The Chinese University of Hong Kong, Hong Kong, China · Hong Kong University of Science and Technology, Hong Kong, China

Abstract

Multimodal Large Language Models (MLLMs) achieve strong multimodal reasoning performance, yet we identify a recurring failure mode in long-form generation: as outputs grow longer, models progressively drift away from image evidence and fall back on textual priors, resulting in ungrounded reasoning and hallucinations. Interestingly, Based on attention analysis, we find that MLLMs have a latent capability for late-stage visual verification that is present but not consistently activated. Motivated by this observation, we propose Visual Re-Examination (VRE), a self-evolving training framework that enables MLLMs to autonomously perform visual introspection during reasoning without additional visual inputs. Rather than distilling visual capabilities from a stronger teacher, VRE promotes iterative self-improvement by leveraging the model itself to generate reflection traces, making visual information actionable through information gain. Extensive experiments across diverse multimodal benchmarks demonstrate that VRE consistently improves reasoning accuracy and perceptual reliability, while substantially reducing hallucinations, especially in long-chain settings. Code is available at https://github.com/Xiaobu-USTC/VRE.

Figures & tables

Figure 1: Overall pipeline of the proposed Visual Reflection Enhancement (VRE) framework. Left: Reflection data construction, where difficulty-aware filtering and information-gain-guided reflection synthesis generate high-quality SFT data. Right: Model optimization, where the cold-start model is iteratively improved through rejection sampling and reinforcement learning under a structured reward scheme comprising format, accuracy, and reflection rewards.

Model	Tool Use	Param Size	MathVista	MathVerse	MathVision	WeMath	LogicVista	MMMU
Closed-Source MLLMs
GPT-4o [ 1 ]	✗	-	63.8	50.8	30.4	69.0	53.2	69.1
Claude 3.7 [ 3 ]	✗	-	66.8	52.0	41.3	72.6	-	-
Open-Source General MLLMs
LLaVA-OneVision [ 25 ]	✗	7B	58.6	19.3	18.3	20.9	33.3	48.8
Qwen2.5-VL [ 5 ]	✗	7B	68.2	49.2	23.7 ∗	62.1	45.9	51.2 ∗

Table 1: Performance on Mathematical and Logical Reasoning Benchmarks. We compare our VRE model with representative closed-source, open-source general, and open-source reasoning MLLMs across mathematical and logical reasoning Benchmarks. ∗ denotes the results are reproduced by ourselves.

Model	Tool Use	Param Size	$V^{*}$ -Bench			ChartQA			OCRBench_v2		RealworldQA
Model	Tool Use	Param Size	Attr	Spatial	Overall	Human	Aug	Overall	en	zh	-
Closed-Source MLLMs
GPT-4o [ 1 ]	✗	-	60.5	67.5	79.5	91.9	85.7	86.7	46.5	32.2	75.5
Open-Source General MLLMs
LLaVA-OneVision [ 25 ]	✗	7B	75.7	75.0	75.4	-	-	80.0	-	-	66.3
InternVL2.5 [ 10 ]	✗	8B	-	-	-	-	-	84.8	49.8	52.1	70.1

Table 2: Performance on High-Resolution Perception, Document Understanding, and Real-world QA. We compare our VRE model with representative closed-source, open-source general, and open-source reasoning MLLMs across high-resolution perception, document understanding, and Real-world QA. ∗ denotes the results are reproduced by ourselves.

Setting	Cold-SFT	RLVR	Iter-SFT	$V^{*}$ -Bench	MathVista	MathVerse	ChartQA	OCRBench_v2 (en/zh)
Base Model	✗	✗	✗	76.4	68.2	49.2	83.7	56.3/57.2
SFT Only	✓	✗	✗	80.1	69.2	49.6	87.8	59.6 / 62.3
SFT + RL	✓	✓	✗	83.8	71.0	52.5	88.7	62.1 / 64.3
VRE	✓	✓	✓	84.3	71.2	53.1	88.8	62.6 / 64.7

Table 3: Component Ablation. Performance on select benchmarks when progressively adding VRE components.

Figure 2: Visualizing Implicit Visual Re-Examination via Attention. The heatmaps display attention weights on visual tokens across inferences steps. (Left) The Base Model exhibits visual decay, where attention to the image vanishes as the textual context grows, leading to ungrounded hallucinations. (Right) The VRE Model demonstrates a spontaneous attention resurgence. Inside the <reflection> block, the model sharply re-allocates attention back to the visual features, proving that the mechanism actively triggers a re-examination behavior to extract missing visual evidence.

Figure 3: Visualizing the Re-examination Mechanism: From Blindness to Grounding. Phase 1: Initial Reasoning. During the initial pass, the model’s attention is dispersed over the background, failing to locate the dustpan. Phase 2: Visual Re-examination. Triggered by the reflection token, the model performs an active visual search. The attention map shows a sharp, targeted focus on the dustpan (red region), successfully retrieving the correct visual evidence to fix the answer.

Figure 4: Evolution of Reflection Paradigms. The stacked bar charts detail the distribution of introspection types across training stages.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Data Source	Cold-Start SFT	RLVR	Post-RL SFT
Thyme-SFT [ 64 ]	$\sim$ 80k	-	-
MM-Eureka [ 36 ]	-	$\sim$ 20k	$\sim$ 4k
$V^{*}$ Dataset [ 51 ]	-	$\sim$ 20k	$\sim$ 4k
ViRL39K [ 46 ]	-	$\sim$ 20k	$\sim$ 4k
Total Data	$\sim$ 80k	$\sim$ 60k	$\sim$ 12k

Appendix

Table 4: Detailed Composition of Training Data. This table summarizes the data sources and the approximate number of samples in our VRE Framework.

SFT Hyperparameters			Inference / Rollout Hyperparameters
Parameter	Cold-Start	Post-RL	Parameter	Value
Global Batch Size	64	64	Max New Tokens	4900
Learning Rate	1e-5	5e-7	Temperature	1.0
LR Scheduler	Cosine	Cosine	Top- $p$	1.0
Max Sequence Len	4900	4900	Top- $k$	-1
Num Epochs	3	1	Rollout Trajectories ( $N$ )	8

Appendix

Table 5: Hyperparameters for SFT and Inference.

Category	Configuration
Generation	max_tokens=7168, min_tokens=50, temp=1.0, top_p=1.0
RL Algorithm	$\gamma=1.0$ , $\lambda=0.95$ , adv_est=group_norm
KL Control	type=fixed, init_coef=0.01, penalty=kl
GRPO	clip_ratio=0.2, group_size=10, $\lambda_{\text{form}}$ =0.4, $\lambda_{\text{acc}}$ =0.6, $\lambda_{\text{refl}}$ =0.4

Appendix

Table 6: Hyperparameter configurations for RLVR.

Panel of Figure 5 — Figure 5: System prompt template for information gains.

Panel of Figure 6 — Figure 6: Final accuracy evaluation prompt. Dynamic inputs {question} (Question), {ground_truth} (Standard Answer) and {predict_str} (Model_answer) are inserted at inference time.

Panel of Figure 7 — Figure 7: Prompt Template for Elaborative Reflection. Dynamic inputs {question} (Question), {initial_reasoning} (Initial Reasoning) are inserted at inference time.

Panel of Figure 8 — Figure 8: Prompt for evaluating whether reflection blocks demonstrate genuine visual re-examination. Dynamic inputs {question} (Question) and {predict_str} (Model_answer) are inserted at inference time.

Panel of Figure 9 — Figure 9: Prompt Template for Corrective Reflection. Dynamic inputs {question} (Question), {initial_reasoning} (Initial Reasoning) are inserted at inference time.

Panel of Figure 10 — Figure 10: Prompt Template for Recheck Reflection. Dynamic inputs {question} (Question), {initial_reasoning} (Initial Reasoning) are inserted at inference time.

Panel of Figure 11 — Figure 11: Scaling Laws in VRE Training. (a) Cold-Start SFT Data Scaling. There is no significant performance improvement after 50k samples, and the performance is approaching saturation. (b) RL Training Step Scaling. The policy achieves optimal reasoning performance at 100 steps. Prolonged training ( $>150$ steps) induces severe over-optimization and capability degradation.

Models	Evaluation Set	All Correct ( $\uparrow$ )	Mixed	All Wrong ( $\downarrow$ )
Cold-Start	Full Set	30.8%	56.5%	12.7%
RL Model	Full Set	55.6%	38.4%	6.0%
Post-RL SFT	Full Set	58.1%	36.2%	5.7%
Post-RL SFT	Cold-Start “All Wrong” Only	9.2%	58.9%	31.9%

Appendix

Table 7: Distributions of Rollout Trajectories Across Training Stages. Full Set denotes the complete dataset used for RL optimization. Cold-Start “All Wrong” Only refers to the specific subset of “hard queries” where the Cold-Start model failed across all sampled trajectories. This subset highlights the model’s ability to breakthrough its initial cognitive boundaries.

Panel of Figure 12 — Figure 12: KL Divergence Analysis on $V^{*}$ -Bench. (a) Sample-wise average KL divergence. (b) A specific case study showing token-level KL divergence over inference steps.

Figure 13: Information Entropy Comparison on

V^{*}

-Bench. VRE consistently maintains higher conditional entropy than the Semantic-back model. The shift toward higher entropy values validates that VRE’s reflection mechanism actively retrieves novel, non-redundant visual evidence.

Panel of Figure 14 — Figure 14: Training Dynamics of GRPO. (a) The reward score steadily increases and converges around step 50, reflecting stable policy optimization. (b) The average response length stabilizes at $\sim$ 600 tokens, indicating that the model extracts visual evidence efficiently without degenerating into verbose patterns.

Figure 15: Training Loss during Post-RL SFT.

Explore similar work

Feb 25, 2026

See It, Say It, Sorted: An Iterative Training-Free Framework for Visually-Grounded Multimodal Reasoning in LVLMs

Recent large vision-language models (LVLMs) have demonstrated impressive reasoning ability by generating long chain-of-thought (CoT) responses. However, CoT reasoning in multimodal contexts is highly vulnerable to visual hallucination propagation: once an intermediate reasoning step becomes inconsistent with the visual evidence, subsequent steps-even if logically valid-can still lead to incorrect final answers. Existing solutions attempt to mitigate this issue by training models to "think with images" via reinforcement learning (RL). While effective, these methods are costly, model-specific, and difficult to generalize across architectures. Differently, we present a lightweight method that bypasses RL training and provides an iterative, training-free, plug-and-play framework for visually-grounded multimodal reasoning. Our key idea is to supervise each reasoning step at test time with visual evidence, ensuring that every decoded token is justified by corresponding visual cues. Concretely, we construct a textual visual-evidence pool that guides the model's reasoning generation. When existing evidence is insufficient, a visual decider module dynamically extracts additional relevant evidence from the image based on the ongoing reasoning context, expanding the pool until the model achieves sufficient visual certainty to terminate reasoning and produce the final answer. Extensive experiments on multiple LVLM backbones and benchmarks demonstrate the effectiveness of our approach. Our method achieves 16.5%-29.5% improvements on TreeBench and 13.7% RH-AUC gains on RH-Bench, substantially reducing hallucination rates while improving reasoning accuracy without additional training.

Yongchang Zhang, Oliver Ma, Tianyi Liu +2

Southeast University · AIIA, Ministry of Education, China · University of Oxford

Multimodal Reasoning VLM Reasoning VLM Hallucination Mitigation Object Hallucination in VLMs

Ask PDF

Feb 21, 2026

Bridging Modality Disconnect in Self-Reflection via Closed-Loop Visually Grounded Verification

In the era of Vision-Language Models (VLMs), enhancing multimodal reasoning capabilities remains a critical challenge, particularly in handling ambiguous or complex visual inputs, where initial inferences often lead to hallucinations or logic errors. Existing VLMs often produce plausible yet ungrounded answers, and even when prompted to "reflect", their corrections may remain detached from the image evidence. To address this, we propose the MIRROR framework for Multimodal Iterative Reasoning via Reflection On visual Regions. By embedding visual reflection as a core mechanism, MIRROR is formulated as a closed-loop process comprising draft, critique, region-based verification, and revision, which are repeated until the output is visually grounded. To facilitate training of this model, we construct ReflectV, a visual reflective dataset for multi-turn supervision that explicitly contains reflection triggers, region-based verification actions, and answer revision grounded in visual evidence. Experiments on both general vision-language benchmarks and representative vision-language reasoning benchmarks show that MIRROR improves correctness and reduces visual hallucinations, demonstrating the value of training reflection as an evidence-seeking, region-aware verification process rather than a purely textual revision step.

Haoyu Zhang, Yuwei Wu, Pengxiang Li +6

Beijing Institute of Technology · Shenzhen MSU-BIT University

Vision-Language Models VLM Hallucination Mitigation Visually Grounded Reasoning Modality Gap in VLMs

Ask PDF

Aug 29, 2026

Think, Look, and Revise: Inconsistency-Aware Visual Self-Correction in MLLMs

Tool-augmented multimodal reasoning integrates external tools (e.g., object detection, depth estimation) into multimodal large language models (MLLMs) to address perceptual bottlenecks in complex visual tasks. However, existing approaches rarely verify tool outputs, limiting their ability to detect and recover from tool failures. We propose ReVISE, a framework that equips MLLMs with verification and dynamic error recovery for tool-augmented reasoning. ReVISE introduces (1) a curated training dataset that supervises reflective behaviors, enabling models to validate tool-derived evidence, reformulate queries when visual mismatches arise, and fall back to intrinsic grounding when external tools are unreliable; and (2) a reinforcement learning based targeted rewards that encourage internal reflection and penalize spatial misalignment. Experiments on several benchmarks demonstrate consistent improvements over existing methods, highlighting the importance of error detection and correction in tool-augmented multimodal reasoning.

Yu Cheng, Arushi Goel, Hakan Bilen

University of Edinburgh · NVIDIA Research

Tool Use in VLMs Multimodal Large Language Models Multimodal Reasoning

Ask PDF