Bridging Modality Disconnect in Self-Reflection via Closed-Loop Visually Grounded Verification

Haoyu Zhang; Yuwei Wu; Pengxiang Li; Xintong Zhang; Zhi Gao; Rui Gao; Mingyang Gao; Che Sun

cs.CVFeb 21, 2026

Bridging Modality Disconnect in Self-Reflection via Closed-Loop Visually Grounded Verification

Authors: Haoyu Zhang, Yuwei Wu, Pengxiang Li, Xintong Zhang, Zhi Gao, Rui Gao, Mingyang Gao, Che Sun, +1 more

Organizations: Beijing Institute of Technology · Shenzhen MSU-BIT University

Abstract

In the era of Vision-Language Models (VLMs), enhancing multimodal reasoning capabilities remains a critical challenge, particularly in handling ambiguous or complex visual inputs, where initial inferences often lead to hallucinations or logic errors. Existing VLMs often produce plausible yet ungrounded answers, and even when prompted to "reflect", their corrections may remain detached from the image evidence. To address this, we propose the MIRROR framework for Multimodal Iterative Reasoning via Reflection On visual Regions. By embedding visual reflection as a core mechanism, MIRROR is formulated as a closed-loop process comprising draft, critique, region-based verification, and revision, which are repeated until the output is visually grounded. To facilitate training of this model, we construct ReflectV, a visual reflective dataset for multi-turn supervision that explicitly contains reflection triggers, region-based verification actions, and answer revision grounded in visual evidence. Experiments on both general vision-language benchmarks and representative vision-language reasoning benchmarks show that MIRROR improves correctness and reduces visual hallucinations, demonstrating the value of training reflection as an evidence-seeking, region-aware verification process rather than a purely textual revision step.

Figures & tables

Figure 1: Comparison of existing multimodal reasoning paradigms. MIRROR explicitly seeks visual evidence to achieve closed-loop visually grounded verification.

Figure 2: MIRROR performs closed-loop: it iteratively generates an initial response, identifies response uncertainties, invokes the visual prompt generator for verification, and revises based on the rendered visual evidence.

Figure 3: Overview of the ReflectV construction pipeline. Multi-turn reflective trajectories are constructed from diverse multimodal datasets and further processed into visually grounded verification trajectories.

Model	General Capabilities			OCR & Document
	MM-Vet	MMStar	SeedBench-2-Plus	TextVQA-Val	OCRBench	ChartQA-Test
LLaVA-OneVision-7B	48.80	61.70	–	76.10	62.10	80.00
InternVL3-2B	54.95	60.70	64.95	77.00	82.20	76.08
InternVL3-8B	64.27	61.50	69.61	80.51	85.00	79.64
Qwen2.5-VL-3B	47.39	55.87	68.81	79.12	82.60	83.20
Qwen2.5-VL-7B	56.60	61.21	70.88	84.90	83.20	86.08

Table 1: Performance on General and OCR & Document benchmarks. The best and second-best results are highlighted in bold and underlined . MIRROR (w/o loop) is an open-loop variant without reflective loop.

Model	Hallucination		Fine-grained Perception			Math
	POPE	HallusionBench	HRBench-4K	MME-RW	VStarBench	MathVision
LLaVA-OneVision-7B	78.10	31.60	63.00	–	72.30	18.30
InternVL3-2B	89.60	42.50	61.75	43.88	68.59	21.71
InternVL3-8B	90.37	49.90	70.00	48.83	68.06	20.39
Qwen2.5-VL-3B	86.21	63.09	50.25	42.15	72.77	25.66
Qwen2.5-VL-7B	86.45	68.66	68.87	44.29	75.39	23.36

Table 2: Performance comparison on Hallucination, Fine-grained Perception, and Math benchmarks. The best and second-best results are highlighted in bold and underlined , respectively. MME-RW indicates MME-RealWorld-Lite.

Figure 4: Qualitative examples of iterative visual reflection. Compared to Qwen2.5-VL and MIRROR (w/o loop) (without reflective loop), MIRROR successfully corrects errors via visually grounded verification.

Figure 5: Visualization of per-token attention maps in the first reasoning round (before tool invocation). Correct target regions (the specific value “56” and the “van”) are highlighted with red boxes in MIRROR’s attention maps.

Method	OCRBench	POPE	MME-RW	MM-Vet
Text Reflection
VL-Rethinker	85.40	84.19	47.21	56.19
Visual Reflection
LookBack (Solution)	87.50	88.20	49.80	63.50
LookBack (Semantic)	88.60	89.80	50.40	65.10
Thinking with Images

Table 3: Performance comparison with SOTA reasoning methods, all fine-tuned on Qwen2.5-VL-7B. Best and second-best results are bold and underlined .

Method	OCRBench	POPE	VStarBench	MM-Vet
Qwen2.5-VL-3B	82.60	86.20	72.77	47.39
+vanilla-SFT	81.60	86.31	75.92	56.33
MIRROR-3B	85.40	87.74	80.63	60.60
Qwen2.5-VL-7B	83.20	86.45	75.39	56.60
+vanilla-SFT	86.90	86.99	78.01	61.01
MIRROR-7B	92.00	94.42	83.77	66.70

Table 4: Ablation study on training paradigms, model sizes, and architectures. Best and second-best results are bold and underlined .

Ratio $\rho$	OCR	Math	Spat	Gen	Rec	Avg.
$\rho=0.5$	61.85	51.54	59.73	45.75	55.61	55.87
$\rho=0.6$	64.81	56.15	62.13	56.75	61.28	60.32
$\rho=1.0$	65.19	56.54	61.47	54.12	59.04	58.81
$\rho=0.75$	69.90	60.77	70.80	61.62	64.40	66.70

Table 5: Ablation study regarding multi-turn sample ratio

\rho

for trajectory adaptation on MM-Vet. Best and second-best results are bold and underlined .

Configuration	Loop	Tools	MM-Vet
Qwen2.5-VL-7B	-	-	56.60
Qwen2.5-VL-7B-Tools	-	✓	57.90
Qwen2.5-VL-7B-MIRROR	✓	✓	57.10
MIRROR (w/o loop)	-	-	59.91
MIRROR-Tools	-	✓	62.70
MIRROR (ours)	✓	✓	66.70

Table 6: Ablation study of the closed-loop reflection mechanism on the MM-Vet benchmark (Average Score). Bold and underlined indicate the best and second-best performance, respectively.

Figure 6: Statistics and distribution of the ReflectV dataset construction. (a). Domain Distribution: The composition of the raw data spans four distinct capabilities: General QA, Document Understanding (Doc), Scene Text (OCR), and Chart Reasoning. (b). Filtering Pipeline: The data volume retention across the three construction stages (Original $\rightarrow$ Response-Filtered $\rightarrow$ GT-Filtered), illustrating the rigorous quality control process described in Section 4.1 . (c). Trajectory Depth: We set the default mixing ratio to $\rho=0.75$ for our training. The distribution of samples based on the number of self-reflection rounds required for convergence illustrates the varying complexity of error correction.

Appendix figures & tables32 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Human		GPT-4o-mini		Average
Dataset	Logic	Visual	Logic	Visual	Logic	Visual
$D_{noise}$	2.14	1.38	2.30	1.45	2.22	1.42
$D_{raw}$	3.56	3.12	3.65	3.20	3.61	3.16
$D_{clean}$	4.78	4.65	4.82	4.70	4.80	4.68

Appendix

Table 7: Comparative quality assessment. We evaluate samples from Discarded (

D_{noise}

), Unfiltered (

D_{raw}

), and Filtered (

D_{clean}

) subsets. The results demonstrate that our pipeline effectively isolates high-quality trajectories.

Figure 7: Distribution of inference trajectory lengths across representative benchmarks. The stacked bars illustrate the percentage of samples resolved within 1, 2, or 3+ rounds.

(a) Part 1 of the case from $D_{\text{noise}}$ .

Panel of Figure 9 — Figure 9: Case 1 from $D_{\text{noise}}$ .

Model	OCR	Math	Spat	Gen	Rec	Overall
MIRROR-Raw	56.39	41.54	55.20	38.63	55.88	55.87
MIRROR-1.0	65.19	56.54	61.47	54.12	59.04	58.81
MIRROR-0.75	69.90	60.77	70.80	61.62	64.40	66.70

Appendix

Table 8: Ablation study on data filtering strategy conducted on MM-Vet. We compare MIRROR-Raw with MIRROR-1.0 and our final MIRROR-0.75 (

\rho=0.75

). The best and second-best results are highlighted in bold and underlined , respectively.

Method	C $\rightarrow$ C	W $\rightarrow$ C $\uparrow$	C $\rightarrow$ W $\downarrow$	W $\rightarrow$ W $\downarrow$
Qwen2.5-VL- 7B-MIRROR	28.0	28.5	15.0	28.5
MIRROR (ours)	25.0	69.0	3.5	2.5

Appendix

Table 9: Reflection outcome analysis on the triggered multi-turn subset of MM-Vet. Distribution of correctness transitions (%) on the triggered multi-turn subset of MM-Vet.

Figure 10: Subject Extraction Prompt: Extracts core subjects and key modifiers from questions to form concise visual descriptions for Chart, OCR, and Doc understanding tasks.

Figure 11: Visual Caption Generation Prompt: A two-step pipeline that first identifies key physical objects from feedback and then synthesizes them into short descriptive captions for QA tasks.

Figure 12: Caption Insertion Prompt: Seamlessly integrates generated visual captions and spatial marks (e.g., points) into textual feedback to ensure visual grounding.

Figure 13: Self-Reflection Conversion Prompt: Rewrites teacher-side evaluations into first-person student self-reflections (e.g., changing "Your response is wrong" to "I think my response is wrong").

(a) Part 1 of system prompt.

Figure 15: System prompt during training.

Figure 16: User prompt during training.

Method	Time (s) $\downarrow$	Tokens $\downarrow$
Text Reflection
VL-Rethinker	5.56	229.33
Visual Reflection
Look-Back (Semantic)	89.34	307.90
Look-Back (Solution)	300.86	427.22
Thinking with Images

Appendix

Table 10: Efficiency comparison on the MM-Vet dataset. We report the average time cost (seconds) and token consumption per sample. Bold and underlined indicate the best and second-best efficiency, respectively.

Configuration	Time (s) $\downarrow$	Tokens $\downarrow$
Qwen2.5-VL-7B	2.81	85.50
Qwen2.5-VL-7B-Tools	6.65	180.40
Qwen2.5-VL-7B-MIRROR	4.69	156.30
MIRROR (w/o loop)	3.25	98.50
MIRROR-Tools	5.74	161.70
MIRROR (ours)	3.73	112.58

Appendix

Table 11: Efficiency breakdown of ablation configurations on MM-Vet. Time and token costs are per sample. Bold and underlined indicate the best and second-best, respectively.

(a) Part 1 of comparison case 1 with Qwen2.5-VL-7B.

Panel of Figure 18 — Figure 18: Comparison case 1 with Qwen2.5-VL-7B.

Figure 19: Comparison case 2 with Qwen2.5-VL-7B.

(a) Part 1 of comparison case 1 with reasoning paradigms.

(a) Part 2 of comparison case 1 with reasoning paradigms.

Figure 22: Comparison case 1 with reasoning paradigms.

(a) Part 1 of comparison case 2 with reasoning paradigms.

(a) Part 2 of comparison case 2 with reasoning paradigms.

Panel of Figure 25 — Figure 25: Comparison case 2 with reasoning paradigms.

(a) Part 1 of comparison case 3 with reasoning paradigms.

(a) Part 2 of comparison case 3 with reasoning paradigms.

Panel of Figure 28 — Figure 28: Comparison case 3 with reasoning paradigms.

(a) Part 1 of comparison case 4 with reasoning paradigms.

Panel of Figure 30 — Figure 30: Comparison case 4 with reasoning paradigms.

(a) Part 1 of Qualitative failure case 1.

Panel of Figure 32 — Figure 32: Limited Applicability in Abstract Domains: When addressing mathematical problems, the model attempts to verify symbolic logic (e.g., radius calculation) visually. However, since abstract derivations cannot be spatially grounded, the resulting visual prompts (red points) are uninformative and do not aid in verifying the calculation.

Figure 33: Coarse-grained Attribute Binding: This case demonstrates the limitations of the current visual verification mechanism.

Explore similar work

Reflect to Inform: Boosting Multimodal Reasoning via Information-Gain-Driven Verification
Mar 27, 2026Shuai Lv, Chang Liu, Feng Tang +5Multimodal Large Language Models Multimodal Reasoning
University of Science and Technology of China, Hefei, China · Huawei Foundation Model Department, Shanghai, China · The Chinese University of Hong Kong, Hong Kong, China +1
See It, Say It, Sorted: An Iterative Training-Free Framework for Visually-Grounded Multimodal Reasoning in LVLMs
Feb 25, 2026Yongchang Zhang, Oliver Ma, Tianyi Liu +2Multimodal Reasoning VLM Reasoning
Southeast University · AIIA, Ministry of Education, China · University of Oxford
Visually Grounded Self-Reflection for Vision-Language Models via Reinforcement Learning
Jul 2, 2026Liyan Tang, Fangcong Yin, Greg DurrettVLM Robustness Vision-Language Models
The University of Texas at Austin · New York University

Mar 27, 2026J/K move · Enter open · S save

Reflect to Inform: Boosting Multimodal Reasoning via Information-Gain-Driven Verification

Shuai Lv, Chang Liu, Feng Tang, Yujie Yuan+4

University of Science and Technology of China, Hefei, China · Huawei Foundation Model Department, Shanghai, China · The Chinese University of Hong Kong, Hong Kong, China+1

Multimodal Large Language Models Multimodal Reasoning VLM Hallucination Mitigation Visual Grounding

Open paper AskPDF arXiv

Multimodal Large Language Models (MLLMs) achieve strong multimodal reasoning performance, yet we identify a recurring failure mode in long-form generation: as outputs grow longer, models progressively drift away from image evidence and fall back on textual priors, resulting in ungrounded reasoning and hallucinations. Interestingly, Based on attention analysis, we find that MLLMs have a latent capability for late-stage visual verification that is present but not consistently activated. Motivated by this observation, we propose Visual Re-Examination (VRE), a self-evolving training framework that enables MLLMs to autonomously perform visual introspection during reasoning without additional visual inputs. Rather than distilling visual capabilities from a stronger teacher, VRE promotes iterative self-improvement by leveraging the model itself to generate reflection traces, making visual information actionable through information gain. Extensive experiments across diverse multimodal benchmarks demonstrate that VRE consistently improves reasoning accuracy and perceptual reliability, while substantially reducing hallucinations, especially in long-chain settings. Code is available at https://github.com/Xiaobu-USTC/VRE.

Abstract

Explore similar work

Reflect to Inform: Boosting Multimodal Reasoning via Information-Gain-Driven Verification

See It, Say It, Sorted: An Iterative Training-Free Framework for Visually-Grounded Multimodal Reasoning in LVLMs

Visually Grounded Self-Reflection for Vision-Language Models via Reinforcement Learning