V-Reflection: Transforming MLLMs from Passive Observers to Active Interrogators
Authors: Jiazhou Zhou, Yucheng Chen, Hongyang Li, Qing Jiang, Hu Zhou, Ying-Cong Chen, Lei Zhang
Organizations: AI Thrust, The Hong Kong University of Science and Technology (Guangzhou) · International Digital Economy Academy · MedVisAI Lab, Lee Kong Chian School of Medicine, Nanyang Technological University, and Centre of AI in Medicine · South China University of Technology · Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University
Multimodal Large Language Models (MLLMs) have achieved remarkable success, yet they remain prone to perception-related hallucinations in fine-grained tasks. This vulnerability arises from a fundamental limitation: their reasoning is largely restricted to the language domain, treating visual input as a static, reasoning-agnostic preamble rather than a dynamic participant. Consequently, current models act as passive observers, unable to re-examine visual details to ground their evolving reasoning states. To overcome this, we propose V-Reflection, a framework that transforms the MLLM into an active interrogator through a "think-then-look" visual reflection mechanism. During reasoning, latent states function as dynamic probes that actively interrogate the visual feature space, grounding each reasoning step for task-critical evidence. Our approach employs a two-stage distillation strategy. First, the Box-Guided Compression Module (BCM) establishes stable pixel-to-latent targets through explicit spatial grounding. Next, a Dynamic Autoregressive Compression (DAC) module maps the model's hidden states into dynamic probes that interrogate the global visual feature map. By distilling the spatial expertise of the BCM teacher into the DAC student, V-Reflection internalizes the ability to localize task-critical evidence. During inference, both modules remain entirely inactive, maintaining a purely end-to-end autoregressive decoding in the latent space with optimal efficiency. Extensive experiments demonstrate the effectiveness of our V-Reflection across six perception-intensive benchmarks, significantly narrowing the fine-grained perception gap. Visualizations confirm that latent reasoning autonomously localizes task-critical visual evidence.
Figures & tables
Figure 1: Conceptual comparison between traditional MLLMs and our V-Reflection . Current MLLMs’ reasoning remains confined to the language domain, treating visual information as a reasoning-agnostic static input rather than an active driver of the thought process, thus leading to perception-related hallucinations (e.g., ‘Kevlar’) where the model prioritizes language priors over actual visual evidence. (b) Our framework internalizes a "think-then-look" visual self-reflection mechanism, where evolving latent states act as dynamic probes ( Qdyn ) to retrace global visual features. This mechanism retrieves task-specific evidence (e.g., accurately localizing the rubber glove), effectively correcting the reasoning trajectory for a precise answer.
Figure 2: The V-Reflection Architecture. A two-stage paradigm establishes a "think-then-look" visual self-reflection reasoning mechanism. (a) Stage 1: The Box-Guided Compression (BCM) module distills regional patches into grounded latent tokens ZT via LBCM . (b) Stage 2: The Dynamic Autoregressive Compression (DAC) module distills the spatial expertise of the BCM module, training the LLM’s hidden states H to act as dynamic probes that autonomously interrogate global features. (c) Inference: Both BCM and DAC remain entirely inactive, as they have been fully internalized to execute a purely end-to-end visual search driven by its latent reasoning process.
Method
Param Size
MMVP
BLINK
V*
Overall
Overall
Counting
IQ-Test
JigSaw
Relative Reflect
Spatial Relation
Overall
V D.A.∗
V R.P.∗
Proprietary Model
GPT-4o [ 10 ]
-
58.33
51.1
51.7
30.0
58.0
38.8
76.9
62.8
-
-
Open-Source Model
Qwen2.5-VL-7B
7B
66.7
54.5
65.8
27.3
52.7
41.0
88.1
78.5
81.7
73.7
+ vanilla SFT
7B
69.5
53.1
60.8
26.7
45.3
33.6
88.8
79.1
82.6
73.7
Table 1: Performance comparison on visual perception and cognitive benchmarks. Comparison results are reported by [ 12 ] .
Method
Param Size
HRBench-4K
HRBench-8K
MME-Real-Lite
Overall
FSP
FCP
Overall
FSP
FCP
Overall
Reasoning
Perception
Proprietary Model
GPT-4o [ 10 ]
-
65.0
66.8
63.3
59.6
60.8
58.5
52.0
48.3
54.4
Open-Source Model
Qwen2.5-VL-7B
7B
68.0
80.3
55.8
63.8
73.8
51.8
45.8
39.7
49.6
+ vanilla SFT
7B
69.2
78.8
59.8
64.5
76.8
52.3
50.1
42.5
54.1
Table 2: Performance on real-world high-resolution perception and reasoning benchmarks. Results marked with † are reported by [ 37 ] , while others are reported by [ 24 ] .
BCM
DAC
MMVP
HRBench-4K
BLINK
-
-
69.5
68.0
53.1
✓
-
70.9
71.1
54.2
-
✓
65.9
66.8
52.2
✓
✓
72.3
72.6
56.4
Table 3: Ablation Studies of V-Reflection . We evaluate V-Reflection across four dimensions: (a) contribution of core modules BCM and DAC; (b) effectiveness of LBCM & LDAC Loss; (c) necessity of the two-stage training paradigm; (d) impact of different Qdyn sources for the DAC module; (e) inference efficiency analysis; and (f) impact of latent steps ( S ).
Figure 3: Visualization of Latent Reasoning during Training. Averaged attention maps across all reasoning steps demonstrate that while the teacher (BCM) is confined to local priors, the student (DAC) successfully transcends bounding box constraints to capture global contextual relationships.
Figure 4: Visualization of Latent Reasoning during Inference. Averaged attention maps across all latent reasoning steps (Col. 3) autonomously pinpointing visual evidence driven by latent states.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
p
MMVP
HRBench-4K
BLINK
0.1
69.5
69.1
53.1
0.3
69.8
70.4
53.7
0.5
70.9
71.1
54.2
0.7
70.4
70.2
53.5
0.9
70.1
69.7
52.9
Appendix
Table 4: Ablation Study on Bernoulli Probability p in the Stochastic Decoupled Alignment Strategy.
Figure 5: Visualization of visual latent distillation during training. While the teacher (BCM) is confined to local priors, the student (DAC) successfully transcends bounding box constraints to capture global contextual relationships.
Figure 6: Visualization of visual latent reasoning during inference. Rows 1–4: cross-image; Rows 5–6: direct attribute.
Figure 7: Visualization of visual latent reasoning during inference. Row 1: OCR; Rows 2–3: tables and diagrams; Rows 4–5: IQ tests.
Figure 8: Visualization of visual latent reasoning during inference. Rows 1–2: monitoring; Rows 3–4: remote sensing; Rows 5-6: autonomous driving.
In the era of Vision-Language Models (VLMs), enhancing multimodal reasoning capabilities remains a critical challenge, particularly in handling ambiguous or complex visual inputs, where initial inferences often lead to hallucinations or logic errors. Existing VLMs often produce plausible yet ungrounded answers, and even when prompted to "reflect", their corrections may remain detached from the image evidence. To address this, we propose the MIRROR framework for Multimodal Iterative Reasoning via Reflection On visual Regions. By embedding visual reflection as a core mechanism, MIRROR is formulated as a closed-loop process comprising draft, critique, region-based verification, and revision, which are repeated until the output is visually grounded. To facilitate training of this model, we construct ReflectV, a visual reflective dataset for multi-turn supervision that explicitly contains reflection triggers, region-based verification actions, and answer revision grounded in visual evidence. Experiments on both general vision-language benchmarks and representative vision-language reasoning benchmarks show that MIRROR improves correctness and reduces visual hallucinations, demonstrating the value of training reflection as an evidence-seeking, region-aware verification process rather than a purely textual revision step.
Haoyu Zhang, Yuwei Wu, Pengxiang Li +6
Beijing Institute of Technology · Shenzhen MSU-BIT University
Multimodal large language models (MLLMs) often suffer from perceptual impairments under extended reasoning modes, particularly in visual question answering (VQA) tasks. We identify attention dispersion as the underlying cause: during multi-step reasoning, the model's visual attention becomes scattered and drifts away from question-relevant regions, effectively "losing focus" on the visual input. To better understand this phenomenon, we analyze the attention maps of MLLMs and observe that reasoning prompts significantly reduce attention to regions critical for answering the question. We further find a strong correlation between the model's overall attention on image tokens and the spatial dispersiveness of its attention within the image. Leveraging this insight, we propose a training-free Visual Region-Guided Attention (VRGA) framework that selects visual heads based on an entropy-focus criterion and reweights their attention, effectively guiding the model to focus on question-relevant regions during reasoning. Extensive experiments on vision-language benchmarks demonstrate that our method effectively alleviates perceptual degradation, leading to improvements in visual grounding and reasoning accuracy while providing interpretable insights into how MLLMs process visual information.
Ruiying Peng, Xueyu Wu, Jing Lei +3
Tsinghua Shenzhen International Graduate School · Huawei Technologies
Multimodal Large Language Models (MLLMs) achieve strong multimodal reasoning performance, yet we identify a recurring failure mode in long-form generation: as outputs grow longer, models progressively drift away from image evidence and fall back on textual priors, resulting in ungrounded reasoning and hallucinations. Interestingly, Based on attention analysis, we find that MLLMs have a latent capability for late-stage visual verification that is present but not consistently activated. Motivated by this observation, we propose Visual Re-Examination (VRE), a self-evolving training framework that enables MLLMs to autonomously perform visual introspection during reasoning without additional visual inputs. Rather than distilling visual capabilities from a stronger teacher, VRE promotes iterative self-improvement by leveraging the model itself to generate reflection traces, making visual information actionable through information gain. Extensive experiments across diverse multimodal benchmarks demonstrate that VRE consistently improves reasoning accuracy and perceptual reliability, while substantially reducing hallucinations, especially in long-chain settings. Code is available at https://github.com/Xiaobu-USTC/VRE.
Shuai Lv, Chang Liu, Feng Tang +5
University of Science and Technology of China, Hefei, China · Huawei Foundation Model Department, Shanghai, China · The Chinese University of Hong Kong, Hong Kong, China +1