Vision-language models (VLMs) have advanced rapidly in multimodal reasoning, yet they remain limited on tasks requiring fine-grained understanding of small or easily overlooked image regions. Recent thinking with images approaches mitigate this limitation by allowing models to inspect images through operations such as zooming, cropping, or code execution. While effective, their multi-turn interactions with VLMs introduce redundant token computation and slow inference, and separate region inspection from the reasoning step that depends on it. To address these issues, we propose Look Before You Reason (LoRe), an implicit visual thinking framework for efficient visual reasoning within a single-turn VLM interaction. Given an image and a question, LoRe predicts the most relevant image region, aligns it with the corresponding image patch features, and strengthens the associated key-value (KV) cache entries, enabling subsequent decoding to directly exploit the selected visual cues without additional visual inputs or explicit tool calls. We further employ reinforcement learning to guide the VLM toward identifying informative image regions and leveraging them during reasoning. To support fine-grained evaluation of multimodal reasoning, we introduce COCO-ObjQA, a large-scale benchmark for object-centric multimodal reasoning. Experiments on three challenging multimodal benchmarks demonstrate that LoRe consistently improves VLM reasoning performance with low additional inference overhead.
Figures & tables
Figure 1: Comparison of visual reasoning paradigms. (a) Conventional VLMs rely on static visual tokens, which limits reasoning ability, particularly for small regions. (b) thinking with images approaches depend on costly multi-turn visual manipulation, incurring significant inference overhead. (c) LoRe achieves efficient single-turn implicit visual reasoning via lightweight KV cache modulation.
Figure 2: Overview of LoRe. The VLM emits a region as coordinate tokens. The region detector denotes this generation-and-parsing process, not a separate detection network. LoRe maps the box to visual-token positions and scales only their value-cache entries before decoding continues in the same request. Keys, position IDs, and token order remain unchanged. No crop, additional visual tokens, or second VLM call is required. RL optimizes these trajectories with answer, format, and localization rewards.
Figure 3: GRPO training dynamics for LoRe with Qwen2.5-VL-7B [ 4 ] . The increasing correctness reward with stable response length indicates that RL improves the LoRe’s reasoning ability.
Figure 4: Qualitative examples from COCO-ObjQA. Boxes visualize target COCO annotations across object scales and are not part of the evaluation input. Models receive the original full image and the multiple-choice question.
Figure 5: Qualitative effects of LoRe. LoRe localizes task-relevant visual evidence across different benchmarks before producing the final answer. Yellow boxes denote the region detection results.
Method
Latency
V*
HR-Bench
COCO-ObjQA
Dir. Attr.
Rel. Pos.
Avg.
4K
8K
Avg.
Small
Medium
Large
Avg.
Representative VLMs
InternVL3-8B [ 59 ]
1.25s
72.1
68.3
70.6
62.7
52.4
57.6
79.8
91.7
92.2
89.5
Qwen2.5-VL-7B [ 4 ]
1.92s
75.2
68.1
72.4
69.1
62.8
66.0
72.3
90.1
91.2
86.9
Qwen3-VL-8B [ 3 ]
2.95s
82.4
80.9
81.8
78.6
74.1
76.4
79.2
92.3
93.1
90.0
State-of-the-art Reasoning Approaches
Table 1: Main comparison on fine-grained multimodal reasoning benchmarks. Explicit reasoning baselines use official checkpoints with different training recipes. Gain rows report speedup over the fastest explicit reasoning baseline and relative accuracy gains over the corresponding base VLM. Bold and underlined scores mark the best and second-best results.
Ablation model variant
V* avg
HR-Bench avg
COCO-ObjQA small
COCO-ObjQA avg
COCO-ObjQA IoU
Implicit reasoning ablations
Qwen2.5-VL-7B base
72.4
66.0
72.3
86.9
0.24
+ implicit thinking, w/o RL
77.4
70.2
78.1
89.4
0.31
+ implicit thinking and RL (LoRe)
82.6
73.6
83.1
91.6
0.43
Reward-term ablations
Full reward
82.6
73.6
83.1
91.6
0.43
Table 2: Training-free LoRe, RL training, and reward ablations. LoRe variants use γ=2 , while the base model uses no modulation. These rows do not constitute a fixed-checkpoint ablation of modulation after RL. COCO-ObjQA IoU is mean box IoU, with invalid predictions counted as zero.
Figure 6: KV cache modulation configurations. Left: LoRe policies are trained and evaluated with the same indicated γ . The non-RL base reference is displayed separately. Right: one region performs best overall among learned settings. The oracle row substitutes reference boxes using the same RL checkpoint and γ=2 . This comparison highlights the potential for more accurate region localization to further improve reasoning performance of LoRe.
Policy
γtrain
γtest
V*
HR-Bench
COCO-ObjQA
RL, no modulation
1
1
78.5
70.6
90.1
LoRe, modulation disabled
2
1
76.9
69.8
90.6
LoRe, default
2
2
82.6
73.6
91.6
Crop-based GRPO
–
–
80.9
71.8
92.3
Table 3: Controlled comparisons with Qwen2.5-VL-7B. The two γtrain=2 rows share a checkpoint. Crop-based GRPO matches LoRe’s backbone, training data, recipe, and objectives. All scores are average accuracy (%).
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Samples
Parsed box
Cache update
Fallback
Avg. selected
Avg. visual
Qwen2.5-VL-7B
200
97.5%
97.5%
2.5%
64.5
331.7
Appendix
Table 4: Parser and cache-update diagnostics before RL. We run Qwen2.5-VL-7B on a 200-sample COCO-ObjQA diagnostic subset with max generation length 512, and γ=2.0 .
Figure 7: COCO-ObjQA examples across target-object scales. We visualize representative full-image samples from the small, medium, and large splits. Boxes mark the target COCO instances used to generate object-centric questions, and labels show the category and target-area ratio. The boxes are used for dataset construction and localization diagnostics, while models are evaluated on the original full image.
Intervention
γ
V*
HR-Bench
COCO-ObjQA
Latency
Δ mass
Δ entropy
Additive score bias
2
76.8
68.2
89.3
2.71
+0.24
+0.007
Value scaling
2
77.4
70.2
89.4
2.16
0.00
0.000
Key–value scaling
1.01
74.3
68.5
89.3
2.33
+0.26
+0.005
Appendix
Table 5: Training-free modulation variants. Accuracy is reported in percent and latency in seconds. The final columns give direct changes in selected-token attention mass (percentage points) and attention entropy (nats).
Policy
Benchmark
Fallback rate
Fallback acc.
Non-fallback acc.
Overall acc.
Pre-RL
V*
11.0
61.9
79.3
77.4
Pre-RL
HR-Bench
8.5
64.7
70.7
70.2
Pre-RL
COCO-ObjQA
2.5
79.8
89.6
89.4
RL-trained
V*
6.8
69.2
83.6
82.6
RL-trained
HR-Bench
5.0
70.0
73.8
73.6
RL-trained
COCO-ObjQA
1.5
66.4
92.0
91.6
Appendix
Table 6: Fallback frequency and answer accuracy for Qwen2.5-VL-7B. All entries are percentages. Fallback and non-fallback accuracy are measured within their respective subsets.
Region source
γtest
V*
HR-Bench
COCO-ObjQA
Predicted box
2
82.6
73.6
91.6
Oracle box
1.5
83.5
75.0
92.5
Oracle box
2
84.1
75.8
93.0
Oracle box
2.5
83.8
75.4
92.7
Oracle box
3
83.2
74.7
92.2
Appendix
Table 7: Boost strength with reference regions. All rows use the same RL checkpoint trained at γ=2 . Scores are average accuracy (%).
Benchmark
Both correct
Predicted only
Oracle only
Both wrong
Recovery
V*
80.5
2.1
3.6
13.8
20.7
HR-Bench
71.5
2.1
4.3
22.1
16.3
COCO-ObjQA
89.8
1.8
3.2
5.2
38.1
Appendix
Table 8: Per-example oracle comparison. The four outcome columns partition each benchmark and sum to 100%. Recovery is conditioned on predicted-box errors.
Method
MathVista-testmini
We-Math
TIR-Bench
Qwen2.5-VL-7B
68.6
61.3
16.0
LoRe (pre-RL)
71.3
64.7
16.2
Appendix
Table 9: Training-free transfer to additional reasoning tasks. Scores are accuracy (%).
Large Vision-Language Models (LVLMs) have shown strong promise for multimodal reasoning, yet often struggle with tasks requiring concepts beyond what is directly observable in the input image. Existing methods generate intermediate images or latent visual tokens to guide reasoning, but these representations can introduce errors and increasingly interfere with textual reasoning as reasoning progresses. We propose Visual-to-Text Chain-of-Thought Distillation (V2T), a framework that enables LVLMs to internalize visual reasoning without generating intermediate visual representations at inference time. V2T first trains a teacher LVLM using interleaved visual and textual chains of thought, and then uses knowledge distillation to train a student LVLM using the teacher's logits and cross-entropy supervision from ground-truth textual reasoning. When reasoning images can be mapped to the original image, V2T can additionally distill the teacher's attention to corresponding regions, while ground-truth bounding boxes can further guide a subsequent reinforcement learning stage. Experiments across multiple multimodal reasoning benchmarks show that V2T consistently outperforms the teacher and existing baselines, improving average accuracy by 14.3% on a held-out set and 2.7% on the broader visual evaluation suite. Moreover, lightweight SFT and substantially reduced RL make V2T up to 42x faster to train than state-of-the-art baselines.
Wenhan Yang, Nilay Naharas, Ali Payani +1
University of California, Los Angeles · Cisco Research
Recent large vision-language models (LVLMs) have demonstrated impressive reasoning ability by generating long chain-of-thought (CoT) responses. However, CoT reasoning in multimodal contexts is highly vulnerable to visual hallucination propagation: once an intermediate reasoning step becomes inconsistent with the visual evidence, subsequent steps-even if logically valid-can still lead to incorrect final answers. Existing solutions attempt to mitigate this issue by training models to "think with images" via reinforcement learning (RL). While effective, these methods are costly, model-specific, and difficult to generalize across architectures. Differently, we present a lightweight method that bypasses RL training and provides an iterative, training-free, plug-and-play framework for visually-grounded multimodal reasoning. Our key idea is to supervise each reasoning step at test time with visual evidence, ensuring that every decoded token is justified by corresponding visual cues. Concretely, we construct a textual visual-evidence pool that guides the model's reasoning generation. When existing evidence is insufficient, a visual decider module dynamically extracts additional relevant evidence from the image based on the ongoing reasoning context, expanding the pool until the model achieves sufficient visual certainty to terminate reasoning and produce the final answer. Extensive experiments on multiple LVLM backbones and benchmarks demonstrate the effectiveness of our approach. Our method achieves 16.5%-29.5% improvements on TreeBench and 13.7% RH-AUC gains on RH-Bench, substantially reducing hallucination rates while improving reasoning accuracy without additional training.
Yongchang Zhang, Oliver Ma, Tianyi Liu +2
Southeast University · AIIA, Ministry of Education, China · University of Oxford
Multimodal Large Language Models (MLLMs) are shifting towards "Thinking with Images" by actively exploring image details. While effective, large-scale training is computationally expensive, which has spurred growing interest in lightweight, training-free solutions. However, existing training-free methods suffer from two flaws: perceptual redundancy from indiscriminate cropping, which increases computational cost and introduces noise; and a drift between semantic intent and spatial attention, which prevents accurate localization of user-focused regions. To address these challenges, we propose LookWise, a framework for adaptive visual reasoning. LookWise follows a two-stage pipeline: a confidence-based module decides when to look more carefully, and a semantic-guided localization module determines where to look. This design enables MLLMs to adaptively acquire fine-grained visual evidence without additional training. Experiments on fine-grained and high-resolution visual reasoning benchmarks show that LookWise consistently improves accuracy over strong baselines while achieving an approximately 4.0× inference speedup over the search-based method ZoomEye, demonstrating robust cross-model generalization.
Yuxiang Shen, Hailong Huang, Zhenkun Gao +6
the Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei 230031, China, and also with the University of Science and Technology of China, Hefei 230026, China · Zhejiang University, Hangzhou 310027, China · East China Normal University, Shanghai 200241, China +3