Look Before You Reason: Implicit Visual Thinking for Efficient Multimodal Reasoning

Zerui Chen; Changrui Chen; Fei Ni; Jiankang Deng

cs.CVOct 19, 2026

Look Before You Reason: Implicit Visual Thinking for Efficient Multimodal Reasoning

Authors: Zerui Chen, Changrui Chen, Fei Ni, Jiankang Deng

Organizations: MVP Lab, Imperial College London

Abstract

Vision-language models (VLMs) have advanced rapidly in multimodal reasoning, yet they remain limited on tasks requiring fine-grained understanding of small or easily overlooked image regions. Recent thinking with images approaches mitigate this limitation by allowing models to inspect images through operations such as zooming, cropping, or code execution. While effective, their multi-turn interactions with VLMs introduce redundant token computation and slow inference, and separate region inspection from the reasoning step that depends on it. To address these issues, we propose Look Before You Reason (LoRe), an implicit visual thinking framework for efficient visual reasoning within a single-turn VLM interaction. Given an image and a question, LoRe predicts the most relevant image region, aligns it with the corresponding image patch features, and strengthens the associated key-value (KV) cache entries, enabling subsequent decoding to directly exploit the selected visual cues without additional visual inputs or explicit tool calls. We further employ reinforcement learning to guide the VLM toward identifying informative image regions and leveraging them during reasoning. To support fine-grained evaluation of multimodal reasoning, we introduce COCO-ObjQA, a large-scale benchmark for object-centric multimodal reasoning. Experiments on three challenging multimodal benchmarks demonstrate that LoRe consistently improves VLM reasoning performance with low additional inference overhead.

Figures & tables

Figure 1: Comparison of visual reasoning paradigms. (a) Conventional VLMs rely on static visual tokens, which limits reasoning ability, particularly for small regions. (b) thinking with images approaches depend on costly multi-turn visual manipulation, incurring significant inference overhead. (c) LoRe achieves efficient single-turn implicit visual reasoning via lightweight KV cache modulation.

Figure 2: Overview of LoRe. The VLM emits a region as coordinate tokens. The region detector denotes this generation-and-parsing process, not a separate detection network. LoRe maps the box to visual-token positions and scales only their value-cache entries before decoding continues in the same request. Keys, position IDs, and token order remain unchanged. No crop, additional visual tokens, or second VLM call is required. RL optimizes these trajectories with answer, format, and localization rewards.

Figure 3: GRPO training dynamics for LoRe with Qwen2.5-VL-7B [ 4 ] . The increasing correctness reward with stable response length indicates that RL improves the LoRe’s reasoning ability.

Figure 4: Qualitative examples from COCO-ObjQA. Boxes visualize target COCO annotations across object scales and are not part of the evaluation input. Models receive the original full image and the multiple-choice question.

Figure 5: Qualitative effects of LoRe. LoRe localizes task-relevant visual evidence across different benchmarks before producing the final answer. Yellow boxes denote the region detection results.

Representative VLMs
Method	Latency	V*			HR-Bench			COCO-ObjQA
		Dir. Attr.	Rel. Pos.	Avg.	4K	8K	Avg.	Small	Medium	Large	Avg.
InternVL3-8B [ 59 ]	1.25s	72.1	68.3	70.6	62.7	52.4	57.6	79.8	91.7	92.2	89.5
Qwen2.5-VL-7B [ 4 ]	1.92s	75.2	68.1	72.4	69.1	62.8	66.0	72.3	90.1	91.2	86.9
Qwen3-VL-8B [ 3 ]	2.95s	82.4	80.9	81.8	78.6	74.1	76.4	79.2	92.3	93.1	90.0
State-of-the-art Reasoning Approaches

Table 1: Main comparison on fine-grained multimodal reasoning benchmarks. Explicit reasoning baselines use official checkpoints with different training recipes. Gain rows report speedup over the fastest explicit reasoning baseline and relative accuracy gains over the corresponding base VLM. Bold and underlined scores mark the best and second-best results.

Implicit reasoning ablations
Ablation model variant	V* avg	HR-Bench avg	COCO-ObjQA small	COCO-ObjQA avg	COCO-ObjQA IoU
Qwen2.5-VL-7B base	72.4	66.0	72.3	86.9	0.24
+ implicit thinking, w/o RL	77.4	70.2	78.1	89.4	0.31
+ implicit thinking and RL (LoRe)	82.6	73.6	83.1	91.6	0.43
Reward-term ablations
Full reward	82.6	73.6	83.1	91.6	0.43

Table 2: Training-free LoRe, RL training, and reward ablations. LoRe variants use

\gamma=2

, while the base model uses no modulation. These rows do not constitute a fixed-checkpoint ablation of modulation after RL. COCO-ObjQA IoU is mean box IoU, with invalid predictions counted as zero.

Figure 6: KV cache modulation configurations. Left: LoRe policies are trained and evaluated with the same indicated

\gamma

. The non-RL base reference is displayed separately. Right: one region performs best overall among learned settings. The oracle row substitutes reference boxes using the same RL checkpoint and

\gamma=2

. This comparison highlights the potential for more accurate region localization to further improve reasoning performance of LoRe.

Policy	$\gamma_{\mathrm{train}}$	$\gamma_{\mathrm{test}}$	V*	HR-Bench	COCO-ObjQA
RL, no modulation	1	1	78.5	70.6	90.1
LoRe, modulation disabled	2	1	76.9	69.8	90.6
LoRe, default	2	2	82.6	73.6	91.6
Crop-based GRPO	–	–	80.9	71.8	92.3

Table 3: Controlled comparisons with Qwen2.5-VL-7B. The two

\gamma_{\mathrm{train}}=2

rows share a checkpoint. Crop-based GRPO matches LoRe’s backbone, training data, recipe, and objectives. All scores are average accuracy (%).

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Samples	Parsed box	Cache update	Fallback	Avg. selected	Avg. visual
Qwen2.5-VL-7B	200	97.5%	97.5%	2.5%	64.5	331.7

Appendix

Table 4: Parser and cache-update diagnostics before RL. We run Qwen2.5-VL-7B on a 200-sample COCO-ObjQA diagnostic subset with max generation length 512, and

\gamma=2.0

Figure 7: COCO-ObjQA examples across target-object scales. We visualize representative full-image samples from the small, medium, and large splits. Boxes mark the target COCO instances used to generate object-centric questions, and labels show the category and target-area ratio. The boxes are used for dataset construction and localization diagnostics, while models are evaluated on the original full image.

Intervention	$\gamma$	V*	HR-Bench	COCO-ObjQA	Latency	$\Delta$ mass	$\Delta$ entropy
Additive score bias	2	76.8	68.2	89.3	2.71	+0.24	+0.007
Value scaling	2	77.4	70.2	89.4	2.16	0.00	0.000
Key–value scaling	1.01	74.3	68.5	89.3	2.33	+0.26	+0.005

Appendix

Table 5: Training-free modulation variants. Accuracy is reported in percent and latency in seconds. The final columns give direct changes in selected-token attention mass (percentage points) and attention entropy (nats).

Policy	Benchmark	Fallback rate	Fallback acc.	Non-fallback acc.	Overall acc.
Pre-RL	V*	11.0	61.9	79.3	77.4
Pre-RL	HR-Bench	8.5	64.7	70.7	70.2
Pre-RL	COCO-ObjQA	2.5	79.8	89.6	89.4
RL-trained	V*	6.8	69.2	83.6	82.6
RL-trained	HR-Bench	5.0	70.0	73.8	73.6
RL-trained	COCO-ObjQA	1.5	66.4	92.0	91.6

Appendix

Table 6: Fallback frequency and answer accuracy for Qwen2.5-VL-7B. All entries are percentages. Fallback and non-fallback accuracy are measured within their respective subsets.

Region source	$\gamma_{\mathrm{test}}$	V*	HR-Bench	COCO-ObjQA
Predicted box	2	82.6	73.6	91.6
Oracle box	1.5	83.5	75.0	92.5
Oracle box	2	84.1	75.8	93.0
Oracle box	2.5	83.8	75.4	92.7
Oracle box	3	83.2	74.7	92.2

Appendix

Table 7: Boost strength with reference regions. All rows use the same RL checkpoint trained at

\gamma=2

. Scores are average accuracy (%).

Benchmark	Both correct	Predicted only	Oracle only	Both wrong	Recovery
V*	80.5	2.1	3.6	13.8	20.7
HR-Bench	71.5	2.1	4.3	22.1	16.3
COCO-ObjQA	89.8	1.8	3.2	5.2	38.1

Appendix

Table 8: Per-example oracle comparison. The four outcome columns partition each benchmark and sum to 100%. Recovery is conditioned on predicted-box errors.

Method	MathVista-testmini	We-Math	TIR-Bench
Qwen2.5-VL-7B	68.6	61.3	16.0
LoRe (pre-RL)	71.3	64.7	16.2

Appendix

Table 9: Training-free transfer to additional reasoning tasks. Scores are accuracy (%).

Multimodal Reasoning

Multimodal reasoning studies how AI systems combine text with images, audio, video, and other signals to solve evidence-dependent tasks.

See all 508 papers

Distilling Visual Reasoning into Text Space
Sep 28, 2026Wenhan Yang, Nilay Naharas, Ali Payani +1CoT Distillation VLM Distillation
University of California, Los Angeles · Cisco Research
See It, Say It, Sorted: An Iterative Training-Free Framework for Visually-Grounded Multimodal Reasoning in LVLMs
Feb 25, 2026Yongchang Zhang, Oliver Ma, Tianyi Liu +2Multimodal Reasoning VLM Reasoning
Southeast University · AIIA, Ministry of Education, China · University of Oxford
LookWise: Knowing When and Where to Look for Fine-Grained Visual Reasoning in Multimodal Large Language Models
Feb 26, 2026Yuxiang Shen, Hailong Huang, Zhenkun Gao +6VLM Reasoning Large Vision-Language Models
the Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei 230031, China, and also with the University of Science and Technology of China, Hefei 230026, China · Zhejiang University, Hangzhou 310027, China · East China Normal University, Shanghai 200241, China +3

Sep 28, 2026J/K move · Enter open · S save

Distilling Visual Reasoning into Text Space

Wenhan Yang, Nilay Naharas, Ali Payani, Baharan Mirzasoleiman

University of California, Los Angeles · Cisco Research

CoT Distillation VLM Distillation Large Vision-Language Models VLM Reasoning

Open paper AskPDF arXiv

Large Vision-Language Models (LVLMs) have shown strong promise for multimodal reasoning, yet often struggle with tasks requiring concepts beyond what is directly observable in the input image. Existing methods generate intermediate images or latent visual tokens to guide reasoning, but these representations can introduce errors and increasingly interfere with textual reasoning as reasoning progresses. We propose Visual-to-Text Chain-of-Thought Distillation (V2T), a framework that enables LVLMs to internalize visual reasoning without generating intermediate visual representations at inference time. V2T first trains a teacher LVLM using interleaved visual and textual chains of thought, and then uses knowledge distillation to train a student LVLM using the teacher's logits and cross-entropy supervision from ground-truth textual reasoning. When reasoning images can be mapped to the original image, V2T can additionally distill the teacher's attention to corresponding regions, while ground-truth bounding boxes can further guide a subsequent reinforcement learning stage. Experiments across multiple multimodal reasoning benchmarks show that V2T consistently outperforms the teacher and existing baselines, improving average accuracy by 14.3% on a held-out set and 2.7% on the broader visual evaluation suite. Moreover, lightweight SFT and substantially reduced RL make V2T up to 42x faster to train than state-of-the-art baselines.

Abstract

Multimodal Reasoning

Explore similar work

Distilling Visual Reasoning into Text Space

See It, Say It, Sorted: An Iterative Training-Free Framework for Visually-Grounded Multimodal Reasoning in LVLMs

LookWise: Knowing When and Where to Look for Fine-Grained Visual Reasoning in Multimodal Large Language Models