Despite recent advances, vision-language models (VLMs) still struggle with fine-grained visual understanding, often failing to effectively utilize critical evidence already encoded in their visual representations. Existing approaches address this limitation through repeated visual processing or explicit reasoning, introducing substantial computational overhead. In this paper, we propose ReAccess, a lightweight framework that preserves and reuses question-relevant visual evidence through compact associative memory. Constructed during a single visual prefill, the memory supports progressive evidence retrieval that steers both attention queries and hidden representations, enabling effective visual grounding without repeated image encoding or explicit reasoning traces. Extensive experiments demonstrate consistent improvements in both general visual understanding and fine-grained perception, while achieving substantially faster inference than existing reasoning-intensive approaches, demonstrating a promising path to efficient multimodal reasoning.
Figures & tables
Figure 1: Visual evidence accessibility in VLMs. Analysis of focused visual evidence and its utilization during language reasoning.
Baselines
Matched Image
Evidence Type
Question Only
Unrelated Image
Full State
PCA (16D)
PCA (64D)
Random (16D)
Visual Appearance
23.5
21.3
64.5
64.3
65.4
44.4
Attribute Grounding
51.5
53.5
85.9
86.3
86.6
84.3
Spatial Structure
53.2
51.0
89.5
90.5
89.8
78.8
Physical Properties
41.6
40.1
75.2
73.7
70.1
45.3
Table 1: Visual evidence recoverability in compact representations.
Figure 2: Overview of ReAccess. Given an image–question pair, ReAccess constructs a compact associative memory from visual features during a single prefill ( Memory Construction ). The memory is progressively accessed through successive reads to retrieve complementary visual evidence, which steers both attention queries and hidden representations of the frozen VLM ( Memory Utility ).
Backbone
GQA
MMBench
MMStar
TextVQA
DocVQA
InfoVQA
ChartQA
AI2D
OCRBench
MMMU
LogicVista
POPE
Qwen2.5-VL-7B [ 4 ]
60.4
82.2
56.7
83.2
93.3
78.6
83.3
78.7
84.3
50.1
39.1
86.1
+Ours
62.1 ↑ 1.7
83.7 ↑ 1.5
61.6 ↑ 4.9
85.0 ↑ 1.8
95.2 ↑ 1.9
80.5 ↑ 1.9
87.6 ↑ 4.3
82.6 ↑ 3.9
86.7 ↑ 2.4
51.0 ↑ 0.9
42.4 ↑ 3.3
87.9 ↑ 1.8
LLaVA-NeXT-7B [ 19 ]
63.0
64.9
33.5
65.4
66.1
30.7
53.5
63.4
52.4
35.8
26.3
86.1
+Ours
64.6 ↑ 1.6
67.4 ↑ 2.5
36.5 ↑ 3.0
67.8 ↑ 2.4
70.2 ↑ 4.1
35.8 ↑ 5.1
55.7 ↑ 2.2
65.2 ↑ 1.8
53.7 ↑ 1.3
37.4 ↑ 1.6
29.2 ↑ 2.9
88.2 ↑ 2.1
Qwen3-VL-8B [ 3 ]
61.8
83.8
61.1
82.7
94.3
80.8
84.6
82.9
85.8
54.0
35.3
87.3
+Ours
63.0 ↑ 1.2
86.4 ↑ 2.6
65.5 ↑ 4.4
84.6 ↑ 1.9
95.8 ↑ 1.5
83.6 ↑ 2.8
89.9 ↑ 5.3
84.3 ↑ 1.4
88.6 ↑ 2.8
55.7 ↑ 1.7
39.5 ↑ 4.2
88.5 ↑ 1.2
Table 2: Performance on general visual understanding across various VLM backbones.
Method
HR-Bench 4K
HR-Bench 8K
V ∗
CV-Bench
Attribute
Spatial
Overall
Multi-turn
Thyme-RL [ 47 ]
77.0
72.0
84.4
81.6
83.3
78.2
DeepEyes [ 48 ]
73.9
71.0
88.7
79.0
84.8
77.8
Two-stage
API [ 45 ]
71.5
68.5
80.9
75.0
78.5
78.1
Table 3: Comparison with representative methods based on Qwen2.5-VL-7B. † denotes single-prefill variants.
Training Objectives
Architecture
Evaluation Benchmarks
Configuration
Lev
Lground
Ldiv
Q
O
Q-Cond.
Prog.
HR-4K
HR-8K
V ∗ -Attr.
V ∗ -Spat.
V ∗ -Overall
TextVQA
POPE
Full
✓
✓
✓
✓
✓
✓
✓
73.3
71.8
84.4
82.9
83.8
85.0
87.9
Training objective ablations
Evidence reconstruction
✗
✓
✓
71.6
69.8
82.6
79.0
81.2
83.4
87.2
Grounding
✓
✗
✓
72.0
67.9
80.9
77.6
79.6
84.2
86.7
Diversity
✓
✓
✗
72.4
70.8
83.5
80.3
82.2
84.6
87.3
Table 4: Component Ablations of ReAccess.
HR-Bench
V*
Memory Setting
4K
8K
Attr.
Spatial
Overall
Original
73.3
71.8
84.4
82.9
83.8
Wrong-Image
69.6
66.1
80.0
76.3
78.5
Wrong-Question
71.3
68.4
82.6
79.0
81.2
Zero-Memory
70.9
66.9
81.7
77.6
80.1
Table 5: Assessing the Contribution of Evidence Memory
Figure 3: Causal evidence intervention on V*Bench.
Memory Size
HR-Bench
V*
TextVQA
4K
8K
Attr.
Spatial
Overall
128×128
71.8
69.8
81.7
79.0
80.6
84.7
64×64(default)
73.3
71.8
84.4
82.9
83.8
85.0
32×32
73.1
71.5
83.5
81.6
82.7
84.6
16×16
72.8
71.0
82.6
81.6
82.2
84.1
Table 6: Effect of memory size on visual understanding.
Figure 4: Qualitative visualization of evidence preservation and utilization through ReAccess memory.
Figure 5: Inference efficiency comparison in different scenario
Depth
HR-Bench
V*
TextVQA
4K
8K
Attr.
Spatial
Overall
Full
73.3
71.8
84.4
82.9
83.8
85.0
Shallow
72.0
68.9
80.9
75.0
78.5
84.6
Middle
72.5
69.0
83.5
80.3
82.2
84.0
Deep
72.1
68.8
81.7
77.6
80.1
82.9
Table 7: Effect of memory insertion depth on performance.
Configuration
Parameter
Value
Core
Backbone
Qwen2.5-VL-7B
Memory dimension
64
Memory layers
28
Memory reads
2
Spatial kernel
Depthwise 3×3
Trainable parameters
57.9M ( ∼ 0.7%)
Table 8: Architecture and training hyperparameters of ReAccess.
HR-Bench
V*
MMMU
Anomalies
Method
8K
4K
Latency
Latency
/400
ReAccess
3.98 (0)
3.34 (0)
0.95 (0)
0.31 (0)
0
Frozen
3.79 (0)
3.17 (0)
0.89 (0)
0.28 (0)
0
LVR
4.26 (0)
3.78 (0)
1.38 (0)
1.00 (0)
0
API
8.26 (0)
4.98 (0)
2.14 (0)
1.24 (0)
0
FGVP-Mask
8.28 (0)
5.93 (0)
2.86 (0)
1.70 (0)
0
Table 9: Inference latency and anomalous generations across benchmarks. Average latency (s/sample) includes anomalous generations and retries. Parentheses indicate anomalous sample counts.
Method
Anomalies
Failure Cause
Thyme-RL
6
Repetitive outputs reaching token limits and triggering repeated retries.
DeepEyes
2
Repeated unsuccessful action requests with excessive token generation.
Vision-R1
29
Overlong, repetitive reasoning and retry loops during sampled decoding.
PixelReasoner
1
Repetitive generation exhausting the output token limit.
HyLaR
10
Erroneous generation continuing until the maximum token limit.
SkiLa
2
Erroneous generation continuing until the maximum token limit.
Table 10: Causes of anomalous inference latency. Anomalies primarily arise from repetitive generation and retry mechanisms.
Vision-language models (VLMs) have advanced rapidly in multimodal reasoning, yet they remain limited on tasks requiring fine-grained understanding of small or easily overlooked image regions. Recent thinking with images approaches mitigate this limitation by allowing models to inspect images through operations such as zooming, cropping, or code execution. While effective, their multi-turn interactions with VLMs introduce redundant token computation and slow inference, and separate region inspection from the reasoning step that depends on it. To address these issues, we propose Look Before You Reason (LoRe), an implicit visual thinking framework for efficient visual reasoning within a single-turn VLM interaction. Given an image and a question, LoRe predicts the most relevant image region, aligns it with the corresponding image patch features, and strengthens the associated key-value (KV) cache entries, enabling subsequent decoding to directly exploit the selected visual cues without additional visual inputs or explicit tool calls. We further employ reinforcement learning to guide the VLM toward identifying informative image regions and leveraging them during reasoning. To support fine-grained evaluation of multimodal reasoning, we introduce COCO-ObjQA, a large-scale benchmark for object-centric multimodal reasoning. Experiments on three challenging multimodal benchmarks demonstrate that LoRe consistently improves VLM reasoning performance with low additional inference overhead.
Multimodal Large Language Models (MLLMs) are shifting towards "Thinking with Images" by actively exploring image details. While effective, large-scale training is computationally expensive, which has spurred growing interest in lightweight, training-free solutions. However, existing training-free methods suffer from two flaws: perceptual redundancy from indiscriminate cropping, which increases computational cost and introduces noise; and a drift between semantic intent and spatial attention, which prevents accurate localization of user-focused regions. To address these challenges, we propose LookWise, a framework for adaptive visual reasoning. LookWise follows a two-stage pipeline: a confidence-based module decides when to look more carefully, and a semantic-guided localization module determines where to look. This design enables MLLMs to adaptively acquire fine-grained visual evidence without additional training. Experiments on fine-grained and high-resolution visual reasoning benchmarks show that LookWise consistently improves accuracy over strong baselines while achieving an approximately 4.0× inference speedup over the search-based method ZoomEye, demonstrating robust cross-model generalization.
Yuxiang Shen, Hailong Huang, Zhenkun Gao +6
the Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei 230031, China, and also with the University of Science and Technology of China, Hefei 230026, China · Zhejiang University, Hangzhou 310027, China · East China Normal University, Shanghai 200241, China +3
Vision language models (VLMs) are increasingly capable of reasoning over images, but robust visual reasoning often requires re-grounding intermediate steps in the underlying visual evidence. Recent approaches typically rely on external image operations such as zooming or cropping to re-access fine-grained details during inference, which requires additional image re-encoding and can disrupt the reasoning trajectory. We argue that VLMs already provide strong internal signals for identifying and reusing visual evidence, and that these signals can be directly leveraged to support image-grounded reasoning. Motivated by this insight, we propose an end-to-end self-revisit framework, SIEVE, that trains models to re-engage image evidence through internal representations. SIEVE automatically extracts embeddings of salient image regions and injects them into the reasoning chain when additional grounding is needed, enabling later steps to condition on relevant visual cues without external tool calls or re-encoding. We use reinforcement learning to teach the model when to trigger visual revisiting and which region embeddings to retrieve and insert during the reasoning process. Experiments on multiple visual reasoning benchmarks, together with perception, reasoning, and hallucination evaluations, show that SIEVE yields consistent gains, improving performance by 8 percent on average across several benchmarks.