Thinking with images (TwI) empowers vision-language models (VLMs) to overcome perceptual bottlenecks by actively cropping and zooming into fine-grained visual details, promising both higher accuracy and interpretable, region-grounded reasoning. However, our analysis shows that the improved accuracy of TwI models does not guarantee faithful tool use; they frequently invoke tools without actually relying on the zoomed region, undermining the reliability of tool-assisted reasoning. We trace this unfaithful behaviour to the training data: when questions are answerable without zooming, the model can answer correctly while ignoring the returned evidence, and this problem is exacerbated during training as the model becomes better at answering without tools. To keep zooming useful throughout training, we adapt the training data to the model's evolving capability and train on questions where correct zoomed evidence can still help the model answer correctly. This selection is supported by a data synthesis pipeline that localises small objects and constructs verifiable questions about their fine details, and complemented by an adaptiveness reward that favours tool-free answers when zooming is unnecessary. The experimental results show that our method substantially improves reasoning faithfulness and achieves state-of-the-art performance on visual search benchmarks (+11.6% on V*, +7.9% on HR-Bench 4K, and +9.4% on HR-Bench 8K). On multimodal hallucination benchmarks, our model also outperforms the compared TwI baselines. These results suggest that improving TwI faithfulness does not sacrifice accuracy and that both can be learnt from the right training data without directly supervising the reasoning process.
Figures & tables
Figure 1: Two faithfulness failure modes in existing TwI models. False positive : replacing the tool-returned crop with a random and a black image, PixelReasoner [ 33 ] and DeepEyes [ 61 ] claim to see the target in the crop. False negative : CodeV [ 10 ] predicts a correct area but denies the target in it, though its answer is correct after additional zooms. All three models answer Mickey Mouse correctly but are not faithfully grounded on the evidence.
Controlled setting
Real-crop setting
Model
FNR ↓
FPR ↓
Acc. ↑
FNR ↓
FPR ↓
Acc. ↑
DeepEyes
1.6
98.8
50.0
0.0
94.1
73.9
PixelReasoner
2.3
56.9
70.8
2.8
91.9
54.6
CodeV
87.9
0.7
55.2
84.1
10.7
31.6
FaithTwI
8.7
2.2
94.5
6.1
18.2
93.3
Table 1: Target recognition on V* (%). The controlled setting (left) replaces the models’ first crops with ground-truth crops v+ or target-absent crops v− ; the real-crop setting (right) uses the models’ own predicted crops, and its details are provided in Appendix A .
Figure 2: Accuracy with and without tools.
Figure 4: Question synthesis workflow. We first localise and sample objects from high-resolution, information-dense images using SAM3 [ 4 ] . Then, we generate questions for randomly sampled objects using the image overview and zoomed evidence. Finally, we verify questions in several steps: target identification with Set-of-Mark prompting [ 50 ] checks if the question refers to unambiguous objects; answer correctness checks if the proposed answer is reliable; visual dependency checks if the question can be guessed without seeing the image.
Model
V*
HR-Bench 4K
HR-Bench 8K
MME-RW-en
MME-RW-cn
ZoomBench
Qwen2.5-VL-7B † [ 2 ]
78.5
71.6
67.9
60.8
58.3
42.5
PixelReasoner [ 33 ]
84.3
72.6
66.1
64.4
60.5
47.8
DeepEyes [ 61 ]
85.6
75.1
72.6
64.1
64.2
46.6
Thyme [ 58 ]
82.2
77.0
72.0
64.8
65.0
44.9
DeepEyesV2 [ 9 ]
81.8
77.9
73.8
64.9
64.8
45.2
Mini o3 [ 15 ]
88.2
77.5
73.3
65.5
64.8
50.3
Table 2: Answer accuracy (%) on the perception-intensive benchmarks. Bold marks the highest score in each column. Results of Qwen2.5-VL-7B are taken from Wei et al. [45] . † denotes models that do not crop or zoom into the image at inference time.
Model
V*
HR-Bench 8K
ZoomBench
FaithTwI
90.1
77.3
56.8
w/o selection
86.4
71.5
48.2
w/o adapt. reward
90.1
74.8
56.3
w/o sel. & adapt. reward
90.1
71.9
55.5
w/o synthesised data
85.5
69.0
49.5
w/o tool training
86.9
70.6
51.0
Table 3: Answer accuracy (%) of the ablation models. Adapt. reward denotes the adaptiveness reward.
Figure 5: Tool use rate and answer accuracy during training. Left: percentage of training responses containing a tool call. Right: HR-Bench 8K accuracy of FaithTwI with tools (solid) and without tools during inference time (dotted).
Model
FNR ↓
FPR ↓
Acc. ↑
Full method
8.7
2.2
94.5
w/o adapt. reward
9.8
4.9
92.6
w/o selection & adapt. reward
8.9
7.4
91.9
w/o unanswerable data
13.3
3.7
91.5
w/o SFT
0.0
96.3
51.9
Table 4: Target-recognition accuracy and error rates after replacing the first returned crop on V* (%).
Model
RePOPE
AMBER
HaloQuest
VizWiz
PixelReasoner
93.6
87.9
54.4
49.1
DeepEyes
88.3
85.2
47.5
46.1
Thyme
89.4
85.2
34.5
44.3
DeepEyesV2
43.7
38.2
23.4
45.8
Mini o3
73.7
75.6
26.2
44.4
CodeV
91.1
84.7
62.2
54.9
Table 5: Faithfulness on multimodal hallucination benchmarks. Bold marks the highest score in each column.
Figure 6: Answer accuracy (%) of FaithTwI trained from three base models: Qwen2.5-VL-7B, Qwen3-VL-4B, and Qwen3.5-4B.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Real crops
Controlled crops
Model
FNR
FPR
FNR (ground truth)
FPR (random)
FPR (grey)
FPR (black)
Acc.
DeepEyes
0.0
94.1
1.6
98.8
97.6
98.0
50.0
PixelReasoner
2.8
91.9
2.3
56.9
95.3
95.1
70.8
CodeV
84.1
10.7
87.9
0.7
0.5
0.0
55.2
FaithTwI
6.1
18.2
8.7
2.2
0.7
0.0
94.5
Appendix
Table 6: First-crop target recognition on V* (%). Real crops are the models’ own first crops. Controlled crops replace them with a ground-truth crop, which shows the target, or with a random crop, a grey image, or a black image, which do not. FNR is measured on ground-truth crops and FPR on each type of crop without the target; accuracy is measured on ground-truth and random crops.
V*
HR-Bench 4K
HR-Bench 8K
Model
With
Without
Tool use
With
Without
Tool use
With
Without
Tool use
DeepEyes
81.8
83.8
100.0
74.5
73.9
100.0
70.9
69.5
99.9
PixelReasoner
82.5
79.9
75.9
72.9
74.6
51.8
65.4
66.5
41.4
CodeV
80.6
77.7
96.3
75.0
72.8
94.2
69.4
68.2
94.2
Appendix
Table 7: Accuracy with and without tools, and tool-use rate (%), evaluated by us with each model’s official settings. Without tools, the prompts still ask the model to reason before answering. Tool use is the percentage of questions on which the model calls the tool. V* is averaged over three runs, and HR-Bench uses one run.
Ability
Questions
Proportion (%)
Spatial relation
3,655
28.1
Colour recognition
3,382
26.0
Text recognition
2,054
15.8
Structure recognition
1,650
12.7
Entity recognition
1,176
9.0
Counting
560
4.3
Appendix
Table 8: Ability distribution of the synthesised training questions.
Figure 7: Selection pass rate of the synthesised questions at each RL step for FaithTwI , i.e., the percentage of checked questions that are used for training.
Model
V*
HR-Bench 4K
HR-Bench 8K
MME-RW-en
MME-RW-cn
ZoomBench
FaithTwI
90.1
79.5
77.3
65.9
67.0
56.8
w/o selection
86.4
75.1
71.5
65.7
65.6
48.2
w/o adapt. reward
90.1
78.8
74.8
63.5
64.9
56.3
w/o selection & adapt. reward
90.1
77.6
71.9
62.9
63.8
55.5
w/o synthesised data
85.5
75.6
69.0
65.1
66.4
49.5
w/o tool training
86.9
76.1
70.6
64.9
65.3
51.0
Appendix
Table 9: Answer accuracy (%) of the ablation models on all six perception-intensive benchmarks. Bold marks the highest score in each column. Adapt. reward denotes the adaptiveness reward.
Model
V*
HR-Bench 4K
HR-Bench 8K
MME-RW-en
MME-RW-cn
ZoomBench
Based on Qwen3-VL-4B
Qwen3-VL-4B-Instruct † [ 1 ]
80.1
78.3
72.9
63.5
63.6
40.2
Qwen3-VL-4B-Thinking † [ 1 ]
79.6
74.5
66.9
60.3
58.8
40.9
ZwZ-4B † [ 45 ]
92.7
81.8
79.5
68.5
68.1
55.7
P2R-4B ‡ [ 18 ]
93.2
81.9
80.5
66.3
66.8
52.5
Imagine-OPD-4B † [ 3 ]
88.0
83.3
78.6
68.1
67.6
43.2
Appendix
Table 10: Answer accuracy (%) of models based on Qwen3-VL-4B and Qwen3.5-4B. Results of Qwen3-VL-4B-Instruct are taken from Wei et al. [45] , and the Qwen3.5-4B baselines use the scores reported in their papers. With Qwen3-VL-4B, FaithTwI starts from Qwen3-VL-4B-Thinking. † denotes models that do not crop or zoom into the image at inference time. ‡ P2R-4B does not call a tool during reasoning; it first predicts region boxes, and then answers from the image with crops of these boxes.
Controlled, human
Controlled, judge
Real crops, human
Model
FNR
FPR
Acc.
FNR
FPR
Acc.
Crop contains target
FNR
FPR
Acc.
DeepEyes
0.0
100.0
50.0
0.0
96.2
51.9
66.3
0.0
87.2
80.8
PixelReasoner
0.0
80.8
59.6
0.0
73.1
63.5
52.4
7.3
91.3
54.2
CodeV
84.5
3.4
56.0
87.9
0.0
56.0
67.5
77.3
12.7
34.6
FaithTwI
12.1
6.9
90.5
19.0
0.0
90.5
90.9
2.1
12.1
97.1
Appendix
Table 11: Human evaluation of first-crop target recognition on V* (%). Controlled: ground-truth crops versus random crops, grey images, and black images, labelled by humans and by the judge on the same samples. Real crops: the models’ own first crops; “Crop contains target” is the percentage of crops that annotators judged to contain the target.
Wrong region
Grey or black image
Model
Not noticed
Zoom again
Other
Not noticed
Zoom again
Other
DeepEyes
90.9
0.0
9.1
100.0
0.0
0.0
PixelReasoner
85.3
5.9
8.8
100.0
0.0
0.0
CodeV
21.2
76.2
2.6
7.9
92.1
0.0
FaithTwI
7.6
92.4
0.0
0.0
100.0
0.0
Appendix
Table 12: Model behaviour when the first crop, or its replacement, does not contain the target, based on human annotation (%). “Wrong region” refers to a first crop that misses the target, which is either the model’s own wrong crop or a random crop that replaces it. “Not noticed” means that the model responds as if the target were in the image; “Other” includes answering from the original image, stating that the question is unanswerable, and guessing an answer after noticing that the target is missing.
Model
Infers an answer before crop
Final prediction matches
DeepEyes
88.0
95.2
PixelReasoner
21.8
96.8
CodeV
8.2
100.0
FaithTwI
20.4
89.5
Appendix
Table 13: How often a model decides its answer before seeing the returned crop on V*. Infers an answer before crop : percentage of questions for which the model infers an answer in its first reasoning turn, before the tool call. Final prediction matches : among these questions, percentage for which the final prediction is the same as this answer.
Figure 8: Proportion of attention over the input while the model writes its response after the first crop (%).
Thinking with images empowers vision-language models (VLMs) to use an external Zoom-in tool for fine-grained visual search. Recent works employ supervised fine-tuning (SFT) and reinforcement learning (RL) with tool-specific designs to enable such capabilities. However, achieving logical reasoning alongside accurate and efficient tool use remains a significant challenge. To address this, we shift the focus away from increasing RL complexity and instead identify SFT data quality as a key bottleneck. We then propose LogT (short for Logically Think with Images), a fully automated SFT data engine that explicitly enforces logic-chain-guided reasoning. LogT generates an SFT dataset featuring challenging, visually grounded questions and structured tool-use traces with human-like logic. We validate our approach by fine-tuning two base models, Qwen2.5-VL-7B and InternVL3.5-8B, on the dataset and evaluating them across seven benchmark test sets. We show that LogT-SFT based on Qwen2.5-VL-7B outperforms the state-of-the-art trained with both SFT and RL, despite using 3.1× less data and >120× fewer GPU hours. Furthermore, applying a simple RL recipe without tool-specific design on LogT-SFT yields a model that surpasses prior methods with complex RL setups by a substantial 9.59-point margin. In addition to strong performance gains, our model exhibits higher tool-use accuracy and efficiency, as well as more structured and logically consistent reasoning traces, directly validating the effectiveness of our approach.
Yanjun Fu, Quanzeng You, Jiadong Guo +6
Meta Superintelligence Labs (MSL) · University of Maryland
Agentic vision-language models (VLMs), which interleave textual reasoning with explicit tool calls such as cropping and code-based image manipulation, have emerged as a compelling paradigm for reliable and interpretable multi-modal reasoning. However, recent studies have revealed that such models often use tools unfaithfully. Many process images are irrelevant to the question (e.g., the crops miss the queried target), yet the tool call still receives full credit and the model still answers correctly. Such decorative or misaligned tool calls waste computation and reveal that the model does not faithfully use the evidence it retrieves. This may stem from two limitations of prevailing methods: the tool reward fails to distinguish useful from useless calls, and tool feedback carries no signal of usefulness. To this end, we introduce FaithEyes, a multi-agent self-judging framework. Concretely, we use a VLM to judge whether each process image helps answer the question. The judgement is injected into the reasoning context as part of the tool observation to help subsequent reasoning, and meanwhile is used to scale the tool reward by the helpful-tool ratio to suppress reward hacking. To keep judgement available at evaluation, we further design a multi-agent framework where the model itself serves as a subagent to judge the tool calls from the main agent, eliminating any dependence on external models at inference. Training via a two-stage SFT + RL pipeline on adapted open-source data, FaithEyes attains competitive or superior accuracy across visual perception and reasoning benchmarks, while substantially improving tool faithfulness and reducing inference cost. The homepage is at https://github.com/Mosi-AI/FaithEyes.
Haoqing Wang, Xingrun Xing, Ziheng Li +2
Samsung Research, Beijing, China · Peking University
Vision-language models (VLMs) exhibit a systematic bias when confronted with classic optical illusions: they overwhelmingly predict the illusion as "real" regardless of whether the image has been counterfactually modified. We present a tool-guided inference framework for the DataCV 2026 Challenge (Tasks I and II) that addresses this failure mode without any model training. An off-the-shelf vision-language model is given access to a small set of generic image manipulation tools: line drawing, region cropping, side-by-side comparison, and channel isolation, together with an illusion-type-routing system prompt that prescribes which tools to invoke for each perceptual question category. Critically, every tool call produces a new, immutable image resource appended to a persistent registry, so the model can reference and compose any prior annotated view throughout its reasoning chain. Rather than hard-coding illusion-specific modules, this generic-tool-plus-routing design yields strong cross-structural generalization: performance remained consistent from the validation set to a test set containing structurally unfamiliar illusion variants (e.g., Mach Bands rotated from vertical to horizontal stacking). We further report three empirical observations that we believe warrant additional investigation: (i) a strong positive-detection bias likely rooted in imbalanced illusion training data, (ii) a striking dissociation between pixel-accurate spatial reasoning and logical inference over self-generated annotations, and (iii) pronounced sensitivity to image compression artifacts that compounds false positives.