DeepScan: A Training-Free Framework for Visually Grounded Reasoning in Large Vision-Language Models

Yangfu Li; Hongjian Zhan; Jiawei Chen; Yuning Gong; Qi Liu; Yue Lu

cs.CVMar 4, 2026

DeepScan: A Training-Free Framework for Visually Grounded Reasoning in Large Vision-Language Models

Authors: Yangfu Li, Hongjian Zhan, Jiawei Chen, Yuning Gong, Qi Liu, Yue Lu

Organizations: East China Normal University · Sichuan University · Shanghai AI Laboratory

Abstract

Humans can robustly localize visual evidence and provide grounded answers even in noisy environments by identifying critical cues and then relating them to the full context in a bottom-up manner. Inspired by this, we propose DeepScan, a training-free framework that combines Hierarchical Scanning, Refocusing, and Evidence-Enhanced Reasoning for visually grounded reasoning in Large Vision-Language Models (LVLMs). Unlike existing methods that pursue one-shot localization of complete evidence, Hierarchical Scanning performs local cue exploration and multi-scale evidence extraction to recover evidence in a bottom-up manner, effectively mitigating the impacts of distractive context. Refocusing then optimizes the localized evidence view through collaboration of LVLMs and visual experts. Finally, Evidence-Enhanced Reasoning aggregates multi-granular views via a hybrid evidence memory and yields accurate and interpretable answers. Experimental results demonstrate that DeepScan significantly boosts LVLMs in diverse visual tasks, especially in fine-grained visual understanding. It achieves 90.6% overall accuracy on V* when integrated with Qwen2.5-VL-7B. Moreover, DeepScan provides consistent improvements for LVLMs across various architectures and model scales without additional adaptation cost.

Figures & tables

Figure 1: Comparison of grounding pipelines in existing methods and the proposed DeepScan. (a) Existing methods depend on one-shot localization of evidence regions, making them sensitive to noisy contexts. (b) DeepScan uses local cue exploration to identify discriminative cues and recovers evidence from these cues in a bottom-up manner, robustly localizing critical visual content even in challenging scenes.

Figure 2: Performance of LVLMs and visually grounded reasoning variants on V*. DeepScan achieves highly competitive results.

Figure 3: Overall architecture of DeepScan : Hierarchical Scanning progressively recovers visual evidence from in-patch cues formulated as point-based proxies using Local Cue Exploration and Multi-Scale Evidence Extraction, e.g . , $c^{1}_{T}\mapsto e_{t-1}$ in step $T$ ; Refocusing further refines the surrounding context for the fused evidence via interactions between LVLMs and visual experts; Evidence-Enhanced Reasoning leverages a Hybrid Evidence Memory to provide multi-granular information to the LVLM, enabling detailed yet comprehensive answers.

Figure 4: Illustration of morphological post-processing. + marks the point-based proxies; $m$ and $m^{+}$ denote evidence masks before and after post-processing, showing improved robustness.

Figure 5: Analysis of ( left ) performance gain w.r.t. target area ratio and ( right ) performance-latency trade-off on V*, where

\infty

mark denotes the case without explicit truncation of candidate count

k

Figure 6: Illustration of Refocusing, where $V^{*}$ denotes its results. It reveals that Refocusing recalibrates the proxy misalignment by adaptively completing ( left ) or further amplifying ( right ) evidence.

	V* [ 42 ]			HR-4K [ 39 ]			HR-8K [ 39 ]
	Avg	Att	Spa	Avg	Sin	Cro	Avg	Sin	Cro
General Large Vision-Language Models
GPT-4o-1120 [ 22 ]	66.0	–	–	59.0	70.0	48.0	55.5	62.0	49.0
LLaVA-OV-7B [ 10 ]	70.7	73.0	60.5	64.3	74.8	53.8	59.8	65.3	54.3
LLaVA-OV-72B	73.8	80.9	63.2	66.3	76.5	56.0	60.9	68.8	53.0
InternVL3-8B [ 53 ]	72.3	73.0	71.1	70.8	79.3	62.3	62.0	64.3	59.8

Table 1: Comparisons on V* Bench and HR-Bench, where all the baselines are developed from Qwen2.5-VL-7B, and the RL-based methods are marked by gray . Results † are self-collected.

			Attributes	Material	Phy. State	Obj. Retr.	OCR	Per. Trans.	Ordering	Con. & Oc.	Spa. Cont.	Comparison
	Overall	mIoU	Perception					Reasoning
General Large Vision-Language Models
Gemini-2.5-Flash-0520 [ 7 ]	45.9	–	48.3	53.9	69.6	68.8	75.0	15.3	19.3	56.1	72.4	43.2
GPT-4o-1120 [ 22 ]	46.9	–	51.7	61.5	65.2	43.8	69.1	18.8	38.6	48.8	72.4	43.2
Gemini-2.5-Pro-0605 [ 8 ]	54.1	–	51.7	61.5	56.5	75.0	83.8	20.0	36.8	65.9	86.2	54.6
GPT-o3-0416 [ 23 ]	54.8	–	69.0	69.2	65.2	68.8	79.4	22.4	38.6	61.0	86.2	50.0

Table 2: Comparison with state-of-the-art alternatives on TreeBench [ 35 ] . All the baselines are developed from Qwen2.5-VL-7B, and the RL-based methods are marked by gray . Results † are self-collected. DeepScan achieves competitive results compared to SOTAs.

Figure 7: Ablations of DeepScan across LVLMs on V*, where the $y$ -axis shows average performance (%), the $x$ -axis reports latency (s), and the shadow shows performance deviation across subsets.

		Overall	Attribute	Spatial	Mem ( $\downarrow$ )	Time ( $\downarrow$ )
BLIP-ITM	base	90.6	93.0	86.8	29.4G	24.5s
	large	90.1	92.2	86.8	32.8G	25.8s
LangSAM	small	89.5	93.0	84.2	31.5G	23.2s
	base +	89.5	91.3	86.8	31.9G	24.0s
	large	90.6	93.0	86.8	32.8G	24.5s

Table 3: Ablation of external experts on V*; Mem denotes the memory footprint of each DeepScan variant in gigabytes (G). Time reports the average per-sample inference latency in seconds (s).

	Overall	Attribute	Spatial	Time ( $\downarrow$ )
Detection [ 13 ]	82.2	81.7	82.9	13.0s
Hierarchical Scanning	90.6	93.0	86.8	24.5s
$w/o$ Post-Processing	87.4	89.6	85.5	32.1s

Table 4: Ablation of Hierarchical Scanning on V*.

Table 5: Ablation of the Local Cue Exploration on V*. left : proxy type, where S and T denote semantic and topological information. right : patch size, where S / M denotes single-/multi-object scenes.

Figure 8: Ablation of the Zoom-Out scale on V*, where the left y-axis shows Hit@1 of evidence detection on the zoomed-out view, while the right y-axis shows the accuracy (Acc) of DeepScan.

Table 6: Left : effects of each action; Right : comparison of search length with different search methods and search space design.

Figure 9: Case study of grounding results and model responses of GPT-4o, DyFo, and the proposed DeepScan. Panels (a–b) are from V* Bench; panels (c–d) are from TreeBench. Correct and incorrect responses are highlighted in blue and red text, respectively. GT, evidence searched by Dyfo, and evidence localized by DeepScan are marked in the image using white , red , and blue bboxes, respectively.

Figure 10: Qualitative analysis of the grounding paradigms with the attention map $S$ of the search expert in (a).

	Overall	Attribute	Spatial	Time ( $\downarrow$ )
One-shot Localization	83.8	83.5	84.2	20.4s
Bottom-up Localization	90.6	93.0	86.8	24.5s

Table 7: Quantitative analysis of the grounding paradigms on V*.

Figure 11: Performance of Qwen2.5-VL series with the varying grounding precision (measured by IoU) on V* Bench.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 12: Case study of grounding failure

Figure 13: Case study of reasoning failure

Methods	Venue	Ease to Scaling	External Experts	Search Strategy	Grounding Paradigm	Evidence Granularity	Inference Latency
Seal [ 42 ]	CVPR’24	✗	✗	LLM-guided Search	Coarse-to-Fine	Fine	High
Dyfo [ 13 ]	CVPR’25	✓	✓	Detection + MCTS	Coarse-to-Fine	Coarse	High
DeepEyes [ 52 ]	NeurIPS’25	✗	✗	Generative Bbox	Coarse-to-Fine	Fine	High
PixelReasoner [ 31 ]	NeurIPS’25	✗	✗	Generative Bbox	Coarse-to-Fine	Coarse	High
ViGoRL [ 28 ]	NeurIPS’25	✗	✗	Generative Bbox	Coarse-to-Fine	Fine	High
ZoomRefine [ 46 ]	NeurIPS’25	✓	✗	Generative Bbox	Coarse-to-Fine	Coarse	Medium

Appendix

Table 8: Comparisons between the proposed DeepScan and existing visually grounded reasoning approaches. Like most test-time scaling paradigms, DeepScan is easier to scale up. Furthermore, DeepScan introduces hierarchical scanning and refocusing for a robust bottom-up evidence localization and recalibration and leverages hybrid granular evidence to enable LVLM to produce higher-quality answers.

	Overall	Attribute	Spatial
o3	95.0	-	-
Seal	74.8	76.3	75.4
Dyfo-L	62.7	53.9	59.2
Dyfo-Q	80.0	82.9	81.2
Qwen2.5-VL-7B	74.3	77.4	69.7
PixelReasoner	80.6	83.5	76.3

Appendix

Table 9: Additional comparison results on V* Bench between our method with existing baselines. We produced the results † with the provided official codes for fair comparisons.

Figure 14: Qualitative analysis of the grounding paradigms with the attention map $S$ of the search expert

	Qwen2.5-VL 7B	DeepEyes	Zoom Refine	Dyfo	Hierarchical Scan	Refocus	DeepScan
Accuracy (%)	75.4	89.0	82.7	83.8	84.8	89.5	90.1
Token Cost (k)	3.5	13	5.1	7.2	6.5	8.1	8.4
End-to-End Latency (s)	0.4	6.9	0.9	2.4	2.2	3.0	3.1

Appendix

Table 10: Performance–efficiency comparison on V* Benchmark. End-to-End Latency is the wall-clock time per sample from input to final output, including all auxiliary steps ( e.g . , Evidence Decompositions, Expert Calls, Post-Processing, Evidence Judgments).

VLM Reasoning

Vision-language model reasoning studies how multimodal systems use visual evidence and language to answer questions, plan, and explain.

See all 853 papers

H-GRPO: Permutation-Invariant Reinforcement Learning for Improving Spatial Grounded Reasoning
Jun 29, 2026Eric Peh, Debaditya Roy, Chen Li +1RL for VLMs Visual Reasoning
Institute of High-Performance Computing, Agency for Science, Technology and Research, Singapore · Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore · Department of Computer Science and Engineering, Indian Institute of Technology Kharagpur, India +1
LOCI: A Locator-Critic with Refinement Loop
Aug 31, 2026Walid Bousselham, Mathilde Caron, Arsha Nagrani +1Visual Question Answering Vision-Language Grounding
Google DeepMind
Look Where It Counts: A Free, Label-Free Visual Evidence Signal for Fine-Grained Vision-Language Reasoning
Sep 21, 2026Santi Ram Tiwari, Nihal Naik, Devbrat Pandey +1Vision-Language Grounding VLM Reasoning
KGraph AI Solutions Pvt. Ltd. Bangalore, India – 560016

Jun 29, 2026J/K move · Enter open · S save

H-GRPO: Permutation-Invariant Reinforcement Learning for Improving Spatial Grounded Reasoning

Eric Peh, Debaditya Roy, Chen Li, Basura Fernando

Institute of High-Performance Computing, Agency for Science, Technology and Research, Singapore · Centre for Frontier AI Research, Agency for Science, Technology and Research, Singapore · Department of Computer Science and Engineering, Indian Institute of Technology Kharagpur, India+1

RL for VLMs Visual Reasoning VLM Interpretability Reinforcement Learning Vision-Language Grounding VLM Reasoning+2

Open paper AskPDF arXiv

Vision-language models (VLMs) have achieved strong performance on visual reasoning tasks, yet a correct final answer does not necessarily imply that the model followed a correct or visually grounded reasoning process. We study how VLMs can be trained to produce explicit intermediate reasoning steps that are both semantically meaningful and grounded in visual evidence. We represent each reasoning step as a structured triplet consisting of a sub-question, sub-answer, and supporting bounding box, enabling the reasoning process to be evaluated along both semantic and spatial dimensions. To optimize this representation, we introduce Hungarian-GRPO (H-GRPO), a reinforcement-learning framework that aligns predicted and reference reasoning steps using permutation-invariant Hungarian matching, avoiding the assumption that valid reasoning steps must follow a fixed ordering. Experiments show that optimizing grounded reasoning improves both intermediate reasoning quality and answer accuracy, with an H-GRPO variant achieving the strongest overall grounding and decomposition performance. Our analyses show that grounding quality and answer correctness are related but distinct, while counterfactual evidence masking confirms that the localized regions contain information important to the model's predictions. These findings motivate evaluating VLMs beyond answer accuracy and demonstrate the value of explicit, spatially grounded reasoning.

Abstract

VLM Reasoning

Explore similar work

H-GRPO: Permutation-Invariant Reinforcement Learning for Improving Spatial Grounded Reasoning

LOCI: A Locator-Critic with Refinement Loop

Look Where It Counts: A Free, Label-Free Visual Evidence Signal for Fine-Grained Vision-Language Reasoning