MLLMs require high-resolution visual inputs for fine-grained tasks like document understanding and dense scene perception. However, current global resolution scaling paradigms indiscriminately flood the quadratic self-attention mechanism with visually redundant tokens, severely bottlenecking inference throughput while ignoring spatial sparsity and query intent. To overcome this, we propose Q-Zoom, a query-aware adaptive high-resolution perception framework that operates in an efficient coarse-to-fine manner. First, a lightweight Dynamic Gating Network safely bypasses high-resolution processing when coarse global features suffice. Second, for queries demanding fine-grained perception, a Self-Distilled Region Proposal Network (SD-RPN) precisely localizes the task-relevant Region-of-Interest (RoI) directly from intermediate feature spaces. To optimize these modules efficiently, the gating network uses a consistency-aware generation strategy to derive deterministic routing labels, while the SD-RPN employs a fully self-supervised distillation paradigm. A continuous spatio-temporal alignment scheme and targeted fine-tuning then seamlessly fuse the dense local RoI with the coarse global layout. Extensive experiments demonstrate that Q-Zoom establishes a dominant Pareto frontier. Using Qwen2.5-VL-7B as a primary testbed, Q-Zoom accelerates inference by 2.52 times on Document & OCR benchmarks and 4.39 times in High-Resolution scenarios while matching the baseline's peak accuracy. Furthermore, when configured for maximum perceptual fidelity, Q-Zoom surpasses the baseline's peak performance by 1.1% and 8.1% on these respective benchmarks. These robust improvements transfer seamlessly to Qwen3-VL, LLaVA, and emerging RL-based thinking-with-image models. Project page is available at https://yuhengsss.github.io/Q-Zoom/.
Figures & tables
Fig. 1 : Comparison of adaptive high-resolution perception paradigms. Training-free methods rely on handcrafted contrastive rules, requiring multiple redundant prefilling passes. RL-based methods use the LLM to auto-regressively generate code or coordinates to find the RoI. Our Q-Zoom framework operates directly on the intermediate feature space during a single prefilling pass, yielding superior efficiency.
Max Limit
Through- put
DocVQA val
OCRBench test
InfoVQA val
V*
Ave.
512
4.6
90.9
83.3
69.1
62.8
76.5
2048
2.2
94.7
84.8
80.4
72.3
83.1
TABLE I : Performance and throughput comparison of Qwen2.5-VL 7B on Document and Vision-Centric benchmarks under different maximum visual token limits. Throughput is measured in samples/second on a single RTX A6000 GPU.
Fig. 2 : Overview of the proposed Adaptive High-Resolution Perception Framework. (a) The framework derives robust supervisory signals through consistency-aware generation to train a lightweight gating module. (b) During inference, the gate dynamically evaluates the textual query. It routes simpler queries for direct, accelerated generation using coarse features, while triggering the SD-RPN for complex queries to extract targeted high-resolution regions.
Fig. 3 : Overview of the conditional Region-of-Interest extraction pipeline. When triggered by the dynamic gating module, the SD-RPN (top) leverages shared intermediate features from the frozen backbone to efficiently generate a dense spatial heatmap. During the training phase (bottom), the network is optimized through a self-distillation paradigm, utilizing denoised cross-modal attention maps from the base MLLM as supervisory pseudo-labels. Superscripts indicate network depth (layers), while subscripts denote the modality or token origin. System prompts are excluded for visual clarity.
Figure 5
Fig. 6 : Overview of the Spatio-Temporal Alignment and Targeted Post-SFT pipeline. The vision encoder and projector are omitted for visual brevity.
Methods
Throughput
DocVQA val
ChartQA test
OCRBench test
InfoVQA val
TextVQA val
Ave.
LLaVA-1.5-7B
1.0 ×
21.5
18.1
31.4
20.4
46.1
27.5
+S 2 [ECCV2024] [ 67 ]
0.70 ×
27.1
18.9
32.6
22.5
52.6
30.7
+ViCrop [ICLR2025] [ 23 ]
0.42 ×
27.0
20.0
33.2
21.4
57.2
31.8
+SD-RPN [ICLR2026] [ 37 ]
0.62 ×
34.2
20.6
37.3
22.3
58.8
34.6
+Q-Zoom
0.64 ×
34.0
20.8
38.0
22.4
58.4
34.7 ↑ 7.2
LLaVA-1.5-13B
1.0 ×
23.5
18.1
33.7
23.4
48.7
29.5
TABLE II : Performance on Document & OCR benchmarks. Dataset subscripts denote the evaluation split. Performance subscripts show the absolute improvement ( ↑ ) over the baseline. Throughput is relative to the baseline, measured on a single NVIDIA A6000 GPU and our results are evaluated under a constraint of 576 maximum visual tokens.
Methods
Tp
V* Bench
MME-RW
HR-Bench 4K
HR-Bench 8K
Ave.
Attr
Spatial
Overall
Lite
FSP
FCP
Overall
FSP
FCP
Overall
LLaVA-1.5-7B
1.00 ×
48.7
52.6
50.3
27.7
39.5
35.5
37.5
32.5
33.8
33.8
37.3
+S 2 [ECCV2024] [ 67 ]
0.63 ×
53.0
59.1
55.5
28.5
49.8
38.3
44.0
40.5
36.5
38.5
41.6
+ViCrop [ICLR2025] [ 23 ]
0.15 ×
53.9
50.0
52.4
27.6
60.8
34.8
47.8
38.5
33.8
36.1
41.0
+SD-RPN [ICLR2026] [ 37 ]
0.57 ×
70.4
71.1
70.7
27.7
59.0
35.5
47.3
48.8
34.5
41.6
46.8
+Q-Zoom
0.58 ×
70.4
71.1
70.7 ↑ 20.4
27.7 ↑ 0.0
58.8
35.8
47.1 ↑ 9.6
48.5
34.5
41.4 ↑ 7.6
46.7 ↑ 9.4
TABLE III : Performance on Vision-Centric and High-Resolution benchmarks. Dataset subscripts denote the specific evaluation split. Performance subscripts indicate the absolute improvement ( ↑ ) of our latest version over the baseline. Tp denotes the relative inference throughput. Averages are computed exclusively across the four Overall metrics. Unless otherwise noted, our results are evaluated under a constraint of 4,096 maximum visual tokens. The † symbol denotes results directly cited from the corresponding original publications.
Fig. 7 : Qualitative comparisons on challenging examples from TextVQA (left) and V* Bench (right). These examples highlight visually demanding scenarios where the target evidence is small or obscured. The baseline Qwen2.5-VL-7B suffers from resolution compression, whereas our Q-Zoom framework successfully leverages the SD-RPN to predict highly accurate RoI heatmaps, cropping the necessary fine-grained details to generate the correct answers.
Fig. 8 : Accuracy vs. Efficiency Trade-offs. We evaluate the Qwen2.5-VL-7B baseline against Q-Zoom by sweeping the maximum visual token limit in a image from 256 to 4,096. Our framework establishes a dominant Pareto frontier on both (a) Document & OCR and (b) High-Resolution benchmark categories. By adaptively localizing RoIs, Q-Zoom surpasses the peak accuracy of the brute-force 4,096-token baseline while reducing the visual token cost and accelerating throughput.
Components
Document & OCR
High-Res & Vision-Centric
General QA
RPN
SFT
Gate
Tp
Doc
Chart
OCR
Info
Text
Ave.
Tp
V*
RW
HR4K
HR8K
Ave.
Tp
MME
MMS
Qwen2.5-VL-7B
-
-
-
1.0 ×
92.0
83.0
82.8
70.1
81.1
81.8
1.0 ×
64.4
35.4
57.9
52.4
52.5
1.0 ×
2297
62.4
✓ †
-
-
0.50 ×
93.6
85.5
82.9
76.9
83.5
84.5
0.47 ×
77.5
40.5
73.3
66.1
64.4
0.40 ×
2335
62.1
✓
-
-
0.59 ×
94.1
85.8
84.9
79.6
83.0
85.5
0.49 ×
80.1
44.6
75.5
66.6
66.7
0.63 ×
2272
62.5
✓
✓
-
0.63 ×
94.5
86.5
85.9
79.9
83.8
86.1
0.52 ×
80.1
46.0
75.8
67.4
67.3
0.66 ×
2322
63.1
TABLE IV : Ablation study on the core components of our proposed framework. We progressively enable the upgraded Self-Distilled Region Proposal Network ( RPN ), targeted Supervised Fine-Tuning ( SFT ), and the Dynamic Gating Network ( Gate ). The † denotes results obtained using the configuration from our preliminary conference version. Tp denotes throughput, which is reported as relative speed ( × ) to baseline.
Method
Document & OCR
High-Res & Vision-Centric
Tp
Doc
Chart
OCR
Info
Text
Ave.
Tp
V*
RW
HR4K
HR8K
Ave.
(0) LLaVA-1.5-7B
1.0 ×
21.5
18.1
31.4
20.4
46.1
27.5
1.0 ×
50.3
27.7
37.5
33.8
37.3
(1) Response-to-Image Attention
0.39 ×
28.4
22.8
32.4
22.2
53.1
31.8
0.38 ×
57.1
27.6
41.8
35.3
40.5
(2) GroundingDINO (1 bbox)
0.37 ×
23.2
19.2
32.5
20.6
52.6
29.6
0.17 ×
56.5
27.9
48.9
44.8
44.5
(3) GroundingDINO (2 bbox)
0.32 ×
23.8
19.2
32.5
20.2
54.5
30.0
0.15 ×
62.3
27.6
48.0
46.3
46.1
(4) +SD-RPN
0.62 ×
34.2
20.6
37.3
22.3
58.8
34.6
0.57 ×
70.7
27.7
47.3
41.6
46.8
TABLE V : Ablation of training-free pseudo-label based ROI strategies. Throughput is reported as relative speed ( × ) to each model’s baseline under the same macro-category.
Value
Doc
Chart
OCR
Info
Text
V*
RW
Ave.
3
92.6
85.0
82.2
70.4
81.2
61.8
36.5
72.8
9
93.0
84.8
83.3
71.0
81.4
71.2
36.0
74.4
15
94.0
85.6
84.4
77.6
82.7
71.7
39.6
76.5
18
94.1
85.8
84.9
79.6
83.0
80.1
44.6
78.9
B
21
93.6
85.7
84.4
76.4
82.2
78.5
41.2
77.4
1
93.6
85.2
84.6
75.5
82.5
71.7
40.4
76.2
TABLE VI : Ablation on the backbone depth ( B ) and the number of RPN layers ( R ) using Qwen2.5-VL 7B baseline. When ablating B , R is fixed to 3; when ablating R , B is fixed to 18.
Data Size
Doc
Chart
OCR
Info
Text
V*
RW
Ave.
10k
93.6
85.4
84.9
78.0
82.7
78.5
40.7
77.7
25k
93.5
85.5
85.1
78.1
83.0
80.1
42.4
78.2
50k
93.4
85.5
84.6
78.8
83.0
78.0
43.7
78.1
100k
93.6
85.3
84.9
79.2
82.9
78.0
45.1
78.4
185k
94.1
85.8
84.9
79.6
83.0
80.1
44.6
78.9
68k †
93.4
85.5
83.6
75.8
83.0
80.1
44.3
78.0
TABLE VII : Ablation on pseudo-label training data size for the SD-RPN. The † denotes a baseline model supervised exclusively by 68K ground-truth (GT) bounding boxes from the Visual CoT dataset.
τfg
τbg
LLaVA-1.5-7B
Qwen2.5-VL-7B
OCR
Text
V*
Ave.
OCR
Text
V*
Ave.
Baseline
31.4
46.1
50.3
42.6
82.8
81.1
64.4
76.1
0.10
0.10
35.1
57.4
65.4
52.6
84.3
83.1
78.0
81.8
0.15
0.10
36.3
57.9
67.5
53.9
84.2
83.1
79.6
82.3
0.20
0.10
37.0
58.7
67.5
54.4
84.5
82.8
79.1
82.1
0.30
0.10
36.6
58.1
64.9
53.2
84.2
82.6
77.5
81.4
TABLE VIII : Ablation study on pseudo-label thresholds τfg and τbg across different model architectures.
Fig. 9 : Ablation of the Consistency-aware Training Sample Generation upon Qwen2.5-VL 7B. (Top) Training loss curves of the dynamic gating network. (Bottom) The Pareto front illustrating the trade-off between perception accuracy and inference efficiency (No-RoI Ratio).
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Config
SD-RPN
Post-SFT
Dynamic Gate
Optimizer
AdamW
Weight decay
0.0
Optimizer momentum
(β1,β2)=(0.9,0.98)
Batch size
128
64
128
Learning rate schedule
cosine decay
Peak learning rate
1×10−4
1×10−6
1×10−4
Appendix
TABLE IX : Training hyperparameters for the three core components of Q-Zoom.
Component
Model
Training Source
Samples
SD-RPN
Qwen-series
GQA
72K
OCR-VQA
80K
VCoT-DocVQA
33K
Total
185K
LLaVA-series
GQA
72K
OCR-VQA
80K
Appendix
TABLE X : Training data used by each core component of Q-Zoom.
Model
Backbone Layer B
LLaVA-1.5-7B
15
LLaVA-1.5-13B
15
Qwen2.5-VL-3B
24
Qwen2.5-VL-7B
18
ZwZ-Qwen2.5-VL-7B
18
Qwen3-VL-4B
24
Appendix
TABLE XI : Backbone split depth B for the models used in the main results. All models use the same branch depth R=3 for both SD-RPN and dynamic gating.
Vision-Language Models (VLMs) struggle as query-relevant objects become smaller. To address this, recent training-free approaches dynamically retrieve and zoom into local image regions. However, we show that indiscriminately applying retrieval ignores a critical vulnerability: the resolution-context trade-off. Patch-based zooming recovers details for small targets, but can split large objects and destroy global spatial context; attention-based retrieval better preserves large objects, but remains less reliable on tiny details; and global perception is often fastest when retrieval is unnecessary. Motivated by these failure modes, we introduce ViRGo (Visual Retrieval or Global Perception), a lightweight framework that formulates visual retrieval as an adaptive routing problem. ViRGo estimates object scale from the VLM's intrinsic localization heads during the initial forward pass and combines it with semantic token confidence to select between global perception, patch-based retrieval, and attention-based retrieval with minimal additional computation. Experiments across multiple VQA benchmarks and object-size groups show that ViRGo improves the accuracy-efficiency trade-off: it matches patch retrieval on small details, leverages attention-based retrieval for larger objects, and reduces inference time by routing to the global baseline when zooming is unnecessary.
Oanh N. Tran, Thanh Quoc Hung Le, Oscar Chew +2
VinUni-Illinois Smart Health Center, VinUniversity · Texas A&M University
Multimodal Large Language Models (MLLMs) are shifting towards "Thinking with Images" by actively exploring image details. While effective, large-scale training is computationally expensive, which has spurred growing interest in lightweight, training-free solutions. However, existing training-free methods suffer from two flaws: perceptual redundancy from indiscriminate cropping, which increases computational cost and introduces noise; and a drift between semantic intent and spatial attention, which prevents accurate localization of user-focused regions. To address these challenges, we propose LookWise, a framework for adaptive visual reasoning. LookWise follows a two-stage pipeline: a confidence-based module decides when to look more carefully, and a semantic-guided localization module determines where to look. This design enables MLLMs to adaptively acquire fine-grained visual evidence without additional training. Experiments on fine-grained and high-resolution visual reasoning benchmarks show that LookWise consistently improves accuracy over strong baselines while achieving an approximately 4.0× inference speedup over the search-based method ZoomEye, demonstrating robust cross-model generalization.
Yuxiang Shen, Hailong Huang, Zhenkun Gao +6
the Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei 230031, China, and also with the University of Science and Technology of China, Hefei 230026, China · Zhejiang University, Hangzhou 310027, China · East China Normal University, Shanghai 200241, China +3
High-resolution (HR) image perception remains a key challenge in multimodal large language models (MLLMs), as fine-grained details are often lost when the image is processed as a whole. Existing methods either require training to teach models where to look or heuristically divide the image into fixed regions, both of which struggle to generalize in complex HR scenes. In this work, we propose Hierarchical Entity Exploration (HEE), a training-free and model-agnostic framework that transforms static image understanding into dynamic, query-guided entity exploration. HEE first evaluates each region using a dual scoring mechanism to determine whether it already contains sufficient evidence to answer the question. If not, it applies object detection within the most promising region to extract fine-grained entities, clusters them into coherent subregions, and organizes them into a multi-level semantic hierarchy for deeper exploration. When deeper regions still fail to yield confident answers, a confidence-guided backtracking mechanism revisits alternative paths to ensure adaptive perception. Extensive results show that HEE outperforms training-free methods like ZoomEye and RAP in both accuracy and efficiency on two complex HR benchmarks (Visual Probe and HR-Bench), across different MLLMs such as Qwen2.5-VL and LLaVA-OneVision. Moreover, HEE demonstrates generalization on the MME-RealWorld benchmark.
Ziyu Ma, Shidong Yang, Yuxiang Ji +5
AMAP, Alibaba Group, China · Monash University, Australia