ConFoThinking: Consolidated Focused Attention Driven Thinking for Visual Question Answering

Zhaodong Wu; Haochen Xue; Qi Cao; Wenqi Mo; Yu Pei; Wenqi Xu; Jionglong Su; Yang Liu

cs.CVFeb 26, 2026

ConFoThinking: Consolidated Focused Attention Driven Thinking for Visual Question Answering

Authors: Zhaodong Wu, Haochen Xue, Qi Cao, Wenqi Mo, Yu Pei, Wenqi Xu, Jionglong Su, Yang Liu

Organizations: Wangxuan Institute of Computer Technology, Peking University, Beijing, China · School of AI and Advanced Computing, Xi’an Jiaotong-Liverpool University, Suzhou, China

Abstract

Thinking with Images improves fine-grained VQA for MLLMs by emphasizing visual cues. However, tool-augmented methods depend on the capacity of grounding, which remains unreliable for MLLMs. In parallel, attention-driven methods to crop the Region of Interest (ROIs) are proposed but they are constrained by (1) fragmented attention signals scattered across layers, leading to suboptimal localization and (2) relying on question- or redundant-text-conditioned attention extraction. Our analysis reveals three patterns: MLLMs may attend to the correct region yet generate incorrect coordinates, where-to-look attention is often fragmented across layers, and attention extraction is query-sensitive. Motivated by these, We propose ConFoThinking, a Consolidated-Focused-Attention-Driven Thinking framework that learns to aggregate attention into a designated intermediate layer, from which we mine and zoom in salient regions for downstream visual understanding. Moreover, we extract attention using concise semantic cues of what to look into, which mitigates the semantic noise introduced by question- or redundant-text-based attention extraction. Experiments across five VQA benchmarks demonstrate ConFoThinking significantly improves perception performance. The code, checkpoints, and dataset will be released after being accepted.

Figures & tables

Figure 1: Limitations of existing methods. (A) In tool-augmented coordinate-output pipelines, MLLMs may “say” incorrect bounding-box coordinates, even though their intermediate vision-language fusion layers still attend to the GT ROI. The line chart shows the attention distribution inside the predicted and GT boxes for Qwen3-VL-8B (with tools) on V* benchmark ( Wu and Xie, 2024 ) , where the model outputs an incorrect answer and the predicted box has IoU $<$ 0.1 with the ground-truth box. (B) In attention-driven methods, where-to-look signals are fragmented across layers, making any fixed-layer choice unreliable. The bar chart shows the distribution of the highest-ROI-attention layer for Qwen3-VL-8B across samples in VisCoT (only 19.3% at a fixed single layer (layer22)). (C) Where-to-look signals are query-sensitive: extracting attention from semantic visual cues is more accurate than extracting it from the raw question. The line chart compares the layer-wise attention distributions of these two approaches on the V* benchmark.

Figure 2: ConFoThinking overview. (A) Training ConFoAttn to produce a <FOCUS> … </FOCUS> span and fixed-layer attention heatmaps. (B) Training AttnDetector to regress ROI boxes from attention heatmaps. (C) Inference pipeline: generate heatmap, localize and zoom the ROI, then answer with the base MLLM using both the original and zoomed images.

Model	High-resolution									OCR	General
	V* Bench			HRBench-4K			HRBench-8K			$\text{InfoVQA}_{\text{Val}}$	GQA
	Attr.	Spatial	Overall	FSP	FCP	Overall	FSP	FCP	Overall	$\text{InfoVQA}_{\text{Val}}$	GQA
Open-source Models
Qwen2.5-VL-7B	78.2	73.6	76.4	85.3	52.3	68.8	78.8	51.8	65.3	77.9	62.1
Qwen3-VL-4B	80.0	75.0	78.0	86.0	62.5	74.3	82.3	58.0	70.1	79.3	69.3
Qwen3-VL-8B	86.1	81.6	84.3	88.3	64.5	76.4	86.3	58.0	72.1	81.7	71.3

Table 1: Results on high-resolution, OCR, and general VQA benchmarks. For Ours , the

\uparrow

value denotes the absolute gain over the corresponding base backbone (e.g., ConFoThinking (Qwen2.5-VL-7B) vs. Qwen2.5-VL-7B).

Method	Qwen2.5-VL-7B	Qwen3-VL-8B
w/o NTP & AC	76.4	84.3
NTP only	81.2	88.0
NTP + AC	86.9	92.1

Table 2: Ablation I: effect of attention-condensation training.

Text Query	Qwen2.5-VL-7B	Qwen3-VL-8B
Question-only	85.9	89.0
All generated text	80.1	84.3
<FOCUS> span	86.9	92.1

Table 3: Ablation II: text query for attention extraction.

Extraction layers	Train	V*
Single layer	$\{22\}$	92.1
3-layer window	$\{21,22,23\}$	91.1
5-layer window	$\{20,21,22,23,24\}$	90.6

Table 4: Ablation III on V* (Qwen3-VL-8B): single-layer extraction vs. neighborhood layer aggregation (the same layer set is used for training and inference).

Figure 3: Comparison for attention condensation.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Layer-wise comparison of attention concentration on

R_{\text{GT}}

vs.

R_{\text{pred}}

for Pixel-Reasoner over grounding-error cases (

\mathrm{IoU}<0.1

Figure 5: Distribution of the peak-attention layer $\ell^{\star}$ on the Validation set, VisCoT. Left: Qwen3-VL-4B (mode at Layer 22). Right: Qwen2.5-VL-7B (mode at Layer 20). Despite the existence of a modal layer, the distribution remains broadly dispersed across layers.

Visual Question Answering

Visual Question Answering (VQA) develops systems that answer natural-language questions about images, videos, and visual documents.

See all 253 papers

R3G: A Reasoning-Retrieval-Reranking Framework for Vision-Centric Answer Generation
Jan 25, 2026Zhuohong Chen, Zhengxian Wu, Zirui Liao +6Multimodal RAG Visual Question Answering
1The Shenzhen International Graduate School, Tsinghua University, China · School of Computer Science and Information Engineering, Hefei University of Technology, China · 2State Key Laboratory of Nuclear Power Safety Technology and Equipment, China
Look Twice: Training-Free Evidence Highlighting for Knowledge-based Visual Question Answering
Apr 1, 2026Marco Morini, Sara Sarto, Marcella Cornia +2Knowledge-Based VQA VLM Inference
University of Modena and Reggio Emilia, Italy
ReVA: A Region-Aware Visual Assistant for Visually Grounded Question Answering
Aug 27, 2026Anoop SenthilVisual Question Answering Vision-Language Grounding
School of Electronic Engineering and Computer Science Queen Mary University of London London, United Kingdom

Jan 25, 2026J/K move · Enter open · S save

R3G: A Reasoning-Retrieval-Reranking Framework for Vision-Centric Answer Generation

Zhuohong Chen, Zhengxian Wu, Zirui Liao, Shenao Jiang+5

1The Shenzhen International Graduate School, Tsinghua University, China · School of Computer Science and Information Engineering, Hefei University of Technology, China · 2State Key Laboratory of Nuclear Power Safety Technology and Equipment, China

Multimodal RAG Visual Question Answering Large Vision-Language Models VLM Reasoning Visually Grounded Reasoning Vision-Language Retrieval+1

Open paper AskPDF arXiv

Vision-centric retrieval for VQA requires retrieving images to supply missing visual cues and integrating them into the reasoning process. However, selecting the right images and integrating them effectively into the model's reasoning remains challenging. To address this challenge, we propose R3G, a modular Reasoning-Retrieval-Reranking framework. It first produces a brief reasoning plan that specifies the required visual cues, then adopts a two-stage strategy, with coarse retrieval followed by fine-grained reranking, to select evidence images. On MRAG-Bench, R3G improves accuracy across six MLLM backbones and nine sub-scenarios, achieving state-of-the-art overall performance. Ablations show that sufficiency-aware reranking and reasoning steps are complementary, helping the model both choose the right images and use them well. We release code and data at https://github.com/czh24/R3G.

Abstract

Visual Question Answering

Explore similar work

R3G: A Reasoning-Retrieval-Reranking Framework for Vision-Centric Answer Generation

Look Twice: Training-Free Evidence Highlighting for Knowledge-based Visual Question Answering

ReVA: A Region-Aware Visual Assistant for Visually Grounded Question Answering