Do MLLMs Really See It: Reinforcing Visual Attention in Multimodal LLMs

Siqu Ou; Tianrui Wan; Zhiyuan Zhao; Junyu Gao; Xuelong Li

cs.AIFeb 9, 2026

Do MLLMs Really See It: Reinforcing Visual Attention in Multimodal LLMs

Authors: Siqu Ou, Tianrui Wan, Zhiyuan Zhao, Junyu Gao, Xuelong Li

Organizations: TeleAI · Shanghai Jiao Tong University · Northwestern Polytechnical University

Abstract

While chain-of-thought (CoT) reasoning has substantially improved multimodal large language models (MLLMs) on complex reasoning tasks, existing approaches largely rely on long textual reasoning trajectories and provide limited mechanisms for learning stable visual attention policies. Our analysis shows that current MLLMs exhibit weak visual focus: early-stage visual misalignment is rarely corrected during subsequent reasoning, leading to error propagation and failed inferences. We argue that this limitation stems from inadequate credit assignment for visual attention during training. To address this issue, we propose SAYO, a visual reasoning model trained with a reinforcement learning (RL) framework that introduces a region-level visual attention-based reward. This reward explicitly aligns optimization signals with visually grounded reasoning steps, enabling the model to learn more reliable attention behaviors. Extensive experiments across multiple multimodal benchmarks demonstrate that SAYO consistently improves performance on diverse reasoning and perception tasks. Project is open at https://github.com/Cratileo/Sayo.

Figures & tables

Figure 1: In the reasoning process of CoT, initial visual focusing errors can mislead the inference processing. We emphasize enhancing the model’s visual capabilities to ensure MLLMs can proactively focus on text-relevant visual regions. The resolution of image is 2752x1824. The attention map displays the average attention of all generated tokens.

Figure 2: Comparison of target attention score (TAS) and accuracy across Models on a part of GQA dataset. The displayed score and accuracy represent the average across all samples. * denotes models based on the Qwen2.5-7B series.

Scale Value	Accuracy
1.0 (Normal)	14.0
0.5	11.0
0.1	10.0
0.0	3.0

Table 1: Comparison of attention weight and accuracy across different scale values on a part of GQA dataset in Qwen3-VL-8B-Instruct.

Figure 3: The workflow for our method, including constructing reasoning data with region visual information and training with region attention reward

Model	General				Math		Chart			Avg
Model	MMERealWorld	M3CoT	V ∗	MMStar	MathVision	We-Math	ChartQA	AI2D	CharXiv	Avg
Close-Source Models
GPT4o	73.06	74.20 †	-	64.70 †	30.40 †	69.00 †	75.32	84.60	48.90 †	-
Gemini 2.5 Pro	-	-	83.80 †	77.50 †	-	80.60 †	83.30 †	90.90 †	-	-
Open-Source General Models
Qwen3-VL-4B	56.80	65.10	79.06	60.80	21.64	55.63	80.12	75.16	38.00	59.15

Table 2: Performance of SAYO across various visual reasoning benchmarks with different tasks.

\dagger

indicates results are taken from the respective models’ official reports. The best results of each benchmark among open-source models are bold and the secondary results are underlined .

Figure 4: Attention weights of model-generated tokens to target visual tokens (last layer) and attention weights of tokens with different entropy values to target visual tokens. The entropy values shown have been normalized across samples, and the displayed attention weights represent the average across all samples.

Model	General				Math		Chart			Avg
Model	MMERealWorld	M3CoT	Vstar	MMStar	MathVision	We-Math	ChartQA	AI2D	CharXiv	Avg
Qwen3-VL-8B	56.23	64.71	81.15	62.60	22.20	52.64	78.96	75.55	42.70	59.64
+ Attn. Reward	62.85	67.86	82.20	65.27	25.26	64.83	81.84	83.06	42.50	63.96
+ Acc. Reward	57.01	65.96	78.01	65.20	23.59	56.61	82.52	78.01	41.40	60.92
Acc. + Attn.	61.59	67.77	82.20	66.00	25.95	66.15	82.56	83.03	43.20	64.27
InternVL3.5-8B	45.34	54.70	73.82	60.27	13.82	20.06	79.12	73.22	34.10	50.49

Table 3: Ablation results for different combinations of rewards and visual attention-based reward on performance improvement. Attn. Reward denotes visual attention-based reward, and Acc. Reward denotes accuracy reward. All combinations include format rewards.

Model	General				Math		Chart			Avg
Model	MMERealWorld	M3CoT	Vstar	MMStar	MathVision	We-Math	ChartQA	AI2D	CharXiv	Avg
Qwen3-VL-8B	56.23	64.71	81.15	62.60	22.20	52.64	78.96	75.55	42.70	59.64
key tokens	62.85	67.86	82.20	65.27	25.26	64.83	81.84	83.06	42.50	63.96
all tokens	56.59	66.01	75.92	64.07	22.57	58.39	82.00	77.43	41.60	60.51
InternVL3.5-8B	45.34	54.70	73.82	60.27	13.82	20.06	79.12	73.22	34.10	50.49
key tokens	50.03	57.20	72.77	62.73	13.62	25.23	82.28	76.88	43.20	53.77

Table 4: Comparative results of visual attention reward for all generated tokens and key generated tokens on reasoning performance improvement.

Figure 5: An example demonstrating how areas of visual attention shift during the reasoning process. The background color of tokens in the figure indicates the magnitude of the target visual attention score.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Parameters	Value
Epochs	4
Per Device Batch Size	64
Warmup	False
Rollout	16
Rollout Temperature	1.0
Rollout Top-P	0.9

Appendix

Table 5: The hyperparameters used during GRPO with visual attention based reward.

Datasets	Size
GQA ( Hudson and Manning, 2019 )	$\sim$ 16k
Refocus_Data ( Fu et al., 2025b )	$\sim$ 4k

Appendix

Table 6: Detailed composition of the datasets used for RL training stage.

Figure 6: Attention weight (last layer) on target visual token in Qwen3-VL-8B-Instruct and SAYO-8B. The entropy values shown have been normalized across samples, and the displayed attention weights represent the average across all samples.

Figure 7: Attention weight (last layer) on all visual tokens in Qwen3-VL-8B-Instruct and SAYO-8B. The entropy values shown have been normalized across samples, and the displayed attention weights represent the average across all samples.

Range	General		Math		Chart
Range	MMERealWorld	M3CoT	MathVision	We-Math	AI2D	CharXiv
Sayo-Qwen-8B	62.85	68.46	25.26	64.83	83.06	42.50
Top 20% tokens	58.36	65.19	17.80	59.08	79.40	40.90
Top 40% tokens	62.48	66.18	20.13	66.84	82.93	39.30

Appendix

Table 7: Ablation results for different ranges of entropy select strategies in visual attention-based reward.

Model	MMIQ	ScienceQA	MMMU-Pro
Qwen3-VL-8B	16.19	88.27	42.41
Sayo-Qwen-8B	19.93	89.46	45.61

Appendix

Table 8: Extend evaluation results on abstract reasoning tasks.

Figure 8: Comparative analysis of model accuracy and average output length across MMERealworld and AI2D.

Target Layers	MMMU_Pro	MMstar
10	42.45	65.60
20	41.92	63.27
36 (last)	45.61	67.60

Appendix

Table 9: Ablation experiments on model’s different layer.

Figure 9: Attention rewards vary with training step. The left figure shows rewards covering the top 30% of high-entropy tokens, while the right figure covers all tokens.

Figure 10: Case 1 of Sayo-Qwen-8B in ChartQA.

Figure 11: Case 2 of Sayo-Qwen-8B in We-Math

Figure 12: Case 3 of Sayo-Qwen-8B in MME-Realworld

Reasoning for VLMs

See all 55 papers

Attention-guided Fine-tuning of Multimodal Large Language Models Improves Chain-of-Thought Reasoning
Jun 1, 2026Sanchit Sinha, Guangzhi Xiong, Bohan Liu +2Multimodal CoT Reasoning Multimodal Large Language Models
University of Virginia
Deeper Thought, Weaker Aim: Understanding and Mitigating Perceptual Impairment during Reasoning in Multimodal Large Language Models
Mar 15, 2026Ruiying Peng, Xueyu Wu, Jing Lei +3Multimodal Large Language Models VLM Reasoning
Tsinghua Shenzhen International Graduate School · Huawei Technologies
Balancing Efficiency and Efficacy: Training-Free Attention-Guided Switching Between Explicit and Latent Thoughts for MLLMs
Aug 4, 2026Haoqian Kang, Liupeng Li, Kuofeng Gao +5Efficient Multimodal Inference Multimodal Large Language Models
Harbin Institute of Technology, Shenzhen Shenzhen, China · Tsinghua Shenzhen International Graduate School, Tsinghua University Shenzhen, China · Peng Cheng Laboratory Shenzhen, China +1

Jun 1, 2026J/K move · Enter open · S save

Attention-guided Fine-tuning of Multimodal Large Language Models Improves Chain-of-Thought Reasoning

Sanchit Sinha, Guangzhi Xiong, Bohan Liu, Zhenghao He+1

University of Virginia

Multimodal CoT Reasoning Multimodal Large Language Models VLM Reasoning Fine-Tuning Visually Grounded Reasoning Multimodal Understanding+1

Open paper AskPDF arXiv

The effectiveness of Chain-of-Thought (CoT) prompting in Multimodal Large Language Models (MLLMs) remains uncertain: across several visual reasoning benchmarks, CoT prompting often degrades performance compared to direct prompting. In this paper, we provide a systematic analysis of CoT behavior in three modern MLLM families across model scales on datasets requiring step-wise visual evidence. Our analysis identifies two recurring failure modes: premature answer commitment and limited direct visual-token access during rationale generation. We further find that standard CoT-style Supervised Fine-Tuning (CoT-SFT) can mitigate these issues only partially, while often increasing reliance on textual priors and reducing counterfactual visual dependence. Motivated by these findings, we propose Attentive-CoT (Att-CoT), an attention-guided fine-tuning objective that encourages CoT trajectories to delay answer commitment while maintaining sustained visual-token access. Att-CoT can be plugged into any CoT-SFT training run without architectural changes. Experiments on three visual reasoning benchmarks across six MLLMs show that Att-CoT enhances CoT performance over standard fine-tuning.

Abstract

Reasoning for VLMs

Explore similar work

Attention-guided Fine-tuning of Multimodal Large Language Models Improves Chain-of-Thought Reasoning

Deeper Thought, Weaker Aim: Understanding and Mitigating Perceptual Impairment during Reasoning in Multimodal Large Language Models

Balancing Efficiency and Efficacy: Training-Free Attention-Guided Switching Between Explicit and Latent Thoughts for MLLMs