Reinforcement learning with verifiable rewards (RLVR) has recently been extended to vision-language models (VLMs), yet its effectiveness remains limited for text-rich reasoning tasks. We highlight a key factor contributing to this limitation: models frequently arrive at correct final answers despite misperceiving visual information during intermediate steps. We define this phenomenon as Correct-but-Visually-Misgrounded Reasoning (CVMR). Because standard RLVR rewards only final-answer correctness, it fails to differentiate between properly grounded and misgrounded reasoning, reinforcing both equally. Through token-level analysis, we reveal that CVMR tokens exhibit distinct attention patterns that differ systematically from well-grounded ones. Based on this observation, we propose GRIT, a training framework that incorporates visual grounding into RLVR to mitigate the impact of CVMR. Furthermore, to address the lack of text-rich reasoning benchmarks that allow for systematic evaluation across text length and reasoning depth, we introduce GRIT-Bench. Experiments demonstrate that GRIT significantly improves both reasoning accuracy and grounding fidelity over existing RLVR methods.
Figures & tables
Figure 1: Performance change over the base model on text-rich reasoning benchmarks. Existing RLVR methods mostly degrade or show marginal improvement, while GRIT achieves substantial gains.
Figure 2: An example of Correct-but-Visually-Misgrounded Reasoning (CVMR). The model correctly identifies the relative ranking, noting that Health Care has the highest representation in the infographic, and arrives at the correct answer. However, despite multiple industries having meaningful percentages, the model misreads all values including Health Care itself. Red-highlighted text indicates visually misgrounded reasoning.
Figure 3: CVMR rate among correct responses by task type. Text-rich reasoning exhibits a substantially higher CVMR rate compared to other task types.
Figure 4: Visual attention mass and entropy of CVMR tokens vs. normal tokens, measured on Visual Tracker Heads. CVMR tokens show lower attention mass and higher attention entropy (two-sample z -test, both p -values <10−4 ).
Simple
Moderate
Complex
Total
Short ( < 1K)
198
396
169
763
Medium (1–2K)
210
476
270
956
Long (2K+)
263
540
333
1,136
Total
671
1,412
772
2,855
Table 1: Distribution of GRIT-Bench samples across difficulty combinations.
Text-Rich Reasoning
Chart Understanding
General
OCR-Reasoning
ChartQA-P
CharXiv
MMStar
HalluB.
Model
Spatial
Numerical
Math
Enum.
Logical
Multi.
Overall
Δ
Reas.
Desc.
Based on Qwen2.5-VL-3B
Qwen2.5-VL-3B
9.17
10.55
10.78
20.11
8.33
13.14
12.07
–
29.8
29.7
65.3
52.9
38.5
GRPO-3B
15.60
9.05
8.82
18.44
9.72
13.87
11.97
-0.10
30.0
29.8
65.3
53.2
38.2
DAPO-3B
11.01
8.79
9.80
20.11
9.72
12.41
11.60
-0.47
30.3
29.4
65.2
53.1
37.8
Table 2: Results across text-rich reasoning and general benchmarks. Best results are in bold , second best are underlined . Δ denotes the difference from the base model.
Short ( < 1K)
Medium (1–2K)
Long (2K+)
Overall
Δ
Simple
Moderate
Complex
Simple
Moderate
Complex
Simple
Moderate
Complex
Model
Acc
RF
Acc
RF
Acc
RF
Acc
RF
Acc
RF
Acc
RF
Acc
RF
Acc
RF
Acc
RF
Acc
RF
Acc
RF
Based on Qwen2.5-VL-3B
Qwen2.5-VL-3B
45.5
61.9
52.0
67.4
49.7
69.0
50.0
57.7
44.5
62.5
40.0
65.3
39.5
52.0
40.9
61.1
39.0
64.0
44.1
62.4
–
–
GRPO-3B
47.0
62.7
51.3
68.0
46.2
68.7
49.5
57.0
43.3
61.0
38.5
65.5
37.3
49.8
38.3
57.2
36.6
62.7
42.6
61.2
-1.5
-1.2
DAPO-3B
51.5
62.8
53.3
67.3
45.6
69.5
44.8
56.6
44.1
62.2
37.4
65.4
39.9
52.2
40.9
58.6
36.0
62.4
43.5
61.8
-0.6
-0.6
Table 3: GRIT-Bench results across text length and reasoning depth. Acc: Accuracy (%), RF: Reasoning Fidelity (%) = faithful / (total_steps + ungrounded_claims). Best results are in bold , second best are underlined . Δ denotes the difference from the base model.
Figure 5: GRIT-Bench analysis on 7B models across text length and reasoning depth. (a) Accuracy by text length. (b) Accuracy by reasoning depth.
Model
OCR-Reasoning
GRIT-Bench
Qwen2.5-VL-7B
20.58
66.0
└ + GRPO
18.99 ( ↓ 1.59)
63.9 ( ↓ 2.1)
└ + DAPO
19.55 ( ↓ 1.03)
64.7 ( ↓ 1.3)
└ + Gradient Masking
24.04 ( ↑ 4.49)
74.9 ( ↑ 10.2)
└ + Attention Reward
25.54 ( ↑ 1.50)
77.7 ( ↑ 2.8)
Table 4: Ablation study on component contributions. Each row adds a component to the configuration it is indented under; arrows denote the change from that configuration.
Masking Strategy
OCR-Reasoning
GRIT-Bench
DAPO-7B
19.55
64.7
└ + Inverted Masking
17.49 ( ↓ 2.06)
59.0 ( ↓ 5.7)
└ + Random Masking
19.46 ( ↓ 0.09)
62.4 ( ↓ 2.3)
└ + GRIT Masking (Ours)
24.04 ( ↑ 4.49)
74.9 ( ↑ 10.2)
Table 5: Ablation on masking strategies (without Attention Reward). Inverted Masking masks the complementary set of tokens selected by GRIT’s filtering. Arrows denote the change from DAPO-7B.
Masking Ratio
Heads
OCR-Reasoning
GRIT-Bench
30/30
5
23.57
75.1
50/50
5
25.54
77.7
70/70
5
24.70
76.4
50/50
3
25.26
77.2
50/50
5
25.54
77.7
50/50
10
24.51
76.7
Table 6: Ablation on masking ratio and number of Visual Tracker Heads. Default setting (50/50 ratio, 5 heads) is highlighted.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Setting
Time/step
Total wall-clock
Relative
GRPO (no attention extraction)
650 s
110.1 h
1.00×
GRIT (all-layer extraction)
1,638 s
277.6 h
2.52×
GRIT (selective extraction, adopted)
838 s
142.1 h
1.29×
Appendix
Table 7: Wall-clock training cost on Qwen2.5-VL-7B using three H200 GPUs. All settings use the same training configuration, and relative cost is normalized by GRPO.
Dataset
Samples
ConvFinQA [ 7 ]
476
WTQ [ 23 ]
460
DROP [ 9 ]
450
TAT-QA [ 42 ]
428
HiTab [ 8 ]
417
FinQA [ 6 ]
349
Appendix
Table 8: Source datasets and sample counts for GRIT-Bench.
Model
OCR-Reasoning
GRIT-Bench
Acc.
RF
Qwen3-VL-4B-Instruct (base)
32.27
71.1
80.3
GRPO-4B
31.99 ( −0.28 )
69.4 ( −1.7 )
78.1 ( −2.2 )
DAPO-4B
32.09 ( −0.18 )
70.9 ( −0.2 )
78.3 ( −2.0 )
GRIT-4B (Ours)
36.01 ( +3.74 )
77.5 ( +6.4 )
83.9 ( +3.6 )
Appendix
Table 9: Results on Qwen3-VL-4B-Instruct. Values in parentheses denote changes from the base model. Best results are in bold.
Figure 6: GRIT-Bench example with rendered image, question, key reasoning steps, and answer.
Figure 7: GRIT-Bench example with rendered image, question, key reasoning steps, and answer.
Figure 8: GRIT-Bench example with rendered image, question, key reasoning steps, and answer.
Figure 9: GRIT-Bench example with rendered image, question, key reasoning steps, and answer.
Figure 10: Visual Tracker Head attention maps on a CVMR example. Each heatmap corresponds to the highlighted token. Red highlights indicate CVMR tokens; green highlights indicate normal tokens.
Figure 11: Visual Tracker Head attention maps on a CVMR example. Each heatmap corresponds to the highlighted token. Red highlights indicate CVMR tokens; green highlights indicate normal tokens.
Figure 12: Visual Tracker Head attention maps on a CVMR example. Each heatmap corresponds to the highlighted token. Red highlights indicate CVMR tokens; green highlights indicate normal tokens.
Vision-Language Models (VLMs) frequently suffer from visual perception errors and hallucinations that compromise answer accuracy in complex reasoning tasks. Reinforcement Learning with Verifiable Rewards (RLVR) offers a promising solution by optimizing policies using answer correctness signals. Despite their effectiveness, prevailing RLVR methods face two critical limitations. First, much of the sampling budget is wasted on trajectories doomed to fail due to early visual description errors. Second, sparse rewards cannot distinguish whether failures stem from visual perception or reasoning stages. We introduce MIRL, a decoupled framework that addresses both limitations by leveraging mutual information (MI) between generated descriptions and visual inputs as a cheap pre-screening signal. This enables intelligent budget allocation toward high-potential trajectories via forking, while decoupled training provides independent MI-based rewards for visual perception optimization, resolving reward blindness. Experiments on six vision-language reasoning benchmarks demonstrate that MIRL achieves 70.22% average accuracy and successfully surpasses the performance of sampling 16 complete trajectories using only 10 pre-samples with top-6 selection (25% fewer complete trajectories). Our code is available at: https://anonymous.4open.science/r/mirl-main/.
Yin Zhang, Jiaxuan Zhao, Zonghan Wu +5
School of Mathematics, Tianjin University, Tianjin, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · Shanghai Advanced Institute of Finance (SAIFS), East China Normal University, Shanghai, China +4
While visually grounded Chain-of-Thought (CoT) has emerged as a promising paradigm to enhance fine-grained perception in multimodal large language models (MLLMs), its efficacy during the inference phase remains underexplored. In this work, we empirically find that mandating explicit object boxes in visually grounded CoT during inference often degrades performance compared to standard textual CoT, which reasons without explicit visual grounding. We hypothesize that the visual localization capability can be internalized into the textual CoT and that the mandatory explicit grounding introduces unnecessary interference with the model's primary objective of answer prediction. To address this problem, we propose Internalizing Visually Grounded Reasoning (iVGR), a novel reinforcement learning framework that transfers localization capabilities into the textual reasoning process. We employ a dual-stream training strategy, where a textual stream is aligned with a high-quality visually grounded stream via a proposed consistency reward, enabling the model to localize accurately without explicit grounding during inference. Extensive experiments demonstrate that our method significantly outperforms existing baselines on fine-grained benchmarks, while maintaining the flexibility to support tool-assisted inference workflows.
Chang-Bin Zhang, Yujie Zhong, Qiang Zhang +1
Visual AI Lab, The University of Hong Kong · Independent Researcher · University of Science and Technology of China
Reinforcement learning has brought strong gains to language model reasoning, but its improvements on visual tasks remain less consistent. One difficulty is that visual task failures can arise from different sources: the same benchmark may be limited by perception for one model and by reasoning for another. We introduce Perception and Reasoning Attribution (PARA), which measures the remaining perceptual and reasoning headroom of each model--benchmark pair by comparing how many current errors are recovered with more accurate visual information versus a stronger reasoner. Across 4 vision-language models and 17 multimodal benchmarks, PARA broadly agrees with the expected characteristics of existing benchmarks while revealing substantial model-specific variation. More importantly, it shows that RL gains become smaller as perceptual headroom increases, but larger when reasoning explains more of the recoverable errors, suggesting that current RL exploits reasoning headroom more effectively than perceptual headroom. Motivated by this gap, we analyze how existing RL objectives affect visual perception and find that reward- and token-level signals influence visual attention only indirectly. We therefore propose GRAD, a group-relative attention distillation objective that uses the best rollout in each group to directly guide the visual attention of weaker rollouts. GRAD consistently improves performance across two model scales and seven benchmarks, and further complements existing RL methods.