GRIT: Gradient Masking for Visually Misgrounded Tokens in Text-Rich Reasoning
Organizations: Machine Intelligence Laboratory, University of Seoul Data Science Interdisciplinary Major
Abstract
Reinforcement learning with verifiable rewards (RLVR) has recently been extended to vision-language models (VLMs), yet its effectiveness remains limited for text-rich reasoning tasks. We highlight a key factor contributing to this limitation: models frequently arrive at correct final answers despite misperceiving visual information during intermediate steps. We define this phenomenon as Correct-but-Visually-Misgrounded Reasoning (CVMR). Because standard RLVR rewards only final-answer correctness, it fails to differentiate between properly grounded and misgrounded reasoning, reinforcing both equally. Through token-level analysis, we reveal that CVMR tokens exhibit distinct attention patterns that differ systematically from well-grounded ones. Based on this observation, we propose GRIT, a training framework that incorporates visual grounding into RLVR to mitigate the impact of CVMR. Furthermore, to address the lack of text-rich reasoning benchmarks that allow for systematic evaluation across text length and reasoning depth, we introduce GRIT-Bench. Experiments demonstrate that GRIT significantly improves both reasoning accuracy and grounding fidelity over existing RLVR methods.
Figures & tables
| Simple | Moderate | Complex | Total | |
|---|---|---|---|---|
| Short ( 1K) | 198 | 396 | 169 | 763 |
| Medium (1–2K) | 210 | 476 | 270 | 956 |
| Long (2K+) | 263 | 540 | 333 | 1,136 |
| Total | 671 | 1,412 | 772 | 2,855 |
| Text-Rich Reasoning | Chart Understanding | General | |||||||||||
| OCR-Reasoning | ChartQA-P | CharXiv | MMStar | HalluB. | |||||||||
| Model | Spatial | Numerical | Math | Enum. | Logical | Multi. | Overall | Reas. | Desc. | ||||
| Based on Qwen2.5-VL-3B | |||||||||||||
| Qwen2.5-VL-3B | 9.17 | 10.55 | 10.78 | 20.11 | 8.33 | 13.14 | 12.07 | – | 29.8 | 29.7 | 65.3 | 52.9 | 38.5 |
| GRPO-3B | 15.60 | 9.05 | 8.82 | 18.44 | 9.72 | 13.87 | 11.97 | -0.10 | 30.0 | 29.8 | 65.3 | 53.2 | 38.2 |
| DAPO-3B | 11.01 | 8.79 | 9.80 | 20.11 | 9.72 | 12.41 | 11.60 | -0.47 | 30.3 | 29.4 | 65.2 | 53.1 | 37.8 |
| Short ( 1K) | Medium (1–2K) | Long (2K+) | Overall | |||||||||||||||||||
| Simple | Moderate | Complex | Simple | Moderate | Complex | Simple | Moderate | Complex | ||||||||||||||
| Model | Acc | RF | Acc | RF | Acc | RF | Acc | RF | Acc | RF | Acc | RF | Acc | RF | Acc | RF | Acc | RF | Acc | RF | Acc | RF |
| Based on Qwen2.5-VL-3B | ||||||||||||||||||||||
| Qwen2.5-VL-3B | 45.5 | 61.9 | 52.0 | 67.4 | 49.7 | 69.0 | 50.0 | 57.7 | 44.5 | 62.5 | 40.0 | 65.3 | 39.5 | 52.0 | 40.9 | 61.1 | 39.0 | 64.0 | 44.1 | 62.4 | – | – |
| GRPO-3B | 47.0 | 62.7 | 51.3 | 68.0 | 46.2 | 68.7 | 49.5 | 57.0 | 43.3 | 61.0 | 38.5 | 65.5 | 37.3 | 49.8 | 38.3 | 57.2 | 36.6 | 62.7 | 42.6 | 61.2 | -1.5 | -1.2 |
| DAPO-3B | 51.5 | 62.8 | 53.3 | 67.3 | 45.6 | 69.5 | 44.8 | 56.6 | 44.1 | 62.2 | 37.4 | 65.4 | 39.9 | 52.2 | 40.9 | 58.6 | 36.0 | 62.4 | 43.5 | 61.8 | -0.6 | -0.6 |
| Model | OCR-Reasoning | GRIT-Bench |
|---|---|---|
| Qwen2.5-VL-7B | 20.58 | 66.0 |
| + GRPO | 18.99 ( 1.59) | 63.9 ( 2.1) |
| + DAPO | 19.55 ( 1.03) | 64.7 ( 1.3) |
| + Gradient Masking | 24.04 ( 4.49) | 74.9 ( 10.2) |
| + Attention Reward | 25.54 ( 1.50) | 77.7 ( 2.8) |
| Masking Strategy | OCR-Reasoning | GRIT-Bench |
|---|---|---|
| DAPO-7B | 19.55 | 64.7 |
| + Inverted Masking | 17.49 ( 2.06) | 59.0 ( 5.7) |
| + Random Masking | 19.46 ( 0.09) | 62.4 ( 2.3) |
| + GRIT Masking (Ours) | 24.04 ( 4.49) | 74.9 ( 10.2) |
| Masking Ratio | Heads | OCR-Reasoning | GRIT-Bench |
|---|---|---|---|
| 30/30 | 5 | 23.57 | 75.1 |
| 50/50 | 5 | 25.54 | 77.7 |
| 70/70 | 5 | 24.70 | 76.4 |
| 50/50 | 3 | 25.26 | 77.2 |
| 50/50 | 5 | 25.54 | 77.7 |
| 50/50 | 10 | 24.51 | 76.7 |
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Setting | Time/step | Total wall-clock | Relative |
|---|---|---|---|
| GRPO (no attention extraction) | 650 s | 110.1 h | |
| GRIT (all-layer extraction) | 1,638 s | 277.6 h | |
| GRIT (selective extraction, adopted) | 838 s | 142.1 h |
| Dataset | Samples |
|---|---|
| ConvFinQA [ 7 ] | 476 |
| WTQ [ 23 ] | 460 |
| DROP [ 9 ] | 450 |
| TAT-QA [ 42 ] | 428 |
| HiTab [ 8 ] | 417 |
| FinQA [ 6 ] | 349 |
| Model | OCR-Reasoning | GRIT-Bench | |
|---|---|---|---|
| Acc. | RF | ||
| Qwen3-VL-4B-Instruct (base) | 32.27 | 71.1 | 80.3 |
| GRPO-4B | 31.99 ( ) | 69.4 ( ) | 78.1 ( ) |
| DAPO-4B | 32.09 ( ) | 70.9 ( ) | 78.3 ( ) |
| GRIT-4B (Ours) | 36.01 ( ) | 77.5 ( ) | 83.9 ( ) |
VLM Reasoning
Vision-language model reasoning studies how multimodal systems use visual evidence and language to answer questions, plan, and explain.
See all 873 papers