Eyes in the Update? Learning Where Rewards Land in Multimodal Reasoning
Organizations: The University of Hong Kong · TranscEngram · Nanyang Technological University
Abstract
Multimodal post-training has moved from broad instruction tuning toward reinforcement learning for image and video reasoning, where outcome rewards can improve final-answer accuracy. Yet these rewards are assigned to complete generated trajectories, while visual evidence is used only at sparse and uneven positions inside the response. Recent work has begun to make multimodal RL more token-sensitive through grounding signals or external token-level weighting. We ask a more specific question: can the allocation of the same trajectory-level reward be inferred from the model's own multimodal computation, rather than from an auxiliary annotator, perception module, or process label? We propose Gaze-RL, a representation-driven policy optimization method that redistributes completion-level advantage through a latent reward landing field. The field is computed from internal cross-modal geometry and local predictive uncertainty, assigning larger update weight to positions that are both visually anchored and decision-bearing while remaining mean-one, so the total credit of each rollout is preserved and only its positionwise allocation changes. Gaze-RL requires no process supervision, auxiliary perception module, or architectural modification, and improves both Qwen2.5-VL and InternVL3.5 across a broad range of image and video benchmarks.
Figures & tables
| VLM in the editor | Overall |
| Qwen2.5-VL-7B (original) | 7.54 |
| + GRPO | 7.55 |
| + Gaze-RL (ours) | 7.60 |
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Asset | Type | Usage in this paper | License / terms |
| Qwen2.5-VL-3B / Qwen2.5-VL-7B [ 4 ] | Model backbone | Main multimodal policy backbones | Apache-2.0 official release |
| InternVL3.5-8B [ 69 ] | Model backbone | Main multimodal policy backbone | Apache-2.0 official release |
| ViRL39K [ 67 ] | Training dataset | Image-side RL curriculum | MIT official dataset release |
| Video-R1 [ 14 ] | Training dataset / baseline | Video-side RL curriculum and video RL baseline | Apache-2.0 official dataset release |
| Qwen-Image-Edit [ 72 ] | Image-editing model | Image-editing transfer probe | Apache-2.0 official release |
| GEdit-Bench [ 39 ] | Evaluation benchmark | Quantitative image-editing transfer probe | Public benchmark terms |
RL for VLMs
Reinforcement learning for vision-language models aligns multimodal reasoning, tool use, and action with task-specific feedback.
See all 503 papers