RL for VLMs

RL: Reinforcement Learning · VLM: Vision-Language Model

This field studies how reinforcement learning can improve models that interpret images or video alongside language, and sometimes take actions in visual environments. Common approaches include group-relative policy optimization, PPO, and reinforcement learning from verifiable rewards, with feedback based on answer correctness, visual evidence, intermediate reasoning, preferences, or tool use. Applications range from image and video question answering, spatial reasoning, and medical analysis to navigation and robotic manipulation. A central challenge is ensuring that better scores reflect genuine use of visual evidence rather than language shortcuts, while designing reliable rewards and efficient training that generalize beyond the training setting.

Latest papers

476 papers68 papers in the last four weeks, up 94% on the four weeks before. 0.4% of all new papers.

ProcessThinker: Enhancing Multi-modal Large Language Models Reasoning via Rollout-based Process Reward
Apr 23, 2026Jingpei Wu, Xiao Han, Weixiang Shen +3Multimodal Understanding Multimodal QA
LMU Munich · Harvard University · University of Cambridge +2
Incentivizing Neuro-symbolic Language-based Reasoning in VLMs via Reinforcement Learning
Apr 23, 2026Karthic PalaniappanVLM Reasoning Neuro-Symbolic Reasoning
Georgia Institute of Technology
Reinforcing 3D Understanding in Point-VLMs via Geometric Reward Credit Assignment
Apr 23, 2026Jingkun Chen, Ruoshi Xu, Mingqi Gao +23D Spatial Reasoning VLM Hallucination
Northwestern Polytechnical University China · Southern University of Science and Technology China · The University of Sheffield United Kingdom +2
V-tableR1: Process-Supervised Multimodal Table Reasoning with Critic-Guided Policy Optimization
Apr 22, 2026Yubo Jiang, Yitong An, Xin Yang +7Multimodal Reasoning Reinforcement Learning with Verifiable Rewards
School of Astronautics, Beihang University, Beijing 102206, China · Longcat Interaction Team, Meituan, Beijing 100102, China · Tianmushan Laboratory, Beihang University, Hangzhou 311115, China
SSL-R1: Self-Supervised Visual Reinforcement Post-Training for Multimodal Large Language Models
Apr 22, 2026Jiahao Xie, Alessio Tonioni, Nathalie Rauschmayr +2Multimodal Large Language Models Reinforcement Learning with Verifiable Rewards
Max Planck Institute for Informatics, SIC · VIA Research Center · Google
ProMMSearchAgent: A Generalizable Multimodal Search Agent Trained with Process-Oriented Rewards
Apr 22, 2026Wentao Yan, Shengqin Wang, Huichi Zhou +4Multimodal IR Process Reward Models
East China Normal University · Huawei Noah’s Ark Lab · University College London +2
Video-ToC: Video Tree-of-Cue Reasoning
Apr 22, 2026Qizhong Tan, Zhuotao Tian, Guangming Lu +2Video Reasoning VLM Reasoning
Harbin Institute of Technology, Shenzhen
HyLaR: Hybrid Latent Reasoning with Decoupled Policy Optimization
Apr 22, 2026Tao Cheng, Shi-Zhe Chen, Hao Zhang +3Multimodal Large Language Models Multimodal Reasoning
Tencent PCG, China · Tencent CSIG, China
Visual Reasoning through Tool-supervised Reinforcement Learning
Apr 21, 2026Qihua Dong, Gozde Sahin, Pei Wang +4Tool Use in VLMs VLM Reasoning
1Northeastern University · 2Amazon AGI
Rethinking Reinforcement Fine-Tuning in LVLM: Convergence, Reward Decomposition, and Generalization
Apr 21, 2026Carter Adams, Rafael Oliveira, Gabriel Almeida +1Tool Use in VLMs Reinforcement Learning
Federal University of Bahia
Multi-modal Reasoning with LLMs for Visual Semantic Arithmetic
Apr 21, 2026Chuou Xu, Liya Ji, Qifeng ChenLarge Vision-Language Models VLM Reasoning
The Hong Kong University of Science and Technology
EVE: Verifiable Self-Evolution of MLLMs via Executable Visual Transformations
Apr 20, 2026Yongrui Heng, Chaoya Jiang, Han Yang +2Synthetic Data Generation Multimodal Large Language Models
School of Control Science and Engineering, Shandong University Jinan, China · Zeekr, Geely Auto Shanghai, China
RS-HyRe-R1: A Hybrid Reward Mechanism to Overcome Perceptual Inertia for Remote Sensing Images Understanding
Apr 19, 2026Gaozhi Zhou, Hu He, Peng Shen +8Vision-Language Models Visual Question Answering
School of Mechanical and Electrical Engineering, Central South University, Changsha 410083, China · School of Geosciences and Info-Physics, Central South University, Changsha 410083, China
Waking Up Blind: Cold-Start Optimization of Supervision-Free Agentic Trajectories for Grounded Visual Perception
Apr 19, 2026Ashutosh Bajpai, Tamal Majumder, Akshay Nambi +1Reinforcement Learning Self-Supervised RL
Indian Institute of Technology Delhi, India · Microsoft Research, India · Indian Institute of Technology Abu Dhabi, UAE
Freshness-Aware Prioritized Experience Replay for LLM/VLM Reinforcement Learning
Apr 18, 2026Weiyu Ma, Yongcheng Zeng, Yan Song +4Reinforcement Learning RL Post-Training
King Abdullah University of Science and Technology (KAUST) · Chinese Academy of Sciences, Institute of Automation (CASIA) · AI Centre, Department of Computer Science, University College London +1
EasyVideoR1: Easier RL for Video Understanding
Apr 18, 2026Chuanyu Qin, Chenxu Yang, Qingyi Si +6Reinforcement Learning Video-Language Models
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China
Q-DeepSight: Incentivizing Thinking with Images for Image Quality Assessment and Refinement
Apr 18, 2026Xudong Li, Jiaxi Tan, Ziyin Zhou +6Tool Use in VLMs Multimodal CoT Reasoning
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · Beijing Institute of Technology
S-GRPO: Unified Post-Training for Large Vision-Language Models
Apr 17, 2026Yuming Yan, Kai Tang, Sihong Chen +4VLM Adaptation Group Relative Policy Optimization
Tencent Shenzhen, Guangdong, China
UniDoc-RL: Coarse-to-Fine Visual RAG with Hierarchical Actions and Dense Rewards
Apr 16, 2026Jun Wang, Shuo Tan, Zelong Sun +5Document Understanding Multimodal RAG
Glint Lab
Targeted Exploration via Unified Entropy Control for Reinforcement Learning
Apr 16, 2026Chen Wang, Lai Wei, Yanzhi Zhang +5Reinforcement Learning RL for Language Model Reasoning
College of Software, Nankai University · Zhongguancun Academy · Shanghai Jiao Tong University +3
Learning Adaptive Reasoning Paths for Efficient Visual Reasoning
Apr 16, 2026Yixu Huang, Tinghui Zhu, Muhao ChenEfficient VLM Inference Visual Reasoning
♠Fudan University · ♣University of California, Davis
Reward Design for Physical Reasoning in Vision-Language Models
Apr 15, 2026Derek Lilienthal, Manisha Mukherjee, Sameera HorawalavithanaVLM Reasoning Physical Reasoning
Pacific Northwest National Laboratory, Richland, WA 99352, USA
CSPO: Alleviating Reward Ambiguity for Structured Table-to-LaTeX Generation
Apr 13, 2026Yunfan Yang, Cuiling Lan, Jitao Sang +1Reward Decomposition Structured Output Generation
Beijing Key Laboratory of Traffic Data Mining and Embodied Intelligence, Beijing Jiaotong University · Microsoft Research Asia · Peng Cheng Laboratory
Cognitive Pivot Points and Visual Anchoring: Unveiling and Rectifying Hallucinations in Multimodal Reasoning Models
Apr 11, 2026Zhe Qian, Yanbiao Ma, Zhuohan Ouyang +7VLM Hallucination Mitigation Multimodal Reasoning
South China Agricultural University · Gaoling School of Artificial Intelligence, Renmin University of China · South China Normal University +4
Improving Medical VQA through Trajectory-Aware Process Supervision
Apr 10, 2026Halil Ibrahim Gulluk, Olivier GevaertMedical VQA Medical VLMs
Electrical Engineering, Stanford University · Stanford Center For Biomedical Informatics Research
VL-Calibration: Decoupled Confidence Calibration for Large Vision-Language Models Reasoning
Apr 10, 2026Wenyi Xiao, Xinchi Xu, Leilei GanVLM Calibration Confidence Estimation for VLMs
Zhejiang University
MedLVR: Latent Visual Reasoning for Reliable Medical Visual Question Answering
Apr 10, 2026Suyang Xi, Songtao Hu, Yuxiang Lai +4Medical VQA Medical VLMs
Department of Radiation Oncology and Winship Cancer Institute, Emory University School of Medicine, Atlanta, 30322, GA, USA · Department of Biostatistics and Bioinformatics, Emory University, Atlanta, 30322, GA, USA
Visually-Guided Policy Optimization for Multimodal Reasoning
Apr 10, 2026Zengbin Wang, Feng Xiong, Liang Lin +5VLM Reasoning RL for VLMs
AMAP, Alibaba Group · SYSU · BUPT
OpenVLThinkerV2: A Generalist Multimodal Reasoning Model for Multi-domain Visual Tasks
Apr 9, 2026Wenbo Hu, Xin Chen, Yan Gao-Tian +3Multimodal Large Language Models Multimodal Reasoning
University of California, Los Angeles (UCLA)
Faithful GRPO: Improving Visual Spatial Reasoning in Multimodal Language Models via Constrained Policy Optimization
Apr 9, 2026Sai Srinivas Kancheti, Aditya Kanade, Rohit Sinha +2Visual Spatial Reasoning Multimodal CoT Reasoning
CSE Dept., IIT Hyderabad, Hyderabad, India · Microsoft Research, Bengaluru, India

Apr 23, 2026J/K move · Enter open · S save

ProcessThinker: Enhancing Multi-modal Large Language Models Reasoning via Rollout-based Process Reward

Jingpei Wu, Xiao Han, Weixiang Shen, Boer Zhang+2

LMU Munich · Harvard University · University of Cambridge+2

Multimodal Understanding Multimodal QA Video Reasoning Multimodal CoT Reasoning Multimodal Foundation Models RL for VLMs+3

Open paper AskPDF arXiv

Visual question answering increasingly requires multi-step reasoning. Recent post-training with reinforcement learning under verifiable rewards (RLVR) and Group Relative Policy Optimization (GRPO) can improve multimodal reasoning, but most approaches rely on sparse outcome-only rewards. As a result, they struggle to tell whether an incorrect answer comes from a small mistake late in the reasoning or from an unhelpful trajectory from the start. A common solution is to train a process reward model (PRM) for step-level supervision, but this typically requires large-scale high-quality chain-of-thought annotations and additional training cost. We propose ProcessThinker, a practical post-training pipeline that provides step-level process rewards without training an explicit PRM. ProcessThinker first rewrites reasoning traces into a step-tagged format for cold-start supervised fine-tuning, then applies GRPO with a standard format reward and our rollout-based process reward. Concretely, for each intermediate step, we sample multiple continuations from that step and use the empirical success rate (final-answer verification) as the step reward. This gives dense credit assignment and encourages reasoning steps that more reliably support a correct conclusion, helping reduce inconsistent or self-contradictory progress across steps -- a key issue in logical reasoning. Across four challenging video benchmarks (Video-MMMU, MMVU, VideoMathQA, and LongVideoBench), ProcessThinker consistently improves over the baseline model Qwen3-VL-8B-Instruct