RL for VLMs

RL: Reinforcement Learning · VLM: Vision-Language Model

This field studies how reinforcement learning can improve models that interpret images or video alongside language, and sometimes take actions in visual environments. Common approaches include group-relative policy optimization, PPO, and reinforcement learning from verifiable rewards, with feedback based on answer correctness, visual evidence, intermediate reasoning, preferences, or tool use. Applications range from image and video question answering, spatial reasoning, and medical analysis to navigation and robotic manipulation. A central challenge is ensuring that better scores reflect genuine use of visual evidence rather than language shortcuts, while designing reliable rewards and efficient training that generalize beyond the training setting.

Latest papers

476 papers68 papers in the last four weeks, up 94% on the four weeks before. 0.4% of all new papers.

Fundus-R1: Training a Fundus-Reading MLLM on Public Data
Apr 9, 2026Yuchuan Deng, Qijie Wei, Kaiheng Qian +6Medical VLMs Medical Imaging
Renmin University of China, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China · Zhejiang Gongshang University, China
MedVR: Annotation-Free Medical Visual Reasoning via Agentic Reinforcement Learning
Apr 9, 2026Zheng Jiang, Heng Guo, Chengyu Fang +4VLM Reasoning Visual Reasoning
Tsinghua University · DAMO Academy, Alibaba Group · Hupan Lab, 310023, Hangzhou, China +2
Decompose, Look, and Reason: Reinforced Latent Reasoning for VLMs
Apr 8, 2026Mengdan Zhu, Senhao Cheng, Liang ZhaoVision-Language Grounding VLM Interpretability
Emory University · University of Michigan, Ann Arbor
Multi-Granularity Reasoning for Image Quality Assessment via Attribute-Aware Reinforcement Learning to Rank
Apr 7, 2026Xiangyong Chen, Xiaochuan Lin, Haoran Liu +3Image Quality Assessment VLM Reasoning
Henan Polytechnic University
Vero: An Open RL Recipe for General Visual Reasoning
Apr 6, 2026Gabriel Sarch, Linrong Cai, Qunzhong Wang +3Reasoning for VLMs Visual Reasoning
Princeton University
Saliency-R1: Enforcing Interpretable and Faithful Vision-language Reasoning via Saliency-map Alignment Reward
Apr 6, 2026Shizhan Gong, Minda Hu, Qiyuan Zhang +2VLM Reasoning Faithfulness in VLMs
The Chinese University of Hong Kong, Hong Kong, China · City University of Hong Kong, Hong Kong, China
Reinforce to Learn, Elect to Reason: A Dual Paradigm for Video Reasoning
Apr 6, 2026Songyuan Yang, Weijiang Yu, Jilin Ma +5Video Reasoning VLM Inference
National University of Defense Technology · Sun Yat-sen University
Understanding the Role of Hallucination in Reinforcement Post-Training of Multimodal Reasoning Models
Apr 3, 2026Gengwei Zhang, Jie Peng, Zhen Tan +6RL for VLMs Multimodal Reasoning
University of North Carolina at Chapel Hill · University of Science and Technology of China · Arizona State University +1
Chart-RL: Policy Optimization Reinforcement Learning for Enhanced Visual Reasoning in Chart Question Answering with Vision Language Models
Apr 3, 2026Yunfei Bai, Amit Dhanda, Shekhar JainChart QA VLM Reasoning
Amazon
Not All Tokens See Equally: Perception-Grounded Policy Optimization for Large Vision-Language Models
Apr 2, 2026Zekai Ye, Qiming Li, Xiaocheng Feng +6RL for VLMs Large Vision-Language Models
Harbin Institute of Technology · Peng Cheng Laboratory · Huawei Technologies Co., Ltd
STRIVE: Structured Spatiotemporal Exploration for Reinforcement Learning in Video Question Answering
Apr 2, 2026Emad Bahrami, Olga Zatsarynna, Parth Pathak +3Video QA RL for Video Reasoning
University of Bonn · Lamarr Institute for Machine Learning and Artificial Intelligence · Meta +1
Reinforcing Consistency in Video MLLMs with Structured Rewards
Apr 1, 2026Yihao Quan, Zeru Shi, Jinman Zhao +1Video-Language Models Object Hallucination in VLMs
Rutgers University · University of Toronto
All Roads Lead to Rome: Incentivizing Divergent Thinking in Vision-Language Models
Apr 1, 2026Xinyu Tian, Shu Zou, Zhaoyuan Yang +3RL for VLMs VLM Reasoning
Australian National University · Shanghai AI Lab · GE Research
Seeing with You: Perception-Reasoning Coevolution for Multimodal Reasoning
Mar 30, 2026Ziqi Miao, Haonan Jia, Lijun Li +4Multimodal Reasoning Multimodal Large Language Models
$AutoDrive\text{-}P^3$ : Unified Chain of Perception-Prediction-Planning Thought via Reinforcement Fine-Tuning
Mar 30, 2026Yuqi Ye, Zijian Zhang, Junhong Lin +3Autonomous Driving Planning VLMs for Autonomous Driving
School of Electronic and Computer Engineering, Peking University
TIR-Agent: Training an Explorative and Efficient Agent for Image Restoration
Mar 29, 2026Guoli Jia, Yisheng Zhang, Haote Hu +11Tool-Using Vision-Language Agents Image Restoration
Tsinghua University, Beijing, China · Data Science & Artificial Intelligence Research Institute, China Unicom, Beijing, China · Hunan University, Changsha, China +2
LVRPO: Language-Visual Alignment with GRPO for Multimodal Understanding and Generation
Mar 29, 2026Shentong Mo, Sukmin YunVision-Language Alignment Unified Multimodal Models
Department of Machine Learning, CMU, USA · Department of Machine Learning, MBZUAI, UAE · Department of Artificial Intelligence, Hanyang University ERICA, South Korea
Learning to Focus and Precise Cropping: A Reinforcement Learning Framework with Information Gaps and Grounding Loss for MLLMs
Mar 29, 2026Xuanpu Zhao, Zhentao Tan, Dianmo Sheng +6Visual Question Answering RL for VLMs
School of Cyber Science and Technology, University of Science and Technology of China · Anhui Province Key Laboratory of Digital Security · Independent Researcher
Difference Feedback: Generating Multimodal Process-Level Supervision for VLM Reinforcement Learning
Mar 29, 2026Feiding, Yongkang Zhang, Yuhao Liao +10RL for VLMs Reasoning for VLMs
Shenzhen International Graduate School, Tsinghua University, China
Bridging Visual Representation and Reinforcement Learning from Verifiable Rewards in Large Vision-Language Models
Mar 28, 2026Yuhang Han, Yuyang Wu, Zhengbo Jiao +6VLM Reasoning RL for VLMs
EPIC Lab, SJTU · HKUST (GZ) · Huawei
Incentivizing Temporal-Awareness in Egocentric Video Understanding Models
Mar 28, 2026Zhiyang Xu, Tian Qin, Bowen Jin +4Egocentric Video Understanding RL for VLMs
Virginia Tech · Harvard University · UIUC +2
VLA-OPD: Bridging Offline SFT and Online RL for Vision-Language-Action Models via On-Policy Distillation
Mar 27, 2026Zhide Zhong, Haodong Yan, Junfeng Li +3Vision-Language-Action Models On-Policy Distillation
HKUST (GZ)
Beyond Where to Look: Trajectory-Guided Reinforcement Learning for Multimodal RLVR
Mar 27, 2026Jinda Lu, Junkang Wu, Jinghan Li +6RL for VLMs Multimodal Reasoning
University of Science and Technology of China · Pecking University · Institute of Software, Chinese Academy of Sciences
Reinforcing Structured Chain-of-Thought for Video Understanding
Mar 26, 2026Peiyao Wang, Haotian Xu, Noranart Vesdapunt +6Structured CoT Reasoning RL for VLMs
Stony Brook University · Amazon
Pixelis: Reasoning in Pixels, from Seeing to Acting
Mar 26, 2026Yunpeng ZhouTool Use in VLMs Visual Reasoning
University of Reading, Reading, UK
Bridging Perception and Reasoning: Token Reweighting for RLVR in Multimodal LLMs
Mar 26, 2026Jinda Lu, Junkang Wu, Jinghan Li +6RL for VLMs VLM Reasoning
University of Science and Technology of China · Peking University · Independent Researcher
VideoTIR: Accurate Understanding for Long Videos with Efficient Tool-Integrated Reasoning
Mar 26, 2026Zhe Gao, Shiyu Shen, Taifeng Chai +7Tool Use in VLMs Efficient VLM Inference
Nanjing University, Nanjing, China · Nankai University, Tianjin, China · East China Normal University, Shanghai, China +3
MoE-GRPO: Optimizing Mixture-of-Experts via Reinforcement Learning in Vision-Language Models
Mar 26, 2026Dohwan Ko, Jinyoung Park, Seoung Choi +3Expert Routing Mixture-of-Experts Models
Korea University · KAIST
Learning to Select Visual In-Context Demonstrations
Mar 24, 2026Eugene Lee, Yu-Chi Lin, Jiajie DiaoIn-Context Example Selection Multimodal ICL
University of Cincinnati · University of California, Los Angeles
RoboAlign: Learning Test-Time Reasoning for Language-Action Alignment in Vision-Language-Action Models
Mar 22, 2026Dongyoung Kim, Sumin Park, Woomin Song +6Vision-Language-Action Models Reasoning for VLA Models
KAIST · RLWRLD · Yonsei University +1

Apr 9, 2026J/K move · Enter open · S save

Fundus-R1: Training a Fundus-Reading MLLM on Public Data

Yuchuan Deng, Qijie Wei, Kaiheng Qian, Jiazhen Liu+5

Renmin University of China, China · The Hong Kong University of Science and Technology, Hong Kong SAR, China · Zhejiang Gongshang University, China

Medical VLMs Medical Imaging Fundus Imaging RL for VLMs Process Reward Models

Open paper AskPDF arXiv

Fundus imaging such as CFP, OCT and UWF is crucial for the early detection of retinal anomalies and diseases. Fundus image understanding, due to its knowledge-intensive nature, poses a challenging vision-language task. An emerging approach to addressing the task is to post-train a generic multimodal large language model (MLLM), either by supervised finetuning (SFT) or by reinforcement learning with verifiable rewards (RLVR), on a considerable amount of in-house samples paired with high-quality clinical reports. However, these valuable samples are not publicly accessible, which not only hinders reproducibility but also practically limits research to few players. To overcome the barrier, we make a novel attempt to train a reasoning-enhanced fundus-reading MLLM, which we term Fundus-R1, using exclusively public datasets, wherein over 94% of the data are annotated with only image-level labels. Our technical contributions are two-fold. First, we propose a RAG-based method for composing image-specific, knowledge-aware reasoning traces. Such auto-generated traces link visual findings identified by a generic MLLM to the image labels in terms of ophthalmic knowledge. Second, we enhance RLVR with a process reward that encourages self-consistency of the generated reasoning trace in each rollout. Extensive experiments on three fundus-reading benchmarks, i.e., FunBench, Omni-Fundus and GMAI-Fundus, show that Fundus-R1 clearly outperforms multiple baselines, including its generic counterpart (Qwen2.5-VL) and a stronger edition post-trained without using the generated traces. This work paves the way for training powerful fundus-reading MLLMs with publicly available data.