RL for VLMs

RL: Reinforcement Learning · VLM: Vision-Language Model

Overview

This field studies how reinforcement learning can improve models that interpret images or video alongside language, and sometimes take actions in visual environments. Common approaches include group-relative policy optimization, PPO, and reinforcement learning from verifiable rewards, with feedback based on answer correctness, visual evidence, intermediate reasoning, preferences, or tool use. Applications range from image and video question answering, spatial reasoning, and medical analysis to navigation and robotic manipulation. A central challenge is ensuring that better scores reflect genuine use of visual evidence rather than language shortcuts, while designing reliable rewards and efficient training that generalize beyond the training setting.

Latest papers

CardsList

507 papers39 papers in the last four weeks, down 43% on the four weeks before. 0.3% of all new papers.

  1. ZeroVLN: Learning to Navigate without Supervised Fine-Tuning of the VLM Brain

    Nov 19, 2026Zixuan Huang, Yang Zhou, Guli Zhang +8Vision-Language NavigationRL for VLMs

  2. EmbodyAgent: Post-Training Embodied VLM Agents via Distillation, SFT, and RL

    Nov 19, 2026Wei Dai, Jinkun Liu, Jiayi Chen +3VLMs for RoboticsRL for VLMs

  3. Goal-Guided Reinforcement Learning via Fine-Tuning Vision-Language Models

    Nov 19, 2026Yajie Qi, Wei Wei, Lin Li +2Goal-Conditioned RLVisual RL

  4. WEVO: World Model-Enabled VLA Optimization for Whole-Body Loco-Manipulation

    Nov 18, 2026Qianzhong Chen, Muyuan Lin, Yuyin Sun +3Humanoid Loco-ManipulationClosed-Loop Control of VLAs

  5. ViStep-RL: Stepwise Reinforcement Learning for Long-Horizon Visual Agents

    Nov 18, 2026Junyang Chen, Yao Qin, Xiao Li +4RL for VLMsLong-Horizon RL

  6. PReMind: When Personal Memory Meets Visual Hazards

    Nov 18, 2026Yeongtak Oh, Donghyeok Lee, Joonhee Oh +2Memory-Augmented VLMsVision-Language Models

  7. Outcome-Balanced PPO and Policy-Induced Flow Rectification for Flow-Based VLA Policies

    Nov 17, 2026Yuhan Wang, Ruiyang Wang, Jiaguan ZhuReinforcement LearningFlow-Based VLAs

  8. Eyes in the Update? Learning Where Rewards Land in Multimodal Reasoning

    Nov 17, 2026Lixuan He, Tianzhe Chu, Shulin Tian +3RL for VLMsCredit Assignment in RL

  9. ViRA: Toward Ultra-High-Resolution Visual Agents via Cross-Input Relative Policy Optimization

    Nov 16, 2026Huawei Lin, Jinting Liu, Tongyan Wang +9Visual Question AnsweringReasoning for VLMs

  10. Competent Autonomous Driving Through Fine Tuning Vision Language Action Models with Symbolic Traffic Rules

    Nov 16, 2026Barkin Dagda, Vinal Asodia, Iman Sharifi +2Autonomous DrivingVLMs for Autonomous Driving

  11. Stochastic Self-Distillation for Flow-Based Vision-Language-Action Models

    Nov 16, 2026Jingru Guo, Alessandra Russo, Joey Bose +1Flow-Based VLAsEfficient VLA Inference

  12. GEM-MM: Reducing VLM Hallucination with Entropy-Guided Preference Alignment

    Nov 15, 2026Yuanshan Chua, Xuejiao ZhaoVision-Language ModelsVLM Hallucination Mitigation

  13. SARGO: Agreement-Gated Reinforcement and Suppression via On-Policy Self-Guidance for Visual Hallucination Mitigation

    Nov 14, 2026Jinyu Li, Zhiming Zhang, Hao Fang +4VLM Hallucination MitigationRL for VLMs

  14. UAV-Flow++: A Unified Suite for Benchmarking and Learning Language-Guided UAV Flight

    Nov 12, 2026Donglin Yang, Xiangyu Wang, Yantong Zhong +6UAV NavigationAerial VLN

  15. Gravity and Flight: Learning to Navigate Fine-Grained Remote Sensing Scenes

    Nov 12, 2026Yuheng Zong, Minghua Wang, Xin Zhao +3Remote Sensing VLMsReasoning for VLMs

  16. DISCOVER: Online Variance-Guided Data Discovery for Budgeted Multimodal GRPO

    Nov 8, 2026Peyman Gholami, Shivam Chandhok, Saber Malekmohammadi +3Reinforcement LearningRL for VLMs

  17. Learning When to Think: Dual-Reference Offline Optimization for Adaptive VLM Reasoning

    Nov 6, 2026Hongbin Lin, Sizhe Zou, Juangui Xu +6Reasoning for VLMsVLM Reasoning

  18. Traffic-VL: Reinforced Vision-Language Models for Generalizable End-to-End Traffic Signal Control

    Nov 4, 2026Xingchen Zou, Maonan Wang, Kemou Jiang +5Traffic Signal ControlRL for VLMs

  19. CRONOS: Benchmarking Non-Episodic Reinforcement Learning for Robotic Manipulation

    Nov 3, 2026Po-Yi Wu, Djengo Cyun-Jyun Fang, Dian Cheng +1Robotic RLRobot Manipulation Benchmarks

  20. Learning Where to Look: Observation Policy Optimization for Thinking with Images

    Nov 3, 2026Junfeng Wang, Jiawei Liu, Yongchao Xu +5Visual Question AnsweringReasoning for VLMs

  21. MIRA: Reinforcing Multimodal Reasoning via Deceptive Contextual Augmentation

    Nov 2, 2026Zhihan Yin, Jianxin Liang, Yifeng Yao +5Reasoning for VLMsVLM Robustness

  22. NAS-PO: Native Attention-Strategy Policy Optimization for Vision-Language Models

    Nov 1, 2026Ke An, Shizhe Xiang, Yue Liu +5RL for VLMsAttention Intervention in VLMs

  23. Learning to Group and Order: Cross-Instance Self-Supervised RL for Vision-Centric MLLMs

    Oct 31, 2026Songming Yang, Hong-Tao Yu, Xiu-Shen WeiRL for VLMsSelf-Supervised RL

  24. Toward Counterfactual Learning for Selective Re-Examination in Agentic Visual Search

    Oct 31, 2026Ao Chen, Shengchao Chen, Yan LuximonVisual SearchRL for VLMs

  25. GRIT: Gradient Masking for Visually Misgrounded Tokens in Text-Rich Reasoning

    Oct 30, 2026Chanseul Cho, Noori Bae, Dongheon Lee +1VLM ReasoningVisually Grounded Reasoning

  26. Aligning Medical Vision Language Models at the Capability Boundary with Entropy Adaptive Rubric Grounding

    Oct 29, 2026Hongyu Bai, Jiaqi Zeng, Chunzheng ZhuMedical VLMsReasoning for VLMs