RL for VLMs

RL: Reinforcement Learning · VLM: Vision-Language Model

This field studies how reinforcement learning can improve models that interpret images or video alongside language, and sometimes take actions in visual environments. Common approaches include group-relative policy optimization, PPO, and reinforcement learning from verifiable rewards, with feedback based on answer correctness, visual evidence, intermediate reasoning, preferences, or tool use. Applications range from image and video question answering, spatial reasoning, and medical analysis to navigation and robotic manipulation. A central challenge is ensuring that better scores reflect genuine use of visual evidence rather than language shortcuts, while designing reliable rewards and efficient training that generalize beyond the training setting.

Latest papers

476 papers68 papers in the last four weeks, up 94% on the four weeks before. 0.4% of all new papers.

GRIP-VLM: Group-Relative Importance Pruning for Efficient Vision-Language Models
May 13, 2026Mingzhe Huang, Weijun Wang, Xin Ding +7Efficient VLM Inference Visual Token Pruning
Institute for AI Industry Research (AIR), Tsinghua University · University of Science and Technology of China · Nanjing University +1
D-VLA: A High-Concurrency Distributed Asynchronous Reinforcement Learning Framework for Vision-Language-Action Models
May 13, 2026Yucheng Guo, Yongjian Guo, Zhong Guan +9Reinforcement Learning Communication-Efficient Distributed Training
JDT AI Infra · Tsinghua University · Tianjin University +2
Reinforcing VLAs in Task-Agnostic World Models
May 12, 2026Yucen Wang, Rui Yu, Fengming Zhang +5World Model Learning World Models
Nanjing University · Interns at Microsoft · University of Illinois at Urbana-Champaign +4
Uni-Synergy: Bridging Understanding and Generation for Personalized Reasoning via Co-operative Reinforcement Learning
May 11, 2026Zijun Shen, Sihan Yang, Ruichuan An +5Unified Multimodal Models Multimodal Reasoning
1Peking University · 3CUHK · 4Zhongguancun Academy
Reflection Anchors for Propagation-Aware Visual Retention in Long-Chain Multimodal Reasoning
May 10, 2026Xuan Gong, Hanbo Huang, Hao Zheng +4Vision-Language Models Vision-Language Grounding
Shanghai Jiao Tong University · Lanzhou University
Perception Without Engagement: Dissecting the Causal Discovery Deficit in LMMs
May 10, 2026Jiafeng Liang, Zhihao Zhu, Zihan Zhang +7Modality Reliance in Multimodal Models Causal Discovery
Harbin Institute of Technology · Harvard University · Peking University +2
DeltaRubric: Generative Multimodal Reward Modeling via Joint Planning and Verification
May 10, 2026Rui Liu, Dian Yu, Zhenwen Liang +6Multimodal Understanding Multimodal Reward Modeling
1Tencent Hunyuan · University of Maryland, College Park · University of North Carolina, Chapel Hill
SeePhys Pro: Diagnosing Modality Transfer and Blind-Training Effects in Multimodal RLVR for Physics Reasoning
May 10, 2026Kun Xiang, Terry Jingchen Zhang, Zirong Liu +15Cross-Modal Learning Multimodal Grounding
Sun Yat-sen University · ETH Zurich · Huawei Technologies Ltd. +1
Reinforcing Multimodal Reasoning Against Visual Degradation
May 10, 2026Rui Liu, Dian Yu, Haolin Liu +6VLM Robustness Image Corruption Robustness
Tencent Hunyuan · University of Maryland, College Park · University of Virginia +1
BalCapRL: A Balanced Framework for RL-Based MLLM Image Captioning
May 8, 2026Shaokai Ye, Vasileios Saveris, Yihao Qian +3Multi-Objective Reinforcement Learning Image Captioning
Apple
RCoT-Seg: Reinforced Chain-of-Thought for Video Reasoning and Segmentation
May 8, 2026Junwei Wen, Deshui Miao, Guangming Lu +2Video Object Segmentation Video Reasoning
Harbin Institute of Technology, Shenzhen · Pengcheng Laboratory · Pazhou Lab (Huangpu)
Structured Role-Aware Policy Optimization for Multimodal Reasoning
May 8, 2026Bingqing Jiang, Difan ZouToken-Level Credit Assignment Multimodal Reasoning
School of Computing & Data Science, The University of Hong Kong · School of Computing & Data Science and Institute of Data Science, The University of Hong Kong
Pest-Thinker: Learning to Think and Reason like Entomologists via Reinforcement Learning
May 7, 2026Xueheng Li, Yu Wang, Tao Hu +6Agricultural Image Analysis VLM Reasoning
Institute of Intelligent Machines, Hefei Institute of Physical Science, Chinese Academy of Sciences · University of Science and Technology of China · 4Hefei University of Technology +1
VISD: Enhancing Video Reasoning via Structured Self-Distillation
May 7, 2026Hao Lin, Kunyang Lv, Xu Jiang +5Video-Language Models Video Reasoning
1HUST · 2Wuhan University · 3Peking University +1
Reward-Decomposed Reinforcement Learning for Immersive Video Role-Playing
May 6, 2026Miao Wang, Yuling Shi, Yijiang Li +8Grounded Text Generation Large Language Model-Based Role-Play Simulation
1Nanjing University · 2Shanghai Jiao Tong University · University of California, San Diego +3
What You Think is What You See: Driving Exploration in VLM Agents via Visual-Linguistic Curiosity
May 5, 2026Haoxi Li, Qinglin Hou, Jianfei Ma +6Intrinsic Motivation RL Exploration
Department of Computer Science and Engineering, Hong Kong University of Science and Technology (HKUST), Hong Kong, China. · Department of Computer Science, University of Southern California, Los Angeles, USA · School of Computing, National University of Singapore, Singapore. +1
Replacing Parameters with Preferences: Federated Alignment of Heterogeneous Vision-Language Models
May 5, 2026Shule Lu, Yujing Wang, Hainan Zhang +5Reward Modeling Preference Optimization
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing · Institute of Artificial Intelligence, Beihang University, China · MAIS, Institute of Automation, Chinese Academy of Sciences, China +2
GRPO-TTA: Test-Time Visual Tuning for Vision-Language Models via GRPO-Driven Reinforcement Learning
May 5, 2026Yujun Li, Hongyuan Zhang, Yuan YuanVLM Adaptation Test-Time Adaptation of VLMs
School of Artificial Intelligence, Optics and Electronics (iOPEN), Northwestern Polytechnical University, Xi’an 710072, P.R. China · The University of Hong Kong
Perceptual Flow Network for Visually Grounded Reasoning
May 4, 2026Yangfu Li, Yuning Gong, Hongjian Zhan +8Object Hallucination in VLMs VLM Reasoning
1ECNU · 5Ant Group · 2SCU +3
Chart-FR1: Visual Focus-Driven Fine-Grained Reasoning on Dense Charts
May 3, 2026Hongkun Pan, Yuwei Wu, Wanyi Hong +8Data Visualization Multimodal CoT Reasoning
Zhejiang University · State Key Lab of CAD&CG, Zhejiang University · HiThink Research
MIRL: Mutual Information-Guided Reinforcement Learning for Vision-Language Models
May 2, 2026Yin Zhang, Jiaxuan Zhao, Zonghan Wu +5VLM Reasoning Vision-Language Models
School of Mathematics, Tianjin University, Tianjin, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · Shanghai Advanced Institute of Finance (SAIFS), East China Normal University, Shanghai, China +4
Beyond Perceptual Shortcuts: Causal-Inspired Debiasing Optimization for Generalizable Video Reasoning in Lightweight MLLMs
May 2, 2026Jingze Wu, Quan Zhang, Hongfei Suo +2VLM Robustness Video Reasoning
Sun Yat-sen University, China
Odysseus: Scaling VLMs to 100+ Turn Decision-Making in Games via Reinforcement Learning
May 1, 2026Chengshuai Shi, Wenzhe Li, Xinran Liang +10Sequential Decision Making Proximal Policy Optimization
Princeton Language and Intelligence, Princeton University · Fudan University · Tsinghua University
LaST-R1: Reinforcing Robotic Manipulation via Adaptive Physical Latent Reasoning
Apr 30, 2026Hao Chen, Jiaming Liu, Zhonghao Yan +10RL for Robotics Robot Foundation Models
1The Chinese University of Hong Kong · 2State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · 3Simplexity Robotics
Beyond SFT-to-RL: Pre-alignment via Black-Box On-Policy Distillation for Multimodal RL
Apr 30, 2026Sudong Wang, Weiquan Huang, Xiaomin Yu +9Multimodal Large Language Models Multimodal Reasoning
Hong Kong University of Science and Technology (Guangzhou) · Nanyang Technological University · Tsinghua University +3
WaferSAGE: Large Language Model-Powered Wafer Defect Analysis via Synthetic Data Generation and Rubric-Guided Reinforcement Learning
Apr 30, 2026Ke Xu, Zhongyuan LianVLM Evaluation Visual Question Answering
Shanghai Huahong Grace Semiconductor Manufacturing Corporation, Shanghai, 201203, China · Dept. of Automation, School of Information Science and Engineering, East China University of Science and Technology, Shanghai, 200237, China
Improving Vision-language Models with Perception-centric Process Reward Models
Apr 27, 2026Yingqian Min, Kun Zhou, Yifan Li +6Process Reward Models Object Hallucination in VLMs
Gaoling School of Artificial Intelligence, Renmin University of China. · Bytedance. · University of California, San Diego. +1
See Further, Think Deeper: Advancing VLM's Reasoning Ability with Low-level Visual Cues and Reflection
Apr 27, 2026Zhiheng Wu, Tong Wang, Shuning Wang +2Tool Use in VLMs VLM Reasoning
1Baidu Inc. · †Independent researcher. · 2Zhejiang University +1
Omni-o3: Deep Nested Omnimodal Deduction for Deliberative Audio-Visual Reasoning
Apr 27, 2026Zhicheng Zhang, Wentao Gu, Weicheng Wang +5Audio-Visual Understanding Tree Search
Nankai University · Kuaishou Technology · Pengcheng Laboratory +1
SOLAR-RL: Semi-Online Long-horizon Assignment Reinforcement Learning
Apr 24, 2026Jichao Wang, Liuyang Bian, Yufeng Zhou +9RL for GUI Agents Credit Assignment in RL
1vivo AI Lab · 2Zhejiang Lab · 5Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences +2

May 13, 2026J/K move · Enter open · S save

GRIP-VLM: Group-Relative Importance Pruning for Efficient Vision-Language Models

Mingzhe Huang, Weijun Wang, Xin Ding, Liang Mi+6

Institute for AI Industry Research (AIR), Tsinghua University · University of Science and Technology of China · Nanjing University+1

Efficient VLM Inference Visual Token Pruning RL for VLMs

Open paper AskPDF arXiv

In Vision-Language Models (VLMs), processing a massive number of visual tokens incurs prohibitive computational overhead. While recent training-aware pruning methods attempt to selectively discard redundant tokens, they largely rely on continuous-gradient relaxations. However, visual token pruning is inherently a discrete, non-convex combinatorial problem; consequently, these continuous approximations frequently trap the optimization in sub-optimal local minima, especially under aggressive compression budgets. To overcome this fundamental bottleneck, we propose GRIP-VLM, a Group-Relative Importance Pruning framework driven by Reinforcement Learning. Rather than relying on smooth-gradient assumptions, GRIP-VLM formulates pruning as a Markov Decision Process, employing a Group Relative Policy Optimization (GRPO) paradigm anchored by supervised warm-up to directly explore the discrete selection space. Integrated with a budget-aware scorer, our lightweight agent dynamically evaluates per-token importance and adapts to arbitrary compression ratios without retraining. Extensive experiments across diverse multimodal benchmarks demonstrate that GRIP-VLM consistently outperforms heuristic and supervised-learning baselines, achieving a superior Pareto frontier and delivering up to a 15% inference speedup at equal accuracy.