RL Fine-Tuning

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

18 papers in the last four weeks, up 260% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 162

All topics

Learning Process Rewards via Success Visitation Matching for Efficient RL
Jun 22, 2026Raymond Tsao, Andrew Wagenmaker, Sergey LevineProcess Reward Models Credit Assignment in RL
UC Berkeley
FlowTTS-GRPO: Online Reinforcement Learning with Multi-Objective Reward Optimization for Flow-Matching Based Text-to-Speech
Jun 22, 2026Haoxu Wang, Biao Tian, Weiqin Li +3Flow Matching TTS Synthesis
Tongyi Lab, Alibaba Group, China
Provable Benefits of RLVR over SFT for Reasoning Models: Learning to Backtrack Efficiently
Jun 22, 2026Stanley Wei, Juno KimRL for Language Model Reasoning Efficient Language Model Reasoning
1Princeton University · University of California, Berkeley.
HiL-ResRL: A Model-Agnostic Finetuning Adapter via Human-in-the-loop Residual Reinforcement Learning
Jun 22, 2026Jingyi Liu, Zhaohong Mai, ShunSen He +5RL for Robotics Vision-Language-Action Models
1CloudRobo Lab, Huawei Cloud Computing Technologies Co., Ltd. · 2South China University of Technology.
DF-ExpEnse: Diffusion Filtered Exploration for Sample Efficient Finetuning
Jun 17, 2026Calvin Luo, Chen Sun, Shuran SongRobot Policy Adaptation Robotic RL
1Stanford University · 2Brown University.
Beyond Reward Engineering: A Data Recipe for Long-Context Reinforcement Learning
Jun 17, 2026Xiaoyue Xu, Sikui Zhang, Xiaorong Wang +2Training Data Curation RL for Language Model Reasoning
OpenBMB · Tsinghua University
Hierarchical Advantage Weighting for Online RL Fine-Tuning of VLAs from Sparse Episode Outcomes
Jun 15, 2026Tongyan Fang, Siyuan Huang, Naiyu Fang +6Credit Assignment in RL Robotic RL
1ACE Robotics · 2Shenzhen International Graduate School, Tsinghua University · 3The Chinese University of Hong Kong
Reinforcement Learning for LLM-based Event Forecasting
Jun 14, 2026Amit Arnold LevyEvent Time Prediction RL for Language Models
University of Oxford
StarOR: Synergizing Tree Search and Test-Time Reinforcement Learning for Optimization Modeling
Jun 13, 2026Jiajun Li, Yu Ding, Shisi Guan +2Reinforcement Learning Large Language Model-Guided Optimization
School of Computer Science and Engineering, Southeast University · Northwest A&F University
Architecture-Aware Reinforcement Learning Makes Sliding-Window Attention Competitive in Math Reasoning
Jun 10, 2026Kai Liu, Peijie Dong, Xinchen Xie +5RL for Language Model Reasoning LLM Inference Acceleration
Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University · Hong Kong University of Science and Technology (Guangzhou) · Shanghai AI Laboratory
GUI-AC: Enhancing Continual Learning in GUI Agents
Jun 9, 2026Can Lin, Tao Feng, Hangjie Yuan +3Non-Stationary RL RL for GUI Agents
Beijing University of Posts and Telecommunications · Tsinghua University · Zhejiang University +1
Emergence of Context Characteristics Sensitivity in Large Language Models
Jun 8, 2026Nadya Yuki Wangsajaya, Haeun Yu, Isabelle AugensteinLLM Fine-Tuning Instruction Tuning
Nanyang Technological University · University of Copenhagen
From Shortcuts to Reasoning: Robust Post-Training of Theory of Mind with Reinforcement Learning
Jun 8, 2026Jike Zhong, Yuxiang Lai, Ming Li +5Theory of Mind RL for Language Model Reasoning
University of Southern California · 2Emory University · University of Maryland, College Park +4
Reinforcement Learning for Flow-Matching Policies with Density Transport
Jun 7, 2026Boshu Lei, Kostas Daniilidis, Antonio LoquercioFlow Matching Robotic RL
University of Pennsylvania · Philadelphia, PA 19103
RREDCoT: Segment-Level Reward Redistribution for Reasoning Models
Jun 4, 2026Mykyta Ielanskyi, Kajetan Schweighofer, Lukas Aichberger +1Reinforcement Learning RL for Language Model Reasoning
ELLIS Unit Linz and LIT AI Lab, Institute for Machine Learning, Johannes Kepler University Linz, Austria · Cognizant AI Lab, San Francisco, USA · Work done while at Johannes Kepler University Linz. +1
TARPO: Token-Wise Latent-Explicit Reasoning via Action-Routing Policy Optimization
Jun 4, 2026Liting Zhang, Shiwan Zhao, Xuyang Zhao +3RL for Language Model Reasoning RL Fine-Tuning
TMCC, College of Computer Science, Nankai University, Tianjin, China
FlowPRO: Reward-Free Reinforced Fine-Tuning of Flow-Matching VLAs via Proximalized Preference Optimization
Jun 3, 2026Yihao Wu, He Zhang, Junbo Tan +2Offline RL Vision-Language-Action Models
Tencent Robotics X · Futian Laboratory · Tsinghua University
MAGNIFIED: RL Fine-tuning of Multimodal Large Language Models for Motion Planning
Jun 2, 2026Letian Chen, Yiren Lu, Justin Fu +5VLMs for Autonomous Driving Multimodal Large Language Models
Waymo LLC.
Towards Precise Intent-Aligned VLA Aerial Navigation via Expert-Guided GRPO
Jun 1, 2026Tianyang Chen, Wenjun Li, Xin zhou +2UAV Navigation Group Relative Policy Optimization
1Zhejiang University · 2Differential Robotics
Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards
Jun 1, 2026Christian Scherer, Joe Watson, Theo Gruner +3Reinforcement Learning Robot Skill Learning
Technical University of Darmstadt · Zuse School ELIZA · University of Oxford +2
DRIFT: Decoupled Rollouts and Importance-Weighted Fine-Tuning for Efficient Multi-Turn Optimization
May 29, 2026Jian Mu, Tianyi Lin, Chengwei Qin +2Supervised Fine-Tuning RL for Language Models
The Hong Kong University of Science and Technology (Guangzhou) · The Chinese University of Hong Kong, Shenzhen.
DARTS: Distribution-Aware Active Rollout Trajectory Shaping for Accelerating LLM Reinforcement Learning
May 29, 2026Yujie Wang, Siwei Chen, Longzan Luo +4RL for Language Models Reinforcement Learning
School of Computer Science & Beijing Key Laboratory of Software and Hardware Cooperative Artificial Intelligence Systems, Peking University, Beijing, China · School of Artificial Intelligence, Shanghai Jiao Tong University, Shanghai, China · Institute of Computational Social Science, Peking University (Qingdao), Qingdao, China
Your Teacher Can't Help You Here: Combating Supervision Fidelity Decay in On-Policy Distillation
May 29, 2026Yanjiang Liu, Jie Lou, Xinyan Guan +7CoT Distillation Language Model Distillation
University of Chinese Academy of Sciences · 2Chinese Information Processing Laboratory Institute of Software, Chinese Academy of Sciences 2University of Chinese Academy of Sciences, Beijing, China · 3Xiaohongshu
Soft-SVeRL: Self-Verified Reinforcement Learning with Soft Rewards
May 27, 2026Saurabh Dash, Pierre Clavier, John Dang +4Reinforcement Learning Instruction Following
Cohere Labs · Cohere · Adaption Labs +1
DenoiseRL: Bootstrapping Reasoning Models to Recover from Noisy Prefixes
May 27, 2026Caijun Xu, Changyi Xiao, Zhongyuan Peng +1LLM Self-Correction RL for Language Model Reasoning
1Fudan University · 2Shanghai Innovation Institute
GeneralThinker: Domain-General Reasoning through Likelihood-Guided Answer-Conditioned Optimization
May 27, 2026Shengmin Piao, Sanghyun ParkToken-Level Credit Assignment RL for Language Model Reasoning
Yonsei University Seoul, South Korea
Trust Region Q Adjoint Matching
May 26, 2026Yonghoon Dong, Kyungmin Lee, Changyeon Kim +2Reinforcement Learning KL-Regularized RL
1KAIST AI · 2Seoul National University · 3RLWRLD
Tournament-GRPO: Group-Wise Tournament Rewards for Reinforcement Learning in Open-Ended Long-Form Generation
May 26, 2026Zixuan Yang, Yiqun Chen, Wei Yang +7Open-Ended Generation Reward Modeling
Renmin University of China · University of Southern California · Zhejiang University +1
Focal Reward: Balanced Reinforcement Learning under Rubric-Based Rewards
May 26, 2026Yu Huang, Zihua Zhao, Zhaoxin Huan +9Multi-Objective Reinforcement Learning RL for Language Models
Shanghai Jiao Tong University · Ant Group
Trait-Aware Policy Optimization for Autoregressive Multi-Trait Essay Scoring
May 25, 2026Zhengyang Wang, Sanwoo Lee, Jiaxin Wang +3Automated Essay Scoring Language Model Post-Training
Peking University · Baidu Inc.

Jun 22, 2026, cs.LGJ/K move · Enter open · S save

Learning Process Rewards via Success Visitation Matching for Efficient RL

Raymond Tsao, Andrew Wagenmaker, Sergey Levine

UC Berkeley

Process Reward Models Credit Assignment in RL Dense Reward RL Distribution Matching Robotic RL Sparse-Reward RL+3

Open paper AskPDF arXiv

In many modern applications of reinforcement learning (RL), the natural reward for a task of interest is inherently sparse: a reward of 0 is given everywhere except when the task is completed, when a reward of +1 is given. Training a policy to maximize such a sparse reward requires solving a challenging credit assignment problem, leading to slow or ineffective RL improvement. We propose a simple approach to transform a sparse outcome reward into a dense process reward. Our approach relies on training a discriminator to distinguish between previous successful and unsuccessful episodes, and using this discriminator to incentivize the RL-learned policy to match the state-action visitations of successful episodes, while avoiding those of unsuccessful episodes. By incentivizing the policy to match the visitations over all states, not just those that correspond to task success, this reward provides dense feedback on whether progress is being made towards task completion, and, we show, provably achieves this without changing the optimal policy. Focusing on finetuning of robotic control policies, we demonstrate that our approach leads to significantly faster RL finetuning performance on both simulated and real-world manipulation tasks, as compared to simply maximizing the sparse outcome reward.

Momentum

Related topics

Latest papers 162