Reward Design for RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

23 papers in the last four weeks, up 64% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 167

All topics

Beyond Outcome Rewards: Constructing and Assigning Retrieval Credit for Search Agents
Oct 7, 2026Wenyu Huang, Xinyu Hou, Pavlos Vougiouklis +2Reward Design for RL RL for IR
University of Edinburgh, UK · Huawei Technologies Research & Development (UK) Limited
When Do Intrinsic Rewards Lead to Exploration?
Oct 1, 2026Scott W. Viteri, Laura Gomezjurado Gonzalez, Clark BarrettIntrinsic Reward Methods RL Exploration
Stanford University
PPO-HRAP: Proximal Policy Optimization with a Hybrid Regime-Aware Policy for Risk-Controlled Trading
Oct 1, 2026Duong Hien Chi Kien, Thanh Trung HuynhReinforcement Learning Risk-Sensitive RL
College of Engineering and Computer Science, VinUniversity Ha Noi, Viet Nam
Dependency-Aware Reward Shaping for Agentic Reinforcement Learning
Oct 1, 2026Ziyi Chen, Yan Zhang, Jianhui Wei +2RL for Language Models Agentic RL
University of Illinois Urbana-Champaign · National University of Singapore · Zhejiang University +1
CATCH: A Controllable Analysis Testbed for Reward Hacking in Coding RL
Sep 30, 2026Shouli Wang, Yanfeng Jia, Zhihao Ou +6RL Benchmarks RL for Code Generation
Tsinghua University · Beihang University · Southern University of Science and Technology +2
CAST: Causal Advantage-Structured Training with Spatially Grounded Compositional Rewards for Diffusion Models
Sep 30, 2026Shu Yu, Chaochao LuDiffusion Model Fine-Tuning Compositional T2I Generation
Shanghai Artificial Intelligence Laboratory, Shanghai, China · Shanghai Innovation Institute, Shanghai, China · Fudan University, Shanghai, China
Group-Marginalized Self-Rewarding RL Drives Zero-Label Self-Evolving
Sep 29, 2026Yiming Wang, Yikang Liu, Qingyuan Tian +4RL for Language Models Reinforcement Learning
School of Computer Science, Shanghai Jiao Tong University · Tencent
Verifier Errors in RLVR: Reward Hacking, Limits of Feedback, and Selective Control
Sep 28, 2026Christian Moya, Elliott Thornley, Guang LinReward Hacking Reinforcement Learning
Purdue University · National University of Singapore
Rubric Rewards from Item Response Theory
Sep 28, 2026Milad Yazdani, Yaser Souri, Xiren Zhou +4Reward Modeling Rubric-Based RL
Department of Electrical and Computer Engineering, University of British Columbia · Microsoft · School of Biomedical Engineering, University of British Columbia
Sufficiency of Zeroth-Order Reward Shaping for Policy Gradient in Stabilization Control
Sep 28, 2026Yisheng Zhang, Tao Wang, Sicun GaoReward Shaping Robotic RL
University of California San Diego, La Jolla, CA, USA · Tsinghua University, Beijing, China
CRISP: Cultural Reward Modeling for Implicit Situated Propriety
Sep 28, 2026Zekun Yuan, Yangfan Ye, Baohang Li +6Reward Modeling RL for Language Models
Harbin Institute of Technology · Huawei Technologies Co., Ltd · Peng Cheng Laboratory
Evolving Support Priorities in Empathetic Reinforcement Learning
Sep 28, 2026Pengyu Huang, Zhiyuan Han, Wenwen Tong +6Reward Modeling RL for Language Models
University of Science and Technology of China, Hefei, China · SenseTime Research, Shanghai, China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
Demonstration-Free Success-Probability Reward Learning for Generalist Robot Policies
Sep 27, 2026Duo Wu, Haifeng Wang, Rongwei Lu +6Robotic RL Sparse-Reward RL
Tsinghua University · Yuanxing Robotics · Pengcheng Laboratory
Reinforcement Learning with Verifiable Rewards for Small Search Agents
Sep 23, 2026Gaurisankar Jayadas, Aske Plaat, Álvaro Serra-Gómez +1Reinforcement Learning Reinforcement Learning with Verifiable Rewards
Leiden Institute of Advanced Computer Science, Leiden University, Leiden, The Netherlands
DCRL: Decoupling and Coupling Reinforcement Learning via Policy-Reward Manifold Alignment
Sep 23, 2026Henan Sun, Zehua Li, Haitao Hu +4Reinforcement Learning RL for Language Model Reasoning
The Hong Kong University of Science and Technology (Guangzhou) · Huawei Noah’s Ark Lab · Tencent HY +1
Reinforcement Learning with Decomposed Subtasks
Sep 22, 2026Mattie Terzolo, Mikolaj Sacha, Ayan Sinha +1Group Relative Policy Optimization Credit Assignment in RL
Upwork
Sometimes You Gotta Run Before You Can Walk: Run-then-Walk Scheduling Strategy for VLM Autonomous Driving
Sep 22, 2026Yuqi Ye, Shangkun Sun, Junhong Lin +6VLMs for Autonomous Driving Group Relative Policy Optimization
Peking University · Minieye Technology Co., Ltd · Fuzhou University
RLVR $^{2}$ : Reinforcement Learning with Verifiable Rubric-based Ranking
Sep 20, 2026Hao Li, Zhengkun Zhang, Gangqiang Hu +4Learning to Rank Rubric-Based RL
ERNIE Team, Baidu Inc.
Mitigating Retaliatory Algorithmic Collusion in Repeated Games
Sep 17, 2026Karthik Sivachandran, Rohan PalejaGame-Playing Agents Repeated Games
Department of Computer Science Purdue University
Flow-Matched Motion Priors: Online Optimal-Transport Rewards for Imitation Learning
Sep 14, 2026Yilin Zou, Chenghua Liu, Chenglong Wu +1Imitation Learning RL for Legged Locomotion
School of Aerospace Engineering, Tsinghua University, Beijing 100084, China · Institute of Software, Chinese Academy of Sciences, Beijing, China
Specifying Reward Functions for RL Without Environment Sampling
Sep 14, 2026Stephane Hatgis-Kessell, W. Bradley Knox, Emma BrunskillLLM-Guided RL Reward Design for RL
1Computer Science Department, Stanford University · 2Computer Science Department, The University of Texas at Austin
EvoRS: On-Policy Self-Evolution of Reward Systems for Open-Ended Reinforcement Learning
Sep 14, 2026Weiyuan Li, Aili Chen, Xintao Wang +11Reward Hacking Reinforcement Learning
School of Data Science, Fudan University · Shanghai Key Laboratory of Data Science · College of Computer Science and Artificial Intelligence, Fudan University +3
From Connectivity to Rewards: Dense Reward Learning with Directed State Graphs
Sep 11, 2026Shuyuan Zhang, Zihan Wang, Xiao-Wen Chang +1Reinforcement Learning Hierarchical RL
School of Computer Science, McGill University Mila – Quebec AI Institute · Google DeepMind, CIFAR Fellow
Entropy-Regularized Rank-Masked Policy Optimization for Test-Time Reinforcement Learning in Code Generation
Sep 8, 2026Jiacheng Xu, Feng Chen, Xiuneng Xu +1RL for Code Generation Policy Optimization
Nanyang Technological University, Singapore
Spurious Advantage Hidden in GRPO
Sep 3, 2026Jiamian Wang, Samyadeep Basu, Koustava Goswami +2Reinforcement Learning Group Relative Policy Optimization
Rochester Institute of Technology · Adobe Research
GDB-Reward: From Evaluation Metrics to Training Rewards for Graphic Design
Sep 2, 2026Adrienne Deganutti, Purvanshi Mehta, Simon Hadfield +1Reward Design for RL Prompt Optimization for T2I Generation
Lica World · University of Surrey
Dense Process Supervision for Search Agents via Fact Utility Estimation
Sep 1, 2026Rongzhi Zhu, Xiangyu Liu, Yi Liu +7Reinforcement Learning Credit Assignment in RL
State Key Laboratory for Novel Software Technology, Nanjing University, China · Ant Group, China · National Institute of Healthcare Data Science, Nanjing University, China
What Emerges and What Breaks in Self-Play Driving
Aug 31, 2026Laur Sisask, Ardi Tampuu, Tambet MatiisenReward Hacking Self-Play RL
University of Tartu, Tartu, Estonia
Confounding Masquerading as Improvement: A Systematic Evaluation of Offline Reinforcement Learning for Stroke Antithrombotic Treatment in a 129,000-Patient Registry
Aug 31, 2026Kihun RheeOff-Policy Evaluation Offline RL
AI Research Center, JLK Inc., Seoul, South Korea
Locally-Guided Actor-Critic: Training a Goal-conditioned Actor with a Subgoal-aware Critic
Aug 31, 2026Olivier Serris, Stéphane Doncieux, Olivier SigaudReinforcement Learning Sparse-Reward RL
ISIR · Sorbonne Université, CNRS

Oct 7, 2026, cs.CLJ/K move · Enter open · S save

Beyond Outcome Rewards: Constructing and Assigning Retrieval Credit for Search Agents

Wenyu Huang, Xinyu Hou, Pavlos Vougiouklis, Ruofei Lai+1

University of Edinburgh, UK · Huawei Technologies Research & Development (UK) Limited

Reward Design for RL RL for IR Step-Level Credit Assignment in RL Reinforcement Learning with Verifiable Rewards

Open paper AskPDF arXiv

Search agents enable Large Language Models (LLMs) to iteratively retrieve and use information for complex multi-hop questions. Reinforcement Learning with Verifiable Rewards (RLVR) offers a promising approach for post-training such agents, but its reliance on sparse, outcome-based supervision can make credit assignment difficult and limit learning efficiency. In this paper, we systematically investigate how intermediate supervision can improve reinforcement learning for search agents. We study a range of reward-shaping and credit-assignment strategies that provide learning signals from intermediate retrieval steps. Building on these insights, we develop a training framework that combines intermediate signals with final outcome rewards to improve learning from multi-step search trajectories. Experiments across multiple benchmarks under matched training conditions demonstrate improvements in aggregate search-agent performance and show that both the choice of intermediate signal and where its credit is assigned affect training behaviour. These findings show that reward design and credit assignment are important design dimensions for training effective search agents.

Momentum

Related topics

Latest papers 167