Reward Design for RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

23 papers in the last four weeks, up 64% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 167

All topics

Beyond Outcome Rewards: Constructing and Assigning Retrieval Credit for Search Agents
Oct 7, 2026Wenyu Huang, Xinyu Hou, Pavlos Vougiouklis +2Reward Design for RL RL for IR
University of Edinburgh, UK · Huawei Technologies Research & Development (UK) Limited
When Do Intrinsic Rewards Lead to Exploration?
Oct 1, 2026Scott W. Viteri, Laura Gomezjurado Gonzalez, Clark BarrettIntrinsic Reward Methods RL Exploration
Stanford University
PPO-HRAP: Proximal Policy Optimization with a Hybrid Regime-Aware Policy for Risk-Controlled Trading
Oct 1, 2026Duong Hien Chi Kien, Thanh Trung HuynhReinforcement Learning Risk-Sensitive RL
College of Engineering and Computer Science, VinUniversity Ha Noi, Viet Nam
Dependency-Aware Reward Shaping for Agentic Reinforcement Learning
Oct 1, 2026Ziyi Chen, Yan Zhang, Jianhui Wei +2RL for Language Models Agentic RL
University of Illinois Urbana-Champaign · National University of Singapore · Zhejiang University +1
CATCH: A Controllable Analysis Testbed for Reward Hacking in Coding RL
Sep 30, 2026Shouli Wang, Yanfeng Jia, Zhihao Ou +6RL Benchmarks RL for Code Generation
Tsinghua University · Beihang University · Southern University of Science and Technology +2
CAST: Causal Advantage-Structured Training with Spatially Grounded Compositional Rewards for Diffusion Models
Sep 30, 2026Shu Yu, Chaochao LuDiffusion Model Fine-Tuning Compositional T2I Generation
Shanghai Artificial Intelligence Laboratory, Shanghai, China · Shanghai Innovation Institute, Shanghai, China · Fudan University, Shanghai, China
Group-Marginalized Self-Rewarding RL Drives Zero-Label Self-Evolving
Sep 29, 2026Yiming Wang, Yikang Liu, Qingyuan Tian +4RL for Language Models Reinforcement Learning
School of Computer Science, Shanghai Jiao Tong University · Tencent
Verifier Errors in RLVR: Reward Hacking, Limits of Feedback, and Selective Control
Sep 28, 2026Christian Moya, Elliott Thornley, Guang LinReward Hacking Reinforcement Learning
Purdue University · National University of Singapore
Rubric Rewards from Item Response Theory
Sep 28, 2026Milad Yazdani, Yaser Souri, Xiren Zhou +4Reward Modeling Rubric-Based RL
Department of Electrical and Computer Engineering, University of British Columbia · Microsoft · School of Biomedical Engineering, University of British Columbia
Sufficiency of Zeroth-Order Reward Shaping for Policy Gradient in Stabilization Control
Sep 28, 2026Yisheng Zhang, Tao Wang, Sicun GaoReward Shaping Robotic RL
University of California San Diego, La Jolla, CA, USA · Tsinghua University, Beijing, China
CRISP: Cultural Reward Modeling for Implicit Situated Propriety
Sep 28, 2026Zekun Yuan, Yangfan Ye, Baohang Li +6Reward Modeling RL for Language Models
Harbin Institute of Technology · Huawei Technologies Co., Ltd · Peng Cheng Laboratory
Evolving Support Priorities in Empathetic Reinforcement Learning
Sep 28, 2026Pengyu Huang, Zhiyuan Han, Wenwen Tong +6Reward Modeling RL for Language Models
University of Science and Technology of China, Hefei, China · SenseTime Research, Shanghai, China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center
Demonstration-Free Success-Probability Reward Learning for Generalist Robot Policies
Sep 27, 2026Duo Wu, Haifeng Wang, Rongwei Lu +6Robotic RL Sparse-Reward RL
Tsinghua University · Yuanxing Robotics · Pengcheng Laboratory
Reinforcement Learning with Verifiable Rewards for Small Search Agents
Sep 23, 2026Gaurisankar Jayadas, Aske Plaat, Álvaro Serra-Gómez +1Reinforcement Learning Reinforcement Learning with Verifiable Rewards
Leiden Institute of Advanced Computer Science, Leiden University, Leiden, The Netherlands
DCRL: Decoupling and Coupling Reinforcement Learning via Policy-Reward Manifold Alignment
Sep 23, 2026Henan Sun, Zehua Li, Haitao Hu +4Reinforcement Learning RL for Language Model Reasoning
The Hong Kong University of Science and Technology (Guangzhou) · Huawei Noah’s Ark Lab · Tencent HY +1
Reinforcement Learning with Decomposed Subtasks
Sep 22, 2026Mattie Terzolo, Mikolaj Sacha, Ayan Sinha +1Group Relative Policy Optimization Credit Assignment in RL
Upwork
Sometimes You Gotta Run Before You Can Walk: Run-then-Walk Scheduling Strategy for VLM Autonomous Driving
Sep 22, 2026Yuqi Ye, Shangkun Sun, Junhong Lin +6VLMs for Autonomous Driving Group Relative Policy Optimization
Peking University · Minieye Technology Co., Ltd · Fuzhou University
RLVR $^{2}$ : Reinforcement Learning with Verifiable Rubric-based Ranking
Sep 20, 2026Hao Li, Zhengkun Zhang, Gangqiang Hu +4Learning to Rank Rubric-Based RL
ERNIE Team, Baidu Inc.
Mitigating Retaliatory Algorithmic Collusion in Repeated Games
Sep 17, 2026Karthik Sivachandran, Rohan PalejaGame-Playing Agents Repeated Games
Department of Computer Science Purdue University
Flow-Matched Motion Priors: Online Optimal-Transport Rewards for Imitation Learning
Sep 14, 2026Yilin Zou, Chenghua Liu, Chenglong Wu +1Imitation Learning RL for Legged Locomotion
School of Aerospace Engineering, Tsinghua University, Beijing 100084, China · Institute of Software, Chinese Academy of Sciences, Beijing, China
Specifying Reward Functions for RL Without Environment Sampling
Sep 14, 2026Stephane Hatgis-Kessell, W. Bradley Knox, Emma BrunskillLLM-Guided RL Reward Design for RL
1Computer Science Department, Stanford University · 2Computer Science Department, The University of Texas at Austin
EvoRS: On-Policy Self-Evolution of Reward Systems for Open-Ended Reinforcement Learning
Sep 14, 2026Weiyuan Li, Aili Chen, Xintao Wang +11Reward Hacking Reinforcement Learning
School of Data Science, Fudan University · Shanghai Key Laboratory of Data Science · College of Computer Science and Artificial Intelligence, Fudan University +3
From Connectivity to Rewards: Dense Reward Learning with Directed State Graphs
Sep 11, 2026Shuyuan Zhang, Zihan Wang, Xiao-Wen Chang +1Reinforcement Learning Hierarchical RL
School of Computer Science, McGill University Mila – Quebec AI Institute · Google DeepMind, CIFAR Fellow
Entropy-Regularized Rank-Masked Policy Optimization for Test-Time Reinforcement Learning in Code Generation
Sep 8, 2026Jiacheng Xu, Feng Chen, Xiuneng Xu +1RL for Code Generation Policy Optimization
Nanyang Technological University, Singapore
Spurious Advantage Hidden in GRPO
Sep 3, 2026Jiamian Wang, Samyadeep Basu, Koustava Goswami +2Reinforcement Learning Group Relative Policy Optimization
Rochester Institute of Technology · Adobe Research
GDB-Reward: From Evaluation Metrics to Training Rewards for Graphic Design
Sep 2, 2026Adrienne Deganutti, Purvanshi Mehta, Simon Hadfield +1Reward Design for RL Prompt Optimization for T2I Generation
Lica World · University of Surrey
Dense Process Supervision for Search Agents via Fact Utility Estimation
Sep 1, 2026Rongzhi Zhu, Xiangyu Liu, Yi Liu +7Reinforcement Learning Credit Assignment in RL
State Key Laboratory for Novel Software Technology, Nanjing University, China · Ant Group, China · National Institute of Healthcare Data Science, Nanjing University, China
What Emerges and What Breaks in Self-Play Driving
Aug 31, 2026Laur Sisask, Ardi Tampuu, Tambet MatiisenReward Hacking Self-Play RL
University of Tartu, Tartu, Estonia
Confounding Masquerading as Improvement: A Systematic Evaluation of Offline Reinforcement Learning for Stroke Antithrombotic Treatment in a 129,000-Patient Registry
Aug 31, 2026Kihun RheeOff-Policy Evaluation Offline RL
AI Research Center, JLK Inc., Seoul, South Korea
Locally-Guided Actor-Critic: Training a Goal-conditioned Actor with a Subgoal-aware Critic
Aug 31, 2026Olivier Serris, Stéphane Doncieux, Olivier SigaudReinforcement Learning Sparse-Reward RL
ISIR · Sorbonne Université, CNRS
Beyond Uncertainty: Multi-Solver Disagreement Rewards for Self-Evolving Reasoning Curricula
Aug 30, 2026Vinoth Selvendran, Zhanming ZhangReward Modeling RL for Language Model Reasoning
Independent Researcher · Palo Alto, CA, USA · New York, NY, USA
An Empirical Study of Reward Specification and Benchmark Reliability in GRPO-based LLM Unlearning
Aug 18, 2026Rubén Balbastre, Juan Manuel Orduña, Mariano PérezGroup Relative Policy Optimization Rubric-Based RL
University of Valencia
Towards Socially Compliant Navigation in Deep Reinforcement Learning via Proxemics-Based Reward Modeling
Aug 13, 2026Takieddine Soualhi, Jacques Saraydaryan, Laetitia MatignonRL for Robotics Social Robot Navigation
CHROMA · Inria, CITI Lab, INRIA–INSA Chroma Team, Villeurbanne, France. · CPE, CHROMA +3
A Framework for Designing Reward Functions: From Objectives to Features to Human-Aligned Reward Functions
Aug 12, 2026Di Yang Shi, W. Bradley KnoxReward Modeling Reward Design for RL
University of Texas at Austin
Rubric Dropout: A Simple Way to Mitigate Reward Hacking in Rubric-as-Reward RL
Aug 12, 2026Minglai Yang, Xinyu Guo, Utkarsh Tyagi +6Reward Hacking Rubric-Based RL
Scale AI · University of Arizona · University of Texas at Dallas
When Offline Evaluation Misleads: A Diagnostic Protocol for Reward and Policy Selection in Delayed-Feedback Contextual Bandits
Aug 12, 2026Sang Su Lee, Vineeth Loganathan, Shishir Dash +1Multi-Armed Bandits Off-Policy Evaluation
Thumbtack, Inc. San Francisco, CA, USA
CoRE: Consensus Rewards via Equilibrium for Test-Time Reinforcement Learning
Aug 10, 2026Ambuj Mehrish, Sebastiano VasconReward Modeling RL for Language Models
CVML Lab Ca’ Foscari University of Venice
RISE-RL: Rubric-Informed Selective Exploration for Open-Ended Reinforcement Learning
Aug 10, 2026Jinkun Hou, Zhuo Liu, Huimin Ren +3Open-Ended Generation RL for Language Models
1Peking University · 2Beijing Institute of Technology · 3Li Auto Inc.
MedCalc-R1: Knowledge-Guided Reward Framework for Medical Mathematical Reasoning
Aug 9, 2026Haotian Wang, Lian Yan, Xingzhi Yao +4Numerical Reasoning in Language Models Reinforcement Learning with Verifiable Rewards
Harbin Institute of Technology
Multi-Agent Reinforcement Learning via Agent-Specific Preference
Aug 9, 2026Ni Mu, Yao Luan, Yiqin Yang +1Decentralized MARL Cooperative MARL
CFINS, Department of Automation, BNRist, Beijing Key Laboratory of Embodied Intelligence Systems, Institute for Embodied Intelligence and Robotics, Tsinghua University, Beijing 100084, China · Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China
A Unified Framework for Dynamic Reward Shaping in Reinforcement Learning
Aug 8, 2026Fouad BahrpeymaReward Shaping Reward Design for RL
Industrial AI Group Professorship for Information Management Faculty of Informatics / Mathematics Dresden University of Applied Sciences Dresden, Germany
RRC: Unlocking Generative Reward Models in LLM Reinforcement Learning via Ranking-Based Reward Construction
Aug 6, 2026Chenglong Wang, Ziming Zhu, Yifu Huo +9Reward Modeling RL for Language Models
School of Computer Science and Engineering, Northeastern University, Shenyang, China · 2NiuTrans Research, Shenyang, China · 3Independent Researcher, Beijing, China +1
Training a Conditioned Video Game Agent on a VLM Annotated Dataset
Aug 6, 2026Katrin Schmid, Iuri FrosioReinforcement Learning Reward Design for RL
NVIDIA, Australia · NVIDIA, Italy
Reward Structure Shapes the Interaction Between Episodic Exploration and Neural Memory in Reinforcement Learning
Aug 5, 2026Jai Malegaonkar, Rohan Patil, Henrik I. ChristensenReinforcement Learning Partially Observable RL
Department of Computer Science and Engineering, UC San Diego, United States
GORDON: Graph-based Object-centric Rewards for Decomposition of Long-Horizon Manipulation
Aug 4, 2026Andrea Protopapa, Davide Buoso, Francesca Pistilli +2Long-Horizon Robotic Manipulation Learning from Demonstration
Politecnico di Torino, Turin, Italy. · Interactive Robot Perception & Learning (PEARL) Lab, Computer Science Dept., TU Darmstadt, Germany. · Hessian.AI, Darmstadt, Germany. +1
DocPO: Advancing Document Policy Optimization via Tailored Step-Aware Rewards
Aug 1, 2026Yunhao Wang, Binghong Wu, Zhenyu Huang +4Reward Shaping Document Parsing
Tencent Hunyuan · Beijing, China · Shanghai, China +1
Stabilized Best-of- $K$ Training for Neural Combinatorial Optimization
Jul 31, 2026Melveena Jolly, Midhun XavierTraveling Salesperson Problem Best-of-N Sampling
1Independent Researcher
LEMUR: Learning to Align with Multi-Objective Reinforcement Learning from Preference Feedback
Jul 31, 2026Manith Adikari, Bei Peng, Samuele Vinanzi +1Multi-Objective Reinforcement Learning Reinforcement Learning
Department of Computer Science, University of Manchester · Centre for Robotics & AI, University of Manchester · School of Computer Science, University of Sheffield +1
Beyond Binary Rewards: A Comparative Study of Reward Design for Reinforcement Unlearning
Jul 30, 2026Efstratios Zaradoukas, Davide Gabrielli, Bardh Prenkaj +1Reinforcement Learning with Verifiable Rewards Machine Unlearning
Technical University of Munich, Germany · Munich Center for Machine Learning, Germany · Sapienza University of Rome, Italy
Reinforcement Learning for Code Optimization
Jul 28, 2026Pierre Chambon, Kunhao Zheng, Juliette Decugis +2RL for Code Generation Reinforcement Learning
FAIR at Meta · Inria · Université Paris Dauphine
Meta-Learned Reward Shaping for Reinforcement Learning from Human Feedback
Jul 28, 2026Yunpeng ChuLLM Alignment Reward Shaping
Stony Brook University
Optimal Reward Shaping: Autonomous Car Parking Case Study
Jul 26, 2026Emre Özkaya, Nicolas R. GaugerDeep Q-Learning Autonomous Parking
Chair for Scientific Computing, RPTU University of Kaiserslautern-Landau Paul-Ehrlich-Strasse 34, 67663 Kaiserslautern
Search-G1: Grounded Search Agents via Representation-Based Intrinsic Rewards
Jul 24, 2026Cheng Ruoxi, Ma Haoxuan, Zhang Hongyi +7Retrieval-Augmented Generation LLM Grounding
1Fudan University · 2Tencent · 3Nanjing University +2
The Dark Room in the Reward Channel: Dense Prediction Rewards Collapse GRPO-Trained LLM Agents -- and The Channel, Not the Content, Decides What Works
Jul 23, 2026Yu WangReward Hacking Reward Shaping
Independent researcher
Multi-turn RL with Structural and Performance Aware Rewards for CUDA Kernel Generation
Jul 23, 2026Quazi Ishtiaque Mahmud, Nesreen K. Ahmed, Ali JannesariRL for Code Generation GPU Kernel Optimization
Iowa State University, Ames, IA, USA · Cisco AI Research, San Jose, CA, USA
CriPO: Enhancing Rubric-based RL via Self-Distillation
Jul 20, 2026Mingxuan Xia, Yuhang Yang, Chao Ye +7Credit Assignment in RL Rubric-Based RL
Zhejiang University · ByteDance
CORAL: Learning Amyloid Fibril Ligand Docking with Cooperative Binding Rewards
Jul 19, 2026Yasheng Sun, Bohan Li, Youqi Tao +1Reward Design for RL Drug Discovery
Center of Excellence for Generative AI, KAUST · MoE Key Lab of Artificial Intelligence, Shanghai Jiao Tong University · Stern Laboratory, Brigham and Women’s Hospital, Harvard University
Principled Direction-Free Intrinsic Motivation through Model-Free Epistemic Free-Energy Estimators
Jul 18, 2026Alireza Furutanpey, Schahram DustdarFree Energy Calculation Intrinsic Reward Methods
Coovally, Barcelona, Spain · Distributed Systems Group, TU Vienna, Austria · ICREA Barcelona, Spain
CLaC@FinMMEval 2026 Task 3: Sentiment-Augmented Deep Reinforcement Learning for Active Trading -- An Alpha-Reward Approach
Jul 17, 2026Andrei Neagu, Eeham Khan, Leila KosseimReinforcement Learning Deep Q-Learning
Department of Computer Science and Software Engineering, Concordia University, Montréal, Canada
Lyapunov Exponent as Physics-Informed Dense Reward: RL Discovery of Stabilization Beyond the Kapitza Pendulum
Jul 15, 2026Slava AndrejevRL Control Reinforcement Learning

Oct 7, 2026, cs.CLJ/K move · Enter open · S save

Beyond Outcome Rewards: Constructing and Assigning Retrieval Credit for Search Agents

Wenyu Huang, Xinyu Hou, Pavlos Vougiouklis, Ruofei Lai+1

University of Edinburgh, UK · Huawei Technologies Research & Development (UK) Limited

Reward Design for RL RL for IR Step-Level Credit Assignment in RL Reinforcement Learning with Verifiable Rewards

Open paper AskPDF arXiv

Search agents enable Large Language Models (LLMs) to iteratively retrieve and use information for complex multi-hop questions. Reinforcement Learning with Verifiable Rewards (RLVR) offers a promising approach for post-training such agents, but its reliance on sparse, outcome-based supervision can make credit assignment difficult and limit learning efficiency. In this paper, we systematically investigate how intermediate supervision can improve reinforcement learning for search agents. We study a range of reward-shaping and credit-assignment strategies that provide learning signals from intermediate retrieval steps. Building on these insights, we develop a training framework that combines intermediate signals with final outcome rewards to improve learning from multi-step search trajectories. Experiments across multiple benchmarks under matched training conditions demonstrate improvements in aggregate search-agent performance and show that both the choice of intermediate signal and where its credit is assigned affect training behaviour. These findings show that reward design and credit assignment are important design dimensions for training effective search agents.

Momentum

Related topics

Latest papers 167