Reward Design for RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

23 papers in the last four weeks, up 64% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 167

All topics

Learning Safe Agent Behaviour from Human Preferences and Justifications via World Models
Jul 14, 2026Ilias Kazantzidis, Timothy J. Norman, Yali Du +1Safe RL Reward Design for RL
University of Southampton, Southampton SO17 1BJ, United Kingdom · King’s College London, London WC2R 2LS, United Kingdom
SCOPE-RL: Optimizing Reasoning Paths Before and After Success
Jul 13, 2026Xiaojian Liu, Han Xu, Jianqiang Xia +6Reward Shaping RL for Language Model Reasoning
Baidu Inc. · Shandong University
STAMP: Provenance-Guided Credit Assignment for Deep Search Agents
Jul 13, 2026Ke Xu, Han Xu, Xinran Chen +6Inference-Time Search Step-Level Credit Assignment in RL
Baidu Inc. · Peking University
Calibration-First Reward-Component Auditing for Reinforcement Learning Control in Smart Greenhouses
Jul 12, 2026Yuhui Bie, Guowei Xu, Yaojun WangRL Control Reinforcement Learning
aCollege of Information and Electrical Engineering, China Agricultural University, Beijing, China
Reinforcement Learning with Verifiable Physics: Post-training LLMs with Continuous Rewards
Jul 11, 2026Pengfei Cai, Utkarsh Utkarsh, Alan Edelman +2RL for Code Generation Scientific Code Generation
Massachusetts Institute of Technology
RLVP: Penalize the Path, Reward the Outcome
Jul 8, 2026Bojie Li, Noah ShiAgentic RL Constrained RL
Pine AI · University of Washington
A Gold-Standard Study of What Makes a Lightweight Game-Playing Agent Strong
Jul 7, 2026Nima Kelidari, Mohammadsaeed Haghi, Mahdi SalmaniReinforcement Learning Game-Playing Agents
University of Southern California
Improving LLM-Generated Process Model Quality Through Reinforcement Learning: The Role of Reward Function Design
Jul 7, 2026Alexander Rombach, Chantale Lauer, Nijat MehdiyevRL for Language Models Multi-Objective Language Model Alignment
German Research Center for Artificial Intelligence (DFKI), Campus D3 2, Saarbrücken, 66123, Saarland, Germany · Saarland University, Campus D3 2, Saarbrücken, 66123, Saarland, Germany
HiFuzz: Hierarchical Reinforcement Learning for Semantic-Aware and Adaptive CPU Fuzzing
Jul 7, 2026Ya Wang, Hanwei Fan, Zhenguo Liu +4Hierarchical RL Reward Design for RL
Hong Kong University of Science and Technology (HKUST), Hong Kong · The Hong Kong University of Science and Technology (Guangzhou), China
RSPO: Reward-Swap Policy Optimization for Multi-Turn LLM Agents
Jul 6, 2026Qiang Liu, Taian Guo, Ruizhi Qiao +1Reinforcement Learning Sparse-Reward RL
Tencent YouTu Lab
Integrated Altruistic and Fairness Preference Induces Advanced Mutual Cooperation in Sequential Social Dilemmas
Jul 6, 2026Yu Wei, Yukiko Ogura, Yoshiyuki Ohmura +3Social Dilemmas Cooperative MARL
The University of Tokyo
On the effectiveness of reward functions in reinforcement learning for confidence calibration of large language models
Jul 5, 2026Chee Heng Tan, Zhuoyi Lin, Mehul Motani +1Reward Hacking Confidence Estimation in Language Models
School of Computing National University of Singapore · Institute of Advanced Intelligence and Computing Agency for Science, Technology and Research · Department of Electrical & Computer Engineering National University of Singapore
SPLC: Social Preference Learning for Crowd Robot Navigation
Jul 2, 2026Zixuan Chen, Hao Fu, Haiwen Hu +1Robot Navigation Social Robot Navigation
Wuhan University of Science and Technology, Wuhan, China · School of Computer Science and Technology, Wuhan University of Science and Technology and also with the Hubei Province Key Laboratory of Intelligent Information Processing and Real-time Industrial System, Wuhan 430081, China
Rank-Then-Act: Reward-Free Control from Frame-Order Progress
Jul 2, 2026Yuriy Maksyuta, George Bredis, Ruslan Rakhimov +1RL Control Reward Modeling
T-Tech
CoRe: Combined Rewards with Vision-Language Model Feedback for Preference-Aligned Reinforcement Learning
Jul 2, 2026Hexian Ni, Tao Lu, Yinghao CaiRobotic RL Multimodal Reward Modeling
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China.
Simulation Based Reward Function Validation for Multi-Agent On Orbit Inspection
Jul 1, 2026Patrick Quinn, Bala Prenith Reddy Gopu, George M. Nehma +1Robotic Inspection Cooperative MARL
Florida Institute of Technology, Melbourne, FL., 32901
VLM-AR3L: Vision-Language Models for Absolute and Relative Rewards in Reinforcement Learning
Jul 1, 2026Kuan-Chen Chen, Winston Chen, Wei-Fang Sun +1Multimodal Reward Modeling Reward Design for RL
Department of Computer Science, National Tsing Hua University · NVIDIA AI Technology Center (NVAITC)
Learning Gait-Aware Quadruped Locomotion with Temporal Logic Specifications
Jul 1, 2026Merve Atasever, Cagan Bakirci, Alfredo Reina Corona +2Reinforcement Learning Robot Skill Learning
University of Southern California, Los Angeles, California, USA
Verifiable Rewards for Calibrated Probabilistic Forecasting
Jun 30, 2026Sadanand Singh, Allam Reddy, Manan ChopraProbability Calibration RL for Language Model Reasoning
Cascade Research
TRIAGE: Role-Typed Credit Assignment for Agentic Reinforcement Learning
Jun 30, 2026Yuanda Xu, Zhengze Zhou, Hejian Sang +6Agentic RL Step-Level Credit Assignment in RL
1LinkedIn Corporation · 2Harvard University · 3Johns Hopkins University +1
Stage-Transition Dense Reward Modeling for Reinforcement Learning
Jun 30, 2026Yang Yang, Bingjie Chen, Zihan Wang +4RL for Robotics Long-Horizon Robotic Manipulation
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China. · Zerith Robotics, Shenzhen 518055, China.
A3M: Adaptive, Adversarial and Multi-Objective Learning for Strategic Bidding in Repeated Auctions
Jun 27, 2026Junhan Li, Yuxin Zhang, Haoran Wang +1Multi-Objective Reinforcement Learning Game Theory
Department of Computer Science Nanjing University
WARP-RM: A Warp-Augmented Relative Progress Reward Model for Data Curation
Jun 26, 2026Justin Yu, Andrew Goldberg, Kavish Kondap +7Long-Horizon Robotic Manipulation Robot Skill Learning
University of California, Berkeley · Stanford University · XDOF
Reinforcement Learning without Ground-Truth Solutions can Improve LLMs
Jun 25, 2026Yingyu Lin, Qiyue Gao, Nikki Lijing Kuang +6RL for Code Generation Reinforcement Learning
University of California, San Diego · Snowflake AI Research
Designing Reward Signals for Portable Query Generation: A Case Study in Industrial Semantic Job Search
Jun 25, 2026Ping Liu, Qianqi Shen, Jianqiang Shen +11Reward Hacking Reward Shaping
LinkedIn Corporation, Mountain View, CA, USA
Automating Potential-based Reward Shaping with Vision Language Model Guidance
Jun 25, 2026Henrik Müller, Daniel KudenkoReward Shaping Sparse-Reward RL
L3S Research Center Leibniz University Hannover Hannover, 30167, Germany
The Verification Horizon: No Silver Bullet for Coding Agent Rewards
Jun 24, 2026Binghai Wang, Chenlong Zhang, Dayiheng Liu +10Reward Modeling Reward Hacking
Qwen Team
MiniOpt: Reasoning to Model and Solve General Optimization Problems with Limited Resources
Jun 24, 2026Ke Zhao, Zixiang Di, Hong Qian +9Large Language Model-Guided Optimization RL for Language Model Reasoning
East China Normal University, Shanghai 200062, China · AntGroup, Hangzhou 310000, China · Southern University of Science and Technology, Shenzhen 518055, China +1
MAPL: Multi-Objective Preference Learning for Robot Locomotion
Jun 24, 2026Xiyue Chen, Muhan Lin, Shuyang Shi +1Multi-Objective Reinforcement Learning Reward Modeling
Department of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47906, USA · Department of Computer Science, Purdue University, West Lafayette, IN 47906, USA · independent researcher
Reward-Conditioned Attention: How Reward Design Shapes What Autonomous Driving Agents See
Jun 23, 2026Mohamed Benabdelouahad, Ahmed Djalal Hacini, Nadir Farhi +1Reward Shaping RL for Autonomous Driving
National School of Artificial Intelligence (ENSIA), Sidi Abdellah Campus, Algiers, Algeria. · Cosys-Grettia, Univ Gustave Eiffel, F-77454 Marne-la-Vallee, France.

Jul 14, 2026, cs.AIJ/K move · Enter open · S save

Learning Safe Agent Behaviour from Human Preferences and Justifications via World Models

Ilias Kazantzidis, Timothy J. Norman, Yali Du, Christopher T. Freeman

University of Southampton, Southampton SO17 1BJ, United Kingdom · King’s College London, London WC2R 2LS, United Kingdom

Safe RL Reward Design for RL Model-Based RL Preference-Based RL

Open paper AskPDF arXiv

We address the problem of safely training an agent policy and deploying a good and safe policy, in settings where the environment dynamics are unknown and no suitable reward function is available. In the context of safety-critical environments, we consider traditional reinforcement learning impractical and resort to the resource of human input. We introduce DROPJ, a human-centred method for both safe training and deployment. We first learn a world model (a learned simulator) from a dataset of prior real-world trajectories. A human then plays the game in this learned simulator to extract several informative simulated trajectories. From these, we sample pairs of simulated trajectory segments and elicit from a human their preference over these segments, as well as a reason (justification) for their choice. We then train a reward model from these justified preferences and use it, together with the world model, to directly deploy the agent using model predictive control. Running real-user experiments, we find that generating informative simulated trajectories from a user significantly reduces the computational cost during training compared to other strategies, and can also improve the performance during deployment. In the context of training within a learned simulator, we show that the use of preferences rather than other types of feedback substantially improves the performance during deployment. We further demonstrate that safety justifications accompanying preferences can significantly enhance safety or prioritise user-prescribed aspects of safety associated with them during deployment.

Momentum

Related topics

Latest papers 167