Reward Design for RL

RL: Reinforcement Learning

Ask about this topic

Overview

Reward design studies how to turn task goals, preferences, and constraints into learning signals, especially when success is sparse or objectives compete. Approaches include potential-based shaping and reward machines, preference and rubric models, visual progress rewards, intrinsic exploration bonuses, and automated reward search using language models. These methods are used in settings from robot manipulation and locomotion to language-model reasoning, tool use, and multi-agent coordination. A central tension is that denser or adaptive rewards can improve credit assignment but also mislead policies, destabilize training, or invite reward hacking, so evaluation must test intended outcomes rather than proxy scores alone.

Latest papers

CardsList

261 papers44 papers in the last four weeks, up 83% on the four weeks before. 0.3% of all new papers.

  1. Occupancy Reward Shaping: Improving Credit Assignment for Offline Goal-Conditioned Reinforcement Learning

    Apr 22, 2026Aravind Venugopal, Jiayu Chen, Xudong Wu +3Reward ShapingTemporal Credit Assignment

    Carnegie Mellon University · The University of Hong Kong · INFIFORCE Intelligent Technology +1

  2. OGER: A Robust Offline-Guided Exploration Reward for Hybrid Reinforcement Learning

    Apr 20, 2026Xinyu Ma, Mingzhou Xu, Xuebo Liu +4Reward ShapingRL for Language Model Reasoning

    Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China · Hithink RoyalFlush Information Network, Hangzhou, China · Computer and Information Science, University of Macau, Macau, China

  3. Learning to Correct: Calibrated Reinforcement Learning for Multi-Attempt Chain-of-Thought

    Apr 20, 2026Muhammed Emrullah Ildiz, Halil Alperen Gozeten, Ege Onur Taga +1CoT ReasoningRL for Language Model Reasoning

    University of Michigan, Ann Arbor · Google Research

  4. Fuzzy Logic Theory-based Adaptive Reward Shaping for Robust Reinforcement Learning (FARS)

    Apr 17, 2026Hürkan Şahin, Van Huyen Dang, Erdi Sayar +2Reward ShapingAutonomous Drone Racing

    Automatic Control Group (RAT), Paderborn University, 33098 Paderborn, Germany

  5. Enhancing Reinforcement Learning for Radiology Report Generation with Evidence-aware Rewards and Self-correcting Preference Learning

    Apr 15, 2026Qin Zhou, Guoyan Liang, Qianyi Yang +4Radiology Report GenerationRL for Generative Models

    Department of Computer Science and Engineering, ECUST, China · Key Laboratory of Smart Manufacturing in Energy Chemical Process, Ministry of Education, P. R. China · Zhejiang University, Hangzhou, China +1

  6. Chain of Uncertain Rewards with Large Language Models for Reinforcement Learning

    Apr 15, 2026Shentong MoReward Design for RLLLM-Guided RL

    Carnegie Mellon University

  7. Robust Optimization for Mitigating Reward Hacking with Correlated Proxies

    Apr 13, 2026Zixuan Liu, Xiaolin Sun, Zizhan ZhengReward HackingRobust RL

    Department of Computer Science Tulane University New Orleans, LA 70118, USA

  8. Utilizing and Calibrating Hindsight Process Rewards via Reinforcement with Mutual Information Self-Evaluation

    Apr 13, 2026Jiashu Yao, Heyan Huang, Zeming Liu +1Reinforcement LearningRL for LLM Agents

    Beijing Institute of Technology · Beihang University

  9. Triviality Corrected Endogenous Reward

    Apr 13, 2026Xinda Wang, Zhengxu Hou, Yangshijie Zhang +6RL for Language ModelsReward Design for RL

    Peking University · Alibaba Group · Lanzhou University

  10. The Past Is Not Past: Memory-Enhanced Dynamic Reward Shaping

    Apr 13, 2026Yang Liu, Enxi Wang, Yufei Gao +6RL for Language ModelsReinforcement Learning

    Fudan University · Shanghai Innovation Institute

  11. PokeRL: Reinforcement Learning for Pokemon Red

    Apr 12, 2026Dheeraj Mudireddy, Sai PatibandlaRL for Game AIReinforcement Learning

    Texas A&M Institute of Data Science Texas A&M University College Station, USA · Dept. of Computer Science Texas A&M University College Station, USA

  12. Learning Preference-Based Objectives from Clinical Narratives for Dynamic Sepsis Treatment

    Apr 12, 2026Daniel J. Tan, Jayne Hui Zhen Chan, Kai Wen Hwang +3Reward Design for RLReinforcement Learning

    Institute of Data Science, National University of Singapore, Singapore · National University Hospital, Singapore · Saw Swee Hock School of Public Health, National University of Singapore, Singapore

  13. On Tackling Complex Tasks with Reward Machines and Signal Temporal Logics

    Apr 9, 2026Ana María Gómez Ruiz, Thao Dang, Alexandre DonzéRL ControlReinforcement Learning

    UGA · VERIMAG, Université Grenoble Alpes · VERIMAG - IMAG, CNRS, UGA +1

  14. SPARD: Self-Paced Curriculum for RL Alignment via Integrating Reward Dynamics and Data Utility

    Apr 9, 2026Xuyang Zhi, Peilun zhou, Chengqiang Lu +10RL Post-TrainingMulti-Objective Language Model Alignment

    University of Science and Technology of China · Xiaohongshu Inc.

  15. Pedagogical Safety in Educational Reinforcement Learning: Formalizing and Detecting Reward Hacking in AI Tutoring Systems

    Apr 5, 2026Oluseyi Olukola, Nick RahimiReward HackingSafe RL

    School of Computing Sciences and Computer Engineering, Hattiesburg, MS, USA

  16. When Adaptive Rewards Hurt: Causal Probing and the Switching-Stability Dilemma in LLM-Guided LEO Satellite Scheduling

    Apr 4, 2026Yuanhang LiReward Design for RLLLM-Guided RL

  17. Multi-Turn Reinforcement Learning for Tool-Calling Agents with Iterative Reward Calibration

    Apr 3, 2026Wachiravit Modecrua, Krittanon Kaewtawee, Krittin Pachtrachai +1Multi-Turn RLRL for Tool Use

    Amity Research and Application Center (ARAC)

  18. Rubrics to Tokens: Bridging Response-level Rubrics and Token-level Rewards in Instruction Following Tasks

    Apr 3, 2026Tianze Xu, Yanzhao Zheng, Pengrui Lu +11RL for Language ModelsRubric-Based RL

    Shanghai Jiao Tong University · GAIR · Alibaba Group +2

  19. From Rebound to Remedy: Understanding and Mitigating Reward Hacking via Representation Engineering

    Apr 1, 2026Rui Wu, Ruixiang TangReward HackingRL Fine-Tuning

    Department of Computer Science Rutgers University

  20. Extending MONA in Camera Dropbox: Reproduction, Learned Approval, and Design Implications for Reward-Hacking Mitigation

    Mar 31, 2026Nathan HeathReward HackingReward Modeling

    Independent Researcher

  21. ShapE-GRPO: Shapley-Enhanced Reward Allocation for Multi-Candidate LLM Training

    Mar 31, 2026Rui Ai, Yu Pan, David Simchi-Levi +1RL Post-TrainingReward Design for RL

    MIT · University of Sydney · University of Texas at Dallas

  22. Generalizable Dense Reward for Long-Horizon Robotic Tasks

    Mar 31, 2026Silong Yong, Stephen Sheng, Carl Qi +6Reinforcement LearningLong-Horizon Robotic Manipulation

    Carnegie Mellon University · Amazon Robotics · UT Austin

  23. Large Language Model Guided Incentive Aware Reward Design for Cooperative Multi-Agent Reinforcement Learning

    Mar 25, 2026Dogan Urgun, Gokhan GungorReward Design for RLCooperative MARL

    Department of Electrical and Electronics Engineering, Karabuk University, 78050 Karabuk, Türkiye · Department of Mechatronics Engineering, Karabuk University, 78050 Karabuk, Türkiye

  24. Experience is the Best Teacher: Motivating Effective Exploration in Reinforcement Learning for LLMs

    Mar 20, 2026Wenjian Zhang, Kongcheng Zhang, Jiaxin Qi +2Hindsight Experience ReplayRL for Language Model Reasoning

    Dalian University of Technology · Zhejiang University · Chinese Academy of Sciences

  25. Decomposable Reward Modeling and Realistic Environment Design for Reinforcement Learning-Based Forex Trading

    Mar 20, 2026Nabeel Ahmad SaiddReinforcement LearningReward Design for RL

    Dr APJ Abdul Kalam Technical University, India

  26. RewardFlow: Topology-Aware Reward Propagation on State Graphs for Agentic RL with Large Language Models

    Mar 19, 2026Xiao Feng, Bo Han, Zhanke Zhou +5RL for LLM AgentsReward Design for RL

    TMLR Group, Hong Kong Baptist University · TCL Corporate Research (HK) Co., Ltd · Cooperative Medianet Innovation Center, Shanghai Jiao Tong University +1

  27. Escaping Offline Pessimism: Vector-Field Reward Shaping for Safe Frontier Exploration

    Mar 18, 2026Amirhossein Roknilamouki, Arnob Ghosh, Eylem Ekici +1RL ExplorationReward Design for RL

    Department of Electrical and Computer Engineering, The Ohio State University · Department of Electrical and Computer Engineering, New Jersey Institute of Technology · Department of Computer Science and Engineering, The Ohio State University.

  28. Rewarding DINO: Predicting Dense Rewards with Vision Foundation Models

    Mar 17, 2026Pierre Krack, Tobias Jülg, Wolfram Burgard +1Reward ModelingReward Design for RL

    Department of Computer Science & Artificial Intelligence, University of Technology Nuremberg, Germany. · TUM School of Computation, Information and Technology, Technical University of Munich, Germany

  29. From the Inside Out: Progressive Distribution Refinement for Confidence Calibration

    Mar 17, 2026Xizhong Yang, Yinan Xia, Huiming Wang +1Test-Time RLReinforcement Learning

    Southeast University · Kuaishou Technology · SUTD

  30. Large Reward Models: Generalizable Online Robot Reward Generation with Vision-Language Models

    Mar 17, 2026Yanru Wu, Weiduo Yuan, Esteban Martinez Licon +4Long-Horizon Robotic ManipulationRobot Skill Learning

    USC Physical Superintelligence Lab