Reward Shaping

Ask about this topic

Overview

Reward shaping studies how to turn sparse or delayed task outcomes into learning signals that help an agent identify useful actions and make progress. Common approaches add potential-based progress estimates, subgoal or process rewards, and signals learned from demonstrations, language models, visual feedback, or model uncertainty; they are used in robotics, tool-using language agents, reasoning, and control. A central challenge is providing dense, reliable credit without changing the intended objective or encouraging reward hacking, verbosity, or unsafe shortcuts. Current work increasingly tests not just whether rewards improve benchmark performance, but also whether they preserve optimal policies, remain stable during optimization, and generalize beyond training settings.

Latest papers

CardsList

142 papers21 papers in the last four weeks, up 75% on the four weeks before. 0.1% of all new papers.

  1. Enhancing Deep Reinforcement Learning for Robotic Motion Planning via Task Decomposition

    Oct 26, 2026Yuanzhi He, Victor Romero-Cano, Jiayu Xu +2Reinforcement LearningRobotic RL

    School of Computational and Mathematical Sciences, Cardiff University, United Kingdom

  2. MPBRS: Multi-Potential-Based Reward Shaping for Unified Reward Configuration in Humanoid Locomotion

    Oct 21, 2026Van Chung Nguyen, Hung Manh LaReward ShapingReward Design for RL

    are with the Advanced Robotics and Automation (ARA) Lab, Department of Computer Science and Engineering, University of Nevada, Reno, NV 89557, USA.

  3. The Predictive--Optimization Gap in GuanDan Self-Play

    Oct 21, 2026Qicheng Zou, Biao Yuan, Siyu Song +2Predict-Then-OptimizeSelf-Play RL

    the University of California San Diego, La Jolla, CA, USA. · Chabiyue (Shanghai) Information Technology Co., Ltd., Shanghai, China.

  4. Mitigating Reward Hacking in Preference Reward Models with Evolving Penalties

    Oct 19, 2026Afonso Baldo, António Farinhas, Marcos Treviso +2Reward HackingRL for Language Models

    Sword Health · Instituto Superior Técnico, Universidade de Lisboa · Gandara AI +2

  5. Progress Reward Model for Reinforcement Learning via Large Language Models

    Oct 15, 2026Xiuhui Zhang, Ning Gao, Xingyu Jiang +4Reward Design for RLLLM-Guided RL

    Beihang University, 37 Xueyuan Road, Haidian District, Beijing

  6. GRID-Planner: Graph-Structured Relational Reward Shaping for Diffusion Planning

    Oct 15, 2026Zhao Yang, Zezhong Qian, Yinan Shi +2Autonomous Driving PlanningDiffusion-Based Planning

    Xi’an Jiaotong University, Xi’an, China · Cytoderm Intelligent Technology, Beijing, China

  7. Diagnosing and Correcting Reward Hacking in Verifiable Instruction Following

    Oct 14, 2026Sangyeon Yoon, Wooseok Seo, Hamish Ivison +5Reward HackingInstruction Following

    Yonsei University · FuriosaAI · University of Washington

  8. Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards

    Oct 13, 2026Wenbo Hu, Shraman Pramanick, Krishna Kumar Singh +3LLM Agent TrainingLong-Horizon LLM Agents

    Adobe Research · University of California, Los Angeles

  9. Training LLMs for Multi-Agent Collaboration with Verifiable Information Rewards

    Oct 13, 2026Keru Chen, Sen Lin, Yingbin Liang +2LLM Agent TrainingMulti-Agent Collaboration

    School of ECEE, Arizona State University · Computer Science Department, University of Houston · Department of ECE, The Ohio State University +1

  10. CodeLift: Your Test Suite Is Secretly a Dense SWE Curriculum

    Oct 12, 2026Junjielong Xu, Huachao Zhu, Alyssa Wu +2RL for Code GenerationSoftware Engineering Agents

    School of Data Science, The Chinese University of Hong Kong, Shenzhen · Tencent

  11. ADAPT: Audio-Depth Action Planning with Transition-Aware Rewards for Lights-Out Industrial Navigation

    Oct 11, 2026Wenjun Huang, Shenghao Fu, Yian Jin +12Robot NavigationRobotic RL

    University of California, Irvine · University of Pennsylvania · Carnegie Mellon University +6

  12. Self-Evolving Visual Agent via Margin Reward

    Oct 10, 2026Wenxi Yang, Yuzhong Zhao, Baochang Zhang +2Tool-Using Vision-Language AgentsRL for Tool Use

    UCAS · Microsoft Research Asia · Beihang University

  13. RewardWeaver: Long-Horizon Interactive Learning for Language Agents via Self-Evolving Reward Adaptation

    Oct 7, 2026Hengbo Xiao, Boyao Zhang, Purui Liu +4RL for LLM AgentsRubric-Based RL

  14. Ask the Expert: LLM-Guided Reinforcement Learning for Autonomous Cyber Defense

    Oct 7, 2026Fernando Martinez, Abhishek Satyam, Tao Li +3Autonomous Cyber DefenseLLM-Guided RL

    Department of Computer and Information Sciences, Fordham University, New York, USA · Department of Systems Engineering, Stevens Institute of Technology, New Jersey, USA

  15. Beyond Scalar IoU: Structured Verification from Rollout Groups for Video Temporal Grounding

    Oct 6, 2026Youngjae Cho, Won Young Jhoo, Jongsuk KimTemporal Video GroundingReward Shaping

    PYLER

  16. Task-Space Imitation Guidance for Efficient Reinforcement Learning

    Oct 5, 2026Salar Asayesh, Hossein Darani, Todd Cao +2Reward ShapingImitation Learning

    Sanctuary AI, Canada

  17. Dependency-Aware Reward Shaping for Agentic Reinforcement Learning

    Oct 1, 2026Ziyi Chen, Yan Zhang, Jianhui Wei +2RL for Language ModelsAgentic RL

    University of Illinois Urbana-Champaign · National University of Singapore · Zhejiang University +1

  18. T2SPO: Trajectory-to-Step Policy Optimization for Agentic Reinforcement Learning

    Sep 30, 2026Bo-Wen Zhang, Junwei He, Maoqi Liu +6Agentic RLReward Shaping

    State Key Laboratory of Novel Software Technology, Nanjing University · School of Intelligence Science and Technology, Nanjing University · ByteDance

  19. ReF-HIL: Shaping the Critic around Human Action Neighborhoods for Efficient Human-in-the-Loop Reinforcement Learning

    Sep 29, 2026Shaoyin Luo, Song Wang, Shibo Xia +5RL for RoboticsReward Shaping

    Department of Mechanical Engineering, Tsinghua University, Beijing 100084, China

  20. StructRL: Online Structured Reinforcement Learning for Long-Horizon Vision-Language-Action Tasks

    Sep 28, 2026Ziyi Yin, Sangmin Woo, Kang Zhou +4Reward ShapingLong-Horizon Robotic Manipulation

    The Pennsylvania State University · Work done during an internship at Amazon · Amazon AWS AI

  21. Quality Determines Direction, Length Shapes Magnitude: Length Control for Open-Ended Reinforcement Learning

    Sep 28, 2026Zijun Weng, Zhongan Bi, Xuanang Gao +5Open-Ended GenerationRL for Language Models

    Fudan University · Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd. · Zhejiang University +1

  22. Sufficiency of Zeroth-Order Reward Shaping for Policy Gradient in Stabilization Control

    Sep 28, 2026Yisheng Zhang, Tao Wang, Sicun GaoReward ShapingRobotic RL

    University of California San Diego, La Jolla, CA, USA · Tsinghua University, Beijing, China

  23. SWE-MILE: Asynchronous Potential-Induced Milestone Credit Assignment for Long-Horizon Software Engineering Agents

    Sep 26, 2026Chaoqun Cui, Hao Zhou, Meiqi Chen +2Software Engineering AgentsStep-Level Credit Assignment in RL

    MAIS, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Tencent Inc., China

  24. Counterfactual Constraint-Conditioned On-Policy Distillation for Multi-Constraint Instruction Following

    Sep 23, 2026Yanzhao Zheng, Yuanqiang Yu, Tianze Xu +6Reward ShapingInstruction Following

    Alibaba Group, Hangzhou, China

  25. HiRE: Hindsight Reward Editing for Policy Finetuning

    Sep 22, 2026Haoyi Niu, Zhengtao Han, Yufeng Ji +2Reward Design for RLRobot Policy Learning

    University of California, Berkeley · The Chinese University of Hong Kong

  26. IncentRL: The Trade-Off Between Preference Guidance and Task Performance

    Sep 18, 2026Xuening Wu, Yanlan Kang, Shenqin YinReinforcement LearningPreference-Based RL

    Fudan University

  27. Improving Offline Goal-Conditioned Reinforcement Learning via Selective Reward Stimulation

    Sep 16, 2026Jing ZhangReward ShapingOffline RL

    The Hong Kong University of Science and Technology

  28. FIERCE: From Generalist Robot Policies to Fast Specialists via Progress-Failure Feedback

    Sep 16, 2026Runjia Tan, Yuang Tu, Yujie Yan +3Reward ModelingRobot Policy Learning

    Nanyang Technological University. · Guangzhou Cloudbutterfly Technology Co., Ltd.

  29. DistAL: Distance-based Advantage Learning for VLA Fine-Tuning

    Sep 16, 2026Reece O'Mahoney, Ioannis HavoutisReinforcement LearningReward Shaping

    Oxford Robotics Institute, University of Oxford, UK

  30. The Filter Metric is Safety-Critical: Phantom Advantages in Group-Relative RL under Shaped Rewards

    Sep 12, 2026Juntao YuReinforcement LearningGroup Relative Policy Optimization

    Fudan University, Shanghai, China.