Reward Shaping

Reward shaping studies how to turn sparse or delayed task outcomes into learning signals that help an agent identify useful actions and make progress. Common approaches add potential-based progress estimates, subgoal or process rewards, and signals learned from demonstrations, language models, visual feedback, or model uncertainty; they are used in robotics, tool-using language agents, reasoning, and control. A central challenge is providing dense, reliable credit without changing the intended objective or encouraging reward hacking, verbosity, or unsafe shortcuts. Current work increasingly tests not just whether rewards improve benchmark performance, but also whether they preserve optimal policies, remain stable during optimization, and generalize beyond training settings.