Reward Shaping

Follow this topic Ask about this topic

Reward shaping studies how to turn sparse or delayed task outcomes into learning signals that help an agent identify useful actions and make progress. Common approaches add potential-based progress estimates, subgoal or process rewards, and signals learned from demonstrations, language models, visual feedback, or model uncertainty; they are used in robotics, tool-using language agents, reasoning, and control. A central challenge is providing dense, reliable credit without changing the intended objective or encouraging reward hacking, verbosity, or unsafe shortcuts. Current work increasingly tests not just whether rewards improve benchmark performance, but also whether they preserve optimal policies, remain stable during optimization, and generalize beyond training settings.

Latest papers

142 papers21 papers in the last four weeks, up 75% on the four weeks before. 0.1% of all new papers.

Tackling Length Inflation Without Trade-offs: Group Relative Reward Rescaling for Reinforcement Learning
Mar 11, 2026Zichao Li, Jie Lou, Fangchen Dong +8RL for Language Models RL Post-Training
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Xiaohongshu Inc.
Reward-Zero: Language Embedding Driven Implicit Reward Mechanisms for Reinforcement Learning
Mar 10, 2026Heng Zhang, Haddy Alchaer, Arash Ajoudani +1Reward Design for RL Dense Reward RL
Human-Robot Interfaces and Interaction Lab, Istituto Italiano di Tecnologia, Genova, Italy · Ph.D. program of national interest in Robotics and Intelligent Machines (DRIM) and Universit ‘a di Genova, Genoa, Italy. · Edwardson School of Industrial Engineering, Purdue University, West Lafayette, IN, USA. +2
ProgAgent:A Continual RL Agent with Progress-Aware Rewards
Mar 8, 2026Jinzhou Tan, Gabriel Adineera, Jinoh KimContinual RL Continual Robot Learning
University of California, San Diego, USA · Texas A&M University-Commerce, USA
LLMs for High-Frequency Decision-Making: Normalized Action Reward-Guided Consistency Policy Optimization
Mar 3, 2026Yang Zhao, Zihao Li, Zhiyu Jiang +3Policy Learning LLM-Guided RL
School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University, Xi’an 710072, China
LongRLVR: Long-Context Reinforcement Learning Requires Verifiable Context Rewards
Mar 2, 2026Guanzheng Chen, Michael Qizhe Shieh, Lidong BingRL for Language Models Reinforcement Learning with Verifiable Rewards
National University of Singapore · MiroMind AI · absolute AI
MVR: Multi-view Video Reward Shaping for Reinforcement Learning
Mar 2, 2026Lirui Luo, Guoxi Zhang, Hongming Xu +3Reward Shaping Visual RL
School of Intelligence Science and Technology, Peking University · State Key Laboratory of General Artificial Intelligence, BIGAI
Keyframe-Guided Structured Rewards for Reinforcement Learning in Long-Horizon Laboratory Robotics
Feb 28, 2026Yibo Qiu, Shu'ang Sun, Haoliang Ye +2Long-Horizon Robotic Manipulation Robotic RL
Suzhou Institute for Advanced Research, University of Science and Technology of China, Suzhou, Jiangsu, China · School of Biomedical Engineering, Division of Life Sciences and Medicine, University of Science and Technology of China, Hefei, Anhui, China
InfoPO: Information-Driven Policy Optimization for User-Centric Agents
Feb 28, 2026Fanqi Kong, Jiayi Zhang, Mingyi Deng +3LLM Agent Training RL for LLM Agents
Peking University · DeepWisdom · The Hong Kong University of Science and Technology (Guangzhou) +1
Search-P1: Path-Centric Reward Shaping for Stable and Efficient Agentic RAG Training
Feb 26, 2026Tianle Xia, Ming Xu, Lingxiang Hu +7RL for RAG Agentic RAG
Tencent
Explore-on-Graph: Incentivizing Autonomous Exploration of Large Language Models on Knowledge Graphs with Path-refined Reward Modeling
Feb 25, 2026Shiqi Yan, Yubo Chen, Ruiqi Zhou +8LLM Reasoning with Graphs Knowledge Graph Question Answering
Zhongguancun Laboratory, Beijing, China · Department of Electronic Engineering, Tsinghua University, Beijing, China · Ant International, Ant Group, Hangzhou, Zhejiang, China +1
Dr. Seg: Revisiting GRPO Training for Visual Large Language Models through Perception-Oriented Design
Feb 25, 2026Haoxiang Sun, Tao Wang, Chenwei Tang +2Large Vision-Language Models RL for VLMs
College of Computer Science, Sichuan University · School of Electronic and Computer Engineering, Peking University
Self-Correcting VLA: Online Action Refinement via Sparse World Imagination
Feb 25, 2026Chenyv Liu, Wentao Tan, Lei Zhu +4Vision-Language-Action Models Robotic RL
Tongji University · University of Technology Sydney · University of Electronic Science and Technology of China +1
SELAUR: Self Evolving LLM Agent via Uncertainty-aware Rewards
Feb 24, 2026Dengjia Zhang, Xiaoou Liu, Lu Cheng +3RL for LLM Agents Reward Shaping
Johns Hopkins University, Baltimore MD, USA · Arizona State University, Tempe AZ, USA · University of Illinois Chicago, Chicago IL, USA +1
The Art of Efficient Reasoning: Data, Reward, and Optimization
Feb 24, 2026Taiqiang Wu, Zenan Xu, Bo Zhou +1Efficient Language Model Reasoning RL for Language Model Reasoning
The University of Hong Kong · LLM Department, Tencent
IG-RFT: An Interaction-Guided RL Framework for VLA Models in Long-Horizon Robotic Manipulation
Feb 24, 2026Zhian Su, Weijie Kong, Haonan Dong +1RL Fine-Tuning VLMs for Robotics
Grasp Laboratory, Mechanical Engineering Department, Zhejiang University, Hangzhou 310027, China. · Torch Kernel Co., Ltd., Hangzhou 310000, China.
MARLEM: A Multi-Agent Reinforcement Learning Simulation Framework for Implicit Cooperation in Decentralized Local Energy Markets
Feb 17, 2026Nelson Salazar-Pena, Alejandra Tabares, Andres Gonzalez-ManceraMulti-Agent Coordination Power Systems
Department of Mechanical Engineering, Universidad de los Andes, 111711, Bogotá D.C., Colombia · Department of Industrial Engineering, Universidad de los Andes, 111711, Bogotá D.C., Colombia
ManeuverNet: A Soft Actor-Critic Framework for Precise Maneuvering of Double-Ackermann-Steering Robots with Optimized Reward Functions
Feb 16, 2026Kohio Deflesselle, Mélodie Daniel, Aly Magassouba +2Robotic RL Agricultural Robotics
Univ. Bordeaux, CNRS, Bordeaux INP, LaBRI, UMR 5800, F-33400 Talence, France. · School of Computer Science, University of Nottingham, UK. · Instituto de Investigación en Ingeniería de Aragón (I3A), Universidad de Zaragoza, 50018 Zaragoza, Spain.
LongR: Unleashing Long-Context Reasoning via Reinforcement Learning with Dense Utility Rewards
Feb 5, 2026Qize Yu, Zijun Chen, Chen Wu +6Long-Context Reasoning in Language Models RL for Language Model Reasoning
Shanghai Jiao Tong University · Tsinghua University · Beijing University of Posts and Telecommunications, Beijing, China +1
TaoSR-AGRL: Adaptive Guided Reinforcement Learning Framework for E-commerce Search Relevance
Oct 9, 2025Jianhui Yang, Yiming Jin, Pengkun Jiao +6RL for Language Models Reward Shaping
Tsinghua University Beijing, China · Taobao & Tmall Group of Alibaba Beijing, China · Taobao & Tmall Group of Alibaba Hangzhou, China +1
A Graph-Based Reinforcement Learning Approach with Frontier Potential Based Reward for Safe Cluttered Environment Exploration
Apr 16, 2025Gabriele Calzolari, Vidya Sumathy, Christoforos Kanellakis +1Reward Shaping Mobile Robotics
Luleå University of Technology · Robotics and AI Group, Department of Computer Science, Electrical and Space Engineering, Luleåa University of Technology, Sweden
Reward-Centered ReST-MCTS: A Robust Decision-Making Framework for Robotic Manipulation in High Uncertainty Environments
Mar 7, 2025Xibai WangReward Shaping Monte Carlo Tree Search
NeuroQuant Labs Limited Hong Kong SAR, China
Leader Reward for POMO-Based Neural Combinatorial Optimization
May 22, 2024Chaoyang Wang, Pengzhi Cheng, Jingze Li +1Traveling Salesperson Problem Reward Shaping
Fudan University

Mar 11, 2026J/K move · Enter open · S save

Tackling Length Inflation Without Trade-offs: Group Relative Reward Rescaling for Reinforcement Learning

Zichao Li, Jie Lou, Fangchen Dong, Zhiyuan Fan+7

Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Xiaohongshu Inc.

RL for Language Models RL Post-Training Reward Shaping

Open paper AskPDF arXiv

Reinforcement learning significantly enhances LLM capabilities but suffers from a critical issue: length inflation, where models adopt verbosity or inefficient reasoning to maximize rewards. Prior approaches struggle to address this challenge in a general and lossless manner, primarily because additive penalties introduce a compensatory effect that creates optimization shortcuts, while heuristic gating strategies lack generality beyond binary feedback. To bridge this gap, we present Group Relative Reward Rescaling (GR

^3

), which reframes length control as a multiplicative rescaling paradigm, effectively establishing a generalized, continuous, and reward-dependent gating mechanism. To further ensure lossless optimization, we incorporate group-relative regularization and advantage-aware calibration, which dynamically adapt length budgets to instance difficulty and preserve the advantage signal of high-quality trajectories. Empirically, across both RLHF and RLVR settings, GR

^3

~maintains training dynamics and downstream performance comparable to standard GRPO while significantly mitigating length inflation, outperforming state-of-the-art length-regularized baselines.