Reward Shaping

Follow this topic Ask about this topic

Momentum

12 papers in the last four weeks, up 300% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 89

All topics

Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learning
Jul 21, 2026Lizhe Fang, Weizhou Shen, Tianyi Tang +1LLM Grounding Reward Shaping
Peking University · Qwen Team of Alibaba Group · Alibaba Group ATH
LenGuard-GPC: Length Guarding with Guided-Prompt Consistency for Spatial Reasoning Reinforce Learning
Jul 19, 2026Xingjian Tao, Yiwei Wang, Yujun Cai +1Visual Spatial Reasoning Reward Shaping
HKUST(GZ) · UC Merced · University of Queensland +1
Foresight Residual RL for Long-Horizon Robot Manipulation with Vision-Language-Action Models
Jul 17, 2026Yuhan Liu, Xinyu Zhang, Litao Liu +1Reward Shaping Long-Horizon Robotic Manipulation
Department of Computer Science, Rutgers University
SCOPE-RL: Optimizing Reasoning Paths Before and After Success
Jul 13, 2026Xiaojian Liu, Han Xu, Jianqiang Xia +6Reward Shaping RL for Language Model Reasoning
Baidu Inc. · Shandong University
Regime-Conditional Stabilisation of LLM-Augmented Cooperative Multi-Agent Reinforcement Learning
Jul 5, 2026Faid Keddouri, Sohaib Houhou, Aissa Boulmerka +1Reward Shaping LLM-Guided RL
National School of Artificial Intelligence (ENSIA), Sidi Abdellah Campus, Algiers, Algeria · Cosys-Grettia, Univ Gustave Eiffel, F-77454 Marne-la-Vallee, France.
UniCoder: Unified Visual-to-Code Generation via Symbolic Rewards and Reference-Guided Code Optimization
Jun 30, 2026Yaozhi Zheng, Yilei Jiang, Manyuan Zhang +5RL for Code Generation Reinforcement Learning
The Chinese University of Hong Kong · Shanghai Artificial Intelligence Laboratory
Designing Reward Signals for Portable Query Generation: A Case Study in Industrial Semantic Job Search
Jun 25, 2026Ping Liu, Qianqi Shen, Jianqiang Shen +11Reward Hacking Reward Shaping
LinkedIn Corporation, Mountain View, CA, USA
Automating Potential-based Reward Shaping with Vision Language Model Guidance
Jun 25, 2026Henrik Müller, Daniel KudenkoReward Shaping Sparse-Reward RL
L3S Research Center Leibniz University Hannover Hannover, 30167, Germany
Sample-efficient Transfer Reinforcement Learning via Adaptive Reward Shaping and Policy-Ratio Reweighting Strategy
Jun 25, 2026Wenjie Huang, Yang Li, Jingjia Teng +6Reinforcement Learning Reward Shaping
State Key Laboratory of Advanced Design and Manufacturing Technology for Vehicle, College of Mechanical and Vehicle Engineering, Hunan University, Changsha 410082, China · Key Laboratory of Intelligent Air-Ground Cooperative Control for Universities in Chongqing, School of Automation, Chongqing University of Posts and Telecommunications, Chongqing 400065, China · Xi’an Institute of High-Tech, Xi’an 710025, China +1
Semantic Consistency Policy Optimization for Reinforcement Learning of LLM Agents
Jun 24, 2026Peng Xu, Sijia Chen, Junzhuo Li +1Reward Shaping Credit Assignment in RL
†The Hong Kong University of Science and Technology (Guangzhou) · ‡The Hong Kong University of Science and Technology
Reward-Conditioned Attention: How Reward Design Shapes What Autonomous Driving Agents See
Jun 23, 2026Mohamed Benabdelouahad, Ahmed Djalal Hacini, Nadir Farhi +1Reward Shaping RL for Autonomous Driving
National School of Artificial Intelligence (ENSIA), Sidi Abdellah Campus, Algiers, Algeria. · Cosys-Grettia, Univ Gustave Eiffel, F-77454 Marne-la-Vallee, France.
Beyond Penalizing Mistakes: Stabilizing Efficiency Training in Large Reasoning Models via Adaptive Correct-Only Rewards
Jun 21, 2026Jungseob Lee, Seungyoon Lee, Seongtae Hong +3Reward Shaping RL for Language Model Reasoning
Korea University · Soongsil University
A Reward-Petri-Net Interpretation of Temporal Behavior Trees
Jun 19, 2026Till Schmeil, Günther Waxenegger-Wilfing, Sebastian SchirmerReinforcement Learning Reward Shaping
University of Würzburg, Germany · German Aerospace Center (DLR), Germany
A Multi-Agent system for Multi-Objective constrained optimization
Jun 18, 2026Federica FilippiniMulti-Objective Reinforcement Learning Multi-Agent System Optimization
University of Milano-Bicocca Milan, Italy
CARE: Competence-Aware Reward Shaping for Adaptive Reasoning Length in Video-MLLMs
Jun 18, 2026Chengwen Liu, Hao Peng, Jisheng Dang +3Reward Shaping Video Reasoning
School of Information Science and Engineering, Lanzhou University, Lanzhou, China · School of Medical Technology, Beijing Institute of Technology, Beijing, China · School of Computing, National University of Singapore, Singapore
Model-Free Reinforcement Learning Control for Resilient Cyber-Physical Systems
Jun 17, 2026Hugo O. Garcés, Alejandro J. Rojas, Bernardo A. Hernández +5RL Control Cyber-Physical Systems
Departamento de Ingeniería Informática y Ciencias de la Computación, Universidad de Concepción, Concepción, Chile · Electrical Engineering Departament, Universidad de Talca · Department of Electrical & Computer Engineering, University of Alberta, Edmonton, T6G 1H9, Alberta, AB, Canada +2
PathRouter: Aligning Rewards with Retrieval Quality in Agentic Graph Retrieval-Augmented Generation
Jun 15, 2026Bo Wang, Heyan Huang, Yaolin Li +6Reward Shaping Graph Retrieval-Augmented Generation
Beijing Institute of Technology · Joy Future Academy
Evolutionary Bilevel Reward Shaping for Generalization in Reinforcement Learning
Jun 15, 2026Ekasit Usaratniwart, Xilin Gao, Marc Ong +1Reinforcement Learning Reward Shaping
University of Tsukuba, Tsukuba, Japan · RIKEN Center for Advanced Intelligence Project, Tokyo, Japan
Retrospective Progress-Aware Self-Refinement for LLM Agent Training
Jun 12, 2026Xinbei Ma, Congmin Zheng, Jiyang Qiu +11Reward Shaping LLM Agent Training
Shanghai Jiao Tong University · OPPO Research Institute
Anatomy of Post-Training: Using Interpretability to Characterize Data and Shape the Learning Signal
Jun 10, 2026Leon Bergen, Usha Bhalla, Sidharth Baskaran +14Reward Shaping LLM Interpretability
Core contributors
Internalizing Geometric Law: Learning from Solver Residuals for Precision-Critical Generation
Jun 8, 2026Rafael Cabral, Pang Zixi, Ziyi Shou +1Reward Shaping LLM-Based Program Synthesis
Huawei Celia Team
Can the Environment Speak for Itself? $T^{2}$ -GRPO: A Turn-Trajectory Group Relative Policy Optimization for Caregiver Agents
Jun 7, 2026Yutong Song, Jiang Wu, Pengfei Zhang +4Reward Shaping Healthcare
University of California, Irvine · Independent Researcher · Kennesaw State University
GIFT: LLM-Guided State-Reward Interface for Financial Reinforcement Learning
Jun 7, 2026Yanyan Wu, Boyi Zhang, Yanlin Liu +10Reinforcement Learning Reward Shaping
East China University of Science and Technology · University of Science and Technology of China · Southwestern University of Finance and Economics +5
ConSteer-RL: Steering Reasoning Capabilities in Large Language Models via Confidence-Aware Reinforcement Learning
Jun 6, 2026Qing Miao, Yiming Zhao, Jing Yang +5Reward Shaping RL for Language Model Reasoning
Xi’an Jiaotong University · University of Science and Technology of China
Predictive Style Matching: Natural and Robust Humanoid Locomotion
Jun 5, 2026Simeon Nedelchev, Ekaterina Chaikovskaia, Egor Davydenko +2Humanoid Robot Locomotion Reward Shaping
Moscow Institute of Physics and Technology (MIPT), Dolgoprudny, Moscow Region, Russia · Innopolis University, Innopolis, Russia · Sber Robotics Center, Moscow, Russia
SlimSearcher: Training Efficiency-Aware Web Agents via Adaptive Reward Gating
Jun 5, 2026Zequn Xie, Junjie Wang, Dan Yang +4Reward Shaping LLM Agent Training
Zhejiang University
Progress-SQL: Improving Reinforcement Learning for Text-to-SQL via Progressive Rewards
Jun 5, 2026Shihao Zhang, Xiaoman Wang, Yuan Liu +2RL for Language Models Reinforcement Learning
East China Normal University
Feat2Go: Visual Feature-Grounded Value Estimation for Embodied Reinforcement Learning
May 29, 2026Junyang Shu, Zhiwei Lin, Bingqing Wei +1Reward Shaping Value Function Estimation
Wangxuan Institute of Computer Technology, Peking University, China
Improving Small Language Models for Code Generation with Reinforcement Learning from Verification Feedback
May 28, 2026Egor Skopin, Evgeny KotelnikovRL for Code Generation Reward Shaping
Momentum Based Reward Design for Low Emission Traffic Signal Control
May 28, 2026Chinmay Mundane, Amith Manoharan, Arun Kumar SinghReinforcement Learning Reward Shaping
Institute of Technology, University of Tartu

Jul 21, 2026, cs.CLJ/K move · Enter open · S save

Copy Less, Ground More: Overcoming Repetitive Copying in Long-Context Reasoning via Evidence-Aware Reinforcement Learning

Lizhe Fang, Weizhou Shen, Tianyi Tang, Yisen Wang

Peking University · Qwen Team of Alibaba Group · Alibaba Group ATH

LLM Grounding Reward Shaping RL for Language Model Reasoning Long-Context Modeling Data Repetition in Language Model Training

Open paper AskPDF arXiv

Large language models that generate step-by-step reasoning traces have achieved strong performance on complex tasks, and extending them to long-context settings has emerged as an important frontier. However, we identify a critical failure mode in this regime: \emph{repetitive copying}, where models extensively copy text from the input into their reasoning traces rather than productively solving the problem. We show that this behavior is pervasive across frontier long-context LLMs and intensifies with context length. By separating each prompt into task-relevant key evidence and irrelevant distractor context, we further show that the root cause is insufficient grounding: models copy from the prompt indiscriminately, and those that fail to focus on key evidence are far more likely to answer incorrectly. Motivated by this diagnosis, we propose GEAR (Grounding Evidence-Aware Reward), a reward shaping method that augments the accuracy signal with a grounding reward for overlap with key evidence and a distractor penalty for overlap with irrelevant context. To enable GEAR on natural-language data, we develop an automated pipeline that constructs evidence-annotated training data from arbitrary documents. We validate GEAR across multiple model scales and benchmarks, showing consistent improvements of up to +4.6 average points over standard RL with accuracy-based rewards, with larger gains at longer contexts, while also reducing repetitive copying and thinking length. Our findings suggest that, even as long-context evaluation shifts from simple retrieval toward complex reasoning, accurate grounding in relevant evidence remains an indispensable capability with substantial room for improvement.

Momentum

Related topics

Latest papers 89