Reward Shaping

Follow this topic Ask about this topic

Momentum

12 papers in the last four weeks, up 300% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 89

All topics

RewardWeaver: Long-Horizon Interactive Learning for Language Agents via Self-Evolving Reward Adaptation
Oct 7, 2026Hengbo Xiao, Boyao Zhang, Purui Liu +4RL for LLM Agents Rubric-Based RL
Ask the Expert: LLM-Guided Reinforcement Learning for Autonomous Cyber Defense
Oct 7, 2026Fernando Martinez, Abhishek Satyam, Tao Li +3Autonomous Cyber Defense LLM-Guided RL
Department of Computer and Information Sciences, Fordham University, New York, USA · Department of Systems Engineering, Stevens Institute of Technology, New Jersey, USA
Task-Space Imitation Guidance for Efficient Reinforcement Learning
Oct 5, 2026Salar Asayesh, Hossein Darani, Todd Cao +2Reward Shaping Imitation Learning
Sanctuary AI, Canada
Dependency-Aware Reward Shaping for Agentic Reinforcement Learning
Oct 1, 2026Ziyi Chen, Yan Zhang, Jianhui Wei +2RL for Language Models Agentic RL
University of Illinois Urbana-Champaign · National University of Singapore · Zhejiang University +1
T2SPO: Trajectory-to-Step Policy Optimization for Agentic Reinforcement Learning
Sep 30, 2026Bo-Wen Zhang, Junwei He, Maoqi Liu +6Agentic RL Reward Shaping
State Key Laboratory of Novel Software Technology, Nanjing University · School of Intelligence Science and Technology, Nanjing University · ByteDance
ReF-HIL: Shaping the Critic around Human Action Neighborhoods for Efficient Human-in-the-Loop Reinforcement Learning
Sep 29, 2026Shaoyin Luo, Song Wang, Shibo Xia +5RL for Robotics Reward Shaping
Department of Mechanical Engineering, Tsinghua University, Beijing 100084, China
StructRL: Online Structured Reinforcement Learning for Long-Horizon Vision-Language-Action Tasks
Sep 28, 2026Ziyi Yin, Sangmin Woo, Kang Zhou +4Reward Shaping Long-Horizon Robotic Manipulation
The Pennsylvania State University · Work done during an internship at Amazon · Amazon AWS AI
Quality Determines Direction, Length Shapes Magnitude: Length Control for Open-Ended Reinforcement Learning
Sep 28, 2026Zijun Weng, Zhongan Bi, Xuanang Gao +5Open-Ended Generation RL for Language Models
Fudan University · Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd. · Zhejiang University +1
Sufficiency of Zeroth-Order Reward Shaping for Policy Gradient in Stabilization Control
Sep 28, 2026Yisheng Zhang, Tao Wang, Sicun GaoReward Shaping Robotic RL
University of California San Diego, La Jolla, CA, USA · Tsinghua University, Beijing, China
Counterfactual Constraint-Conditioned On-Policy Distillation for Multi-Constraint Instruction Following
Sep 23, 2026Yanzhao Zheng, Yuanqiang Yu, Tianze Xu +6Reward Shaping Instruction Following
Alibaba Group, Hangzhou, China
Improving Offline Goal-Conditioned Reinforcement Learning via Selective Reward Stimulation
Sep 16, 2026Jing ZhangReward Shaping Offline RL
The Hong Kong University of Science and Technology
FIERCE: From Generalist Robot Policies to Fast Specialists via Progress-Failure Feedback
Sep 16, 2026Runjia Tan, Yuang Tu, Yujie Yan +3Reward Modeling Robot Policy Learning
Nanyang Technological University. · Guangzhou Cloudbutterfly Technology Co., Ltd.
DistAL: Distance-based Advantage Learning for VLA Fine-Tuning
Sep 16, 2026Reece O'Mahoney, Ioannis HavoutisReinforcement Learning Reward Shaping
Oxford Robotics Institute, University of Oxford, UK
Long-Horizon Language Model Reinforcement Learning via Progressive Point Matching
Sep 7, 2026Preston Fu, Kevin Frans, Oleh Rybkin +2Reinforcement Learning Reward Shaping
UC Berkeley · Carnegie Mellon University
Phase-and-First-Arrival VLM Feedback for Sparse-Reward Reinforcement Learning in Surgical Manipulation
Sep 7, 2026Wanli Liuchen, Fangyuan Wang, Bin Li +4Reward Shaping Temporal Credit Assignment
The Hong Kong Polytechnic University · The Chinese University of Hong Kong · Mohamed bin Zayed University of Artificial Intelligence +1
Gradients Know What Outcomes Don't: Unlocking Reinforcement Learning for LLM Reasoning with Gradient-Aligned Rewards
Sep 3, 2026Leqi Zheng, Jinbo Su, Fang Niu +8Numerical Reasoning in Language Models Reward Shaping
Tsinghua University · Renmin University of China · Institute of Information Engineering, CAS +4
Cliff: Learning Process Rewards from the First Mistake
Sep 2, 2026Peixuan Han, Runhui Wang, Ketan Ramaneti +3Reinforcement Learning with Verifiable Rewards LLM Reasoning
1Amazon Web Services · University of Illinois Urbana-Champaign
ERR+: Sequential Entropy Resolution for Efficient and Decisive LLM Reasoning
Aug 28, 2026Xin Jiang, Minhao Wang, Wen Wu +4Reward Shaping RL for Language Model Reasoning
School of Computer Science and Technology, East China Normal University, Shanghai 200241, China · State Key Laboratory of Estuarine and Coastal Research, School of Computer Science and Technology, East China Normal University, Shanghai 200241, China
A Unified Framework for Dynamic Reward Shaping in Reinforcement Learning
Aug 8, 2026Fouad BahrpeymaReward Shaping Reward Design for RL
Industrial AI Group Professorship for Information Management Faculty of Informatics / Mathematics Dresden University of Applied Sciences Dresden, Germany
Enhancing Social Intelligence in LLMs with Hierarchical Reasoning and Utterance-Level Goal Rewarding
Aug 6, 2026Xiaofeng Wang, Kakam Chong, Shuai Xiao +9Reward Shaping Hierarchical RL
Independent Researcher · Alibaba Group · Shanghai Jiao Tong University
Calibrating Artificial Guilt: Neurally Grounded Reward Shaping for Prosocial Multi-Agent Reinforcement Learning
Aug 5, 2026Aaditya Mehta, Arya ShahReinforcement Learning Reward Shaping
Mahatma Gandhi International School, Ahmedabad, India · Indian Institute of Technology, Gandhinagar, India
Agentic Reinforcement Learning with Self-Distilled Reward Shaping
Aug 4, 2026Ranxu Zhang, Guinan Chen, Chenshaodong +5Reward Shaping Token-Level Credit Assignment
Alibaba Group · University of Science and Technology of China Hefei, China · Alibaba Group Hangzhou, China
ValueFormer: A Causal Transformer Value Function with Stage-Aware Labels for Semi-Autonomous Vision-Language-Action Policies
Aug 3, 2026Inkyu Sa, Konstantin Stulov, Rajat BhageriaReward Shaping Value Function Estimation
Chef Robotics
PROGRESS: Coverage-guided RL to Train Search-augmented LLM Agent
Aug 2, 2026Sudipta Paul, Vijay Srinivasan, Vivek Kulkarni +4Reward Shaping LLM-Guided RL
AI Center-Mountain View, Samsung Electronics
Minute-Scale Training for Microrobot Navigation
Aug 1, 2026Yinghan Sun, Aoji Zhu, Xiang Ji +6Reward Shaping Robot Navigation
State Key Laboratory of Ultra-precision Machining Technology, Department of Industrial and Systems Engineering, The Hong Kong Polytechnic University, Hong Kong, China · Department of Mechanical and Automation Engineering, The Chinese University of Hong Kong, Hong Kong, China · Shenzhen Loop Area Institute, Shenzhen, China +3
DocPO: Advancing Document Policy Optimization via Tailored Step-Aware Rewards
Aug 1, 2026Yunhao Wang, Binghong Wu, Zhenyu Huang +4Reward Shaping Document Parsing
Tencent Hunyuan · Beijing, China · Shanghai, China +1
TaPR: Test-Aware Policy Refinement for Feedback-Conditioned Code Generation
Aug 1, 2026Aofan Liu, Jingxiang Meng, Fangxin Liu +1RL for Code Generation Reward Shaping
∗These authors contributed equally. · †Corresponding author.
RMSWeb: Reflection, Failure-Mode Mining, and Salvage-DS for Web Agent Reinforcement Learning
Jul 31, 2026Chengbo Liu, Lifang Zhou, Ruijie Yan +8Reward Shaping Offline RL
1Microsoft, Beijing, China · 2Southeast University, Nanjing, China
Meta-Learned Reward Shaping for Reinforcement Learning from Human Feedback
Jul 28, 2026Yunpeng ChuLLM Alignment Reward Shaping
Stony Brook University
The Dark Room in the Reward Channel: Dense Prediction Rewards Collapse GRPO-Trained LLM Agents -- and The Channel, Not the Content, Decides What Works
Jul 23, 2026Yu WangReward Hacking Reward Shaping
Independent researcher

Oct 7, 2026, cs.AIJ/K move · Enter open · S save

RewardWeaver: Long-Horizon Interactive Learning for Language Agents via Self-Evolving Reward Adaptation

Hengbo Xiao, Boyao Zhang, Purui Liu, Yuxuan Zheng+3

RL for LLM Agents Rubric-Based RL Reward Shaping Process Reward Models Reinforcement Learning with Verifiable Rewards Credit Assignment in RL

Open paper AskPDF arXiv

Reinforcement learning with verifiable rewards (RLVR) has driven substantial progress in domains where task outcomes can be reliably evaluated, but long-horizon interaction remains challenging due to sparse terminal feedback and difficult credit assignment. Process rewards provide denser supervision, yet the capabilities most relevant for training can change as the policy evolves: a behavior that is easy to evaluate or frequently deficient need not be the bottleneck currently limiting task success. We introduce RewardWeaver, a self-evolving reward adaptation framework for language agents in long-horizon interaction. RewardWeaver maintains a validated capability space in which the semantics of admitted Rubrics remain fixed, and closes the loop between policy optimization, task evaluation, failure attribution, and reward adaptation. After each training stage, it performs outcome-grounded backward attribution on low-outcome trajectories, aggregates recurrent and policy-controlled capability bottlenecks, and dynamically selects the corresponding process rewards for the next stage. Recurrent failures not covered by the existing capability space trigger a separate, controlled expansion procedure. We evaluate REWARDWEAVER on SOTOPIA, Amazon?HistoryPrice, and a newly constructed Sales Benchmark. Across social interaction, bilateral bargaining, and domain-specific sales, REWARDWEAVER establishes new state-of-the-art (SOTA) results. Ablations further demonstrate the importance of dynamic reward allocation, failure-grounded attribution, and stable semantics for admitted capabilities.

Momentum

Related topics

Latest papers 89