Dense Reward RL

RL: Reinforcement Learning

Ask about this topic

Overview

Dense reward reinforcement learning studies how to give agents useful feedback during a task, rather than only scoring the final outcome, to improve learning and credit assignment. Approaches include progress and stage-based rewards, models learned from demonstrations or videos, language- and vision-based signals, and trajectory, graph, or token-level feedback. These methods are used in robotic manipulation, long-horizon language-model agents, coding, and mathematical reasoning, often alongside sparse outcome rewards. A central challenge is making intermediate signals accurate and generalizable without encouraging reward hacking or destabilizing training; in some settings, noisy or poorly delivered dense rewards can perform worse than sparse feedback.

Latest papers

CardsList

29 papers3 papers in the last four weeks, down 40% on the four weeks before. 0.0% of all new papers.

  1. Expert-Guided Rubrics as Dense Reward Signals for Reinforcement Learning in Non-Verifiable Domains

    Oct 17, 2026Ashish Bansal, Hossam Amer, Keerat Kaur Guliani +3RL for LLM AgentsReward Design for RL

    The Vanguard Group Malvern, Pennsylvania, USA · The Vanguard Group Toronto, Ontario, Canada

  2. Tev: Scaling Long-Horizon Agentic Environments with Dense Progress Rewards

    Oct 13, 2026Wenbo Hu, Shraman Pramanick, Krishna Kumar Singh +3LLM Agent TrainingLong-Horizon LLM Agents

    Adobe Research · University of California, Los Angeles

  3. Disentangling Task-Relevant Latent Dynamics for Reward Modeling

    Oct 9, 2026Dayou Li, Jionghao Wang, Peihao Wang +9Dense Reward RLMultimodal Reward Modeling

    Texas A&M University · University of Texas at Austin · Google DeepMind +1

  4. Before the Rollout Ends: Early Terminal Reward Prediction for Long-horizon Coding Agents

    Sep 25, 2026Jihan Yao, Sihan Zeng, Shangbin Feng +3Long-Horizon Agent EvaluationReward Modeling

    University of Washington · Independent Researcher · HKUST

  5. HiRE: Hindsight Reward Editing for Policy Finetuning

    Sep 22, 2026Haoyi Niu, Zhengtao Han, Yufeng Ji +2Reward Design for RLRobot Policy Learning

    University of California, Berkeley · The Chinese University of Hong Kong

  6. From Connectivity to Rewards: Dense Reward Learning with Directed State Graphs

    Sep 9, 2026Shuyuan Zhang, Zihan Wang, Xiao-Wen Chang +1Reinforcement LearningHierarchical RL

    School of Computer Science, McGill University Mila – Quebec AI Institute · Google DeepMind, CIFAR Fellow

  7. Long-Horizon Language Model Reinforcement Learning via Progressive Point Matching

    Sep 7, 2026Preston Fu, Kevin Frans, Oleh Rybkin +2Dense Reward RLReward Shaping

    UC Berkeley · Carnegie Mellon University

  8. Dense Process Supervision for Search Agents via Fact Utility Estimation

    Sep 1, 2026Rongzhi Zhu, Xiangyu Liu, Yi Liu +7Reinforcement LearningCredit Assignment in RL

    State Key Laboratory for Novel Software Technology, Nanjing University, China · Ant Group, China · National Institute of Healthcare Data Science, Nanjing University, China

  9. Dream2Reward: Transition-Alignment Reward Models from Positive Demonstrations for Robotic Manipulation

    Aug 19, 2026Haoyu Zhang, Zecui Zeng, Bin Wang +3Reward Design for RLDense Reward RL

    School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · JD Explore Academy, China

  10. DHRCL:Training Code LLMs with Dense Hierarchical Rewards and Curriculum Learning

    Jul 29, 2026Shuhang Wang, Ziming Li, Hui ChengRL for Code GenerationReinforcement Learning

  11. The Dark Room in the Reward Channel: Dense Prediction Rewards Collapse GRPO-Trained LLM Agents -- and The Channel, Not the Content, Decides What Works

    Jul 23, 2026Yu WangReward HackingReward Shaping

    Independent researcher

  12. Lyapunov Exponent as Physics-Informed Dense Reward: RL Discovery of Stabilization Beyond the Kapitza Pendulum

    Jul 15, 2026Slava AndrejevRL ControlReinforcement Learning

  13. TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents

    Jul 15, 2026Leitian Tao, Baolin Peng, Wenlin Yao +5Credit Assignment in RLStep-Level Credit Assignment in RL

    University of Wisconsin–Madison · Microsoft Research

  14. Stage-Transition Dense Reward Modeling for Reinforcement Learning

    Jun 30, 2026Yang Yang, Bingjie Chen, Zihan Wang +4RL for RoboticsLong-Horizon Robotic Manipulation

    Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen 518055, China. · Zerith Robotics, Shenzhen 518055, China.

  15. Learning Process Rewards via Success Visitation Matching for Efficient RL

    Jun 22, 2026Raymond Tsao, Andrew Wagenmaker, Sergey LevineProcess Reward ModelsCredit Assignment in RL

    UC Berkeley

  16. RARM: Confidence-Gated Progress Reward Modeling for RL in Manipulation

    Jun 20, 2026Pengzhi Yang, Xinyu Wang, Pengyu Jing +7Process Reward ModelsLong-Horizon Robotic Manipulation

    NUS Human-Centered Robotic Lab · ‡ Booking.com · University of Cambridge +1

  17. Momentum for Reasoning: Dense Intrinsic Signals in Policy Optimization

    Jun 7, 2026Hao Chen, Zhanming Shen, Liyao Li +8Intrinsic Reward MethodsReinforcement Learning

    Zhejiang University · The Chinese University of Hong Kong · Eastern Institute of Technology

  18. Domain-Adaptable Reinforcement Learning for Code Generation with Dense Rewards

    May 20, 2026Erfan Aghadavoodi Jolfaei, Daniel Maninger, Abhinav Anand +2RL for Code GenerationReinforcement Learning

    Technische Universität Darmstadt, Darmstadt, Germany · Hessian Center for Artificial Intelligence (hessian.AI), Darmstadt, Germany · National Research Center for Applied Cybersecurity ATHENE, Darmstadt, Germany.

  19. PokeRL: Reinforcement Learning for Pokemon Red

    Apr 12, 2026Dheeraj Mudireddy, Sai PatibandlaRL for Game AIReinforcement Learning

    Texas A&M Institute of Data Science Texas A&M University College Station, USA · Dept. of Computer Science Texas A&M University College Station, USA

  20. Skip-Connected Policy Optimization for Implicit Advantage

    Apr 9, 2026Fengwei Teng, Jinyi Bai, Xinhao Yao +3RL for Language Model ReasoningReinforcement Learning with Verifiable Rewards

    Mohamed bin Zayed University of Artificial Intelligence · Renmin University of China · Carnegie Mellon University +1

  21. ARM: Advantage Reward Modeling for Long-Horizon Manipulation

    Apr 3, 2026Yiming Mao, Zixi Yu, Weixin Mao +5Reward ModelingLong-Horizon Robotic Manipulation

    LimX Dynamics · Beijing University of Posts and Telecommunications · Zhejiang University

  22. Generalizable Dense Reward for Long-Horizon Robotic Tasks

    Mar 31, 2026Silong Yong, Stephen Sheng, Carl Qi +6Reinforcement LearningLong-Horizon Robotic Manipulation

    Carnegie Mellon University · Amazon Robotics · UT Austin

  23. RewardFlow: Topology-Aware Reward Propagation on State Graphs for Agentic RL with Large Language Models

    Mar 19, 2026Xiao Feng, Bo Han, Zhanke Zhou +5RL for LLM AgentsReward Design for RL

    TMLR Group, Hong Kong Baptist University · TCL Corporate Research (HK) Co., Ltd · Cooperative Medianet Innovation Center, Shanghai Jiao Tong University +1

  24. Rewarding DINO: Predicting Dense Rewards with Vision Foundation Models

    Mar 17, 2026Pierre Krack, Tobias Jülg, Wolfram Burgard +1Reward ModelingReward Design for RL

    Department of Computer Science & Artificial Intelligence, University of Technology Nuremberg, Germany. · TUM School of Computation, Information and Technology, Technical University of Munich, Germany

  25. SQL-ASTRA: Alleviating Sparse Feedback in Agentic SQL via Column-Set Matching and Trajectory Aggregation

    Mar 17, 2026Long Li, Zhijian Zhou, Jiangxuan Long +5Text-to-SQLStep-Level Credit Assignment in RL

    Griffith University, Brisbane, Australia · Fudan University, Shanghai, China · Shanghai Innovation Institute, China +4

  26. Listening to the Echo: User-Reaction Aware Policy Optimization via Scalar-Verbal Hybrid Reinforcement Learning

    Mar 16, 2026Jing Ye, Xinpei Zhao, Lu Xiang +2Emotional Support ConversationEmotion-Aware Dialogue Generation

    State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, CAS, Beijing, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Independent Researcher

  27. Reward-Zero: Language Embedding Driven Implicit Reward Mechanisms for Reinforcement Learning

    Mar 10, 2026Heng Zhang, Haddy Alchaer, Arash Ajoudani +1Reward Design for RLDense Reward RL

    Human-Robot Interfaces and Interaction Lab, Istituto Italiano di Tecnologia, Genova, Italy · Ph.D. program of national interest in Robotics and Intelligent Machines (DRIM) and Universit ‘a di Genova, Genoa, Italy. · Edwardson School of Industrial Engineering, Purdue University, West Lafayette, IN, USA. +2

  28. PRISM: Parallel Reward Integration with Symmetry for MORL

    Feb 20, 2026Finn van der Knaap, Kejiang Qian, Zheng Xu +1Equivariant Policy LearningReinforcement Learning

    University of Edinburgh · Meta Superintelligence Labs.

  29. SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards

    Nov 10, 2025Hunar Batra, Haoqin Tu, Hardy Chen +3Multimodal UnderstandingScene Graph Generation

    University of Oxford · University of California, Santa Cruz