Reward Redistribution

Reward redistribution in reinforcement learning addresses the challenge of assigning credit for actions taken in environments with delayed or sparse rewards, improving learning efficiency and interpretability. Current research focuses on developing algorithms that decompose overall trajectory rewards into per-step signals, often employing techniques like least-squares methods, causal inference, and attention mechanisms to achieve this. These methods aim to create more informative reward signals for training agents, leading to better performance in various applications, particularly in long-horizon tasks and multi-agent settings. The resulting improvements in agent training efficiency and the enhanced interpretability of learned policies are significant advancements for the field.

Papers

November 13, 2024

R3HF: Reward Redistribution for Enhancing Reinforcement Learning from Human Feedback
Jiahui Li, Tai-wei Chang, Fengda Zhang, Kun Kuang, Long Chen
Large Language Model Reinforcement Learning Human Feedback Reward Prediction Reward Redistribution

September 5, 2024

ELO-Rated Sequence Rewards: Advancing Reinforcement Learning Models
Qi Ju, Falin Hei, Zhemei Fang, Yunfeng Luo
Reinforcement Learning Deep Learning Reinforcement Learning Model Utility Based Learning Reward Reward Redistribution

May 28, 2023

Interpretable Reward Redistribution in Reinforcement Learning: A Causal Approach
Yudi Zhang, Yali Du, Biwei Huang, Ziyan Wang, Jun Wang, Meng Fang, Mykola Pechenizkiy
Reinforcement Learning Causal Approach Markovian Reward Reward Redistribution

June 13, 2022

Provably Efficient Offline Reinforcement Learning with Trajectory-Wise Reward
Tengyu Xu, Yue Wang, Shaofeng Zou, Yingbin Liang
Offline Reinforcement Learning Proxy Reward Trajectory Wise Reward rEward Decomposition Reward Redistribution

January 12, 2022

Agent-Temporal Attention for Reward Redistribution in Episodic Multi-Agent Reinforcement Learning
Baicen Xiao, Bhaskar Ramasubramanian, Radha Poovendran
Multi Agent Reinforcement Learning Temporal Attention Reward Redistribution Episodic Reward

November 26, 2021

Learning Long-Term Reward Redistribution via Randomized Return Decomposition
Zhizhou Ren, Ruihan Guo, Yuan Zhou, Jian Peng
Reinforcement Learning Episodic Reinforcement Learning Proxy Reward Reward Redistribution Episodic Reward Return Decomposition