Return Decomposition

Return decomposition in reinforcement learning addresses the challenge of assigning credit to individual actions within episodes where rewards are only received at the end. Current research focuses on developing algorithms that effectively decompose the final reward into step-wise proxy rewards, often employing techniques like attention mechanisms or Shapley value estimations to handle complex temporal and multi-agent interactions. These methods aim to improve sample efficiency and learning performance in scenarios with delayed or sparse rewards, particularly relevant for real-world applications with long horizons. Improved return decomposition promises to enhance the applicability of reinforcement learning to complex, real-world problems.

Papers

December 17, 2023

Episodic Return Decomposition by Difference of Implicitly Assigned Sub-Trajectory Reward
Haoxin Lin, Hongqiu Wu, Jiaji Zhang, Yihao Sun, Junyin Ye, Yang Yu
Qualitative Difference Proxy Reward Trajectory Wise Reward Episodic Reward Return Decomposition

April 15, 2023

STAS: Spatial-Temporal Return Decomposition for Multi-agent Reinforcement Learning
Sirui Chen, Zhaowei Zhang, Yaodong Yang, Yali Du
Multi Agent Reinforcement Learning Temporal Attention Episodic Reinforcement Learning Temporal Credit Assignment Global Reward Return Decomposition

November 26, 2021

Learning Long-Term Reward Redistribution via Randomized Return Decomposition
Zhizhou Ren, Ruihan Guo, Yuan Zhou, Jian Peng
Reinforcement Learning Episodic Reinforcement Learning Proxy Reward Reward Redistribution Episodic Reward Return Decomposition

Return Decomposition

Papers

Episodic Return Decomposition by Difference of Implicitly Assigned Sub-Trajectory Reward

STAS: Spatial-Temporal Return Decomposition for Multi-agent Reinforcement Learning

Learning Long-Term Reward Redistribution via Randomized Return Decomposition