Sparse Reward

Sparse reward reinforcement learning tackles the challenge of training agents in environments where positive feedback is infrequent, hindering efficient learning. Current research focuses on improving exploration strategies through techniques like optimistic Thompson sampling and intrinsic reward shaping, often employing deep deterministic policy gradients (DDPG), transformers, and generative flow networks (GFlowNets) to address the problem. These advancements aim to enhance sample efficiency and improve the performance of reinforcement learning agents in complex, real-world scenarios characterized by sparse rewards, such as robotics and multi-agent systems. The resulting improvements in sample efficiency and robustness have significant implications for various applications, including robotics, personalized recommendations, and human-AI collaboration.

Papers

November 3, 2024

Learning World Models for Unconstrained Goal Navigation
Yuanlin Duan, Wensen Mao, He Zhu
Reinforcement Learning World Model Sparse Reward Navigation Task Active Exploration Exploration Efficiency Goal Exploration

October 30, 2024

Online Intrinsic Rewards for Decision Making Agents from Large Language Model Feedback
Qinqing Zheng, Mikael Henaff, Amy Zhang, Aditya Grover, Brandon Amos
Reinforcement Learning Agent Smith Sparse Reward Large Language Online Service Intrinsic Reward Model Improvement

October 27, 2024

Efficient Diversity-based Experience Replay for Deep Reinforcement Learning
Kaiyan Zhao, Yiming Wang, Yuyang Chen, Xiaoguang Niu, Yan Li, Leong Hou U
LeArning Abstract Deep Reinforcement Learning High Efficiency Sparse Reward Deep Reinforcement Experience Replay Sparse Reward Environment

October 25, 2024

On-Robot Reinforcement Learning with Goal-Contrastive Rewards
Ondrej Biza, Thomas Weng, Lingfeng Sun, Karl Schmeckpeper, Tarik Kelestemur, Yecheng Jason Ma, Robert Platt, Jan-Willem van de Meent, Lawson L. S. Wong
Reinforcement Learning Sparse Reward Robot Policy Learning Contrastive Reward

October 19, 2024

October 7, 2024

October 4, 2024

Predictive Coding for Decision Transformer
Tung M. Luu, Donghoon Lee, Chang D. Yoo
Offline Reinforcement Learning Policy Learning Sparse Reward Decision Transformer Predictive Coding Goal Conditioned Reinforcement Learning

September 13, 2024

Quasimetric Value Functions with Dense Rewards
Khadichabonu Valieva, Bikramjit Banerjee
Sparse Reward Dense Reward Quasimetric Learning

September 7, 2024

LMGT: Optimizing Exploration-Exploitation Balance in Reinforcement Learning through Language Model Guided Trade-offs
Yongxin Deng, Xihe Qiu, Xiaoyu Tan, Wei Chu, Yinghui Xu
Reinforcement Learning Sparse Reward Industrial Recommender System Exploration Exploitation

September 6, 2024

Sparse Rewards Can Self-Train Dialogue Agents
Barrett Martin Lattimer, Varun Gangal, Ryan McDonald, Yi Yang
Large Language Model Human Feedback Sparse Reward Multi Turn Dialogue Sparse Reward Environment

August 27, 2024

What makes math problems hard for reinforcement learning: a case study
Ali Shehper, Anibal M. Medina-Mardones, Bartłomiej Lewandowski, Angus Gruen, Piotr Kucharski, Sergei Gukov
Reinforcement Learning Case Study Sparse Reward Math Problem Conjecture Refuting Algorithm Long Standing Conjecture

August 15, 2024

IReCa: Intrinsic Reward-enhanced Context-aware Reinforcement Learning for Human-AI Coordination
Xin Hao, Bahareh Nakisa, Mohmmad Naim Rastgoo, Richard Dazeley
Reinforcement Learning Algorithm Sparse Reward Intrinsic Reward Human AI Coordination

August 6, 2024

Highly Efficient Self-Adaptive Reward Shaping for Reinforcement Learning
Haozhe Ma, Zhengding Luo, Thanh Vinh Vo, Kuankuan Sima, Tze-Yun Leong
Reinforcement Learning Sparse Reward Reward Shaping Reward Signal Self Adaptation

July 16, 2024

RobotKeyframing: Learning Locomotion with High-Level Objectives via Mixture of Dense and Sparse Rewards
Fatemeh Zargarbashi, Jin Cheng, Dongho Kang, Robert Sumner, Stelian Coros
Legged Robot Learning Based Sparse Reward Human Like Locomotion Behavior Locomotion Skill

June 29, 2024

Revisiting Sparse Rewards for Goal-Reaching Reinforcement Learning
Gautham Vasan, Yan Wang, Fahim Shahriar, James Bergstra, Martin Jagersand, A. Rupam Mahmood
Robot Learning Sparse Reward Episodic Reinforcement Learning Conditioned Policy Goal Oriented Reinforcement Learning

June 25, 2024

EXTRACT: Efficient Policy Learning by Extracting Transferable Robot Skills from Offline Data
Jesse Zhang, Minho Heo, Zuxin Liu, Erdem Biyik, Joseph J Lim, Yao Liu, Rasool Fakoor
Action Space Sparse Reward Offline Data Efficient Policy Learning Parameterized Skill Robotic Skill Skill Based Reinforcement Learning

June 3, 2024

Looking Backward: Retrospective Backward Synthesis for Goal-Conditioned GFlowNets
Haoran He, Can Chang, Huazhe Xu, Ling Pan
Sparse Reward Generative Flow Network Forward Pas Goal Conditioned GFlowNets

June 2, 2024

FuRL: Visual-Language Models as Fuzzy Rewards for Reinforcement Learning
Yuwei Fu, Haichao Zhang, Di Wu, Wei Xu, Benoit Boulet
Reinforcement Learning Vision Language Visual Language Model Sparse Reward Online Reinforcement Learning