Sparse Reward Environment

Sparse reward environments, characterized by infrequent or delayed feedback signals, pose a significant challenge for reinforcement learning (RL) agents. Current research focuses on improving exploration strategies through intrinsic rewards (e.g., novelty-based methods, information-theoretic approaches), hierarchical RL architectures to decompose complex tasks, and leveraging auxiliary models like vision-language models to provide richer reward signals. These advancements aim to enhance sample efficiency and enable RL agents to solve complex, real-world problems where dense reward functions are impractical or impossible to design, impacting fields like robotics and human-computer interaction.

Papers

October 27, 2024

Efficient Diversity-based Experience Replay for Deep Reinforcement Learning
Kaiyan Zhao, Yiming Wang, Yuyang Chen, Xiaoguang Niu, Yan Li, Leong Hou U
LeArning Abstract Deep Reinforcement Learning High Efficiency Sparse Reward Deep Reinforcement Experience Replay Sparse Reward Environment

October 16, 2024

Potential-Based Intrinsic Motivation: Preserving Optimality With Complex, Non-Markovian Shaping Rewards
Grant C. Forbes, Leonardo Villalobos-Arias, Jianxun Wang, Arnav Jhala, David L. Roberts
Near Optimality Reward Shaping Sparse Reward Environment Complexity Level Intrinsic Motivation

October 15, 2024

DIAR: Diffusion-model-guided Implicit Q-learning with Adaptive Revaluation
Jaehyun Park, Yunho Kim, Sejin Kim, Byung-Jun Lee, Sundong Kim
Offline Reinforcement Learning Adaptive Importance Sparse Reward Environment Adaptive Decision Making Implicit Q Learning

October 7, 2024

ETGL-DDPG: A Deep Deterministic Policy Gradient Algorithm for Sparse Reward Continuous Control
Ehsan Futuhi, Shayan Karimi, Chao Gao, Martin Müller
Sparse Reward Deep Deterministic Policy Gradient Sparse Reward Environment Replay Buffer Sparse Reward Continuous Control

September 24, 2024

The Dark Side of Rich Rewards: Understanding and Mitigating Noise in VLM Rewards
Sukai Huang, Nir Lipovetzky, Trevor Cohn
Reward Model Sparse Reward Environment Robust Reward Auxiliary Reward

September 21, 2024

R-AIF: Solving Sparse-Reward Robotic Tasks from Pixels with Active Inference and World Models
Viet Dung Nguyen, Zhizhuo Yang, Christopher L. Buckley, Alexander Ororbia
Markov Decision Process World Model Tetromino Pixel Active Inference Partially Observable Markov Decision Process Sparse Reward Environment

September 6, 2024

Sparse Rewards Can Self-Train Dialogue Agents
Barrett Martin Lattimer, Varun Gangal, Ryan McDonald, Yi Yang
Large Language Model Human Feedback Sparse Reward Multi Turn Dialogue Sparse Reward Environment

July 31, 2024

Black box meta-learning intrinsic rewards for sparse-reward environments
Octavio Pappalardo, Rodrigo Ramele, Juan Miguel Santos
Meta Learning Black Box Intrinsic Reward Sparse Reward Environment Meta Gradient

June 12, 2024

When Do Skills Help Reinforcement Learning? A Theoretical Analysis of Temporal Abstractions
Zhening Li, Gabriel Poesia, Armando Solar-Lezama
Reinforcement Learning Theoretical Analysis Sparse Reward Environment Robust Skill Temporal Abstraction Parameterized Skill

June 11, 2024

World Models with Hints of Large Language Models for Goal Achieving
Zeyuan Liu, Ziyu Huan, Xiyao Wang, Jiafei Lyu, Jian Tao, Xiu Li, Furong Huang, Huazhe Xu
Reinforcement Learning World Model Intrinsic Reward Sparse Reward Environment Natural Language Hint Visual Goal Long Horizon Decision Making

May 28, 2024

Individual Contributions as Intrinsic Exploration Scaffolds for Multi-agent Reinforcement Learning
Xinran Li, Zifan Liu, Shibo Chen, Jun Zhang
Multi Agent Reinforcement Learning Intrinsic Reward Client Contribution Sparse Reward Environment Intrinsic Exploration

May 23, 2024

Adaptive Teaching in Heterogeneous Agents: Balancing Surprise in Sparse Reward Scenarios
Emma Clark, Kanghyun Ryu, Negar Mehr
Sparse Reward Efficient Learning Textual Demonstration Heterogeneous Agent Sparse Reward Environment Agent Capability Surprise Bound Adaptive Teaching Incentivized Exploration

April 20, 2024

PIPER: Primitive-Informed Preference-based Hierarchical Reinforcement Learning via Hindsight Relabeling
Utsav Singh, Wesley A. Suttle, Brian M. Sadler, Vinay P. Namboodiri, Amrit Singh Bedi
Sparse Reward Hierarchical Reinforcement Learning Sparse Reward Environment Hindsight Goal Relabeling Pose Invariant Face Recognition

April 19, 2024

Random Network Distillation Based Deep Reinforcement Learning for AGV Path Planning
Huilin Yin, Shengkai Su, Yinjia Lin, Pengju Zhen, Karin Festl, Daniel Watzenig
Deep Reinforcement Learning Sparse Reward Environment Guided Vehicle Random Network Distillation

February 12, 2024

Potential-Based Reward Shaping For Intrinsic Motivation
Grant C. Forbes, Nitish Gupta, Leonardo Villalobos-Arias, Colin M. Potts, Arnav Jhala, David L. Roberts
Reward Shaping Sparse Reward Environment Intrinsic Motivation

February 7, 2024

Code as Reward: Empowering Reinforcement Learning with VLMs
David Venuto, Sami Nur Islam, Martin Klissarov, Doina Precup, Sherry Yang, Ankit Anand
Reinforcement Learning Vision Language Model Real World Code Reward Report Sparse Reward Environment

February 6, 2024

TopoNav: Topological Navigation for Efficient Exploration in Sparse Reward Environments
Jumman Hossain, Abu-Zaher Faridee, Nirmalya Roy, Jade Freeman, Timothy Gregory, Theron T. Trout
Obstacle Avoidance Autonomous Robot Hierarchical Reinforcement Learning Efficient Exploration Sparse Reward Environment Goal Conditioned Exploration

January 25, 2024

Sample Efficient Reinforcement Learning by Automatically Learning to Compose Subtasks
Shuai Han, Mehdi Dastani, Shihan Wang
Reinforcement Learning Machine Learning Reward Function Sparse Reward Environment Sample Efficient Reinforcement Learning Reward Structure

December 30, 2023

Policy Optimization with Smooth Guidance Learned from State-Only Demonstrations
Guojian Wang, Faguo Wu, Xiao Zhang, Tianyuan Chen, Zhiming Zheng
Reward Function Policy OpTimization Sparse Reward Environment

December 10, 2023

Efficient Sparse-Reward Goal-Conditioned Reinforcement Learning with a High Replay Ratio and Regularization
Takuya Hiraoka
Reinforcement Learning Sparse Reward Environment Double Q Learning Dense Reward

Sparse Reward Environment

Papers

Efficient Diversity-based Experience Replay for Deep Reinforcement Learning

Potential-Based Intrinsic Motivation: Preserving Optimality With Complex, Non-Markovian Shaping Rewards

DIAR: Diffusion-model-guided Implicit Q-learning with Adaptive Revaluation

ETGL-DDPG: A Deep Deterministic Policy Gradient Algorithm for Sparse Reward Continuous Control

The Dark Side of Rich Rewards: Understanding and Mitigating Noise in VLM Rewards

R-AIF: Solving Sparse-Reward Robotic Tasks from Pixels with Active Inference and World Models

Sparse Rewards Can Self-Train Dialogue Agents

Black box meta-learning intrinsic rewards for sparse-reward environments

When Do Skills Help Reinforcement Learning? A Theoretical Analysis of Temporal Abstractions

World Models with Hints of Large Language Models for Goal Achieving

Individual Contributions as Intrinsic Exploration Scaffolds for Multi-agent Reinforcement Learning

Adaptive Teaching in Heterogeneous Agents: Balancing Surprise in Sparse Reward Scenarios

PIPER: Primitive-Informed Preference-based Hierarchical Reinforcement Learning via Hindsight Relabeling

Random Network Distillation Based Deep Reinforcement Learning for AGV Path Planning

Potential-Based Reward Shaping For Intrinsic Motivation

Code as Reward: Empowering Reinforcement Learning with VLMs

TopoNav: Topological Navigation for Efficient Exploration in Sparse Reward Environments

Sample Efficient Reinforcement Learning by Automatically Learning to Compose Subtasks

Policy Optimization with Smooth Guidance Learned from State-Only Demonstrations

Efficient Sparse-Reward Goal-Conditioned Reinforcement Learning with a High Replay Ratio and Regularization