Hindsight Experience Replay

Hindsight Experience Replay (HER) is a reinforcement learning technique designed to improve sample efficiency in tasks with sparse rewards, where successful experiences are rare. Current research focuses on enhancing HER's performance through various strategies, including prioritized experience replay, model-based approaches, and integration with other techniques like curriculum learning and emergent communication, often within the context of robotic manipulation and language model training. These advancements aim to make reinforcement learning more practical by reducing the massive amounts of data typically required for training, leading to more efficient and robust agents in diverse applications.

Papers

October 31, 2024

Maximum Entropy Hindsight Experience Replay
Douglas C. Crowder, Matthew L. Trappett, Darrien M. McKenzie, Frances S. Chance
Reinforcement Learning Proximal Policy Optimization Policy Reinforcement Learning Hindsight Experience Replay

October 29, 2024

Hindsight Experience Replay Accelerates Proximal Policy Optimization
Douglas C. Crowder, Darrien M. McKenzie, Matthew L. Trappett, Frances S. Chance
Proximal Policy Optimization Policy Reinforcement Learning Hindsight Experience Replay

July 30, 2024

Diffusion Augmented Agents: A Framework for Efficient Exploration and Transfer Learning
Norman Di Palo, Leonard Hasenclever, Jan Humplik, Arunkumar Byravan
Reinforcement Learning Vision Language Model Transfer Learning New Framework Efficient Exploration Hindsight Experience Replay Diffusion Based Augmentation

February 7, 2024

CodeIt: Self-Improving Language Models with Prioritized Hindsight Replay
Natasha Butt, Blazej Manczak, Auke Wiggers, Corrado Rainone, David Zhang, Michaël Defferrard, Taco Cohen
Large Language Model Language Model Cross Task Generalization Reasoning Corpus Hindsight Experience Replay GIT Net

December 14, 2023

HiER: Highlight Experience Replay and Easy2Hard Curriculum Learning for Boosting Off-Policy Reinforcement Learning Agents
Dániel Horváth, Jesús Bujalance Martín, Ferenc Gábor Erdős, Zoltán Istenes, Fabien Moutarde
Robotics Domain Action Space Policy Reinforcement Learning Experience Replay Hindsight Experience Replay Initial State Distribution

December 5, 2023

Contact Energy Based Hindsight Experience Prioritization
Erdi Sayar, Zhenshan Bing, Carlo D'Eramo, Ozgur S. Oguz, Alois Knoll
Reinforcement Learning Manipulation Task Sparse Reward Hindsight Experience Replay

October 24, 2023

Learning Agility and Adaptive Legged Locomotion via Curricular Hindsight Reinforcement Learning
Sicen Li, Yiming Pang, Panju Bai, Zhaojin Liu, Jiawei Li, Shihao Hu, Liquan Wang, Gang Wang
Robust Locomotion Curriculum Reinforcement Learning Agile Maneuver Hindsight Experience Replay Legged System

October 3, 2023

Learning and reusing primitive behaviours to improve Hindsight Experience Replay sample efficiency
Francisco Roldan Sanchez, Qiang Wang, David Cordova Bulens, Kevin McGuinness, Stephen Redmond, Noel O'Connor
Reinforcement Learning LeArning Abstract Hindsight Experience Replay Replay Strategy Behavior Primitive

July 28, 2023

ETHER: Aligning Emergent Communication for Hindsight Experience Replay
Kevin Denamganaï, Daniel Hernandez, Ozan Vardal, Sondess Missaoui, James Alfred Walker
Reinforcement Learning Language Conditioned Emergent Communication Emergent Language Hindsight Experience Replay

June 28, 2023

MRHER: Model-based Relay Hindsight Experience Replay for Sequential Object Manipulation Tasks with Sparse Rewards
Yuming Huang, Bin Ren, Ziming Xu, Lianghong Wu
Sparse Reward Goal Conditioned Reinforcement Learning Sequential Manipulation Hindsight Experience Replay

October 24, 2022

AACHER: Assorted Actor-Critic Deep Reinforcement Learning with Hindsight Experience Replay
Adarsh Sehgal, Muskan Sehgal, Hung Manh La
Actor Critic Deep Deterministic Policy Gradient Critic Model Multi Agent Actor Critic Hindsight Experience Replay

September 22, 2022

Minimizing Human Assistance: Augmenting a Single Demonstration for Deep Reinforcement Learning
Abraham George, Alison Bartsch, Amir Barati Farimani
Reinforcement Learning Deep Reinforcement Learning Video Demonstration Deep Deterministic Policy Gradient Human Performance Hindsight Experience Replay 3D Simulator

September 19, 2022

Towards advanced robotic manipulation
Francisco Roldan Sanchez, Stephen Redmond, Kevin McGuinness, Noel O'Connor
Robotic Manipulation Real World Environment Real World Application Hindsight Experience Replay Goal Generation

August 31, 2022

Cluster-based Sampling in Hindsight Experience Replay for Robotic Tasks (Student Abstract)
Taeyoung Kim, Dongsoo Har
Sparse Reward Robotic Task Experience Replay Sampling Strategy Hindsight Experience Replay Multi Goal Reinforcement Learning Based Sampling

August 1, 2022

Relay Hindsight Experience Replay: Self-Guided Continual Reinforcement Learning for Sequential Object Manipulation Tasks with Sparse Rewards
Yongle Luo, Yuxin Wang, Kun Dong, Qiang Zhang, Erkang Cheng, Zhiyong Sun, Bo Song
Reinforcement Learning Sparse Reward Continual Reinforcement Learning Guided Exploration Sequential Manipulation Hindsight Experience Replay Unsupervised Exploration

July 3, 2022

USHER: Unbiased Sampling for Hindsight Experience Replay
Liam Schramm, Yunfu Deng, Edgar Granados, Abdeslam Boularias
Reinforcement Learning High Dimensional Sparse Reward Importance Sampling Hindsight Experience Replay Unbiased Sampling

April 7, 2022

Automatic Parameter Optimization Using Genetic Algorithm in Deep Reinforcement Learning for Robotic Manipulation Tasks
Adarsh Sehgal, Nicholas Ward, Hung La, Sushil Louis
Reinforcement Learning Deep Reinforcement Learning Genetic Algorithm Manipulation Task Deep Deterministic Policy Gradient Learning Agent Hindsight Experience Replay Parameter Optimization

February 28, 2022

GA+DDPG+HER: Genetic Algorithm-Based Function Optimizer in Deep Reinforcement Learning for Robotic Manipulation Tasks
Adarsh Sehgal, Nicholas Ward, Hung Manh La, Christos Papachristos, Sushil Louis
Reinforcement Learning Deep Reinforcement Learning Genetic Algorithm Manipulation Task Deep Deterministic Policy Gradient Hindsight Experience Replay

December 20, 2021

Proving Theorems using Incremental Learning and Hindsight Experience Replay
Eser Aygün, Laurent Orseau, Ankit Anand, Xavier Glorot, Vlad Firoiu, Lei M. Zhang, Doina Precup, Shibl Mourad
Incremental Learning Theorem Proving First Order Logic Theorem Provers Hindsight Experience Replay Conflict Driven Clause Learning

Hindsight Experience Replay

Papers

Maximum Entropy Hindsight Experience Replay

Hindsight Experience Replay Accelerates Proximal Policy Optimization

Diffusion Augmented Agents: A Framework for Efficient Exploration and Transfer Learning

CodeIt: Self-Improving Language Models with Prioritized Hindsight Replay

HiER: Highlight Experience Replay and Easy2Hard Curriculum Learning for Boosting Off-Policy Reinforcement Learning Agents

Contact Energy Based Hindsight Experience Prioritization

Learning Agility and Adaptive Legged Locomotion via Curricular Hindsight Reinforcement Learning

Learning and reusing primitive behaviours to improve Hindsight Experience Replay sample efficiency

ETHER: Aligning Emergent Communication for Hindsight Experience Replay

MRHER: Model-based Relay Hindsight Experience Replay for Sequential Object Manipulation Tasks with Sparse Rewards

AACHER: Assorted Actor-Critic Deep Reinforcement Learning with Hindsight Experience Replay

Minimizing Human Assistance: Augmenting a Single Demonstration for Deep Reinforcement Learning

Towards advanced robotic manipulation

Cluster-based Sampling in Hindsight Experience Replay for Robotic Tasks (Student Abstract)

Relay Hindsight Experience Replay: Self-Guided Continual Reinforcement Learning for Sequential Object Manipulation Tasks with Sparse Rewards

USHER: Unbiased Sampling for Hindsight Experience Replay

Automatic Parameter Optimization Using Genetic Algorithm in Deep Reinforcement Learning for Robotic Manipulation Tasks

GA+DDPG+HER: Genetic Algorithm-Based Function Optimizer in Deep Reinforcement Learning for Robotic Manipulation Tasks

Proving Theorems using Incremental Learning and Hindsight Experience Replay