Hierarchical Reinforcement Learning

Hierarchical Reinforcement Learning (HRL) addresses the challenge of training agents to solve complex, long-horizon tasks by decomposing them into simpler sub-tasks managed by a hierarchy of policies. Current research focuses on improving HRL's efficiency and robustness through techniques like subgoal generation, option frameworks, and the integration of large language models and other AI methods to guide learning and improve decision-making. This approach is proving valuable in diverse applications, including robotics, recommendation systems, and even high-frequency trading, by enabling more efficient learning and better generalization to unseen situations. The development of more efficient and theoretically grounded HRL algorithms is a significant area of ongoing research.

Papers

June 28, 2023

Action and Trajectory Planning for Urban Autonomous Driving with Hierarchical Reinforcement Learning
Xinyang Lu, Flint Xiaofeng Fan, Tianying Wang
Reinforcement Learning Autonomous Vehicle Trajectory Planning Hierarchical Reinforcement Learning Action Feature Urban Driving Urban Autonomous Driving

June 20, 2023

Int-HRL: Towards Intention-based Hierarchical Reinforcement Learning
Anna Penzkofer, Simon Schaefer, Florian Strohm, Mihai Bâce, Stefan Leutenegger, Andreas Bulling
Deep Reinforcement Learning Hierarchical Reinforcement Learning Sub Goal

June 14, 2023

Skill-Critic: Refining Learned Skills for Reinforcement Learning
Ce Hao, Catherine Weaver, Chen Tang, Kenta Kawamoto, Masayoshi Tomizuka, Wei Zhan
Reinforcement Learning Hierarchical Reinforcement Learning Sparse Reward Environment Robust Skill Refining Method Policy Fine Tuning Skill Mention

June 10, 2023

PEAR: Primitive enabled Adaptive Relabeling for boosting Hierarchical Reinforcement Learning
Utsav Singh, Vinay P. Namboodiri
Reinforcement Learning Policy Reinforcement Learning Hierarchical Reinforcement Learning Hierarchical Agent Negative Primitive

June 2, 2023

Hierarchical Reinforcement Learning for Modeling User Novelty-Seeking Intent in Recommender Systems
Pan Li, Yuyan Wang, Ed H. Chi, Minmin Chen
Recommender System Hierarchical Reinforcement Learning Human Intent USER Provided Name Novelty Search Novelty Reaction Intent Taxonomy

May 26, 2023

A Hierarchical Approach to Population Training for Human-AI Collaboration
Yi Loo, Chen Gong, Malika Meghjani
Deep Reinforcement Learning Human Ai Collaboration Hierarchical Reinforcement Learning Hierarchical Model Best Response Population Based Training

May 10, 2023

An Option-Dependent Analysis of Regret Minimization Algorithms in Finite-Horizon Semi-Markov Decision Processes
Gianluca Drappo, Alberto Maria Metelli, Marcello Restelli
Hierarchical Reinforcement Learning Finite Horizon Real World Reinforcement Learning Planning Horizon Semi Markov Decision Process Path Dependent Option

April 19, 2023

H-TSP: Hierarchically Solving the Large-Scale Travelling Salesman Problem
Xuanhao Pan, Yan Jin, Yuandong Ding, Mingxiao Feng, Li Zhao, Lei Song, Jiang Bian
Hierarchical Reinforcement Learning Salesman Problem Deep Reinforcement Learning Approach Traveling Salesman Problem

April 14, 2023

Learning to Learn Group Alignment: A Self-Tuning Credo Framework with Multiagent Teams
David Radke, Kyle Tilbury
LeArning Abstract Multi Agent Hierarchical Reinforcement Learning Auto Tuning Agent Team Incentive Security Social Alignment

April 7, 2023

CRISP: Curriculum Inducing Primitive Informed Subgoal Prediction for Hierarchical Reinforcement Learning
Utsav Singh, Vinay P. Namboodiri
Reinforcement Learning Imitation Learning Hierarchical Reinforcement Learning Robotic Task Proximal Curriculum Maze Navigation Subgoal Search

February 28, 2023

Hierarchical Reinforcement Learning in Complex 3D Environments
Bernardo Avila Pires, Feryal Behbahani, Hubert Soyer, Kyriacos Nikiforou, Thomas Keck, Satinder Singh
Hierarchical Reinforcement Learning Complex Environment 3D Environment REinforcement Learning Hierarchical Agent

February 21, 2023

Handling Long and Richly Constrained Tasks through Constrained Hierarchical Reinforcement Learning
Yuxiao Lu, Arunesh Sinha, Pradeep Varakantham
Reinforcement Learning Hierarchical Reinforcement Learning Safety Constraint Long Span

December 24, 2022

SHIRO: Soft Hierarchical Reinforcement Learning
Kandai Watanabe, Mathew Strong, Omer Eldar
Hierarchical Reinforcement Learning Efficient Exploration REinforcement Learning Robotics Benchmark

December 14, 2022

Explaining Agent's Decision-making in a Hierarchical Reinforcement Learning Scenario
Hugo Muñoz, Ernesto Portugal, Angel Ayala, Bruno Fernandes, Francisco Cruz
Reinforcement Learning Decision Making Hierarchical Reinforcement Learning Episodic Memory Explainable Reinforcement Learning Memory Based

November 17, 2022

Planning Irregular Object Packing via Hierarchical Reinforcement Learning
Sichao Huang, Ziwei Wang, Jie Zhou, Jiwen Lu
Hierarchical Reinforcement Learning REinforcement Learning Online Packing Irregular Shape Packing

November 11, 2022

Emergency action termination for immediate reaction in hierarchical reinforcement learning
Michał Bortkiewicz, Jakub Łyskawa, Paweł Wawrzyński, Mateusz Ostaszewski, Artur Grudkowski, Tomasz Trzciński
Reinforcement Learning Large Scale Hierarchical Reinforcement Learning Early Termination Hierarchical Decomposition

October 19, 2022

Hierarchical Reinforcement Learning for Furniture Layout in Virtual Indoor Scenes
Xinhan Di, Pengqian Yu
Hierarchical Reinforcement Learning Indoor Dataset Furniture Layout

October 13, 2022

Causality-driven Hierarchical Structure Discovery for Reinforcement Learning
Shaohui Peng, Xing Hu, Rui Zhang, Ke Tang, Jiaming Guo, Qi Yi, Ruizhi Chen, Xishan Zhang, Zidong Du, Ling Li, Qi Guo, Yunji Chen
Reinforcement Learning Causal Discovery Hierarchical Reinforcement Learning Random Exploration

October 11, 2022

DHRL: A Graph-Based Approach for Long-Horizon and Sparse Hierarchical Reinforcement Learning
Seungjae Lee, Jigang Kim, Inkyu Jang, H. Jin Kim
Reinforcement Learning Long Term Hierarchical Reinforcement Learning Graph Based Approach Temporal Abstraction Efficient Locomotion

October 3, 2022

Hierarchical reinforcement learning for in-hand robotic manipulation using Davenport chained rotations
Francisco Roldan Sanchez, Qiang Wang, David Cordova Bulens, Kevin McGuinness, Stephen Redmond, Noel O'Connor
Reinforcement Learning Manipulation Task Hierarchical Reinforcement Learning Rotation Angle 3D Rotation Robotic in Hand Manipulation

Hierarchical Reinforcement Learning

Papers

Action and Trajectory Planning for Urban Autonomous Driving with Hierarchical Reinforcement Learning

Int-HRL: Towards Intention-based Hierarchical Reinforcement Learning

Skill-Critic: Refining Learned Skills for Reinforcement Learning

PEAR: Primitive enabled Adaptive Relabeling for boosting Hierarchical Reinforcement Learning

Hierarchical Reinforcement Learning for Modeling User Novelty-Seeking Intent in Recommender Systems

A Hierarchical Approach to Population Training for Human-AI Collaboration

An Option-Dependent Analysis of Regret Minimization Algorithms in Finite-Horizon Semi-Markov Decision Processes

H-TSP: Hierarchically Solving the Large-Scale Travelling Salesman Problem

Learning to Learn Group Alignment: A Self-Tuning Credo Framework with Multiagent Teams

CRISP: Curriculum Inducing Primitive Informed Subgoal Prediction for Hierarchical Reinforcement Learning

Hierarchical Reinforcement Learning in Complex 3D Environments

Handling Long and Richly Constrained Tasks through Constrained Hierarchical Reinforcement Learning

SHIRO: Soft Hierarchical Reinforcement Learning

Explaining Agent's Decision-making in a Hierarchical Reinforcement Learning Scenario

Planning Irregular Object Packing via Hierarchical Reinforcement Learning

Emergency action termination for immediate reaction in hierarchical reinforcement learning

Hierarchical Reinforcement Learning for Furniture Layout in Virtual Indoor Scenes

Causality-driven Hierarchical Structure Discovery for Reinforcement Learning

DHRL: A Graph-Based Approach for Long-Horizon and Sparse Hierarchical Reinforcement Learning

Hierarchical reinforcement learning for in-hand robotic manipulation using Davenport chained rotations