Hierarchical Reinforcement Learning

Hierarchical Reinforcement Learning (HRL) addresses the challenge of training agents to solve complex, long-horizon tasks by decomposing them into simpler sub-tasks managed by a hierarchy of policies. Current research focuses on improving HRL's efficiency and robustness through techniques like subgoal generation, option frameworks, and the integration of large language models and other AI methods to guide learning and improve decision-making. This approach is proving valuable in diverse applications, including robotics, recommendation systems, and even high-frequency trading, by enabling more efficient learning and better generalization to unseen situations. The development of more efficient and theoretically grounded HRL algorithms is a significant area of ongoing research.

Papers

June 26, 2024

Bidirectional-Reachable Hierarchical Reinforcement Learning with Mutually Responsive Policies
Yu Luo, Fuchun Sun, Tianying Ji, Xianyuan Zhan
Hierarchical Reinforcement Learning Diverse Policy Hierarchical Policy Learning Goal Reachability Exploration Characteristic

June 24, 2024

June 21, 2024

A Provably Efficient Option-Based Algorithm for both High-Level and Low-Level Learning
Gianluca Drappo, Alberto Maria Metelli, Marcello Restelli
High Efficiency Practical Algorithm Hierarchical Reinforcement Learning Finite Horizon High Level Semi Markov Decision Process Policy Selection

June 20, 2024

MacroHFT: Memory Augmented Context-aware Reinforcement Learning On High Frequency Trading
Chuqiao Zong, Chaojie Wang, Molei Qin, Lei Feng, Xinrun Wang, Bo An
Reinforcement Learning Hierarchical Reinforcement Learning High Frequency Algorithmic Trading

June 16, 2024

DIPPER: Direct Preference Optimization to Accelerate Primitive-Enabled Hierarchical Reinforcement Learning
Utsav Singh, Souradip Chakraborty, Wesley A. Suttle, Brian M. Sadler, Vinay P Namboodiri, Amrit Singh Bedi
Reinforcement Learning Direct Preference Optimization Hierarchical Reinforcement Learning Paraphrase Generation

June 12, 2024

Hierarchical Reinforcement Learning for Swarm Confrontation with High Uncertainty
Qizhen Wu, Kexin Liu, Lei Chen, Jinhu Lü
Deep Reinforcement Learning Hierarchical Reinforcement Learning Swarm Robotics Pursuit Evasion Game Swarm Confrontation Human Uncertainty

June 11, 2024

RACon: Retrieval-Augmented Simulated Character Locomotion Control
Yuxuan Mu, Shihao Zou, Kangning Yin, Zheng Tian, Li Cheng, Weinan Zhang, Jun Wang
Hierarchical Reinforcement Learning Virtual Character Dynamic Locomotion Whole Body Locomotion Motion Controller

June 9, 2024

LGR2: Language Guided Reward Relabeling for Accelerating Hierarchical Reinforcement Learning
Utsav Singh, Pramit Bhattacharyya, Vinay P. Namboodiri
Hierarchical Reinforcement Learning Complex Robotic Task Language Reward Low Level Sparse Reward Continuous Control

June 1, 2024

Exploring the limits of Hierarchical World Models in Reinforcement Learning
Robin Schiewer, Anand Subramoney, Laurenz Wiskott
Reinforcement Learning Continuum Limit World Model Hierarchical Reinforcement Learning Goal Conditioned Reinforcement Learning

May 30, 2024

From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems
Jianliang He, Siyu Chen, Fengzhuo Zhang, Zhuoran Yang
Word List Hierarchical Reinforcement Learning Sublinear Regret Observable Markov Decision Process Past Action Theoretical Foundation Greedy Exploration

April 20, 2024

PIPER: Primitive-Informed Preference-based Hierarchical Reinforcement Learning via Hindsight Relabeling
Utsav Singh, Wesley A. Suttle, Brian M. Sadler, Vinay P. Namboodiri, Amrit Singh Bedi
Sparse Reward Hierarchical Reinforcement Learning Sparse Reward Environment Hindsight Goal Relabeling Pose Invariant Face Recognition

March 24, 2024

MQE: Unleashing the Power of Interaction with Multi-agent Quadruped Environment
Ziyan Xiong, Bo Chen, Shiyu Huang, Wei-Wei Tu, Zhaofeng He, Yang Gao
Deep Reinforcement Learning Multi Agent Reinforcement Learning Real Power Hierarchical Reinforcement Learning Interaction Generation Quadruped Robot

March 21, 2024

Learning to Change: Choreographing Mixed Traffic Through Lateral Control and Hierarchical Reinforcement Learning
Dawei Wang, Weizi Li, Lei Zhu, Jia Pan
LeArning Abstract Hierarchical Reinforcement Learning Human Driven Vehicle Mixed Traffic Future Traffic Lateral Control Mixed Traffic Control

March 16, 2024

Reinforcement Learning with Options and State Representation
Ayoub Ghriss, Masashi Sugiyama, Alessandro Lazaric
Reinforcement Learning Markov Decision Process Hierarchical Reinforcement Learning Complex Environment Continuous Option Hierarchical Policy Learning

March 4, 2024

SMAUG: A Sliding Multidimensional Task Window-Based MARL Framework for Adaptive Real-Time Subtask Recognition
Wenjing Zhang, Wei Zhang
Multi Agent Reinforcement Learning Hierarchical Reinforcement Learning Multi Agent Reinforcement Subtask Assignment

February 22, 2024

MENTOR: Guiding Hierarchical Reinforcement Learning with Human Feedback and Dynamic Distance Constraint
Xinglin Zhou, Yifu Yuan, Shaofu Yang, Jianye Hao
Human Feedback Sparse Reward Hierarchical Reinforcement Learning

February 6, 2024

TopoNav: Topological Navigation for Efficient Exploration in Sparse Reward Environments
Jumman Hossain, Abu-Zaher Faridee, Nirmalya Roy, Jade Freeman, Timothy Gregory, Theron T. Trout
Obstacle Avoidance Autonomous Robot Hierarchical Reinforcement Learning Efficient Exploration Sparse Reward Environment Goal Conditioned Exploration

February 3, 2024

Emergency Computing: An Adaptive Collaborative Inference Method Based on Hierarchical Reinforcement Learning
Weiqi Fu, Lianming Xu, Xin Wu, Li Wang, Aiguo Fei
Hierarchical Reinforcement Learning Disaster Response Collaborative Inference Public Emergency

January 25, 2024

Hierarchical Continual Reinforcement Learning via Large Language Model
Chaofan Pan, Xin Yang, Hao Wang, Wei Wei, Tianrui Li
Large Language Model Reinforcement Learning Hierarchical Reinforcement Learning Continual Reinforcement Learning Policy Making

Hierarchical Reinforcement Learning

Papers

Bidirectional-Reachable Hierarchical Reinforcement Learning with Mutually Responsive Policies

Probabilistic Subgoal Representations for Hierarchical Reinforcement learning

KEHRL: Learning Knowledge-Enhanced Language Representations with Hierarchical Reinforcement Learning

A Provably Efficient Option-Based Algorithm for both High-Level and Low-Level Learning

MacroHFT: Memory Augmented Context-aware Reinforcement Learning On High Frequency Trading

DIPPER: Direct Preference Optimization to Accelerate Primitive-Enabled Hierarchical Reinforcement Learning

Hierarchical Reinforcement Learning for Swarm Confrontation with High Uncertainty

RACon: Retrieval-Augmented Simulated Character Locomotion Control

LGR2: Language Guided Reward Relabeling for Accelerating Hierarchical Reinforcement Learning

Exploring the limits of Hierarchical World Models in Reinforcement Learning

From Words to Actions: Unveiling the Theoretical Underpinnings of LLM-Driven Autonomous Systems

PIPER: Primitive-Informed Preference-based Hierarchical Reinforcement Learning via Hindsight Relabeling

MQE: Unleashing the Power of Interaction with Multi-agent Quadruped Environment

Learning to Change: Choreographing Mixed Traffic Through Lateral Control and Hierarchical Reinforcement Learning

Reinforcement Learning with Options and State Representation

SMAUG: A Sliding Multidimensional Task Window-Based MARL Framework for Adaptive Real-Time Subtask Recognition

MENTOR: Guiding Hierarchical Reinforcement Learning with Human Feedback and Dynamic Distance Constraint

TopoNav: Topological Navigation for Efficient Exploration in Sparse Reward Environments

Emergency Computing: An Adaptive Collaborative Inference Method Based on Hierarchical Reinforcement Learning

Hierarchical Continual Reinforcement Learning via Large Language Model