Hierarchical Reinforcement Learning

Hierarchical Reinforcement Learning (HRL) addresses the challenge of training agents to solve complex, long-horizon tasks by decomposing them into simpler sub-tasks managed by a hierarchy of policies. Current research focuses on improving HRL's efficiency and robustness through techniques like subgoal generation, option frameworks, and the integration of large language models and other AI methods to guide learning and improve decision-making. This approach is proving valuable in diverse applications, including robotics, recommendation systems, and even high-frequency trading, by enabling more efficient learning and better generalization to unseen situations. The development of more efficient and theoretically grounded HRL algorithms is a significant area of ongoing research.

Papers

November 1, 2024

Hierarchical Preference Optimization: Learning to achieve goals via feasible subgoals prediction
Utsav Singh, Souradip Chakraborty, Wesley A. Suttle, Brian M. Sadler, Anit Kumar Sahu, Mubarak Shah, Vinay P. Namboodiri, Amrit Singh Bedi
Robot Navigation Direct Preference Optimization Hierarchical Reinforcement Learning Pseudo Goal Maximum Entropy Reinforcement Learning Goal Model

October 31, 2024

Demystifying Linear MDPs and Novel Dynamics Aggregation Framework
Joongkyu Lee, Min-hwan Oh
Hierarchical Reinforcement Learning C POMDPs Linear MDPs Q Aggregation

October 30, 2024

VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planning
Yichao Liang, Nishanth Kumar, Hao Tang, Adrian Weller, Joshua B. Tenenbaum, Tom Silver, João F. Henriques, Kevin Ellis
Neuro Symbolic World Model Hierarchical Reinforcement Learning Robot Planning Lex Leader Symmetry Breaking Predicate Knowledge Embeddings Visual Predictive Vision Language Planning

October 26, 2024

Copyright-Aware Incentive Scheme for Generative Art Models Using Hierarchical Reinforcement Learning
Zhuan Shi, Yifei Song, Xiaoli Tang, Lingjuan Lyu, Boi Faltings
Image Generation Text to Image Hierarchical Reinforcement Learning Generative Art

October 19, 2024

Hierarchical Reinforced Trader (HRT): A Bi-Level Approach for Optimizing Stock Selection and Execution
Zijie Zhao, Roy E. Welsch
Deep Reinforcement Learning Hierarchical Reinforcement Learning Trading Strategy Layered Approach

October 15, 2024

Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learning
Jiaheng Hu, Zizhao Wang, Peter Stone, Roberto Martín-Martín
Reinforcement Learning Hierarchical Reinforcement Learning Skill Discovery Hierarchical Framework Skill Mention

October 12, 2024

HG2P: Hippocampus-inspired High-reward Graph and Model-Free Q-Gradient Penalty for Path Planning and Motion Control
Haoran Wang, Yaoru Sun, Zeshen Tang
Path Planning Hierarchical Reinforcement Learning Motion Control

October 11, 2024

Hierarchical Universal Value Function Approximators
Rushiv Arora
Reinforcement Learning Value Function Hierarchical Reinforcement Learning

October 10, 2024

Meta-Learning Integration in Hierarchical Reinforcement Learning for Advanced Task Complexity
Arash Khajooeinejad, Masoumeh Chapariniya
Hierarchical Reinforcement Learning Efficient Exploration Task Adaptation Hierarchical Agent Hierarchical Policy

October 7, 2024

Unsupervised Skill Discovery for Robotic Manipulation through Automatic Task Generation
Paul Jansonnie, Bingbing Wu, Julien Perez, Jan Peters
Robotic Manipulation Manipulation Task Hierarchical Reinforcement Learning Self Play Skill Discovery

October 3, 2024

Diffusion Meets Options: Hierarchical Generative Skill Composition for Temporally-Extended Tasks
Zeyu Feng, Hao Luan, Kevin Yuchen Ma, Harold Soh
Robot Navigation Temporal Logic Hierarchical Reinforcement Learning Long Horizon Option Pricing Temporal Task Latent Skill

September 20, 2024

September 11, 2024

Hierarchical Reinforcement Learning for Temporal Abstraction of Listwise Recommendation
Luo Ji, Gao Liu, Mingyang Yin, Hongxia Yang, Jingren Zhou
Reinforcement Learning Hierarchical Reinforcement Learning Modern Recommender System Temporal Abstraction User Feedback

August 23, 2024

Mastering the Digital Art of War: Developing Intelligent Combat Simulation Agents for Wargaming Using Hierarchical Reinforcement Learning
Scotty Black
Reinforcement Learning Decision Making Hierarchical Reinforcement Learning Hybrid Reinforcement Learning Digital Art

August 21, 2024

Subgoal-based Hierarchical Reinforcement Learning for Multi-Agent Collaboration
Cheng Xu, Changtian Zhang, Yuchen Shi, Ran Wang, Shihong Duan, Yadong Wan, Xiaotong Zhang
Reinforcement Learning Hierarchical Reinforcement Learning Multi Agent Collaboration Multi Agent Environment Dynamic Goal

August 12, 2024

Hierarchical in-Context Reinforcement Learning with Hindsight Modular Reflections for Planning
Chuanneng Sun, Songjun Huang, Dario Pompili
Reinforcement Learning Task Planning Hierarchical Reinforcement Learning Hindsight Instruction LLM Policy

August 3, 2024

Walk Wisely on Graph: Knowledge Graph Reasoning with Dual Agents via Efficient Guidance-Exploration
Zijian Wang, Bin Wang, Haifeng Jing, Huayu Li, Hongbo Dou
Graph Drawing Hierarchical Reinforcement Learning Multi Hop Reasoning Reasoning Path Malicious Agent Knowledge Graph Reasoning Exploration Policy

July 21, 2024

Temporal Abstraction in Reinforcement Learning with Offline Data
Ranga Shaarad Ayyagari, Anurita Ghosh, Ambedkar Dukkipati
Reinforcement Learning Hierarchical Reinforcement Learning Offline Data Multi Agent MuJoCo Standard Reinforcement Learning Temporal Abstraction

July 9, 2024

Hierarchical Average-Reward Linearly-solvable Markov Decision Processes
Guillermo Infante, Anders Jonsson, Vicenç Gómez
Hierarchical Reinforcement Learning Average Reward Markov Decision Process Average Reward Reinforcement Learning High Level Task

Hierarchical Reinforcement Learning

Papers

Hierarchical Preference Optimization: Learning to achieve goals via feasible subgoals prediction

Demystifying Linear MDPs and Novel Dynamics Aggregation Framework

VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planning

Copyright-Aware Incentive Scheme for Generative Art Models Using Hierarchical Reinforcement Learning

Hierarchical Reinforced Trader (HRT): A Bi-Level Approach for Optimizing Stock Selection and Execution

Disentangled Unsupervised Skill Discovery for Efficient Hierarchical Reinforcement Learning

HG2P: Hippocampus-inspired High-reward Graph and Model-Free Q-Gradient Penalty for Path Planning and Motion Control

Hierarchical Universal Value Function Approximators

Meta-Learning Integration in Hierarchical Reinforcement Learning for Advanced Task Complexity

Unsupervised Skill Discovery for Robotic Manipulation through Automatic Task Generation

Diffusion Meets Options: Hierarchical Generative Skill Composition for Temporally-Extended Tasks

Adaptive Task Allocation in Multi-Human Multi-Robot Teams under Team Heterogeneity and Dynamic Information Uncertainty

Selective Exploration and Information Gathering in Search and Rescue Using Hierarchical Learning Guided by Natural Language Input

Hierarchical Reinforcement Learning for Temporal Abstraction of Listwise Recommendation

Mastering the Digital Art of War: Developing Intelligent Combat Simulation Agents for Wargaming Using Hierarchical Reinforcement Learning

Subgoal-based Hierarchical Reinforcement Learning for Multi-Agent Collaboration

Hierarchical in-Context Reinforcement Learning with Hindsight Modular Reflections for Planning

Walk Wisely on Graph: Knowledge Graph Reasoning with Dual Agents via Efficient Guidance-Exploration

Temporal Abstraction in Reinforcement Learning with Offline Data

Hierarchical Average-Reward Linearly-solvable Markov Decision Processes