Hierarchical RL

RL: Reinforcement Learning

Ask about this topic

Overview

Hierarchical reinforcement learning helps agents handle long tasks by grouping primitive actions into skills or planning through intermediate goals. Common approaches include options and manager-worker policies, learned subgoal selection, action chunking, offline skill reuse, and model-based planning. These methods are applied to sparse-reward control, robotics, multi-agent coordination, and language-model agents, where they can improve exploration, credit assignment, and transfer. A central challenge is learning abstractions that are useful and stable without hand-designed boundaries, while keeping high-level plans aligned with what low-level controllers can reliably execute.

Latest papers

CardsList

160 papers25 papers in the last four weeks, up 32% on the four weeks before. 0.1% of all new papers.

  1. DYNAS: A Dynamic Network Penetration Testing Framework Using Deep Hierarchical Reinforcement Learning

    Oct 29, 2026Md. Rakibul Hassan, Majedul Islam, Md. Ashraful Alam +2Automated Penetration TestingHierarchical RL

    BRAC University, Bangladesh

  2. MATCH: Execution-Aware Hierarchical Reinforcement Learning for Vision-and-Language Navigation in Continuous Environment

    Oct 28, 2026Shuhao Ye, Yufei Wei, Demao Wang +3Vision-Language NavigationLanguage-Conditioned Robot Planning

    Zhejiang University, Hangzhou, China

  3. Interpretable Meta-Policies for Neural Decision Making

    Oct 27, 2026Raban Emunds, Jannis Blüml, Quentin Delfosse +1Interpretability in RLNeuro-Symbolic RL

    Department of Computer Science, Technical University Darmstadt, Germany · Hessian Center for Artificial Intelligence (hessian.AI) · Centre for Cognitive Science, Darmstadt +1

  4. Intrinsic Goals for Autonomous Agents: Hierarchical Exploration Captures Future Volitional Action in Drosophila

    Oct 25, 2026Declan Kutscher, Reece Keller, Alyn Kirsch +2Intrinsic MotivationIntrinsic Reward Methods

    Department of Computer Science, University of Maryland, College Park · Machine Learning Department, Carnegie Mellon University · Neuroscience Institute, Carnegie Mellon University +1

  5. SpecMAGIC: Specification-Guided Multi-Task Generalization in Multi-Agent Reinforcement Learning

    Oct 21, 2026Zijian Guo, İlker Işık, H. M. Sabbir Ahmad +1Multi-Agent Reinforcement LearningMulti-Agent Coordination

    Division of Systems Engineering, Boston University · Department of Electrical and Computer Engineering, Boston University

  6. SOMA: Stack-of-Objects Manipulation using a Quadruped Robot

    Oct 14, 2026Stergios E. Bachoumas, Panagiotis ArtemiadisNon-Prehensile Robotic ManipulationRobotic Manipulation

  7. Non-asymptotic Convergence of Average-reward Q-learning with Options

    Oct 14, 2026Kintan Saha, Ahana Deb, Anders Jonsson +1Q-LearningAverage-Reward RL

    Indian Institute of Science · Universitat Pompeu Fabra · Univ. Lille, Inria, CNRS- CRIStAL, Centrale Lille

  8. Learning Navigation from Multimodal Demonstrations with Geometric Candidate Modes

    Oct 14, 2026Chengyang Peng, Zhihao Zhang, Upama Pant +3Robot NavigationRobot Imitation Learning

    Mechanical and Aerospace Engineering, The Ohio State University, Columbus, OH, USA. · Electrical and Computer Engineering, The Ohio State University, Columbus, OH, USA.

  9. HiFun: a Hierarchical Framework for Efficient Functional Dexterous Manipulation Learning

    Oct 14, 2026Linyi Huang, Guowei Huai, Weibin Liu +5Robotic ManipulationRobot Skill Learning

    HKUST (Guangzhou), China · HKUST, Hong Kong (China) · ETH Zurich, Switzerland

  10. From Orbit Matching to Docking: A Hierarchical Reinforcement-Learning Approach to Multiphase Rendezvous Trajectory Design

    Oct 12, 2026Mariko Storey-Matsutani, Anatoly Zlotnik, Lee Holguin +2Spacecraft Rendezvous and Proximity OperationsRL Control

    Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge, MA 02139, USA · Los Alamos National Laboratory, Los Alamos, NM, USA

  11. Q-Shaped Options for Hierarchical Reinforcement Learning

    Oct 8, 2026Clarisse Wibault, Antoine Gorceix, Antonio Léon Villares +5Hierarchical RLRepresentation Learning for RL

    University of Oxford

  12. From Solo to Ensemble: A Hierarchical Framework for Composable Multi-Agent Human-Object Interaction

    Oct 8, 2026Zekai Deng, Kangyi Chen, Ye Shi +1Skill DistillationHOI Generation

    ShanghaiTech University · InstAdapt

  13. RoboAware: Learning to Coordinate Embodied Skills from Counterfactual Outcomes

    Oct 8, 2026Bohan Zhou, Xingbei Chen, Emily Huang +17Robot Policy LearningLarge Language Model-Based Robot Planning

    The Chinese University of Hong Kong · The Hong Kong University of Science and Technology · Knowin AI +3

  14. Energy-Efficient Gait Adaptation via Hierarchical Reinforcement Learning for Quadrupedal Locomotion Across Diverse Terrains

    Oct 7, 2026Ammar Issa, Anubhav Singh, Anton Tsaritsin +1Legged RoboticsEnergy-Efficient Robot Locomotion

    Faculty of Control Systems and Robotics, ITMO University, and the Biomechatronics and Energy-Efficient Robotics Lab (BE2R), St. Petersburg, Russia

  15. Hierarchical Reinforcement Learning for Collision-Free Locomotion of an Underactuated Biped

    Oct 5, 2026Jagannath Prasad Sahoo, Saurabh Kumar, Surya Prakash S. K. +3Legged RoboticsHierarchical RL

    CAIR, Indian Institute of Technology Mandi, Mandi, India · SMME, Indian Institute of Technology Mandi, Mandi, India

  16. Hierarchical Reinforcement Learning with Stable Temporal Abstraction for Language Model Agents

    Oct 4, 2026Shayan Mohajer Hamidi, Yize Cheng, Yuanda Xu +2Reinforcement LearningHierarchical RL

    LinkedIn Corporation · University of Maryland

  17. Hierarchical Time-aware Bootstrapping for Off-Policy Subgoal Value Learning

    Oct 4, 2026Bingyun Liu, Yuheng JingHierarchical RLTemporal-Difference Learning

    Institute of Automation Chinese Academy of Sciences

  18. On Semi-Markov Suboptimality in Hierarchical Reinforcement Learning

    Oct 4, 2026Bingyun Liu, Yuheng JingReinforcement LearningSequential Decision Making

    Institute of Automation Chinese Academy of Sciences

  19. Learning Modular Policy for Multi-Floor Object Navigation:A Factorized Framework for Diagnostic Study

    Oct 3, 2026Shichao Zhai, Shuhao Ye, Rong Xiong +1Object-Goal NavigationEmbodied Navigation

    State Key Laboratory of Industrial Control and Technology, Zhejiang University, Hangzhou, China.

  20. Mastering Atari 2600 Games with Discovered Options

    Oct 2, 2026Erik M. Lintunen, Marlos C. MachadoRL for Game AIHierarchical RL

    University of Alberta · Alberta Machine Intelligence Institute (Amii) · Canada CIFAR AI Chair

  21. Learning Transferable Policies from Action-free Time Series Through Dynamical Embeddings

    Oct 2, 2026Niklas Emonds, Georgia KoppeTransfer Learning in RLReinforcement Learning

    Hector Institute for AI in Psychiatry (HITKIP) & Department of Psychiatry and Psychotherapy, Central Institute of Mental Health (CIMH), Medical Faculty Mannheim, Heidelberg University · Interdisciplinary Center for Scientific Computing (IWR), Faculty of Mathematics and Computer Science, Heidelberg University · Hertie Institute for AI in Brain Health, University of Tübingen, Germany

  22. Learning What to Investigate Next: Meta-Reasoning for Long-Horizon Research Agents

    Oct 1, 2026Ankur Samanta, Yonathan Efroni, Paul Sajda +2Autonomous Research AgentsLong-Horizon Agent Tasks

    Meta AI · Columbia University · Work done at Meta +1

  23. Towards Hierarchical Cyber Defense with Large Language Models: From Planning to Execution

    Sep 30, 2026Harshith Doppalapudi, Nathaniel D. Bastian, Ankit ShahAutonomous Cyber DefenseLanguage Model-Based Control

    Indiana University Bloomington, IN, USA · Johns Hopkins University Baltimore, MD, USA

  24. Game-Guided Skill Discovery through Self-Play for Playable Agent Control

    Sep 30, 2026Seungeun Rho, Jeonghwan Kim, Xue Bin Peng +1Robot Skill LearningHierarchical RL

    Georgia Institute of Technology · Simon Fraser University and NVIDIA

  25. Patient-Centered Treatment Planning for Chronic Multimorbidity: A Hierarchical Reinforcement Learning Framework for Preference Modeling

    Sep 30, 2026Nafiseh Payani, Soham Das, G. Anthony Wilson +1HealthcareHierarchical RL

    Department of Industrial and Systems Engineering at the University of Tennessee, Knoxville, TN, USA · Department of Industrial Engineering at the University of Arkansas, Fayetteville, AR, USA

  26. Validity-Preserving Hierarchical RL for Joint Routing and Switch Placement in EDA

    Sep 30, 2026Dorian Gailhard, Ugo Lecerf, Enzo Tartaglione +2Electronic Design AutomationHierarchical RL

    LTCI, Télécom Paris, Institut Polytechnique de Paris, France · Arteris IP

  27. Going Beyond State-Reaching: Learning Abstractions for Intrinsically Motivated Option Discovery

    Sep 29, 2026Akhil Bagaria, Anita De Mello Koch, George KonidarisHierarchical RLRL Exploration

    Brown University

  28. Diffusion Subgoal Planning for Long-Horizon Offline Goal-Conditioned Reinforcement Learning

    Sep 28, 2026Hengrui Zhang, Yuhu Cheng, C. L. Philip Chen +1Diffusion-Based PlanningHierarchical RL

    School of Information and Control Engineering, China University of Mining and Technology · School of Computer Science and Engineering, South China University of Technology

  29. Hierarchical Multi-agent Reinforcement Learning for Warehouse Robot Coordination under Communication Loss

    Sep 27, 2026Weihao Sun, Gehui Xu, Andreas A. MalikopoulosHierarchical RLMulti-Robot Coordination

    Systems Engineering Program, Cornell University, Ithaca, NY 14850 USA · Department of Electrical and Electronic Engineering, Imperial College London, London SW7 2AZ, UK · Applied Mathematics, Systems Engineering, Mechanical Engineering, Electrical & Computer Engineering, and School of Civil & Environmental Engineering, Cornell University, Ithaca, NY, USA

  30. CodeSkill: Latent Skill Abstraction for Long-Horizon Code Agents

    Sep 27, 2026Song-Li Wu, Jingyi Wang, Zhaocheng Du +2LLM Agent Skill LearningLong-Horizon LLM Agents

    Tsinghua University · Huawei Noah’s Ark Lab · Shanghai Jiao Tong University