Goal-Conditioned RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

13 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 70

All topics

Q-Shaped Options for Hierarchical Reinforcement Learning
Oct 8, 2026Clarisse Wibault, Antoine Gorceix, Antonio Léon Villares +5Hierarchical RL Representation Learning for RL
University of Oxford
World-Model Policy Arbiter for Goal-Conditioned Reinforcement Learning
Oct 7, 2026Junwei Quan, Evgenii Opryshko, Nicholas Rhinehart +1Goal-Conditioned RL Offline RL
University of Toronto · University of Toronto, Vector Institute
An Informational Curse of Horizon in Goal-Conditioned Policy Learning
Oct 7, 2026John L. Zhou, Yuxuan Dong, Jonathan C. KaoGoal-Conditioned RL Generalization in RL
University of California, Los Angeles
Directed Temporal Representations for Offline Visual Control
Oct 6, 2026Chenyang Yuan, Haoyu Wang, Zhuo Sun +1Visual RL Trajectory Representation Learning
University of Sheffield · Shanghai Jiao Tong University · Shanghai University of Finance and Economics +1
Hierarchical Time-aware Bootstrapping for Off-Policy Subgoal Value Learning
Oct 4, 2026Bingyun Liu, Yuheng JingHierarchical RL Temporal-Difference Learning
Institute of Automation Chinese Academy of Sciences
Direction-Conditioned Policies for Online Goal-Conditioned Reinforcement Learning
Oct 4, 2026S K Swaminathan, Damiya Gondha, Theyanesh Eswaramoorthy Rajahkrishnan +1Reinforcement Learning Contrastive RL
Department of Computer Science and Engineering Indian Institute of Technology Kharagpur, India · Department of Mechanical Engineering Indian Institute of Technology Kharagpur, India
Learning Multiple Timescales for Goal-Conditioned Reinforcement Learning
Oct 1, 2026Pedro Robles Dutenhefner, Dikshant Shehmar, Wagner Meira +1Reinforcement Learning Offline RL
Universidade Federal de Minas Gerais · University of Alberta · Alberta Machine Intelligence Institute (Amii) +1
Learning Goal-Reaching Quasimetric Geometry From Finite-Time Reachability
Sep 30, 2026Daisuke Yamada, Travis Pence, Vikas SinghMetric Learning Offline RL
Department of Computer Science University of Wisconsin–Madison · Department of Biostatistics and Medical Informatics University of Wisconsin–Madison
Do Better Goal Representations Improve Goal-Conditioned Reinforcement Learning?
Sep 30, 2026Syed Nazmus Sakib, Abdul Monaf Chowdhury, Nafiul Haque +2Offline RL Representation Learning for RL
Department of Robotics and Mechatronics Engineering, University of Dhaka, Bangladesh · Department of Computer Science, University of Oxford, UK
EVOKE: Eliciting World Knowledge in Agents for Transferable Decision-Making
Sep 29, 2026Yuhan Guo, Jinming Liu, Liang Xu +8LLM Agent Training Generalization in RL
Shanghai Jiaotong University · Eastern Institute of Technology, Ningbo · Zhongguancun Academy, Beijing, China +1
ChronoSRL: Temporal Geometry for Self-Supervised Reinforcement Learning
Sep 28, 2026Nico Bohlinger, Jan PetersRL for Robotics Representation Learning
Technical University of Darmstadt, Germany · Robotics Institute Germany (RIG); German Research Center for AI (DFKI); hessian.AI
Diffusion Subgoal Planning for Long-Horizon Offline Goal-Conditioned Reinforcement Learning
Sep 28, 2026Hengrui Zhang, Yuhu Cheng, C. L. Philip Chen +1Diffusion-Based Planning Hierarchical RL
School of Information and Control Engineering, China University of Mining and Technology · School of Computer Science and Engineering, South China University of Technology
GTRL: Grounding Divide-and-Conquer Value Learning with Temporal Differences
Sep 27, 2026Abdul Monaf Chowdhury, MD Sameer Iqbal Chowdhury, Shifat E Arman +1Reinforcement Learning Temporal-Difference Learning
Department of Robotics and Mechatronics Engineering, University of Dhaka, Bangladesh · Department of Computer Science, Texas State University, USA · Department of Computer Science, University of Oxford, UK
Improving Offline Goal-Conditioned Reinforcement Learning via Selective Reward Stimulation
Sep 16, 2026Jing ZhangReward Shaping Offline RL
The Hong Kong University of Science and Technology
Topological Necessities: Mechanism-Invariant Strategic Subgoals for Cross-Embodiment Goal-Conditioned Control
Sep 12, 2026Hao Shi, Xi LiHierarchical RL Cross-Embodiment Robot Learning
Army Engineering University of PLA (Shijiazhuang Campus), Shijiazhuang, China
From Connectivity to Rewards: Dense Reward Learning with Directed State Graphs
Sep 11, 2026Shuyuan Zhang, Zihan Wang, Xiao-Wen Chang +1Reinforcement Learning Hierarchical RL
School of Computer Science, McGill University Mila – Quebec AI Institute · Google DeepMind, CIFAR Fellow
SUN: Reaching for Novelty in Reinforcement Learning
Sep 8, 2026Wenyan Yang, Arsenii Mustafin, Dominik Baumann +2Intrinsic Motivation RL Exploration
Aalto University · Tampere University
Locally-Guided Actor-Critic: Training a Goal-conditioned Actor with a Subgoal-aware Critic
Aug 31, 2026Olivier Serris, Stéphane Doncieux, Olivier SigaudReinforcement Learning Sparse-Reward RL
ISIR · Sorbonne Université, CNRS
Enhancing Social Intelligence in LLMs with Hierarchical Reasoning and Utterance-Level Goal Rewarding
Aug 6, 2026Xiaofeng Wang, Kakam Chong, Shuai Xiao +9Reward Shaping Hierarchical RL
Independent Researcher · Alibaba Group · Shanghai Jiao Tong University
An Emerging Retail Portfolio Management Application: Personalized, Tax-Aware Reinforcement Learning with Natural Language Goals
Aug 5, 2026Ramin PishehvarReinforcement Learning Portfolio Optimization
State2State: Environment-Derived Mid-Training for LLM Agents
Aug 5, 2026Xuanyu Lei, Yiqi Zhu, Chenliang Li +6LLM Agent Training Goal-Conditioned RL
Institute for AI Industry Research (AIR), Tsinghua University, Beijing, China · Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University, Beijing, China · Institute of Intelligent Computing, Alibaba Group
ATLAS: Adaptive Topological Learning with Abstract Successors for Continual Learning
Aug 5, 2026R. Blake Lawlor, Daniel S. BrownNon-Stationary RL Reinforcement Learning
1Kahlert School of Computing, University of Utah
Action-Conditioned World Model for Goal Plane Probe Guidance in Robotic Ultrasound
Jul 24, 2026Siqi Fan, Mingcong Chen, Ran Liu +5World Model Learning Action-Conditioned World Models
Department of Mechanical and Automation Engineering, Chinese University of Hong Kong, Hong Kong SAR · Centre for Artificial Intelligence and Robotics Hong Kong Institute of Science & Innovation, Chinese Academy of Sciences, Hong Kong SAR · Department of Biomedical Engineering, City University of Hong Kong, Hong Kong SAR +3
Offline RL with Hierarchical Action Chunking
Jul 23, 2026Ahad JawaidHierarchical RL Offline RL
The University of Texas at Dallas
DAGR: State-Conditioned Goal Representations via Difference-Aware Goal Cross-Attention
Jul 15, 2026Xing Lei, Wenyan Yang, Xuetao Zhang +1Reinforcement Learning Attention Mechanisms
Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University · Department of Electrical Engineering and Automation, Aalto University · School of Engineering, Westlake University
RoamFlow: Reinforcement-Aligned One-Step Action MeanFlow Policy for Image-Goal Navigation
Jun 29, 2026Zixuan Zhang, Yuqi Chen, Junjie Gao +4Visual Navigation Robot Navigation
Nanyang Technological University, Singapore
CORE: Common Outcome Regularities from Action-Free Visual Demonstrations for Robot Manipulation
Jun 28, 2026Juyi Sheng, Jincheng Li, Mingxin Tan +1Robot Skill Learning Cross-Embodiment Robot Learning
1Peking University, Shenzhen Graduate School, Shenzhen, China
Humanoid-DART: Humanoid Loco-Manipulation using Diffusion-guided Augmentation through Relabeling and Tracking
Jun 25, 2026Pranav Debbad, Kanish Thiagarajan, Victor Dhédin +2Humanoid Loco-Manipulation Robot Imitation Learning
Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University of Munich (TUM), Germany.
Inverting the Bellman Equation: From $Q$ -Values to World Models
Jun 19, 2026Alistair Letcher, Mattie Fellows, Alexander D. Goldie +3Reinforcement Learning Model-Free RL
FLAIR, University of Oxford · Google DeepMind · Mila, University of Montreal
Direction-Conditioned Policies via Compositional Subgoal Scoring for Online Goal-Conditioned Reinforcement Learning
Jun 15, 2026Swaminathan S K, Damiya Gondha, Theyanesh Eswaramoorthy Rajahkrishnan +1Contrastive RL Policy Learning
Department of Computer Science and Engineering, Indian Institute of Technology Kharagpur, Kharagpur, India · Department of Mechanical Engineering, Indian Institute of Technology Kharagpur, Kharagpur, India.

Oct 8, 2026, cs.AIJ/K move · Enter open · S save

Q-Shaped Options for Hierarchical Reinforcement Learning

Clarisse Wibault, Antoine Gorceix, Antonio Léon Villares, Alexey Zakharov+4

University of Oxford

Hierarchical RL Representation Learning for RL Goal-Conditioned RL

Open paper AskPDF arXiv

Learning to tackle long-horizon, goal-conditioned tasks requires an agent to reason over extended timescales and act across a broad range of states. In principle, Hierarchical Reinforcement Learning (HRL) addresses both challenges through the interaction between action (temporal) and state (spatial) abstraction. First, using an action abstraction to represent temporally extended behaviour as options reduces the effective decision horizon. Second, enabling different state abstractions at each level of the decision process permits greater data aggregation for learning. However, realising these two benefits of a hierarchical policy depends on learning an appropriate action abstraction. Current HRL algorithms fail in one of two ways. Some discard distinctions between options needed for optimal control, undermining hierarchy altogether. Others retain unnecessary distinctions, preserving horizon reduction, but forfeiting coarser state abstraction. In this work, we characterise three desiderata for an action abstraction. We introduce Q-Shaped Options (QSO) to address all three. QSO builds on an architecture with distinct state-value functions, Q functions and policies at each level of the hierarchy. It learns the action abstraction between consecutive levels as a shared encoder shaped by their respective Q functions. The low-level Q function uses the option as a goal, encouraging the abstraction to retain distinctions necessary for optimal control. The high-level Q function uses it as an action, encouraging unnecessary distinctions to be discarded. Across offline goal-conditioned locomotion and manipulation environments, QSO learns semantically meaningful option spaces and outperforms baselines, achieving non-zero performance in tasks where all other evaluated algorithms fail.

Momentum

Related topics

Latest papers 70