Policy Decomposition

Policy decomposition is a technique in reinforcement learning that simplifies complex control problems by breaking down a single policy into smaller, more manageable sub-policies. Current research focuses on developing efficient algorithms, such as two-stage approaches and policy-decoupled methods, to learn these sub-policies, often leveraging clustering or sparsity-inducing representations to improve performance and generalization. This approach addresses the challenges of high-dimensional action spaces and limited data, leading to more efficient training and improved adaptation to new environments, with applications ranging from robotics to multi-agent systems. The resulting improvements in scalability and sample efficiency have significant implications for deploying reinforcement learning in real-world scenarios.

Papers

January 7, 2025

Explainable Reinforcement Learning via Temporal Policy Decomposition
Franco Ruggeri, Alessio Russo, Rafia Inam, Karl Henrik Johansson
Reinforcement Learning High Explainability Explainability Method Finite Horizon Explainable Reinforcement Learning Policy Decomposition

February 9, 2024

POTEC: Off-Policy Learning for Large Action Spaces via Two-Stage Policy Decomposition
Yuta Saito, Jihan Yao, Thorsten Joachims
Policy Gradient Action Space Policy OpTimization Policy Learning Policy Decomposition

September 6, 2023

Pre- and post-contact policy decomposition for non-prehensile manipulation with zero-shot sim-to-real transfer
Minchan Kim, Junhyek Han, Jaehyung Kim, Beomjoon Kim
Deep Reinforcement Learning Action Space Efficient Exploration Non Prehensile Manipulation Zero Shot Sim to Real Policy Decomposition

June 10, 2023

HIPODE: Enhancing Offline Reinforcement Learning with High-Quality Synthetic Data from a Policy-Decoupled Approach
Shixi Lian, Yi Ma, Jinyi Liu, Yan Zheng, Zhaopeng Meng
Synthetic Data Offline Reinforcement Learning Reinforcement Learning Model Augmentation Policy Policy Decomposition

September 15, 2022

Sparsity Inducing Representations for Policy Decompositions
Ashwin Khadke, Hartmut Geyer
Optimal Policy Meaningful Representation Sparsity Increase Policy Decomposition

April 6, 2022

PAnDR: Fast Adaptation to New Environments from Offline Experiences via Decoupling Policy and Environment Representations
Tong Sang, Hongyao Tang, Yi Ma, Jianye Hao, Yan Zheng, Zhaopeng Meng, Boyan Li, Zhen Wang
Deep Reinforcement Learning Action Free Offline Fast Adaptation Policy Adaptation Environment Representation New Environment Policy Decomposition

March 29, 2022

Search Methods for Policy Decompositions
Ashwin Khadke, Hartmut Geyer
Optimal Control Complex Dynamical System Approximation Method Policy Decomposition

March 4, 2022

Plan Your Target and Learn Your Skills: Transferable State-Only Imitation Learning via Decoupled Policy Optimization
Minghuan Liu, Zhengbang Zhu, Yuzheng Zhuang, Weinan Zhang, Jianye Hao, Yong Yu, Jun Wang
Imitation Learning Policy Gradient High Level Plan Target Amazon Ally Robust Skill Action Mapping Policy Decomposition

November 10, 2021

DeCOM: Decomposed Policy for Constrained Cooperative Multi-Agent Reinforcement Learning
Zhaoxing Yang, Rong Ding, Haiming Jin, Yifei Wei, Haoyi You, Guiyun Fan, Xiaoying Gan, Xinbing Wang
Multi Agent Reinforcement Learning Multi Agent System Participation Constraint Constrained Optimization Cooperative Multi Agent Reinforcement Learning Policy Decomposition

Policy Decomposition

Papers

Explainable Reinforcement Learning via Temporal Policy Decomposition

POTEC: Off-Policy Learning for Large Action Spaces via Two-Stage Policy Decomposition

Pre- and post-contact policy decomposition for non-prehensile manipulation with zero-shot sim-to-real transfer

HIPODE: Enhancing Offline Reinforcement Learning with High-Quality Synthetic Data from a Policy-Decoupled Approach

Sparsity Inducing Representations for Policy Decompositions

PAnDR: Fast Adaptation to New Environments from Offline Experiences via Decoupling Policy and Environment Representations

Search Methods for Policy Decompositions

Plan Your Target and Learn Your Skills: Transferable State-Only Imitation Learning via Decoupled Policy Optimization

DeCOM: Decomposed Policy for Constrained Cooperative Multi-Agent Reinforcement Learning