Efficient Policy Learning

Efficient policy learning in reinforcement learning (RL) aims to rapidly train agents to perform complex tasks with minimal data and computational resources. Current research emphasizes improving sample efficiency through techniques like incorporating prior knowledge into policy initialization, leveraging offline data for skill extraction and transfer, and developing novel exploration strategies that focus retraining on areas of policy weakness. These advancements, often implemented using actor-critic methods and incorporating entropy regularization or uncertainty-aware perception, are crucial for deploying RL in resource-constrained real-world applications such as robotics and industrial automation. The resulting improvements in sample efficiency and generalization capability are driving significant progress in the field.

Papers

November 5, 2024

VQ-ACE: Efficient Policy Search for Dexterous Robotic Manipulation via Action Chunking Embedding
Chenyu Yang, Davide Liconti, Robert K. Katzschmann
Manipulation Task Dexterous Manipulation Hand Manipulation Dexterous Robotic Manipulation Efficient Policy Learning Action Chunking Action Embeddings

September 30, 2024

Continuously Improving Mobile Manipulation with Autonomous Real-World RL
Russell Mendonca, Emmanuel Panov, Bernadette Bucher, Jiuguang Wang, Deepak Pathak
Autonomous System Mobile Manipulation Real World Reinforcement Learning Efficient Policy Learning

August 5, 2024

Perception Matters: Enhancing Embodied AI with Uncertainty-Aware Semantic Segmentation
Sai Prasanna, Daniel Honerkamp, Kshitij Sirohi, Tim Welschehold, Wolfram Burgard, Abhinav Valada
Perception Aware Embodied AI Perception Model Efficient Policy Learning Perception Data

July 25, 2024

Maximum Entropy On-Policy Actor-Critic via Entropy Advantage Estimation
Jean Seong Bjorn Choe, Jong-Kook Kim
Proximal Policy Optimization Actor Critic Maximum Entropy Reinforcement Learning Trust Region Policy Optimization Efficient Policy Learning

June 25, 2024

EXTRACT: Efficient Policy Learning by Extracting Transferable Robot Skills from Offline Data
Jesse Zhang, Minho Heo, Zuxin Liu, Erdem Biyik, Joseph J Lim, Yao Liu, Rasool Fakoor
Action Space Sparse Reward Offline Data Efficient Policy Learning Parameterized Skill Robotic Skill Skill Based Reinforcement Learning

June 12, 2024

Improving Policy Optimization via $\varepsilon$-Retrain
Luca Marzari, Changliu Liu, Priya L. Donti, Enrico Marchesini
Model Retraining Exploration Strategy Efficient Policy Learning

June 3, 2024

Deep reinforcement learning for weakly coupled MDP's with continuous actions
Francisco Robledo, Urtzi Ayesta, Konstantin Avrachenkov
Deep Reinforcement Learning Action Space Reinforcement Learning Algorithm Lagrangian Relaxation Efficient Policy Learning Continuous Action

April 9, 2024

Diverse Randomized Value Functions: A Provably Pessimistic Approach for Offline Reinforcement Learning
Xudong Yu, Chenjia Bai, Hongyi Guo, Changhong Wang, Zhen Wang
Offline Reinforcement Learning Uncertainty Quantification Efficient Policy Learning Value Estimation

February 22, 2024

Learning an Actionable Discrete Diffusion Policy via Large-Scale Actionless Video Pre-Training
Haoran He, Chenjia Bai, Ling Pan, Weinan Zhang, Bin Zhao, Xuelong Li
Generative Pre Training Discrete Diffusion Efficient Policy Learning Robot Representation Silent Video

December 6, 2023

SDSRA: A Skill-Driven Skill-Recombination Algorithm for Efficient Policy Learning
Eric H. Jiang, Andrew Lizarraga
Faster Convergence Skill Discovery Reinforcement Learning Task Efficient Policy Learning

December 19, 2022

Policy learning "without" overlap: Pessimism and generalized empirical Bernstein's inequality
Ying Jin, Zhimei Ren, Zhuoran Yang, Zhaoran Wang
Optimal Policy Policy Learning Efficient Policy Learning Policy Learning Method Soft Inequality Individualized Decision

November 25, 2022

Policy-Adaptive Estimator Selection for Off-Policy Evaluation
Takuma Udagawa, Haruka Kiyohara, Yusuke Narita, Yuta Saito, Kei Tateno
Policy Evaluation Efficient Estimation Efficient Policy Learning

November 23, 2022

Representation Learning for Continuous Action Spaces is Beneficial for Efficient Policy Learning
Tingting Zhao, Ying Wang, Wei Sun, Yarui Chen, Gang Niub, Masashi Sugiyama
Reinforcement Learning Deep Reinforcement Learning Representation Learning Action Space Policy Learning Efficient Policy Learning

July 29, 2022

Contrastive UCB: Provably Efficient Contrastive Self-Supervised Learning in Online Reinforcement Learning
Shuang Qiu, Lingxiao Wang, Chenjia Bai, Zhuoran Yang, Zhaoran Wang
Reinforcement Learning Contrastive Learning Online Reinforcement Learning Contrastive Self Supervised Learning Upper Confidence Bound Efficient Policy Learning Confidence Aware Contrastive

June 26, 2022

Improving Policy Optimization with Generalist-Specialist Learning
Zhiwei Jia, Xuanlin Li, Zhan Ling, Shuang Liu, Yiran Wu, Hao Su
Deep Reinforcement Learning Strong Generalization Policy Learning Efficient Policy Learning Training Point

February 15, 2022

Beyond the Policy Gradient Theorem for Efficient Policy Updates in Actor-Critic Algorithms
Romain Laroche, Remi Tachet
Reinforcement Learning Policy Gradient Policy Optimization Actor Critic Algorithm Global Optimality Efficient Policy Learning

February 14, 2022

Reinforcement Learning in Presence of Discrete Markovian Context Evolution
Hang Ren, Aivar Sootla, Taher Jafferjee, Junxiao Shen, Jun Wang, Haitham Bou-Ammar
Reinforcement Learning Variational Inference Speech Presence Efficient Policy Learning Context Distillation

December 11, 2021

Learning Efficient Policies for Picking Entangled Wire Harnesses: An Approach to Industrial Bin Picking
Xinyi Zhang, Yukiyasu Domae, Weiwei Wan, Kensuke Harada
Robot Person Constructive Approach Wire Harness Efficient Policy Learning Arbitrary Trajectory

Efficient Policy Learning

Papers

VQ-ACE: Efficient Policy Search for Dexterous Robotic Manipulation via Action Chunking Embedding

Continuously Improving Mobile Manipulation with Autonomous Real-World RL

Perception Matters: Enhancing Embodied AI with Uncertainty-Aware Semantic Segmentation

Maximum Entropy On-Policy Actor-Critic via Entropy Advantage Estimation

EXTRACT: Efficient Policy Learning by Extracting Transferable Robot Skills from Offline Data

Improving Policy Optimization via $\varepsilon$-Retrain

Deep reinforcement learning for weakly coupled MDP's with continuous actions

Diverse Randomized Value Functions: A Provably Pessimistic Approach for Offline Reinforcement Learning

Learning an Actionable Discrete Diffusion Policy via Large-Scale Actionless Video Pre-Training

SDSRA: A Skill-Driven Skill-Recombination Algorithm for Efficient Policy Learning

Policy learning "without" overlap: Pessimism and generalized empirical Bernstein's inequality

Policy-Adaptive Estimator Selection for Off-Policy Evaluation

Representation Learning for Continuous Action Spaces is Beneficial for Efficient Policy Learning

Contrastive UCB: Provably Efficient Contrastive Self-Supervised Learning in Online Reinforcement Learning

Improving Policy Optimization with Generalist-Specialist Learning

Beyond the Policy Gradient Theorem for Efficient Policy Updates in Actor-Critic Algorithms

Reinforcement Learning in Presence of Discrete Markovian Context Evolution

Learning Efficient Policies for Picking Entangled Wire Harnesses: An Approach to Industrial Bin Picking