Policy Actor Critic

Policy Actor-Critic (PAC) methods are a class of reinforcement learning algorithms aiming to efficiently learn optimal policies by simultaneously updating a policy (actor) and a value function (critic). Current research focuses on improving sample efficiency and robustness of off-policy PAC algorithms, exploring techniques like multi-step learning, pessimism/optimism control, and unique experience replay to optimize data usage and mitigate overestimation bias. These advancements are significant for addressing challenges in continuous control tasks and enabling applications in robotics, autonomous driving, and other domains requiring efficient learning from complex, high-dimensional environments.

Papers

October 19, 2024

Offline-to-online Reinforcement Learning for Image-based Grasping with Scarce Demonstrations
Bryan Chan, Anson Leung, James Bergstra
Behavior Cloning Suboptimal Demonstration Policy Actor Critic Offline to Online Reinforcement Learning RL Algorithm

October 15, 2024

Mitigating Suboptimality of Deterministic Policy Gradients in Complex Q-functions
Ayush Jain, Norio Kosaka, Xinhu Li, Kyung-Min Kim, Erdem Bıyık, Joseph J. Lim
Reinforcement Learning Complexity Level Q Function Policy Actor Critic Deterministic Policy Gradient Sub Optimality Gap

October 10, 2024

Rethinking Adversarial Inverse Reinforcement Learning: From the Angles of Policy Imitation and Transferable Reward Recovery
Yangchun Zhang, Wang Zhou, Yirui Zhou
Proximal Policy Optimization Reinforcement Learning Algorithm Inverse Reinforcement Learning Tilting Angle Policy Actor Critic Imitation Policy Adversarial Inverse Reinforcement Learning

September 6, 2024

Goal-Reaching Policy Learning from Non-Expert Observations via Effective Subgoal Guidance
RenMing Huang, Shaochong Liu, Yunqiang Pei, Peng Wang, Guoqing Wang, Yang Yang, Hengtao Shen
Efficient Exploration Policy Actor Critic Subgoal Search Goal Oriented Reinforcement Learning

June 6, 2024

Exploring Pessimism and Optimism Dynamics in Deep Reinforcement Learning
Bahareh Tasdighi, Nicklas Werge, Yi-Shan Wu, Melih Kandemir
Deep Reinforcement Learning Policy Actor Critic State Action Value Optimism Bias

March 15, 2024

Quality-Diversity Actor-Critic: Learning High-Performing and Diverse Behaviors via Value and Successor Features Critics
Luca Grillotti, Maxence Faldor, Borja G. León, Antoine Cully
Deep Reinforcement Learning Value Function Net Present Value Quality Diversity High Performance Diverse Behavior Policy Actor Critic Successor Feature

February 22, 2024

ACE : Off-Policy Actor-Critic with Causality-Aware Entropy Regularization
Tianying Ji, Yongyuan Liang, Yan Zeng, Yu Luo, Guowei Xu, Jiawei Guo, Ruijie Zheng, Furong Huang, Fuchun Sun, Huazhe Xu
Model Free Policy Actor Critic Ace Opencpop Causal Entropy

February 5, 2024

Frugal Actor-Critic: Sample Efficient Off-Policy Deep Reinforcement Learning Using Unique Experiences
Nikhil Kumar Singh, Indranil Saha
Actor Critic Reinforcement Learning Policy Actor Critic Model Free Policy State Reward

December 9, 2023

Position control of an acoustic cavitation bubble by reinforcement learning
Kálmán Klapcsik, Bálint Gyires-Tóth, Juan Manuel Rosselló, Ferenc Hegedűs
Reinforcement Learning Policy Actor Critic Asset Price Bubble Position Control Cavitation Intensity Recognition Pressure Signal Normalization

November 30, 2023

Handling Cost and Constraints with Off-Policy Deep Reinforcement Learning
Jared Markowitz, Jesse Silverberg, Gary Collins
Reward Function Action Space Participation Constraint Hidden CoST Policy Actor Critic Policy Deep Reinforcement Learning

May 29, 2023

DoMo-AC: Doubly Multi-step Off-policy Actor-Critic Algorithm
Yunhao Tang, Tadashi Kozuno, Mark Rowland, Anna Harutyunyan, Rémi Munos, Bernardo Ávila Pires, Michal Valko
Policy Gradient Policy Learning Policy Improvement Policy Actor Critic Multi Step Off Policy

February 22, 2023

Behavior Proximal Policy Optimization
Zifeng Zhuang, Kun Lei, Jinxin Liu, Donglin Wang, Yilang Guo
Offline Reinforcement Learning Proximal Policy Optimization Offline Policy Policy Actor Critic

December 15, 2022

Driver Assistance Eco-driving and Transmission Control with Deep Reinforcement Learning
Lindsey Kerbel, Beshah Ayalew, Andrej Ivanco, Keith Loiselle
Deep Reinforcement Learning Policy Optimization Model Free Deep Reinforcement Learning Policy Actor Critic Eco Driving

November 7, 2022

Max-Min Off-Policy Actor-Critic Method Focusing on Worst-Case Robustness to Model Misspecification
Takumi Tanabe, Rei Sato, Kazuto Fukuchi, Jun Sakuma, Youhei Akimoto
Reinforcement Learning Actor Critic Deep Deterministic Policy Gradient Model Misspecification Policy Algorithm Policy Actor Critic Uncertainty Set Worst Case Robustness Multi Joint

October 1, 2022

Boosting Exploration in Actor-Critic Algorithms by Incentivizing Plausible Novel States
Chayan Banerjee, Zhiyong Chen, Nasimul Noman
Policy OpTimization Actor Critic Algorithm Intrinsic Reward Model Free Deep Reinforcement Learning Exploration Performance Policy Actor Critic Path Entropy

September 1, 2022

Actor Prioritized Experience Replay
Baturay Saglam, Furkan B. Mutlu, Dogan C. Cicek, Suleyman S. Kozat
Deep Reinforcement Learning Actor Critic Algorithm Experience Replay Policy Actor Critic

June 23, 2022

CGAR: Critic Guided Action Redistribution in Reinforcement Leaning
Tairan Huang, Xu Li, Hao Li, Mingming Sun, Ping Li
Positive Reinforcement Policy Actor Critic Policy Distribution Intrinsic Exploration Understanding Reinforcement

May 24, 2022

Concurrent Credit Assignment for Data-efficient Reinforcement Learning
Emmanuel Daucé
Action Space Data Efficient Credit Assignment Policy Actor Critic Variational Optimization Effective Reinforcement Learning Occupancy Model

May 8, 2022

Simultaneous Double Q-learning with Conservative Advantage Learning for Actor-Critic Methods
Qing Li, Wengang Zhou, Zhenbo Lu, Houqiang Li
Actor Critic Policy Actor Critic Double Q Learning Advantage Learning

February 18, 2022

DARL1N: Distributed multi-Agent Reinforcement Learning with One-hop Neighbors
Baoqian Wang, Junfei Xie, Nikolay Atanasov
Multi Agent Reinforcement Learning Policy Actor Critic Hop Neighbor

Policy Actor Critic

Papers

Offline-to-online Reinforcement Learning for Image-based Grasping with Scarce Demonstrations

Mitigating Suboptimality of Deterministic Policy Gradients in Complex Q-functions

Rethinking Adversarial Inverse Reinforcement Learning: From the Angles of Policy Imitation and Transferable Reward Recovery

Goal-Reaching Policy Learning from Non-Expert Observations via Effective Subgoal Guidance

Exploring Pessimism and Optimism Dynamics in Deep Reinforcement Learning

Quality-Diversity Actor-Critic: Learning High-Performing and Diverse Behaviors via Value and Successor Features Critics

ACE : Off-Policy Actor-Critic with Causality-Aware Entropy Regularization

Frugal Actor-Critic: Sample Efficient Off-Policy Deep Reinforcement Learning Using Unique Experiences

Position control of an acoustic cavitation bubble by reinforcement learning

Handling Cost and Constraints with Off-Policy Deep Reinforcement Learning

DoMo-AC: Doubly Multi-step Off-policy Actor-Critic Algorithm

Behavior Proximal Policy Optimization

Driver Assistance Eco-driving and Transmission Control with Deep Reinforcement Learning

Max-Min Off-Policy Actor-Critic Method Focusing on Worst-Case Robustness to Model Misspecification

Boosting Exploration in Actor-Critic Algorithms by Incentivizing Plausible Novel States

Actor Prioritized Experience Replay

CGAR: Critic Guided Action Redistribution in Reinforcement Leaning

Concurrent Credit Assignment for Data-efficient Reinforcement Learning

Simultaneous Double Q-learning with Conservative Advantage Learning for Actor-Critic Methods

DARL1N: Distributed multi-Agent Reinforcement Learning with One-hop Neighbors