Policy Gradient Algorithm

Policy gradient algorithms are a class of reinforcement learning methods aiming to optimize a policy's parameters by directly maximizing expected cumulative rewards or more general utility functions. Current research focuses on improving sample efficiency, addressing challenges in high-dimensional spaces through function approximation techniques like linear regression and neural networks (e.g., Implicit Quantile Networks), and enhancing robustness and safety via methods such as variance reduction, risk-averse formulations (using Gini deviation), and probabilistic logic constraints. These advancements are significant for tackling complex real-world problems across diverse domains, including robotics, traffic control, and financial risk management, by enabling more efficient and reliable learning of optimal policies in challenging environments.

Papers

October 5, 2024

On the Sample Complexity of a Policy Gradient Algorithm with Occupancy Approximation for General Utility Reinforcement Learning
Anas Barakat, Souradip Chakraborty, Peihong Yu, Pratap Tokekar, Amrit Singh Bedi
Reinforcement Learning Sample Complexity Action Space Policy OpTimization Policy Gradient Algorithm Utility Learning

October 3, 2024

Beyond Expected Returns: A Policy Gradient Algorithm for Cumulative Prospect Theoretic Reinforcement Learning
Olivier Lepel, Anas Barakat
Policy Gradient Policy Gradient Algorithm Stock Return Prediction Prospect Theory

September 9, 2024

Towards Fast Rates for Federated and Multi-Task Reinforcement Learning
Feng Zhu, Robert W. Heath Jr., Aritra Mitra
Markov Decision Process Policy Gradient Optimal Policy Federated Prompt Cooperation Multi Task Reinforcement Learning Fast Rate Policy Gradient Algorithm

July 18, 2024

PG-Rainbow: Using Distributional Reinforcement Learning in Policy Gradient Methods
WooJae Jeon, KangJun Lee, Jeewoo Lee
Policy Gradient Proximal Policy Optimization Distributional Reinforcement Learning Reinforcement Learning Task Policy Gradient Algorithm

June 21, 2024

Catastrophic-risk-aware reinforcement learning with extreme-value-theory-based policy gradients
Parisa Davar, Frédéric Godin, Jose Garrido
Policy Gradient Sequential Decision Extreme Value Theory Policy Gradient Algorithm Tail Risk Risk Aware Reinforcement Learning Dynamic Hedging

June 4, 2024

Building Socially-Equitable Public Models
Yejia Liu, Jianyi Yang, Pengfei Li, Tongxin Li, Shaolei Ren
Fair Model Policy Gradient Algorithm Public Model

May 5, 2024

ClothPPO: A Proximal Policy Optimization Enhancing Framework for Robotic Cloth Manipulation with Observation-Aligned Action Spaces
Libing Yang, Yang Li, Long Chen
Action Space Proximal Policy Optimization Policy Gradient Algorithm Robotic Cloth Manipulation Soft Body Manipulation

April 12, 2024

RLEMMO: Evolutionary Multimodal Optimization Assisted By Deep Reinforcement Learning
Hongqiao Lian, Zeyuan Ma, Hongshu Guo, Ting Huang, Yue-Jiao Gong
Deep Reinforcement Learning Multi Objective Black Box Optimization Policy Gradient Algorithm Competitive Algorithm Multi Modal Optimization Evolutionary Multimodal Optimization

February 4, 2024

INViT: A Generalizable Routing Problem Solver with Invariant Nested View Transformer
Han Fang, Zhihao Song, Paul Weng, Yutong Ban
Deep Reinforcement Learning Multi View Policy Gradient Algorithm Learning Based Solver Admissible Heuristic

February 3, 2024

Learning General Parameterized Policies for Infinite Horizon Average Reward Constrained MDPs via Primal-Dual Policy Gradient Algorithm
Qinbo Bai, Washim Uddin Mondal, Vaneet Aggarwal
Optimal Policy Primal Dual Constrained Markov Decision Process Regret Guarantee Simple Regret Policy Gradient Algorithm Average Reward Markov Decision Process Policy Parametrization

January 31, 2024

Behind the Myth of Exploration in Policy Gradients
Adrien Bolland, Gaspard Lambrechts, Damien Ernst
Policy Gradient Optimal Policy Environment Exploration Near Optimal Policy Policy Gradient Algorithm Exploration Strategy Many Myth Effective Reinforcement Learning

October 25, 2023

Over-the-air Federated Policy Gradient
Huiwen Yang, Lingying Huang, Subhrakanti Dey, Ling Shi
Optimization Purpose Distortion Aware Air Federated Learning Policy Gradient Algorithm Air Aggregation Novel Over the Air Second

October 15, 2023

Enhancing Task Performance of Learned Simplified Models via Reinforcement Learning
Hien Bui, Michael Posa
Reinforcement Learning Predictive Control Robotic Hand Model Learning Stochastic Policy Policy Gradient Algorithm Task Performance Contact Dynamic

September 5, 2023

Regret Analysis of Policy Gradient Algorithm for Infinite Horizon Average Reward Markov Decision Processes
Qinbo Bai, Washim Uddin Mondal, Vaneet Aggarwal
Regret Analysis O$ Regret Policy Gradient Algorithm Average Reward Markov Decision Process Linear Markov Decision Process

July 17, 2023

An Alternative to Variance: Gini Deviation for Risk-averse Policy Gradient
Yudong Luo, Guiliang Liu, Pascal Poupart, Yangchen Pan
Policy Learning Variance Information Policy Gradient Algorithm Risk Averse Reinforcement Learning Risk Averse Policy Gini Index

June 2, 2023

Reinforcement Learning with General Utilities: Simpler Variance Reduction and Large State-Action Space
Anas Barakat, Ilyas Fatkhullin, Niao He
Reinforcement Learning Action Space Task Utility Variance Reduction Global Optimality Policy Gradient Algorithm

May 11, 2023

Policy Gradient Algorithms Implicitly Optimize by Continuation
Adrien Bolland, Gilles Louppe, Damien Ernst
Reinforcement Learning Policy Gradient Deterministic Policy Policy Gradient Algorithm Direct Policy Gaussian Policy

March 15, 2023

Policy Gradient Converges to the Globally Optimal Policy for Nearly Linear-Quadratic Regulators
Yinbin Han, Meisam Razaviyayn, Renyuan Xu
Reinforcement Learning Optimal Policy Linear Quadratic Regulator Nonlinear Control Policy Gradient Algorithm

March 8, 2023

Policy Mirror Descent Inherently Explores Action Space
Yan Li, Guanghui Lan
Policy Gradient Policy Gradient Algorithm Policy Mirror Descent Explicit Communication

March 6, 2023

Safe Reinforcement Learning via Probabilistic Logic Shields
Wen-Chi Yang, Giuseppe Marra, Gavin Rens, Luc De Raedt
Policy Gradient Safe Reinforcement Learning Deep RL Policy Gradient Algorithm Model Based Shielding Model Based Safe Reinforcement Learning

Policy Gradient Algorithm

Papers

On the Sample Complexity of a Policy Gradient Algorithm with Occupancy Approximation for General Utility Reinforcement Learning

Beyond Expected Returns: A Policy Gradient Algorithm for Cumulative Prospect Theoretic Reinforcement Learning

Towards Fast Rates for Federated and Multi-Task Reinforcement Learning

PG-Rainbow: Using Distributional Reinforcement Learning in Policy Gradient Methods

Catastrophic-risk-aware reinforcement learning with extreme-value-theory-based policy gradients

Building Socially-Equitable Public Models

ClothPPO: A Proximal Policy Optimization Enhancing Framework for Robotic Cloth Manipulation with Observation-Aligned Action Spaces

RLEMMO: Evolutionary Multimodal Optimization Assisted By Deep Reinforcement Learning

INViT: A Generalizable Routing Problem Solver with Invariant Nested View Transformer

Learning General Parameterized Policies for Infinite Horizon Average Reward Constrained MDPs via Primal-Dual Policy Gradient Algorithm

Behind the Myth of Exploration in Policy Gradients

Over-the-air Federated Policy Gradient

Enhancing Task Performance of Learned Simplified Models via Reinforcement Learning

Regret Analysis of Policy Gradient Algorithm for Infinite Horizon Average Reward Markov Decision Processes

An Alternative to Variance: Gini Deviation for Risk-averse Policy Gradient

Reinforcement Learning with General Utilities: Simpler Variance Reduction and Large State-Action Space

Policy Gradient Algorithms Implicitly Optimize by Continuation

Policy Gradient Converges to the Globally Optimal Policy for Nearly Linear-Quadratic Regulators

Policy Mirror Descent Inherently Explores Action Space

Safe Reinforcement Learning via Probabilistic Logic Shields