Policy Gradient

Policy gradient methods are a core component of reinforcement learning, aiming to optimize policies by directly estimating the gradient of expected cumulative rewards. Current research emphasizes improving sample efficiency and addressing challenges like high-dimensional state spaces and non-convex optimization landscapes through techniques such as residual policy learning, differentiable simulation, and novel policy architectures (e.g., tree-based, low-rank matrix models). These advancements are significant for both theoretical understanding of reinforcement learning algorithms and practical applications in robotics, control systems, and other domains requiring efficient and robust decision-making under uncertainty.

Papers

May 12, 2023

Quantile-Based Deep Reinforcement Learning using Two-Timescale Policy Gradient Algorithms
Jinyang Jiang, Jiaqiao Hu, Yijie Peng
Reinforcement Learning Deep Reinforcement Learning Policy Gradient Quantile Estimation

May 11, 2023

Policy Gradient Algorithms Implicitly Optimize by Continuation
Adrien Bolland, Gilles Louppe, Damien Ernst
Reinforcement Learning Policy Gradient Deterministic Policy Policy Gradient Algorithm Direct Policy Gaussian Policy

May 9, 2023

Policy Gradient Methods in the Presence of Symmetries and State Abstractions
Prakash Panangaden, Sahand Rezaei-Shoshtari, Rosie Zhao, David Meger, Doina Precup
Reinforcement Learning Markov Decision Process Policy Gradient Speech Presence Policy OpTimization Approximate Symmetry State Abstraction Abstraction Learning

April 29, 2023

A Coupled Flow Approach to Imitation Learning
Gideon Freund, Elad Sarafian, Sarit Kraus
Imitation Learning Policy Gradient Normalizing Flow State Action Distribution

April 28, 2023

Optimal Scheduling in IoT-Driven Smart Isolated Microgrids Based on Deep Reinforcement Learning
Jiaju Qi, Lei Lei, Kan Zheng, Simon X. Yang, Xuemin, Shen
Deep Reinforcement Learning Policy Gradient Finite Horizon DC Microgrid Application Renewable Generation Optimal Scheduling Energy Dispatch

April 26, 2023

Quantum Natural Policy Gradients: Towards Sample-Efficient Reinforcement Learning
Nico Meyer, Daniel D. Scherer, Axel Plinge, Christopher Mutschler, Michael J. Hartmann
Reinforcement Learning Policy Gradient Variational Quantum Circuit Sample Efficient Reinforcement Learning Quantum Fisher Information

April 21, 2023

A Cubic-regularized Policy Newton Algorithm for Reinforcement Learning
Mizhaan Prajit Maniyar, Akash Mondal, Prashanth L. A., Shalabh Bhatnagar
Reinforcement Learning Policy Gradient Efficient Gradient Cubic Regularization

April 14, 2023

Towards Controllable Diffusion Models via Reward-Guided Exploration
Hengtong Zhang, Tingyang Xu
Diffusion Model Policy Gradient Conditional Diffusion Model Controllable Diffusion Model Reward Aware

March 29, 2023

Policy Gradient Methods for Discrete Time Linear Quadratic Regulator With Random Parameters
Deyue Li
Reinforcement Learning Policy Gradient Optimal Control Discrete Time Random Cost Coefficient

March 23, 2023

Policy Evaluation in Distributional LQR
Zifan Wang, Yulong Gao, Siyi Wang, Michael M. Zavlanos, Alessandro Abate, Karl H. Johansson
Policy Gradient Policy Evaluation Risk Sensitive Distributional Reinforcement Learning

March 22, 2023

Matryoshka Policy Gradient for Entropy-Regularized RL: Convergence and Global Optimality
François Ged, Maria Han Veiga
Policy Gradient Early Stage Convergence Maximum Entropy Global Optimality Entropy Regularized Maximum Entropy Reinforcement Learning Softmax Policy

March 15, 2023

Latent-Conditioned Policy Gradient for Multi-Objective Deep Reinforcement Learning
Takuya Kanazawa, Chetan Gupta
Policy Gradient Multi Objective Reinforcement Learning Sequential Decision Making Pareto Set Multi Objective Deep Reinforcement Learning Pareto Optimal Policy

March 8, 2023

Policy Mirror Descent Inherently Explores Action Space
Yan Li, Guanghui Lan
Policy Gradient Policy Gradient Algorithm Policy Mirror Descent Explicit Communication

March 7, 2023

MAP-Elites with Descriptor-Conditioned Gradients and Archive Distillation into a Single Policy
Maxence Faldor, Félix Chalumeau, Manon Flageat, Antoine Cully
Deep Reinforcement Learning Policy Gradient Evolutionary Computation Diverse Policy Conditional Gradient Distilled Model Evolutionary Robotics MAP Elite

March 6, 2023

Safe Reinforcement Learning via Probabilistic Logic Shields
Wen-Chi Yang, Giuseppe Marra, Gavin Rens, Luc De Raedt
Policy Gradient Safe Reinforcement Learning Deep RL Policy Gradient Algorithm Model Based Shielding Model Based Safe Reinforcement Learning

March 2, 2023

Multi-Start Team Orienteering Problem for UAS Mission Re-Planning with Data-Efficient Deep Reinforcement Learning
Dong Ho Lee, Jaemyung Ahn
Deep Reinforcement Learning Unmanned Aerial Vehicle Policy Gradient Data Efficient Maximum Entropy Orienteering Problem

February 25, 2023

Revisiting LQR Control from the Perspective of Receding-Horizon Policy Gradient
Xiangyuan Zhang, Tamer Başar
Policy Gradient Visual Perspective Discrete Time Linear Quadratic Regulator Free Learning

February 20, 2023

Improving Deep Policy Gradients with Value Function Search
Enrico Marchesini, Christopher Amato
Policy Gradient Value Function Value Network Gradient Based Policy

February 15, 2023

Scalable Multi-Agent Reinforcement Learning with General Utilities
Donghao Ying, Yuhao Ding, Alec Koppel, Javad Lavaei
Multi Agent Reinforcement Learning Policy Gradient Task Utility Policy Gradient Algorithm Multi Agent RL

February 11, 2023

A Policy Gradient Framework for Stochastic Optimal Control Problems with Global Convergence Guarantee
Mo Zhou, Jianfeng Lu
Policy Gradient Gradient Flow Stochastic Optimal Control Global Convergence Guarantee