Policy Optimization

Policy optimization is a core area of reinforcement learning focused on efficiently finding optimal policies, or strategies, for agents interacting with an environment to maximize rewards. Current research emphasizes improving sample efficiency and robustness, particularly through algorithms like Proximal Policy Optimization (PPO) and its variants, as well as exploring new approaches such as Direct Preference Optimization (DPO) and incorporating techniques like diffusion models and dual regularization. These advancements are significant for both theoretical understanding of reinforcement learning and practical applications across diverse fields, including robotics, natural language processing, and resource management.

Papers

February 15, 2022

Beyond the Policy Gradient Theorem for Efficient Policy Updates in Actor-Critic Algorithms
Romain Laroche, Remi Tachet
Reinforcement Learning Policy Gradient Policy Optimization Actor Critic Algorithm Global Optimality Efficient Policy Learning

February 11, 2022

Understanding Curriculum Learning in Policy Optimization for Online Combinatorial Optimization
Runlong Zhou, Zelin He, Yuandong Tian, Yi Wu, Simon S. Du
Reinforcement Learning Curriculum Learning Policy OpTimization Policy Optimization Natural Policy Gradient Online Combinatorial Latent Markov Decision Process

February 7, 2022

Policy Optimization for Stochastic Shortest Path
Liyu Chen, Haipeng Luo, Aviv Rosenberg
Reinforcement Learning Policy OpTimization Optimal Regret Policy Optimization Stochastic Shortest Path

January 31, 2022

January 18, 2022

Conservative Distributional Reinforcement Learning with Safety Constraints
Hengrui Zhang, Youfang Lin, Sheng Han, Shuo Wang, Kai Lv
Policy Reinforcement Learning Safety Constraint Policy Optimization Distributional Reinforcement Learning Constrained Markov Decision Process Policy Algorithm

December 28, 2021

Efficient Performance Bounds for Primal-Dual Reinforcement Learning from Demonstrations
Angeliki Kamoutsi, Goran Banjac, John Lygeros
Inverse Reinforcement Learning Noisy Demonstration Policy Optimization Stochastic Convex Optimization Low Regret Performance Bound Primal Dual Reinforcement Learning

December 21, 2021

Nearly Optimal Policy Optimization with Stable at Any Time Guarantee
Tianhao Wu, Yunchang Yang, Han Zhong, Liwei Wang, Simon S. Du, Jiantao Jiao
Reinforcement Learning Optimal Policy Policy Optimization Stable Code Minimax Q Learning

December 8, 2021

DiPS: Differentiable Policy for Sketching in Recommender Systems
Aritra Ghosh, Saayan Mitra, Andrew Lan
Recommender System Sequential Recommendation Policy Optimization Sequential Recommender System

November 16, 2021

CLARA: A Constrained Reinforcement Learning Based Resource Allocation Framework for Network Slicing
Yongshuai Liu, Jiaxin Ding, Zhi-Li Zhang, Xin Liu
Reinforcement Learning Policy Optimization Network Slicing Network Slice Constrained Reinforcement LeArning

November 7, 2021

Coordinated Proximal Policy Optimization
Zifan Wu, Chao Yu, Deheng Ye, Junge Zhang, Haiyin Piao, Hankz Hankui Zhuo
Proximal Policy Optimization Policy Optimization Multi Agent Setting Multi Agent Proximal Policy Optimization Matrix Game

Policy Optimization

Papers

Beyond the Policy Gradient Theorem for Efficient Policy Updates in Actor-Critic Algorithms

Understanding Curriculum Learning in Policy Optimization for Online Combinatorial Optimization

Policy Optimization for Stochastic Shortest Path

Learning Infinite-Horizon Average-Reward Markov Decision Processes with Constraints

You May Not Need Ratio Clipping in PPO

Conservative Distributional Reinforcement Learning with Safety Constraints

Efficient Performance Bounds for Primal-Dual Reinforcement Learning from Demonstrations

Nearly Optimal Policy Optimization with Stable at Any Time Guarantee

DiPS: Differentiable Policy for Sketching in Recommender Systems

CLARA: A Constrained Reinforcement Learning Based Resource Allocation Framework for Network Slicing

Coordinated Proximal Policy Optimization