Policy Gradient Methods

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, up 133% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 108

All topics

Bayesian policy gradient and actor-critic algorithms
Apr 30, 2026Mohammad Ghavamzadeh, Yaakov Engel, Michal ValkoBayesian RL Temporal-Difference Learning
Adobe Research & Inria · Rafael Advanced Defence System, Israel · Inria Lille — SequeL team, France
K-Score: Kalman Filter as a Principled Alternative to Reward Normalization in Reinforcement Learning
Apr 24, 2026Zixuan Xia, Quanxi LiReinforcement Learning Policy Gradient Methods
University of Bern, Bern, Switzerland.
Dynamical Priors as a Training Objective in Reinforcement Learning
Apr 23, 2026Sukesh SubaharanReinforcement Learning Policy Gradient Methods
Coimbatore Medical College Coimbatore, TN India
EVPO: Explained Variance Policy Optimization for Adaptive Critic Utilization in LLM Post-Training
Apr 21, 2026Chengjun Pan, Shichun Liu, Jiahang Lin +10RL for Language Models Policy Gradient Methods
Peking University · Fudan University · Shanghai AI Lab +1
Intentional Updates for Streaming Reinforcement Learning
Apr 21, 2026Arsalan Sharifnassab, Mohamed Elsayed, Kris De Asis +2Reinforcement Learning Temporal-Difference Learning
Openmind Research Institute · Department of Computing Science, University of Alberta.
Policy Gradient Primal-Dual Method for Safe Reinforcement Learning from Human Feedback
Apr 21, 2026Qiang Liu, Adrienne Kline, Ermin WeiConstrained RL Policy Gradient Methods
Department of Electrical and Computer Engineering, Northwestern University, Evanston, IL, USA · Center for Artificial Intelligence, Bluhm Cardiovascular Institute, Northwestern Medicine, Chicago, IL, USA · Department of Surgery, Northwestern University, Chicago, IL, USA +3
COSAC: Counterfactual Credit Assignment in Sequential Cooperative Teams
Apr 20, 2026Shripad Deshmukh, Jayakumar Subramanian, Raghavendra Addanki +1Sequential MARL Credit Assignment in RL
College of Information and Computer Sciences University of Massachusetts Amherst · Adobe Research San Jose, CA
Beyond Importance Sampling: Rejection-Gated Policy Optimization
Apr 16, 2026Ziwu Sun, Zhen Gao, Jiyong Zhang +1Importance Sampling Policy Optimization
Softmax gradient policy for variance minimization and risk-averse multi armed bandits
Mar 31, 2026Gabriel TuriniciMulti-Armed Bandits Policy Gradient Methods
CEREMADE, Université Paris Dauphine - PSL, CNRS, Paris, France
ARLArena: A Unified Framework for Stable Agentic Reinforcement Learning
Feb 25, 2026Xiaoxuan Wang, Han Zhang, Haixin Wang +11Agentic RL Reinforcement Learning
Department of Computer Science, University of California, Los Angeles, California, USA · University of Wisconsin–Madison, Madison, Wisconsin, USA
Reinforcement learning with an expectile-based objective
Feb 10, 2026Shrey Rakeshkumar Patel, Sumedh Gupte, Soumen Pachal +2Reinforcement Learning Risk-Sensitive RL
Indian Institute of Technology Madras · TCS Research, India
The Optimal Token Baseline: Variance Reduction for Long-Horizon LLM-RL
Feb 6, 2026Yingru Li, Jiawei Xu, Ziniu Li +10RL for Language Model Reasoning Policy Gradient Methods
The Chinese University of Hong Kong, Shenzhen · Fudan University · Hong Kong University of Science and Technology +4
Rethinking the Design Space of Reinforcement Learning for Diffusion Models: On the Importance of Likelihood Estimation Beyond Loss Design
Feb 4, 2026Jaemoo Choi, Yuchen Zhu, Wei Guo +6T2I Generation Diffusion Model Fine-Tuning
Georgia Institute of Technology · National University of Singapore · Nanjing university.
Model-Free Output Feedback Stabilization via Policy Gradient Methods
Jan 27, 2026Ankang Zhang, Ming Chi, Xiaoling Wang +1Dynamical Systems Partially Observable RL
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China · College of Automation, Nanjing University of Posts and Telecommunications, Nanjing 210023, China
Trust Region Masking for Long-Horizon LLM Reinforcement Learning
Dec 28, 2025Yingru Li, Jiacai Liu, Jiawei Xu +4RL for Language Models Reinforcement Learning
Fudan University · The Chinese University of Hong Kong, Shenzhen
Rethinking the Global Convergence of Softmax Policy Gradient with Linear Function Approximation: The Case of Multi-Armed Bandits
May 6, 2025Max Qiushi Lin, Jincheng Mei, Matin Aghaei +6Multi-Armed Bandits Policy Gradient
Simon Fraser University · Google DeepMind · Google Research +1
Reflective Policy Optimization
Jun 6, 2024Yaozhong Gan, Renye Yan, Zhe Wu +1Policy Optimization Policy Gradient Methods
QiYuan Lab.
On the Approximation and Convergence of Distributional Policy Gradient Algorithms for Risk-Sensitive Reinforcement Learning
May 23, 2024Xian Yu, Minheng Xiao, Lei YingStochastic Optimization Distributional RL
Department of Integrated Systems Engineering, The Ohio State University, Columbus, OH, USA · Department of Electrical Engineering and Computer Science, University of Michigan, Ann Arbor, MI, USA

Apr 30, 2026, cs.LGJ/K move · Enter open · S save

Bayesian policy gradient and actor-critic algorithms

Mohammad Ghavamzadeh, Yaakov Engel, Michal Valko

Adobe Research & Inria · Rafael Advanced Defence System, Israel · Inria Lille — SequeL team, France

Bayesian RL Temporal-Difference Learning Policy Gradient Methods Actor-Critic Methods Policy Learning

Open paper AskPDF arXiv

Policy gradient methods are reinforcement learning algorithms that adapt a parameterized policy by following a performance gradient estimate. Conventional policy gradient methods use Monte-Carlo techniques to estimate the gradient, which tend to have high variance, requiring many samples and resulting in slow convergence. We first propose a Bayesian framework for policy gradient, based on modeling the policy gradient as a Gaussian process. This reduces the number of samples needed to obtain accurate gradient estimates. Moreover, estimates of the natural gradient and a measure of the uncertainty in the gradient estimates, namely, the gradient covariance, are provided at little extra cost. Since the proposed framework considers system trajectories as its basic observable unit, it does not require the dynamics within trajectories to be of any particular form, and can be extended to partially observable problems. On the downside, it cannot exploit the Markov property when the system is Markovian. To address this, we supplement our Bayesian policy gradient framework with a new actor-critic learning model in which a Bayesian class of non-parametric critics, based on Gaussian process temporal difference learning, is used. Such critics model the action-value function as a Gaussian process, allowing Bayes rule to be used to compute the posterior distribution over action-value functions, conditioned on the observed data. Appropriate choices of the policy parameterization and of the prior covariance (kernel) between action-values yield closed-form expressions for the posterior of the gradient of the expected return with respect to the policy parameters. We perform detailed experimental comparisons of the proposed Bayesian policy gradient and actor-critic algorithms with classic Monte-Carlo based policy gradient methods, on a number of reinforcement learning problems.

Momentum

Related topics

Latest papers 108