Overview

Policy gradient methods optimize a policy directly, using reward or advantage estimates to adjust the likelihood of actions that perform well. Common approaches include REINFORCE, actor–critic and natural policy gradient, alongside deterministic or pathwise gradients and variance-reduction techniques. They are used in bandits, continuous control, multi-agent learning and language-model post-training with verifiable rewards. Theory gives convergence and sample-efficiency guarantees in settings such as tabular MDPs, but exploration, function approximation and noisy or biased gradient estimates remain central challenges.

Latest papers

CardsList

70 papers4 papers in the last four weeks, down 64% on the four weeks before. 0.0% of all new papers.

  1. On Advantage Estimates for Max@K Policy Gradients

    Jun 4, 2026Shota Takashiro, Soichiro Nishimori, Paavo Parmas +6Policy GradientPolicy Gradient Methods

  2. SALT: When More Rollouts Don't Help in Group-Based Policy Optimization and How to Make Them Matter

    Jun 4, 2026Powei Chang, Jinpeng Zhang, Chaoqun Sun +6Policy GradientGroup Relative Policy Optimization

  3. Self-Distilled Policy Gradient

    Jun 2, 2026Yifeng Liu, Shiyuan Zhang, Yifan Zhang +1Policy GradientOn-Policy Self-Distillation

  4. Agentic Transformers Provably Learn to Search via Reinforcement Learning

    May 29, 2026Tong Yang, Yu Huang, Yingbin Liang +1Policy GradientAgentic RL

  5. Cross-Entropy Games and Frost Training

    May 26, 2026Arthur Renard, Franck Gabriel, Valentin Hartmann +1RL for Language ModelsPolicy Gradient

  6. Completion vs Optimality: Policy Gradient in Long-Horizon Cumulative-Damage Problems

    May 26, 2026Wolfgang Maass, Sabine JanzenPolicy GradientProximal Policy Optimization

  7. Value-Gradient Hypothesis of RL for LLMs

    May 20, 2026Arip Asadulaev, Daniil Ognev, Karim Salta +1RL for Language ModelsPolicy Gradient

  8. DelTA: Discriminative Token Credit Assignment for Reinforcement Learning from Verifiable Rewards

    May 20, 2026Kaiyi Zhang, Wei Wu, Yankai LinPolicy GradientToken-Level Credit Assignment

  9. The Dynamics of Policy Gradient in Social Dilemmas with Partner Selection

    May 18, 2026Benedict Russell, Chin-wing Leung, Paolo TurriniSocial DilemmasPolicy Gradient

  10. Distributed Zeroth-Order Policy Gradient for Networked Multi-agent Reinforcement Learning from Human Feedback

    May 15, 2026Pengcheng Dai, He Wang, Dongming Wang +2Policy GradientDecentralized MARL

  11. Achieving ε−2ε^{-2} Sample Complexity for Single-Loop Actor-Critic under Minimal Assumptions

    May 13, 2026Ishaq Hamza, Zaiwei ChenPolicy GradientLast-Iterate Convergence

  12. Delightful Gradients Accelerate Corner Escape

    May 12, 2026Jincheng Mei, Ian OsbandMulti-Armed BanditsPolicy Gradient

  13. Revisiting Policy Gradients for Restricted Policy Classes: Escaping Myopic Local Optima with kk-step Policy Gradients

    May 11, 2026Alex DeWeese, Guannan QuPolicy GradientPolicy Optimization

  14. Natural Policy Gradient as Doubly Smoothed Policy Iteration: A Bellman-Operator Framework

    May 11, 2026Phalguni Nanda, Zaiwei ChenPolicy GradientReinforcement Learning

  15. Soft Deterministic Policy Gradient with Gaussian Smoothing

    May 7, 2026Hyunjun Na, Donghwan LeeRL ControlPolicy Gradient

  16. Non-Myopic Active Feature Acquisition via Pathwise Policy Gradients

    May 6, 2026Linus Aronsson, Morteza Haghir ChehreghaniPolicy GradientActive Learning

  17. Explaining and Preventing Alignment Collapse in Iterative RLHF

    May 5, 2026Etienne Gauthier, Francis Bach, Michael I. JordanReward HackingPolicy Gradient

  18. Vanishing L2 regularization for the softmax Multi Armed Bandit

    May 5, 2026Stefana-Lucia Anita, Gabriel TuriniciMulti-Armed BanditsPolicy Gradient

  19. When Errors Can Be Beneficial: A Categorization of Imperfect Rewards for Policy Gradient

    Apr 28, 2026Shuning Shang, Hubert Strauss, Stanley Wei +2Reward ModelingPolicy Gradient

  20. Does "Do Differentiable Simulators Give Better Policy Gradients?" Give Better Policy Gradients?

    Apr 20, 2026Ku Onoda, Paavo Parmas, Manato Yaguchi +1Policy GradientRobotic RL

  21. Functional Natural Policy Gradients

    Mar 30, 2026Aurelien Bibaut, Houssam Zenati, Thibaud Rahier +1Cross-FittingNatural Policy Gradient