Policy Gradient Methods

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, up 133% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 108

All topics

Randomized Transport Maps for Model-Free Policy-Gradient Mean-Field Control
Oct 8, 2026Adonis Jamal, Samy Mekkaoui, Yadh Hafsi +1Policy Gradient Methods Model-Free RL
École Polytechnique
Temporally Interpretable Differentiable Decision Trees
Oct 7, 2026Eisuke Hirota, Aarav Sane, Rohan PalejaInterpretable ML Interpretability in RL
Department of Computer Science, Purdue University · Thomas Lord Department of Computer Science, University of Southern California
A Closed-Loop Non-Asymptotic Convergence Analysis of PPO with Learned Critics and Clipping
Oct 7, 2026Junwei Su, Mengfan Liu, Yanyong Zhang +1Proximal Policy Optimization Actor-Critic Methods
School of Artificial Intelligence and Data Science, University of Science and Technology of China · Department of Computer Science, University of Hong Kong
Convex-Concave Reinforcement Learning
Oct 6, 2026Shripad V. Deshmukh, Yaswanth Chittepu, Dhawal Gupta +2Reinforcement Learning Policy Gradient Methods
University of Massachusetts · Institute of Foundation Models
OPTS-TTPO: Enhancing Finite-Sample Policy-Gradient Learning with Tree Search
Sep 30, 2026Junyu Lu, Shichao Weng, Zhiqiang Wang +10Tree Search Policy Gradient Methods
Dobot Robotics · Independent Researcher · Zhejiang University +2
Regularized policy gradient with learned mixtures of Gaussians for games with continuous actions
Sep 29, 2026Ondřej Kubíček, Viliam Lisý, Tuomas SandholmGame-Playing Agents Self-Play RL
Czech Technical University in Prague · Carnegie Mellon University · Artificial Intelligence Center
ORPG: Reconciling Multiple Reward Objectives through Objective-wise Policy Gradients
Sep 28, 2026Shicheng Fang, Yiwen Zhao, Wenbo Tian +4Multi-Objective Reinforcement Learning Gradient Interference
Fudan University · Shanghai Innovation Institute
Sufficiency of Zeroth-Order Reward Shaping for Policy Gradient in Stabilization Control
Sep 28, 2026Yisheng Zhang, Tao Wang, Sicun GaoReward Shaping Robotic RL
University of California San Diego, La Jolla, CA, USA · Tsinghua University, Beijing, China
OSCC: Certified Observation-Safe Coupling Optimization for Gradient-Noise Control in Imperfect-Information Learning
Sep 27, 2026Miaobo Hu, Shuhao Hu, Xiaobo Guo +5Partially Observable RL Policy Gradient Methods
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China
Luck Is Not Skill: When Do Paired Rollouts Help Group-Relative RL of LLM Agents?
Sep 21, 2026Nazmus SakibGroup Relative Policy Optimization Policy Gradient Methods
BRAC University, Dhaka, Bangladesh
Continuous Delayed-Memory Stochastic Gradient Descent and Continuous-Time Reinforcement Learning from History of Astrophysical Time Series Studies
Sep 17, 2026Juncheng YiContinuous-Time RL Policy Gradient Methods
Designing Reinforcement Learning for Diffusion Models: A Unified Path-Space View
Aug 14, 2026Yixian Xu, Yuanrui Zhang, Shengjie Luo +2Diffusion Model Alignment Policy Gradient Methods
State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · ByteDance Seed
The Time Value of Evolution
Aug 13, 2026Matthew Siper, Ahmed Khalifa, Julian TogeliusEvolutionary Optimization Policy Gradient Methods
Nof1, New York University · Nof1, University of Malta
Boundary-Seeking Policy Gradient for Safe Reinforcement Learning
Aug 10, 2026Chenhua Fan, Jiahui Zhu, Yuhang Zhang +1Markov Decision Processes Constrained RL
School of EECS, Washington State University, Pullman, WA 99163, USA
Wasserstein Policy Gradient for Entropy-Regularized Linear-Quadratic Control
Aug 7, 2026Zhaoyu Zhu, Rui Gao, Shuang LiMaximum Entropy RL Policy Gradient Methods
University of Illinois Urbana-Champaign · The University of Texas at Austin · The Chinese University of Hong Kong, Shenzhen
LC-GRPO: Bridging Train-Inference Gap for Flow-Based GRPO with Langevin Correction
Aug 6, 2026Yingqing Guo, Hui Yuan, Zijian He +2RL for Video Generation RL for Image Generation
Convergence and Regret of the Policy Gradient for Multi-Armed Bandits in Diffusion Environment
Jul 31, 2026Yanwei Jia, Du OuyangMulti-Armed Bandits Regret Minimization in RL
Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong SAR · Department of Mathematical Sciences, Tsinghua University, Beijing, Beijing 100084, China
CoRT: Counterfactual Replay for Token-Level Rubric-Guided Policy Optimization
Jul 28, 2026Bo-Wen Zhang, Junwei He, Wen Wang +5Token-Level Credit Assignment Rubric-Based RL
State Key Laboratory of Novel Software Technology, Nanjing University · School of Intelligence Science and Technology, Nanjing University · ByteDance +1
Finite-Time Analysis of the Natural Policy Gradient in Finite-Horizon Markov Decision Processes
Jul 25, 2026Asha Barua, Sajad KhodadadianPolicy Gradient Markov Decision Processes
Grado Department of Industrial and Systems Engineering, Virginia Polytechnic Institute and State University, Blacksburg, Virginia 24061, USA.
Exposure-Based Reinforcement Learning to Rank
Jul 21, 2026Harrie Oosterhuis, Rolf Jagerman, Zhen Qin +1Reinforcement Learning Learning to Rank
University of Amsterdam · Google DeepMind
Scalable Policy Optimization for Networked Multi-Agent Reinforcement Learning with Continuous State-Action Spaces
Jul 20, 2026Dongming Wang, Pengcheng Dai, Wenwu Yu +1Multi-Agent Control Temporal-Difference Learning
Department of Electrical and Computer Engineering, University of California, Riverside, CA 92521, USA · Engineering Systems and Design Pillar, Singapore University of Technology and Design, Singapore 487372 · Frontiers Science Center for Mobile Information Communication and Security, School of Mathematics, Southeast University, Nanjing 211102, China +1
CLaC@FinMMEval 2026 Task 3: Sentiment-Augmented Deep Reinforcement Learning for Active Trading -- An Alpha-Reward Approach
Jul 17, 2026Andrei Neagu, Eeham Khan, Leila KosseimReinforcement Learning Deep Q-Learning
Department of Computer Science and Software Engineering, Concordia University, Montréal, Canada
Mask-Aware Policy Gradients for Diffusion Language Models
Jul 16, 2026Haran Raajesh, Kulin Shah, Adam Klivans +1Masked Diffusion Models RL for Language Model Reasoning
Department of Computer Science The University of Texas at Austin
Branching Policy Optimization: Sandbox-Native Language Agent Reinforcement Learning
Jul 15, 2026Bowei He, Yankai Chen, Xiaokun Zhang +1Reinforcement Learning Policy Gradient Methods
Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · McGill Univeristy, Montreal, Canada · City University of Hong Kong, Hong Kong SAR.
UP: Unbounded Positive Asymmetric Optimization for Breaking the Exploration-Stability Dilemma
Jul 8, 2026Chongyu Fan, Pengfei Liu, Jingjia Huang +2RL for Language Model Reasoning Policy Optimization
ByteDance Seed · Michigan State University
ACPO: Adaptive Credit Policy Optimization via Fine-Grained Surrogate Entropy
Jul 3, 2026Zijun Xie, Yuyang You, Yongzhi Li +6Token-Level Credit Assignment RL for Language Model Reasoning
Peking University · Baidu Inc. · Kuaishou Inc.
One More Time: Revisiting Neural Quantum States from a Reinforcement Learning Perspective
Jul 2, 2026Juan Agustín Duque, Sergio García Heredia, Vinicius Hernandes +4Neural Quantum States Second-Order Optimization
Mila Quebec AI Institute, Université de Montréal, Montréal, Canada · Applied Quantum Algorithms ⟨aQaL⟩, LIACS & LION, Leiden University, Leiden, Netherlands · QuTech and Kavli Institute of Nanoscience, Delft University of Technology, Delft, Netherlands +1
Mean Field Reinforcement Learning
Jul 1, 2026René Carmona, Mathieu LaurièreReinforcement Learning Q-Learning
Department of Operations Research and Financial Engineering & Program in Applied and Computational Mathematics, Princeton NJ 08544, USA · Shanghai Frontiers Science Center of Artificial Intelligence and Deep Learning; NYU-ECNU Institute of Mathematical Sciences at NYU Shanghai; NYU Shanghai, 567 West Yangsi Road, Shanghai, 200126, People’s Republic of China
GRPO, Dr. GRPO, and DAPO Are Three Operations on One Number: The Group-Standard-Deviation Identity
Jun 30, 2026Yong Yi Bay, Kathleen A. YearickReinforcement Learning RL for Language Model Reasoning
PhD, University of Illinois at Urbana-Champaign
Smart charging of large fleets of Electric Vehicles: Independent Multi-Agent Reinforcement Learning approaches
Jun 30, 2026Xavier Rate, Eloann Le Guern, Raphaël Féraud +8Multi-Agent Coordination Electric Vehicle Charging
Orange · ENS Rennes · IMT Atlantique +1

Oct 8, 2026, math.OCJ/K move · Enter open · S save

Randomized Transport Maps for Model-Free Policy-Gradient Mean-Field Control

Adonis Jamal, Samy Mekkaoui, Yadh Hafsi, Huyên Pham

École Polytechnique

Policy Gradient Methods Model-Free RL

Open paper AskPDF arXiv

We develop a model-free policy gradient method for discrete-time mean-field control (MFC). In MFC, the policy affects the objective both through the controlled dynamics and through the population distribution. Standard REINFORCE estimators capture the first effect but not the second. We introduce Transport REINFORCE, a transport map-based approach that perturbs a suitable transformation of the population distribution to estimate this missing mean-field contribution. The method applies to both finite and continuous state spaces. In finite state spaces, we perturb the population distribution directly on the probability simplex through a convex combination of the current population weights and random weights. In continuous state spaces, we project the population distribution onto the manifold of Gaussian mixtures, and then randomize it via a transport map that ensures the perturbed law remains within this manifold. We prove consistency of the perturbed objective and gradient as the perturbation vanishes, and derive bias and mean-square error bounds for the resulting sample-based gradient estimator. Numerical experiments on several MFC benchmarks show that Transport REINFORCE improves over standard REINFORCE.

Momentum

Related topics

Latest papers 108