Policy Gradient

Follow this topic Ask about this topic

Momentum

10 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 75

All topics

OPTS-TTPO: Enhancing Finite-Sample Policy-Gradient Learning with Tree Search
Sep 30, 2026Junyu Lu, Shichao Weng, Zhiqiang Wang +10Frictive Policy Optimization Policy Gradient
Dobot Robotics · Independent Researcher · Zhejiang University +2
DAMPER: Return-Prioritized Gradient Control for Smooth Policies
Sep 30, 2026Seokmin Ko, Taewon Goo, Kihyuk HongSoft Actor-Critic Policy Gradient
KAIST
Sufficiency of Zeroth-Order Reward Shaping for Policy Gradient in Stabilization Control
Sep 28, 2026Yisheng Zhang, Tao Wang, Sicun GaoPotential-Based Reward Shaping Policy Gradient
University of California San Diego, La Jolla, CA, USA · Tsinghua University, Beijing, China
HiLoRe: What to Store, Compress, or Recompute for Efficient GRPO Training
Sep 27, 2026Xinrui Chen, Mengyang Li, Ou Wu +1Policy Gradient
Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou, China · Tianjin Key Laboratory of Wireless Mobile Communications and Power Transmission, Tianjin Normal University, Tianjin, China · University of Southern Queensland
OSCC: Certified Observation-Safe Coupling Optimization for Gradient-Noise Control in Imperfect-Information Learning
Sep 27, 2026Miaobo Hu, Shuhao Hu, Xiaobo Guo +5Optimal Learner Policy Gradient
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China
Elucidating the Design Space of Regression-based Diffusion Reinforcement Learning
Sep 27, 2026Toyota Li, David Zhao, Alan ZhaoDiffusion-Based Reinforcement Learning Methods Reward Gradients
Tencent
Reinforcement Learning with Decomposed Subtasks
Sep 22, 2026Mattie Terzolo, Mikolaj Sacha, Ayan Sinha +1Policy Gradient Agentic Benchmarks
Upwork
PACT: From Credit Assignment to Critic Alignment
Sep 22, 2026Jiayan Fu, Hang Xu, Yong Zhang +4Critic-Free Reinforcement Learning Credit Assignment
AllSpark Team
CAST: Alternating State-Value Targets and Expanded Policy Gradients for Model-Based Reinforcement Learning
Sep 8, 2026Pietro Noah Crestaz, Mohamed Yassine Kabouri, Nicolas Mansard +1Model-Based Reinforcement Learning Policy Gradient
1Industrial Engineering Department, University of Trento, Trento, Italy. · 2LAAS-CNRS, Universit´e de Toulouse, CNRS, Toulouse, France. · 4Artificial and Natural Intelligence Toulouse Institute (ANITI), Toulouse. +1
Emergent Charging Coordination in Electric Delivery Fleets
Sep 7, 2026Javier Vales-Alonso, Juan J. AlcarazElectric Vehicles Demand Response
Department of Information and Communication Technologies, Universidad Politécnica de Cartagena, Cartagena, Spain
Boundary-Seeking Policy Gradient for Safe Reinforcement Learning
Aug 10, 2026Chenhua Fan, Jiahui Zhu, Yuhang Zhang +1Policy Gradient Safety Constraints
School of EECS, Washington State University, Pullman, WA 99163, USA
Wasserstein Policy Gradient for Entropy-Regularized Linear-Quadratic Control
Aug 7, 2026Zhaoyu Zhu, Rui Gao, Shuang LiEntropy Regularized Reinforcement Learning Linear Quadratic Regulator
University of Illinois Urbana-Champaign · The University of Texas at Austin · The Chinese University of Hong Kong, Shenzhen
Convergence and Regret of the Policy Gradient for Multi-Armed Bandits in Diffusion Environment
Jul 31, 2026Yanwei Jia, Du OuyangPolicy Gradient Stochastic Multi-Armed Bandits
Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong SAR · Department of Mathematical Sciences, Tsinghua University, Beijing, Beijing 100084, China
Policy Gradient Steering: Interventions from Behavioral Objectives
Jul 30, 2026Yoann Poupart, Aurélie Beynier, Nicolas MaudetPolicy Gradient Steering
LIP6, Sorbonne University Paris, France
Early Verdicts, Better Budgets: Sequential Adaptive Rollout Allocation for Compute-Efficient RLVR
Jul 28, 2026Pixel Nomand, Elena Voss, Marcus Hale +1Reinforcement Learning With Verifiable Reward Token Budget Allocation
University of Wisconsin–Madison · University of Washington
On the Impossibility of Unbiased and Length-Invariant Policy Optimization with Outcome Rewards
Jul 25, 2026Fei Ding, Yongkang Zhang, Yuhao Liao +2Policy Gradient Frictive Policy Optimization
Alibaba Group · Tsinghua University
Finite-Time Analysis of the Natural Policy Gradient in Finite-Horizon Markov Decision Processes
Jul 25, 2026Asha Barua, Sajad KhodadadianPolicy Gradient Markov Decision Processes
Grado Department of Industrial and Systems Engineering, Virginia Polytechnic Institute and State University, Blacksburg, Virginia 24061, USA.
Cross-Domain Off-Policy Evaluation and Learning for Contextual Bandits
Jul 24, 2026Yuta Natsubori, Masataka Ushiku, Yuta SaitoContextual Bandit Framework Off-Policy Learning
Hakuhodo DY Holdings, Inc. · Cornell University
Optimizing Regret
Jul 21, 2026Irene AldridgeRegret Policy Gradient
Risk AI Center
Mask-Aware Policy Gradients for Diffusion Language Models
Jul 16, 2026Haran Raajesh, Kulin Shah, Adam Klivans +1Masked Diffusion Language Models Large Language Model Reinforcement Learning
Department of Computer Science The University of Texas at Austin
Environment Parameter Gradient Theorem for Policy-Environment Co-Design in Reinforcement Learning
Jul 14, 2026Amber SrivastavaPolicy Gradient Value Functions
Faculty of Mechanical Engineering, Indian Institute of Technology Delhi, India
SafeExplorer: An Unbiased Policy Gradient for Reinforcement Learning with Recovery Interventions
Jul 9, 2026Elham Daneshmand, Majid Khadiv, Glen Berseth +1Proximal Policy Optimization Policy Gradient
McGill University · Mila · Technical University of Munich +1
Mean Field Reinforcement Learning
Jul 1, 2026René Carmona, Mathieu LaurièreStochastic Optimal Control Multi-Agent Reinforcement Learning
Department of Operations Research and Financial Engineering & Program in Applied and Computational Mathematics, Princeton NJ 08544, USA · Shanghai Frontiers Science Center of Artificial Intelligence and Deep Learning; NYU-ECNU Institute of Mathematical Sciences at NYU Shanghai; NYU Shanghai, 567 West Yangsi Road, Shanghai, 200126, People’s Republic of China
Superhuman AI for Generals.io Using Self-Play Reinforcement Learning
Jun 22, 2026Matej Straka, Viliam Lisý, Martin SchmidSelf-Play Simulation-Based Reinforcement Learning
Department of Applied Mathematics, Charles University, Prague, Czech Republic · Department of Computer Science, Faculty of Electrical Engineering, Czech Technical University in Prague, Czech Republic · Charles University +1
Modularized Reinforcement Learning on LLMs: From MDP Creation to Exploration and Learning
Jun 20, 2026Zhao Yang, Yuxuan Jiang, Ting-Chih Chen +18Large Language Model Reinforcement Learning Offline Reinforcement Learning
VU Amsterdam, The Netherlands · University of Maryland, Baltimore County, USA · Leiden University, The Netherlands +4
Backpropagating Through Simulation: Analytic Policy Gradients for Sample and Learning Efficient Differentiable Continuous Control
Jun 19, 2026Yueci DengPolicy Gradient Frictive Policy Optimization
School of Data Science, The Chinese University of Hong Kong, Shenzhen
Redesigning Regularization for Effective Policy Smoothing
Jun 11, 2026Taisuke Kobayashi, Naoto YamanakaSmoothing Policy Gradient
National Institute of Informatics (NII) and with The Graduate University for Advanced Studies (SOKENDAI), 2-1-2 Hitotsubashi, Chiyoda-ku, Tokyo, 101-8430, Japan
Keep Policy Gradient in Charge: Sibling-Guided Credit Distillation for Long-Horizon Tool-Use Agents
Jun 10, 2026Tianyu Ding, Jianhong Xin, Juan Pablo De la Cruz WeinsteinBetter Call Group Relative Policy Optimization Policy Gradient
Amazon Web Services
On Advantage Estimates for Max@K Policy Gradients
Jun 4, 2026Shota Takashiro, Soichiro Nishimori, Paavo Parmas +6Policy Gradient Step-Level Advantage Estimation
The University of Tokyo
Selective-Advantage Entropy-Adaptive Horizon GRPO: Asymmetric Token-Level Discounting for Efficient Reinforcement Learning of Language Models
Jun 3, 2026Chirag Chawla, Rohan Charudatt Salvi, Madhav S. BaidyaPolicy Gradient Offline Reinforcement Learning
Indian Institute of Technology (BHU), Varanasi, India · Department of Computer Science, University of Illinois Chicago, IL 60607, USA
Policy Gradient for Continuous-Time Robust Markov Decision Processes
Jun 3, 2026Tanya Veeravalli, David M. Bossens, Atsushi NitandaMarkov Decision Processes Policy Gradient
Centre for Frontier AI Research, Agency for Science, Technology and Research (A*STAR) · Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR) · College of Computing and Data Science, Nanyang Technological University
EchoRL: Reinforcement Learning via Rollout Echoing
May 29, 2026Jinhe Bi, Aniri, Minglai Yang +9Reinforcement Learning With Verifiable Reward Large Language Model Training
Huawei Heisenberg Research Center · LMU Munich · Munich Center for Machine Learning +5
Zero Collapse: A Failure Mode of Policy Gradient Methods in Discontinuous Reward Environments
May 29, 2026Nishant Kumar, Enrique Areyan Viqueira, Amy GreenwaldPolicy Gradient
Convergence of Steepest Descent and Adam under Non-Uniform Smoothness
May 28, 2026Sharan Vaswani, Yifan Sun, Reza BabanezhadGradient Descent Smoothness
1Simon Fraser University · 2Stony Brook University · 3Samsung AI, Montreal.
ProRL: Effective Reinforcement Learning for Proactive Recommendation via Rectified Policy Gradient Estimation
May 27, 2026Hongru Hou, Tiehua Mei, Denghui Geng +5Sequential Recommendation Reward Functions
Training-Inference Kernel Contracts: Bounding Divergence in Post-Training and Deployment
May 26, 2026Bruce Changlong Xu, Lan WuReinforcement Learning Post-Training Post-Training
Completion vs Optimality: Policy Gradient in Long-Horizon Cumulative-Damage Problems
May 26, 2026Wolfgang Maass, Sabine JanzenPolicy Gradient Optimality
Saarland University & German Research Center for Artificial Intelligence (DFKI) · German Research Center for Artificial Intelligence (DFKI)
Sample Complexity of Policy Gradient for Log-Growth Control
May 26, 2026Qiuhua Pan, Yukai Shen, Liwei Zhang +2Policy Gradient Stochastic Optimal Control
State Key Laboratory of Submarine Geoscience, School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai 200240, China · Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai 200240, China · Shanghai Key Laboratory of Perception and Control in Industrial Network Systems, Shanghai 200240, China +1
Efficient On-policy Visual-RL via Stochastic Decoupled Policy Gradient
May 26, 2026Haoxiang You, Yilang Liu, Davis Zong +5Visuomotor Policy Frictive Policy Optimization
Yale University, Department of Mechanical Engineering. · Yale University, Department of Computer Science. · Shanghai Jiao Tong University, School of Computer Science. +1
Credit-assigned Policy Gradient for Early Stage Retrieval in Two-stage Ranking
May 25, 2026Haruka Kiyohara, Mihaela Curmei, Ariel Evnine +7Policy Gradient Gradient
Computer Science Department, Cornell University, Ithaca, NY, USA · Central Applied Science, Meta, Menlo Park, CA, USA.
Global Convergence of Wasserstein Policy Gradient for Entropy-Regularized Reinforcement Learning
May 25, 2026Zhaoyu Zhu, Rui Gao, Shuang LiEntropy Regularized Reinforcement Learning Policy Gradient
Shanghai Jiao Tong University · The University of Texas at Austin · The Chinese University of Hong Kong, Shenzhen
DeepSeekMath Meets Order Book: Group-Aware Policy Optimization for High-Frequency Directional Trading
May 25, 2026Sayak Charabarty, Souradip PalPolicy Gradient Reinforcement Learning Baseline
Department of Computer Science Northwestern University Evanston, IL 60208, USA · School of Electrical and Computer Engineering Purdue University West Lafayette, IN 47906, USA
Global linear convergence of entropy-regularized softmax policy gradient beyond tabular MDPs
May 24, 2026Ziyue Chen, David Šiška, Lukasz SzpruchEntropy Regularized Reinforcement Learning Policy Gradient
DelTA: Discriminative Token Credit Assignment for Reinforcement Learning from Verifiable Rewards
May 20, 2026Kaiyi Zhang, Wei Wu, Yankai LinReinforcement Learning With Verifiable Reward Credit Assignment
Gaoling School of Artificial Intelligence, Renmin University of China · Ant International
Randomized Advantage Transformation (RAT): Computing Natural Policy Gradients via Direct Backpropagation
May 18, 2026Mingfei SunPolicy Gradient Backpropagation
The University of Manchester, United Kingdom.
The Dynamics of Policy Gradient in Social Dilemmas with Partner Selection
May 18, 2026Benedict Russell, Chin-wing Leung, Paolo TurriniCooperation Policy Gradient
Mathematics Institute Warwick University · Department of Computer Science Warwick University
Equilibrium Selection in Multi-Agent Policy Gradients via Opponent-Aware Basin Entry
May 18, 2026Yevhen Shcherbinin, Arina Redina, Maxim Kalpin +1Nash Equilibrium Equilibrium
Bloomsbury Technology, London, United Kingdom · London School of Economics and Political Science, London, United Kingdom · University of Bristol, Bristol, United Kingdom +2
Scalable Bi-causal Optimal Transport via KL Relaxation and Policy Gradients
May 17, 2026Haoyang Cao, Jesse Hoekstra, Renyuan Xu +2Optimal Transport Approach Stochastic Optimization
Department of Applied Mathematics and Statistics, Data Science and AI Institute, and Mathematical Institute for Data Science, Johns Hopkins University. · Department of Statistics, Oxford-Man Institute of Quantitative Finance, and Nuffield College, University of Oxford. · Management Science & Engineering Department, Stanford University. +2
Collaborative Yet Personalized Policy Training: Single-Timescale Federated Actor-Critic
May 14, 2026Leo Muxing Wang, Pengkun Yang, Lili SuSoft Actor-Critic Multi-Agent Reinforcement Learning
Northeastern University · Tsinghua University
AIS: Adaptive Importance Sampling for Quantized RL
May 13, 2026Jiajun Zhou, Wei Shao, Lingchao Zheng +2Large Language Model Reinforcement Learning Adaptive Sampling
Huawei · The University of Hong Kong
Delightful Gradients Accelerate Corner Escape
May 12, 2026Jincheng Mei, Ian OsbandPolicy Gradient Optimal Policies
Google DeepMind
Epistemic Uncertainty for Test-Time Discovery
May 11, 2026Kainat Riaz, Muhammad Ahmed Mohsin, Ahsan Bilal +5Model Discovery Stochastic Exploration
Stanford University · University of Oklahoma · National University of Sciences and Technology
Revisiting Policy Gradients for Restricted Policy Classes: Escaping Myopic Local Optima with $k$ -step Policy Gradients
May 11, 2026Alex DeWeese, Guannan QuPolicy Gradient Optimal Policies
Department of Electrical and Computer Engineering Carnegie Mellon University Pittsburgh, PA 15213-3890, USA
Policy Gradient Methods for Non-Markovian Reinforcement Learning
May 11, 2026Avik Kar, Siddharth Chandak, Rahul Singh +4Markov Decision Processes Policy Gradient
Department of Computer Science and Automation, Indian Institute of Science, Bengaluru, India. · Department of Electrical Engineering, Stanford University, Stanford, CA, USA. · Department of Electrical and Electronics Engineering, Nanyang Technological University, Singapore. +1
Natural Policy Gradient as Doubly Smoothed Policy Iteration: A Bellman-Operator Framework
May 11, 2026Phalguni Nanda, Zaiwei ChenPolicy Gradient Boltzmann Policies
Edwardson School of Industrial Engineering, Purdue University
Iterative Critique-and-Routing Controller for Multi-Agent Systems with Heterogeneous LLMs
May 9, 2026Wenzhi Fang, Liangqi Yuan, Guangchen Lan +2Multi-Agent Large Language Model Systems Multi-Agent Coordination
Purdue University · Yonsei University
The Reciprocity Gradient
May 8, 2026Yue Lin, Pascal Poupart, Shuhui Zhu +5Policy Gradient Reputation
The Chinese University of Hong Kong, Shenzhen · LIGHTSPEED · University of Waterloo +2
Asymmetric On-Policy Distillation: Bridging Exploitation and Imitation at the Token Level
May 7, 2026Nan Jia, Haojin Yang, Xing Ma +6Efficient On-Policy Distillation Policy Gradient
Huazhong University of Science and Technology · Peking University · Meituan
Soft Deterministic Policy Gradient with Gaussian Smoothing
May 7, 2026Hyunjun Na, Donghwan LeePolicy Gradient Value Functions
School of Electrical Engineering, KAIST 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Korea
Vanishing L2 regularization for the softmax Multi Armed Bandit
May 5, 2026Stefana-Lucia Anita, Gabriel TuriniciEntropy Regularized Reinforcement Learning Policy Gradient
CEREMADE, Universit´e Paris Dauphine - PSL, CNRS, Paris, France · ‘Octav Mayer” Institute of Mathematics of the Romanian Academy, Bd. Carol I 8, Ias¸i 700505, Romania

Sep 30, 2026, cs.CLJ/K move · Enter open · S save

OPTS-TTPO: Enhancing Finite-Sample Policy-Gradient Learning with Tree Search

Junyu Lu, Shichao Weng, Zhiqiang Wang, Haojie Luo+9

Dobot Robotics · Independent Researcher · Zhejiang University+2

Frictive Policy Optimization Policy Gradient Tree Search On-Policy

Open paper AskPDF arXiv

The policy-gradient theorem gives the exact gradient under the current policy, but finite on-policy samples may miss rare high-return trajectories. We study whether tree search improves their coverage within a fixed budget while controlling gradient bias. We introduce On-Policy Parallel Tree Search (OPTS) and Tree Trajectory Policy Optimization (TTPO) using on-policy tree trajectories, which sample new suffixes from the current policy at visited states. This needs no action-distribution correction, although branching changes state visitation. Our Branch Aggregation Lemma shows that branch-weighted tree statistics recover chain expectations when branch choices and weights are fixed before outgoing transitions are sampled. OPTS selects expansion states using estimated performance differences. Under deterministic dynamics, exact values, and max-backup advantages, the induced search policy's expected return improves monotonically with the budget. We bound the gradient bias from adaptive expansion and show that max backup assigns prefix credit to actions leading to better discovered suffixes. Against a finite chain reference, TTPG's measured bias stays near its no-branching level, while NaivePG's bias grows from 0.1251 to 0.4884. At matched budgets, reward- and value-guided OPTS improve correct-answer coverage and majority-vote accuracy over independent sampling. At matched branch counts, OPTS + TTPG gains coverage with a modest bias increase relative to Fixed-branch + TTPG. Under matched interaction or rollout budgets, OPTS-TTPO improves MuJoCo tail returns over PPO by up to 28.6%, achieves a 34-22-1 win-loss-tie record against PPO on Atari-57 under the last-100-log mean-return metric, and improves micro-averaged avg@32 and pass@32 over PPO across all four Qwen3 models.

Momentum

Related topics

Latest papers 75