Policy Learning

Follow this topic Ask about this topic

Momentum

79 papers in the last four weeks, up 316% on the four weeks before. 0.8% of all new papers.

Jul 13Week of Sep 28

Latest papers 416

All topics

SALT: When More Rollouts Don't Help in Group-Based Policy Optimization and How to Make Them Matter
Jun 4, 2026Powei Chang, Jinpeng Zhang, Chaoqun Sun +6Policy Gradient Group Relative Policy Optimization
Bilibili Inc. · Fudan University · Zhejiang University
Agentic Monte Carlo: Simulating Reinforcement Learning for Black-Box Agents
Jun 3, 2026Dae Yon Hwang, Raunaq Suri, Valentin Villecroze +4Reinforcement Learning Bayesian RL
Layer 6 AI, Toronto, Canada.
Reinforcement Learning from Rich Feedback with Distributional DAgger
Jun 3, 2026Rishabh Agrawal, Jacob Fein-Ashley, Paria RashidinejadDistributional RL RL for Language Model Reasoning
University of Southern California
SocraticPO: Policy Optimization via Interactive Guidance
Jun 3, 2026Zirui Liu, Jie Ouyang, Qi Liu +8RL for Language Model Reasoning LLM-Guided RL
State Key Laboratory of Cognitive Intelligence, University of Science and Technology of China · iFLYTEK AI Research (Central China), iFLYTEK Co., Ltd
Rollout-Level Advantage-Prioritized Experience Replay for GRPO
Jun 3, 2026Gyeongtae Yoo, Sanghyeok Park, Soohyuk Jang +2RL for Language Model Reasoning Group Relative Policy Optimization
Department of Electrical and Computer Engineering, Seoul National University · Interdisciplinary Program in AI, Seoul National University · AIIS, ASRI, INMC, and ISRC, Seoul National University
Policy Gradient for Continuous-Time Robust Markov Decision Processes
Jun 3, 2026Tanya Veeravalli, David M. Bossens, Atsushi NitandaRobust Markov Decision Processes Markov Decision Processes
Centre for Frontier AI Research, Agency for Science, Technology and Research (A*STAR) · Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR) · College of Computing and Data Science, Nanyang Technological University
Policy-based Foveated Imaging and Perception
Jun 1, 2026Howard Xiao, Jan Ackermann, Boyang Deng +1Active Perception Policy Learning
Stanford University, USA
Coherent Off-Policy Improvement of Large Behavior Models with Learned Rewards
Jun 1, 2026Christian Scherer, Joe Watson, Theo Gruner +3Reinforcement Learning Robot Skill Learning
Technical University of Darmstadt · Zuse School ELIZA · University of Oxford +2
World-Task Factorization for Robot Learning
Jun 1, 2026Eduardo Sebastián, Adrian Pfisterer, Vito Mengers +2Robotic Control Robot Policy Learning
Department of Computer Science and Technology, University of Cambridge, United Kingdom · Robotics and Biology Laboratory, Technische Universität Berlin, Germany · Robotics and Biology Laboratory, Technische Universität Berlin +1
Training-Free Imitation Learning with Closed-Form Diffusion Policies
May 31, 2026Raghav Mishra, Ian R. ManchesterDiffusion Model Guidance Visuomotor Policy Learning
Australian Center for Robotics, ARIAM Hub, and School of Aerospace, Mechanical and Mechatronic Engineering University of Sydney
Lagrangian Perturbation Diffusion Steering: Latent Reinforcement Learning for Generative Policies
May 31, 2026Hikmet Simsir, Ozgur S. OguzRobot Policy Adaptation Reinforcement Learning
Department of Computer Engineering, Bilkent University, Ankara, Türkiye.
Decision-Focused On-Policy Learning for Contextual Linear Optimization with Partial Feedback
May 31, 2026Wyame Benslimane, Tinghan Ye, Pascal Van Hentenryck +1Predict-Then-Optimize Decision-Focused Learning
Department of Industrial Engineering and Operations Research University of California, Berkeley · H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology
Decoupled Behavioral Cloning for Scalable Inductive Generalization in RL from Specifications
May 30, 2026Vignesh Subramanian, Subhajit Roy, Suguman BansalReinforcement Learning Meta-Reinforcement Learning
School of Computer Science, Georgia Institute of Technology, USA · Department of Computer Science and Engineering, Indian Institute of Technology Kanpur, India
Drift Q-Learning
May 29, 2026Anas Houssaini, Mohamad H. Danesh, Amin Abyaneh +3Reinforcement Learning Q-Learning
McGill University · Mila - Quebec AI Institute
From Noise to Control: Parameterized Diffusion Policies
May 29, 2026Renhao Zhang, Haotian Fu, Mingxi Jia +3Robot Policy Learning Robot Policy Adaptation
University of Massachusetts · 2Brown University · 3Harvard
Skill Reuse as Compression in Agentic RL
May 29, 2026Zhikun Xu, Yu Feng, Jacob Dineen +3Agentic RL Minimum Description Length
Arizona State University · University of Pennsylvania · University of Southern California
Agentic Transformers Provably Learn to Search via Reinforcement Learning
May 29, 2026Tong Yang, Yu Huang, Yingbin Liang +1Policy Gradient Agentic RL
Department of Electrical and Computer Engineering, Carnegie Mellon University · Department of Statistics and Data Science, Wharton School, University of Pennsylvania · Department of Electrical and Computer Engineering, The Ohio State University +1
Trust-Region Behavior Blending for On-Policy Distillation
May 29, 2026Daniil Plyusov, Alexey Gorbatovski, Alexey Malakhov +4Language Model Distillation Trust-Region Optimization
T-Tech
Safe Equilibrium Policy Optimization for Strategic Agent Policies
May 29, 2026Karthika Arumugam, Kiran Kumar Manku, Amit DhandaMulti-Agent System Optimization Game-Playing Agents
†Amazon, USA
Emergence of Exploration in Policy Gradient Reinforcement Learning via Retrying
May 29, 2026Soichiro Nishimori, Paavo Parmas, Sotetsu Koyamada +4RL Exploration Policy Gradient Methods
The University of Tokyo, Tokyo, Japan · Kobe University, Kobe, Japan · Kyoto University, Kyoto, Japan +3
FLAG: Flow Policy MaxEnt-RL by Latent Augmented Guidance
May 29, 2026Sungha Kim, Gawon Lee, Jusuk Lee +3RL Control Maximum Entropy RL
Seoul National University · Georgia Institute of Technology
When are LLMs Sufficient Policy Optimizers for Sequential RL Tasks?
May 29, 2026Stephane Hatgis-Kessell, Emma BrunskillLanguage Model-Based Control Policy Optimization
Department of Computer Science, Stanford University
HPO: Hysteretic Policy Optimization for Stable and Efficient Training under Sparse-Reward Regime
May 28, 2026Mohamed Sana, Nicola Piovesan, Antonio De Domenico +2Group Relative Policy Optimization Policy Optimization
Paris Research Center, Huawei Technologies, Boulogne-Billancourt, France · Huawei Technologies, China
ESPO: Early-Stopping Proximal Policy Optimization
May 28, 2026Zihang Li, Rui Zhou, Yingcheng Shi +8Early Stopping Reinforcement Learning
1Tongyi Lab , Alibaba Group · 2Peking University
Variance-Adaptive Optimal Algorithm for Reinforcement Learning with Multinomial Logit Function Approximation
May 27, 2026Wonyoung Kim, Min-Hwan Oh, Garud Iyengar +1Regret Minimization in RL Reinforcement Learning
Chung-Ang University · Seoul National University · Columbia University
Commit to the Bit: Reactive Reinforcement Learning Done Right
May 27, 2026Onno Eberhard, Claire Vernade, Michael MuehlebachReinforcement Learning Partially Observable RL
Max Planck Institute for Intelligent Systems, Tübingen, Germany · University of Tübingen · University of Technology Nuremberg.
Off-Policy Learning to Reason Works Because It Is More Pessimistic Than You Think
May 27, 2026Otmane Sakhi, Aleksei Arzhantsev, Imad Aouali +1Reinforcement Learning RL for Language Model Reasoning
Criteo AI Lab · Criteo AI Lab, CREST IP Paris
Learning to Assign Prediction Tasks to Agents with Capacity Constraints
May 27, 2026Shang Wu, Saatvik Kher, Padhraic SmythLearning to Defer Contextual Bandits
Department of Computer Science University of California, Irvine Irvine, CA 92697, USA
SPAR: Support-Preserving Action Rectification
May 27, 2026Jiaxin Zhao, Weihang Pan, Xun Liang +1Imitation Learning Offline RL
Zhejiang University.
BASIS: Batchwise Advantage Estimation from Single-Rollout Information Sharing for LLM Reasoning
May 26, 2026Shijin Gong, Erhan Xu, Kai Ye +3RL for Language Model Reasoning Critic-Free RL
School of Management, University of Science and Technology of China · Department of Statistics, London School of Economics and Political Science · Department of Engineering Science, University of Oxford

Jun 4, 2026, cs.LGJ/K move · Enter open · S save

SALT: When More Rollouts Don't Help in Group-Based Policy Optimization and How to Make Them Matter

Powei Chang, Jinpeng Zhang, Chaoqun Sun, MiniWell Tsao+5

Bilibili Inc. · Fudan University · Zhejiang University

Policy Gradient Group Relative Policy Optimization Reinforcement Learning with Verifiable Rewards Policy Learning

Open paper AskPDF arXiv

Reinforcement learning with verifiable rewards (RLVR) often adopts GRPO-style group-relative updates, sampling multiple rollouts per prompt to construct normalized learning signals. However, merely increasing the number of rollouts does not reliably strengthen learning: under GRPO-style group normalization, per-rollout policy-gradient features can concentrate into a low-rank, signed geometry, causing substantial cancellation during aggregation and weakening the effective update. We address this failure mode with SALT, a Subspace-Adaptive geometry pLug-in componenT that uses sample-wise gradient geometry to reweight the coefficients of group-relative updates. SALT estimates a dominant shared subspace from the mini-batch Gram geometry, decomposes group-relative coefficients into shared and residual channels, and adaptively amplifies the residual channel when signed cancellation is severe. Across diverse reasoning-oriented RLVR benchmarks and model scales, SALT improves effective update geometry and performance without modifying the reward model or the rollout sampling procedure

Momentum

Related topics

Latest papers 416