Policy Learning

Follow this topic Ask about this topic

Momentum

79 papers in the last four weeks, up 316% on the four weeks before. 0.8% of all new papers.

Jul 13Week of Sep 28

Latest papers 416

All topics

Asymmetric On-Policy Distillation: Bridging Exploitation and Imitation at the Token Level
May 7, 2026Nan Jia, Haojin Yang, Xing Ma +6RL for Language Model Reasoning Language Model Distillation
Huazhong University of Science and Technology · Peking University · Meituan
On Training in Imagination
May 7, 2026Nadav Timor, Ravid Shwartz-Ziv, Micah Goldblum +2Reward Modeling Reinforcement Learning
Weizmann Institute of Science · New York University · Columbia University
Skill1: Unified Evolution of Skill-Augmented Agents via Reinforcement Learning
May 7, 2026Yaorui Shi, Yuxin Chen, Zhengxi Lu +6LLM Agent Skill Learning LLM Agent Skill Retrieval
University of Science and Technology of China · 2Meituan · 3National University of Singapore +2
Milestone-Guided Policy Learning for Long-Horizon Language Agents
May 7, 2026Zixuan Wang, Yuchen Yan, Hongxing Li +7Credit Assignment in RL Long-Horizon Agent Tasks
Zhejiang University · Baidu Inc.
Transformers Provably Implement In-Context Reinforcement Learning with Policy Improvement
May 7, 2026Haodong Liang, Lifeng LaiReinforcement Learning Transformer-Based RL
Department of Electrical and Computer Engineering, University of California, Davis
Adaptive Q-Chunking for Offline-to-Online Reinforcement Learning
May 7, 2026Nandiraju Gireesh, Yuanliang Ju, He WangQ-Learning Offline RL
Peking University · Galbot · University of Toronto
Non-Myopic Active Feature Acquisition via Pathwise Policy Gradients
May 6, 2026Linus Aronsson, Morteza Haghir ChehreghaniPolicy Gradient Active Learning
Department of Computer Science and Engineering · Chalmers University of Technology & University of Gothenburg · Gothenburg, Sweden
Approximate Next Policy Sampling: Replacing Conservative Target Policy Updates in Deep RL
May 6, 2026Dillon Sandhu, Ronald ParrPolicy Optimization Policy Iteration
Department of Computer Science, Duke University, USA
LANTERN: LLM-Augmented Neurosymbolic Transfer with Experience-Gated Reasoning Networks
May 6, 2026Mahyar Alinejad, Yue Wang, Amrit Singh Bedi +1Reinforcement Learning LLM-Guided RL
Department of Electrical and Computer Engineering, University of Central Florida, Orlando, Florida, USA · Department of Computer Science, University of Central Florida, Orlando, Florida, USA
Hidden States as Value Gradients: The Pontryagin Structure of Recurrent Policies
May 6, 2026David Leeftink, Max Hinne, Marcel van GervenRL Control Reinforcement Learning
Department of Machine Learning and Neural Computating Donders Institute for Brain, Cognition and Behaviour Radboud University, Nijmegen, the Netherlands.
On-line Learning in Tree MDPs by Treating Policies as Bandit Arms
May 6, 2026Anvay Shah, Ramsundar Anandanarayanan, Sharayu Moharir +1Markov Decision Processes Markov Models
Indian Institute of Technology Bombay · Mumbai, India
SPHERE: Mitigating the Loss of Spectral Plasticity in Mixture-of-Experts for Deep Reinforcement Learning
May 6, 2026Lirui Luo, Guoxi Zhang, Hongming Xu +2Loss of Plasticity Reinforcement Learning
State Key Lab of General AI, School of Intelligence Science and Technology, Peking University · State Key Laboratory of General Artificial Intelligence, BIGAI · Institute for Artificial Intelligence, Peking University
Hierarchical Support Vector State Partitioning for Distilling Black Box Reinforcement Learning Policies
May 5, 2026Senne Deproost, Mehrdad Asadi, Ann NowéReinforcement Learning Interpretability in RL
aVrije Universiteit Brussel · bFlanders Make
Vanishing L2 regularization for the softmax Multi Armed Bandit
May 5, 2026Stefana-Lucia Anita, Gabriel TuriniciMulti-Armed Bandits Policy Gradient
CEREMADE, Universit´e Paris Dauphine - PSL, CNRS, Paris, France · ‘Octav Mayer” Institute of Mathematics of the Romanian Academy, Bd. Carol I 8, Ias¸i 700505, Romania
T $^2$ PO: Uncertainty-Guided Exploration Control for Stable Multi-Turn Agentic Reinforcement Learning
May 4, 2026Haixin Wang, Hejie Cui, Chenwei Zhang +7Agentic RL Reinforcement Learning
University of California, Los Angeles · 2Amazon.com Inc.
Towards Efficient and Expressive Offline RL via Flow-Anchored Noise-conditioned Q-Learning
May 3, 2026Sungyoung Lee, Dohyeong Kim, Eshan Balachandar +2Distributional RL Q-Learning
The University of Texas at Austin, Austin, TX, USA · Independent Researcher, Seoul, South Korea.
Breaking the Computational Barrier: Provably Efficient Actor-Critic for Low-Rank MDPs
May 2, 2026Ruiquan Huang, Donghao Li, Yingbin Liang +1Markov Decision Processes Reinforcement Learning
University of Kentucky. · University of Virginia. · Ohio State University..
Data Deletion Can Help in Adaptive RL
Apr 30, 2026Param Budhraja, Aditya Gangrade, Alex Olshevsky +1Non-Stationary RL Reinforcement Learning
Department of Electrical and Computer Engineering, Boston University, Boston, MA 02215, USA · Faculty of Electrical and Computer Engineering, Systems Engineering, Boston University, Boston, MA 02215, USA
Can Tabular Foundation Models Guide Exploration in Robot Policy Learning?
Apr 30, 2026Buqing Ou, Frederike DümbgenTabular Foundation Models Robot Policy Learning
Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA · Inria, Département d’informatique de l’ENS, CNRS, PSL Research University, Paris, France
Bayesian policy gradient and actor-critic algorithms
Apr 30, 2026Mohammad Ghavamzadeh, Yaakov Engel, Michal ValkoBayesian RL Temporal-Difference Learning
Adobe Research & Inria · Rafael Advanced Defence System, Israel · Inria Lille — SequeL team, France
When Errors Can Be Beneficial: A Categorization of Imperfect Rewards for Policy Gradient
Apr 28, 2026Shuning Shang, Hubert Strauss, Stanley Wei +2Reward Modeling Policy Gradient
Princeton Language and Intelligence, Princeton University
Sample-efficient Neuro-symbolic Proximal Policy Optimization
Apr 28, 2026Simone Murari, Celeste Veronese, Daniele MeliReinforcement Learning Proximal Policy Optimization
Dept. of Computer Science, University of Verona, Italy
Dyna-Style Safety Augmented Reinforcement Learning: Staying Safe in the Face of Uncertainty
Apr 28, 2026Artur Eisele, Bernd Frauenknecht, Friedrich Solowjow +1Robot Safety Safety Filtering
Institute for Data Science in Mechanical Engineering, RWTH Aachen University, Aachen, Germany
Spark Policy Toolkit: Semantic Contracts and Scalable Execution for Policy Learning in Spark
Apr 27, 2026Zeyu BaiEfficient Inference Policy Learning
Hierarchical Behaviour Spaces
Apr 27, 2026Michael Tryfan Matthews, Anssi Kanervisto, Jakob Foerster +3Reinforcement Learning Hierarchical RL
Meta Superintelligence Labs · University of Oxford
Breaking Lock-In: Preserving Steerability under Low-Data VLA Post-Training
Apr 25, 2026Suning Huang, Jiaqi Shao, Ke Wang +5Vision-Language Grounding Vision-Language-Action Models
Stanford University
K-Score: Kalman Filter as a Principled Alternative to Reward Normalization in Reinforcement Learning
Apr 24, 2026Zixuan Xia, Quanxi LiReinforcement Learning Policy Gradient Methods
University of Bern, Bern, Switzerland.
GCImOpt: Learning efficient goal-conditioned policies by imitating optimal trajectories
Apr 24, 2026Jon Goikoetxea, Jesús F. PalaciánRobotic Control Imitation Learning
Department of Statistics, Mathematics and Computer Science, Public University of Navarre · Ensimag, Institut National Polytechnique de Grenoble, Universit´e Grenoble Alpes · Institute for Advanced Materials and Mathematics (InaMat2)
Preserve Support, Not Correspondence: Dynamic Routing for Offline Reinforcement Learning
Apr 24, 2026Zhancun Mu, Guangyu Zhao, Yiwu Zhong +1Reinforcement Learning Offline RL
School of Intelligence Science and Technology, Peking University
Insect-inspired modular architectures as inductive biases for reinforcement learning
Apr 23, 2026Anne E. StaplesRL Control Reinforcement Learning
Department of Mechanical Engineering, Virginia Tech, Blacksburg, Virginia 24061, USA

May 7, 2026, cs.LGJ/K move · Enter open · S save

Asymmetric On-Policy Distillation: Bridging Exploitation and Imitation at the Token Level

Nan Jia, Haojin Yang, Xing Ma, Jiesong Lian+5

Huazhong University of Science and Technology · Peking University · Meituan

RL for Language Model Reasoning Language Model Distillation Policy Gradient Methods On-Policy Distillation Teacher-Student Learning Policy Learning

Open paper AskPDF arXiv

On-policy distillation (OPD) trains a student on its own trajectories with token-level teacher feedback and often outperforms off-policy distillation and standard reinforcement learning. However, we find that its standard advantage weighted policy gradient suffers from three structural weaknesses, including high variance updates, vanishing gradients in zero-advantage regions, and exploration bottlenecks when corrective signals are insufficient. We therefore propose Asymmetric On-Policy Distillation (AOPD), which replaces ineffective negative reinforcement with localized divergence minimization in non-positive advantage regions while preserving positive reinforcement learning. Experiments on mathematical reasoning benchmarks show that AOPD consistently outperforms standard OPD, with average gains of 4.09 / 8.34 under strong / weak initialization, respectively. AOPD also maintains higher policy entropy during training and better capability retention during sequential tool-use adaptation.

Momentum

Related topics

Latest papers 416