Policy Learning

Follow this topic Ask about this topic

Momentum

79 papers in the last four weeks, up 316% on the four weeks before. 0.8% of all new papers.

Jul 13Week of Sep 28

Latest papers 416

All topics

Higher-Order Action Supervision Makes A Strong Policy Class
Oct 8, 2026Peng Cheng, Yunxian Hou, Zhi Zhou +3Policy Learning Robust RL
Institute for AI Industry Research (AIR), Tsinghua University · Zenfex AI Lab · University of Electronic Science and Technology of China +1
Learning How to Search for Plans with Exponentially Less Space
Oct 7, 2026Dominik Drexler, Simon Ståhlberg, Markus Fritzsche +1Symbolic Planning Policy Learning
Linköping University Linköping, Sweden · RWTH Aachen University Aachen, Germany · Universidad Simón Bolívar Caracas, Venezuela
Policy Learning with Weak Signals
Oct 7, 2026Benedikt Koch, Winston Chou, Aurélien Bibaut +1Policy Learning Causal Effect Estimation
Harvard University · Netflix
Learning Situation-Conditioned Thinking Policies for Long-Term LLM Agents
Oct 7, 2026Hong SuLLM Agent Memory Long-Horizon LLM Agents
School of Computer Science, Chengdu University of Information Technology, Chengdu, China
Towards the Automatic Synthesis of Interpretable Chess Tactics
Oct 6, 2026Abhijeet Krishnan, Chris MartensGame-Playing Agents Chess
North Carolina State University Venture IV, 1730 Varsity Dr, Raleigh, NC 27606
Learning Explainable Representations of Complex Game-playing Strategies
Oct 6, 2026Abhijeet Krishnan, Colin M. Potts, Arnav Jhala +3Game-Playing Agents Program Synthesis
Venture IV 420, North Carolina State University, Raleigh, NC 27606 USA · Department of Aerospace Engineering, Indian Institute of Technology Bombay, Powai, Mumbai 400076, Maharashtra, India · Tata Research Development and Design Centre, Pune 411013, Maharashtra, India +1
FERPO: Forward Entropy-Regularized Policy Optimization
Oct 1, 2026Sebastian Sanokowski, Alireza Sarmadi, Majid KhadivKL-Regularized RL Policy Optimization
Applied and Theoretical Aspects of Robot Intelligence (ATARI) Lab, Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University of Munich
Q-Learning for Reachability in MEC-Free MDPs
Oct 1, 2026Lu-Chin Chang, Suguman BansalMarkov Decision Processes Reinforcement Learning
Georgia Institute of Technology
Towards Optimal Policy Improvement
Oct 1, 2026Yaniv Oren, Viliam Vadocz, Wiktor Zabka +7Markov Decision Processes Policy Optimization
Department of Intelligent Systems, TU Delft · Centrum Wiskunde & Informatica, Amsterdam · Department of Computer Science, ETH Zürich +3
Reusing Past Samples in Proximal Policy Optimization: When and How Does It Help?
Oct 1, 2026Alessandro Montenegro, Riccardo Venturelli, Marco Mussi +2Experience Replay Proximal Policy Optimization
Politecnico di Milano, Milan, Italy · Università degli Studi di Milano, Milan, Italy
Learning Multiple Timescales for Goal-Conditioned Reinforcement Learning
Oct 1, 2026Pedro Robles Dutenhefner, Dikshant Shehmar, Wagner Meira +1Reinforcement Learning Offline RL
Universidade Federal de Minas Gerais · University of Alberta · Alberta Machine Intelligence Institute (Amii) +1
Reward as Observation: Learning Reward-Based Policies for Rapid Adaptation
Sep 30, 2026Morgan Byrd, Jacob Blevins, Maks Sorokin +2Robot Policy Learning Robot Policy Adaptation
Georgia Institute of Technology, Atlanta, GA, 30308, USA · Georgia Tech Research Institute, Atlanta, GA, 30308, USA
Make Sparse Rewards Count: Density-Aware Reward Aggregation for Multi-Reward RL
Sep 30, 2026Tong Zheng, Skylar Zhai, Zhan Cheng +7Multi-Objective Reinforcement Learning Reinforcement Learning
University of Chinese Academy of Sciences · University of Minnesota Twin Cities · University of Wisconsin–Madison +5
PivotOPD: Learning to Recover from Pivotal Mistakes in Multi-Turn Agents
Sep 30, 2026Yinghui He, Yapei Chang, Khushi Bhardwaj +4Language Model Distillation LLM Agent Training
Princeton University · NVIDIA · University of Maryland
Role-Adaptive Policy Optimization for Offline Reinforcement Learning
Sep 30, 2026Seonvin Cho, Soohyun Choi, Songnam HongPolicy Optimization Offline RL
Department of Electronic Engineering, Hanyang University Seoul, Republic of Korea
OPTS-TTPO: Enhancing Finite-Sample Policy-Gradient Learning with Tree Search
Sep 30, 2026Junyu Lu, Shichao Weng, Zhiqiang Wang +10Tree Search Policy Gradient Methods
Dobot Robotics · Independent Researcher · Zhejiang University +2
Fast Regularized Policy Mirror Descent with One-Step TD Updates
Sep 30, 2026Qipei Chen, Wenye Li, Yule Sun +1Markov Decision Processes Policy Optimization
School of Mathematical Sciences, Fudan University · School of Data Science, Fudan University
Beyond Policy Alignment: Closing the Planning-Learning Loop for Robot Control with Learned World Models
Sep 30, 2026Kowndinya Boyalakuntla, Yuhan Liu, Abdeslam BoulariasRobotic Control World Models for Robotics
Rutgers University
Free Everywhere, Exact on Trees: PPO's Dropped Correction Buys Sample Efficiency Under Aggressive Reuse
Sep 30, 2026Nima H. SiboniImportance Sampling Proximal Policy Optimization
Juna.ai, Kastanienallee 32, 10435 Berlin, Germany
T2SPO: Trajectory-to-Step Policy Optimization for Agentic Reinforcement Learning
Sep 30, 2026Bo-Wen Zhang, Junwei He, Maoqi Liu +6Agentic RL Reward Shaping
State Key Laboratory of Novel Software Technology, Nanjing University · School of Intelligence Science and Technology, Nanjing University · ByteDance
From Imitation to Reward Discovery: On-Policy Warmup for Agentic RL
Sep 30, 2026Yitong Qiao, Tiantian He, Lei Liu +4Agentic RL Reinforcement Learning with Verifiable Rewards
Zhejiang University · Ant Healthcare, Ant Group
Linear Recurrent Memory Suffices to Distil a World-Model Policy for Robot Air Hockey
Sep 30, 2026F. Olivia Fan, Oliver ObstRecurrent Neural Networks Robotic RL
UNSW Sydney, NSW 2052, Australia
DAMPER: Return-Prioritized Gradient Control for Smooth Policies
Sep 30, 2026Seokmin Ko, Taewon Goo, Kihyuk HongRL Control Reinforcement Learning
KAIST
Understanding Off- vs On-Policy Distillation: A Tale of Distinct Training Objectives
Sep 29, 2026Qiwei Di, Xuheng Li, Kaixuan Ji +3Multi-Teacher Knowledge Distillation On-Policy Distillation
Department of Computer Science, University of California, Los Angeles, CA 90095, USA
Learning What to Remember: Long-horizon Counterfactual Memory Optimization
Sep 29, 2026Jiaming Tang, Mingyan Liu, Armin SarabiMemory-Augmented Language Models Persistent Memory for Language Models
Department of Electrical Engineering and Computer Science, University of Michigan
Guide, Then Let Go: Gap-Adaptive Teacher Scheduling for Sparse-Reward Agentic RL
Sep 29, 2026Youling Huang, Tiankuo Xu, Jiaji Liu +10Agentic RL LLM-Guided RL
DUT · Kuaishou · XJTU +5
Video2STL: Grounding VLM-Generated Temporal Specifications for Robot Learning
Sep 29, 2026Merve Atasever, Keyan Azbijari, Cagan Bakirci +6Robot Policy Learning Signal Temporal Logic
Department of Computer Science, University of Southern California Los Angeles, CA, USA · University of Florida Gainesville, FL, USA
Unlocking the Critic: Reward-Free Policy Optimization for LLM Post-Training
Sep 29, 2026Hongyang Li, Xiao Li, Caesar Wu +3Reinforcement Learning Value Function Estimation
University of Luxembourg · Seafill Open-Source Community · Université Paris-Saclay
Learning from Viable Failure Prefixes: Milestone Viability Potential Policy Optimization for Long-Horizon LLM Agents
Sep 29, 2026Qi Zhou, Yuanfan LiCredit Assignment in RL Long-Horizon LLM Agents
Xi’an Jiaotong University
State Trace Rationale As Auxiliary Task in Reinforcement Learning
Sep 29, 2026Muhammad U. Nasir, Alex Vogt, Steven D. James +1Reinforcement Learning Representation Learning for RL
University of the Witwatersrand Johannesburg, South Africa · New York University New York, USA

Oct 8, 2026, cs.ROJ/K move · Enter open · S save

Higher-Order Action Supervision Makes A Strong Policy Class

Peng Cheng, Yunxian Hou, Zhi Zhou, Qian Zhang+2

Institute for AI Industry Research (AIR), Tsinghua University · Zenfex AI Lab · University of Electronic Science and Technology of China+1

Policy Learning Robust RL Offline RL

Open paper AskPDF arXiv

Modern data-driven decision-making methods, such as imitation learning (IL) and reinforcement learning (RL), have achieved great success in solving many complex tasks. However, these methods often suffer from serious control instability and robustness issues when applied in real-world applications such as robotics and autonomous driving, posing notable challenges for their practical deployment. We argue that this instability issue stems largely from their limitations in solely supervising and optimizing zeroth-order actions (i.e., the action labels), failing to account for higher-order action dynamics and temporal consistency. In this paper, we show that simultaneously supervising both zeroth- and first-order actions can dramatically enhance policies' performance and control robustness. To achieve this, we introduce a novel and elegant loss scheme supported by formal theoretical guarantees that can equip any off-the-shelf policy model (e.g., deterministic, stochastic, or flow policies) with the capability for higher-order action supervision, without requiring any structural modifications. Moreover, our proposed method can serve as a lightweight plug-and-play module that seamlessly integrates with a broad spectrum of existing offline RL frameworks. Extensive evaluations on OGBench and D4RL demonstrate that our approach yields substantial performance and robustness improvements across a wide range of continuous control environments. Notably, our method can also enhance policies' out-of-distribution (OOD) generalization capability in the challenging low-data regime, making it an ideal tool in tackling many real-world control problems.

Momentum

Related topics

Latest papers 416