Policy Learning

Follow this topic Ask about this topic

Momentum

79 papers in the last four weeks, up 316% on the four weeks before. 0.8% of all new papers.

Jul 13Week of Sep 28

Latest papers 416

All topics

Diffusion Policy Improvement with Proposal-Conditioned Refinement Flows
Sep 29, 2026Junhyun Ha, Juho Lee, Byoungwoo ParkPolicy Optimization Offline RL
KAIST
Regularized policy gradient with learned mixtures of Gaussians for games with continuous actions
Sep 29, 2026Ondřej Kubíček, Viliam Lisý, Tuomas SandholmGame-Playing Agents Self-Play RL
Czech Technical University in Prague · Carnegie Mellon University · Artificial Intelligence Center
T $^2$ Mem: Learning Test-Time Memory for Robotics
Sep 29, 2026Yize Liu, Huang Huang, Yining Hong +4Robot Policy Learning Test-Time Adaptation of VLA Models
Stanford University · NVIDIA · University of Michigan, Ann Arbor
Learning to Explore Hidden Kinematics for Articulated Object Manipulation
Sep 29, 2026Ruiyao Liu, Boshu Lei, Zhuoyang Pan +1RL for Robotics Active Perception
GRASP Lab, University of Pennsylvania, Philadelphia, PA 19104, USA
Reward-rate Policy Gradient for Efficient Machine Learning Engineering Agents
Sep 28, 2026Muhang Tian, Sherry YangPolicy Gradient Agentic RL
New York University
Provable Benefits of Regularization: Fast Rates for Adversarial Imitation Learning
Sep 28, 2026Hanbin Zhou, Shangzhe Li, Alexander Braverman +1Reinforcement Learning Adversarial Imitation Learning
HKUST(GZ) · UNC Chapel Hill · Cornell University
Agent Priors-guided Policy Learning
Sep 28, 2026Puming Jiang, Tianrun Hu, Haozhe Du +4Robot Skill Learning Robot Imitation Learning
National University of Singapore
Behavioral Foundation Models for Quality Diversity
Sep 28, 2026Nazim Bendib, Nicolas Perrin-Gilbert, Olivier SigaudReinforcement Learning Quality-Diversity Optimization
Sorbonne Université, ISIR Paris, France
Beyond Teacher Assignment: Domain-Normalized Multi-Teacher On-Policy Distillation
Sep 28, 2026Xin Li, Hao Jiang, Xin Gao +6Language Model Distillation Multi-Teacher Knowledge Distillation
Nanyang Technological University · Yale University · University of Manchester
PIVOT: Pivot-Aware On Policy Self Distillation for Multi-Turn VLM Agents
Sep 28, 2026Jiazhou Zhou, Hu Zhou, Yucheng Chen +3On-Policy Self-Distillation Reinforcement Learning with Verifiable Rewards
AI Thrust, The Hong Kong University of Science and Technology (Guangzhou) · International Digital Economy Academy (IDEA) · The Hong Kong Polytechnic University +2
On-Policy or Off-Policy Learning? A Systematic Study of Distillation Dynamics
Sep 28, 2026Julianna Piskorz, Antonin Berthon, Mihaela van der SchaarLanguage Model Distillation On-Policy Distillation
University of Cambridge Cambridge, UK
ASCT: Attentive Search over Counterfactual Trees for Credit Assignment in Agentic Reinforcement Learning
Sep 28, 2026Yang Li, Jinhan Yang, hai liu +8Agentic RL Credit Assignment in RL
University of Hong Kong · The Chinese University of Hong Kong · Jiangxi Science and Technology Normal University +1
GraphHCA: Closed-Form Hindsight Credit Assignment for Long-Horizon LLM Agents
Sep 28, 2026Haodong Zhu, Yangyang Ren, Changbai Li +4Credit Assignment in RL Step-Level Credit Assignment in RL
Beihang University · Zhongguancun Academy · Communication University of China +1
Cross-Rollout Bellman Closure for Long-Horizon Agentic Reinforcement Learning
Sep 28, 2026Yangyang Ren, Haodong Zhu, Linlin Yang +3Agentic RL Step-Level Credit Assignment in RL
Beihang University · Zhongguancun Academy · Communication University of China +2
When Sparse Reward Meets Dense Distillation: Training Dynamics of On-Policy Distillation
Sep 28, 2026Xinke Jiang, Tao Feng, Zhibang Yang +4Gradient Interference Reinforcement Learning with Verifiable Rewards
National Engineering Research Center of Software Engineering, Peking University, Beijing, China · School of Computer Science, Peking University, Beijing, China · Key Laboratory of High Confidence Software Technologies, Ministry of Education, Beijing, China +1
UniOPSD: Unifying Outcome and Hindsight Feedback for Agentic Reinforcement Learning
Sep 28, 2026Zenghuang Fu, Zhaoyang Li, Qiuyuan Ai +8Agentic RL Credit Assignment in RL
University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences · Peking University +3
SIPO: Selective-Inference Policy Optimization for Tree-Structured Agentic RL
Sep 28, 2026Zenghuang Fu, Ningqi Chen, Mingda Jia +10Agentic RL Reinforcement Learning
University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences · The University of Hong Kong +4
SOLAR: A State-Driven Online Learning Rate Scheduler for LLM Pretraining
Sep 28, 2026Qiulin Shang, Binyu Wang, Yongqi Qiao +3Language Model Pretraining Learning Rate Scheduling
Peking University · Nanjing University
QAMM: Adjoint MeanFlow Matching for Few-Step Offline Reinforcement Learning
Sep 28, 2026Yuehu Gong, Shutong Ding, Mokai Pan +4Flow Matching MeanFlow
School of Data Science, Fudan University · ShanghaiTech University · Shanghai Innovation Institute
FailPatch: Failure Residual Patching for Vision-Language-Action Models
Sep 28, 2026Peng Yu, Jiacheng Wang, Ziheng Zhang +6Robot Policy Learning Robot Policy Adaptation
Xi’an Jiaotong University · Dexmal · Nanjing University
Learning Perturbation Robust Policies for LLM Agents with Stable Optimization
Sep 28, 2026Pengxin Wang, Yuanzhe LI, Yuxin Ren +2RL Post-Training Robust RL
Department of Electrical and Computer Engineering University of Arizona
Estimate, Don't Imitate: Reusing Differentiable State-Based Policies for Visuomotor Control
Sep 27, 2026Denis Shcherba, Adrian Abel, Eckart Cobo-Briesewitz +3Visuomotor Policy Learning Robotic Manipulation
Fraunhofer Heinrich-Hertz-Institut, Berlin, Germany · Technische Universität Berlin, Germany
ICMAPE: In-Context Multiagent Pure Exploration
Sep 27, 2026Xinyi Hu, Alessio Russo, Aldo PacchianoRL Exploration Decentralized MARL
Boston University · Broad Institute of MIT and Harvard
Future Information-Directed Sampling for Bayesian Nonstationary Bandits
Sep 27, 2026Yichen Song, Alessio Russo, Aldo PacchianoMulti-Armed Bandits Non-Stationary Bandits
Boston University · Broad Institute of MIT and Harvard
Demonstration-Free Success-Probability Reward Learning for Generalist Robot Policies
Sep 27, 2026Duo Wu, Haifeng Wang, Rongwei Lu +6Robotic RL Sparse-Reward RL
Tsinghua University · Yuanxing Robotics · Pengcheng Laboratory
HiLoRe: What to Store, Compress, or Recompute for Efficient GRPO Training
Sep 27, 2026Xinrui Chen, Mengyang Li, Ou Wu +1Group Relative Policy Optimization Policy Learning
Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou, China · Tianjin Key Laboratory of Wireless Mobile Communications and Power Transmission, Tianjin Normal University, Tianjin, China · University of Southern Queensland
NLPG: Natural-Language Policy Gradients for Self-Evolving Language Agents
Sep 27, 2026Xu Liu, WenZhang Wei, Jun Cao +4Continual Learning for LLM Agents Policy Optimization
School of Electronic Information, Wuhan University, Wuhan 430079, China · School of Remote Sensing and Information Engineering, Wuhan University, Wuhan 430079, China · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China
Action Shaping: Policies Absorb What They Can Express
Sep 26, 2026Yanjun Chen, Jinghan Wang, Xiaoyu Shen +2Policy Optimization Policy Learning
Eastern Institute of Technology · The Hong Kong Polytechnic University · Harbin Institute of Technology
DRAM: Delta-rule Recurrent Associative Memory for Robot Manipulation Policies
Sep 26, 2026Xinyu Zhao, Yixiang Shan, Tao Yang +5Long-Horizon Robotic Manipulation Delta-Rule Linear Attention
Simplexity Robotics
Learning from Mixed-Quality Deployment Experience for Robot Manipulation
Sep 24, 2026Yangang Ren, Yujie Yan, Zirui Li +6Robot Policy Learning Offline RL
School of Mechanical and Aerospace Engineering, Nanyang Technological University, Singapore 639798 · Chongqing Changan Automobile Co., Ltd, Chongqing 400023, China · Guangzhou Cloudbutterfly Technology Co., Ltd., Guangzhou 510220, China

Sep 29, 2026, cs.LGJ/K move · Enter open · S save

Diffusion Policy Improvement with Proposal-Conditioned Refinement Flows

Junhyun Ha, Juho Lee, Byoungwoo Park

KAIST

Policy Optimization Offline RL Policy Learning Offline-To-Online RL

Open paper AskPDF arXiv

Diffusion and flow policies can model complex behaviors in offline reinforcement learning (RL). However, penalizing their KL divergence from the behavior policy can discourage actions having high critic values with low behavior density. Directly refining behavior proposals may be an alternative, yet Gaussian or deterministic editors limit expressiveness to represent multiple separated modes for the same proposal. In this work, we introduce Proposal-Conditioned Refinement Flows (PReFlow), a policy extraction method combining critic-based proposal selection with a conditional refinement flow. To optimize proposal selection and refinement together, we formulate a KL-regularized objective whose optimum induces a Gibbs policy over final actions under a Gaussian-smoothed behavior prior. The refinement flow can represent multiple high value modes, while a proposal-centered Gaussian reference regulates large action changes. This Gaussian reference further enables us to make use of simulation-free, closed form adjoint matching targets from sampled endpoints and critic gradients, yielding a single velocity regression loss without a backward adjoint solve. On 50 OGBench tasks, PReFlow achieves competitive offline performance and the highest aggregate score among the compared methods after online fine-tuning, reaching 91% after 500K environment steps.

Momentum

Related topics

Latest papers 416