Policy Learning

Follow this topic Ask about this topic

Momentum

79 papers in the last four weeks, up 316% on the four weeks before. 0.8% of all new papers.

Jul 13Week of Sep 28

Latest papers 416

All topics

HindSearch: Trajectory-Level Hindsight Critique for Search-Augmented Reinforcement Learning
Aug 3, 2026Haowei Liu, Jiamian Wang, Hsin-Tai Wu +2RL for Language Models Agentic Search
Santa Clara University Santa Clara, CA · Rochester Institute of Technology Rochester, NY · Independent Researcher Sunnyvale, CA
Reusing Rollouts under Policy Lag: Prefix-Normalized Policy Optimization for LLM Reinforcement Learning
Aug 2, 2026Wenhao Zhang, Yibo Xie, Rui Wang +9Reinforcement Learning RL for Language Model Reasoning
1Tencent · 2Harbin Institute of Technology · 3Jinan University +1
ReBRAC-v2: The Return of the King
Aug 2, 2026Denis Tarasov, Robert K. KatzschmannReinforcement Learning Offline RL
Soft Robotics Lab, D-MAVT ETH Zurich, Switzerland
Learning Not to Optimize: Physics-Informed Action-Space Reshaping for Intent-Based Network Control
Aug 2, 2026Zuyuan Zhang, Vaneet Aggarwal, Tian LanReinforcement Learning Policy Optimization
The George Washington University · Purdue University
Personalizing Large Language Model Agents with Small Policy Models
Jul 31, 2026Dian Jin, Zhi Zhang, Huichao Li +3LLM Personalization LLM Agents
Department of Statistics and Data Science, National University of Singapore · Department of Statistics and Data Science, University of California, Los Angeles · Department of Data Science, Fudan University
When Does On-Policy Interaction Help? Representational Tradeoffs in Value-Based Imitation Learning
Jul 31, 2026Luca Viano, Antoine Moulin, Audrey Huang +3Imitation Learning Interactive Imitation Learning
Microsoft
Convergence and Regret of the Policy Gradient for Multi-Armed Bandits in Diffusion Environment
Jul 31, 2026Yanwei Jia, Du OuyangMulti-Armed Bandits Regret Minimization in RL
Department of Systems Engineering and Engineering Management, The Chinese University of Hong Kong, Shatin, New Territories, Hong Kong SAR · Department of Mathematical Sciences, Tsinghua University, Beijing, Beijing 100084, China
Temporal Policy: History-Initialized Action Generation for Robotic Learning from Demonstration
Jul 31, 2026Dylan Miller, Martin JagersandFlow Matching Robot Policy Learning
Department of Computing Science, University of Alberta, Canada
Sample Efficient Hierarchical Reinforcement Learning via Best Policy Identification
Jul 31, 2026Anders Jonsson, Emilie Kaufmann, Gianmarco Tedeschi +1Hierarchical RL Policy Learning
Department of Engineering Universitat Pompeu Fabra · Univ. Lille, CNRS, Inria Centrale Lille, UMR 9189-CRIStAL · Dept. Electronics, Information, and Bioengineering Politecnico di Milano
Don't Mix Rewards, Mix Policies: Policy Decomposition and Optimization for Multi-Reward RL
Jul 31, 2026Ruiming Liang, Yi Zhong, Yizhen Yuan +6Multi-Objective Reinforcement Learning RL for Language Models
1Fundation Model Research Center, CASIA · School of Artificial Intelligence, UCAS · Institute for AI Industry Research (AIR), Tsinghua University +1
$β$ -OPSD: Deriving with Policy Optimization, Training with Self-Distillation
Jul 30, 2026Jiawei Xu, Minghui Liu, Juzheng Zhang +2KL-Regularized RL On-Policy Self-Distillation
University of Maryland, College Park
On-Policy and Off-Policy Learning for Large Action Spaces
Jul 30, 2026Imad AoualiThompson Sampling Multi-Armed Bandits
Institut Polytechnique de Paris · Ecole nationale de la statistique et de l’administration économique · Ecole doctorale n◦574 Ecole Doctorale de Mathématique Hadamard (EDMH)
Hierarchical Multilevel Monte Carlo for Order-Optimal Neural Actor-Critic in Average-Reward CMDPs
Jul 30, 2026Ankur Naskar, Vaneet AggarwalAverage-Reward RL Constrained RL
Indian Institute of Science, India · Purdue University, USA
LoRA Scaffolded Policy Optimization (LSPO): A Sampling-Time Low-Rank Scaffold for Recovering Reinforcement-Learning Gradient on Zero-Reward Cliff Prompts
Jul 30, 2026Ken DingReinforcement Learning RL for Language Model Reasoning
NVIDIA
Do You Really Need to Pretrain Q-Functions for Online RL Fine-Tuning?
Jul 29, 2026Perry Dong, Ron Polonsky, Dorsa Sadigh +1Continuous Control RL Fine-Tuning
Stanford University
Pass the Baton: Trajectory-Relayed On-Policy Distillation
Jul 28, 2026Haolei Xu, Xiaowen Xu, Haiwen Hong +5Language Model Distillation On-Policy Distillation
1Zhejiang University · 2Yuvion Team, Alibaba Group
$π\mathbf{R}^2$ : Reactive Real-time Flow Policies
Jul 28, 2026Sungjae Park, Shubham TulsianiRobotic Control Flow Matching
Carnegie Mellon University
PAC-DP: PAC-Bayesian Diffusion Policy Learning
Jul 27, 2026Mohammad Hasan Yeganegi, Dian Yu, Andrea Del Prete +2PAC-Bayesian Generalization Bounds Robot Policy Learning
Department of Industrial Engineering, University of Trento, Italy. · Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University of Munich (TUM), Germany.
On the Impossibility of Unbiased and Length-Invariant Policy Optimization with Outcome Rewards
Jul 25, 2026Fei Ding, Yongkang Zhang, Yuhao Liao +2Policy Gradient RL for Language Model Reasoning
Alibaba Group · Tsinghua University
Finite-Time Analysis of the Natural Policy Gradient in Finite-Horizon Markov Decision Processes
Jul 25, 2026Asha Barua, Sajad KhodadadianPolicy Gradient Markov Decision Processes
Grado Department of Industrial and Systems Engineering, Virginia Polytechnic Institute and State University, Blacksburg, Virginia 24061, USA.
Scale Up Strategically: Learning Compositional Generalization via Bias-Aware Evaluation and Data Collection for Robotic Manipulation
Jul 23, 2026Yu Qi, Zhang Ye, Xinyi Xu +6Robot Skill Learning Language-Conditioned Robot Manipulation
Northeastern University · NVIDIA
PATS: Policy-Aware Training Scaffolding for Agentic Reinforcement Learning
Jul 23, 2026Yipeng Shi, Zhipeng Ma, Yue Wang +4Agentic RL LLM Agent Skill Learning
School of Software & Microelectronics, Peking University · 2Tencent
The Dark Room in the Reward Channel: Dense Prediction Rewards Collapse GRPO-Trained LLM Agents -- and The Channel, Not the Content, Decides What Works
Jul 23, 2026Yu WangReward Hacking Reward Shaping
Independent researcher
Relative Value Learning
Jul 23, 2026Marc Höftmann, Jan Robine, Stefan HarmelingValue Function Estimation Temporal-Difference Learning
Department of Computer Science, Technical University of Dortmund, Germany · Lamarr Institute for Machine Learning and Artificial Intelligence
AttriMem: Attribution-Guided Process Feedback for Agent Memory Construction
Jul 23, 2026Qinfeng Li, Yuntai Bao, Xinyan Yu +7LLM Agent Memory Policy Learning
1Zhejiang University · 2Washington University in St. Louis · University of Science and Technology of China
Molt: A Scalable PyTorch-Native Training Framework for Agentic Reinforcement Learning
Jul 22, 2026Jian Hu, Huiying Li, Hao Zhang +8Agentic RL Reinforcement Learning
NVIDIA
Progress-conditioned Group Policy Optimization for Long-Horizon Agentic Tasks
Jul 22, 2026Kaibing Yang, Guangfeng Cai, Shengtian Yang +6Group Relative Policy Optimization RL Exploration
Southeast University · Kuaishou Technology · Work done during an internship at Kuaishou Technology, supervised by Jun Xu. +1
From Trajectories to Instructions: Language-Conditioned Meta-Reinforcement Learning
Jul 21, 2026Garvit Singla, Uma Maheswari Natarajan, Raghuram Bharadwaj DiddigiReinforcement Learning Meta-Learning
International Institute of Information Technology Bangalore, India
OR Else: A Differentiable Trust Region for Policy Optimization
Jul 20, 2026Chinmay Rane, Kanishka Tyagi, Michael ManryRL for Language Models Group Relative Policy Optimization
Quantiphi Inc Marlborough, MA, 01752 · Self Machines Inc San Francisco, California, 94402 · Department of Electrical Engineering The University of Texas at Arlington Arlington, TX, 76010
Theoretical Foundations of $\max$ @ $k$ Reinforcement Learning
Jul 20, 2026Riccardo Poiani, Martino Bernasconi, Andrea CelliReinforcement Learning Policy Learning
Bocconi University

Aug 3, 2026, cs.LGJ/K move · Enter open · S save

HindSearch: Trajectory-Level Hindsight Critique for Search-Augmented Reinforcement Learning

Haowei Liu, Jiamian Wang, Hsin-Tai Wu, Zhiqiang Tao+1

Santa Clara University Santa Clara, CA · Rochester Institute of Technology Rochester, NY · Independent Researcher Sunnyvale, CA

RL for Language Models Agentic Search Self-Distillation Policy Learning RL for Tool Use

Open paper AskPDF arXiv

Search-augmented LM agents are typically trained with a binary exact-match reward, which throws away most of what a failed trajectory tells us about why it failed. We introduce HindSearch, a hindsight self-distillation procedure for GRPO: after each rollout, a frozen judge writes a short critique of every failed trajectory using the gold answer, and the critique supplies an auxiliary on-policy distillation signal on the student's search actions. On the standard seven-benchmark suite with Qwen2.5-3B-Instruct, HindSearch reaches 39.4% average EM, outperforming prior search-RL baselines. Removing the judge's access to the gold answer erases most of the gain, isolating hindsight as the source of the improvement.

Momentum

Related topics

Latest papers 416