Policy Learning

Follow this topic Ask about this topic

Momentum

79 papers in the last four weeks, up 316% on the four weeks before. 0.8% of all new papers.

Jul 13Week of Sep 28

Latest papers 416

All topics

Exploration and Online Transfer with Behavioral Foundation Models
Jun 29, 2026Louis Bagot, Mathieu Lefort, Laëtitia MatignonReinforcement Learning RL Exploration
SyCoSMA · Université Lyon 1, CNRS, INSA Lyon LIRIS - UMR5205, 69622 Villeurbanne, France · LIRIS, SyCoSMA, IRISA, MALT, UR +1
STEAM: Self-Supervised Temporal Ensemble Advantage Modeling for Real-World Robot Learning
Jun 29, 2026Zhihao Liu, Qiuyi Gu, Yitao Wang +16Robot Policy Learning Robot Skill Learning
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Zhongguancun Academy +8
Behavior Uncloning: Distilling Mode Redirection into Policy Weights without Inference-Time Steering
Jun 28, 2026Hao Wang, Jiuzhou Lei, Dayou Li +5Robot Policy Learning Robot Policy Adaptation
Texas A&M University · University of Wisconsin · Northwestern University +1
Regularized Reward-Punishment Reinforcement Learning
Jun 26, 2026Jiexin Wang, Eiji UchibeReinforcement Learning KL-Regularized RL
Dept. of Brain Robot Interface, ATR Computational Neuroscience Laboratories, 2-2-2, Hikaridai, Seikacho, Soraku-gun, Kyoto, 619-0288, Japan
Continual Robot Policy Learning via Variational Neural Dynamics
Jun 25, 2026Jiaxu Xing, Zhiyuan Zhu, Yunfan Ren +4Robot Policy Learning Robot Policy Adaptation
Robotics and Perception Group, University of Zurich
Automating Potential-based Reward Shaping with Vision Language Model Guidance
Jun 25, 2026Henrik Müller, Daniel KudenkoReward Shaping Sparse-Reward RL
L3S Research Center Leibniz University Hannover Hannover, 30167, Germany
Joint Learning of Experiential Rules and Policies for Large Language Model Agents
Jun 25, 2026Shicheng Ye, Chao YuContinual Learning for LLM Agents LLM Agent Training
Sun Yat-sen University
OPID: On-Policy Skill Distillation for Agentic Reinforcement Learning
Jun 25, 2026Shuo Yang, Jinyang Wu, Zhengxi Lu +8Agentic RL LLM Agent Skill Learning
1Tsinghua University · 2Zhejiang University · 3The Chinese University of Hong Kong +2
Finding the Time to Think: Learning Planning Budgets in Real-Time RL
Jun 24, 2026Aneesh Muppidi, Firas Darwish, Dylan Cope +2Reinforcement Learning Policy Learning
British Open-ended Learning and Discovery Lab (BOLD), University of Oxford · Department of Statistics, University of Oxford · VGG, University of Oxford
EVOM: Agentic Meta-Evolution of Actor-Critic Architectures for Reinforcement Learning
Jun 24, 2026Boyun Zhang, Chao Wang, Kai WuLarge Language Model-Guided NAS Neural Architecture Search
Xidian University
Memory-Efficient Policy Libraries with Low-Rank Adaptation in Reinforcement Learning
Jun 24, 2026Samuel Valland Lyngset, Tor Viljen Raanaas, Gard Sveipe +4RL for Robotics Low-Rank Adaptation
Department of Informatics, University of Oslo
Low-Complexity Policy Tessellations in Structured Markov Decision Processes
Jun 24, 2026Fredy PokouMarkov Decision Processes Reinforcement Learning
CRIStAL · Inria, University of Lille, CNRS, Centrale Lille Villeneuve-d’Ascq, France
MAPL: Multi-Objective Preference Learning for Robot Locomotion
Jun 24, 2026Xiyue Chen, Muhan Lin, Shuyang Shi +1Multi-Objective Reinforcement Learning Reward Modeling
Department of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47906, USA · Department of Computer Science, Purdue University, West Lafayette, IN 47906, USA · independent researcher
Efficient Adaptive Data Acquisition via Pretrained Belief Representations
Jun 23, 2026Daolang Huang, Zhuoyue Huang, Conor Hassan +3Bayesian Optimization Active Learning
ELLIS Institute Finland · Department of Computer Science, Aalto University, Finland · Department of Statistics, University of Oxford, UK +2
Minimax PAC Bounds for Learning in Exogenous Contextual MDPs
Jun 23, 2026Corentin Pla, Hugo Richard, Marc Abeille +1Markov Decision Processes Reinforcement Learning
CREST, ENSAE · Criteo AI Lab · FairPlay Joint Team
Bias-Controlled Primal-Dual Natural Actor-Critic: Optimal Rates for Constrained Multi-Objective Average-Reward RL
Jun 23, 2026Ankur Naskar, Swetha Ganesh, Vaneet AggarwalMulti-Objective Reinforcement Learning Reinforcement Learning
Indian Institute of Science, Bengaluru, India · Purdue University, USA
ExTra: Exploratory Trajectory Optimization for Language Model Reinforcement Learning
Jun 23, 2026Wenyang Hu, Junxiang Jia, Zhen Shu +3RL for Language Models RL for Language Model Reasoning
1National University of Singapore · 2SAP
LaGO: Latent Action Guidance for Online Reinforcement Learning
Jun 23, 2026Kuan-Yen Liu, Ren-Jyun Huang, Ti-Rong WuLatent Action Learning LLM-Guided RL
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign, USA · Department of Computer Science, National Yang Ming Chiao Tung University, Taiwan · Institute of Information Science, Academia Sinica, Taiwan.
Blockwise Policy-Drift Gating for On-Policy Distillation
Jun 23, 2026Liwen Zheng, Haiyun JiangLanguage Model Distillation On-Policy Distillation
Independent Researcher
EMAgnet: Parameter-Space EMA Regularization for Policy Gradient Self-Play in Large Games
Jun 22, 2026Tristan Maidment, JB Lanier, Chase McDonald +5Self-Play RL Imperfect-Information Games
1Riot Games · 3New York University · University of California, Irvine
KLip-PPO: A per-sample KL perspective on PPO-Clip
Jun 22, 2026Riccardo Colletti, Robin HolzingerKL-Regularized RL Policy Gradient Methods
University of California, Berkeley
Learning Process Rewards via Success Visitation Matching for Efficient RL
Jun 22, 2026Raymond Tsao, Andrew Wagenmaker, Sergey LevineProcess Reward Models Credit Assignment in RL
UC Berkeley
On the Position Bias of On-Policy Distillation
Jun 21, 2026Yan Xie, Sijie Zhu, Tiansheng Wen +2Importance Weighting Language Model Distillation
Xidian University · Georgia Institute of Technology · Amazon AGI SF Lab
Drowning in Routine: Signal Dilution in Multi-Turn Agent Training
Jun 20, 2026Yann Pernot, Vi RetaultReinforcement Learning Credit Assignment in RL
Mila - Québec AI Institute · Polytechnique Montréal
Modularized Reinforcement Learning on LLMs: From MDP Creation to Exploration and Learning
Jun 20, 2026Zhao Yang, Yuxuan Jiang, Ting-Chih Chen +18RL for Language Models Credit Assignment in RL
VU Amsterdam, The Netherlands · University of Maryland, Baltimore County, USA · Leiden University, The Netherlands +4
Backpropagating Through Simulation: Analytic Policy Gradients for Sample and Learning Efficient Differentiable Continuous Control
Jun 19, 2026Yueci DengRobotic RL Policy Gradient Methods
School of Data Science, The Chinese University of Hong Kong, Shenzhen
Self-Improvement Can Self-Regress: The Rise-and-Collapse Failure Mode of LLM Self-Training
Jun 17, 2026Jianzhe LinRL for Code Generation Reinforcement Learning
MetaAI · *Work done at Meta
Pareto Q-Learning with Reward Machines
Jun 17, 2026Arnaud Lequen, Clément Legrand-Lixon, Léo SaulièresMulti-Objective Reinforcement Learning Reinforcement Learning
Linköping University, Sweden · Univ. Lille, CNRS, Centrale Lille, UMR 9189 CRIStal, F-59000 Lille, France · Univ. Toulouse, INRAE-MIAT, Toulouse, France
From Trainee to Trainer: LLM-Designed Training Environment for RL with Multi-Agent Reasoning
Jun 16, 2026Chao Chen, Chengzu Li, Zhiwei Li +2Reinforcement Learning LLM-Guided RL
1LARK, HKUST (GZ) · University of Cambridge · 3HKUST
EnvRL: Learn from Environment Dynamics in Agentic Reinforcement Learning
Jun 16, 2026Zhitong Wang, Songze Li, Hao Peng +4Agentic RL World Model Learning
Department of Computer Science and Technology, Tsinghua University · Shanghai AI Laboratory

Jun 29, 2026, cs.AIJ/K move · Enter open · S save

Exploration and Online Transfer with Behavioral Foundation Models

Louis Bagot, Mathieu Lefort, Laëtitia Matignon

SyCoSMA · Université Lyon 1, CNRS, INSA Lyon LIRIS - UMR5205, 69622 Villeurbanne, France · LIRIS, SyCoSMA, IRISA, MALT, UR+1

Reinforcement Learning RL Exploration Policy Learning

Open paper AskPDF arXiv

Zero-shot Transfer in Reinforcement Learning (RL) aims to train an agent that can generate optimal policies for any reward function, without additional learning at transfer time, while training only on reward-free trajectories. For their generality over tasks, such models are sometimes called ``Behavioral Foundation Models'' (BFMs). While they have shown strong performances and improvements in recent years, the current framework and algorithms still assume that, during the transfer phase, the agent is informed offline about the reward (the task to solve) through a dataset of state-reward pairs, which it uses to pick the best policy to deploy. However, in practice if the reward is a black-box (e.g. direct user feedback), it is not possible to generate such a dataset: it is necessary to observe the reward through interactions with the environment. In other words, the current framework of offline transfer is not aligned with the traditional RL setting of online learning through trial-and-error, which requires exploration in order to find rewards. This paper proposes to tackle this new online transfer in zero-shot RL, with the key insight that the BFM itself can be used to generate exploration policies. We show that it is possible to frame this online learning problem in terms of a bandit-like exploration-exploitation problem. More precisely, at each step the bandit algorithm recommends a policy, the BFM executes it in the environment, which yields a reward and a new state; we repeat the process until we converge to the optimal policy. In the popular context of linear reward approximation, we derive a formulation inspired by Upper Confidence Bound and show that exploration can be achieved through the minimization of the eigenvalues of an uncertainty matrix. We evaluate qualitatively and quantitatively our framework on a simple environment to validate the concept of our method.

Momentum

Related topics

Latest papers 416