RL Exploration

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

17 papers in the last four weeks, up 325% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 139

All topics

TMRL: Diffusion Timestep-Modulated Pretraining Enables Exploration for Efficient Policy Finetuning
May 12, 2026Matthew M. Hong, Jesse Zhang, Anusha Nagabandi +1Robot Policy Adaptation Robot Skill Learning
University of Washington · Amazon FAR
Learning What Matters: Adaptive Information-Theoretic Objectives for Robot Exploration
May 12, 2026Youwei Yu, Jionghao Wang, Zhengming Yu +2Robotic Data Collection Robot Navigation
Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN, USA · Department of Computer Science & Engineering, Texas A&M University, College Station, TX, USA
fg-expo: Frontier-guided exploration-prioritized policy optimization via adaptive kl and gaussian curriculum
May 12, 2026Mingxiong Lin, Zhangquan Gong, Maowen Tang +6Reinforcement Learning RL for Language Model Reasoning
OPPO AI Center
Epistemic Uncertainty for Test-Time Discovery
May 11, 2026Kainat Riaz, Muhammad Ahmed Mohsin, Ahsan Bilal +5RL for Language Models RL Exploration
Stanford University · University of Oklahoma · National University of Sciences and Technology
expo: Exploration-prioritized policy optimization via adaptive kl regulation and gaussian curriculum sampling
May 11, 2026Mingxiong Lin, Zhangquan Gong, Maowen Tang +6Reinforcement Learning KL-Regularized RL
OPPO AI Center
Learning to Explore: Scaling Agentic Reasoning via Exploration-Aware Policy Optimization
May 9, 2026Xingyuan Hua, Sheng Yue, Ju RenRL for Language Model Reasoning Test-Time Scaling
Department of Computer Science and Technology, Tsinghua University, Beijing, China · State Key Laboratory of Internet Architecture, Tsinghua University, Beijing, China · School of Cyber Science and Technology, Sun Yat-sen University Shenzhen Campus, Shenzhen, China
How You Begin is How You Reason: Driving Exploration in RLVR via Prefix-Tuned Priors
May 9, 2026Yifan Xu, Junren Chen, Yifan ChenRL for Language Model Reasoning Prompt Tuning
Hong Kong Baptist University · University of Maryland, College Park
Model-Driven Policy Optimization in Differentiable Simulators via Stochastic Exploration
May 8, 2026Yuval Aroosh, Ayal TaitlerPolicy Optimization RL Exploration
Department of Industrial Engineering and Management Ben-Gurion University of the Negev Beer Sheva, Israel
Measuring Learning Progress via Gradient-Momentum Coupling
May 7, 2026Samuel Blad, Martin Längkvist, Amy LoutfiReinforcement Learning Intrinsic Motivation
Örebro University
Data-dependent Exploration for Online Reinforcement Learning from Human Feedback
May 6, 2026Zhen-Yu Zhang, Yuting Tang, Jiandong Zhang +2RL Exploration Sample-Efficient RL
Center for Advanced Intelligence Project, RIKEN · Graduate School of Frontier Sciences, The University of Tokyo · Northeastern University +1
What You Think is What You See: Driving Exploration in VLM Agents via Visual-Linguistic Curiosity
May 5, 2026Haoxi Li, Qinglin Hou, Jianfei Ma +6Intrinsic Motivation RL Exploration
Department of Computer Science and Engineering, Hong Kong University of Science and Technology (HKUST), Hong Kong, China. · Department of Computer Science, University of Southern California, Los Angeles, USA · School of Computing, National University of Singapore, Singapore. +1
T $^2$ PO: Uncertainty-Guided Exploration Control for Stable Multi-Turn Agentic Reinforcement Learning
May 4, 2026Haixin Wang, Hejie Cui, Chenwei Zhang +7Agentic RL Reinforcement Learning
University of California, Los Angeles · 2Amazon.com Inc.
Quality-Aware Exploration Budget Allocation for Cooperative Multi-Agent Reinforcement Learning
May 3, 2026Dahyun Oh, Minhyuk Yoon, H. Jin KimMulti-Agent Coordination Intrinsic Reward Methods
Department of Aerospace Engineering, Seoul National University, Seoul, Republic of Korea
NonZero: Interaction-Guided Exploration for Multi-Agent Monte Carlo Tree Search
May 1, 2026Sizhe Tang, Zuyuan Zhang, Mahdi Imani +1Multi-Agent System Optimization Multi-Agent Coordination
The George Washington University · Northeastern University
Exploration Hacking: Can LLMs Learn to Resist RL Training?
Apr 30, 2026Eyon Jang, Damon Falck, Joschka Braun +6RL Exploration LLM Safety
MATS · UC San Diego · Anthropic +1
Can Tabular Foundation Models Guide Exploration in Robot Policy Learning?
Apr 30, 2026Buqing Ou, Frederike DümbgenTabular Foundation Models Robot Policy Learning
Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA 15213, USA · Inria, Département d’informatique de l’ENS, CNRS, PSL Research University, Paris, France
Safe-Support Q-Learning: Learning without Unsafe Exploration
Apr 28, 2026Yeeun Lim, Narim Jeong, Donghwan LeeReinforcement Learning KL-Regularized RL
HD Korea Shipbuilding & Offshore Engineering · Department of Electrical Engineering, KAIST, Daejeon, South Korea
Hierarchical Behaviour Spaces
Apr 27, 2026Michael Tryfan Matthews, Anssi Kanervisto, Jakob Foerster +3Reinforcement Learning Hierarchical RL
Meta Superintelligence Labs · University of Oxford
DPEPO: Diverse Parallel Exploration Policy Optimization for LLM-based Agents
Apr 27, 2026Junshuo Zhang, Chengrui Huang, Feng Guo +6Reinforcement Learning RL Exploration
University of Electronic Science and Technology of China · DiDi
Curiosity-Critic: Cumulative Prediction Error Improvement as a Tractable Intrinsic Reward for World Model Training
Apr 20, 2026Vin Bhaskara, Haicheng WangIntrinsic Reward Methods Reinforcement Learning
Department of Computer Science, University of Toronto, Toronto, Canada.
OGER: A Robust Offline-Guided Exploration Reward for Hybrid Reinforcement Learning
Apr 20, 2026Xinyu Ma, Mingzhou Xu, Xuebo Liu +4Reward Shaping RL for Language Model Reasoning
Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China · Hithink RoyalFlush Information Network, Hangzhou, China · Computer and Information Science, University of Macau, Macau, China
HEALing Entropy Collapse: Enhancing Exploration in Few-Shot RLVR via Hybrid-Domain Entropy Dynamics Alignment
Apr 20, 2026Zhanyu Liu, Qingguo Hu, Ante Wang +5RL for Language Model Reasoning Few-Shot Learning
Key Laboratory of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian and Taiwan (Xiamen University), Ministry of Culture and Tourism, China · Tsinghua University · Xiamen Unisound Intelligence Technology Co., Ltd
Poly-EPO: Training Exploratory Reasoning Models
Apr 19, 2026Ifdita Hasan Orney, Jubayer Ibn Hamid, Shreya S Ramanujam +5Reinforcement Learning RL for Language Model Reasoning
Stanford University
SPS: Steering Probability Squeezing for Better Exploration in Reinforcement Learning for Large Language Models
Apr 18, 2026Yifu Huo, Chenglong Wang, Ziming Zhu +9Reinforcement Learning RL for Language Model Reasoning
1Northeastern University, Shenyang, China · 2CAS Key Laboratory of Behavioral Science, Beijing, China · 3Independent Researcher, Beijing, China +1
Flexible Empowerment at Reasoning with Extended Best-of-N Sampling
Apr 17, 2026Taisuke KobayashiReinforcement Learning Best-of-N Sampling
National Institute of Informatics and The Graduate University for Advanced Studies, SOKENDAI Japan
Targeted Exploration via Unified Entropy Control for Reinforcement Learning
Apr 16, 2026Chen Wang, Lai Wei, Yanzhi Zhang +5Reinforcement Learning RL for Language Model Reasoning
College of Software, Nankai University · Zhongguancun Academy · Shanghai Jiao Tong University +3
ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Driving
Apr 3, 2026Zihao Sheng, Xin Ye, Jingru Luo +2World Model Learning VLMs for Autonomous Driving
Bosch Research North America & Bosch Center for Artificial Intelligence (BCAI) · University of Wisconsin–Madison
ContactExplorer: Contact Coverage-Guided Exploration for General-Purpose Dexterous Manipulation
Mar 11, 2026Zixuan Liu, Ruoyi Qiao, Chenrui Tie +5Contact-Rich Robotic Manipulation RL Exploration
School of Computing, National University of Singapore · RoboScience
Conformal Policy Control
Mar 2, 2026Drew Prinster, Clara Fannjiang, Ji Won Park +4Constrained RL RL Exploration
Prescient Design, Genentech, U.S.A. · Johns Hopkins University, Baltimore, U.S.A. · New York University, NYC, U.S.A. +1
Unveiling Implicit Advantage Symmetry: Why GRPO Struggles with Exploration and Difficulty Adaptation
Feb 5, 2026Zhiqi Yu, Zhangquan Chen, Mengting Liu +2RL for Language Model Reasoning Group Relative Policy Optimization
University of Hong Kong · Tsinghua University · Sun Yat-sen University

May 12, 2026, cs.ROJ/K move · Enter open · S save

TMRL: Diffusion Timestep-Modulated Pretraining Enables Exploration for Efficient Policy Finetuning

Matthew M. Hong, Jesse Zhang, Anusha Nagabandi, Abhishek Gupta

University of Washington · Amazon FAR

Robot Policy Adaptation Robot Skill Learning Robotic RL Diffusion Policy RL Exploration Data-Efficient Robot Learning+4

Open paper AskPDF arXiv

Fine-tuning pre-trained robot policies with reinforcement learning (RL) often inherits the bottlenecks introduced by pre-training with behavioral cloning (BC), which produces narrow action distributions that lack the coverage necessary for downstream exploration. We present a unified framework that enables the exploration necessary to enable efficient robot policy finetuning by bridging BC pre-training and RL fine-tuning. Our pre-training method, Context-Smoothed Pre-training (CSP), injects forward-diffusion noise into policy inputs, creating a continuum between precise imitation and broad action coverage. We then fine-tune pre-trained policies via Timestep-Modulated Reinforcement Learning (TMRL), which trains the agent to dynamically adjust this conditioning during fine-tuning by modulating the diffusion timestep, granting explicit control over exploration. Integrating seamlessly with arbitrary policy inputs, e.g., states, 3D point clouds, or image-based VLA policies, we show that TMRL improves RL fine-tuning sample efficiency. Notably, TMRL enables successful real-world fine-tuning on complex manipulation tasks in under one hour. Videos and code available at https://weirdlabuw.github.io/tmrl/.

Momentum

Related topics

Latest papers 139