RL Exploration

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

17 papers in the last four weeks, up 325% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 139

All topics

Decoupling Exploration from Optimization in RLVR
Oct 7, 2026Saif Punjwani, Micah GoldblumRL for Language Model Reasoning RL Exploration
Department of Computer Science, Columbia University · Department of Electrical Engineering, Columbia University
Constraint Tree Exploration for Learning from Language Feedback
Oct 6, 2026Shaoang Li, Daniel R. Jiang, Jian LiRL Exploration RL from Human Feedback
Stony Brook University · Meta Platforms
EigenDEXplore: Structured Exploration for Dexterous Manipulation with Human Priors
Oct 6, 2026Harsh Gupta, Tyler Ga Wei Lum, Changhao Wang +4Robotic Grasping Contact-Rich Robotic Manipulation
Stanford University
When Do Intrinsic Rewards Lead to Exploration?
Oct 1, 2026Scott W. Viteri, Laura Gomezjurado Gonzalez, Clark BarrettIntrinsic Reward Methods RL Exploration
Stanford University
ExploreNet: Learning Where to Explore in Diffusion GRPO
Sep 29, 2026Shuyue Stella Li, Xiaochuang Han, Yulia Tsvetkov +1Diffusion Model Fine-Tuning RL Exploration
University of Washington
Human-inspired, Task-Dimension-Guided Exploration for Efficient Learning in High Dimensions
Sep 29, 2026Fanyu Zhu, Jiahui An, Ni JiExploration-Exploitation Tradeoff RL Exploration
Beijing Institute for Brain Research, Chinese Academy of Medical Sciences & Peking Union Medical College, Chinese Institute for Brain Research, Beijing, Beijing Key Laboratory of Brain Science and Brain-Machine Interface · State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University
Going Beyond State-Reaching: Learning Abstractions for Intrinsically Motivated Option Discovery
Sep 29, 2026Akhil Bagaria, Anita De Mello Koch, George KonidarisHierarchical RL RL Exploration
Brown University
Deep Epistemic Value Functions for Optimistic Exploration
Sep 28, 2026Leander Diaz-Bone, Marco Bagatella, Jonas Hübotter +1Value Function Estimation RL Exploration
ETH Zürich, Switzerland · Max Planck Institute for Intelligent Systems, Germany
ICMAPE: In-Context Multiagent Pure Exploration
Sep 27, 2026Xinyi Hu, Alessio Russo, Aldo PacchianoRL Exploration Decentralized MARL
Boston University · Broad Institute of MIT and Harvard
TGRL: Temperature-Grouped Reinforcement Learning for Efficient Exploration in LLMs
Sep 27, 2026Zihan Lin, Xiaohan Wang, Jie Cao +4RL for Language Models RL for Language Model Reasoning
University of Chinese Academy of Sciences · Meituan · MAIS&NLPR, Institute of Automation, Chinese Academy of Sciences
Beyond Conservatism: Recoverability-Conditioned Exploration for Model-Based Imitation Learning
Sep 27, 2026Xuanlin Chen, Ziyue Wang, Xunlan Zhou +3Imitation Learning RL Exploration
School of Intelligent Science and Technology, Nanjing University, China · National Key Laboratory for Novel Software Technology, School of Artificial Intelligence, Nanjing University, China
AlphaDiverse: Post-Training Local Quantitative Research Agents for Diverse Exploration in Alpha Factor Mining
Sep 24, 2026Qingzhuo Wang, Zikun Wei, Zhihua Wei +1Multi-Agent LLM Systems Quantitative Finance
Tongji University · Shanghai Non-convex Intelligent Technology
Uncertainty-Gated Exploration Noise Suppresses Task Collapse in Online RL Fine-Tuning of a Flow-Matching Vision-Language-Action Policy
Sep 23, 2026Mehmet Turan Yardımcı, Yunus Emre ÇoğurcuRobot Policy Learning RL Exploration
Karlsruhe Institute of Technology · Çukurova University
Anchor and Perturb: Lazy Agent Remediation by Exploration Injection
Sep 23, 2026Chengxi Zhong, Yongzhe ChangMulti-Agent Coordination RL Exploration
Don't Mask the Environment: Observation Supervision Changes How Agents Explore Under RL
Sep 17, 2026Juzheng Zhang, Disha Makhija, Manoj Ghuhan Arivazhagan +2Supervised Fine-Tuning Reinforcement Learning
University of Maryland · AWS AI Labs
Learning to Solve Stochastic Controls with Unknown Drifts and Running Rewards: Theory, Algorithms and Convergence
Sep 14, 2026Jin Ma, Gaozhan Wang, Jianfeng Zhang +1Reinforcement Learning Stochastic Optimal Control
Department of Mathematics, University of Southern California, Los Angeles, CA 90089 · Department of Industrial Engineering and Operations Research, Columbia University, New York, NY 10027
Dream-RSI: Recursive Self-Improvement through Evolving Worlds
Sep 14, 2026Tong Zheng, Xidong Wu, Zheng Zhang +14RL Exploration Self-Improving Agents
Google · University of Maryland, College Park · Google Deepmind +1
Difficulty-Adaptive Tree-Structured Policy Optimization for Expanding Reasoning Coverage in RLVR
Sep 8, 2026Youngjun Yu, Sanghwan Jang, Hwanjo YuRL for Language Model Reasoning RL Exploration
Pohang University of Science and Technology (POSTECH)
SUN: Reaching for Novelty in Reinforcement Learning
Sep 8, 2026Wenyan Yang, Arsenii Mustafin, Dominik Baumann +2Intrinsic Motivation RL Exploration
Aalto University · Tampere University
Efficient Exploration Is Enough
Sep 7, 2026Mikel Malagón, Jon Vadillo, Josu Ceberio +2Intrinsic Reward Methods RL Exploration
University of the Basque Country (UPV/EHU) · University of Alberta · Alberta Machine Intelligence Institute +2
CIG-RL: Curiosity-Driven Information-Guided Reinforcement Learning for Source Term Estimation in Uncertain Environments
Aug 31, 2026Junhee Lee, Seunghwan Kim, Hongro Jang +4RL Exploration Robust RL
Department of Mechanical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea. · Department of Mechanical Engineering, Ulsan National Institute of Science and Technology (UNIST), Ulsan 44919, Republic of Korea.
Exploration-Driven Personalized Federated Reinforcement Learning via Intrinsic Motivation
Aug 11, 2026Md Rafid Islam, Rafsan Jany, Zahid Hasan +1Federated RL Intrinsic Motivation
Department of Electrical and Computer Engineering North South University Dhaka, Bangladesh · Digital Health Research Division Korea Institute of Oriental Medicine Daejeon, South Korea · Department of Electrical and Computer Engineering The University of Alabama in Huntsville Huntsville, AL, USA
Parameter Exploration for RLVR via Variational Learning
Aug 10, 2026Vatsal Venkatkrishna, Nico Daheim, Iryna GurevychRL for Language Model Reasoning RL Exploration
INSAIT, Sofia University “St. Kliment Ohridski”, Bulgaria · Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science, Technical University of Darmstadt and National Research Center for Applied Cybersecurity ATHENE, Germany
RISE-RL: Rubric-Informed Selective Exploration for Open-Ended Reinforcement Learning
Aug 10, 2026Jinkun Hou, Zhuo Liu, Huimin Ren +3Open-Ended Generation RL for Language Models
1Peking University · 2Beijing Institute of Technology · 3Li Auto Inc.
Bootstrap-Conditioned Action Selection with Tabular Foundation Models
Aug 6, 2026Devansh Gupta, Shiv Tavker, Dmitry Efimov +3Tabular Foundation Models RL Exploration
Amazon Science
Reward Structure Shapes the Interaction Between Episodic Exploration and Neural Memory in Reinforcement Learning
Aug 5, 2026Jai Malegaonkar, Rohan Patil, Henrik I. ChristensenReinforcement Learning Partially Observable RL
Department of Computer Science and Engineering, UC San Diego, United States
Explore Beyond the Boundary Using Entropic Information
Jul 31, 2026Bumgeun Park, Donghwan LeeReinforcement Learning RL Exploration
Korea Advanced Institute of Science and Technology, Daejeon 34051, Korea
WARL: Wrench-Augmented Reinforcement Learning for Task-Agnostic Learning in Legged Robots
Jul 27, 2026Keita Yoneda, Kento Kawaharazuka, Kei OkadaCurriculum RL RL Exploration
Department of Mechano-Informatics, Graduate School of Information Science and Technology, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8656, Japan. · AI Center, Graduate School of Information Science and Technology, The University of Tokyo, Japan.
Try Once, Then Optimal: De-Redundified Procedure Memory for Cross-Episode Exploration Amortization
Jul 26, 2026Haizhou Ge, Haochen Ouyang, Zhixing Chen +6RL Exploration Robotic Manipulation
1Tsinghua University. · 3Northeast Agricultural University. · 2DISCOVER Robotics.
Progress-conditioned Group Policy Optimization for Long-Horizon Agentic Tasks
Jul 22, 2026Kaibing Yang, Guangfeng Cai, Shengtian Yang +6Group Relative Policy Optimization RL Exploration
Southeast University · Kuaishou Technology · Work done during an internship at Kuaishou Technology, supervised by Jun Xu. +1
Information-Based Exploration via Random Features for Reinforcement Learning
Jul 20, 2026Waris Radji, Odalric-Ambrym MaillardRL Exploration
Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189-CRIStAL, F-59000 Lille, France
Non-Asymptotic Best Policy Identification Guarantees in Online Reinforcement Learning
Jul 19, 2026Joseph Lazzaro, Alessio Russo, Aldo PacchianoMarkov Decision Processes RL Exploration
Imperial College London · Boston University · Broad Institute of MIT and Harvard
When Can Safe Controllers Adapt? Information before Commitment
Jul 18, 2026Venkatesh SaligramaRL Exploration Optimal Control
Department of Electrical and Computer Engineering, Boston University, MA 02215
Principled Direction-Free Intrinsic Motivation through Model-Free Epistemic Free-Energy Estimators
Jul 18, 2026Alireza Furutanpey, Schahram DustdarFree Energy Calculation Intrinsic Reward Methods
Coovally, Barcelona, Spain · Distributed Systems Group, TU Vienna, Austria · ICREA Barcelona, Spain
Process Reward Informed Tree Rollout for Effective Multi-Turn RL
Jul 17, 2026Xintong Li, Sha Li, Yuwei Zhang +8Reinforcement Learning RL Exploration
1UC San Diego · 2Amazon · 3MIT Alumni
ExToken: Structured Exploration for Efficient Vision-Language-Action Reinforcement Fine-tuning
Jul 14, 2026Yilun Kong, Yunpeng Qing, Guozheng Ma +4RL Exploration Robotic Manipulation
Nanyang Technological University · ACE Robotics · Zhejiang University +1
Beyond Euclidean Clipping: Overcoming Exploration Collapse in LLM RL via Riemannian Isometric Policy Optimization
Jul 11, 2026Zhicheng Cai, Xinyuan Guo, Hanlin Wu +4Reinforcement Learning RL for Language Model Reasoning
Institute for AI Industry Research (AIR), Tsinghua University · 2ByteDance Seed 3SIA-Lab of Tsinghua AIR and ByteDance Seed.
TopoExplore: Topological Discrimination for Archive-Based Exploration
Jul 10, 2026Jason CarlsonRL Exploration Frontier-Based Exploration
Prompt-Driven Exploration
Jul 9, 2026Sunshine Jiang, John Marangola, David Zhang +6LLM-Guided RL RL Exploration
Massachusetts Institute of Technology · Improbable AI Lab · MIT-IBM Computing Research Lab
UP: Unbounded Positive Asymmetric Optimization for Breaking the Exploration-Stability Dilemma
Jul 8, 2026Chongyu Fan, Pengfei Liu, Jingjia Huang +2RL for Language Model Reasoning Policy Optimization
ByteDance Seed · Michigan State University
TREK: Distill to Explore, Reinforce to Refine
Jul 6, 2026Yuanda Xu, Zhengze Zhou, Kayhan Behdin +10RL for Language Model Reasoning LLM-Guided RL
LinkedIn Corporation · Harvard University · Georgia Institute of Technology
Learning Task-Sufficient World Models by Synergizing Agentic Exploration and Structured Modeling
Jul 5, 2026Fan Feng, Yujia Zheng, Minghao Fu +5World Model Learning World Models
1UCSD · 2MBZUAI · 3CMU +1
Exploration and Online Transfer with Behavioral Foundation Models
Jun 29, 2026Louis Bagot, Mathieu Lefort, Laëtitia MatignonReinforcement Learning RL Exploration
SyCoSMA · Université Lyon 1, CNRS, INSA Lyon LIRIS - UMR5205, 69622 Villeurbanne, France · LIRIS, SyCoSMA, IRISA, MALT, UR +1
Dual-Flow Reinforcement Learning with State-Aware Exploration
Jun 29, 2026Qijun Li, Zheng Fu, Qi Song +4Distributional RL Reinforcement Learning
School of Vehicle and Mobility and the State Key Laboratory of Intelligent Green Vehicle and Mobility, Tsinghua University, Beijing 100084, China
Implementation of reinforcement learning in chemical reaction networks: application to phototaxis as curiosity-driven exploration
Jun 24, 2026Ruyi Tang, Grégoire Sergeant-Perthuis, David ColliauxDynamical Systems Reinforcement Learning
LCQB-AG · 1CQSB, Sorbonne University, France · 2Sony CSL, France
Modularized Reinforcement Learning on LLMs: From MDP Creation to Exploration and Learning
Jun 20, 2026Zhao Yang, Yuxuan Jiang, Ting-Chih Chen +18RL for Language Models Credit Assignment in RL
VU Amsterdam, The Netherlands · University of Maryland, Baltimore County, USA · Leiden University, The Netherlands +4
Reward-free Pretraining for Reinforcement Learning via Occupancy Coverage Maximization
Jun 19, 2026Marco Pratticò, Pietro Novelli, Massimiliano Pontil +1Reinforcement Learning Exploration-Exploitation Tradeoff
Computational Statistics and Machine Learning - Istituto Italiano di Tecnologia, 16100 Genova, Italy · 2AI Centre, Computer Science Department, University College London, London, UK.
Quantile of Means: A Bonus-Free Ensemble Method for Minimax Optimal Reinforcement Learning
Jun 18, 2026Asaf Cassel, Aviv RosenbergRegret Minimization in RL Reinforcement Learning
Google Research
Reward as An Agent for Embodied World Models
Jun 18, 2026Pu Li, Zhigang Lin, Qiang Wu +3Reward Modeling Reward Hacking
Kairos Team
DF-ExpEnse: Diffusion Filtered Exploration for Sample Efficient Finetuning
Jun 17, 2026Calvin Luo, Chen Sun, Shuran SongRobot Policy Adaptation Robotic RL
1Stanford University · 2Brown University.
Can In-Context Learning Support Intrinsic Curiosity?
Jun 17, 2026Eric Elmoznino, Sangnie Bhardwaj, Johannes von Oswald +5Intrinsic Reward Methods Intrinsic Motivation
1Google – Paradigms of Intelligence Team, 2Google DeepMind
UBP2: Uncertainty-Balanced Preference Planning for Efficient Preference-based Reinforcement Learning
Jun 17, 2026Mohamed Nabail, Leo Kaixuan Cheng, Jingmin Wang +1Pairwise Preference Evaluation RL Exploration
Learning, Embodied Autonomy, and Forecasting (LEAF) Lab, University of Toronto
Infant Spontaneous Movement Noise Improves Exploration in Deep RL
Jun 15, 2026Francisco M. López, Markus R. Ernst, Francisco Cruz +2Reinforcement Learning RL Exploration
Frankfurt Institute for Advanced Studies, Germany, and with the School of Computer Science and Engineering, University of New South Wales, Australia · School of Computer Science and Engineering, University of New South Wales, Australia · School of Computer Science and Engineering, University of New South Wales, Australia, and with Escuela de Ingenier´ıa, Universidad Central de Chile, Chile +2
Repeated Bilateral Trade: The Quest for Fairness
Jun 13, 2026François Bachoc, Roberto Colomboni, Emilie KaufmannMechanism Design Algorithmic Fairness
University of Lille Institut Universitaire de France (IUF) Lille, France · School of Mathematics University of Bristol Fry Building, Woodland Road Bristol BS8 1UG, United Kingdom · Univ. Lille, CNRS, Inria, Centrale Lille, UMR 9189-CRIStAL F-59000 Lille, France
Exploring Starts Are Not Enough: Counterexamples and a Fix for Monte Carlo Exploring Starts
Jun 13, 2026Octave Oliviers, Glenn VinnicombeRL Control Reinforcement Learning
Department of Engineering · University of Cambridge · Cambridge, UK
VANDERER: Map-Free Exploration using Future-Aware and Visual-Curiosity-Guided Diffusion Policy
Jun 12, 2026Venkata Naren Devarakonda, Raktim Gautam Goswami, Prashanth Krishnamurthy +1Diffusion-Based Planning Robot Navigation
Control/Robotics Research Laboratory (CRRL), Department of Electrical and Computer Engineering, NYU Tandon School of Engineering, Brooklyn, NY, 11201.
Reasoning or Memorization? Direction-Aware Diversity Exploration in LLM Reinforcement Learning
Jun 9, 2026Jiangnan Xia, Yucheng Shi, Yu Yang +3Memorization in Language Models RL for Language Model Reasoning
University of Georgia · Tencent AI Lab · The Education University of Hong Kong +1
SHAPO: Sharpness-Aware Policy Optimization for Safe Exploration
Jun 8, 2026Kaustubh Mani, Yann Pequignot, Vincent Mai +1Sharpness-Aware Minimization RL Exploration
Universit´e de Montr´eal · Mila - Qu´ebec AI Institute · Universit´e Laval +2
An Information-Theoretic Definition for Open-Ended Learning
Jun 6, 2026Wanqiao Xu, Yifan Zhu, Benjamin Van RoyLifelong Learning Agents RL Exploration
Retry Policy Gradients in Continuous Action Spaces
Jun 4, 2026Soichiro Nishimori, Paavo ParmasPolicy Optimization RL Exploration
The University of Tokyo, Japan

Oct 7, 2026, cs.LGJ/K move · Enter open · S save

Decoupling Exploration from Optimization in RLVR

Saif Punjwani, Micah Goldblum

Department of Computer Science, Columbia University · Department of Electrical Engineering, Columbia University

RL for Language Model Reasoning RL Exploration Reinforcement Learning with Verifiable Rewards

Open paper AskPDF arXiv

Modern language models undergo reinforcement learning with verifiable rewards (RLVR) on top of already-trained checkpoints. A key promise of RLVR is the discovery of new reasoning strategies. In principle, a model can sample novel ideas absent from its prior training data. In practice, however, augmenting RLVR with strong novelty incentives has seen limited success and can degrade model quality. Because verifiable rewards supervise only a narrow slice of the model's knowledge and behavior, such degradations are difficult to recover from. Instead, we decouple exploration from optimization in a framework we call Exploration-Distillation (ExpDis). We train one or more explorer policies with a novelty bonus in the reward, filter their trajectories for correctness and quality, and distill them into a separate student policy. The student policy is then trained without a novelty bonus. We repeat the above procedure for several rounds, alternating between exploration and optimization. This decoupling allows us to aggressively scale exploration without degrading the student policy. Across seven mathematical reasoning benchmarks and two model families, ExpDis outperforms DAPO at the same wall-clock budget. Moreover, we observe improved pass@

k

scaling, indicating that ExpDis produces models that generate more diverse correct solutions.

Momentum

Related topics

Latest papers 139