RL Exploration

RL: Reinforcement Learning

Exploration studies how an agent can gather useful experience when rewards are sparse, the environment is uncertain, or its current policy visits only a narrow part of the state space. Common approaches use uncertainty bonuses, novelty and entropy rewards, model-based planning, goal or skill discovery, memory, and structured or language-guided exploration. These ideas are tested in settings from bandits and sparse-reward navigation to robotics, multi-agent tasks, and language-model reasoning and tool use. A central challenge is to seek information that improves future decisions without wasting experience on irrelevant novelty or destabilizing learning, while maintaining safety and useful behavior.

Latest papers

221 papers29 papers in the last four weeks, up 81% on the four weeks before. 0.2% of all new papers.

Guided-RLVR: Guided Exploration in Reinforcement Learning with Verifiable Rewards
Oct 27, 2026Bo Yang, Zishuai Zhang, Weihua Li +3RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University · School of Artificial Intelligence, Beihang University
Multi-Horizon Future Discovery for Physics-Informed Exploration
Oct 26, 2026Shuwei Wang, Qiang Guan, Brian Qiu +5RL Exploration Intrinsic Motivation
University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences (CASIA) · School of Artificial Intelligence, University of Chinese Academy of Sciences +2
VISTA: Active Exploration for Self-Improvement in Embodied Agents
Oct 26, 2026Zhang Zhang, Shengjun Zhang, Simin Huang +4Self-Improving Agents Agent Harness Optimization
Tsinghua University · National University of Singapore
Asymmetric On-Policy Distillation Learns In-Context Exploration
Oct 21, 2026Sriyash Poddar, Yanda Bao, Jacob Krantz +5RL Exploration Partially Observable RL
Paul G. Allen School of Computer Science and Engineering, University of Washington · Work partially done while at Meta
Second Thoughts: Exploring and Planning through Comparative Imagination in World Models
Oct 20, 2026Hieu Tran, Hong YuWorld Model-Based Planning Model-Based RL
University of Massachusetts Amherst · University of Massachusetts Lowell
Exploration via Activation-Space Steering for Reinforcement Learning with Verifiable Rewards
Oct 19, 2026Yunan Zeng, Jinhao Jing, Xi Cheng +1RL for Language Model Reasoning RL Exploration
Fudan University · The Chinese University of Hong Kong, Shenzhen · Kunlun Inc. +1
Productive State-Space Exploration for Multi-turn RL in LLM Agents
Oct 19, 2026Mohammed Haroon Dupty, Yong Liang Goh, Song Guo +1RL for LLM Agents RL Exploration
Institute of Advanced Computing and Intelligence, A*STAR · School of Computing, National University of Singapore
Measuring Empowerment from Offline Data for Intrinsically Motivated Exploration
Oct 17, 2026Ishir Garg, Vivek Myers, Qiyang Li +3Empowerment RL Exploration
UC Berkeley · Princeton University
Average-Reward Policy Identification from a Single Continuing Trajectory
Oct 14, 2026Tongxi WangReinforcement Learning Average-Reward RL
Southeast University
What Information Buys in Active Inference
Oct 13, 2026Melih Kandemir, Mustafa Mert ÇelikokRL Exploration Active Inference
Department of Mathematics and Computer Science, University of Southern Denmark
Explore with Memory, Learn without It: In-Context Exploration and Self-Distillation for LLM Reasoning
Oct 13, 2026Yutong Chen, Zhichao Xu, Rui Liu +1RL for Language Model Reasoning RL Exploration
Department of Computer Science, Clemson University · Kahlert School of Computing, University of Utah
MemAct: Streaming Spatial Understanding with Memory and Active Exploration
Oct 13, 2026Yuxi Wei, Wei Huang, Qirui Chen +2Memory-Augmented Agents Embodied Reasoning
The University of Hong Kong · Shanghai Jiao Tong University · Yinwang Intelligent Technology Co. Ltd
Beyond Policy Rollouts: Embedding-Space Directional Exploration with Anchored Noise for LLM RLVR
Oct 12, 2026Daijiao Liu, Huiyi Wang, Lina Yao +1RL for Language Model Reasoning RL Exploration
University of New South Wales (UNSW Sydney)
CFG-Policy: Unified Diffusion Policy Learning Framework via Classifier-free Guidance
Oct 10, 2026Shutong Ding, Zhaoyang Liu, Yimiao Zhou +3Diffusion Policy Robotic RL
ShanghaiTech University, Shanghai, China · InstAdapt, Shanghai, China
Travel Back To Move Forward: Retrospective Exploration and Evolving for Long Horizon Agents
Oct 9, 2026Ziang Ye, Wentao Shi, Lei Chen +7Long-Horizon LLM Agents LLM Agent Planning
University of Science and Technology of China · Shanghai Innovation Institute · Meituan +4
Can Jev be Your Q or Policy in Reinforcement Learning?
Oct 8, 2026Yi Ma, Tianpei Yang, Yaodong Yang +2LLM-Guided RL Sample-Efficient RL
Shanxi University · Nanjing University · The Chinese University of Hong Kong +2
Decoupling Exploration from Optimization in RLVR
Oct 7, 2026Saif Punjwani, Micah GoldblumRL for Language Model Reasoning RL Exploration
Department of Computer Science, Columbia University · Department of Electrical Engineering, Columbia University
Constraint Tree Exploration for Learning from Language Feedback
Oct 6, 2026Shaoang Li, Daniel R. Jiang, Jian LiError Feedback RL Exploration
Stony Brook University · Meta Platforms
EigenDEXplore: Structured Exploration for Dexterous Manipulation with Human Priors
Oct 6, 2026Harsh Gupta, Tyler Ga Wei Lum, Changhao Wang +4Robotic Grasping Contact-Rich Robotic Manipulation
Stanford University
Variance-Aware Fine-Grained Gap-Dependent Bounds for Online Reinforcement Learning
Oct 3, 2026Haochen Zhang, Lingzhou Xue, Zhong ZhengReinforcement Learning Model-Free RL
Department of Statistics, The Pennsylvania State University · Department of Applied and Computational Mathematics and Statistics, University of Notre Dame
Hierarchical Credit Assignment for RLVR on Fused Gromov-Wasserstein Geometry
Oct 3, 2026Qi Yu, Ruizhong Qiu, Zhichen Zeng +6RL for Language Model Reasoning Credit Assignment in RL
University of Illinois Urbana-Champaign · Meta
Exploration-Preserving Policy Optimization
Oct 2, 2026Hangzhan jin, Mohammad Hamdaqa, Doina PrecupRL for Language Model Reasoning RL Exploration
Mila - Quebec AI Institute · Polytechnique Montr´eal · McGill University +2
Mastering Atari 2600 Games with Discovered Options
Oct 2, 2026Erik M. Lintunen, Marlos C. MachadoRL for Game AI Hierarchical RL
University of Alberta · Alberta Machine Intelligence Institute (Amii) · Canada CIFAR AI Chair
Bidirectional Voronoi-biased Exploration Curriculum for Reinforcement Learning
Oct 2, 2026Juri Pfammatter, Kaixian Qu, Clemens Schwarke +2Curriculum RL RL Exploration
Robotic Systems Lab, ETH Zürich, Zürich, Switzerland · NVIDIA
Instance-Dependent Regret for CMDPs with Step-Wise Constraints
Oct 1, 2026Qian Zuo, Francesco Emanuele Stradi, Leyang Xue +1Constrained RL Safe RL
University of Edinburgh · Politecnico di Milano · University College London
When Do Intrinsic Rewards Lead to Exploration?
Oct 1, 2026Scott W. Viteri, Laura Gomezjurado Gonzalez, Clark BarrettIntrinsic Reward Methods RL Exploration
Stanford University
DexPolicy: Scheduled Exploration for Trajectory-Guided Dexterous Manipulation
Sep 30, 2026Haoyu Wang, Siyuan Qian, Yanjun Li +4Robotic Manipulation RL Exploration
School of Computer Science, Peking University · AI2 Robotics
ExploreNet: Learning Where to Explore in Diffusion GRPO
Sep 29, 2026Shuyue Stella Li, Xiaochuang Han, Yulia Tsvetkov +1Diffusion Model Fine-Tuning RL Exploration
University of Washington
Where the Model Changes Its Mind: Hindsight-Divergence Localization for Efficient Reinforcement Learning with Verifiable Rewards
Sep 29, 2026Fanchao Chen, Hengyu Fu, Shivaram Venkataraman +1RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards
University of Wisconsin–Madison · University of California, Berkeley · ETH Zurich +1
Human-inspired, Task-Dimension-Guided Exploration for Efficient Learning in High Dimensions
Sep 29, 2026Fanyu Zhu, Jiahui An, Ni JiExploration-Exploitation Tradeoff RL Exploration
Beijing Institute for Brain Research, Chinese Academy of Medical Sciences & Peking Union Medical College, Chinese Institute for Brain Research, Beijing, Beijing Key Laboratory of Brain Science and Brain-Machine Interface · State Key Laboratory of Cognitive Neuroscience and Learning, Beijing Normal University

Oct 27, 2026J/K move · Enter open · S save

Guided-RLVR: Guided Exploration in Reinforcement Learning with Verifiable Rewards

Bo Yang, Zishuai Zhang, Weihua Li, Qinnan Zhang+2

Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing, Beihang University · School of Artificial Intelligence, Beihang University

RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards LLM Reasoning RL Exploration RL for Code Generation

Open paper AskPDF arXiv

Reinforcement learning with verifiable rewards (RLVR) has shown strong empirical gains in improving the reasoning ability of large language models (LLMs) on tasks such as mathematics and programming. However, recent studies have shown that RLVR rollouts often collapse onto a narrow set of reasoning trajectories, limiting the explored reasoning space. Motivated by findings that thinking tokens at critical steps serve as critical structural elements in LLM reasoning pathways, we propose Guided-RLVR, a novel approach that augments RLVR with a lightweight controller. The controller strategically injects semantically meaningful steering tokens into the sampling distribution, encouraging the model to approach problems from multiple reasoning perspectives and thereby expanding the exploration space. Experiments on mathematical reasoning and code generation benchmarks show that Guided-RLVR consistently outperforms existing RLVR methods while producing more diverse reasoning trajectories.