RL Exploration

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

17 papers in the last four weeks, up 325% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 140

All topics

Retry Policy Gradients in Continuous Action Spaces
Jun 4, 2026Soichiro Nishimori, Paavo ParmasPolicy Optimization RL Exploration
The University of Tokyo, Japan
MineXplore: An Open-Source Reinforcement Learning Exploration Benchmark for GNSS-Denied Underground Environment
Jun 3, 2026Abhishek S, Badrikanath Praharaj, Sreeram MVRL Benchmarks Robot Navigation
BuildMachineLabs
Easy-to-Use Shielding for Reinforcement Learning
Jun 2, 2026Stefan Pranger, Bettina KönighoferReinforcement Learning RL Exploration
Institute of Information Security, Graz University of Technology
Tool-Aware Optimization with Entropy Guidance for Efficient Agentic Reinforcement Learning
Jun 2, 2026Hongye Cao, Nuo Yan, Haoyuan Deng +5Agentic RL RL for Language Model Reasoning
National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing 210093, China · Nanyang Technological University, Singapore · China Mobile NineVerse Artificial Intelligence Technology (Beijing) Co., Ltd., and Institute of Artificial Intelligence, NineVerse, Beijing 100032, China
Constraint-Enhanced Physical Search through Correlation Matching
Jun 2, 2026Song-Ju KimMulti-Armed Bandits RL Exploration
SOBIN Institute LLC, Kawanishi, Hyogo, Japan
Randomized Least Squares Value Iteration itself is Joint Differentially Private
Jun 1, 2026Haiyang Lu, Pratik Gajane, Shaojie Bai +1Markov Decision Processes RL Exploration
1Laboratoire d’Informatique Fondamentale d’Orléans (LIFO), Université d’Orléans · College of Control Science and Engineering, Zhejiang University · Department of Computer Science, University of Copenhagen
Joint Agent Memory and Exploration Learning via Novelty Signals
Jun 1, 2026Shizuo Tian, Xiaohong Weng, Rui Kong +9RL Exploration Memory-Augmented Language Model Agents
Tsinghua University · Sun Yat-sen University · Baidu Inc. +2
Efficient Exploration for Iterative Nash Preference Optimization
May 31, 2026Tianlong Nan, Xiaopeng Li, Christian Kroer +1Nash Equilibrium LLM Alignment
Columbia University⋄ · The Chinese University of Hong Kong, Shenzhen†
Emergence of Exploration in Policy Gradient Reinforcement Learning via Retrying
May 29, 2026Soichiro Nishimori, Paavo Parmas, Sotetsu Koyamada +4RL Exploration Policy Gradient Methods
The University of Tokyo, Tokyo, Japan · Kobe University, Kobe, Japan · Kyoto University, Kyoto, Japan +3
Uncertainty-Aware and Temporally Regulated Expert Advice in Reinforcement Learning for Autonomous Driving
May 28, 2026Ahmed Abouelazm, Felix Klingebiel, Philip Schörner +1Reinforcement Learning Risk-Sensitive RL
FZI Research Center for Information Technology, Germany · Karlsruhe Institute of Technology, Germany
Information-Directed Offline-to-Online Reinforcement Learning
May 28, 2026Keru ChenRegret Minimization in RL Bayesian RL
School of Electrical, Computer and Energy Engineering, Arizona State University
Exploratory Experience Shapes the Geometry of Predictive Representations
May 27, 2026Kseniia Shilova, Abdelrahman Sharafeldin, Advay Balakrishnan +1Computational Cognitive Modeling Predictive Coding
School of Mathematics Georgia Institute of Technology Atlanta, GA 30332 · College of Computing Georgia Institute of Technology Atlanta, GA 30332
Breaking the Epistemic Trap: Active Perception Under Compound Uncertainty
May 26, 2026Chayan Banerjee, Ethan GoanActive Perception Constrained RL
School of Electrical Engineering and Robotics, Queensland University of Technology, Brisbane, Australia.
Evolving Robustness--Exploration Trade-off in Online Reinforcement Learning via Quantile Bayesian Risk MDPs
May 23, 2026Meichen Song, Yuhao Wang, Enlu ZhouReinforcement Learning Bayesian RL
School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, GA 30332, USA
Pure Exploration for a Good Policy in Reinforcement Learning with Bandit Feedback
May 22, 2026Zitian Li, Wang Chi CheungReinforcement Learning RL Exploration
Department of Industrial Systems Engineering & Management National University of Singapore Engineering Drive 2 Block E1A #06-25 Singapore 117576
Remember to be Curious: Episodic Context and Persistent Worlds for 3D Exploration
May 21, 2026Lily Goli, Justin Kerr, Daniele Reda +3Reinforcement Learning Embodied Navigation
University of Toronto · Wayve · Vector Institute +2
Kernel-Based Safe Exploration in Deep Reinforcement Learning
May 21, 2026Rupak Majumdar, Nikhil Singh, Sadegh SoudjaniControl Barrier Functions RL Exploration
Max Planck Institute for Software Systems, Germany
Mind the Sim-to-Real Gap & Think Like a Scientist
May 20, 2026Harsh Parikh, Gabriel Levin-Konigsberg, Dominique Perrault-Joncas +1Sim-to-Real Transfer RL Exploration
Amazon SCOT, Seattle, USA · Yale University, New Haven, USA · Duke University, Durham, USA
CIG: Exploration via Conditional Information Gain
May 20, 2026Tim Joseph, Marcus Fechner, Philipp Stegmaier +2Intrinsic Reward Methods Intrinsic Motivation
FZI · KIT
A conceptual framework for learning to listen by reward: Curiosity-driven search for novel sources
May 19, 2026Andreas Triantafyllopoulos, Jakub Šťastný, Alexios Terpinas +3Reinforcement Learning Intrinsic Motivation
CHI – Chair of Health Informatics, Technical University of Munich, Munich, Germany · MCML – Munich Center for Machine Learning, Munich, Germany · MDSI – Munich Data Science Institute, Munich, Germany +1
JAXenstein: Accelerated Benchmarking for First-Person Environments
May 19, 2026Ruo Yu Tao, George KonidarisRL Benchmarks Partially Observable RL
Department of Computer Science, Brown University, USA
Beyond Mode Collapse: Distribution Matching for Diverse Reasoning
May 19, 2026Xiaozhe Li, Yang Li, Xinyu Fang +10KL-Regularized RL RL for Language Model Reasoning
1Tongji University, China · 6Independent · 2Shanghai AI Laboratory, China +3
Generative Auto-Bidding with Unified Modeling and Exploration
May 19, 2026Mingming Zhang, Feiqing Zhuang, Na Li +7Automated Bidding Online Advertising
Key Laboratory of Aerospace Information Security and Trusted Computing, Ministry of Education, School of Cyber Science and Engineering, Wuhan University · Taobao & Tmall Group of Alibaba
FBOS-RL: Feedback-Driven Bi-Objective Synergistic Reinforcement Learning
May 18, 2026Xikai Zhang, Yongzhi Li, Likang Xiao +6RL for Language Model Reasoning RL Exploration
Hangzhou International Innovation Institute, Beihang University · Kuaishou Technology · School of Artificial Intelligence, Beihang University
Look Before You Leap: Autonomous Exploration for LLM Agents
May 15, 2026Ziang Ye, Wentao Shi, Yuxin Liu +6RL Exploration LLM Agents
University of Science and Technology of China · Meituan
Mind Dreamer: Untethering Imagination via Active Causal Intervention on Latent Manifolds
May 15, 2026Shaojun Xu, Xiaoling Zhou, Yihan Lin +4Reinforcement Learning Credit Assignment in RL
Center for Brain-Inspired Computing Research, Department of Precision Instrument, Tsinghua University, Beijing, China · College of Computer Science and Technology, Zhejiang University, Hangzhou, China · Pen-Tung Sah Institute of Micro-Nano Science and Technology, Xiamen University, Xiamen, China +1
ScreenSearch: Uncertainty-Aware OS Exploration
May 15, 2026Michael Solodko, Justin WagleComputer-Use Agents Monte Carlo Tree Search
Microsoft
Nudging Beyond the Comfort Zone: Efficient Strategy-Guided Exploration for RLVR
May 15, 2026Chanuk Lee, Sangwoo Park, Minki Kang +1RL for Language Model Reasoning RL Exploration
KAIST
Data-Augmented Game Starts for Accelerating Self-Play Exploration in Imperfect Information Games
May 14, 2026JB Lanier, Nathan Monette, Pierre Baldi +1RL Benchmarks Self-Play RL
University of California Irvine · University of Oxford
Delightful Exploration
May 13, 2026Ian OsbandMulti-Armed Bandits Value of Information
Google DeepMind

Jun 4, 2026, cs.AIJ/K move · Enter open · S save

Retry Policy Gradients in Continuous Action Spaces

Soichiro Nishimori, Paavo Parmas

The University of Tokyo, Japan

Policy Optimization RL Exploration Policy Gradient Methods Actor-Critic Methods Policy Learning

Open paper AskPDF arXiv

Retry-based objectives such as pass@K and max@K optimize the best return obtained from multiple sampled trajectories, and recent work has shown that they can promote exploration without explicit exploration bonuses. In discrete action spaces, ReMax was shown to do so by adapting to return uncertainty. In this work, we introduce pathwise derivative estimators for retry objectives and use them to extend ReMax to continuous action spaces. We study the resulting learning dynamics and show that, even with deterministic rewards, ReMax can encourage stochastic exploration by reshaping the policy-gradient landscape. In particular, it alters gradients both in direction, biasing updates toward higher policy entropy, and in magnitude, damping gradients and slowing convergence. We further show that Adam's adaptive normalization can mitigate this damping, depending on its numerical stabilization parameter. Empirically, we instantiate this objective as ReMax Actor-Critic (ReMAC), an off-policy actor--critic algorithm that optimizes the ReMax objective using a pathwise derivative estimator. Our experiments show that ReMAC can promote higher policy entropy without entropy regularization and achieves performance comparable to SAC.

Momentum

Related topics

Latest papers 140