Uncertainty-Aware Exploration

Latest papers

23 papers6 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Uncertainty-Driven Self-Play for Robust Multi-Agent Driving Policies
Oct 14, 2026Sajad Marvi, Julian Schmidt, Alessandro Canevaro +2Uncertainty-Aware Exploration Autonomous Driving Planning
Mercedes-Benz AG, Germany. · Department of Computer Science, University of Freiburg, Germany.
What Information Buys in Active Inference
Oct 13, 2026Melih Kandemir, Mustafa Mert ÇelikokRL Exploration Active Inference
Department of Mathematics and Computer Science, University of Southern Denmark
Understanding Hidden Spatial Preferences in Bayesian Optimization
Oct 10, 2026Maria Bånkestad, Jens SjölundBayesian Optimization Uncertainty-Aware Exploration
Deep Epistemic Value Functions for Optimistic Exploration
Sep 28, 2026Leander Diaz-Bone, Marco Bagatella, Jonas Hübotter +1Value Function Estimation RL Exploration
ETH Zürich, Switzerland · Max Planck Institute for Intelligent Systems, Germany
ICMAPE: In-Context Multiagent Pure Exploration
Sep 27, 2026Xinyi Hu, Alessio Russo, Aldo PacchianoRL Exploration Decentralized MARL
Boston University · Broad Institute of MIT and Harvard
Future Information-Directed Sampling for Bayesian Nonstationary Bandits
Sep 27, 2026Yichen Song, Alessio Russo, Aldo PacchianoMulti-Armed Bandits Non-Stationary Bandits
Boston University · Broad Institute of MIT and Harvard
Where World Models Break: Natural-Input Failure Discovery
Aug 23, 2026Zhanpeng Shi, Zi Liang, Rong Feng +3Adversarial Attacks on World Models Uncertainty-Aware Exploration
School of Computer Science, Northwestern Polytechnical University · The Hong Kong Polytechnic University · Department of Data Science, City University of Hong Kong +3
Variance Driven Exploration: A Provable and Efficient Methodology for Pure Exploration in Highly Stochastic Environments
Aug 22, 2026Khang Luong, Nam Nguyen, Hoang Ta +2Uncertainty-Aware Exploration RL Exploration
Hanoi University of Science and Technology, Hanoi, Vietnam · Quantum AI & Cyber Security Institute, FPT Corporation, Vietnam.
Information-Based Exploration via Random Features for Reinforcement Learning
Jul 20, 2026Waris Radji, Odalric-Ambrym MaillardRL Exploration Uncertainty-Aware Exploration
Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189-CRIStAL, F-59000 Lille, France
UBP2: Uncertainty-Balanced Preference Planning for Efficient Preference-based Reinforcement Learning
Jun 17, 2026Mohamed Nabail, Leo Kaixuan Cheng, Jingmin Wang +1Preference-Based RL Model-Based RL
Learning, Embodied Autonomy, and Forecasting (LEAF) Lab, University of Toronto
TreeSeeker: Tree-Structured Trial, Error, and Return in Deep Search
Jun 10, 2026Zhuofan Shi, Mingzhe Ma, Lu Wang +8Inference-Time Search Exploration-Exploitation Tradeoff
1Microsoft · 2East China Normal University
SHAPO: Sharpness-Aware Policy Optimization for Safe Exploration
Jun 8, 2026Kaustubh Mani, Yann Pequignot, Vincent Mai +1Sharpness-Aware Minimization RL Exploration
Universit´e de Montr´eal · Mila - Qu´ebec AI Institute · Universit´e Laval +2
Uncertainty-Aware and Temporally Regulated Expert Advice in Reinforcement Learning for Autonomous Driving
May 28, 2026Ahmed Abouelazm, Felix Klingebiel, Philip Schörner +1Reinforcement Learning Risk-Sensitive RL
FZI Research Center for Information Technology, Germany · Karlsruhe Institute of Technology, Germany
Not all uncertainty is alike: volatility, stochasticity, and exploration
May 19, 2026Payam PirayMulti-Armed Bandits Exploration-Exploitation Tradeoff
Department of Psychology, University of Southern California
ScreenSearch: Uncertainty-Aware OS Exploration
May 15, 2026Michael Solodko, Justin WagleComputer-Use Agents Monte Carlo Tree Search
Microsoft
Delightful Exploration
May 13, 2026Ian OsbandMulti-Armed Bandits Value of Information
Google DeepMind
Learning What Matters: Adaptive Information-Theoretic Objectives for Robot Exploration
May 12, 2026Youwei Yu, Jionghao Wang, Zhengming Yu +2Robotic Data Collection Robot Navigation
Luddy School of Informatics, Computing, and Engineering, Indiana University, Bloomington, IN, USA · Department of Computer Science & Engineering, Texas A&M University, College Station, TX, USA
T $^2$ PO: Uncertainty-Guided Exploration Control for Stable Multi-Turn Agentic Reinforcement Learning
May 4, 2026Haixin Wang, Hejie Cui, Chenwei Zhang +7RL for LLM Agents Reinforcement Learning
University of California, Los Angeles · 2Amazon.com Inc.
Uncertainty Guided Exploratory Trajectory Optimization for Sampling-Based Model Predictive Control
Apr 13, 2026O. Goktug Poyrazoglu, Yukang Cao, Rahul Moorthy +1Uncertainty-Aware Exploration Trajectory Optimization
University of Minnesota, Minneapolis, MN 55455, USA. · 2The University of Texas at Austin, Austin, TX 78712, USA.
Long-Horizon Plan Execution in Large Tool Spaces through Entropy-Guided Branching
Apr 13, 2026Rongzhe Wei, Ge Shi, Min Cheng +5Uncertainty-Aware Exploration LLM Agent Planning
Georgia Institute of Technology · Amazon
Decoupling Exploration and Policy Optimization: Uncertainty Guided Tree Search for Hard Exploration
Mar 23, 2026Zakaria Mhammedi, James CohanUncertainty-Aware Exploration RL Exploration
Google Research, NYC
Telogenesis: Goal Is All U Need
Mar 10, 2026Zhuoran Deng, Yizhi Zhang, Ziyi Zhang +1Uncertainty-Aware Exploration Attention Mechanisms
Independent Research
Active Exploration via Autoregressive Generation of Missing Data
May 29, 2024Tiffany Tianhui Cai, Hongseok Namkoong, Daniel Russo +1RL Exploration Autoregressive Generation
Department of Statistics Columbia University New York, NY 10025, USA · Decision, Risk, and Operations Division Columbia Business School New York, NY 10025, USA · Department of Mathematics Imperial College London London SW7 2AZ, United Kingdom

Oct 14, 2026J/K move · Enter open · S save

Uncertainty-Driven Self-Play for Robust Multi-Agent Driving Policies

Sajad Marvi, Julian Schmidt, Alessandro Canevaro, Julian Jordan+1

Mercedes-Benz AG, Germany. · Department of Computer Science, University of Freiburg, Germany.

Uncertainty-Aware Exploration Autonomous Driving Planning RL for Autonomous Driving Self-Play RL Multi-Agent Reinforcement Learning Autonomous Driving Scenario Generation+1

Open paper AskPDF arXiv

Learning robust driving policies that generalize beyond their training domain remains a central challenge in motion planning. We propose an uncertainty-driven self-play scheme, in which half of the agents are rewarded for increasing their neighbors' epistemic uncertainty, while the other half learn to drive safely as the exploring agents continuously seek out their weaknesses. Epistemic uncertainty indicates where the policy is least confident and serves as an automatic proxy for scenario difficulty. Exploring agents use this signal to seek out hard cases, generating a challenging curriculum without manual scenario design. The resulting policy outperforms standard self-play, achieving best-in-class zero-shot transfer to nuplan Val14 (99.12% vs. 97.97%) and Argoverse 2 (82.39% vs. 78.03%) without fine-tuning, while also improving performance over standard self-play on Waymo. Our diversity-driven approach achieves 85% higher LSTM hidden-state coverage than the baseline, indicating a much richer set of behaviors. Further analysis shows that epistemic uncertainty serves as a calibrated difficulty detector, with highest-uncertain agents exhibiting 14.8 times the combined failure rate across collision, off-road, and DNF, of lowest-uncertain agents, enabling automatic prioritization of challenging scenarios without ground-truth labels.