Bayesian RL

RL: Reinforcement Learning

Bayesian reinforcement learning studies how agents can learn and act when rewards, dynamics, or task goals are uncertain. Common approaches include posterior sampling such as Thompson sampling, Bayes-adaptive planning over beliefs, and Gaussian-process or variational models that use uncertainty to direct exploration. Methods are applied to tabular and linear Markov decision processes, partially observed tasks, continuous control, robotics, and offline learning, with regret guarantees and evaluations on control and manipulation benchmarks. A central tension is balancing informative exploration and safe, robust behavior while keeping inference tractable and avoiding misleading beliefs when the model is misspecified.

Latest papers

34 papers8 papers in the last four weeks, up 167% on the four weeks before. 0.0% of all new papers.

Approximate Universal Prediction under Interaction
Oct 24, 2026Richard Cornelius SuwandiBayesian RL
School of Artificial Intelligence The Chinese University of Hong Kong, Shenzhen
Reinforcement Learning with Multi-Step Transition Predictions: A Bellman Theory
Oct 23, 2026Chenbei Lu, Zaiwei Chen, Tongxin Li +2Reinforcement Learning Model-Based RL
Duffield College of Engineering, Cornell University · Edwardson School of Industrial Engineering, Purdue University · School of Data Science, The Chinese University of Hong Kong (Shenzhen) +2
Is In-Context Reinforcement Learning A Posterior Sampler? An Empirical Study
Oct 20, 2026Weiqin Chen, Xinlong Du, Gesualdo Scutari +1In-Context RL Transformer-Based RL
Edwardson School of Industrial Engineering, Purdue University West Lafayette, IN 47907, USA
Bayesian Neural Networks via Dirichlet Process Priors on the Data-Generating Distribution for Randomized Value Functions in Deep Reinforcement Learning
Oct 20, 2026Sumit VashishthaBayesian RL Uncertainty Estimation in RL
Independent Researcher
Towards instance-dependent regret optimality in Episodic MDPs with Posterior Sampling
Oct 19, 2026Victor Boone, Cyrille Kone, Waris Radji +2Regret Minimization in RL Markov Decision Processes
IRIT, Université de Toulouse, CNRS, Toulouse INP, Toulouse, France · LAAS-CNRS, Université de Toulouse, CNRS, Toulouse, France · Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189-CRIStAL, F-59000 Lille, France +1
Alliance of Stochastic Routing and Reinforcement Fusion for Multi-Model Reasoning
Oct 16, 2026Selim Furkan Tekin, Ling LiuMultimodal Model Routing Adaptive Model Routing
Georgia Institute of Technology
What Information Buys in Active Inference
Oct 13, 2026Melih Kandemir, Mustafa Mert ÇelikokRL Exploration Active Inference
Department of Mathematics and Computer Science, University of Southern Denmark
MaPP: A Unified Marginalized Posterior-Predictive Framework for Data-Efficient RLVR
Sep 28, 2026Yangyang Ren, Haodong Zhu, Sheng Xu +4Prompt Selection for RL Bayesian RL
Beihang University · Zhongguancun Academy · Communication University of China +4
Minimax-Optimality of Posterior Sampling for Reinforcement Learning
Sep 27, 2026Taewon Goo, Kihyuk HongRegret Minimization in RL Markov Decision Processes
KAIST
Efficient Bayes-Adaptive Reinforcement Learning with Temporal Logic Specifications
Sep 17, 2026Jonathan Hau, Alessandro AbateBayesian RL Safe RL
Department of Computer Science, University of Oxford, United Kingdom
Subspace Inference Enables Efficient Active Reward Learning from Preferences
Sep 3, 2026Yutai Zhou, Erdem BıyıkBayesian RL Active Preference Learning
Thomas Lord Department of Computer Science University of Southern California, Los Angeles, USA
Quantifying Risk Under Evolving Uncertainty: Belief-Dependent Robustness for Safe Sequential Decision Making
Aug 18, 2026Deep Kumar Ganguly, Jan KretinskyRisk-Sensitive Planning Robust RL
Technical University of Munich, Germany · Masaryk University, Brno, Czech Republic
Q-based Variational Inverse Reinforcement Learning
Aug 17, 2026Ondrej Bajgar, Peter Tisnikar, Alessandro Abate +2Reinforcement Learning Bayesian RL
University of Oxford · Artificial Intelligence Center, Czech Technical University in Prague · Villanova University
Full Bayesian Reinforcement Learning via LF-IBIS
Jul 2, 2026Stefano Masini, Cecilia Viscardi, Michela BacciniSimulation-Based Inference Reinforcement Learning
Department of Computer Science, University of Pisa, Italy · Department of Economics and Statistics, University of Salerno, Italy · Department of Statistics, Computer Science, Applications ”G.Parenti”, University of Florence, Italy
Agentic Monte Carlo: Simulating Reinforcement Learning for Black-Box Agents
Jun 3, 2026Dae Yon Hwang, Raunaq Suri, Valentin Villecroze +4Reinforcement Learning Bayesian RL
Layer 6 AI, Toronto, Canada.
Bayesian learning for the stochastic shortest path problem
Jun 3, 2026Chon Wai Ho, Sumeetpal S. Singh, Jiaqi GuoMarkov Decision Processes Reinforcement Learning
Department of Engineering, University of Cambridge, UK · School of Mathematics and Physics, University of Wollongong, Wollongong, Australia
Regularized Offline Policy Optimization with Posterior Hybrid Bayesian Belief
May 30, 2026Hongqiang Lin, Pengfei Wang, Nenggan ZhengReinforcement Learning Bayesian RL
Zhejiang University, Hangzhou, China.
Information-Directed Offline-to-Online Reinforcement Learning
May 28, 2026Keru ChenRegret Minimization in RL Bayesian RL
School of Electrical, Computer and Energy Engineering, Arizona State University
Evolving Robustness--Exploration Trade-off in Online Reinforcement Learning via Quantile Bayesian Risk MDPs
May 23, 2026Meichen Song, Yuhao Wang, Enlu ZhouReinforcement Learning Bayesian RL
School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, GA 30332, USA
BAPR: Bayesian amnesic piecewise-robust reinforcement learning for non-stationary continuous control
May 15, 2026Yifan Zhang, Liang Zheng, Qifan ZhangRL Control Non-Stationary RL
Central South University
Bayesian policy gradient and actor-critic algorithms
Apr 30, 2026Mohammad Ghavamzadeh, Yaakov Engel, Michal ValkoBayesian RL Temporal-Difference Learning
Adobe Research & Inria · Rafael Advanced Defence System, Israel · Inria Lille — SequeL team, France
Golden Handcuffs make safer AI agents
Apr 15, 2026Aram Ebtekar, Michael K. CohenAI Agent Safety Safe RL
University of California, Berkeley
Integrating Deep RL and Bayesian Inference for ObjectNav in Mobile Robotics
Mar 26, 2026João Castelo-Branco, José Santos-Victor, Alexandre BernardinoObject-Goal Navigation Robot Navigation
Instituto Superior Técnico, Lisbon, Portugal · Institute For Systems and Robotics, Lisbon, Portugal
Bayesian Learning in Episodic Zero-Sum Games
Mar 21, 2026Chang-Wei Yueh, Andy Zhao, Ashutosh Nayyar +1Zero-Sum Games Game-Theoretic RL
Department of Electrical and Computer Engineering at the University of Southern California
Posterior Sampling Reinforcement Learning with Gaussian Processes for Continuous Control: Sublinear Regret Bounds for Unbounded State Spaces
Mar 9, 2026Hamish Flynn, Joe Watson, Ingmar Posner +1RL Control Regret Minimization in RL
Carnegie Mellon University · University of Oxford · Technical University of Darmstadt +2
A Model-Free Universal AI
Feb 26, 2026Yegon Kim, Juho LeeReinforcement Learning Bayesian RL
Graduate School of AI, KAIST, Seoul, South Korea
VariBASed: Variational Bayes-Adaptive Sequential Monte-Carlo Planning for Deep Reinforcement Learning
Feb 21, 2026Joery A. de Vries, Jinke He, Yaniv Oren +3Bayesian RL Sample-Efficient RL
Trent AI Ltd, London, United Kingdom · EEMCS, Delft University of Technology, Delft, the Netherlands
Dynamic Decision-Making under Model Misspecification: A Stochastic Stability Approach
Feb 19, 2026Xinyu Dai, Daniel Chen, Yian QianThompson Sampling Model Misspecification
Department of Economics, Brown University. · Department of Applied Mathematics, Brown University.
MAVRL: Learning Reward Functions from Multiple Feedback Types with Amortized Variational Inference
Feb 16, 2026Raphaël Baur, Yannick Metz, Maria Gkoulta +3Bayesian RL
ETH AI Center, ETH Zurich, Zurich, Switzerland · Department of Computer Science, ETH Zurich, Zurich, Switzerland · Department of Informatics, University of Zurich, Zurich, Switzerland
Meta-RL with Bayesian Linear Task Models
Dec 24, 2025Jingyang You, Hanna KurniawatiRepresentation Learning Reinforcement Learning
School of Computing Australian National University

Oct 24, 2026J/K move · Enter open · S save

Approximate Universal Prediction under Interaction

Richard Cornelius Suwandi

School of Artificial Intelligence The Chinese University of Hong Kong, Shenzhen

Bayesian RL

Open paper AskPDF arXiv

Bayesian mixtures, and Solomonoff induction in particular, provide an idealized model of prediction learned by minimizing population log loss, and they underlie universal agents such as AIXI. An agent that plans with an approximation of such a predictor decides where that predictor is evaluated, and a planner can be drawn toward branches on which its model is optimistic. We study which approximation guarantees survive this feedback. For normalized finite-horizon causal mixtures, the transfer coefficient factors into a policy likelihood ratio and task evidence available before each prediction, so task rarity and action-branch rarity can compound. Within one fixed universal mixture, infinitely many computable environments admit excess prediction error of order

κη\,2^{K(μ)}

, where

η

is the training KL budget and

κ

measures policy coverage. This order is sharp, and for each target a computable predictor attains it at a suitable finite horizon, so computability alone does not remove the penalty. For bounded returns, value error is controlled by the probability of entering a poorly covered branch plus a square-root term in

η

. Both terms are necessary, and finite maximum coverage ratios are not required. Motivated by the mentor override of Golden Handcuffs, we derive margins for preserving deferral decisions and conditional KL budgets with an unavoidable inverse-history-probability penalty, and a counterexample separates agreement of deferral decisions from task-wise event avoidance. The guarantees apply to any predictor architecture that meets explicit population approximation and coverage conditions, which gives concrete targets for implementing idealized universal agents.