Bayesian Regret

Bayesian regret quantifies the difference in cumulative reward between an optimal strategy and a learning algorithm using Bayesian methods, focusing on minimizing this regret over time. Current research emphasizes efficient exploration strategies within various frameworks, including linear bandits, reinforcement learning (RL), and contextual bandits, often employing algorithms like Thompson Sampling and posterior sampling to balance exploration and exploitation. These advancements yield improved regret bounds and enhanced performance in diverse applications such as revenue management, robotics, and online decision-making, contributing to a deeper understanding of optimal learning under uncertainty. The resulting theoretical and algorithmic improvements have significant implications for the design of efficient and robust learning systems across numerous fields.

Papers

September 10, 2024

Modified Meta-Thompson Sampling for Linear Bandits and Its Bayes Regret Analysis
Hao Li, Dong Liang, Zheng Xie
Meta Learning Linear Bandit Thompson Sampling Linear Contextual Bandit Contextual Multi Armed Bandit Bayesian Regret

May 29, 2024

May 8, 2024

Learning with Posterior Sampling for Revenue Management under Time-varying Demand
Kazuma Shimizu, Junya Honda, Shinji Ito, Shinji Nakadai
LeArning Abstract Posterior Sampling Linear Programming Bayesian Regret Time Varying Demand Revenue Management Demand Distribution

March 17, 2024

Prior-dependent analysis of posterior sampling reinforcement learning with function approximation
Yingru Li, Zhi-Quan Luo
Reinforcement Learning Function Approximation Model Based Prior Bayesian Regret Linear Mixture Markov Decision Process Prior Dependent

March 5, 2024

Chained Information-Theoretic bounds and Tight Regret Rate for Linear Bandit Problems
Amaury Gouverneur, Borja Rodríguez-Gálvez, Tobias J. Oechtering, Mikael Skoglund
Information Theoretic Thompson Sampling Bandit Problem Bayesian Regret Finite Time Regret

January 21, 2024

Thompson Sampling for Stochastic Bandits with Noisy Contexts: An Information-Theoretic Regret Analysis
Sharu Theresa Jose, Shana Moothedath
Thompson Sampling Noisy Environment Stochastic Bandit Contextual Linear Bandit Bayesian Regret Dependent Regret

December 20, 2023

Bayesian Analysis of Combinatorial Gaussian Process Bandits
Jack Sandberg, Niklas Åkerblom, Morteza Haghir Chehreghani
Application Proficiency Theoretical Understanding Contextual Bandit Bayesian Regret Gaussian Process Bandit Bayesian Setting

October 16, 2023

Posterior Sampling-based Online Learning for Episodic POMDPs
Dengwang Tang, Dongze Ye, Rahul Jain, Ashutosh Nayyar, Pierluigi Nuzzo
Multi Agent C POMDPs Episodic Markov Decision Process Posterior Mode Bayesian Regret

October 11, 2023

Exploiting Causal Graph Priors with Posterior Sampling for Reinforcement Learning
Mirco Mutti, Riccardo De Santi, Marcello Restelli, Alexander Marx, Giorgia Ramponi
Reinforcement Learning Causal Graph Posterior Sampling Bayesian Regret Causal System

September 27, 2023

Provably Efficient Exploration in Constrained Reinforcement Learning:Posterior Sampling Is All You Need
Danil Provodin, Pratik Gajane, Mykola Pechenizkiy, Maurits Kaptein
Efficient Exploration Posterior Sampling Constrained Reinforcement Learning Constrained Markov Decision Process Bayesian Regret

September 20, 2023

STARNet: Sensor Trustworthiness and Anomaly Recognition via Approximated Likelihood Regret for Robust Edge Autonomy
Nastaran Darabi, Sina Tayebati, Sureshkumar S., Sathya Ravi, Theja Tulabandhula, Amit R. Trivedi
Data Stream Bayesian Regret Sensor Fault Edge Robotics General Anomaly Enhancement Network

June 2, 2023

Bayesian Regret Minimization in Offline Bandits
Mohammad Ghavamzadeh, Marek Petrik, Guy Tennenholtz
Regret Minimization Upper Bound Bayesian Regret Offline Contextual Bandit Conic Optimization

May 27, 2023

Hierarchical Deep Counterfactual Regret Minimization
Jiayu Chen, Tian Lan, Vaneet Aggarwal
Imperfect Information Game Bayesian Regret Counterfactual Regret Minimization

March 28, 2023

qEUBO: A Decision-Theoretic Acquisition Function for Preferential Bayesian Optimization
Raul Astudillo, Zhiyuan Jerry Lin, Eytan Bakshy, Peter I. Frazier
Bayes Rule Bayesian Regret Bayes Optimal Policy Preferential Bayesian Optimization

March 6, 2023

Thompson Sampling for Linear Bandit Problems with Normal-Gamma Priors
Björn Lindenberg, Karl-Olof Lindahl
Thompson Sampling Bayesian Regret Bayesian Formulation

January 28, 2023

STEERING: Stein Information Directed Exploration for Model-Based Reinforcement Learning
Souradip Chakraborty, Amrit Singh Bedi, Alec Koppel, Mengdi Wang, Furong Huang, Dinesh Manocha
Reinforcement Learning Model Based Reinforcement Learning Steering Technique Bayesian Regret Directed Exploration Incentivized Exploration

November 29, 2022

Posterior Sampling for Continuing Environments
Wanqiao Xu, Shi Dong, Benjamin Van Roy
Reinforcement Learning Posterior Inference Posterior Sampling Continuous Environment Bayesian Regret Agent Environment Interaction

October 30, 2022

On Rate-Distortion Theory in Capacity-Limited Cognition & Reinforcement Learning
Dilip Arumugam, Mark K. Ho, Noah D. Goodman, Benjamin Van Roy
Reinforcement Learning Rate Distortion Decision Making Agent Bayesian Regret Client Availability

May 30, 2022

Mixed-Effect Thompson Sampling
Imad Aouali, Branislav Kveton, Sumeet Katariya
Online Learning Contextual Bandit Thompson Sampling Mixed Effect Model Bayesian Regret

Bayesian Regret

Papers

Modified Meta-Thompson Sampling for Linear Bandits and Its Bayes Regret Analysis

Efficient Exploration in Average-Reward Constrained Reinforcement Learning: Achieving Near-Optimal Regret With Posterior Sampling

Approximate Thompson Sampling for Learning Linear Quadratic Regulators with $O(\sqrt{T})$ Regret

Learning with Posterior Sampling for Revenue Management under Time-varying Demand

Prior-dependent analysis of posterior sampling reinforcement learning with function approximation

Chained Information-Theoretic bounds and Tight Regret Rate for Linear Bandit Problems

Thompson Sampling for Stochastic Bandits with Noisy Contexts: An Information-Theoretic Regret Analysis

Bayesian Analysis of Combinatorial Gaussian Process Bandits

Posterior Sampling-based Online Learning for Episodic POMDPs

Exploiting Causal Graph Priors with Posterior Sampling for Reinforcement Learning

Provably Efficient Exploration in Constrained Reinforcement Learning:Posterior Sampling Is All You Need

STARNet: Sensor Trustworthiness and Anomaly Recognition via Approximated Likelihood Regret for Robust Edge Autonomy

Bayesian Regret Minimization in Offline Bandits

Hierarchical Deep Counterfactual Regret Minimization

qEUBO: A Decision-Theoretic Acquisition Function for Preferential Bayesian Optimization

Thompson Sampling for Linear Bandit Problems with Normal-Gamma Priors

STEERING: Stein Information Directed Exploration for Model-Based Reinforcement Learning

Posterior Sampling for Continuing Environments

On Rate-Distortion Theory in Capacity-Limited Cognition & Reinforcement Learning

Mixed-Effect Thompson Sampling