Contextual Bandit Setting

Contextual bandit settings address the challenge of sequentially selecting actions in environments where rewards depend on both the chosen action and contextual information. Current research focuses on robustly learning optimal policies from potentially biased or incomplete offline data, employing techniques like offline policy learning, causal inference, and decision transformers, often with a focus on mitigating the effects of confounding variables and large action spaces. These advancements are significant for applications such as personalized medicine (e.g., optimizing Warfarin dosage) and online recommendation systems, where efficient and reliable decision-making under uncertainty is crucial. Furthermore, research explores how to effectively leverage collaborative learning and handle adversarial agents in these settings.

Papers

June 7, 2024

Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learning
Subhojyoti Mukherjee, Josiah P. Hanna, Qiaomin Xie, Robert Nowak
Multi Task Decision Transformer Near Optimal Policy Reward Prediction Structured Bandit Contextual Bandit Setting

February 16, 2024

Optimizing Warfarin Dosing Using Contextual Bandit: An Offline Policy Learning and Evaluation Method
Yong Huang, Charles A. Downs, Amir M. Rahmani
Reinforcement Learning Evaluation Method Based Policy Offline Policy Learning Contextual Bandit Setting Dosing Model Warfarin Dosing

December 20, 2023

Robustly Improving Bandit Algorithms with Confounded and Selection Biased Offline Data: A Causal Approach
Wen Huang, Xintao Wu
Bandit Algorithm Regret Analysis Causal Approach Systematic Bias Causal Constraint Offline Dataset Contextual Bandit Setting

August 7, 2023

Doubly Robust Estimator for Off-Policy Evaluation with Large Action Spaces
Tatsuhiro Shimizu, Laura Forastiere
Action Space Policy Evaluation Robust Estimation Benchmark Data Inverse Propensity Contextual Bandit Setting

July 24, 2023

Contextual Bandits and Imitation Learning via Preference-Based Active Queries
Ayush Sekhari, Karthik Sridharan, Wen Sun, Runzhe Wu
Imitation Learning Contextual Bandit Noisy Preference Contextual Bandit Setting

July 18, 2023

Online Learning with Costly Features in Non-stationary Environments
Saeed Ghoorchian, Evgenii Kortukov, Setareh Maghsudi
Long Term Online Learning Sublinear Regret Sequential Decision Making Problem Non Stationary Environment Effective Feature Dual Learning Contextual Bandit Setting

December 4, 2022

Counterfactual Learning with General Data-generating Policies
Yusuke Narita, Kyohei Okumura, Akihiro Shimizu, Kohei Yata
Policy Evaluation Diverse Policy Counterfactual Learning Bandit Model Contextual Bandit Setting

June 16, 2022

How to talk so AI will learn: Instructions, descriptions, and autonomy
Theodore R Sumers, Robert D Hawkins, Mark K Ho, Thomas L Griffiths, Dylan Hadfield-Menell
Artificial Intelligence Reward Function Human Instruction Legal Autonomy Description Library Traditional Reinforcement Learning Social Learning Contextual Bandit Setting Behavioural Experiment

June 6, 2022

Collaborative Linear Bandits with Adversarial Agents: Near-Optimal Regret Bounds
Aritra Mitra, Arman Adibi, George J. Pappas, Hamed Hassani
Optimal Regret Linear Bandit Adversarial Agent Contextual Bandit Setting Collaborative Bandit

January 30, 2022

Coordinated Attacks against Contextual Bandits: Fundamental Limits and Defense Mechanisms
Jeongyeol Kwon, Yonathan Efroni, Constantine Caramanis, Shie Mannor
Optimal Policy Contextual Bandit Adversarial Agent Fundamental Limit DEfense Mechanism Multiple Attack Contextual Bandit Setting

December 23, 2021

Model Selection in Batch Policy Optimization
Jonathan N. Lee, George Tucker, Ofir Nachum, Bo Dai
Human Feedback Policy OpTimization Model Selection Contextual Bandit Setting Batch Policy