Contextual Bandits

Follow this topic Ask about this topic

Contextual bandits study how to choose an action using information about the current situation when feedback reveals the reward only for the action taken. Common approaches include linear and generalized linear models with UCB or Thompson sampling, alongside kernel, neural, and policy-based methods for richer reward patterns. Offline learning and evaluation, nonstationary or corrupted feedback, and constraints such as budgets, privacy, and safety extend these methods to settings including recommendation, pricing, and LLM routing. A central challenge is balancing exploration with reliable decisions when models are misspecified, feedback is sparse or delayed, and computational or operational costs matter.

Latest papers

152 papers20 papers in the last four weeks, up 150% on the four weeks before. 0.1% of all new papers.

A Context Augmented Multi-Play Multi-Armed Bandit Algorithm for Fast Channel Allocation in Opportunistic Spectrum Access
May 25, 2026Ruiyu Li, Guangxia Li, Xiao Lu +2Multi-Armed Bandits Robust Multi-Armed Bandits
School of Computer Science and Technology, Xidian University, China · 2Research and Development, Ericsson, Canada · 3Hainayun IoT Technology Ltd, China
Active Learning for Stochastic Contextual Linear Bandits
May 24, 2026Emma Brunskill, Ishani Karmarkar, Zhaoqi LiActive Learning Adaptive Sampling
Stanford University
Human-in-the-Loop Contextual Bandits for Short-Term Rental Dynamic Pricing: Structural Equivalence of Historical Warm-Up and Approval-Gated Live Learning
May 22, 2026Oleg MiroshnichenkoContextual Bandits Dynamic Pricing
Active Context Selection Improves Simple Regret in Contextual Bandits
May 19, 2026Mohammad Shahverdikondori, Jalal Etesami, Negar KiyavashMulti-Armed Bandits Best-Arm Identification
College of Management of Technology, EPFL · Department of Computer Science, TU Munich
Catching a Moving Subspace: Low-Rank Bandits Beyond Stationarity
May 18, 2026Hamed Khosravi, Xiaoming HuoMulti-Armed Bandits Non-Stationary Bandits
H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, GA 30332
Offline Contextual Bandits in the Presence of New Actions
May 18, 2026Ren Kishimoto, Tatsuhiro Shimizu, Kazuki Kawamura +6Offline-to-Online Contextual Bandits Off-Policy Evaluation
Institute of Science Tokyo · Yale University · Sony Group Corporation +1
Don't Let Bandit Feedback Pull Continual LLM-Recommender Updates Off Target
May 17, 2026Taesan Kim, Hyeongjun Yun, Jaegul Choo +1Large Language Model-Based Recommendation Off-Policy RL
1SK Telecom · 2KAIST
Pessimistic Risk-Aware Policy Learning in Contextual Bandits
May 15, 2026Yilong Wan, Yuqiang Li, Xianyi WuOffline-to-Online Contextual Bandits Risk-Sensitive RL
Latency-Quality Routing for Functionally Equivalent Tools in LLM Agents
May 14, 2026Kexin Chu, Dawei Xiang, Wei ZhangTool-Augmented Language Model Agents LLM Tool Use
University of Connecticut
Contextual Bandits for Resource-Constrained Devices using Probabilistic Learning
May 13, 2026Marco Angioli, Kevin Johansson, Antonello Rosato +2Hyperdimensional Computing Contextual Bandits
Sapienza University of Rome · Orebro University · Linkoping University +1
Robust Sequential Experimental Design for A/B Testing
May 13, 2026Qianglin Wen, Xiangkun Wu, Chengchun Shi +4Causal Effect Estimation Model Misspecification
Yunnan Key Laboratory of Statistical Modeling and Data Analysis, Yunnan University, Kunming, China · School of Mathematical Sciences, Zhejiang University, Hangzhou, China · Department of Statistics, London School of Economics and Political Science, London, United Kingdom +3
Multi-Task Representation Learning for Conservative Linear Bandits
May 12, 2026Jiabin Lin, Shana MoothedathMulti-Armed Bandits Multi-Task Learning
Department of Computer Science and Technology, Qingdao University, China · Department of Electrical and Computer Engineering, Iowa State University, USA
Signature Approach for Contextual Bandits with Nonlinear and Path-dependent Rewards
May 11, 2026Xin Guo, Grace He, Xinyu LiContextual Bandits UCB Algorithms
University of California, Berkeley, Berkeley, CA · University of Oxford, Oxford, UK
Learning to Sparsify Stochastic Linear Bandits
May 11, 2026Zhengmiao Wang, Ming Chi, Zhi-Wei Liu +2Multi-Armed Bandits Exploration-Exploitation Tradeoff
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology, Wuhan 430074, China · Department of Electronics and Telecommunications, Politecnico di Torino, 10129 Torino, Italy
PFN-TS: Thompson Sampling for Contextual Bandits via Prior-Data Fitted Networks
May 11, 2026Yan Shuo Tan, Kenyon Ng, Ruizhe Deng +3Thompson Sampling Prior-Data Fitted Networks
National University of Singapore · Monash University · Renmin University of China +1
Optimal Regret for Single Index Bandits
May 10, 2026Devdan Dey, Sujoy Bhore, Avishek GhoshMulti-Armed Bandits Single-Index Models
Department of Computer Science and Engineering, Indian Institute of Technology Bombay.
Fast Rates for Offline Contextual Bandits with Forward-KL Regularization under Single-Policy Concentrability
May 9, 2026Qingyue Zhao, Kaixuan Ji, Heyang Zhao +1Offline-to-Online Contextual Bandits KL-Regularized RL
Department of Computer Science, University of California, Los Angeles, CA 90095, USA
Constrained Contextual Bandits with Adversarial Contexts
May 7, 2026Dhruv Sarkar, Abhishek SinhaContextual Bandits Adversarial Bandits
Indian Institute of Technology Kharagpur, India · Tata Institute of Fundamental Research, Mumbai, India
Optimal Contextual Pricing under Agnostic Non-Lipschitz Demand
May 7, 2026Jianyu Xu, Yu-Xiang WangContextual Bandits UCB Algorithms
Carnegie Mellon University Pittsburgh, PA 15213
Robust Linear Dueling Bandits with Post-serving Context under Unknown Delays and Adversarial Corruptions
May 3, 2026Youngmin OhMulti-Armed Bandits Bandits with Delayed Feedback
InfiniTree, Republic of Korea
Scaling Federated Linear Contextual Bandits via Sketching
May 1, 2026Hantao Yang, Hong Xie, Xutong Liu +1Communication-Efficient Distributed Training Contextual Bandits
University of Science and Technology of China · University of Washington Tacoma
Learning When to Remember: Risk-Sensitive Contextual Bandits for Abstention-Aware Memory Retrieval in LLM-Based Coding Agents
Apr 30, 2026Mehmet IscanSelective Prediction AI Coding Agents
PythaLab, Yildiz Technical University, Istanbul, Türkiye.
The Bandit's Blind Spot: The Critical Role of User State Representation in Recommender Systems
Apr 29, 2026Pedro R. Pires, Gregorio F. Azevedo, Rafael T. Sereicikas +2User Modeling Recommender Systems
Federal University of São Carlos Sorocaba, São Paulo, Brazil
Budget-Constrained Causal Bandits: Bridging Uplift Modeling and Sequential Decision-Making
Apr 28, 2026Abhirami PillaiMulti-Armed Bandits Causal Effect Estimation
utdallas
Direction-Aware Offline-to-Online Learning in Linear Contextual Bandits
Apr 27, 2026Zean Han, Ruihan Lin, Zezhen Ding +1Offline-to-Online Contextual Bandits Contextual Bandits
Department of Industrial Engineering and Decision Analytics The Hong Kong University of Science and Technology
Logistic Bandits with $\tilde{O}(\sqrt{dT})$ Regret without Context Diversity Assumptions
Apr 24, 2026Seoungbin Bae, Dabeen LeeMulti-Armed Bandits Contextual Bandits
Department of Industrial & Systems Engineering, KAIST · Department of Mathematical Sciences, Seoul National University
Replicable Bandits with UCB based Exploration
Apr 21, 2026Rohan Deb, Udaya Ghai, Karan Singh +1Multi-Armed Bandits Contextual Bandits
University of Illinois Urbana-Champaign · Amazon · Carnegie Mellon University
POLAR: Online Learning for LoRA Adapter Caching and Routing in Edge LLM Serving
Apr 17, 2026Shaoang Li, Jian LiLLM Serving Low-Rank Adaptation
Stony Brook University
Calibration-Gated LLM Pseudo-Observations for Online Contextual Bandits
Apr 16, 2026Maksim Pershin, Ivan Golovanov, Pavel Baltabaev +1Contextual Bandits Linear Contextual Bandits
A Direct Approach for Handling Contextual Bandits with Latent State Dynamics
Apr 9, 2026Zhen Li, Gilles StoltzContextual Bandits Linear Contextual Bandits
LMO, CELESTE, HEC Paris · BNP Paribas Corporate and Institutional Banking, Paris, France · Université Paris-Saclay, CNRS, Inria, Laboratoire de mathématiques d’Orsay, 91405, Orsay, France +1

May 25, 2026J/K move · Enter open · S save

A Context Augmented Multi-Play Multi-Armed Bandit Algorithm for Fast Channel Allocation in Opportunistic Spectrum Access

Ruiyu Li, Guangxia Li, Xiao Lu, Jichao Liu+1

School of Computer Science and Technology, Xidian University, China · 2Research and Development, Ericsson, Canada · 3Hainayun IoT Technology Ltd, China

Multi-Armed Bandits Robust Multi-Armed Bandits Wireless Resource Allocation Contextual Bandits

Open paper AskPDF arXiv

We study the restless contextual multi-play multi-armed bandit (MP-MAB) problem for channel allocation in the opportunity spectrum access (OSA) scenario. Most existing MP-MAB methods are impractical for real-world OSA systems as they assume many ideal conditions, incur a heavy computational cost, and most importantly, ignore the impact of channel noise which is directly related to the quality of service. In this study, we embody this impact by modeling channel noise as a perturbation of the arm's reward function in MP-MAB. As there is an implicit correlation between channel state information and channel noise, we take the former as a context for MP-MAB to present the perturbation caused by the latter. We investigate two types of correlation between the context and the perturbation -- linear and nonlinear, and derive two index policies, respectively. These policies learn the correlations through a linear model and a neural network, and use estimated noise value to adjust the upper confidence bound. Numerical experiments demonstrate that the proposed policies can achieve lower regret and select sub-optimal arms in a more reasonable way.