Contextual Bandits

Follow this topic Ask about this topic

Contextual bandits study how to choose an action using information about the current situation when feedback reveals the reward only for the action taken. Common approaches include linear and generalized linear models with UCB or Thompson sampling, alongside kernel, neural, and policy-based methods for richer reward patterns. Offline learning and evaluation, nonstationary or corrupted feedback, and constraints such as budgets, privacy, and safety extend these methods to settings including recommendation, pricing, and LLM routing. A central challenge is balancing exploration with reliable decisions when models are misspecified, feedback is sparse or delayed, and computational or operational costs matter.

Latest papers

152 papers20 papers in the last four weeks, up 150% on the four weeks before. 0.1% of all new papers.

Safety by Design: Realized-Cost Constraints for Contextual Bandits with Continuous Actions
Aug 27, 2026Spyros Dragazis, Aldo PacchianoContextual Bandits Multi-Armed Bandits
Boston University · Broad Institute of MIT and Harvard
Gated Decoupled Compositional Bandits: A Unified Theory of Contextual Bandits with Supervised-Calibrated Action Scaling and Pre-Execution Gating
Aug 22, 2026Oleg MiroshnichenkoMulti-Armed Bandits Contextual Bandits
Learning What to Fail On: Failure-Mode Contextual Bandits for Adversarial Data Curation
Aug 19, 2026Roie Kazoom, Ofir Cohen, Rami Puzis +2Adversarial Robustness Training Data Curation
Electrical and Computers Engineering, Ben Gurion University of The Negev · Software and Information Systems Engineering, Ben Gurion University of The Negev
Fast A/B/n Testing: Exact Multi-Policy Comparison via Tree-Coupled Feedback Sharing
Aug 13, 2026Yuxiao WenPolicy Evaluation A/b Testing
Courant Institute of Mathematical Sciences, New York University
When Offline Evaluation Misleads: A Diagnostic Protocol for Reward and Policy Selection in Delayed-Feedback Contextual Bandits
Aug 12, 2026Sang Su Lee, Vineeth Loganathan, Shishir Dash +1Multi-Armed Bandits Off-Policy Evaluation
Thumbtack, Inc. San Francisco, CA, USA
Reoptimization Algorithms for Contextual Bandits with Knapsack Constraints
Aug 11, 2026Zhen XuContextual Bandits UCB Algorithms
Management School, University of Liverpool, Liverpool, United Kingdom
Offline Policy Evaluation as a decision support tool for designing Adaptive Experiments
Aug 10, 2026João Victor Ferreira Alves, Eduardo Rocha Laurentino, Gustavo de Oliveira Kanno +1Contextual Bandits A/b Testing
Instituto de Ciência e Tecnologia Itaú (ICTi), Brasil
Progressive Content Refinement with Decaying Reward Joint LinUCB
Aug 7, 2026Shion Ishikawa, Pablo Loyola, Young-joo Chung +1LLM Self-Refinement Large Language Model-Guided Optimization
Rakuten Group, Inc. Tokyo, Japan · Rakuten Group, Inc. San Mateo, USA
Bootstrap-Conditioned Action Selection with Tabular Foundation Models
Aug 6, 2026Devansh Gupta, Shiv Tavker, Dmitry Efimov +3Tabular Foundation Models RL Exploration
Amazon Science
Minimax-Optimal Semiparametric Contextual Dynamic Pricing with Multimodal Revenue
Aug 4, 2026Xueping Gong, Zhuoluo Zhang, Zhaowei Miao +1Contextual Bandits Dynamic Pricing
School of Management, Xiamen University · Department of Industrial Engineering and Decision Analytics, The Hong Kong University of Science and Technology
402Pilot: An x402 Decision Layer for Autonomous Agent Micropayments
Aug 2, 2026Yin Li, Yanbo He, Boo-Ho Yang +5Multi-Armed Bandits Non-Stationary Bandits
The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China · MOVENSYS Inc., Seongnam-si, Republic of Korea · Schneider Electric, Shanghai, China +2
Personalizing Large Language Model Agents with Small Policy Models
Jul 31, 2026Dian Jin, Zhi Zhang, Huichao Li +3LLM Personalization LLM Agents
Department of Statistics and Data Science, National University of Singapore · Department of Statistics and Data Science, University of California, Los Angeles · Department of Data Science, Fudan University
On-Policy and Off-Policy Learning for Large Action Spaces
Jul 30, 2026Imad AoualiThompson Sampling Multi-Armed Bandits
Institut Polytechnique de Paris · Ecole nationale de la statistique et de l’administration économique · Ecole doctorale n◦574 Ecole Doctorale de Mathématique Hadamard (EDMH)
Kairos: Numerically Robust News Recommendation under Item Cold-Start via Cholesky-based LinUCB
Jul 29, 2026Finn HertschCold-Start Recommendation Recommender Systems
DHBW Ravensburg, School of Business, Data Science and AI, Marienplatz 2, 88212 Ravensburg, Germany
WISERouter: LLM Routing with Workload Budget Constraint
Jul 26, 2026Yifei Li, Zihui Gao, Laks V. S. LakshmananLLM Routing Contextual Bandits
The University of British Columbia
Cross-Domain Off-Policy Evaluation and Learning for Contextual Bandits
Jul 24, 2026Yuta Natsubori, Masataka Ushiku, Yuta SaitoOff-Policy Evaluation Contextual Bandits
Hakuhodo DY Holdings, Inc. · Cornell University
A Predict-then-Correct Loop Based on Few-Shot Continuous Contextual Bandit for Demand Forecasting
Jul 17, 2026Zhiwei Lei, Benedict Jun Ma, Ilya JacksonTime Series Forecasting Contextual Bandits
Thrust of Intelligent Transportation, Hong Kong University of Science and Technology (Guangzhou), China · Center for Transportation and Logistics, Massachusetts Institute of Technology, USA
Information-Directed Sampling for Causal Bandits
Jul 17, 2026Muhammad Qasim Elahi, Murat Kocaoglu, Mahsa GhasemiThompson Sampling Multi-Armed Bandits
School of Electrical and Computer Engineering, Purdue University · School of Computer Science, Johns Hopkins University
Kernel weighted importance sampling for off-policy evaluation in contextual bandits
Jul 16, 2026Joshua Spear, Matthieu Komorowski, Rebecca Pope +2Off-Policy Evaluation Importance Sampling
Institute of Child Health, University College London · Strive Health Ltd. · National Institute for Health Research +3
Optimal and Efficient Contextual Combinatorial Semi-bandits with General Function Approximation
Jul 15, 2026Hao Qin, Chicheng ZhangCombinatorial Semi-Bandits Contextual Bandits
The University of Arizona
Diversified Multinomial Logit Contextual Bandits
Jul 13, 2026Heesang Ann, Taehyun Hwang, Min-hwan OhMulti-Armed Bandits Discrete Choice Modeling
Seoul National University
Correlation-Aware Contextual Bandits with Surrogate Rewards for LLM Routing
Jul 10, 2026Ajay Narayanan Sridhar, Ronak Singh, Mehrdad Mahdavi +1Multi-Armed Bandits LLM Routing
The Pennsylvania State University
Stochastic Linear Bandits with Partially Observed Actions
Jul 9, 2026Gautam Dasarathy, Vineet Gattani, Lalit JainMulti-Armed Bandits Contextual Bandits
Arizona State University · GE Vernova · Google
Nonlinear Bandit
Jul 8, 2026Tianshuo Zheng, Ting Wu, Zhi-Hua Zhou +1Multi-Armed Bandits Heavy-Tailed Noise
School of Mathematics, Nanjing University, Nanjing, 210093, China · School of Artificial Intelligence, Nanjing University, National Key Laboratory for Novel Software Technology, Nanjing, 210023, China · School of Mathematics and Physics, Xi’an Jiaotong-Liverpool University, Suzhou, 215123, China
Contextual Procurement Auctions with Bandit Learning
Jul 7, 2026Yiling Chen, Shi Feng, Sadie ZhaoMechanism Design Regret Minimization
Harvard University
Dynamic Regret for Non-Stationary Linear Bandits via Misspecification Reductions
Jul 3, 2026Zihao Hu, Yuan Yao, Jiheng Zhang +1Dynamic Regret Minimization Non-Stationary Bandits
Department of Mathematics, The Hong Kong University of Science and Technology
A Contextual-Bandit Oversight Game with Two-Sided Informational Asymmetry
Jun 30, 2026Yunjin TongReinforcement Learning Imperfect-Information Games
Stanford Graduate School of Business
Contextual Slate GLM Bandits with Limited Adaptivity
Jun 30, 2026Tanmay Goyal, Sukruta Prakash Midigeshi, Gaurav SinhaMulti-Armed Bandits Contextual Bandits
Microsoft Research India
Smart charging of large fleets of Electric Vehicles: Independent Multi-Agent Reinforcement Learning approaches
Jun 30, 2026Xavier Rate, Eloann Le Guern, Raphaël Féraud +8Multi-Agent Coordination Electric Vehicle Charging
Orange · ENS Rennes · IMT Atlantique +1
Learning to Bid in Discriminatory Auctions with Budget Constraints
Jun 28, 2026Negin Golrezaei, Sourav SahooGame Theory Automated Bidding
Sloan School of Management, Massachusetts Institute of Technology · Operations Research Center, Massachusetts Institute of Technology

Aug 27, 2026J/K move · Enter open · S save

Safety by Design: Realized-Cost Constraints for Contextual Bandits with Continuous Actions

Spyros Dragazis, Aldo Pacchiano

Boston University · Broad Institute of MIT and Harvard

Contextual Bandits Multi-Armed Bandits UCB Algorithms Linear Contextual Bandits

Open paper AskPDF arXiv

Contextual bandits are a standard framework for sequential decision-making under uncertainty, with applications in clinical trials, dosage selection, recommendation systems, and autonomous systems. Safety is central in many of these applications, since a single unsafe decision in settings such as dosage selection or autonomous driving can have catastrophic consequences. A common way to model safety in bandit problems is to associate each action with both a reward signal and a cost signal, and to optimize reward subject to constraints on cost. Most existing safety-constrained bandit models enforce safety by requiring the expected cost of each action to remain below a prescribed threshold. However, this may be insufficient in heteroscedastic settings, where the chosen action affects not only the expected reward and cost, but also the variability of the observed outcomes. We study contextual bandits with one-dimensional continuous actions and stage-wise high-probability constraints on the realized cost. We propose High-Probability Constrained UCB, an optimistic-pessimistic algorithm that explores for reward while conservatively estimating the safe action set. For linear reward and cost models, we prove a tight

\tilde{\mathcal{O}}(d\sqrt{T})

regret bound, and we extend the analysis to general function classes using the eluder dimension. Experiments show that enforcing realized-cost safety substantially reduces violations compared with expected-cost constrained baselines.