Overview

Contextual bandits study how to choose an action using information about the current situation when feedback reveals the reward only for the action taken. Common approaches include linear and generalized linear models with UCB or Thompson sampling, alongside kernel, neural, and policy-based methods for richer reward patterns. Offline learning and evaluation, nonstationary or corrupted feedback, and constraints such as budgets, privacy, and safety extend these methods to settings including recommendation, pricing, and LLM routing. A central challenge is balancing exploration with reliable decisions when models are misspecified, feedback is sparse or delayed, and computational or operational costs matter.

Latest papers

CardsList

160 papers9 papers in the last four weeks, down 61% on the four weeks before. 0.1% of all new papers.

  1. Contextual Non-Convex Bandit Optimization with Two-Point Feedback

    Nov 13, 2026Khaled Eldowa, Pierre GaillardContextual BanditsMulti-Armed Bandits

  2. Near-Optimal Regret for Oracle-Efficient Adversarial Contextual Bandits

    Nov 12, 2026Kiarash Banihashem, MohammadTaghi HajiaghayiContextual BanditsAdversarial Bandits

  3. Action-Centered Causal Bandits for Budget-Constrained Promotion Targeting

    Nov 6, 2026Yifan Lin, Kaixu Yang, Di MoContextual BanditsMulti-Armed Bandits

  4. Multi-Turn Semantic Caching for Low-Cost LLM Serving

    Oct 31, 2026Baran Atalar, Selly Beyene, Xutong Liu +1Semantic CachingLLM Serving

  5. Optimal Learning in Fairness-Aware Contextual Bandits

    Oct 31, 2026Gengyu Xue, Fan Wang, Feiyu Jiang +2Algorithmic FairnessContextual Bandits

  6. Optimal Contextual Pricing in Non-Stationary Environments

    Oct 29, 2026Yiding Feng, Alireza RezaeimoghadamContextual Dynamic PricingDynamic Pricing

  7. Conservative Dueling Bandits: Preference Learning under Baseline Constraints

    Oct 27, 2026Rohan Deb, Mohammad Ghavamzadeh, Arindam BanerjeeDueling BanditsContextual Bandits

  8. Conversational Generalized Linear Bandits with Hybrid Feedback

    Oct 26, 2026Yurui Zou, Qirun Zeng, Jinhang ZuoBandit LearningContextual Bandits

  9. Learning Which Device to Call: Online Tool Grounding for LLM Agents in IoT

    Oct 18, 2026Yichen Liu, Xiaochong Jiang, Haoran Yu +1LLM Tool UseTool Selection for Language Model Agents

  10. Learning Personalized Memory Gating from Delayed One-Sided Feedback

    Oct 16, 2026Kaixuan Ji, Zixiang Chen, Wenting Zhao +9Memory Management for LLM AgentsContextual Bandits

  11. High-Probability Regret Bounds for Feel-Good Thompson Sampling

    Oct 14, 2026Xuheng Li, Qingyue Zhao, Quanquan GuThompson SamplingContextual Bandits

  12. Optimal Online Regret for Context-Lumpable Bandits

    Oct 13, 2026Chenyu Lu, Zihan ZhangContextual Bandits

  13. Online LLM Personalization from Pairwise Preferences Across Users

    Oct 12, 2026Romain Puech, Georgy Salakhutdinov, Dimitris BertsimasEducational QALLM Personalization

  14. Budgeted Multi-Source Counterfactual Annotation for Off-Policy Evaluation

    Oct 7, 2026Biao Xiang, Ali Eshragh, Yuexing Li +1Off-Policy EvaluationCounterfactual Evaluation

  15. Amortized Off-Policy Evaluation for LLMs

    Oct 7, 2026Younwoo Choi, Leo Feng, Vincent Liu +1Off-Policy EvaluationLLM Evaluation

  16. Variance-Optimal Off-Policy Evaluation with Conjunct Effect Modeling

    Oct 6, 2026Nicolò Felicioni, Michael Benigni, Maurizio Ferrari Dacrema +1Causal Effect EstimationOff-Policy Evaluation

  17. The Surrogate Is Not the Reward: Post-Surrogate Primary-Outcome Acquisition in Contextual Bandits

    Oct 5, 2026Kyungbok Lee, Michael R. KosorokContextual Bandits

  18. Sampling Allocation of LinUCB: Optimal Design Limits in the Small-Gap Regime

    Oct 5, 2026Yujie Liu, Vincent Y. F. Tan, Yunbei XuAdaptive InferenceContextual Bandits

  19. Learning While Scheduling Jobs under Context-Dependent Service Rates: An Anytime Rate-Optimal Algorithm

    Oct 5, 2026Seoungbin Bae, Dabeen LeeMulti-Armed BanditsContextual Bandits

  20. Bandits via Additive Quantized Representations

    Oct 1, 2026Ami Tavory, Noam Touitou, Tal Sarig +2Contextual BanditsResidual VQ

  21. Reserve-Aware Contrast Certificates for Conservative Bandits with Uncertain Baselines

    Sep 30, 2026Qinchuan ChengMulti-Armed BanditsContextual Bandits

  22. Risk-Aware Adaptive Evaluation: Finding High-Impact Failures Under Limited Budgets

    Sep 30, 2026Priyanath Maji, Spandan Ghose ChowdhuryThompson SamplingAI Agent Evaluation

  23. Nonpreemptive Scheduling While Learning Context-Dependent Service Rates

    Sep 29, 2026Wansoo Choi, Seoungbin Bae, Dabeen LeeQueueing TheoryContextual Bandits

  24. Pointwise or Pairwise: When Do Pairwise Losses Help Reward Learning, Provably?

    Sep 29, 2026Junghyun Lee, Minsoo Ha, Sanghwa Kim +4Offline-to-Online Contextual BanditsContextual Bandits

  25. Corruption-Robust Sparse Linear Contextual Bandits with Knapsack Constraints

    Sep 29, 2026Yige Wang, Hanyang Li, Yiming Zong +2Contextual BanditsLinear Contextual Bandits

  26. Elicitation and Decision Geometry in Single-Index Bandits

    Sep 28, 2026Sakshi Arya, Cheng Soon OngMulti-Armed BanditsSingle-Index Models

  27. Vanilla Policy Optimization Is Both Optimal and Differentially Private for Stochastic Contextual Bandits

    Sep 27, 2026Idan Attias, Orin Levy, Alexander Ryabchenko +2Policy OptimizationDifferential Privacy