Linear Contextual Bandits

Follow this topic Ask about this topic

Linear contextual bandits study how to choose actions using observed context features when rewards are unknown and feedback arrives only for selected actions. Methods such as LinUCB and optimism-based elimination, Thompson sampling, and adversarial online learning estimate linear reward models while trading off exploration and cumulative regret. Current work extends these methods to non-stationary, noisy, corrupted, resource-constrained, and offline-to-online settings, while also reducing computation and communication costs. A central tension is how to gain stronger guarantees or exploit structure without relying on restrictive assumptions about context coverage, drift, noise, or action availability.

Latest papers

33 papers8 papers in the last four weeks, up 100% on the four weeks before. 0.0% of all new papers.

Explore Once, Evaluate Many: Multiple Policy Evaluation for Stochastic Linear Bandits
Oct 15, 2026Spyros Dragazis, Yilei Chen, Aldo PacchianoPolicy Evaluation Linear Contextual Bandits
Boston University · Broad Institute of MIT and Harvard
High-Probability Regret Bounds for Feel-Good Thompson Sampling
Oct 14, 2026Xuheng Li, Qingyue Zhao, Quanquan GuThompson Sampling Contextual Bandits
Department of Computer Science, University of California, Los Angeles, CA 90095, USA
Accelerating Linear Bandits with Persistent Candidate Sets
Oct 9, 2026Bingkui Tong, Daria Korovaitceva, Marina Sheshukova +2Multi-Armed Bandits Linear Contextual Bandits
Mohamed bin Zayed University of Artificial Intelligence · HSE University · RIKEN AIP
Gaussian-Width Bounds for Offline-to-Online Linear Bandits
Oct 9, 2026Kushagra Chandak, Volodymyr Tkachuk, Xiaoqi Tan +1Linear Contextual Bandits
Department of Computing Science, University of Alberta, Canada
Sampling Allocation of LinUCB: Optimal Design Limits in the Small-Gap Regime
Oct 5, 2026Yujie Liu, Vincent Y. F. Tan, Yunbei XuAdaptive Inference Contextual Bandits
National University of Singapore
Bandits via Additive Quantized Representations
Oct 1, 2026Ami Tavory, Noam Touitou, Tal Sarig +2Contextual Bandits Residual VQ
Meta Platforms
Pointwise or Pairwise: When Do Pairwise Losses Help Reward Learning, Provably?
Sep 29, 2026Junghyun Lee, Minsoo Ha, Sanghwa Kim +4Offline-to-Online Contextual Bandits Contextual Bandits
Kim Jaechul Graduate School of AI, KAIST Seoul 02455, Republic of Korea · Graduate School of AI, POSTECH Pohang 37673, Republic of Korea · Samsung Research Seoul 06765, Republic of Korea +1
Corruption-Robust Sparse Linear Contextual Bandits with Knapsack Constraints
Sep 29, 2026Yige Wang, Hanyang Li, Yiming Zong +2Contextual Bandits Linear Contextual Bandits
Department of Industrial Engineering and Decision Analytics, The Hong Kong University of Science and Technology · Department of Statistics and Data Science, The Wharton School, University of Pennsylvania
Nearly Minimax-Optimal Regret for Linear Contextual Bandits with Arbitrary Adaptive Action Sets
Sep 14, 2026Tianyuan JinMulti-Armed Bandits Minimax Regret
Data Science and Analytics Thrust The Hong Kong University of Science and Technology (Guangzhou)
Meta-LinEXP3: Online-within-Online Learning for Adversarial Linear Contextual Bandits
Sep 9, 2026Hao Li, Jie Xu, Zheng XieMulti-Armed Bandits Contextual Bandits
College of Science, National University of Defense Technology, Changsha 410073, China
Learning to Price and Stock Under Contextual and Censored Demand
Sep 5, 2026Zean Han, Zezhen Ding, Jiheng ZhangDynamic Pricing Supply Chain Management
The Hong Kong University of Science and Technology
Online Learning with LLM Experts from Limited Feedback
Sep 5, 2026Wang Wei, Soumyabrata Pal, Koyel Mukherjee +4LLM Routing Multi-Armed Bandits
Virginia Tech · Adobe Research
Safety by Design: Realized-Cost Constraints for Contextual Bandits with Continuous Actions
Aug 27, 2026Spyros Dragazis, Aldo PacchianoContextual Bandits Multi-Armed Bandits
Boston University · Broad Institute of MIT and Harvard
Generalized Linear Bandits with Memory
Aug 16, 2026Heesang Ann, Hyunjun Choi, Taehyun Hwang +3Generalized Linear Bandits Non-Stationary Bandits
Seoul National University, Seoul, Republic of Korea · Shinsegae, Seoul, Republic of Korea
Reoptimization Algorithms for Contextual Bandits with Knapsack Constraints
Aug 11, 2026Zhen XuContextual Bandits UCB Algorithms
Management School, University of Liverpool, Liverpool, United Kingdom
Progressive Content Refinement with Decaying Reward Joint LinUCB
Aug 7, 2026Shion Ishikawa, Pablo Loyola, Young-joo Chung +1LLM Self-Refinement Large Language Model-Guided Optimization
Rakuten Group, Inc. Tokyo, Japan · Rakuten Group, Inc. San Mateo, USA
Dynamic Regret for Non-Stationary Linear Bandits via Misspecification Reductions
Jul 3, 2026Zihao Hu, Yuan Yao, Jiheng Zhang +1Dynamic Regret Minimization Non-Stationary Bandits
Department of Mathematics, The Hong Kong University of Science and Technology
Graph Dimensionality Reduction for Contextual Bandits: Structure-Specific Regret Bounds under Approximate Smoothness and Noisy Eigenspaces
Jun 26, 2026Joyanta Jyoti Mondal, Ibne Farabi Shihab, Anuj SharmaMulti-Armed Bandits Graph Bandits
Department of Computer and Information Sciences, University of Delaware, USA · Department of Computer Science, Iowa State University, USA · Department of Civil, Construction & Environmental Engineering, Iowa State University, USA
Flow-Corrected Thompson Sampling for Non-Stationary Contextual Bandits
Jun 22, 2026AmirHossein Naghdi, Ali BaheriThompson Sampling Non-Stationary Bandits
Sharif University of Technology, Iran · Rochester Institute of Technology, USA
Set Membership Estimation for Stochastic Linear Contextual Bandits
Jun 18, 2026Haonan Xu, Yingying LiMulti-Armed Bandits Contextual Bandits
The Grainger College of Engineering University of Illinois Urbana-Champaign Champaign, IL, 61801
Bandits for Efficient Experimentation: Adapting to Control Group, Preferences, and Context Drifts
Jun 8, 2026Udvas Das, Waris Radji, Debabrota Basu +1Multi-Armed Bandits Non-Stationary Bandits
Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 – CRIStAL F-59000 Lille, France
Practical and Optimal Algorithm for Linear Contextual Bandits with Rare Parameter Updates
May 31, 2026Sanghoon Yu, Min-hwan OhBatched Bandits Contextual Bandits
Seoul National University, Seoul, Republic of Korea
Active Learning for Stochastic Contextual Linear Bandits
May 24, 2026Emma Brunskill, Ishani Karmarkar, Zhaoqi LiActive Learning Adaptive Sampling
Stanford University
Catching a Moving Subspace: Low-Rank Bandits Beyond Stationarity
May 18, 2026Hamed Khosravi, Xiaoming HuoMulti-Armed Bandits Non-Stationary Bandits
H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology Atlanta, GA 30332
Scaling Federated Linear Contextual Bandits via Sketching
May 1, 2026Hantao Yang, Hong Xie, Xutong Liu +1Communication-Efficient Distributed Training Contextual Bandits
University of Science and Technology of China · University of Washington Tacoma
Direction-Aware Offline-to-Online Learning in Linear Contextual Bandits
Apr 27, 2026Zean Han, Ruihan Lin, Zezhen Ding +1Offline-to-Online Contextual Bandits Contextual Bandits
Department of Industrial Engineering and Decision Analytics The Hong Kong University of Science and Technology
Calibration-Gated LLM Pseudo-Observations for Online Contextual Bandits
Apr 16, 2026Maksim Pershin, Ivan Golovanov, Pavel Baltabaev +1Contextual Bandits Linear Contextual Bandits
A Direct Approach for Handling Contextual Bandits with Latent State Dynamics
Apr 9, 2026Zhen Li, Gilles StoltzContextual Bandits Linear Contextual Bandits
LMO, CELESTE, HEC Paris · BNP Paribas Corporate and Institutional Banking, Paris, France · Université Paris-Saclay, CNRS, Inria, Laboratoire de mathématiques d’Orsay, 91405, Orsay, France +1
Simple Projection-Free Algorithm for Contextual Recommendation with Logarithmic Regret and Robustness
Mar 21, 2026Shinsaku SakaueContextual Bandits Recommender Systems
Center for Advanced Intelligence Project, RIKEN, Tokyo, Japan
Robust and Computationally Efficient Linear Contextual Bandits under Adversarial Corruption and Heavy-Tailed Noise
Mar 16, 2026Naoto Tani, Futoshi FutamiContextual Bandits Linear Contextual Bandits
The University of Osaka · RIKEN AIP · The University of Tokyo

Oct 15, 2026J/K move · Enter open · S save

Explore Once, Evaluate Many: Multiple Policy Evaluation for Stochastic Linear Bandits

Spyros Dragazis, Yilei Chen, Aldo Pacchiano

Boston University · Broad Institute of MIT and Harvard

Policy Evaluation Linear Contextual Bandits Off-Policy Evaluation

Open paper AskPDF arXiv

We study how to evaluate

K

target policies simultaneously in stochastic contextual bandits with linear reward structure. Given a known

d

-dimensional feature map and reward-free access to policy queries, we propose PRISMA (Policy evaluation through Regularized Information Sharing and Mixture Allocation), an algorithm that estimates every target policy's value to accuracy

\varepsilon

with probability at least

1-δ

. The algorithm uses

\widetilde{\mathcal{O}}\!\left( \frac{(L^2S^2+β^2)(1\veeρ^\star)}{\varepsilon^2} \cdot \log(K) \right)

context draws and reward observations, where

ρ^\star\le d

is an instance-dependent measure of the difficulty of designing a shared sampling distribution. Here,

L

S

, and

β

bound the feature norm, reward parameter norm, and reward noise, respectively, and the suppressed factors are logarithmic in

K

d

, and

1/δ

. Our method combines reward-free planning, a regularized sampling design, and a single ridge regression estimator shared across all target policies. An adaptive doubling procedure selects the regularization scale without knowledge of

ρ^\star

. The guarantees require no numerical lower bound on second-moment eigenvalues and have no explicit dependence on the cardinalities of the context and action spaces. Policy-query costs are accounted for separately, with

K

queries per shared planning context.