UCB Algorithms

UCB: Upper Confidence Bound

Overview

UCB algorithms choose actions by combining estimated rewards with confidence bonuses, balancing exploration of uncertain options against exploiting promising ones. The approach spans multi-armed and linear or contextual bandits, as well as Gaussian-process optimization, with variants for nonstationary rewards, constraints, delayed feedback, and multiple objectives. Analyses typically establish regret guarantees, while benchmarks test performance in settings such as pricing, ranking, resource allocation, and adaptive model selection. A central challenge is adapting confidence bounds to realistic noise and structure without making decisions too costly or compromising safety, fairness, or valid statistical inference.

Latest papers

CardsList

62 papers5 papers in the last four weeks, down 50% on the four weeks before. 0.0% of all new papers.

  1. UCB-min: Combining Bandits Under Variance Heterogeneity (Extended Version)

    Nov 9, 2026Lilia Izri, Benoit Groz, Silviu ManiuMulti-Armed BanditsUCB Algorithms

  2. Does LinUCB Need Valid Confidence Sets?

    Nov 3, 2026Ahmed Ben Yahmed, Marc Abeille, David Janz +1Linear Contextual BanditsUCB Algorithms

  3. Closing the Log Gap: Minimax-Optimal Regret for Episodic RL

    Oct 29, 2026Guy Zamir, Victor Boone, Matthew Zurek +1Episodic RLReinforcement Learning

  4. Cascading Bandits with Noisy Observations: Algorithms and Regret Bounds

    Oct 21, 2026Yashaswini Kaluguri, Nikhil KaramchandaniMulti-Armed BanditsBandit Learning

  5. Near-Optimal Regret Bounds for UCB-Style Gaussian Process Bandits via Data Thinning

    Oct 20, 2026Shion Takeno, Shogo Iwazaki, Ichiro TakeuchiGaussian Process BanditsRegret Minimization

  6. The Tilted Sampling Problem: Optimism, Posterior Sampling, and Logarithmic Regret

    Oct 15, 2026Mohammad Pedramfar, Siamak RavanbakhshPosterior SamplingThompson Sampling

  7. Transfer Learning in Bandits from Unknown Adaptive Policies: Problem-Dependent Lower and Upper Bounds

    Oct 15, 2026Adrien Prevost, Timothee Mathieu, Waris Radji +1Multi-Armed BanditsTransfer Learning

  8. Accelerating Linear Bandits with Persistent Candidate Sets

    Oct 9, 2026Bingkui Tong, Daria Korovaitceva, Marina Sheshukova +2Multi-Armed BanditsLinear Contextual Bandits

  9. Sampling Allocation of LinUCB: Optimal Design Limits in the Small-Gap Regime

    Oct 5, 2026Yujie Liu, Vincent Y. F. Tan, Yunbei XuAdaptive InferenceContextual Bandits

  10. Ranking Bandits for Carousel Interfaces with Observable Browsing Depth

    Oct 4, 2026Takuma Yasuda, Atsuyoshi Nakamura, Dai Hai NguyenThompson SamplingMulti-Armed Bandits

  11. Block Optimism for Nonstationary Bandits with Latent Linear Dynamics

    Oct 1, 2026Taehyun Hwang, Hyunjun Choi, Heesang Ann +1Multi-Armed BanditsStochastic Linear Dynamical Systems

  12. Optimal Nonparametric Dynamic Pricing with Censored Demand and Adversarial Inventory

    Sep 26, 2026Mengxiao Zhang, Yingfei Wang, Haipeng LuoDynamic PricingMinimax Regret

  13. Robust Bayesian Optimization with Q-Exponential Surrogates

    Sep 26, 2026Richard Cornelius Suwandi, Zhidi Lin, Feng Yin +1Bayesian OptimizationUCB Algorithms

  14. A Unified Optimism-Agnostic Framework for Linear Bandits over Spherical Action Sets

    Sep 26, 2026Arda Güçlü, Subhonmesh Bose, John R. BirgeMulti-Armed BanditsUCB Algorithms

  15. Safety by Design: Realized-Cost Constraints for Contextual Bandits with Continuous Actions

    Aug 27, 2026Spyros Dragazis, Aldo PacchianoContextual BanditsMulti-Armed Bandits

  16. Risk-Sensitive Reinforcement Learning with Smoothed Quantile Objectives

    Aug 23, 2026Mohammad Alipour-Vaezi, Huaiyang Zhong, Sajad KhodadadianRisk-Sensitive RLRegret Minimization in RL

  17. Sharper Regret Bounds for Time-Varying Gaussian Process Bandits with Constant Exploration

    Aug 19, 2026Matthias Mandl, Hanne KekkonenGaussian Process BanditsBayesian Optimization

  18. Toward the Optimal Regret-Instability Trade-off in Multi-Armed Bandits

    Aug 18, 2026Kaifei Wang, Yinyu Ye, Han ZhongMulti-Armed BanditsUCB Algorithms

  19. Reoptimization Algorithms for Contextual Bandits with Knapsack Constraints

    Aug 11, 2026Zhen XuContextual BanditsUCB Algorithms

  20. Sharp Characterization of Bias in Post-Bandit Inference

    Aug 2, 2026Lisu Wang, Yilun Chen, Jiaqi LuMulti-Armed BanditsAlgorithmic Bias

  21. Price of Fairness in Bandits: A Tight Minimax Characterization

    Jul 15, 2026Dhruv Sarkar, Soumyadeep Dutta, Sayak Ray ChowdhuryMulti-Armed BanditsMinimax Regret

  22. Randomized Exploration for Linear Bandits via Absolute Perturbations

    Jun 26, 2026Toshinori Kitamura, Shuai Liu, Csaba SzepesváriThompson SamplingMulti-Armed Bandits

  23. Learning in Markovian bandits with non-observable states and constrained decision epochs

    Jun 25, 2026Thomas Hira, Victor Boone, Urtzi Ayesta +1Multi-Armed BanditsMarkov Models

  24. Selective Ensemble Based on Preference-Directed Multi-Objective Bandits

    Jun 20, 2026Lanjihong Ma, Zhen-Yu Zhang, Masashi Sugiyama +1Multi-Armed BanditsEnsemble Learning

  25. Set Membership Estimation for Stochastic Linear Contextual Bandits

    Jun 18, 2026Haonan Xu, Yingying LiMulti-Armed BanditsContextual Bandits

  26. Efficient Multinomial Logistic Bandit via Frequent Directions

    Jun 10, 2026Linzhe He, Yu-Jie Zhang, Sifan Yang +1Multi-Armed BanditsContextual Bandits

  27. Bellman-sufficient Information Complexity

    Jun 9, 2026Yunbei XuMulti-Armed BanditsMinimax Regret

  28. Algorithm for Contextual Queueing Bandits with Rate-Optimal Queue Length Regret

    Jun 8, 2026Seoungbin Bae, Dabeen LeeExploration-Exploitation TradeoffRegret Minimization

  29. Multi-Armed Bandits with Arriving Arms: Sequential Screening, Dynamic Regret, and Sublinear Guarantees

    Jun 8, 2026Deqi Zheng, Xiaoyang Xu, Yuhong YangMulti-Armed BanditsDynamic Regret Minimization