Nearly Minimax Optimal Regret for Adversarial Linear Bandits with Strong Corruptions
Organizations: Department of Computer Science, University of California, Los Angeles, CA 90095, USA
Abstract
We study adversarial linear bandits with strong corruption, in which an adaptive adversary chooses the reward vectors and a strong adversary corrupts each observed reward after seeing the learner's action, subject to a known total corruption budget . Strong corruption breaks the importance-weighted reward estimates used by adversarial bandit algorithms: by corrupting only the rounds in which a rarely sampled action is drawn, the adversary amplifies each unit of corruption by the inverse of its sampling probability. The best known bound for this problem, by Liu et al. (2024), is a factor above the corruption-free rate in its leading term, and whether regret is achievable was left open. We propose CRW-FTRL, which runs follow-the-regularized-leader with a radius-weighted entropy regularizer and estimates rewards with a normalized kernel estimator whose sensitivity to corruption is governed by per-action radii instead of inverse sampling probabilities. The radii are nondecreasing and are chosen jointly with the sampling distribution through a monotone log-determinant barrier problem, so that a negative linear term in the regularizer cancels both the corruption bias and the estimation error of every comparator. For actions in , CRW-FTRL achieves regret with high probability against an adaptive adversary, which matches the best known corruption-free rate and the lower bound up to logarithmic factors. Applied to a discretization of a general compact action set, it yields regret, which settles the statistical rate of corrupted adversarial linear bandits under strong corruption.
Figures & tables
| Algorithm | Rewards | Measure | Regret |
| Robust Phased Elimination ( Bogunovic et al., 2021 ) | Stoch. | ||
| CW-OFUL ( He et al., 2022 ) | Stoch. | ||
| Randomized Phased Elimination ( Liu et al., 2024 ) | Stoch. | ||
| FTRL with log-det ( Liu et al., 2024 ) | Adv. | ||
| Continuous Exp. Weights ( Liu et al., 2024 ) | Adv. | ||
| CRW-FTRL (Algorithm 1 , ours), | Adv. |
Adversarial Bandits
Adversarial bandits study how to choose actions and learn from partial feedback when rewards or losses can change strategically over time.
See all 54 papers