cs.LGAug 15, 2026

Does 1/2-Tsallis-INF Also Work Well for Best-Arm Identification?

Authors: Jingxin Zhan, Yuze Han, Zhihua Zhang

Organizations: School of Mathematical Sciences, Peking University · Center for Applied Statistics and School of Statistics, Renmin University of China

Abstract

Regret minimization (RM) and best-arm identification (BAI) are two fundamental objectives in multi-armed bandits. Among regret-minimizing algorithms, 1/21/2-Tsallis-INF is a canonical best-of-both-worlds FTRL algorithm: it achieves logarithmic pseudo-regret in stochastic bandits while retaining minimax-optimal regret in adversarial bandits, without knowing the environment in advance. This raises a natural question: can the same algorithm, without additional exploration, also identify the best arm reliably? We study this question in stochastic bandits by analyzing the failure probability Err⁡t\operatorname{Err}_t, defined as the probability that the empirical best arm determined by the cumulative importance-weighted loss estimates of 1/2-Tsallis-INF differs from the true optimal arm. The main difficulty is that, at the logarithmic-regret scale, suboptimal arms are sampled with probability heuristically of order 1/t1/t. Consequently, importance weighting causes the cumulative estimator to fluctuate on the same linear scale as its mean separation. To overcome this obstacle, guided by a diffusion toy model, we construct a Lyapunov function for the gap process between the estimated cumulative loss of the optimal arm and that of the best competing arm. This leads to polynomial upper bounds on Err⁡t\operatorname{Err}_t: for learning rate ηt=α/tη_t=α/\sqrt t, Err⁡t\operatorname{Err}_t decays at rate t−2+α2μi∗/4+ρt^{-2+α^2μ_{i_*}/4+ρ} for any ρ>0ρ>0, where μi∗μ_{i_*} denotes the mean loss of the true optimal arm. We also establish a lower bound Ω(t−2−ε)Ω(t^{-2-\varepsilon}) for any ε>0\varepsilon>0, showing that the exponent 22 is essentially tight.

Explore similar work

Mar 11, 2026

On The Complexity of Best-Arm Identification in Non-Stationary Linear Bandits

We study the fixed-budget best-arm identification (BAI) problem in non-stationary linear bandits. Concretely, given a fixed time budget T∈NT\in \mathbb{N}, finite arm set X⊂Rd\mathcal{X} \subset \mathbb{R}^d, and a potentially adversarial sequence of unknown parameters {θt}t=1T\lbrace θ_t\rbrace_{t=1}^{T} (hence non-stationary), a learner aims to identify the arm with the largest cumulative reward x∗=arg⁡max⁡x∈Xx⊤∑t=1Tθtx_* = \arg\max_{x \in \mathcal{X}} x^\top\sum_{t=1}^T θ_t with high probability. In this setting, it is well-known that uniformly sampling arms from the G-optimal design yields a minimax-optimal error probability of exp⁡(−Θ(T/HG))\exp\left(-Θ\left(T / H_{G}\right)\right), where HGH_{G} scales proportionally with the dimension dd. However, this notion of complexity is overly pessimistic, as it is derived from a lower bound in which the arm set consists only of the standard basis vectors, thus masking any potential advantages arising from arm sets with richer geometric structure. To address this, we establish an arm-set-dependent lower bound that, in contrast, holds for any arm set. Motivated by the ideas underlying our lower bound, we propose the Adjacent-optimal design, a specialization of the well-known XY\mathcal{X}\mathcal{Y}-optimal design, and develop the Adjacent-BAI\textsf{Adjacent-BAI} algorithm. We prove that the error probability of Adjacent-BAI\textsf{Adjacent-BAI} matches our lower bound up to constants, verifying the tightness of our lower bound, and establishing the arm-set-dependent complexity of this setting.
May 1, 2026

Trading off rewards and errors in multi-armed bandits

In multi-armed bandits, the most-explored arms are the most informative, while reward maximization typically pulls only the best arm. We study the tradeoff between identifying arm means accurately and accumulating reward, and present an algorithm with regret guarantees that interpolates between the two objectives. We provide both upper and lower bounds and validate empirically.
Apr 16, 2026

Best of both worlds: Stochastic & adversarial best-arm identification

We study bandit best-arm identification with arbitrary and potentially adversarial rewards. A simple random uniform learner obtains the optimal rate of error in the adversarial scenario. However, this type of strategy is suboptimal when the rewards are sampled stochastically. Therefore, we ask: Can we design a learner that performs optimally in both the stochastic and adversarial problems while not being aware of the nature of the rewards? First, we show that designing such a learner is impossible in general. In particular, to be robust to adversarial rewards, we can only guarantee optimal rates of error on a subset of the stochastic problems. We give a lower bound that characterizes the optimal rate in stochastic problems if the strategy is constrained to be robust to adversarial rewards. Finally, we design a simple parameter-free algorithm and show that its probability of error matches (up to log factors) the lower bound in stochastic problems, and it is also robust to adversarial ones.