Multi-Armed Bandits

Multi-armed bandits model repeated choices among actions whose rewards are initially unknown, asking how to learn while avoiding too many poor choices. Classic methods such as upper confidence bounds, Thompson sampling, and successive elimination balance exploration and reward; newer work extends them to contextual, linear, and kernel models, as well as combinatorial choices and changing environments. Researchers also study best-arm identification, resource and fairness constraints, delayed or partial feedback, and robust guarantees under heavy-tailed noise or adversarial changes. A central challenge is adapting to structure and practical limits such as memory, computation, and feedback without relying on assumptions that fail in real settings or sacrificing reliable regret and inference guarantees.