Online Mirror Descent

Online Mirror Descent (OMD) studies how to make a sequence of decisions using feedback as it arrives while keeping cumulative loss close to that of a fixed comparator. It uses a mirror map and Bregman divergence to choose update geometry: Euclidean geometry gives projected gradient descent, while entropy yields exponentiated-gradient methods, closely related to Follow-the-Regularized-Leader. OMD is used in online convex optimization, bandits with generalized linear or heavy-tailed rewards, delayed and nonstationary feedback, reinforcement learning policy updates, and decentralized games. Regret guarantees are well developed, but choosing geometry and adapting to unknown noise, drift, or delay remains challenging; in reinforcement learning, convergence also depends on critic accuracy and policy-update stability.