cs.LGJul 29, 2026

Parameter-Free Dynamic Regret under Heavy-Tailed Noise

Authors: Vaneet Aggarwal

Abstract

We study online convex optimization with one unbiased stochastic subgradient per round and noise having a finite pp-th central moment, where p∈(1,2]p\in(1,2] is unknown. For a bounded convex domain of diameter DD, subgradients bounded by GG, noise scale σσ, and comparator path length PTP_T, let ΛT=1+PT/DΛ_T=1+P_T/D. A single algorithm, using none of G,σ,p,PTG,σ,p,P_T, attains expected dynamic regret Op(min⁡{GDTΛT+σDT1/pΛT(p−1)/p, GDT})O_p\left(\min\{GD\sqrt{TΛ_T}+σDT^{1/p}Λ_T^{(p-1)/p},\,GDT\}\right) against every fixed comparator sequence. Restarted AdaGrad experts produce the noise-path exponent (p−1)/p(p-1)/p, and a prior favoring longer restart intervals removes horizon-dependent logarithmic overhead. We give an explicit bound uniform in pp; its logarithm-free form has noise coefficient O(1+log⁡(p/(p−1)))O(1+\log(p/(p-1))), while the static-regret constant is universal. The analysis requires only marginal noise moments and permits dependent errors. Complete pathwise proofs retain both the expert-loss range and the gradient energies preceding comparator movement. Matching lower bounds hold on every bounded convex domain of positive diameter, under the same gradient-only information model. Together with a path-budget-tuned upper bound, they characterize the minimax rate with universal constants, including its linear-regret saturation.

Explore similar work

CardsList