math.PRSep 8, 2026

Gaussian Approximation for Multivariate Martingale Sums from Uniformly Ergodic Markov Chains

Authors: Yixuan ZhangQiaomin Xie

Abstract

We develop Gaussian approximation bounds in higher-order Wasserstein distance WpW_p, p2p\geq2, for sums of multivariate martingale differences generated by a uniformly ergodic Markov chain. Under an L(2+η)pL^{(2+η)p}-moment condition with η>0η>0, we establish the explicit bound O(p3A42+pd1/4A21/2A42)O\left( p^3 \|A\|_4^2 + pd^{1/4}\|A\|_2^{1/2}\|A\|_4^2 \right) where ARnA\in\mathbb{R}^n collects the L(2+η)pL^{(2+η)p}-sizes of the nn individual martingale increments. In the balanced-increment regime where the individual increments have comparable sizes of order n1/2n^{-1/2}, it yields the first optimal O(n1/2)O(n^{-1/2}) Gaussian approximation rate for fixed pp and dd. Consequently, we also obtain the first optimal O(n1/2)O(n^{-1/2}) WpW_p Gaussian approximation rate for multivariate additive functionals of uniformly ergodic Markov chains. Our analysis develops two techniques for addressing the interplay between higher-order Wasserstein distance and temporal dependence. First, building on the Ornstein--Uhlenbeck relative-score approach of Fang and Koike (2023), we formulate the bound in terms of antisymmetric Stein couplings while retaining the conditional tensor structure. Second, we develop a refresh-then-maximal coupling that combines an independent first-step resampling, which preserves the desired Stein identity, with a subsequent maximal coupling that provides effective control of the coupling increment. These tools may be useful more broadly for Gaussian approximation under temporal dependence.

Explore similar work

Feb 14, 2025cs.LG

Nonasymptotic CLT and Error Bounds for Linear Two-Time-Scale Stochastic Approximation

We consider linear two-time-scale stochastic approximation algorithms driven by martingale noise. Recent applications in machine learning motivate the need to understand finite-time error rates, but conventional stochastic approximation analyses focus on either asymptotic convergence in distribution or finite-time bounds that are far from optimal. Prior work on asymptotic central limit theorems (CLTs) suggests that two-time-scale algorithms may be able to achieve 1/K1/\sqrt{K} error in expectation, with a constant given by the expected norm of the limiting Gaussian vector. However, the best known finite-time rates are much slower. We derive the first nonasymptotic Wasserstein-1 CLT for linear two-time-scale stochastic approximation with Polyak-Ruppert averaging driven by martingale difference noise. As a corollary, we show that the expected error achieved by Polyak-Ruppert averaging decays at rate 1/K1/\sqrt{K}, which significantly improves on the rates of convergence in prior works.
Seo Taek Kong, Sihan Zeng, Thinh T. Doan +1
May 20, 2026math.PR

Concentration of General Stochastic Approximation Under Heavy-Tailed Markovian Noise

We establish maximal concentration bounds for the iterates generated by stochastic approximation algorithms with general step sizes, where the noise has a finite-state Markovian component plus a Martingale-difference component. When the Martingale-difference noise is bounded, we show that the tail of the error can be sub-Gaussian, sub-Weibull, or something lighter than any Pareto but heavier than any Weibull, depending on the step size sequence and on whether the random operator is almost surely contractive, almost surely non-expansive, or expansive with positive probability. Our analysis relies on a novel Lyapunov function involving the moment-generating function of the solution to a Poisson equation, together with an auxiliary projected algorithm. We complement the upper bounds with worst-case examples showing that qualitatively sharper bounds are impossible. We further study the case of unbounded Martingale-difference noise when the average operator is contractive, and the step sizes are of order 1/k1/k. In this setting, we show that if the random operator is almost surely non-expansive, then the error tail is at most three times heavier than the noise tail, whereas if the random operator is expansive with positive probability, then the error may have substantially heavier tails. These results are obtained through a novel black-box truncation argument that reduces the unbounded-noise setting to the bounded-noise case.
Shubhada Agrawal, Siva Theja Maguluri, Martin Zubeldia
Feb 15, 2026cs.LG

Constant-Stepsize Stochastic Approximation: Finite-Time Convergence, Gaussian Approximation, and Tail Bounds

Constant-stepsize stochastic approximation (SA) is widely used in learning for computational efficiency, yet the distribution of the iterates is typically intractable. Classical asymptotics results give Xk(α)X(α)x+αYX_k^{(α)} \approx X^{(α)} \approx x^\star+\sqrtαY, where X(α)X^{(α)} is the steady state and YY is an appropriate Gaussian limit, by progressively taking the time kk\uparrow\infty and stepsize α0α\downarrow0. Such limit results, however, do not quantify finite-time, finite-stepsize errors. We develop an explicit pre-limit characterization for SA with i.i.d.\ and Markovian noise. We establish existence and uniqueness of the stationary law, a geometric Wasserstein convergence to stationarity, and almost-sure and L3L^3 convergence of the steady state to the root xx^\star, identifying the scale α\sqrtα as first-order fluctuation. At this scale, we derive a higher-order quantitative Gaussian approximation with a Wasserstein error, using Stein's method and Poisson equation techniques. We further obtain non-uniform Berry--Esseen-type tail bounds, incorporating both steady-state approximation and finite-time convergence errors. We instantiate the theory for strongly convex smooth SGD, linear SA, and nonlinear contractive SA. Beyond strong convexity, for general convex SGD, we identify a Gibbs limiting law and prove a pre-limit Wasserstein approximation error under stability and Stein-equation hypothesis, which are validated numerically.
Zedong Wang, Yuyang Wang, Ijay Narang +3