Dimension-free regimes for unadjusted Langevin in ridge-separable models: A subspace robust Wasserstein analysis

Yuansi Chen; Francesco Pedrotti; Peter A. Whalley

stat.MLOct 22, 2026

Dimension-free regimes for unadjusted Langevin in ridge-separable models: A subspace robust Wasserstein analysis

Authors: Yuansi Chen, Francesco Pedrotti, Peter A. Whalley

Organizations: Seminar for Statistics, ETH Zürich · Department of Statistics, University of Warwick

Abstract

The Unadjusted Langevin Algorithm (ULA) is widely used for high-dimensional Bayesian inference. Classical analyses in Wasserstein-2 distance predict an asymptotic bias and a mixing time that grow with dimension, but this metric is stronger than necessary for many statistical tasks. We instead analyze the bias in subspace robust Wasserstein distance, which controls the mean and arbitrary low-dimensional marginals and is therefore well suited to observables typically estimated in practice. For ridge-separable potentials formed with i.i.d. data, we prove high-probability bounds showing a bell-shaped mixing profile as the number of data points $N$ varies relative to the dimension $d$ . In the overparameterized regime, the effective dimension is the rank $N$ of the data-dependent perturbation, leading to complexity depending on $N$ rather than $d$ . In the large-sample regime, Gaussian-perturbation estimates yield dimension-free complexity once $N$ is sufficiently large. In contrast, near the proportional regime $\asymp d$ , we prove that dimension-free guarantees are impossible in general.

Figures & tables

Figure 1:

20

-dimensional Bayesian logistic regression with a mean-zero and variance one Gaussian prior, simulated data

a_{i}\overset{\text{i.i.d.}}{\sim}\mathcal{N}(0,\frac{1}{d}I_{d}),

for

i=1,\dots,N

, with corresponding simulated labels defined via

y_{i}\mid a_{i},w_{\mathrm{true}}\sim\mathrm{Bernoulli}\left(p_{i}\right)

with probabilities,

p_{i}=\sigma(a_{i}^{\top}w_{\mathrm{true}}):=1/(1+\exp{\left(-a_{i}^{\top}w_{\mathrm{true}}\right)})

, where

w_{\mathrm{true}}\sim\mathcal{N}\!\left(0,\,\frac{1}{d}I_{d}\right)

. Left: Bell-shaped profile for bias in the mean. Middle: Stepsize scaling is in accordance with Lemma 3.1 . Right: Bell-shaped profile for rescaled bias in the mean.

Figure 2: Illustration of Theorem 1

Figure 3: Bias of ULA as a function of the dimension

d

for two choices of the number of “data points”:

N=d

and

N=10

fixed. For each

d\in\{25,50,100,200\}

, we study the target with potential

V(x)=\frac{1}{2}\|x\|^{2}+\sum_{i=1}^{N}\left(0.7\sin\left(\left\langle x,a_{i}\right\rangle\right)+\left\lvert\left\langle x,a_{i}\right\rangle\right\rvert^{2}\right)

, where

a_{i}\sim\mathcal{N}(0,I_{d}/d),

using one sampled matrix

A\in\mathbb{R}^{N\times d}

for each value of

d

. The reference mean

m

is approximated by long MALA simulations with a tuned step size targeting acceptance

0.574

, and

\hat{m}

is the ULA estimate obtained with step size

h=0.2

. The figure reports the error

\left\lvert m-\hat{m}_{h}\right\rvert

, averaged over independent chains, for the same set of dimensions in both regimes.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 4:

18

-dimensional Bayesian logistic regression on the UCI SUSY dataset, with a standard Gaussian prior on the transformed coefficients. Left: Bell-shaped profile for the estimated error in the mean. Middle: Step-size scaling as a function of

N

. Right: Bell-shaped profile for the rescaled error in the mean.

Langevin Dynamics

Langevin dynamics uses drift and random noise to sample probability distributions and optimize objectives.

See all 59 papers

Aug 3, 2026

Wasserstein mixing time of the unadjusted Langevin algorithm

We provide new estimates in Wasserstein distance for the asymptotic bias of the unadjusted Langevin algorithm, in the classical setting of log-smooth strongly log-concave measures. Our bound implies a Wasserstein mixing time of order

κ\sqrt{d}/\varepsilon

, where

κ

is the condition number,

d

is the dimension, and

\varepsilon

is the target precision: this improves by a factor of

\sqrt{d}/\varepsilon

over the previous state-of-the-art results.

Francesco Pedrotti, Peter A. Whalley

ETH Zürich

Markov Chain Monte Carlo Langevin Dynamics Log-Concave Sampling Wasserstein Distance

Ask PDF

Mar 2, 2026

Dimension-Independent Convergence of Underdamped Langevin Monte Carlo in KL Divergence

Underdamped Langevin dynamics (ULD) is a widely-used sampler for Gibbs distributions

π\propto e^{-V}

, and is often empirically effective in high dimensions. However, existing non-asymptotic convergence guarantees for discretized ULD typically scale polynomially with the ambient dimension

d

, leading to vacuous bounds when

d

is large. The main known dimension-free result concerns the randomized midpoint discretization in Wasserstein-2 distance (Liu et al.,2023), while dimension-independent guarantees for ULD discretizations in KL divergence have remained open. We close this gap by proving the first dimension-free KL divergence bounds for discretized ULD. Our analysis refines the KL local error framework (Altschuler et al., 2025) to a dimension-free setting and yields bounds that depend on

\mathrm{tr}(\mathbf{H})

, where

\mathbf{H}

upper bounds the Hessian of

V

, rather than on

d

. As a consequence, we obtain improved iteration complexity for underdamped Langevin Monte Carlo relative to overdamped Langevin methods in regimes where

\mathrm{tr}(\mathbf{H})\ll d

Shiyuan Zhang, Qiwei Di, Xuheng Li +1

Department of Computer Science, University of California, Los Angeles, CA 90095, USA

Kullback-Leibler Divergence Markov Chain Monte Carlo Langevin Dynamics Convergence Analysis

Ask PDF

Sep 30, 2026

Near-Linear Accuracy Bounds for Moreau--Yosida Unadjusted Langevin Sampling

We establish near-linear accuracy bounds for the classical Moreau--Yosida unadjusted Langevin algorithm (MYULA). The target is

π\propto e^{-f-g}

, where

f\in C^2(\mathbb{R}^d)

m

-strongly convex with Lipschitz gradient and

g

is convex and globally Lipschitz. Under an explicit parameter-dependent step-size condition, we bound the invariant-measure bias relative to the Moreau-smoothed target by

\widetilde O(h)

, with only logarithmic dependence on the inverse smoothing parameter in the error coefficient. Combining this estimate with the Moreau approximation bias and Wasserstein contraction gives

\widetilde O(\varepsilon^{-1})

iterations to make the

N

th-iterate law

μ_N

satisfy

\sqrt m\,W_2(μ_N,π)\le\varepsilon

, for fixed model parameters and initialization. We bound the stationary error directly, without assuming third derivatives or a Lipschitz Hessian. Each iteration uses one gradient evaluation and one exact proximal evaluation. The key idea in our analysis is to convert a second-order stationary residual into a Wasserstein bound using a Poisson-based estimate.

Yuchen Xin, Zhihua Zhang

School of Mathematical Sciences, Peking University

Markov Chain Monte Carlo Langevin Dynamics Log-Concave Sampling Convergence Analysis

Ask PDF

Abstract

Langevin Dynamics

Explore similar work

Wasserstein mixing time of the unadjusted Langevin algorithm

Dimension-Independent Convergence of Underdamped Langevin Monte Carlo in KL Divergence

Near-Linear Accuracy Bounds for Moreau--Yosida Unadjusted Langevin Sampling