Quantitative Target Convergence and Uniform-in-Time Propagation of Chaos for Langevin-Regularized SVGD

Sayan Banerjee; Dohyeon Kim

stat.MLAug 28, 2026

Quantitative Target Convergence and Uniform-in-Time Propagation of Chaos for Langevin-Regularized SVGD

Authors: Sayan Banerjee, Dohyeon Kim

Organizations: Department of Statistics and Operations Research, University of North Carolina at Chapel Hill, 318 Hanes Hall, CB# 3260, Chapel Hill, NC 27599-3260, USA. · Department of Computing and Mathematical Sciences, California Institute of Technology, 1200 East California Boulevard, MC 305-16, Pasadena, CA 91125, USA.

Abstract

We establish quantitative convergence to the target and uniform-in-time propagation of chaos for Langevin-regularized Stein variational gradient descent. The Stein interaction need not be small relative to the confining Langevin drift and does not generally yield a contractive particle coupling. At the mean-field level, the Stein and Langevin components dissipate the same relative entropy in the kernel-induced Stein and $2$ -Wasserstein geometries, producing the squared kernel Stein discrepancy and relative Fisher information. Under a log-Sobolev inequality for the target, this yields exponential last-iterate convergence. We also derive a finite-particle entropy identity relative to the product target, giving exponential-in-time convergence of the empirical measure up to polynomial sampling errors. For propagation of chaos, we develop two complementary finite-time approaches. A synchronous coupling, combined with exponential moment estimates for the nonlinear mean-field diffusion, yields explicit single-exponential bounds in Wasserstein distance and kernel Stein discrepancy (KSD). Moving-product entropy gives joint-law relative entropy control relative to the evolving mean-field product law and, through entropy superadditivity and concentration, fixed-marginal relative entropy and total variation bounds and empirical KSD estimates. Under an additional $T_2$ inequality for the initial law, it also yields Wasserstein bounds. Combining these finite-time estimates with target convergence at a logarithmic cutoff time gives polynomial uniform-in-time propagation of chaos rates in expectation for empirical KSD and $W_2^2$ , and for fixed-marginal total variation and $W_2^2$ . All bounds control the last iterate in physical time. We also compare the two finite-time mechanisms and identify regimes in which each gives the sharper polynomial exponent.

Figures & tables

Work	Dynamics	Principal conditions	Quantitative conclusion
SPOS [ 43 ]	Closely related noisy particle-optimization/SVGD system	Strong monotonicity and a positive contraction margin	Continuous-time one-particle uniform-in-time $W_{1}=O(N^{-1/2})$ , together with discrete-time approximation bounds
Priser–Bianchi–Salim [ 37 ]	Decreasing-step noisy-SVGD recursion associated with the continuous Langevin-regularized dynamics ( 1.2 )	Qualitative well-posedness and long-time assumptions	Characterization of the empirical-measure limit set and its large- $N$ target consistency; no explicit rate in $N$
Deterministic cutoff [ 3 ]	Deterministic SVGD	Target–kernel-specific finite- and long-time estimates	Uniform-in-averaging-time rates in general discrepancies; uniform-in-physical-time parametric rates for special finite-rank target–kernel pairs
Hong–Liu–Yang [ 22 ]	One-dimensional stochastic-SVGD-type dynamics with superlinear polynomial kernels	Pseudo/local monotonicity and sufficiently high polynomial moments	Quantitative strong tagged-particle propagation of chaos on every fixed $[0,T]$ ; no target-convergence or infinite-horizon result
Current paper	Langevin-regularized SVGD ( 1.2 )	Target LSI, smooth bounded kernels, and concentration assumptions for strong finite-time bounds; no small-interaction condition	Polynomial uniform-in-physical-time rates for expected empirical KSD and $W_{2}^{2}$ , and for fixed-marginal TV and $W_{2}^{2}$ ; finite-time fixed-marginal KL control

Table 1: Comparison with closely related work and the main-text results of the present paper.

Discrepancy and kernel condition	Finite-time estimate	Uniform-in-time conclusion
Empirical Langevin KSD KSD kernel condition	$\displaystyle\mathbb{E}\mathrm{KSD}_{\pi}(\mu_{t}^{N},\mu_{t})$ $\displaystyle\lesssim N^{-1/2}e^{\left(\kappa\wedge(L_{\mathrm{cpl}}/2)\right)t}$	$\displaystyle\sup_{t\geq 0}\mathbb{E}\mathrm{KSD}_{\pi}(\mu_{t}^{N},\mu_{t})$ $\displaystyle\lesssim N^{-\varepsilon\alpha/[2(\kappa\wedge(L_{\mathrm{cpl}}/2)+\varepsilon\alpha)]}+N^{-1/2}$
Empirical $W_{2}^{2}$ Wasserstein kernel condition	$\displaystyle\mathbb{E}W_{2}^{2}(\mu_{t}^{N},\mu_{t})$ $\displaystyle\lesssim N^{-1}(e^{L_{\mathrm{cpl}}t}-1)+r_{N,d}$	$\displaystyle\sup_{t\geq 0}\mathbb{E}W_{2}^{2}(\mu_{t}^{N},\mu_{t})$ $\displaystyle\lesssim N^{-2\varepsilon\alpha/(L_{\mathrm{cpl}}+2\varepsilon\alpha)}+r_{N,d}$
Fixed- $k$ $W_{2}^{2}$ Wasserstein kernel condition	$\displaystyle W_{2}^{2}(P_{N,t}^{(k)},\mu_{t}^{\otimes k})$ $\displaystyle\lesssim\frac{k}{N}(e^{L_{\mathrm{cpl}}t}-1)$	$\displaystyle\sup_{t\geq 0}W_{2}^{2}(P_{N,t}^{(k)},\mu_{t}^{\otimes k})$ $\displaystyle\leq C_{k}\left\{N^{-2\varepsilon\alpha/(L_{\mathrm{cpl}}+2\varepsilon\alpha)}+N^{-1}\right\}$
Fixed- $k$ total variation Either kernel condition	$\displaystyle\\|P_{N,t}^{(k)}-\mu_{t}^{\otimes k}\\|_{\mathrm{TV}}$ $\displaystyle\lesssim\sqrt{\frac{k}{N}}\,(e^{\kappa t}-1)^{1/2}$	$\displaystyle\sup_{t\geq 0}\\|P_{N,t}^{(k)}-\mu_{t}^{\otimes k}\\|_{\mathrm{TV}}$ $\displaystyle\leq C_{k}\left\{N^{-\varepsilon\alpha/(\kappa+2\varepsilon\alpha)}+N^{-1/2}\right\}$
Entropy-derived $W_{2}^{2}$ empirical and fixed- $k$ Wasserstein kernel condition and $\mu_{0}\in T_{2}(C_{0})$	$\displaystyle\mathbb{E}W_{2}^{2}(\mu_{t}^{N},\mu_{t})$ $\displaystyle\lesssim N^{-1}e^{L_{\mathrm{ent}}t}+r_{N,d}$ $\displaystyle W_{2}^{2}(P_{N,t}^{(k)},\mu_{t}^{\otimes k})\lesssim\frac{k}{N}e^{L_{\mathrm{ent}}t}$	$\displaystyle\sup_{t\geq 0}\mathbb{E}W_{2}^{2}(\mu_{t}^{N},\mu_{t})$ $\displaystyle\lesssim N^{-2\varepsilon\alpha/(L_{\mathrm{ent}}+2\varepsilon\alpha)}+r_{N,d}$ $\displaystyle\sup_{t\geq 0}W_{2}^{2}(P_{N,t}^{(k)},\mu_{t}^{\otimes k})$ $\displaystyle\leq C_{k}\left\{N^{-2\varepsilon\alpha/(L_{\mathrm{ent}}+2\varepsilon\alpha)}+N^{-1}\right\}$

Table 2: Main metric propagation of chaos bounds.

Explore similar work

Jul 16, 2026

Riesz-Kernel Stein Variational Gradient Descent: Renormalized Entropy and Long-Time Particle Limits

Stein variational gradient descent (SVGD) transports interacting particles toward a target distribution through deterministic kernelized dynamics. Singular Riesz kernels are attractive because they can provide quantitative population-level convergence, but at the finite-particle level the corresponding Stein energy has infinite self-interaction. We study periodic Riesz SVGD with self-interaction removed and prove a many-particle, long-time sampling theorem. Throughout the range in which the singular Stein energy is locally integrable, under a uniform bound on the initial relative entropy per particle, the time-averaged empirical-measure law converges weakly to the point mass

δ_π

at the target as the particle number and any diverging averaging horizon tend to infinity. We also show that the empirical-measure laws induced by invariant particle laws of finite relative entropy converge weakly to

δ_π

, without a uniform entropy bound. Below the logarithmic singularity threshold, we obtain an explicit algebraic finite-particle error bound. These results extend the joint-entropy approach for smooth-kernel SVGD to singular interactions.

Trevor Teolis, Maarten V. de Hoop

Rice University

Kernel Methods Stein Variational Gradient Descent

Ask PDF

Sep 13, 2024

Improved Finite-Particle Convergence Rates for Stein Variational Gradient Descent

We provide finite-particle convergence rates for the Stein Variational Gradient Descent (SVGD) algorithm in the Kernelized Stein Discrepancy (

\mathsf{KSD}

) and Wasserstein-2 metrics. Our key insight is that the time derivative of the relative entropy between the joint density of

N

particle locations and the

N

-fold product target measure, starting from a regular initial distribution, splits into a dominant negative part' proportional to $N$ times the expected $\mathsf{KSD}^2$ and a smaller positive part'. This observation leads to

\mathsf{KSD}

rates of order

1/\sqrt{N}

, in both continuous and discrete time, providing a near optimal (in the sense of matching the corresponding i.i.d. rates) double exponential improvement over the recent result by Shi and Mackey (2024). Under mild assumptions on the kernel and potential, these bounds also grow polynomially in the dimension

d

. By adding a bilinear component to the kernel, the above approach is used to further obtain Wasserstein-2 convergence in continuous time. For the case of `bilinear + Matérn' kernels, we derive Wasserstein-2 rates that exhibit a curse-of-dimensionality similar to the i.i.d. setting. We also obtain marginal convergence and long-time propagation of chaos results for the time-averaged particle laws.

Sayan Banerjee, Krishnakumar Balasubramanian, Promit Ghosal

Department of Statistics and Operations Research, University of North Carolina, Chapel Hill. · Department of Statistics, University of California, Davis. · Department of Statistics, University of Chicago.

Kernel Methods Variational Inference Stein Variational Gradient Descent Wasserstein Distance

Ask PDF

May 10, 2026

Quantitative Local Convergence of Mean-Field Stein Variational Gradient Flow

Stein Variational Gradient Descent (SVGD) is a deterministic interacting-particle method for sampling from a target probability measure given access to its score function. In the mean-field and continuous-time limit, it is known that the flow converges weakly toward the target, but no quantitative rate is known for the last iterate. In this paper, we establish quantitative local convergence in strong norms for this dynamics, when the interaction kernel is of Riesz type on the

d

-dimensional torus. Specifically, assuming that the initial density and the target are smooth and close in

L^2

-norm, we obtain explicit polynomial convergence rates in

L^2

-norm that depend on the dimension and on the regularity parameters of the kernel, the initialization and the target. We further show that these rates are sharp in certain regimes, and support the theory with numerical experiments. In the edge case of kernels with a Coulomb singularity, we recover the global exponential convergence result established in prior work. Our analysis is inspired by recent results on Wasserstein gradient flows of kernel mean discrepancies.

Lénaïc Chizat, Maria Colombo, Roberto Colombo +1

Dynamical Systems Optimization Convergence Analysis Mean-Field Theory Variational Inference Stein Variational Gradient Descent

Ask PDF