On the Optimisation Landscape of Overparameterised Weighted MMD

Tharrshann Jayan Logarajah; Jeremias Knoblauch; Louis Sharrock

stat.MLOct 14, 2026

On the Optimisation Landscape of Overparameterised Weighted MMD

Authors: Tharrshann Jayan Logarajah, Jeremias Knoblauch, Louis Sharrock

Organizations: University College London

Abstract

We study the optimisation landscape of the maximum mean discrepancy (MMD) over weighted atomic measures. For the one-dimensional Gaussian kernel, we show that, for any compactly supported target distribution, sufficiently many particles ensure that every local minimum has arbitrarily small MMD. Moreover, any stationary point whose MMD remains bounded away from zero is unstable under an unweighted gradient flow over the particle locations. These results hold both when the weights are constrained to the simplex, and when they are unconstrained but optimised at each step. When the target is itself finitely supported, we further strengthen this result: beyond an explicit particle threshold, there are no spurious local minima, and every spurious stationary point is unstable. To our knowledge, these are the first finite-particle overparameterisation guarantees of their kind for weighted MMD optimisation.

Figures & tables

Setting and result	Particle threshold	Guarantees
Relaxed weights $K$ -atom target, ETP kernel (Theorem 4 )	$N\geq K$	No spurious local minima. Every spurious stationary point is unstable.
Relaxed weights Target with compact support of diameter at most $D$ , Gaussian kernel (Theorem 5 )	$N>N_{\varepsilon}(D)=O_{D}\!\left(\frac{\log(1/\varepsilon)}{\log\log(1/\varepsilon)}\right)$	Every local minimum has $\MMD<\varepsilon$ . Stationary points with $\MMD\geq\varepsilon$ are unstable.
Simplex weights Target with compact support of diameter at most $D$ , Gaussian kernel (Theorem 6 )	$N>N_{\varepsilon}(D)=O_{D}\!\left(\varepsilon^{-4}\log(1/\varepsilon)\right)$	Every local minimum has $\MMD<\varepsilon$ . Stationary points with $\MMD\geq\varepsilon$ are unstable.
Simplex weights Finite atomic target, Gaussian kernel (Theorem 7 )	$N>N(P^{*})$	No spurious local minima. Every spurious stationary point is unstable.

Table 1: Summary of the one-dimensional landscape and instability guarantees. Instability refers to the unweighted particle flow

\dot{\theta}_{i}=-\Delta^{\prime}(\theta_{i})

. Relaxed weights are unrestricted and optimised at each particle configuration; the simplex results concern the joint objective in the weights and positions. The rates are asymptotic as

\varepsilon\to 0

, with

D

and the Gaussian bandwidth fixed.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 1: A numerical illustration of the finite-particle landscapes. (a) Largest MMD among the retained local minima, for each of the 30 well-specified three-atom targets. Dots correspond to individual targets, while boxes show medians and quartiles with

1.5

-IQR whiskers. For

N\geq 3

, the target can be represented exactly; hence a plotted value of zero means that the numerical search retained no positive-error local minimum. (b) MMD of the retained local minima for the continuous target

P^{*}=\operatorname{Unif}[0,12]

, shown on a logarithmic scale. The

N=1

solutions are obtained analytically.

Overparameterization

See all 22 papers

A Gradient Flow Perspective on Minimum MMD Estimation
Jul 4, 2026Sophia Seulkee Kang, Louis Sharrock, Xiaoyuan Cheng +2Parameter Estimation Maximum Mean Discrepancy
Independent Researcher · University College London
Difference of Convex Programming in the Wasserstein Space with Applications to MMD Optimization
Jun 26, 2026Clément Bonet, Pierre-Cyril Aubin-Frankowski, Youssef MrouehMaximum Mean Discrepancy Nonconvex Optimization
CMAP, CNRS, Ecole Polytechnique, IP Paris · CERMICS, CNRS, ENPC, IP Paris · IBM Research
Local linear convergence of gradient methods for overparameterized Gaussian mixtures
May 29, 2026Jingxing Wang, Vasileios Charisopoulos, Maryam FazelGradient Descent Gaussian Mixture Models
Electrical & Computer Engineering, University of Washington, Seattle, WA · National Institute for Theory and Mathematics in Biology, Chicago, IL · Amazon, Inc.

Jul 4, 2026J/K move · Enter open · S save

A Gradient Flow Perspective on Minimum MMD Estimation

Sophia Seulkee Kang, Louis Sharrock, Xiaoyuan Cheng, François-Xavier Briol+1

Independent Researcher · University College London

Parameter Estimation Maximum Mean Discrepancy Gradient Descent Stochastic Optimization Convergence

Open paper AskPDF arXiv

Minimum maximum mean discrepancy (MMD) estimation has emerged as a robust and likelihood-free alternative to maximum likelihood estimation for parameter estimation. Yet, despite its practical success, the associated optimization problem remains poorly understood, with theoretical guarantees for existing algorithms hinging on convexity assumptions that rarely hold in practice. We address this gap by proposing a preconditioned gradient descent (PGD) scheme, establishing its asymptotic global convergence under explicit gradient-dominance and projection-residual conditions. Our approach is inspired by recent progress on MMD gradient flows, a nonparametric descent scheme on the space of probability measures. We provide extensive empirical evidence that our PGD scheme outperforms standard gradient descent across a range of challenging parameter estimation and composite hypothesis testing problems.

Abstract

Overparameterization

Explore similar work

A Gradient Flow Perspective on Minimum MMD Estimation

Difference of Convex Programming in the Wasserstein Space with Applications to MMD Optimization

Local linear convergence of gradient methods for overparameterized Gaussian mixtures