MiniMMD: Minimization of MMD for Sampling, Inference and Integration

Zonghao Chen

stat.MLOct 28, 2026

MiniMMD: Minimization of MMD for Sampling, Inference and Integration

Authors: Zonghao Chen

Abstract

Many problems in computational statistics and machine learning can be formulated as approximating a target distribution by minimizing a discrepancy over a restricted class of candidate distributions. This minimum-discrepancy viewpoint appears, for example, in numerical integration, sampling, generative modelling, and statistical inference, but these instances are often studied separately in the literature. This thesis develops the first unified treatment of these many problems through the minimization of the maximum mean discrepancy (MMD), a kernel-based discrepancy between probability measures. The central theme is that MMD is not only a statistically meaningful distance between probability distributions, but also a tractable optimization objective to design practical and efficient algorithms. This thesis summarizes the recent theoretical and empirical advancements of such minimum-MMD based algorithms.

Figures & tables

Notation	Meaning
$\mathbb{N}$	Natural numbers, $\{0,1,\ldots\}$ .
$\mathrm{Id}$ or $\mathrm{Id}_{d}$	Identity operator or identity matrix, depending on context; $\mathrm{Id}_{d}$ denotes the $d\times d$ identity matrix.
$x_{1:N}\in\mathbb{R}^{N\times d}$	Vector when $d=1$ , or matrix $[x_{1}\;x_{2}\;\ldots\;x_{N}]^{\top}$ , for $x_{1},\ldots,x_{N}\in\mathcal{X}\subseteq\mathbb{R}^{d}$ .
$T_{\#}\mathbb{P}$	Pushforward of $\mathbb{P}$ under a measurable map $T$ , defined by $(T_{\#}\mathbb{P})(A)=\mathbb{P}(T^{-1}(A))$ for measurable sets $A$ .
$L_{2}(\mathbb{P})$	Hilbert space of square-integrable functions under the probability measure $\mathbb{P}$ , with norm $\\|f\\|_{L_{2}(\mathbb{P})}^{2}=\int\|f(x)\|^{2}\,\mathrm{d}\mathbb{P}(x)$ .
$L_{2}^{d}(\mathbb{P})$	Vector-valued $L_{2}$ space of functions $f=(f_{1},\ldots,f_{d})$ with $f_{i}\in L_{2}(\mathbb{P})$ , equipped with $\langle f,g\rangle_{L_{2}^{d}(\mathbb{P})}=\sum_{i=1}^{d}\int f_{i}(x)g_{i}(x)\,\mathrm{d}\mathbb{P}(x)$ .

Table 1.1 : Frequently used function spaces and probability-measure notation.

Method	Cost	Method	Cost
NMC	$\mathcal{O}(\Delta^{-3})$ or $\mathcal{O}(\Delta^{-4})$	NQMC	$\mathcal{O}(\Delta^{-2.5})$
MLMC	$\mathcal{O}(\Delta^{-2})$	NKQ ( Corollary 3.2 )	$\tilde{\mathcal{O}}\Big(\Delta^{-\frac{d_{\mathcal{X}}}{s_{\mathcal{X}}}-\frac{d_{\Theta}}{s_{\Theta}}}\Big)$

Table 3.1 : Cost of methods for nested expectations, measured through the number of function evaluations required to ensure

|I-\hat{I}|\leq\Delta

. NMC rate is taken from Theorem 3 of Rainforth et al. (2018) , NQMC rate is taken from Proposition 4 of Bartuska et al. (2023) , MLMC rate is taken from Section 3.1 of Giles (2018) . Smaller exponents

r

\Delta^{-r}

indicate a cheaper method.

\displaystyle y_{n+1}^{(i)}=y_{n}^{(i)}-\gamma(1+\lambda_{n})\nabla h_{\hat{\mu}_{n},\hat{\pi}}^{\ast}(y_{n}^{(i)}).

Algorithm 1

\operatorname{DrMMD}

particle descent (Algorithm 1 of Chen et al. (2025c) )

Method	Type-I error ( $\downarrow$ )	Time	Power ( $\uparrow$ )	Time
PGD ( $\mathcal{I}=1$ )	0.050 $\pm$ 0.018	8.6s	0.829 $\pm$ 0.032	3.6s
GD ( $\mathcal{I}=400$ )	0.107 $\pm$ 0.026	10.4s	0.600 $\pm$ 0.041	4.9s
GD ( $\mathcal{I}=1$ )	0.071 $\pm$ 0.022	2.6s	0.279 $\pm$ 0.038	1.1s

Table 5.1 : Composite goodness-of-fit test. Critical values are computed by parametric bootstrap with

B=50

bootstrap samples. The nominal Type-I error is

\alpha=0.05

; lower Type-I error and higher power indicate better performance. All experiments are repeated over

140

random seeds; results report the mean

\pm

standard error.

Maximum Mean Discrepancy

Maximum Mean Discrepancy (MMD) compares probability distributions through distances between their kernel mean embeddings.

See all 34 papers

Scalable Kernel-Based Distances for Statistical Inference and Integration
Feb 25, 2026Masha NaslidnykNumerical Integration Maximum Mean Discrepancy
Department of Computer Science University College London
A Gradient Flow Perspective on Minimum MMD Estimation
Jul 4, 2026Sophia Seulkee Kang, Louis Sharrock, Xiaoyuan Cheng +2Parameter Estimation Maximum Mean Discrepancy
Independent Researcher · University College London
Minimax Lower Bounds of Kernel Discrepancy Estimation: MMD, HSIC, KSD
Jul 27, 2026Jose Cribeiro-Ramallo, Florian Kalinke, Zoltán SzabóMaximum Mean Discrepancy Minimax Estimation
Chair of Information Systems, Karlsruhe Institute of Technology, Am Fasanengarten 5, 76131 Karlsruhe, Germany · Department of Statistics, London School of Economics, Houghton Street, London, WC2A 2AE, UK

Feb 25, 2026J/K move · Enter open · S save

Scalable Kernel-Based Distances for Statistical Inference and Integration

Masha Naslidnyk

Department of Computer Science University College London

Numerical Integration Maximum Mean Discrepancy Integral Probability Metrics

Open paper AskPDF arXiv

Representing, comparing, and measuring the distance between probability distributions is a key task in computational statistics and machine learning. The choice of representation and the associated distance determine properties of the methods in which they are used: for example, certain distances can allow one to encode robustness or smoothness of the problem. Kernel methods offer flexible and rich Hilbert space representations of distributions that allow the modeller to enforce properties through the choice of kernel, and estimate associated distances at efficient nonparametric rates. In particular, the maximum mean discrepancy (MMD), a kernel-based distance constructed by comparing Hilbert space mean functions, has received significant attention due to its computational tractability and is favoured by practitioners. In this thesis, we conduct a thorough study of kernel-based distances with a focus on efficient computation, with core contributions in Chapters 3 to 6. Part I of the thesis is focused on the MMD, specifically on improved MMD estimation. In Chapter 3 we propose a theoretically sound, improved estimator for MMD in simulation-based inference. Then, in Chapter 4, we propose an MMD-based estimator for conditional expectations, a ubiquitous task in statistical computation. Closing Part I, in Chapter 5 we study the problem of calibration when MMD is applied to the task of integration. In Part II, motivated by the recent developments in kernel embeddings beyond the mean, we introduce a family of novel kernel-based discrepancies: kernel quantile discrepancies. These address some of the pitfalls of MMD, and are shown through both theoretical results and an empirical study to offer a competitive alternative to MMD and its fast approximations. We conclude with a discussion on broader lessons and future work emerging from the thesis.

Abstract

Maximum Mean Discrepancy

Explore similar work

Scalable Kernel-Based Distances for Statistical Inference and Integration

A Gradient Flow Perspective on Minimum MMD Estimation

Minimax Lower Bounds of Kernel Discrepancy Estimation: MMD, HSIC, KSD