Kernel Tests of Equivalence

Xing Liu; Axel Gandy

stat.MLMar 11, 2026

Kernel Tests of Equivalence

Authors: Xing Liu, Axel Gandy

Organizations: QuantCo, London, UK · Department of Mathematics, Imperial College London, London, UK

Abstract

We propose novel kernel-based tests for assessing the equivalence between distributions. Traditional goodness-of-fit testing is inappropriate for concluding the absence of distributional differences, because failure to reject the null hypothesis may simply be a result of lack of test power, also known as the Type-II error. This motivates equivalence testing, which aims to assess the absence of a statistically meaningful effect under controlled error rates. However, existing equivalence tests are either limited to parametric distributions or focus only on specific moments rather than the full distribution. We address these limitations using two kernel-based statistical discrepancies: the kernel Stein discrepancy and the Maximum Mean Discrepancy. The null hypothesis of our proposed tests assumes the candidate distribution differs from the nominal distribution by at least a pre-defined margin, which is measured by these discrepancies. We propose two approaches for computing the critical values of the tests, one using an asymptotic normality approximation, and another based on bootstrapping. Numerical experiments are conducted to assess the performance of these tests.

Figures & tables

Equivalence Test	Discrepancy	Setting	Method	Reference
E-KSD-Normal	KSD	One-sample	CLT	Theorem 2
E-KSD-Boot	KSD	One-sample	Bootstrapping	Theorem 4
E-MMD-Normal	MMD	Two-sample	CLT	Chen et al. (2023) and Theorem 6
E-MMD-Boot	MMD	Two-sample	Bootstrapping	Theorem 8

Table 1: Summary of kernel-based equivalence tests proposed in this paper, and their connections to existing literature.

Figure 1: Comparison of different hypothesis testing paradigms. The shaded area represents the space of distributions of interest. The null sets are shown in orange, and the alternative sets in grey. Left. standard testing with a point null hypothesis. Middle. Robust testing based on a statistical discrepancy $D$ . Right. Equivalence testing.

Figure 2: Gaussian mean-shift experiments with varying sample sizes. The black dotted vertical line is the equivalence margin

\theta

Figure 3: Gaussian mean-shift experiments with varying equivalence margins

\theta

, selected to be the population KSD values for different mean shifts.

Figure 4: Gaussian mean-shift experiments with

\theta

selected using a power guarantee with

\beta=0.2

. Top . KSD-based equivalence tests. Bottom . MMD-based equivalence tests.

Figure 5: Gaussian-Bernoulli RBM experiment with

\theta

selected using a Type-II error guarantee with

\beta=0.2

. The same results are plotted against the KSD values (left) and the standard deviations of the noise (right).

Figure 6: Testing equivalence of MNIST samples. The

\theta

is selected using a power guarantee with

\beta=0.2

Statistical Hypothesis Testing

Statistical hypothesis testing develops methods for assessing evidence against a stated null while controlling false-positive and false-negative rates.

See all 101 papers

Conditional Kernel Stein Discrepancy
Oct 8, 2026Federico Matteucci, Florian KalinkeKernel Methods Statistical Hypothesis Testing
Karlsruhe Institute of Technology
Computationally efficient goodness-of-fit tests through kernelized Stein discrepancy
Dec 23, 2025Zhihan Huang, Ziang NiuKernel Methods Statistical Hypothesis Testing
Department of Statistics and Data Science University of Pennsylvania
Nyström Kernel Stein Discrepancy Tests
May 24, 2026Florian Kalinke, Zoltán Szabó, Bharath K. SriperumbudurNyström Method Statistical Hypothesis Testing
Chair of Information Systems Karlsruhe Institute of Technology Am Fasanengarten 5, 76131 Karlsruhe, Germany · Department of Statistics London School of Economics Houghton Street, London, WC2A 2AE, UK · Department of Statistics The Pennsylvania State UniversityMay University Park, PA 16802, USA

Oct 8, 2026J/K move · Enter open · S save

Conditional Kernel Stein Discrepancy

Federico Matteucci, Florian Kalinke

Karlsruhe Institute of Technology

Kernel Methods Statistical Hypothesis Testing

Open paper AskPDF arXiv

Kernel Stein discrepancies (KSDs) provide a versatile tool for comparing distributions. One of their main applications is in quantifying the goodness-of-fit (GoF) between a data-generating distribution and a prescribed target distribution. In this work, we study the related problem of conditional GoF quantification: given only a (possibly non-normalized) conditional target model, without information on the distribution of its covariates, and samples from a joint distribution, the goal is to assess how well the conditional distribution of the samples matches the target. To tackle this setting, we present a framework that allows lifting unconditional KSDs to the conditional setting through an operator-valued kernel on the covariate space, going beyond the known Euclidean case. We establish that our suggested statistic vanishes if and only if the conditional model and the true conditional distribution agree for almost all covariates and deploy it to test conditional GoF on smooth manifolds and on discrete spaces. Our experiments on level, power, and runtime demonstrate the viability of testing on these domains using the proposed statistic.

Abstract

Statistical Hypothesis Testing

Explore similar work

Conditional Kernel Stein Discrepancy

Computationally efficient goodness-of-fit tests through kernelized Stein discrepancy

Nyström Kernel Stein Discrepancy Tests