Generative Neural Networks for Sinkhorn Distributionally Robust Hypothesis Testing

Fenglin Zhang; Teyan Liu; Jie Wang

stat.MLAug 24, 2026

Generative Neural Networks for Sinkhorn Distributionally Robust Hypothesis Testing

Authors: Fenglin Zhang, Teyan Liu, Jie Wang

Organizations: School of Artificial Intelligence and School of Data Science, The Chinese University of Hong Kong, Shenzhen · Department of Decision Analytics and Operations, College of Business, City University of Hong Kong

Abstract

This paper studies the Sinkhorn distributionally robust hypothesis testing (SDRHT) problem, seeking a robust detector against least-favorable distributions in Sinkhorn discrepancy-based ambiguity sets centered at the empirical distributions. Existing approaches solve this problem by solving large-scale conic programs, which are not scalable. To overcome this, we propose a generative framework that learns least-favorable distributions and supports efficient training and end-to-end sampling. For the Sinkhorn discrepancy-based ambiguity sets, we first derive an equivalent conditional-KL-divergence representation with respect to kernel-smoothed reference distributions. This property allows us to prove strong duality for both constrained and unconstrained minimax SDRHT formulations. Based on the closed-form optimal detector and Brenier's theorem, we reformulate the max-min dual formulation as a maximization problem over convex potentials whose gradients characterize invertible transport maps between kernel-smoothed distributions and their least-favorable counterparts. We efficiently approximate these potentials using Hyper Input Convex Neural Networks (HyCNNs) equipped with stochastic gradient estimators and prove the representation power of HyCNNs and the distributional universality of their induced transport maps. Numerical results show that the proposed method achieves superior accuracy and robustness across different sample sizes and dimensions, while avoiding the scalability limitations of classical SDRHT methods.

Figures & tables

Figure 1: Illustration of the training and testing processes of our least-favorable detector.

$\ell(t)$	Optimal $\phi$	Corresponding optimal $1-\Phi(\mathbb{P}_{0},\mathbb{P}_{1})/2$
$\exp(t)$	$\log\sqrt{p_{0}/p_{1}}$	$H^{2}(\mathbb{P}_{0},\mathbb{P}_{1})$
$\log(1+\exp(t))/\log 2$	$\log(p_{0}/p_{1})$	$\text{JS}(\mathbb{P}_{0},\mathbb{P}_{1})/\log 2$
$(t+1)^{2}$	$1-2p_{0}/(p_{0}+p_{1})$	$\chi^{2}(\mathbb{P}_{0},\mathbb{P}_{1})$
$(t+1)_{+}$	$\operatorname{sign}(p_{0}-p_{1})$	$\text{TV}(\mathbb{P}_{0},\mathbb{P}_{1})$

Table 1: Common choices of generating function (first column), together with their corresponding optimal detector (second column) and surrogate risk (third column).

Figure 2: The architecture of HyCNN

	$\displaystyle\nabla_{\bm{\theta}_{k,t}}F(\bm{\theta}_{0,t},\bm{\theta}_{1,t})\leftarrow$	$\displaystyle\mathsf{Risk\_Gradient\_Estimator}(\{\bm{\omega}_{0}^{(i)}\}_{i=1}^{N},\{\bm{\omega}_{1}^{(i)}\}_{i=1}^{N},\widetilde{\varphi}_{0,t},\widetilde{\varphi}_{1,t},\bm{\theta}_{k,t})$
		$\displaystyle-\nabla_{\bm{\theta}_{k,t}}\sumop\displaylimits_{k\in\mathbb{K}}\lambda_{k}\cdot\mathsf{Sinkhorn\_Estimator}(\{\bm{x}_{k}^{(i)}\}_{i=1}^{n_{k}},\{\bm{\omega}_{k,t}^{(i)}\}_{i=1}^{N}),\quad\forall k\in\mathbb{K};$

Algorithm 1 Convex Potentials Approximation based on HyCNNs.

	$\displaystyle\bm{E}\leftarrow\bm{E}$	$\displaystyle+\frac{h_{k}(\bm{\omega}_{k}^{(i)},\bm{\theta}_{0},\bm{\theta}_{1})}{2}\cdot\Bigg\{\Big[\frac{\partial\Big(\bm{\omega}_{k}^{(i)}-\nabla\widetilde{\bm{\varphi}}_{\|1-k\|}(\bar{\bm{z}}_{k}^{(i)})\Big)}{\partial\bm{\theta}}\Big]^{\top}\bm{u}_{k}^{(i)}$
		$\displaystyle+\mathsf{GLDH\_Estimator}\Big(\nabla^{2}\widetilde{\varphi}_{k}(\bm{z}_{k}^{(i)}),\bm{\theta}\Big)-\mathsf{GLDH\_Estimator}\Big(\nabla^{2}\widetilde{\varphi}_{\|1-k\|}(\bar{\bm{z}}_{k}^{(i)}),\bm{\theta}\Big)\Bigg\};$

Algorithm 3

\mathsf{Risk\_Gradient\_Estimator}

: Gradient Estimator for the Risk Function.

Panel of Figure 3 — Figure 3: Error rate comparison on GMM dataset, averaged over 100 trials. SDRO-H : SDRHT method with HyCNN; SDRO-C : SDRHT method with standard ICNN.

$d$	Computation Time (s)
$d$	SDRO-H	SDRO-C	WDRO	GMM	SVM	3NN
4	1.906	1.749	0.283	0.075	$<$ 0.000	0.052
10	1.507	1.802	0.275	0.002	0.001	0.040
30	1.292	1.900	0.271	0.002	$<$ 0.000	0.034
50	1.753	1.897	0.285	0.002	0.001	0.033
70	1.614	1.907	0.290	0.002	$<$ 0.000	0.026

Table 2: Computation time (s) of methods on Gaussian mixture dataset, averaged over 100 trials.

Panel of Figure 4 — Figure 4: Average accuracy (%) of SDRO-H and WDRO on the GMM dataset with different hyperparameters ( $n_{0}=n_{1}=10,d=100$ ).

Method	PGD- $L_{2}$ Attack				PGD- $L_{\infty}$ Attack
Method	$\Delta$ = 0.025	$\Delta$ = 0.05	$\Delta$ = 0.075	$\Delta$ = 0.1	$\Delta$ = 0.05	$\Delta$ = 0.1	$\Delta$ = 0.15	$\Delta$ = 0.2
SDRO-H	85.801	85.769	85.736	85.737	85.467	85.165	84.906	84.811
SDRO-C	82.333	82.347	82.332	82.309	82.175	82.018	81.867	81.774
WDRO	69.693	68.902	68.108	67.924	62.440	54.711	48.096	44.256
GMM	73.401	72.392	71.367	71.121	63.100	51.579	41.804	36.414
SVM	64.477	63.628	62.774	62.561	55.957	47.171	40.166	36.166

Table 3: Robustness comparison on the GMM dataset after PGD attack, averaged over

m\in[10]

and 5 trials (accuracy %,

n_{0}=n_{1}=10,d=50

). Bold represents the highest accuracy in each column.

Panel of Figure 5 — Figure 5: Error rate comparison on the MNIST dataset, averaged over 100 trials.

$\bm{n}^{\prime}$	Computation Time (s)
$\bm{n}^{\prime}$	SDRO	FDRO	WDRO	LR	SVM	3NN
2	1.943	4.821	0.022	0.008	$<$ 0.001	0.167
5	1.477	8.999	0.045	0.011	0.001	0.231
10	1.465	17.153	0.135	0.018	0.001	0.269

Table 4: Computation time (s) of methods on the MNIST dataset, averaged over 100 trials.

Method	PGD- $L_{2}$ Attack				PGD- $L_{\infty}$ Attack
Method	$\Delta$ = 0.025	$\Delta$ = 0.05	$\Delta$ = 0.075	$\Delta$ = 0.1	$\Delta$ = 0.05	$\Delta$ = 0.1	$\Delta$ = 0.15	$\Delta$ = 0.2
SDRO	95.75	95.70	95.67	95.67	93.53	90.60	87.46	85.47
FDRO	91.04	90.93	90.87	90.87	86.51	80.92	75.28	71.91
WDRO	94.54	94.47	94.42	94.42	91.18	85.63	79.42	75.56
LR	91.15	90.98	90.89	90.89	84.79	75.80	67.27	62.34
SVM	76.67	76.50	76.44	76.44	72.65	69.02	66.21	64.73

Table 5: Robustness comparison on MNIST dataset after PGD attack, averaged over

m\in[10]

and 5 trials (accuracy %,

n_{0}=n_{1}=25

Panel of Figure 6 — Figure 6: Adversarial attacks on digits by trained convex gradients. The first column displays the original images, while the following six columns show the attacked images from Epochs 1 to 6.

$n_{0}=n_{1}$	Accuracy (%)					Computation Time (s)
( $\times 10^{3}$ )	SDRO	FDRO	LR	SVM	3NN	SDRO	FDRO	LR	SVM	3NN
1	68.09	62.07	59.73	64.13	57.92	1.11	2.09	0.02	0.18	5.09
2	71.32	65.56	60.45	66.67	58.70	1.06	2.06	0.01	0.58	11.06
3	72.83	67.20	60.68	67.87	58.97	1.13	2.69	0.01	0.74	9.86
4	73.88	68.03	60.95	68.88	60.17	1.64	3.65	0.02	1.51	11.21
5	74.62	68.77	61.01	69.56	61.00	1.79	3.56	0.02	2.20	11.16

Table 6: Accuracy (

\%

) and computation time (s) of methods on the Higgs dataset, averaged over observations

m\in[10]

and 10 trials. Bold represents the highest accuracy in each row.

Method	PGD- $L_{2}$ Attack				PGD- $L_{\infty}$ Attack
Method	$\Delta$ = 0.025	$\Delta$ = 0.05	$\Delta$ = 0.075	$\Delta$ = 0.1	$\Delta$ = 0.025	$\Delta$ = 0.05	$\Delta$ = 0.075	$\Delta$ = 0.1
SDRO	72.342	71.998	71.887	71.891	73.352	72.758	72.436	72.284
FDRO	66.999	64.888	62.736	60.835	62.780	55.396	48.914	44.461
LR	62.945	60.913	59.193	57.582	59.708	54.046	48.528	44.412
SVM	67.669	65.849	64.191	62.716	64.524	58.350	53.250	49.614
3NN	58.204	54.669	50.782	48.015	50.264	38.658	30.418	26.266

Table 7: Robustness comparison on Higgs dataset after PGD attack, averaged over

m\in[10]

and 5 trials (accuracy %,

n_{0}=n_{1}=2,000

Panel of Figure 7 — Figure 7: The trajectory of image generation between Female (Class 0) and Male (Class 1). The first column displays the original images, while the following columns show the images generated during 30 epochs.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Spaces	Description
	Closed sample space, $\Omega\subseteq\mathbb{R}^{d}$ .
$\mathcal{F}$	Functional space, which is not necessarily compact.
Basic Sets
$\mathbb{K}$	Indices set of hypotheses, $\mathbb{K}=\{0,1\}$ .
$\mathscr{P}\left(\Omega\right)$	The set of all probability distributions on $\Omega$ .
$\mathcal{M}\left(\Omega\right)$	The set of measures on $\Omega$ .

Appendix

Table 8: Notations List

Functions	Description
$c:\Omega\times\Omega\to\mathbb{R}$	Transport cost function satisfying Assumption 1 , we choose $c(\bm{x},\bm{z}):=\frac{1}{2}\\|\bm{x}-\bm{z}\\|_{2}^{2}$ .
$\ell:\mathbb{R}\to\mathbb{R}_{+}\cup\{\infty\}$	Generating function (see Definition 1 ), we choose $\ell(t)=\exp(t)$ .
$\phi:\Omega\to\mathbb{R}$	Detector for hypothesis testing, which is measurable and upper semicontinuous.
$\phi_{\text{opt}}:\Omega\to\mathbb{R}$	Optimal detector for generating function $\ell(t)=\exp(t)$ , $\phi_{\text{opt}}=\frac{1}{2}\log(p_{0}/p_{1})$ .
$\varphi_{1},\varphi_{2}:\mathbb{R}^{d}\to\mathbb{R}$	Convex functions, decision variables in the problem ( 7 ).
$\varphi^{\ast}:\mathbb{R}^{d}\to\mathbb{R}\cup\{\infty\}$	Convex conjugate of a convex function $\varphi:\mathbb{R}^{d}\to\mathbb{R}\cup\{\infty\}$ .

Appendix

Table 9: Functions List

Statistical Hypothesis Testing

Statistical hypothesis testing develops methods for assessing evidence against a stated null while controlling false-positive and false-negative rates.

See all 101 papers

Jul 27, 2026

Generative Distributionally Robust Optimization

Generative models are increasingly adopted in distributionally robust optimization (DRO), but existing approaches trade off model compatibility and adversarial structure: methods that accept arbitrary samplers do not restrict worst-case laws to a generator family, while generator-parameterized adversaries rely on model-specific access such as likelihoods, scores, or training data. We propose Generative Distributionally Robust Optimization (GDRO), a principled framework that accepts any sampleable conditional generator as the nominal model and restricts worst-case laws to a chosen conditional generator family. The key is the sampler-Sinkhorn pairing: samplers represent the conditional laws exactly, while Sinkhorn divergence compares their induced distributions without likelihood access and can be estimated from samples alone. The resulting population problem admits a direct finite-sample approximation and differentiable primal-dual implementation at the active decision context. For Lipschitz losses, the population Sinkhorn radius bounds downstream degradation. Across explicit and implicit generators, our method reduces rare-context inventory regret by 60% and SocialGAN navigation collisions by 50% relative to nominal decisions.

Ziwei Zhang, Jonathan Yu-Meng Li, Zhihao Jin

1Telfer School of Management, University of Ottawa · Department of Electrical and Computer Engineering, Western University

Sinkhorn Divergence Distributionally Robust Optimization

Ask PDF

Aug 12, 2026

A Local Sinkhorn Framework for Conditional Distribution Reconstruction of Multidimensional Random Fields

In this paper, we propose a local Sinkhorn divergence framework for conditional distribution reconstruction of multidimensional random fields. By utilizing the debiased Sinkhorn divergence, our proposed approach develops a differentiable and computationally efficient local distribution matching objective to train stochastic neural networks (SNNs). Furthermore, we establish theoretical generalization error estimates for our local Sinkhorn divergence framework, which explicitly characterizes the trade-off between approximation bias and statistical efficiency controlled by the regularization parameter and reveals how our proposed local Sinkhorn divergence loss function can be efficiently applied to learning multidimensional random field models. The proposed framework provides a scalable alternative to exact local optimal transport for conditional distribution reconstruction, offering a practical compromise between geometric fidelity, statistical efficiency, and computational scalability for uncertainty quantification and probabilistic scientific machine learning. Through various numerical examples, we compare our proposed local Sinkhorn divergence framework with other loss functions to train SNNs and with other machine-learning-based uncertainty quantification frameworks, demonstrating that the proposed local Sinkhorn divergence framework achieves an effective balance between reconstruction accuracy and computational efficiency while maintaining good scalability for multidimensional stochastic systems.

Mingtao Xia, Qijing Shen

School of Mathematics, University of Birmingham, Watson Building, Edgbaston, Birmingham, B15 2TT, United Kingdom · Department of Mathematics, University of Houston, 3551 Cullen Blvd, Houston, 77204, Texas, United States · Nuffield Department of Medicine, University of Oxford, United Kingdom, Old Road Campus, Oxford, OX3 7BN, United Kingdom

Uncertainty Quantification Sinkhorn Divergence Conditional Distribution Estimation Wasserstein Distance

Ask PDF

Dec 29, 2024

Distributionally Robust Optimization via Iterative Algorithms in Continuous Probability Spaces

We study distributionally robust optimization (DRO) for robust inference when the worst-case distribution is continuous, leading to significant computational challenges due to the infinite-dimensional nature of the optimization problem. Unlike traditional discrete DRO approaches, which often suffer from scalability issues, limited generalization, and costly worst-case inference, our framework exploits Brenier's theorem to characterize the least favorable distribution as the pushforward of a transport map from a continuous reference measure. This characterization motivates our study of the minimax problem in Wasserstein space. We propose an iterative algorithmic framework with multiple variants and establish global convergence guarantees under mild assumptions, deriving complexity bounds in terms of subgradient evaluations and inexact Jordan-Kinderlehrer-Otto updates. Numerical results with neural network-based transport maps demonstrate that the proposed method enables both stable training of robust classifiers and effective worst-case inference for classification tasks.

Linglingzhi Zhu, Yunqin Zhu, Yao Xie

1H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology

Distributionally Robust Optimization Wasserstein Distributionally Robust Optimization Robust Classification

Ask PDF

Abstract

Statistical Hypothesis Testing

Explore similar work

Generative Distributionally Robust Optimization

A Local Sinkhorn Framework for Conditional Distribution Reconstruction of Multidimensional Random Fields

Distributionally Robust Optimization via Iterative Algorithms in Continuous Probability Spaces