Generative Neural Networks for Sinkhorn Distributionally Robust Hypothesis Testing

Fenglin Zhang; Teyan Liu; Jie Wang

stat.MLAug 24, 2026

Generative Neural Networks for Sinkhorn Distributionally Robust Hypothesis Testing

Authors: Fenglin Zhang, Teyan Liu, Jie Wang

Organizations: School of Artificial Intelligence and School of Data Science, The Chinese University of Hong Kong, Shenzhen · Department of Decision Analytics and Operations, College of Business, City University of Hong Kong

Abstract

This paper studies the Sinkhorn distributionally robust hypothesis testing (SDRHT) problem, seeking a robust detector against least-favorable distributions in Sinkhorn discrepancy-based ambiguity sets centered at the empirical distributions. Existing approaches solve this problem by solving large-scale conic programs, which are not scalable. To overcome this, we propose a generative framework that learns least-favorable distributions and supports efficient training and end-to-end sampling. For the Sinkhorn discrepancy-based ambiguity sets, we first derive an equivalent conditional-KL-divergence representation with respect to kernel-smoothed reference distributions. This property allows us to prove strong duality for both constrained and unconstrained minimax SDRHT formulations. Based on the closed-form optimal detector and Brenier's theorem, we reformulate the max-min dual formulation as a maximization problem over convex potentials whose gradients characterize invertible transport maps between kernel-smoothed distributions and their least-favorable counterparts. We efficiently approximate these potentials using Hyper Input Convex Neural Networks (HyCNNs) equipped with stochastic gradient estimators and prove the representation power of HyCNNs and the distributional universality of their induced transport maps. Numerical results show that the proposed method achieves superior accuracy and robustness across different sample sizes and dimensions, while avoiding the scalability limitations of classical SDRHT methods.

Figures & tables

Figure 1: Illustration of the training and testing processes of our least-favorable detector.

$\ell(t)$	Optimal $\phi$	Corresponding optimal $1-\Phi(\mathbb{P}_{0},\mathbb{P}_{1})/2$
$\exp(t)$	$\log\sqrt{p_{0}/p_{1}}$	$H^{2}(\mathbb{P}_{0},\mathbb{P}_{1})$
$\log(1+\exp(t))/\log 2$	$\log(p_{0}/p_{1})$	$\text{JS}(\mathbb{P}_{0},\mathbb{P}_{1})/\log 2$
$(t+1)^{2}$	$1-2p_{0}/(p_{0}+p_{1})$	$\chi^{2}(\mathbb{P}_{0},\mathbb{P}_{1})$
$(t+1)_{+}$	$\operatorname{sign}(p_{0}-p_{1})$	$\text{TV}(\mathbb{P}_{0},\mathbb{P}_{1})$

Table 1: Common choices of generating function (first column), together with their corresponding optimal detector (second column) and surrogate risk (third column).

Figure 2: The architecture of HyCNN

	$\displaystyle\nabla_{\bm{\theta}_{k,t}}F(\bm{\theta}_{0,t},\bm{\theta}_{1,t})\leftarrow$	$\displaystyle\mathsf{Risk\_Gradient\_Estimator}(\{\bm{\omega}_{0}^{(i)}\}_{i=1}^{N},\{\bm{\omega}_{1}^{(i)}\}_{i=1}^{N},\widetilde{\varphi}_{0,t},\widetilde{\varphi}_{1,t},\bm{\theta}_{k,t})$
		$\displaystyle-\nabla_{\bm{\theta}_{k,t}}\sumop\displaylimits_{k\in\mathbb{K}}\lambda_{k}\cdot\mathsf{Sinkhorn\_Estimator}(\{\bm{x}_{k}^{(i)}\}_{i=1}^{n_{k}},\{\bm{\omega}_{k,t}^{(i)}\}_{i=1}^{N}),\quad\forall k\in\mathbb{K};$

Algorithm 1 Convex Potentials Approximation based on HyCNNs.

	$\displaystyle\bm{E}\leftarrow\bm{E}$	$\displaystyle+\frac{h_{k}(\bm{\omega}_{k}^{(i)},\bm{\theta}_{0},\bm{\theta}_{1})}{2}\cdot\Bigg\{\Big[\frac{\partial\Big(\bm{\omega}_{k}^{(i)}-\nabla\widetilde{\bm{\varphi}}_{\|1-k\|}(\bar{\bm{z}}_{k}^{(i)})\Big)}{\partial\bm{\theta}}\Big]^{\top}\bm{u}_{k}^{(i)}$
		$\displaystyle+\mathsf{GLDH\_Estimator}\Big(\nabla^{2}\widetilde{\varphi}_{k}(\bm{z}_{k}^{(i)}),\bm{\theta}\Big)-\mathsf{GLDH\_Estimator}\Big(\nabla^{2}\widetilde{\varphi}_{\|1-k\|}(\bar{\bm{z}}_{k}^{(i)}),\bm{\theta}\Big)\Bigg\};$

Algorithm 3

\mathsf{Risk\_Gradient\_Estimator}

: Gradient Estimator for the Risk Function.

Panel of Figure 3 — Figure 3: Error rate comparison on GMM dataset, averaged over 100 trials. SDRO-H : SDRHT method with HyCNN; SDRO-C : SDRHT method with standard ICNN.

$d$	Computation Time (s)
$d$	SDRO-H	SDRO-C	WDRO	GMM	SVM	3NN
4	1.906	1.749	0.283	0.075	$<$ 0.000	0.052
10	1.507	1.802	0.275	0.002	0.001	0.040
30	1.292	1.900	0.271	0.002	$<$ 0.000	0.034
50	1.753	1.897	0.285	0.002	0.001	0.033
70	1.614	1.907	0.290	0.002	$<$ 0.000	0.026

Table 2: Computation time (s) of methods on Gaussian mixture dataset, averaged over 100 trials.

Panel of Figure 4 — Figure 4: Average accuracy (%) of SDRO-H and WDRO on the GMM dataset with different hyperparameters ( $n_{0}=n_{1}=10,d=100$ ).

Method	PGD- $L_{2}$ Attack				PGD- $L_{\infty}$ Attack
Method	$\Delta$ = 0.025	$\Delta$ = 0.05	$\Delta$ = 0.075	$\Delta$ = 0.1	$\Delta$ = 0.05	$\Delta$ = 0.1	$\Delta$ = 0.15	$\Delta$ = 0.2
SDRO-H	85.801	85.769	85.736	85.737	85.467	85.165	84.906	84.811
SDRO-C	82.333	82.347	82.332	82.309	82.175	82.018	81.867	81.774
WDRO	69.693	68.902	68.108	67.924	62.440	54.711	48.096	44.256
GMM	73.401	72.392	71.367	71.121	63.100	51.579	41.804	36.414
SVM	64.477	63.628	62.774	62.561	55.957	47.171	40.166	36.166

Table 3: Robustness comparison on the GMM dataset after PGD attack, averaged over

m\in[10]

and 5 trials (accuracy %,

n_{0}=n_{1}=10,d=50

). Bold represents the highest accuracy in each column.

Panel of Figure 5 — Figure 5: Error rate comparison on the MNIST dataset, averaged over 100 trials.

$\bm{n}^{\prime}$	Computation Time (s)
$\bm{n}^{\prime}$	SDRO	FDRO	WDRO	LR	SVM	3NN
2	1.943	4.821	0.022	0.008	$<$ 0.001	0.167
5	1.477	8.999	0.045	0.011	0.001	0.231
10	1.465	17.153	0.135	0.018	0.001	0.269

Table 4: Computation time (s) of methods on the MNIST dataset, averaged over 100 trials.

Method	PGD- $L_{2}$ Attack				PGD- $L_{\infty}$ Attack
Method	$\Delta$ = 0.025	$\Delta$ = 0.05	$\Delta$ = 0.075	$\Delta$ = 0.1	$\Delta$ = 0.05	$\Delta$ = 0.1	$\Delta$ = 0.15	$\Delta$ = 0.2
SDRO	95.75	95.70	95.67	95.67	93.53	90.60	87.46	85.47
FDRO	91.04	90.93	90.87	90.87	86.51	80.92	75.28	71.91
WDRO	94.54	94.47	94.42	94.42	91.18	85.63	79.42	75.56
LR	91.15	90.98	90.89	90.89	84.79	75.80	67.27	62.34
SVM	76.67	76.50	76.44	76.44	72.65	69.02	66.21	64.73

Table 5: Robustness comparison on MNIST dataset after PGD attack, averaged over

m\in[10]

and 5 trials (accuracy %,

n_{0}=n_{1}=25

Panel of Figure 6 — Figure 6: Adversarial attacks on digits by trained convex gradients. The first column displays the original images, while the following six columns show the attacked images from Epochs 1 to 6.

$n_{0}=n_{1}$	Accuracy (%)					Computation Time (s)
( $\times 10^{3}$ )	SDRO	FDRO	LR	SVM	3NN	SDRO	FDRO	LR	SVM	3NN
1	68.09	62.07	59.73	64.13	57.92	1.11	2.09	0.02	0.18	5.09
2	71.32	65.56	60.45	66.67	58.70	1.06	2.06	0.01	0.58	11.06
3	72.83	67.20	60.68	67.87	58.97	1.13	2.69	0.01	0.74	9.86
4	73.88	68.03	60.95	68.88	60.17	1.64	3.65	0.02	1.51	11.21
5	74.62	68.77	61.01	69.56	61.00	1.79	3.56	0.02	2.20	11.16

Table 6: Accuracy (

\%

) and computation time (s) of methods on the Higgs dataset, averaged over observations

m\in[10]

and 10 trials. Bold represents the highest accuracy in each row.

Method	PGD- $L_{2}$ Attack				PGD- $L_{\infty}$ Attack
Method	$\Delta$ = 0.025	$\Delta$ = 0.05	$\Delta$ = 0.075	$\Delta$ = 0.1	$\Delta$ = 0.025	$\Delta$ = 0.05	$\Delta$ = 0.075	$\Delta$ = 0.1
SDRO	72.342	71.998	71.887	71.891	73.352	72.758	72.436	72.284
FDRO	66.999	64.888	62.736	60.835	62.780	55.396	48.914	44.461
LR	62.945	60.913	59.193	57.582	59.708	54.046	48.528	44.412
SVM	67.669	65.849	64.191	62.716	64.524	58.350	53.250	49.614
3NN	58.204	54.669	50.782	48.015	50.264	38.658	30.418	26.266

Table 7: Robustness comparison on Higgs dataset after PGD attack, averaged over

m\in[10]

and 5 trials (accuracy %,

n_{0}=n_{1}=2,000

Panel of Figure 7 — Figure 7: The trajectory of image generation between Female (Class 0) and Male (Class 1). The first column displays the original images, while the following columns show the images generated during 30 epochs.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Spaces	Description
	Closed sample space, $\Omega\subseteq\mathbb{R}^{d}$ .
$\mathcal{F}$	Functional space, which is not necessarily compact.
Basic Sets
$\mathbb{K}$	Indices set of hypotheses, $\mathbb{K}=\{0,1\}$ .
$\mathscr{P}\left(\Omega\right)$	The set of all probability distributions on $\Omega$ .
$\mathcal{M}\left(\Omega\right)$	The set of measures on $\Omega$ .

Appendix

Table 8: Notations List

Functions	Description
$c:\Omega\times\Omega\to\mathbb{R}$	Transport cost function satisfying Assumption 1 , we choose $c(\bm{x},\bm{z}):=\frac{1}{2}\\|\bm{x}-\bm{z}\\|_{2}^{2}$ .
$\ell:\mathbb{R}\to\mathbb{R}_{+}\cup\{\infty\}$	Generating function (see Definition 1 ), we choose $\ell(t)=\exp(t)$ .
$\phi:\Omega\to\mathbb{R}$	Detector for hypothesis testing, which is measurable and upper semicontinuous.
$\phi_{\text{opt}}:\Omega\to\mathbb{R}$	Optimal detector for generating function $\ell(t)=\exp(t)$ , $\phi_{\text{opt}}=\frac{1}{2}\log(p_{0}/p_{1})$ .
$\varphi_{1},\varphi_{2}:\mathbb{R}^{d}\to\mathbb{R}$	Convex functions, decision variables in the problem ( 7 ).
$\varphi^{\ast}:\mathbb{R}^{d}\to\mathbb{R}\cup\{\infty\}$	Convex conjugate of a convex function $\varphi:\mathbb{R}^{d}\to\mathbb{R}\cup\{\infty\}$ .

Appendix

Table 9: Functions List

Statistical Hypothesis Testing

Statistical hypothesis testing develops methods for assessing evidence against a stated null while controlling false-positive and false-negative rates.

See all 101 papers

Generative Distributionally Robust Optimization
Jul 27, 2026Ziwei Zhang, Jonathan Yu-Meng Li, Zhihao JinSinkhorn Divergence Distributionally Robust Optimization
1Telfer School of Management, University of Ottawa · Department of Electrical and Computer Engineering, Western University
A Local Sinkhorn Framework for Conditional Distribution Reconstruction of Multidimensional Random Fields
Aug 12, 2026Mingtao Xia, Qijing ShenUncertainty Quantification Sinkhorn Divergence
School of Mathematics, University of Birmingham, Watson Building, Edgbaston, Birmingham, B15 2TT, United Kingdom · Department of Mathematics, University of Houston, 3551 Cullen Blvd, Houston, 77204, Texas, United States · Nuffield Department of Medicine, University of Oxford, United Kingdom, Old Road Campus, Oxford, OX3 7BN, United Kingdom
Distributionally Robust Optimization via Iterative Algorithms in Continuous Probability Spaces
Dec 29, 2024Linglingzhi Zhu, Yunqin Zhu, Yao XieDistributionally Robust Optimization Wasserstein Distributionally Robust Optimization
1H. Milton Stewart School of Industrial and Systems Engineering Georgia Institute of Technology

Jul 27, 2026J/K move · Enter open · S save

Generative Distributionally Robust Optimization

Ziwei Zhang, Jonathan Yu-Meng Li, Zhihao Jin

1Telfer School of Management, University of Ottawa · Department of Electrical and Computer Engineering, Western University

Sinkhorn Divergence Distributionally Robust Optimization

Open paper AskPDF arXiv

Generative models are increasingly adopted in distributionally robust optimization (DRO), but existing approaches trade off model compatibility and adversarial structure: methods that accept arbitrary samplers do not restrict worst-case laws to a generator family, while generator-parameterized adversaries rely on model-specific access such as likelihoods, scores, or training data. We propose Generative Distributionally Robust Optimization (GDRO), a principled framework that accepts any sampleable conditional generator as the nominal model and restricts worst-case laws to a chosen conditional generator family. The key is the sampler-Sinkhorn pairing: samplers represent the conditional laws exactly, while Sinkhorn divergence compares their induced distributions without likelihood access and can be estimated from samples alone. The resulting population problem admits a direct finite-sample approximation and differentiable primal-dual implementation at the active decision context. For Lipschitz losses, the population Sinkhorn radius bounds downstream degradation. Across explicit and implicit generators, our method reduces rare-context inventory regret by 60% and SocialGAN navigation collisions by 50% relative to nominal decisions.

Abstract

Statistical Hypothesis Testing

Explore similar work

Generative Distributionally Robust Optimization

A Local Sinkhorn Framework for Conditional Distribution Reconstruction of Multidimensional Random Fields

Distributionally Robust Optimization via Iterative Algorithms in Continuous Probability Spaces