Beyond Empirical Support: Structured Outlier Generation via Sinkhorn Optimal Transport

Haixiang Sun; Andrew L. Liu

stat.MLSep 25, 2026

Beyond Empirical Support: Structured Outlier Generation via Sinkhorn Optimal Transport

Authors: Haixiang Sun, Andrew L. Liu

Organizations: Edwardson School of Industrial Engineering Purdue University West Lafayette, IN, 47906

Abstract

Outliers are essential for evaluating and improving the robustness of machine learning systems, especially when future distributions may differ significantly from historical training data. In high-stakes applications, robustness often depends on rare cases that finite datasets fail to capture, making simple resampling or perturbation insufficient for stress scenario generation. Existing outlier synthesis methods typically rely on sparse neighborhoods, low support latent regions, or classifier boundary crossings, which can be heuristic, unstable, and tied to specific modalities or architectures. We therefore propose Sinkhorn Boundary Outlier Generation (SBOG), a structured framework for latent-space outlier generation that couples Sinkhorn optimal transport geometry with distributionally robust boundary modeling. The resulting Sinkhorn-induced support cost guides the sampler toward weakly supported boundary regions, while semantic constraints prevent uncontrolled drift from the intended context, yielding controlled deviations from the in-distribution reference measure rather than arbitrary sparse-region samples. Experiments on time series anomaly generation and image outlier synthesis show that our framework produces informative, semantically controlled outliers and improves downstream robustness evaluation across modalities, providing a foundation for stress scenario generation beyond empirical support.

Figures & tables

Figure 1: Pipeline of Sinkhorn Boundary Outlier Generation (SBOG).

Methods	SMD		MSL		SMAP		SWaT		WADI		Average
Methods	AU-PR	F1	AU-PR	F1	AU-PR	F1	AU-PR	F1	AU-PR	F1	AU-PR	F1
AnomalyTransformer Xu et al. (2022)	0.082	0.137	0.129	0.218	0.203	0.307	0.375	0.531	0.064	0.145	0.171	0.268
DCdetector Yang et al. (2023)	0.044	0.086	0.115	0.204	0.131	0.225	0.401	0.537	0.059	0.109	0.150	0.232
TranAD Tuli et al. (2022)	0.389	0.450	0.204	0.314	0.196	0.308	$0.464^{\dagger}$	$0.684^{\dagger}$	0.040	0.109	0.259	0.373
COUTA Xu et al. (2024)	$0.409^{\dagger}$	$0.461^{\dagger}$	0.321	0.403	0.309	0.382	0.303	0.535	0.207	0.266	0.310	0.410
TS2Vec Yue et al. (2022)	0.113	0.186	0.145	0.253	0.165	0.276	0.337	0.532	0.057	0.114	0.163	0.272

Table 1: Comparison results on SMD, MSL, SMAP, SWaT, and WADI datasets. Bold indicates the best result among the synthesis baselines, and

\dagger

indicates the best result among all baselines.

Figure 2: Component ablation of the time-series sampler.

$\tau_{E}$	Avg AUPR	Avg F1	Gap	Fill
$0.50$	0.279	0.387	0.225	1.000
$0.70$	0.295	0.409	0.224	1.000
$0.90$	0.310	0.430	0.217	1.000
$\mathbf{0.95}$	$\mathbf{0.312}$	$\mathbf{0.432}$	0.126	1.000
$0.98$	0.305	0.422	0.109	0.999
$0.99$	0.277	0.384	0.079	0.945

Table 2: Sensitivity to the threshold

\tau_{E}

Methods	INATURALIST		PLACES		SUN		TEXTURES		Average
Methods	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$
GAN Lee et al. (2018)	83.10	71.35	83.20	69.85	84.40	67.56	91.00	59.16	85.42	66.98
VOS Du et al. (2022)	43.00	93.77	47.60	91.77	39.40	93.17	66.10	81.42	49.02	90.03
NPOS Tao et al. ()	53.84	86.52	59.66	83.50	53.54	87.99	8.98	98.13	44.00	89.04
Dream-OOD Du et al. (2023)	24.10	96.10	39.87	93.11	36.88	93.31	53.99	85.56	38.76	92.02
NCIS Doorenbos et al. (2024)	20.70	96.56	34.60	94.07	35.43	94.13	44.83	88.50	33.89	93.32

Table 3: Comparative evaluation of outlier detection performance on ImageNet-100 as the ID dataset.

Methods	SVHN		PLACES365		LSUN-R		iSUN		TEXTURES		Average
Methods	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$	FPR95 $\downarrow$	AUROC $\uparrow$
GAN Lee et al. (2018)	89.45	66.95	88.75	66.76	82.35	75.87	83.45	73.49	92.80	62.99	87.36	69.21
VOS Du et al. (2022)	78.50	73.11	84.55	75.85	59.05	85.72	72.45	82.66	75.35	80.08	73.98	79.48
NPOS Tao et al. ()	11.14	97.84	79.08	71.30	56.27	82.43	51.72	85.48	35.20	92.44	46.68	85.90
DreamOOD Du et al. (2023)	58.75	87.01	70.85	79.94	24.25	95.23	1.10	99.73	46.60	88.82	40.31	90.15
BOOD* Liao et al. (2025)	5.42	98.43	40.55	90.76	2.06	99.25	0.22	99.91	5.10	98.74	10.67	97.42

Table 4: Comparative evaluation of outlier detection performance on CIFAR-100 as the ID dataset.

Strategy	Token cos.	Diversity	Yield
$k$ -NN	0.490	0.635	0.277
KDE-RBF	0.464	0.626	0.272
SBOG	0.691	0.835	0.490

Table 5: Anchor-selection ablation.

Figure 3: Finite-sample stability of the outlier-energy decision under varying

\epsilon

and

m

Metric	w/o filter	SBOG
Purity $\uparrow$	0.381	0.824
Token cos. $\uparrow$	0.329	0.602
Class margin $\uparrow$	-0.074	0.220

Table 6: Effect of semantic anchoring.

Figure 4: Visualization of anchors.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Method	BPrec $\uparrow$	SemValid $\uparrow$	BCov $\uparrow$	WorstCov $\uparrow$	SW2-to- oracle $\downarrow$	Oracle AUPR $\uparrow$
RandomNeg	0.229 $\pm$ 0.027	0.950 $\pm$ 0.001	0.091 $\pm$ 0.036	0.011 $\pm$ 0.004	0.2349 $\pm$ 0.0034	0.455 $\pm$ 0.005
kNNNeg	0.298 $\pm$ 0.036	0.977 $\pm$ 0.014	0.033 $\pm$ 0.005	0.002 $\pm$ 0.002	0.1453 $\pm$ 0.0062	0.684 $\pm$ 0.014
KDENeg	0.305 $\pm$ 0.043	0.972 $\pm$ 0.011	0.070 $\pm$ 0.008	0.005 $\pm$ 0.003	0.1489 $\pm$ 0.0089	0.830 $\pm$ 0.012
SBOG	0.629 $\pm$ 0.032	0.983 $\pm$ 0.013	0.236 $\pm$ 0.006	0.012 $\pm$ 0.003	0.1405 $\pm$ 0.0069	0.896 $\pm$ 0.052

Appendix

Table 7: Controlled boundary-generation evaluation against a method-independent population oracle.

Variant	Fidelity $\uparrow$	Contam. $\downarrow$	Margin $\uparrow$	MMD 2 $\uparrow$	Diversity $\uparrow$
Random	0.339 $\pm$ 0.014	0.661 $\pm$ 0.014	-0.000 $\pm$ 0.002	0.140 $\pm$ 0.043	0.051 $\pm$ 0.074
w/o top-A anchor	0.329 $\pm$ 0.025	0.671 $\pm$ 0.025	0.000 $\pm$ 0.002	0.467 $\pm$ 0.121	0.223 $\pm$ 0.062
w/o $\rho$ threshold	0.988 $\pm$ 0.004	0.012 $\pm$ 0.004	0.034 $\pm$ 0.038	1.076 $\pm$ 0.227	0.182 $\pm$ 0.026
w/o argmax select	0.989 $\pm$ 0.004	0.011 $\pm$ 0.004	0.034 $\pm$ 0.039	1.049 $\pm$ 0.247	0.099 $\pm$ 0.022
$k$ -NN	0.990 $\pm$ 0.031	0.010 $\pm$ 0.031	0.025 $\pm$ 0.030	0.934 $\pm$ 0.301	0.195 $\pm$ 0.027
KDE	0.992 $\pm$ 0.028	0.008 $\pm$ 0.028	0.028 $\pm$ 0.032	0.978 $\pm$ 0.265	0.198 $\pm$ 0.024

Appendix

Table 8: Component ablation of the TS outlier-synthesis sampler.

Figure 5: Generation quality validation

Synthetic Anomaly Generation

Synthetic anomaly generation creates controlled abnormal examples to train, test, and evaluate detectors when real anomalies are scarce.

See all 65 papers

Geometrically Constrained Outlier Synthesis
Mar 9, 2026Daniil Karzanov, Marcin DetynieckiOOD Detection Synthetic Anomaly Generation
AXA AI Research · EPFL, Lausanne, Switzerland · TRAIL, Sorbonne Université, Paris, France +1
Score-based Outlier Generation via Controlling the Radon-Nikodym Derivative
Sep 10, 2026Amartya Mukherjee, Tristan Milne, Kry Yik-Chau Lui +2Diffusion Sampling Synthetic Anomaly Generation
Department of Applied Mathematics, University of Waterloo, Waterloo, Ontario, Canada N2L 3G1 · Royal Bank of Canada, 1 Place Ville Marie, Montreal, Quebec, H3C 3A9
Robust Out-of-Distribution Stochastic Optimization
Apr 22, 2026Xianyu Li, Huan Xu, Xiaolin Huang +1Stochastic Optimization OOD Generalization
Department of Automation, Tsinghua University · Antai College of Economics & Management, Shanghai Jiao Tong University · Institute of Image Processing and Pattern Recognition, Shanghai Jiao Tong University

Mar 9, 2026J/K move · Enter open · S save

Geometrically Constrained Outlier Synthesis

Daniil Karzanov, Marcin Detyniecki

AXA AI Research · EPFL, Lausanne, Switzerland · TRAIL, Sorbonne Université, Paris, France+1

OOD Detection Synthetic Anomaly Generation

Open paper AskPDF arXiv

Deep neural networks for image classification often exhibit overconfidence on out-of-distribution (OOD) samples. To address this, we introduce Geometrically Constrained Outlier Synthesis (GCOS), a training-time regularization framework aimed at improving OOD robustness during inference. GCOS addresses a limitation of prior synthesis methods by generating virtual outliers in the hidden feature space that respect the learned manifold structure of in-distribution (ID) data. The synthesis proceeds in two stages: (i) a dominant-variance subspace extracted from the training features identifies geometrically informed, off-manifold directions; (ii) a conformally-inspired shell, defined by the empirical quantiles of a nonconformity score from a calibration set, adaptively controls the synthesis magnitude to produce boundary samples. The shell ensures that generated outliers are neither trivially detectable nor indistinguishable from in-distribution data, facilitating smoother learning of robust features. This is combined with a contrastive regularization objective that promotes separability of ID and OOD samples in a chosen score space, such as Mahalanobis or energy-based. Experiments demonstrate that GCOS outperforms state-of-the-art methods using standard energy-based inference on near-OOD benchmarks, defined as tasks where outliers share the same semantic domain as in-distribution data. As an exploratory extension, the framework naturally transitions to conformal OOD inference, which translates uncertainty scores into statistically valid p-values and enables thresholds with formal error guarantees, providing a pathway toward more predictable and reliable OOD detection.

Abstract

Synthetic Anomaly Generation

Explore similar work

Geometrically Constrained Outlier Synthesis

Score-based Outlier Generation via Controlling the Radon-Nikodym Derivative

Robust Out-of-Distribution Stochastic Optimization