Geometrically Constrained Outlier Synthesis

Daniil Karzanov; Marcin Detyniecki

cs.LGMar 9, 2026

Geometrically Constrained Outlier Synthesis

Authors: Daniil Karzanov, Marcin Detyniecki

Organizations: AXA AI Research · EPFL, Lausanne, Switzerland · TRAIL, Sorbonne Université, Paris, France · Polish Academy of Science, IBS PAN, Warsaw, Poland

Abstract

Deep neural networks for image classification often exhibit overconfidence on out-of-distribution (OOD) samples. To address this, we introduce Geometrically Constrained Outlier Synthesis (GCOS), a training-time regularization framework aimed at improving OOD robustness during inference. GCOS addresses a limitation of prior synthesis methods by generating virtual outliers in the hidden feature space that respect the learned manifold structure of in-distribution (ID) data. The synthesis proceeds in two stages: (i) a dominant-variance subspace extracted from the training features identifies geometrically informed, off-manifold directions; (ii) a conformally-inspired shell, defined by the empirical quantiles of a nonconformity score from a calibration set, adaptively controls the synthesis magnitude to produce boundary samples. The shell ensures that generated outliers are neither trivially detectable nor indistinguishable from in-distribution data, facilitating smoother learning of robust features. This is combined with a contrastive regularization objective that promotes separability of ID and OOD samples in a chosen score space, such as Mahalanobis or energy-based. Experiments demonstrate that GCOS outperforms state-of-the-art methods using standard energy-based inference on near-OOD benchmarks, defined as tasks where outliers share the same semantic domain as in-distribution data. As an exploratory extension, the framework naturally transitions to conformal OOD inference, which translates uncertainty scores into statistically valid p-values and enables thresholds with formal error guarantees, providing a pathway toward more predictable and reliable OOD detection.

Figures & tables

Figure 1: GCOS Training Procedure Schematic. Illustration of the data flow in our online synthesis and regularization method. Epoch-level calibration on $\mathcal{D}_{calib}$ produces class-conditional subspace models $\mathcal{M}_{calib}$ and Mahalanobis quantiles $q$ . During batch-level training, features are used to update a queue that generates proposer subspace models $\mathcal{M}_{train}$ and identifies off-manifold directions $v$ . Outliers $\mathbf{z}_{ood}$ are synthesized to match the target quantiles $q$ , as evaluated by $\mathcal{M}_{calib}$ . The final regularization loss, $\mathcal{L}_{reg}$ , is a contrastive objective computed on the energy scores of in-distribution batch features and the synthesized outliers $\mathbf{z}_{ood}$ and added to cross-entropy loss, $\mathcal{L}_{CE}$ , from the main classification task.

	C-MNIST			Dogs			MVTec			Retino
Method	AUROC	AUPR	FPR95	AUROC	AUPR	FPR95	AUROC	AUPR	FPR95	AUROC	AUPR	FPR95	Avg. AUC
No Reg.	85.60	99.47	32.50	88.35	97.72	80.00	94.86	98.91	27.69	69.74	98.43	81.50	84.64
VOS ( Du et al., 2022 )	94.71	99.87	18.50	99.25	99.85	5.00	80.37	95.83	70.77	70.52	98.56	80.00	86.21
MSP ( Hendrycks and Gimpel, 2017 )	92.10	89.88	21.12	56.15	13.81	96.96	56.58	34.72	82.20	67.02	94.09	90.52	67.96
MaxLogit ( Hendrycks et al., 2022 )	88.88	85.68	27.59	58.42	14.27	93.69	55.30	39.20	90.24	77.21	96.41	81.39	69.95
ReAct ( Sun et al., 2021 )	89.41	86.60	24.69	57.77	14.08	94.04	57.31	36.74	87.09	77.37	96.43	75.84	70.47

Table 1: OOD detection performance across four datasets. Best values per column shown in bold.

Figure 2: UMAP Projection of Learned Features. Top: the overall feature space, showing that classes form varying shapes and are largely well separated. Bottom: a zoomed-in view highlighting the distribution of GCOS outliers in off-manifold regions and VOS outliers near cluster edges for two clusters. The panels illustrate how GCOS generates points in challenging regions beyond the main clusters, while VOS outliers remain close to class boundaries.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

	MVTec			Colored MNIST			Stanford Dogs			Retinopathy
	AUROC	AUPR	FPR95	AUROC	AUPR	FPR95	AUROC	AUPR	FPR95	AUROC	AUPR	FPR95
GCOS + $\mathcal{L}_{\text{uncertainty}}$	50.92	83.68	100.00	98.75	99.97	2.00	50.00	83.33	100.00	67.60	98.37	77.00
GCOS + $\mathcal{L}_{reg}$ (Mahalanobis)	63.67	88.07	100.00	98.93	99.98	1.50	50.00	83.33	100.00	37.58	96.27	97.00
GCOS + $\mathcal{L}_{reg}$ (Energy)	70.62	88.50	72.31	76.00	97.85	34.00	2.40	65.43	100.00	31.11	95.30	98.00

Appendix

Table 2: Results using formal hypothesis testing ( 7 ) as OOD classification head.

Dataset	ID ${}_{\text{train}}$	ID ${}_{\text{test}}$	OOD	Total
Colored MNIST	60,000	10,000	10,000	80,000
MVTec	1,319	326	635	2,280
Stanford Dogs	1,394	100	856	2,350
Retinopathy	28,086	7,022	865	35,973

Appendix

Table 3: Number of ID and OOD images in each dataset.

Panel of Figure 4 — Figure 4: Example images from four datasets. First row in each box: in-distribution; second row: outliers.

		CIFAR-100			LSUN-C			Places			Textures			Mean
		AUPR	AUROC	FPR95	AUPR	AUROC	FPR95	AUPR	AUROC	FPR95	AUPR	AUROC	FPR95	AUPR	AUROC	FPR95
Gaussian OOD	+ $\mathcal{L}_{\text{uncertainty}}$	99.61	87.08	49.0	99.97	98.56	6.5	99.66	89.1	39.5	99.68	88.34	49.0	99.73	90.77	36.0
GCOS OOD	+ $\mathcal{L}_{\text{uncertainty}}$	99.48	83.63	56.5	99.97	98.74	7.0	99.6	88.36	42.5	99.48	82.39	60.5	99.64	88.28	41.63
GCOS OOD	+ $\mathcal{L}_{reg}$	99.54	82.48	72.5	99.96	98.12	8.5	99.82	92.3	41.5	99.76	90.19	50.5	99.77	90.77	43.25

Appendix

Table 4: Comparison of out-of-distribution detection performance trained on CIFAR-10.

		CIFAR-10			LSUN-C			Places			Textures			Mean
		AUPR	AUROC	FPR95	AUPR	AUROC	FPR95	AUPR	AUROC	FPR95	AUPR	AUROC	FPR95	AUPR	AUROC	FPR95
Gaussian OOD	+ $\mathcal{L}_{\text{uncertainty}}$	99.35	76.47	80.0	99.92	96.54	21.5	99.32	75.51	84.0	99.3	74.53	84.5	99.47	80.76	67.5
GCOS OOD	+ $\mathcal{L}_{\text{uncertainty}}$	99.33	76.13	83.5	99.94	97.1	17.0	99.37	77.18	81.0	99.23	72.96	87.5	99.47	80.84	67.25
GCOS OOD	+ $\mathcal{L}_{reg}$	99.16	72.53	89.5	99.79	91.63	38.0	99.16	72.95	84.5	99.3	76.79	78.0	99.35	78.47	72.5

Appendix

Table 5: Comparison of out-of-distribution detection performance trained on CIFAR-100.

	Textures		Places365
	AUROC	FPR95	AUROC	FPR95
Free Energy ( Liu et al., 2020 )	85.35	52.46	90.02	40.11
ASH ( Djurisic et al., 2023 )	86.07	50.90	89.79	40.89
VOS ( Du et al., 2022 )	88.34	49.00	89.10	39.50
GCOS + $\mathcal{L}_{reg}$	90.19	50.50	92.3	41.50

Appendix

Table 6: Results on CIFAR-10.

	Textures		Places365
	AUROC	FPR95	AUROC	FPR95
Free Energy ( Liu et al., 2020 )	76.35	79.63	75.65	80.18
ASH ( Djurisic et al., 2023 )	83.59	63.69	74.87	79.70
VOS ( Du et al., 2022 )	74.53	84.50	75.51	84.00
GCOS + $\mathcal{L}_{reg}$	76.79	78.00	72.95	84.50

Appendix

Table 7: Results on CIFAR-100.

Dataset	Mean AUROC	Mean AUPR	Mean FPR95	Std(AUROC)	Std(AUPR)
Colored MNIST	92.90%	99.81%	23.50%	$\pm$ 3.01%	$\pm$ 0.10%
Stanford Dogs	97.39%	99.51%	24.00%	$\pm$ 2.11%	$\pm$ 0.39%
Retinopathy	76.60%	99.05%	80.00%	$\pm$ 0.55%	$\pm$ 0.02%
MVTec	98.71%	99.66%	3.08%	$\pm$ 0.29%	$\pm$ 0.12%

Appendix

Table 8: Multi-Seed Validation.

Method	FPR95 $\downarrow$	AUROC $\uparrow$	mAP (ID) $\uparrow$
	OOD: MS-COCO / OpenImages
MSP ( Hendrycks and Gimpel, 2017 )	70.99 / 73.13	83.45 / 81.91	48.7
ODIN ( Liang et al., 2018 )	59.82 / 63.14	82.20 / 82.59	48.7
Mahalanobis ( Lee et al., 2018b )	96.46 / 96.27	59.25 / 57.42	48.7
Energy score ( Liu et al., 2020 )	56.89 / 58.69	83.69 / 82.98	48.7
Gram matrices ( Sastry and Oore, 2020 )	62.75 / 67.42	79.88 / 77.62	48.7

Appendix

Table 9: OOD detection comparison on PASCAL-VOC (In-Distribution) with MS-COCO and OpenImages as OOD datasets.

$K$	PCA (ms)	Shell Calib (ms)	Synthesis (ms)	Total (ms)
10	16.7	2.3	35.8	54.8
20	33.6	4.4	71.3	109.3
50	83.7	11.4	183.8	278.9
100	165.4	23.1	379.9	568.3
200	332.1	46.3	764.8	1143.3
1000	1663.7	239.6	3845.3	5748.6

Appendix

Table 10: Per-batch GCOS overhead by number of classes

K

. NVIDIA RTX 6000 Ada Generation; 100-iteration average.

OOD Dataset	GCOS AUROC	VOS AUROC	GCOS FPR@95	VOS FPR@95
iNaturalist	91.6	90.8	41.5	46.0
SUN	94.7	94.6	24.0	28.5
Places	95.0	94.7	22.5	22.5
Textures	95.9	96.4	22.5	16.5
Mean	94.3	94.1	27.6	28.4

Appendix

Table 11: OOD detection on ImageNet-100.

	MVTec			Colored MNIST			Stanford Dogs			Retinopathy
	AUROC	AUPR	FPR95	AUROC	AUPR	FPR95	AUROC	AUPR	FPR95	AUROC	AUPR	FPR95
GCOS + $\mathcal{L}_{\text{uncertainty}}$	93.07	98.56	33.85	93.21	99.82	25.50	99.42	99.89	0.86	80.05	99.23	71.00
GCOS + $\mathcal{L}_{reg}$ (Mahalanobis)	97.56	99.51	20.00	95.65	99.91	25.00	97.90	99.61	20.00	77.89	99.17	79.00

Appendix

Table 12: Ablation study of proposed contributions . OOD detection results on MVTec, Colored MNIST, Stanford Dogs and Retinopathy datasets. First row: GCOS outliers combined with the standard energy-based separation loss

\mathcal{L}_{\text{uncertainty}}

in ( 2 ). Second row: GCOS outliers combined with the geometric loss

\mathcal{L}_{\text{reg}}

in ( 5 ) using the Mahalanobis score function

\mathcal{S}_{\mathcal{L}}=\mathcal{S}_{\text{Mahal}}

Figure 5: Hyperparameter ablation study.

OOD Detection

Out-of-distribution (OOD) detection identifies inputs unlike a model’s training data so systems can flag uncertainty, reject inputs, or adapt safely.

See all 223 papers

Learning to Explore: Policy-Guided Outlier Synthesis for Graph Out-of-Distribution Detection
Feb 28, 2026Li Sun, Lanxu Yang, Jiayu Tian +6OOD Detection Synthetic Anomaly Generation
North China Electric Power University · Beijing University of Posts and Telecommunications · Tsinghua University +4
GAMR: Geometric-Aware Manifold Regularization with Virtual Outlier Synthesis for Learning with Noisy Labels
May 20, 2026Ningkang Peng, Jingyang Mao, Xiaoqian Peng +4Geometric Representation Learning Manifold Learning
Nanjing Normal University Nanjing, China · Nanjing University of Chinese Medicine Nanjing, China
Out-of-Distribution Object Detection in Street Scenes via Synthetic Outlier Exposure and Transfer Learning
Mar 17, 2026Sadia Ilyas, Annika Mütze, Klaus Friedrichs +2Object Detection OOD Detection
University of Wuppertal · Aptiv Services Deutschland GmbH · University of Osnabrück, Germany

Feb 28, 2026J/K move · Enter open · S save

Learning to Explore: Policy-Guided Outlier Synthesis for Graph Out-of-Distribution Detection

Li Sun, Lanxu Yang, Jiayu Tian, Bowen Fang+5

North China Electric Power University · Beijing University of Posts and Telecommunications · Tsinghua University+4

OOD Detection Synthetic Anomaly Generation

Open paper AskPDF arXiv

Detecting out-of-distribution (OOD) graphs is crucial for ensuring the safety and reliability of Graph Neural Networks. In unsupervised graph-level OOD detection, models are typically trained using only in-distribution (ID) data, resulting in incomplete feature space characterization and weak decision boundaries. Although synthesizing outliers offers a promising solution, existing approaches rely on fixed, non-adaptive sampling heuristics (e.g., distance- or density-based), limiting their ability to explore informative OOD regions. We propose a Policy-Guided Outlier Synthesis (PGOS) framework that replaces static heuristics with a learned exploration strategy. Specifically, PGOS trains a reinforcement learning agent to navigate low-density regions in a structured latent space and sample representations that most effectively refine the OOD decision boundary. These representations are then decoded into high-quality pseudo-OOD graphs to improve detector robustness. Extensive experiments demonstrate that PGOS achieves state-of-the-art performance on multiple graph OOD and anomaly detection benchmarks.

Abstract

OOD Detection

Explore similar work

Learning to Explore: Policy-Guided Outlier Synthesis for Graph Out-of-Distribution Detection

GAMR: Geometric-Aware Manifold Regularization with Virtual Outlier Synthesis for Learning with Noisy Labels

Out-of-Distribution Object Detection in Street Scenes via Synthetic Outlier Exposure and Transfer Learning