Revisiting GAN with Bayes-Optimal Discrimination

Mohammadreza Tavasoli Naeini; Ali Bereyhi; Morteza Noshad; Ben Liang; Alfred O. Hero

cs.LGOct 29, 2025

Revisiting GAN with Bayes-Optimal Discrimination

Authors: Mohammadreza Tavasoli Naeini, Ali Bereyhi, Morteza Noshad, Ben Liang, Alfred O. Hero

Organizations: University of Toronto · Stanford University · University of Michigan

Abstract

We approach generative adversarial network (GAN) training from its original real-versus-fake classification viewpoint, replacing the cross-entropy discriminator objective with an upper bound on the Bayes error rate (BER). Using the Bayes optimal learning threshold (BOLT) loss, we formulate an adversarial game in which the discriminator tightens this bound and the generator maximizes it. We characterize the unrestricted optimal discriminator and show that, under balanced class priors, the population game is equivalent to minimizing total variation between the data and generator distributions. Restricting the bounded discriminator to be 1-Lipschitz yields BOLT-GAN. For balanced priors and the full discriminator class, we establish upper and lower bounds relating its population discrepancy to Wasserstein-1 distance on bounded domains. Controlled experiments on four image-generation benchmarks show improved sample quality and coverage over WGAN-GP under matched architectures, optimization settings, and gradient penalties. At a common 20-epoch budget, BOLT-GAN reduces Fréchet Inception Distance (FID) from 60.0 to 44.2 on CIFAR-10 and from 102.5 to 28.4 on LSUN Bedroom-64.

Figures & tables

(a) Unconstrained versus GP-regularized BOLT
Dataset	Max-BER FID	BOLT-GAN FID
CIFAR-10	$>300$	$\mathbf{44.2}$
CelebA-64	$>300$	$\mathbf{9.2}$
LSUN Bedroom-64	$>400$	$\mathbf{28.4}$
(b) Real/fake prior sensitivity
Prior $\pi$	FID at $20$ epochs	FID at $100$ epochs

Table 1: Discriminator-continuity and hyperparameter comparisons, separated by purpose. Lower FID is better. Panel (a) uses the same

20

-epoch budget; panels (b,c) use CIFAR-10. Panel (b) reports mean

\pm

SD over three seeds. The default

\lambda_{\rm GP}=10

is retained in Table 2 , independently of the sweep optimum.

(a) Common architecture, optimizer, schedule, and GP
	FID ( $\downarrow$ )		P ( $\uparrow$ )		R ( $\uparrow$ )
Dataset	WGAN-GP	BOLT-GAN	WGAN-GP	BOLT-GAN	WGAN-GP	BOLT-GAN
CIFAR-10	$60.0$	$\mathbf{44.2}$	$.714$	$\mathbf{.763}$	$.183$	$\mathbf{.352}$
LSUN Church-64	$43.5$	$\mathbf{14.8}$	$.681$	$\mathbf{.750}$	$.176$	$\mathbf{.411}$
LSUN Bedroom-64	$102.5$	$\mathbf{28.4}$	$.388$	$\mathbf{.598}$	$.098$	$\mathbf{.450}$
CelebA-64	$10.3$	$\mathbf{9.2}$	$.672$	$\mathbf{.840}$	$.104$	$\mathbf{.548}$

Table 2: Fixed-budget comparisons at

20

epochs. Panel (a) holds GP and its coefficient

\lambda_{\rm GP}=10

fixed. Precision (P) and Recall (R) use

n=2000

k=3

. Panel (b) presents HINGE-GAN with spectral normalization (SN). Entries are point estimates.

(a) Discriminator changes: FID ( $\downarrow$ )
Dataset	Discriminator	WGAN-GP	BOLT-GAN
Tiny ImageNet	Residual CNN	$74.8$	$\mathbf{68.9}$
Tiny ImageNet	ViT-style	$70.6$	$\mathbf{64.1}$
STL-10	Residual CNN	$49.7$	$\mathbf{44.8}$
STL-10	ViT-style	$45.5$	$\mathbf{40.9}$

Table 3: Additional experiments separated by purpose. Arrows specify the preferred metric direction. Entries are point estimates unless marked otherwise. The conditional results use the final checkpoint at epoch

50

; the best checkpoints are reported separately in Appendix G.7 .

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 3 — Figure 3: Representative BOLT-GAN samples after $20$ epochs on CIFAR-10, CelebA-64, LSUN Bedroom-64, and LSUN Church-64.

Panel of Figure 4 — Figure 4: Gradient-penalty sensitivity for BOLT-GAN . Curves show mean FID trajectories for different $\lambda_{\rm GP}$ .

Dataset	Epochs	Method	Precision	Recall
CIFAR-10	20	BOLT-GAN	$\mathbf{0.763}$	$\mathbf{0.352}$
	20	WGAN-GP	$0.714$	$0.183$
	100	BOLT-GAN	$\mathbf{0.812}$	$\mathbf{0.469}$
	100	WGAN-GP	$0.764$	$0.102$
LSUN Church-64	20	BOLT-GAN	$\mathbf{0.750}$	$\mathbf{0.411}$
	20	WGAN-GP	$0.681$	$0.176$

Appendix

Table 4: Improved Precision/Recall (

\uparrow

) with

n=2000

k=3

Panel of Figure 5 — Figure 5: Synthetic 25-Gaussians: mode-coverage and histogram-discrepancy trajectories with representative samples. A mode is counted when at least one generated sample lies within radius $0.15$ of its center; the histogram discrepancy compares empirical bin masses.

Grid	Mean histogram TV	Sample SD	Central $95\%$ of null draws
$20\times 20$	0.0556	0.0060	$[0.0442,0.0680]$
$40\times 40$	0.0829	0.0060	$[0.0714,0.0948]$
$80\times 80$	0.1368	0.0061	$[0.1252,0.1494]$
$160\times 160$	0.2445	0.0061	$[0.2330,0.2570]$

Appendix

Table 5: Null histogram-TV diagnostic for identical target distributions. SD is computed across

1{,}000

repetitions; the range contains the central

95\%

of simulated values.

Ratio	$M$	Empirical mean	$M\widehat{\operatorname{Var}}$	$M\operatorname{Var}$	$M$ bound
Oracle ratio	128	0.15902	0.03417	0.03292	0.06250
Oracle ratio	512	0.15872	0.03259	0.03292	0.06250
Oracle ratio	2048	0.15872	0.03200	0.03292	0.06250
Oracle ratio	8192	0.15866	0.03267	0.03292	0.06250
$1.2\,U$	128	0.17314	0.03401	0.03495	0.06250
$1.2\,U$	512	0.17353	0.03546	0.03495	0.06250

Appendix

Table 6: Gaussian plug-in experiment:

4{,}000

independent evaluation repetitions per row, with each ratio held fixed. Empirical variance uses the sample-variance convention. Exact moments and the theorem bound are shown separately.

Ratio	Exact conditional mean	Absolute bias	$q_{1}\varepsilon_{0}$
Oracle ratio	0.158655	0.000000	0.000000
$1.2\,U$	0.173419	0.014764	0.100000
Fitted Gaussian ratio	0.160628	0.001972	0.100141

Appendix

Table 7: Exact conditional bias and the

q_{1}\varepsilon_{0}

bound in the same Gaussian study. Ratio errors are evaluated analytically for these exponential-affine Gaussian models.

Regularizer (BOLT objective fixed)	FID@10k $\downarrow$
None ( Max-BER )	$\sim 380$ (diverges)
Weight clipping	$96.4$
SN only	$51.2$
GP only (paper default)	$\mathbf{43.7}$
SN + GP ( $\lambda_{\rm GP}{=}2$ )	$44.0$

Appendix

Table 8: CIFAR-10 ablation of Lipschitz enforcement with the BOLT objective fixed. FID@10k is measured after

30

epochs and averaged over

3

seeds; lower is better.

Panel of Figure 6 — Figure 6: Class-conditional CIFAR-10 with a projection discriminator. Panel (a) shows that BOLT-GAN reaches a lower FID@10k than HINGE-GAN . Panels (b) and (c) show class-conditional samples from the same projection-discriminator architecture, with one row per class. WGAN-GP is omitted because it diverges under matched hyperparameters (D loss $\to{-}10^{6}$ ; see text).

TV Distance Computation

See all 8 papers

Population-Based Multi-Objective Training of Discriminators for Semi-Supervised GANs
Jul 2, 2026Francisco Sedeño, Francisco Chicano, Jamal ToutouhEvolutionary Optimization Semi-Supervised Learning
ITIS Software, University of Malaga Malaga, Spain · CSAIL MIT, Cambridge, MA, USA
EmbeddGAN: A Novel GAN Framework Using an Embedding Network and Gini Distance Correlation
Sep 18, 2026MaTais Caldwell, Yixin Chen, Xin Dang +1Generative Adversarial Networks Adversarial Training
Department of Computer and Information Science University of Mississippi Oxford, MS 38677 USA · Department of Mathematics University of Mississippi Oxford, MS 38677 USA
Deciding When to Switch: E-Processes for Adaptive Minimax Training for Generative Adversarial Nets
Aug 10, 2026Hyunjoo Kim, Sicheng Wu, Agastya Venkatraman +2Min-Max Optimization Sequential Hypothesis Testing
Department of Statistics, Ewha Womans University · Department of Mathematics, Purdue University · Department of Computer Science, Purdue University

Jul 2, 2026J/K move · Enter open · S save

Population-Based Multi-Objective Training of Discriminators for Semi-Supervised GANs

Francisco Sedeño, Francisco Chicano, Jamal Toutouh

ITIS Software, University of Malaga Malaga, Spain · CSAIL MIT, Cambridge, MA, USA

Evolutionary Optimization Semi-Supervised Learning Multi-Objective Optimization Generative Adversarial Networks

Open paper AskPDF arXiv

Semi-supervised generative adversarial networks (SSL-GANs) can exploit large unlabeled datasets while retaining a classifier in the discriminator, but their training is often unstable. This paper proposes a population-based evolutionary training strategy in which discriminator learning is formulated as a multi-objective optimization problem. Instead of aggregating the supervised and unsupervised components of the SSL objective into a single scalar loss, the method maintains a population of discriminators ranked by Pareto dominance, enabling the exploration of different trade-offs between classification accuracy and real/fake discrimination. This formulation aims to improve both roles of SSL-GANs: learning accurate classifiers and training generators capable of producing realistic samples. We analyze several variants, including an elitist strategy and a mono-objective ablation, to assess the role of multi-objective selection. Experiments on MNIST with limited labels show improved training robustness compared to SSL-GAN and CE-SSL-GAN state-of-the-art baselines, while the elitist variant consistently achieves the highest classification accuracy.

Abstract

TV Distance Computation

Explore similar work

Population-Based Multi-Objective Training of Discriminators for Semi-Supervised GANs

EmbeddGAN: A Novel GAN Framework Using an Embedding Network and Gini Distance Correlation

Deciding When to Switch: E-Processes for Adaptive Minimax Training for Generative Adversarial Nets