Exposing Diversity Bias in Deep Generative Models: Statistical Origins and Correction of Diversity Error

Farzan Farnia; Mohammad Jalali; Azim Ospanov

cs.LGFeb 16, 2026

Exposing Diversity Bias in Deep Generative Models: Statistical Origins and Correction of Diversity Error

Authors: Farzan Farnia, Mohammad Jalali, Azim Ospanov

Organizations: Department of Computer Science and Engineering, The Chinese University of Hong Kong.

Abstract

Deep generative models have achieved great success in producing high-quality samples, making them a central tool across machine learning applications. Beyond sample quality, an important yet less systematically studied question is whether trained generative models faithfully capture the diversity of the underlying data distribution. In this work, we address this question by directly comparing the diversity of samples generated by state-of-the-art models with that of test samples drawn from the target data distribution, using recently proposed reference-free entropy-based diversity scores, Vendi and RKE. Across multiple benchmark datasets, we find that test data consistently attains substantially higher Vendi and RKE diversity scores than the generated samples, suggesting a systematic downward diversity bias in modern generative models. To understand the origin of this bias, we analyze the finite-sample behavior of entropy-based diversity scores and show that their expected values increase with sample size, implying that diversity estimated from finite training sets could inherently underestimate the diversity of the true distribution. As a result, optimizing the generators to minimize divergence to empirical data distributions would induce a loss of diversity. Finally, we discuss potential diversity-aware regularization and guidance strategies based on Vendi and RKE as principled directions for mitigating this bias, and provide empirical evidence suggesting their potential to improve the results.

Figures & tables

Figure 1: Numerical evaluation of Vendi scores at different sample sizes (averaged over 100 independent random trials with confidence interval) for the validation set of ImageNet, the pre-trained latent diffusion model (LDM) of DiT-XL-2, and the SPRAKE-guided LDM with entropy diversity regularization proposed in ( Jalali et al., 2025 ) . The entropy regularization in the SPARKE Guidance not only improves the Vendi and Coverage diversity scores, but further improves the FD and KD overall evaluation scores.

Figure 2: Exponential of Shannon entropy vs. sample size

N

for varying alphabet sizes

R

. Error bars represent 95% confidence intervals calculated over 10 independent trials.

Figure 3: Vendi score curves (mean and confidence intervals over 10 independent sample sets) for ImageNet, FFHQ, and MSCOCO (sample size values of

n\leq 20\text{K}

), computed using DINOv2 embeddings and a Gaussian (RBF) kernel with bandwidth

\sigma=35

. The Vendi scores continue to increase at a significant rate across all sample sizes below the 20000-computational-feasible size limit for exact Vendi computation.

Figure 4: Comparison of Vendi scores of the test sample set (the dashed red curve) and the generated samples by pre-trained generative models across four datasets. The backbone embedding is DINOv2 embeddings using Gaussian (RBF) kernel with bandwidth

\sigma=35

Figure 5: Entropy level sets in distribution space. $\widehat{P}_{n}$ and $Q_{{\theta}^{*}}$ lie in a lower-entropy region; $P_{\mathrm{data}}$ lies higher. Projection onto the entropy superlevel set yields $Q_{{\theta}^{\mathrm{proj}}}$ , closer to $P_{\mathrm{data}}$ within the model family.

Figure 6: Vendi Score comparison across varying training set sizes. Results are shown for StyleGAN-XL (shaded red) and LDMs (shaded blue), with the training dataset baseline indicated by a dashed red line. Corresponding RKE, Recall, and Coverage metrics are also presented.

Figure 7: Comparison of the diffusion model (DDIM) with RKE regularized DDIM on an underlying 2D-Gaussian mixture model.

Figure 8: Vendi and RKE diversity scores vs. size $N$ for $N$ uniformly weighted generated samples and their post-hoc Vendi-projected reweighting. (Upper) LDM trained on FFHQ. (Lower) BigGAN trained on ImageNet. The Vendi-based projection yields higher diversity scores for the reweighted empirical distribution.

	Model	Sampling	Vendi $\uparrow$	RKE $\uparrow$	KD $\downarrow$	FD $\downarrow$
FFHQ	ProjectedGAN	Uniform	193.64	9.31	1.958	586.43
		Projected-Vendi	237.34	10.88	1.642	562.92
	LDM	Uniform	210.32	10.21	0.587	223.32
		Projected-Vendi	248.01	11.92	0.462	213.87
ImageNet	BigGAN	Uniform	885.68	32.56	0.539	403.57
		Projected-Vendi	947.32	33.27	0.517	386.34

Table 1: Evaluated scores for comparison of Uniform sampling vs. Vendi-Projected sampling.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 9: Vendi Score convergence for

d

-dimensional Gaussian Mixtures. The score is evaluated across varying sample sizes

n

for

2^{d}

isotropic Gaussians (

\sigma_{\text{std}}=10^{-4}

) centered on the vertices of a

d

-dimensional cube. We set Gaussian kernel bandwidth

\sigma

to 0.1. Error bars represent 95% confidence intervals calculated over 10 independent trials.

Figure 10: Vendi score curves (mean and 95% confidence intervals over 5 independent sample sets) for ImageNet, FFHQ, MSCOCO, LSUN Bedroom (sample size values of $n\leq 20\text{K}$ ) and AFHQ-v2 (sample size values $n\leq 16000$ ), computed using DINOv2 embeddings and a Gaussian (RBF) kernel with bandwidth $\sigma=35$ .

Panel of Figure 11 — Figure 11: Comparison of Vendi scores of the test sample set (the dashed red curve) and the generated samples by pre-trained generative models across three datasets. The backbone embedding is DINOv2 embeddings using Gaussian kernel with bandwidth $\sigma=35$ .

Panel of Figure 12 — Figure 12: Comparison of Vendi scores of the test sample set (the dashed red curve) and the generated samples by pre-trained generative models across three datasets. The backbone embedding is CLIP embeddings using Gaussian kernel with bandwidth $\sigma=8.0$ .

Panel of Figure 13 — Figure 13: Comparison of Vendi scores of the test sample set (the dashed red curve) and the generated samples by pre-trained generative models across three datasets. The backbone embedding is InceptionV3 embeddings using Gaussian kernel with bandwidth $\sigma=9.0$ .

Panel of Figure 14 — Figure 14: Comparison of Vendi scores of the test sample set (the dashed red curve) and the generated samples by pre-trained generative models across three datasets. The backbone embedding is DINOv3-ViTS16 embeddings using Gaussian kernel with bandwidth $\sigma=8.0$ .

Figure 15: Comparison of generative model architectures across four datasets. We evaluate models on ImageNet, FFHQ, LSUN Bedroom, and MSCOCO using DINOv2 embeddings with cosine similarity kernel.

Method	Recall $\uparrow$	Coverage $\uparrow$	RKE $\uparrow$	Vendi $\uparrow$
Dataset	-	-	36.82	2533.93
ADM Dhariwal and Nichol [2021]	0.79	0.89	29.74	2279.73
BigGAN Brock et al. [2018]	0.44	0.57	26.91	1655.41
GigaGAN Kang et al. [2023]	0.74	0.70	30.59	2174.02
LDM Rombach et al. [2022]	0.76	0.93	34.13	2155.19
RQ-Transformer Lee et al. [2022]	0.76	0.59	29.23	2214.47

Appendix

Table 2: Comparison of RKE, Coverage, and Recall across different model architectures trained on ImageNet. Coverage and Recall metrics were computed using Stein et al. [2023] implementation. Best model scores are underlined.

Method	Recall $\uparrow$	Coverage $\uparrow$	RKE $\uparrow$	Vendi $\uparrow$
Dataset	-	-	14.46	512.84
LDM Rombach et al. [2022]	0.44	0.74	10.21	295.47
Eff. vdVAE Hazami et al. [2022]	0.14	0.52	7.21	153.45
InsGen Yang et al. [2021]	0.14	0.51	10.38	291.21
Projected GAN Sauer et al. [2021]	0.07	0.30	9.34	260.41
StyleGAN XL Sauer et al. [2022]	0.42	0.61	10.53	343.63

Appendix

Table 3: Comparison of RKE, Coverage, and Recall across different model architectures trained on FFHQ. Coverage and Recall metrics were computed using Stein et al. [2023] implementation. Best model scores are underlined.

Method	Recall $\uparrow$	Coverage $\uparrow$	RKE $\uparrow$	Vendi $\uparrow$
Dataset	-	-	11.60	274.90
ADMNet Dropout Dhariwal and Nichol [2021]	0.75	0.90	10.23	226.85
DDPM Ho et al. [2020]	0.61	0.68	9.61	239.00
Diffusion-Projected GAN Wang et al. [2022]	0.28	0.28	8.31	204.06
iDDPM Nichol and Dhariwal [2021]	0.64	0.76	9.61	224.55
Projected GAN Sauer et al. [2021]	0.22	0.24	8.12	199.45

Appendix

Table 4: Comparison of RKE, Coverage, and Recall across different model architectures trained on LSUN-Bedroom. Coverage and Recall metrics were computed using Stein et al. [2023] implementation. Dataset scores are bolded, best model scores are underlined.

Figure 16: Vendi Score comparison for UNet, StyleGAN-XL, LDM. The models were trained on varying subsets of ImageNet, ImageNet, FFHQ respectively (indicated by fraction of the original size). Dashed red line represents score of the corresponding training dataset. The evaluation results are done using DINOv2 with

\sigma=35

Figure 17: Vendi Score comparison for UNet, StyleGAN-XL, LDM. The models were trained on varying subsets of ImageNet, ImageNet, FFHQ respectively (indicated by fraction of the original size). Dashed red line represents score of the corresponding training dataset. The evaluation results are done using CLIP with

\sigma=8.0

Method	Recall $\uparrow$	Coverage $\uparrow$	RKE $\uparrow$	Vendi $\uparrow$
Dataset	-	-	36.82	2533.93
StyleGAN-XL: full dataset	0.74	0.96	31.27	2203.39
StyleGAN-XL: 1/2 dataset	0.59	0.94	20.80	1432.20
StyleGAN-XL: 1/10 dataset	0.42	0.93	14.95	255.44

Appendix

Table 5: Comparison of RKE, Coverage, and Recall across StyleGAN-XL models trained on varying sizes of ImageNet.

Method	Recall $\uparrow$	Coverage $\uparrow$	RKE $\uparrow$	Vendi $\uparrow$
Dataset	-	-	14.46	514.02
LDM: full dataset	0.44	0.74	10.21	294.60
LDM: 1/2 dataset	0.22	0.74	9.17	260.64
LDM: 1/4 dataset	0.03	0.52	7.06	170.44

Appendix

Table 6: Comparison of RKE, Coverage, and Recall across LDM models trained on varying sizes of FFHQ.

Figure 18: Comparison of DDIM with and without RKE regularization in two 2D GMM distributions: (1) modes are placed in a circular pattern, (2) modes are placed in a grid.

Method	Vendi	RKE	KD $\times$ 10 2	FD	Precision	Recall	Density	Coverage
Dataset	16.49	12.86	-	-	-	-	-	-
JS-GAN+RKE Reg.	15.20	12.18	0.098	0.081	0.98	0.89	1.02	0.78
JS-GAN	12.61	10.92	0.767	0.654	0.98	0.83	0.99	0.49

Appendix

Table 7: Comparison of Vendi scores, RKE, KD, and FD across different sampling settings. on 2D Gaussian Mixtures with 10 components.

Figure 19: Comparison of JS-GAN generated GMM modes with and without diversity regularization during training. We show two mode placement examples: (1) modes are in a circular pattern, (2) modes are in the $4\times 4$ grid pattern.

Figure 20: Qualitative Comparison of samples generated by DiT-XL-2 and DiT-XL-2 with Vendi guidance (entropy regularization).

Method	Vendi $\uparrow$	RKE $\uparrow$	KD $\downarrow$	FD $\downarrow$	Precision $\uparrow$	Recall $\uparrow$	Density $\uparrow$	Coverage $\uparrow$
Dataset	1747.60	36.37	-	-	-	-	-	-
Vendi guidance	672.24	33.94	0.1628	328.16	0.95	0.15	1.63	0.73
RKE guidance	751.09	34.50	0.1629	319.56	0.96	0.16	1.54	0.74
No guidance	603.68	33.71	0.2174	422.15	0.94	0.11	1.50	0.64

Appendix

Table 8: Comparison of Vendi scores, RKE, KD, and FD across different sampling settings. on DiT (trained on ImageNet) using DINOv2 feature embeddings. (

\sigma=35

)

Generative Modeling

Generative modeling learns data distributions to create new samples, from images and text to molecules, simulations, and structured records.

See all 487 papers

Mine, Sample, Render: Test-Time Diversity Alignment Of Generative Models
Oct 15, 2026Kostas Triaridis, Morris E. Sharp, Nebojsa JojicGenerative Modeling Diversity-Aware Sampling
Stony Brook University · Microsoft
Conditional Vendi Score: Prompt-Aware Diversity Evaluation for Generative AI Models and LLMs
Nov 5, 2024Mohammad Jalali, Azim Ospanov, Amin Gohari +1Generative AI Evaluation Language Model Generation Evaluation
Department of Computer Science and Engineering, The Chinese University of Hong Kong · Department of Information Engineering, The Chinese University of Hong Kong
Imaginative Generative AI: Crossing the Entropy Wall into Worlds Beyond Imitation
Aug 10, 2026Farzan Farnia, Hossein Goli, Amin GohariConstrained Generative Modeling Diffusion Model Guidance
Department of Computer Science and Engineering, The Chinese University of Hong Kong · Department of Information Engineering, The Chinese University of Hong Kong

Oct 15, 2026J/K move · Enter open · S save

Mine, Sample, Render: Test-Time Diversity Alignment Of Generative Models

Kostas Triaridis, Morris E. Sharp, Nebojsa Jojic

Stony Brook University · Microsoft

Generative Modeling Diversity-Aware Sampling Diverse Text Generation Constrained Generative Modeling

Open paper AskPDF arXiv

Modern generative models produce high-quality individual outputs but struggle to sample diversely, even when explicitly instructed to do so. We introduce MINE-SAMPLE-RENDER (MSR), a test-time approach that externalizes the diversity into an explicit, corpus-derived sampling procedure. Our pipeline (i) mines an interpretable categorical schema from a corpus of real artifacts and supplies statistics for sampling; (ii) samples attribute configurations from the corpus-derived statistics; (iii) uses a generative model purely as a conditional renderer of a sampled configuration. Across weather report generation, user simulation, webpage design, text-to-image generation, and agent-generated 3D assets, we evaluate the diversity and output quality of MSR-guided generation and show that MSR improves alignment with corpus distributions and diversity of generated outputs.

Abstract

Generative Modeling

Explore similar work

Mine, Sample, Render: Test-Time Diversity Alignment Of Generative Models

Conditional Vendi Score: Prompt-Aware Diversity Evaluation for Generative AI Models and LLMs

Imaginative Generative AI: Crossing the Entropy Wall into Worlds Beyond Imitation