Sample-Optimal Estimation of the Fréchet Inception Distance

Ziyun Chen; Jerry Li; Kevin Tian; Yusong Zhu

cs.LGOct 5, 2026

Sample-Optimal Estimation of the Fréchet Inception Distance

Authors: Ziyun Chen, Jerry Li, Kevin Tian, Yusong Zhu

Organizations: University of Washington · University of Texas at Austin

Abstract

The Fréchet Inception Distance (FID) is widely used to evaluate generative models, but its empirical plug-in estimator suffers from finite-sample bias [BSAG18, CF20]. We study the sample complexity $n$ of estimating FID to error $ε$ between $d$ -dimensional Gaussians with bounded mean distance and covariances, when one distribution is known. Our contributions are threefold. (1) We establish tight finite-sample $Θ(\frac{d^2}{n})$ bias and $Θ(\frac{d}{n} + \frac {d^2} {n^2})$ variance bounds for the empirical plug-in estimator, establishing a $\gtrsim d^2$ sample complexity. (2) To debias the empirical plug-in estimator, we generalize the ${\rm FID}_\infty$ estimator of [CF20] to extrapolation methods of arbitrary order $k$ . We further prove tight bias and variance bounds of $Θ(\frac{d^{k + 2}}{n^{k + 1}})$ and $Θ(\frac d n + \frac{d^2}{n^2})$ for any order- $k$ extrapolation under our framework. (3) We introduce Relative Taylor Debiasing (RTD), a new, computationally efficient FID estimation algorithm using debiasing techniques inspired by U-statistics. We show that RTD achieves an $O(\frac d {ε^2})$ sample complexity, and prove that this is optimal. We provide a complementary empirical evaluation of our new estimators. Our experiments on synthetic Gaussians validate the predicted residual bias and support the tightness of our bounds. On ImageNet with Inception embeddings, RTD achieves the lowest mean estimation error at the standard 50K sample budget, while our second-order variance-aware extrapolation estimator (VALE $_2$ ) uses only 10K samples to achieve accuracy comparable to FID $_\infty$ at 50K samples.

Figures & tables

Method	Error at sample size $n$	Runtime at sample size $n$
Empirical plug-in [ 10 ]	$\displaystyle\par\sqrt{\frac{d}{n}}+\frac{d^{2}}{n}$ (Theorem 1 )	$\displaystyle\par nd^{\omega-1}+d^{\omega}$
$\mathrm{FID}_{\infty}$ [ 7 ]	$\displaystyle\par\sqrt{\frac{d}{n}}+\frac{d^{3}}{n^{2}}$ (Corollary 1 )	$\displaystyle\par nd^{\omega-1}+d^{\omega}$
FID-VALE, constant order $k$ ( Section 4.3 )	$\displaystyle\par\sqrt{\frac{d}{n}}+\frac{d^{k+2}}{n^{k+1}}$ (Theorem 2 )	$\displaystyle\par nd^{\omega-1}+d^{\omega}$
FID-RTD (Section 5 )	$\displaystyle\par\sqrt{\frac{d}{n}}$ (Theorem 3 )	$\displaystyle n^{1+\gamma}d^{\omega-1}+d^{\omega+\gamma}$
Information-theoretic lower bound	$\displaystyle\par\sqrt{\frac{d}{n}}$ (Proposition 1 )	N/A

Table 1: Sample complexity and runtime comparison of FID estimators. We report the sample complexity necessary to achieve constant failure probability for Gaussian data, and suppress constant factors;

n

is the sample size,

\omega<2.373

is the matrix multiplication exponent, and

\gamma>0

is an arbitrarily small constant. All results come with matching lower bounds (Propositions 1 , 4 ).

Figure 1: Representative Gaussian and ImageNet comparisons. (a) Haar log-uniform covariances at

N=50{,}000

; each point is the mean across ten covariance pairs of the median absolute error over five trials. (b) DDO/EDM2-L 512 with Inception features; each point is the median of five estimates from the first

N

development samples.

	Empirical	OLS 1	OLS 2	OLS 3	RTD
Mean estimate	37.8610	-9.1300	6.9197	4.7519	4.9033
Signed error	32.8610	-14.1300	1.9197	-0.2481	-0.0967
Standard Deviation	0.0139	0.0316	0.0290	0.0447	0.1164

Table 2: Isotropic experiment with

D_{0}=5

d=5000

, and

N=100{,}000

. Mean and standard deviation are reported across 50 trial sample trajectories.

Figure 2: Estimation error versus dimension for Haar log-uniform covariances. At each

(d,N)

, we compute the median absolute error over five trials for each of ten independent population pairs. Curves show the mean of these ten medians, with shaded bands indicating

\pm 1

sample SD.

Panel of Figure 3 — Figure 3: Sample-budget dependence for StyleGAN-XL at $256\times 256$ resolution with Inception-V3 (left), DINOv2 (middle), and CLIP (right). Horizontal axes show $N$ (thousands); vertical axes show FID estimates. FID ∞ and RTD report medians of five estimates.

Embedding	Estimator	Empirical proxy $Q_{c}^{\mathrm{plug}}$	FID ∞ proxy $Q_{c}^{\infty}$	RTD proxy $Q_{c}$
Inception	Empirical	$0.5108\pm 0.0264$	$0.6389\pm 0.0267$	$0.6287\pm 0.0268$
	FID ∞	$0.2531\pm 0.0456$	$0.1250\pm 0.0423$	$0.1352\pm 0.0471$
	VALE 1	$0.2410\pm 0.0406$	$0.1129\pm 0.0369$	$0.1231\pm 0.0412$
	VALE 2	$0.2224\pm 0.0372$	$0.0944\pm 0.0347$	$0.1046\pm 0.0356$
	VALE 3	$0.2328\pm 0.0383$	$0.1048\pm 0.0345$	$0.1150\pm 0.0376$
	RTD	$\mathbf{0.1972\pm 0.0422}$	$\mathbf{0.0692\pm 0.0418}$	$\mathbf{0.0794\pm 0.0416}$

Table 3: Held-out utility and sensitivity to the proxy at

N=50\mathrm{K}

. For each checkpoint and proxy, we take the median absolute error over five held-out trials; entries report means

\pm

sample SD.

Configuration	Inception	DINOv2	CLIP
FID ∞ , OLS 2 , OLS 3	$0.0468$	$1.0702$	$0.1121$
$+$ VALE weighting	$0.0346\;(\downarrow 26.1\%)$	$0.8696\;(\downarrow 18.7\%)$	$0.0930\;(\downarrow 17.0\%)$
$+$ Regression strategy	$\mathbf{0.0308}\;(\downarrow 11.2\%)$	$\mathbf{0.7679}\;(\downarrow 11.7\%)$	$\mathbf{0.0878}\;(\downarrow 5.6\%)$

Table 4: Successive ablations on 50K development-pool trials. RMSE is averaged over seven checkpoints and orders

k=1,2,3

; parentheses show reductions from preceding row.

Embedding	Empirical	FID ∞	VALE 1	VALE 2	VALE 3	RTD
Inception	$64.9\pm 0.2$	$915.3\pm 3.0$	$123.1\pm 0.3$	$890.2\pm 2.1$	$1181.1\pm 3.6$	$1976.7\pm 5.3$
DINOv2	$25.9\pm 0.1$	$368.8\pm 1.1$	$50.3\pm 0.3$	$358.9\pm 1.1$	$478.0\pm 1.5$	$559.3\pm 2.0$
CLIP	$19.2\pm 0.1$	$272.6\pm 1.8$	$262.8\pm 2.4$	$354.9\pm 7.7$	$362.3\pm 2.4$	$217.7\pm 12.2$

Table 5: Estimator runtime (ms) for 50K trials, measured by mean

\pm

sample SD over all 35 runs.

$N$	Empirical	FID ∞	VALE 1	VALE 2	VALE 3	RTD
10K	$3.588\pm 0.134$	$0.211\pm 0.058$	$0.223\pm 0.073$	$\mathbf{0.137}\pm 0.053$	$1.014\pm 0.579$	$101.592\pm 23.510$
20K	$1.724\pm 0.060$	$0.159\pm 0.054$	$0.153\pm 0.048$	$\mathbf{0.107}\pm 0.053$	$0.172\pm 0.067$	$0.490\pm 0.119$
30K	$1.107\pm 0.032$	$0.144\pm 0.030$	$0.137\pm 0.028$	$0.108\pm 0.050$	$\mathbf{0.100}\pm 0.046$	$0.149\pm 0.059$
40K	$0.804\pm 0.026$	$0.141\pm 0.033$	$0.130\pm 0.033$	$\mathbf{0.112}\pm 0.040$	$0.123\pm 0.038$	$0.118\pm 0.044$
50K	$0.629\pm 0.027$	$0.135\pm 0.047$	$0.123\pm 0.041$	$0.105\pm 0.036$	$0.115\pm 0.038$	$\mathbf{0.079}\pm 0.042$

Table 6: Sample-budget sensitivity for Inception. Entries report the mean

\pm

sample SD across seven checkpoints of the median absolute error over five held-out trials against the RTD proxy.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Family	Order	Points $J$	Uniform in $n$	Uniform in $\frac{1}{n}$	Chebyshev in $\frac{1}{n}$
OLS	$k=1$	8	1.507	2.148	1.681
OLS	$k=1$	15	1.342 †	1.743	1.407
VA	$k=1$	8	1.218	1.231	1.190
VA	$k=1$	15	1.141	1.225	1.123
VA	$k=2$	8	1.778	1.567	1.784
VA	$k=2$	15	1.470	1.543	1.472

Appendix

Table 7: Expected common-pool variance factor

{\mathbb{E}}\Gamma_{\mathrm{pool}}

for different extrapolation methods.

Figure 4: Estimation error versus dimension for the spiked-bulk family at four fixed sample budgets.

Figure 5: Estimation error versus dimension for the rotated Toeplitz family at fixed sample budgets.

Figure 6: Estimation error for singular covariances with

D=5

Figure 7: Estimation error for ill-conditioned covariances with

D=5

Model	Resolution	Inception $Q_{c}$	DINOv2 $Q_{c}$	CLIP $Q_{c}$	Published FID
StyleGAN-XL [ 25 ]	64	1.936	206.696	9.852	1.51
DDO/EDM2-S [ 35 ]	64	0.333	63.356	3.270	0.97
StyleGAN-XL [ 25 ]	256	1.581	202.413	27.528	2.30
VAR-d30 [ 30 ]	256	1.127	65.289	19.089	1.73
StyleGAN-XL [ 25 ]	512	1.748	210.961	30.260	2.41
DDO/EDM2-L [ 35 ]	512	0.641	39.898	9.402	1.21

Appendix

Table 8: Model checkpoints and full-300K RTD proxy values.

Estimator	$N$	Inception	DINOv2	CLIP
FID ∞	30K	0.0439	0.7538	0.0278
FID ∞	300K	0.0102	0.1334	0.0082
Empirical	300K	0.1178	1.4370	0.0405

Appendix

Table 9: Mean absolute differences from RTD at the same sample budget across seven generators.

Figure 8: Estimates across sample budgets for the six additional generators using Inception.

Figure 9: Estimates across sample budgets for the six additional generators using DINOv2.

Figure 10: Estimates across sample budgets for the six additional generators using CLIP.

Figure 11: OLS k order sweep. Hollow markers show the case-level statistics for seven generators; solid lines show their means. Markers are slightly offset horizontally for visibility.

Embedding	$k$	OLS k	VALE k	OLS k	VALE k	RMSE drop (%)	Win rate
		SD		RMSE
Inception	1	$0.0277\pm 0.0068$	$0.0233\pm 0.0065$	$0.0366\pm 0.0044$	$0.0289\pm 0.0048$	21.21	7/7
	2	$0.0378\pm 0.0109$	$0.0289\pm 0.0092$	$0.0380\pm 0.0101$	$0.0291\pm 0.0081$	23.48	7/7
	3	$0.0678\pm 0.0202$	$0.0469\pm 0.0115$	$0.0659\pm 0.0206$	$0.0458\pm 0.0115$	28.62	7/7
	4	$0.1569\pm 0.0301$	$0.1224\pm 0.0227$	$0.1553\pm 0.0328$	$0.1212\pm 0.0240$	21.71	7/7
DINOv2	1	$0.7256\pm 0.2274$	$0.6771\pm 0.1986$	$0.7901\pm 0.2031$	$0.6994\pm 0.1968$	11.34	6/7

Appendix

Table 10: Matched OLS k –VALE k comparison. RMSE drop is the mean casewise percentage reduction; win rate counts cases with lower RMSE.

Embedding	Order $k$	Schedule	Points $\ell$	RMSE
Inception	1	Endpoints	2	$0.0275\pm 0.0045$
	2	Chebyshev in $\frac{1}{n}$	15	$0.0254\pm 0.0079$
	3	Chebyshev in $\frac{1}{n}$	20	$0.0393\pm 0.0111$
DINOv2	1	Endpoints	2	$0.6675\pm 0.1947$
	2	Chebyshev in $\frac{1}{n}$	15	$0.6871\pm 0.1925$
	3	Chebyshev in $\frac{1}{n}$	20	$0.9491\pm 0.3099$

Appendix

Table 11: Selected VALE k configurations.

Order $k$	Points $\ell$	Uniform $n$	Uniform $\frac{1}{n}$	Chebyshev $\frac{1}{n}$
1	2	$\mathbf{0.0275\pm 0.0045}$	$\mathbf{0.0275\pm 0.0045}$	$\mathbf{0.0275\pm 0.0045}$
1	10	$0.0291\pm 0.0060$	$0.0324\pm 0.0049$	$0.0317\pm 0.0058$
1	15	$0.0289\pm 0.0048$	$0.0336\pm 0.0078$	$0.0327\pm 0.0042$
1	20	$0.0279\pm 0.0057$	$0.0340\pm 0.0065$	$0.0309\pm 0.0035$
2	3	$0.0415\pm 0.0171$	$0.0371\pm 0.0083$	$0.0371\pm 0.0083$
2	10	$0.0324\pm 0.0115$	$0.0307\pm 0.0090$	$0.0323\pm 0.0115$

Appendix

Table 12: FID-VALE k regression-point ablation for Inception.

Order $k$	Points $\ell$	Uniform $n$	Uniform $\frac{1}{n}$	Chebyshev $\frac{1}{n}$
1	2	$\mathbf{0.6675\pm 0.1947}$	$\mathbf{0.6675\pm 0.1947}$	$\mathbf{0.6675\pm 0.1947}$
1	10	$0.7062\pm 0.2072$	$0.7179\pm 0.2516$	$0.7403\pm 0.2734$
1	15	$0.6994\pm 0.1968$	$0.7498\pm 0.1934$	$0.7276\pm 0.2032$
1	20	$0.7186\pm 0.2074$	$0.7277\pm 0.1825$	$0.6941\pm 0.1803$
2	3	$0.9551\pm 0.4135$	$0.9665\pm 0.2765$	$0.9665\pm 0.2765$
2	10	$0.8848\pm 0.1940$	$0.8011\pm 0.2110$	$0.8931\pm 0.2122$

Appendix

Table 13: VALE k regression-point ablation for DINOv2.

Order $k$	Points $\ell$	Uniform $n$	Uniform $\frac{1}{n}$	Chebyshev $\frac{1}{n}$
1	2	$0.0772\pm 0.0333$	$0.0772\pm 0.0333$	$0.0772\pm 0.0333$
1	10	$0.0805\pm 0.0309$	$0.0785\pm 0.0382$	$0.0798\pm 0.0333$
1	15	$0.0790\pm 0.0363$	$\mathbf{0.0742\pm 0.0360}$	$0.0786\pm 0.0318$
1	20	$0.0790\pm 0.0362$	$0.0797\pm 0.0373$	$0.0780\pm 0.0349$
2	3	$0.1129\pm 0.0500$	$0.0983\pm 0.0361$	$0.0983\pm 0.0361$
2	10	$0.1025\pm 0.0457$	$0.0927\pm 0.0329$	$0.0918\pm 0.0306$

Appendix

Table 14: VALE k regression-point ablation for CLIP.

DINOv2
$N$	Empirical	FID ∞	VALE 1	VALE 2	VALE 3	RTD
10K	$40.557\pm 1.051$	$2.722\pm 1.084$	$\mathbf{2.273}\pm 1.066$	$4.705\pm 1.863$	$20.916\pm 11.920$	$20.571\pm 4.880$
20K	$19.545\pm 0.540$	$1.675\pm 0.469$	$1.482\pm 0.475$	$1.872\pm 1.087$	$3.934\pm 1.491$	$\mathbf{1.125}\pm 0.397$
30K	$12.574\pm 0.245$	$1.625\pm 0.605$	$1.581\pm 0.503$	$1.425\pm 0.513$	$1.761\pm 0.995$	$\mathbf{1.259}\pm 0.399$
40K	$9.067\pm 0.219$	$1.727\pm 0.187$	$1.661\pm 0.245$	$1.332\pm 0.445$	$1.345\pm 0.322$	$\mathbf{1.295}\pm 0.315$
50K	$6.943\pm 0.238$	$1.582\pm 0.301$	$1.549\pm 0.281$	$\mathbf{1.257}\pm 0.399$	$1.468\pm 0.591$	$1.357\pm 0.354$

Appendix

Table 15: Sample-budget sensitivity for DINOv2 and CLIP, using the same error statistic and RTD reference proxy as Table 6 . Bold entries have the lowest mean within each embedding and budget.

Explore similar work

MIND: Monge Inception Distance for Generative Models Evaluation
May 7, 2026Quentin Berthet, Yu-Han Wu, Clement Crepy +3Image Generation Evaluation Sliced Wasserstein Distance
Google DeepMind · LPSM, Sorbonne Université
Rethinking FID Through the Geometry of the Reference Dataset
May 28, 2026Yunghee Lee, Byeonghyun PakImage Generation Evaluation
The FID Lottery: Quantifying Hidden Randomness in Generative-Model Evaluation
Jun 18, 2026Nicolas Dufour, Alexei A. Efros, Patrick PérezML Reproducibility Image Generation Evaluation
Kyutai · UC Berkeley

May 7, 2026, cs.LGJ/K move · Enter open · S save

MIND: Monge Inception Distance for Generative Models Evaluation

Quentin Berthet, Yu-Han Wu, Clement Crepy, Romuald Elie+2

Google DeepMind · LPSM, Sorbonne Université

Image Generation Evaluation Sliced Wasserstein Distance Generative AI Evaluation

Open paper AskPDF arXiv

We propose the Monge Inception Distance (MIND), a metric for evaluating generative models that addresses key limitations of the widely adopted Fréchet Inception Distance (FID). The MIND metric leverages the sliced Wasserstein distance to compare distributions by averaging one-dimensional optimal transport distances, efficiently computed via sorting. This approach circumvents the estimation of high-dimensional means and covariance matrices, which underlie FID's poor sample complexity and vulnerability to adversarial attacks. We empirically demonstrate three primary advantages: (i) it is more sample-efficient by one order of magnitude, (ii) it is faster to compute by two orders of magnitude, (iii) it is more robust to adversarial attacks such as moment-matching. We show that MIND with 5k samples can replace the evaluation performance of FID with 50k samples, providing high correlation with this standard benchmark and superior discriminative performance. We further demonstrate that even smaller sample sizes (e.g., 1k or 2k) remain highly informative for rapid model iteration.

Abstract

Explore similar work

MIND: Monge Inception Distance for Generative Models Evaluation

Rethinking FID Through the Geometry of the Reference Dataset

The FID Lottery: Quantifying Hidden Randomness in Generative-Model Evaluation