Sample-Optimal Estimation of the Fréchet Inception Distance

Ziyun Chen; Jerry Li; Kevin Tian; Yusong Zhu

cs.LGOct 5, 2026

Sample-Optimal Estimation of the Fréchet Inception Distance

Authors: Ziyun Chen, Jerry Li, Kevin Tian, Yusong Zhu

Organizations: University of Washington · University of Texas at Austin

Abstract

The Fréchet Inception Distance (FID) is widely used to evaluate generative models, but its empirical plug-in estimator suffers from finite-sample bias [BSAG18, CF20]. We study the sample complexity $n$ of estimating FID to error $ε$ between $d$ -dimensional Gaussians with bounded mean distance and covariances, when one distribution is known. Our contributions are threefold. (1) We establish tight finite-sample $Θ(\frac{d^2}{n})$ bias and $Θ(\frac{d}{n} + \frac {d^2} {n^2})$ variance bounds for the empirical plug-in estimator, establishing a $\gtrsim d^2$ sample complexity. (2) To debias the empirical plug-in estimator, we generalize the ${\rm FID}_\infty$ estimator of [CF20] to extrapolation methods of arbitrary order $k$ . We further prove tight bias and variance bounds of $Θ(\frac{d^{k + 2}}{n^{k + 1}})$ and $Θ(\frac d n + \frac{d^2}{n^2})$ for any order- $k$ extrapolation under our framework. (3) We introduce Relative Taylor Debiasing (RTD), a new, computationally efficient FID estimation algorithm using debiasing techniques inspired by U-statistics. We show that RTD achieves an $O(\frac d {ε^2})$ sample complexity, and prove that this is optimal. We provide a complementary empirical evaluation of our new estimators. Our experiments on synthetic Gaussians validate the predicted residual bias and support the tightness of our bounds. On ImageNet with Inception embeddings, RTD achieves the lowest mean estimation error at the standard 50K sample budget, while our second-order variance-aware extrapolation estimator (VALE $_2$ ) uses only 10K samples to achieve accuracy comparable to FID $_\infty$ at 50K samples.

Figures & tables

Method	Error at sample size $n$	Runtime at sample size $n$
Empirical plug-in [ 10 ]	$\displaystyle\par\sqrt{\frac{d}{n}}+\frac{d^{2}}{n}$ (Theorem 1 )	$\displaystyle\par nd^{\omega-1}+d^{\omega}$
$\mathrm{FID}_{\infty}$ [ 7 ]	$\displaystyle\par\sqrt{\frac{d}{n}}+\frac{d^{3}}{n^{2}}$ (Corollary 1 )	$\displaystyle\par nd^{\omega-1}+d^{\omega}$
FID-VALE, constant order $k$ ( Section 4.3 )	$\displaystyle\par\sqrt{\frac{d}{n}}+\frac{d^{k+2}}{n^{k+1}}$ (Theorem 2 )	$\displaystyle\par nd^{\omega-1}+d^{\omega}$
FID-RTD (Section 5 )	$\displaystyle\par\sqrt{\frac{d}{n}}$ (Theorem 3 )	$\displaystyle n^{1+\gamma}d^{\omega-1}+d^{\omega+\gamma}$
Information-theoretic lower bound	$\displaystyle\par\sqrt{\frac{d}{n}}$ (Proposition 1 )	N/A

Table 1: Sample complexity and runtime comparison of FID estimators. We report the sample complexity necessary to achieve constant failure probability for Gaussian data, and suppress constant factors;

n

is the sample size,

\omega<2.373

is the matrix multiplication exponent, and

\gamma>0

is an arbitrarily small constant. All results come with matching lower bounds (Propositions 1 , 4 ).

Figure 1: Representative Gaussian and ImageNet comparisons. (a) Haar log-uniform covariances at

N=50{,}000

; each point is the mean across ten covariance pairs of the median absolute error over five trials. (b) DDO/EDM2-L 512 with Inception features; each point is the median of five estimates from the first

N

development samples.

	Empirical	OLS 1	OLS 2	OLS 3	RTD
Mean estimate	37.8610	-9.1300	6.9197	4.7519	4.9033
Signed error	32.8610	-14.1300	1.9197	-0.2481	-0.0967
Standard Deviation	0.0139	0.0316	0.0290	0.0447	0.1164

Table 2: Isotropic experiment with

D_{0}=5

d=5000

, and

N=100{,}000

. Mean and standard deviation are reported across 50 trial sample trajectories.

Figure 2: Estimation error versus dimension for Haar log-uniform covariances. At each

(d,N)

, we compute the median absolute error over five trials for each of ten independent population pairs. Curves show the mean of these ten medians, with shaded bands indicating

\pm 1

sample SD.

Panel of Figure 3 — Figure 3: Sample-budget dependence for StyleGAN-XL at $256\times 256$ resolution with Inception-V3 (left), DINOv2 (middle), and CLIP (right). Horizontal axes show $N$ (thousands); vertical axes show FID estimates. FID ∞ and RTD report medians of five estimates.

Embedding	Estimator	Empirical proxy $Q_{c}^{\mathrm{plug}}$	FID ∞ proxy $Q_{c}^{\infty}$	RTD proxy $Q_{c}$
Inception	Empirical	$0.5108\pm 0.0264$	$0.6389\pm 0.0267$	$0.6287\pm 0.0268$
	FID ∞	$0.2531\pm 0.0456$	$0.1250\pm 0.0423$	$0.1352\pm 0.0471$
	VALE 1	$0.2410\pm 0.0406$	$0.1129\pm 0.0369$	$0.1231\pm 0.0412$
	VALE 2	$0.2224\pm 0.0372$	$0.0944\pm 0.0347$	$0.1046\pm 0.0356$
	VALE 3	$0.2328\pm 0.0383$	$0.1048\pm 0.0345$	$0.1150\pm 0.0376$
	RTD	$\mathbf{0.1972\pm 0.0422}$	$\mathbf{0.0692\pm 0.0418}$	$\mathbf{0.0794\pm 0.0416}$

Table 3: Held-out utility and sensitivity to the proxy at

N=50\mathrm{K}

. For each checkpoint and proxy, we take the median absolute error over five held-out trials; entries report means

\pm

sample SD.

Configuration	Inception	DINOv2	CLIP
FID ∞ , OLS 2 , OLS 3	$0.0468$	$1.0702$	$0.1121$
$+$ VALE weighting	$0.0346\;(\downarrow 26.1\%)$	$0.8696\;(\downarrow 18.7\%)$	$0.0930\;(\downarrow 17.0\%)$
$+$ Regression strategy	$\mathbf{0.0308}\;(\downarrow 11.2\%)$	$\mathbf{0.7679}\;(\downarrow 11.7\%)$	$\mathbf{0.0878}\;(\downarrow 5.6\%)$

Table 4: Successive ablations on 50K development-pool trials. RMSE is averaged over seven checkpoints and orders

k=1,2,3

; parentheses show reductions from preceding row.

Embedding	Empirical	FID ∞	VALE 1	VALE 2	VALE 3	RTD
Inception	$64.9\pm 0.2$	$915.3\pm 3.0$	$123.1\pm 0.3$	$890.2\pm 2.1$	$1181.1\pm 3.6$	$1976.7\pm 5.3$
DINOv2	$25.9\pm 0.1$	$368.8\pm 1.1$	$50.3\pm 0.3$	$358.9\pm 1.1$	$478.0\pm 1.5$	$559.3\pm 2.0$
CLIP	$19.2\pm 0.1$	$272.6\pm 1.8$	$262.8\pm 2.4$	$354.9\pm 7.7$	$362.3\pm 2.4$	$217.7\pm 12.2$

Table 5: Estimator runtime (ms) for 50K trials, measured by mean

\pm

sample SD over all 35 runs.

$N$	Empirical	FID ∞	VALE 1	VALE 2	VALE 3	RTD
10K	$3.588\pm 0.134$	$0.211\pm 0.058$	$0.223\pm 0.073$	$\mathbf{0.137}\pm 0.053$	$1.014\pm 0.579$	$101.592\pm 23.510$
20K	$1.724\pm 0.060$	$0.159\pm 0.054$	$0.153\pm 0.048$	$\mathbf{0.107}\pm 0.053$	$0.172\pm 0.067$	$0.490\pm 0.119$
30K	$1.107\pm 0.032$	$0.144\pm 0.030$	$0.137\pm 0.028$	$0.108\pm 0.050$	$\mathbf{0.100}\pm 0.046$	$0.149\pm 0.059$
40K	$0.804\pm 0.026$	$0.141\pm 0.033$	$0.130\pm 0.033$	$\mathbf{0.112}\pm 0.040$	$0.123\pm 0.038$	$0.118\pm 0.044$
50K	$0.629\pm 0.027$	$0.135\pm 0.047$	$0.123\pm 0.041$	$0.105\pm 0.036$	$0.115\pm 0.038$	$\mathbf{0.079}\pm 0.042$

Table 6: Sample-budget sensitivity for Inception. Entries report the mean

\pm

sample SD across seven checkpoints of the median absolute error over five held-out trials against the RTD proxy.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Family	Order	Points $J$	Uniform in $n$	Uniform in $\frac{1}{n}$	Chebyshev in $\frac{1}{n}$
OLS	$k=1$	8	1.507	2.148	1.681
OLS	$k=1$	15	1.342 †	1.743	1.407
VA	$k=1$	8	1.218	1.231	1.190
VA	$k=1$	15	1.141	1.225	1.123
VA	$k=2$	8	1.778	1.567	1.784
VA	$k=2$	15	1.470	1.543	1.472

Appendix

Table 7: Expected common-pool variance factor

{\mathbb{E}}\Gamma_{\mathrm{pool}}

for different extrapolation methods.

Figure 4: Estimation error versus dimension for the spiked-bulk family at four fixed sample budgets.

Figure 5: Estimation error versus dimension for the rotated Toeplitz family at fixed sample budgets.

Figure 6: Estimation error for singular covariances with

D=5

Figure 7: Estimation error for ill-conditioned covariances with

D=5

Model	Resolution	Inception $Q_{c}$	DINOv2 $Q_{c}$	CLIP $Q_{c}$	Published FID
StyleGAN-XL [ 25 ]	64	1.936	206.696	9.852	1.51
DDO/EDM2-S [ 35 ]	64	0.333	63.356	3.270	0.97
StyleGAN-XL [ 25 ]	256	1.581	202.413	27.528	2.30
VAR-d30 [ 30 ]	256	1.127	65.289	19.089	1.73
StyleGAN-XL [ 25 ]	512	1.748	210.961	30.260	2.41
DDO/EDM2-L [ 35 ]	512	0.641	39.898	9.402	1.21

Appendix

Table 8: Model checkpoints and full-300K RTD proxy values.

Estimator	$N$	Inception	DINOv2	CLIP
FID ∞	30K	0.0439	0.7538	0.0278
FID ∞	300K	0.0102	0.1334	0.0082
Empirical	300K	0.1178	1.4370	0.0405

Appendix

Table 9: Mean absolute differences from RTD at the same sample budget across seven generators.

Figure 8: Estimates across sample budgets for the six additional generators using Inception.

Figure 9: Estimates across sample budgets for the six additional generators using DINOv2.

Figure 10: Estimates across sample budgets for the six additional generators using CLIP.

Figure 11: OLS k order sweep. Hollow markers show the case-level statistics for seven generators; solid lines show their means. Markers are slightly offset horizontally for visibility.

Embedding	$k$	OLS k	VALE k	OLS k	VALE k	RMSE drop (%)	Win rate
		SD		RMSE
Inception	1	$0.0277\pm 0.0068$	$0.0233\pm 0.0065$	$0.0366\pm 0.0044$	$0.0289\pm 0.0048$	21.21	7/7
	2	$0.0378\pm 0.0109$	$0.0289\pm 0.0092$	$0.0380\pm 0.0101$	$0.0291\pm 0.0081$	23.48	7/7
	3	$0.0678\pm 0.0202$	$0.0469\pm 0.0115$	$0.0659\pm 0.0206$	$0.0458\pm 0.0115$	28.62	7/7
	4	$0.1569\pm 0.0301$	$0.1224\pm 0.0227$	$0.1553\pm 0.0328$	$0.1212\pm 0.0240$	21.71	7/7
DINOv2	1	$0.7256\pm 0.2274$	$0.6771\pm 0.1986$	$0.7901\pm 0.2031$	$0.6994\pm 0.1968$	11.34	6/7

Appendix

Table 10: Matched OLS k –VALE k comparison. RMSE drop is the mean casewise percentage reduction; win rate counts cases with lower RMSE.

Embedding	Order $k$	Schedule	Points $\ell$	RMSE
Inception	1	Endpoints	2	$0.0275\pm 0.0045$
	2	Chebyshev in $\frac{1}{n}$	15	$0.0254\pm 0.0079$
	3	Chebyshev in $\frac{1}{n}$	20	$0.0393\pm 0.0111$
DINOv2	1	Endpoints	2	$0.6675\pm 0.1947$
	2	Chebyshev in $\frac{1}{n}$	15	$0.6871\pm 0.1925$
	3	Chebyshev in $\frac{1}{n}$	20	$0.9491\pm 0.3099$

Appendix

Table 11: Selected VALE k configurations.

Order $k$	Points $\ell$	Uniform $n$	Uniform $\frac{1}{n}$	Chebyshev $\frac{1}{n}$
1	2	$\mathbf{0.0275\pm 0.0045}$	$\mathbf{0.0275\pm 0.0045}$	$\mathbf{0.0275\pm 0.0045}$
1	10	$0.0291\pm 0.0060$	$0.0324\pm 0.0049$	$0.0317\pm 0.0058$
1	15	$0.0289\pm 0.0048$	$0.0336\pm 0.0078$	$0.0327\pm 0.0042$
1	20	$0.0279\pm 0.0057$	$0.0340\pm 0.0065$	$0.0309\pm 0.0035$
2	3	$0.0415\pm 0.0171$	$0.0371\pm 0.0083$	$0.0371\pm 0.0083$
2	10	$0.0324\pm 0.0115$	$0.0307\pm 0.0090$	$0.0323\pm 0.0115$

Appendix

Table 12: FID-VALE k regression-point ablation for Inception.

Order $k$	Points $\ell$	Uniform $n$	Uniform $\frac{1}{n}$	Chebyshev $\frac{1}{n}$
1	2	$\mathbf{0.6675\pm 0.1947}$	$\mathbf{0.6675\pm 0.1947}$	$\mathbf{0.6675\pm 0.1947}$
1	10	$0.7062\pm 0.2072$	$0.7179\pm 0.2516$	$0.7403\pm 0.2734$
1	15	$0.6994\pm 0.1968$	$0.7498\pm 0.1934$	$0.7276\pm 0.2032$
1	20	$0.7186\pm 0.2074$	$0.7277\pm 0.1825$	$0.6941\pm 0.1803$
2	3	$0.9551\pm 0.4135$	$0.9665\pm 0.2765$	$0.9665\pm 0.2765$
2	10	$0.8848\pm 0.1940$	$0.8011\pm 0.2110$	$0.8931\pm 0.2122$

Appendix

Table 13: VALE k regression-point ablation for DINOv2.

Order $k$	Points $\ell$	Uniform $n$	Uniform $\frac{1}{n}$	Chebyshev $\frac{1}{n}$
1	2	$0.0772\pm 0.0333$	$0.0772\pm 0.0333$	$0.0772\pm 0.0333$
1	10	$0.0805\pm 0.0309$	$0.0785\pm 0.0382$	$0.0798\pm 0.0333$
1	15	$0.0790\pm 0.0363$	$\mathbf{0.0742\pm 0.0360}$	$0.0786\pm 0.0318$
1	20	$0.0790\pm 0.0362$	$0.0797\pm 0.0373$	$0.0780\pm 0.0349$
2	3	$0.1129\pm 0.0500$	$0.0983\pm 0.0361$	$0.0983\pm 0.0361$
2	10	$0.1025\pm 0.0457$	$0.0927\pm 0.0329$	$0.0918\pm 0.0306$

Appendix

Table 14: VALE k regression-point ablation for CLIP.

DINOv2
$N$	Empirical	FID ∞	VALE 1	VALE 2	VALE 3	RTD
10K	$40.557\pm 1.051$	$2.722\pm 1.084$	$\mathbf{2.273}\pm 1.066$	$4.705\pm 1.863$	$20.916\pm 11.920$	$20.571\pm 4.880$
20K	$19.545\pm 0.540$	$1.675\pm 0.469$	$1.482\pm 0.475$	$1.872\pm 1.087$	$3.934\pm 1.491$	$\mathbf{1.125}\pm 0.397$
30K	$12.574\pm 0.245$	$1.625\pm 0.605$	$1.581\pm 0.503$	$1.425\pm 0.513$	$1.761\pm 0.995$	$\mathbf{1.259}\pm 0.399$
40K	$9.067\pm 0.219$	$1.727\pm 0.187$	$1.661\pm 0.245$	$1.332\pm 0.445$	$1.345\pm 0.322$	$\mathbf{1.295}\pm 0.315$
50K	$6.943\pm 0.238$	$1.582\pm 0.301$	$1.549\pm 0.281$	$\mathbf{1.257}\pm 0.399$	$1.468\pm 0.591$	$1.357\pm 0.354$

Appendix

Table 15: Sample-budget sensitivity for DINOv2 and CLIP, using the same error statistic and RTD reference proxy as Table 6 . Bold entries have the lowest mean within each embedding and budget.

Explore similar work

May 7, 2026cs.LG

MIND: Monge Inception Distance for Generative Models Evaluation

We propose the Monge Inception Distance (MIND), a metric for evaluating generative models that addresses key limitations of the widely adopted Fréchet Inception Distance (FID). The MIND metric leverages the sliced Wasserstein distance to compare distributions by averaging one-dimensional optimal transport distances, efficiently computed via sorting. This approach circumvents the estimation of high-dimensional means and covariance matrices, which underlie FID's poor sample complexity and vulnerability to adversarial attacks. We empirically demonstrate three primary advantages: (i) it is more sample-efficient by one order of magnitude, (ii) it is faster to compute by two orders of magnitude, (iii) it is more robust to adversarial attacks such as moment-matching. We show that MIND with 5k samples can replace the evaluation performance of FID with 50k samples, providing high correlation with this standard benchmark and superior discriminative performance. We further demonstrate that even smaller sample sizes (e.g., 1k or 2k) remain highly informative for rapid model iteration.

Quentin Berthet, Yu-Han Wu, Clement Crepy +3

Google DeepMind · LPSM, Sorbonne Université

Fréchet Inception Distance Generative Models Distance Discriminative Models Discriminative

Ask PDF

May 28, 2026cs.CV

Rethinking FID Through the Geometry of the Reference Dataset

Fréchet Inception Distance (FID) is widely used to evaluate image generators, yet lower FID does not always correspond to better sample quality. We show that this mismatch depends in part on the geometry of the reference dataset. In a controlled study across six datasets, distributional density and effective rank significantly explain how FID changes as sample quality improves. Concentrated datasets tend to yield more favorable FID trends, whereas more dispersed datasets can make FID worsen despite better samples. Attribution to precision and recall and ablations with alternative feature spaces and distances support the same conclusion. These results suggest that distributional metrics should be interpreted together with the geometry of the reference dataset for more reliable benchmarking.

Yunghee Lee, Byeonghyun Pak

Fréchet Inception Distance Real-World Dataset Image Generation

Ask PDF

Jun 18, 2026cs.CV

The FID Lottery: Quantifying Hidden Randomness in Generative-Model Evaluation

The Frechet Inception Distance (FID) is the de facto arbiter of image generation, yet most papers report just a single number from a single trained model using a single sampling seed. How reproducible is that number if we retrain the model, or merely resample from it? In this paper, we treat FID as a random variable on a two-axis panel of training and generation seeds, and measure its variance directly on several hundred SiT networks trained on class-conditional ImageNet 256x256. We report surprising findings: (a) Retraining the model using the same recipe with a different seed moves FID 3.2x more (in Inception feature space) than redrawing samples from a fixed network. (b) That gap is driven by three factors: random initialisation, data ordering, and the per-step Gaussian noise of the flow-matching loss. (c) Increasing compute or model size barely tightens the spread, holding the FID coefficient of variation (CoV) inside a 1-2% band. (d) Per-cell classifier-free-guidance tuning halves the spread but reshuffles which seeds work best, and a lucky training seed reaches the same FID with up to 2x less compute than an unlucky one. Based on these findings, we recommend a new FID evaluation protocol: evaluate under per-cell optimal guidance, treat any FID gap below the empirically measured ~1.3% CoV as inconclusive, and report an error bar over several training seeds rather than a single FID number.

Nicolas Dufour, Alexei A. Efros, Patrick Pérez

Kyutai · UC Berkeley

Fréchet Inception Distance Generative Models Image Generation Random

Ask PDF