It is well known that the reconstruction FID (rFID) of a VAE is poorly correlated with the generation FID (gFID) of a latent diffusion model. We propose interpolated FID (iFID), a simple variant of rFID that exhibits a strong correlation with gFID. Specifically, for each dataset element, we retrieve its nearest neighbor in latent space, interpolate between their latent representations, decode the interpolated latent, and compute the FID between the decoded samples and the original dataset. We provide an intuitive explanation for why iFID correlates well with gFID, and why reconstruction metrics can be negatively correlated with gFID, by connecting iFID to recent results on diffusion generalization and hallucination. Theoretically, we show that iFID evaluates decoded interpolations aligned with the ridge set around which diffusion samples concentrate, thereby measuring a quantity closely related to diffusion sample quality. Empirically, iFID is the first metric shown to strongly correlate with diffusion gFID across diverse VAEs, achieving Pearson and Spearman correlations of approximately 0.85. The project page is available at https://tongdaxu.github.io/pages/ifid.html.
Figures & tables
Figure 1: Left two plots : The rFID values of VAEs are uncorrelated with, or even negatively correlated with, the gFID values of diffusion models. Right two plots : The iFID metric exhibits a strong positive correlation with the gFID values of diffusion models.
Figure 2: Toy example illustrating how latent-space geometry affects diffusion samples. Left two plots : An isolated 25-component Gaussian mixture arranged on a two-dimensional square grid has poor iFID because the interpolated latent z^ lies off the data manifold. Diffusion samples interpolating between nearby modes also fall outside the data manifold, leading to hallucination. Right two plots : A connected 25-component Gaussian mixture has good iFID because the interpolated latent z^ remains on the data manifold. Diffusion samples interpolating between nearby modes therefore stay within the data manifold, reducing hallucination.
Figure 3: Ridge sets of the source distributions p(z) from Figure 2 and Figure 4 , computed with β=0.0 and t=0.1 . The ridge sets approximate the manifolds around which diffusion samples concentrate and visually resemble lines connecting nearby training samples.
Figure 4: Toy example illustrating the difference between reconstruction-oriented and diffusion-oriented latent spaces. Left two plots : The latent distribution is an isolated two-mode Gaussian mixture with poor interpolatability, since the interpolated latent z^ does not lie on the data manifold. In this case, diffusion samples interpolating between nearby modes also fall outside the data manifold, leading to hallucinations. Right two plots : The latent distribution is an overlapping two-mode Gaussian mixture with good interpolatability, as the interpolated latent z^ remains on the data manifold. Consequently, diffusion samples interpolating between nearby modes stay within the data manifold, reducing hallucinations.
VAE Name
Latent Dim.
Arch.
Training
SD-VAE ( Rombach et al., 2022 )
4×32×32
UNet
Recon.
IN-VAE ( Yao and Wang, 2025 )
32×16×16
UNet
Recon.
FLUX-VAE ( Labs, 2024 )
16×32×32
UNet
Recon.
QwenImage-VAE ( Wu et al., 2025a )
16×32×32
UNet
Recon.
SD3-VAE ( Esser et al., 2024 )
16×32×32
UNet
Recon.
EQ-VAE ( Kouzelis et al., 2025 )
4×32×32
UNet
Recon. + Equivariance
Table 1: VAEs included in our study. We evaluate 13 VAEs with publicly available checkpoints and train corresponding diffusion models in their respective latent spaces.
Metric
gFID SiT-B
gFID SiT-XL
w/o CFG
w/ CFG
w/o CFG
w/ CFG
PCC ↑
SRCC ↑
PCC ↑
SRCC ↑
PCC ↑
SRCC ↑
PCC ↑
SRCC ↑
Reconstruction metrics
-PSNR
-0.81 ∗
-0.81 ∗
-0.83 ∗
-0.82 ∗
-0.79 ∗
-0.78 ∗
-0.79 ∗
-0.85 ∗
-SSIM
-0.78 ∗
-0.81 ∗
-0.80 ∗
-0.84 ∗
-0.77 ∗
-0.78 ∗
-0.77 ∗
-0.85 ∗
LPIPS
-0.73 ∗
-0.74 ∗
-0.72 ∗
-0.76 ∗
-0.73 ∗
-0.72 ∗
-0.73 ∗
-0.79 ∗
Table 2: Correlation between VAE metrics and diffusion gFID. Correlations are computed using metrics obtained from SiT models trained with 13 pre-trained VAEs. EQ Loss and SE Loss are not evaluated for VAEs with 1D latents. Following ( Zheng et al., 2025 ) , RAE is not evaluated with SiT-B. PCC denotes the Pearson correlation coefficient, and SRCC denotes Spearman’s rank correlation coefficient. Bold : best. ∗ : p<0.05 .
Figure 5: Visualization of nearest-neighbor latents NN(z) and interpolated latents z^ . For reconstruction-oriented VAEs, NN(z) is often semantically different from z , causing the decoded interpolated latents to produce invalid images. In contrast, for diffusion-oriented VAEs, NN(z) is semantically similar to z , and the decoded interpolated latents produce realistic images.
Setting
Interpolation
# Images
K for NN(⋅)
gFID SiT-XL
w/o CFG
w/ CFG
PCC ↑
SRCC ↑
PCC ↑
SRCC ↑
(a)
Linear
50k
1
0.78
0.78
0.80
0.82
(b)
Mask
50k
1
0.74
0.77
0.75
0.76
(c)
Spherical
50k
1
0.84
0.85
0.86
0.81
(d)
Spherical
200k
1
0.88
0.89
0.86
0.87
Table 3: Sensitivity analysis of the latent interpolation method, the number of images used for nearest-neighbor search, and the number of nearest neighbors K . iFID is robust to these choices and consistently shows a strong correlation with gFID.
Interpolation strength α
rFID
gFID SiT-XL
w/o CFG
w/ CFG
PCC ↑
SRCC ↑
PCC ↑
SRCC ↑
PCC ↑
SRCC ↑
α=0.0 (rFID)
1.00
1.00
-0.06
-0.21
-0.15
-0.31
α=0.1
0.21
0.67
0.37
0.22
0.38
0.36
α=0.2
-0.14
0.14
0.63
0.53
0.62
0.52
α=0.3
-0.14
0.11
0.66
0.59
0.65
0.58
Table 4: Sensitivity analysis of the iFID interpolation strength α . As α increases from 0 to 0.5 , iFID becomes more correlated with gFID and less correlated with rFID. The correlation with gFID becomes reasonably strong when α≥0.2 .
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: The relationship between the GMM loss and the diffusion loss is strong, which supports the theoretical results presented in [ 9 ] .
Figure 7: The relationship between reconstruction metrics and gFID for SiT/XL. The results indicate that reconstruction metrics are often negatively correlated with gFID.
Figure 8: The relationship between non-reconstruction metrics and gFID for SiT/XL. It is shown that our iFID is the only metric that is strongly correlated with gFID.
Figure 9: Visualization of the decoded nearest neighbour latent NN( z ) and the interpolated latent z^ . For reconstruction-oriented VAEs, NN( z ) is semantically different from z , and the interpolated z^ are invalid images. In contrast, for diffusion-oriented VAEs, NN( z ) is semantically similar to z , and the interpolated z^ are realistic images.
In latent generative models, reconstruction quality is often assumed to correlate with generative performance. However, reconstruction FID (rFID) can exhibit weak or even negative correlation with generation FID (gFID). We attribute this misalignment to a latent distribution mismatch: reconstruction evaluates the decoder on encoder-induced latents, whereas generation uses the same decoder on latents produced by the generative model. To characterize this shift, we introduce generation-aware reconstruction (GAR), which constructs a continuous trajectory from standard reconstruction toward generation by perturbing encoder latents with noise and denoising them through the generative model before decoding. GAR probes the decoder behavior along this trajectory, making the transition from encoder to generation-time latent distributions observable and diagnosable. The resulting trajectory-based diagnostic, GAR-FID, exhibits strong empirical correlation with gFID across diverse tokenizers and scales. Importantly, intermediate GAR latents become more generation-aware while preserving correspondence with their source images, thereby retaining paired supervision that is absent for fully generated latents. This correspondence enables decoder adaptation on intermediate GAR latents, consistently improving generative quality across model scales. Overall, latent distribution mismatch provides a useful perspective for evaluating and improving latent generative models.
Fréchet Inception Distance (FID) is widely used to evaluate image generators, yet lower FID does not always correspond to better sample quality. We show that this mismatch depends in part on the geometry of the reference dataset. In a controlled study across six datasets, distributional density and effective rank significantly explain how FID changes as sample quality improves. Concentrated datasets tend to yield more favorable FID trends, whereas more dispersed datasets can make FID worsen despite better samples. Attribution to precision and recall and ablations with alternative feature spaces and distances support the same conclusion. These results suggest that distributional metrics should be interpreted together with the geometry of the reference dataset for more reliable benchmarking.
The Frechet Inception Distance (FID) is the de facto arbiter of image generation, yet most papers report just a single number from a single trained model using a single sampling seed. How reproducible is that number if we retrain the model, or merely resample from it? In this paper, we treat FID as a random variable on a two-axis panel of training and generation seeds, and measure its variance directly on several hundred SiT networks trained on class-conditional ImageNet 256x256. We report surprising findings: (a) Retraining the model using the same recipe with a different seed moves FID 3.2x more (in Inception feature space) than redrawing samples from a fixed network. (b) That gap is driven by three factors: random initialisation, data ordering, and the per-step Gaussian noise of the flow-matching loss. (c) Increasing compute or model size barely tightens the spread, holding the FID coefficient of variation (CoV) inside a 1-2% band. (d) Per-cell classifier-free-guidance tuning halves the spread but reshuffles which seeds work best, and a lucky training seed reaches the same FID with up to 2x less compute than an unlucky one. Based on these findings, we recommend a new FID evaluation protocol: evaluate under per-cell optimal guidance, treat any FID gap below the empirically measured ~1.3% CoV as inconclusive, and report an error bar over several training seeds rather than a single FID number.