Evaluating generative models requires measuring not only sample fidelity but also how well a generated set covers the real data distribution. Recent work has therefore introduced recall-oriented metrics for distribution-level evaluation. However, existing recall metrics often reduce distributional coverage to geometry-based coverage. This simplifying assumption weakens their diagnostic value and interpretability in two ways: these metrics provide only a single aggregate score that cannot distinguish essential diversity failures such as mode dropping and mode collapse, and geometry-based coverage can provide unreliable estimates by assigning high scores to poorly covered generative distributions. We therefore introduce CADMA, a capacity-aware framework for recall-oriented evaluation and diagnosis. CADMA evaluates distributional coverage through local capacity matching between real and generated samples, separating geometry-based coverage from effective coverage. This yields multi-dimensional coverage diagnostics that quantify mode dropping and mode collapse while reducing the overly optimistic estimates produced by geometry-based metrics. Through extensive experiments on toy data, real-world datasets, and generative models, we show that CADMA provides more reliable coverage estimates and more fine-grained, interpretable diagnostic signals of diversity failures than existing metrics. Our code is available at https://github.com/Open-Debin/CADMA_NeurIPS2026.
Figures & tables
Figure 1
Figure 3: Main experiments on Gaussian and CIFAR-10 mode dropping, mode collapse, and outlier pollution. We compare the proposed capacity-aware coverage with existing baselines and report CADMA diagnostic scores for mode dropping and mode collapse.
Method
Gaussian
CIFAR-10
Avg.
Mode Collapse ( ↓ )
Outlier Pollution ( ↓ )
Mode Collapse ( ↓ )
Outlier Pollution ( ↓ )
Recall Kynkäänniemi et al. (2019)
0.206
0.214
0.252
0.179
0.213
Coverage Naeem et al. (2020)
0.204
0.203
0.239
0.189
0.209
symRecall Khayatkhoei and AbdAlmageed (2023)
0.180
0.183
0.227
0.175
0.191
P-recall Park and Kim (2023)
0.255
0.283
0.080
0.079
0.174
RecallCover Cheema and Urner (2023)
0.088
0.085
0.108
0.090
0.093
Table 1: Mean absolute error. Best and second-best results are bolded and underlined.
Figure 4Figure 5
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Supplementary illustration of the controlled experimental settings. Panel (a) shows the Gaussian translation setting with real and generated outliers, and panel (b) shows simultaneous mode dropping where generated mass is reassigned to the remaining mode.
#
Family
Model (Year)
I-Precision ↑
Ccap↑
Ddrop↓
Dcollapse↓
1
Diffusion
ADM (2021)
0.512
0.492
0.308
0.027
2
Diffusion
ADM-CFG (2021)
0.589
0.674
0.164
0.034
3
Diffusion
ADM-CFG-ADMU (2021)
0.619
0.710
0.146
0.039
4
Diffusion
LDM (2022)
0.690
0.757
0.093
0.049
5
Diffusion
DiT-XL-2 (2023)
0.731
0.786
0.087
0.042
6
Diffusion
DiT-XL-2-CFG (2023)
0.926
0.826
0.035
0.145
Appendix
Table 2: Evaluation of 11 generators on ImageNet-256. Each model uses 25,000 real and 25,000 generated samples.
#
Family
Model (Year)
I-Precision ↑
Ccap↑
Ddrop↓
Dcollapse↓
1
VAE
Efficient-VDVAE (2022)
0.448
0.417
0.154
0.223
2
Diffusion
LDM (2022)
0.430
0.606
0.143
0.067
3
Discrete-token
Unleashing Transformers (2022)
0.285
0.395
0.209
0.102
4
GAN
StyleGAN2-ADA (2020)
0.159
0.283
0.190
0.129
5
GAN
InsGen (2021)
0.203
0.355
0.218
0.101
6
GAN
Projected-GAN (2021)
0.109
0.191
0.170
0.107
Appendix
Table 3: Evaluation of nine generators on FFHQ. Best and second-best results are bolded and underlined, respectively.
#
Family
Model (Year)
I-Precision ↑
Ccap↑
Ddrop↓
Dcollapse↓
1
Diffusion
DDPM (2020)
0.306
0.439
0.260
0.031
2
Diffusion
iDDPM (2021)
0.383
0.530
0.201
0.028
3
Diffusion
ADMNet-dropout (2021)
0.521
0.687
0.085
0.015
4
Consistency
Consistency (2023)
0.377
0.288
0.282
0.268
5
Discrete-token
Unleashing Transformers (2022)
0.209
0.234
0.305
0.108
6
GAN
StyleGAN (2019)
0.418
0.465
0.217
0.111
Appendix
Table 4: Evaluation of eight generators on LSUN Bedroom. Best and second-best results are bolded and underlined, respectively.
Model (Year)
Sampling
I-Precision ↑
Ccap↑
Ddrop↓
Dcollapse↓
VAR-d16 (2024)
Original
0.636
0.703
0.139
0.039
VAR-d16 (2024)
Smooth
0.758
0.798
0.059
0.069
VAR-d20 (2024)
Original
0.694
0.769
0.103
0.040
VAR-d20 (2024)
Smooth
0.819
0.832
0.043
0.067
VAR-d24 (2024)
Original
0.735
0.800
0.085
0.036
VAR-d24 (2024)
Smooth
0.844
0.849
0.038
0.068
Appendix
Table 5: VAR evaluation on ImageNet-256 under original and smooth sampling, with DiT-XL-2 at CFG scale 1.5 as a reference. Bold values indicate the better result between the two sampling strategies at each VAR depth.
N
CADMA (s)
Tneigh (s)
Tadd (s)
Tadd/TCADMA
ClippedCoverage (s)
Coverage (s)
1K
0.225
0.223427
0.001189
0.529%
0.205
0.158
2.5K
0.662
0.658441
0.003792
0.572%
0.795
0.625
5K
2.166
2.157869
0.007753
0.358%
2.641
2.094
7.5K
4.792
4.779245
0.012223
0.255%
5.674
4.466
10K
8.266
8.230676
0.034747
0.420%
10.252
7.947
20K
31.757
31.645776
0.110717
0.349%
39.798
30.817
Appendix
Table 6: Wall-clock runtime of CADMA and baseline metrics, measured in seconds using precomputed features. N denotes the number of samples in each of the real and generated sets.
N
Feature Memory
CADMA
ClippedCoverage
Coverage
1K
167.3
8.3 (5.0%)
3.5 (2.1%)
5.2 (3.1%)
2.5K
191.6
11.1 (5.8%)
3.7 (1.9%)
9.9 (5.2%)
5K
234.0
16.8 (7.3%)
6.1 (2.6%)
16.3 (6.9%)
7.5K
274.7
24.4 (8.9%)
4.1 (1.5%)
17.0 (6.2%)
10K
314.3
28.5 (9.1%)
3.9 (1.3%)
15.7 (5.0%)
20K
478.7
39.5 (8.3%)
10.2 (2.1%)
24.2 (5.1%)
Appendix
Table 7: Memory usage in MB. Percentages in parentheses are relative to the Feature Memory column.