CADMA: Capacity-Aware Coverage Decomposition for Generative Model Assessment
Organizations: Queen Mary University of London · University of Bedfordshire
Abstract
Evaluating generative models requires measuring not only sample fidelity but also how well a generated set covers the real data distribution. Recent work has therefore introduced recall-oriented metrics for distribution-level evaluation. However, existing recall metrics often reduce distributional coverage to geometry-based coverage. This simplifying assumption weakens their diagnostic value and interpretability in two ways: these metrics provide only a single aggregate score that cannot distinguish essential diversity failures such as mode dropping and mode collapse, and geometry-based coverage can provide unreliable estimates by assigning high scores to poorly covered generative distributions. We therefore introduce CADMA, a capacity-aware framework for recall-oriented evaluation and diagnosis. CADMA evaluates distributional coverage through local capacity matching between real and generated samples, separating geometry-based coverage from effective coverage. This yields multi-dimensional coverage diagnostics that quantify mode dropping and mode collapse while reducing the overly optimistic estimates produced by geometry-based metrics. Through extensive experiments on toy data, real-world datasets, and generative models, we show that CADMA provides more reliable coverage estimates and more fine-grained, interpretable diagnostic signals of diversity failures than existing metrics. Our code is available at https://github.com/Open-Debin/CADMA_NeurIPS2026.
Figures & tables
| Method | Gaussian | CIFAR-10 | Avg. | ||
|---|---|---|---|---|---|
| Mode Collapse ( ) | Outlier Pollution ( ) | Mode Collapse ( ) | Outlier Pollution ( ) | ||
| Recall Kynkäänniemi et al. (2019) | 0.206 | 0.214 | 0.252 | 0.179 | 0.213 |
| Coverage Naeem et al. (2020) | 0.204 | 0.203 | 0.239 | 0.189 | 0.209 |
| symRecall Khayatkhoei and AbdAlmageed (2023) | 0.180 | 0.183 | 0.227 | 0.175 | 0.191 |
| P-recall Park and Kim (2023) | 0.255 | 0.283 | 0.080 | 0.079 | 0.174 |
| RecallCover Cheema and Urner (2023) | 0.088 | 0.085 | 0.108 | 0.090 | 0.093 |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| # | Family | Model (Year) | I-Precision | |||
|---|---|---|---|---|---|---|
| 1 | Diffusion | ADM (2021) | 0.512 | 0.492 | 0.308 | 0.027 |
| 2 | Diffusion | ADM-CFG (2021) | 0.589 | 0.674 | 0.164 | 0.034 |
| 3 | Diffusion | ADM-CFG-ADMU (2021) | 0.619 | 0.710 | 0.146 | 0.039 |
| 4 | Diffusion | LDM (2022) | 0.690 | 0.757 | 0.093 | 0.049 |
| 5 | Diffusion | DiT-XL-2 (2023) | 0.731 | 0.786 | 0.087 | 0.042 |
| 6 | Diffusion | DiT-XL-2-CFG (2023) | 0.926 | 0.826 | 0.035 | 0.145 |
| # | Family | Model (Year) | I-Precision | |||
|---|---|---|---|---|---|---|
| 1 | VAE | Efficient-VDVAE (2022) | 0.448 | 0.417 | 0.223 | |
| 2 | Diffusion | LDM (2022) | 0.606 | 0.143 | ||
| 3 | Discrete-token | Unleashing Transformers (2022) | 0.285 | 0.395 | 0.209 | 0.102 |
| 4 | GAN | StyleGAN2-ADA (2020) | 0.159 | 0.283 | 0.190 | 0.129 |
| 5 | GAN | InsGen (2021) | 0.203 | 0.355 | 0.218 | 0.101 |
| 6 | GAN | Projected-GAN (2021) | 0.109 | 0.191 | 0.170 | 0.107 |
| # | Family | Model (Year) | I-Precision | |||
|---|---|---|---|---|---|---|
| 1 | Diffusion | DDPM (2020) | 0.306 | 0.439 | 0.260 | 0.031 |
| 2 | Diffusion | iDDPM (2021) | 0.383 | 0.201 | ||
| 3 | Diffusion | ADMNet-dropout (2021) | 0.521 | 0.687 | 0.085 | 0.015 |
| 4 | Consistency | Consistency (2023) | 0.377 | 0.288 | 0.282 | 0.268 |
| 5 | Discrete-token | Unleashing Transformers (2022) | 0.209 | 0.234 | 0.305 | 0.108 |
| 6 | GAN | StyleGAN (2019) | 0.465 | 0.217 | 0.111 |
| Model (Year) | Sampling | I-Precision | |||
|---|---|---|---|---|---|
| VAR-d16 (2024) | Original | 0.636 | 0.703 | 0.139 | 0.039 |
| VAR-d16 (2024) | Smooth | 0.758 | 0.798 | 0.059 | 0.069 |
| VAR-d20 (2024) | Original | 0.694 | 0.769 | 0.103 | 0.040 |
| VAR-d20 (2024) | Smooth | 0.819 | 0.832 | 0.043 | 0.067 |
| VAR-d24 (2024) | Original | 0.735 | 0.800 | 0.085 | 0.036 |
| VAR-d24 (2024) | Smooth | 0.844 | 0.849 | 0.038 | 0.068 |
| CADMA (s) | (s) | (s) | ClippedCoverage (s) | Coverage (s) | ||
|---|---|---|---|---|---|---|
| 1K | 0.225 | 0.223427 | 0.001189 | 0.529% | 0.205 | 0.158 |
| 2.5K | 0.662 | 0.658441 | 0.003792 | 0.572% | 0.795 | 0.625 |
| 5K | 2.166 | 2.157869 | 0.007753 | 0.358% | 2.641 | 2.094 |
| 7.5K | 4.792 | 4.779245 | 0.012223 | 0.255% | 5.674 | 4.466 |
| 10K | 8.266 | 8.230676 | 0.034747 | 0.420% | 10.252 | 7.947 |
| 20K | 31.757 | 31.645776 | 0.110717 | 0.349% | 39.798 | 30.817 |
| Feature Memory | CADMA | ClippedCoverage | Coverage | |
|---|---|---|---|---|
| 1K | 167.3 | 8.3 (5.0%) | 3.5 (2.1%) | 5.2 (3.1%) |
| 2.5K | 191.6 | 11.1 (5.8%) | 3.7 (1.9%) | 9.9 (5.2%) |
| 5K | 234.0 | 16.8 (7.3%) | 6.1 (2.6%) | 16.3 (6.9%) |
| 7.5K | 274.7 | 24.4 (8.9%) | 4.1 (1.5%) | 17.0 (6.2%) |
| 10K | 314.3 | 28.5 (9.1%) | 3.9 (1.3%) | 15.7 (5.0%) |
| 20K | 478.7 | 39.5 (8.3%) | 10.2 (2.1%) | 24.2 (5.1%) |
Generative AI Evaluation
Generative AI evaluation measures the quality, reliability, diversity, and real-world usefulness of generated outputs and AI-assisted workflows.
See all 334 papers