DELTA: Robustly Training Label-Conditional Diffusion Models with Weak Annotations
Organizations: The University of Tokyo, Tokyo, Japan · RIKEN AIP, Tokyo, Japan · Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE
Abstract
While label-conditional diffusion models have recently exhibited remarkable generative capabilities, their success heavily relies on massive, cleanly labeled datasets. In practice, categorical supervision is rarely perfect: it is often corrupted by noise, clouded by ambiguity, or partially missing. Training directly on such weak annotations severely degrades generation quality, and existing robust methods offer only fragmented solutions that depend on scarce auxiliary priors. To overcome these limitations, we propose DELTA, a unified framework that robustly trains Diffusion modELs across all three weak annoTAtion types. By treating the unknown true label as a latent variable, DELTA optimizes a principled variational objective that jointly recovers the clean data distribution and infers the true label posteriors. To make this joint training computationally tractable, we further introduce a median-centered timestep sampling strategy that efficiently concentrates estimation where the diffusion process is most informative. Extensive experiments on synthetic and real-world benchmarks demonstrate that DELTA produces high-fidelity, class-consistent samples across diverse weak supervision scenarios, outperforming specialized baselines while demanding strictly less prior knowledge.
Figures & tables
| Metric | Noisy label | Ambiguous label | Semi-supervised data | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Sym-40% | Asym-40% | Random | Class-dept. | 1% Labels | 10% Labels | Clean | ||||||||||
| Vanilla | DELTA | Vanilla | DELTA | Vanilla | DELTA | Vanilla | DELTA | Vanilla | DELTA | Vanilla | DELTA | |||||
| CIFAR-10 | uncond | FID | 3.33 | 3.47 | 3.23 | 3.10 | 7.76 | 2.26 | 11.75 | 2.77 | 3.16 | 3.12 | 2.93 | 2.89 | 2.05 | |
| IS | 9.56 | 9.68 | 9.02 | 9.73 | 9.09 | 9.80 | 9.62 | 9.68 | 10.03 | 10.57 | 9.80 | 9.83 | 10.61 | |||
| Density | 101.39 | 109.75 | 100.06 | 109.69 | 103.21 | 106.49 | 108.76 | 109.06 | 97.19 | 108.18 | 99.96 | 108.87 | 112.59 | |||
| Coverage | 81.12 | 81.21 | 80.71 | 81.30 | 68.45 | 82.69 | 64.90 | 81.52 | 78.44 | 81.00 | 81.85 | 82.00 | 83.27 | |||
| Sym-40% | Asym-40% | |||||||||||||
| Method | FID | IS | Dens. | Cov. | CW-FID | CW-Dens. | CW-Cov. | FID | IS | Dens. | Cov. | CW-FID | CW-Dens. | CW-Cov. |
| Vanilla | 3.33 | 9.56 | 101.39 | 81.12 | 29.84 | 72.98 | 73.39 | 3.23 | 9.02 | 100.06 | 80.71 | 14.70 | 90.85 | 79.63 |
| CAD [ 27 ] | 4.10 | 9.08 | 101.48 | 77.04 | 19.66 | 93.51 | 76.43 | 3.87 | 9.16 | 98.27 | 75.81 | 14.33 | 99.25 | 78.12 |
| TDSM [ 12 ] | 3.85 | 9.40 | 106.32 | 79.66 | 16.42 | 101.27 | 78.63 | 3.96 | 10.12 | 107.85 | 80.47 | 14.96 | 102.67 | 78.99 |
| SBDC [ 30 ] | 4.25 | 8.95 | 97.83 | 74.92 | 21.30 | 89.83 | 75.37 | 4.02 | 9.05 | 95.14 | 73.29 | 15.70 | 96.14 | 77.25 |
| RTDC [ 28 ] | 4.00 | 9.20 | 104.51 | 78.35 | 17.92 | 98.13 | 77.70 | 3.85 | 9.30 | 103.63 | 78.06 | 13.92 | 101.03 | 78.68 |
| Dataset | Type | CE | Mixup [ 56 ] | Coteaching [ 57 ] | ELR [ 46 ] | PENCIL [ 58 ] | Vanilla | DELTA |
|---|---|---|---|---|---|---|---|---|
| Sym-40% | 76.18 0.26 | 92.21 0.03 | 92.17 0.34 | 93.13 0.13 | 90.85 0.58 | 90.11 1.24 | 93.40 0.40 | |
| Asym-40% | 82.01 0.06 | 92.01 1.02 | 92.78 0.25 | 92.82 0.09 | 91.77 0.69 | 85.41 0.96 | 93.20 0.30 | |
| Sym-40% | 67.22 0.26 | 84.26 0.64 | 86.54 0.57 | 85.68 0.13 | 85.91 0.26 | 80.22 0.10 | 88.63 0.12 | |
| Asym-40% | 76.98 0.42 | 83.21 0.85 | 79.38 0.39 | 81.32 0.31 | 84.89 0.49 | 86.31 0.10 | 88.83 0.33 | |
| Type | PRODEN [ 47 ] | IDGP [ 59 ] | PiCO [ 60 ] | CRDPLL [ 61 ] | DIRK [ 62 ] | Vanilla | DELTA | |
| Random | 93.31 0.07 | 92.26 0.25 | 93.32 0.12 | 94.03 0.14 | 94.11 0.22 | 80.20 1.29 | 94.27 0.55 |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Symbol | Definition |
|---|---|
| Data sample | |
| Random variable of data samples | |
| Latent clean class label | |
| Random variable of clean class labels | |
| Observed weak annotation | |
| Random variable of weak annotations |
Conditional Diffusion Models
Conditional diffusion models generate or infer data by steering iterative denoising with observations, labels, text, physical constraints, or other inputs.
See all 297 papers