DUO: Dual Readouts of Diffusion Models for OOD Detection
Organizations: University of California, San Diego · Washington University in St. Louis
Abstract
Diffusion models are powerful generative methods for learning and sampling from complex data distributions. Safely deploying these models requires detecting inputs that fall outside their training distribution. Because diffusion models recover clean samples from noisy inputs, reconstruction error is often used for out-of-distribution (OOD) detection. This approach assumes that in-distribution (ID) inputs are reconstructed more accurately than OOD inputs. However, OOD inputs may be reconstructed equally well or even better, making this score unreliable. We therefore propose the leading sensitivity spectrum of intermediate denoising features as a complementary OOD signal and demonstrate that it remains informative when reconstruction error is misleading. We introduce DUal-readout of diffusion models for OOD detection (DUO), a fully unsupervised, training-free OOD detector that combines reconstruction error and feature sensitivity. DUO improves average AUROC by approximately 6% points over the strongest baselines while substantially reducing inference cost, and achieves SOTA on near-OOD and high-resolution benchmarks.
Figures & tables
| InD | CelebA vs. | CIFAR-10 vs. | CIFAR-100 vs. | SVHN vs. | Avg | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| OOD | CIFAR-10 | CIFAR-100 | SVHN | CIFAR-100 | SVHN | CelebA | CIFAR-10 | CelebA | SVHN | CIFAR-10 | CIFAR-100 | CelebA | |
| DoS | 0.630 | 0.615 | 0.808 | 0.504 | 0.752 | 0.456 | 0.491 | 0.520 | 0.777 | 0.911 | 0.904 | 0.956 | 0.693 |
| TT | 0.676 | 0.655 | 0.773 | 0.558 | 0.714 | 0.469 | 0.538 | 0.464 | 0.648 | 0.957 | 0.961 | 0.994 | 0.701 |
| WAIC | 0.589 | 0.569 | 0.793 | 0.476 | 0.760 | 0.469 | 0.502 | 0.530 | 0.782 | 0.978 | 0.974 | 0.955 | 0.698 |
| Method | LMD | DDPM-OOD | DiffPath | GEPC | EigenScore | DUO (ours) |
|---|---|---|---|---|---|---|
| NFE | 10,000 | 350 | 10 | 16 | 7,000 | 197 |
| Latency (s) | 190.85 | 3.62 | 0.29 | 0.15 | 9.48 | 0.28 |
Appendix figures & tables24 assets
Supplementary material from the paper’s appendix.
Appendix
| ID | ||||
|---|---|---|---|---|
| CIFAR-10 | ||||
| CIFAR-100 | ||||
| SVHN | ||||
| CelebA |
| InD | CIFAR-10 vs. | CIFAR-100 vs. | Avg | ||
|---|---|---|---|---|---|
| OOD | C100 | TinyImageNet | C10 | TinyImageNet | |
| DDPM-OOD | 0.618 | 0.570 | 0.462 | 0.457 | 0.527 |
| LMD | 0.601 | 0.592 | 0.569 | 0.558 | 0.580 |
| GEPC | 0.582 | 0.634 | 0.427 | 0.629 | 0.568 |
| DiffPath | 0.554 | 0.993 | 0.475 | 0.995 | 0.754 |
| EigenScore | 0.880 | 0.899 | 0.642 | 0.755 | 0.794 |
| Method | FFHQ vs. | ||
|---|---|---|---|
| AFHQ | Microscopy | Avg | |
| LMD | 0.485 | 0.552 | 0.519 |
| GEPC | 0.818 | 0.834 | 0.826 |
| DiffPath | 0.593 | 0.998 | 0.796 |
| EigenScore | 0.532 | 0.899 | 0.716 |
| DUO (ours) | 0.999 | 0.999 | 0.999 |
| Method | ImageNet vs. | ||
|---|---|---|---|
| SVHN | Texture | Avg | |
| DDPM-OOD | 0.208 | 0.570 | 0.389 |
| LMD | 0.901 | 0.486 | 0.693 |
| DiffPath | 0.753 | 0.699 | 0.726 |
| DiffPathV2 | 0.547 | 0.860 | 0.703 |
| EigenScore | 0.755 | 0.709 | 0.732 |
| Method | CIFAR-10 vs. | CIFAR-100 vs. | Avg | ||||
|---|---|---|---|---|---|---|---|
| LSUN | iSUN | Textures | LSUN | iSUN | Textures | ||
| GEPC | 0.672 | 0.694 | 0.831 | 0.574 | 0.568 | 0.721 | 0.677 |
| DiffPath | 0.618 | 0.911 | 0.780 | 0.609 | 0.869 | 0.761 | 0.758 |
| EigenScore | 0.977 | 0.974 | 0.756 | 0.877 | 0.879 | 0.627 | 0.848 |
| DUO (ours) | 0.952 | 0.955 | 0.947 | 0.830 | 0.844 | 0.832 | 0.893 |
| InD | CelebA vs. | CIFAR-10 vs. | CIFAR-100 vs. | SVHN vs. | Avg | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| OOD | CIFAR-10 | CIFAR-100 | SVHN | CIFAR-100 | SVHN | CelebA | CIFAR-10 | CelebA | SVHN | CIFAR-10 | CIFAR-100 | CelebA | |
| DUO | 1.000 | 1.000 | 1.000 | 0.876 | 0.986 | 0.872 | 0.657 | 0.544 | 0.839 | 0.995 | 0.994 | 0.996 | 0.897 |
| DUO-OODVal | 1.000 | 1.000 | 1.000 | 0.898 | 0.982 | 0.941 | 0.645 | 0.769 | 0.817 | 0.996 | 0.995 | 1.000 | 0.920 |
| Environment | ID quality | Random | Medium | Med.-Replay | Expert | Avg. |
|---|---|---|---|---|---|---|
| HalfCheetah | Medium | 0.999 | – | 0.869 | 1.000 | 0.956 |
| Medium-Replay | 0.997 | 0.513 | – | 0.999 | 0.837 | |
| Hopper | Medium | 1.000 | – | 0.991 | 1.000 | 0.997 |
| Medium-Replay | 1.000 | 0.579 | – | 0.998 | 0.859 | |
| Walker2d | Medium | 1.000 | – | 0.940 | 0.993 | 0.978 |
| Medium-Replay | 1.000 | 0.516 | – | 0.882 | 0.799 |
| Probes | CelebA vs. | CIFAR-10 vs. | CIFAR-100 vs. | SVHN vs. | Avg | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CIFAR-10 | CIFAR-100 | SVHN | CIFAR-100 | SVHN | CelebA | CIFAR-10 | CelebA | SVHN | CIFAR-10 | CIFAR-100 | CelebA | ||
| 24 | 1.000 | 0.999 | 1.000 | 0.871 | 0.985 | 0.859 | 0.650 | 0.516 | 0.838 | 0.995 | 0.994 | 0.996 | 0.892 |
| 32 | 1.000 | 1.000 | 1.000 | 0.876 | 0.986 | 0.872 | 0.657 | 0.544 | 0.839 | 0.995 | 0.994 | 0.996 | 0.897 |
| 48 | 1.000 | 1.000 | 1.000 | 0.881 | 0.986 | 0.883 | 0.662 | 0.573 | 0.838 | 0.995 | 0.994 | 0.996 | 0.901 |
| Eigenvalues | CelebA vs. | CIFAR-10 vs. | CIFAR-100 vs. | SVHN vs. | Avg | ||||||||
| CIFAR-10 | CIFAR-100 | SVHN | CIFAR-100 | SVHN | CelebA | CIFAR-10 | CelebA | SVHN | CIFAR-10 | CIFAR-100 | CelebA | ||
| Weight | CelebA vs. | CIFAR-10 vs. | CIFAR-100 vs. | SVHN vs. | Avg | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CIFAR-10 | CIFAR-100 | SVHN | CIFAR-100 | SVHN | CelebA | CIFAR-10 | CelebA | SVHN | CIFAR-10 | CIFAR-100 | CelebA | ||
| 0.991 | 0.978 | 0.997 | 0.861 | 0.988 | 0.800 | 0.633 | 0.501 | 0.899 | 0.861 | 0.866 | 0.900 | 0.856 | |
| 1.000 | 0.999 | 1.000 | 0.854 | 0.956 | 0.918 | 0.630 | 0.614 | 0.679 | 0.997 | 0.996 | 0.998 | 0.887 | |
| 0.996 | 0.993 | 0.558 | 0.601 | 0.148 | 0.827 | 0.524 | 0.697 | 0.122 | 0.993 | 0.994 | 1.000 | 0.704 | |
| (deployed) | 1.000 | 1.000 | 1.000 | 0.876 | 0.986 | 0.872 | 0.657 | 0.544 | 0.839 | 0.995 | 0.994 | 0.996 | 0.897 |
| Ensemble | CelebA vs. | CIFAR-10 vs. | CIFAR-100 vs. | SVHN vs. | Avg | ||||||||
| CelebA | CIFAR-10 | CIFAR-100 | SVHN | Avg | |
|---|---|---|---|---|---|
| AUROC |
Diffusion-Based OOD Detection
Diffusion-based out-of-distribution detection uses diffusion models to identify inputs that differ from the data a system was trained to handle.
See all 16 papers