A Difference-in-Difference Approach to Detecting AI-Generated Images
Organizations: The London School of Economics and Political Science · University of Birmingham
Abstract
Diffusion models are able to produce AI-generated images that are almost indistinguishable from real ones. This raises concerns about their potential misuse and poses substantial challenges for detecting them. Many existing detectors rely on reconstruction error -- the difference between the input image and its reconstructed version -- as the basis for distinguishing real from fake images. However, these detectors become less effective as modern AI-generated images become increasingly similar to real ones. To address this challenge, we propose a novel difference-in-difference method. Instead of directly using the reconstruction error (a first-order difference), we compute the difference in reconstruction error -- a second-order difference -- for variance reduction and improving detection accuracy. Extensive experiments demonstrate that our method achieves strong generalization performance, enabling reliable detection of AI-generated images in the era of generative AI.
Figures & tables
| Train set & Gen. model | ImageNet & ADM | |||||
|---|---|---|---|---|---|---|
| Eval set | Gen. model | DID | DIRE | LaRE 2 | AERO. | UFD |
| LSUN-B | ADM | 99.5 | 99.5 | 50.0 | 52.3 | 63.9 |
| PNDM | 97.6 | 97.6 | 55.4 | 49.5 | 91.8 | |
| DDPM | 99.5 | 99.8 | 56.6 | 56.7 | 76.5 | |
| iDDPM | 99.4 | 99.6 | 50.0 | 50.3 | 85.1 | |
| SDv1 | 99.7 | 99.9 | 100 | 50.6 | 94.5 | |
| Train set & Gen. model | LAION & Kan.3 | Imp. | LAION & SDXL | Imp. | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Eval set | Gen. model | DID | DIRE | LaRE 2 | AERO. | UFD | DID | DIRE | LaRE 2 | AERO. | UFD | ||
| LSUN-B | ADM | 90.7 | 88.1 | 50.3 | 50.0 | 50.8 | 21.85% | 95.0 | 94.3 | 50.2 | 50.0 | 50.3 | 13.04% |
| PNDM | 78.2 | 73.5 | 71.2 | 49.8 | 51.5 | 17.55% | 81.7 | 74.6 | 51.6 | 49.8 | 50.9 | 27.95% | |
| DDPM | 88.7 | 85.2 | 56.5 | 56.4 | 51.5 | 24.05% | 95.1 | 93.3 | 57.1 | 56.4 | 50.5 | 27.73% | |
| iDDPM | 80.1 | 77.5 | 50.0 | 49.8 | 54.2 | 11.33% | 89.0 | 86.8 | 50.0 | 49.8 | 53.3 | 16.29% | |
| SDv1 | 98.6 | 97.8 | 99.9 | 49.8 | 49.7 | - | 98.4 | 98.3 | 99.7 | 49.8 | 50.5 | - | |
| Train set & Gen. model | Eval set | Gen. model | Accuracy (%) | Imp. | ||||
|---|---|---|---|---|---|---|---|---|
| DID | DIRE | LaRE 2 | AERO. | UFD | ||||
| LAION & Kan.3 | LSUN-B | StyleGAN | 95.8 | 94.3 | 50.4 | 49.8 | 50.8 | 26.32% |
| ProjectedGAN | 93.4 | 91.0 | 51.2 | 49.8 | 50.7 | 26.52% | ||
| Diff-ProjectedGAN | 92.7 | 90.2 | 51.9 | 49.8 | 51.1 | 25.51% | ||
| Avg. | 93.95 | 91.82 | 51.12 | 49.75 | 50.85 | 26.07% | ||
| LAION & SDXL | LSUN-B | StyleGAN | 96.8 | 96.3 | 50.7 | 49.8 | 50.9 | 12.16% |
| Eval set | Gen. model | Accuracy (%) | ||
|---|---|---|---|---|
| DID | DIRE | |||
| LSUN-B | ADM | 86.5 | 69.0 | 84.0 |
| PNDM | 83.0 | 81.5 | 67.5 | |
| DDPM | 82.0 | 78.0 | 71.5 | |
| ImageNet | ADM | 98.9 | 98.4 | 91.0 |
| SDv1 | 98.9 | 98.7 | 93.5 | |
| Gen. model | LAION & Kan.3 (PNG-only training) | LAION & SDXL | ||||
|---|---|---|---|---|---|---|
| PNG | JPG95 | JPG75 | PNG | JPG95 | JPG75 | |
| Kan.3 | 97.2 | 98.9 | 95.1 | 90.6 | 85.4 | 70.4 |
| SDXL | 93.2 | 88.9 | 76.3 | 97.3 | 97.1 | 85.6 |
| Vega | 89.1 | 84.3 | 72.3 | 95.2 | 93.0 | 76.3 |
| Playground v2.5 | 94.9 | 92.9 | 82.7 | 91.6 | 88.0 | 73.4 |
| Stable Cascade | 96.2 | 95.6 | 89.4 | 94.4 | 87.2 | 71.7 |
| Eval set | Gen. model | DID 0.29 | DID 0.5 | Effort | DDA |
|---|---|---|---|---|---|
| GenImage | ADM | 99.90 / 1.000 / 0.00 | 100.00 / 1.000 / 0.00 | 98.55/ 0.999 /0.40 | 99.30/ 1.000 / 0.20 |
| BigGAN | 99.90 / 1.000 / 0.00 | 99.95 / 1.000 / 0.00 | 95.45/0.995/2.60 | 98.70/ 0.999 / 0.50 | |
| GLIDE | 99.90 / 1.000 / 0.00 | 100.00 / 1.000 / 0.00 | 98.35/ 0.999 / 0.20 | 99.20/ 1.000 / 0.20 | |
| Midjourney | 99.70 / 1.000 / 0.00 | 99.70 / 1.000 / 0.00 | 57.25 /0.744/80.00 | 54.95/ 0.845 / 56.30 | |
| SD-v1.4 | 99.45 / 1.000 / 0.00 | 99.50 / 1.000 / 0.00 | 86.70/ 0.975 / 13.00 | 72.81/0.965/15.60 | |
| SD-v1.5 | 99.50 / 1.000 / 0.00 | 99.40 / 1.000 / 0.00 | 83.95/ 0.975 / 11.30 | 72.65/0.962/16.50 |
| Eval set | Gen. model | DID 0.29 | DID 0.5 | Effort | DDA |
| GenImage | ADM | 99.20 / 1.000 / 0.00 | 99.50 / 1.000 / 0.00 | 86.30/ 0.986 /6.70 | 98.30/ 0.999 / 0.10 |
| BigGAN | 99.10 / 1.000 / 0.00 | 99.40 / 1.000 / 0.00 | 80.15/0.878/ 39.00 | 98.50/ 1.000 / 0.00 | |
| GLIDE | 99.20 / 1.000 / 0.00 | 99.50 / 1.000 / 0.00 | 85.85/0.973/12.80 | 97.50/ 0.998 / 0.90 | |
| Midjourney | 99.20 / 1.000 / 0.00 | 99.50 / 1.000 / 0.00 | 57.00/0.597/ 90.20 | 56.15/ 0.810 / 58.80 | |
| SD-v1.4 | 99.20 / 1.000 / 0.00 | 99.50 / 1.000 / 0.00 | 85.30/0.958/19.10 | 82.91/ 0.969 / 13.30 | |
| SD-v1.5 | 99.20 / 1.000 / 0.00 | 99.50 / 1.000 / 0.00 | 84.75/0.950/22.10 | 83.60/ 0.967 / 17.00 |
| Method | Reconstruction | Testing |
|---|---|---|
| Time (Memory) | Time (Memory) | |
| LaRE2 | 0.09 (33.34) | 0.66 (9.74) |
| DIRE | 1.01 (396.48) | 0.34 (21.77) |
| DID | 2.01 (398.39) | 0.45 (22.55) |
| Train set | Eval set | LAION | ImageNet |
|---|---|---|---|
| ImageNet | DID 0.29 | 99.29/1.00/0.00 | 99.85/1.00/0.00 |
| DID SC | 99.03/1.00/0.00 | 99.45/1.00/0.00 |