As AI-generated images proliferate across digital platforms, reliable detection methods have become critical for combating misinformation and maintaining content authenticity. While numerous deepfake detection methods have been proposed, existing benchmarks predominantly evaluate fine-tuned models, leaving a critical gap in understanding out-of-the-box performance -- the most common deployment scenario for practitioners. We present the first comprehensive zero-shot evaluation of 16 state-of-the-art detection methods, comprising 23 pretrained detector variants (due to multiple released versions of certain detectors), across 12 diverse datasets, comprising 2.6million image samples spanning 291 unique generators including modern diffusion models. Our systematic analysis reveals striking findings: (1)no universal winner exists, with detector rankings exhibiting substantial instability (Spearmanρ: 0.01 -- 0.87 across dataset pairs); (2)a 37percentage-point performance gap separates the best detector (75.0% mean accuracy) from the worst (37.5%); (3)training data alignment critically impacts generalization, causing up to 20--60% performance variance within architecturally identical detector families; (4)modern commercial generators (FluxDev, Fireflyv4, Midjourneyv7) defeat most detectors, achieving only 18--30% average accuracy; and (5)we identify three systematic failure patterns affecting cross-dataset generalization. Statistical analysis confirms significant performance differences between detectors (Friedman test: χ2=121.01, p<10−16, KendallW=0.524). Our findings challenge the ``one-size-fits-all'' detector paradigm and provide actionable deployment guidelines, demonstrating that practitioners must carefully select detectors based on their specific threat landscape rather than relying on published benchmark performance.
Figures & tables
Method
Architecture
Training Data
Key Features
CNN-based Detectors
PatchCraft [ 38 ]
EfficientNet-B4
GAN + Diffusion images
Patch-level texture analysis
AIDE_progan [ 39 ]
ResNet-50
ProGAN
Training data alignment analysis
AIDE_GenImage [ 39 ]
ResNet-50
GenImage dataset
Training data alignment analysis
AIDE_sd14 [ 39 ]
ResNet-50
Stable Diffusion v1.4
Training data alignment analysis
CNNSpot [ 40 ] †
ResNet-50
GAN-based datasets
Classical CNN baseline
Table 1: Overview of AI-generated image detectors evaluated in this study, including recent 2025 state-of-the-art methods. All detectors are evaluated in their pre-trained form to assess zero-shot generalization.
Dataset
Generation Source (Models)
Data Provenance
#Gen.
Year
GenImage [ 52 ]
GANs and diffusion models
Official dataset (ImageNet-based)
8
2023
AIGCDetectionBench [ 40 ]
GANs and diffusion models
Community-curated benchmark
18
2023
WildFake [ 53 ]
Diverse generative models
Community-curated & in-house collection
diverse
2024
OpenFake [ 54 ]
Diffusion-based generators
Community-curated & third-party collected
hundreds
2023
Diffusion1kstep [ 55 ]
Stable Diffusion variants
Community-collected from official models
6
2023
SynthBuster [ 56 ]
Commercial text-to-image models
Third-party collected from official tools
9
2024
Table 2: Overview of datasets used in our benchmark, highlighting generation sources and data provenance. Evaluation is performed on uniformly subsampled splits (Section 3).
Figure 1: Zero-shot performance of 23 detectors across 12 datasets. Color scale: red = poor ( < 30%), yellow = random guess ( 50%), green = excellent ( > 90%). Methods and datasets are ordered by average performance. Substantial horizontal variation (dataset-dependent performance) and vertical stratification (detector quality gaps) are evident.
Detector
Mean
Median
Std
Range
Top-5 Detectors
Community-Forensics [ 47 ]
0.780
0.821
0.155
[0.50, 0.97]
SAFE [ 17 ]
0.688
0.812
0.298
[0.03, 1.00]
PatchCraft [ 38 ]
0.675
0.742
0.190
[0.39, 0.90]
DRCT CLIP-ViT (SD v2.0) [ 45 ]
0.718
0.696
0.171
[0.52, 0.94]
DRCT CLIP-ViT (SD v1.4) [ 45 ]
0.721
0.677
0.184
[0.33, 0.98]
Table 3: Performance summary: Top-5 and bottom-5 detectors by median accuracy. Mean accuracy calculated after excluding Community-Forensics v0.2 and v0.3.
Figure 2: Performance distribution of 23 detectors across all datasets. Box boundaries: 25th/75th percentiles; center line: median; whiskers: min/max. Red dashed line: random guess baseline (50%). Wide boxes indicate unstable performance; narrow boxes suggest consistency. Top detectors maintain high median accuracy, while bottom detectors cluster near chance level.
Figure 3: Detector ranking trajectories across 12 datasets. Top-5 methods (solid lines, circles) and bottom-5 methods (dashed lines, squares) shown. Steep slopes indicate dataset-dependent rankings. Most detectors shift by more than 10 positions across datasets, with maximum fluctuation up to 18 positions (e.g., DRCT_clip_vit variants). Community-Forensics shows relative stability (rank std=1.27).
Figure 4: Spearman rank correlation matrix between all dataset pairs. Green = high correlation ( > 0.7); red = low correlation ( < 0.3). Moderate correlations (median ρ =0.52) indicate partially overlapping but distinct evaluation dimensions.
Figure 5: Dataset difficulty ranking by mean detector accuracy averaged across all evaluated detectors . Error bars denote the standard deviation across detectors. The red dashed line indicates random guessing (50%). Dataset difficulty is defined by the average performance across detectors rather than peak performance of individual methods . Easiest datasets (GenImage, AIGCDetectionBench) exceed 80% mean accuracy, while the hardest datasets (community_forensics_test, MNW_fake, Nano-banana) fall below 45%.
Figure 6: AUC vs. accuracy correlation across all experiments. Red dashed diagonal: perfect calibration (AUC = accuracy). Strong positive correlation (Pearson r =0.82, p<0.001 ) confirms general metric agreement.
Figure 7: Training data alignment impact on detector performance. Left: AIDE family (ResNet-50 backbone) with ProGAN, GenImage, and SD v1.4 training data. Right: DRCT family (CLIP-ViT and ConvNeXt backbones) with SD v1.4 and v2.0 training. Identical architectures show 20–60% performance variance across datasets depending on training data alignment with test generators.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Detector
GenImage
AIGCDetectionBench
wildfake
OpenFake
Diffusion1kstep
GPT_4o
synthbuster
Chameleon
AI-GenBench
Nano-banana
MNW_fake
community_test
Community-Forensics
1.000
0.994
0.969
0.928
0.962
1.000
0.975
0.896
0.964
0.812
0.879
–
Community-Forensics
0.969
0.987
0.947
0.886
0.902
0.998
0.947
0.821
0.878
0.782
0.787
0.596
Community-Forensics orig
0.903
0.963
0.925
0.821
0.861
0.945
0.892
0.754
0.821
0.660
0.712
0.500
SAFE
0.998
0.952
0.984
0.867
0.881
0.956
0.092
0.812
0.724
0.538
0.625
0.032
PatchCraft
0.904
0.879
0.789
0.945
0.742
0.863
0.789
0.742
0.663
0.497
0.576
0.393
AIDE progan
0.985
0.967
0.932
0.812
0.715
0.887
0.764
0.643
0.459
0.459
0.047
0.154
Appendix
Table 4: Complete performance matrix: accuracy of all 19 detectors across all 12 datasets. Best performance per dataset in bold . Missing values indicate detector-dataset combinations not evaluated.
AI-generated image detection identifies synthetic or AI-edited images and develops methods that generalize across generators, domains, and image transformations.
Recent advances in visual generative models have enabled the creation of highly realistic, fully AI-generated images without relying on real source content. While beneficial for many applications, these models also pose significant societal risks, as they can be easily exploited to produce convincing Deepfakes. Detecting them represents a foundational yet challenging problem in AI media forensics, requiring detectors to reliably extract the inherent artifacts imprinted by generative architectures. In this Review, we provide a systematic overview of fully AI-generated image detection. Following the standard detector design pipeline, we focus on two key components: dataset construction and artifact extraction. We analyze how dataset design influences the generalization and robustness of learned artifacts, and categorize existing artifact extraction methods based on the primary inductive priors leveraged to isolate artifacts. Within this framework, we systematically review existing works. Finally, we highlight open problems and envision several future directions for developing more robust and generalizable detectors. Reviewed works in this survey can be found at https://github.com/zju-pi/Awesome-Fully-AI-Generated-Image-Detection.
Qijie Xu, Can Wang, Jiawei Chen +2
State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China. · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, Hangzhou, China. · Institute for Artificial Intelligence and Data Science, University at Buffalo, State University of New York, Buffalo, USA.
The rapid advancement of generative models has blurred the boundary between synthetic and real imagery, creating an urgent need for reliable deepfake detection. Yet most existing approaches rely on massive real--fake datasets, which are increasingly difficult to maintain as new generators continue to emerge. In this work, we investigate how much information about image authenticity is already encoded in modern multimodal vision representations. We find that frozen multimodal encoders naturally separate real and synthetic images in their embedding space, enabling a simple linear classifier to achieve strong performance without task-specific fine-tuning. Motivated by this observation, we develop a representation-aware data curation strategy that selects a compact set of representative generators for training. The resulting training set contains only 10K images, compared to 288K in AIGIBench and 4M in OpenFake, while improving robustness to unseen generators and distribution shifts. We additionally introduce RealWorldBench, a benchmark consisting of modern camera photographs, contemporary stock images, and outputs from recent commercial generators. Experiments across multiple benchmarks show that combining frozen multimodal representations with carefully curated training data provides a simple and effective approach to AI-generated image detection.
The rapid progress of text-to-image models has made AI-generated images increasingly realistic, posing significant challenges for accurate detection of generated content. While training-based detectors often suffer from limited generalization to unseen images, training-free approaches offer better robustness, yet struggle to capture subtle discrepancies between real and synthetic images. In this work, we propose a training-free AI-generated image detection method that measures representation sensitivity to structured frequency perturbations, enabling detection of minute manipulations. The proposed method is computationally lightweight, as perturbation generation requires only a single Fourier transform for an input image. As a result, it achieves one to two orders of magnitude faster inference than most training-free detectors.Extensive experiments on challenging benchmarks demonstrate the efficacy of our method over state-of-the-art (SoTA). In particular, on OpenFake benchmark, our method improves AUC by nearly 10% compared to SoTA, while maintaining substantially lower computational cost.
Ryosuke Sonoda, Ramya Srinivasan
Fujitsu Ltd., Japan · Fujitsu Research of America, Inc., USA