As AI generated images are getting popular these days, discussions on such generations are widely existed in current public forums, software repositories, and photography tutorials. Moreover, with the help of LLM-assisted code translation, malicious actors can adopt such information to evade detectors without specialized knowledge or local GPU execution of the resulting tools. To evaluate such risks, we introduce WildFakeToolBench, a benchmark suite of 156 executable image-processing tools derived from 220 public sources across 142 domains. Each tool retains its source evidence and passes syntax, execution, and output-distortion checks without using feedback from the evaluated detectors. On 105 generated images from five generators, we evaluate six state-of-the-art detectors with images processed by the tools in our collected bench to assess their effectiveness against such in-the-wild risks. The strongest individual tools achieve attack success rates of 25.25%--100% across detectors, exceeding the strongest of six selected image-processing baselines for each detector. Evaluating subsets of an eight-tool pool yields worst-case coverage above 80% for every detector. Effectiveness is sparse and detector-specific: high coverage across candidates does not imply reliable one-shot evasion. These findings not only highlight the potential of such web-sourced risks, but also show the potential on establishing AI-image detector robustness evaluation framework based on those in-the-wild information.
Figures & tables
Figure 1: WildFakeToolBench overview. Threat Model (Left): a non-expert user searches public web experience and uses LLM-assisted translation to obtain CPU-compatible image-processing tools for detector evasion. WildFakeToolBench (Right): we organize the recovered methods into a benchmark of 156 tools across ten processing classes. The paired images illustrate the software print_rescan transformation.
Figure 2: Source and tool composition by source category. Each label reports the count and its percentage of all 220 collected sources (upper) or 156 validated tools (lower). Colors identify the same categories in both charts. Percentages are rounded to one decimal place. Full source counts, per-source yields, and largest contributors are retained in Table 2 .
Detector
∣Bd∣
Max ASR
Mean ASR
Base Max
N>base
VLM
FakeVLM
101
45.54%
3.22%
6.93%
16
SIDA-13B
99
25.25%
4.84%
12.12%
10
FakeShield
64
46.88%
11.60%
39.06%
3
GPT-5.6-luna
72
66.67%
21.87%
33.80%
14
Classifier
CommForensics
48
95.83%
21.77%
41.67%
23
AIDE
85
100.00%
23.10%
55.29%
25
Table 1: Single-tool ASR on initially detected images. ∣Bd∣ is number of images which are detectable for sepcific detectors among 105 inputs. Max and Mean summarize the web-derived suite; Base Max is the strongest of six selected baselines; N>base counts tools exceeding Base Max. Resizing tools are excluded from the FakeShield comparison, following the original evaluation’s output-quality exclusion.
Source category
Src.
Tools
Tools/Src
Largest contributor
Reddit
34
20
0.59
reddit.com (20)
X / Twitter
4
11
2.75
x.com (11)
GitHub
6
11
1.83
github.com (11)
Medium
5
8
1.60
medium.com (7)
AI image-tool sites
34
58
1.71
imagera.ai (10)
Diffusion guidance
4
8
2.00
stable-diffusion-art.com (8)
Table 2: Full benchmark source statistics. Named platforms are listed separately; other sites are grouped by content type. Tools/Src may exceed one because a source can provide several methods.
Two
Three
Four
Detector
Max
> 25%
> 50%
Max
> 25%
> 50%
Max
> 25%
> 50%
VLM
FakeVLM
63.37%
16
6
84.16%
46
30
92.08%
65
63
SIDA-13B
49.49%
15
0
58.59%
55
12
68.69%
70
52
FakeShield
64.06%
20
4
71.88%
55
27
79.69%
70
70
GPT-5.6-luna
69.01%
28
23
81.69%
56
56
87.32%
70
70
Neural
CommForensics
100.00%
28
27
100.00%
56
56
100.00%
70
70
Table 3: Candidate-coverage results for all two-, three-, and four-tool available sets selected from the fixed eight-tool pool defined in Section 4.3 . The evaluation includes 28 , 56 , and 70 sets of the respective sizes. Max : the highest stacked ASR among available tool sets of that size. > 25% / > 50% : the number of sets exceeding the corresponding ASR threshold.
Figure 3: Maximum candidate coverage (stacked ASR) over k -tool available sets from the performance-selected eight-tool pool.
Figure 4: Best reported ASR for applying all k tools sequentially in a chain. Longer chains need not improve performance.
Detector
Single
ASR
Two
ASR
Three
ASR
Four
ASR
FakeVLM
AC
45.54%
AC, DS
63.37%
RS, DS, DR
84.16%
RS, CP, DS, DR
92.08%
SIDA-13B
AC
25.25%
CP, AC
49.49%
CP, AC, DR
58.59%
CP, CW, AC, DR
68.69%
FakeShield
DS
46.88%
CP, CW
64.06%
CP, AC, CW
71.88%
CP, AC, CW
79.69%
GPT-5.6-luna
LT
66.67%
CP, DR
69.01%
CP, CW, DR
81.69%
CP, CW, AC, DR
87.32%
CommForensics
CW
95.83%
DR, CW
100.00%
DS, DR, CW
100.00%
AC, DS, DR, CW
100.00%
AIDE
RS
100.00%
RS, DR
100.00%
RS, DS, DR
100.00%
CP, AC, DS, DR
100.00%
Table 4: Operation categories represented in the highest-ASR available sets. For each detector, Single reports the category and ASR of the strongest individual tool. The two-/three-/four-tool columns report the distinct categories represented in the highest-ASR available tool set of each size and its corresponding stacked ASR. AC=Analog/capture, DS=De-stylization, DR=Detail recovery, CP=Composition, CW=Color/white balance, RS=Resampling, and LT=Local texture. Categories are defined in Appendix B .
Detector
Single
ASR
Two
ASR
Three
ASR
Four
ASR
FakeVLM
AI
45.54%
AI, MD
63.37%
AI, MD, RD
84.16%
AI, RD, MD
92.08%
SIDA-13B
RD
25.25%
RD
49.49%
RD
58.59%
RD, AI
68.69%
FakeShield
AI
46.88%
RD, ES
64.06%
RD, AI, ES
71.88%
RD, AI, ES
79.69%
CommForensics
AI
95.83%
RD, ES
100.00%
MD, RD, ES
100.00%
RD, MD, ES
100.00%
AIDE
AI
100.00%
AI, RD
100.00%
AI, MD, RD
100.00%
RD, MD
100.00%
GPT-5.6-luna
TM
66.67%
RD
69.01%
RD, AI
81.69%
RD, AI
87.32%
Table 5: Source platforms represented in the highest-ASR available sets. For each detector, Single reports the source platform and ASR of the strongest individual tool. The two-/three-/four-tool columns report the source platforms represented in the highest-ASR available tool set of each size and its corresponding stacked ASR. AI=AI tool sites, RD=Reddit, MD=Medium, ES=Editing software, and TM=Tech media.
Figure 5: Tool counts across ten assigned processing categories. Categories describe intended operations rather than verified perceptual improvements.
Figure 6: Original images and the images processed by our collected print_rescan in the tool bench.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Number of released tools whose single-tool ASR exceeds the strongest of the 6 academic-sourced baseline tools, per detector.
AI-generated images linked to news events and online claims can mislead audiences, enable fraud, or cause public controversy. We introduce WildAIGC-Bench, built through recurring collection across news websites, fact-checking sources, and related social media content. The pipeline combines broad coverage, timely updates, and human-verified labels through scheduled discovery, automatic screening and preliminary annotation, and human review of image-specific evidence. The benchmark contains 407 verified AI-generated or edited images, each accompanied by a source webpage link and an evidence summary. Images retain the conditions encountered online, including compression, screenshots, and recapture. Evaluation of dedicated visual detectors, multimodal large language models, and a simple Perception Encoder baseline reveals substantial recall gaps relative to existing benchmarks. WildAIGC-Bench provides both a practical detection challenge and a basis for continued data expansion and source-based verification. The dataset is released at https://github.com/YuZijian/WildAIGC-Bench.
As AI-generated images proliferate across digital platforms, reliable detection methods have become critical for combating misinformation and maintaining content authenticity. While numerous deepfake detection methods have been proposed, existing benchmarks predominantly evaluate fine-tuned models, leaving a critical gap in understanding out-of-the-box performance -- the most common deployment scenario for practitioners. We present the first comprehensive zero-shot evaluation of 16 state-of-the-art detection methods, comprising 23 pretrained detector variants (due to multiple released versions of certain detectors), across 12 diverse datasets, comprising 2.6million image samples spanning 291 unique generators including modern diffusion models. Our systematic analysis reveals striking findings: (1)no universal winner exists, with detector rankings exhibiting substantial instability (Spearmanρ: 0.01 -- 0.87 across dataset pairs); (2)a 37percentage-point performance gap separates the best detector (75.0% mean accuracy) from the worst (37.5%); (3)training data alignment critically impacts generalization, causing up to 20--60% performance variance within architecturally identical detector families; (4)modern commercial generators (FluxDev, Fireflyv4, Midjourneyv7) defeat most detectors, achieving only 18--30% average accuracy; and (5)we identify three systematic failure patterns affecting cross-dataset generalization. Statistical analysis confirms significant performance differences between detectors (Friedman test: χ2=121.01, p<10−16, KendallW=0.524). Our findings challenge the ``one-size-fits-all'' detector paradigm and provide actionable deployment guidelines, demonstrating that practitioners must carefully select detectors based on their specific threat landscape rather than relying on published benchmark performance.
Recent AI-generated image (AIGI) detectors perform well on natural-image benchmarks, but their behavior on text-rich forgeries, such as fabricated screenshots, documents, and news pages prevalent in misinformation, remains untested. We introduce TextFake, a 20,000-image benchmark for text-rich AIGI detection spanning 28 languages, 4 topic categories, and 2 scene modalities. Fake images are synthesized via a four-stage pipeline that annotates real images along three controlled dimensions and generates counterparts through distribution-aligned structured prompting, ruling out covariate shortcuts. Zero-shot evaluation of 14 specialized detectors and 3 frontier VLM APIs reveals a large systematic gap: no method exceeds 80% accuracy, with some dropping over 60% from natural-image benchmarks. Diagnostic evaluations identify three failure modes: the Text Density Curse, where dense glyphs overwhelm low-level detectors; Cloaking via Rendering Fidelity, where stronger text rendering suppresses enerative artifacts; and Threshold Collapse, where routine perturbations drive detectors toward chance-level performance.
Yuning Zhang, Changtao Miao, Mingyu Liao +5
School of Cyber Science and Technology, University of Science and Technology of China · Anhui Province Key Laboratory of Digital Security · Individual Researcher