Beyond Natural Images: Rethinking AI-Generated Image Detection in Documents
Authors: Zhangjie Fu, Jiazhen Yan, Yuanwen Chen, Xinquan Yu, Yanzhe Li, Hui Jiang, Lei Gao, Chenfu Bao
Organizations: Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology · Baidu Inc. · School of Computer Science and Engineering, MoE Key Laboratory of Information Technology, Guangdong Province Key Laboratory of Information Security Technology, Sun Yat-sen University · Tsinghua University
AI-generated image detection has attracted increasing attention, but existing evaluations mainly focus on natural images, leaving AI-generated document images largely underexplored. This omission is concerning because documents often appear in sensitive real-world scenarios, such as invoices, expense reports, certificates, and medical records. In this paper, we first construct a controlled diagnostic benchmark, AIGDoc-Pilot, and reveal that existing detectors suffer substantial performance degradation on AI-generated document images, with the mean AUC dropping by more than 7%. Based on this, we further reveal two document-specific properties behind this gap: generation artifacts exhibit strong spatial inconsistency across local regions, and text density significantly affects real-synthetic separability, where text-dense regions offer stronger discriminative evidence. Motivated by these findings, we construct AIGDoc, a larger document-centric dataset containing diverse real-world documents and AI-generated counterparts produced by multiple advanced generation and editing models. Extensive experiments on AIGDoc demonstrate that existing detectors still struggle to reliably identify AI-generated documents, while document-based training partially narrows the gap. Together, these results offer valuable insights for developing dependable and generalizable detectors in document-centric scenarios. The code and datasets will be made publicly available upon acceptance of the paper.
Figures & tables
Figure 1: (a) Diagnosis and Findings. Using the controlled diagnostic benchmark AIGDoc-Pilot, we reveal that existing detectors suffer substantial performance degradation on AI-generated document images. Further diagnosis shows that generated document artifacts exhibit strong spatial inconsistency across local regions, while text-dense regions provide stronger discriminative evidence for real-synthetic separability. (b) Datasets and Evaluations. We construct AIGDoc, a large-scale benchmark for AI-generated document image detection, comprising 27K synthetic images across 17 subsets, covering 13 generation methods from three categories.
Method
SDv2.1-VAE
HiDream- E1-Full
FLUX.1-dev
Qwen-Image
SDv3.5-Large
Mean
0.2
0.4
0.6
0.8
0.2
0.4
0.2
0.4
F.Acc.
AUC
13.21
15.41
6.61
9.17
17.98
29.54
11.56
17.06
10.64
12.48
14.37
69.78
DRCT
15.23
9.54
7.89
11.56
19.82
35.96
24.06
37.19
14.63
13.70
18.96+4.59
68.36-1.42
1.83
1.10
0.55
2.02
4.40
7.89
4.77
4.77
1.10
0.55
2.90
55.73
Effort
1.65
1.10
1.10
2.20
5.87
15.96
10.94
12.50
0.75
0.94
5.30+2.40
40.46-15.27
98.53
70.83
27.89
22.94
32.11
51.38
77.98
82.75
93.21
95.41
65.30
88.08
Table 1: Detection Performance Gap between AI-Generated Natural and Document Images on AIGDoc-Pilot. For each detector, the decision threshold is calibrated to maintain 95% Real Acc. Gray rows denote results on document images, while white rows denote results on natural images.
Method
Region
SDv2.1- VAE
HiDream- E1-Full
FLUX.1-dev
Qwen-Image
SDv3.5-Large
Mean
0.2
0.4
0.6
0.8
0.2
0.4
0.2
0.4
F.Acc.
AUC
No
72.33
39.00
14.67
17.00
34.00
28.00
62.67
65.67
45.67
47.67
42.67
82.16
Low
84.67
48.33
18.67
23.33
35.00
38.00
61.33
72.00
46.33
51.00
47.87 +5.20
83.56 +1.40
Mid
86.67
48.00
21.00
27.00
47.00
53.00
63.00
74.00
53.33
58.67
53.17 +10.50
85.02 +2.86
B-Free
High
96.33
47.00
28.00
34.00
53.00
56.67
69.67
77.00
56.33
67.33
58.53 +15.86
86.42 +4.26
No
80.00
47.33
14.00
16.67
13.00
28.00
62.67
66.00
51.67
66.33
44.57
81.67
Table 3: Document-Region Fake Acc. under Different Text Densities at 95% Real Acc. Each generator reports F.Acc. (%). We also report the mean F.Acc. and mean AUC across generation settings. Rows under each detector correspond to non-text, low-text, medium-text, and high-text regions.
Generator
UnivFD
DRCT
Effort
B-Free
DDA
MIRROR
OmniAID
PROBE
SICA
PE-LoRA
PE-LoRA †
PE-LoRA ‡
Text2Image
Z-Image-Turbo
49.4
61.8
85.2
52.3
61.4
49.9
49.8
55.1
44.9
68.0
92.8
90.5
HiDream-O1-Image
56.5
62.1
72.7
97.0
95.2
66.8
63.2
97.9
75.8
95.2
97.9
93.5
Seedream 4.5
50.1
51.7
58.2
95.2
91.8
52.8
53.3
99.5
61.8
98.0
97.4
95.7
Nano-Banana-2
50.0
46.2
60.2
66.0
58.9
50.4
51.0
63.9
55.4
59.8
67.5
60.5
GPT-Image-2
50.0
49.5
74.2
51.3
47.0
49.8
50.3
51.2
47.4
49.4
59.7
83.9
Table 5: Detection Accuracy across Different AI-Generated Document Image Settings. Each detector reports Acc. (%) for each generation setting.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 2: Qualitative Visualization of AIGDoc-Pilot Generation Results. For each real image, we show the generated counterpart and its difference map. Compared with natural images, document images exhibit much sparser generation-induced differences, which are mainly concentrated around text, table lines, and other high-frequency structures.
Method
SDv2.1-VAE
HiDream- E1-Full
FLUX.1-dev
Qwen-Image
SDv3.5-Large
Mean AUC
0.2
0.4
0.6
0.8
0.2
0.4
0.2
0.4
69.25
69.79
56.93
65.91
73.19
81.69
70.52
74.90
65.13
70.52
69.78
DRCT
67.88
57.76
61.23
58.91
66.63
79.09
75.30
80.94
67.28
68.54
68.36-1.42
48.80
49.02
41.53
55.24
64.54
72.91
61.74
64.30
47.95
51.31
55.73
Effort
30.70
27.26
29.86
36.48
48.73
64.40
57.17
59.23
24.46
26.32
40.46-15.27
99.63
90.46
74.47
70.94
74.83
83.55
94.24
95.68
98.14
98.80
88.08
Appendix
Table 6: Detection Performance Gap between AI-Generated Natural and Document Images on AIGDoc-Pilot. Each generator setting reports AUC (%). Gray rows denote results on document images, while white rows denote results on natural images.
Figure 3: Visualization of text-density-based crop regions and their corresponding residual maps. Low-text-density regions exhibit weaker and less discernible differences between generated and real images, whereas text-dense regions reveal more pronounced generation traces.
Method
Region
SDv2.1- VAE
HiDream- E1-Full
FLUX.1-dev
Qwen-Image
SDv3.5-Large
Mean
0.2
0.4
0.6
0.8
0.2
0.4
0.2
0.4
No
94.03
82.70
66.38
67.15
78.94
80.44
90.23
91.21
83.84
86.64
82.16
Low
96.63
82.89
67.51
68.90
79.69
84.52
90.09
93.23
83.92
88.26
83.56 +1.40
Mid
96.76
83.63
68.63
71.98
83.36
85.43
91.09
93.32
86.24
89.73
85.02 +2.86
B-Free
High
98.40
81.83
70.48
74.92
84.84
88.03
93.00
94.80
87.00
90.84
86.42 +4.26
No
95.78
85.87
67.37
64.15
68.58
75.90
89.94
91.51
86.22
91.34
81.67
Appendix
Table 7: Document-Region AUC under Different Text Densities. Each generator reports AUC (%). Rows under each detector correspond to non-text, low-text, medium-text, and high-text regions.
Method
Region
SDv2.1- VAE
HiDream- E1-Full
FLUX.1-dev
Qwen-Image
SDv3.5-Large
Mean
0.2
0.4
0.6
0.8
0.2
0.4
0.2
0.4
F.Acc.
AUC
No
86.33
54.67
29.00
29.67
44.33
49.33
66.67
70.00
60.00
60.33
55.03
82.16
Low
85.00
45.67
20.00
21.33
37.00
43.33
56.33
69.33
46.67
50.33
47.50 -7.53
83.56 +1.40
Mid
80.00
39.67
17.33
22.67
36.00
46.67
55.33
65.00
48.67
49.67
46.10 -8.93
85.02 +2.86
B-Free
High
82.00
38.00
18.00
22.33
33.00
42.00
56.00
65.33
42.33
49.33
44.83 -10.20
86.42 +4.26
No
83.00
52.67
20.00
17.33
17.33
26.00
49.33
52.00
59.00
66.00
44.27
81.67
Appendix
Table 8: Document-Region Fake Acc. under Different Text Densities with the 0.5 Threshold. Each generator reports F.Acc. (%). We also report the mean F.Acc. and mean AUC across generation settings. Rows under each detector correspond to non-text, low-text, medium-text, and high-text regions.
Generator
UnivFD
DRCT
Effort
B-Free
DDA
MIRROR
OmniAID
PROBE
SICA
PE-LoRA
PE-LoRA †
PE-LoRA ‡
Text2Image
Z-Image-Turbo
48.0
69.5
93.2
60.9
78.4
56.4
31.8
86.1
42.9
84.2
97.7
97.2
HiDream-O1-Image
82.6
66.7
83.5
99.6
99.1
93.5
87.1
99.9
78.9
99.3
99.9
98.7
Seedream 4.5
67.8
51.4
64.0
99.0
97.8
67.2
41.0
100.0
64.8
99.6
99.9
99.8
Nano-Banana-2
55.9
45.2
63.6
70.6
63.6
67.8
47.0
78.5
57.5
58.2
79.0
64.5
GPT-Image-2
67.7
51.0
83.3
47.3
46.9
64.4
38.2
75.8
48.3
35.8
72.7
94.3
Appendix
Table 9: Detection AUC across Different AI-Generated Document Image Settings. Each detector reports AUC (%) for each generation setting.
Evaluation Split
Method
Type
Time
Resolution
Data Scale
Text2Image
Z-Image-Turbo
Open
2025.12
1024 ∼ 1024
1,000
HiDream-O1-Image
Open
2026.05
2048 ∼ 2048
1,000
Seedream 4.5
Closed
2025.12
1184 ∼ 3616
1,000
Nano-Banana-2
Closed
2026.02
338 ∼ 2816
1,000
GPT-Image-2
Closed
2026.04
724 ∼ 2172
1,000
Image-Edit
FLUX.1-Kontext
Open
2025.06
512 ∼ 1984
2,000
Appendix
Table 10: Data Scale of AI-Generated Document Images.
Figure 4: Prompt Templates for Image-to-Text Generation.
AI-generated image detection identifies synthetic or AI-edited images and develops methods that generalize across generators, domains, and image transformations.
Recent AI-generated image (AIGI) detectors perform well on natural-image benchmarks, but their behavior on text-rich forgeries, such as fabricated screenshots, documents, and news pages prevalent in misinformation, remains untested. We introduce TextFake, a 20,000-image benchmark for text-rich AIGI detection spanning 28 languages, 4 topic categories, and 2 scene modalities. Fake images are synthesized via a four-stage pipeline that annotates real images along three controlled dimensions and generates counterparts through distribution-aligned structured prompting, ruling out covariate shortcuts. Zero-shot evaluation of 14 specialized detectors and 3 frontier VLM APIs reveals a large systematic gap: no method exceeds 80% accuracy, with some dropping over 60% from natural-image benchmarks. Diagnostic evaluations identify three failure modes: the Text Density Curse, where dense glyphs overwhelm low-level detectors; Cloaking via Rendering Fidelity, where stronger text rendering suppresses enerative artifacts; and Threshold Collapse, where routine perturbations drive detectors toward chance-level performance.
Yuning Zhang, Changtao Miao, Mingyu Liao +5
School of Cyber Science and Technology, University of Science and Technology of China · Anhui Province Key Laboratory of Digital Security · Individual Researcher
The rapid advancement in generative AI models has enabled the creation of photorealistic images. At the same time, there are growing concerns about the potential misuse and dangers of generated content, as well as a pressing need for effective AI-generated image detectors. However, current training-based detection techniques are typically computationally costly and can hardly be generalized to unseen data domains, while training-free methods fall short in detection performance. To bridge this gap, we propose a search-based method employing data embedding sensitivity in intermediate layers to detect AI-generated images. Given a set of real and AI-generated images, our method examines the similarity between original image embeddings and perturbed image embeddings, and detects AI-generated images based on the similarity. We examine the proposed method on two comprehensive benchmarks: GenImage and Forensics Small. Our method exhibits improved performance across different datasets compared to both training-free and training-based state-of-the-art methods. On average, our method achieves the largest performance gain on the Forensics Small benchmark by 39.61% compared to the best training-free method and 5.14% compared to the best training-based method in AUROC score.
Zhenhan Huang, Pin-Yu Chen, Tejaswini Pedapati +1
Department of Computer Science · Rensselaer Polytechnic Institute · Troy +2
Recent advances in visual generative models have enabled the creation of highly realistic, fully AI-generated images without relying on real source content. While beneficial for many applications, these models also pose significant societal risks, as they can be easily exploited to produce convincing Deepfakes. Detecting them represents a foundational yet challenging problem in AI media forensics, requiring detectors to reliably extract the inherent artifacts imprinted by generative architectures. In this Review, we provide a systematic overview of fully AI-generated image detection. Following the standard detector design pipeline, we focus on two key components: dataset construction and artifact extraction. We analyze how dataset design influences the generalization and robustness of learned artifacts, and categorize existing artifact extraction methods based on the primary inductive priors leveraged to isolate artifacts. Within this framework, we systematically review existing works. Finally, we highlight open problems and envision several future directions for developing more robust and generalizable detectors. Reviewed works in this survey can be found at https://github.com/zju-pi/Awesome-Fully-AI-Generated-Image-Detection.
Qijie Xu, Can Wang, Jiawei Chen +2
State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China. · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, Hangzhou, China. · Institute for Artificial Intelligence and Data Science, University at Buffalo, State University of New York, Buffalo, USA.