Beyond Natural Images: Rethinking AI-Generated Image Detection in Documents

Zhangjie Fu; Jiazhen Yan; Yuanwen Chen; Xinquan Yu; Yanzhe Li; Hui Jiang; Lei Gao; Chenfu Bao

cs.CVSep 13, 2026

Beyond Natural Images: Rethinking AI-Generated Image Detection in Documents

Authors: Zhangjie Fu, Jiazhen Yan, Yuanwen Chen, Xinquan Yu, Yanzhe Li, Hui Jiang, Lei Gao, Chenfu Bao

Organizations: Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology · Baidu Inc. · School of Computer Science and Engineering, MoE Key Laboratory of Information Technology, Guangdong Province Key Laboratory of Information Security Technology, Sun Yat-sen University · Tsinghua University

Abstract

AI-generated image detection has attracted increasing attention, but existing evaluations mainly focus on natural images, leaving AI-generated document images largely underexplored. This omission is concerning because documents often appear in sensitive real-world scenarios, such as invoices, expense reports, certificates, and medical records. In this paper, we first construct a controlled diagnostic benchmark, AIGDoc-Pilot, and reveal that existing detectors suffer substantial performance degradation on AI-generated document images, with the mean AUC dropping by more than 7%. Based on this, we further reveal two document-specific properties behind this gap: generation artifacts exhibit strong spatial inconsistency across local regions, and text density significantly affects real-synthetic separability, where text-dense regions offer stronger discriminative evidence. Motivated by these findings, we construct AIGDoc, a larger document-centric dataset containing diverse real-world documents and AI-generated counterparts produced by multiple advanced generation and editing models. Extensive experiments on AIGDoc demonstrate that existing detectors still struggle to reliably identify AI-generated documents, while document-based training partially narrows the gap. Together, these results offer valuable insights for developing dependable and generalizable detectors in document-centric scenarios. The code and datasets will be made publicly available upon acceptance of the paper.

Figures & tables

Figure 1: (a) Diagnosis and Findings. Using the controlled diagnostic benchmark AIGDoc-Pilot, we reveal that existing detectors suffer substantial performance degradation on AI-generated document images. Further diagnosis shows that generated document artifacts exhibit strong spatial inconsistency across local regions, while text-dense regions provide stronger discriminative evidence for real-synthetic separability. (b) Datasets and Evaluations. We construct AIGDoc, a large-scale benchmark for AI-generated document image detection, comprising 27K synthetic images across 17 subsets, covering 13 generation methods from three categories.

Method	SDv2.1-VAE	HiDream- E1-Full	FLUX.1-dev				Qwen-Image		SDv3.5-Large		Mean
Method	SDv2.1-VAE	HiDream- E1-Full	0.2	0.4	0.6	0.8	0.2	0.4	0.2	0.4	F.Acc.	AUC
	13.21	15.41	6.61	9.17	17.98	29.54	11.56	17.06	10.64	12.48	14.37	69.78
DRCT	15.23	9.54	7.89	11.56	19.82	35.96	24.06	37.19	14.63	13.70	$\textbf{18.96}_{\text{+4.59}}$	$68.36_{\text{-1.42}}$
	1.83	1.10	0.55	2.02	4.40	7.89	4.77	4.77	1.10	0.55	2.90	55.73
Effort	1.65	1.10	1.10	2.20	5.87	15.96	10.94	12.50	0.75	0.94	$\textbf{5.30}_{\text{+2.40}}$	$40.46_{\text{-15.27}}$
	98.53	70.83	27.89	22.94	32.11	51.38	77.98	82.75	93.21	95.41	65.30	88.08

Table 1: Detection Performance Gap between AI-Generated Natural and Document Images on AIGDoc-Pilot. For each detector, the decision threshold is calibrated to maintain 95% Real Acc. Gray rows denote results on document images, while white rows denote results on natural images.

Method	Region	SDv2.1- VAE	HiDream- E1-Full	FLUX.1-dev				Qwen-Image		SDv3.5-Large		Mean
Method	Region	SDv2.1- VAE	HiDream- E1-Full	0.2	0.4	0.6	0.8	0.2	0.4	0.2	0.4	F.Acc.	AUC
	No	72.33	39.00	14.67	17.00	34.00	28.00	62.67	65.67	45.67	47.67	42.67	82.16
	Low	84.67	48.33	18.67	23.33	35.00	38.00	61.33	72.00	46.33	51.00	47.87 ${}_{\text{+5.20}}$	83.56 ${}_{\text{+1.40}}$
	Mid	86.67	48.00	21.00	27.00	47.00	53.00	63.00	74.00	53.33	58.67	53.17 ${}_{\text{+10.50}}$	85.02 ${}_{\text{+2.86}}$
B-Free	High	96.33	47.00	28.00	34.00	53.00	56.67	69.67	77.00	56.33	67.33	58.53 ${}_{\text{+15.86}}$	86.42 ${}_{\text{+4.26}}$
	No	80.00	47.33	14.00	16.67	13.00	28.00	62.67	66.00	51.67	66.33	44.57	81.67

Table 3: Document-Region Fake Acc. under Different Text Densities at 95% Real Acc. Each generator reports F.Acc. (%). We also report the mean F.Acc. and mean AUC across generation settings. Rows under each detector correspond to non-text, low-text, medium-text, and high-text regions.

Generator	UnivFD	DRCT	Effort	B-Free	DDA	MIRROR	OmniAID	PROBE	SICA	PE-LoRA	PE-LoRA †	PE-LoRA ‡
Text2Image
Z-Image-Turbo	49.4	61.8	85.2	52.3	61.4	49.9	49.8	55.1	44.9	68.0	92.8	90.5
HiDream-O1-Image	56.5	62.1	72.7	97.0	95.2	66.8	63.2	97.9	75.8	95.2	97.9	93.5
Seedream 4.5	50.1	51.7	58.2	95.2	91.8	52.8	53.3	99.5	61.8	98.0	97.4	95.7
Nano-Banana-2	50.0	46.2	60.2	66.0	58.9	50.4	51.0	63.9	55.4	59.8	67.5	60.5
GPT-Image-2	50.0	49.5	74.2	51.3	47.0	49.8	50.3	51.2	47.4	49.4	59.7	83.9

Table 5: Detection Accuracy across Different AI-Generated Document Image Settings. Each detector reports Acc. (%) for each generation setting.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 2: Qualitative Visualization of AIGDoc-Pilot Generation Results. For each real image, we show the generated counterpart and its difference map. Compared with natural images, document images exhibit much sparser generation-induced differences, which are mainly concentrated around text, table lines, and other high-frequency structures.

Method	SDv2.1-VAE	HiDream- E1-Full	FLUX.1-dev				Qwen-Image		SDv3.5-Large		Mean AUC
Method	SDv2.1-VAE	HiDream- E1-Full	0.2	0.4	0.6	0.8	0.2	0.4	0.2	0.4	Mean AUC
	69.25	69.79	56.93	65.91	73.19	81.69	70.52	74.90	65.13	70.52	69.78
DRCT	67.88	57.76	61.23	58.91	66.63	79.09	75.30	80.94	67.28	68.54	$68.36_{\text{-1.42}}$
	48.80	49.02	41.53	55.24	64.54	72.91	61.74	64.30	47.95	51.31	55.73
Effort	30.70	27.26	29.86	36.48	48.73	64.40	57.17	59.23	24.46	26.32	$40.46_{\text{-15.27}}$
	99.63	90.46	74.47	70.94	74.83	83.55	94.24	95.68	98.14	98.80	88.08

Appendix

Table 6: Detection Performance Gap between AI-Generated Natural and Document Images on AIGDoc-Pilot. Each generator setting reports AUC (%). Gray rows denote results on document images, while white rows denote results on natural images.

Figure 3: Visualization of text-density-based crop regions and their corresponding residual maps. Low-text-density regions exhibit weaker and less discernible differences between generated and real images, whereas text-dense regions reveal more pronounced generation traces.

Method	Region	SDv2.1- VAE	HiDream- E1-Full	FLUX.1-dev				Qwen-Image		SDv3.5-Large		Mean
Method	Region	SDv2.1- VAE	HiDream- E1-Full	0.2	0.4	0.6	0.8	0.2	0.4	0.2	0.4	Mean
	No	94.03	82.70	66.38	67.15	78.94	80.44	90.23	91.21	83.84	86.64	82.16
	Low	96.63	82.89	67.51	68.90	79.69	84.52	90.09	93.23	83.92	88.26	83.56 ${}_{\text{+1.40}}$
	Mid	96.76	83.63	68.63	71.98	83.36	85.43	91.09	93.32	86.24	89.73	85.02 ${}_{\text{+2.86}}$
B-Free	High	98.40	81.83	70.48	74.92	84.84	88.03	93.00	94.80	87.00	90.84	86.42 ${}_{\text{+4.26}}$
	No	95.78	85.87	67.37	64.15	68.58	75.90	89.94	91.51	86.22	91.34	81.67

Appendix

Table 7: Document-Region AUC under Different Text Densities. Each generator reports AUC (%). Rows under each detector correspond to non-text, low-text, medium-text, and high-text regions.

Method	Region	SDv2.1- VAE	HiDream- E1-Full	FLUX.1-dev				Qwen-Image		SDv3.5-Large		Mean
Method	Region	SDv2.1- VAE	HiDream- E1-Full	0.2	0.4	0.6	0.8	0.2	0.4	0.2	0.4	F.Acc.	AUC
	No	86.33	54.67	29.00	29.67	44.33	49.33	66.67	70.00	60.00	60.33	55.03	82.16
	Low	85.00	45.67	20.00	21.33	37.00	43.33	56.33	69.33	46.67	50.33	47.50 ${}_{\text{-7.53}}$	83.56 ${}_{\text{+1.40}}$
	Mid	80.00	39.67	17.33	22.67	36.00	46.67	55.33	65.00	48.67	49.67	46.10 ${}_{\text{-8.93}}$	85.02 ${}_{\text{+2.86}}$
B-Free	High	82.00	38.00	18.00	22.33	33.00	42.00	56.00	65.33	42.33	49.33	44.83 ${}_{\text{-10.20}}$	86.42 ${}_{\text{+4.26}}$
	No	83.00	52.67	20.00	17.33	17.33	26.00	49.33	52.00	59.00	66.00	44.27	81.67

Appendix

Table 8: Document-Region Fake Acc. under Different Text Densities with the 0.5 Threshold. Each generator reports F.Acc. (%). We also report the mean F.Acc. and mean AUC across generation settings. Rows under each detector correspond to non-text, low-text, medium-text, and high-text regions.

Generator	UnivFD	DRCT	Effort	B-Free	DDA	MIRROR	OmniAID	PROBE	SICA	PE-LoRA	PE-LoRA †	PE-LoRA ‡
Text2Image
Z-Image-Turbo	48.0	69.5	93.2	60.9	78.4	56.4	31.8	86.1	42.9	84.2	97.7	97.2
HiDream-O1-Image	82.6	66.7	83.5	99.6	99.1	93.5	87.1	99.9	78.9	99.3	99.9	98.7
Seedream 4.5	67.8	51.4	64.0	99.0	97.8	67.2	41.0	100.0	64.8	99.6	99.9	99.8
Nano-Banana-2	55.9	45.2	63.6	70.6	63.6	67.8	47.0	78.5	57.5	58.2	79.0	64.5
GPT-Image-2	67.7	51.0	83.3	47.3	46.9	64.4	38.2	75.8	48.3	35.8	72.7	94.3

Appendix

Table 9: Detection AUC across Different AI-Generated Document Image Settings. Each detector reports AUC (%) for each generation setting.

Evaluation Split	Method	Type	Time	Resolution	Data Scale
Text2Image	Z-Image-Turbo	Open	2025.12	1024 $\sim$ 1024	1,000
	HiDream-O1-Image	Open	2026.05	2048 $\sim$ 2048	1,000
	Seedream 4.5	Closed	2025.12	1184 $\sim$ 3616	1,000
	Nano-Banana-2	Closed	2026.02	338 $\sim$ 2816	1,000
	GPT-Image-2	Closed	2026.04	724 $\sim$ 2172	1,000
Image-Edit	FLUX.1-Kontext	Open	2025.06	512 $\sim$ 1984	2,000

Appendix

Table 10: Data Scale of AI-Generated Document Images.

Figure 4: Prompt Templates for Image-to-Text Generation.

Figure 5: Prompt Templates for Image Editing.

Figure 6: Prompt Examples and Generated Images.

AI-Generated Image Detection

AI-generated image detection identifies synthetic or AI-edited images and develops methods that generalize across generators, domains, and image transformations.

See all 194 papers

TextFake: Benchmarking AI-Generated Image Detection on Text-Rich Images
May 31, 2026Yuning Zhang, Changtao Miao, Mingyu Liao +5AI-Generated Image Detection Adversarial Attacks on AI-Generated Image Detectors
School of Cyber Science and Technology, University of Science and Technology of China · Anhui Province Key Laboratory of Digital Security · Individual Researcher
Intermediate Representations are Strong AI-Generated Image Detectors
May 5, 2026Zhenhan Huang, Pin-Yu Chen, Tejaswini Pedapati +1AI-Generated Image Detection
Department of Computer Science · Rensselaer Polytechnic Institute · Troy +2
Fully AI-Generated Image Detection: Definition, Recent Advances and Challenges
Feb 27, 2025Qijie Xu, Can Wang, Jiawei Chen +2Digital Image Forensics AI-Generated Image Detection
State Key Laboratory of Blockchain and Data Security, Zhejiang University, Hangzhou, China. · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, Hangzhou, China. · Institute for Artificial Intelligence and Data Science, University at Buffalo, State University of New York, Buffalo, USA.

May 31, 2026J/K move · Enter open · S save

TextFake: Benchmarking AI-Generated Image Detection on Text-Rich Images

Yuning Zhang, Changtao Miao, Mingyu Liao, Tingyu Liu+4

School of Cyber Science and Technology, University of Science and Technology of China · Anhui Province Key Laboratory of Digital Security · Individual Researcher

AI-Generated Image Detection Adversarial Attacks on AI-Generated Image Detectors

Open paper AskPDF arXiv

Recent AI-generated image (AIGI) detectors perform well on natural-image benchmarks, but their behavior on text-rich forgeries, such as fabricated screenshots, documents, and news pages prevalent in misinformation, remains untested. We introduce TextFake, a 20,000-image benchmark for text-rich AIGI detection spanning 28 languages, 4 topic categories, and 2 scene modalities. Fake images are synthesized via a four-stage pipeline that annotates real images along three controlled dimensions and generates counterparts through distribution-aligned structured prompting, ruling out covariate shortcuts. Zero-shot evaluation of 14 specialized detectors and 3 frontier VLM APIs reveals a large systematic gap: no method exceeds 80% accuracy, with some dropping over 60% from natural-image benchmarks. Diagnostic evaluations identify three failure modes: the Text Density Curse, where dense glyphs overwhelm low-level detectors; Cloaking via Rendering Fidelity, where stronger text rendering suppresses enerative artifacts; and Threshold Collapse, where routine perturbations drive detectors toward chance-level performance.

Abstract

AI-Generated Image Detection

Explore similar work

TextFake: Benchmarking AI-Generated Image Detection on Text-Rich Images

Intermediate Representations are Strong AI-Generated Image Detectors

Fully AI-Generated Image Detection: Definition, Recent Advances and Challenges