Naïve Exposure of Generative AI Capabilities Undermines Deepfake Detection

Sunpill Kim; Chanwoo Hwang; Minsu Kim; Jae Hong Seo

cs.CRMar 11, 2026

Naïve Exposure of Generative AI Capabilities Undermines Deepfake Detection

Authors: Sunpill Kim, Chanwoo Hwang, Minsu Kim, Jae Hong Seo

Organizations: Department of Mathematics & Research Institute for Natural Sciences, Hanyang University

Abstract

Generative AI systems increasingly expose powerful reasoning and image refinement capabilities through user-facing chatbot interfaces. In this work, we show that the naïve exposure of such capabilities fundamentally undermines modern deepfake detectors. Rather than proposing a new image manipulation technique, we study a realistic and already-deployed usage scenario in which an adversary uses only benign, policy-compliant prompts and commercial generative AI systems. We demonstrate that state-of-the-art deepfake detection methods fail under semantic-preserving image refinement. Specifically, we show that generative AI systems articulate explicit authenticity criteria and inadvertently externalize them through unrestricted reasoning, enabling their direct reuse as refinement objectives. As a result, refined images simultaneously evade detection, preserve identity as verified by commercial face recognition APIs, and exhibit substantially higher perceptual quality. Importantly, we find that widely accessible commercial chatbot services pose a significantly greater security risk than open-source models, as their superior realism, semantic controllability, and low-barrier interfaces enable effective evasion by non-expert users. Our findings reveal a structural mismatch between the threat models assumed by current detection frameworks and the actual capabilities of real-world generative AI. While detection baselines are largely shaped by prior benchmarks, deployed systems expose unrestricted authenticity reasoning and refinement despite stringent safety controls in other domains.

Figures & tables

Panel of Figure 1 — Figure 1: Semantic-preserving refinement alters authenticity judgments without changing identity from commercial APIs.

Figure 2: Summary of observed GAI behaviors across authenticity assessment and refinement stages, illustrating how structured reasoning transitions into benign, semantics-preserving image edits (Finding 1–5).

Service / Family	Access Channel			License	Specific API Model ID	Release	Cost
Service / Family	WEB	APP	API	License	Specific API Model ID	(YYYY.MM)	($/img)
Qwen (Alibaba)	✓	–	✓	Open Weights	qwen-image-edit	2024.10	$0
Qwen (Alibaba)	✓	–	✓	Open Weights	qwen-image-edit-2511	2025.11	$0
Flux AI (Black Forest Labs)	✓	–	✓	Hybrid †	flux-2-max	2025.12	$0.070
ChatGPT (OpenAI)	✓	✓	✓	Commercial	gpt-image-1.5	2025.10	$0.133
Gemini (Google)	✓	✓	✓	Commercial	gemini-3-pro-image-preview	2025.11	$0.200

Table 1: Detailed comparison of image refinement services and model specifications (As of Feb 2026).

Target	Feature Extractor	Training Dataset	Access
GenD † [ 72 ]	CLIP:ViT-L [ 48 ] / Perceptual [ 5 ] / DINOv3 [ 55 ]	FaceForensics++ [ 51 ]	Open
M2F2-Det ‡ [ 17 ]	EfficientNet-B4 [ 59 ] + CLIP:ViT-L [ 48 ]	FaceForensics++ [ 51 ] + DD-VQA ⋆ [ 76 ]	Open
Hive-DF [ 20 ]	EfficientNet-B4 [ 59 ] + YoloV8	Proprietary (Internal)	API
UnivFD [ 44 ]	CLIP:ViT-L [ 48 ]	ProGAN (LSUN) [ 24 , 73 ]	Open
D 3 [ 71 ]	CLIP:ViT-L [ 48 ]	ProGAN (LSUN) + GenImage [ 79 ]	Open
Hive-AI [ 19 ]	EfficientNet-B4 [ 59 ]	Proprietary (Internal)	API

Table 2: Detailed comparison of evaluation targets. Rows are color-coded by detection scope: Deepfake and AI-generated .

Artifact Category	Top Frequent Descriptors (Count)
Skin Texture	smooth (158), plastic (102), texture (116), airbrushed (56), pores (51), sheen (40), waxy (17)
Eyes & Gaze	catchlights (90), iris (54), sclera (37), reflections (32), glassy (22), mismatch (9)
Hair & Boundaries	strands (88), hairline (69), halo (44), clumps (30), flyaways (22), merge (17)
Facial Geometry	edges (111), nostrils (73), contours (69), lip (63), ear (53), jawline (47), eyelid (35)
Scene & Background	background (166), depth (84), camera (65), blur (44), gradient (40), bokeh (39)

Table 3: Taxonomy of visual artifacts frequently identified by GAI systems during deepfake assessment (Top-K keywords aggregated from 100 records).

Detector	Original		Qwen-v1 ( qwen-image-edit )										Qwen-v2 ( qwen-image-edit-2511 )
	N/A		P1		P2		P3		P4		P5 (IAP)		P1		P2		P3		P4		P5 (IAP)
	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$
GenD (C)	75	95	68	88	65	90	45	91	28	85	28	89	76	95	79	94	34	78	5	40	22	89
GenD (P)	85	88	67	80	72	84	20	67	8	42	3	46	79	96	86	98	27	56	3	13	7	30
GenD (D)	84	93	81	92	69	89	25	74	11	70	7	61	88	97	90	98	30	58	3	18	11	45
M2F2	31	61	19	45	11	35	7	22	3	12	2	11	18	51	21	57	11	40	1	17	18	68

Table 4: DRs (%) of deepfake and AI-generated detectors before and after refinement using Qwen models with prompt variations. Columns report performance on Original (FF++) vs. refined outputs produced by Qwen under Prompt 1–5 (increasingly detailed).

Detector	Original		P4 (short version of IAP)								P5 (IAP)								P6 (ISP)
	N/A		Qwen-v1		Flux AI		ChatGPT		Gemini		Qwen-v1		Flux AI		ChatGPT		Gemini		ChatGPT		Gemini
	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$
GenD (C)	75	95	28	85	12	59	0	0	0	0	28	89	10	62	0	0	0	0	0	0	0	1
GenD (P)	85	88	8	42	1	24	0	0	0	0	3	46	0	14	0	0	0	0	0	0	0	0
GenD (D)	84	93	11	70	6	33	0	0	0	0	7	61	6	35	0	0	0	0	0	0	0	0
M2F2	31	61	3	12	1	21	3	29	0	5	2	11	5	33	6	41	0	15	0	17	0	2

Table 5: Detection rates of deepfake and AI-generated image detectors before and after refinement with commercial GAI systems.

Figure 3: Semantic preservation with IAP and ISP.

Panel of Figure 4 — Figure 4: Qualitative examples. Each row shows one source image and corresponding outputs from different systems.

Detector	Original		ChatGPT				Gemini
	N/A		P4		P5		P4		P5
	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$	$\tau_{99}$	$\tau_{90}$
D 3	50	96	18	67	10	59	36	72	39	81
UnivFD	38	50	8	20	12	27	23	39	33	46
Hive-AI	83	94	14	90	28	86	38	86	41	83

Table 6: DRs (%) of AI-generated image detectors on original images and images refined by commercial GAI systems.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Evaluation Dimension	Operational Focus in Policies
Synthesized Output	Categorical filtering of prohibited content, such as sexual, violent, non-consensual imagery; hate speech; or illegal activities.
Prompt-level Intent	Detection of explicit malicious or illicit intent from surface-level wording in individual user instructions.

Appendix

Table 7: Risk dimensions in existing AI safety policies.

Service	Provider	Primary Capability
Moderation API	OpenAI	Text-based safety filtering with optional multimodal extension
AI Content Safety	Microsoft Azure	Native text and image safety filtering

Appendix

Table 8: Automated safety filtering services used in our experimental setup.

Prompt Type	Max Signal
Criteria Elicitation	Illicit (0.0166)
Image Analysis	Sexual ( $2.6\times 10^{-5}$ )
Image Refinement (Instance-Agnostic)	Violence ( $5.2\times 10^{-4}$ )
Image Refinement (Instance-Specific)	Violence ( $5.2\times 10^{-4}$ )

Appendix

Table 9: OpenAI safety filtering outcomes for text-based prompts using omni-moderation-latest .(No prompt was flagged by the moderation system.)

Prompt Type	Max Severity
Criteria Elicitation	0
Image Analysis	0
Image Refinement (Instance-Agnostic)	0
Image Refinement (Instance-Specific)	0

Appendix

Table 10: Azure AI Content Safety outcomes for text-based prompts.

Image Set	# of Severity $>0$	Max Severity
Original	0 (0.00%)	0
Flux AI	0 (0.00%)	0
ChatGPT	1 (1.00%)	2
Gemini	0 (0.00%)	0
Qwen-v1	2 (2.00%)	2
Qwen-v2	1 (1.00%)	2

Appendix

Table 11: Image-based safety filtering outcomes using Azure AI Content Safety. For each image set, we report the proportion of images exhibiting any non-zero severity and the maximum observed severity across all categories.

Figure 5: Semantic preservation with IAP and ISP on Tencent CompareFace. (Note that the default thresholds in Tencent CompareFace is $50$ )

Figure 7: Criterion-based image analysis. A GAI system applies the articulated criteria to analyze a facial image that is a known deepfake, producing a structured forensic explanation and final judgment.

Deepfake Detection

Deepfake detection identifies manipulated or AI-generated media to support trustworthy communication, identity verification, and forensic analysis.

See all 241 papers

How well are open sourced AI-generated image detection models out-of-the-box: A comprehensive benchmark study
Feb 8, 2026Simiao Ren, Yuchen Zhou, Xingyu Shen +9AI-Generated Image Detection
Neo · Dennis
Adversarial Robustness of AI-Generated Image Detectors in the Real World
Oct 2, 2024Sina Mavali, Jonas Ricker, David Pape +2Adversarial Attacks Adversarial Robustness
CISPA Helmholtz Center for Information Security, Saarbrücken, Germany · Ruhr University Bochum, Bochum, Germany
Perception, Verdict, and Evolution: Hindsight-Driven Self-Refining Forensics Agent for AI-Generated Image Detection
Jun 25, 2026Yangjun Wu, Keyu Yan, Yu Liu +5LLM Self-Refinement AI-Generated Image Detection
Zhejiang University, Zhejiang, China · Alibaba Group

Feb 8, 2026J/K move · Enter open · S save

How well are open sourced AI-generated image detection models out-of-the-box: A comprehensive benchmark study

Simiao Ren, Yuchen Zhou, Xingyu Shen, Kidus Zewde+8

Neo · Dennis

AI-Generated Image Detection

Open paper AskPDF arXiv

As AI-generated images proliferate across digital platforms, reliable detection methods have become critical for combating misinformation and maintaining content authenticity. While numerous deepfake detection methods have been proposed, existing benchmarks predominantly evaluate fine-tuned models, leaving a critical gap in understanding out-of-the-box performance -- the most common deployment scenario for practitioners. We present the first comprehensive zero-shot evaluation of 16 state-of-the-art detection methods, comprising 23 pretrained detector variants (due to multiple released versions of certain detectors), across 12 diverse datasets, comprising 2.6million image samples spanning 291 unique generators including modern diffusion models. Our systematic analysis reveals striking findings: (1)~~no universal winner exists, with detector rankings exhibiting substantial instability (Spearman~~ $ρ$ : 0.01 -- 0.87 across dataset pairs); (2)~~a 37~~percentage-point performance gap separates the best detector (75.0% mean accuracy) from the worst (37.5%); (3)training data alignment critically impacts generalization, causing up to 20--60% performance variance within architecturally identical detector families; (4)~~modern commercial generators (Flux~~Dev, Fireflyv4, Midjourneyv7) defeat most detectors, achieving only 18--30% average accuracy; and (5)we identify three systematic failure patterns affecting cross-dataset generalization. Statistical analysis confirms significant performance differences between detectors (Friedman test: $χ^2$ =121.01, $p<10^{-16}$ , Kendall

W

=0.524). Our findings challenge the ``one-size-fits-all'' detector paradigm and provide actionable deployment guidelines, demonstrating that practitioners must carefully select detectors based on their specific threat landscape rather than relying on published benchmark performance.

Abstract

Deepfake Detection

Explore similar work

How well are open sourced AI-generated image detection models out-of-the-box: A comprehensive benchmark study

Adversarial Robustness of AI-Generated Image Detectors in the Real World

Perception, Verdict, and Evolution: Hindsight-Driven Self-Refining Forensics Agent for AI-Generated Image Detection