Defake-o3: From Speculative Rationales to Verifiable Evidence for Explainable AIGI Detection

Bowen Deng; Jiahui Zhan; Yikun Ji; Haozhen Yan; Jianfu Zhang

cs.CVAug 17, 2026

Defake-o3: From Speculative Rationales to Verifiable Evidence for Explainable AIGI Detection

Authors: Bowen Deng, Jiahui Zhan, Yikun Ji, Haozhen Yan, Jianfu Zhang

Organizations: Shanghai Jiao Tong University Shanghai, China

Abstract

The rapid progress of image generation models calls for AI-generated image (AIGI) detectors that are not only accurate but also explainable and reliable. While MLLM-based detectors can provide natural language explanations, existing methods often generate speculative rationales: they rely on vague or hallucinated artifacts, miss subtle localized flaws from the latest generators, and fail to provide evidence that can be visually verified. We present Defake-o3, an explainable AIGI detector that moves from speculative rationales to verifiable evidence. It combines interactive visual search with verifier-guided evidence alignment: the model iteratively zooms into suspicious regions to inspect fine-grained details, while an Evidence Verifier, trained from human verification annotations, provides reinforcement learning rewards that favor grounded evidence and penalize baseless claims. To support this objective, we construct GroundFake, a dataset designed for grounded explainable detection, with localized bounding-box evidence, human verification based on visual grounding and artifact specificity, corrected reasoning trajectories, and valid/invalid evidence supervision. We further introduce FakeFrontier, an out-of-distribution benchmark built from real images and outputs of 10 recent generators, together with an MLLM-based protocol for evaluating evidence quality and persuasiveness. Experiments on GroundFake, FakeFrontier, and additional out-of-distribution benchmarks show that Defake-o3 improves both detection accuracy and explanation quality, producing more localized, verifiable, and persuasive evidence.

Figures & tables

Figure 1. Overview of Defake-o3. Compared with existing single-pass MLLM detectors, Defake-o3 iteratively zooms into suspicious regions to verify candidate artifacts and produces a structured verdict supported by global evidence and localized key evidence. A comparison of existing single-pass MLLM detectors and Defake-o3. Existing methods inspect the full image once and may produce speculative rationales. Defake-o3 repeatedly selects and magnifies suspicious regions, verifies localized artifacts, and returns an authenticity verdict supported by global evidence and bounding-box-level local evidence.

Figure 2. GroundFake construction pipeline. Starting from a balanced, bias-controlled image pool, we generate label-conditioned candidate reasoning trajectories and localized evidence, manually verify localized key evidence for fake images, and rewrite the trajectories into annotation-consistent supervision transcripts. The resulting dataset supports both SFT and Evidence Verifier training. A four-stage pipeline for constructing GroundFake: collecting a balanced and bias-controlled image pool, generating candidate reasoning and localized evidence, manually verifying the localized evidence, and correcting reasoning trajectories for SFT and Evidence Verifier training.

Figure 3. Overview of Defake-o3. The model performs interactive visual search by iteratively zooming into suspicious regions and then outputs a structured verdict with global and localized evidence. For every correctly classified fake image, the total reward combines the classification reward with rule-based evidence matching and model-based scoring from the Evidence Verifier. The training pipeline of Defake-o3, where the model alternates between viewing the image and selecting zoom-in boxes, then outputs a verdict with localized evidence, and receives GRPO rewards from an evidence verifier.

Method	Acc	F1	BLEU-1	BLEU-2	ROUGE-L	IoU
CNNSpot	0.976	0.976	–	–	–	–
NPR	0.838	0.830	–	–	–	–
AIDE	0.924	0.922	–	–	–	–
FakeVLM	0.857	0.854	0.105	0.045	0.110	–
LEGION	0.502	0.024	0.003	0.001	0.002	0.000
FakeShield	0.494	0.275	0.029	0.013	0.026	0.017

Table 1. Results on the GroundFake test set.

Figure 4. Qualitative examples from the GroundFake test set. Defake-o3 first performs coarse scene-level inspection and then uses targeted zoom-in steps to verify candidate artifacts, yielding a small set of localized, visually verifiable evidence items. Compared with FakeVLM ( Wen et al., 2025 ) and FakeShield ( Xu et al., 2025b ) , its outputs are more directly grounded in explicit, image-specific artifacts. FakeShield masks are omitted for brevity. Qualitative comparisons on several GroundFake images. For each image, the figure contrasts evidence produced by FakeVLM, FakeShield, and Defake-o3, and shows how Defake-o3 zooms into suspicious regions and identifies localized visual artifacts.

Method	Avg #Evi	Qwen3-VL-235B (Prior-Acc = 0.2421)			Kimi K2.5 (Prior-Acc = 0.3526)			GLM-4.6V (Prior-Acc = 0.0737)
Method	Avg #Evi	QS	Hit@Img	Hit@Evi	QS	Hit@Img	Hit@Evi	QS	Hit@Img	Hit@Evi
Defake-o3	3.0804	0.5378	0.7839	0.6362	0.4334	0.7471	0.5895	0.6812	0.9045	0.8249
Defake-CoT	2.9397	0.4841	0.7755	0.6239	0.3811	0.7119	0.5704	0.5980	0.8961	0.7584
Defake-Direct	3.1910	0.3721	0.6047	0.4861	0.2877	0.5645	0.4504	0.4431	0.7085	0.5554
FakeVLM	3.6421	0.1268	0.3702	0.2558	0.1204	0.4526	0.3531	0.2438	0.5456	0.5092
FakeShield	2.2161	0.0875	0.2312	0.3296	0.0763	0.2747	0.4218	0.1393	0.2747	0.4255

Table 2. Evaluation with the MLLM-based protocol on the fake-image subset of FakeFrontier.

Method	Real Acc	Fake Acc	Acc	F1
Defake-o3	0.9020	0.9446	0.9232	0.9246
Defake-CoT	0.8675	0.9532	0.9102	0.9137
Defake-Direct	0.8485	0.7826	0.8157	0.8088
FakeVLM	0.7127	0.7102	0.7114	0.7108
FakeShield	0.8135	0.4107	0.6127	0.5139
LEGION	0.9930	0.0045	0.5004	0.0090

Table 3. Classification results on FakeFrontier benchmark.

Test Set	Defake-o3	FakeVLM	LEGION	FakeShield	CNNSpot	NPR	AIDE
AIGI-Now	0.9180	0.8596	0.4997	0.8307	0.7730	0.6925	0.8473
EvalGEN	0.9872	0.9668	0.0907	0.9513	0.7660	0.2430	0.7045
MNW	0.8871	0.8417	0.0265	0.5072	0.6435	0.3775	0.7712

Table 4. Accuracy on other OoD test sets.

Figure 5. Output value distribution of the Evidence Verifier on generated evidence. A distribution plot of the Evidence Verifier scores assigned to generated evidence, illustrating the separation between evidence judged invalid and evidence judged valid.

$\alpha_{M}$	$\lambda_{iou}/\lambda_{bleu}$	Acc	F1	BLEU-1	BLEU-2	ROUGE-L	IoU
1	0	0.984	0.984	0.275	0.150	0.222	0.195
0.75	1	0.910	0.902	0.303	0.176	0.235	0.260
0.5	1	0.992	0.992	0.364	0.215	0.281	0.311
0.25	1	0.962	0.960	0.330	0.194	0.270	0.318
0	1	0.986	0.986	0.363	0.218	0.291	0.252

Table 5. Ablation of reward hyperparameters evaluated on the GroundFake test set.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Figure S1. Comparison of validation loss between Defake-o3 and Defake-CoT in the SFT stage.

Figure S2. Average number of tool calls and other metrics during the RL stage of Defake-o3.

Perturbation	Defake-o3	FakeVLM	LEGION	FakeShield
Original	0.9232	0.7114	0.5004	0.6127
JPEG Compression	0.8924	0.6891	0.5004	0.5952
Gaussian Blur	0.8746	0.6888	0.4991	0.6090
Resize	0.9182	0.6682	0.5004	0.5867

Appendix

Table S1. Robustness evaluation on FakeFrontier.

$\alpha_{M}$	$\lambda_{iou}/\lambda_{bleu}$	Acc	F1	BLEU-1	BLEU-2	ROUGE-L	IoU
0.5	1	0.992	0.992	0.364	0.215	0.281	0.311
0	1	0.986	0.986	0.363	0.218	0.291	0.252
0	2	0.952	0.950	0.321	0.190	0.266	0.237
0	4	0.974	0.973	0.341	0.202	0.286	0.258

Appendix

Table S2. Ablation of rule-based reward hyperparameters evaluated on the GroundFake test set.

Model	Acc	F1	BLEU-1	BLEU-2	ROUGE-L	IoU
Qwen3-VL-8B-Instruct	0.824	0.823	0.108	0.051	0.159	0.143
Defake-Direct	0.958	0.957	0.333	0.195	0.252	0.262
Defake-o3	0.992	0.992	0.364	0.215	0.281	0.311

Appendix

Table S3. Performance of the base model compared to our trained models on GroundFake.

Method	Throughput (samples/second)
Defake-Direct	10.668
Defake-CoT	8.180
Defake-o3	2.606
FakeVLM	3.008
LEGION	7.398
FakeShield	0.535

Appendix

Table S4. Throughput comparison during inference.

Figure S3. Examples of images from the GroundFake dataset.

Invalid Filtering	RL Training	GroundFake		FakeFrontier
Invalid Filtering	RL Training	Acc	IoU	Real Acc	Fake Acc	Acc
✗	✗	0.984	0.201	0.836	0.916	0.876
✓	✗	0.972	0.241	0.883	0.904	0.894
✓	✓	0.992	0.311	0.902	0.945	0.923

Appendix

Table S5. Impact of removing invalid evidence during training.

Method	Qwen3-VL-235B		Kimi K2.5		GLM-4.6V
Method	Hit@Img	Hit@Evi	Hit@Img	Hit@Evi	Hit@Img	Hit@Evi
Defake-o3	0.9347	0.8825	0.8643	0.7292	0.9397	0.9527
Defake-CoT	0.9548	0.8684	0.8844	0.7521	0.9548	0.8974
Defake-Direct	0.7538	0.7165	0.6683	0.5890	0.7688	0.6803
FakeVLM	0.5895	0.4740	0.5474	0.4870	0.7053	0.7428
FakeShield	0.3166	0.5420	0.3116	0.5465	0.3266	0.5646

Appendix

Table S6. Persuasion Evaluation Results under Prompt I.

Method	Qwen3-VL-235B		Kimi K2.5		GLM-4.6V
Method	Hit@Img	Hit@Evi	Hit@Img	Hit@Evi	Hit@Img	Hit@Evi
Defake-o3	0.8945	0.7455	0.8392	0.7080	0.9497	0.9462
Defake-CoT	0.9146	0.7607	0.8693	0.7197	0.9598	0.8923
Defake-Direct	0.6935	0.5543	0.6482	0.5449	0.7688	0.6787
FakeVLM	0.4158	0.2384	0.5316	0.4176	0.6632	0.6561
FakeShield	0.2764	0.3741	0.3116	0.4807	0.3266	0.5692

Appendix

Table S7. Persuasion Evaluation Results under Prompt II.

Method	Qwen3-VL-235B		Kimi K2.5		GLM-4.6V
Method	Hit@Img	Hit@Evi	Hit@Img	Hit@Evi	Hit@Img	Hit@Evi
Defake-o3	0.5226	0.2806	0.5377	0.3312	0.8241	0.5759
Defake-CoT	0.4573	0.2427	0.3819	0.2393	0.7739	0.4855
Defake-Direct	0.3668	0.1874	0.3769	0.2173	0.5879	0.3071
FakeVLM	0.1053	0.0549	0.2789	0.1546	0.2684	0.1286
FakeShield	0.1005	0.0726	0.2010	0.2381	0.1709	0.1429

Appendix

Table S8. Persuasion Evaluation Results under Prompt III.

Figure S4. Additional qualitative comparison of Defake-o3 against FakeVLM and FakeShield on the GroundFake test set (Part 1). The bounding boxes are from the outputs of Defake-o3.

Figure S5. Additional qualitative comparison of Defake-o3 against FakeVLM and FakeShield on the GroundFake test set (Part 2).

Figure S6. Additional qualitative comparison of Defake-o3 against FakeVLM and FakeShield on the GroundFake test set (Part 3).

AI-Generated Image Detection

AI-generated image detection identifies synthetic or AI-edited images and develops methods that generalize across generators, domains, and image transformations.

See all 194 papers

Aug 13, 2026

SPARED: Reasoning-Based AI-Generated Image Detection via Adversarially Edited Data

Detecting AI-generated images is only half the task: a deployed detector must also justify its verdict, yet existing detectors inherit three failure modes from their training data: real and fake images collected from different sources invite provenance shortcuts, supervised explanation corpora teach templated rationales, and a static forgery corpus leaves the decision boundary standing still while generators keep moving. We introduce \methodname{}, an adversarial reinforcement learning framework that pits two heterogeneous models against each other. A diffusion image editor learns to edit real photographs into fake counterparts of those same photographs that fool the current detector, while a reasoning MLLM learns to expose them with a verdict grounded in free-form reasoning. Both rewards are shortcut-proof by design: the attacker is credited only when its edit is faithfully executed, and the defender only when its verdict is correct. As the two models alternate, each round's attacker regenerates a harder training pool aimed at the current detector's blind spots, so the detector must generalize rather than memorize any fixed artifact distribution. Although the explanation is never rewarded, its quality rises round over round as a side effect of accuracy-only training. A detector trained within this loop improves monotonically across rounds on each of three external benchmarks.

Yicheng Bao, Xiahui Guo, Xuhong Wang +1

1East China Normal University · 2Shanghai Artificial Intelligence Laboratory

Adversarial Training Adversarial Examples AI-Generated Image Detection Adversarial Attacks on AI-Generated Image Detectors

Ask PDF

Mar 24, 2026

AgentFoX: LLM Agent-Guided Fusion with eXplainability for AI-Generated Image Detection

The realism of AI-generated images (AIGI) poses increasing challenges for reliable forensic detection, where heterogeneous expert detectors may produce conflicting predictions across diverse generative sources and post-processing conditions. Existing multi-expert fusion methods rely on fixed rules or learned fusion strategies, offering limited ability to assess sample-specific reliability, execute rigorous adjudication of conflicts, and provide evidence-grounded explanations. We propose AgentFoX, an LLM-driven agentic multi-expert framework for AIGI detection that employs a command-and-reasoning core to perform evidence fusion. Following predefined guidelines, the core coordinates designated subtasks to collect semantic and signal-level evidence, reason over structured contexts to determine authenticity, and generate an auditable report for explainability. During this process, Expert Profiles are constructed for model-centric reliability assessment, while Clustering Profiles are built for data-centric contextual analysis, jointly establishing evidence contexts for conflict resolution. Extensive evaluations across diverse benchmarks demonstrate the robustness and generalizability of AgentFoX under complex conditions.

Yangxin Yu, Yue Zhou, Bin Li +4

Guangdong Provincial Key Laboratory of Intelligent Information Processing, Shenzhen Key Laboratory of Media Security, and SZU-AFS Joint Innovation Center for AI Technology, Shenzhen University, Shenzhen 518060, China · School of Cyber Science and Technology, Sun Yat-sen University, China · Smart City Research Institute of China Electronics Technology Group Corporation, Shenzhen, China

Explainable Artificial Intelligence Digital Image Forensics LLM Agents AI-Generated Image Detection

Ask PDF

May 7, 2026

AI-Generated Images: What Humans and Machines See When They Look at the Same Image

The misuse of generative AI in online disinformation campaigns highlights the urgent need for transparent and explainable detection systems. In this work, we investigate how detectors for AI-generated images can be more effective in providing human-understandable explanations for their predictions. To this end, we develop a suite of detectors with various architectures and fine-tuning strategies, trained on our large-scale photorealistic fake image dataset, AIText2Image, and assess their performance on state-of-the-art text-to-image AI generators. We integrate 16 different explainable AI (XAI) methods into our detection framework, and the visual explanations are comprehensively refined and evaluated through a novel approach that prioritizes human understanding of AI-generated images, using both textual and visual responses collected from a survey of 100 participants. This framework offers insights into visual-language cues in fake image detection and into the clarity of XAI methods from a human perspective, measuring the alignment of XAI outputs with human preferences.

Silvia Poletti, Justin Ilyes, Marcel Hasenbalg +2

Austrian Institute of Technology, Vienna, Austria

Explainable Artificial Intelligence AI-Generated Image Detection Explainability Evaluation

Ask PDF

Abstract

AI-Generated Image Detection

Explore similar work

SPARED: Reasoning-Based AI-Generated Image Detection via Adversarially Edited Data

AgentFoX: LLM Agent-Guided Fusion with eXplainability for AI-Generated Image Detection

AI-Generated Images: What Humans and Machines See When They Look at the Same Image