Despite recent advances in diffusion models, AI generated images still often contain visual artifacts that compromise realism. Although more thorough pre-training and bigger models might reduce artifacts, there is no assurance that they can be completely eliminated, which makes artifact mitigation a highly crucial area of study. Previous artifact-aware methodologies depend on human-labeled artifact datasets, which are costly and difficult to scale, underscoring the need for an automated approach to reliably acquire artifact-annotated datasets. In this paper, we propose ArtiAgent, which efficiently creates pairs of real and artifact-injected images. It comprises three agents: a perception agent that recognizes and grounds entities and subentities from real images, a synthesis agent that introduces artifacts via artifact injection tools through novel patch-wise embedding manipulation within a diffusion transformer, and a curation agent that filters the synthesized artifacts and generates both local and global explanations for each instance. Using ArtiAgent, we synthesize 100K images with rich artifact annotations and demonstrate both efficacy and versatility across diverse applications. Code is available at link.
Figures & tables
Figure 1: Overview of our challenges and approach. The red boxes indicate the regions with visual artifacts. (a) Examples of structural visual artifacts in state-of-the-art diffusion models and the inability of VLMs to recognize or explain them. (b) Overview of ArtiAgent, a novel agentic framework that synthesizes artifacts for arbitrary visual contexts without human intervention. (c) Example of VLM-based artifact comprehension via detection, explanation, and localization. (d) Application to reward-guided text-to-image generation. (e) Application to image correction, where artifact-aware VLM-guided inpainting removes the flawed regions.
Figure 2
Figure 3: ArtiAgent consists of three coordinated agents: (1) the perception agent detects entities and subentities using Grounded-SAM; (2) the synthesis agent injects artifacts through patch mapping tool and the inversion-injection paradigm; and (3) the curation agent filters low-quality results and generates localized and global textual explanations.
Figure 4: Visualization of each target-reference patch mapping and its resulting artifact-injected image.
Figure 5: Inversion-injection module. In this example, the right arm from the reference patches is added to target patches below it.
Benchmark
Source Models
Sample
Bin.
Loc.
Exp.
Oldest
Newest
RichHF
SD2.1
Dreamlike
955
✓
Photoreal
LOKI
pix2pix
FLUX
229
✓
✓
SynthScars
Midjourney
DALL ⋅ E3
1K
✓
✓
ArtiBench
SD3.5
Nano-Banana
1K
✓
✓
✓
Table 2: Comparison of artifact benchmark datasets. If a benchmark reused another dataset as a source, we describe the generative models used in that source. The table with the full citations is provided in Appendix B.1 .
Methods
(a) Binary detection
(b) Localization
(c) Explanation
ArtiBench
RichHF
LOKI
SynthScars
ArtiBench
LOKI
SynthScars
ArtiBench
Acc
F1
mIoU
F1
mIoU
F1
mIoU
F1
mIoU
F1
ROUGE
CSS
ROUGE
CSS
ROUGE
CSS
PAL
-
-
0.079
0.028
0.021
0.037
0.035
0.053
0.040
0.066
-
-
-
-
-
-
DiffDoctor
-
-
0.077
0.139
0.175
0.274
0.083
0.136
0.081
0.137
-
-
-
-
-
-
LEGION
-
-
0.067
0.112
0.100
0.158
0.106 †
0.152 †
0.062
0.099
0.133
0.314
0.247 †
0.589 †
0.143
0.332
GPT-4o
0.619
0.601
0.086
0.040
0.037
0.056
0.032
0.052
0.049
0.084
0.107
0.266
0.125
0.404
0.143
0.433
Table 3: Artifact understanding performance across (a) binary detection, (b) localization, and (c) explanation. The benchmarks are listed in the order in which they were published.
Figure 6: Scaling effect of data generated by ArtiAgent with Qwen2.5-VL-7B. We average the results of all the benchmarks.
Methods
(a) Binary detection
(b) Localization
(c) Explanation
Acc
F1
mIoU
F1
ROUGE
CSS
SynthScars
0.555
0.548
0.094
0.147
0.156
0.521
ArtiAgent
0.555
0.550
0.074
0.121
0.222
0.606
Table 4: ArtiBench result of the VLM trained on a 1K dataset.
Figure 7: Reward-guided generation. ArtiAgent can train a reward model that guides diffusion to generate artifact-free images.
Figure 8: Image correction. The ArtiAgent-trained VLM can effectively guide image inpainting models to correct artifact regions.
Appendix figures & tables17 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 9: Attention Visualization. We compare the attention maps of InternVL3.5-8B before (base) and after (fine-tuned) training on ArtiAgent. The fine-tuned model reliably focuses on genuine artifact features across both synthesized images and real-world images.
Steps
Acc
F1
5/25
0.513
0.377
10/25
0.583
0.565
15/25
0.586
0.570
20/25
0.540
0.477
Appendix
Table 5: Ablation study on injection steps. Performance of Qwen2.5-VL-7B on ArtiBench binary detection when trained with 1K ArtiAgent samples generated using varying numbers of injection steps out of 25 total steps.
Figure 10: Visualizations of artifacts injected with ArtiAgent. The first and third rows show the original images, whereas the second and fourth rows show the output images with artifacts injected.
Figure 11: An instance of ArtiAgent with the annotation.
Figure 12: Guidance and rules for generating entity-subentity vocabulary sets.
Figure 13: Full prompt for generating entity-subentity vocabulary sets.
Figure 14: Type descriptions and instructions by artifact type.
Figure 15: Full prompt and example for data filtering.
Figure 16: Full prompt and example for regional explanation generation.
Figure 17: Full prompt and example for global explanation generation.
Figure 18: Hyperparameter study on PE injection and value injection steps (1). The image in the red box shows our selected configuration.
Table 6: Comparison of artifact benchmark datasets, their generative sources, and evaluation tasks. Highlighted entries denote dataset sources that were reused by subsequent benchmarks.
Figure 19: Artifact examples of the four benchmarks, listed in the order of publishment.
Figure 20: Evaluation prompts for VLMs.
Task
Template
Bin.
Q: Does this image contain any visual artifacts?
A: No.
Loc. 1
Q: Locate the ⟨entity⟩’s ⟨subentity⟩.
A: ⟨bbox⟩
Loc. 2
Q: Is there a ⟨entity⟩’s ⟨subentity⟩ in ⟨bbox⟩?
A: Yes / No
Appendix
Table 7: VQA templates for clean images.
Task
Template
Bin.
Q: Does this image contain any visual artifacts?
A: Yes.
Loc. 1
Q: Provide bounding boxes for all artifact regions.
A: [⟨bbox⟩]
Loc. 2
Q: Explain why region ⟨bbox⟩ is an artifact.
A: ⟨label⟩
Appendix
Table 8: VQA templates for artifact-injected images.
Prompt
Instruction Text
Localization
“Provide the bounding box for the artifact region. Format as [x_min, y_min, x_max, y_max]. Only output the bounding box.”
Captioning
“Describe what the image would look like if it had no artifacts. Provide a short caption of the clean scene.”
Verifying
“Is there an artifact within ⟨B⟩? Answer Yes or No.”
Appendix
Table 9: Prompts used by the VLM in the artifact-correction loop.
Diffusion-based image synthesis has made AI-generated images (AIGI) increasingly photorealistic, raising urgent concerns about authenticity in applications such as misinformation detection, digital forensics, and content moderation. Despite the substantial advances in AIGI detection, how to correct detected AI-generated images with visible artifacts and restore realistic appearance remains largely underexplored. Moreover, few existing work has established the connection between AIGI detection and artifact correction. To fill this gap, we propose GenShield, a unified autoregressive framework that jointly performs explainable AIGI detection and controllable artifact correction in a closed loop from diagnosis to restoration, revealing a mutually reinforcing relationship between these two tasks. We further introduce a Visual Chain-of-Thought based curriculum learning strategy that enables self-explained, multi-step diagnose-then-repair'' correction with an explicit stopping criterion. A high-quality dataset with large-scale artifact-restored'' pairs is also constructed alongside a unified evaluation pipeline. Extensive experiments on our correction benchmark and mainstream AIGI detection benchmarks demonstrate state-of-the-art performance and strong generalization of our method. The code is available at https://github.com/zhipeixu/GenShield.
Zhipei Xu, Xuanyu Zhang, Youmin Xu +5
School of Electronic and Computer Engineering, Peking University · Tencent Youtu Lab
Vision-language models (VLMs) are increasingly used to detect whether AI-generated images contain visible artifacts, yet their ability to analyze such artifacts remains poorly understood. A correct image-level decision can still hide important failures: a model may correctly flag an artifact while relying on the wrong visual cue, selecting the wrong region, or describing a defect that the image does not support. To evaluate these behaviors directly, we introduce SalArt-VQA, a diagnostic benchmark for fine-grained SALient ARTifact understanding in AI-generated images. SalArt-VQA contains 950 images and 3,681 human-authored multiple-choice questions spanning artifact images, matched real reference images, and paired generated reference images. Four aligned question types evaluate presence detection, semantic localization, spatial grounding, and evidence-grounded defect identification, while the reference splits test calibration and abstention when the annotated defect is absent. Across 20 VLMs, SalArt-VQA reveals failures that image-level detection accuracy hides: the strongest model reaches 99.37% detection recall on artifact images but answers all four artifact-side questions correctly on only 53.26% of images. Comparing artifact images with artifact-free references reveals a sensitivity-calibration tradeoff: sensitive models often make unsupported artifact claims, while conservative models avoid false alarms largely by missing real artifacts. These results show that high artifact detection accuracy alone does not imply grounded artifact understanding. SalArt-VQA exposes these hidden failure modes and provides a fine-grained evaluation of whether VLM artifact claims are supported by local visual evidence.
Ruotian Zhang, Xiaoxiao Sun, Junzhe Huang +2
Zhejiang University · The University of Queensland
Generative AI can now produce highly realistic images, yet current models still exhibit subtle but critical defects that undermine their reliability. While existing AI-generated image (AGI) evaluation benchmarks have made notable progress, comprehensive AGI defect diagnosis remains underexplored. To bridge this gap, we introduce AGIDefect-4K, a richly annotated dataset of 4,000 images from 15 state-of-the-art generative models spanning both open-source and closed-source systems. AGIDefect-4K features hierarchical defect annotations: (1) detection labels identifying whether defects exist, (2) pixel-level segmentation masks localizing defective regions, and (3) detailed textual explanations characterizing defect types and their perceptual impact. Each image is further annotated with an overall quality score. Building on this, we present AGIDA (AGI Defect Assistant), a baseline framework leveraging Multimodal Large Language Models (MLLMs) for joint defect detection, localization, explanation, and quality prediction. Comprehensive benchmarking on AGIDefect-4K reveals that AGI defect understanding remains challenging, underscoring the value of this dataset. The dataset is publicly available at https://github.com/sxfly99/AGIDefect-4K.