UC-VLM: Consistency-Driven Learning for AI-Generated Image Detection with Vision-Language Large Models

Lei Tan; Shuwei Li; Mohan Kankanhalli; Robby T. Tan

cs.CVAug 15, 2026

UC-VLM: Consistency-Driven Learning for AI-Generated Image Detection with Vision-Language Large Models

Authors: Lei Tan, Shuwei Li, Mohan Kankanhalli, Robby T. Tan

Organizations: National University of Singapore, Singapore

Abstract

Vision-Language Large Models (VLLMs) are promising for AI-generated image (AIGI) detection because they can produce both a prediction and a natural-language output. However, most existing VLLM-based detectors primarily fine-tune the language side while giving limited attention to low-level visual forensic cues. They also often depend on manually crafted prompts or human-annotated rationales, which limits scalability.We present UC-VLM, a unified multi-stage framework for AIGI detection that relies solely on binary supervision. UC-VLM first identifies effective instruction variants automatically. It then reuses the same binary label within a multi-stage training framework: (i) a visual discrimination objective that strengthens sensitivity to non-semantic forensic cues, and (ii) a label-conditioned generation objective that uses the binary label to supervise textual outputs. This design turns weak binary supervision into a shared supervision signal for both the visual pathway and the language output. Our key novelty is a unified multi-stage binary-supervised framework that consistently reuses the same authenticity labels for visual adaptation and label-conditioned text generation, while leveraging automatically optimized instructions to reduce prompt sensitivity without requiring human-written rationales or hand-crafted prompts.Experiments show that UC-VLM achieves 96.1% average accuracy on GenImage, exceeding the strongest prior result by 4.6%, and obtains 69.6% / 77.9% accuracy on Chameleon under ProGAN / SDV1.4 training, surpassing the best baseline by 11.2% / 15.3%, respectively.

Figures & tables

Figure 1: Comparison of paradigms for AI-generated image detection. (a) Conventional classifiers predict real/fake from image data. (b) Existing VLLM-based detectors often rely on manually crafted prompts and human-annotated rationales to produce a label with accompanying text. (c) UC-VLM learns from binary authentic/generated labels only, combining visual adaptation, instruction optimization, and label-conditioned generation to improve robustness to prompt choice and provide scalable supervision for textual outputs.

Figure 2: Overview of UC-VLM. UC-VLM is built upon Qwen2.5-VL-7B and learns AI-generated image detection under binary authentic/generated supervision. The framework includes three coordinated components: (i) Visual Adaptation , which enhances the vision encoder’s sensitivity to local non-semantic forensic cues; (ii) Instruction Optimization , which automatically searches for effective instruction variants for authenticity assessment; and (iii) Label-Conditioned Regeneration , which conditions text generation on the binary label to provide scalable supervision for textual outputs while freezing the vision tower.

Prompt ID	Accuracy (%)
P-1: Authentic/Generated QA	55.5
P-2: Real/Fake QA	59.8
P-3: Answer with Explanation	63.6

Table 1: Effect of prompt formulation on the Chameleon dataset using the pretrained Qwen2.5-VL-7B model. Structured prompts (e.g., P-3) yield higher detection accuracy, indicating that VLM performance is strongly influenced by prompt formulation.

Method	Venue	Testing Subset								Avg Accuracy (%)
Method	Venue	Midjourney	SDV1.4	SDV1.5	ADM	GLIDE	Wukong	VQDM	BigGAN	Avg Accuracy (%)
ResNet-50 [ 14 ]	CVPR’16	54.9	99.9	99.7	53.5	61.9	98.2	56.6	52.0	72.1
DeiT-S [ 58 ]	ICML’21	55.6	99.9	99.8	49.8	58.1	98.9	56.9	53.5	71.6
Swin-T [ 32 ]	ICCV’21	62.1	99.9	99.8	49.8	67.6	99.1	62.3	57.6	74.8
CNNSpot [ 60 ]	CVPR’20	52.8	96.3	95.9	50.1	39.8	78.6	53.4	46.8	64.2
Spec [ 73 ]	WIFS’19	52.0	99.4	99.2	49.7	49.8	94.8	55.6	49.8	68.8

Table 2: Comparisons between the proposed UC-VLM and state-of-the-art methods on the GenImage testing set. The symbol †indicates our reproduction using the source code.

Training Set	Methods
Training Set	CNNSpot [ 60 ]	FreDect [ 12 ]	Fusing [ 18 ]	GramNet [ 34 ]	LNP [ 28 ]	UniFD [ 44 ]	DIRE [ 63 ]	Patchcraft [ 75 ]	NPR [ 55 ]	AIDE [ 67 ]	UC-VLM
ProGAN	56.9	55.6	57.0	58.9	57.1	57.2	58.2	53.8	57.3	58.4	69.6
SDV1.4	60.1	56.9	57.1	61.0	55.6	55.6	59.7	56.3	58.1	62.6	77.9

Table 3: Comparisons between the proposed UC-VLM and state-of-the-art methods in terms of accuracy on the Chameleon dataset.

Setting	GenImage (%)	Chameleon (%)
Baseline Methods
Baseline (w/o fine-tuning)	51.9	55.5
Baseline (Visual)	76.8	50.9
Baseline (LLM)	80.3	71.0
Baseline (Visual + LLM)	77.6	52.0
Component-wise Ablations

Table 4: Ablation study of UC-VLM components. Baseline (Visual) and Baseline (LLM) denote fine-tuning only the vision encoder or the language module, respectively, while Baseline (Visual + LLM) denotes naive joint fine-tuning of both.

Figure 3: Visualization of results on Chameleon under different step and Top-

k

. According to the results, we finally set the step to 10 and the

k

to 2.

Figure 4: Instruction comparison on the Chameleon dataset. After instruction optimization, UC-VLM’s evolved prompt shifts from generic artifact descriptions to more specific visual cues, demonstrating stronger grounding in visual information.

Figure 5: Qualitative results of UC-VLM on Chameleon dataset. UC-VLM provides structured four-step explanations for both authentic and AI-generated images, often describing texture irregularities, lighting inconsistencies, depth errors, and semantic-physics mismatches. These examples illustrate its qualitative behavior.

Figure 6: Failure Cases of UC-VLM on Chameleon dataset. UC-VLM may misclassify highly realistic generated photos as authentic when artifacts are visually subtle, and may also mistake overly pristine or heavily post-processed authentic images. It is also uncertain on artistic or stylized imagery, where coherent generated artworks can be judged as authentic while authentic paintings can be judged as generated.

AI-Generated Image Detection

AI-generated image detection identifies synthetic or AI-edited images and develops methods that generalize across generators, domains, and image transformations.

See all 194 papers

Sep 30, 2026

Typographic Attack Against VLM-based AI-generated Image Detection

Vision-language models (VLMs) are increasingly used for AI-generated image (AIGI) detection, providing natural-language explanations for authenticity judgments. However, their ability to interpret text within images may also expose these judgments to misleading semantic cues. We systematically evaluate typographic attack strategies across detection-oriented, open-weight, and commercial VLMs, considering both real-to-fake and fake-to-real attacks. Our results show that reasoning modes generally exhibit greater vulnerability than direct modes and that attack effectiveness exhibits pronounced directional asymmetry. Moreover, larger models tend to exhibit higher clean detection accuracy but also higher attack success rates. We further examine attack robustness under image and text transformations and investigate whether overlays indicating the correct class can aid error correction. Together, these analyses characterize how typographic attacks influence authenticity judgments and expose limitations of current VLM-based AIGI detection systems.

Eunmin Lee, Jungwoo Kim, Jong-Seok Lee

School of Integrated Technology, Yonsei University

VLM Evaluation VLM Robustness Adversarial Attacks on VLMs AI-Generated Image Detection

Ask PDF

Aug 5, 2026

Unleashing the Potential of Vision-Language Models for Generalizable AI-Generated Image Detection

Recent work has shown that a simple linear probe on frozen representations from modern vision foundation models (VFMs) can achieve state-of-the-art AIGI detection performance, substantially outperforming specialized detectors in challenging in-the-wild scenarios. This finding has established DINOv3 as the dominant foundation-model baseline for subsequent improvements. However, we find that the vision-language model Perception Encoder (PE) holds greater potential for AIGI detection, because its language-aligned representation preserves high-level provenance semantics. Specifically, PE exhibits stronger local provenance organization than DINOv3 in its frozen feature space. However, semantic-agnostic linear probing fails to exploit this structure, as PE-Linear still underperforms DINOv3-Linear by 4.1% on In-the-Wild. Based on this observation, we propose Semantic Prototype Calibration (SPC), which constructs category prototypes from forensic semantic information and calibrates them with supervised data. We apply SPC to PE and refer to the resulting detector as PE-SPC. Our analysis shows that this simple design achieves stronger generalization. Across cross-generator, post-processing, and in-the-wild benchmarks, PE-SPC surpasses the previous DINOv3 baseline and achieves new state-of-the-art results.

Weihan Cai, Hao Tan, Zichang Tan +2

State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China · Sangfor Technologies Inc., Shenzhen 518055, China +1

Prototype-Based Classification Vision-Language Models AI-Generated Image Detection

Ask PDF

Apr 29, 2026

TAP into the Patch Tokens: Leveraging Vision Foundation Model Features for AI-Generated Image Detection

Recent methods demonstrate that large-scale pretrained models, such as CLIP vision transformers, effectively detect AI-generated images (AIGIs) from unseen generative models when used as feature extractors. Many state-of-the-art methods for AI-generated image detection build upon the original CLIP-ViT to enhance this generalization. Since CLIP's release, numerous vision foundation models (VFMs) have emerged, incorporating architectural improvements and different training paradigms. Despite these advances, their potential for AIGI detection and AI image forensics remains largely unexplored. In this work, we present a comprehensive benchmark across multiple VFM families, covering diverse pretraining objectives, input resolutions, and model scales. We systematically evaluate their out-of-the-box performance for detecting fully-generated AI-images and AI-inpainted images, and discover that the best model outperforms the original CLIP by more than 12% in accuracy, beating established approaches in the process. To fully leverage the features of a modern VFM, we propose a simple redesign of the classifier head by utilizing tunable attention pooling (TAP), which aggregates output tokens into a refined global representation. Integrating TAP with the latest VFMs yields substantial performance gains across several AIGI detection benchmarks, establishing a new state-of-the-art on two challenging benchmarks for in-the-wild detection of AI-generated and -inpainted images.

Ahmed Abdullah, Nikolas Ebert, Oliver Wasenmüller

Mannheim University of Applied Sciences, Germany

Vision Foundation Models Vision Foundation Model Adaptation AI-Generated Image Detection

Ask PDF

Abstract

AI-Generated Image Detection

Explore similar work

Typographic Attack Against VLM-based AI-generated Image Detection

Unleashing the Potential of Vision-Language Models for Generalizable AI-Generated Image Detection

TAP into the Patch Tokens: Leveraging Vision Foundation Model Features for AI-Generated Image Detection