Beyond Semantics: Uncovering the Physics of Fakes via Universal Physical Descriptors for Cross-Modal Synthetic Detection

Mei Qiu; Jianqiang Zhao; Yanyun Qu

cs.CVApr 6, 2026

Beyond Semantics: Uncovering the Physics of Fakes via Universal Physical Descriptors for Cross-Modal Synthetic Detection

Authors: Mei Qiu, Jianqiang Zhao, Yanyun Qu

Organizations: SDIC Intelligence Information Technology Co., Ltd. · Xia’men University, China

Abstract

The rapid advancement of AI generated content (AIGC) has blurred the boundaries between real and synthetic images, exposing the limitations of existing deepfake detectors that often overfit to specific generative models. This adaptability crisis calls for a fundamental reexamination of the intrinsic physical characteristics that distinguish natural from AI-generated images. In this paper, we address two critical research questions: (1) What physical features can stably and robustly discriminate AI generated images across diverse datasets and generative architectures? (2) Can these objective pixel-level features be integrated into multimodal models like CLIP to enhance detection performance while mitigating the unreliability of language-based information? To answer these questions, we conduct a comprehensive exploration of 15 physical features across more than 20 datasets generated by various GANs and diffusion models. We propose a novel feature selection algorithm that identifies five core physical features including Laplacian variance, Sobel statistics, and residual noise variance that exhibit consistent discriminative power across all tested datasets. These features are then converted into text encoded values and integrated with semantic captions to guide image text representation learning in CLIP. Extensive experiments demonstrate that our method achieves state-of-the-art performance on multiple Genimage benchmarks, with near-perfect accuracy (99.8%) on datasets such as Wukong and SDv1.4. By bridging pixel level authenticity with semantic understanding, this work pioneers the use of physically grounded features for trustworthy vision language modeling and opens new directions for mitigating hallucinations and textual inaccuracies in large multimodal models.

Figures & tables

Figure 1: Overall workflow of the proposed synthetic image detection framework. (a) Enhanced caption preparation: Integrate input images’ core physical features into captions to enrich representation. (b) Training: Use enhanced captions and class prompts to train the model for real/AI-generated image discrimination. (c) Testing: Input images into the trained model to predict AI generation. The training and testing procedure are totally the same with C2pClip [ 27 ] .

$F_{1}\!=\!\frac{1}{HW}\sum_{i,j}\text{HSV}_{i,j,1}$	$F_{2}\!=\!\frac{1}{HW}\sum_{i,j}\text{HSV}_{i,j,2}$	$F_{3}\!=\!\log\!\left(1+\operatorname{Var}(\nabla^{2}I_{g})\right)$
$F_{4}\!=\!10\!\cdot\!\frac{1}{HW}\sum_{i,j}\sqrt{S_{x}^{2}\!+\!S_{y}^{2}}$	$F_{5}\!=\!10\!\cdot\!\sqrt{\frac{1}{HW}\sum_{i,j}(M_{i,j}\!-\!\mu_{M})^{2}}$	$F_{6}\!=\!\log\!\left(1+\frac{1}{N}\sum_{k}\operatorname{Var}(\operatorname{DCT}(B_{k}))\right)$
$F_{7}\!=\!\log\!\left(1+1000\operatorname{Var}(R)\right)$	$F_{8}\!=\!-\sum_{b}p_{b}\log_{2}p_{b}$	$F_{9}\!=\!\frac{\operatorname{Cov}(R,G)}{\sqrt{\operatorname{Var}(R)\operatorname{Var}(G)}}$
$F_{10}\!=\!\frac{\operatorname{Cov}(R,B)}{\sqrt{\operatorname{Var}(R)\operatorname{Var}(B)}}$	$F_{11}\!=\!-\sum_{v}p_{v}\log_{2}p_{v}$	$F_{12}\!=\!-\sum_{v}p_{v}\log_{2}p_{v}$
$F_{13}\!=\!\log\!\left(1+\operatorname{Var}(\text{HSV}_{:,:,0})\right)$	$F_{14}\!=\!\frac{1}{HW}\sum_{i,j}\frac{E_{i,j}}{255}$	$F_{15}\!=\!\log\!\left(1+\|\operatorname{Kurt}(B)\|\right)$

Table 1: Mathematical definitions of 15 synthetic image detection features. Key symbol definitions:

i,j

= 2D pixel coordinates (height/width);

H,W

= image height/width;

\text{HSV}_{i,j,c}

= HSV color space value at pixel

(i,j)

for channel

c

(1=saturation, 2=brightness, 0=hue);

I_{g}

= grayscale image;

\nabla^{2}

= Laplacian operator (second-order derivative for sharpness);

S_{x}/S_{y}

= Sobel horizontal/vertical edge responses;

M_{i,j}

= Sobel edge magnitude at

(i,j)

;

\mu_{M}

= mean edge magnitude;

\operatorname{DCT}(\cdot)

= Discrete Cosine Transform;

B_{k}

k

-th 8×8 image block;

N=1000

(sampled blocks for efficiency);

R

= residual noise (image minus non-local means denoised version);

p_{b}

= normalized frequency of Local Binary Pattern (LBP) bin

b

;

\operatorname{Cov}(\cdot,\cdot)

= covariance;

\operatorname{Var}(\cdot)

= variance;

R/G/B

= red/green/blue color channels;

p_{v}

= normalized pixel value frequency for chrominance channels (Cr/Cb);

E_{i,j}

= Canny edge detector output (0=non-edge, 255=edge);

\operatorname{Kurt}(\cdot)

= kurtosis (tailedness of intensity distribution).

Figure 2: Stability ( $S_{s}$ , left) and discriminability ( $S_{d}$ , right) scores of image features for fake image detection. Based on the thresholds, set features to Core Feature(green), Usable Feature(orange), red Unstable High-Discrim(red), and Unusable Feature(grey).

Figure 3: Density distributions of four core features (Laplacian variance, Sobel magnitude mean/std, LBP variance) over ADM/Midjourney (GenImage) and BigGAN/CRN/Glide (UniversalFakeDetect). Blue = real images, red = fake images.

Figure 4: Image-text pairs with original texts generated by ClipCap and enhanced by core features’ description.

Method	Training Dataset	Test Datasets (Genimage)								mAcc
Method	Training Dataset	Midjourney	SDv1.4	SDv1.5	ADM	GLIDE	Wukong	VQDM	BigGAN	mAcc
C2pClip	Midjourney	97.2	96.5	96.2	79.8	96.6	93.6	87.0	92.7	92.4
	SDv1.4	88.2	90.9	97.9	96.4	99.0	98.8	96.5	98.7	95.8
	SDv1.5	96.2	98.8	98.5	92.8	98.5	98.4	97	97.4	97.2
	ADM	86.4	96.8	96.4	98.4	98.6	95.1	98.2	98.2	96.0
	wukong	96.4	98.9	98.4	92.9	98.4	98.5	97.9	97.2	97.3

Table 2: Impact of Training Dataset on Test Accuracy (Genimage Dataset). All methods are evaluated on the same 8 fixed test datasets (Midjourney, SDv1.4, SDv1.5, ADM, GLIDE, Wukong, VQDM, BigGAN) but trained on distinct datasets (Midjourney, SDv1.4, SDv1.5). Results highlight how training data choice affects cross-model generalization for C2pClip and our method (Ours), with bold values indicating our method achieve better performance on the corresponding test dataset .

Method	Ref	Midjourney	SDv1.4	SDv1.5	ADM	GLIDE	Wukong	VQDM	BigGAN	mAcc
UniFD(2023)	CVPR2023	93.9	96.4	96.2	71.9	85.4	94.3	81.6	90.5	88.8
NPR (2024c)	CVPR2024	81.0	98.2	97.9	76.9	89.8	96.9	84.1	84.2	88.6
FreqNet(2024b)	AAAI2024	89.6	98.8	98.6	66.8	86.5	87.3	75.8	81.4	86.8
FatFormer(2024)	CVPR2024	92.7	100.0	99.9	75.9	88.0	99.9	98.8	55.8	88.9
C2pClip (Trump.Biden)	AAAI2025	82.2	95.1	95.5	95.1	98.9	98.7	93.8	98.3	94.7
C2pClip (Deepfake.Camera)	AAAI2025	88.2	90.9	97.9	96.4	99.8	98.8	96.5	98.7	95.8

Table 3: Cross-model Accuracy (Acc) Performance on the Genimage Dataset.

Figure 5: Quantitative evaluation of image-text similarity based on pre-trained Clip model.

Figure 6: Cosine similarity comparison between two caption styles across 7 datasets. Blue/orange: results using caption “The major features are:”. Green/red: results using caption “The physical features are:”.

Panel of Figure 7 — Figure 7: Image-text cosine similarity distributions of ADM

Panel of Figure 8 — Figure 8: Image-text cosine similarity distributions of BigGAN

Panel of Figure 9 — Figure 9: Image-text cosine similarity distributions of SDv1.5

Panel of Figure 10 — Figure 10: Image-text cosine similarity distributions of VQDM

Panel of Figure 11 — Figure 11: Image-text cosine similarity distributions of Wukong

Panel of Figure 12 — Figure 12: Image-text cosine similarity distributions of Midjourney

Panel of Figure 13 — Figure 13: Image-text cosine similarity distributions of Glide

AI-Generated Image Detection

AI-generated image detection identifies synthetic or AI-edited images and develops methods that generalize across generators, domains, and image transformations.

See all 194 papers

May 28, 2026

FakeVLM-R1: Internalizing Physical Laws via CoT for Synthetic Image Detection

The development of generative artificial intelligence technologies has propelled the visual realism of synthetic images to an unprecedented level. Although current interpretable detection methods based on Large Multimodal Models (LMMs) have made certain progress, they still rely on imitation learning derived from massive volumes of forged data. Consequently, they lack genuine causal reasoning capabilities and are prone to explanatory hallucinations. To overcome this bottleneck, we propose FakeVLM-R1, aiming to endow the model with human-like critical thinking capabilities when performing synthetic detection tasks. Building upon Supervised Fine-Tuning (SFT), this framework integrates Group Relative Policy Optimization (GRPO) with a Critical Thinking Chain-of-Thought (CoT) mechanism. During the inference phase, the model executes a "bidirectional dialectical reasoning" process: while proposing a forgery hypothesis, it must simultaneously invoke physical commonsense to construct an authenticity counter-proof. Furthermore, we constructed the FakeClue++ dataset with high-quality samples, which extensively introduces annotations guided by the physical laws of authentic images, providing a unified authenticity anchor for the model. Experiments confirm that FakeVLM-R1 achieves SOTA performance the evaluated models across multiple benchmarks. It not only achieves high-precision, logically interpretable detection but also resolves the over-rejection bias of existing methods against real images, demonstrating generalization and robustness against perturbations.

Leqi Zhu, Junyan Ye, Kaiqing Lin +3

Shanghai AI Lab · Nanjing University · Sun Yat-Sen University +3

Explainable Deepfake Detection CoT Reasoning VLM Reasoning AI-Generated Image Detection

Ask PDF

Jun 7, 2026

SSAFE: Simple and Strong AI-Generated Image Detection via Frozen Vision Encoders

The rapid advancement of generative models has blurred the boundary between synthetic and real imagery, creating an urgent need for reliable deepfake detection. Yet most existing approaches rely on massive real--fake datasets, which are increasingly difficult to maintain as new generators continue to emerge. In this work, we investigate how much information about image authenticity is already encoded in modern multimodal vision representations. We find that frozen multimodal encoders naturally separate real and synthetic images in their embedding space, enabling a simple linear classifier to achieve strong performance without task-specific fine-tuning. Motivated by this observation, we develop a representation-aware data curation strategy that selects a compact set of representative generators for training. The resulting training set contains only 10K images, compared to 288K in AIGIBench and 4M in OpenFake, while improving robustness to unseen generators and distribution shifts. We additionally introduce RealWorldBench, a benchmark consisting of modern camera photographs, contemporary stock images, and outputs from recent commercial generators. Experiments across multiple benchmarks show that combining frozen multimodal representations with carefully curated training data provides a simple and effective approach to AI-generated image detection.

Seunghyun Lee, Byoungkwon Kim, Jaehyun Nam +2

1KAIST · † Work done at KAIST. · 2Google Cloud AI

Domain Generalization Training Data Selection AI-Generated Image Detection Multimodal Deepfake Detection

Ask PDF

Aug 2, 2025

NS-Net: Decoupling CLIP Semantic Information through NULL-Space for Generalizable AI-Generated Image Detection

The rapid progress of generative models, such as GANs and diffusion models, has facilitated the creation of highly realistic images, raising growing concerns over their misuse in security-sensitive domains. While existing detectors perform well under known generative settings, they often fail to generalize to unknown generative models, especially when semantic content between real and fake images is closely aligned. In this paper, we revisit the use of CLIP features for AI-generated image detection and uncover a critical limitation: the high-level semantic information embedded in CLIP's visual features hinders effective discrimination. To address this, we propose the NS-Net, a novel detection framework that leverages NULL-Space projection to decouple semantic information from CLIP's visual features, followed by contrastive learning to capture intrinsic distributional differences between real and generated images. Furthermore, we design a Patch Selection strategy to preserve fine-grained artifacts by selecting patches with the highest and lowest spectral entropy. Extensive experiments on an open-world benchmark comprising images generated by 47 diverse generative models show that NS-Net outperforms existing state-of-the-art methods, achieving a 5.0% improvement in detection accuracy, thereby demonstrating strong generalization across both GAN- and diffusion-based image generation techniques. Our code is available at https://github.com/HorizonTEL/NS-Net.

Jiazhen Yan, Fan Wang, Ziwen He +2

Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology, Nanjing, 210044, China. · University of Macau, Macau, 999078, China.

AI-Generated Image Detection

Ask PDF

Abstract

AI-Generated Image Detection

Explore similar work

FakeVLM-R1: Internalizing Physical Laws via CoT for Synthetic Image Detection

SSAFE: Simple and Strong AI-Generated Image Detection via Frozen Vision Encoders

NS-Net: Decoupling CLIP Semantic Information through NULL-Space for Generalizable AI-Generated Image Detection