Deepfakes increasingly undermine trust in digital media, yet existing detectors often struggle to generalize across unseen manipulation techniques, compression levels, and dataset shifts. Most approaches rely on RGB appearance, frequency-domain artifacts, or their combination, learning cues present in the final synthesized image. While effective in-distribution, such cues are frequently generator-specific and can diminish as synthesis models evolve. We argue that a complementary perspective can be obtained through implicit neural representations (INRs), which expose how an image is reconstructed rather than merely how it appears. By fitting an INR to an image and analyzing its internal activations, one can reveal reconstruction dynamics that are difficult to observe in the image domain alone and may capture more intrinsic differences between genuine and synthetic content. To this end, we propose Spectra, a deepfake detection framework that combines semantic visual representations with artifact-aware cues derived from INR reconstruction. Spectra extracts multi-level reconstruction heatmaps from image-specific INRs and jointly models them with semantic features through a bidirectional fusion mechanism. This enables direct interaction between image semantics and reconstruction artifacts, allowing the model to identify forgery cues that are less dependent on specific generation pipelines. Extensive experiments on multiple deepfake benchmarks demonstrate improved robustness and cross-dataset generalization, consistently outperforming detectors based solely on RGB appearance or semantic representations. Furthermore, we show that the proposed representation extends beyond deepfake detection to face presentation attack detection, highlighting its potential as a unified framework for facial attack detection.
Figures & tables
Figure 1: Spectra provides a unified framework for two authenticity tasks: Spectra employs a unified INR-guided artifact representation to address both deepfake detection and presentation attack detection (PAD). The framework achieves state-of-the-art average AUC on deepfake detection ( 86.6% ) and PAD ( 97.17% ), demonstrating strong generalization across both digital and physical forgery domains.
Figure 2: Overview of Spectra . Given an input face image, a frozen CLIP vision encoder extracts multiscale semantic representations from selected transformer layers, while an INR fitted to the same image produces layer-wise heatmaps that capture reconstruction dynamics and frequency-specific artifacts. The heatmaps are processed by spatial and frequency encoders and fused through a gated aggregation module to obtain artifact-aware heatmap tokens. In parallel, multiscale CLIP features are combined through weighted fusion to produce semantic tokens. To model interactions between semantic content and reconstruction artifacts, we employ both cross-attention (CLIP queries heatmap features) and reverse cross-attention (heatmap queries CLIP features). The resulting representations, together with global CLIP and heatmap descriptors, are fused and passed to an MLP for real/fake classification. A projection head attached to the shared representation enables supervised contrastive learning, encouraging stronger separation between genuine and manipulated samples.
Figure 3: Layer-wise PCA decomposition of SIREN reconstruction dynamics. Top: a genuine face from CDF v2 ; bottom: a deepfake. Although both are reconstructed with comparable fidelity (rightmost column), their internal activations diverge sharply across layers
Figure 4: ROC curves. Comparison of the proposed Spectra model and the RGB-only ablation on the Celeb-DF v1, and v2 benchmarks.
Figure 5: t-SNE visualization of feature embeddings on the CDFv1 dataset. The proposed Spectra model produces more discriminative feature representations than the RGB -only baseline, resulting in improved separation between real and fake samples.
Method
CDF v1
CDF v2
FSh
DFD
Avg
General Methods
MesoNet’18 [ 1 ]
0.735
0.609
0.566
0.548
0.615
MesoInc.’18 [ 1 ]
0.736
0.696
0.643
0.607
0.671
Xception’18 [ 12 ]
0.779
0.736
0.624
0.816
0.739
DSP-FWA’18 [ 34 ]
0.789
0.668
0.555
0.740
0.688
EffNet’19 [ 53 ]
0.790
0.748
0.616
0.815
0.742
Table 1: Cross-dataset frame-level AUC. Best and second-best results are shown in bold and underlined , respectively.
Figure 6: INR layer-wise decomposition. Genuine samples produce smooth activations across layers, while attacks yield noisier intermediate responses and less consistent structure.
Method
M
C
I
O
Avg
RP’21 [ 59 ]
87.43
86.76
86.38
92.43
88.25
FG’22 [ 50 ]
91.25
80.58
95.34
89.85
89.25
FDGAL’24 [ 26 ]
93.29
88.10
91.92
90.54
90.96
MRGA’23 [ 63 ]
89.83
88.36
94.02
90.86
90.77
ST’24 [ 32 ]
90.00
89.98
87.81
92.73
90.13
HPDR’24 [ 25 ]
96.02
94.42
92.49
95.26
94.55
Table 3: Cross-dataset PAD performance (% AUC). M, C, I, and O denote the four evaluation protocols. Best and second-best results are highlighted in bold and underlined , respectively.
Existing image forgery detectors often suffer from generalization to unseen manipulation methods due to the limited ability to capture transferable forensic cues. Recent cross-reconstruction based methods attempt to improve generalization through semantic-artifact disentanglement, but typically align heterogeneous artifacts across generators and exclude artifact representations during reconstruction, which may overlook the inherent diversity and visual cues of manipulation artifacts. In this work, we revisit cross-reconstruction and introduce an artifact-oriented disentanglement framework for robust image forgery detection. We argue that artifact diversity, i.e., the intrinsic variations of manipulation artifacts introduced by different generation processes, contains complementary forensic cues rather than undesirable domain variations. Instead of enforcing explicit artifact alignment, our framework preserves diverse artifact characteristics through semantically aligned cross-generator reconstruction. Furthermore, we incorporate artifact representations into the reconstruction process and introduce a masked frequency-aware reconstruction strategy to emphasize manipulation-related residuals while reducing semantic interference. This design enables the model to learn transferable forensic representations from diverse artifacts. Extensive experiments on multiple benchmark datasets demonstrate improvements under both cross-dataset and cross-generator evaluation settings. Further analysis and ablation studies validate the effectiveness of artifact diversity preservation and artifact-aware cross-reconstruction.
Bingjian Yang, Shilei Zhao, Zheng Wang
National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University, China
Detecting AI-generated images remains a significant challenge because detectors trained on specific generators often fail to generalize to unseen models; however, while pixel-level artifacts vary across models, frequency-domain signatures exhibit greater consistency, providing a promising foundation for cross-generator detection. To address this, we propose SPARK-IL, a retrieval-augmented framework that combines dual-path spectral analysis with incremental learning by utilizing a partially frozen ViT-L/14 encoder for semantic representations alongside a parallel path for raw RGB pixel embeddings. Both paths undergo multi-band Fourier decomposition into four frequency bands, which are individually processed by Kolmogorov-Arnold Networks (KAN) with mixture-of-experts for band-specific transformations before the resulting spectral embeddings are fused via cross-attention with residual connections. During inference, this fused embedding retrieves the k nearest labeled signatures from a Milvus database using cosine similarity to facilitate predictions via majority voting, while an incremental learning strategy expands the database and employs elastic weight consolidation to preserve previously learned transformations. Evaluated on the UniversalFakeDetect benchmark across 19 generative models -- including GANs, face-swapping, and diffusion methods -- SPARK-IL achieves a 94.6% mean accuracy, with the code to be publicly released at https://github.com/HessenUPHF/SPARK-IL.
Univ. Polytechnique Hauts-de-France, Valenciennes, France · Institute of Applied Sciences and Intelligent Systems – Lecce, Italy · Department of Innovation Engineering, University of Salento, Italy +1
Pretrained self-supervised representations have emerged as a core component of current deepfake detection methods, yet it remains unclear which of their properties make real and fake media distinguishable. In this work, we uncover a surprisingly consistent phenomenon: across multiple pretrained models, datasets, and both image and video domains, fake samples systematically produce lower-magnitude representations than their real counterparts. Motivated by this finding, we formulate deepfake detection as an anomaly detection problem and show that simple statistics of feature magnitude achieve competitive performance with far more sophisticated deepfake detection methods. We further investigate the origin of this effect and demonstrate that reduced feature magnitude is primarily associated with semantic shifts introduced by fake content, while low-level generative fingerprints play a comparatively smaller role. Finally, we show that this discriminative signal strengthens as the size of the underlying foundation model grows, suggesting that advances in representation learning naturally translate into stronger zero-shot deepfake detectors.
Stefan Smeu, Dragos-Alexandru Boldisor, Elisabeta Oneata +1