Spectra: Multi-Scale Implicit Decomposition for Deepfake Detection

Dhananjaya Jayasundara; Kartik Narayan; Vishal M. Patel

cs.CVOct 22, 2026

Spectra: Multi-Scale Implicit Decomposition for Deepfake Detection

Authors: Dhananjaya Jayasundara, Kartik Narayan, Vishal M. Patel

Organizations: *All authors are with Johns Hopkins University

Abstract

Deepfakes increasingly undermine trust in digital media, yet existing detectors often struggle to generalize across unseen manipulation techniques, compression levels, and dataset shifts. Most approaches rely on RGB appearance, frequency-domain artifacts, or their combination, learning cues present in the final synthesized image. While effective in-distribution, such cues are frequently generator-specific and can diminish as synthesis models evolve. We argue that a complementary perspective can be obtained through implicit neural representations (INRs), which expose how an image is reconstructed rather than merely how it appears. By fitting an INR to an image and analyzing its internal activations, one can reveal reconstruction dynamics that are difficult to observe in the image domain alone and may capture more intrinsic differences between genuine and synthetic content. To this end, we propose Spectra, a deepfake detection framework that combines semantic visual representations with artifact-aware cues derived from INR reconstruction. Spectra extracts multi-level reconstruction heatmaps from image-specific INRs and jointly models them with semantic features through a bidirectional fusion mechanism. This enables direct interaction between image semantics and reconstruction artifacts, allowing the model to identify forgery cues that are less dependent on specific generation pipelines. Extensive experiments on multiple deepfake benchmarks demonstrate improved robustness and cross-dataset generalization, consistently outperforming detectors based solely on RGB appearance or semantic representations. Furthermore, we show that the proposed representation extends beyond deepfake detection to face presentation attack detection, highlighting its potential as a unified framework for facial attack detection.

Figures & tables

Figure 1: Spectra provides a unified framework for two authenticity tasks: Spectra employs a unified INR-guided artifact representation to address both deepfake detection and presentation attack detection (PAD). The framework achieves state-of-the-art average AUC on deepfake detection ( $86.6\%$ ) and PAD ( $97.17\%$ ), demonstrating strong generalization across both digital and physical forgery domains.

Figure 2: Overview of Spectra . Given an input face image, a frozen CLIP vision encoder extracts multiscale semantic representations from selected transformer layers, while an INR fitted to the same image produces layer-wise heatmaps that capture reconstruction dynamics and frequency-specific artifacts. The heatmaps are processed by spatial and frequency encoders and fused through a gated aggregation module to obtain artifact-aware heatmap tokens. In parallel, multiscale CLIP features are combined through weighted fusion to produce semantic tokens. To model interactions between semantic content and reconstruction artifacts, we employ both cross-attention (CLIP queries heatmap features) and reverse cross-attention (heatmap queries CLIP features). The resulting representations, together with global CLIP and heatmap descriptors, are fused and passed to an MLP for real/fake classification. A projection head attached to the shared representation enables supervised contrastive learning, encouraging stronger separation between genuine and manipulated samples.

Figure 3: Layer-wise PCA decomposition of SIREN reconstruction dynamics. Top: a genuine face from CDF v2 ; bottom: a deepfake. Although both are reconstructed with comparable fidelity (rightmost column), their internal activations diverge sharply across layers

Figure 4: ROC curves. Comparison of the proposed Spectra model and the RGB-only ablation on the Celeb-DF v1, and v2 benchmarks.

Figure 5: t-SNE visualization of feature embeddings on the CDFv1 dataset. The proposed Spectra model produces more discriminative feature representations than the RGB -only baseline, resulting in improved separation between real and fake samples.

General Methods
Method	CDF v1	CDF v2	FSh	DFD	Avg
MesoNet’18 [ 1 ]	0.735	0.609	0.566	0.548	0.615
MesoInc.’18 [ 1 ]	0.736	0.696	0.643	0.607	0.671
Xception’18 [ 12 ]	0.779	0.736	0.624	0.816	0.739
DSP-FWA’18 [ 34 ]	0.789	0.668	0.555	0.740	0.688
EffNet’19 [ 53 ]	0.790	0.748	0.616	0.815	0.742

Table 1: Cross-dataset frame-level AUC. Best and second-best results are shown in bold and underlined , respectively.

Benchmark	Silhouette $\uparrow$		DB $\downarrow$
Benchmark	Spectra	RGB	Spectra	RGB
CDF v1	0.1337	0.0561	2.517	3.939
CDF v2	0.1155	0.0421	2.632	4.620
DFD	0.2306	0.0808	1.690	3.178
FaceShifter	0.1059	0.0330	2.741	5.153
Mean	0.1465	0.0530	2.395	4.222

Table 2: Cluster-separation metrics. Silhouette (

\uparrow

) and Davies–Bouldin (DB,

\downarrow

) scores

Figure 6: INR layer-wise decomposition. Genuine samples produce smooth activations across layers, while attacks yield noisier intermediate responses and less consistent structure.

Method	M	C	I	O	Avg
RP’21 [ 59 ]	87.43	86.76	86.38	92.43	88.25
FG’22 [ 50 ]	91.25	80.58	95.34	89.85	89.25
FDGAL’24 [ 26 ]	93.29	88.10	91.92	90.54	90.96
MRGA’23 [ 63 ]	89.83	88.36	94.02	90.86	90.77
ST’24 [ 32 ]	90.00	89.98	87.81	92.73	90.13
HPDR’24 [ 25 ]	96.02	94.42	92.49	95.26	94.55

Table 3: Cross-dataset PAD performance (% AUC). M, C, I, and O denote the four evaluation protocols. Best and second-best results are highlighted in bold and underlined , respectively.

Deepfake Detection

Deepfake detection identifies manipulated or AI-generated media to support trustworthy communication, identity verification, and forensic analysis.

See all 241 papers

Revisiting Cross-Reconstruction for Generalizable Deepfake Detection
Oct 1, 2026Bingjian Yang, Shilei Zhao, Zheng WangImage Forgery Detection Digital Image Forensics
National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University, China
SPARK-IL: Spectral Retrieval-Augmented RAG for Knowledge-driven Deepfake Detection via Incremental Learning
Apr 4, 2026Hessen Bougueffa Eutamene, Abdellah Zakaria Sellam, Abdelmalik Taleb-Ahmed +1Deepfake Detection AI-Generated Image Detection
Univ. Polytechnique Hauts-de-France, Valenciennes, France · Institute of Applied Sciences and Intelligent Systems – Lecce, Italy · Department of Innovation Engineering, University of Salento, Italy +1
Foundation Models are Implicit Deepfake Detectors
Aug 10, 2026Stefan Smeu, Dragos-Alexandru Boldisor, Elisabeta Oneata +1Vision Foundation Models Deepfake Detection
1Bitdefender, Romania · 2POLITEHNICA Bucharest

Oct 1, 2026J/K move · Enter open · S save

Revisiting Cross-Reconstruction for Generalizable Deepfake Detection

Bingjian Yang, Shilei Zhao, Zheng Wang

National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University, China

Image Forgery Detection Digital Image Forensics Representation Disentanglement Deepfake Detection

Open paper AskPDF arXiv

Existing image forgery detectors often suffer from generalization to unseen manipulation methods due to the limited ability to capture transferable forensic cues. Recent cross-reconstruction based methods attempt to improve generalization through semantic-artifact disentanglement, but typically align heterogeneous artifacts across generators and exclude artifact representations during reconstruction, which may overlook the inherent diversity and visual cues of manipulation artifacts. In this work, we revisit cross-reconstruction and introduce an artifact-oriented disentanglement framework for robust image forgery detection. We argue that artifact diversity, i.e., the intrinsic variations of manipulation artifacts introduced by different generation processes, contains complementary forensic cues rather than undesirable domain variations. Instead of enforcing explicit artifact alignment, our framework preserves diverse artifact characteristics through semantically aligned cross-generator reconstruction. Furthermore, we incorporate artifact representations into the reconstruction process and introduce a masked frequency-aware reconstruction strategy to emphasize manipulation-related residuals while reducing semantic interference. This design enables the model to learn transferable forensic representations from diverse artifacts. Extensive experiments on multiple benchmark datasets demonstrate improvements under both cross-dataset and cross-generator evaluation settings. Further analysis and ablation studies validate the effectiveness of artifact diversity preservation and artifact-aware cross-reconstruction.

Abstract

Deepfake Detection

Explore similar work

Revisiting Cross-Reconstruction for Generalizable Deepfake Detection

SPARK-IL: Spectral Retrieval-Augmented RAG for Knowledge-driven Deepfake Detection via Incremental Learning

Foundation Models are Implicit Deepfake Detectors