DiffusionPrint: Learning Generative Fingerprints for Diffusion-Based Inpainting Localization

Paschalis Giakoumoglou; Symeon Papadopoulos

cs.CVApr 14, 2026

DiffusionPrint: Learning Generative Fingerprints for Diffusion-Based Inpainting Localization

Authors: Paschalis Giakoumoglou, Symeon Papadopoulos

Organizations: Information Technologies Institute, CERTH

Abstract

Modern diffusion-based inpainting models pose significant challenges for image forgery localization (IFL), as their full regeneration pipelines reconstruct the entire image via a latent decoder, disrupting the camera-level noise patterns that existing forensic methods rely on. We propose DiffusionPrint, a patch-level contrastive learning framework that learns a forensic signal robust to the spectral distortions introduced by latent decoding. It exploits the fact that inpainted regions generated by the same model share a consistent generative fingerprint, using this as a self-supervisory signal. DiffusionPrint trains a convolutional backbone via a MoCo-style objective with cross-category hard negative mining and a generator-aware classification head, producing a forensic feature map that serves as a highly discriminative secondary modality in fusion-based IFL frameworks. Integrated into TruFor, MMFusion, and a lightweight fusion baseline, DiffusionPrint consistently improves localization across multiple generative models, with gains of up to +28% on mask types unseen during fine-tuning and confirmed generalization to unseen generative architectures. Code is available at https://github.com/mever-team/diffusionprint

Figures & tables

Figure 1: Positive pair construction for the two image categories in our contrastive framework. An image generated by a generator $g_{i}$ is inpainted using the corresponding inpainting variant of $g_{i}$ . Natural images are also inpainted. For generated images, we extract the inpainted patches and pair them as positives with patches from the exact same spatial position in the source image. For natural images, positive pairs are formed by sampling patches outside the inpainting mask in both the source and the inpainted image.

Figure 2: Overview of DiffusionPrint. An anchor patch is encoded by $f_{\theta}$ and its positive by the momentum encoder $f_{\bar{\theta}}$ , updated via exponential moving average. Negatives are drawn from a cross-category queue, where each anchor receives the top- $k$ most similar embeddings from the opposite category. The InfoNCE loss and a generator-aware classification head are jointly optimized.

Panel of Figure 3 — Figure 3: Radial power spectra in the mid-to-high frequency band $[0.2,0.5]$ for Flux (left), SDXL (center), and SD 2.1 (right). Four patch types are shown: original real ( real_src ), reconstructed real ( real_regen ), original generated ( gen_src ), and inpainted generated ( gen_inpainted ) patches. Across all three models, latent reconstruction attenuates high-frequency content in real patches, shifting their spectral profile toward that of generated content.

	Configuration	All (%)	IID (%)
Pairing	MoCo	75.21	78.33
Pairing	Ours	86.35	87.69
Channels $C$	1 (flatten)	80.36	81.10
	64 (GAP)	86.35	87.69
	256 (GAP)	85.00	85.41
Top- $k$	64	85.83	86.45

Table 1: Ablation studies on DiffusionPrint pretraining. All: overall linear probing accuracy. IID: accuracy on in-domain models (SD 2.1, SDXL, Flux). Bold denotes the selected configuration.

		SD 2.1		SDXL		Flux schnell		Flux dev		Flux fill
Framework	Modality	sem	rand	sem	rand	sem	rand	sem	rand	sem	rand
Lite Baseline	NP++	52%	23%	60%	61%	75%	72%	65%	62%	64%	59%
	DP	54% +2	25% +2	64% +4	68% +7	85% +10	77% +5	80% +15	73% +11	73% +9	73% +14
	DP †	52% +0	25% +2	65% +5	70% +9	88% +13	76% +4	81% +16	71% +9	76% +12	77% +18
TruFor [ 23 ]	NP++	50%	23%	61%	62%	80%	62%	69%	46%	65%	59%
	DP	53% +3	25% +2	67% +6	73% +11	89% +9	85% +23	83% +14	74% +28	75% +10	74% +15

Table 2: F1 score (%) on TGIF fully regenerated (FR) subset at threshold 0.5 for semantic (sem) and random (rand) mask subsets. NP++: Noiseprint++ modality. DP: DiffusionPrint modality trained on SD 2.1, SDXL, and Flux. DP † : DiffusionPrint contrastive pretraining on SD 2.1 and SDXL only; Flux variants are unseen during pretraining. Lite Baseline: simplified fusion baseline using Segformer for RGB and a CNN for the secondary signal with concatenation. All fusion frameworks are fine-tuned on semantic FR images only; random mask results reflect generalization to unseen mask types. +

\Delta

indicates improvement over NP++ baseline.

Panel of Figure 4 — Figure 4: Qualitative forgery localization results. Comparison between traditional noise-based extractors (NP++) and our proposed DiffusionPrint (DP) integrated into TruFor, MMFusion, and a Lite Baseline (Concat). DiffusionPrint consistently produces cleaner, more accurate localization maps across all frameworks.

Panel of Figure 5 — Figure 5: Failure cases. Top: an SD 2.1 inpainting correctly localized, but with an activation on an authentic object too. Bottom: a case exhibiting object bias, where the model activates on salient non-inpainted regions rather than the manipulated area. Input images show the inpainted region outlined by the ground-truth mask.

Augmentation Strategy	Overall Accuracy (%)
None	84.90
Geometric	86.35
Geometric + JPEG	83.40
Geometric + HP Filter	64.10

Table 3: Ablation on pretraining augmentations. All metrics denote overall linear probing accuracy. While geometric augmentations improve representation quality, applying high-pass filters or JPEG compression severely degrades the learned forensic signal.

Image Forgery Localization

Image forgery localization identifies manipulated or AI-generated regions in images, often alongside image-level authenticity detection.

See all 79 papers

FUSED: Forensic-Semantic Mixture-of-Experts for AI Inpainting Detection and Localization
Aug 28, 2026Anton Nuzhdin, Marcel Worring, Ivona NajdenkoskaAI-Generated Image Detection Image Forgery Detection
Informatics Institute, University of Amsterdam Amsterdam, The Netherlands
Progressive Learning of a Diffusion-based Inpainting Model for Separating Overlapped Fingerprints
Aug 4, 2026Noor Hussein, Anil K. Jain, Karthik NandakumarBiometric Identification Image Inpainting
Michigan State University, East Lansing, MI 48824, USA
DINO-FIL: Multi-Layer Transformer Fusion for Generative Inpainting Forgery Localization
Oct 15, 2026Simon Freitter, David Fischinger, Silvia Poletti +2Image Forgery Localization Feature Fusion
AIT Austrian Institute of Technology GmbH, Giefinggasse 4, 1210 Vienna, Austria

Aug 28, 2026J/K move · Enter open · S save

FUSED: Forensic-Semantic Mixture-of-Experts for AI Inpainting Detection and Localization

Anton Nuzhdin, Marcel Worring, Ivona Najdenkoska

Informatics Institute, University of Amsterdam Amsterdam, The Netherlands

AI-Generated Image Detection Image Forgery Detection Image Forgery Localization

Open paper AskPDF arXiv

Diffusion-based inpainting models modify only a localized part of an image, while many AI-image detectors rely on global artifacts and do not localize. These artifacts vary across generators, limiting detector transfer under distribution shifts. Recent work shows that restoring the authentic pixels outside the inpainted region removes these cues and can degrade pretrained detectors. To address this, we present FUSED, a unified framework for the joint detection and localization of AI-generated inpainting. FUSED combines low-level forensic cues with high-level semantic features using a sparsely-gated Mixture-of-Experts architecture, enabling the model to adaptively prioritize the most relevant signal for each token. For each input, FUSED predicts both an image-level manipulation score and a pixel-level mask of the inpainted area. On the OpenSDID cross-generator benchmark, FUSED achieves the best average detection and localization, with the largest gains on unseen generators. The same model transfers directly to the held-out AutoSplice and CocoGlide benchmarks, more than doubling localization performance. Evaluating each held-out benchmark with and without the global generator artifact further shows that all evaluated methods, ours included, partly read the artifact as evidence of manipulation, and FUSED remains the strongest under both conditions. Code and pretrained models are available at https://github.com/AntonNuzhdin/FUSED.

Abstract

Image Forgery Localization

Explore similar work

FUSED: Forensic-Semantic Mixture-of-Experts for AI Inpainting Detection and Localization

Progressive Learning of a Diffusion-based Inpainting Model for Separating Overlapped Fingerprints

DINO-FIL: Multi-Layer Transformer Fusion for Generative Inpainting Forgery Localization