DINO-FIL: Multi-Layer Transformer Fusion for Generative Inpainting Forgery Localization

Simon Freitter; David Fischinger; Silvia Poletti; Marcel Hasenbalg; Martin Boyer

cs.CVOct 15, 2026

DINO-FIL: Multi-Layer Transformer Fusion for Generative Inpainting Forgery Localization

Authors: Simon Freitter, David Fischinger, Silvia Poletti, Marcel Hasenbalg, Martin Boyer

Organizations: AIT Austrian Institute of Technology GmbH, Giefinggasse 4, 1210 Vienna, Austria

Abstract

We propose DINO-FIL, a transformer-based framework for pixel-level localization of generative image forgeries. The method leverages multi-layer representations from a pretrained DINOv3 ViT-H+/16 encoder combined with a lightweight fusion decoder to capture both semantic structure and fine-grained texture inconsistencies introduced by image editing. By aggregating intermediate transformer features across depths, the model improves sensitivity to subtle, spatially localized artifacts that are often missed by standard single-layer approaches. We evaluate DINO-FIL in a cross-dataset setting, training on B-Free and testing on Beyond the Brush, CocoGlide, TGIF, and SAGI-D. The model achieves strong generalization, outperforming existing baselines on average and on four of five evaluated datasets, while maintaining a high true negative rate on pristine images.

Figures & tables

Panel of Figure 1 — Figure 1: Overview of DINO-FIL. The batch of resized images is encoded with DINOv3, and hidden states from layers $8,16,24,$ and $32$ are converted into spatial patch maps $f_{\ell}$ by reshaping and discarding one CLS token and four register tokens. The projection adapter maps $f_{\ell}$ to $p_{\ell}\in\mathbb{R}^{B\times C_{d}\times q\times q}$ . The context block produces $g_{\ell}\in\mathbb{R}^{B\times C_{d}\times q\times q}$ using lightweight multi-scale depthwise-separable convolutions. Before fusion, $g_{\ell}$ is optionally regularized with spatial dropout, yielding $\tilde{g}_{\ell}$ during training and the identity mapping otherwise. The resulting $\tilde{g}_{\ell}$ maps are fused by a horizontal depth-wise top-down decoder on the fixed $q\times q$ patch grid. A $1{\times}1$ prediction head produces patch-level logits, which are upsampled and passed through a sigmoid to obtain the final inpainting probability mask. Here, $q=S/P$ , $P=16$ , $C_{b}=1280$ , and $C_{d}=1024$ .

Test image resolution	Unfrozen layers	BtB		CocoGlide		TGIF		SAGI-D		B-Free		Average
Test image resolution	Unfrozen layers	IoU	F1	IoU	F1	IoU	F1	IoU	F1	IoU	F1	IoU	F1
512px	0	0.23	0.31	0.75	0.82	0.27	0.33	0.47	0.55	0.82	0.89	0.51	0.58
	6	0.33	0.41	0.77	0.85	0.38	0.45	0.52	0.61	0.86	0.92	0.57	0.65
	12	0.36	0.44	0.81	0.87	0.38	0.44	0.58	0.66	0.89	0.94	0.60	0.67
	16	0.36	0.43	0.83	0.88	0.38	0.43	0.59	0.67	0.90	0.94	0.61	0.67
	24	0.32	0.38	0.84	0.89	0.33	0.37	0.57	0.64	0.91	0.95	0.60	0.65

Table 1: Comparison of DINO-FIL variants under different numbers of unfrozen encoder layers and test image resolutions (training resolution is fixed at 512px), evaluated using average pixel-level IoU and F1 score on image forgery localization. The best results are shown in bold, and the second-best results are underlined.

Model	BtB		CocoGlide		TGIF		SAGI-D		B-Free		Average
Model	IoU	F1	IoU	F1	IoU	F1	IoU	F1	IoU	F1	IoU	F1
DINO-FIL (ours)	0.59	0.68	0.87	0.92	0.57	0.63	0.66	0.73	0.87	0.92	0.71	0.77
DinoLizer	0.42	0.56	0.57	0.69	0.39	0.49	0.38	0.49	0.70	0.81	0.49	0.61
TruFor	0.22	0.30	0.29	0.36	0.70	0.80	0.35	0.41	0.22	0.30	0.36	0.43

Table 2: Comparison of DINO-FIL (ours) with DinoLizer and TruFor, the two strongest-performing methods reported in the DinoLizer benchmark [ 2 ] . Performance is reported using average pixel-level IoU and F1 score for image forgery localization. The SAGI-D results for DinoLizer and TruFor are computed as image-count-weighted averages of the reported SAGI-SP and SAGI-FR results. The best results are shown in bold, and the second-best results are underlined.

Dataset	Number of Images	Pixel-level FP Rate	Image-level TN Rate
COCO	2,922	0.0019	0.9767
Open Images	5,579	0.0040	0.9684
RAISE	580	0.0021	0.9810
Average	9,081	0.0032	0.9719

Table 3: False-positive evaluation of DINO-FIL on original, non-manipulated images. We report the pixel-level false-positive rate and the image-level true-negative rate. The average is computed over all images across the three datasets.

Train Resolution	Test Resolution	Beyond the Brush		TGIF		SAGI-D		Average
Train Resolution	Test Resolution	IoU	F1	IoU	F1	IoU	F1	IoU	F1
512px	512px	0.37	0.44	0.41	0.44	0.76	0.81	0.51	0.56
512px	1024px	0.68	0.76	0.62	0.69	0.83	0.88	0.71	0.78
1024px	1024px	0.64	0.71	0.55	0.62	0.84	0.88	0.68	0.74

Table 4: Comparison of DINO-FIL variants trained at different resolutions (512px and 1024px) and evaluated at corresponding test resolutions for image forgery localization. Performance is reported in terms of average pixel-level IoU and F1 score across datasets. The best results are shown in bold, and the second-best results are underlined.

Image Forgery Localization

Image forgery localization identifies manipulated or AI-generated regions in images, often alongside image-level authenticity detection.

See all 79 papers

Order within Chaos: Capturing Intrinsic Energy Anomalies for AI-Manipulated Image Forgery Localization
Jun 1, 2026Yiming Wang, Baiqi Wu, Qingming Li +3Image Forgery Detection Digital Image Forensics
Zhejiang University, Hangzhou, China.
DiffusionPrint: Learning Generative Fingerprints for Diffusion-Based Inpainting Localization
Apr 14, 2026Paschalis Giakoumoglou, Symeon PapadopoulosImage Forgery Localization Digital Image Forensics
Information Technologies Institute, CERTH
FUSED: Forensic-Semantic Mixture-of-Experts for AI Inpainting Detection and Localization
Aug 28, 2026Anton Nuzhdin, Marcel Worring, Ivona NajdenkoskaAI-Generated Image Detection Image Forgery Detection
Informatics Institute, University of Amsterdam Amsterdam, The Netherlands

Jun 1, 2026J/K move · Enter open · S save

Order within Chaos: Capturing Intrinsic Energy Anomalies for AI-Manipulated Image Forgery Localization

Yiming Wang, Baiqi Wu, Qingming Li, Jiahao Chen+2

Zhejiang University, Hangzhou, China.

Image Forgery Detection Digital Image Forensics Image Forgery Localization Anomaly Detection

Open paper AskPDF arXiv

Recent advancements in generative AI have led to image editing models capable of producing realistic forgeries that evade traditional image forgery localization methods, as these approaches depend on physical noise absent in synthetic data. To address this challenge, we theoretically demonstrate that the diffusion process inherently suppresses local high-frequency variance, creating a statistical energy gap that is distinguishable from the natural entropy of optical imaging. Guided by this insight, we propose FLAME, a unified framework that utilizes a LAD map to capture these intrinsic anomalies, coupled with a parameter-efficient adapter for SAM to achieve precise, pixel-level forgery localization. Furthermore, to bridge the lag between forensic benchmarks and evolving generative models, we introduce EditStream, an automated pipeline for continuous, instruction-based training data synthesis. Extensive experiments demonstrate that FLAME establishes a new state-of-the-art, significantly outperforming previous methods on AI-generated forgery datasets while effectively generalizing to unseen generative architectures. Our code is available at https://github.com/phoenixnir/FLAME.

Abstract

Image Forgery Localization

Explore similar work

Order within Chaos: Capturing Intrinsic Energy Anomalies for AI-Manipulated Image Forgery Localization

DiffusionPrint: Learning Generative Fingerprints for Diffusion-Based Inpainting Localization

FUSED: Forensic-Semantic Mixture-of-Experts for AI Inpainting Detection and Localization