Frozen DINO Localizes Image Edits Without a Localizer
Authors: Zane Kumar, Vishal Jain, Bernhard Kainz
Organizations: Department of Computing St Paul’s School London London, United Kingdom · Department of Computing Imperial College London London, United Kingdom · Computing / AIBE Imperial College London / FAU London, UK / Erlangen, Germany
Localized image edits can change a photograph's meaning while leaving most of it authentic, so forensic analysis must identify where an edit occurred. We show that patch-level perturbation responses from frozen DINO encoders are themselves localization maps. Training-free Localization of AI-image Edits from patch-token Drift (TRAIL) applies one global Haar perturbation and maps cosine drift between corresponding patch tokens. On 80 source-disjoint CocoGlide test images, TRAIL reaches .903 patch AUROC versus .912 for the mask-supervised Detective SAM; fixed-threshold Dice is .619 versus .709, while an oracle threshold raises TRAIL to .790. Transferred unchanged to Poisson image interpolation, TRAIL reaches .855 AUROC versus .864, showing that the cue persists without a generator. Across sixteen DINO encoders, the best block lies at normalized depth .80-.94. Global context matters: AUROC falls from .903 globally to .857 for local-in-canvas perturbations and .735 for independently encoded crops. Frozen DINO patch tokens therefore contain a strong late-layer localization signal whose visibility depends on the perturbation and preserved context. Code: https://github.com/VishalJ99/trail-image-edit-localization.
Figures & tables
Fig. 1: Three illustrative CocoGlide examples; aggregate performance is reported in Table I . Columns show the edited input, the globally Haar-attenuated input, the TRAIL patch-drift map, the binary Detective SAM output at its released threshold, and the ground-truth mask. TRAIL maps are normalized independently for display; brighter colors indicate greater relative drift. Dice labels use each method’s fixed operating point. Ground truth is used only for evaluation.
Patch AUROC
Fixed-operating-point Dice
Per-image oracle Dice
TRAIL
Dataset / regime
N
TRAIL
Det. SAM
Δ
TRAIL
Det. SAM
Δ
TRAIL
Det. SAM
Δ
F–R
CocoGlide
80
.903
.912
−.009†
.619
.709
−.090
.790
—
—
+.206
TGIF2 semantic
400
.811
.904
−.092
.212
.357
−.145
.363
.540
−.178
+.068
TGIF2 random
320
.727
.758
−.031†
.256
.281
−.026†
.405
.408
−.003†
+.164
FPI
80
.850
.894
−.044†
.482
.694
−.211
.697
.829
−.131
+.151
PII
79
.855
.864
−.009†
.558
.613
−.056
.769
.800
−.032
+.157
TABLE I: Frozen TRAIL and mask-supervised Detective SAM are compared at matched 28×28 resolution across generative and classical edits; higher values are better.
Fig. 2: Depth and scale across sixteen DINO-family encoders. (a) Development AUROC relative to each model’s own maximum; family-specific marker lanes show all selected blocks, which lie at normalized depth .80–.94. (b) Edited-image patch-AUROC at those blocks. (c) Paired edited-minus-aligned-real patch-AUROC, not a real-versus-fake classifier. Panels (b,c) show 95% source-bootstrap intervals using identical source resamples for both metrics of each encoder. From DINOv3-B to 7B, the point-estimate ranking difference is +.018 and the increment difference is +.066.
Perturbation
Application
CocoGlide
TGIF2 dev.
Haar attenuation
Global
.903
.809
Local canvas
.857
.735
Independent crops
.735
.453
Gaussian noise
Global
.798
.481
Local canvas
.738
.418
Independent crops
.527
.226
TABLE II: The prespecified perturbation/application sweep is evaluated by patch AUROC on CocoGlide and TGIF2.
Forensic analysis of AI-edited images requires more than binary real-versus-fake prediction: a useful system should localize the edit, identify its semantic type, and ground its decisions in visual evidence. Existing image-forensics datasets typically emphasize detection or localization, while reasoning-supervised vision-language datasets rarely target image manipulation and often rely on LLM-generated rationales whose faithfulness is difficult to verify. We introduce EditSleuth, a dataset of 257,725 image-edit triplets constructed from existing image-editing corpora for grounded image-edit forensic reasoning. Each example includes an edited image, its source image, a binary edit mask, a 12-class edit taxonomy label, a difficulty score, and a six-step reasoning chain. EditSleuth chains are generated deterministically from triplet-grounded upstream artifacts, with each statement tied to a specific computable source of evidence. Our analysis reveals that a naive four-component difficulty formulation suffers from a rank-2 correlation collapse among magnitude features; a simplified three-component formulation substantially increases score dispersion on both Pico-Banana and MagicBrush. Difficulty also varies meaningfully within most edit categories, indicating that the score is not a proxy for edit type. As an initial learning study, we fine-tune Qwen2-VL-2B with LoRA and find that chain-as-target supervision matches a label-only baseline on classification accuracy among parseable answers, while additionally yielding grounded explanatory prose that label-only supervision cannot produce. We release the dataset, the deterministic construction pipeline, and pilot training scripts.
We propose DINO-FIL, a transformer-based framework for pixel-level localization of generative image forgeries. The method leverages multi-layer representations from a pretrained DINOv3 ViT-H+/16 encoder combined with a lightweight fusion decoder to capture both semantic structure and fine-grained texture inconsistencies introduced by image editing. By aggregating intermediate transformer features across depths, the model improves sensitivity to subtle, spatially localized artifacts that are often missed by standard single-layer approaches. We evaluate DINO-FIL in a cross-dataset setting, training on B-Free and testing on Beyond the Brush, CocoGlide, TGIF, and SAGI-D. The model achieves strong generalization, outperforming existing baselines on average and on four of five evaluated datasets, while maintaining a high true negative rate on pristine images.
Simon Freitter, David Fischinger, Silvia Poletti +2
AIT Austrian Institute of Technology GmbH, Giefinggasse 4, 1210 Vienna, Austria
With the rapid advancement of deep generative models, realistic fake images have become increasingly accessible, yet existing localization methods rely on complex designs and still struggle to generalize across manipulation types and imaging conditions. We present a simple but strong baseline based on DINOv3 with LoRA adaptation and a lightweight convolutional decoder. Under the CAT-Net protocol, our best model improves average pixel-level F1 by 17.0 points over the previous state of the art on four standard benchmarks using only 9.1,M trainable parameters on top of a frozen ViT-L backbone, and even our smallest variant surpasses all prior specialized methods. LoRA consistently outperforms full fine-tuning across all backbone scales. Under the data-scarce MVSS-Net protocol, LoRA reaches an average F1 of 0.774 versus 0.530 for the strongest prior method, while full fine-tuning becomes highly unstable, suggesting that pre-trained representations encode forensic information that is better preserved than overwritten. The baseline also exhibits strong robustness to Gaussian noise, JPEG re-compression, and Gaussian blur. We hope this work can serve as a reliable baseline for the research community and a practical starting point for future image-forensic applications. Code is available at https://github.com/Irennnne/DINOv3-IML.
Jieming Yu, Qiuxiao Feng, Zhuohan Wang +1
The Hong Kong University of Science and Technology · Harvard University