In recent years, advanced image editing and generation methods have rapidly evolved, making detecting and locating forged image content increasingly challenging. Most existing image forgery detection methods rely on identifying the edited traces left in the image. However, because the traces of different forgeries are distinct, these methods can identify familiar forgeries included in the training data but struggle to handle unseen ones. In response, we present an approach for Generalizable Image Forgery Localization (GIFL). Once trained, our model can detect both seen and unseen forgeries, providing a more practical and efficient solution to counter false information in the era of generative AI. Our method focuses on learning general features from the pristine content rather than traces of specific forgeries, which are relatively consistent across different types of forgeries and therefore can be used as universal features to locate unseen forgeries. Additionally, as existing image forgery datasets are still dominated by traditional hand-crafted forgeries, we construct a new dataset consisting of images edited by various popular deep generative image editing methods to further encourage research in detecting images manipulated by deep generative models. Extensive experimental results show that the proposed approach outperforms state-of-the-art methods in the detection of unseen forgeries and also demonstrates competitive results for seen forgeries. The code and dataset are available at https://github.com/ZhaoHengrun/GIFL.
Figures & tables
Fig. 1: Illustration of the traditional classification-based image forgery detection pipeline (left) and our proposed GIFL method (right).
Fig. 3: Forged images generated by different forgery methods in Forgery ADE dataset.
CASIA 1.0
CASIA 2.0
COVERAGE
IMD2020
tampCOCO
CocoGlide
GRE
Forgery ADE Train
Forgery ADE Test
Mean
0.0871
0.0896
0.1117
0.0129
0.0966
0.2522
0.0570
0.1988
0.1292
Variance
0.0064
0.0190
0.0036
0.0000
0.0133
0.0545
0.0747
0.0358
0.0182
Max
0.6953
0.9630
0.2787
0.0329
0.9726
0.9977
0.7601
0.9915
0.9410
Min
0.0014
0.0002
0.0027
0.0100
0.0000
0.0197
0.0005
0.0001
0.0003
TABLE I: Forgery region statistics for Forgery ADE and several existing datasets.
Module
Params (M)
FLOP (G)
Memory (MB)
Encoder
304.4
723.7
12318.3
Predictor
718.0
1676.4
Total
1022.4
2400.1
TABLE II: Computational Complexity Analysis.
Prior
Forgery
Metric
ManTra-Net
MVSS-Net
PSCC-Net
CAT-Net
IID-Net
IF-OSN
IML-ViT
TruFor
FOCAL
EITLNet
GIFL
Trained
Deepfill v2
F1
0.8855
0.9117
0.8765
0.8532
0.8583
0.9069
0.9035
0.8779
0.8239
0.8956
0.8539
IOU
0.7071
0.7719
0.7009
0.6523
0.6633
0.7656
0.7166
0.7057
0.6086
0.7362
0.6446
ACC
0.9805
0.9881
0.9815
0.9665
0.9795
0.9888
0.9865
0.9852
0.9519
0.9847
0.9756
AUC
0.8905
0.9133
0.8718
0.8724
0.8565
0.9010
0.9082
0.8611
0.8251
0.9030
0.8647
LDM
F1
0.5623
0.7063
0.6382
0.5019
0.5951
0.6480
0.5193
0.6069
0.5238
0.7218
0.7078
IOU
0.1262
0.3703
0.3108
0.0547
0.1979
0.2768
0.0795
0.2106
0.0821
0.4015
0.3655
TABLE III: Quantitative comparison. Bold for best results.
Fig. 4: Visual comparison of our results and those of previous methods. Red boxes indicate the results on seen forgeries and blue for unseen ones. Zoom-in to see the details.
Prior
Forgery
Metric
Baseline
I
II
III
IV
V
VI
VII
VIII
IX
Trained
Deepfill v2
F1
0.8932
0.9033
0.9007
0.8683
0.9011
0.8920
0.8421
0.8680
0.8754
0.8736
IOU
0.7195
0.7495
0.7364
0.6594
0.7341
0.7133
0.6186
0.6621
0.6759
0.6810
ACC
0.9650
0.9719
0.9681
0.9552
0.9692
0.9652
0.9289
0.9589
0.9569
0.9545
AUC
0.9093
0.9061
0.9039
0.8810
0.9054
0.8956
0.8979
0.8704
0.8947
0.9058
LDM
F1
0.7164
0.7391
0.7333
0.6287
0.7087
0.6823
0.6828
0.6631
0.6694
0.7100
IOU
0.3936
0.4203
0.4181
0.2535
0.3829
0.3348
0.3623
0.3018
0.3262
0.3963
TABLE IV: Ablation study on different configurations of GIFL.
Fig. 5: Visualization of target features and reconstruction features of different types of forged images. Zoom-in to see the details.
Prior
Metric
DINOv2-L
DINOv2-B
MAE-B
ViT-B
Trained
F1
0.7815
0.6982
0.6490
0.6307
IOU
0.5037
0.3596
0.3115
0.2486
ACC
0.9498
0.9216
0.9332
0.8956
AUC
0.7953
0.7258
0.6843
0.6566
Unseen
F1
0.6703
0.6326
0.6290
0.6076
IOU
0.3212
0.2565
0.2531
0.2165
TABLE V: The impact of different pre-trained encoders.
Forgery
Metric
Baseline
Data Diversity
False Positives
Masking
I
II
III
IV
V
VI
VII
VIII
IX
X
XI
Deepfill v2
F1
0.8932
0.9051
0.7320
0.8874
0.8859
0.8810
0.8968
0.9035
0.8982
0.8641
0.8722
0.9029
IOU
0.7195
0.7433
0.4243
0.7075
0.7025
0.6909
0.7285
0.7419
0.7271
0.6647
0.6758
0.7415
ACC
0.9650
0.9695
0.9007
0.9617
0.9620
0.9608
0.9678
0.9698
0.9677
0.9595
0.9510
0.9688
AUC
0.9093
0.9161
0.7564
0.9055
0.9069
0.8979
0.9117
0.9122
0.9065
0.8576
0.8985
0.9124
CTSDG
F1
0.8032
0.7500
0.7216
0.8731
0.8629
0.8621
0.7940
0.8079
0.7864
0.8061
0.8303
0.8720
TABLE VI: Performance comparison under different dataset settings, bold for trained forgeries.
Training set
Semantic
Test set
Semantic
F1
IOU
ACC
AUC
GRE
✓
CocoGlide
✓
0.4570
0.0493
0.7488
0.5194
COVERAGE
0.4748
0.0103
0.8731
0.4962
IMD2020
0.5038
0.0646
0.9472
0.5319
Forgery ADE
0.6909
0.3406
0.8961
0.6870
ADE
CocoGlide
✓
0.4785
0.0800
0.7419
0.5276
COVERAGE
0.4971
0.0383
0.8665
0.5103
TABLE VII: The impact of dataset semantic correlation on performance.
Forgery
Metric
Baseline
Compression
Resizing
Sharpen
Blur
Motion Blur
Gamma
Noise
Deepfill v2
F1
0.8932
0.8552
0.8790
0.8658
0.8244
0.8821
0.8904
0.8659
IOU
0.7195
0.6506
0.6932
0.6675
0.5937
0.6999
0.7120
0.6655
ACC
0.9650
0.9526
0.9608
0.9544
0.9429
0.9606
0.9638
0.9569
AUC
0.9093
0.8712
0.8977
0.8859
0.8442
0.8952
0.9064
0.8853
CTSDG
F1
0.8032
0.7963
0.8041
0.7627
0.7918
0.8023
0.7889
0.8022
IOU
0.5464
0.5336
0.5427
0.4720
0.5351
0.5415
0.5215
0.5397
TABLE VIII: Detection results under various image degradation.
Methods
w/o Authentic Image
w/ Authentic Image
p-ACC
i-ACC
p-ACC
i-ACC
ManTra-Net
0.9334
0.0800
0.9978
0.6000
MVSS-Net
0.9885
0.6400
1.0000
1.0000
PSCC-Net
0.9363
0.3200
0.9892
0.9800
CAT-Net
0.9897
0.2200
0.9928
0.4800
IID-Net
0.9936
0.8000
1.0000
1.0000
TABLE IX: The performance of each method on authentic images after training on datasets with and without negative samples.
Fig. 6: GIFL predictions on authentic Forgery ADE images with different proportions of negative sample training. Zoom-in to see the details.
Fig. 7: Typical failure cases. Zoom-in to see the details.