Organizations: Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology, Nanjing, 210044, China · College of Computer Science and Technology, Zhejiang University, and KOKONI3D, Moxin (Hangzhou) Technology Company Ltd., Hangzhou 310027, China
Image Manipulation Localization (IML) aims to identify edited regions in an image. However, with the increasing use of modern image editing and generative models, many manipulations no longer exhibit obvious low-level artifacts. Instead, they often involve subtle but meaning-altering edits to an object's attributes, state, or relationships while remaining highly consistent with the surrounding content. This makes conventional IML methods less effective because they mainly rely on artifact detection rather than semantic sensitivity. To address this issue, we introduce Semantic Manipulation Localization (SML), a new task that focuses on localizing subtle semantic edits that significantly change image interpretation. We further construct a dedicated fine-grained benchmark for SML using a semantics-driven manipulation pipeline with pixel-level annotations. Based on this task, we propose TRACE (Targeted Reasoning of Attributed Cognitive Edits), an end-to-end framework that models semantic sensitivity through three progressively coupled components: semantic anchoring, semantic perturbation sensing, and semantic-constrained reasoning. Specifically, TRACE first identifies semantically meaningful regions that support image understanding, then injects perturbation-sensitive frequency cues to capture subtle edits under strong visual consistency, and finally verifies candidate regions through joint reasoning over semantic content and semantic scope. Extensive experiments show that TRACE consistently outperforms existing IML methods on our benchmark and produces more complete, compact, and semantically coherent localization results. These results demonstrate the necessity of moving beyond artifact-based localization and provide a new direction for image forensics in complex semantic editing scenarios.
Figures & tables
Fig. 1: Distinction between image manipulation localization (IML) and semantic manipulation localization (SML). (a) Authentic images; (b) Manipulated images; (c) Local enlarged areas of manipulated images; (d) Ground-truth.
Fig. 2: The pipeline of two-stage automatic semantically decisive region identification and semantic tampering.
Datasets
C
Tampering Type
Size
Total
Selected
SP
CM
RM
CASIA [ 36 ]
M
✓
✓
410×309
6041
734
CMFD [ 37 ]
M
✓
1024×768
80
2
COVERAGE [ 5 ]
M
✓
✓
469×403
100
53
data-tifs-2016-map [ 38 ]
M
✓
480×270
136
9
IMD2020 [ 39 ]
M
✓
✓
1040×876
3781
914
TABLE I: Dataset composition source. ‘S’ means saliency and ‘M’ means manipulation. ‘C’ represents category. ‘SP’ denotes splicing, ‘CM’ denotes copy-move, and ‘RM’ denotes removal.
Fig. 3: Framework of the proposed TRACE. “WT” denotes the wavelet transform. TRACE decomposes localization into three progressively coupled stages: semantic anchoring, semantic perturbation sensing, and semantic-constrained reasoning. Specifically, TRACE first grounds high-level interpretation to semantically meaningful objects and their spatial scope, then injects perturbation-sensitive frequency cues to capture subtle semantic edits, and finally refines the prediction through explicit semantic-scope reasoning.
Fig. 4: Framework of the semantic-scope coherence reasoning (SSCR). Starting from the mask and edge features, SSCR first constructs four interleaved semantic-scope sequences under row-forward, column-forward, row-backward, and column-backward scan orders (cross-scan reasoning). These sequences are then processed by a shared MixerModel composed of stacked Mamba (mixer modeling). Finally, each directional output is split by position back into mask and edge subsequences, and the four directional responses are aggregated separately with learnable channel-wise weights to obtain the refined mask and edge features (split and weighted aggregation).
Method
Full Test Set
SDRI+ST Dataset
IoU
Dice
Precision
Recall
Accuracy
MAE
AUC
IoU
Dice
Precision
Recall
Accuracy
MAE
AUC
MVSS-Net [ 8 ]
0.601
0.673
0.855
0.648
0.934
0.066
0.926
0.622
0.702
0.733
0.743
0.962
0.038
0.913
CFL-Net [ 9 ]
0.459
0.578
0.638
0.616
0.898
0.102
0.895
0.467
0.571
0.548
0.689
0.938
0.062
0.883
IML-ViT [ 15 ]
0.672
0.737
0.935
0.702
0.944
0.056
0.974
0.680
0.761
0.781
0.797
0.969
0.031
0.953
EITLNet [ 16 ]
0.635
0.694
0.913
0.678
0.936
0.064
0.955
0.742
0.802
0.840
0.812
0.978
0.022
0.959
MMFusion-IML [ 14 ]
0.693
0.779
0.855
0.792
0.946
0.054
0.966
0.716
0.799
0.787
0.869
0.968
0.032
0.977
TABLE II: Quantitative comparison on the full test set and on the dataset constructed using the Semantically Decisive Region Identification and Semantic Tampering (SDRI-ST) pipeline. All values are rounded to three decimals. The best results are shown in bold , and the second-best results are underlined. RI denotes the relative improvement of our method over the second-best method, computed as (Ours−Second)/Second for all metrics except MAE, for which we use (Second−Ours)/Second since lower is better.
Fig. 5: Qualitative results of our Trace compared with other SoTAs on our proposed dataset. In addition, We provide negative semantics generated by LLMs to help clarify how cognitive changes occur in images.
TABLE III: Ablation study of the proposed framework. We progressively evaluate the contribution on semantic manipulation localization.
Fig. 7: Visualization results of SPS. (a) Original image; (b) Manipulated image; (c) Ground truth; (d) SRM result of ‘R’ channel; (e) SRM result of ‘G’ channel; (f) SRM result of ‘B’ channel; (g) Wavelet result of ‘R’ channel; (h) Wavelet result of ‘G’ channel; (i) Wavelet result of ‘B’ channel.
Fig. 8: Visualization results of SSCR. (a) (i) Manipulated image; (b) (j) Ground truth; (c) (k) Predicted result; (d) (l) Predicted edge; (e) (m) Visualization feature of row-forward sequence; (f) (n) Visualization feature of row-backward sequence; (g) (o) Visualization feature of column-forward sequence; (h) (p) Visualization feature of column-backward sequence.
Post-processing method
Parameter value
Methods
Ours
SAM2-UNET
CFL-Net
EITLNet
HDFNet
IML-ViT
MMFunsion
MVSSnet
Gaussian blur
kernel=3
0.790
0.532
0.225
0.319
0.433
0.288
0.510
0.132
kernel=9
0.745
0.315
0.251
0.187
0.432
0.215
0.436
0.081
kernel=15
0.666
0.285
0.259
0.160
0.423
0.295
0.408
0.100
JPEG compression
quality=50
0.717
0.657
0.417
0.576
0.457
0.569
0.597
0.462
quality=75
0.735
0.663
0.448
0.612
0.461
0.630
0.630
0.520
TABLE IV: Robustness comparison in terms of IoU under different post-processing perturbations. The best results are shown in bold.
Fig. 9: Visualization results of SSGM. (a) Manipulated image; (b) Ground truth; (c) Correction gate map, which takes the mean of the channels and restores them to the 2D dimension; (d) Overlay of gate image and tampered image; (e) Result.
Fig. 10: Comparison of accuracy–efficiency trade-offs among different methods. The x-axis denotes model parameters (in millions) in log scale, and the y-axis shows IoU on the full test set. Bubble size encodes computational cost (GFLOPs) with square-root scaling to mitigate the large dynamic range across models. Trace achieves the best IoU, highlighting its effectiveness despite increased computational complexity.