Organizations: Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology, Nanjing, 210044, China · College of Computer Science and Technology, Zhejiang University, and KOKONI3D, Moxin (Hangzhou) Technology Company Ltd., Hangzhou 310027, China
Image Manipulation Localization (IML) aims to identify edited regions in an image. However, with the increasing use of modern image editing and generative models, many manipulations no longer exhibit obvious low-level artifacts. Instead, they often involve subtle but meaning-altering edits to an object's attributes, state, or relationships while remaining highly consistent with the surrounding content. This makes conventional IML methods less effective because they mainly rely on artifact detection rather than semantic sensitivity. To address this issue, we introduce Semantic Manipulation Localization (SML), a new task that focuses on localizing subtle semantic edits that significantly change image interpretation. We further construct a dedicated fine-grained benchmark for SML using a semantics-driven manipulation pipeline with pixel-level annotations. Based on this task, we propose TRACE (Targeted Reasoning of Attributed Cognitive Edits), an end-to-end framework that models semantic sensitivity through three progressively coupled components: semantic anchoring, semantic perturbation sensing, and semantic-constrained reasoning. Specifically, TRACE first identifies semantically meaningful regions that support image understanding, then injects perturbation-sensitive frequency cues to capture subtle edits under strong visual consistency, and finally verifies candidate regions through joint reasoning over semantic content and semantic scope. Extensive experiments show that TRACE consistently outperforms existing IML methods on our benchmark and produces more complete, compact, and semantically coherent localization results. These results demonstrate the necessity of moving beyond artifact-based localization and provide a new direction for image forensics in complex semantic editing scenarios.
Figures & tables
Fig. 1: Distinction between image manipulation localization (IML) and semantic manipulation localization (SML). (a) Authentic images; (b) Manipulated images; (c) Local enlarged areas of manipulated images; (d) Ground-truth.
Fig. 2: The pipeline of two-stage automatic semantically decisive region identification and semantic tampering.
Datasets
C
Tampering Type
Size
Total
Selected
SP
CM
RM
CASIA [ 36 ]
M
✓
✓
410×309
6041
734
CMFD [ 37 ]
M
✓
1024×768
80
2
COVERAGE [ 5 ]
M
✓
✓
469×403
100
53
data-tifs-2016-map [ 38 ]
M
✓
480×270
136
9
IMD2020 [ 39 ]
M
✓
✓
1040×876
3781
914
TABLE I: Dataset composition source. ‘S’ means saliency and ‘M’ means manipulation. ‘C’ represents category. ‘SP’ denotes splicing, ‘CM’ denotes copy-move, and ‘RM’ denotes removal.
Fig. 3: Framework of the proposed TRACE. “WT” denotes the wavelet transform. TRACE decomposes localization into three progressively coupled stages: semantic anchoring, semantic perturbation sensing, and semantic-constrained reasoning. Specifically, TRACE first grounds high-level interpretation to semantically meaningful objects and their spatial scope, then injects perturbation-sensitive frequency cues to capture subtle semantic edits, and finally refines the prediction through explicit semantic-scope reasoning.
Fig. 4: Framework of the semantic-scope coherence reasoning (SSCR). Starting from the mask and edge features, SSCR first constructs four interleaved semantic-scope sequences under row-forward, column-forward, row-backward, and column-backward scan orders (cross-scan reasoning). These sequences are then processed by a shared MixerModel composed of stacked Mamba (mixer modeling). Finally, each directional output is split by position back into mask and edge subsequences, and the four directional responses are aggregated separately with learnable channel-wise weights to obtain the refined mask and edge features (split and weighted aggregation).
Method
Full Test Set
SDRI+ST Dataset
IoU
Dice
Precision
Recall
Accuracy
MAE
AUC
IoU
Dice
Precision
Recall
Accuracy
MAE
AUC
MVSS-Net [ 8 ]
0.601
0.673
0.855
0.648
0.934
0.066
0.926
0.622
0.702
0.733
0.743
0.962
0.038
0.913
CFL-Net [ 9 ]
0.459
0.578
0.638
0.616
0.898
0.102
0.895
0.467
0.571
0.548
0.689
0.938
0.062
0.883
IML-ViT [ 15 ]
0.672
0.737
0.935
0.702
0.944
0.056
0.974
0.680
0.761
0.781
0.797
0.969
0.031
0.953
EITLNet [ 16 ]
0.635
0.694
0.913
0.678
0.936
0.064
0.955
0.742
0.802
0.840
0.812
0.978
0.022
0.959
MMFusion-IML [ 14 ]
0.693
0.779
0.855
0.792
0.946
0.054
0.966
0.716
0.799
0.787
0.869
0.968
0.032
0.977
TABLE II: Quantitative comparison on the full test set and on the dataset constructed using the Semantically Decisive Region Identification and Semantic Tampering (SDRI-ST) pipeline. All values are rounded to three decimals. The best results are shown in bold , and the second-best results are underlined. RI denotes the relative improvement of our method over the second-best method, computed as (Ours−Second)/Second for all metrics except MAE, for which we use (Second−Ours)/Second since lower is better.
Fig. 5: Qualitative results of our Trace compared with other SoTAs on our proposed dataset. In addition, We provide negative semantics generated by LLMs to help clarify how cognitive changes occur in images.
TABLE III: Ablation study of the proposed framework. We progressively evaluate the contribution on semantic manipulation localization.
Fig. 7: Visualization results of SPS. (a) Original image; (b) Manipulated image; (c) Ground truth; (d) SRM result of ‘R’ channel; (e) SRM result of ‘G’ channel; (f) SRM result of ‘B’ channel; (g) Wavelet result of ‘R’ channel; (h) Wavelet result of ‘G’ channel; (i) Wavelet result of ‘B’ channel.
Fig. 8: Visualization results of SSCR. (a) (i) Manipulated image; (b) (j) Ground truth; (c) (k) Predicted result; (d) (l) Predicted edge; (e) (m) Visualization feature of row-forward sequence; (f) (n) Visualization feature of row-backward sequence; (g) (o) Visualization feature of column-forward sequence; (h) (p) Visualization feature of column-backward sequence.
Post-processing method
Parameter value
Methods
Ours
SAM2-UNET
CFL-Net
EITLNet
HDFNet
IML-ViT
MMFunsion
MVSSnet
Gaussian blur
kernel=3
0.790
0.532
0.225
0.319
0.433
0.288
0.510
0.132
kernel=9
0.745
0.315
0.251
0.187
0.432
0.215
0.436
0.081
kernel=15
0.666
0.285
0.259
0.160
0.423
0.295
0.408
0.100
JPEG compression
quality=50
0.717
0.657
0.417
0.576
0.457
0.569
0.597
0.462
quality=75
0.735
0.663
0.448
0.612
0.461
0.630
0.630
0.520
TABLE IV: Robustness comparison in terms of IoU under different post-processing perturbations. The best results are shown in bold.
Fig. 9: Visualization results of SSGM. (a) Manipulated image; (b) Ground truth; (c) Correction gate map, which takes the mean of the channels and restores them to the 2D dimension; (d) Overlay of gate image and tampered image; (e) Result.
Fig. 10: Comparison of accuracy–efficiency trade-offs among different methods. The x-axis denotes model parameters (in millions) in log scale, and the y-axis shows IoU on the full test set. Bubble size encodes computational cost (GFLOPs) with square-root scaling to mitigate the large dynamic range across models. Trace achieves the best IoU, highlighting its effectiveness despite increased computational complexity.
As generative image editing advances, image manipulation localization (IML) must handle both traditional manipulations with conspicuous forensic artifacts and diffusion-generated edits that appear locally realistic. Existing methods typically rely on either low-level forensic cues or high-level semantics alone, leading to a fundamental micro--macro gap. To bridge this gap, we propose FASA, a unified framework for localizing both traditional and diffusion-generated manipulations. Specifically, we extract manipulation-sensitive frequency cues through an adaptive dual-band DCT module and learn manipulation-aware semantic priors via patch-level contrastive alignment on frozen CLIP representations. We then inject these priors into a hierarchical frequency pathway through a semantic-frequency side adapter for multi-scale feature interaction, and employ a prototype-guided, frequency-gated mask decoder to integrate semantic consistency with boundary-aware localization for tampered region prediction. Extensive experiments on OpenSDI and multiple traditional manipulation benchmarks demonstrate state-of-the-art localization performance, strong cross-generator and cross-dataset generalization, and robust performance under common image degradations.
Xiaojie Liang, Zhimin Chen, Ziqi Sheng +1
Sun Yat-sen University, School of Computer Science and Engineering, Guangzhou, Guangdong, China
Image Manipulation Localization (IML) is commonly formulated as a fully supervised learning task that estimates the optimal manipulation mask y for a given image x. In this work, we first reveal the latent nature of artifacts and thus reinterpret IML as a latent-variable problem, P(y∣x)=∫P(y∣z)P(z∣x)dz, where z denotes the artifacts. Following this interpretation, we pinpoint the cause for the current IML models' insufficiency as their implicit artifacts modeling strategy, highlighting the necessity of modeling z in an explicit manner. Without direct labels, feature disentanglement is the most appropriate solution for this explicit modeling. Accordingly, we propose a two-stage learning paradigm with the Pairwise Artifacts Learning (PAL) and Standard Localization (SL) phases to estimate P(z∣x) and P(y∣z) via edit relations. To support our edit-relation-based learning, we further curate EditGroup-45K, a source-anchored dataset organized into edit groups for pair construction. Extensive experiments show that our PAL paradigm yields consistent improvements across diverse IML architectures, and empirical analyses further verify that PAL does capture artifacts explicitly through feature disentanglement. Code and dataset are available at https://github.com/venus-guangjian/PAL
Xuekang Zhu, Kaiwen Feng, Ruifeng Wang +10
Sichuan University · Ant Group · The Hong Kong University of Science and Technology +4
Recent advances in generative image editing have improved the realism and controllability of localized image manipulation, raising new challenges for image manipulation detection and localization (IMDL). However, existing IMDL benchmarks still have limitations in visual realism, manipulation diversity, and generator coverage, making it difficult to reflect recent trends in image manipulation. To address these limitations, we introduce Impostor, a high-quality AI-edited image manipulation localization dataset containing 100K manipulated images. Impostor is constructed by CraftAgent, a closed-loop agent framework that integrates scene perception, editing planning, manipulation execution, quality validation, and iterative reflection to automatically generate diverse and visually realistic manipulated images. Moreover, Impostor contains images generated by seven recent AIGC models across three manipulation types and includes multiple manipulated regions, providing a more comprehensive benchmark for AIGC-based IMDL. Furthermore, we propose PhaseAware-Net (PANet), a semantic-forensic framework that introduces local phase modeling and semantic-forensic consistency learning to better localize semantically plausible yet forensically disrupted manipulated regions. Extensive experiments show that Impostor poses significant challenges to existing large vision-language models (LVLMs) and specialized IMDL methods, while PANet achieves superior performance on Impostor and multiple public benchmarks.
Zhenliang Li, Yutao Hu, Qixiong Wang +5
Southeast University · Xiaohongshu Inc · Xiaohongshu Inc. +1