Can We Model the Artifacts Explicitly? Disentangle Artifacts via Pairwise Edit Relations for Image Manipulation Localization

Xuekang Zhu; Kaiwen Feng; Ruifeng Wang; Xiwen Wang; Xiaochen Ma; Bo Du; Changjiang Jiang; Chenfan Qu

cs.CVOct 6, 2026

Can We Model the Artifacts Explicitly? Disentangle Artifacts via Pairwise Edit Relations for Image Manipulation Localization

Authors: Xuekang Zhu, Kaiwen Feng, Ruifeng Wang, Xiwen Wang, Xiaochen Ma, Bo Du, Changjiang Jiang, Chenfan Qu, +5 more

Organizations: Sichuan University · Ant Group · The Hong Kong University of Science and Technology · Wuhan University · South China University of Technology · University of Southern California · Xiamen University of Technology

Abstract

Image Manipulation Localization (IML) is commonly formulated as a fully supervised learning task that estimates the optimal manipulation mask $y$ for a given image $x$ . In this work, we first reveal the latent nature of artifacts and thus reinterpret IML as a latent-variable problem, $P(y|x)=\int P(y|z)\,P(z|x)\,dz$ , where $z$ denotes the artifacts. Following this interpretation, we pinpoint the cause for the current IML models' insufficiency as their implicit artifacts modeling strategy, highlighting the necessity of modeling $z$ in an explicit manner. Without direct labels, feature disentanglement is the most appropriate solution for this explicit modeling. Accordingly, we propose a two-stage learning paradigm with the Pairwise Artifacts Learning (PAL) and Standard Localization (SL) phases to estimate $P(z|x)$ and $P(y|z)$ via edit relations. To support our edit-relation-based learning, we further curate EditGroup-45K, a source-anchored dataset organized into edit groups for pair construction. Extensive experiments show that our PAL paradigm yields consistent improvements across diverse IML architectures, and empirical analyses further verify that PAL does capture artifacts explicitly through feature disentanglement. Code and dataset are available at https://github.com/venus-guangjian/PAL

Figures & tables

Dataset	Auth.	Manip.	Anchor	Tech.
CASIA v2 Dong et al. (2013)	7,491	5,123	✗	Manual
NIST’16 Guan et al. (2019)	0	564	✗	Manual
IMD2020 Novozamsky et al. (2020)	414	2,010	✗	Manual
COVERAGE Wen et al. (2016)	100	100	✓	Manual
Columbia Hsu and Chang (2006)	183	180	✓	Manual
Fantastic Reality Kniaz et al. (2019)	16,592	19,423	✓	Manual

Table 2 : Comparison with representative IML datasets. ‘Anchor’ denotes source availability.

Figure 2 : Overview of the proposed two-stage paradigm. (a) EditGroup-based training pair construction. (b) Pairwise Artifacts Learning (PAL) with a shared backbone and pairwise BCE supervision. (c) Standard Localization with the PAL-optimized backbone.

Table 4

Figure 3 : Qualitative visualization of PAL-derived Artifacts.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Intra-class Compactness	Inter-class Difference	Epochs	In-Avg	Cross-Avg	All-Avg
EditGroup-45K Property	No	Yes	–	–	–	–
PAL-Con (Contrastive)	Required	Required	30	0.7953	0.5548	0.5951
PAL (BCE)	Not Required	Required	12	0.8256	0.5868	0.6266

Appendix

Table 6 : Comparison between BCE and contrastive objectives. EditGroup-45K provides reliable inter-class differences but does not assume intra-class compactness among heterogeneous manipulation artifacts. PAL-Con converges more slowly and reaches a lower performance ceiling than BCE-based PAL.

Learning Rate	Test AUC
$2\times 10^{-4}$	0.8873
$\mathbf{1\times 10^{-4}}$	$\mathbf{0.8926}$
$2\times 10^{-5}$	0.8897
$1\times 10^{-6}$	0.8862

Appendix

Table 7 : Effect of Learning Rate. The model is trained for 12 epochs with a 1:1 sampling ratio. The best performance is observed at

1\times 10^{-4}

Epochs	Test AUC
10	0.8747
$\mathbf{12}$	$\mathbf{0.8926}$
15	0.8920

Appendix

Table 8 : Effect of Training Duration. One epoch corresponds to sampling 100% of positive pairs. The model saturates at epoch 12.

Ratio ( $Pos:Neg$ )	Test AUC
$\mathbf{1:1}$	$\mathbf{0.8926}$
$1:2$	0.8837

Appendix

Table 9 : Effect of Positive-to-Negative Ratio. Comparing balanced sampling (

1:1

) versus increased negative sampling (

1:2

). Balanced sampling yields superior results.

Figure 4 : Distribution of edit frequency per real image.

Figure 5 : Proportion of single-source vs. multi-source manipulations.

Figure 6 : Distribution of image pairs per Edit Group.

Figure 7 : Categorical distribution of Structural Similarity (SSIM).

Figure 8 : Continuous probability distribution of SSIM across the dataset.

Model	In-Domain	Cross-Domain						Average
Model	CASIAv1	COVERAGE	Columbia	NIST16	Autosplice	CocoGlide	Average	Average
Baseline	0.7369	0.5241	0.8976	0.3754	0.4430	0.4198	0.5320	0.5661
Baseline-ABS	0.8457	0.5888	0.9361	0.3650	0.3637	0.3927	0.5292	0.5820
PAL(Ours)	0.8256	0.6242	0.9043	0.3849	0.4830	0.5375	0.5868	0.6266
w/o Diff	0.8073	0.426	0.9233	0.3763	0.4794	0.4617	0.5333	0.5790
w/o AvgPool	0.7814	0.5826	0.8618	0.3312	0.4415	0.5017	0.5438	0.5834

Appendix

Table 10 : Detailed numerical results of the ablation study evaluating different supervision forms, representation designs, and negative pair compositions. We report the pixel-level F1 score on the Standard Localization task under the Protocol-CAT benchmark. In-Domain performance is evaluated on the source dataset (CASIAv1), while Cross-Domain performance is measured across five unseen datasets (COVERAGE, Columbia, NIST16, AutoSplice, and COCO-Glide). The Average column denotes the mean score across the five cross-domain datasets, and the final column ( All-Avg ) represents the overall mean across all six evaluation datasets.

Perturbation	Model	Kernel Size							F1
Perturbation	Model	None	3	7	11	15	19	23	F1
GaussBlur	Baseline	0.7369	0.6690	0.5853	0.4938	0.3741	0.1746	0.0490	0.4404
GaussBlur	Baseline+PAL	0.8256	0.7590	0.7166	0.6324	0.5157	0.3619	0.2030	0.5735
		QualityFactors
		None	100	90	80	70	60	50
JpegCompression	Baseline	0.7369	0.7124	0.6880	0.6301	0.6268	0.5943	0.5038	0.6418

Appendix

Table 11 : Robustness analysis under common post-processing perturbations. We evaluate the robustness of the Baseline and Baseline+PAL models under Gaussian blur with increasing kernel sizes and JPEG compression with decreasing quality factors. PAL consistently improves robustness across all perturbation levels, exhibiting significantly slower performance degradation as perturbations intensify. The reported F1 score (rightmost column) summarizes the average performance over each perturbation setting.

Figure 9 : Scaling behavior with different pre-training ratios of EditGroup-45K. PAL benefits consistently from a larger data scale, while ABS saturates and slightly degrades when using the full dataset.

Figure 10 : Qualitative comparison between PAL and ImageNet initialization. Rows show representative copy-move, splicing, and inpainting examples. PAL produces localization results closer to GT and fewer false positives than ImageNet initialization.

Explore similar work

Towards Generalized Image Manipulation Localization via Score-based Model
May 16, 2026Yunfei Wang, Bo Du, Zhe Yang +4Multimedia Forensics Conditional Variational Autoencoder
Sichuan University · Chengdu, China
The Courtroom Trial of Pixels: Robust Image Manipulation Localization via Adversarial Evidence and Reinforcement Learning Judgment
Apr 16, 2026Songlin Li, Zhiqing Guo, Dan Ma +2Pixels Adversarial Robustness
School of Computer Science and Technology, Xinjiang University, Urumqi 830046, China
SIGMA: Semantic-Difference Instruction-Grounding Mask Annotator for Text-Driven Image Manipulation Localization
May 27, 2026Peiyu Zhuang, Jianquan Yang, Haodong Li +6Neural Mask Estimation
Shenzhen Campus of Sun Yat-sen University, China · Guangdong Provincial Key Laboratory of Intelligent Information Processing and Shenzhen Key Laboratory of Media Security, Shenzhen University, Shenzhen, China · Shenzhen University of Advanced Technology and Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, China +2

May 16, 2026, cs.CVJ/K move · Enter open · S save

Towards Generalized Image Manipulation Localization via Score-based Model

Yunfei Wang, Bo Du, Zhe Yang, Xin Liu+3

Sichuan University · Chengdu, China

Multimedia Forensics Conditional Variational Autoencoder

Open paper AskPDF arXiv

With the rapid evolution of synthetic media, Image Manipulation Localization (IML) has emerged as a critical component in multimedia forensics for ensuring the integrity of digital content. However, generalization remains a core challenge, as existing discriminative methods typically learn a fixed decision boundary that tends to overfit to specific training artifacts and fails to adapt to unseen manipulation types. To address this, we propose DiffIML, a novel framework that introduces score-based generative modeling to IML. Diverging from the direct estimation of hard boundaries, DiffIML approximates the score function, the gradient of the log-likelihood, to capture the intrinsic geometric topology of mask distributions. This paradigm leverages structural priors to iteratively recover coherent masks from noise, thereby circumventing the brittleness associated with discriminative models. Under this formulation, diffusion models serve as an effective numerical solver for the learned score function.To ensure practicality, we respectively resolve the efficiency and stability bottlenecks of standard diffusion by: (1) utilizing a Lightweight Mask-Specific VAE for fast latent-space process and a decoupled architecture with a lightweight denoising UNet, (2) edge supervision and error prior to mitigate error accumulation during sampling. Extensive experiments of two distinct protocols on eight non-generative and three generative benchmarks demonstrate that DiffIML consistently outperforms state-of-the-art methods, yielding remarkable generalization improvements on diverse unseen datasets. The code is publicly available at https://github.com/scu-zjz/DiffIML.

Abstract

Explore similar work

Towards Generalized Image Manipulation Localization via Score-based Model

The Courtroom Trial of Pixels: Robust Image Manipulation Localization via Adversarial Evidence and Reinforcement Learning Judgment

SIGMA: Semantic-Difference Instruction-Grounding Mask Annotator for Text-Driven Image Manipulation Localization