Organizations: State Key Laboratory of Internet of Things for Smart City, University of Macau · School of Information and Software Engineering, University of Electronic Science and Technology of China
As AI-assisted image editing becomes increasingly prevalent, Text-Centric Image Forgery Localization (TFL) is essential for protecting trust in financial and legal records. However, we observe that existing TFL detectors predominantly rely on Full-Parameter Fine-Tuning (FPFT) and fail to generalize to unseen forgery types because FPFT can induce low-rank feature collapse and overfit training artifacts. Meanwhile, prior detectors capture diverse forgery traces with a single unified representation, making subtle traces entangled and difficult to learn. To address these challenges, we propose RiSE, a Residual Subspace Expert model that preserves pre-trained priors by freezing principal SVD components and adapting only the residual subspaces with a ViT-only segmentor to avoid feature collapse. RiSE introduces forgery-aware residual experts, where each expert is trained independently on a training subset with a particular forgery type, and we show that cross-domain localization performance improves consistently as more experts are added. At inference, we introduce Latent Perturbation Confidence (LPC), which selects the most confident expert by analytically computing the latent-perturbed output. LPC enables efficient confidence estimation with a single forward pass, avoiding repeated forward passes on perturbed samples and improving expert selection. By training on synthetic forgery data, RiSE with 1 expert outperforms prior methods on real-world forgery images by 20.4% while reducing training steps by 10 times. Scaling the number of experts to 9 yields a 35.8% gain. RiSE remains effective with DCT feature, even when the majority of the parameters is frozen with RGB-only pretraining. Code: https://github.com/KahimWong/RiSE.
Figures & tables
Figure 1: Effective-rank comparison of penultimate features across 8 datasets. Prior TFL detectors (DTD, FFDN, and ADCD-Net) concentrate most representation variance in a very small number of directions, while RiSE-1 (one expert RiSE) maintains a substantially higher effective rank, indicating a less collapsed and more expressive representation.
Figure 2: More residual experts improve (a) training optimization and (b) cross-domain localization. RiSE- x denotes the RiSE with x residual experts. In (b), RiSE-3 and RiSE-9 report oracle expert selection, i.e., the upper bound obtained by choosing the best expert with the ground-truth mask.
Figure 3: Training overview of RiSE. (a) The training set is partitioned into non-overlapping subsets using forgery synthesis type and feature-cluster index, and each subset trains one expert independently. (b) Each expert uses the same ViT-only segmenter with a shared, optional DCT branch, a shared frozen principal subspace, and expert-specific residual parameters.
Figure 4: Inference overview of RiSE. (a) RiSE evaluates all experts in parallel on the same input and selects the final localization map from the most confident expert. (b) Latent Perturbation Confidence perturbs the latent features after the VFM backbone, averages the resulting mask predictions, and scores the averaged output with SDC for expert selection.
Methods
Venue
In-Domain (Synthetic Data)
Cross-Domain (Real Data)
Avg
FCD
SCD
Test
AvgI
T-SROIE
OSTF
TPIC-13
RTM
AvgC
TruFor
CVPR’23
0.261
0.312
0.252
0.275
0.213
0.046
0.104
0.034
0.099
0.175
MVSS-Net
ICCV’23
0.207
0.371
0.381
0.320
0.187
0.037
0.113
0.027
0.091
0.189
Mesorch
AAAI’25
0.423
0.459
0.465
0.449
0.294
0.139
0.241
0.041
0.178
0.294
IML-ViT
arXiv
0.469
0.512
0.449
0.476
0.427
0.210
0.256
0.076
0.242
0.343
PSCC-Net
TCSVT’22
0.386
0.393
0.497
0.425
0.517
0.441
0.550
0.126
0.408
0.416
Table 5
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Dataset
Usage
Setting
#Sample
Description
Training Set [ Qu et al., 2023 ]
Train
Source domain
120,000
A synthetic training set of images with copy-move, splicing, and print-based edits on contracts, receipts, and books.
Test [ Qu et al., 2023 ]
Test
In-domain
30,000
A general synthetic test set with copy-move, splicing, and print-based edits.
FCD [ Qu et al., 2023 ]
Test
In-domain
2,000
A synthetic test set with copy-move, splicing, and print-based edits, whose source images come from the Noisy Office Dataset.
SCD [ Qu et al., 2023 ]
Test
In-domain
18,000
A synthetic test set with copy-move, splicing, and print-based edits, whose source images come from Huawei Cloud.
T-SROIE [ Wang et al., 2022b ]
Test
Cross-domain
1,579
Scanned receipts tampered using the AIGC text editing model SR-Net.
OSTF [ Qu et al., 2025 ]
Test
Cross-domain
3,046
Natural scene text images tampered using eight different AIGC-based text editing models.
Appendix
Table 5: Summary of datasets used in our experiments.
Operation
Formulation
Notes
Convolution
μy=W∗μx+b
W is the convolution kernel and ∗ denotes convolution. The matching operation torch.einsum("bqc,bchw->bqhw", Q_mask, Z_mask) can be viewed as query-wise 1×1 convolution with deterministic weights Qmask .
GELU
μy≈π1∑kwkg(2vxξk+μx)
It is a Gauss–Hermite quadrature approximation under the Gaussian input, using nodes and weights (ξk,wk) with default K=5 and g is GELU.
LayerNorm2d
μy,c=sLN2(μx,c−μˉx)γc+βc
μˉx=d1∑cμx,c is the input channel mean, sLN2≈Varc(μx,c)+(1−d1)d1∑cvx,c is the expected channel variance and γc,βc are the LayerNorm affine parameters.
Sigmoid
μy≈21[σ(μx−vx)+σ(μx+vx)]
This is the 2-point Gauss–Hermite quadrature for E[σ(z)] with z∼N(μx,vx) : after rescaling the Gaussian variable, the standard nodes become μx±vx and both have weight 21 .
Appendix
Table 6: ADF mean propagation rules in the mask module.
Operation
Formulation
Notes
Convolution
vy=(W⊙W)∗vx
⊙ denotes element-wise multiplication. The matching operation torch.einsum("bqc,bchw->bqhw", Q_mask, Z_mask) likewise follows this rule with squared deterministic query weights Qmask⊙Qmask , which yields the variance update used in Pseudocode 2 .
GELU
vy≈π1∑kwkg(2vxξk+μx)2−μy2
μy is the GELU mean approximation from the mean table.
LayerNorm2d
vy,c=sLN2v~x,cγc2
v~x,c=(1−d2)vx,c+d21∑c′vx,c′ is the variance after channel centering
Appendix
Table 7: ADF variance propagation rules in the mask module.
Figure 7: RiSE-1 query embeddings for 3,000 balanced training images. The same points are colored by forgery type (left) and K=3 cluster (right).
Figure 8: The same embeddings colored by forgery type (left) and K=9 cluster (right).
Metric
K=3
K=9
Silhouette score
0.1232±0.0000
0.0789±0.0001
Appendix
Table 8: Silhouette scores of the embedding clusters over five random seeds (mean ± standard deviation).
Dataset
Metadata-based / LPC
Metadata-based / oracle
Metadata-free / LPC
Metadata-free / oracle
FCD
0.710
0.746
0.771
0.801
SCD
0.643
0.707
0.614
0.669
Test
0.721
0.757
0.683
0.723
AvgI
0.691
0.737
0.689
0.731
T-SROIE
0.760
0.815
0.783
0.820
OSTF
0.625
0.658
0.588
0.615
Appendix
Table 9: RiSE-3 F1 with metadata-based and metadata-free partitions.
Expert
Test
FCD
SCD
T-SROIE
TPIC-13
RTM
OSTF
Avg.
EGE
0.393
0.600
0.445
0.743
0.763
0.113
0.631
0.527
ECM
0.605
0.634
0.556
0.605
0.231
0.148
0.227
0.430
ESP
0.571
0.669
0.483
0.519
0.248
0.114
0.286
0.413
Appendix
Table 10: Per-expert F1 for RiSE-3. Each row is one expert trained on the generation (GE), copy-move (CM), or splicing (SP) subset. Dataset order matches Table 11
Expert
Test
FCD
SCD
T-SROIE
TPIC-13
RTM
OSTF
Avg.
EGE,0
0.335
0.496
0.433
0.783
0.672
0.089
0.542
0.479
EGE,1
0.244
0.449
0.195
0.519
0.698
0.096
0.580
0.397
EGE,2
0.407
0.594
0.494
0.749
0.798
0.122
0.683
0.550
ECM,0
0.513
0.553
0.505
0.365
0.182
0.113
0.222
0.350
ECM,1
0.587
0.727
0.466
0.598
0.216
0.136
0.237
0.424
ECM,2
0.452
0.480
0.520
0.630
0.176
0.117
0.140
0.359
Appendix
Table 11: Per-expert F1 scores for RiSE-9. Each row is one expert Et,j for forgery type t∈{GE,CM,SP} and within-type cluster j∈{0,1,2}
Dataset
RiSE-3 Top-1
RiSE-9 Top-1
RiSE-9 Top-3
RiSE-9 Top-5
Test
0.620
0.483
0.754
0.887
FCD
0.678
0.644
0.856
0.938
SCD
0.572
0.311
0.695
0.851
T-SROIE
0.630
0.346
0.734
0.889
TPIC-13
0.855
0.544
0.946
0.978
RTM
0.425
0.221
0.491
0.689
Appendix
Table 12: LPC routing accuracy against the oracle-selected expert.
Figure 9: Expert selection counts across seven test sets: LPC assignments and ground-truth oracle choices for (a) RiSE-3 and (b) RiSE-9.
Method
NIST16
Columbia
DSO
CASIA
CASIA2
IMD2020
Avg.
DTD
0.059
0.198
0.116
0.001
0.014
0.076
0.077
FFDN
0.026
0.016
0.001
0.006
0.010
0.017
0.013
ADCD-Net
0.103
0.030
0.047
0.019
0.038
0.060
0.049
RiSE-1
0.052
0.003
0.005
0.060
0.046
0.045
0.035
RiSE-3
0.117
0.031
0.026
0.141
0.110
0.155
0.097
RiSE-9
0.216
0.072
0.030
0.280
0.148
0.190
0.156
Appendix
Table 13: Zero-shot F1 on six natural-image forgery datasets. All methods are trained only on DocTamper.
Dataset
Method
Clean
J90
J80
J70
R90
R80
R70
Avg.
T-SROIE
ADCD-Net
0.623
0.571
0.538
0.508
0.757
0.748
0.755
0.646
RiSE-9-DCT
0.729
0.679
0.638
0.608
0.774
0.780
0.750
0.705
RiSE-9
0.786
0.697
0.660
0.631
0.762
0.772
0.759
0.713
TPIC-13
ADCD-Net
0.579
0.599
0.499
0.436
0.516
0.515
0.513
0.513
RiSE-9-DCT
0.713
0.714
0.625
0.555
0.693
0.682
0.676
0.658
RiSE-9
0.788
0.777
0.722
0.655
0.768
0.761
0.759
0.740
Appendix
Table 14: Robustness under JPEG recompression (J90, J80, J70: quality factors) and resizing (R90, R80, R70: retained linear scale in percent). Avg. averages the six transformed conditions; Clean is reported separately.
Model
AvgC F1
Latency (s)
DTD
0.247
0.0407±0.0006
FFDN
0.300
0.0451±0.0009
RiSE-1
0.542
0.0474±0.0002
ADCD-Net
0.450
0.1171±0.0001
RiSE-3
0.575
0.1165±0.0004
RiSE-9
0.611
0.3571±0.0005
Appendix
Table 15: Cross-domain AvgC F1 and single-image latency on one NVIDIA RTX 4090. Latency is reported as mean ± standard deviation over 100 inputs.
Method
#Total Params (M)
#Trainable Params (M)
Training Time (hour)
Inference Time (s/sample)
RiSE-1
315.14
12.20
1.12
0.0219
RiSE-3
316.02
13.09
2.98
0.0353
RiSE-9
318.68
15.75
7.93
0.0927
Appendix
Table 16: Efficiency comparison of non-DCT RiSE variants measured on 4 NVIDIA RTX 4090 GPUs. Training time is reported in hours; inference time is converted from samples-per-second throughput to seconds per sample.
Figure 10: Visualization of training samples and expert assignments. Each forgery synthesis type is further divided into three feature clusters, producing nine expert assignments in total. Within each subplot, each row shows samples from a single feature cluster.
AI-assisted image editing threatens trust in financial, legal, and identity records. The GenText-Forensics Challenge at ACM MM 2026 addresses this by requiring structured forensic reports, in which integrating detection, pixel-level localization, and natural language explanation for multilingual text-centric forgery images. We present SEED, a modular system with three components. First, a similarity-guided pipeline augments training with diverse synthetic forgeries. Second, a single ViT, built on DINOv3 with LoRA adaptation, jointly performs detection and pixel-level localization while preserving pre-trained priors with minimal trainable parameters. Third, an evolving harness takes the detector's predictions and generates a complete forensic report via an MLLM, iteratively improved through a proposer-evaluator loop optimizing report quality. SEED ranked 3rd in the GenText-Forensics Challenge. Code and data are available at https://github.com/KahimWong/GenText-Forensics-3rd-Place.
Kahim Wong, Kemou Li, Yiming Chen +2
State Key Laboratory of Internet of Things for Smart City, Department of Computer and Information Science, University of Macau · School of Information and Software Engineering, University of Electronic Science and Technology of China
Image forgery localization remains challenging due to diverse manipulation techniques and distribution shifts. Existing forgery localization models achieve high accuracy on benchmarks but often struggle with cross-domain generalization and robustness. In this paper, we propose SARIF (Segment Anything for Robust Image Forensics), a framework that leverages the Segment Anything Model (SAM), which has a promptable architecture and strong generalization ability. SARIF introduces a feedback-guided mask decoder and a dual-encoder design that extracts forgery-specific information to capture forensic traces while exploiting the SAM architecture. To localize manipulated regions, we design a block-wise prompting mechanism that derives forgery-specific cues from residual features between an adapted encoder and its frozen counterpart. These features are fused with the previous mask prompt to drive a feedback-based mask refinement process, enabling automatic forgery segmentation without manual input. Extensive experiments on standard forgery-localization benchmarks show that SARIF achieves strong average cross-dataset performance and robustness to common image corruptions.
Dong-Hyun Moon, Ju-Hyeon Nam, Sang-Chul Lee
Department of Electrical and Computer Engineering, Inha University, 100 Inha-ro, Michuhol-gu, Incheon, 22212, Republic of Korea
Recent advancements in generative AI have led to image editing models capable of producing realistic forgeries that evade traditional image forgery localization methods, as these approaches depend on physical noise absent in synthetic data. To address this challenge, we theoretically demonstrate that the diffusion process inherently suppresses local high-frequency variance, creating a statistical energy gap that is distinguishable from the natural entropy of optical imaging. Guided by this insight, we propose FLAME, a unified framework that utilizes a LAD map to capture these intrinsic anomalies, coupled with a parameter-efficient adapter for SAM to achieve precise, pixel-level forgery localization. Furthermore, to bridge the lag between forensic benchmarks and evolving generative models, we introduce EditStream, an automated pipeline for continuous, instruction-based training data synthesis. Extensive experiments demonstrate that FLAME establishes a new state-of-the-art, significantly outperforming previous methods on AI-generated forgery datasets while effectively generalizing to unseen generative architectures. Our code is available at https://github.com/phoenixnir/FLAME.