RiSE: Residual Subspace Expert for Generalizable Text-Centric Image Forgery Localization

Kahim Wong; Kemou Li; Yiming Chen; Haiwei Wu; Jiantao Zhou

cs.CVOct 17, 2026

RiSE: Residual Subspace Expert for Generalizable Text-Centric Image Forgery Localization

Authors: Kahim Wong, Kemou Li, Yiming Chen, Haiwei Wu, Jiantao Zhou

Organizations: State Key Laboratory of Internet of Things for Smart City, University of Macau · School of Information and Software Engineering, University of Electronic Science and Technology of China

Abstract

As AI-assisted image editing becomes increasingly prevalent, Text-Centric Image Forgery Localization (TFL) is essential for protecting trust in financial and legal records. However, we observe that existing TFL detectors predominantly rely on Full-Parameter Fine-Tuning (FPFT) and fail to generalize to unseen forgery types because FPFT can induce low-rank feature collapse and overfit training artifacts. Meanwhile, prior detectors capture diverse forgery traces with a single unified representation, making subtle traces entangled and difficult to learn. To address these challenges, we propose RiSE, a Residual Subspace Expert model that preserves pre-trained priors by freezing principal SVD components and adapting only the residual subspaces with a ViT-only segmentor to avoid feature collapse. RiSE introduces forgery-aware residual experts, where each expert is trained independently on a training subset with a particular forgery type, and we show that cross-domain localization performance improves consistently as more experts are added. At inference, we introduce Latent Perturbation Confidence (LPC), which selects the most confident expert by analytically computing the latent-perturbed output. LPC enables efficient confidence estimation with a single forward pass, avoiding repeated forward passes on perturbed samples and improving expert selection. By training on synthetic forgery data, RiSE with 1 expert outperforms prior methods on real-world forgery images by 20.4% while reducing training steps by 10 times. Scaling the number of experts to 9 yields a 35.8% gain. RiSE remains effective with DCT feature, even when the majority of the parameters is frozen with RGB-only pretraining. Code: https://github.com/KahimWong/RiSE.

Figures & tables

Figure 1: Effective-rank comparison of penultimate features across 8 datasets. Prior TFL detectors (DTD, FFDN, and ADCD-Net) concentrate most representation variance in a very small number of directions, while RiSE-1 (one expert RiSE) maintains a substantially higher effective rank, indicating a less collapsed and more expressive representation.

Panel of Figure 2 — Figure 2: More residual experts improve (a) training optimization and (b) cross-domain localization. RiSE- $x$ denotes the RiSE with $x$ residual experts. In (b), RiSE-3 and RiSE-9 report oracle expert selection, i.e., the upper bound obtained by choosing the best expert with the ground-truth mask.

Figure 3: Training overview of RiSE. (a) The training set is partitioned into non-overlapping subsets using forgery synthesis type and feature-cluster index, and each subset trains one expert independently. (b) Each expert uses the same ViT-only segmenter with a shared, optional DCT branch, a shared frozen principal subspace, and expert-specific residual parameters.

Figure 4: Inference overview of RiSE. (a) RiSE evaluates all experts in parallel on the same input and selects the final localization map from the most confident expert. (b) Latent Perturbation Confidence perturbs the latent features after the VFM backbone, averages the resulting mask predictions, and scores the averaged output with SDC for expert selection.

Methods	Venue	In-Domain (Synthetic Data)				Cross-Domain (Real Data)					Avg
		FCD	SCD	Test	AvgI	T-SROIE	OSTF	TPIC-13	RTM	AvgC
TruFor	CVPR’23	0.261	0.312	0.252	0.275	0.213	0.046	0.104	0.034	0.099	0.175
MVSS-Net	ICCV’23	0.207	0.371	0.381	0.320	0.187	0.037	0.113	0.027	0.091	0.189
Mesorch	AAAI’25	0.423	0.459	0.465	0.449	0.294	0.139	0.241	0.041	0.178	0.294
IML-ViT	arXiv	0.469	0.512	0.449	0.476	0.427	0.210	0.256	0.076	0.242	0.343
PSCC-Net	TCSVT’22	0.386	0.393	0.497	0.425	0.517	0.441	0.550	0.126	0.408	0.416

Table 5

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Usage	Setting	#Sample	Description
Training Set [ Qu et al., 2023 ]	Train	Source domain	120,000	A synthetic training set of images with copy-move, splicing, and print-based edits on contracts, receipts, and books.
Test [ Qu et al., 2023 ]	Test	In-domain	30,000	A general synthetic test set with copy-move, splicing, and print-based edits.
FCD [ Qu et al., 2023 ]	Test	In-domain	2,000	A synthetic test set with copy-move, splicing, and print-based edits, whose source images come from the Noisy Office Dataset.
SCD [ Qu et al., 2023 ]	Test	In-domain	18,000	A synthetic test set with copy-move, splicing, and print-based edits, whose source images come from Huawei Cloud.
T-SROIE [ Wang et al., 2022b ]	Test	Cross-domain	1,579	Scanned receipts tampered using the AIGC text editing model SR-Net.
OSTF [ Qu et al., 2025 ]	Test	Cross-domain	3,046	Natural scene text images tampered using eight different AIGC-based text editing models.

Appendix

Table 5: Summary of datasets used in our experiments.

Operation	Formulation	Notes
Convolution	$\boldsymbol{\mu}_{y}=\mathbf{W}*\boldsymbol{\mu}_{x}+\mathbf{b}$	$\mathbf{W}$ is the convolution kernel and $*$ denotes convolution. The matching operation torch.einsum("bqc,bchw->bqhw", Q_mask, Z_mask) can be viewed as query-wise $1\times 1$ convolution with deterministic weights $\mathbf{Q}_{\mathrm{mask}}$ .
GELU	$\boldsymbol{\mu}_{y}\approx\frac{1}{\sqrt{\pi}}\sum_{k}w_{k}g(\sqrt{2\mathbf{v}_{x}}\,\xi_{k}+\boldsymbol{\mu}_{x})$	It is a Gauss–Hermite quadrature approximation under the Gaussian input, using nodes and weights $(\xi_{k},w_{k})$ with default $K=5$ and $g$ is GELU.
LayerNorm2d	$\boldsymbol{\mu}_{y,c}=\frac{(\boldsymbol{\mu}_{x,c}-\bar{\boldsymbol{\mu}}_{x})\gamma_{c}}{\sqrt{\mathbf{s}_{\mathrm{LN}}^{2}}}+\beta_{c}$	$\bar{\boldsymbol{\mu}}_{x}=\frac{1}{d}\sum_{c}\boldsymbol{\mu}_{x,c}$ is the input channel mean, $\mathbf{s}_{\mathrm{LN}}^{2}\approx\operatorname{Var}_{c}(\boldsymbol{\mu}_{x,c})+(1-\frac{1}{d})\frac{1}{d}\sum_{c}\mathbf{v}_{x,c}$ is the expected channel variance and $\gamma_{c},\beta_{c}$ are the LayerNorm affine parameters.
Sigmoid	$\boldsymbol{\mu}_{y}\approx\frac{1}{2}\left[\sigma\!\left(\boldsymbol{\mu}_{x}-\sqrt{\mathbf{v}_{x}}\right)+\sigma\!\left(\boldsymbol{\mu}_{x}+\sqrt{\mathbf{v}_{x}}\right)\right]$	This is the 2-point Gauss–Hermite quadrature for $\mathbb{E}[\sigma(z)]$ with $z\sim\mathcal{N}(\boldsymbol{\mu}_{x},\mathbf{v}_{x})$ : after rescaling the Gaussian variable, the standard nodes become $\boldsymbol{\mu}_{x}\pm\sqrt{\mathbf{v}_{x}}$ and both have weight $\frac{1}{2}$ .

Appendix

Table 6: ADF mean propagation rules in the mask module.

Operation	Formulation	Notes
Convolution	$\mathbf{v}_{y}=(\mathbf{W}\odot\mathbf{W})*\mathbf{v}_{x}$	$\odot$ denotes element-wise multiplication. The matching operation torch.einsum("bqc,bchw->bqhw", Q_mask, Z_mask) likewise follows this rule with squared deterministic query weights $\mathbf{Q}_{\mathrm{mask}}\odot\mathbf{Q}_{\mathrm{mask}}$ , which yields the variance update used in Pseudocode 2 .
GELU	$\mathbf{v}_{y}\approx\frac{1}{\sqrt{\pi}}\sum_{k}w_{k}g(\sqrt{2\mathbf{v}_{x}}\,\xi_{k}+\boldsymbol{\mu}_{x})^{2}-\boldsymbol{\mu}_{y}^{2}$	$\boldsymbol{\mu}_{y}$ is the GELU mean approximation from the mean table.
LayerNorm2d	$\mathbf{v}_{y,c}=\frac{\tilde{\mathbf{v}}_{x,c}\gamma_{c}^{2}}{\mathbf{s}_{\mathrm{LN}}^{2}}$	$\tilde{\mathbf{v}}_{x,c}=(1-\frac{2}{d})\mathbf{v}_{x,c}+\frac{1}{d^{2}}\sum_{c^{\prime}}\mathbf{v}_{x,c^{\prime}}$ is the variance after channel centering

Appendix

Table 7: ADF variance propagation rules in the mask module.

Figure 7: RiSE-1 query embeddings for 3,000 balanced training images. The same points are colored by forgery type (left) and $K=3$ cluster (right).

Figure 8: The same embeddings colored by forgery type (left) and $K=9$ cluster (right).

Metric	$K=3$	$K=9$
Silhouette score	$0.1232\pm 0.0000$	$0.0789\pm 0.0001$

Appendix

Table 8: Silhouette scores of the embedding clusters over five random seeds (mean

\pm

standard deviation).

Dataset	Metadata-based / LPC	Metadata-based / oracle	Metadata-free / LPC	Metadata-free / oracle
FCD	0.710	0.746	0.771	0.801
SCD	0.643	0.707	0.614	0.669
Test	0.721	0.757	0.683	0.723
AvgI	0.691	0.737	0.689	0.731
T-SROIE	0.760	0.815	0.783	0.820
OSTF	0.625	0.658	0.588	0.615

Appendix

Table 9: RiSE-3 F1 with metadata-based and metadata-free partitions.

Expert	Test	FCD	SCD	T-SROIE	TPIC-13	RTM	OSTF	Avg.
$E_{\rm GE}$	0.393	0.600	0.445	0.743	0.763	0.113	0.631	0.527
$E_{\rm CM}$	0.605	0.634	0.556	0.605	0.231	0.148	0.227	0.430
$E_{\rm SP}$	0.571	0.669	0.483	0.519	0.248	0.114	0.286	0.413

Appendix

Table 10: Per-expert F1 for RiSE-3. Each row is one expert trained on the generation (GE), copy-move (CM), or splicing (SP) subset. Dataset order matches Table 11

Expert	Test	FCD	SCD	T-SROIE	TPIC-13	RTM	OSTF	Avg.
$E_{\rm GE,0}$	0.335	0.496	0.433	0.783	0.672	0.089	0.542	0.479
$E_{\rm GE,1}$	0.244	0.449	0.195	0.519	0.698	0.096	0.580	0.397
$E_{\rm GE,2}$	0.407	0.594	0.494	0.749	0.798	0.122	0.683	0.550
$E_{\rm CM,0}$	0.513	0.553	0.505	0.365	0.182	0.113	0.222	0.350
$E_{\rm CM,1}$	0.587	0.727	0.466	0.598	0.216	0.136	0.237	0.424
$E_{\rm CM,2}$	0.452	0.480	0.520	0.630	0.176	0.117	0.140	0.359

Appendix

Table 11: Per-expert F1 scores for RiSE-9. Each row is one expert

E_{t,j}

for forgery type

t\in\{\rm GE,CM,SP\}

and within-type cluster

j\in\{0,1,2\}

Dataset	RiSE-3 Top-1	RiSE-9 Top-1	RiSE-9 Top-3	RiSE-9 Top-5
Test	0.620	0.483	0.754	0.887
FCD	0.678	0.644	0.856	0.938
SCD	0.572	0.311	0.695	0.851
T-SROIE	0.630	0.346	0.734	0.889
TPIC-13	0.855	0.544	0.946	0.978
RTM	0.425	0.221	0.491	0.689

Appendix

Table 12: LPC routing accuracy against the oracle-selected expert.

Panel of Figure 9 — Figure 9: Expert selection counts across seven test sets: LPC assignments and ground-truth oracle choices for (a) RiSE-3 and (b) RiSE-9.

Method	NIST16	Columbia	DSO	CASIA	CASIA2	IMD2020	Avg.
DTD	0.059	0.198	0.116	0.001	0.014	0.076	0.077
FFDN	0.026	0.016	0.001	0.006	0.010	0.017	0.013
ADCD-Net	0.103	0.030	0.047	0.019	0.038	0.060	0.049
RiSE-1	0.052	0.003	0.005	0.060	0.046	0.045	0.035
RiSE-3	0.117	0.031	0.026	0.141	0.110	0.155	0.097
RiSE-9	0.216	0.072	0.030	0.280	0.148	0.190	0.156

Appendix

Table 13: Zero-shot F1 on six natural-image forgery datasets. All methods are trained only on DocTamper.

Dataset	Method	Clean	J90	J80	J70	R90	R80	R70	Avg.
T-SROIE	ADCD-Net	0.623	0.571	0.538	0.508	0.757	0.748	0.755	0.646
	RiSE-9-DCT	0.729	0.679	0.638	0.608	0.774	0.780	0.750	0.705
	RiSE-9	0.786	0.697	0.660	0.631	0.762	0.772	0.759	0.713
TPIC-13	ADCD-Net	0.579	0.599	0.499	0.436	0.516	0.515	0.513	0.513
	RiSE-9-DCT	0.713	0.714	0.625	0.555	0.693	0.682	0.676	0.658
	RiSE-9	0.788	0.777	0.722	0.655	0.768	0.761	0.759	0.740

Appendix

Table 14: Robustness under JPEG recompression (J90, J80, J70: quality factors) and resizing (R90, R80, R70: retained linear scale in percent). Avg. averages the six transformed conditions; Clean is reported separately.

Model	AvgC F1	Latency (s)
DTD	0.247	$0.0407\pm 0.0006$
FFDN	0.300	$0.0451\pm 0.0009$
RiSE-1	0.542	$0.0474\pm 0.0002$
ADCD-Net	0.450	$0.1171\pm 0.0001$
RiSE-3	0.575	$0.1165\pm 0.0004$
RiSE-9	0.611	$0.3571\pm 0.0005$

Appendix

Table 15: Cross-domain AvgC F1 and single-image latency on one NVIDIA RTX 4090. Latency is reported as mean

\pm

standard deviation over 100 inputs.

Method	#Total Params (M)	#Trainable Params (M)	Training Time (hour)	Inference Time (s/sample)
RiSE-1	315.14	12.20	1.12	0.0219
RiSE-3	316.02	13.09	2.98	0.0353
RiSE-9	318.68	15.75	7.93	0.0927

Appendix

Table 16: Efficiency comparison of non-DCT RiSE variants measured on

4

NVIDIA RTX 4090 GPUs. Training time is reported in hours; inference time is converted from samples-per-second throughput to seconds per sample.

Panel of Figure 10 — Figure 10: Visualization of training samples and expert assignments. Each forgery synthesis type is further divided into three feature clusters, producing nine expert assignments in total. Within each subplot, each row shows samples from a single feature cluster.

Image Forgery Localization

Image forgery localization identifies manipulated or AI-generated regions in images, often alongside image-level authenticity detection.

See all 79 papers

SEED: Simple ViT and Evolving Harness for Explainable Text Forgery Detection
Jun 19, 2026Kahim Wong, Kemou Li, Yiming Chen +2Image Forgery Detection Digital Image Forensics
State Key Laboratory of Internet of Things for Smart City, Department of Computer and Information Science, University of Macau · School of Information and Software Engineering, University of Electronic Science and Technology of China
SARIF: Segment Anything for Robust Image Forensics
Jun 19, 2026Dong-Hyun Moon, Ju-Hyeon Nam, Sang-Chul LeeDigital Image Forensics Domain Generalization
Department of Electrical and Computer Engineering, Inha University, 100 Inha-ro, Michuhol-gu, Incheon, 22212, Republic of Korea
Order within Chaos: Capturing Intrinsic Energy Anomalies for AI-Manipulated Image Forgery Localization
Jun 1, 2026Yiming Wang, Baiqi Wu, Qingming Li +3Image Forgery Detection Digital Image Forensics
Zhejiang University, Hangzhou, China.

Jun 19, 2026J/K move · Enter open · S save

SEED: Simple ViT and Evolving Harness for Explainable Text Forgery Detection

Kahim Wong, Kemou Li, Yiming Chen, Haiwei Wu+1

State Key Laboratory of Internet of Things for Smart City, Department of Computer and Information Science, University of Macau · School of Information and Software Engineering, University of Electronic Science and Technology of China

Image Forgery Detection Digital Image Forensics Digital Forensics Image Forgery Localization Text Forgery Detection

Open paper AskPDF arXiv

AI-assisted image editing threatens trust in financial, legal, and identity records. The GenText-Forensics Challenge at ACM MM 2026 addresses this by requiring structured forensic reports, in which integrating detection, pixel-level localization, and natural language explanation for multilingual text-centric forgery images. We present SEED, a modular system with three components. First, a similarity-guided pipeline augments training with diverse synthetic forgeries. Second, a single ViT, built on DINOv3 with LoRA adaptation, jointly performs detection and pixel-level localization while preserving pre-trained priors with minimal trainable parameters. Third, an evolving harness takes the detector's predictions and generates a complete forensic report via an MLLM, iteratively improved through a proposer-evaluator loop optimizing report quality. SEED ranked 3rd in the GenText-Forensics Challenge. Code and data are available at https://github.com/KahimWong/GenText-Forensics-3rd-Place.

Abstract

Image Forgery Localization

Explore similar work

SEED: Simple ViT and Evolving Harness for Explainable Text Forgery Detection

SARIF: Segment Anything for Robust Image Forensics

Order within Chaos: Capturing Intrinsic Energy Anomalies for AI-Manipulated Image Forgery Localization