Leveraging Contrastive Learning for a Similarity-Guided Tampered Document Data Generation Pipeline

Mohamed Dhouib; Davide Buscaldi; Sonia Vanier; Aymen Shabou

cs.CVFeb 19, 2026

Leveraging Contrastive Learning for a Similarity-Guided Tampered Document Data Generation Pipeline

Authors: Mohamed Dhouib, Davide Buscaldi, Sonia Vanier, Aymen Shabou

Organizations: LIX, École Polytechnique, IP Paris, France · LIPN, Université Sorbonne Paris Nord, France · DataLab Groupe, Crédit Agricole S.A, France

Abstract

Detecting tampered text in document images is a challenging task due to data scarcity. To address this, previous work has attempted to generate tampered documents using rule-based methods. However, the resulting documents often suffer from limited variety and poor visual quality, typically leaving highly visible artifacts that are rarely observed in real-world manipulations. This undermines the model's ability to learn robust, generalizable features and results in poor performance on real-world data. Motivated by this discrepancy, we propose a novel method for generating high-quality tampered document images. We first train an auxiliary network to compare text crops, leveraging contrastive learning with a novel strategy for defining positive pairs and their corresponding negatives. We also train a second auxiliary network to evaluate whether a crop tightly encloses the intended characters, without cutting off parts of characters or including parts of adjacent ones. Using a carefully designed generation pipeline that leverages both networks, we introduce a framework capable of producing diverse, high-quality tampered document images. We assess the effectiveness of our data generation pipeline by training multiple models on datasets derived from the same source images, generated using our method and existing approaches, under identical training protocols. Evaluating these models on various open-source datasets shows that our pipeline yields consistent performance improvements across architectures and datasets.

Figures & tables

Panel of Figure 1 — Figure 1: Illustration of some failure cases in tampered document generation caused by a visual mismatch between the source and target crops, resulting in obvious visual artifacts.

Panel of Figure 2 — Figure 2: Illustration of positive and negative pairs in our contrastive setting. The red box denotes the anchor, the green its positive, and the blue boxes the negatives.

Figure 3: Similarity scores from $\mathcal{F}_{\theta}$ between a target crop (left) and candidate crops. The score below each crop indicates its similarity to the target crop and is scaled by a factor of 10.

Panel of Figure 4 — Figure 4: Illustration of failure cases in document tampering due to poor bounding boxes: (A) an ill‑defined source box cutting through characters; (B) an ill‑defined target box overlapping adjacent text.

Figure 5: A simplified illustration of our tampered document generation process.

Model	Data Gen.	RTM						FindItAgain						FindIt
Model	Data Gen.	Image			Pixel			Image			Pixel			Image			Pixel
		P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
PSCC-Net	[ 25 ]	38.5	27.9	32.3	7.5	10.0	8.6	47.8	31.4	37.9	8.7	9.2	8.9	61.3	53.5	57.1	11.2	11.9	11.5
	[ 6 ]	33.8	34.1	33.9	7.9	8.1	8.0	51.8	36.6	42.9	7.5	8.7	8.1	60.8	52.0	56.1	8.9	9.7	9.3
	Ours	39.3	40.2	39.7	10.5	10.8	10.6	56.7	60.0	58.3	10.6	12.8	11.6	62.9	55.0	58.7	12.0	12.1	12.0
	$\Delta$ (%)	+2.1%	+17.9%	+17.1%	+32.9%	+8.0%	+23.3%	+9.5%	+63.9%	+35.9%	+21.8%	+39.1%	+30.3%	+2.6%	+2.8%	+2.8%	+7.1%	+1.7%	+4.3%

Table 1: Zero-shot document tampering detection results on RTM, FindItAgain, and FindIt for several models trained on data generated using different data generation approaches.

\Delta(\%)

indicates the relative performance gain of our generation method over the best of [ 25 ] and [ 6 ] . P, R, and F1 correspond to precision, recall, and F1 score, while Image and Pixel denote image-level classification and pixel-level localization performance. Note that the source images used to generate the training data are the same across models and data generation methods as described in Sec. 4.1 , and all models were trained using the same training protocol as described in Sec. 4.2.3 .

Training Data	Image-level			Pixel-level
	Prec	Rec	F1	Prec	Rec	F1
DocTamper	39.4	36.7	37.3	11.7	8.3	9.4
Ours	53.2	52.6	52.0	18.2	14.2	15.7

Table 2: Comparison of models trained on data generated by our approach and on the DocTamper dataset. Results are averaged across models and evaluation datasets.

Data Gen.	Image-level			Pixel-level
	Prec	Rec	F1	Prec	Rec	F1
[ 25 ]	65.6	64.6	63.7	29.4	26.3	27.1
[ 6 ]	62.5	61.8	61.2	27.9	26.3	26.5
Ours	73.5	65.1	67.5	34.6	27.3	30.0

Table 3: Impact of data-generation strategy on document tampering detection and localization after fine-tuning. Models are pretrained on synthetic data generated by different approaches and then fine-tuned on each target dataset’s training split. Results are averaged across models and evaluation datasets.

Model	Ours		w/o $G_{\theta}$		w/o $F_{\theta}$		w/o $F_{\theta},G_{\theta}$
Model	Img	Pix	Img	Pix	Img	Pix	Img	Pix
CAT-Net	41.3	9.7	41.1	9.6	40.6	9.5	40.4	9.5
DTD	52.5	13.2	51.5	12.9	49.1	12.3	48.3	12.0
PSCC-Net	52.2	11.4	51.3	11.0	45.9	10.3	43.8	9.8
ASC-Former	51.5	17.2	50.9	16.8	49.0	14.5	48.4	14.1
FFDN	62.3	27.0	61.4	26.6	57.8	24.0	55.0	23.5

Table 4: Ablation study on the auxiliary networks

F_{\theta}

and

G_{\theta}

used in the data generation process. When ablating

F_{\theta}

, Sauvola thresholding was used, as in [ 25 ] . Results are averaged across models and evaluation datasets and reported in the zero-shot setting.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Similarity degradation curves for several visual transformations. Each curve shows the average percentage of retained similarity as the intensity of a single perturbation increases.

Data Gen.	Image-level			Pixel-level
	Prec	Rec	F1	Prec	Rec	F1
Baseline (Google OCR)	53.2	52.6	52.0	18.2	14.2	15.7
Tesseract OCR	52.4	52.5	51.6	17.9	13.9	15.3
Perturbed OCR boxes	52.7	52.0	51.6	17.6	13.5	15.0

Appendix

Table 5: Robustness of our data generation pipeline to OCR quality. We report average zero-shot performance across models and evaluation datasets when using Google OCR, when replacing the OCR engine with Tesseract, and when perturbing the Google OCR bounding boxes prior to generation.

Generation	Pixel-level
Setting	Prec	Rec	F1
ours	67.6	29.3	40.1
w/o $\mathcal{G}_{\theta}$	68.4	39.8	49.7
w/o $\mathcal{F}_{\theta}$	69.9	71.5	68.5
w/o $\mathcal{F}_{\theta}$ , $\mathcal{G}_{\theta}$	85.1	90.2	87.4

Appendix

Table 6: Detection performance on synthetic tampered datasets generated with and without

\mathcal{F}_{\theta}

and

\mathcal{G}_{\theta}

. Lower F1 indicates more realistic and challenging tampering. Averages across different models are shown.

Method	Per-image (ms)	Throughput (img/s)	Slowdown ratio
Crop-quality net (ours)	0.159 $\pm$ 0.000	6306.8	1.00 $\times$
Niblack	1.213 $\pm$ 0.018	824.3	7.65 $\times$
Adaptive mean	0.536 $\pm$ 0.007	1866.4	3.38 $\times$
Adaptive Gaussian	0.638 $\pm$ 0.009	1567.0	4.02 $\times$
Sauvola	1.783 $\pm$ 0.021	560.8	11.25 $\times$

Appendix

Table 7: Inference speed comparison between the crop-quality network and classical baseline algorithms. Lower is better for latency; higher is better for throughput. The slowdown ratio is defined as (classic / model) per-image time. The evaluation is done with a batch size of 1024.

Method	Accuracy well-defined	Accuracy ill-defined
Crop-quality network	98.6%	97.9 %

Appendix

Table 8: Crop-quality classification results on data derived from the DIBCO datasets.

Model	Data Gen.	RTM						FindItAgain						FindIt
Model	Data Gen.	Image			Pixel			Image			Pixel			Image			Pixel
		P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
PSCC-Net	[ 25 ]	54.8	88.5	67.7	15.2	17.8	16.4	59.8	54.0	56.8	14.1	31.3	19.4	74.6	73.7	74.2	35.3	34.4	34.8
	[ 6 ]	52.1	88.2	65.5	14.3	16.7	15.4	58.5	53.4	55.8	13.3	28.7	18.2	71.1	69.1	70.1	32.4	34.2	33.3
	Ours	53.3	90.2	67.0	15.9	20.4	17.9	73.0	54.2	62.2	30.1	22.6	25.8	71.6	72.5	72.1	34.3	32.7	33.5
CAT-Net	[ 25 ]	63.7	75.5	69.1	23.6	18.3	20.6	35.1	37.1	36.1	6.3	5.6	5.9	60.2	77.5	67.8	47.8	27.3	34.8

Appendix

Table 9: Document tampering detection and localization results after fine-tuning. Models are pretrained on synthetic data generated by different approaches and then fine-tuned on each target dataset’s training split. Results are reported on RTM, FindItAgain, and FindIt. P, R, and F1 correspond to precision, recall, and F1 score, while Image and Pixel denote image-level classification and pixel-level localization performance.

Method	Image-level			Pixel-level
	Prec	Rec	F1	Prec	Rec	F1
Ours	84.1	99.5	90.9	79.9	87.2	83.1

Appendix

Table 10: Zero-shot performance on AI-generated tampering. Evaluation was conducted on 5,000 FLUX-Text tampered samples and 5,000 AnyText tampered samples, combined with 10,000 untampered images and averaged across models.

Model	Data Gen.	RTM						FindItAgain						FindIt
Model	Data Gen.	Image			Pixel			Image			Pixel			Image			Pixel
		P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1	P	R	F1
PSCC-Net	Ours	39.3	40.2	39.7	10.5	10.8	10.6	56.7	60.0	58.3	10.6	12.8	11.6	62.9	55.0	58.7	12.0	12.1	12.0
PSCC-Net	w/o $F_{\theta},G_{\theta}$	38.8	39.7	39.2	6.8	9.9	8.1	42.4	40.0	41.2	8.3	10.7	9.4	46.0	57.5	51.1	11.9	11.8	11.8
CAT-Net	Ours	40.0	35.0	37.4	9.2	11.0	10.0	48.0	34.2	40.0	11.0	5.6	7.4	51.5	42.5	46.6	13.0	10.5	11.7
CAT-Net	w/o $F_{\theta},G_{\theta}$	38.6	34.8	36.6	9.7	10.1	9.9	47.5	33.9	39.5	10.5	5.6	7.3	51.5	40.0	45.0	12.7	10.1	11.2

Appendix

Table 11: Ablation results on RTM, FindItAgain, and FindIt.

Panel of Figure 7 — Figure 7: Visual comparison of tampered regions generated under four settings: with both $\mathcal{F}_{\theta}$ and $\mathcal{G}_{\theta}$ enabled ( full ), with the similarity network $\mathcal{F}_{\theta}$ disabled ( no_similarity ), with the crop-quality network $\mathcal{G}_{\theta}$ disabled ( no_quality ), and with both disabled ( no_sim_no_quality ). These examples illustrate the impact of each auxiliary network on tampering realism.

Figure 8: Qualitative comparison of the FFDN segmentation maps when the model is trained on data generated by our approach ( Ours ) versus the best of models trained on data generated by [ 25 ] and [ 6 ] , which we denote collectively as Baseline in the figure. Each block shows the input image (left), the predicted tampering mask of our model (red), the baseline prediction (green), and the ground-truth mask (blue). Images and ground-truth masks are from the RTM dataset.

Panel of Figure 9 — Figure 9: Precision–recall curves averaged across datasets for ASC-Former and FFDN.

Panel of Figure 10 — Figure 10: Examples of generated tampered documents and their corresponding masks.

Synthetic Data Augmentation

Synthetic data augmentation creates or modifies training examples to improve learning when real data are scarce, imbalanced, costly, or sensitive.

See all 454 papers

TextShield-R1: Reinforced Reasoning for Tampered Text Detection
Feb 23, 2026Chenfan Qu, Yiwu Zhong, Jian Liu +3Text Forgery Detection Image Forgery Detection
South China University of Technology · Ant Group · Peking University
Team MSU GenText-Forensics Challenge 2026 Technical Report
Sep 29, 2026Kirill Koltsov, Aleksandr Gushchin, Dmitriy Vatolin +1Image Forgery Detection Digital Forensics
Lomonosov Moscow State University Moscow, Russia · MSU Institute for Artificial Intelligence Moscow, Russia
Beyond Natural Images: Rethinking AI-Generated Image Detection in Documents
Sep 13, 2026Zhangjie Fu, Jiazhen Yan, Yuanwen Chen +5AI-Generated Image Detection Digital Image Forensics
Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology · Baidu Inc. · School of Computer Science and Engineering, MoE Key Laboratory of Information Technology, Guangdong Province Key Laboratory of Information Security Technology, Sun Yat-sen University +1

Feb 23, 2026J/K move · Enter open · S save

TextShield-R1: Reinforced Reasoning for Tampered Text Detection

Chenfan Qu, Yiwu Zhong, Jian Liu, Xuekang Zhu+2

South China University of Technology · Ant Group · Peking University

Text Forgery Detection Image Forgery Detection Image Forgery Localization

Open paper AskPDF arXiv

The growing prevalence of tampered images poses serious security threats, highlighting the urgent need for reliable detection methods. Multimodal large language models (MLLMs) demonstrate strong potential in analyzing tampered images and generating interpretations. However, they still struggle with identifying micro-level artifacts, exhibit low accuracy in localizing tampered text regions, and heavily rely on expensive annotations for forgery interpretation. To this end, we introduce TextShield-R1, the first reinforcement learning based MLLM solution for tampered text detection and reasoning. Specifically, our approach introduces Forensic Continual Pre-training, an easy-to-hard curriculum that well prepares the MLLM for tampered text detection by harnessing the large-scale cheap data from natural image forensic and OCR tasks. During fine-tuning, we perform Group Relative Policy Optimization with novel reward functions to reduce annotation dependency and improve reasoning capabilities. At inference time, we enhance localization accuracy via OCR Rectification, a method that leverages the MLLM's strong text recognition abilities to refine its predictions. Furthermore, to support rigorous evaluation, we introduce the Text Forensics Reasoning (TFR) benchmark, comprising over 45k real and tampered images across 16 languages, 10 tampering techniques, and diverse domains. Rich reasoning-style annotations are included, allowing for comprehensive assessment. Our TFR benchmark simultaneously addresses seven major limitations of existing benchmarks and enables robust evaluation under cross-style, cross-method, and cross-language conditions. Extensive experiments demonstrate that TextShield-R1 significantly advances the state of the art in interpretable tampered text detection.

Abstract

Synthetic Data Augmentation

Explore similar work

TextShield-R1: Reinforced Reasoning for Tampered Text Detection

Team MSU GenText-Forensics Challenge 2026 Technical Report

Beyond Natural Images: Rethinking AI-Generated Image Detection in Documents